2024-07-16T09:35:40.691 INFO:root:teuthology version: 0.0.1.dev355+g5bc6960 2024-07-16T09:35:40.691 DEBUG:teuthology.run:Teuthology command: teuthology --verbose --description orch:cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn pg_health syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} --name adking-2024-07-16_04:15:24-orch:cephadm-wip-adk4-testing-2024-07-15-1832-squid-distro-default-smithi --owner scheduled_adking@teuthology --archive /home/teuthworker/archive/adking-2024-07-16_04:15:24-orch:cephadm-wip-adk4-testing-2024-07-15-1832-squid-distro-default-smithi/7803693 -- /home/teuthworker/archive/adking-2024-07-16_04:15:24-orch:cephadm-wip-adk4-testing-2024-07-15-1832-squid-distro-default-smithi/7803693/orig.config.yaml 2024-07-16T09:35:40.743 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-07-16T09:35:40.831 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/adking-2024-07-16_04:15:24-orch:cephadm-wip-adk4-testing-2024-07-15-1832-squid-distro-default-smithi/7803693 branch: wip-adk4-testing-2024-07-15-1832-squid description: orch:cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn pg_health syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} email: adking@redhat.com first_in_suite: false job_id: '7803693' kernel: kdb: 1 sha1: distro last_in_suite: false machine_type: smithi meta: - desc: 'setup ceph/quincy ' name: adking-2024-07-16_04:15:24-orch:cephadm-wip-adk4-testing-2024-07-15-1832-squid-distro-default-smithi no_nested_subset: false os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-adk4-testing-2024-07-15-1832-squid ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: mon pg warn min per osd: 0 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - OSD_DOWN - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - MON_DOWN - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - Degraded data redundancy sha1: a6ca461be7cc1aff3658c2c6df4cff975a24449a ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: a6ca461be7cc1aff3658c2c6df4cff975a24449a kclient: syntax: v1 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-adk4-testing-2024-07-15-1832-squid sha1: a6ca461be7cc1aff3658c2c6df4cff975a24449a owner: scheduled_adking@teuthology priority: 80 repo: https://git.ceph.com/ceph-ci.git roles: - - host.a - client.0 - osd.0 - osd.1 - osd.2 - - host.b - client.1 - osd.3 - osd.4 - osd.5 seed: 6515 sha1: a6ca461be7cc1aff3658c2c6df4cff975a24449a sleep_before_teardown: 0 subset: 1/20 suite: orch:cephadm suite_branch: wip-adk4-testing-2024-07-15-1832-squid suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_a6ca461be7cc1aff3658c2c6df4cff975a24449a/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: a6ca461be7cc1aff3658c2c6df4cff975a24449a targets: smithi156.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOvlhZHzEDyxvHU7qyiCNw3TeqRv4LSeEIJpX+bteQr2mMFYfaHFdT67f5zpbYAIt8M1pEVdgneerL0/HwhkkgA= smithi176.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHfZ12J58ssRHHH+DpEgzYXNjiytsdRKuktT8725yO8/JTFjP0yaYpefyXfwJ/aif7dN79pRvReAV4BVnGTZAqw= tasks: - install: branch: quincy exclude_packages: - ceph-volume - print: '**** done install task...' - cephadm: cephadm_branch: quincy cephadm_git_url: https://github.com/ceph/ceph conf: osd: osd_class_default_list: '*' osd_class_load_list: '*' image: quay.ceph.io/ceph-ci/ceph:quincy roleless: true - print: '**** done end installing quincy cephadm ...' - cephadm.shell: host.a: - ceph config set mgr mgr/cephadm/use_repo_digest true --force - print: '**** done cephadm.shell ceph config set mgr...' - cephadm.shell: host.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: host.a: - ceph fs volume create cephfs --placement=4 - ceph fs dump - cephadm.shell: host.a: - ceph fs set cephfs max_mds 2 - cephadm.shell: host.a: - ceph fs set cephfs allow_standby_replay true - cephadm.shell: host.a: - ceph fs set cephfs inline_data false - cephadm.shell: host.a: - ceph fs dump - ceph --format=json fs dump | jq -e ".filesystems | length == 1" - while ! ceph --format=json mds versions | jq -e ". | add == 4"; do sleep 1; done - fs.pre_upgrade_save: null - ceph-fuse: null - print: '**** done client' - parallel: - upgrade-tasks - workload-tasks - cephadm.shell: host.a: - ceph fs dump - fs.post_upgrade_checks: null teuthology: fragments_dropped: - /cephfs/home/adking/src/git.ceph.com_ceph-c_wip-adk4-testing-2024-07-15-1832-squid/qa/suites/orch/cephadm/mds_upgrade_sequence/tasks/3-upgrade-mgr-staggered.yaml meta: {} postmerge: [] variables: fail_fs: false teuthology_branch: main teuthology_sha1: 5bc696040f54eb999ca02a20c94bfd735f22b93c timestamp: 2024-07-16_04:15:24 tube: smithi upgrade-tasks: sequential: - cephadm.shell: env: - sha1 host.a: - ceph config set mgr mgr/orchestrator/fail_fs false || true - cephadm.shell: env: - sha1 host.a: - ceph config set mon mon_warn_on_insecure_global_id_reclaim false --force - ceph config set mon mon_warn_on_insecure_global_id_reclaim_allowed false --force - ceph config set global log_to_journald false --force - ceph orch upgrade start --image quay.ceph.io/ceph-ci/ceph:$sha1 - cephadm.shell: env: - sha1 host.a: - while ceph orch upgrade status | jq '.in_progress' | grep true && ! ceph orch upgrade status | jq '.message' | grep Error ; do ceph orch ps ; ceph versions ; ceph fs dump; ceph orch upgrade status ; ceph health detail ; sleep 30 ; done - ceph orch ps - ceph orch upgrade status - ceph health detail - ceph versions - echo "wait for servicemap items w/ changing names to refresh" - sleep 60 - ceph orch ps - ceph versions - ceph versions | jq -e '.overall | length == 1' - ceph versions | jq -e '.overall | keys' | grep $sha1 user: adking verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.621193 workload-tasks: sequential: - workunit: clients: all: - suites/fsstress.sh 2024-07-16T09:35:40.831 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_a6ca461be7cc1aff3658c2c6df4cff975a24449a/qa; will attempt to use it 2024-07-16T09:35:40.832 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_a6ca461be7cc1aff3658c2c6df4cff975a24449a/qa/tasks 2024-07-16T09:35:40.832 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-07-16T09:35:40.834 INFO:teuthology.task.internal:Checking packages... 2024-07-16T09:35:40.857 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'a6ca461be7cc1aff3658c2c6df4cff975a24449a' 2024-07-16T09:35:40.858 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-07-16T09:35:40.858 INFO:teuthology.packaging:ref: None 2024-07-16T09:35:40.858 INFO:teuthology.packaging:tag: None 2024-07-16T09:35:40.858 INFO:teuthology.packaging:branch: wip-adk4-testing-2024-07-15-1832-squid 2024-07-16T09:35:40.858 INFO:teuthology.packaging:sha1: a6ca461be7cc1aff3658c2c6df4cff975a24449a 2024-07-16T09:35:40.858 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-adk4-testing-2024-07-15-1832-squid 2024-07-16T09:35:40.995 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-3495.ga6ca461b 2024-07-16T09:35:40.996 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-07-16T09:35:41.002 INFO:teuthology.task.internal:no buildpackages task found 2024-07-16T09:35:41.002 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-07-16T09:35:41.007 INFO:teuthology.task.internal:Saving configuration 2024-07-16T09:35:41.027 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-07-16T09:35:41.033 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-07-16T09:35:41.059 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi156.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/adking-2024-07-16_04:15:24-orch:cephadm-wip-adk4-testing-2024-07-15-1832-squid-distro-default-smithi/7803693', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-07-16 09:30:37.000549', 'locked_by': 'scheduled_adking@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOvlhZHzEDyxvHU7qyiCNw3TeqRv4LSeEIJpX+bteQr2mMFYfaHFdT67f5zpbYAIt8M1pEVdgneerL0/HwhkkgA='} 2024-07-16T09:35:41.081 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi176.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/adking-2024-07-16_04:15:24-orch:cephadm-wip-adk4-testing-2024-07-15-1832-squid-distro-default-smithi/7803693', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-07-16 09:30:36.998807', 'locked_by': 'scheduled_adking@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHfZ12J58ssRHHH+DpEgzYXNjiytsdRKuktT8725yO8/JTFjP0yaYpefyXfwJ/aif7dN79pRvReAV4BVnGTZAqw='} 2024-07-16T09:35:41.082 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-07-16T09:35:41.087 INFO:teuthology.task.internal:roles: ubuntu@smithi156.front.sepia.ceph.com - ['host.a', 'client.0', 'osd.0', 'osd.1', 'osd.2'] 2024-07-16T09:35:41.088 INFO:teuthology.task.internal:roles: ubuntu@smithi176.front.sepia.ceph.com - ['host.b', 'client.1', 'osd.3', 'osd.4', 'osd.5'] 2024-07-16T09:35:41.088 INFO:teuthology.run_tasks:Running task console_log... 2024-07-16T09:35:41.222 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f68f1543ac0>, signals=[15]) 2024-07-16T09:35:41.222 INFO:teuthology.run_tasks:Running task internal.connect... 2024-07-16T09:35:41.228 INFO:teuthology.task.internal:Opening connections... 2024-07-16T09:35:41.228 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi156.front.sepia.ceph.com 2024-07-16T09:35:41.229 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi156.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-07-16T09:35:41.314 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi176.front.sepia.ceph.com 2024-07-16T09:35:41.315 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi176.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-07-16T09:35:41.388 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-07-16T09:35:41.395 DEBUG:teuthology.orchestra.run.smithi156:> uname -m 2024-07-16T09:35:41.413 INFO:teuthology.orchestra.run.smithi156.stdout:x86_64 2024-07-16T09:35:41.413 DEBUG:teuthology.orchestra.run.smithi156:> cat /etc/os-release 2024-07-16T09:35:41.471 INFO:teuthology.orchestra.run.smithi156.stdout:NAME="CentOS Stream" 2024-07-16T09:35:41.471 INFO:teuthology.orchestra.run.smithi156.stdout:VERSION="9" 2024-07-16T09:35:41.471 INFO:teuthology.orchestra.run.smithi156.stdout:ID="centos" 2024-07-16T09:35:41.471 INFO:teuthology.orchestra.run.smithi156.stdout:ID_LIKE="rhel fedora" 2024-07-16T09:35:41.471 INFO:teuthology.orchestra.run.smithi156.stdout:VERSION_ID="9" 2024-07-16T09:35:41.472 INFO:teuthology.orchestra.run.smithi156.stdout:PLATFORM_ID="platform:el9" 2024-07-16T09:35:41.472 INFO:teuthology.orchestra.run.smithi156.stdout:PRETTY_NAME="CentOS Stream 9" 2024-07-16T09:35:41.472 INFO:teuthology.orchestra.run.smithi156.stdout:ANSI_COLOR="0;31" 2024-07-16T09:35:41.472 INFO:teuthology.orchestra.run.smithi156.stdout:LOGO="fedora-logo-icon" 2024-07-16T09:35:41.472 INFO:teuthology.orchestra.run.smithi156.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-07-16T09:35:41.472 INFO:teuthology.orchestra.run.smithi156.stdout:HOME_URL="https://centos.org/" 2024-07-16T09:35:41.473 INFO:teuthology.orchestra.run.smithi156.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-07-16T09:35:41.473 INFO:teuthology.orchestra.run.smithi156.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-07-16T09:35:41.473 INFO:teuthology.orchestra.run.smithi156.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-07-16T09:35:41.473 INFO:teuthology.lock.ops:Updating smithi156.front.sepia.ceph.com on lock server 2024-07-16T09:35:41.499 DEBUG:teuthology.orchestra.run.smithi176:> uname -m 2024-07-16T09:35:41.517 INFO:teuthology.orchestra.run.smithi176.stdout:x86_64 2024-07-16T09:35:41.517 DEBUG:teuthology.orchestra.run.smithi176:> cat /etc/os-release 2024-07-16T09:35:41.574 INFO:teuthology.orchestra.run.smithi176.stdout:NAME="CentOS Stream" 2024-07-16T09:35:41.574 INFO:teuthology.orchestra.run.smithi176.stdout:VERSION="9" 2024-07-16T09:35:41.575 INFO:teuthology.orchestra.run.smithi176.stdout:ID="centos" 2024-07-16T09:35:41.575 INFO:teuthology.orchestra.run.smithi176.stdout:ID_LIKE="rhel fedora" 2024-07-16T09:35:41.575 INFO:teuthology.orchestra.run.smithi176.stdout:VERSION_ID="9" 2024-07-16T09:35:41.575 INFO:teuthology.orchestra.run.smithi176.stdout:PLATFORM_ID="platform:el9" 2024-07-16T09:35:41.575 INFO:teuthology.orchestra.run.smithi176.stdout:PRETTY_NAME="CentOS Stream 9" 2024-07-16T09:35:41.575 INFO:teuthology.orchestra.run.smithi176.stdout:ANSI_COLOR="0;31" 2024-07-16T09:35:41.575 INFO:teuthology.orchestra.run.smithi176.stdout:LOGO="fedora-logo-icon" 2024-07-16T09:35:41.575 INFO:teuthology.orchestra.run.smithi176.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-07-16T09:35:41.575 INFO:teuthology.orchestra.run.smithi176.stdout:HOME_URL="https://centos.org/" 2024-07-16T09:35:41.575 INFO:teuthology.orchestra.run.smithi176.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-07-16T09:35:41.575 INFO:teuthology.orchestra.run.smithi176.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-07-16T09:35:41.575 INFO:teuthology.orchestra.run.smithi176.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-07-16T09:35:41.576 INFO:teuthology.lock.ops:Updating smithi176.front.sepia.ceph.com on lock server 2024-07-16T09:35:41.599 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-07-16T09:35:41.608 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-07-16T09:35:41.614 INFO:teuthology.task.internal:Checking for old test directory... 2024-07-16T09:35:41.614 DEBUG:teuthology.orchestra.run.smithi156:> test '!' -e /home/ubuntu/cephtest 2024-07-16T09:35:41.616 DEBUG:teuthology.orchestra.run.smithi176:> test '!' -e /home/ubuntu/cephtest 2024-07-16T09:35:41.632 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-07-16T09:35:41.638 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-07-16T09:35:41.639 DEBUG:teuthology.orchestra.run.smithi156:> test -z $(ls -A /var/lib/ceph) 2024-07-16T09:35:41.673 DEBUG:teuthology.orchestra.run.smithi176:> test -z $(ls -A /var/lib/ceph) 2024-07-16T09:35:41.768 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-07-16T09:35:41.820 INFO:teuthology.run_tasks:Running task kernel... 2024-07-16T09:35:41.834 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-07-16T09:35:41.834 INFO:teuthology.task.kernel:config {'host.a': {'kdb': 1, 'sha1': 'distro'}, 'host.b': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-07-16T09:35:41.834 DEBUG:teuthology.orchestra.run.smithi156:> test -f /run/.containerenv -o -f /.dockerenv 2024-07-16T09:35:41.835 DEBUG:teuthology.orchestra.run.smithi176:> test -f /run/.containerenv -o -f /.dockerenv 2024-07-16T09:35:41.851 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-07-16T09:35:41.851 DEBUG:teuthology.orchestra.run.smithi156:> uname -r 2024-07-16T09:35:41.852 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-07-16T09:35:41.852 DEBUG:teuthology.orchestra.run.smithi176:> uname -r 2024-07-16T09:35:41.909 INFO:teuthology.orchestra.run.smithi156.stdout:5.14.0-437.el9.x86_64 2024-07-16T09:35:41.909 INFO:teuthology.task.kernel:Running kernel on smithi156: 5.14.0-437.el9.x86_64 2024-07-16T09:35:41.909 DEBUG:teuthology.orchestra.run.smithi156:> sudo yum install -y kernel 2024-07-16T09:35:41.910 INFO:teuthology.orchestra.run.smithi176.stdout:5.14.0-437.el9.x86_64 2024-07-16T09:35:41.910 INFO:teuthology.task.kernel:Running kernel on smithi176: 5.14.0-437.el9.x86_64 2024-07-16T09:35:41.910 DEBUG:teuthology.orchestra.run.smithi176:> sudo yum install -y kernel 2024-07-16T09:35:44.389 INFO:teuthology.orchestra.run.smithi156.stdout:CentOS Stream 9 - BaseOS 9.9 MB/s | 8.2 MB 00:00 2024-07-16T09:35:46.761 INFO:teuthology.orchestra.run.smithi176.stdout:CentOS Stream 9 - BaseOS 2.5 MB/s | 8.2 MB 00:03 2024-07-16T09:35:49.680 INFO:teuthology.orchestra.run.smithi176.stdout:CentOS Stream 9 - AppStream 11 MB/s | 20 MB 00:01 2024-07-16T09:35:53.856 INFO:teuthology.orchestra.run.smithi156.stdout:CentOS Stream 9 - AppStream 2.4 MB/s | 20 MB 00:08 2024-07-16T09:35:55.387 INFO:teuthology.orchestra.run.smithi176.stdout:CentOS Stream 9 - CRB 3.4 MB/s | 6.5 MB 00:01 2024-07-16T09:35:57.004 INFO:teuthology.orchestra.run.smithi176.stdout:CentOS Stream 9 - Extras packages 68 kB/s | 17 kB 00:00 2024-07-16T09:35:58.081 INFO:teuthology.orchestra.run.smithi156.stdout:CentOS Stream 9 - CRB 16 MB/s | 6.5 MB 00:00 2024-07-16T09:35:58.099 INFO:teuthology.orchestra.run.smithi176.stdout:Extra Packages for Enterprise Linux 23 MB/s | 22 MB 00:00 2024-07-16T09:36:00.049 INFO:teuthology.orchestra.run.smithi156.stdout:CentOS Stream 9 - Extras packages 26 kB/s | 17 kB 00:00 2024-07-16T09:36:01.019 INFO:teuthology.orchestra.run.smithi156.stdout:Extra Packages for Enterprise Linux 27 MB/s | 22 MB 00:00 2024-07-16T09:36:03.413 INFO:teuthology.orchestra.run.smithi176.stdout:lab-extras 32 kB/s | 1.7 kB 00:00 2024-07-16T09:36:05.469 INFO:teuthology.orchestra.run.smithi176.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-07-16T09:36:05.470 INFO:teuthology.orchestra.run.smithi176.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-07-16T09:36:05.566 INFO:teuthology.orchestra.run.smithi176.stdout:Dependencies resolved. 2024-07-16T09:36:05.576 INFO:teuthology.orchestra.run.smithi176.stdout:================================================================================ 2024-07-16T09:36:05.576 INFO:teuthology.orchestra.run.smithi176.stdout: Package Architecture Version Repository Size 2024-07-16T09:36:05.576 INFO:teuthology.orchestra.run.smithi176.stdout:================================================================================ 2024-07-16T09:36:05.576 INFO:teuthology.orchestra.run.smithi176.stdout:Installing: 2024-07-16T09:36:05.577 INFO:teuthology.orchestra.run.smithi176.stdout: kernel x86_64 5.14.0-479.el9 baseos 1.5 M 2024-07-16T09:36:05.577 INFO:teuthology.orchestra.run.smithi176.stdout:Installing dependencies: 2024-07-16T09:36:05.577 INFO:teuthology.orchestra.run.smithi176.stdout: kernel-core x86_64 5.14.0-479.el9 baseos 17 M 2024-07-16T09:36:05.577 INFO:teuthology.orchestra.run.smithi176.stdout: kernel-modules x86_64 5.14.0-479.el9 baseos 36 M 2024-07-16T09:36:05.577 INFO:teuthology.orchestra.run.smithi176.stdout: kernel-modules-core x86_64 5.14.0-479.el9 baseos 30 M 2024-07-16T09:36:05.577 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-07-16T09:36:05.577 INFO:teuthology.orchestra.run.smithi176.stdout:Transaction Summary 2024-07-16T09:36:05.577 INFO:teuthology.orchestra.run.smithi176.stdout:================================================================================ 2024-07-16T09:36:05.577 INFO:teuthology.orchestra.run.smithi176.stdout:Install 4 Packages 2024-07-16T09:36:05.577 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-07-16T09:36:05.578 INFO:teuthology.orchestra.run.smithi176.stdout:Total download size: 84 M 2024-07-16T09:36:05.578 INFO:teuthology.orchestra.run.smithi176.stdout:Installed size: 126 M 2024-07-16T09:36:05.578 INFO:teuthology.orchestra.run.smithi176.stdout:Downloading Packages: 2024-07-16T09:36:06.400 INFO:teuthology.orchestra.run.smithi156.stdout:lab-extras 43 kB/s | 1.7 kB 00:00 2024-07-16T09:36:06.498 INFO:teuthology.orchestra.run.smithi176.stdout:(1/4): kernel-5.14.0-479.el9.x86_64.rpm 1.8 MB/s | 1.5 MB 00:00 2024-07-16T09:36:07.190 INFO:teuthology.orchestra.run.smithi176.stdout:(2/4): kernel-core-5.14.0-479.el9.x86_64.rpm 11 MB/s | 17 MB 00:01 2024-07-16T09:36:07.848 INFO:teuthology.orchestra.run.smithi176.stdout:(3/4): kernel-modules-core-5.14.0-479.el9.x86_6 22 MB/s | 30 MB 00:01 2024-07-16T09:36:08.182 INFO:teuthology.orchestra.run.smithi176.stdout:(4/4): kernel-modules-5.14.0-479.el9.x86_64.rpm 14 MB/s | 36 MB 00:02 2024-07-16T09:36:08.185 INFO:teuthology.orchestra.run.smithi176.stdout:-------------------------------------------------------------------------------- 2024-07-16T09:36:08.185 INFO:teuthology.orchestra.run.smithi176.stdout:Total 32 MB/s | 84 MB 00:02 2024-07-16T09:36:08.448 INFO:teuthology.orchestra.run.smithi156.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-07-16T09:36:08.448 INFO:teuthology.orchestra.run.smithi156.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-07-16T09:36:08.525 INFO:teuthology.orchestra.run.smithi156.stdout:Dependencies resolved. 2024-07-16T09:36:08.534 INFO:teuthology.orchestra.run.smithi156.stdout:================================================================================ 2024-07-16T09:36:08.534 INFO:teuthology.orchestra.run.smithi156.stdout: Package Architecture Version Repository Size 2024-07-16T09:36:08.534 INFO:teuthology.orchestra.run.smithi156.stdout:================================================================================ 2024-07-16T09:36:08.534 INFO:teuthology.orchestra.run.smithi156.stdout:Installing: 2024-07-16T09:36:08.534 INFO:teuthology.orchestra.run.smithi156.stdout: kernel x86_64 5.14.0-479.el9 baseos 1.5 M 2024-07-16T09:36:08.534 INFO:teuthology.orchestra.run.smithi156.stdout:Installing dependencies: 2024-07-16T09:36:08.534 INFO:teuthology.orchestra.run.smithi156.stdout: kernel-core x86_64 5.14.0-479.el9 baseos 17 M 2024-07-16T09:36:08.534 INFO:teuthology.orchestra.run.smithi156.stdout: kernel-modules x86_64 5.14.0-479.el9 baseos 36 M 2024-07-16T09:36:08.534 INFO:teuthology.orchestra.run.smithi156.stdout: kernel-modules-core x86_64 5.14.0-479.el9 baseos 30 M 2024-07-16T09:36:08.535 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-07-16T09:36:08.535 INFO:teuthology.orchestra.run.smithi156.stdout:Transaction Summary 2024-07-16T09:36:08.535 INFO:teuthology.orchestra.run.smithi156.stdout:================================================================================ 2024-07-16T09:36:08.535 INFO:teuthology.orchestra.run.smithi156.stdout:Install 4 Packages 2024-07-16T09:36:08.535 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-07-16T09:36:08.535 INFO:teuthology.orchestra.run.smithi156.stdout:Total download size: 84 M 2024-07-16T09:36:08.536 INFO:teuthology.orchestra.run.smithi156.stdout:Installed size: 126 M 2024-07-16T09:36:08.536 INFO:teuthology.orchestra.run.smithi156.stdout:Downloading Packages: 2024-07-16T09:36:08.924 INFO:teuthology.orchestra.run.smithi176.stdout:Running transaction check 2024-07-16T09:36:09.254 INFO:teuthology.orchestra.run.smithi176.stdout:Transaction check succeeded. 2024-07-16T09:36:09.254 INFO:teuthology.orchestra.run.smithi176.stdout:Running transaction test 2024-07-16T09:36:09.476 INFO:teuthology.orchestra.run.smithi156.stdout:(1/4): kernel-5.14.0-479.el9.x86_64.rpm 1.7 MB/s | 1.5 MB 00:00 2024-07-16T09:36:10.160 INFO:teuthology.orchestra.run.smithi156.stdout:(2/4): kernel-core-5.14.0-479.el9.x86_64.rpm 11 MB/s | 17 MB 00:01 2024-07-16T09:36:10.270 INFO:teuthology.orchestra.run.smithi176.stdout:Transaction test succeeded. 2024-07-16T09:36:10.271 INFO:teuthology.orchestra.run.smithi176.stdout:Running transaction 2024-07-16T09:36:11.160 INFO:teuthology.orchestra.run.smithi156.stdout:(3/4): kernel-modules-5.14.0-479.el9.x86_64.rpm 14 MB/s | 36 MB 00:02 2024-07-16T09:36:11.435 INFO:teuthology.orchestra.run.smithi156.stdout:(4/4): kernel-modules-core-5.14.0-479.el9.x86_6 15 MB/s | 30 MB 00:01 2024-07-16T09:36:11.438 INFO:teuthology.orchestra.run.smithi156.stdout:-------------------------------------------------------------------------------- 2024-07-16T09:36:11.438 INFO:teuthology.orchestra.run.smithi156.stdout:Total 29 MB/s | 84 MB 00:02 2024-07-16T09:36:12.173 INFO:teuthology.orchestra.run.smithi156.stdout:Running transaction check 2024-07-16T09:36:12.541 INFO:teuthology.orchestra.run.smithi156.stdout:Transaction check succeeded. 2024-07-16T09:36:12.542 INFO:teuthology.orchestra.run.smithi156.stdout:Running transaction test 2024-07-16T09:36:12.723 INFO:teuthology.orchestra.run.smithi176.stdout: Preparing : 1/1 2024-07-16T09:36:13.414 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : kernel-modules-core-5.14.0-479.el9.x86_64 1/4 2024-07-16T09:36:13.506 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : kernel-core-5.14.0-479.el9.x86_64 2/4 2024-07-16T09:36:13.548 INFO:teuthology.orchestra.run.smithi156.stdout:Transaction test succeeded. 2024-07-16T09:36:13.549 INFO:teuthology.orchestra.run.smithi156.stdout:Running transaction 2024-07-16T09:36:16.031 INFO:teuthology.orchestra.run.smithi156.stdout: Preparing : 1/1 2024-07-16T09:36:16.162 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: kernel-core-5.14.0-479.el9.x86_64 2/4 2024-07-16T09:36:16.298 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : kernel-modules-5.14.0-479.el9.x86_64 3/4 2024-07-16T09:36:16.730 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : kernel-modules-core-5.14.0-479.el9.x86_64 1/4 2024-07-16T09:36:16.836 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : kernel-core-5.14.0-479.el9.x86_64 2/4 2024-07-16T09:36:19.539 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: kernel-core-5.14.0-479.el9.x86_64 2/4 2024-07-16T09:36:19.708 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : kernel-modules-5.14.0-479.el9.x86_64 3/4 2024-07-16T09:36:21.163 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: kernel-modules-5.14.0-479.el9.x86_64 3/4 2024-07-16T09:36:21.273 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : kernel-5.14.0-479.el9.x86_64 4/4 2024-07-16T09:36:24.552 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: kernel-modules-5.14.0-479.el9.x86_64 3/4 2024-07-16T09:36:24.666 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : kernel-5.14.0-479.el9.x86_64 4/4 2024-07-16T09:36:26.155 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: kernel-modules-core-5.14.0-479.el9.x86_64 4/4 2024-07-16T09:36:29.522 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: kernel-modules-core-5.14.0-479.el9.x86_64 4/4 2024-07-16T09:36:55.911 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: kernel-core-5.14.0-479.el9.x86_64 4/4 2024-07-16T09:36:55.952 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: kernel-modules-5.14.0-479.el9.x86_64 4/4 2024-07-16T09:36:56.303 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: kernel-5.14.0-479.el9.x86_64 4/4 2024-07-16T09:36:56.303 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : kernel-5.14.0-479.el9.x86_64 1/4 2024-07-16T09:36:56.303 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : kernel-core-5.14.0-479.el9.x86_64 2/4 2024-07-16T09:36:56.303 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : kernel-modules-5.14.0-479.el9.x86_64 3/4 2024-07-16T09:36:57.136 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : kernel-modules-core-5.14.0-479.el9.x86_64 4/4 2024-07-16T09:36:57.136 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-07-16T09:36:57.136 INFO:teuthology.orchestra.run.smithi176.stdout:Installed: 2024-07-16T09:36:57.136 INFO:teuthology.orchestra.run.smithi176.stdout: kernel-5.14.0-479.el9.x86_64 2024-07-16T09:36:57.137 INFO:teuthology.orchestra.run.smithi176.stdout: kernel-core-5.14.0-479.el9.x86_64 2024-07-16T09:36:57.137 INFO:teuthology.orchestra.run.smithi176.stdout: kernel-modules-5.14.0-479.el9.x86_64 2024-07-16T09:36:57.137 INFO:teuthology.orchestra.run.smithi176.stdout: kernel-modules-core-5.14.0-479.el9.x86_64 2024-07-16T09:36:57.137 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-07-16T09:36:57.137 INFO:teuthology.orchestra.run.smithi176.stdout:Complete! 2024-07-16T09:36:57.587 DEBUG:teuthology.orchestra.run.smithi176:> rpm -q kernel | sort -rV | head -n 1 2024-07-16T09:36:57.680 INFO:teuthology.orchestra.run.smithi176.stdout:kernel-5.14.0-479.el9.x86_64 2024-07-16T09:36:57.681 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-479.el9.x86_64 2024-07-16T09:36:57.681 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-437.el9.x86_64 Expected: 5.14.0-479.el9.x86_64 2024-07-16T09:36:57.681 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-07-16T09:36:57.681 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-07-16T09:36:57.681 INFO:teuthology.task.kernel:Installing distro kernel on host.b... 2024-07-16T09:36:57.681 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi176.front.sepia.ceph.com, path=None, version=distro) 2024-07-16T09:36:57.681 DEBUG:teuthology.orchestra.run.smithi176:> rpm -q kernel | sort -rV | head -n 1 2024-07-16T09:36:57.737 INFO:teuthology.orchestra.run.smithi176.stdout:kernel-5.14.0-479.el9.x86_64 2024-07-16T09:36:57.738 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-479.el9.x86_64 2024-07-16T09:36:57.738 DEBUG:teuthology.orchestra.run.smithi176:> sudo rpm -qi grub2-tools 2024-07-16T09:36:57.768 INFO:teuthology.orchestra.run.smithi176.stdout:Name : grub2-tools 2024-07-16T09:36:57.768 INFO:teuthology.orchestra.run.smithi176.stdout:Epoch : 1 2024-07-16T09:36:57.768 INFO:teuthology.orchestra.run.smithi176.stdout:Version : 2.06 2024-07-16T09:36:57.768 INFO:teuthology.orchestra.run.smithi176.stdout:Release : 68.el9 2024-07-16T09:36:57.768 INFO:teuthology.orchestra.run.smithi176.stdout:Architecture: x86_64 2024-07-16T09:36:57.768 INFO:teuthology.orchestra.run.smithi176.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-07-16T09:36:57.769 INFO:teuthology.orchestra.run.smithi176.stdout:Group : Unspecified 2024-07-16T09:36:57.769 INFO:teuthology.orchestra.run.smithi176.stdout:Size : 8264254 2024-07-16T09:36:57.769 INFO:teuthology.orchestra.run.smithi176.stdout:License : GPLv3+ 2024-07-16T09:36:57.769 INFO:teuthology.orchestra.run.smithi176.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-07-16T09:36:57.769 INFO:teuthology.orchestra.run.smithi176.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-07-16T09:36:57.769 INFO:teuthology.orchestra.run.smithi176.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-07-16T09:36:57.769 INFO:teuthology.orchestra.run.smithi176.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-07-16T09:36:57.769 INFO:teuthology.orchestra.run.smithi176.stdout:Packager : builder@centos.org 2024-07-16T09:36:57.769 INFO:teuthology.orchestra.run.smithi176.stdout:Vendor : CentOS 2024-07-16T09:36:57.769 INFO:teuthology.orchestra.run.smithi176.stdout:URL : http://www.gnu.org/software/grub/ 2024-07-16T09:36:57.769 INFO:teuthology.orchestra.run.smithi176.stdout:Summary : Support tools for GRUB. 2024-07-16T09:36:57.769 INFO:teuthology.orchestra.run.smithi176.stdout:Description : 2024-07-16T09:36:57.769 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-07-16T09:36:57.770 INFO:teuthology.orchestra.run.smithi176.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-07-16T09:36:57.770 INFO:teuthology.orchestra.run.smithi176.stdout:customizable bootloader with modular architecture. It supports a rich 2024-07-16T09:36:57.770 INFO:teuthology.orchestra.run.smithi176.stdout:variety of kernel formats, file systems, computer architectures and 2024-07-16T09:36:57.770 INFO:teuthology.orchestra.run.smithi176.stdout:hardware devices. 2024-07-16T09:36:57.770 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-07-16T09:36:57.770 INFO:teuthology.orchestra.run.smithi176.stdout:This subpackage provides tools for support of all platforms. 2024-07-16T09:36:57.771 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-07-16T09:36:57.771 INFO:teuthology.task.kernel:Updating grub on smithi176 to boot 5.14.0-479.el9.x86_64 2024-07-16T09:36:57.771 DEBUG:teuthology.orchestra.run.smithi176:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-07-16T09:36:58.442 INFO:teuthology.orchestra.run.smithi176.stderr:Generating grub configuration file ... 2024-07-16T09:36:59.338 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: kernel-core-5.14.0-479.el9.x86_64 4/4 2024-07-16T09:36:59.379 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: kernel-modules-5.14.0-479.el9.x86_64 4/4 2024-07-16T09:36:59.754 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: kernel-5.14.0-479.el9.x86_64 4/4 2024-07-16T09:36:59.754 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : kernel-5.14.0-479.el9.x86_64 1/4 2024-07-16T09:36:59.754 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : kernel-core-5.14.0-479.el9.x86_64 2/4 2024-07-16T09:36:59.754 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : kernel-modules-5.14.0-479.el9.x86_64 3/4 2024-07-16T09:36:59.784 INFO:teuthology.orchestra.run.smithi176.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-07-16T09:36:59.818 INFO:teuthology.orchestra.run.smithi176.stderr:done 2024-07-16T09:36:59.820 DEBUG:teuthology.orchestra.run.smithi176:> mktemp 2024-07-16T09:36:59.841 INFO:teuthology.orchestra.run.smithi176.stdout:/tmp/tmp.Yt7sEzaB19 2024-07-16T09:36:59.841 DEBUG:teuthology.orchestra.run.smithi176:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.Yt7sEzaB19 2024-07-16T09:36:59.907 DEBUG:teuthology.orchestra.run.smithi176:> sudo chmod 0666 /tmp/tmp.Yt7sEzaB19 2024-07-16T09:37:00.042 DEBUG:teuthology.orchestra.remote:smithi176:/tmp/tmp.Yt7sEzaB19 is 6KB 2024-07-16T09:37:00.089 DEBUG:teuthology.orchestra.run.smithi176:> rm -fr /tmp/tmp.Yt7sEzaB19 2024-07-16T09:37:00.104 DEBUG:teuthology.orchestra.run.smithi176:> sudo /bin/ls /boot/loader/entries || true 2024-07-16T09:37:00.165 INFO:teuthology.orchestra.run.smithi176.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-07-16T09:37:00.165 INFO:teuthology.orchestra.run.smithi176.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-07-16T09:37:00.166 INFO:teuthology.orchestra.run.smithi176.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-07-16T09:37:00.166 INFO:teuthology.orchestra.run.smithi176.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-479.el9.x86_64.conf 2024-07-16T09:37:00.167 DEBUG:teuthology.orchestra.run.smithi176:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-5.14.0-479.el9.x86_64 2024-07-16T09:37:00.318 DEBUG:teuthology.orchestra.run.smithi176:> sudo shutdown -r now 2024-07-16T09:37:00.554 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : kernel-modules-core-5.14.0-479.el9.x86_64 4/4 2024-07-16T09:37:00.555 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-07-16T09:37:00.555 INFO:teuthology.orchestra.run.smithi156.stdout:Installed: 2024-07-16T09:37:00.555 INFO:teuthology.orchestra.run.smithi156.stdout: kernel-5.14.0-479.el9.x86_64 2024-07-16T09:37:00.555 INFO:teuthology.orchestra.run.smithi156.stdout: kernel-core-5.14.0-479.el9.x86_64 2024-07-16T09:37:00.555 INFO:teuthology.orchestra.run.smithi156.stdout: kernel-modules-5.14.0-479.el9.x86_64 2024-07-16T09:37:00.555 INFO:teuthology.orchestra.run.smithi156.stdout: kernel-modules-core-5.14.0-479.el9.x86_64 2024-07-16T09:37:00.555 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-07-16T09:37:00.555 INFO:teuthology.orchestra.run.smithi156.stdout:Complete! 2024-07-16T09:37:00.965 DEBUG:teuthology.orchestra.run.smithi156:> rpm -q kernel | sort -rV | head -n 1 2024-07-16T09:37:01.043 INFO:teuthology.orchestra.run.smithi156.stdout:kernel-5.14.0-479.el9.x86_64 2024-07-16T09:37:01.044 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-479.el9.x86_64 2024-07-16T09:37:01.044 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-437.el9.x86_64 Expected: 5.14.0-479.el9.x86_64 2024-07-16T09:37:01.044 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-07-16T09:37:01.044 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-07-16T09:37:01.044 INFO:teuthology.task.kernel:Installing distro kernel on host.a... 2024-07-16T09:37:01.044 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi156.front.sepia.ceph.com, path=None, version=distro) 2024-07-16T09:37:01.044 DEBUG:teuthology.orchestra.run.smithi156:> rpm -q kernel | sort -rV | head -n 1 2024-07-16T09:37:01.101 INFO:teuthology.orchestra.run.smithi156.stdout:kernel-5.14.0-479.el9.x86_64 2024-07-16T09:37:01.101 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-479.el9.x86_64 2024-07-16T09:37:01.101 DEBUG:teuthology.orchestra.run.smithi156:> sudo rpm -qi grub2-tools 2024-07-16T09:37:01.139 INFO:teuthology.orchestra.run.smithi156.stdout:Name : grub2-tools 2024-07-16T09:37:01.139 INFO:teuthology.orchestra.run.smithi156.stdout:Epoch : 1 2024-07-16T09:37:01.139 INFO:teuthology.orchestra.run.smithi156.stdout:Version : 2.06 2024-07-16T09:37:01.139 INFO:teuthology.orchestra.run.smithi156.stdout:Release : 68.el9 2024-07-16T09:37:01.139 INFO:teuthology.orchestra.run.smithi156.stdout:Architecture: x86_64 2024-07-16T09:37:01.140 INFO:teuthology.orchestra.run.smithi156.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-07-16T09:37:01.140 INFO:teuthology.orchestra.run.smithi156.stdout:Group : Unspecified 2024-07-16T09:37:01.140 INFO:teuthology.orchestra.run.smithi156.stdout:Size : 8264254 2024-07-16T09:37:01.140 INFO:teuthology.orchestra.run.smithi156.stdout:License : GPLv3+ 2024-07-16T09:37:01.140 INFO:teuthology.orchestra.run.smithi156.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-07-16T09:37:01.140 INFO:teuthology.orchestra.run.smithi156.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-07-16T09:37:01.140 INFO:teuthology.orchestra.run.smithi156.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-07-16T09:37:01.140 INFO:teuthology.orchestra.run.smithi156.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-07-16T09:37:01.140 INFO:teuthology.orchestra.run.smithi156.stdout:Packager : builder@centos.org 2024-07-16T09:37:01.140 INFO:teuthology.orchestra.run.smithi156.stdout:Vendor : CentOS 2024-07-16T09:37:01.140 INFO:teuthology.orchestra.run.smithi156.stdout:URL : http://www.gnu.org/software/grub/ 2024-07-16T09:37:01.140 INFO:teuthology.orchestra.run.smithi156.stdout:Summary : Support tools for GRUB. 2024-07-16T09:37:01.140 INFO:teuthology.orchestra.run.smithi156.stdout:Description : 2024-07-16T09:37:01.140 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-07-16T09:37:01.141 INFO:teuthology.orchestra.run.smithi156.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-07-16T09:37:01.141 INFO:teuthology.orchestra.run.smithi156.stdout:customizable bootloader with modular architecture. It supports a rich 2024-07-16T09:37:01.141 INFO:teuthology.orchestra.run.smithi156.stdout:variety of kernel formats, file systems, computer architectures and 2024-07-16T09:37:01.141 INFO:teuthology.orchestra.run.smithi156.stdout:hardware devices. 2024-07-16T09:37:01.141 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-07-16T09:37:01.141 INFO:teuthology.orchestra.run.smithi156.stdout:This subpackage provides tools for support of all platforms. 2024-07-16T09:37:01.142 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-07-16T09:37:01.142 INFO:teuthology.task.kernel:Updating grub on smithi156 to boot 5.14.0-479.el9.x86_64 2024-07-16T09:37:01.142 DEBUG:teuthology.orchestra.run.smithi156:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-07-16T09:37:01.766 INFO:teuthology.orchestra.run.smithi156.stderr:Generating grub configuration file ... 2024-07-16T09:37:03.197 INFO:teuthology.orchestra.run.smithi156.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-07-16T09:37:03.227 INFO:teuthology.orchestra.run.smithi156.stderr:done 2024-07-16T09:37:03.229 DEBUG:teuthology.orchestra.run.smithi156:> mktemp 2024-07-16T09:37:03.254 INFO:teuthology.orchestra.run.smithi156.stdout:/tmp/tmp.IcNGMjG3Qx 2024-07-16T09:37:03.255 DEBUG:teuthology.orchestra.run.smithi156:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.IcNGMjG3Qx 2024-07-16T09:37:03.319 DEBUG:teuthology.orchestra.run.smithi156:> sudo chmod 0666 /tmp/tmp.IcNGMjG3Qx 2024-07-16T09:37:03.462 DEBUG:teuthology.orchestra.remote:smithi156:/tmp/tmp.IcNGMjG3Qx is 6KB 2024-07-16T09:37:03.510 DEBUG:teuthology.orchestra.run.smithi156:> rm -fr /tmp/tmp.IcNGMjG3Qx 2024-07-16T09:37:03.524 DEBUG:teuthology.orchestra.run.smithi156:> sudo /bin/ls /boot/loader/entries || true 2024-07-16T09:37:03.588 INFO:teuthology.orchestra.run.smithi156.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-07-16T09:37:03.589 INFO:teuthology.orchestra.run.smithi156.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-07-16T09:37:03.589 INFO:teuthology.orchestra.run.smithi156.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-07-16T09:37:03.589 INFO:teuthology.orchestra.run.smithi156.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-479.el9.x86_64.conf 2024-07-16T09:37:03.590 DEBUG:teuthology.orchestra.run.smithi156:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-5.14.0-479.el9.x86_64 2024-07-16T09:37:03.731 DEBUG:teuthology.orchestra.run.smithi156:> sudo shutdown -r now 2024-07-16T09:37:30.346 INFO:teuthology.task.kernel:Checking client host.b for new kernel version... 2024-07-16T09:37:30.347 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi176.front.sepia.ceph.com' 2024-07-16T09:37:30.348 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi176.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-07-16T09:37:33.738 INFO:teuthology.task.kernel:Checking client host.a for new kernel version... 2024-07-16T09:37:33.738 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi156.front.sepia.ceph.com' 2024-07-16T09:37:33.739 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi156.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-07-16T09:38:08.291 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.156 2024-07-16T09:38:17.298 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi156.front.sepia.ceph.com' 2024-07-16T09:38:17.299 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi156.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-07-16T09:38:30.366 DEBUG:teuthology.orchestra.remote:timed out 2024-07-16T09:38:35.678 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.156 2024-07-16T09:38:39.370 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi176.front.sepia.ceph.com' 2024-07-16T09:38:39.371 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi176.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-07-16T09:38:46.777 DEBUG:teuthology.orchestra.run.smithi176:> true 2024-07-16T09:38:47.176 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi176.front.sepia.ceph.com' 2024-07-16T09:38:47.176 INFO:teuthology.task.kernel:Checking kernel version of host.b, want "5.14.0-479.el9.x86_64"... 2024-07-16T09:38:47.177 DEBUG:teuthology.orchestra.run.smithi176:> uname -r 2024-07-16T09:38:47.191 INFO:teuthology.orchestra.run.smithi176.stdout:5.14.0-479.el9.x86_64 2024-07-16T09:38:47.191 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-479.el9.x86_64 vs 5.14.0-479.el9.x86_64 2024-07-16T09:38:47.191 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-07-16T09:38:47.192 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-07-16T09:38:47.679 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi156.front.sepia.ceph.com' 2024-07-16T09:38:47.680 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi156.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-07-16T09:38:48.192 INFO:teuthology.task.kernel:Enabling kdb on host.b... 2024-07-16T09:38:48.192 DEBUG:teuthology.orchestra.run.smithi176:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-07-16T09:38:48.286 INFO:teuthology.orchestra.run.smithi176.stdout:ttyS1 2024-07-16T09:38:48.333 DEBUG:teuthology.parallel:result is None 2024-07-16T09:38:49.002 DEBUG:teuthology.orchestra.run.smithi156:> true 2024-07-16T09:38:49.508 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi156.front.sepia.ceph.com' 2024-07-16T09:38:49.509 INFO:teuthology.task.kernel:Checking kernel version of host.a, want "5.14.0-479.el9.x86_64"... 2024-07-16T09:38:49.509 DEBUG:teuthology.orchestra.run.smithi156:> uname -r 2024-07-16T09:38:49.523 INFO:teuthology.orchestra.run.smithi156.stdout:5.14.0-479.el9.x86_64 2024-07-16T09:38:49.523 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-479.el9.x86_64 vs 5.14.0-479.el9.x86_64 2024-07-16T09:38:49.523 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-07-16T09:38:49.523 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-07-16T09:38:50.524 INFO:teuthology.task.kernel:Enabling kdb on host.a... 2024-07-16T09:38:50.524 DEBUG:teuthology.orchestra.run.smithi156:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-07-16T09:38:50.605 INFO:teuthology.orchestra.run.smithi156.stdout:ttyS1 2024-07-16T09:38:50.652 DEBUG:teuthology.parallel:result is None 2024-07-16T09:38:50.652 INFO:teuthology.run_tasks:Running task internal.base... 2024-07-16T09:38:50.661 INFO:teuthology.task.internal:Creating test directory... 2024-07-16T09:38:50.662 DEBUG:teuthology.orchestra.run.smithi156:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-07-16T09:38:50.664 DEBUG:teuthology.orchestra.run.smithi176:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-07-16T09:38:50.702 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-07-16T09:38:50.709 INFO:teuthology.run_tasks:Running task internal.archive... 2024-07-16T09:38:50.719 INFO:teuthology.task.internal:Creating archive directory... 2024-07-16T09:38:50.720 DEBUG:teuthology.orchestra.run.smithi156:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-07-16T09:38:50.722 DEBUG:teuthology.orchestra.run.smithi176:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-07-16T09:38:50.800 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-07-16T09:38:50.905 INFO:teuthology.task.internal:Enabling coredump saving... 2024-07-16T09:38:50.906 DEBUG:teuthology.orchestra.run.smithi156:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-07-16T09:38:50.908 DEBUG:teuthology.orchestra.run.smithi176:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-07-16T09:38:50.952 INFO:teuthology.orchestra.run.smithi156.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-07-16T09:38:50.963 INFO:teuthology.orchestra.run.smithi176.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-07-16T09:38:50.963 INFO:teuthology.orchestra.run.smithi156.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-07-16T09:38:50.975 INFO:teuthology.orchestra.run.smithi176.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-07-16T09:38:50.977 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-07-16T09:38:50.985 INFO:teuthology.task.internal:Configuring sudo... 2024-07-16T09:38:50.985 DEBUG:teuthology.orchestra.run.smithi156:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-07-16T09:38:50.987 DEBUG:teuthology.orchestra.run.smithi176:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-07-16T09:38:51.012 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-07-16T09:38:51.021 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-07-16T09:38:51.021 DEBUG:teuthology.orchestra.run.smithi156:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-07-16T09:38:51.053 DEBUG:teuthology.orchestra.run.smithi176:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-07-16T09:38:51.068 DEBUG:teuthology.orchestra.run.smithi156:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-07-16T09:38:51.143 DEBUG:teuthology.orchestra.run.smithi156:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-07-16T09:38:51.204 DEBUG:teuthology.orchestra.run.smithi156:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-07-16T09:38:51.286 DEBUG:teuthology.orchestra.run.smithi156:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-07-16T09:38:51.327 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2024-07-16T09:38:51.327 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-07-16T09:38:51.393 DEBUG:teuthology.orchestra.run.smithi176:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-07-16T09:38:51.426 DEBUG:teuthology.orchestra.run.smithi176:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-07-16T09:38:51.553 DEBUG:teuthology.orchestra.run.smithi176:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-07-16T09:38:51.592 DEBUG:teuthology.orchestra.run.smithi176:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-07-16T09:38:51.636 DEBUG:teuthology.orchestra.run.smithi176:> set -ex 2024-07-16T09:38:51.636 DEBUG:teuthology.orchestra.run.smithi176:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-07-16T09:38:51.701 DEBUG:teuthology.orchestra.run.smithi156:> sudo service rsyslog restart 2024-07-16T09:38:51.703 DEBUG:teuthology.orchestra.run.smithi176:> sudo service rsyslog restart 2024-07-16T09:38:51.741 INFO:teuthology.orchestra.run.smithi156.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-07-16T09:38:51.791 INFO:teuthology.orchestra.run.smithi176.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-07-16T09:38:52.028 INFO:teuthology.run_tasks:Running task internal.timer... 2024-07-16T09:38:52.035 INFO:teuthology.task.internal:Starting timer... 2024-07-16T09:38:52.035 INFO:teuthology.run_tasks:Running task pcp... 2024-07-16T09:38:52.045 INFO:teuthology.run_tasks:Running task selinux... 2024-07-16T09:38:52.054 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-07-16T09:38:52.054 DEBUG:teuthology.orchestra.run.smithi156:> sudo service auditd rotate 2024-07-16T09:38:52.138 INFO:teuthology.orchestra.run.smithi156.stdout:Rotating logs: 2024-07-16T09:38:52.140 DEBUG:teuthology.orchestra.run.smithi176:> sudo service auditd rotate 2024-07-16T09:38:52.209 INFO:teuthology.orchestra.run.smithi176.stdout:Rotating logs: 2024-07-16T09:38:52.211 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-07-16T09:38:52.211 DEBUG:teuthology.orchestra.run.smithi156:> /usr/sbin/getenforce 2024-07-16T09:38:52.245 INFO:teuthology.orchestra.run.smithi156.stdout:Permissive 2024-07-16T09:38:52.245 DEBUG:teuthology.orchestra.run.smithi176:> /usr/sbin/getenforce 2024-07-16T09:38:52.278 INFO:teuthology.orchestra.run.smithi176.stdout:Permissive 2024-07-16T09:38:52.278 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi156.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi176.front.sepia.ceph.com': 'permissive'} 2024-07-16T09:38:52.278 DEBUG:teuthology.orchestra.run.smithi156:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-07-16T09:38:52.310 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-07-16T09:38:52.310 DEBUG:teuthology.orchestra.run.smithi176:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-07-16T09:38:52.343 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-07-16T09:38:52.343 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-07-16T09:38:52.344 DEBUG:teuthology.orchestra.run.smithi156:> sudo /usr/sbin/setenforce permissive 2024-07-16T09:38:52.381 DEBUG:teuthology.orchestra.run.smithi176:> sudo /usr/sbin/setenforce permissive 2024-07-16T09:38:52.409 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-07-16T09:38:52.419 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-07-16T09:38:52.432 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-07-16T09:38:52.433 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi156.front.sepia.ceph.com,smithi176.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-07-16T09:43:19.998 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi156.front.sepia.ceph.com'), Remote(name='ubuntu@smithi176.front.sepia.ceph.com')] 2024-07-16T09:43:19.999 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi156.front.sepia.ceph.com' 2024-07-16T09:43:20.000 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi156.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-07-16T09:43:20.083 DEBUG:teuthology.orchestra.run.smithi156:> true 2024-07-16T09:43:20.161 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi156.front.sepia.ceph.com' 2024-07-16T09:43:20.162 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi176.front.sepia.ceph.com' 2024-07-16T09:43:20.162 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi176.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-07-16T09:43:20.239 DEBUG:teuthology.orchestra.run.smithi176:> true 2024-07-16T09:43:20.322 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi176.front.sepia.ceph.com' 2024-07-16T09:43:20.323 INFO:teuthology.run_tasks:Running task clock... 2024-07-16T09:43:20.333 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-07-16T09:43:20.333 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-07-16T09:43:20.333 DEBUG:teuthology.orchestra.run.smithi156:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-07-16T09:43:20.336 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-07-16T09:43:20.336 DEBUG:teuthology.orchestra.run.smithi176:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-07-16T09:43:20.363 INFO:teuthology.orchestra.run.smithi156.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-07-16T09:43:20.377 INFO:teuthology.orchestra.run.smithi156.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-07-16T09:43:20.392 INFO:teuthology.orchestra.run.smithi176.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-07-16T09:43:20.400 INFO:teuthology.orchestra.run.smithi156.stderr:sudo: ntpd: command not found 2024-07-16T09:43:20.407 INFO:teuthology.orchestra.run.smithi176.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-07-16T09:43:20.411 INFO:teuthology.orchestra.run.smithi156.stdout:506 Cannot talk to daemon 2024-07-16T09:43:20.425 INFO:teuthology.orchestra.run.smithi156.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-07-16T09:43:20.432 INFO:teuthology.orchestra.run.smithi176.stderr:sudo: ntpd: command not found 2024-07-16T09:43:20.440 INFO:teuthology.orchestra.run.smithi156.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-07-16T09:43:20.444 INFO:teuthology.orchestra.run.smithi176.stdout:506 Cannot talk to daemon 2024-07-16T09:43:20.457 INFO:teuthology.orchestra.run.smithi176.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-07-16T09:43:20.474 INFO:teuthology.orchestra.run.smithi176.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-07-16T09:43:20.493 INFO:teuthology.orchestra.run.smithi156.stderr:bash: line 1: ntpq: command not found 2024-07-16T09:43:20.497 INFO:teuthology.orchestra.run.smithi156.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-07-16T09:43:20.497 INFO:teuthology.orchestra.run.smithi156.stdout:=============================================================================== 2024-07-16T09:43:20.497 INFO:teuthology.orchestra.run.smithi156.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-07-16T09:43:20.497 INFO:teuthology.orchestra.run.smithi156.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-07-16T09:43:20.497 INFO:teuthology.orchestra.run.smithi156.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-07-16T09:43:20.497 INFO:teuthology.orchestra.run.smithi156.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-07-16T09:43:20.526 INFO:teuthology.orchestra.run.smithi176.stderr:bash: line 1: ntpq: command not found 2024-07-16T09:43:20.529 INFO:teuthology.orchestra.run.smithi176.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-07-16T09:43:20.529 INFO:teuthology.orchestra.run.smithi176.stdout:=============================================================================== 2024-07-16T09:43:20.530 INFO:teuthology.orchestra.run.smithi176.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-07-16T09:43:20.530 INFO:teuthology.orchestra.run.smithi176.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-07-16T09:43:20.530 INFO:teuthology.orchestra.run.smithi176.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-07-16T09:43:20.530 INFO:teuthology.orchestra.run.smithi176.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-07-16T09:43:20.530 INFO:teuthology.run_tasks:Running task install... 2024-07-16T09:43:20.538 DEBUG:teuthology.task.install:project ceph 2024-07-16T09:43:20.538 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'a6ca461be7cc1aff3658c2c6df4cff975a24449a'}} 2024-07-16T09:43:20.538 DEBUG:teuthology.task.install:config {'branch': 'quincy', 'exclude_packages': ['ceph-volume'], 'flavor': 'default', 'sha1': 'a6ca461be7cc1aff3658c2c6df4cff975a24449a'} 2024-07-16T09:43:20.538 INFO:teuthology.task.install:Using flavor: default 2024-07-16T09:43:20.544 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-07-16T09:43:20.544 INFO:teuthology.task.install:extra packages: [] 2024-07-16T09:43:20.544 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': 'quincy', 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': ['ceph-volume'], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'a6ca461be7cc1aff3658c2c6df4cff975a24449a', 'tag': None, 'wait_for_package': False} 2024-07-16T09:43:20.545 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-07-16T09:43:20.545 INFO:teuthology.packaging:ref: None 2024-07-16T09:43:20.545 INFO:teuthology.packaging:tag: None 2024-07-16T09:43:20.545 INFO:teuthology.packaging:branch: quincy 2024-07-16T09:43:20.545 INFO:teuthology.packaging:sha1: a6ca461be7cc1aff3658c2c6df4cff975a24449a 2024-07-16T09:43:20.545 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=quincy 2024-07-16T09:43:20.547 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': 'quincy', 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': ['ceph-volume'], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'a6ca461be7cc1aff3658c2c6df4cff975a24449a', 'tag': None, 'wait_for_package': False} 2024-07-16T09:43:20.547 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-07-16T09:43:20.548 INFO:teuthology.packaging:ref: None 2024-07-16T09:43:20.548 INFO:teuthology.packaging:tag: None 2024-07-16T09:43:20.548 INFO:teuthology.packaging:branch: quincy 2024-07-16T09:43:20.548 INFO:teuthology.packaging:sha1: a6ca461be7cc1aff3658c2c6df4cff975a24449a 2024-07-16T09:43:20.548 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=quincy 2024-07-16T09:43:20.757 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/quincy/e6231b04b52ebf3b9b31803282dd3074439e51ac/centos/9/flavors/default/ 2024-07-16T09:43:20.758 INFO:teuthology.task.install.rpm:Package version is 17.2.7-1104.ge6231b04 2024-07-16T09:43:20.767 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/quincy/e6231b04b52ebf3b9b31803282dd3074439e51ac/centos/9/flavors/default/ 2024-07-16T09:43:20.767 INFO:teuthology.task.install.rpm:Package version is 17.2.7-1104.ge6231b04 2024-07-16T09:43:20.868 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/quincy/e6231b04b52ebf3b9b31803282dd3074439e51ac/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/quincy/e6231b04b52ebf3b9b31803282dd3074439e51ac/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/quincy/e6231b04b52ebf3b9b31803282dd3074439e51ac/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-07-16T09:43:20.868 DEBUG:teuthology.orchestra.run.smithi176:> set -ex 2024-07-16T09:43:20.868 DEBUG:teuthology.orchestra.run.smithi176:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-07-16T09:43:20.885 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/quincy/e6231b04b52ebf3b9b31803282dd3074439e51ac/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/quincy/e6231b04b52ebf3b9b31803282dd3074439e51ac/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/quincy/e6231b04b52ebf3b9b31803282dd3074439e51ac/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-07-16T09:43:20.885 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2024-07-16T09:43:20.885 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-07-16T09:43:20.894 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-07-16T09:43:20.895 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-07-16T09:43:20.895 INFO:teuthology.packaging:ref: None 2024-07-16T09:43:20.895 INFO:teuthology.packaging:tag: None 2024-07-16T09:43:20.895 INFO:teuthology.packaging:branch: quincy 2024-07-16T09:43:20.895 INFO:teuthology.packaging:sha1: a6ca461be7cc1aff3658c2c6df4cff975a24449a 2024-07-16T09:43:20.895 DEBUG:teuthology.orchestra.run.smithi176:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;ref/quincy/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-07-16T09:43:20.911 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-07-16T09:43:20.911 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-07-16T09:43:20.911 INFO:teuthology.packaging:ref: None 2024-07-16T09:43:20.911 INFO:teuthology.packaging:tag: None 2024-07-16T09:43:20.912 INFO:teuthology.packaging:branch: quincy 2024-07-16T09:43:20.912 INFO:teuthology.packaging:sha1: a6ca461be7cc1aff3658c2c6df4cff975a24449a 2024-07-16T09:43:20.912 DEBUG:teuthology.orchestra.run.smithi156:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;ref/quincy/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-07-16T09:43:20.960 DEBUG:teuthology.orchestra.run.smithi176:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-07-16T09:43:20.977 DEBUG:teuthology.orchestra.run.smithi156:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-07-16T09:43:21.035 DEBUG:teuthology.orchestra.run.smithi176:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-07-16T09:43:21.052 DEBUG:teuthology.orchestra.run.smithi156:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-07-16T09:43:21.099 INFO:teuthology.orchestra.run.smithi176.stdout:check_obsoletes = 1 2024-07-16T09:43:21.100 DEBUG:teuthology.orchestra.run.smithi176:> sudo yum clean all 2024-07-16T09:43:21.116 INFO:teuthology.orchestra.run.smithi156.stdout:check_obsoletes = 1 2024-07-16T09:43:21.118 DEBUG:teuthology.orchestra.run.smithi156:> sudo yum clean all 2024-07-16T09:43:21.331 INFO:teuthology.orchestra.run.smithi176.stdout:45 files removed 2024-07-16T09:43:21.344 INFO:teuthology.orchestra.run.smithi156.stdout:45 files removed 2024-07-16T09:43:21.358 DEBUG:teuthology.orchestra.run.smithi176:> sudo yum -y install ceph-radosgw 2024-07-16T09:43:21.371 DEBUG:teuthology.orchestra.run.smithi156:> sudo yum -y install ceph-radosgw 2024-07-16T09:43:22.215 INFO:teuthology.orchestra.run.smithi156.stdout:ceph packages for x86_64 137 kB/s | 77 kB 00:00 2024-07-16T09:43:22.217 INFO:teuthology.orchestra.run.smithi176.stdout:ceph packages for x86_64 133 kB/s | 77 kB 00:00 2024-07-16T09:43:22.782 INFO:teuthology.orchestra.run.smithi156.stdout:ceph noarch packages 21 kB/s | 11 kB 00:00 2024-07-16T09:43:22.784 INFO:teuthology.orchestra.run.smithi176.stdout:ceph noarch packages 21 kB/s | 11 kB 00:00 2024-07-16T09:43:23.307 INFO:teuthology.orchestra.run.smithi156.stdout:ceph source packages 3.7 kB/s | 1.8 kB 00:00 2024-07-16T09:43:23.335 INFO:teuthology.orchestra.run.smithi176.stdout:ceph source packages 3.5 kB/s | 1.8 kB 00:00 2024-07-16T09:43:23.849 INFO:teuthology.orchestra.run.smithi156.stdout:CentOS Stream 9 - BaseOS 16 MB/s | 8.2 MB 00:00 2024-07-16T09:43:24.779 INFO:teuthology.orchestra.run.smithi176.stdout:CentOS Stream 9 - BaseOS 5.8 MB/s | 8.2 MB 00:01 2024-07-16T09:43:27.159 INFO:teuthology.orchestra.run.smithi156.stdout:CentOS Stream 9 - AppStream 9.0 MB/s | 20 MB 00:02 2024-07-16T09:43:32.642 INFO:teuthology.orchestra.run.smithi156.stdout:CentOS Stream 9 - CRB 3.6 MB/s | 6.5 MB 00:01 2024-07-16T09:43:33.618 INFO:teuthology.orchestra.run.smithi176.stdout:CentOS Stream 9 - AppStream 2.6 MB/s | 20 MB 00:07 2024-07-16T09:43:34.326 INFO:teuthology.orchestra.run.smithi156.stdout:CentOS Stream 9 - Extras packages 37 kB/s | 17 kB 00:00 2024-07-16T09:43:35.264 INFO:teuthology.orchestra.run.smithi156.stdout:Extra Packages for Enterprise Linux 28 MB/s | 22 MB 00:00 2024-07-16T09:43:37.727 INFO:teuthology.orchestra.run.smithi176.stdout:CentOS Stream 9 - CRB 16 MB/s | 6.5 MB 00:00 2024-07-16T09:43:39.578 INFO:teuthology.orchestra.run.smithi176.stdout:CentOS Stream 9 - Extras packages 35 kB/s | 17 kB 00:00 2024-07-16T09:43:40.549 INFO:teuthology.orchestra.run.smithi176.stdout:Extra Packages for Enterprise Linux 27 MB/s | 22 MB 00:00 2024-07-16T09:43:40.670 INFO:teuthology.orchestra.run.smithi156.stdout:lab-extras 28 kB/s | 1.7 kB 00:00 2024-07-16T09:43:43.061 INFO:teuthology.orchestra.run.smithi156.stdout:Dependencies resolved. 2024-07-16T09:43:43.070 INFO:teuthology.orchestra.run.smithi156.stdout:================================================================================ 2024-07-16T09:43:43.070 INFO:teuthology.orchestra.run.smithi156.stdout: Package Arch Version Repository Size 2024-07-16T09:43:43.070 INFO:teuthology.orchestra.run.smithi156.stdout:================================================================================ 2024-07-16T09:43:43.070 INFO:teuthology.orchestra.run.smithi156.stdout:Installing: 2024-07-16T09:43:43.070 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-radosgw x86_64 2:17.2.7-1104.ge6231b04.el9 ceph 14 M 2024-07-16T09:43:43.070 INFO:teuthology.orchestra.run.smithi156.stdout:Upgrading: 2024-07-16T09:43:43.070 INFO:teuthology.orchestra.run.smithi156.stdout: librados2 x86_64 2:17.2.7-1104.ge6231b04.el9 ceph 3.1 M 2024-07-16T09:43:43.071 INFO:teuthology.orchestra.run.smithi156.stdout: librbd1 x86_64 2:17.2.7-1104.ge6231b04.el9 ceph 2.9 M 2024-07-16T09:43:43.071 INFO:teuthology.orchestra.run.smithi156.stdout:Installing dependencies: 2024-07-16T09:43:43.071 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-base x86_64 2:17.2.7-1104.ge6231b04.el9 ceph 5.0 M 2024-07-16T09:43:43.071 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-common x86_64 2:17.2.7-1104.ge6231b04.el9 ceph 22 M 2024-07-16T09:43:43.071 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-selinux x86_64 2:17.2.7-1104.ge6231b04.el9 ceph 25 k 2024-07-16T09:43:43.071 INFO:teuthology.orchestra.run.smithi156.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-07-16T09:43:43.071 INFO:teuthology.orchestra.run.smithi156.stdout: libcephfs2 x86_64 2:17.2.7-1104.ge6231b04.el9 ceph 637 k 2024-07-16T09:43:43.071 INFO:teuthology.orchestra.run.smithi156.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-07-16T09:43:43.071 INFO:teuthology.orchestra.run.smithi156.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-07-16T09:43:43.071 INFO:teuthology.orchestra.run.smithi156.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-07-16T09:43:43.071 INFO:teuthology.orchestra.run.smithi156.stdout: libradosstriper1 x86_64 2:17.2.7-1104.ge6231b04.el9 ceph 457 k 2024-07-16T09:43:43.072 INFO:teuthology.orchestra.run.smithi156.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-07-16T09:43:43.072 INFO:teuthology.orchestra.run.smithi156.stdout: librgw2 x86_64 2:17.2.7-1104.ge6231b04.el9 ceph 4.9 M 2024-07-16T09:43:43.072 INFO:teuthology.orchestra.run.smithi156.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-07-16T09:43:43.072 INFO:teuthology.orchestra.run.smithi156.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-07-16T09:43:43.072 INFO:teuthology.orchestra.run.smithi156.stdout: python3-ceph-argparse x86_64 2:17.2.7-1104.ge6231b04.el9 ceph 45 k 2024-07-16T09:43:43.072 INFO:teuthology.orchestra.run.smithi156.stdout: python3-ceph-common x86_64 2:17.2.7-1104.ge6231b04.el9 ceph 118 k 2024-07-16T09:43:43.072 INFO:teuthology.orchestra.run.smithi156.stdout: python3-cephfs x86_64 2:17.2.7-1104.ge6231b04.el9 ceph 157 k 2024-07-16T09:43:43.072 INFO:teuthology.orchestra.run.smithi156.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-07-16T09:43:43.072 INFO:teuthology.orchestra.run.smithi156.stdout: python3-rados x86_64 2:17.2.7-1104.ge6231b04.el9 ceph 320 k 2024-07-16T09:43:43.072 INFO:teuthology.orchestra.run.smithi156.stdout: python3-rbd x86_64 2:17.2.7-1104.ge6231b04.el9 ceph 294 k 2024-07-16T09:43:43.072 INFO:teuthology.orchestra.run.smithi156.stdout: python3-rgw x86_64 2:17.2.7-1104.ge6231b04.el9 ceph 99 k 2024-07-16T09:43:43.072 INFO:teuthology.orchestra.run.smithi156.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-07-16T09:43:43.073 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-07-16T09:43:43.073 INFO:teuthology.orchestra.run.smithi156.stdout:Transaction Summary 2024-07-16T09:43:43.073 INFO:teuthology.orchestra.run.smithi156.stdout:================================================================================ 2024-07-16T09:43:43.073 INFO:teuthology.orchestra.run.smithi156.stdout:Install 22 Packages 2024-07-16T09:43:43.073 INFO:teuthology.orchestra.run.smithi156.stdout:Upgrade 2 Packages 2024-07-16T09:43:43.073 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-07-16T09:43:43.073 INFO:teuthology.orchestra.run.smithi156.stdout:Total download size: 58 M 2024-07-16T09:43:43.074 INFO:teuthology.orchestra.run.smithi156.stdout:Downloading Packages: 2024-07-16T09:43:44.580 INFO:teuthology.orchestra.run.smithi156.stdout:(1/24): ceph-common-17.2.7-1104.ge6231b04.el9.x 16 MB/s | 22 MB 00:01 2024-07-16T09:43:44.663 INFO:teuthology.orchestra.run.smithi156.stdout:(2/24): ceph-base-17.2.7-1104.ge6231b04.el9.x86 3.4 MB/s | 5.0 MB 00:01 2024-07-16T09:43:44.689 INFO:teuthology.orchestra.run.smithi156.stdout:(3/24): ceph-selinux-17.2.7-1104.ge6231b04.el9. 232 kB/s | 25 kB 00:00 2024-07-16T09:43:44.747 INFO:teuthology.orchestra.run.smithi156.stdout:(4/24): libcephfs2-17.2.7-1104.ge6231b04.el9.x8 7.4 MB/s | 637 kB 00:00 2024-07-16T09:43:44.772 INFO:teuthology.orchestra.run.smithi156.stdout:(5/24): libradosstriper1-17.2.7-1104.ge6231b04. 5.4 MB/s | 457 kB 00:00 2024-07-16T09:43:44.964 INFO:teuthology.orchestra.run.smithi156.stdout:(6/24): ceph-radosgw-17.2.7-1104.ge6231b04.el9. 7.9 MB/s | 14 MB 00:01 2024-07-16T09:43:44.990 INFO:teuthology.orchestra.run.smithi156.stdout:(7/24): python3-ceph-argparse-17.2.7-1104.ge623 208 kB/s | 45 kB 00:00 2024-07-16T09:43:45.026 INFO:teuthology.orchestra.run.smithi156.stdout:(8/24): python3-ceph-common-17.2.7-1104.ge6231b 2.0 MB/s | 118 kB 00:00 2024-07-16T09:43:45.057 INFO:teuthology.orchestra.run.smithi156.stdout:(9/24): python3-cephfs-17.2.7-1104.ge6231b04.el 2.3 MB/s | 157 kB 00:00 2024-07-16T09:43:45.132 INFO:teuthology.orchestra.run.smithi156.stdout:(10/24): librgw2-17.2.7-1104.ge6231b04.el9.x86_ 13 MB/s | 4.9 MB 00:00 2024-07-16T09:43:45.157 INFO:teuthology.orchestra.run.smithi156.stdout:(11/24): python3-rbd-17.2.7-1104.ge6231b04.el9. 2.9 MB/s | 294 kB 00:00 2024-07-16T09:43:45.183 INFO:teuthology.orchestra.run.smithi156.stdout:(12/24): python3-rados-17.2.7-1104.ge6231b04.el 2.0 MB/s | 320 kB 00:00 2024-07-16T09:43:45.233 INFO:teuthology.orchestra.run.smithi156.stdout:(13/24): python3-rgw-17.2.7-1104.ge6231b04.el9. 990 kB/s | 99 kB 00:00 2024-07-16T09:43:45.258 INFO:teuthology.orchestra.run.smithi156.stdout:(14/24): librabbitmq-0.11.0-7.el9.x86_64.rpm 601 kB/s | 45 kB 00:00 2024-07-16T09:43:45.283 INFO:teuthology.orchestra.run.smithi156.stdout:(15/24): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.2 MB/s | 160 kB 00:00 2024-07-16T09:43:45.351 INFO:teuthology.orchestra.run.smithi156.stdout:(16/24): python3-prettytable-0.7.2-27.el9.noarc 634 kB/s | 42 kB 00:00 2024-07-16T09:43:45.376 INFO:teuthology.orchestra.run.smithi156.stdout:(17/24): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.4 MB/s | 292 kB 00:00 2024-07-16T09:43:45.468 INFO:teuthology.orchestra.run.smithi156.stdout:(18/24): librdkafka-1.6.1-102.el9.x86_64.rpm 2.8 MB/s | 662 kB 00:00 2024-07-16T09:43:45.526 INFO:teuthology.orchestra.run.smithi156.stdout:(19/24): liboath-2.6.7-2.el9.x86_64.rpm 394 kB/s | 49 kB 00:00 2024-07-16T09:43:45.552 INFO:teuthology.orchestra.run.smithi156.stdout:(20/24): gperftools-libs-2.9.1-2.el9.x86_64.rpm 1.5 MB/s | 309 kB 00:00 2024-07-16T09:43:45.569 INFO:teuthology.orchestra.run.smithi156.stdout:(21/24): libunwind-1.6.2-1.el9.x86_64.rpm 687 kB/s | 67 kB 00:00 2024-07-16T09:43:45.619 INFO:teuthology.orchestra.run.smithi156.stdout:(22/24): thrift-0.15.0-2.el9.x86_64.rpm 18 MB/s | 1.6 MB 00:00 2024-07-16T09:43:45.811 INFO:teuthology.orchestra.run.smithi156.stdout:(23/24): librados2-17.2.7-1104.ge6231b04.el9.x8 12 MB/s | 3.1 MB 00:00 2024-07-16T09:43:45.862 INFO:teuthology.orchestra.run.smithi176.stdout:lab-extras 21 kB/s | 1.7 kB 00:00 2024-07-16T09:43:45.944 INFO:teuthology.orchestra.run.smithi156.stdout:(24/24): librbd1-17.2.7-1104.ge6231b04.el9.x86_ 7.8 MB/s | 2.9 MB 00:00 2024-07-16T09:43:45.950 INFO:teuthology.orchestra.run.smithi156.stdout:-------------------------------------------------------------------------------- 2024-07-16T09:43:45.950 INFO:teuthology.orchestra.run.smithi156.stdout:Total 20 MB/s | 58 MB 00:02 2024-07-16T09:43:46.010 INFO:teuthology.orchestra.run.smithi156.stdout:Running transaction check 2024-07-16T09:43:46.043 INFO:teuthology.orchestra.run.smithi156.stdout:Transaction check succeeded. 2024-07-16T09:43:46.043 INFO:teuthology.orchestra.run.smithi156.stdout:Running transaction test 2024-07-16T09:43:46.582 INFO:teuthology.orchestra.run.smithi156.stdout:Transaction test succeeded. 2024-07-16T09:43:46.583 INFO:teuthology.orchestra.run.smithi156.stdout:Running transaction 2024-07-16T09:43:47.743 INFO:teuthology.orchestra.run.smithi156.stdout: Preparing : 1/1 2024-07-16T09:43:48.131 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/26 2024-07-16T09:43:48.189 INFO:teuthology.orchestra.run.smithi156.stdout: Upgrading : librados2-2:17.2.7-1104.ge6231b04.el9.x86_64 2/26 2024-07-16T09:43:48.279 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: librados2-2:17.2.7-1104.ge6231b04.el9.x86_64 2/26 2024-07-16T09:43:48.339 INFO:teuthology.orchestra.run.smithi176.stdout:Dependencies resolved. 2024-07-16T09:43:48.354 INFO:teuthology.orchestra.run.smithi176.stdout:================================================================================ 2024-07-16T09:43:48.355 INFO:teuthology.orchestra.run.smithi176.stdout: Package Arch Version Repository Size 2024-07-16T09:43:48.355 INFO:teuthology.orchestra.run.smithi176.stdout:================================================================================ 2024-07-16T09:43:48.355 INFO:teuthology.orchestra.run.smithi176.stdout:Installing: 2024-07-16T09:43:48.355 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-radosgw x86_64 2:17.2.7-1104.ge6231b04.el9 ceph 14 M 2024-07-16T09:43:48.355 INFO:teuthology.orchestra.run.smithi176.stdout:Upgrading: 2024-07-16T09:43:48.355 INFO:teuthology.orchestra.run.smithi176.stdout: librados2 x86_64 2:17.2.7-1104.ge6231b04.el9 ceph 3.1 M 2024-07-16T09:43:48.355 INFO:teuthology.orchestra.run.smithi176.stdout: librbd1 x86_64 2:17.2.7-1104.ge6231b04.el9 ceph 2.9 M 2024-07-16T09:43:48.355 INFO:teuthology.orchestra.run.smithi176.stdout:Installing dependencies: 2024-07-16T09:43:48.355 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-base x86_64 2:17.2.7-1104.ge6231b04.el9 ceph 5.0 M 2024-07-16T09:43:48.355 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-common x86_64 2:17.2.7-1104.ge6231b04.el9 ceph 22 M 2024-07-16T09:43:48.355 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-selinux x86_64 2:17.2.7-1104.ge6231b04.el9 ceph 25 k 2024-07-16T09:43:48.356 INFO:teuthology.orchestra.run.smithi176.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-07-16T09:43:48.356 INFO:teuthology.orchestra.run.smithi176.stdout: libcephfs2 x86_64 2:17.2.7-1104.ge6231b04.el9 ceph 637 k 2024-07-16T09:43:48.356 INFO:teuthology.orchestra.run.smithi176.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-07-16T09:43:48.356 INFO:teuthology.orchestra.run.smithi176.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-07-16T09:43:48.356 INFO:teuthology.orchestra.run.smithi176.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-07-16T09:43:48.356 INFO:teuthology.orchestra.run.smithi176.stdout: libradosstriper1 x86_64 2:17.2.7-1104.ge6231b04.el9 ceph 457 k 2024-07-16T09:43:48.356 INFO:teuthology.orchestra.run.smithi176.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-07-16T09:43:48.356 INFO:teuthology.orchestra.run.smithi176.stdout: librgw2 x86_64 2:17.2.7-1104.ge6231b04.el9 ceph 4.9 M 2024-07-16T09:43:48.356 INFO:teuthology.orchestra.run.smithi176.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-07-16T09:43:48.356 INFO:teuthology.orchestra.run.smithi176.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-07-16T09:43:48.356 INFO:teuthology.orchestra.run.smithi176.stdout: python3-ceph-argparse x86_64 2:17.2.7-1104.ge6231b04.el9 ceph 45 k 2024-07-16T09:43:48.356 INFO:teuthology.orchestra.run.smithi176.stdout: python3-ceph-common x86_64 2:17.2.7-1104.ge6231b04.el9 ceph 118 k 2024-07-16T09:43:48.357 INFO:teuthology.orchestra.run.smithi176.stdout: python3-cephfs x86_64 2:17.2.7-1104.ge6231b04.el9 ceph 157 k 2024-07-16T09:43:48.357 INFO:teuthology.orchestra.run.smithi176.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-07-16T09:43:48.357 INFO:teuthology.orchestra.run.smithi176.stdout: python3-rados x86_64 2:17.2.7-1104.ge6231b04.el9 ceph 320 k 2024-07-16T09:43:48.357 INFO:teuthology.orchestra.run.smithi176.stdout: python3-rbd x86_64 2:17.2.7-1104.ge6231b04.el9 ceph 294 k 2024-07-16T09:43:48.357 INFO:teuthology.orchestra.run.smithi176.stdout: python3-rgw x86_64 2:17.2.7-1104.ge6231b04.el9 ceph 99 k 2024-07-16T09:43:48.357 INFO:teuthology.orchestra.run.smithi176.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-07-16T09:43:48.357 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-07-16T09:43:48.357 INFO:teuthology.orchestra.run.smithi176.stdout:Transaction Summary 2024-07-16T09:43:48.357 INFO:teuthology.orchestra.run.smithi176.stdout:================================================================================ 2024-07-16T09:43:48.357 INFO:teuthology.orchestra.run.smithi176.stdout:Install 22 Packages 2024-07-16T09:43:48.358 INFO:teuthology.orchestra.run.smithi176.stdout:Upgrade 2 Packages 2024-07-16T09:43:48.358 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-07-16T09:43:48.358 INFO:teuthology.orchestra.run.smithi176.stdout:Total download size: 58 M 2024-07-16T09:43:48.358 INFO:teuthology.orchestra.run.smithi176.stdout:Downloading Packages: 2024-07-16T09:43:48.358 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-rados-2:17.2.7-1104.ge6231b04.el9.x86_64 3/26 2024-07-16T09:43:48.382 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : libcephfs2-2:17.2.7-1104.ge6231b04.el9.x86_64 4/26 2024-07-16T09:43:48.475 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: libcephfs2-2:17.2.7-1104.ge6231b04.el9.x86_64 4/26 2024-07-16T09:43:48.517 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : thrift-0.15.0-2.el9.x86_64 5/26 2024-07-16T09:43:48.562 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 6/26 2024-07-16T09:43:48.992 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 7/26 2024-07-16T09:43:49.014 INFO:teuthology.orchestra.run.smithi176.stdout:(1/24): ceph-base-17.2.7-1104.ge6231b04.el9.x86 11 MB/s | 5.0 MB 00:00 2024-07-16T09:43:49.026 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : librgw2-2:17.2.7-1104.ge6231b04.el9.x86_64 8/26 2024-07-16T09:43:49.072 INFO:teuthology.orchestra.run.smithi176.stdout:(2/24): ceph-selinux-17.2.7-1104.ge6231b04.el9. 433 kB/s | 25 kB 00:00 2024-07-16T09:43:49.081 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: librgw2-2:17.2.7-1104.ge6231b04.el9.x86_64 8/26 2024-07-16T09:43:49.109 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : liboath-2.6.7-2.el9.x86_64 9/26 2024-07-16T09:43:49.153 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-ceph-argparse-2:17.2.7-1104.ge6231b04.el9. 10/26 2024-07-16T09:43:49.165 INFO:teuthology.orchestra.run.smithi176.stdout:(3/24): libcephfs2-17.2.7-1104.ge6231b04.el9.x8 6.8 MB/s | 637 kB 00:00 2024-07-16T09:43:49.190 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-cephfs-2:17.2.7-1104.ge6231b04.el9.x86_64 11/26 2024-07-16T09:43:49.248 INFO:teuthology.orchestra.run.smithi176.stdout:(4/24): libradosstriper1-17.2.7-1104.ge6231b04. 5.4 MB/s | 457 kB 00:00 2024-07-16T09:43:49.251 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-rgw-2:17.2.7-1104.ge6231b04.el9.x86_64 12/26 2024-07-16T09:43:49.280 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : libradosstriper1-2:17.2.7-1104.ge6231b04.el9.x86_6 13/26 2024-07-16T09:43:49.345 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: libradosstriper1-2:17.2.7-1104.ge6231b04.el9.x86_6 13/26 2024-07-16T09:43:49.385 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 14/26 2024-07-16T09:43:49.421 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 15/26 2024-07-16T09:43:49.454 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 16/26 2024-07-16T09:43:49.573 INFO:teuthology.orchestra.run.smithi176.stdout:(5/24): librgw2-17.2.7-1104.ge6231b04.el9.x86_6 15 MB/s | 4.9 MB 00:00 2024-07-16T09:43:49.624 INFO:teuthology.orchestra.run.smithi176.stdout:(6/24): python3-ceph-argparse-17.2.7-1104.ge623 906 kB/s | 45 kB 00:00 2024-07-16T09:43:49.748 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 17/26 2024-07-16T09:43:49.809 INFO:teuthology.orchestra.run.smithi156.stdout: Upgrading : librbd1-2:17.2.7-1104.ge6231b04.el9.x86_64 18/26 2024-07-16T09:43:49.832 INFO:teuthology.orchestra.run.smithi176.stdout:(7/24): ceph-common-17.2.7-1104.ge6231b04.el9.x 17 MB/s | 22 MB 00:01 2024-07-16T09:43:49.857 INFO:teuthology.orchestra.run.smithi176.stdout:(8/24): python3-ceph-common-17.2.7-1104.ge6231b 503 kB/s | 118 kB 00:00 2024-07-16T09:43:49.883 INFO:teuthology.orchestra.run.smithi176.stdout:(9/24): python3-cephfs-17.2.7-1104.ge6231b04.el 3.1 MB/s | 157 kB 00:00 2024-07-16T09:43:49.898 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: librbd1-2:17.2.7-1104.ge6231b04.el9.x86_64 18/26 2024-07-16T09:43:49.908 INFO:teuthology.orchestra.run.smithi176.stdout:(10/24): python3-rados-17.2.7-1104.ge6231b04.el 6.2 MB/s | 320 kB 00:00 2024-07-16T09:43:49.933 INFO:teuthology.orchestra.run.smithi176.stdout:(11/24): python3-rbd-17.2.7-1104.ge6231b04.el9. 5.7 MB/s | 294 kB 00:00 2024-07-16T09:43:49.958 INFO:teuthology.orchestra.run.smithi176.stdout:(12/24): python3-rgw-17.2.7-1104.ge6231b04.el9. 1.9 MB/s | 99 kB 00:00 2024-07-16T09:43:49.972 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-rbd-2:17.2.7-1104.ge6231b04.el9.x86_64 19/26 2024-07-16T09:43:50.020 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-ceph-common-2:17.2.7-1104.ge6231b04.el9.x8 20/26 2024-07-16T09:43:50.101 INFO:teuthology.orchestra.run.smithi176.stdout:(13/24): ceph-radosgw-17.2.7-1104.ge6231b04.el9 9.1 MB/s | 14 MB 00:01 2024-07-16T09:43:50.168 INFO:teuthology.orchestra.run.smithi176.stdout:(14/24): libpmemobj-1.12.1-1.el9.x86_64.rpm 685 kB/s | 160 kB 00:00 2024-07-16T09:43:50.193 INFO:teuthology.orchestra.run.smithi176.stdout:(15/24): librabbitmq-0.11.0-7.el9.x86_64.rpm 193 kB/s | 45 kB 00:00 2024-07-16T09:43:50.235 INFO:teuthology.orchestra.run.smithi176.stdout:(16/24): lttng-ust-2.12.0-6.el9.x86_64.rpm 4.3 MB/s | 292 kB 00:00 2024-07-16T09:43:50.260 INFO:teuthology.orchestra.run.smithi176.stdout:(17/24): python3-prettytable-0.7.2-27.el9.noarc 632 kB/s | 42 kB 00:00 2024-07-16T09:43:50.344 INFO:teuthology.orchestra.run.smithi176.stdout:(18/24): librdkafka-1.6.1-102.el9.x86_64.rpm 2.7 MB/s | 662 kB 00:00 2024-07-16T09:43:50.394 INFO:teuthology.orchestra.run.smithi176.stdout:(19/24): liboath-2.6.7-2.el9.x86_64.rpm 364 kB/s | 49 kB 00:00 2024-07-16T09:43:50.419 INFO:teuthology.orchestra.run.smithi176.stdout:(20/24): gperftools-libs-2.9.1-2.el9.x86_64.rpm 1.6 MB/s | 309 kB 00:00 2024-07-16T09:43:50.444 INFO:teuthology.orchestra.run.smithi176.stdout:(21/24): libunwind-1.6.2-1.el9.x86_64.rpm 686 kB/s | 67 kB 00:00 2024-07-16T09:43:50.494 INFO:teuthology.orchestra.run.smithi176.stdout:(22/24): thrift-0.15.0-2.el9.x86_64.rpm 16 MB/s | 1.6 MB 00:00 2024-07-16T09:43:50.645 INFO:teuthology.orchestra.run.smithi176.stdout:(23/24): librbd1-17.2.7-1104.ge6231b04.el9.x86_ 15 MB/s | 2.9 MB 00:00 2024-07-16T09:43:50.695 INFO:teuthology.orchestra.run.smithi176.stdout:(24/24): librados2-17.2.7-1104.ge6231b04.el9.x8 11 MB/s | 3.1 MB 00:00 2024-07-16T09:43:50.700 INFO:teuthology.orchestra.run.smithi176.stdout:-------------------------------------------------------------------------------- 2024-07-16T09:43:50.700 INFO:teuthology.orchestra.run.smithi176.stdout:Total 25 MB/s | 58 MB 00:02 2024-07-16T09:43:50.761 INFO:teuthology.orchestra.run.smithi176.stdout:Running transaction check 2024-07-16T09:43:50.793 INFO:teuthology.orchestra.run.smithi176.stdout:Transaction check succeeded. 2024-07-16T09:43:50.794 INFO:teuthology.orchestra.run.smithi176.stdout:Running transaction test 2024-07-16T09:43:51.335 INFO:teuthology.orchestra.run.smithi176.stdout:Transaction test succeeded. 2024-07-16T09:43:51.335 INFO:teuthology.orchestra.run.smithi176.stdout:Running transaction 2024-07-16T09:43:52.454 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: ceph-common-2:17.2.7-1104.ge6231b04.el9.x86_64 21/26 2024-07-16T09:43:52.492 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-common-2:17.2.7-1104.ge6231b04.el9.x86_64 21/26 2024-07-16T09:43:52.517 INFO:teuthology.orchestra.run.smithi176.stdout: Preparing : 1/1 2024-07-16T09:43:52.872 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/26 2024-07-16T09:43:52.947 INFO:teuthology.orchestra.run.smithi176.stdout: Upgrading : librados2-2:17.2.7-1104.ge6231b04.el9.x86_64 2/26 2024-07-16T09:43:53.021 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: ceph-common-2:17.2.7-1104.ge6231b04.el9.x86_64 21/26 2024-07-16T09:43:53.039 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: librados2-2:17.2.7-1104.ge6231b04.el9.x86_64 2/26 2024-07-16T09:43:53.062 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-base-2:17.2.7-1104.ge6231b04.el9.x86_64 22/26 2024-07-16T09:43:53.117 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-rados-2:17.2.7-1104.ge6231b04.el9.x86_64 3/26 2024-07-16T09:43:53.140 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : libcephfs2-2:17.2.7-1104.ge6231b04.el9.x86_64 4/26 2024-07-16T09:43:53.144 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: ceph-base-2:17.2.7-1104.ge6231b04.el9.x86_64 22/26 2024-07-16T09:43:53.145 INFO:teuthology.orchestra.run.smithi156.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-07-16T09:43:53.145 INFO:teuthology.orchestra.run.smithi156.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-07-16T09:43:53.145 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-07-16T09:43:53.172 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-selinux-2:17.2.7-1104.ge6231b04.el9.x86_64 23/26 2024-07-16T09:43:53.227 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: libcephfs2-2:17.2.7-1104.ge6231b04.el9.x86_64 4/26 2024-07-16T09:43:53.275 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : thrift-0.15.0-2.el9.x86_64 5/26 2024-07-16T09:43:53.307 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 6/26 2024-07-16T09:43:53.766 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 7/26 2024-07-16T09:43:53.793 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : librgw2-2:17.2.7-1104.ge6231b04.el9.x86_64 8/26 2024-07-16T09:43:53.856 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: librgw2-2:17.2.7-1104.ge6231b04.el9.x86_64 8/26 2024-07-16T09:43:53.884 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : liboath-2.6.7-2.el9.x86_64 9/26 2024-07-16T09:43:53.928 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-ceph-argparse-2:17.2.7-1104.ge6231b04.el9. 10/26 2024-07-16T09:43:53.964 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-cephfs-2:17.2.7-1104.ge6231b04.el9.x86_64 11/26 2024-07-16T09:43:54.026 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-rgw-2:17.2.7-1104.ge6231b04.el9.x86_64 12/26 2024-07-16T09:43:54.054 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : libradosstriper1-2:17.2.7-1104.ge6231b04.el9.x86_6 13/26 2024-07-16T09:43:54.119 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: libradosstriper1-2:17.2.7-1104.ge6231b04.el9.x86_6 13/26 2024-07-16T09:43:54.160 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 14/26 2024-07-16T09:43:54.212 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 15/26 2024-07-16T09:43:54.245 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 16/26 2024-07-16T09:43:54.591 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 17/26 2024-07-16T09:43:54.633 INFO:teuthology.orchestra.run.smithi176.stdout: Upgrading : librbd1-2:17.2.7-1104.ge6231b04.el9.x86_64 18/26 2024-07-16T09:43:54.714 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: librbd1-2:17.2.7-1104.ge6231b04.el9.x86_64 18/26 2024-07-16T09:43:54.788 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-rbd-2:17.2.7-1104.ge6231b04.el9.x86_64 19/26 2024-07-16T09:43:54.818 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-ceph-common-2:17.2.7-1104.ge6231b04.el9.x8 20/26 2024-07-16T09:43:57.283 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: ceph-common-2:17.2.7-1104.ge6231b04.el9.x86_64 21/26 2024-07-16T09:43:57.315 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : ceph-common-2:17.2.7-1104.ge6231b04.el9.x86_64 21/26 2024-07-16T09:43:57.847 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: ceph-common-2:17.2.7-1104.ge6231b04.el9.x86_64 21/26 2024-07-16T09:43:57.885 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : ceph-base-2:17.2.7-1104.ge6231b04.el9.x86_64 22/26 2024-07-16T09:43:57.986 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: ceph-base-2:17.2.7-1104.ge6231b04.el9.x86_64 22/26 2024-07-16T09:43:57.987 INFO:teuthology.orchestra.run.smithi176.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-07-16T09:43:57.987 INFO:teuthology.orchestra.run.smithi176.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-07-16T09:43:57.987 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-07-16T09:43:58.011 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : ceph-selinux-2:17.2.7-1104.ge6231b04.el9.x86_64 23/26 2024-07-16T09:44:02.757 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: ceph-selinux-2:17.2.7-1104.ge6231b04.el9.x86_64 23/26 2024-07-16T09:44:02.757 INFO:teuthology.orchestra.run.smithi156.stdout:skipping the directory /sys 2024-07-16T09:44:02.757 INFO:teuthology.orchestra.run.smithi156.stdout:skipping the directory /proc 2024-07-16T09:44:02.757 INFO:teuthology.orchestra.run.smithi156.stdout:skipping the directory /mnt 2024-07-16T09:44:02.757 INFO:teuthology.orchestra.run.smithi156.stdout:skipping the directory /var/tmp 2024-07-16T09:44:02.757 INFO:teuthology.orchestra.run.smithi156.stdout:skipping the directory /home 2024-07-16T09:44:02.757 INFO:teuthology.orchestra.run.smithi156.stdout:skipping the directory /root 2024-07-16T09:44:02.757 INFO:teuthology.orchestra.run.smithi156.stdout:skipping the directory /tmp 2024-07-16T09:44:02.757 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-07-16T09:44:04.051 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-radosgw-2:17.2.7-1104.ge6231b04.el9.x86_64 24/26 2024-07-16T09:44:04.130 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: ceph-radosgw-2:17.2.7-1104.ge6231b04.el9.x86_64 24/26 2024-07-16T09:44:04.130 INFO:teuthology.orchestra.run.smithi156.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-07-16T09:44:04.130 INFO:teuthology.orchestra.run.smithi156.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-07-16T09:44:04.130 INFO:teuthology.orchestra.run.smithi156.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-07-16T09:44:04.130 INFO:teuthology.orchestra.run.smithi156.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-07-16T09:44:04.130 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-07-16T09:44:04.132 INFO:teuthology.orchestra.run.smithi156.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 25/26 2024-07-16T09:44:04.229 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 25/26 2024-07-16T09:44:04.229 INFO:teuthology.orchestra.run.smithi156.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 26/26 2024-07-16T09:44:05.417 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 26/26 2024-07-16T09:44:05.417 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-base-2:17.2.7-1104.ge6231b04.el9.x86_64 1/26 2024-07-16T09:44:05.417 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-common-2:17.2.7-1104.ge6231b04.el9.x86_64 2/26 2024-07-16T09:44:05.417 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-radosgw-2:17.2.7-1104.ge6231b04.el9.x86_64 3/26 2024-07-16T09:44:05.417 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-selinux-2:17.2.7-1104.ge6231b04.el9.x86_64 4/26 2024-07-16T09:44:05.417 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libcephfs2-2:17.2.7-1104.ge6231b04.el9.x86_64 5/26 2024-07-16T09:44:05.417 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libradosstriper1-2:17.2.7-1104.ge6231b04.el9.x86_6 6/26 2024-07-16T09:44:05.418 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : librgw2-2:17.2.7-1104.ge6231b04.el9.x86_64 7/26 2024-07-16T09:44:05.418 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-ceph-argparse-2:17.2.7-1104.ge6231b04.el9. 8/26 2024-07-16T09:44:05.418 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-ceph-common-2:17.2.7-1104.ge6231b04.el9.x8 9/26 2024-07-16T09:44:05.418 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-cephfs-2:17.2.7-1104.ge6231b04.el9.x86_64 10/26 2024-07-16T09:44:05.418 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-rados-2:17.2.7-1104.ge6231b04.el9.x86_64 11/26 2024-07-16T09:44:05.418 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-rbd-2:17.2.7-1104.ge6231b04.el9.x86_64 12/26 2024-07-16T09:44:05.418 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-rgw-2:17.2.7-1104.ge6231b04.el9.x86_64 13/26 2024-07-16T09:44:05.418 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 14/26 2024-07-16T09:44:05.418 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 15/26 2024-07-16T09:44:05.418 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 16/26 2024-07-16T09:44:05.418 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 17/26 2024-07-16T09:44:05.418 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 18/26 2024-07-16T09:44:05.419 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 19/26 2024-07-16T09:44:05.419 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 20/26 2024-07-16T09:44:05.419 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 21/26 2024-07-16T09:44:05.419 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 22/26 2024-07-16T09:44:05.419 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : librados2-2:17.2.7-1104.ge6231b04.el9.x86_64 23/26 2024-07-16T09:44:05.419 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 24/26 2024-07-16T09:44:05.419 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : librbd1-2:17.2.7-1104.ge6231b04.el9.x86_64 25/26 2024-07-16T09:44:06.126 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 26/26 2024-07-16T09:44:06.127 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-07-16T09:44:06.127 INFO:teuthology.orchestra.run.smithi156.stdout:Upgraded: 2024-07-16T09:44:06.127 INFO:teuthology.orchestra.run.smithi156.stdout: librados2-2:17.2.7-1104.ge6231b04.el9.x86_64 2024-07-16T09:44:06.127 INFO:teuthology.orchestra.run.smithi156.stdout: librbd1-2:17.2.7-1104.ge6231b04.el9.x86_64 2024-07-16T09:44:06.127 INFO:teuthology.orchestra.run.smithi156.stdout:Installed: 2024-07-16T09:44:06.127 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-base-2:17.2.7-1104.ge6231b04.el9.x86_64 2024-07-16T09:44:06.127 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-common-2:17.2.7-1104.ge6231b04.el9.x86_64 2024-07-16T09:44:06.127 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-radosgw-2:17.2.7-1104.ge6231b04.el9.x86_64 2024-07-16T09:44:06.127 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-selinux-2:17.2.7-1104.ge6231b04.el9.x86_64 2024-07-16T09:44:06.127 INFO:teuthology.orchestra.run.smithi156.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-07-16T09:44:06.128 INFO:teuthology.orchestra.run.smithi156.stdout: libcephfs2-2:17.2.7-1104.ge6231b04.el9.x86_64 2024-07-16T09:44:06.128 INFO:teuthology.orchestra.run.smithi156.stdout: liboath-2.6.7-2.el9.x86_64 2024-07-16T09:44:06.128 INFO:teuthology.orchestra.run.smithi156.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-07-16T09:44:06.128 INFO:teuthology.orchestra.run.smithi156.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-07-16T09:44:06.128 INFO:teuthology.orchestra.run.smithi156.stdout: libradosstriper1-2:17.2.7-1104.ge6231b04.el9.x86_64 2024-07-16T09:44:06.128 INFO:teuthology.orchestra.run.smithi156.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-07-16T09:44:06.128 INFO:teuthology.orchestra.run.smithi156.stdout: librgw2-2:17.2.7-1104.ge6231b04.el9.x86_64 2024-07-16T09:44:06.128 INFO:teuthology.orchestra.run.smithi156.stdout: libunwind-1.6.2-1.el9.x86_64 2024-07-16T09:44:06.128 INFO:teuthology.orchestra.run.smithi156.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-07-16T09:44:06.128 INFO:teuthology.orchestra.run.smithi156.stdout: python3-ceph-argparse-2:17.2.7-1104.ge6231b04.el9.x86_64 2024-07-16T09:44:06.128 INFO:teuthology.orchestra.run.smithi156.stdout: python3-ceph-common-2:17.2.7-1104.ge6231b04.el9.x86_64 2024-07-16T09:44:06.128 INFO:teuthology.orchestra.run.smithi156.stdout: python3-cephfs-2:17.2.7-1104.ge6231b04.el9.x86_64 2024-07-16T09:44:06.128 INFO:teuthology.orchestra.run.smithi156.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-07-16T09:44:06.129 INFO:teuthology.orchestra.run.smithi156.stdout: python3-rados-2:17.2.7-1104.ge6231b04.el9.x86_64 2024-07-16T09:44:06.129 INFO:teuthology.orchestra.run.smithi156.stdout: python3-rbd-2:17.2.7-1104.ge6231b04.el9.x86_64 2024-07-16T09:44:06.129 INFO:teuthology.orchestra.run.smithi156.stdout: python3-rgw-2:17.2.7-1104.ge6231b04.el9.x86_64 2024-07-16T09:44:06.129 INFO:teuthology.orchestra.run.smithi156.stdout: thrift-0.15.0-2.el9.x86_64 2024-07-16T09:44:06.129 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-07-16T09:44:06.129 INFO:teuthology.orchestra.run.smithi156.stdout:Complete! 2024-07-16T09:44:06.540 DEBUG:teuthology.orchestra.run.smithi156:> sudo yum -y install ceph-test 2024-07-16T09:44:07.130 INFO:teuthology.orchestra.run.smithi156.stdout:Last metadata expiration check: 0:00:27 ago on Tue 16 Jul 2024 09:43:40 AM UTC. 2024-07-16T09:44:07.256 INFO:teuthology.orchestra.run.smithi156.stdout:Dependencies resolved. 2024-07-16T09:44:07.256 INFO:teuthology.orchestra.run.smithi156.stdout:================================================================================ 2024-07-16T09:44:07.256 INFO:teuthology.orchestra.run.smithi156.stdout: Package Arch Version Repository Size 2024-07-16T09:44:07.257 INFO:teuthology.orchestra.run.smithi156.stdout:================================================================================ 2024-07-16T09:44:07.257 INFO:teuthology.orchestra.run.smithi156.stdout:Installing: 2024-07-16T09:44:07.257 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-test x86_64 2:17.2.7-1104.ge6231b04.el9 ceph 48 M 2024-07-16T09:44:07.257 INFO:teuthology.orchestra.run.smithi156.stdout:Installing dependencies: 2024-07-16T09:44:07.257 INFO:teuthology.orchestra.run.smithi156.stdout: libcephsqlite x86_64 2:17.2.7-1104.ge6231b04.el9 ceph 163 k 2024-07-16T09:44:07.257 INFO:teuthology.orchestra.run.smithi156.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-07-16T09:44:07.257 INFO:teuthology.orchestra.run.smithi156.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-07-16T09:44:07.257 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-07-16T09:44:07.257 INFO:teuthology.orchestra.run.smithi156.stdout:Transaction Summary 2024-07-16T09:44:07.257 INFO:teuthology.orchestra.run.smithi156.stdout:================================================================================ 2024-07-16T09:44:07.257 INFO:teuthology.orchestra.run.smithi156.stdout:Install 4 Packages 2024-07-16T09:44:07.257 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-07-16T09:44:07.258 INFO:teuthology.orchestra.run.smithi156.stdout:Total download size: 48 M 2024-07-16T09:44:07.258 INFO:teuthology.orchestra.run.smithi156.stdout:Installed size: 192 M 2024-07-16T09:44:07.258 INFO:teuthology.orchestra.run.smithi156.stdout:Downloading Packages: 2024-07-16T09:44:07.612 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: ceph-selinux-2:17.2.7-1104.ge6231b04.el9.x86_64 23/26 2024-07-16T09:44:07.613 INFO:teuthology.orchestra.run.smithi176.stdout:skipping the directory /sys 2024-07-16T09:44:07.613 INFO:teuthology.orchestra.run.smithi176.stdout:skipping the directory /proc 2024-07-16T09:44:07.613 INFO:teuthology.orchestra.run.smithi176.stdout:skipping the directory /mnt 2024-07-16T09:44:07.613 INFO:teuthology.orchestra.run.smithi176.stdout:skipping the directory /var/tmp 2024-07-16T09:44:07.613 INFO:teuthology.orchestra.run.smithi176.stdout:skipping the directory /home 2024-07-16T09:44:07.613 INFO:teuthology.orchestra.run.smithi176.stdout:skipping the directory /root 2024-07-16T09:44:07.613 INFO:teuthology.orchestra.run.smithi176.stdout:skipping the directory /tmp 2024-07-16T09:44:07.613 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-07-16T09:44:07.701 INFO:teuthology.orchestra.run.smithi156.stdout:(1/4): socat-1.7.4.1-6.el9.x86_64.rpm 1.7 MB/s | 304 kB 00:00 2024-07-16T09:44:07.743 INFO:teuthology.orchestra.run.smithi156.stdout:(2/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.5 MB/s | 64 kB 00:00 2024-07-16T09:44:07.918 INFO:teuthology.orchestra.run.smithi156.stdout:(3/4): libcephsqlite-17.2.7-1104.ge6231b04.el9. 417 kB/s | 163 kB 00:00 2024-07-16T09:44:08.869 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : ceph-radosgw-2:17.2.7-1104.ge6231b04.el9.x86_64 24/26 2024-07-16T09:44:08.952 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: ceph-radosgw-2:17.2.7-1104.ge6231b04.el9.x86_64 24/26 2024-07-16T09:44:08.952 INFO:teuthology.orchestra.run.smithi176.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-07-16T09:44:08.952 INFO:teuthology.orchestra.run.smithi176.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-07-16T09:44:08.952 INFO:teuthology.orchestra.run.smithi176.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-07-16T09:44:08.952 INFO:teuthology.orchestra.run.smithi176.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-07-16T09:44:08.952 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-07-16T09:44:08.954 INFO:teuthology.orchestra.run.smithi176.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 25/26 2024-07-16T09:44:09.039 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 25/26 2024-07-16T09:44:09.039 INFO:teuthology.orchestra.run.smithi176.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 26/26 2024-07-16T09:44:10.328 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 26/26 2024-07-16T09:44:10.328 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : ceph-base-2:17.2.7-1104.ge6231b04.el9.x86_64 1/26 2024-07-16T09:44:10.328 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : ceph-common-2:17.2.7-1104.ge6231b04.el9.x86_64 2/26 2024-07-16T09:44:10.328 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : ceph-radosgw-2:17.2.7-1104.ge6231b04.el9.x86_64 3/26 2024-07-16T09:44:10.328 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : ceph-selinux-2:17.2.7-1104.ge6231b04.el9.x86_64 4/26 2024-07-16T09:44:10.329 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : libcephfs2-2:17.2.7-1104.ge6231b04.el9.x86_64 5/26 2024-07-16T09:44:10.329 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : libradosstriper1-2:17.2.7-1104.ge6231b04.el9.x86_6 6/26 2024-07-16T09:44:10.329 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : librgw2-2:17.2.7-1104.ge6231b04.el9.x86_64 7/26 2024-07-16T09:44:10.329 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-ceph-argparse-2:17.2.7-1104.ge6231b04.el9. 8/26 2024-07-16T09:44:10.329 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-ceph-common-2:17.2.7-1104.ge6231b04.el9.x8 9/26 2024-07-16T09:44:10.329 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-cephfs-2:17.2.7-1104.ge6231b04.el9.x86_64 10/26 2024-07-16T09:44:10.329 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-rados-2:17.2.7-1104.ge6231b04.el9.x86_64 11/26 2024-07-16T09:44:10.329 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-rbd-2:17.2.7-1104.ge6231b04.el9.x86_64 12/26 2024-07-16T09:44:10.329 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-rgw-2:17.2.7-1104.ge6231b04.el9.x86_64 13/26 2024-07-16T09:44:10.329 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 14/26 2024-07-16T09:44:10.329 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 15/26 2024-07-16T09:44:10.329 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 16/26 2024-07-16T09:44:10.329 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 17/26 2024-07-16T09:44:10.330 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 18/26 2024-07-16T09:44:10.330 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 19/26 2024-07-16T09:44:10.330 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 20/26 2024-07-16T09:44:10.330 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 21/26 2024-07-16T09:44:10.330 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 22/26 2024-07-16T09:44:10.330 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : librados2-2:17.2.7-1104.ge6231b04.el9.x86_64 23/26 2024-07-16T09:44:10.330 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 24/26 2024-07-16T09:44:10.330 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : librbd1-2:17.2.7-1104.ge6231b04.el9.x86_64 25/26 2024-07-16T09:44:10.535 INFO:teuthology.orchestra.run.smithi156.stdout:(4/4): ceph-test-17.2.7-1104.ge6231b04.el9.x86_ 16 MB/s | 48 MB 00:03 2024-07-16T09:44:10.537 INFO:teuthology.orchestra.run.smithi156.stdout:-------------------------------------------------------------------------------- 2024-07-16T09:44:10.538 INFO:teuthology.orchestra.run.smithi156.stdout:Total 15 MB/s | 48 MB 00:03 2024-07-16T09:44:10.566 INFO:teuthology.orchestra.run.smithi156.stdout:Running transaction check 2024-07-16T09:44:10.586 INFO:teuthology.orchestra.run.smithi156.stdout:Transaction check succeeded. 2024-07-16T09:44:10.586 INFO:teuthology.orchestra.run.smithi156.stdout:Running transaction test 2024-07-16T09:44:10.950 INFO:teuthology.orchestra.run.smithi156.stdout:Transaction test succeeded. 2024-07-16T09:44:10.950 INFO:teuthology.orchestra.run.smithi156.stdout:Running transaction 2024-07-16T09:44:11.065 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 26/26 2024-07-16T09:44:11.065 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-07-16T09:44:11.065 INFO:teuthology.orchestra.run.smithi176.stdout:Upgraded: 2024-07-16T09:44:11.066 INFO:teuthology.orchestra.run.smithi176.stdout: librados2-2:17.2.7-1104.ge6231b04.el9.x86_64 2024-07-16T09:44:11.066 INFO:teuthology.orchestra.run.smithi176.stdout: librbd1-2:17.2.7-1104.ge6231b04.el9.x86_64 2024-07-16T09:44:11.066 INFO:teuthology.orchestra.run.smithi176.stdout:Installed: 2024-07-16T09:44:11.066 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-base-2:17.2.7-1104.ge6231b04.el9.x86_64 2024-07-16T09:44:11.066 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-common-2:17.2.7-1104.ge6231b04.el9.x86_64 2024-07-16T09:44:11.066 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-radosgw-2:17.2.7-1104.ge6231b04.el9.x86_64 2024-07-16T09:44:11.066 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-selinux-2:17.2.7-1104.ge6231b04.el9.x86_64 2024-07-16T09:44:11.066 INFO:teuthology.orchestra.run.smithi176.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-07-16T09:44:11.066 INFO:teuthology.orchestra.run.smithi176.stdout: libcephfs2-2:17.2.7-1104.ge6231b04.el9.x86_64 2024-07-16T09:44:11.066 INFO:teuthology.orchestra.run.smithi176.stdout: liboath-2.6.7-2.el9.x86_64 2024-07-16T09:44:11.066 INFO:teuthology.orchestra.run.smithi176.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-07-16T09:44:11.066 INFO:teuthology.orchestra.run.smithi176.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-07-16T09:44:11.067 INFO:teuthology.orchestra.run.smithi176.stdout: libradosstriper1-2:17.2.7-1104.ge6231b04.el9.x86_64 2024-07-16T09:44:11.067 INFO:teuthology.orchestra.run.smithi176.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-07-16T09:44:11.067 INFO:teuthology.orchestra.run.smithi176.stdout: librgw2-2:17.2.7-1104.ge6231b04.el9.x86_64 2024-07-16T09:44:11.067 INFO:teuthology.orchestra.run.smithi176.stdout: libunwind-1.6.2-1.el9.x86_64 2024-07-16T09:44:11.067 INFO:teuthology.orchestra.run.smithi176.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-07-16T09:44:11.067 INFO:teuthology.orchestra.run.smithi176.stdout: python3-ceph-argparse-2:17.2.7-1104.ge6231b04.el9.x86_64 2024-07-16T09:44:11.067 INFO:teuthology.orchestra.run.smithi176.stdout: python3-ceph-common-2:17.2.7-1104.ge6231b04.el9.x86_64 2024-07-16T09:44:11.067 INFO:teuthology.orchestra.run.smithi176.stdout: python3-cephfs-2:17.2.7-1104.ge6231b04.el9.x86_64 2024-07-16T09:44:11.067 INFO:teuthology.orchestra.run.smithi176.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-07-16T09:44:11.067 INFO:teuthology.orchestra.run.smithi176.stdout: python3-rados-2:17.2.7-1104.ge6231b04.el9.x86_64 2024-07-16T09:44:11.067 INFO:teuthology.orchestra.run.smithi176.stdout: python3-rbd-2:17.2.7-1104.ge6231b04.el9.x86_64 2024-07-16T09:44:11.067 INFO:teuthology.orchestra.run.smithi176.stdout: python3-rgw-2:17.2.7-1104.ge6231b04.el9.x86_64 2024-07-16T09:44:11.067 INFO:teuthology.orchestra.run.smithi176.stdout: thrift-0.15.0-2.el9.x86_64 2024-07-16T09:44:11.068 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-07-16T09:44:11.068 INFO:teuthology.orchestra.run.smithi176.stdout:Complete! 2024-07-16T09:44:11.424 DEBUG:teuthology.orchestra.run.smithi176:> sudo yum -y install ceph-test 2024-07-16T09:44:11.503 INFO:teuthology.orchestra.run.smithi156.stdout: Preparing : 1/1 2024-07-16T09:44:11.597 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-07-16T09:44:11.643 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 2/4 2024-07-16T09:44:11.670 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : libcephsqlite-2:17.2.7-1104.ge6231b04.el9.x86_64 3/4 2024-07-16T09:44:12.014 INFO:teuthology.orchestra.run.smithi176.stdout:Last metadata expiration check: 0:00:27 ago on Tue 16 Jul 2024 09:43:45 AM UTC. 2024-07-16T09:44:12.143 INFO:teuthology.orchestra.run.smithi176.stdout:Dependencies resolved. 2024-07-16T09:44:12.144 INFO:teuthology.orchestra.run.smithi176.stdout:================================================================================ 2024-07-16T09:44:12.144 INFO:teuthology.orchestra.run.smithi176.stdout: Package Arch Version Repository Size 2024-07-16T09:44:12.144 INFO:teuthology.orchestra.run.smithi176.stdout:================================================================================ 2024-07-16T09:44:12.145 INFO:teuthology.orchestra.run.smithi176.stdout:Installing: 2024-07-16T09:44:12.145 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-test x86_64 2:17.2.7-1104.ge6231b04.el9 ceph 48 M 2024-07-16T09:44:12.145 INFO:teuthology.orchestra.run.smithi176.stdout:Installing dependencies: 2024-07-16T09:44:12.145 INFO:teuthology.orchestra.run.smithi176.stdout: libcephsqlite x86_64 2:17.2.7-1104.ge6231b04.el9 ceph 163 k 2024-07-16T09:44:12.145 INFO:teuthology.orchestra.run.smithi176.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-07-16T09:44:12.145 INFO:teuthology.orchestra.run.smithi176.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-07-16T09:44:12.145 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-07-16T09:44:12.145 INFO:teuthology.orchestra.run.smithi176.stdout:Transaction Summary 2024-07-16T09:44:12.145 INFO:teuthology.orchestra.run.smithi176.stdout:================================================================================ 2024-07-16T09:44:12.145 INFO:teuthology.orchestra.run.smithi176.stdout:Install 4 Packages 2024-07-16T09:44:12.145 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-07-16T09:44:12.145 INFO:teuthology.orchestra.run.smithi176.stdout:Total download size: 48 M 2024-07-16T09:44:12.145 INFO:teuthology.orchestra.run.smithi176.stdout:Installed size: 192 M 2024-07-16T09:44:12.146 INFO:teuthology.orchestra.run.smithi176.stdout:Downloading Packages: 2024-07-16T09:44:12.446 INFO:teuthology.orchestra.run.smithi176.stdout:(1/4): socat-1.7.4.1-6.el9.x86_64.rpm 1.8 MB/s | 304 kB 00:00 2024-07-16T09:44:12.487 INFO:teuthology.orchestra.run.smithi176.stdout:(2/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.5 MB/s | 64 kB 00:00 2024-07-16T09:44:12.546 INFO:teuthology.orchestra.run.smithi176.stdout:(3/4): libcephsqlite-17.2.7-1104.ge6231b04.el9. 614 kB/s | 163 kB 00:00 2024-07-16T09:44:13.821 INFO:teuthology.orchestra.run.smithi176.stdout:(4/4): ceph-test-17.2.7-1104.ge6231b04.el9.x86_ 31 MB/s | 48 MB 00:01 2024-07-16T09:44:13.823 INFO:teuthology.orchestra.run.smithi176.stdout:-------------------------------------------------------------------------------- 2024-07-16T09:44:13.824 INFO:teuthology.orchestra.run.smithi176.stdout:Total 29 MB/s | 48 MB 00:01 2024-07-16T09:44:13.851 INFO:teuthology.orchestra.run.smithi176.stdout:Running transaction check 2024-07-16T09:44:13.870 INFO:teuthology.orchestra.run.smithi176.stdout:Transaction check succeeded. 2024-07-16T09:44:13.870 INFO:teuthology.orchestra.run.smithi176.stdout:Running transaction test 2024-07-16T09:44:14.234 INFO:teuthology.orchestra.run.smithi176.stdout:Transaction test succeeded. 2024-07-16T09:44:14.235 INFO:teuthology.orchestra.run.smithi176.stdout:Running transaction 2024-07-16T09:44:14.758 INFO:teuthology.orchestra.run.smithi176.stdout: Preparing : 1/1 2024-07-16T09:44:14.845 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-07-16T09:44:14.886 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 2/4 2024-07-16T09:44:14.915 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : libcephsqlite-2:17.2.7-1104.ge6231b04.el9.x86_64 3/4 2024-07-16T09:44:16.080 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: libcephsqlite-2:17.2.7-1104.ge6231b04.el9.x86_64 3/4 2024-07-16T09:44:16.160 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-test-2:17.2.7-1104.ge6231b04.el9.x86_64 4/4 2024-07-16T09:44:18.265 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: ceph-test-2:17.2.7-1104.ge6231b04.el9.x86_64 4/4 2024-07-16T09:44:18.266 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-test-2:17.2.7-1104.ge6231b04.el9.x86_64 1/4 2024-07-16T09:44:18.266 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libcephsqlite-2:17.2.7-1104.ge6231b04.el9.x86_64 2/4 2024-07-16T09:44:18.266 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 3/4 2024-07-16T09:44:18.517 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-07-16T09:44:18.522 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-07-16T09:44:18.522 INFO:teuthology.orchestra.run.smithi156.stdout:Installed: 2024-07-16T09:44:18.522 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-test-2:17.2.7-1104.ge6231b04.el9.x86_64 2024-07-16T09:44:18.522 INFO:teuthology.orchestra.run.smithi156.stdout: libcephsqlite-2:17.2.7-1104.ge6231b04.el9.x86_64 2024-07-16T09:44:18.522 INFO:teuthology.orchestra.run.smithi156.stdout: socat-1.7.4.1-6.el9.x86_64 2024-07-16T09:44:18.522 INFO:teuthology.orchestra.run.smithi156.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-07-16T09:44:18.522 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-07-16T09:44:18.522 INFO:teuthology.orchestra.run.smithi156.stdout:Complete! 2024-07-16T09:44:18.811 DEBUG:teuthology.orchestra.run.smithi156:> sudo yum -y install ceph 2024-07-16T09:44:19.288 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: libcephsqlite-2:17.2.7-1104.ge6231b04.el9.x86_64 3/4 2024-07-16T09:44:19.370 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : ceph-test-2:17.2.7-1104.ge6231b04.el9.x86_64 4/4 2024-07-16T09:44:19.403 INFO:teuthology.orchestra.run.smithi156.stdout:Last metadata expiration check: 0:00:39 ago on Tue 16 Jul 2024 09:43:40 AM UTC. 2024-07-16T09:44:19.536 INFO:teuthology.orchestra.run.smithi156.stdout:Dependencies resolved. 2024-07-16T09:44:19.539 INFO:teuthology.orchestra.run.smithi156.stdout:================================================================================ 2024-07-16T09:44:19.539 INFO:teuthology.orchestra.run.smithi156.stdout: Package Arch Version Repository Size 2024-07-16T09:44:19.539 INFO:teuthology.orchestra.run.smithi156.stdout:================================================================================ 2024-07-16T09:44:19.539 INFO:teuthology.orchestra.run.smithi156.stdout:Installing: 2024-07-16T09:44:19.539 INFO:teuthology.orchestra.run.smithi156.stdout: ceph x86_64 2:17.2.7-1104.ge6231b04.el9 ceph 6.5 k 2024-07-16T09:44:19.539 INFO:teuthology.orchestra.run.smithi156.stdout:Installing dependencies: 2024-07-16T09:44:19.539 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mds x86_64 2:17.2.7-1104.ge6231b04.el9 ceph 2.1 M 2024-07-16T09:44:19.539 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mgr x86_64 2:17.2.7-1104.ge6231b04.el9 ceph 1.4 M 2024-07-16T09:44:19.539 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mgr-modules-core noarch 2:17.2.7-1104.ge6231b04.el9 ceph-noarch 240 k 2024-07-16T09:44:19.539 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mon x86_64 2:17.2.7-1104.ge6231b04.el9 ceph 3.9 M 2024-07-16T09:44:19.540 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-osd x86_64 2:17.2.7-1104.ge6231b04.el9 ceph 15 M 2024-07-16T09:44:19.540 INFO:teuthology.orchestra.run.smithi156.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-07-16T09:44:19.540 INFO:teuthology.orchestra.run.smithi156.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-07-16T09:44:19.540 INFO:teuthology.orchestra.run.smithi156.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-07-16T09:44:19.540 INFO:teuthology.orchestra.run.smithi156.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-07-16T09:44:19.540 INFO:teuthology.orchestra.run.smithi156.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-07-16T09:44:19.540 INFO:teuthology.orchestra.run.smithi156.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-07-16T09:44:19.540 INFO:teuthology.orchestra.run.smithi156.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-07-16T09:44:19.540 INFO:teuthology.orchestra.run.smithi156.stdout: python3-jaraco-collections 2024-07-16T09:44:19.540 INFO:teuthology.orchestra.run.smithi156.stdout: noarch 3.0.0-8.el9 epel 23 k 2024-07-16T09:44:19.540 INFO:teuthology.orchestra.run.smithi156.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-07-16T09:44:19.540 INFO:teuthology.orchestra.run.smithi156.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-07-16T09:44:19.540 INFO:teuthology.orchestra.run.smithi156.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-07-16T09:44:19.541 INFO:teuthology.orchestra.run.smithi156.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-07-16T09:44:19.541 INFO:teuthology.orchestra.run.smithi156.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-07-16T09:44:19.541 INFO:teuthology.orchestra.run.smithi156.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-07-16T09:44:19.541 INFO:teuthology.orchestra.run.smithi156.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-07-16T09:44:19.541 INFO:teuthology.orchestra.run.smithi156.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-07-16T09:44:19.541 INFO:teuthology.orchestra.run.smithi156.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-07-16T09:44:19.541 INFO:teuthology.orchestra.run.smithi156.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-07-16T09:44:19.541 INFO:teuthology.orchestra.run.smithi156.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-07-16T09:44:19.541 INFO:teuthology.orchestra.run.smithi156.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-07-16T09:44:19.541 INFO:teuthology.orchestra.run.smithi156.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-07-16T09:44:19.541 INFO:teuthology.orchestra.run.smithi156.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-07-16T09:44:19.541 INFO:teuthology.orchestra.run.smithi156.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-07-16T09:44:19.542 INFO:teuthology.orchestra.run.smithi156.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-07-16T09:44:19.542 INFO:teuthology.orchestra.run.smithi156.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-07-16T09:44:19.542 INFO:teuthology.orchestra.run.smithi156.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-07-16T09:44:19.542 INFO:teuthology.orchestra.run.smithi156.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-07-16T09:44:19.542 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-07-16T09:44:19.542 INFO:teuthology.orchestra.run.smithi156.stdout:Transaction Summary 2024-07-16T09:44:19.542 INFO:teuthology.orchestra.run.smithi156.stdout:================================================================================ 2024-07-16T09:44:19.542 INFO:teuthology.orchestra.run.smithi156.stdout:Install 33 Packages 2024-07-16T09:44:19.542 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-07-16T09:44:19.543 INFO:teuthology.orchestra.run.smithi156.stdout:Total download size: 27 M 2024-07-16T09:44:19.543 INFO:teuthology.orchestra.run.smithi156.stdout:Installed size: 96 M 2024-07-16T09:44:19.544 INFO:teuthology.orchestra.run.smithi156.stdout:Downloading Packages: 2024-07-16T09:44:20.077 INFO:teuthology.orchestra.run.smithi156.stdout:(1/33): ceph-17.2.7-1104.ge6231b04.el9.x86_64.r 33 kB/s | 6.5 kB 00:00 2024-07-16T09:44:20.295 INFO:teuthology.orchestra.run.smithi156.stdout:(2/33): ceph-mgr-17.2.7-1104.ge6231b04.el9.x86_ 3.4 MB/s | 1.4 MB 00:00 2024-07-16T09:44:20.336 INFO:teuthology.orchestra.run.smithi156.stdout:(3/33): ceph-mds-17.2.7-1104.ge6231b04.el9.x86_ 4.6 MB/s | 2.1 MB 00:00 2024-07-16T09:44:20.420 INFO:teuthology.orchestra.run.smithi156.stdout:(4/33): ceph-mgr-modules-core-17.2.7-1104.ge623 2.8 MB/s | 240 kB 00:00 2024-07-16T09:44:20.629 INFO:teuthology.orchestra.run.smithi156.stdout:(5/33): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.2 MB/s | 253 kB 00:00 2024-07-16T09:44:20.754 INFO:teuthology.orchestra.run.smithi156.stdout:(6/33): python3-cryptography-36.0.1-4.el9.x86_6 9.8 MB/s | 1.2 MB 00:00 2024-07-16T09:44:20.804 INFO:teuthology.orchestra.run.smithi156.stdout:(7/33): python3-ply-3.11-14.el9.noarch.rpm 2.1 MB/s | 106 kB 00:00 2024-07-16T09:44:20.979 INFO:teuthology.orchestra.run.smithi156.stdout:(8/33): ceph-osd-17.2.7-1104.ge6231b04.el9.x86_ 22 MB/s | 15 MB 00:00 2024-07-16T09:44:21.004 INFO:teuthology.orchestra.run.smithi156.stdout:(9/33): python3-pycparser-2.20-6.el9.noarch.rpm 675 kB/s | 135 kB 00:00 2024-07-16T09:44:21.088 INFO:teuthology.orchestra.run.smithi156.stdout:(10/33): ceph-mon-17.2.7-1104.ge6231b04.el9.x86 3.9 MB/s | 3.9 MB 00:01 2024-07-16T09:44:21.113 INFO:teuthology.orchestra.run.smithi156.stdout:(11/33): python3-urllib3-1.26.5-6.el9.noarch.rp 1.9 MB/s | 215 kB 00:00 2024-07-16T09:44:21.155 INFO:teuthology.orchestra.run.smithi156.stdout:(12/33): python3-requests-2.25.1-8.el9.noarch.r 711 kB/s | 125 kB 00:00 2024-07-16T09:44:21.239 INFO:teuthology.orchestra.run.smithi156.stdout:(13/33): python3-markupsafe-1.1.1-12.el9.x86_64 277 kB/s | 35 kB 00:00 2024-07-16T09:44:21.272 INFO:teuthology.orchestra.run.smithi156.stdout:(14/33): python3-mako-1.1.4-6.el9.noarch.rpm 935 kB/s | 172 kB 00:00 2024-07-16T09:44:21.289 INFO:teuthology.orchestra.run.smithi156.stdout:(15/33): python3-pytz-2021.1-5.el9.noarch.rpm 381 kB/s | 51 kB 00:00 2024-07-16T09:44:21.365 INFO:teuthology.orchestra.run.smithi156.stdout:(16/33): python3-toml-0.10.2-6.el9.noarch.rpm 332 kB/s | 42 kB 00:00 2024-07-16T09:44:21.381 INFO:teuthology.orchestra.run.smithi156.stdout:(17/33): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 400 kB/s | 43 kB 00:00 2024-07-16T09:44:21.424 INFO:teuthology.orchestra.run.smithi156.stdout:(18/33): python3-jaraco-8.2.1-3.el9.noarch.rpm 613 kB/s | 11 kB 00:00 2024-07-16T09:44:21.449 INFO:teuthology.orchestra.run.smithi156.stdout:(19/33): python3-cheroot-8.6.0-4.el9.noarch.rpm 1.1 MB/s | 172 kB 00:00 2024-07-16T09:44:21.466 INFO:teuthology.orchestra.run.smithi156.stdout:(20/33): python3-jaraco-classes-3.2.1-5.el9.noa 447 kB/s | 18 kB 00:00 2024-07-16T09:44:21.483 INFO:teuthology.orchestra.run.smithi156.stdout:(21/33): python3-jaraco-collections-3.0.0-8.el9 692 kB/s | 23 kB 00:00 2024-07-16T09:44:21.508 INFO:teuthology.orchestra.run.smithi156.stdout:(22/33): python3-jaraco-functools-3.5.0-2.el9.n 464 kB/s | 19 kB 00:00 2024-07-16T09:44:21.533 INFO:teuthology.orchestra.run.smithi156.stdout:(23/33): python3-jaraco-text-3.2.0-6.el9.noarch 393 kB/s | 20 kB 00:00 2024-07-16T09:44:21.558 INFO:teuthology.orchestra.run.smithi156.stdout:(24/33): python3-logutils-0.3.5-21.el9.noarch.r 922 kB/s | 46 kB 00:00 2024-07-16T09:44:21.584 INFO:teuthology.orchestra.run.smithi156.stdout:(25/33): python3-more-itertools-8.12.0-2.el9.no 1.5 MB/s | 79 kB 00:00 2024-07-16T09:44:21.609 INFO:teuthology.orchestra.run.smithi156.stdout:(26/33): python3-cherrypy-18.6.1-2.el9.noarch.r 1.4 MB/s | 358 kB 00:00 2024-07-16T09:44:21.634 INFO:teuthology.orchestra.run.smithi156.stdout:(27/33): python3-portend-3.1.0-2.el9.noarch.rpm 327 kB/s | 16 kB 00:00 2024-07-16T09:44:21.651 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: ceph-test-2:17.2.7-1104.ge6231b04.el9.x86_64 4/4 2024-07-16T09:44:21.651 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : ceph-test-2:17.2.7-1104.ge6231b04.el9.x86_64 1/4 2024-07-16T09:44:21.651 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : libcephsqlite-2:17.2.7-1104.ge6231b04.el9.x86_64 2/4 2024-07-16T09:44:21.651 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 3/4 2024-07-16T09:44:21.659 INFO:teuthology.orchestra.run.smithi156.stdout:(28/33): python3-pyOpenSSL-21.0.0-1.el9.noarch. 1.8 MB/s | 90 kB 00:00 2024-07-16T09:44:21.685 INFO:teuthology.orchestra.run.smithi156.stdout:(29/33): python3-pecan-1.4.2-3.el9.noarch.rpm 2.1 MB/s | 272 kB 00:00 2024-07-16T09:44:21.710 INFO:teuthology.orchestra.run.smithi156.stdout:(30/33): python3-tempora-5.0.0-2.el9.noarch.rpm 471 kB/s | 36 kB 00:00 2024-07-16T09:44:21.735 INFO:teuthology.orchestra.run.smithi156.stdout:(31/33): python3-webob-1.8.7-6.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-07-16T09:44:21.761 INFO:teuthology.orchestra.run.smithi156.stdout:(32/33): python3-zc-lockfile-2.0-10.el9.noarch. 398 kB/s | 20 kB 00:00 2024-07-16T09:44:21.786 INFO:teuthology.orchestra.run.smithi156.stdout:(33/33): python3-werkzeug-2.0.3-3.el9.1.noarch. 4.2 MB/s | 427 kB 00:00 2024-07-16T09:44:21.789 INFO:teuthology.orchestra.run.smithi156.stdout:-------------------------------------------------------------------------------- 2024-07-16T09:44:21.789 INFO:teuthology.orchestra.run.smithi156.stdout:Total 12 MB/s | 27 MB 00:02 2024-07-16T09:44:21.902 INFO:teuthology.orchestra.run.smithi156.stdout:Running transaction check 2024-07-16T09:44:21.908 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-07-16T09:44:21.908 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-07-16T09:44:21.908 INFO:teuthology.orchestra.run.smithi176.stdout:Installed: 2024-07-16T09:44:21.908 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-test-2:17.2.7-1104.ge6231b04.el9.x86_64 2024-07-16T09:44:21.908 INFO:teuthology.orchestra.run.smithi176.stdout: libcephsqlite-2:17.2.7-1104.ge6231b04.el9.x86_64 2024-07-16T09:44:21.908 INFO:teuthology.orchestra.run.smithi176.stdout: socat-1.7.4.1-6.el9.x86_64 2024-07-16T09:44:21.908 INFO:teuthology.orchestra.run.smithi176.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-07-16T09:44:21.908 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-07-16T09:44:21.908 INFO:teuthology.orchestra.run.smithi176.stdout:Complete! 2024-07-16T09:44:21.938 INFO:teuthology.orchestra.run.smithi156.stdout:Transaction check succeeded. 2024-07-16T09:44:21.938 INFO:teuthology.orchestra.run.smithi156.stdout:Running transaction test 2024-07-16T09:44:22.195 DEBUG:teuthology.orchestra.run.smithi176:> sudo yum -y install ceph 2024-07-16T09:44:22.299 INFO:teuthology.orchestra.run.smithi156.stdout:Transaction test succeeded. 2024-07-16T09:44:22.299 INFO:teuthology.orchestra.run.smithi156.stdout:Running transaction 2024-07-16T09:44:22.790 INFO:teuthology.orchestra.run.smithi176.stdout:Last metadata expiration check: 0:00:37 ago on Tue 16 Jul 2024 09:43:45 AM UTC. 2024-07-16T09:44:22.919 INFO:teuthology.orchestra.run.smithi176.stdout:Dependencies resolved. 2024-07-16T09:44:22.922 INFO:teuthology.orchestra.run.smithi176.stdout:================================================================================ 2024-07-16T09:44:22.922 INFO:teuthology.orchestra.run.smithi176.stdout: Package Arch Version Repository Size 2024-07-16T09:44:22.923 INFO:teuthology.orchestra.run.smithi176.stdout:================================================================================ 2024-07-16T09:44:22.923 INFO:teuthology.orchestra.run.smithi176.stdout:Installing: 2024-07-16T09:44:22.923 INFO:teuthology.orchestra.run.smithi176.stdout: ceph x86_64 2:17.2.7-1104.ge6231b04.el9 ceph 6.5 k 2024-07-16T09:44:22.923 INFO:teuthology.orchestra.run.smithi176.stdout:Installing dependencies: 2024-07-16T09:44:22.923 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-mds x86_64 2:17.2.7-1104.ge6231b04.el9 ceph 2.1 M 2024-07-16T09:44:22.924 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-mgr x86_64 2:17.2.7-1104.ge6231b04.el9 ceph 1.4 M 2024-07-16T09:44:22.924 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-mgr-modules-core noarch 2:17.2.7-1104.ge6231b04.el9 ceph-noarch 240 k 2024-07-16T09:44:22.924 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-mon x86_64 2:17.2.7-1104.ge6231b04.el9 ceph 3.9 M 2024-07-16T09:44:22.924 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-osd x86_64 2:17.2.7-1104.ge6231b04.el9 ceph 15 M 2024-07-16T09:44:22.924 INFO:teuthology.orchestra.run.smithi176.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-07-16T09:44:22.924 INFO:teuthology.orchestra.run.smithi176.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-07-16T09:44:22.925 INFO:teuthology.orchestra.run.smithi176.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-07-16T09:44:22.925 INFO:teuthology.orchestra.run.smithi176.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-07-16T09:44:22.925 INFO:teuthology.orchestra.run.smithi176.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-07-16T09:44:22.925 INFO:teuthology.orchestra.run.smithi176.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-07-16T09:44:22.925 INFO:teuthology.orchestra.run.smithi176.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-07-16T09:44:22.926 INFO:teuthology.orchestra.run.smithi176.stdout: python3-jaraco-collections 2024-07-16T09:44:22.926 INFO:teuthology.orchestra.run.smithi176.stdout: noarch 3.0.0-8.el9 epel 23 k 2024-07-16T09:44:22.926 INFO:teuthology.orchestra.run.smithi176.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-07-16T09:44:22.926 INFO:teuthology.orchestra.run.smithi176.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-07-16T09:44:22.926 INFO:teuthology.orchestra.run.smithi176.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-07-16T09:44:22.927 INFO:teuthology.orchestra.run.smithi176.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-07-16T09:44:22.927 INFO:teuthology.orchestra.run.smithi176.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-07-16T09:44:22.927 INFO:teuthology.orchestra.run.smithi176.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-07-16T09:44:22.927 INFO:teuthology.orchestra.run.smithi176.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-07-16T09:44:22.928 INFO:teuthology.orchestra.run.smithi176.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-07-16T09:44:22.928 INFO:teuthology.orchestra.run.smithi176.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-07-16T09:44:22.928 INFO:teuthology.orchestra.run.smithi176.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-07-16T09:44:22.928 INFO:teuthology.orchestra.run.smithi176.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-07-16T09:44:22.928 INFO:teuthology.orchestra.run.smithi176.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-07-16T09:44:22.929 INFO:teuthology.orchestra.run.smithi176.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-07-16T09:44:22.929 INFO:teuthology.orchestra.run.smithi176.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-07-16T09:44:22.929 INFO:teuthology.orchestra.run.smithi176.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-07-16T09:44:22.929 INFO:teuthology.orchestra.run.smithi176.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-07-16T09:44:22.929 INFO:teuthology.orchestra.run.smithi176.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-07-16T09:44:22.930 INFO:teuthology.orchestra.run.smithi176.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-07-16T09:44:22.930 INFO:teuthology.orchestra.run.smithi176.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-07-16T09:44:22.930 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-07-16T09:44:22.930 INFO:teuthology.orchestra.run.smithi176.stdout:Transaction Summary 2024-07-16T09:44:22.930 INFO:teuthology.orchestra.run.smithi176.stdout:================================================================================ 2024-07-16T09:44:22.931 INFO:teuthology.orchestra.run.smithi176.stdout:Install 33 Packages 2024-07-16T09:44:22.931 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-07-16T09:44:22.932 INFO:teuthology.orchestra.run.smithi176.stdout:Total download size: 27 M 2024-07-16T09:44:22.932 INFO:teuthology.orchestra.run.smithi176.stdout:Installed size: 96 M 2024-07-16T09:44:22.932 INFO:teuthology.orchestra.run.smithi176.stdout:Downloading Packages: 2024-07-16T09:44:23.348 INFO:teuthology.orchestra.run.smithi176.stdout:(1/33): ceph-17.2.7-1104.ge6231b04.el9.x86_64.r 35 kB/s | 6.5 kB 00:00 2024-07-16T09:44:23.454 INFO:teuthology.orchestra.run.smithi156.stdout: Preparing : 1/1 2024-07-16T09:44:23.507 INFO:teuthology.orchestra.run.smithi176.stdout:(2/33): ceph-mds-17.2.7-1104.ge6231b04.el9.x86_ 6.0 MB/s | 2.1 MB 00:00 2024-07-16T09:44:23.514 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/33 2024-07-16T09:44:23.545 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/33 2024-07-16T09:44:23.549 INFO:teuthology.orchestra.run.smithi176.stdout:(3/33): ceph-mgr-17.2.7-1104.ge6231b04.el9.x86_ 3.6 MB/s | 1.4 MB 00:00 2024-07-16T09:44:23.587 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 3/33 2024-07-16T09:44:23.641 INFO:teuthology.orchestra.run.smithi176.stdout:(4/33): ceph-mgr-modules-core-17.2.7-1104.ge623 2.6 MB/s | 240 kB 00:00 2024-07-16T09:44:23.678 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 4/33 2024-07-16T09:44:23.755 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 5/33 2024-07-16T09:44:23.798 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 6/33 2024-07-16T09:44:23.833 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 7/33 2024-07-16T09:44:23.862 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 8/33 2024-07-16T09:44:23.897 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 9/33 2024-07-16T09:44:23.930 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 10/33 2024-07-16T09:44:23.971 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 11/33 2024-07-16T09:44:23.974 INFO:teuthology.orchestra.run.smithi176.stdout:(5/33): ceph-osd-17.2.7-1104.ge6231b04.el9.x86_ 33 MB/s | 15 MB 00:00 2024-07-16T09:44:24.014 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 12/33 2024-07-16T09:44:24.049 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 13/33 2024-07-16T09:44:24.086 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 14/33 2024-07-16T09:44:24.108 INFO:teuthology.orchestra.run.smithi176.stdout:(6/33): ceph-mon-17.2.7-1104.ge6231b04.el9.x86_ 5.2 MB/s | 3.9 MB 00:00 2024-07-16T09:44:24.159 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 15/33 2024-07-16T09:44:24.299 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 16/33 2024-07-16T09:44:24.325 INFO:teuthology.orchestra.run.smithi176.stdout:(7/33): python3-cffi-1.14.5-5.el9.x86_64.rpm 370 kB/s | 253 kB 00:00 2024-07-16T09:44:24.378 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 17/33 2024-07-16T09:44:24.408 INFO:teuthology.orchestra.run.smithi176.stdout:(8/33): python3-ply-3.11-14.el9.noarch.rpm 355 kB/s | 106 kB 00:00 2024-07-16T09:44:24.432 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 18/33 2024-07-16T09:44:24.470 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 19/33 2024-07-16T09:44:24.517 INFO:teuthology.orchestra.run.smithi176.stdout:(9/33): python3-cryptography-36.0.1-4.el9.x86_6 2.3 MB/s | 1.2 MB 00:00 2024-07-16T09:44:24.518 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-ply-3.11-14.el9.noarch 20/33 2024-07-16T09:44:24.542 INFO:teuthology.orchestra.run.smithi176.stdout:(10/33): python3-pycparser-2.20-6.el9.noarch.rp 623 kB/s | 135 kB 00:00 2024-07-16T09:44:24.568 INFO:teuthology.orchestra.run.smithi176.stdout:(11/33): python3-requests-2.25.1-8.el9.noarch.r 787 kB/s | 125 kB 00:00 2024-07-16T09:44:24.582 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 21/33 2024-07-16T09:44:24.626 INFO:teuthology.orchestra.run.smithi176.stdout:(12/33): python3-urllib3-1.26.5-6.el9.noarch.rp 1.9 MB/s | 215 kB 00:00 2024-07-16T09:44:24.676 INFO:teuthology.orchestra.run.smithi176.stdout:(13/33): python3-markupsafe-1.1.1-12.el9.x86_64 321 kB/s | 35 kB 00:00 2024-07-16T09:44:24.760 INFO:teuthology.orchestra.run.smithi176.stdout:(14/33): python3-toml-0.10.2-6.el9.noarch.rpm 501 kB/s | 42 kB 00:00 2024-07-16T09:44:24.768 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 22/33 2024-07-16T09:44:24.785 INFO:teuthology.orchestra.run.smithi176.stdout:(15/33): python3-mako-1.1.4-6.el9.noarch.rpm 793 kB/s | 172 kB 00:00 2024-07-16T09:44:24.810 INFO:teuthology.orchestra.run.smithi176.stdout:(16/33): python3-pytz-2021.1-5.el9.noarch.rpm 277 kB/s | 51 kB 00:00 2024-07-16T09:44:24.816 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 23/33 2024-07-16T09:44:24.893 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 24/33 2024-07-16T09:44:24.919 INFO:teuthology.orchestra.run.smithi176.stdout:(17/33): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 273 kB/s | 43 kB 00:00 2024-07-16T09:44:24.944 INFO:teuthology.orchestra.run.smithi176.stdout:(18/33): python3-jaraco-8.2.1-3.el9.noarch.rpm 471 kB/s | 11 kB 00:00 2024-07-16T09:44:24.969 INFO:teuthology.orchestra.run.smithi176.stdout:(19/33): python3-cheroot-8.6.0-4.el9.noarch.rpm 933 kB/s | 172 kB 00:00 2024-07-16T09:44:24.982 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 25/33 2024-07-16T09:44:24.994 INFO:teuthology.orchestra.run.smithi176.stdout:(20/33): python3-jaraco-classes-3.2.1-5.el9.noa 352 kB/s | 18 kB 00:00 2024-07-16T09:44:25.020 INFO:teuthology.orchestra.run.smithi176.stdout:(21/33): python3-jaraco-collections-3.0.0-8.el9 460 kB/s | 23 kB 00:00 2024-07-16T09:44:25.032 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 26/33 2024-07-16T09:44:25.045 INFO:teuthology.orchestra.run.smithi176.stdout:(22/33): python3-jaraco-functools-3.5.0-2.el9.n 387 kB/s | 19 kB 00:00 2024-07-16T09:44:25.079 INFO:teuthology.orchestra.run.smithi176.stdout:(23/33): python3-jaraco-text-3.2.0-6.el9.noarch 334 kB/s | 20 kB 00:00 2024-07-16T09:44:25.105 INFO:teuthology.orchestra.run.smithi176.stdout:(24/33): python3-logutils-0.3.5-21.el9.noarch.r 785 kB/s | 46 kB 00:00 2024-07-16T09:44:25.130 INFO:teuthology.orchestra.run.smithi176.stdout:(25/33): python3-more-itertools-8.12.0-2.el9.no 1.5 MB/s | 79 kB 00:00 2024-07-16T09:44:25.145 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 27/33 2024-07-16T09:44:25.155 INFO:teuthology.orchestra.run.smithi176.stdout:(26/33): python3-cherrypy-18.6.1-2.el9.noarch.r 1.2 MB/s | 358 kB 00:00 2024-07-16T09:44:25.180 INFO:teuthology.orchestra.run.smithi176.stdout:(27/33): python3-portend-3.1.0-2.el9.noarch.rpm 327 kB/s | 16 kB 00:00 2024-07-16T09:44:25.197 INFO:teuthology.orchestra.run.smithi176.stdout:(28/33): python3-pyOpenSSL-21.0.0-1.el9.noarch. 2.1 MB/s | 90 kB 00:00 2024-07-16T09:44:25.222 INFO:teuthology.orchestra.run.smithi176.stdout:(29/33): python3-tempora-5.0.0-2.el9.noarch.rpm 854 kB/s | 36 kB 00:00 2024-07-16T09:44:25.248 INFO:teuthology.orchestra.run.smithi176.stdout:(30/33): python3-pecan-1.4.2-3.el9.noarch.rpm 1.9 MB/s | 272 kB 00:00 2024-07-16T09:44:25.273 INFO:teuthology.orchestra.run.smithi176.stdout:(31/33): python3-webob-1.8.7-6.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-07-16T09:44:25.298 INFO:teuthology.orchestra.run.smithi176.stdout:(32/33): python3-zc-lockfile-2.0-10.el9.noarch. 399 kB/s | 20 kB 00:00 2024-07-16T09:44:25.299 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-mgr-modules-core-2:17.2.7-1104.ge6231b04.el9. 28/33 2024-07-16T09:44:25.323 INFO:teuthology.orchestra.run.smithi176.stdout:(33/33): python3-werkzeug-2.0.3-3.el9.1.noarch. 4.2 MB/s | 427 kB 00:00 2024-07-16T09:44:25.325 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-mgr-2:17.2.7-1104.ge6231b04.el9.x86_64 29/33 2024-07-16T09:44:25.328 INFO:teuthology.orchestra.run.smithi176.stdout:-------------------------------------------------------------------------------- 2024-07-16T09:44:25.328 INFO:teuthology.orchestra.run.smithi176.stdout:Total 11 MB/s | 27 MB 00:02 2024-07-16T09:44:25.350 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: ceph-mgr-2:17.2.7-1104.ge6231b04.el9.x86_64 29/33 2024-07-16T09:44:25.351 INFO:teuthology.orchestra.run.smithi156.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-07-16T09:44:25.351 INFO:teuthology.orchestra.run.smithi156.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-07-16T09:44:25.351 INFO:teuthology.orchestra.run.smithi156.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-07-16T09:44:25.351 INFO:teuthology.orchestra.run.smithi156.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-07-16T09:44:25.351 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-07-16T09:44:25.442 INFO:teuthology.orchestra.run.smithi176.stdout:Running transaction check 2024-07-16T09:44:25.478 INFO:teuthology.orchestra.run.smithi176.stdout:Transaction check succeeded. 2024-07-16T09:44:25.478 INFO:teuthology.orchestra.run.smithi176.stdout:Running transaction test 2024-07-16T09:44:25.840 INFO:teuthology.orchestra.run.smithi176.stdout:Transaction test succeeded. 2024-07-16T09:44:25.841 INFO:teuthology.orchestra.run.smithi176.stdout:Running transaction 2024-07-16T09:44:26.660 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-osd-2:17.2.7-1104.ge6231b04.el9.x86_64 30/33 2024-07-16T09:44:26.696 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: ceph-osd-2:17.2.7-1104.ge6231b04.el9.x86_64 30/33 2024-07-16T09:44:26.696 INFO:teuthology.orchestra.run.smithi156.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-07-16T09:44:26.697 INFO:teuthology.orchestra.run.smithi156.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-07-16T09:44:26.697 INFO:teuthology.orchestra.run.smithi156.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-07-16T09:44:26.697 INFO:teuthology.orchestra.run.smithi156.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-07-16T09:44:26.697 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-07-16T09:44:26.986 INFO:teuthology.orchestra.run.smithi176.stdout: Preparing : 1/1 2024-07-16T09:44:27.084 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/33 2024-07-16T09:44:27.086 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-mon-2:17.2.7-1104.ge6231b04.el9.x86_64 31/33 2024-07-16T09:44:27.117 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/33 2024-07-16T09:44:27.125 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: ceph-mon-2:17.2.7-1104.ge6231b04.el9.x86_64 31/33 2024-07-16T09:44:27.125 INFO:teuthology.orchestra.run.smithi156.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-07-16T09:44:27.125 INFO:teuthology.orchestra.run.smithi156.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-07-16T09:44:27.125 INFO:teuthology.orchestra.run.smithi156.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-07-16T09:44:27.125 INFO:teuthology.orchestra.run.smithi156.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-07-16T09:44:27.125 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-07-16T09:44:27.149 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 3/33 2024-07-16T09:44:27.240 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 4/33 2024-07-16T09:44:27.319 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 5/33 2024-07-16T09:44:27.345 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-mds-2:17.2.7-1104.ge6231b04.el9.x86_64 32/33 2024-07-16T09:44:27.359 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 6/33 2024-07-16T09:44:27.370 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: ceph-mds-2:17.2.7-1104.ge6231b04.el9.x86_64 32/33 2024-07-16T09:44:27.370 INFO:teuthology.orchestra.run.smithi156.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-07-16T09:44:27.370 INFO:teuthology.orchestra.run.smithi156.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-07-16T09:44:27.370 INFO:teuthology.orchestra.run.smithi156.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-07-16T09:44:27.370 INFO:teuthology.orchestra.run.smithi156.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-07-16T09:44:27.370 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-07-16T09:44:27.395 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 7/33 2024-07-16T09:44:27.425 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 8/33 2024-07-16T09:44:27.444 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-2:17.2.7-1104.ge6231b04.el9.x86_64 33/33 2024-07-16T09:44:27.459 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 9/33 2024-07-16T09:44:27.493 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 10/33 2024-07-16T09:44:27.534 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 11/33 2024-07-16T09:44:27.568 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 12/33 2024-07-16T09:44:27.603 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 13/33 2024-07-16T09:44:27.652 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 14/33 2024-07-16T09:44:27.722 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 15/33 2024-07-16T09:44:27.869 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 16/33 2024-07-16T09:44:27.949 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 17/33 2024-07-16T09:44:28.002 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 18/33 2024-07-16T09:44:28.041 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 19/33 2024-07-16T09:44:28.099 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-ply-3.11-14.el9.noarch 20/33 2024-07-16T09:44:28.153 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 21/33 2024-07-16T09:44:28.340 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 22/33 2024-07-16T09:44:28.388 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 23/33 2024-07-16T09:44:28.464 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 24/33 2024-07-16T09:44:28.553 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 25/33 2024-07-16T09:44:28.594 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 26/33 2024-07-16T09:44:28.712 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 27/33 2024-07-16T09:44:28.864 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : ceph-mgr-modules-core-2:17.2.7-1104.ge6231b04.el9. 28/33 2024-07-16T09:44:28.897 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : ceph-mgr-2:17.2.7-1104.ge6231b04.el9.x86_64 29/33 2024-07-16T09:44:28.922 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: ceph-mgr-2:17.2.7-1104.ge6231b04.el9.x86_64 29/33 2024-07-16T09:44:28.923 INFO:teuthology.orchestra.run.smithi176.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-07-16T09:44:28.923 INFO:teuthology.orchestra.run.smithi176.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-07-16T09:44:28.923 INFO:teuthology.orchestra.run.smithi176.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-07-16T09:44:28.923 INFO:teuthology.orchestra.run.smithi176.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-07-16T09:44:28.923 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-07-16T09:44:29.075 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: ceph-2:17.2.7-1104.ge6231b04.el9.x86_64 33/33 2024-07-16T09:44:29.076 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-2:17.2.7-1104.ge6231b04.el9.x86_64 1/33 2024-07-16T09:44:29.076 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-mds-2:17.2.7-1104.ge6231b04.el9.x86_64 2/33 2024-07-16T09:44:29.076 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-mgr-2:17.2.7-1104.ge6231b04.el9.x86_64 3/33 2024-07-16T09:44:29.076 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-mon-2:17.2.7-1104.ge6231b04.el9.x86_64 4/33 2024-07-16T09:44:29.076 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-osd-2:17.2.7-1104.ge6231b04.el9.x86_64 5/33 2024-07-16T09:44:29.076 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-mgr-modules-core-2:17.2.7-1104.ge6231b04.el9. 6/33 2024-07-16T09:44:29.076 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/33 2024-07-16T09:44:29.076 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/33 2024-07-16T09:44:29.076 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/33 2024-07-16T09:44:29.076 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/33 2024-07-16T09:44:29.076 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/33 2024-07-16T09:44:29.077 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 12/33 2024-07-16T09:44:29.077 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 13/33 2024-07-16T09:44:29.077 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 14/33 2024-07-16T09:44:29.077 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 15/33 2024-07-16T09:44:29.077 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 16/33 2024-07-16T09:44:29.077 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 17/33 2024-07-16T09:44:29.077 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 18/33 2024-07-16T09:44:29.077 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 19/33 2024-07-16T09:44:29.077 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 20/33 2024-07-16T09:44:29.077 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 21/33 2024-07-16T09:44:29.077 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 22/33 2024-07-16T09:44:29.077 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 23/33 2024-07-16T09:44:29.078 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 24/33 2024-07-16T09:44:29.078 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 25/33 2024-07-16T09:44:29.078 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 26/33 2024-07-16T09:44:29.078 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 27/33 2024-07-16T09:44:29.078 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 28/33 2024-07-16T09:44:29.078 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 29/33 2024-07-16T09:44:29.078 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 30/33 2024-07-16T09:44:29.078 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 31/33 2024-07-16T09:44:29.078 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 32/33 2024-07-16T09:44:30.030 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 33/33 2024-07-16T09:44:30.030 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-07-16T09:44:30.030 INFO:teuthology.orchestra.run.smithi156.stdout:Installed: 2024-07-16T09:44:30.030 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-2:17.2.7-1104.ge6231b04.el9.x86_64 2024-07-16T09:44:30.031 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mds-2:17.2.7-1104.ge6231b04.el9.x86_64 2024-07-16T09:44:30.031 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mgr-2:17.2.7-1104.ge6231b04.el9.x86_64 2024-07-16T09:44:30.031 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mgr-modules-core-2:17.2.7-1104.ge6231b04.el9.noarch 2024-07-16T09:44:30.031 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mon-2:17.2.7-1104.ge6231b04.el9.x86_64 2024-07-16T09:44:30.031 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-osd-2:17.2.7-1104.ge6231b04.el9.x86_64 2024-07-16T09:44:30.031 INFO:teuthology.orchestra.run.smithi156.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-07-16T09:44:30.031 INFO:teuthology.orchestra.run.smithi156.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-07-16T09:44:30.031 INFO:teuthology.orchestra.run.smithi156.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-07-16T09:44:30.031 INFO:teuthology.orchestra.run.smithi156.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-07-16T09:44:30.031 INFO:teuthology.orchestra.run.smithi156.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-07-16T09:44:30.031 INFO:teuthology.orchestra.run.smithi156.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-07-16T09:44:30.031 INFO:teuthology.orchestra.run.smithi156.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-07-16T09:44:30.031 INFO:teuthology.orchestra.run.smithi156.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-07-16T09:44:30.032 INFO:teuthology.orchestra.run.smithi156.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-07-16T09:44:30.032 INFO:teuthology.orchestra.run.smithi156.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-07-16T09:44:30.032 INFO:teuthology.orchestra.run.smithi156.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-07-16T09:44:30.032 INFO:teuthology.orchestra.run.smithi156.stdout: python3-mako-1.1.4-6.el9.noarch 2024-07-16T09:44:30.032 INFO:teuthology.orchestra.run.smithi156.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-07-16T09:44:30.032 INFO:teuthology.orchestra.run.smithi156.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-07-16T09:44:30.032 INFO:teuthology.orchestra.run.smithi156.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-07-16T09:44:30.032 INFO:teuthology.orchestra.run.smithi156.stdout: python3-ply-3.11-14.el9.noarch 2024-07-16T09:44:30.032 INFO:teuthology.orchestra.run.smithi156.stdout: python3-portend-3.1.0-2.el9.noarch 2024-07-16T09:44:30.032 INFO:teuthology.orchestra.run.smithi156.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-07-16T09:44:30.032 INFO:teuthology.orchestra.run.smithi156.stdout: python3-pycparser-2.20-6.el9.noarch 2024-07-16T09:44:30.032 INFO:teuthology.orchestra.run.smithi156.stdout: python3-pytz-2021.1-5.el9.noarch 2024-07-16T09:44:30.032 INFO:teuthology.orchestra.run.smithi156.stdout: python3-requests-2.25.1-8.el9.noarch 2024-07-16T09:44:30.033 INFO:teuthology.orchestra.run.smithi156.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-07-16T09:44:30.033 INFO:teuthology.orchestra.run.smithi156.stdout: python3-toml-0.10.2-6.el9.noarch 2024-07-16T09:44:30.033 INFO:teuthology.orchestra.run.smithi156.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-07-16T09:44:30.033 INFO:teuthology.orchestra.run.smithi156.stdout: python3-webob-1.8.7-6.el9.noarch 2024-07-16T09:44:30.033 INFO:teuthology.orchestra.run.smithi156.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-07-16T09:44:30.033 INFO:teuthology.orchestra.run.smithi156.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-07-16T09:44:30.033 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-07-16T09:44:30.033 INFO:teuthology.orchestra.run.smithi156.stdout:Complete! 2024-07-16T09:44:30.231 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : ceph-osd-2:17.2.7-1104.ge6231b04.el9.x86_64 30/33 2024-07-16T09:44:30.259 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: ceph-osd-2:17.2.7-1104.ge6231b04.el9.x86_64 30/33 2024-07-16T09:44:30.259 INFO:teuthology.orchestra.run.smithi176.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-07-16T09:44:30.259 INFO:teuthology.orchestra.run.smithi176.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-07-16T09:44:30.259 INFO:teuthology.orchestra.run.smithi176.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-07-16T09:44:30.259 INFO:teuthology.orchestra.run.smithi176.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-07-16T09:44:30.259 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-07-16T09:44:30.377 DEBUG:teuthology.orchestra.run.smithi156:> sudo yum -y install ceph-base 2024-07-16T09:44:30.649 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : ceph-mon-2:17.2.7-1104.ge6231b04.el9.x86_64 31/33 2024-07-16T09:44:30.675 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: ceph-mon-2:17.2.7-1104.ge6231b04.el9.x86_64 31/33 2024-07-16T09:44:30.675 INFO:teuthology.orchestra.run.smithi176.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-07-16T09:44:30.675 INFO:teuthology.orchestra.run.smithi176.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-07-16T09:44:30.675 INFO:teuthology.orchestra.run.smithi176.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-07-16T09:44:30.675 INFO:teuthology.orchestra.run.smithi176.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-07-16T09:44:30.676 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-07-16T09:44:30.891 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : ceph-mds-2:17.2.7-1104.ge6231b04.el9.x86_64 32/33 2024-07-16T09:44:30.916 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: ceph-mds-2:17.2.7-1104.ge6231b04.el9.x86_64 32/33 2024-07-16T09:44:30.916 INFO:teuthology.orchestra.run.smithi176.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-07-16T09:44:30.916 INFO:teuthology.orchestra.run.smithi176.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-07-16T09:44:30.916 INFO:teuthology.orchestra.run.smithi176.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-07-16T09:44:30.917 INFO:teuthology.orchestra.run.smithi176.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-07-16T09:44:30.917 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-07-16T09:44:30.965 INFO:teuthology.orchestra.run.smithi156.stdout:Last metadata expiration check: 0:00:50 ago on Tue 16 Jul 2024 09:43:40 AM UTC. 2024-07-16T09:44:30.981 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : ceph-2:17.2.7-1104.ge6231b04.el9.x86_64 33/33 2024-07-16T09:44:31.050 INFO:teuthology.orchestra.run.smithi156.stdout:Package ceph-base-2:17.2.7-1104.ge6231b04.el9.x86_64 is already installed. 2024-07-16T09:44:31.092 INFO:teuthology.orchestra.run.smithi156.stdout:Dependencies resolved. 2024-07-16T09:44:31.093 INFO:teuthology.orchestra.run.smithi156.stdout:Nothing to do. 2024-07-16T09:44:31.093 INFO:teuthology.orchestra.run.smithi156.stdout:Complete! 2024-07-16T09:44:31.157 DEBUG:teuthology.orchestra.run.smithi156:> sudo yum -y install cephadm 2024-07-16T09:44:31.746 INFO:teuthology.orchestra.run.smithi156.stdout:Last metadata expiration check: 0:00:51 ago on Tue 16 Jul 2024 09:43:40 AM UTC. 2024-07-16T09:44:31.874 INFO:teuthology.orchestra.run.smithi156.stdout:Dependencies resolved. 2024-07-16T09:44:31.875 INFO:teuthology.orchestra.run.smithi156.stdout:================================================================================ 2024-07-16T09:44:31.875 INFO:teuthology.orchestra.run.smithi156.stdout: Package Arch Version Repository Size 2024-07-16T09:44:31.875 INFO:teuthology.orchestra.run.smithi156.stdout:================================================================================ 2024-07-16T09:44:31.875 INFO:teuthology.orchestra.run.smithi156.stdout:Installing: 2024-07-16T09:44:31.876 INFO:teuthology.orchestra.run.smithi156.stdout: cephadm noarch 2:17.2.7-1104.ge6231b04.el9 ceph-noarch 82 k 2024-07-16T09:44:31.876 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-07-16T09:44:31.876 INFO:teuthology.orchestra.run.smithi156.stdout:Transaction Summary 2024-07-16T09:44:31.876 INFO:teuthology.orchestra.run.smithi156.stdout:================================================================================ 2024-07-16T09:44:31.876 INFO:teuthology.orchestra.run.smithi156.stdout:Install 1 Package 2024-07-16T09:44:31.876 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-07-16T09:44:31.876 INFO:teuthology.orchestra.run.smithi156.stdout:Total download size: 82 k 2024-07-16T09:44:31.876 INFO:teuthology.orchestra.run.smithi156.stdout:Installed size: 372 k 2024-07-16T09:44:31.876 INFO:teuthology.orchestra.run.smithi156.stdout:Downloading Packages: 2024-07-16T09:44:32.103 INFO:teuthology.orchestra.run.smithi156.stdout:cephadm-17.2.7-1104.ge6231b04.el9.noarch.rpm 363 kB/s | 82 kB 00:00 2024-07-16T09:44:32.103 INFO:teuthology.orchestra.run.smithi156.stdout:-------------------------------------------------------------------------------- 2024-07-16T09:44:32.103 INFO:teuthology.orchestra.run.smithi156.stdout:Total 360 kB/s | 82 kB 00:00 2024-07-16T09:44:32.104 INFO:teuthology.orchestra.run.smithi156.stdout:Running transaction check 2024-07-16T09:44:32.109 INFO:teuthology.orchestra.run.smithi156.stdout:Transaction check succeeded. 2024-07-16T09:44:32.109 INFO:teuthology.orchestra.run.smithi156.stdout:Running transaction test 2024-07-16T09:44:32.111 INFO:teuthology.orchestra.run.smithi156.stdout:Transaction test succeeded. 2024-07-16T09:44:32.111 INFO:teuthology.orchestra.run.smithi156.stdout:Running transaction 2024-07-16T09:44:32.180 INFO:teuthology.orchestra.run.smithi156.stdout: Preparing : 1/1 2024-07-16T09:44:32.556 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: ceph-2:17.2.7-1104.ge6231b04.el9.x86_64 33/33 2024-07-16T09:44:32.556 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : ceph-2:17.2.7-1104.ge6231b04.el9.x86_64 1/33 2024-07-16T09:44:32.556 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : ceph-mds-2:17.2.7-1104.ge6231b04.el9.x86_64 2/33 2024-07-16T09:44:32.556 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : ceph-mgr-2:17.2.7-1104.ge6231b04.el9.x86_64 3/33 2024-07-16T09:44:32.557 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : ceph-mon-2:17.2.7-1104.ge6231b04.el9.x86_64 4/33 2024-07-16T09:44:32.557 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : ceph-osd-2:17.2.7-1104.ge6231b04.el9.x86_64 5/33 2024-07-16T09:44:32.557 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : ceph-mgr-modules-core-2:17.2.7-1104.ge6231b04.el9. 6/33 2024-07-16T09:44:32.557 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/33 2024-07-16T09:44:32.557 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/33 2024-07-16T09:44:32.557 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/33 2024-07-16T09:44:32.557 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/33 2024-07-16T09:44:32.557 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/33 2024-07-16T09:44:32.557 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 12/33 2024-07-16T09:44:32.557 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 13/33 2024-07-16T09:44:32.557 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 14/33 2024-07-16T09:44:32.557 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 15/33 2024-07-16T09:44:32.558 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 16/33 2024-07-16T09:44:32.558 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 17/33 2024-07-16T09:44:32.558 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 18/33 2024-07-16T09:44:32.558 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 19/33 2024-07-16T09:44:32.558 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 20/33 2024-07-16T09:44:32.558 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 21/33 2024-07-16T09:44:32.558 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 22/33 2024-07-16T09:44:32.558 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 23/33 2024-07-16T09:44:32.558 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 24/33 2024-07-16T09:44:32.558 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 25/33 2024-07-16T09:44:32.558 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 26/33 2024-07-16T09:44:32.558 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 27/33 2024-07-16T09:44:32.558 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 28/33 2024-07-16T09:44:32.559 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 29/33 2024-07-16T09:44:32.559 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 30/33 2024-07-16T09:44:32.559 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 31/33 2024-07-16T09:44:32.559 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 32/33 2024-07-16T09:44:32.585 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: cephadm-2:17.2.7-1104.ge6231b04.el9.noarch 1/1 2024-07-16T09:44:32.674 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : cephadm-2:17.2.7-1104.ge6231b04.el9.noarch 1/1 2024-07-16T09:44:33.009 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: cephadm-2:17.2.7-1104.ge6231b04.el9.noarch 1/1 2024-07-16T09:44:33.252 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : cephadm-2:17.2.7-1104.ge6231b04.el9.noarch 1/1 2024-07-16T09:44:33.252 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-07-16T09:44:33.252 INFO:teuthology.orchestra.run.smithi156.stdout:Installed: 2024-07-16T09:44:33.252 INFO:teuthology.orchestra.run.smithi156.stdout: cephadm-2:17.2.7-1104.ge6231b04.el9.noarch 2024-07-16T09:44:33.252 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-07-16T09:44:33.252 INFO:teuthology.orchestra.run.smithi156.stdout:Complete! 2024-07-16T09:44:33.466 DEBUG:teuthology.orchestra.run.smithi156:> sudo yum -y install ceph-immutable-object-cache 2024-07-16T09:44:33.569 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 33/33 2024-07-16T09:44:33.569 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-07-16T09:44:33.569 INFO:teuthology.orchestra.run.smithi176.stdout:Installed: 2024-07-16T09:44:33.569 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-2:17.2.7-1104.ge6231b04.el9.x86_64 2024-07-16T09:44:33.569 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-mds-2:17.2.7-1104.ge6231b04.el9.x86_64 2024-07-16T09:44:33.569 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-mgr-2:17.2.7-1104.ge6231b04.el9.x86_64 2024-07-16T09:44:33.569 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-mgr-modules-core-2:17.2.7-1104.ge6231b04.el9.noarch 2024-07-16T09:44:33.569 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-mon-2:17.2.7-1104.ge6231b04.el9.x86_64 2024-07-16T09:44:33.569 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-osd-2:17.2.7-1104.ge6231b04.el9.x86_64 2024-07-16T09:44:33.570 INFO:teuthology.orchestra.run.smithi176.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-07-16T09:44:33.570 INFO:teuthology.orchestra.run.smithi176.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-07-16T09:44:33.570 INFO:teuthology.orchestra.run.smithi176.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-07-16T09:44:33.570 INFO:teuthology.orchestra.run.smithi176.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-07-16T09:44:33.570 INFO:teuthology.orchestra.run.smithi176.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-07-16T09:44:33.570 INFO:teuthology.orchestra.run.smithi176.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-07-16T09:44:33.570 INFO:teuthology.orchestra.run.smithi176.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-07-16T09:44:33.570 INFO:teuthology.orchestra.run.smithi176.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-07-16T09:44:33.570 INFO:teuthology.orchestra.run.smithi176.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-07-16T09:44:33.570 INFO:teuthology.orchestra.run.smithi176.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-07-16T09:44:33.570 INFO:teuthology.orchestra.run.smithi176.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-07-16T09:44:33.570 INFO:teuthology.orchestra.run.smithi176.stdout: python3-mako-1.1.4-6.el9.noarch 2024-07-16T09:44:33.570 INFO:teuthology.orchestra.run.smithi176.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-07-16T09:44:33.571 INFO:teuthology.orchestra.run.smithi176.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-07-16T09:44:33.571 INFO:teuthology.orchestra.run.smithi176.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-07-16T09:44:33.571 INFO:teuthology.orchestra.run.smithi176.stdout: python3-ply-3.11-14.el9.noarch 2024-07-16T09:44:33.571 INFO:teuthology.orchestra.run.smithi176.stdout: python3-portend-3.1.0-2.el9.noarch 2024-07-16T09:44:33.571 INFO:teuthology.orchestra.run.smithi176.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-07-16T09:44:33.571 INFO:teuthology.orchestra.run.smithi176.stdout: python3-pycparser-2.20-6.el9.noarch 2024-07-16T09:44:33.571 INFO:teuthology.orchestra.run.smithi176.stdout: python3-pytz-2021.1-5.el9.noarch 2024-07-16T09:44:33.571 INFO:teuthology.orchestra.run.smithi176.stdout: python3-requests-2.25.1-8.el9.noarch 2024-07-16T09:44:33.571 INFO:teuthology.orchestra.run.smithi176.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-07-16T09:44:33.571 INFO:teuthology.orchestra.run.smithi176.stdout: python3-toml-0.10.2-6.el9.noarch 2024-07-16T09:44:33.571 INFO:teuthology.orchestra.run.smithi176.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-07-16T09:44:33.571 INFO:teuthology.orchestra.run.smithi176.stdout: python3-webob-1.8.7-6.el9.noarch 2024-07-16T09:44:33.571 INFO:teuthology.orchestra.run.smithi176.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-07-16T09:44:33.571 INFO:teuthology.orchestra.run.smithi176.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-07-16T09:44:33.572 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-07-16T09:44:33.572 INFO:teuthology.orchestra.run.smithi176.stdout:Complete! 2024-07-16T09:44:33.955 DEBUG:teuthology.orchestra.run.smithi176:> sudo yum -y install ceph-base 2024-07-16T09:44:34.061 INFO:teuthology.orchestra.run.smithi156.stdout:Last metadata expiration check: 0:00:54 ago on Tue 16 Jul 2024 09:43:40 AM UTC. 2024-07-16T09:44:34.195 INFO:teuthology.orchestra.run.smithi156.stdout:Dependencies resolved. 2024-07-16T09:44:34.196 INFO:teuthology.orchestra.run.smithi156.stdout:================================================================================ 2024-07-16T09:44:34.196 INFO:teuthology.orchestra.run.smithi156.stdout: Package Arch Version Repo Size 2024-07-16T09:44:34.196 INFO:teuthology.orchestra.run.smithi156.stdout:================================================================================ 2024-07-16T09:44:34.196 INFO:teuthology.orchestra.run.smithi156.stdout:Installing: 2024-07-16T09:44:34.196 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-immutable-object-cache x86_64 2:17.2.7-1104.ge6231b04.el9 ceph 137 k 2024-07-16T09:44:34.196 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-07-16T09:44:34.196 INFO:teuthology.orchestra.run.smithi156.stdout:Transaction Summary 2024-07-16T09:44:34.196 INFO:teuthology.orchestra.run.smithi156.stdout:================================================================================ 2024-07-16T09:44:34.197 INFO:teuthology.orchestra.run.smithi156.stdout:Install 1 Package 2024-07-16T09:44:34.197 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-07-16T09:44:34.197 INFO:teuthology.orchestra.run.smithi156.stdout:Total download size: 137 k 2024-07-16T09:44:34.197 INFO:teuthology.orchestra.run.smithi156.stdout:Installed size: 379 k 2024-07-16T09:44:34.197 INFO:teuthology.orchestra.run.smithi156.stdout:Downloading Packages: 2024-07-16T09:44:34.441 INFO:teuthology.orchestra.run.smithi156.stdout:ceph-immutable-object-cache-17.2.7-1104.ge6231b 561 kB/s | 137 kB 00:00 2024-07-16T09:44:34.442 INFO:teuthology.orchestra.run.smithi156.stdout:-------------------------------------------------------------------------------- 2024-07-16T09:44:34.442 INFO:teuthology.orchestra.run.smithi156.stdout:Total 558 kB/s | 137 kB 00:00 2024-07-16T09:44:34.442 INFO:teuthology.orchestra.run.smithi156.stdout:Running transaction check 2024-07-16T09:44:34.453 INFO:teuthology.orchestra.run.smithi156.stdout:Transaction check succeeded. 2024-07-16T09:44:34.454 INFO:teuthology.orchestra.run.smithi156.stdout:Running transaction test 2024-07-16T09:44:34.506 INFO:teuthology.orchestra.run.smithi156.stdout:Transaction test succeeded. 2024-07-16T09:44:34.506 INFO:teuthology.orchestra.run.smithi156.stdout:Running transaction 2024-07-16T09:44:34.543 INFO:teuthology.orchestra.run.smithi176.stdout:Last metadata expiration check: 0:00:49 ago on Tue 16 Jul 2024 09:43:45 AM UTC. 2024-07-16T09:44:34.629 INFO:teuthology.orchestra.run.smithi176.stdout:Package ceph-base-2:17.2.7-1104.ge6231b04.el9.x86_64 is already installed. 2024-07-16T09:44:34.643 INFO:teuthology.orchestra.run.smithi156.stdout: Preparing : 1/1 2024-07-16T09:44:34.672 INFO:teuthology.orchestra.run.smithi176.stdout:Dependencies resolved. 2024-07-16T09:44:34.673 INFO:teuthology.orchestra.run.smithi176.stdout:Nothing to do. 2024-07-16T09:44:34.673 INFO:teuthology.orchestra.run.smithi176.stdout:Complete! 2024-07-16T09:44:34.693 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-immutable-object-cache-2:17.2.7-1104.ge6231b04. 1/1 2024-07-16T09:44:34.718 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: ceph-immutable-object-cache-2:17.2.7-1104.ge6231b04. 1/1 2024-07-16T09:44:34.718 INFO:teuthology.orchestra.run.smithi156.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-07-16T09:44:34.718 INFO:teuthology.orchestra.run.smithi156.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-07-16T09:44:34.718 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-07-16T09:44:34.749 DEBUG:teuthology.orchestra.run.smithi176:> sudo yum -y install cephadm 2024-07-16T09:44:35.327 INFO:teuthology.orchestra.run.smithi176.stdout:Last metadata expiration check: 0:00:50 ago on Tue 16 Jul 2024 09:43:45 AM UTC. 2024-07-16T09:44:35.454 INFO:teuthology.orchestra.run.smithi176.stdout:Dependencies resolved. 2024-07-16T09:44:35.455 INFO:teuthology.orchestra.run.smithi176.stdout:================================================================================ 2024-07-16T09:44:35.455 INFO:teuthology.orchestra.run.smithi176.stdout: Package Arch Version Repository Size 2024-07-16T09:44:35.455 INFO:teuthology.orchestra.run.smithi176.stdout:================================================================================ 2024-07-16T09:44:35.455 INFO:teuthology.orchestra.run.smithi176.stdout:Installing: 2024-07-16T09:44:35.455 INFO:teuthology.orchestra.run.smithi176.stdout: cephadm noarch 2:17.2.7-1104.ge6231b04.el9 ceph-noarch 82 k 2024-07-16T09:44:35.455 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-07-16T09:44:35.455 INFO:teuthology.orchestra.run.smithi176.stdout:Transaction Summary 2024-07-16T09:44:35.455 INFO:teuthology.orchestra.run.smithi176.stdout:================================================================================ 2024-07-16T09:44:35.455 INFO:teuthology.orchestra.run.smithi176.stdout:Install 1 Package 2024-07-16T09:44:35.455 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-07-16T09:44:35.455 INFO:teuthology.orchestra.run.smithi176.stdout:Total download size: 82 k 2024-07-16T09:44:35.456 INFO:teuthology.orchestra.run.smithi176.stdout:Installed size: 372 k 2024-07-16T09:44:35.456 INFO:teuthology.orchestra.run.smithi176.stdout:Downloading Packages: 2024-07-16T09:44:35.666 INFO:teuthology.orchestra.run.smithi176.stdout:cephadm-17.2.7-1104.ge6231b04.el9.noarch.rpm 390 kB/s | 82 kB 00:00 2024-07-16T09:44:35.667 INFO:teuthology.orchestra.run.smithi176.stdout:-------------------------------------------------------------------------------- 2024-07-16T09:44:35.667 INFO:teuthology.orchestra.run.smithi176.stdout:Total 387 kB/s | 82 kB 00:00 2024-07-16T09:44:35.667 INFO:teuthology.orchestra.run.smithi176.stdout:Running transaction check 2024-07-16T09:44:35.672 INFO:teuthology.orchestra.run.smithi176.stdout:Transaction check succeeded. 2024-07-16T09:44:35.672 INFO:teuthology.orchestra.run.smithi176.stdout:Running transaction test 2024-07-16T09:44:35.673 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-immutable-object-cache-2:17.2.7-1104.ge6231b04. 1/1 2024-07-16T09:44:35.673 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-07-16T09:44:35.673 INFO:teuthology.orchestra.run.smithi156.stdout:Installed: 2024-07-16T09:44:35.673 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-immutable-object-cache-2:17.2.7-1104.ge6231b04.el9.x86_64 2024-07-16T09:44:35.673 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-07-16T09:44:35.673 INFO:teuthology.orchestra.run.smithi156.stdout:Complete! 2024-07-16T09:44:35.674 INFO:teuthology.orchestra.run.smithi176.stdout:Transaction test succeeded. 2024-07-16T09:44:35.675 INFO:teuthology.orchestra.run.smithi176.stdout:Running transaction 2024-07-16T09:44:35.743 INFO:teuthology.orchestra.run.smithi176.stdout: Preparing : 1/1 2024-07-16T09:44:35.904 DEBUG:teuthology.orchestra.run.smithi156:> sudo yum -y install ceph-mgr 2024-07-16T09:44:36.141 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: cephadm-2:17.2.7-1104.ge6231b04.el9.noarch 1/1 2024-07-16T09:44:36.227 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : cephadm-2:17.2.7-1104.ge6231b04.el9.noarch 1/1 2024-07-16T09:44:36.504 INFO:teuthology.orchestra.run.smithi156.stdout:Last metadata expiration check: 0:00:56 ago on Tue 16 Jul 2024 09:43:40 AM UTC. 2024-07-16T09:44:36.590 INFO:teuthology.orchestra.run.smithi156.stdout:Package ceph-mgr-2:17.2.7-1104.ge6231b04.el9.x86_64 is already installed. 2024-07-16T09:44:36.602 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: cephadm-2:17.2.7-1104.ge6231b04.el9.noarch 1/1 2024-07-16T09:44:36.632 INFO:teuthology.orchestra.run.smithi156.stdout:Dependencies resolved. 2024-07-16T09:44:36.632 INFO:teuthology.orchestra.run.smithi156.stdout:Nothing to do. 2024-07-16T09:44:36.633 INFO:teuthology.orchestra.run.smithi156.stdout:Complete! 2024-07-16T09:44:36.709 DEBUG:teuthology.orchestra.run.smithi156:> sudo yum -y install ceph-mgr-dashboard 2024-07-16T09:44:36.843 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : cephadm-2:17.2.7-1104.ge6231b04.el9.noarch 1/1 2024-07-16T09:44:36.843 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-07-16T09:44:36.843 INFO:teuthology.orchestra.run.smithi176.stdout:Installed: 2024-07-16T09:44:36.843 INFO:teuthology.orchestra.run.smithi176.stdout: cephadm-2:17.2.7-1104.ge6231b04.el9.noarch 2024-07-16T09:44:36.843 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-07-16T09:44:36.843 INFO:teuthology.orchestra.run.smithi176.stdout:Complete! 2024-07-16T09:44:37.089 DEBUG:teuthology.orchestra.run.smithi176:> sudo yum -y install ceph-immutable-object-cache 2024-07-16T09:44:37.297 INFO:teuthology.orchestra.run.smithi156.stdout:Last metadata expiration check: 0:00:57 ago on Tue 16 Jul 2024 09:43:40 AM UTC. 2024-07-16T09:44:37.433 INFO:teuthology.orchestra.run.smithi156.stdout:Dependencies resolved. 2024-07-16T09:44:37.433 INFO:teuthology.orchestra.run.smithi156.stdout:================================================================================ 2024-07-16T09:44:37.434 INFO:teuthology.orchestra.run.smithi156.stdout: Package Arch Version Repository Size 2024-07-16T09:44:37.434 INFO:teuthology.orchestra.run.smithi156.stdout:================================================================================ 2024-07-16T09:44:37.434 INFO:teuthology.orchestra.run.smithi156.stdout:Installing: 2024-07-16T09:44:37.434 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mgr-dashboard noarch 2:17.2.7-1104.ge6231b04.el9 ceph-noarch 3.4 M 2024-07-16T09:44:37.434 INFO:teuthology.orchestra.run.smithi156.stdout:Installing dependencies: 2024-07-16T09:44:37.434 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-grafana-dashboards noarch 2:17.2.7-1104.ge6231b04.el9 ceph-noarch 24 k 2024-07-16T09:44:37.434 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-prometheus-alerts noarch 2:17.2.7-1104.ge6231b04.el9 ceph-noarch 15 k 2024-07-16T09:44:37.434 INFO:teuthology.orchestra.run.smithi156.stdout: python3-jwt noarch 2.4.0-1.el9 epel 41 k 2024-07-16T09:44:37.434 INFO:teuthology.orchestra.run.smithi156.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-07-16T09:44:37.434 INFO:teuthology.orchestra.run.smithi156.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-07-16T09:44:37.434 INFO:teuthology.orchestra.run.smithi156.stdout:Installing weak dependencies: 2024-07-16T09:44:37.435 INFO:teuthology.orchestra.run.smithi156.stdout: python3-jwt+crypto noarch 2.4.0-1.el9 epel 9.0 k 2024-07-16T09:44:37.435 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-07-16T09:44:37.435 INFO:teuthology.orchestra.run.smithi156.stdout:Transaction Summary 2024-07-16T09:44:37.435 INFO:teuthology.orchestra.run.smithi156.stdout:================================================================================ 2024-07-16T09:44:37.435 INFO:teuthology.orchestra.run.smithi156.stdout:Install 7 Packages 2024-07-16T09:44:37.435 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-07-16T09:44:37.435 INFO:teuthology.orchestra.run.smithi156.stdout:Total download size: 3.7 M 2024-07-16T09:44:37.436 INFO:teuthology.orchestra.run.smithi156.stdout:Installed size: 80 M 2024-07-16T09:44:37.436 INFO:teuthology.orchestra.run.smithi156.stdout:Downloading Packages: 2024-07-16T09:44:37.679 INFO:teuthology.orchestra.run.smithi176.stdout:Last metadata expiration check: 0:00:52 ago on Tue 16 Jul 2024 09:43:45 AM UTC. 2024-07-16T09:44:37.779 INFO:teuthology.orchestra.run.smithi156.stdout:(1/7): ceph-prometheus-alerts-17.2.7-1104.ge623 50 kB/s | 15 kB 00:00 2024-07-16T09:44:37.804 INFO:teuthology.orchestra.run.smithi156.stdout:(2/7): ceph-grafana-dashboards-17.2.7-1104.ge62 76 kB/s | 24 kB 00:00 2024-07-16T09:44:37.809 INFO:teuthology.orchestra.run.smithi176.stdout:Dependencies resolved. 2024-07-16T09:44:37.809 INFO:teuthology.orchestra.run.smithi176.stdout:================================================================================ 2024-07-16T09:44:37.810 INFO:teuthology.orchestra.run.smithi176.stdout: Package Arch Version Repo Size 2024-07-16T09:44:37.810 INFO:teuthology.orchestra.run.smithi176.stdout:================================================================================ 2024-07-16T09:44:37.810 INFO:teuthology.orchestra.run.smithi176.stdout:Installing: 2024-07-16T09:44:37.810 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-immutable-object-cache x86_64 2:17.2.7-1104.ge6231b04.el9 ceph 137 k 2024-07-16T09:44:37.810 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-07-16T09:44:37.810 INFO:teuthology.orchestra.run.smithi176.stdout:Transaction Summary 2024-07-16T09:44:37.810 INFO:teuthology.orchestra.run.smithi176.stdout:================================================================================ 2024-07-16T09:44:37.810 INFO:teuthology.orchestra.run.smithi176.stdout:Install 1 Package 2024-07-16T09:44:37.810 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-07-16T09:44:37.811 INFO:teuthology.orchestra.run.smithi176.stdout:Total download size: 137 k 2024-07-16T09:44:37.811 INFO:teuthology.orchestra.run.smithi176.stdout:Installed size: 379 k 2024-07-16T09:44:37.811 INFO:teuthology.orchestra.run.smithi176.stdout:Downloading Packages: 2024-07-16T09:44:37.879 INFO:teuthology.orchestra.run.smithi156.stdout:(3/7): python3-jwt+crypto-2.4.0-1.el9.noarch.rp 90 kB/s | 9.0 kB 00:00 2024-07-16T09:44:37.904 INFO:teuthology.orchestra.run.smithi156.stdout:(4/7): python3-repoze-lru-0.7-16.el9.noarch.rpm 1.3 MB/s | 31 kB 00:00 2024-07-16T09:44:37.933 INFO:teuthology.orchestra.run.smithi156.stdout:(5/7): python3-jwt-2.4.0-1.el9.noarch.rpm 318 kB/s | 41 kB 00:00 2024-07-16T09:44:37.963 INFO:teuthology.orchestra.run.smithi156.stdout:(6/7): python3-routes-2.5.1-5.el9.noarch.rpm 3.2 MB/s | 188 kB 00:00 2024-07-16T09:44:38.022 INFO:teuthology.orchestra.run.smithi156.stdout:(7/7): ceph-mgr-dashboard-17.2.7-1104.ge6231b04 6.2 MB/s | 3.4 MB 00:00 2024-07-16T09:44:38.025 INFO:teuthology.orchestra.run.smithi156.stdout:-------------------------------------------------------------------------------- 2024-07-16T09:44:38.025 INFO:teuthology.orchestra.run.smithi156.stdout:Total 6.2 MB/s | 3.7 MB 00:00 2024-07-16T09:44:38.026 INFO:teuthology.orchestra.run.smithi156.stdout:Running transaction check 2024-07-16T09:44:38.030 INFO:teuthology.orchestra.run.smithi176.stdout:ceph-immutable-object-cache-17.2.7-1104.ge6231b 626 kB/s | 137 kB 00:00 2024-07-16T09:44:38.031 INFO:teuthology.orchestra.run.smithi176.stdout:-------------------------------------------------------------------------------- 2024-07-16T09:44:38.031 INFO:teuthology.orchestra.run.smithi176.stdout:Total 622 kB/s | 137 kB 00:00 2024-07-16T09:44:38.032 INFO:teuthology.orchestra.run.smithi176.stdout:Running transaction check 2024-07-16T09:44:38.033 INFO:teuthology.orchestra.run.smithi156.stdout:Transaction check succeeded. 2024-07-16T09:44:38.034 INFO:teuthology.orchestra.run.smithi156.stdout:Running transaction test 2024-07-16T09:44:38.039 INFO:teuthology.orchestra.run.smithi176.stdout:Transaction check succeeded. 2024-07-16T09:44:38.039 INFO:teuthology.orchestra.run.smithi176.stdout:Running transaction test 2024-07-16T09:44:38.096 INFO:teuthology.orchestra.run.smithi176.stdout:Transaction test succeeded. 2024-07-16T09:44:38.096 INFO:teuthology.orchestra.run.smithi176.stdout:Running transaction 2024-07-16T09:44:38.096 INFO:teuthology.orchestra.run.smithi156.stdout:Transaction test succeeded. 2024-07-16T09:44:38.096 INFO:teuthology.orchestra.run.smithi156.stdout:Running transaction 2024-07-16T09:44:38.241 INFO:teuthology.orchestra.run.smithi176.stdout: Preparing : 1/1 2024-07-16T09:44:38.331 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : ceph-immutable-object-cache-2:17.2.7-1104.ge6231b04. 1/1 2024-07-16T09:44:38.356 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: ceph-immutable-object-cache-2:17.2.7-1104.ge6231b04. 1/1 2024-07-16T09:44:38.357 INFO:teuthology.orchestra.run.smithi176.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-07-16T09:44:38.357 INFO:teuthology.orchestra.run.smithi176.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-07-16T09:44:38.357 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-07-16T09:44:38.457 INFO:teuthology.orchestra.run.smithi156.stdout: Preparing : 1/1 2024-07-16T09:44:38.515 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-jwt-2.4.0-1.el9.noarch 1/7 2024-07-16T09:44:38.550 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-jwt+crypto-2.4.0-1.el9.noarch 2/7 2024-07-16T09:44:38.595 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/7 2024-07-16T09:44:38.636 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/7 2024-07-16T09:44:38.665 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-prometheus-alerts-2:17.2.7-1104.ge6231b04.el9.n 5/7 2024-07-16T09:44:39.300 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : ceph-immutable-object-cache-2:17.2.7-1104.ge6231b04. 1/1 2024-07-16T09:44:39.300 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-07-16T09:44:39.300 INFO:teuthology.orchestra.run.smithi176.stdout:Installed: 2024-07-16T09:44:39.300 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-immutable-object-cache-2:17.2.7-1104.ge6231b04.el9.x86_64 2024-07-16T09:44:39.300 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-07-16T09:44:39.300 INFO:teuthology.orchestra.run.smithi176.stdout:Complete! 2024-07-16T09:44:39.500 DEBUG:teuthology.orchestra.run.smithi176:> sudo yum -y install ceph-mgr 2024-07-16T09:44:39.663 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-grafana-dashboards-2:17.2.7-1104.ge6231b04.el9. 6/7 2024-07-16T09:44:39.701 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-mgr-dashboard-2:17.2.7-1104.ge6231b04.el9.noarc 7/7 2024-07-16T09:44:40.097 INFO:teuthology.orchestra.run.smithi176.stdout:Last metadata expiration check: 0:00:55 ago on Tue 16 Jul 2024 09:43:45 AM UTC. 2024-07-16T09:44:40.186 INFO:teuthology.orchestra.run.smithi176.stdout:Package ceph-mgr-2:17.2.7-1104.ge6231b04.el9.x86_64 is already installed. 2024-07-16T09:44:40.226 INFO:teuthology.orchestra.run.smithi176.stdout:Dependencies resolved. 2024-07-16T09:44:40.227 INFO:teuthology.orchestra.run.smithi176.stdout:Nothing to do. 2024-07-16T09:44:40.228 INFO:teuthology.orchestra.run.smithi176.stdout:Complete! 2024-07-16T09:44:40.294 DEBUG:teuthology.orchestra.run.smithi176:> sudo yum -y install ceph-mgr-dashboard 2024-07-16T09:44:40.882 INFO:teuthology.orchestra.run.smithi176.stdout:Last metadata expiration check: 0:00:55 ago on Tue 16 Jul 2024 09:43:45 AM UTC. 2024-07-16T09:44:40.894 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: ceph-mgr-dashboard-2:17.2.7-1104.ge6231b04.el9.noarc 7/7 2024-07-16T09:44:40.894 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-grafana-dashboards-2:17.2.7-1104.ge6231b04.el9. 1/7 2024-07-16T09:44:40.894 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-mgr-dashboard-2:17.2.7-1104.ge6231b04.el9.noarc 2/7 2024-07-16T09:44:40.894 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-prometheus-alerts-2:17.2.7-1104.ge6231b04.el9.n 3/7 2024-07-16T09:44:40.894 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-jwt+crypto-2.4.0-1.el9.noarch 4/7 2024-07-16T09:44:40.894 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-jwt-2.4.0-1.el9.noarch 5/7 2024-07-16T09:44:40.894 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 6/7 2024-07-16T09:44:41.014 INFO:teuthology.orchestra.run.smithi176.stdout:Dependencies resolved. 2024-07-16T09:44:41.014 INFO:teuthology.orchestra.run.smithi176.stdout:================================================================================ 2024-07-16T09:44:41.015 INFO:teuthology.orchestra.run.smithi176.stdout: Package Arch Version Repository Size 2024-07-16T09:44:41.015 INFO:teuthology.orchestra.run.smithi176.stdout:================================================================================ 2024-07-16T09:44:41.015 INFO:teuthology.orchestra.run.smithi176.stdout:Installing: 2024-07-16T09:44:41.015 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-mgr-dashboard noarch 2:17.2.7-1104.ge6231b04.el9 ceph-noarch 3.4 M 2024-07-16T09:44:41.015 INFO:teuthology.orchestra.run.smithi176.stdout:Installing dependencies: 2024-07-16T09:44:41.015 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-grafana-dashboards noarch 2:17.2.7-1104.ge6231b04.el9 ceph-noarch 24 k 2024-07-16T09:44:41.015 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-prometheus-alerts noarch 2:17.2.7-1104.ge6231b04.el9 ceph-noarch 15 k 2024-07-16T09:44:41.015 INFO:teuthology.orchestra.run.smithi176.stdout: python3-jwt noarch 2.4.0-1.el9 epel 41 k 2024-07-16T09:44:41.015 INFO:teuthology.orchestra.run.smithi176.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-07-16T09:44:41.015 INFO:teuthology.orchestra.run.smithi176.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-07-16T09:44:41.015 INFO:teuthology.orchestra.run.smithi176.stdout:Installing weak dependencies: 2024-07-16T09:44:41.016 INFO:teuthology.orchestra.run.smithi176.stdout: python3-jwt+crypto noarch 2.4.0-1.el9 epel 9.0 k 2024-07-16T09:44:41.016 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-07-16T09:44:41.016 INFO:teuthology.orchestra.run.smithi176.stdout:Transaction Summary 2024-07-16T09:44:41.016 INFO:teuthology.orchestra.run.smithi176.stdout:================================================================================ 2024-07-16T09:44:41.016 INFO:teuthology.orchestra.run.smithi176.stdout:Install 7 Packages 2024-07-16T09:44:41.016 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-07-16T09:44:41.016 INFO:teuthology.orchestra.run.smithi176.stdout:Total download size: 3.7 M 2024-07-16T09:44:41.017 INFO:teuthology.orchestra.run.smithi176.stdout:Installed size: 80 M 2024-07-16T09:44:41.017 INFO:teuthology.orchestra.run.smithi176.stdout:Downloading Packages: 2024-07-16T09:44:41.180 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 7/7 2024-07-16T09:44:41.181 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-07-16T09:44:41.181 INFO:teuthology.orchestra.run.smithi156.stdout:Installed: 2024-07-16T09:44:41.181 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-grafana-dashboards-2:17.2.7-1104.ge6231b04.el9.noarch 2024-07-16T09:44:41.181 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mgr-dashboard-2:17.2.7-1104.ge6231b04.el9.noarch 2024-07-16T09:44:41.181 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-prometheus-alerts-2:17.2.7-1104.ge6231b04.el9.noarch 2024-07-16T09:44:41.181 INFO:teuthology.orchestra.run.smithi156.stdout: python3-jwt-2.4.0-1.el9.noarch 2024-07-16T09:44:41.181 INFO:teuthology.orchestra.run.smithi156.stdout: python3-jwt+crypto-2.4.0-1.el9.noarch 2024-07-16T09:44:41.181 INFO:teuthology.orchestra.run.smithi156.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-07-16T09:44:41.181 INFO:teuthology.orchestra.run.smithi156.stdout: python3-routes-2.5.1-5.el9.noarch 2024-07-16T09:44:41.181 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-07-16T09:44:41.181 INFO:teuthology.orchestra.run.smithi156.stdout:Complete! 2024-07-16T09:44:41.392 DEBUG:teuthology.orchestra.run.smithi156:> sudo yum -y install ceph-mgr-diskprediction-local 2024-07-16T09:44:41.492 INFO:teuthology.orchestra.run.smithi176.stdout:(1/7): ceph-grafana-dashboards-17.2.7-1104.ge62 82 kB/s | 24 kB 00:00 2024-07-16T09:44:41.517 INFO:teuthology.orchestra.run.smithi176.stdout:(2/7): ceph-prometheus-alerts-17.2.7-1104.ge623 45 kB/s | 15 kB 00:00 2024-07-16T09:44:41.593 INFO:teuthology.orchestra.run.smithi176.stdout:(3/7): python3-jwt+crypto-2.4.0-1.el9.noarch.rp 90 kB/s | 9.0 kB 00:00 2024-07-16T09:44:41.618 INFO:teuthology.orchestra.run.smithi176.stdout:(4/7): python3-repoze-lru-0.7-16.el9.noarch.rpm 1.3 MB/s | 31 kB 00:00 2024-07-16T09:44:41.643 INFO:teuthology.orchestra.run.smithi176.stdout:(5/7): python3-jwt-2.4.0-1.el9.noarch.rpm 326 kB/s | 41 kB 00:00 2024-07-16T09:44:41.660 INFO:teuthology.orchestra.run.smithi176.stdout:(6/7): python3-routes-2.5.1-5.el9.noarch.rpm 4.4 MB/s | 188 kB 00:00 2024-07-16T09:44:41.785 INFO:teuthology.orchestra.run.smithi176.stdout:(7/7): ceph-mgr-dashboard-17.2.7-1104.ge6231b04 5.7 MB/s | 3.4 MB 00:00 2024-07-16T09:44:41.789 INFO:teuthology.orchestra.run.smithi176.stdout:-------------------------------------------------------------------------------- 2024-07-16T09:44:41.789 INFO:teuthology.orchestra.run.smithi176.stdout:Total 4.7 MB/s | 3.7 MB 00:00 2024-07-16T09:44:41.789 INFO:teuthology.orchestra.run.smithi176.stdout:Running transaction check 2024-07-16T09:44:41.797 INFO:teuthology.orchestra.run.smithi176.stdout:Transaction check succeeded. 2024-07-16T09:44:41.797 INFO:teuthology.orchestra.run.smithi176.stdout:Running transaction test 2024-07-16T09:44:41.859 INFO:teuthology.orchestra.run.smithi176.stdout:Transaction test succeeded. 2024-07-16T09:44:41.860 INFO:teuthology.orchestra.run.smithi176.stdout:Running transaction 2024-07-16T09:44:41.978 INFO:teuthology.orchestra.run.smithi156.stdout:Last metadata expiration check: 0:01:01 ago on Tue 16 Jul 2024 09:43:40 AM UTC. 2024-07-16T09:44:42.106 INFO:teuthology.orchestra.run.smithi156.stdout:Dependencies resolved. 2024-07-16T09:44:42.107 INFO:teuthology.orchestra.run.smithi156.stdout:======================================================================================= 2024-07-16T09:44:42.107 INFO:teuthology.orchestra.run.smithi156.stdout: Package Arch Version Repository Size 2024-07-16T09:44:42.107 INFO:teuthology.orchestra.run.smithi156.stdout:======================================================================================= 2024-07-16T09:44:42.107 INFO:teuthology.orchestra.run.smithi156.stdout:Installing: 2024-07-16T09:44:42.107 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mgr-diskprediction-local noarch 2:17.2.7-1104.ge6231b04.el9 ceph-noarch 7.4 M 2024-07-16T09:44:42.107 INFO:teuthology.orchestra.run.smithi156.stdout:Installing dependencies: 2024-07-16T09:44:42.107 INFO:teuthology.orchestra.run.smithi156.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-07-16T09:44:42.107 INFO:teuthology.orchestra.run.smithi156.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-07-16T09:44:42.108 INFO:teuthology.orchestra.run.smithi156.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-07-16T09:44:42.108 INFO:teuthology.orchestra.run.smithi156.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-07-16T09:44:42.108 INFO:teuthology.orchestra.run.smithi156.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-07-16T09:44:42.108 INFO:teuthology.orchestra.run.smithi156.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-07-16T09:44:42.108 INFO:teuthology.orchestra.run.smithi156.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-07-16T09:44:42.108 INFO:teuthology.orchestra.run.smithi156.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-07-16T09:44:42.108 INFO:teuthology.orchestra.run.smithi156.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-07-16T09:44:42.108 INFO:teuthology.orchestra.run.smithi156.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-07-16T09:44:42.108 INFO:teuthology.orchestra.run.smithi156.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-07-16T09:44:42.108 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-07-16T09:44:42.108 INFO:teuthology.orchestra.run.smithi156.stdout:Transaction Summary 2024-07-16T09:44:42.108 INFO:teuthology.orchestra.run.smithi156.stdout:======================================================================================= 2024-07-16T09:44:42.108 INFO:teuthology.orchestra.run.smithi156.stdout:Install 12 Packages 2024-07-16T09:44:42.109 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-07-16T09:44:42.109 INFO:teuthology.orchestra.run.smithi156.stdout:Total download size: 38 M 2024-07-16T09:44:42.109 INFO:teuthology.orchestra.run.smithi156.stdout:Installed size: 209 M 2024-07-16T09:44:42.109 INFO:teuthology.orchestra.run.smithi156.stdout:Downloading Packages: 2024-07-16T09:44:42.223 INFO:teuthology.orchestra.run.smithi176.stdout: Preparing : 1/1 2024-07-16T09:44:42.295 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-jwt-2.4.0-1.el9.noarch 1/7 2024-07-16T09:44:42.338 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-jwt+crypto-2.4.0-1.el9.noarch 2/7 2024-07-16T09:44:42.393 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/7 2024-07-16T09:44:42.424 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/7 2024-07-16T09:44:42.453 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : ceph-prometheus-alerts-2:17.2.7-1104.ge6231b04.el9.n 5/7 2024-07-16T09:44:43.255 INFO:teuthology.orchestra.run.smithi156.stdout:(1/12): ceph-mgr-diskprediction-local-17.2.7-11 12 MB/s | 7.4 MB 00:00 2024-07-16T09:44:43.363 INFO:teuthology.orchestra.run.smithi156.stdout:(2/12): flexiblas-3.0.4-8.el9.x86_64.rpm 303 kB/s | 33 kB 00:00 2024-07-16T09:44:43.449 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : ceph-grafana-dashboards-2:17.2.7-1104.ge6231b04.el9. 6/7 2024-07-16T09:44:43.458 INFO:teuthology.orchestra.run.smithi156.stdout:(3/12): libquadmath-11.4.1-3.el9.x86_64.rpm 243 kB/s | 195 kB 00:00 2024-07-16T09:44:43.482 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : ceph-mgr-dashboard-2:17.2.7-1104.ge6231b04.el9.noarc 7/7 2024-07-16T09:44:43.572 INFO:teuthology.orchestra.run.smithi156.stdout:(4/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 156 kB/s | 18 kB 00:00 2024-07-16T09:44:43.656 INFO:teuthology.orchestra.run.smithi156.stdout:(5/12): openblas-0.3.21-2.el9.x86_64.rpm 422 kB/s | 35 kB 00:00 2024-07-16T09:44:43.689 INFO:teuthology.orchestra.run.smithi156.stdout:(6/12): libgfortran-11.4.1-3.el9.x86_64.rpm 772 kB/s | 803 kB 00:01 2024-07-16T09:44:43.889 INFO:teuthology.orchestra.run.smithi156.stdout:(7/12): python3-devel-3.9.18-3.el9.x86_64.rpm 1.2 MB/s | 244 kB 00:00 2024-07-16T09:44:44.481 INFO:teuthology.orchestra.run.smithi156.stdout:(8/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 2.7 MB/s | 3.0 MB 00:01 2024-07-16T09:44:44.623 INFO:teuthology.orchestra.run.smithi156.stdout:(9/12): python3-numpy-f2py-1.20.1-5.el9.x86_64. 1.7 MB/s | 250 kB 00:00 2024-07-16T09:44:44.663 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: ceph-mgr-dashboard-2:17.2.7-1104.ge6231b04.el9.noarc 7/7 2024-07-16T09:44:44.663 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : ceph-grafana-dashboards-2:17.2.7-1104.ge6231b04.el9. 1/7 2024-07-16T09:44:44.663 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : ceph-mgr-dashboard-2:17.2.7-1104.ge6231b04.el9.noarc 2/7 2024-07-16T09:44:44.664 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : ceph-prometheus-alerts-2:17.2.7-1104.ge6231b04.el9.n 3/7 2024-07-16T09:44:44.664 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-jwt+crypto-2.4.0-1.el9.noarch 4/7 2024-07-16T09:44:44.664 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-jwt-2.4.0-1.el9.noarch 5/7 2024-07-16T09:44:44.664 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 6/7 2024-07-16T09:44:44.950 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 7/7 2024-07-16T09:44:44.950 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-07-16T09:44:44.950 INFO:teuthology.orchestra.run.smithi176.stdout:Installed: 2024-07-16T09:44:44.950 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-grafana-dashboards-2:17.2.7-1104.ge6231b04.el9.noarch 2024-07-16T09:44:44.950 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-mgr-dashboard-2:17.2.7-1104.ge6231b04.el9.noarch 2024-07-16T09:44:44.950 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-prometheus-alerts-2:17.2.7-1104.ge6231b04.el9.noarch 2024-07-16T09:44:44.950 INFO:teuthology.orchestra.run.smithi176.stdout: python3-jwt-2.4.0-1.el9.noarch 2024-07-16T09:44:44.950 INFO:teuthology.orchestra.run.smithi176.stdout: python3-jwt+crypto-2.4.0-1.el9.noarch 2024-07-16T09:44:44.950 INFO:teuthology.orchestra.run.smithi176.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-07-16T09:44:44.950 INFO:teuthology.orchestra.run.smithi176.stdout: python3-routes-2.5.1-5.el9.noarch 2024-07-16T09:44:44.951 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-07-16T09:44:44.951 INFO:teuthology.orchestra.run.smithi176.stdout:Complete! 2024-07-16T09:44:45.161 DEBUG:teuthology.orchestra.run.smithi176:> sudo yum -y install ceph-mgr-diskprediction-local 2024-07-16T09:44:45.532 INFO:teuthology.orchestra.run.smithi156.stdout:(10/12): openblas-openmp-0.3.21-2.el9.x86_64.rp 2.5 MB/s | 4.7 MB 00:01 2024-07-16T09:44:45.738 INFO:teuthology.orchestra.run.smithi176.stdout:Last metadata expiration check: 0:01:00 ago on Tue 16 Jul 2024 09:43:45 AM UTC. 2024-07-16T09:44:45.866 INFO:teuthology.orchestra.run.smithi176.stdout:Dependencies resolved. 2024-07-16T09:44:45.867 INFO:teuthology.orchestra.run.smithi176.stdout:======================================================================================= 2024-07-16T09:44:45.868 INFO:teuthology.orchestra.run.smithi176.stdout: Package Arch Version Repository Size 2024-07-16T09:44:45.868 INFO:teuthology.orchestra.run.smithi176.stdout:======================================================================================= 2024-07-16T09:44:45.868 INFO:teuthology.orchestra.run.smithi176.stdout:Installing: 2024-07-16T09:44:45.868 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-mgr-diskprediction-local noarch 2:17.2.7-1104.ge6231b04.el9 ceph-noarch 7.4 M 2024-07-16T09:44:45.868 INFO:teuthology.orchestra.run.smithi176.stdout:Installing dependencies: 2024-07-16T09:44:45.868 INFO:teuthology.orchestra.run.smithi176.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-07-16T09:44:45.868 INFO:teuthology.orchestra.run.smithi176.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-07-16T09:44:45.868 INFO:teuthology.orchestra.run.smithi176.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-07-16T09:44:45.868 INFO:teuthology.orchestra.run.smithi176.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-07-16T09:44:45.868 INFO:teuthology.orchestra.run.smithi176.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-07-16T09:44:45.868 INFO:teuthology.orchestra.run.smithi176.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-07-16T09:44:45.869 INFO:teuthology.orchestra.run.smithi176.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-07-16T09:44:45.869 INFO:teuthology.orchestra.run.smithi176.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-07-16T09:44:45.869 INFO:teuthology.orchestra.run.smithi176.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-07-16T09:44:45.869 INFO:teuthology.orchestra.run.smithi176.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-07-16T09:44:45.869 INFO:teuthology.orchestra.run.smithi176.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-07-16T09:44:45.869 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-07-16T09:44:45.869 INFO:teuthology.orchestra.run.smithi176.stdout:Transaction Summary 2024-07-16T09:44:45.869 INFO:teuthology.orchestra.run.smithi176.stdout:======================================================================================= 2024-07-16T09:44:45.869 INFO:teuthology.orchestra.run.smithi176.stdout:Install 12 Packages 2024-07-16T09:44:45.869 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-07-16T09:44:45.870 INFO:teuthology.orchestra.run.smithi176.stdout:Total download size: 38 M 2024-07-16T09:44:45.870 INFO:teuthology.orchestra.run.smithi176.stdout:Installed size: 209 M 2024-07-16T09:44:45.870 INFO:teuthology.orchestra.run.smithi176.stdout:Downloading Packages: 2024-07-16T09:44:46.007 INFO:teuthology.orchestra.run.smithi156.stdout:(11/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 2.4 MB/s | 5.1 MB 00:02 2024-07-16T09:44:46.352 INFO:teuthology.orchestra.run.smithi176.stdout:(1/12): libquadmath-11.4.1-3.el9.x86_64.rpm 1.3 MB/s | 195 kB 00:00 2024-07-16T09:44:46.377 INFO:teuthology.orchestra.run.smithi176.stdout:(2/12): libgfortran-11.4.1-3.el9.x86_64.rpm 4.6 MB/s | 803 kB 00:00 2024-07-16T09:44:46.469 INFO:teuthology.orchestra.run.smithi176.stdout:(3/12): flexiblas-3.0.4-8.el9.x86_64.rpm 280 kB/s | 33 kB 00:00 2024-07-16T09:44:46.527 INFO:teuthology.orchestra.run.smithi176.stdout:(4/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 312 kB/s | 18 kB 00:00 2024-07-16T09:44:46.578 INFO:teuthology.orchestra.run.smithi176.stdout:(5/12): openblas-0.3.21-2.el9.x86_64.rpm 702 kB/s | 35 kB 00:00 2024-07-16T09:44:46.870 INFO:teuthology.orchestra.run.smithi176.stdout:(6/12): ceph-mgr-diskprediction-local-17.2.7-11 11 MB/s | 7.4 MB 00:00 2024-07-16T09:44:47.104 INFO:teuthology.orchestra.run.smithi176.stdout:(7/12): python3-devel-3.9.18-3.el9.x86_64.rpm 1.0 MB/s | 244 kB 00:00 2024-07-16T09:44:47.387 INFO:teuthology.orchestra.run.smithi176.stdout:(8/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 3.0 MB/s | 3.0 MB 00:01 2024-07-16T09:44:47.529 INFO:teuthology.orchestra.run.smithi176.stdout:(9/12): python3-numpy-f2py-1.20.1-5.el9.x86_64. 1.7 MB/s | 250 kB 00:00 2024-07-16T09:44:48.513 INFO:teuthology.orchestra.run.smithi176.stdout:(10/12): openblas-openmp-0.3.21-2.el9.x86_64.rp 2.4 MB/s | 4.7 MB 00:01 2024-07-16T09:44:49.287 INFO:teuthology.orchestra.run.smithi176.stdout:(11/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 2.3 MB/s | 5.1 MB 00:02 2024-07-16T09:44:51.304 INFO:teuthology.orchestra.run.smithi176.stdout:(12/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 4.3 MB/s | 16 MB 00:03 2024-07-16T09:44:51.305 INFO:teuthology.orchestra.run.smithi176.stdout:-------------------------------------------------------------------------------- 2024-07-16T09:44:51.305 INFO:teuthology.orchestra.run.smithi176.stdout:Total 7.0 MB/s | 38 MB 00:05 2024-07-16T09:44:51.662 INFO:teuthology.orchestra.run.smithi176.stdout:Running transaction check 2024-07-16T09:44:51.688 INFO:teuthology.orchestra.run.smithi176.stdout:Transaction check succeeded. 2024-07-16T09:44:51.688 INFO:teuthology.orchestra.run.smithi176.stdout:Running transaction test 2024-07-16T09:44:52.171 INFO:teuthology.orchestra.run.smithi176.stdout:Transaction test succeeded. 2024-07-16T09:44:52.172 INFO:teuthology.orchestra.run.smithi176.stdout:Running transaction 2024-07-16T09:44:52.198 INFO:teuthology.orchestra.run.smithi156.stdout:(12/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 2.1 MB/s | 16 MB 00:07 2024-07-16T09:44:52.199 INFO:teuthology.orchestra.run.smithi156.stdout:-------------------------------------------------------------------------------- 2024-07-16T09:44:52.199 INFO:teuthology.orchestra.run.smithi156.stdout:Total 3.8 MB/s | 38 MB 00:10 2024-07-16T09:44:52.556 INFO:teuthology.orchestra.run.smithi156.stdout:Running transaction check 2024-07-16T09:44:52.584 INFO:teuthology.orchestra.run.smithi156.stdout:Transaction check succeeded. 2024-07-16T09:44:52.584 INFO:teuthology.orchestra.run.smithi156.stdout:Running transaction test 2024-07-16T09:44:52.926 INFO:teuthology.orchestra.run.smithi176.stdout: Preparing : 1/1 2024-07-16T09:44:52.996 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-07-16T09:44:53.041 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 2/12 2024-07-16T09:44:53.094 INFO:teuthology.orchestra.run.smithi156.stdout:Transaction test succeeded. 2024-07-16T09:44:53.094 INFO:teuthology.orchestra.run.smithi156.stdout:Running transaction 2024-07-16T09:44:53.135 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 3/12 2024-07-16T09:44:53.178 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 4/12 2024-07-16T09:44:53.356 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : openblas-0.3.21-2.el9.x86_64 5/12 2024-07-16T09:44:53.394 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-07-16T09:44:53.473 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-07-16T09:44:53.855 INFO:teuthology.orchestra.run.smithi156.stdout: Preparing : 1/1 2024-07-16T09:44:53.933 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-07-16T09:44:53.980 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 2/12 2024-07-16T09:44:54.088 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 3/12 2024-07-16T09:44:54.128 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-07-16T09:44:54.130 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 4/12 2024-07-16T09:44:54.308 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : openblas-0.3.21-2.el9.x86_64 5/12 2024-07-16T09:44:54.338 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-07-16T09:44:54.425 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-07-16T09:44:54.448 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-07-16T09:44:55.086 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-07-16T09:44:55.209 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-07-16T09:44:55.695 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-07-16T09:44:56.467 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-07-16T09:44:56.715 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-07-16T09:44:56.740 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : ceph-mgr-diskprediction-local-2:17.2.7-1104.ge6231 12/12 2024-07-16T09:44:57.498 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-07-16T09:44:57.531 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-mgr-diskprediction-local-2:17.2.7-1104.ge6231 12/12 2024-07-16T09:44:59.172 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:17.2.7-1104.ge6231 12/12 2024-07-16T09:44:59.172 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : ceph-mgr-diskprediction-local-2:17.2.7-1104.ge6231 1/12 2024-07-16T09:44:59.172 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 2/12 2024-07-16T09:44:59.172 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 3/12 2024-07-16T09:44:59.172 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-07-16T09:44:59.172 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-07-16T09:44:59.173 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-07-16T09:44:59.173 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 7/12 2024-07-16T09:44:59.173 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 8/12 2024-07-16T09:44:59.173 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 9/12 2024-07-16T09:44:59.173 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-07-16T09:44:59.173 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-07-16T09:44:59.558 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-07-16T09:44:59.558 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-07-16T09:44:59.558 INFO:teuthology.orchestra.run.smithi176.stdout:Installed: 2024-07-16T09:44:59.558 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-mgr-diskprediction-local-2:17.2.7-1104.ge6231b04.el9.noarch 2024-07-16T09:44:59.558 INFO:teuthology.orchestra.run.smithi176.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-07-16T09:44:59.558 INFO:teuthology.orchestra.run.smithi176.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-07-16T09:44:59.558 INFO:teuthology.orchestra.run.smithi176.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-07-16T09:44:59.558 INFO:teuthology.orchestra.run.smithi176.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-07-16T09:44:59.558 INFO:teuthology.orchestra.run.smithi176.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-07-16T09:44:59.559 INFO:teuthology.orchestra.run.smithi176.stdout: openblas-0.3.21-2.el9.x86_64 2024-07-16T09:44:59.559 INFO:teuthology.orchestra.run.smithi176.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-07-16T09:44:59.559 INFO:teuthology.orchestra.run.smithi176.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-07-16T09:44:59.559 INFO:teuthology.orchestra.run.smithi176.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-07-16T09:44:59.559 INFO:teuthology.orchestra.run.smithi176.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-07-16T09:44:59.559 INFO:teuthology.orchestra.run.smithi176.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-07-16T09:44:59.559 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-07-16T09:44:59.559 INFO:teuthology.orchestra.run.smithi176.stdout:Complete! 2024-07-16T09:44:59.859 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:17.2.7-1104.ge6231 12/12 2024-07-16T09:44:59.859 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-mgr-diskprediction-local-2:17.2.7-1104.ge6231 1/12 2024-07-16T09:44:59.859 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 2/12 2024-07-16T09:44:59.859 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 3/12 2024-07-16T09:44:59.859 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-07-16T09:44:59.859 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-07-16T09:44:59.860 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-07-16T09:44:59.860 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 7/12 2024-07-16T09:44:59.860 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 8/12 2024-07-16T09:44:59.860 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 9/12 2024-07-16T09:44:59.860 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-07-16T09:44:59.860 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-07-16T09:44:59.919 DEBUG:teuthology.orchestra.run.smithi176:> sudo yum -y install ceph-mgr-rook 2024-07-16T09:45:00.281 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-07-16T09:45:00.281 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-07-16T09:45:00.281 INFO:teuthology.orchestra.run.smithi156.stdout:Installed: 2024-07-16T09:45:00.281 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mgr-diskprediction-local-2:17.2.7-1104.ge6231b04.el9.noarch 2024-07-16T09:45:00.282 INFO:teuthology.orchestra.run.smithi156.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-07-16T09:45:00.282 INFO:teuthology.orchestra.run.smithi156.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-07-16T09:45:00.282 INFO:teuthology.orchestra.run.smithi156.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-07-16T09:45:00.282 INFO:teuthology.orchestra.run.smithi156.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-07-16T09:45:00.282 INFO:teuthology.orchestra.run.smithi156.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-07-16T09:45:00.282 INFO:teuthology.orchestra.run.smithi156.stdout: openblas-0.3.21-2.el9.x86_64 2024-07-16T09:45:00.282 INFO:teuthology.orchestra.run.smithi156.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-07-16T09:45:00.283 INFO:teuthology.orchestra.run.smithi156.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-07-16T09:45:00.283 INFO:teuthology.orchestra.run.smithi156.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-07-16T09:45:00.283 INFO:teuthology.orchestra.run.smithi156.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-07-16T09:45:00.283 INFO:teuthology.orchestra.run.smithi156.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-07-16T09:45:00.283 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-07-16T09:45:00.283 INFO:teuthology.orchestra.run.smithi156.stdout:Complete! 2024-07-16T09:45:00.547 INFO:teuthology.orchestra.run.smithi176.stdout:Last metadata expiration check: 0:01:15 ago on Tue 16 Jul 2024 09:43:45 AM UTC. 2024-07-16T09:45:00.682 INFO:teuthology.orchestra.run.smithi176.stdout:Dependencies resolved. 2024-07-16T09:45:00.683 INFO:teuthology.orchestra.run.smithi176.stdout:================================================================================ 2024-07-16T09:45:00.683 INFO:teuthology.orchestra.run.smithi176.stdout: Package Arch Version Repository Size 2024-07-16T09:45:00.684 INFO:teuthology.orchestra.run.smithi176.stdout:================================================================================ 2024-07-16T09:45:00.684 INFO:teuthology.orchestra.run.smithi176.stdout:Installing: 2024-07-16T09:45:00.684 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-mgr-rook noarch 2:17.2.7-1104.ge6231b04.el9 ceph-noarch 47 k 2024-07-16T09:45:00.684 INFO:teuthology.orchestra.run.smithi176.stdout:Installing dependencies: 2024-07-16T09:45:00.684 INFO:teuthology.orchestra.run.smithi176.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-07-16T09:45:00.684 INFO:teuthology.orchestra.run.smithi176.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-07-16T09:45:00.684 INFO:teuthology.orchestra.run.smithi176.stdout: python3-google-auth noarch 1:2.30.0-1.el9 epel 213 k 2024-07-16T09:45:00.684 INFO:teuthology.orchestra.run.smithi176.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-07-16T09:45:00.684 INFO:teuthology.orchestra.run.smithi176.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-07-16T09:45:00.685 INFO:teuthology.orchestra.run.smithi176.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-07-16T09:45:00.685 INFO:teuthology.orchestra.run.smithi176.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-07-16T09:45:00.685 INFO:teuthology.orchestra.run.smithi176.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-07-16T09:45:00.685 INFO:teuthology.orchestra.run.smithi176.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-07-16T09:45:00.685 INFO:teuthology.orchestra.run.smithi176.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-07-16T09:45:00.685 INFO:teuthology.orchestra.run.smithi176.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-07-16T09:45:00.685 INFO:teuthology.orchestra.run.smithi176.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-07-16T09:45:00.685 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-07-16T09:45:00.685 INFO:teuthology.orchestra.run.smithi176.stdout:Transaction Summary 2024-07-16T09:45:00.685 INFO:teuthology.orchestra.run.smithi176.stdout:================================================================================ 2024-07-16T09:45:00.685 INFO:teuthology.orchestra.run.smithi176.stdout:Install 13 Packages 2024-07-16T09:45:00.685 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-07-16T09:45:00.686 INFO:teuthology.orchestra.run.smithi176.stdout:Total download size: 2.2 M 2024-07-16T09:45:00.686 INFO:teuthology.orchestra.run.smithi176.stdout:Installed size: 26 M 2024-07-16T09:45:00.686 INFO:teuthology.orchestra.run.smithi176.stdout:Downloading Packages: 2024-07-16T09:45:00.728 DEBUG:teuthology.orchestra.run.smithi156:> sudo yum -y install ceph-mgr-rook 2024-07-16T09:45:01.290 INFO:teuthology.orchestra.run.smithi176.stdout:(1/13): ceph-mgr-rook-17.2.7-1104.ge6231b04.el9 130 kB/s | 47 kB 00:00 2024-07-16T09:45:01.329 INFO:teuthology.orchestra.run.smithi156.stdout:Last metadata expiration check: 0:01:21 ago on Tue 16 Jul 2024 09:43:40 AM UTC. 2024-07-16T09:45:01.458 INFO:teuthology.orchestra.run.smithi156.stdout:Dependencies resolved. 2024-07-16T09:45:01.459 INFO:teuthology.orchestra.run.smithi156.stdout:================================================================================ 2024-07-16T09:45:01.460 INFO:teuthology.orchestra.run.smithi156.stdout: Package Arch Version Repository Size 2024-07-16T09:45:01.460 INFO:teuthology.orchestra.run.smithi156.stdout:================================================================================ 2024-07-16T09:45:01.460 INFO:teuthology.orchestra.run.smithi156.stdout:Installing: 2024-07-16T09:45:01.460 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mgr-rook noarch 2:17.2.7-1104.ge6231b04.el9 ceph-noarch 47 k 2024-07-16T09:45:01.460 INFO:teuthology.orchestra.run.smithi156.stdout:Installing dependencies: 2024-07-16T09:45:01.460 INFO:teuthology.orchestra.run.smithi156.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-07-16T09:45:01.460 INFO:teuthology.orchestra.run.smithi156.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-07-16T09:45:01.460 INFO:teuthology.orchestra.run.smithi156.stdout: python3-google-auth noarch 1:2.30.0-1.el9 epel 213 k 2024-07-16T09:45:01.460 INFO:teuthology.orchestra.run.smithi156.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-07-16T09:45:01.460 INFO:teuthology.orchestra.run.smithi156.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-07-16T09:45:01.460 INFO:teuthology.orchestra.run.smithi156.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-07-16T09:45:01.460 INFO:teuthology.orchestra.run.smithi156.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-07-16T09:45:01.461 INFO:teuthology.orchestra.run.smithi156.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-07-16T09:45:01.461 INFO:teuthology.orchestra.run.smithi156.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-07-16T09:45:01.461 INFO:teuthology.orchestra.run.smithi156.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-07-16T09:45:01.461 INFO:teuthology.orchestra.run.smithi156.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-07-16T09:45:01.461 INFO:teuthology.orchestra.run.smithi156.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-07-16T09:45:01.461 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-07-16T09:45:01.461 INFO:teuthology.orchestra.run.smithi156.stdout:Transaction Summary 2024-07-16T09:45:01.461 INFO:teuthology.orchestra.run.smithi156.stdout:================================================================================ 2024-07-16T09:45:01.461 INFO:teuthology.orchestra.run.smithi156.stdout:Install 13 Packages 2024-07-16T09:45:01.461 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-07-16T09:45:01.462 INFO:teuthology.orchestra.run.smithi156.stdout:Total download size: 2.2 M 2024-07-16T09:45:01.462 INFO:teuthology.orchestra.run.smithi156.stdout:Installed size: 26 M 2024-07-16T09:45:01.462 INFO:teuthology.orchestra.run.smithi156.stdout:Downloading Packages: 2024-07-16T09:45:01.481 INFO:teuthology.orchestra.run.smithi176.stdout:(2/13): python3-jsonpatch-1.21-16.el9.noarch.rp 46 kB/s | 26 kB 00:00 2024-07-16T09:45:01.498 INFO:teuthology.orchestra.run.smithi176.stdout:(3/13): python3-jsonpointer-2.0-4.el9.noarch.rp 33 kB/s | 19 kB 00:00 2024-07-16T09:45:02.057 INFO:teuthology.orchestra.run.smithi176.stdout:(4/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 290 kB/s | 222 kB 00:00 2024-07-16T09:45:02.116 INFO:teuthology.orchestra.run.smithi156.stdout:(1/13): ceph-mgr-rook-17.2.7-1104.ge6231b04.el9 227 kB/s | 47 kB 00:00 2024-07-16T09:45:02.207 INFO:teuthology.orchestra.run.smithi176.stdout:(5/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 219 kB/s | 159 kB 00:00 2024-07-16T09:45:02.257 INFO:teuthology.orchestra.run.smithi176.stdout:(6/13): python3-cachetools-4.2.4-1.el9.noarch.r 649 kB/s | 32 kB 00:00 2024-07-16T09:45:02.283 INFO:teuthology.orchestra.run.smithi156.stdout:(2/13): python3-jsonpatch-1.21-16.el9.noarch.rp 68 kB/s | 26 kB 00:00 2024-07-16T09:45:02.283 INFO:teuthology.orchestra.run.smithi176.stdout:(7/13): python3-pyasn1-modules-0.4.8-6.el9.noar 356 kB/s | 279 kB 00:00 2024-07-16T09:45:02.308 INFO:teuthology.orchestra.run.smithi156.stdout:(3/13): python3-jsonpointer-2.0-4.el9.noarch.rp 48 kB/s | 19 kB 00:00 2024-07-16T09:45:02.308 INFO:teuthology.orchestra.run.smithi176.stdout:(8/13): python3-certifi-2023.05.07-4.el9.noarch 281 kB/s | 14 kB 00:00 2024-07-16T09:45:02.392 INFO:teuthology.orchestra.run.smithi176.stdout:(9/13): python3-google-auth-2.30.0-1.el9.noarch 3.5 MB/s | 213 kB 00:00 2024-07-16T09:45:02.417 INFO:teuthology.orchestra.run.smithi176.stdout:(10/13): python3-rsa-4.9-2.el9.noarch.rpm 2.6 MB/s | 59 kB 00:00 2024-07-16T09:45:02.442 INFO:teuthology.orchestra.run.smithi176.stdout:(11/13): python3-websocket-client-1.2.3-2.el9.n 3.6 MB/s | 90 kB 00:00 2024-07-16T09:45:02.484 INFO:teuthology.orchestra.run.smithi176.stdout:(12/13): python3-kubernetes-26.1.0-2.el9.noarch 6.7 MB/s | 1.0 MB 00:00 2024-07-16T09:45:02.516 INFO:teuthology.orchestra.run.smithi156.stdout:(4/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 558 kB/s | 222 kB 00:00 2024-07-16T09:45:02.567 INFO:teuthology.orchestra.run.smithi156.stdout:(5/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 560 kB/s | 159 kB 00:00 2024-07-16T09:45:02.634 INFO:teuthology.orchestra.run.smithi156.stdout:(6/13): python3-cachetools-4.2.4-1.el9.noarch.r 485 kB/s | 32 kB 00:00 2024-07-16T09:45:02.659 INFO:teuthology.orchestra.run.smithi156.stdout:(7/13): python3-pyasn1-modules-0.4.8-6.el9.noar 794 kB/s | 279 kB 00:00 2024-07-16T09:45:02.684 INFO:teuthology.orchestra.run.smithi156.stdout:(8/13): python3-requests-oauthlib-1.3.0-12.el9. 321 kB/s | 54 kB 00:00 2024-07-16T09:45:02.709 INFO:teuthology.orchestra.run.smithi156.stdout:(9/13): python3-certifi-2023.05.07-4.el9.noarch 188 kB/s | 14 kB 00:00 2024-07-16T09:45:02.726 INFO:teuthology.orchestra.run.smithi176.stdout:(13/13): python3-requests-oauthlib-1.3.0-12.el9 80 kB/s | 54 kB 00:00 2024-07-16T09:45:02.729 INFO:teuthology.orchestra.run.smithi176.stdout:-------------------------------------------------------------------------------- 2024-07-16T09:45:02.729 INFO:teuthology.orchestra.run.smithi176.stdout:Total 1.1 MB/s | 2.2 MB 00:02 2024-07-16T09:45:02.785 INFO:teuthology.orchestra.run.smithi156.stdout:(10/13): python3-rsa-4.9-2.el9.noarch.rpm 2.2 MB/s | 59 kB 00:00 2024-07-16T09:45:02.791 INFO:teuthology.orchestra.run.smithi176.stdout:Running transaction check 2024-07-16T09:45:02.806 INFO:teuthology.orchestra.run.smithi176.stdout:Transaction check succeeded. 2024-07-16T09:45:02.806 INFO:teuthology.orchestra.run.smithi176.stdout:Running transaction test 2024-07-16T09:45:02.810 INFO:teuthology.orchestra.run.smithi156.stdout:(11/13): python3-websocket-client-1.2.3-2.el9.n 3.9 MB/s | 90 kB 00:00 2024-07-16T09:45:02.835 INFO:teuthology.orchestra.run.smithi156.stdout:(12/13): python3-google-auth-2.30.0-1.el9.noarc 1.4 MB/s | 213 kB 00:00 2024-07-16T09:45:02.877 INFO:teuthology.orchestra.run.smithi156.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 5.3 MB/s | 1.0 MB 00:00 2024-07-16T09:45:02.881 INFO:teuthology.orchestra.run.smithi156.stdout:-------------------------------------------------------------------------------- 2024-07-16T09:45:02.881 INFO:teuthology.orchestra.run.smithi156.stdout:Total 1.6 MB/s | 2.2 MB 00:01 2024-07-16T09:45:02.891 INFO:teuthology.orchestra.run.smithi176.stdout:Transaction test succeeded. 2024-07-16T09:45:02.893 INFO:teuthology.orchestra.run.smithi176.stdout:Running transaction 2024-07-16T09:45:02.943 INFO:teuthology.orchestra.run.smithi156.stdout:Running transaction check 2024-07-16T09:45:02.958 INFO:teuthology.orchestra.run.smithi156.stdout:Transaction check succeeded. 2024-07-16T09:45:02.959 INFO:teuthology.orchestra.run.smithi156.stdout:Running transaction test 2024-07-16T09:45:03.043 INFO:teuthology.orchestra.run.smithi156.stdout:Transaction test succeeded. 2024-07-16T09:45:03.044 INFO:teuthology.orchestra.run.smithi156.stdout:Running transaction 2024-07-16T09:45:03.401 INFO:teuthology.orchestra.run.smithi176.stdout: Preparing : 1/1 2024-07-16T09:45:03.561 INFO:teuthology.orchestra.run.smithi156.stdout: Preparing : 1/1 2024-07-16T09:45:03.567 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-07-16T09:45:03.628 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-07-16T09:45:03.691 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-07-16T09:45:03.724 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-07-16T09:45:03.728 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-07-16T09:45:03.769 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 5/13 2024-07-16T09:45:03.778 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-07-16T09:45:03.842 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-07-16T09:45:03.875 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-07-16T09:45:03.879 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-07-16T09:45:03.921 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 5/13 2024-07-16T09:45:03.982 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-google-auth-1:2.30.0-1.el9.noarch 7/13 2024-07-16T09:45:04.018 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-07-16T09:45:04.038 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-07-16T09:45:04.135 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-google-auth-1:2.30.0-1.el9.noarch 7/13 2024-07-16T09:45:04.189 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-07-16T09:45:04.765 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-07-16T09:45:04.810 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-07-16T09:45:04.844 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-07-16T09:45:04.890 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-07-16T09:45:04.906 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-07-16T09:45:04.913 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : ceph-mgr-rook-2:17.2.7-1104.ge6231b04.el9.noarch 13/13 2024-07-16T09:45:04.962 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-07-16T09:45:04.995 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-07-16T09:45:05.041 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-07-16T09:45:05.065 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-mgr-rook-2:17.2.7-1104.ge6231b04.el9.noarch 13/13 2024-07-16T09:45:05.659 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: ceph-mgr-rook-2:17.2.7-1104.ge6231b04.el9.noarch 13/13 2024-07-16T09:45:05.659 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : ceph-mgr-rook-2:17.2.7-1104.ge6231b04.el9.noarch 1/13 2024-07-16T09:45:05.659 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-07-16T09:45:05.659 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-07-16T09:45:05.659 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-07-16T09:45:05.659 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-07-16T09:45:05.659 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-07-16T09:45:05.660 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-07-16T09:45:05.660 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-07-16T09:45:05.660 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 9/13 2024-07-16T09:45:05.660 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-google-auth-1:2.30.0-1.el9.noarch 10/13 2024-07-16T09:45:05.660 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-07-16T09:45:05.660 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-07-16T09:45:05.846 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: ceph-mgr-rook-2:17.2.7-1104.ge6231b04.el9.noarch 13/13 2024-07-16T09:45:05.846 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-mgr-rook-2:17.2.7-1104.ge6231b04.el9.noarch 1/13 2024-07-16T09:45:05.846 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-07-16T09:45:05.847 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-07-16T09:45:05.847 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-07-16T09:45:05.847 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-07-16T09:45:05.847 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-07-16T09:45:05.847 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-07-16T09:45:05.847 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-07-16T09:45:05.847 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 9/13 2024-07-16T09:45:05.847 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-google-auth-1:2.30.0-1.el9.noarch 10/13 2024-07-16T09:45:05.847 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-07-16T09:45:05.847 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-07-16T09:45:05.980 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-07-16T09:45:05.980 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-07-16T09:45:05.980 INFO:teuthology.orchestra.run.smithi176.stdout:Installed: 2024-07-16T09:45:05.980 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-mgr-rook-2:17.2.7-1104.ge6231b04.el9.noarch 2024-07-16T09:45:05.980 INFO:teuthology.orchestra.run.smithi176.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-07-16T09:45:05.980 INFO:teuthology.orchestra.run.smithi176.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-07-16T09:45:05.980 INFO:teuthology.orchestra.run.smithi176.stdout: python3-google-auth-1:2.30.0-1.el9.noarch 2024-07-16T09:45:05.980 INFO:teuthology.orchestra.run.smithi176.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-07-16T09:45:05.981 INFO:teuthology.orchestra.run.smithi176.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-07-16T09:45:05.981 INFO:teuthology.orchestra.run.smithi176.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-07-16T09:45:05.981 INFO:teuthology.orchestra.run.smithi176.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-07-16T09:45:05.981 INFO:teuthology.orchestra.run.smithi176.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-07-16T09:45:05.981 INFO:teuthology.orchestra.run.smithi176.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-07-16T09:45:05.981 INFO:teuthology.orchestra.run.smithi176.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-07-16T09:45:05.981 INFO:teuthology.orchestra.run.smithi176.stdout: python3-rsa-4.9-2.el9.noarch 2024-07-16T09:45:05.981 INFO:teuthology.orchestra.run.smithi176.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-07-16T09:45:05.981 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-07-16T09:45:05.981 INFO:teuthology.orchestra.run.smithi176.stdout:Complete! 2024-07-16T09:45:06.160 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-07-16T09:45:06.160 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-07-16T09:45:06.160 INFO:teuthology.orchestra.run.smithi156.stdout:Installed: 2024-07-16T09:45:06.160 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mgr-rook-2:17.2.7-1104.ge6231b04.el9.noarch 2024-07-16T09:45:06.160 INFO:teuthology.orchestra.run.smithi156.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-07-16T09:45:06.161 INFO:teuthology.orchestra.run.smithi156.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-07-16T09:45:06.161 INFO:teuthology.orchestra.run.smithi156.stdout: python3-google-auth-1:2.30.0-1.el9.noarch 2024-07-16T09:45:06.161 INFO:teuthology.orchestra.run.smithi156.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-07-16T09:45:06.161 INFO:teuthology.orchestra.run.smithi156.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-07-16T09:45:06.161 INFO:teuthology.orchestra.run.smithi156.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-07-16T09:45:06.161 INFO:teuthology.orchestra.run.smithi156.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-07-16T09:45:06.161 INFO:teuthology.orchestra.run.smithi156.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-07-16T09:45:06.161 INFO:teuthology.orchestra.run.smithi156.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-07-16T09:45:06.161 INFO:teuthology.orchestra.run.smithi156.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-07-16T09:45:06.161 INFO:teuthology.orchestra.run.smithi156.stdout: python3-rsa-4.9-2.el9.noarch 2024-07-16T09:45:06.161 INFO:teuthology.orchestra.run.smithi156.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-07-16T09:45:06.161 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-07-16T09:45:06.162 INFO:teuthology.orchestra.run.smithi156.stdout:Complete! 2024-07-16T09:45:06.248 DEBUG:teuthology.orchestra.run.smithi176:> sudo yum -y install ceph-mgr-cephadm 2024-07-16T09:45:06.416 DEBUG:teuthology.orchestra.run.smithi156:> sudo yum -y install ceph-mgr-cephadm 2024-07-16T09:45:06.827 INFO:teuthology.orchestra.run.smithi176.stdout:Last metadata expiration check: 0:01:21 ago on Tue 16 Jul 2024 09:43:45 AM UTC. 2024-07-16T09:45:06.954 INFO:teuthology.orchestra.run.smithi176.stdout:Dependencies resolved. 2024-07-16T09:45:06.955 INFO:teuthology.orchestra.run.smithi176.stdout:================================================================================ 2024-07-16T09:45:06.955 INFO:teuthology.orchestra.run.smithi176.stdout: Package Arch Version Repository Size 2024-07-16T09:45:06.955 INFO:teuthology.orchestra.run.smithi176.stdout:================================================================================ 2024-07-16T09:45:06.955 INFO:teuthology.orchestra.run.smithi176.stdout:Installing: 2024-07-16T09:45:06.955 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-mgr-cephadm noarch 2:17.2.7-1104.ge6231b04.el9 ceph-noarch 123 k 2024-07-16T09:45:06.955 INFO:teuthology.orchestra.run.smithi176.stdout:Installing dependencies: 2024-07-16T09:45:06.955 INFO:teuthology.orchestra.run.smithi176.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-07-16T09:45:06.955 INFO:teuthology.orchestra.run.smithi176.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-07-16T09:45:06.956 INFO:teuthology.orchestra.run.smithi176.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-07-16T09:45:06.956 INFO:teuthology.orchestra.run.smithi176.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-07-16T09:45:06.956 INFO:teuthology.orchestra.run.smithi176.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-07-16T09:45:06.956 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-07-16T09:45:06.956 INFO:teuthology.orchestra.run.smithi176.stdout:Transaction Summary 2024-07-16T09:45:06.956 INFO:teuthology.orchestra.run.smithi176.stdout:================================================================================ 2024-07-16T09:45:06.956 INFO:teuthology.orchestra.run.smithi176.stdout:Install 6 Packages 2024-07-16T09:45:06.956 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-07-16T09:45:06.957 INFO:teuthology.orchestra.run.smithi176.stdout:Total download size: 7.0 M 2024-07-16T09:45:06.957 INFO:teuthology.orchestra.run.smithi176.stdout:Installed size: 33 M 2024-07-16T09:45:06.957 INFO:teuthology.orchestra.run.smithi176.stdout:Downloading Packages: 2024-07-16T09:45:06.998 INFO:teuthology.orchestra.run.smithi156.stdout:Last metadata expiration check: 0:01:26 ago on Tue 16 Jul 2024 09:43:40 AM UTC. 2024-07-16T09:45:07.125 INFO:teuthology.orchestra.run.smithi156.stdout:Dependencies resolved. 2024-07-16T09:45:07.126 INFO:teuthology.orchestra.run.smithi156.stdout:================================================================================ 2024-07-16T09:45:07.126 INFO:teuthology.orchestra.run.smithi156.stdout: Package Arch Version Repository Size 2024-07-16T09:45:07.126 INFO:teuthology.orchestra.run.smithi156.stdout:================================================================================ 2024-07-16T09:45:07.126 INFO:teuthology.orchestra.run.smithi156.stdout:Installing: 2024-07-16T09:45:07.126 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mgr-cephadm noarch 2:17.2.7-1104.ge6231b04.el9 ceph-noarch 123 k 2024-07-16T09:45:07.126 INFO:teuthology.orchestra.run.smithi156.stdout:Installing dependencies: 2024-07-16T09:45:07.126 INFO:teuthology.orchestra.run.smithi156.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-07-16T09:45:07.127 INFO:teuthology.orchestra.run.smithi156.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-07-16T09:45:07.127 INFO:teuthology.orchestra.run.smithi156.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-07-16T09:45:07.127 INFO:teuthology.orchestra.run.smithi156.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-07-16T09:45:07.127 INFO:teuthology.orchestra.run.smithi156.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-07-16T09:45:07.127 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-07-16T09:45:07.127 INFO:teuthology.orchestra.run.smithi156.stdout:Transaction Summary 2024-07-16T09:45:07.127 INFO:teuthology.orchestra.run.smithi156.stdout:================================================================================ 2024-07-16T09:45:07.127 INFO:teuthology.orchestra.run.smithi156.stdout:Install 6 Packages 2024-07-16T09:45:07.127 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-07-16T09:45:07.128 INFO:teuthology.orchestra.run.smithi156.stdout:Total download size: 7.0 M 2024-07-16T09:45:07.128 INFO:teuthology.orchestra.run.smithi156.stdout:Installed size: 33 M 2024-07-16T09:45:07.128 INFO:teuthology.orchestra.run.smithi156.stdout:Downloading Packages: 2024-07-16T09:45:07.547 INFO:teuthology.orchestra.run.smithi156.stdout:(1/6): python3-jinja2-2.11.3-6.el9.noarch.rpm 1.7 MB/s | 248 kB 00:00 2024-07-16T09:45:07.614 INFO:teuthology.orchestra.run.smithi156.stdout:(2/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 8.1 MB/s | 548 kB 00:00 2024-07-16T09:45:07.640 INFO:teuthology.orchestra.run.smithi156.stdout:(3/6): python3-natsort-7.1.1-5.el9.noarch.rpm 2.2 MB/s | 58 kB 00:00 2024-07-16T09:45:07.665 INFO:teuthology.orchestra.run.smithi156.stdout:(4/6): python3-typing-extensions-4.4.0-2.el9.no 2.0 MB/s | 51 kB 00:00 2024-07-16T09:45:07.677 INFO:teuthology.orchestra.run.smithi176.stdout:(1/6): ceph-mgr-cephadm-17.2.7-1104.ge6231b04.e 410 kB/s | 123 kB 00:00 2024-07-16T09:45:07.706 INFO:teuthology.orchestra.run.smithi156.stdout:(5/6): ceph-mgr-cephadm-17.2.7-1104.ge6231b04.e 407 kB/s | 123 kB 00:00 2024-07-16T09:45:07.745 INFO:teuthology.orchestra.run.smithi176.stdout:(2/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 8.0 MB/s | 548 kB 00:00 2024-07-16T09:45:07.770 INFO:teuthology.orchestra.run.smithi176.stdout:(3/6): python3-natsort-7.1.1-5.el9.noarch.rpm 2.3 MB/s | 58 kB 00:00 2024-07-16T09:45:07.795 INFO:teuthology.orchestra.run.smithi176.stdout:(4/6): python3-typing-extensions-4.4.0-2.el9.no 2.0 MB/s | 51 kB 00:00 2024-07-16T09:45:07.798 INFO:teuthology.orchestra.run.smithi156.stdout:(6/6): python3-babel-2.9.1-2.el9.noarch.rpm 15 MB/s | 6.0 MB 00:00 2024-07-16T09:45:07.800 INFO:teuthology.orchestra.run.smithi156.stdout:-------------------------------------------------------------------------------- 2024-07-16T09:45:07.800 INFO:teuthology.orchestra.run.smithi156.stdout:Total 10 MB/s | 7.0 MB 00:00 2024-07-16T09:45:07.870 INFO:teuthology.orchestra.run.smithi156.stdout:Running transaction check 2024-07-16T09:45:07.882 INFO:teuthology.orchestra.run.smithi156.stdout:Transaction check succeeded. 2024-07-16T09:45:07.882 INFO:teuthology.orchestra.run.smithi156.stdout:Running transaction test 2024-07-16T09:45:07.980 INFO:teuthology.orchestra.run.smithi156.stdout:Transaction test succeeded. 2024-07-16T09:45:07.982 INFO:teuthology.orchestra.run.smithi156.stdout:Running transaction 2024-07-16T09:45:08.053 INFO:teuthology.orchestra.run.smithi176.stdout:(5/6): python3-jinja2-2.11.3-6.el9.noarch.rpm 367 kB/s | 248 kB 00:00 2024-07-16T09:45:08.282 INFO:teuthology.orchestra.run.smithi156.stdout: Preparing : 1/1 2024-07-16T09:45:08.464 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 1/6 2024-07-16T09:45:08.517 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-07-16T09:45:09.042 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-07-16T09:45:09.129 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-07-16T09:45:09.192 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 5/6 2024-07-16T09:45:09.221 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-mgr-cephadm-2:17.2.7-1104.ge6231b04.el9.noarch 6/6 2024-07-16T09:45:09.973 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: ceph-mgr-cephadm-2:17.2.7-1104.ge6231b04.el9.noarch 6/6 2024-07-16T09:45:09.974 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-mgr-cephadm-2:17.2.7-1104.ge6231b04.el9.noarch 1/6 2024-07-16T09:45:09.974 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-07-16T09:45:09.974 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 3/6 2024-07-16T09:45:09.974 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-07-16T09:45:09.974 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-07-16T09:45:10.242 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 6/6 2024-07-16T09:45:10.242 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-07-16T09:45:10.242 INFO:teuthology.orchestra.run.smithi156.stdout:Installed: 2024-07-16T09:45:10.242 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mgr-cephadm-2:17.2.7-1104.ge6231b04.el9.noarch 2024-07-16T09:45:10.243 INFO:teuthology.orchestra.run.smithi156.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-07-16T09:45:10.243 INFO:teuthology.orchestra.run.smithi156.stdout: python3-babel-2.9.1-2.el9.noarch 2024-07-16T09:45:10.243 INFO:teuthology.orchestra.run.smithi156.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-07-16T09:45:10.243 INFO:teuthology.orchestra.run.smithi156.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-07-16T09:45:10.243 INFO:teuthology.orchestra.run.smithi156.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-07-16T09:45:10.243 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-07-16T09:45:10.243 INFO:teuthology.orchestra.run.smithi156.stdout:Complete! 2024-07-16T09:45:10.504 DEBUG:teuthology.orchestra.run.smithi156:> sudo yum -y install ceph-fuse 2024-07-16T09:45:11.053 INFO:teuthology.orchestra.run.smithi176.stdout:(6/6): python3-babel-2.9.1-2.el9.noarch.rpm 1.6 MB/s | 6.0 MB 00:03 2024-07-16T09:45:11.055 INFO:teuthology.orchestra.run.smithi176.stdout:-------------------------------------------------------------------------------- 2024-07-16T09:45:11.056 INFO:teuthology.orchestra.run.smithi176.stdout:Total 1.7 MB/s | 7.0 MB 00:04 2024-07-16T09:45:11.086 INFO:teuthology.orchestra.run.smithi156.stdout:Last metadata expiration check: 0:01:31 ago on Tue 16 Jul 2024 09:43:40 AM UTC. 2024-07-16T09:45:11.126 INFO:teuthology.orchestra.run.smithi176.stdout:Running transaction check 2024-07-16T09:45:11.138 INFO:teuthology.orchestra.run.smithi176.stdout:Transaction check succeeded. 2024-07-16T09:45:11.138 INFO:teuthology.orchestra.run.smithi176.stdout:Running transaction test 2024-07-16T09:45:11.213 INFO:teuthology.orchestra.run.smithi156.stdout:Dependencies resolved. 2024-07-16T09:45:11.214 INFO:teuthology.orchestra.run.smithi156.stdout:================================================================================ 2024-07-16T09:45:11.214 INFO:teuthology.orchestra.run.smithi156.stdout: Package Arch Version Repository Size 2024-07-16T09:45:11.214 INFO:teuthology.orchestra.run.smithi156.stdout:================================================================================ 2024-07-16T09:45:11.214 INFO:teuthology.orchestra.run.smithi156.stdout:Installing: 2024-07-16T09:45:11.214 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-fuse x86_64 2:17.2.7-1104.ge6231b04.el9 ceph 814 k 2024-07-16T09:45:11.214 INFO:teuthology.orchestra.run.smithi156.stdout:Installing dependencies: 2024-07-16T09:45:11.214 INFO:teuthology.orchestra.run.smithi156.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-07-16T09:45:11.214 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-07-16T09:45:11.214 INFO:teuthology.orchestra.run.smithi156.stdout:Transaction Summary 2024-07-16T09:45:11.214 INFO:teuthology.orchestra.run.smithi156.stdout:================================================================================ 2024-07-16T09:45:11.214 INFO:teuthology.orchestra.run.smithi156.stdout:Install 2 Packages 2024-07-16T09:45:11.215 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-07-16T09:45:11.215 INFO:teuthology.orchestra.run.smithi156.stdout:Total download size: 894 k 2024-07-16T09:45:11.215 INFO:teuthology.orchestra.run.smithi156.stdout:Installed size: 2.6 M 2024-07-16T09:45:11.215 INFO:teuthology.orchestra.run.smithi156.stdout:Downloading Packages: 2024-07-16T09:45:11.240 INFO:teuthology.orchestra.run.smithi176.stdout:Transaction test succeeded. 2024-07-16T09:45:11.240 INFO:teuthology.orchestra.run.smithi176.stdout:Running transaction 2024-07-16T09:45:11.385 INFO:teuthology.orchestra.run.smithi156.stdout:(1/2): fuse-2.9.9-16.el9.x86_64.rpm 608 kB/s | 80 kB 00:00 2024-07-16T09:45:11.545 INFO:teuthology.orchestra.run.smithi176.stdout: Preparing : 1/1 2024-07-16T09:45:11.576 INFO:teuthology.orchestra.run.smithi156.stdout:(2/2): ceph-fuse-17.2.7-1104.ge6231b04.el9.x86_ 2.5 MB/s | 814 kB 00:00 2024-07-16T09:45:11.577 INFO:teuthology.orchestra.run.smithi156.stdout:-------------------------------------------------------------------------------- 2024-07-16T09:45:11.577 INFO:teuthology.orchestra.run.smithi156.stdout:Total 2.4 MB/s | 894 kB 00:00 2024-07-16T09:45:11.588 INFO:teuthology.orchestra.run.smithi156.stdout:Running transaction check 2024-07-16T09:45:11.610 INFO:teuthology.orchestra.run.smithi156.stdout:Transaction check succeeded. 2024-07-16T09:45:11.610 INFO:teuthology.orchestra.run.smithi156.stdout:Running transaction test 2024-07-16T09:45:11.687 INFO:teuthology.orchestra.run.smithi156.stdout:Transaction test succeeded. 2024-07-16T09:45:11.687 INFO:teuthology.orchestra.run.smithi156.stdout:Running transaction 2024-07-16T09:45:11.716 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 1/6 2024-07-16T09:45:11.763 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-07-16T09:45:11.875 INFO:teuthology.orchestra.run.smithi156.stdout: Preparing : 1/1 2024-07-16T09:45:12.014 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : fuse-2.9.9-16.el9.x86_64 1/2 2024-07-16T09:45:12.075 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-fuse-2:17.2.7-1104.ge6231b04.el9.x86_64 2/2 2024-07-16T09:45:12.291 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-07-16T09:45:12.382 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-07-16T09:45:12.437 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 5/6 2024-07-16T09:45:12.467 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : ceph-mgr-cephadm-2:17.2.7-1104.ge6231b04.el9.noarch 6/6 2024-07-16T09:45:12.716 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: ceph-fuse-2:17.2.7-1104.ge6231b04.el9.x86_64 2/2 2024-07-16T09:45:12.716 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-fuse-2:17.2.7-1104.ge6231b04.el9.x86_64 1/2 2024-07-16T09:45:12.978 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 2/2 2024-07-16T09:45:12.978 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-07-16T09:45:12.978 INFO:teuthology.orchestra.run.smithi156.stdout:Installed: 2024-07-16T09:45:12.978 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-fuse-2:17.2.7-1104.ge6231b04.el9.x86_64 fuse-2.9.9-16.el9.x86_64 2024-07-16T09:45:12.978 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-07-16T09:45:12.978 INFO:teuthology.orchestra.run.smithi156.stdout:Complete! 2024-07-16T09:45:13.237 DEBUG:teuthology.orchestra.run.smithi156:> sudo yum -y install librados-devel 2024-07-16T09:45:13.306 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: ceph-mgr-cephadm-2:17.2.7-1104.ge6231b04.el9.noarch 6/6 2024-07-16T09:45:13.306 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : ceph-mgr-cephadm-2:17.2.7-1104.ge6231b04.el9.noarch 1/6 2024-07-16T09:45:13.306 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-07-16T09:45:13.306 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 3/6 2024-07-16T09:45:13.306 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-07-16T09:45:13.306 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-07-16T09:45:13.561 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 6/6 2024-07-16T09:45:13.561 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-07-16T09:45:13.561 INFO:teuthology.orchestra.run.smithi176.stdout:Installed: 2024-07-16T09:45:13.561 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-mgr-cephadm-2:17.2.7-1104.ge6231b04.el9.noarch 2024-07-16T09:45:13.561 INFO:teuthology.orchestra.run.smithi176.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-07-16T09:45:13.561 INFO:teuthology.orchestra.run.smithi176.stdout: python3-babel-2.9.1-2.el9.noarch 2024-07-16T09:45:13.561 INFO:teuthology.orchestra.run.smithi176.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-07-16T09:45:13.561 INFO:teuthology.orchestra.run.smithi176.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-07-16T09:45:13.562 INFO:teuthology.orchestra.run.smithi176.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-07-16T09:45:13.562 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-07-16T09:45:13.562 INFO:teuthology.orchestra.run.smithi176.stdout:Complete! 2024-07-16T09:45:13.807 DEBUG:teuthology.orchestra.run.smithi176:> sudo yum -y install ceph-fuse 2024-07-16T09:45:13.834 INFO:teuthology.orchestra.run.smithi156.stdout:Last metadata expiration check: 0:01:33 ago on Tue 16 Jul 2024 09:43:40 AM UTC. 2024-07-16T09:45:13.966 INFO:teuthology.orchestra.run.smithi156.stdout:Dependencies resolved. 2024-07-16T09:45:13.967 INFO:teuthology.orchestra.run.smithi156.stdout:================================================================================ 2024-07-16T09:45:13.967 INFO:teuthology.orchestra.run.smithi156.stdout: Package Arch Version Repo Size 2024-07-16T09:45:13.967 INFO:teuthology.orchestra.run.smithi156.stdout:================================================================================ 2024-07-16T09:45:13.968 INFO:teuthology.orchestra.run.smithi156.stdout:Installing: 2024-07-16T09:45:13.968 INFO:teuthology.orchestra.run.smithi156.stdout: librados-devel x86_64 2:17.2.7-1104.ge6231b04.el9 ceph 126 k 2024-07-16T09:45:13.968 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-07-16T09:45:13.968 INFO:teuthology.orchestra.run.smithi156.stdout:Transaction Summary 2024-07-16T09:45:13.968 INFO:teuthology.orchestra.run.smithi156.stdout:================================================================================ 2024-07-16T09:45:13.968 INFO:teuthology.orchestra.run.smithi156.stdout:Install 1 Package 2024-07-16T09:45:13.968 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-07-16T09:45:13.968 INFO:teuthology.orchestra.run.smithi156.stdout:Total download size: 126 k 2024-07-16T09:45:13.968 INFO:teuthology.orchestra.run.smithi156.stdout:Installed size: 448 k 2024-07-16T09:45:13.968 INFO:teuthology.orchestra.run.smithi156.stdout:Downloading Packages: 2024-07-16T09:45:14.223 INFO:teuthology.orchestra.run.smithi156.stdout:librados-devel-17.2.7-1104.ge6231b04.el9.x86_64 494 kB/s | 126 kB 00:00 2024-07-16T09:45:14.224 INFO:teuthology.orchestra.run.smithi156.stdout:-------------------------------------------------------------------------------- 2024-07-16T09:45:14.224 INFO:teuthology.orchestra.run.smithi156.stdout:Total 492 kB/s | 126 kB 00:00 2024-07-16T09:45:14.224 INFO:teuthology.orchestra.run.smithi156.stdout:Running transaction check 2024-07-16T09:45:14.230 INFO:teuthology.orchestra.run.smithi156.stdout:Transaction check succeeded. 2024-07-16T09:45:14.230 INFO:teuthology.orchestra.run.smithi156.stdout:Running transaction test 2024-07-16T09:45:14.293 INFO:teuthology.orchestra.run.smithi156.stdout:Transaction test succeeded. 2024-07-16T09:45:14.293 INFO:teuthology.orchestra.run.smithi156.stdout:Running transaction 2024-07-16T09:45:14.394 INFO:teuthology.orchestra.run.smithi176.stdout:Last metadata expiration check: 0:01:29 ago on Tue 16 Jul 2024 09:43:45 AM UTC. 2024-07-16T09:45:14.442 INFO:teuthology.orchestra.run.smithi156.stdout: Preparing : 1/1 2024-07-16T09:45:14.521 INFO:teuthology.orchestra.run.smithi176.stdout:Dependencies resolved. 2024-07-16T09:45:14.522 INFO:teuthology.orchestra.run.smithi176.stdout:================================================================================ 2024-07-16T09:45:14.522 INFO:teuthology.orchestra.run.smithi176.stdout: Package Arch Version Repository Size 2024-07-16T09:45:14.522 INFO:teuthology.orchestra.run.smithi176.stdout:================================================================================ 2024-07-16T09:45:14.522 INFO:teuthology.orchestra.run.smithi176.stdout:Installing: 2024-07-16T09:45:14.522 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-fuse x86_64 2:17.2.7-1104.ge6231b04.el9 ceph 814 k 2024-07-16T09:45:14.522 INFO:teuthology.orchestra.run.smithi176.stdout:Installing dependencies: 2024-07-16T09:45:14.522 INFO:teuthology.orchestra.run.smithi176.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-07-16T09:45:14.522 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-07-16T09:45:14.522 INFO:teuthology.orchestra.run.smithi176.stdout:Transaction Summary 2024-07-16T09:45:14.523 INFO:teuthology.orchestra.run.smithi176.stdout:================================================================================ 2024-07-16T09:45:14.523 INFO:teuthology.orchestra.run.smithi176.stdout:Install 2 Packages 2024-07-16T09:45:14.523 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-07-16T09:45:14.523 INFO:teuthology.orchestra.run.smithi176.stdout:Total download size: 894 k 2024-07-16T09:45:14.523 INFO:teuthology.orchestra.run.smithi176.stdout:Installed size: 2.6 M 2024-07-16T09:45:14.524 INFO:teuthology.orchestra.run.smithi176.stdout:Downloading Packages: 2024-07-16T09:45:14.529 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : librados-devel-2:17.2.7-1104.ge6231b04.el9.x86_64 1/1 2024-07-16T09:45:14.746 INFO:teuthology.orchestra.run.smithi176.stdout:(1/2): fuse-2.9.9-16.el9.x86_64.rpm 429 kB/s | 80 kB 00:00 2024-07-16T09:45:14.928 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: librados-devel-2:17.2.7-1104.ge6231b04.el9.x86_64 1/1 2024-07-16T09:45:15.130 INFO:teuthology.orchestra.run.smithi176.stdout:(2/2): ceph-fuse-17.2.7-1104.ge6231b04.el9.x86_ 1.4 MB/s | 814 kB 00:00 2024-07-16T09:45:15.130 INFO:teuthology.orchestra.run.smithi176.stdout:-------------------------------------------------------------------------------- 2024-07-16T09:45:15.131 INFO:teuthology.orchestra.run.smithi176.stdout:Total 1.4 MB/s | 894 kB 00:00 2024-07-16T09:45:15.143 INFO:teuthology.orchestra.run.smithi176.stdout:Running transaction check 2024-07-16T09:45:15.168 INFO:teuthology.orchestra.run.smithi176.stdout:Transaction check succeeded. 2024-07-16T09:45:15.169 INFO:teuthology.orchestra.run.smithi176.stdout:Running transaction test 2024-07-16T09:45:15.209 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : librados-devel-2:17.2.7-1104.ge6231b04.el9.x86_64 1/1 2024-07-16T09:45:15.209 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-07-16T09:45:15.209 INFO:teuthology.orchestra.run.smithi156.stdout:Installed: 2024-07-16T09:45:15.209 INFO:teuthology.orchestra.run.smithi156.stdout: librados-devel-2:17.2.7-1104.ge6231b04.el9.x86_64 2024-07-16T09:45:15.210 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-07-16T09:45:15.210 INFO:teuthology.orchestra.run.smithi156.stdout:Complete! 2024-07-16T09:45:15.246 INFO:teuthology.orchestra.run.smithi176.stdout:Transaction test succeeded. 2024-07-16T09:45:15.247 INFO:teuthology.orchestra.run.smithi176.stdout:Running transaction 2024-07-16T09:45:15.430 INFO:teuthology.orchestra.run.smithi176.stdout: Preparing : 1/1 2024-07-16T09:45:15.469 DEBUG:teuthology.orchestra.run.smithi156:> sudo yum -y install libcephfs2 2024-07-16T09:45:15.567 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : fuse-2.9.9-16.el9.x86_64 1/2 2024-07-16T09:45:15.646 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : ceph-fuse-2:17.2.7-1104.ge6231b04.el9.x86_64 2/2 2024-07-16T09:45:16.070 INFO:teuthology.orchestra.run.smithi156.stdout:Last metadata expiration check: 0:01:36 ago on Tue 16 Jul 2024 09:43:40 AM UTC. 2024-07-16T09:45:16.158 INFO:teuthology.orchestra.run.smithi156.stdout:Package libcephfs2-2:17.2.7-1104.ge6231b04.el9.x86_64 is already installed. 2024-07-16T09:45:16.204 INFO:teuthology.orchestra.run.smithi156.stdout:Dependencies resolved. 2024-07-16T09:45:16.205 INFO:teuthology.orchestra.run.smithi156.stdout:Nothing to do. 2024-07-16T09:45:16.205 INFO:teuthology.orchestra.run.smithi156.stdout:Complete! 2024-07-16T09:45:16.275 DEBUG:teuthology.orchestra.run.smithi156:> sudo yum -y install libcephfs-devel 2024-07-16T09:45:16.291 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: ceph-fuse-2:17.2.7-1104.ge6231b04.el9.x86_64 2/2 2024-07-16T09:45:16.291 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : ceph-fuse-2:17.2.7-1104.ge6231b04.el9.x86_64 1/2 2024-07-16T09:45:16.556 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 2/2 2024-07-16T09:45:16.556 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-07-16T09:45:16.556 INFO:teuthology.orchestra.run.smithi176.stdout:Installed: 2024-07-16T09:45:16.556 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-fuse-2:17.2.7-1104.ge6231b04.el9.x86_64 fuse-2.9.9-16.el9.x86_64 2024-07-16T09:45:16.556 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-07-16T09:45:16.556 INFO:teuthology.orchestra.run.smithi176.stdout:Complete! 2024-07-16T09:45:16.811 DEBUG:teuthology.orchestra.run.smithi176:> sudo yum -y install librados-devel 2024-07-16T09:45:16.854 INFO:teuthology.orchestra.run.smithi156.stdout:Last metadata expiration check: 0:01:36 ago on Tue 16 Jul 2024 09:43:40 AM UTC. 2024-07-16T09:45:16.982 INFO:teuthology.orchestra.run.smithi156.stdout:Dependencies resolved. 2024-07-16T09:45:16.982 INFO:teuthology.orchestra.run.smithi156.stdout:================================================================================ 2024-07-16T09:45:16.983 INFO:teuthology.orchestra.run.smithi156.stdout: Package Arch Version Repo Size 2024-07-16T09:45:16.983 INFO:teuthology.orchestra.run.smithi156.stdout:================================================================================ 2024-07-16T09:45:16.983 INFO:teuthology.orchestra.run.smithi156.stdout:Installing: 2024-07-16T09:45:16.983 INFO:teuthology.orchestra.run.smithi156.stdout: libcephfs-devel x86_64 2:17.2.7-1104.ge6231b04.el9 ceph 25 k 2024-07-16T09:45:16.983 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-07-16T09:45:16.983 INFO:teuthology.orchestra.run.smithi156.stdout:Transaction Summary 2024-07-16T09:45:16.983 INFO:teuthology.orchestra.run.smithi156.stdout:================================================================================ 2024-07-16T09:45:16.983 INFO:teuthology.orchestra.run.smithi156.stdout:Install 1 Package 2024-07-16T09:45:16.983 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-07-16T09:45:16.983 INFO:teuthology.orchestra.run.smithi156.stdout:Total download size: 25 k 2024-07-16T09:45:16.983 INFO:teuthology.orchestra.run.smithi156.stdout:Installed size: 110 k 2024-07-16T09:45:16.983 INFO:teuthology.orchestra.run.smithi156.stdout:Downloading Packages: 2024-07-16T09:45:17.161 INFO:teuthology.orchestra.run.smithi156.stdout:libcephfs-devel-17.2.7-1104.ge6231b04.el9.x86_6 140 kB/s | 25 kB 00:00 2024-07-16T09:45:17.162 INFO:teuthology.orchestra.run.smithi156.stdout:-------------------------------------------------------------------------------- 2024-07-16T09:45:17.162 INFO:teuthology.orchestra.run.smithi156.stdout:Total 139 kB/s | 25 kB 00:00 2024-07-16T09:45:17.162 INFO:teuthology.orchestra.run.smithi156.stdout:Running transaction check 2024-07-16T09:45:17.165 INFO:teuthology.orchestra.run.smithi156.stdout:Transaction check succeeded. 2024-07-16T09:45:17.165 INFO:teuthology.orchestra.run.smithi156.stdout:Running transaction test 2024-07-16T09:45:17.167 INFO:teuthology.orchestra.run.smithi156.stdout:Transaction test succeeded. 2024-07-16T09:45:17.167 INFO:teuthology.orchestra.run.smithi156.stdout:Running transaction 2024-07-16T09:45:17.245 INFO:teuthology.orchestra.run.smithi156.stdout: Preparing : 1/1 2024-07-16T09:45:17.350 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : libcephfs-devel-2:17.2.7-1104.ge6231b04.el9.x86_64 1/1 2024-07-16T09:45:17.409 INFO:teuthology.orchestra.run.smithi176.stdout:Last metadata expiration check: 0:01:32 ago on Tue 16 Jul 2024 09:43:45 AM UTC. 2024-07-16T09:45:17.541 INFO:teuthology.orchestra.run.smithi176.stdout:Dependencies resolved. 2024-07-16T09:45:17.542 INFO:teuthology.orchestra.run.smithi176.stdout:================================================================================ 2024-07-16T09:45:17.542 INFO:teuthology.orchestra.run.smithi176.stdout: Package Arch Version Repo Size 2024-07-16T09:45:17.542 INFO:teuthology.orchestra.run.smithi176.stdout:================================================================================ 2024-07-16T09:45:17.542 INFO:teuthology.orchestra.run.smithi176.stdout:Installing: 2024-07-16T09:45:17.542 INFO:teuthology.orchestra.run.smithi176.stdout: librados-devel x86_64 2:17.2.7-1104.ge6231b04.el9 ceph 126 k 2024-07-16T09:45:17.542 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-07-16T09:45:17.542 INFO:teuthology.orchestra.run.smithi176.stdout:Transaction Summary 2024-07-16T09:45:17.542 INFO:teuthology.orchestra.run.smithi176.stdout:================================================================================ 2024-07-16T09:45:17.543 INFO:teuthology.orchestra.run.smithi176.stdout:Install 1 Package 2024-07-16T09:45:17.543 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-07-16T09:45:17.543 INFO:teuthology.orchestra.run.smithi176.stdout:Total download size: 126 k 2024-07-16T09:45:17.543 INFO:teuthology.orchestra.run.smithi176.stdout:Installed size: 448 k 2024-07-16T09:45:17.543 INFO:teuthology.orchestra.run.smithi176.stdout:Downloading Packages: 2024-07-16T09:45:17.707 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: libcephfs-devel-2:17.2.7-1104.ge6231b04.el9.x86_64 1/1 2024-07-16T09:45:17.768 INFO:teuthology.orchestra.run.smithi176.stdout:librados-devel-17.2.7-1104.ge6231b04.el9.x86_64 562 kB/s | 126 kB 00:00 2024-07-16T09:45:17.768 INFO:teuthology.orchestra.run.smithi176.stdout:-------------------------------------------------------------------------------- 2024-07-16T09:45:17.769 INFO:teuthology.orchestra.run.smithi176.stdout:Total 558 kB/s | 126 kB 00:00 2024-07-16T09:45:17.769 INFO:teuthology.orchestra.run.smithi176.stdout:Running transaction check 2024-07-16T09:45:17.774 INFO:teuthology.orchestra.run.smithi176.stdout:Transaction check succeeded. 2024-07-16T09:45:17.774 INFO:teuthology.orchestra.run.smithi176.stdout:Running transaction test 2024-07-16T09:45:17.837 INFO:teuthology.orchestra.run.smithi176.stdout:Transaction test succeeded. 2024-07-16T09:45:17.837 INFO:teuthology.orchestra.run.smithi176.stdout:Running transaction 2024-07-16T09:45:17.932 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libcephfs-devel-2:17.2.7-1104.ge6231b04.el9.x86_64 1/1 2024-07-16T09:45:17.932 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-07-16T09:45:17.932 INFO:teuthology.orchestra.run.smithi156.stdout:Installed: 2024-07-16T09:45:17.932 INFO:teuthology.orchestra.run.smithi156.stdout: libcephfs-devel-2:17.2.7-1104.ge6231b04.el9.x86_64 2024-07-16T09:45:17.932 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-07-16T09:45:17.932 INFO:teuthology.orchestra.run.smithi156.stdout:Complete! 2024-07-16T09:45:17.988 INFO:teuthology.orchestra.run.smithi176.stdout: Preparing : 1/1 2024-07-16T09:45:18.090 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : librados-devel-2:17.2.7-1104.ge6231b04.el9.x86_64 1/1 2024-07-16T09:45:18.115 DEBUG:teuthology.orchestra.run.smithi156:> sudo yum -y install librados2 2024-07-16T09:45:18.505 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: librados-devel-2:17.2.7-1104.ge6231b04.el9.x86_64 1/1 2024-07-16T09:45:18.695 INFO:teuthology.orchestra.run.smithi156.stdout:Last metadata expiration check: 0:01:38 ago on Tue 16 Jul 2024 09:43:40 AM UTC. 2024-07-16T09:45:18.759 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : librados-devel-2:17.2.7-1104.ge6231b04.el9.x86_64 1/1 2024-07-16T09:45:18.759 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-07-16T09:45:18.759 INFO:teuthology.orchestra.run.smithi176.stdout:Installed: 2024-07-16T09:45:18.759 INFO:teuthology.orchestra.run.smithi176.stdout: librados-devel-2:17.2.7-1104.ge6231b04.el9.x86_64 2024-07-16T09:45:18.760 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-07-16T09:45:18.760 INFO:teuthology.orchestra.run.smithi176.stdout:Complete! 2024-07-16T09:45:18.781 INFO:teuthology.orchestra.run.smithi156.stdout:Package librados2-2:17.2.7-1104.ge6231b04.el9.x86_64 is already installed. 2024-07-16T09:45:18.822 INFO:teuthology.orchestra.run.smithi156.stdout:Dependencies resolved. 2024-07-16T09:45:18.823 INFO:teuthology.orchestra.run.smithi156.stdout:Nothing to do. 2024-07-16T09:45:18.823 INFO:teuthology.orchestra.run.smithi156.stdout:Complete! 2024-07-16T09:45:18.894 DEBUG:teuthology.orchestra.run.smithi156:> sudo yum -y install librbd1 2024-07-16T09:45:18.978 DEBUG:teuthology.orchestra.run.smithi176:> sudo yum -y install libcephfs2 2024-07-16T09:45:19.472 INFO:teuthology.orchestra.run.smithi156.stdout:Last metadata expiration check: 0:01:39 ago on Tue 16 Jul 2024 09:43:40 AM UTC. 2024-07-16T09:45:19.557 INFO:teuthology.orchestra.run.smithi156.stdout:Package librbd1-2:17.2.7-1104.ge6231b04.el9.x86_64 is already installed. 2024-07-16T09:45:19.577 INFO:teuthology.orchestra.run.smithi176.stdout:Last metadata expiration check: 0:01:34 ago on Tue 16 Jul 2024 09:43:45 AM UTC. 2024-07-16T09:45:19.599 INFO:teuthology.orchestra.run.smithi156.stdout:Dependencies resolved. 2024-07-16T09:45:19.600 INFO:teuthology.orchestra.run.smithi156.stdout:Nothing to do. 2024-07-16T09:45:19.600 INFO:teuthology.orchestra.run.smithi156.stdout:Complete! 2024-07-16T09:45:19.663 INFO:teuthology.orchestra.run.smithi176.stdout:Package libcephfs2-2:17.2.7-1104.ge6231b04.el9.x86_64 is already installed. 2024-07-16T09:45:19.668 DEBUG:teuthology.orchestra.run.smithi156:> sudo yum -y install python3-rados 2024-07-16T09:45:19.708 INFO:teuthology.orchestra.run.smithi176.stdout:Dependencies resolved. 2024-07-16T09:45:19.709 INFO:teuthology.orchestra.run.smithi176.stdout:Nothing to do. 2024-07-16T09:45:19.709 INFO:teuthology.orchestra.run.smithi176.stdout:Complete! 2024-07-16T09:45:19.779 DEBUG:teuthology.orchestra.run.smithi176:> sudo yum -y install libcephfs-devel 2024-07-16T09:45:20.253 INFO:teuthology.orchestra.run.smithi156.stdout:Last metadata expiration check: 0:01:40 ago on Tue 16 Jul 2024 09:43:40 AM UTC. 2024-07-16T09:45:20.340 INFO:teuthology.orchestra.run.smithi156.stdout:Package python3-rados-2:17.2.7-1104.ge6231b04.el9.x86_64 is already installed. 2024-07-16T09:45:20.359 INFO:teuthology.orchestra.run.smithi176.stdout:Last metadata expiration check: 0:01:35 ago on Tue 16 Jul 2024 09:43:45 AM UTC. 2024-07-16T09:45:20.383 INFO:teuthology.orchestra.run.smithi156.stdout:Dependencies resolved. 2024-07-16T09:45:20.384 INFO:teuthology.orchestra.run.smithi156.stdout:Nothing to do. 2024-07-16T09:45:20.384 INFO:teuthology.orchestra.run.smithi156.stdout:Complete! 2024-07-16T09:45:20.460 DEBUG:teuthology.orchestra.run.smithi156:> sudo yum -y install python3-rgw 2024-07-16T09:45:20.487 INFO:teuthology.orchestra.run.smithi176.stdout:Dependencies resolved. 2024-07-16T09:45:20.488 INFO:teuthology.orchestra.run.smithi176.stdout:================================================================================ 2024-07-16T09:45:20.488 INFO:teuthology.orchestra.run.smithi176.stdout: Package Arch Version Repo Size 2024-07-16T09:45:20.488 INFO:teuthology.orchestra.run.smithi176.stdout:================================================================================ 2024-07-16T09:45:20.488 INFO:teuthology.orchestra.run.smithi176.stdout:Installing: 2024-07-16T09:45:20.488 INFO:teuthology.orchestra.run.smithi176.stdout: libcephfs-devel x86_64 2:17.2.7-1104.ge6231b04.el9 ceph 25 k 2024-07-16T09:45:20.488 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-07-16T09:45:20.488 INFO:teuthology.orchestra.run.smithi176.stdout:Transaction Summary 2024-07-16T09:45:20.488 INFO:teuthology.orchestra.run.smithi176.stdout:================================================================================ 2024-07-16T09:45:20.488 INFO:teuthology.orchestra.run.smithi176.stdout:Install 1 Package 2024-07-16T09:45:20.489 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-07-16T09:45:20.489 INFO:teuthology.orchestra.run.smithi176.stdout:Total download size: 25 k 2024-07-16T09:45:20.489 INFO:teuthology.orchestra.run.smithi176.stdout:Installed size: 110 k 2024-07-16T09:45:20.489 INFO:teuthology.orchestra.run.smithi176.stdout:Downloading Packages: 2024-07-16T09:45:20.646 INFO:teuthology.orchestra.run.smithi176.stdout:libcephfs-devel-17.2.7-1104.ge6231b04.el9.x86_6 159 kB/s | 25 kB 00:00 2024-07-16T09:45:20.647 INFO:teuthology.orchestra.run.smithi176.stdout:-------------------------------------------------------------------------------- 2024-07-16T09:45:20.647 INFO:teuthology.orchestra.run.smithi176.stdout:Total 157 kB/s | 25 kB 00:00 2024-07-16T09:45:20.647 INFO:teuthology.orchestra.run.smithi176.stdout:Running transaction check 2024-07-16T09:45:20.650 INFO:teuthology.orchestra.run.smithi176.stdout:Transaction check succeeded. 2024-07-16T09:45:20.650 INFO:teuthology.orchestra.run.smithi176.stdout:Running transaction test 2024-07-16T09:45:20.652 INFO:teuthology.orchestra.run.smithi176.stdout:Transaction test succeeded. 2024-07-16T09:45:20.652 INFO:teuthology.orchestra.run.smithi176.stdout:Running transaction 2024-07-16T09:45:20.726 INFO:teuthology.orchestra.run.smithi176.stdout: Preparing : 1/1 2024-07-16T09:45:20.826 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : libcephfs-devel-2:17.2.7-1104.ge6231b04.el9.x86_64 1/1 2024-07-16T09:45:21.041 INFO:teuthology.orchestra.run.smithi156.stdout:Last metadata expiration check: 0:01:41 ago on Tue 16 Jul 2024 09:43:40 AM UTC. 2024-07-16T09:45:21.127 INFO:teuthology.orchestra.run.smithi156.stdout:Package python3-rgw-2:17.2.7-1104.ge6231b04.el9.x86_64 is already installed. 2024-07-16T09:45:21.169 INFO:teuthology.orchestra.run.smithi156.stdout:Dependencies resolved. 2024-07-16T09:45:21.170 INFO:teuthology.orchestra.run.smithi156.stdout:Nothing to do. 2024-07-16T09:45:21.170 INFO:teuthology.orchestra.run.smithi156.stdout:Complete! 2024-07-16T09:45:21.214 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: libcephfs-devel-2:17.2.7-1104.ge6231b04.el9.x86_64 1/1 2024-07-16T09:45:21.240 DEBUG:teuthology.orchestra.run.smithi156:> sudo yum -y install python3-cephfs 2024-07-16T09:45:21.427 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : libcephfs-devel-2:17.2.7-1104.ge6231b04.el9.x86_64 1/1 2024-07-16T09:45:21.427 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-07-16T09:45:21.427 INFO:teuthology.orchestra.run.smithi176.stdout:Installed: 2024-07-16T09:45:21.427 INFO:teuthology.orchestra.run.smithi176.stdout: libcephfs-devel-2:17.2.7-1104.ge6231b04.el9.x86_64 2024-07-16T09:45:21.427 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-07-16T09:45:21.427 INFO:teuthology.orchestra.run.smithi176.stdout:Complete! 2024-07-16T09:45:21.617 DEBUG:teuthology.orchestra.run.smithi176:> sudo yum -y install librados2 2024-07-16T09:45:21.818 INFO:teuthology.orchestra.run.smithi156.stdout:Last metadata expiration check: 0:01:41 ago on Tue 16 Jul 2024 09:43:40 AM UTC. 2024-07-16T09:45:21.904 INFO:teuthology.orchestra.run.smithi156.stdout:Package python3-cephfs-2:17.2.7-1104.ge6231b04.el9.x86_64 is already installed. 2024-07-16T09:45:21.947 INFO:teuthology.orchestra.run.smithi156.stdout:Dependencies resolved. 2024-07-16T09:45:21.948 INFO:teuthology.orchestra.run.smithi156.stdout:Nothing to do. 2024-07-16T09:45:21.948 INFO:teuthology.orchestra.run.smithi156.stdout:Complete! 2024-07-16T09:45:22.014 DEBUG:teuthology.orchestra.run.smithi156:> sudo yum -y install python3-rbd 2024-07-16T09:45:22.194 INFO:teuthology.orchestra.run.smithi176.stdout:Last metadata expiration check: 0:01:37 ago on Tue 16 Jul 2024 09:43:45 AM UTC. 2024-07-16T09:45:22.279 INFO:teuthology.orchestra.run.smithi176.stdout:Package librados2-2:17.2.7-1104.ge6231b04.el9.x86_64 is already installed. 2024-07-16T09:45:22.320 INFO:teuthology.orchestra.run.smithi176.stdout:Dependencies resolved. 2024-07-16T09:45:22.321 INFO:teuthology.orchestra.run.smithi176.stdout:Nothing to do. 2024-07-16T09:45:22.321 INFO:teuthology.orchestra.run.smithi176.stdout:Complete! 2024-07-16T09:45:22.388 DEBUG:teuthology.orchestra.run.smithi176:> sudo yum -y install librbd1 2024-07-16T09:45:22.597 INFO:teuthology.orchestra.run.smithi156.stdout:Last metadata expiration check: 0:01:42 ago on Tue 16 Jul 2024 09:43:40 AM UTC. 2024-07-16T09:45:22.685 INFO:teuthology.orchestra.run.smithi156.stdout:Package python3-rbd-2:17.2.7-1104.ge6231b04.el9.x86_64 is already installed. 2024-07-16T09:45:22.728 INFO:teuthology.orchestra.run.smithi156.stdout:Dependencies resolved. 2024-07-16T09:45:22.729 INFO:teuthology.orchestra.run.smithi156.stdout:Nothing to do. 2024-07-16T09:45:22.729 INFO:teuthology.orchestra.run.smithi156.stdout:Complete! 2024-07-16T09:45:22.802 DEBUG:teuthology.orchestra.run.smithi156:> sudo yum -y install rbd-fuse 2024-07-16T09:45:22.969 INFO:teuthology.orchestra.run.smithi176.stdout:Last metadata expiration check: 0:01:37 ago on Tue 16 Jul 2024 09:43:45 AM UTC. 2024-07-16T09:45:23.056 INFO:teuthology.orchestra.run.smithi176.stdout:Package librbd1-2:17.2.7-1104.ge6231b04.el9.x86_64 is already installed. 2024-07-16T09:45:23.098 INFO:teuthology.orchestra.run.smithi176.stdout:Dependencies resolved. 2024-07-16T09:45:23.099 INFO:teuthology.orchestra.run.smithi176.stdout:Nothing to do. 2024-07-16T09:45:23.099 INFO:teuthology.orchestra.run.smithi176.stdout:Complete! 2024-07-16T09:45:23.173 DEBUG:teuthology.orchestra.run.smithi176:> sudo yum -y install python3-rados 2024-07-16T09:45:23.378 INFO:teuthology.orchestra.run.smithi156.stdout:Last metadata expiration check: 0:01:43 ago on Tue 16 Jul 2024 09:43:40 AM UTC. 2024-07-16T09:45:23.503 INFO:teuthology.orchestra.run.smithi156.stdout:Dependencies resolved. 2024-07-16T09:45:23.504 INFO:teuthology.orchestra.run.smithi156.stdout:================================================================================ 2024-07-16T09:45:23.504 INFO:teuthology.orchestra.run.smithi156.stdout: Package Architecture Version Repository Size 2024-07-16T09:45:23.504 INFO:teuthology.orchestra.run.smithi156.stdout:================================================================================ 2024-07-16T09:45:23.504 INFO:teuthology.orchestra.run.smithi156.stdout:Installing: 2024-07-16T09:45:23.505 INFO:teuthology.orchestra.run.smithi156.stdout: rbd-fuse x86_64 2:17.2.7-1104.ge6231b04.el9 ceph 82 k 2024-07-16T09:45:23.505 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-07-16T09:45:23.505 INFO:teuthology.orchestra.run.smithi156.stdout:Transaction Summary 2024-07-16T09:45:23.505 INFO:teuthology.orchestra.run.smithi156.stdout:================================================================================ 2024-07-16T09:45:23.505 INFO:teuthology.orchestra.run.smithi156.stdout:Install 1 Package 2024-07-16T09:45:23.505 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-07-16T09:45:23.505 INFO:teuthology.orchestra.run.smithi156.stdout:Total download size: 82 k 2024-07-16T09:45:23.505 INFO:teuthology.orchestra.run.smithi156.stdout:Installed size: 222 k 2024-07-16T09:45:23.505 INFO:teuthology.orchestra.run.smithi156.stdout:Downloading Packages: 2024-07-16T09:45:23.732 INFO:teuthology.orchestra.run.smithi156.stdout:rbd-fuse-17.2.7-1104.ge6231b04.el9.x86_64.rpm 361 kB/s | 82 kB 00:00 2024-07-16T09:45:23.732 INFO:teuthology.orchestra.run.smithi156.stdout:-------------------------------------------------------------------------------- 2024-07-16T09:45:23.732 INFO:teuthology.orchestra.run.smithi156.stdout:Total 359 kB/s | 82 kB 00:00 2024-07-16T09:45:23.733 INFO:teuthology.orchestra.run.smithi156.stdout:Running transaction check 2024-07-16T09:45:23.739 INFO:teuthology.orchestra.run.smithi156.stdout:Transaction check succeeded. 2024-07-16T09:45:23.739 INFO:teuthology.orchestra.run.smithi156.stdout:Running transaction test 2024-07-16T09:45:23.757 INFO:teuthology.orchestra.run.smithi176.stdout:Last metadata expiration check: 0:01:38 ago on Tue 16 Jul 2024 09:43:45 AM UTC. 2024-07-16T09:45:23.801 INFO:teuthology.orchestra.run.smithi156.stdout:Transaction test succeeded. 2024-07-16T09:45:23.801 INFO:teuthology.orchestra.run.smithi156.stdout:Running transaction 2024-07-16T09:45:23.844 INFO:teuthology.orchestra.run.smithi176.stdout:Package python3-rados-2:17.2.7-1104.ge6231b04.el9.x86_64 is already installed. 2024-07-16T09:45:23.886 INFO:teuthology.orchestra.run.smithi176.stdout:Dependencies resolved. 2024-07-16T09:45:23.888 INFO:teuthology.orchestra.run.smithi176.stdout:Nothing to do. 2024-07-16T09:45:23.888 INFO:teuthology.orchestra.run.smithi176.stdout:Complete! 2024-07-16T09:45:23.950 INFO:teuthology.orchestra.run.smithi156.stdout: Preparing : 1/1 2024-07-16T09:45:23.953 DEBUG:teuthology.orchestra.run.smithi176:> sudo yum -y install python3-rgw 2024-07-16T09:45:24.049 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : rbd-fuse-2:17.2.7-1104.ge6231b04.el9.x86_64 1/1 2024-07-16T09:45:24.424 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: rbd-fuse-2:17.2.7-1104.ge6231b04.el9.x86_64 1/1 2024-07-16T09:45:24.538 INFO:teuthology.orchestra.run.smithi176.stdout:Last metadata expiration check: 0:01:39 ago on Tue 16 Jul 2024 09:43:45 AM UTC. 2024-07-16T09:45:24.625 INFO:teuthology.orchestra.run.smithi176.stdout:Package python3-rgw-2:17.2.7-1104.ge6231b04.el9.x86_64 is already installed. 2024-07-16T09:45:24.668 INFO:teuthology.orchestra.run.smithi176.stdout:Dependencies resolved. 2024-07-16T09:45:24.669 INFO:teuthology.orchestra.run.smithi176.stdout:Nothing to do. 2024-07-16T09:45:24.669 INFO:teuthology.orchestra.run.smithi176.stdout:Complete! 2024-07-16T09:45:24.689 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : rbd-fuse-2:17.2.7-1104.ge6231b04.el9.x86_64 1/1 2024-07-16T09:45:24.689 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-07-16T09:45:24.689 INFO:teuthology.orchestra.run.smithi156.stdout:Installed: 2024-07-16T09:45:24.690 INFO:teuthology.orchestra.run.smithi156.stdout: rbd-fuse-2:17.2.7-1104.ge6231b04.el9.x86_64 2024-07-16T09:45:24.690 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-07-16T09:45:24.690 INFO:teuthology.orchestra.run.smithi156.stdout:Complete! 2024-07-16T09:45:24.739 DEBUG:teuthology.orchestra.run.smithi176:> sudo yum -y install python3-cephfs 2024-07-16T09:45:24.955 DEBUG:teuthology.orchestra.run.smithi156:> sudo yum -y install rbd-mirror 2024-07-16T09:45:25.313 INFO:teuthology.orchestra.run.smithi176.stdout:Last metadata expiration check: 0:01:40 ago on Tue 16 Jul 2024 09:43:45 AM UTC. 2024-07-16T09:45:25.405 INFO:teuthology.orchestra.run.smithi176.stdout:Package python3-cephfs-2:17.2.7-1104.ge6231b04.el9.x86_64 is already installed. 2024-07-16T09:45:25.447 INFO:teuthology.orchestra.run.smithi176.stdout:Dependencies resolved. 2024-07-16T09:45:25.448 INFO:teuthology.orchestra.run.smithi176.stdout:Nothing to do. 2024-07-16T09:45:25.448 INFO:teuthology.orchestra.run.smithi176.stdout:Complete! 2024-07-16T09:45:25.518 DEBUG:teuthology.orchestra.run.smithi176:> sudo yum -y install python3-rbd 2024-07-16T09:45:25.555 INFO:teuthology.orchestra.run.smithi156.stdout:Last metadata expiration check: 0:01:45 ago on Tue 16 Jul 2024 09:43:40 AM UTC. 2024-07-16T09:45:25.686 INFO:teuthology.orchestra.run.smithi156.stdout:Dependencies resolved. 2024-07-16T09:45:25.687 INFO:teuthology.orchestra.run.smithi156.stdout:================================================================================ 2024-07-16T09:45:25.687 INFO:teuthology.orchestra.run.smithi156.stdout: Package Arch Version Repo Size 2024-07-16T09:45:25.688 INFO:teuthology.orchestra.run.smithi156.stdout:================================================================================ 2024-07-16T09:45:25.688 INFO:teuthology.orchestra.run.smithi156.stdout:Installing: 2024-07-16T09:45:25.688 INFO:teuthology.orchestra.run.smithi156.stdout: rbd-mirror x86_64 2:17.2.7-1104.ge6231b04.el9 ceph 2.9 M 2024-07-16T09:45:25.688 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-07-16T09:45:25.688 INFO:teuthology.orchestra.run.smithi156.stdout:Transaction Summary 2024-07-16T09:45:25.688 INFO:teuthology.orchestra.run.smithi156.stdout:================================================================================ 2024-07-16T09:45:25.688 INFO:teuthology.orchestra.run.smithi156.stdout:Install 1 Package 2024-07-16T09:45:25.688 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-07-16T09:45:25.688 INFO:teuthology.orchestra.run.smithi156.stdout:Total download size: 2.9 M 2024-07-16T09:45:25.688 INFO:teuthology.orchestra.run.smithi156.stdout:Installed size: 12 M 2024-07-16T09:45:25.688 INFO:teuthology.orchestra.run.smithi156.stdout:Downloading Packages: 2024-07-16T09:45:26.104 INFO:teuthology.orchestra.run.smithi176.stdout:Last metadata expiration check: 0:01:41 ago on Tue 16 Jul 2024 09:43:45 AM UTC. 2024-07-16T09:45:26.191 INFO:teuthology.orchestra.run.smithi176.stdout:Package python3-rbd-2:17.2.7-1104.ge6231b04.el9.x86_64 is already installed. 2024-07-16T09:45:26.234 INFO:teuthology.orchestra.run.smithi176.stdout:Dependencies resolved. 2024-07-16T09:45:26.235 INFO:teuthology.orchestra.run.smithi176.stdout:Nothing to do. 2024-07-16T09:45:26.235 INFO:teuthology.orchestra.run.smithi176.stdout:Complete! 2024-07-16T09:45:26.303 DEBUG:teuthology.orchestra.run.smithi176:> sudo yum -y install rbd-fuse 2024-07-16T09:45:26.692 INFO:teuthology.orchestra.run.smithi156.stdout:rbd-mirror-17.2.7-1104.ge6231b04.el9.x86_64.rpm 2.9 MB/s | 2.9 MB 00:01 2024-07-16T09:45:26.693 INFO:teuthology.orchestra.run.smithi156.stdout:-------------------------------------------------------------------------------- 2024-07-16T09:45:26.693 INFO:teuthology.orchestra.run.smithi156.stdout:Total 2.9 MB/s | 2.9 MB 00:01 2024-07-16T09:45:26.694 INFO:teuthology.orchestra.run.smithi156.stdout:Running transaction check 2024-07-16T09:45:26.703 INFO:teuthology.orchestra.run.smithi156.stdout:Transaction check succeeded. 2024-07-16T09:45:26.703 INFO:teuthology.orchestra.run.smithi156.stdout:Running transaction test 2024-07-16T09:45:26.784 INFO:teuthology.orchestra.run.smithi156.stdout:Transaction test succeeded. 2024-07-16T09:45:26.784 INFO:teuthology.orchestra.run.smithi156.stdout:Running transaction 2024-07-16T09:45:26.886 INFO:teuthology.orchestra.run.smithi176.stdout:Last metadata expiration check: 0:01:41 ago on Tue 16 Jul 2024 09:43:45 AM UTC. 2024-07-16T09:45:27.012 INFO:teuthology.orchestra.run.smithi176.stdout:Dependencies resolved. 2024-07-16T09:45:27.013 INFO:teuthology.orchestra.run.smithi176.stdout:================================================================================ 2024-07-16T09:45:27.013 INFO:teuthology.orchestra.run.smithi176.stdout: Package Architecture Version Repository Size 2024-07-16T09:45:27.013 INFO:teuthology.orchestra.run.smithi176.stdout:================================================================================ 2024-07-16T09:45:27.013 INFO:teuthology.orchestra.run.smithi176.stdout:Installing: 2024-07-16T09:45:27.013 INFO:teuthology.orchestra.run.smithi176.stdout: rbd-fuse x86_64 2:17.2.7-1104.ge6231b04.el9 ceph 82 k 2024-07-16T09:45:27.013 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-07-16T09:45:27.013 INFO:teuthology.orchestra.run.smithi176.stdout:Transaction Summary 2024-07-16T09:45:27.014 INFO:teuthology.orchestra.run.smithi176.stdout:================================================================================ 2024-07-16T09:45:27.014 INFO:teuthology.orchestra.run.smithi176.stdout:Install 1 Package 2024-07-16T09:45:27.014 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-07-16T09:45:27.014 INFO:teuthology.orchestra.run.smithi176.stdout:Total download size: 82 k 2024-07-16T09:45:27.014 INFO:teuthology.orchestra.run.smithi176.stdout:Installed size: 222 k 2024-07-16T09:45:27.014 INFO:teuthology.orchestra.run.smithi176.stdout:Downloading Packages: 2024-07-16T09:45:27.194 INFO:teuthology.orchestra.run.smithi156.stdout: Preparing : 1/1 2024-07-16T09:45:27.252 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : rbd-mirror-2:17.2.7-1104.ge6231b04.el9.x86_64 1/1 2024-07-16T09:45:27.276 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: rbd-mirror-2:17.2.7-1104.ge6231b04.el9.x86_64 1/1 2024-07-16T09:45:27.277 INFO:teuthology.orchestra.run.smithi156.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-07-16T09:45:27.277 INFO:teuthology.orchestra.run.smithi156.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-07-16T09:45:27.277 INFO:teuthology.orchestra.run.smithi156.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-07-16T09:45:27.277 INFO:teuthology.orchestra.run.smithi156.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-07-16T09:45:27.277 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-07-16T09:45:27.474 INFO:teuthology.orchestra.run.smithi176.stdout:rbd-fuse-17.2.7-1104.ge6231b04.el9.x86_64.rpm 178 kB/s | 82 kB 00:00 2024-07-16T09:45:27.474 INFO:teuthology.orchestra.run.smithi176.stdout:-------------------------------------------------------------------------------- 2024-07-16T09:45:27.474 INFO:teuthology.orchestra.run.smithi176.stdout:Total 178 kB/s | 82 kB 00:00 2024-07-16T09:45:27.475 INFO:teuthology.orchestra.run.smithi176.stdout:Running transaction check 2024-07-16T09:45:27.481 INFO:teuthology.orchestra.run.smithi176.stdout:Transaction check succeeded. 2024-07-16T09:45:27.481 INFO:teuthology.orchestra.run.smithi176.stdout:Running transaction test 2024-07-16T09:45:27.546 INFO:teuthology.orchestra.run.smithi176.stdout:Transaction test succeeded. 2024-07-16T09:45:27.546 INFO:teuthology.orchestra.run.smithi176.stdout:Running transaction 2024-07-16T09:45:27.689 INFO:teuthology.orchestra.run.smithi176.stdout: Preparing : 1/1 2024-07-16T09:45:27.791 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : rbd-fuse-2:17.2.7-1104.ge6231b04.el9.x86_64 1/1 2024-07-16T09:45:28.161 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: rbd-fuse-2:17.2.7-1104.ge6231b04.el9.x86_64 1/1 2024-07-16T09:45:28.407 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : rbd-mirror-2:17.2.7-1104.ge6231b04.el9.x86_64 1/1 2024-07-16T09:45:28.407 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-07-16T09:45:28.407 INFO:teuthology.orchestra.run.smithi156.stdout:Installed: 2024-07-16T09:45:28.407 INFO:teuthology.orchestra.run.smithi156.stdout: rbd-mirror-2:17.2.7-1104.ge6231b04.el9.x86_64 2024-07-16T09:45:28.407 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-07-16T09:45:28.407 INFO:teuthology.orchestra.run.smithi156.stdout:Complete! 2024-07-16T09:45:28.424 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : rbd-fuse-2:17.2.7-1104.ge6231b04.el9.x86_64 1/1 2024-07-16T09:45:28.424 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-07-16T09:45:28.424 INFO:teuthology.orchestra.run.smithi176.stdout:Installed: 2024-07-16T09:45:28.424 INFO:teuthology.orchestra.run.smithi176.stdout: rbd-fuse-2:17.2.7-1104.ge6231b04.el9.x86_64 2024-07-16T09:45:28.424 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-07-16T09:45:28.424 INFO:teuthology.orchestra.run.smithi176.stdout:Complete! 2024-07-16T09:45:28.641 DEBUG:teuthology.orchestra.run.smithi156:> sudo yum -y install rbd-nbd 2024-07-16T09:45:28.653 DEBUG:teuthology.orchestra.run.smithi176:> sudo yum -y install rbd-mirror 2024-07-16T09:45:29.238 INFO:teuthology.orchestra.run.smithi156.stdout:Last metadata expiration check: 0:01:49 ago on Tue 16 Jul 2024 09:43:40 AM UTC. 2024-07-16T09:45:29.256 INFO:teuthology.orchestra.run.smithi176.stdout:Last metadata expiration check: 0:01:44 ago on Tue 16 Jul 2024 09:43:45 AM UTC. 2024-07-16T09:45:29.375 INFO:teuthology.orchestra.run.smithi156.stdout:Dependencies resolved. 2024-07-16T09:45:29.376 INFO:teuthology.orchestra.run.smithi156.stdout:================================================================================ 2024-07-16T09:45:29.376 INFO:teuthology.orchestra.run.smithi156.stdout: Package Architecture Version Repository Size 2024-07-16T09:45:29.376 INFO:teuthology.orchestra.run.smithi156.stdout:================================================================================ 2024-07-16T09:45:29.376 INFO:teuthology.orchestra.run.smithi156.stdout:Installing: 2024-07-16T09:45:29.376 INFO:teuthology.orchestra.run.smithi156.stdout: rbd-nbd x86_64 2:17.2.7-1104.ge6231b04.el9 ceph 165 k 2024-07-16T09:45:29.376 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-07-16T09:45:29.376 INFO:teuthology.orchestra.run.smithi156.stdout:Transaction Summary 2024-07-16T09:45:29.376 INFO:teuthology.orchestra.run.smithi156.stdout:================================================================================ 2024-07-16T09:45:29.376 INFO:teuthology.orchestra.run.smithi156.stdout:Install 1 Package 2024-07-16T09:45:29.377 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-07-16T09:45:29.377 INFO:teuthology.orchestra.run.smithi156.stdout:Total download size: 165 k 2024-07-16T09:45:29.377 INFO:teuthology.orchestra.run.smithi156.stdout:Installed size: 477 k 2024-07-16T09:45:29.377 INFO:teuthology.orchestra.run.smithi156.stdout:Downloading Packages: 2024-07-16T09:45:29.395 INFO:teuthology.orchestra.run.smithi176.stdout:Dependencies resolved. 2024-07-16T09:45:29.395 INFO:teuthology.orchestra.run.smithi176.stdout:================================================================================ 2024-07-16T09:45:29.395 INFO:teuthology.orchestra.run.smithi176.stdout: Package Arch Version Repo Size 2024-07-16T09:45:29.395 INFO:teuthology.orchestra.run.smithi176.stdout:================================================================================ 2024-07-16T09:45:29.396 INFO:teuthology.orchestra.run.smithi176.stdout:Installing: 2024-07-16T09:45:29.396 INFO:teuthology.orchestra.run.smithi176.stdout: rbd-mirror x86_64 2:17.2.7-1104.ge6231b04.el9 ceph 2.9 M 2024-07-16T09:45:29.396 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-07-16T09:45:29.396 INFO:teuthology.orchestra.run.smithi176.stdout:Transaction Summary 2024-07-16T09:45:29.396 INFO:teuthology.orchestra.run.smithi176.stdout:================================================================================ 2024-07-16T09:45:29.396 INFO:teuthology.orchestra.run.smithi176.stdout:Install 1 Package 2024-07-16T09:45:29.396 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-07-16T09:45:29.397 INFO:teuthology.orchestra.run.smithi176.stdout:Total download size: 2.9 M 2024-07-16T09:45:29.397 INFO:teuthology.orchestra.run.smithi176.stdout:Installed size: 12 M 2024-07-16T09:45:29.397 INFO:teuthology.orchestra.run.smithi176.stdout:Downloading Packages: 2024-07-16T09:45:29.819 INFO:teuthology.orchestra.run.smithi156.stdout:rbd-nbd-17.2.7-1104.ge6231b04.el9.x86_64.rpm 374 kB/s | 165 kB 00:00 2024-07-16T09:45:29.819 INFO:teuthology.orchestra.run.smithi156.stdout:-------------------------------------------------------------------------------- 2024-07-16T09:45:29.820 INFO:teuthology.orchestra.run.smithi156.stdout:Total 372 kB/s | 165 kB 00:00 2024-07-16T09:45:29.820 INFO:teuthology.orchestra.run.smithi156.stdout:Running transaction check 2024-07-16T09:45:29.828 INFO:teuthology.orchestra.run.smithi156.stdout:Transaction check succeeded. 2024-07-16T09:45:29.828 INFO:teuthology.orchestra.run.smithi156.stdout:Running transaction test 2024-07-16T09:45:29.895 INFO:teuthology.orchestra.run.smithi156.stdout:Transaction test succeeded. 2024-07-16T09:45:29.895 INFO:teuthology.orchestra.run.smithi156.stdout:Running transaction 2024-07-16T09:45:30.062 INFO:teuthology.orchestra.run.smithi156.stdout: Preparing : 1/1 2024-07-16T09:45:30.151 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : rbd-nbd-2:17.2.7-1104.ge6231b04.el9.x86_64 1/1 2024-07-16T09:45:30.375 INFO:teuthology.orchestra.run.smithi176.stdout:rbd-mirror-17.2.7-1104.ge6231b04.el9.x86_64.rpm 3.0 MB/s | 2.9 MB 00:00 2024-07-16T09:45:30.376 INFO:teuthology.orchestra.run.smithi176.stdout:-------------------------------------------------------------------------------- 2024-07-16T09:45:30.376 INFO:teuthology.orchestra.run.smithi176.stdout:Total 3.0 MB/s | 2.9 MB 00:00 2024-07-16T09:45:30.376 INFO:teuthology.orchestra.run.smithi176.stdout:Running transaction check 2024-07-16T09:45:30.385 INFO:teuthology.orchestra.run.smithi176.stdout:Transaction check succeeded. 2024-07-16T09:45:30.385 INFO:teuthology.orchestra.run.smithi176.stdout:Running transaction test 2024-07-16T09:45:30.464 INFO:teuthology.orchestra.run.smithi176.stdout:Transaction test succeeded. 2024-07-16T09:45:30.464 INFO:teuthology.orchestra.run.smithi176.stdout:Running transaction 2024-07-16T09:45:30.571 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: rbd-nbd-2:17.2.7-1104.ge6231b04.el9.x86_64 1/1 2024-07-16T09:45:30.833 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : rbd-nbd-2:17.2.7-1104.ge6231b04.el9.x86_64 1/1 2024-07-16T09:45:30.833 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-07-16T09:45:30.833 INFO:teuthology.orchestra.run.smithi156.stdout:Installed: 2024-07-16T09:45:30.833 INFO:teuthology.orchestra.run.smithi156.stdout: rbd-nbd-2:17.2.7-1104.ge6231b04.el9.x86_64 2024-07-16T09:45:30.833 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-07-16T09:45:30.833 INFO:teuthology.orchestra.run.smithi156.stdout:Complete! 2024-07-16T09:45:30.872 INFO:teuthology.orchestra.run.smithi176.stdout: Preparing : 1/1 2024-07-16T09:45:30.942 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : rbd-mirror-2:17.2.7-1104.ge6231b04.el9.x86_64 1/1 2024-07-16T09:45:30.968 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: rbd-mirror-2:17.2.7-1104.ge6231b04.el9.x86_64 1/1 2024-07-16T09:45:30.968 INFO:teuthology.orchestra.run.smithi176.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-07-16T09:45:30.968 INFO:teuthology.orchestra.run.smithi176.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-07-16T09:45:30.968 INFO:teuthology.orchestra.run.smithi176.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-07-16T09:45:30.969 INFO:teuthology.orchestra.run.smithi176.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-07-16T09:45:30.969 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-07-16T09:45:31.083 DEBUG:teuthology.parallel:result is None 2024-07-16T09:45:32.008 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : rbd-mirror-2:17.2.7-1104.ge6231b04.el9.x86_64 1/1 2024-07-16T09:45:32.008 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-07-16T09:45:32.008 INFO:teuthology.orchestra.run.smithi176.stdout:Installed: 2024-07-16T09:45:32.009 INFO:teuthology.orchestra.run.smithi176.stdout: rbd-mirror-2:17.2.7-1104.ge6231b04.el9.x86_64 2024-07-16T09:45:32.009 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-07-16T09:45:32.009 INFO:teuthology.orchestra.run.smithi176.stdout:Complete! 2024-07-16T09:45:32.238 DEBUG:teuthology.orchestra.run.smithi176:> sudo yum -y install rbd-nbd 2024-07-16T09:45:32.842 INFO:teuthology.orchestra.run.smithi176.stdout:Last metadata expiration check: 0:01:47 ago on Tue 16 Jul 2024 09:43:45 AM UTC. 2024-07-16T09:45:32.971 INFO:teuthology.orchestra.run.smithi176.stdout:Dependencies resolved. 2024-07-16T09:45:32.972 INFO:teuthology.orchestra.run.smithi176.stdout:================================================================================ 2024-07-16T09:45:32.972 INFO:teuthology.orchestra.run.smithi176.stdout: Package Architecture Version Repository Size 2024-07-16T09:45:32.972 INFO:teuthology.orchestra.run.smithi176.stdout:================================================================================ 2024-07-16T09:45:32.972 INFO:teuthology.orchestra.run.smithi176.stdout:Installing: 2024-07-16T09:45:32.972 INFO:teuthology.orchestra.run.smithi176.stdout: rbd-nbd x86_64 2:17.2.7-1104.ge6231b04.el9 ceph 165 k 2024-07-16T09:45:32.972 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-07-16T09:45:32.972 INFO:teuthology.orchestra.run.smithi176.stdout:Transaction Summary 2024-07-16T09:45:32.972 INFO:teuthology.orchestra.run.smithi176.stdout:================================================================================ 2024-07-16T09:45:32.972 INFO:teuthology.orchestra.run.smithi176.stdout:Install 1 Package 2024-07-16T09:45:32.972 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-07-16T09:45:32.973 INFO:teuthology.orchestra.run.smithi176.stdout:Total download size: 165 k 2024-07-16T09:45:32.973 INFO:teuthology.orchestra.run.smithi176.stdout:Installed size: 477 k 2024-07-16T09:45:32.973 INFO:teuthology.orchestra.run.smithi176.stdout:Downloading Packages: 2024-07-16T09:45:33.414 INFO:teuthology.orchestra.run.smithi176.stdout:rbd-nbd-17.2.7-1104.ge6231b04.el9.x86_64.rpm 374 kB/s | 165 kB 00:00 2024-07-16T09:45:33.415 INFO:teuthology.orchestra.run.smithi176.stdout:-------------------------------------------------------------------------------- 2024-07-16T09:45:33.415 INFO:teuthology.orchestra.run.smithi176.stdout:Total 373 kB/s | 165 kB 00:00 2024-07-16T09:45:33.415 INFO:teuthology.orchestra.run.smithi176.stdout:Running transaction check 2024-07-16T09:45:33.423 INFO:teuthology.orchestra.run.smithi176.stdout:Transaction check succeeded. 2024-07-16T09:45:33.423 INFO:teuthology.orchestra.run.smithi176.stdout:Running transaction test 2024-07-16T09:45:33.488 INFO:teuthology.orchestra.run.smithi176.stdout:Transaction test succeeded. 2024-07-16T09:45:33.488 INFO:teuthology.orchestra.run.smithi176.stdout:Running transaction 2024-07-16T09:45:33.648 INFO:teuthology.orchestra.run.smithi176.stdout: Preparing : 1/1 2024-07-16T09:45:33.754 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : rbd-nbd-2:17.2.7-1104.ge6231b04.el9.x86_64 1/1 2024-07-16T09:45:34.174 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: rbd-nbd-2:17.2.7-1104.ge6231b04.el9.x86_64 1/1 2024-07-16T09:45:34.424 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : rbd-nbd-2:17.2.7-1104.ge6231b04.el9.x86_64 1/1 2024-07-16T09:45:34.424 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-07-16T09:45:34.424 INFO:teuthology.orchestra.run.smithi176.stdout:Installed: 2024-07-16T09:45:34.425 INFO:teuthology.orchestra.run.smithi176.stdout: rbd-nbd-2:17.2.7-1104.ge6231b04.el9.x86_64 2024-07-16T09:45:34.425 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-07-16T09:45:34.425 INFO:teuthology.orchestra.run.smithi176.stdout:Complete! 2024-07-16T09:45:34.659 DEBUG:teuthology.parallel:result is None 2024-07-16T09:45:34.660 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-07-16T09:45:34.660 INFO:teuthology.packaging:ref: None 2024-07-16T09:45:34.660 INFO:teuthology.packaging:tag: None 2024-07-16T09:45:34.660 INFO:teuthology.packaging:branch: quincy 2024-07-16T09:45:34.660 INFO:teuthology.packaging:sha1: a6ca461be7cc1aff3658c2c6df4cff975a24449a 2024-07-16T09:45:34.660 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=quincy 2024-07-16T09:45:34.868 DEBUG:teuthology.orchestra.run.smithi156:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-07-16T09:45:34.894 INFO:teuthology.orchestra.run.smithi156.stdout:17.2.7-1104.ge6231b04.el9 2024-07-16T09:45:34.895 INFO:teuthology.packaging:The installed version of ceph is 17.2.7-1104.ge6231b04.el9 2024-07-16T09:45:34.895 INFO:teuthology.task.install:The correct ceph version 17.2.7-1104.ge6231b04 is installed. 2024-07-16T09:45:34.897 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-07-16T09:45:34.897 INFO:teuthology.packaging:ref: None 2024-07-16T09:45:34.897 INFO:teuthology.packaging:tag: None 2024-07-16T09:45:34.897 INFO:teuthology.packaging:branch: quincy 2024-07-16T09:45:34.897 INFO:teuthology.packaging:sha1: a6ca461be7cc1aff3658c2c6df4cff975a24449a 2024-07-16T09:45:34.897 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=quincy 2024-07-16T09:45:35.106 DEBUG:teuthology.orchestra.run.smithi176:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-07-16T09:45:35.131 INFO:teuthology.orchestra.run.smithi176.stdout:17.2.7-1104.ge6231b04.el9 2024-07-16T09:45:35.131 INFO:teuthology.packaging:The installed version of ceph is 17.2.7-1104.ge6231b04.el9 2024-07-16T09:45:35.131 INFO:teuthology.task.install:The correct ceph version 17.2.7-1104.ge6231b04 is installed. 2024-07-16T09:45:35.133 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-07-16T09:45:35.133 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2024-07-16T09:45:35.133 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-07-16T09:45:35.165 DEBUG:teuthology.orchestra.run.smithi176:> set -ex 2024-07-16T09:45:35.165 DEBUG:teuthology.orchestra.run.smithi176:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-07-16T09:45:35.199 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-07-16T09:45:35.200 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2024-07-16T09:45:35.200 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd of=/usr/bin/daemon-helper 2024-07-16T09:45:35.234 DEBUG:teuthology.orchestra.run.smithi156:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-07-16T09:45:35.301 DEBUG:teuthology.orchestra.run.smithi176:> set -ex 2024-07-16T09:45:35.301 DEBUG:teuthology.orchestra.run.smithi176:> sudo dd of=/usr/bin/daemon-helper 2024-07-16T09:45:35.328 DEBUG:teuthology.orchestra.run.smithi176:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-07-16T09:45:35.395 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-07-16T09:45:35.395 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2024-07-16T09:45:35.396 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd of=/usr/bin/adjust-ulimits 2024-07-16T09:45:35.423 DEBUG:teuthology.orchestra.run.smithi156:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-07-16T09:45:35.489 DEBUG:teuthology.orchestra.run.smithi176:> set -ex 2024-07-16T09:45:35.490 DEBUG:teuthology.orchestra.run.smithi176:> sudo dd of=/usr/bin/adjust-ulimits 2024-07-16T09:45:35.516 DEBUG:teuthology.orchestra.run.smithi176:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-07-16T09:45:35.582 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-07-16T09:45:35.582 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2024-07-16T09:45:35.582 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd of=/usr/bin/stdin-killer 2024-07-16T09:45:35.608 DEBUG:teuthology.orchestra.run.smithi156:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-07-16T09:45:35.675 DEBUG:teuthology.orchestra.run.smithi176:> set -ex 2024-07-16T09:45:35.675 DEBUG:teuthology.orchestra.run.smithi176:> sudo dd of=/usr/bin/stdin-killer 2024-07-16T09:45:35.701 DEBUG:teuthology.orchestra.run.smithi176:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-07-16T09:45:35.766 INFO:teuthology.run_tasks:Running task print... 2024-07-16T09:45:35.775 INFO:teuthology.task.print:**** done install task... 2024-07-16T09:45:35.776 INFO:teuthology.run_tasks:Running task cephadm... 2024-07-16T09:45:35.866 INFO:tasks.cephadm:Config: {'cephadm_branch': 'quincy', 'cephadm_git_url': 'https://github.com/ceph/ceph', 'conf': {'osd': {'osd_class_default_list': '*', 'osd_class_load_list': '*', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'mon pg warn min per osd': 0}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}}, 'image': 'quay.ceph.io/ceph-ci/ceph:quincy', 'roleless': True, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'OSD_DOWN', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy'], 'sha1': 'a6ca461be7cc1aff3658c2c6df4cff975a24449a'} 2024-07-16T09:45:35.866 INFO:tasks.cephadm:Cluster image is quay.ceph.io/ceph-ci/ceph:quincy 2024-07-16T09:45:35.867 INFO:tasks.cephadm:Cluster fsid is 26c91c26-4358-11ef-bcad-c7b262605968 2024-07-16T09:45:35.867 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-07-16T09:45:35.867 INFO:tasks.cephadm:No mon roles; fabricating mons 2024-07-16T09:45:35.867 INFO:tasks.cephadm:Monitor IPs: {'mon.smithi156': '172.21.15.156', 'mon.smithi176': '172.21.15.176'} 2024-07-16T09:45:35.867 INFO:tasks.cephadm:Normalizing hostnames... 2024-07-16T09:45:35.867 DEBUG:teuthology.orchestra.run.smithi156:> sudo hostname $(hostname -s) 2024-07-16T09:45:35.894 DEBUG:teuthology.orchestra.run.smithi176:> sudo hostname $(hostname -s) 2024-07-16T09:45:35.919 INFO:tasks.cephadm:Downloading cephadm (repo https://github.com/ceph/ceph ref quincy)... 2024-07-16T09:45:35.919 DEBUG:teuthology.orchestra.run.smithi156:> curl --silent https://raw.githubusercontent.com/ceph/ceph/quincy/src/cephadm/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-07-16T09:45:36.181 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r--. 1 ubuntu ubuntu 375153 Jul 16 09:45 /home/ubuntu/cephtest/cephadm 2024-07-16T09:45:36.181 DEBUG:teuthology.orchestra.run.smithi176:> curl --silent https://raw.githubusercontent.com/ceph/ceph/quincy/src/cephadm/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-07-16T09:45:36.437 INFO:teuthology.orchestra.run.smithi176.stdout:-rw-r--r--. 1 ubuntu ubuntu 375153 Jul 16 09:45 /home/ubuntu/cephtest/cephadm 2024-07-16T09:45:36.438 DEBUG:teuthology.orchestra.run.smithi156:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-07-16T09:45:36.454 DEBUG:teuthology.orchestra.run.smithi176:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-07-16T09:45:36.482 INFO:tasks.cephadm:Pulling image quay.ceph.io/ceph-ci/ceph:quincy on all hosts... 2024-07-16T09:45:36.482 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy pull 2024-07-16T09:45:36.497 DEBUG:teuthology.orchestra.run.smithi176:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy pull 2024-07-16T09:45:37.352 INFO:teuthology.orchestra.run.smithi156.stderr:Pulling container image quay.ceph.io/ceph-ci/ceph:quincy... 2024-07-16T09:45:37.412 INFO:teuthology.orchestra.run.smithi176.stderr:Pulling container image quay.ceph.io/ceph-ci/ceph:quincy... 2024-07-16T09:46:09.070 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-07-16T09:46:09.071 INFO:teuthology.orchestra.run.smithi176.stdout: "ceph_version": "ceph version 17.2.7-1104-ge6231b04 (e6231b04b52ebf3b9b31803282dd3074439e51ac) quincy (stable)", 2024-07-16T09:46:09.071 INFO:teuthology.orchestra.run.smithi176.stdout: "image_id": "d5de5f953b95b9d8fc22e2775f009f50c30fbc9c8f71f43c04eb71cd5289ebad", 2024-07-16T09:46:09.071 INFO:teuthology.orchestra.run.smithi176.stdout: "repo_digests": [ 2024-07-16T09:46:09.071 INFO:teuthology.orchestra.run.smithi176.stdout: "quay.ceph.io/ceph-ci/ceph@sha256:7b86ef6bffe0efc5f1aefc9fef8c854a67a511c50993acee3ac684f23bde83df" 2024-07-16T09:46:09.071 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-07-16T09:46:09.071 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-07-16T09:46:14.753 INFO:teuthology.orchestra.run.smithi156.stdout:{ 2024-07-16T09:46:14.753 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph_version": "ceph version 17.2.7-1104-ge6231b04 (e6231b04b52ebf3b9b31803282dd3074439e51ac) quincy (stable)", 2024-07-16T09:46:14.753 INFO:teuthology.orchestra.run.smithi156.stdout: "image_id": "d5de5f953b95b9d8fc22e2775f009f50c30fbc9c8f71f43c04eb71cd5289ebad", 2024-07-16T09:46:14.753 INFO:teuthology.orchestra.run.smithi156.stdout: "repo_digests": [ 2024-07-16T09:46:14.753 INFO:teuthology.orchestra.run.smithi156.stdout: "quay.ceph.io/ceph-ci/ceph@sha256:7b86ef6bffe0efc5f1aefc9fef8c854a67a511c50993acee3ac684f23bde83df" 2024-07-16T09:46:14.753 INFO:teuthology.orchestra.run.smithi156.stdout: ] 2024-07-16T09:46:14.753 INFO:teuthology.orchestra.run.smithi156.stdout:} 2024-07-16T09:46:14.772 DEBUG:teuthology.orchestra.run.smithi156:> sudo mkdir -p /etc/ceph 2024-07-16T09:46:14.826 DEBUG:teuthology.orchestra.run.smithi176:> sudo mkdir -p /etc/ceph 2024-07-16T09:46:14.859 DEBUG:teuthology.orchestra.run.smithi156:> sudo chmod 777 /etc/ceph 2024-07-16T09:46:14.884 DEBUG:teuthology.orchestra.run.smithi176:> sudo chmod 777 /etc/ceph 2024-07-16T09:46:14.927 INFO:tasks.cephadm:Writing seed config... 2024-07-16T09:46:14.928 INFO:tasks.cephadm: override: [osd] osd_class_default_list = * 2024-07-16T09:46:14.928 INFO:tasks.cephadm: override: [osd] osd_class_load_list = * 2024-07-16T09:46:14.928 INFO:tasks.cephadm: override: [osd] bdev async discard = True 2024-07-16T09:46:14.928 INFO:tasks.cephadm: override: [osd] bdev enable discard = True 2024-07-16T09:46:14.928 INFO:tasks.cephadm: override: [osd] bluestore allocator = bitmap 2024-07-16T09:46:14.928 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-07-16T09:46:14.928 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-07-16T09:46:14.929 INFO:tasks.cephadm: override: [osd] debug bluefs = 1/20 2024-07-16T09:46:14.929 INFO:tasks.cephadm: override: [osd] debug bluestore = 1/20 2024-07-16T09:46:14.929 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-07-16T09:46:14.929 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-07-16T09:46:14.929 INFO:tasks.cephadm: override: [osd] debug rocksdb = 4/10 2024-07-16T09:46:14.929 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-07-16T09:46:14.929 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-07-16T09:46:14.929 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-07-16T09:46:14.929 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-07-16T09:46:14.929 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-07-16T09:46:14.929 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-07-16T09:46:14.929 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-07-16T09:46:14.929 INFO:tasks.cephadm: override: [client] debug client = 20 2024-07-16T09:46:14.930 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-07-16T09:46:14.930 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-07-16T09:46:14.930 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-07-16T09:46:14.930 INFO:tasks.cephadm: override: [global] mon pg warn min per osd = 0 2024-07-16T09:46:14.930 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-07-16T09:46:14.930 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-07-16T09:46:14.930 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-07-16T09:46:14.930 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-07-16T09:46:14.930 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-07-16T09:46:14.930 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-07-16T09:46:14.930 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-07-16T09:46:14.930 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-07-16T09:46:14.930 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-07-16T09:46:14.931 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-07-16T09:46:14.931 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-07-16T09:46:14.931 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-07-16T09:46:14.931 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-07-16T09:46:14.931 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-07-16T09:46:14.931 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-07-16T09:46:14.931 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-07-16T09:46:14.931 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-07-16T09:46:14.931 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-07-16T09:46:14.932 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2024-07-16T09:46:14.932 DEBUG:teuthology.orchestra.run.smithi156:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-07-16T09:46:14.948 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 26c91c26-4358-11ef-bcad-c7b262605968 mon pg warn min per osd = 0 [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = true bdev debug aio = true osd sloppy crc = true osd_class_default_list = * osd_class_load_list = * bdev async discard = True bdev enable discard = True bluestore allocator = bitmap bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 1/20 debug bluestore = 1/20 debug ms = 1 debug osd = 20 debug rocksdb = 4/10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-07-16T09:46:14.948 DEBUG:teuthology.orchestra.run.smithi156:mon.smithi156> sudo journalctl -f -n 0 -u ceph-26c91c26-4358-11ef-bcad-c7b262605968@mon.smithi156.service 2024-07-16T09:46:14.991 INFO:tasks.cephadm:Bootstrapping... 2024-07-16T09:46:14.991 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy -v bootstrap --fsid 26c91c26-4358-11ef-bcad-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-ip 172.21.15.156 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-07-16T09:46:15.193 INFO:teuthology.orchestra.run.smithi156.stdout:-------------------------------------------------------------------------------- 2024-07-16T09:46:15.194 INFO:teuthology.orchestra.run.smithi156.stdout:cephadm ['--image', 'quay.ceph.io/ceph-ci/ceph:quincy', '-v', 'bootstrap', '--fsid', '26c91c26-4358-11ef-bcad-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-ip', '172.21.15.156', '--skip-admin-label'] 2024-07-16T09:46:15.219 INFO:teuthology.orchestra.run.smithi156.stdout:/bin/podman: stdout 5.1.1 2024-07-16T09:46:15.220 INFO:teuthology.orchestra.run.smithi156.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-07-16T09:46:15.220 INFO:teuthology.orchestra.run.smithi156.stdout:Verifying podman|docker is present... 2024-07-16T09:46:15.244 INFO:teuthology.orchestra.run.smithi156.stdout:/bin/podman: stdout 5.1.1 2024-07-16T09:46:15.244 INFO:teuthology.orchestra.run.smithi156.stdout:Verifying lvm2 is present... 2024-07-16T09:46:15.244 INFO:teuthology.orchestra.run.smithi156.stdout:Verifying time synchronization is in place... 2024-07-16T09:46:15.252 INFO:teuthology.orchestra.run.smithi156.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-07-16T09:46:15.252 INFO:teuthology.orchestra.run.smithi156.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-07-16T09:46:15.259 INFO:teuthology.orchestra.run.smithi156.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-07-16T09:46:15.259 INFO:teuthology.orchestra.run.smithi156.stdout:systemctl: stdout inactive 2024-07-16T09:46:15.265 INFO:teuthology.orchestra.run.smithi156.stdout:systemctl: stdout enabled 2024-07-16T09:46:15.272 INFO:teuthology.orchestra.run.smithi156.stdout:systemctl: stdout active 2024-07-16T09:46:15.272 INFO:teuthology.orchestra.run.smithi156.stdout:Unit chronyd.service is enabled and running 2024-07-16T09:46:15.272 INFO:teuthology.orchestra.run.smithi156.stdout:Repeating the final host check... 2024-07-16T09:46:15.296 INFO:teuthology.orchestra.run.smithi156.stdout:/bin/podman: stdout 5.1.1 2024-07-16T09:46:15.296 INFO:teuthology.orchestra.run.smithi156.stdout:podman (/bin/podman) version 5.1.1 is present 2024-07-16T09:46:15.297 INFO:teuthology.orchestra.run.smithi156.stdout:systemctl is present 2024-07-16T09:46:15.297 INFO:teuthology.orchestra.run.smithi156.stdout:lvcreate is present 2024-07-16T09:46:15.303 INFO:teuthology.orchestra.run.smithi156.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-07-16T09:46:15.303 INFO:teuthology.orchestra.run.smithi156.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-07-16T09:46:15.309 INFO:teuthology.orchestra.run.smithi156.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-07-16T09:46:15.310 INFO:teuthology.orchestra.run.smithi156.stdout:systemctl: stdout inactive 2024-07-16T09:46:15.316 INFO:teuthology.orchestra.run.smithi156.stdout:systemctl: stdout enabled 2024-07-16T09:46:15.323 INFO:teuthology.orchestra.run.smithi156.stdout:systemctl: stdout active 2024-07-16T09:46:15.323 INFO:teuthology.orchestra.run.smithi156.stdout:Unit chronyd.service is enabled and running 2024-07-16T09:46:15.323 INFO:teuthology.orchestra.run.smithi156.stdout:Host looks OK 2024-07-16T09:46:15.323 INFO:teuthology.orchestra.run.smithi156.stdout:Cluster fsid: 26c91c26-4358-11ef-bcad-c7b262605968 2024-07-16T09:46:15.323 INFO:teuthology.orchestra.run.smithi156.stdout:Acquiring lock 140577419936192 on /run/cephadm/26c91c26-4358-11ef-bcad-c7b262605968.lock 2024-07-16T09:46:15.323 INFO:teuthology.orchestra.run.smithi156.stdout:Lock 140577419936192 acquired on /run/cephadm/26c91c26-4358-11ef-bcad-c7b262605968.lock 2024-07-16T09:46:15.324 INFO:teuthology.orchestra.run.smithi156.stdout:Verifying IP 172.21.15.156 port 3300 ... 2024-07-16T09:46:15.324 INFO:teuthology.orchestra.run.smithi156.stdout:Verifying IP 172.21.15.156 port 6789 ... 2024-07-16T09:46:15.324 INFO:teuthology.orchestra.run.smithi156.stdout:Base mon IP(s) is [172.21.15.156:3300, 172.21.15.156:6789], mon addrv is [v2:172.21.15.156:3300,v1:172.21.15.156:6789] 2024-07-16T09:46:15.327 INFO:teuthology.orchestra.run.smithi156.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp src 172.21.15.156 metric 100 2024-07-16T09:46:15.327 INFO:teuthology.orchestra.run.smithi156.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.156 metric 100 2024-07-16T09:46:15.330 INFO:teuthology.orchestra.run.smithi156.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-07-16T09:46:15.330 INFO:teuthology.orchestra.run.smithi156.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 256 pref medium 2024-07-16T09:46:15.331 INFO:teuthology.orchestra.run.smithi156.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 1024 expires 1748sec hoplimit 64 pref medium 2024-07-16T09:46:15.333 INFO:teuthology.orchestra.run.smithi156.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-07-16T09:46:15.333 INFO:teuthology.orchestra.run.smithi156.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-07-16T09:46:15.333 INFO:teuthology.orchestra.run.smithi156.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-07-16T09:46:15.333 INFO:teuthology.orchestra.run.smithi156.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-07-16T09:46:15.333 INFO:teuthology.orchestra.run.smithi156.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:fe88:6dfd/64 scope link 2024-07-16T09:46:15.333 INFO:teuthology.orchestra.run.smithi156.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-07-16T09:46:15.334 INFO:teuthology.orchestra.run.smithi156.stdout:Mon IP `172.21.15.156` is in CIDR network `172.21.0.0/20` 2024-07-16T09:46:15.334 INFO:teuthology.orchestra.run.smithi156.stdout:Mon IP `172.21.15.156` is in CIDR network `172.21.0.0/20` 2024-07-16T09:46:15.334 INFO:teuthology.orchestra.run.smithi156.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-07-16T09:46:15.334 INFO:teuthology.orchestra.run.smithi156.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-07-16T09:46:15.335 INFO:teuthology.orchestra.run.smithi156.stdout:Pulling container image quay.ceph.io/ceph-ci/ceph:quincy... 2024-07-16T09:46:15.837 INFO:teuthology.orchestra.run.smithi156.stdout:/bin/podman: stdout d5de5f953b95b9d8fc22e2775f009f50c30fbc9c8f71f43c04eb71cd5289ebad 2024-07-16T09:46:15.838 INFO:teuthology.orchestra.run.smithi156.stdout:/bin/podman: stderr Trying to pull quay.ceph.io/ceph-ci/ceph:quincy... 2024-07-16T09:46:15.838 INFO:teuthology.orchestra.run.smithi156.stdout:/bin/podman: stderr Getting image source signatures 2024-07-16T09:46:15.838 INFO:teuthology.orchestra.run.smithi156.stdout:/bin/podman: stderr Copying blob sha256:2e703a944bea3d2d0d7289c598cef1a71bc393544444494885a4c835e17c4e7a 2024-07-16T09:46:15.838 INFO:teuthology.orchestra.run.smithi156.stdout:/bin/podman: stderr Copying blob sha256:b905d32bc4055f91853c9bcef9a03c0a841e8663d68ab135b4ffcd7db72982c0 2024-07-16T09:46:15.838 INFO:teuthology.orchestra.run.smithi156.stdout:/bin/podman: stderr Copying config sha256:d5de5f953b95b9d8fc22e2775f009f50c30fbc9c8f71f43c04eb71cd5289ebad 2024-07-16T09:46:15.838 INFO:teuthology.orchestra.run.smithi156.stdout:/bin/podman: stderr Writing manifest to image destination 2024-07-16T09:46:16.750 INFO:teuthology.orchestra.run.smithi156.stdout:ceph: stdout ceph version 17.2.7-1104-ge6231b04 (e6231b04b52ebf3b9b31803282dd3074439e51ac) quincy (stable) 2024-07-16T09:46:16.750 INFO:teuthology.orchestra.run.smithi156.stdout:Ceph version: ceph version 17.2.7-1104-ge6231b04 (e6231b04b52ebf3b9b31803282dd3074439e51ac) quincy (stable) 2024-07-16T09:46:16.750 INFO:teuthology.orchestra.run.smithi156.stdout:Extracting ceph user uid/gid from container image... 2024-07-16T09:46:17.716 INFO:teuthology.orchestra.run.smithi156.stdout:stat: stdout 167 167 2024-07-16T09:46:17.716 INFO:teuthology.orchestra.run.smithi156.stdout:Creating initial keys... 2024-07-16T09:46:18.613 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph-authtool: stdout AQBqQZZmiUe4AhAAozs42YdoTXgzH2BzlQVaKQ== 2024-07-16T09:46:19.636 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph-authtool: stdout AQBqQZZm12wmNxAAT6glix4R4httjMKT3kUroQ== 2024-07-16T09:46:20.525 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph-authtool: stdout AQBrQZZmFssIORAAZGx3BAUYmfoOKKxkJXEWtg== 2024-07-16T09:46:20.525 INFO:teuthology.orchestra.run.smithi156.stdout:Creating initial monmap... 2024-07-16T09:46:21.423 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-07-16T09:46:21.423 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = octopus 2024-07-16T09:46:21.423 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 26c91c26-4358-11ef-bcad-c7b262605968 2024-07-16T09:46:21.423 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-07-16T09:46:21.423 INFO:teuthology.orchestra.run.smithi156.stdout:monmaptool for smithi156 [v2:172.21.15.156:3300,v1:172.21.15.156:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-07-16T09:46:21.424 INFO:teuthology.orchestra.run.smithi156.stdout:setting min_mon_release = octopus 2024-07-16T09:46:21.424 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/monmaptool: set fsid to 26c91c26-4358-11ef-bcad-c7b262605968 2024-07-16T09:46:21.424 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-07-16T09:46:21.424 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-07-16T09:46:21.424 INFO:teuthology.orchestra.run.smithi156.stdout:Creating mon... 2024-07-16T09:46:22.513 INFO:teuthology.orchestra.run.smithi156.stdout:create mon.smithi156 on 2024-07-16T09:46:22.729 INFO:teuthology.orchestra.run.smithi156.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-07-16T09:46:22.871 INFO:teuthology.orchestra.run.smithi156.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-07-16T09:46:23.022 INFO:teuthology.orchestra.run.smithi156.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-26c91c26-4358-11ef-bcad-c7b262605968.target → /etc/systemd/system/ceph-26c91c26-4358-11ef-bcad-c7b262605968.target. 2024-07-16T09:46:23.023 INFO:teuthology.orchestra.run.smithi156.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-26c91c26-4358-11ef-bcad-c7b262605968.target → /etc/systemd/system/ceph-26c91c26-4358-11ef-bcad-c7b262605968.target. 2024-07-16T09:46:23.209 INFO:teuthology.orchestra.run.smithi156.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-26c91c26-4358-11ef-bcad-c7b262605968@mon.smithi156 2024-07-16T09:46:23.209 INFO:teuthology.orchestra.run.smithi156.stdout:systemctl: stderr Failed to reset failed state of unit ceph-26c91c26-4358-11ef-bcad-c7b262605968@mon.smithi156.service: Unit ceph-26c91c26-4358-11ef-bcad-c7b262605968@mon.smithi156.service not loaded. 2024-07-16T09:46:23.380 INFO:teuthology.orchestra.run.smithi156.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-26c91c26-4358-11ef-bcad-c7b262605968.target.wants/ceph-26c91c26-4358-11ef-bcad-c7b262605968@mon.smithi156.service → /etc/systemd/system/ceph-26c91c26-4358-11ef-bcad-c7b262605968@.service. 2024-07-16T09:46:23.912 INFO:teuthology.orchestra.run.smithi156.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-07-16T09:46:23.912 INFO:teuthology.orchestra.run.smithi156.stdout:systemctl: stdout disabled 2024-07-16T09:46:23.919 INFO:teuthology.orchestra.run.smithi156.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-07-16T09:46:23.919 INFO:teuthology.orchestra.run.smithi156.stdout:systemctl: stdout inactive 2024-07-16T09:46:23.919 INFO:teuthology.orchestra.run.smithi156.stdout:firewalld.service is not enabled 2024-07-16T09:46:23.920 INFO:teuthology.orchestra.run.smithi156.stdout:Not possible to enable service . firewalld.service is not available 2024-07-16T09:46:23.920 INFO:teuthology.orchestra.run.smithi156.stdout:Waiting for mon to start... 2024-07-16T09:46:23.920 INFO:teuthology.orchestra.run.smithi156.stdout:Waiting for mon... 2024-07-16T09:46:24.946 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout cluster: 2024-07-16T09:46:24.946 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout id: 26c91c26-4358-11ef-bcad-c7b262605968 2024-07-16T09:46:24.946 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-07-16T09:46:24.946 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout 2024-07-16T09:46:24.946 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout services: 2024-07-16T09:46:24.946 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum smithi156 (age 0.538784s) 2024-07-16T09:46:24.946 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-07-16T09:46:24.947 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-07-16T09:46:24.947 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout 2024-07-16T09:46:24.947 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout data: 2024-07-16T09:46:24.947 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-07-16T09:46:24.947 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-07-16T09:46:24.947 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-07-16T09:46:24.947 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout pgs: 2024-07-16T09:46:24.947 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout 2024-07-16T09:46:24.947 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:24.300+0000 7f4169ab3640 1 Processor -- start 2024-07-16T09:46:24.947 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:24.300+0000 7f4169ab3640 1 -- start start 2024-07-16T09:46:24.947 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:24.301+0000 7f4169ab3640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4164075ba0 0x7f4164106e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:46:24.947 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:24.301+0000 7f4169ab3640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4164102820 con 0x7f4164075ba0 2024-07-16T09:46:24.947 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:24.301+0000 7f41637fe640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4164075ba0 0x7f4164106e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:46:24.948 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:24.301+0000 7f41637fe640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4164075ba0 0x7f4164106e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:42710/0 (socket says 172.21.15.156:42710) 2024-07-16T09:46:24.948 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:24.301+0000 7f41637fe640 1 -- 172.21.15.156:0/1944994909 learned_addr learned my addr 172.21.15.156:0/1944994909 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T09:46:24.948 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:24.301+0000 7f41637fe640 1 -- 172.21.15.156:0/1944994909 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4164107370 con 0x7f4164075ba0 2024-07-16T09:46:24.948 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:24.303+0000 7f41637fe640 1 --2- 172.21.15.156:0/1944994909 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4164075ba0 0x7f4164106e30 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f4154009840 tx=0x7f41540091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f4d426835511de1 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:46:24.948 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:24.303+0000 7f41627fc640 1 -- 172.21.15.156:0/1944994909 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4154032440 con 0x7f4164075ba0 2024-07-16T09:46:24.948 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:24.303+0000 7f41627fc640 1 -- 172.21.15.156:0/1944994909 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41540325a0 con 0x7f4164075ba0 2024-07-16T09:46:24.948 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:24.303+0000 7f41627fc640 1 -- 172.21.15.156:0/1944994909 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f41540328d0 con 0x7f4164075ba0 2024-07-16T09:46:24.948 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:24.304+0000 7f4169ab3640 1 -- 172.21.15.156:0/1944994909 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4164075ba0 msgr2=0x7f4164106e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:46:24.948 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:24.304+0000 7f4169ab3640 1 --2- 172.21.15.156:0/1944994909 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4164075ba0 0x7f4164106e30 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f4154009840 tx=0x7f41540091d0 comp rx=0 tx=0).stop 2024-07-16T09:46:24.948 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:24.304+0000 7f4169ab3640 1 -- 172.21.15.156:0/1944994909 shutdown_connections 2024-07-16T09:46:24.948 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:24.304+0000 7f4169ab3640 1 --2- 172.21.15.156:0/1944994909 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4164075ba0 0x7f4164106e30 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:46:24.948 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:24.304+0000 7f4169ab3640 1 -- 172.21.15.156:0/1944994909 >> 172.21.15.156:0/1944994909 conn(0x7f41640fc520 msgr2=0x7f41640fe930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:46:24.949 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:24.304+0000 7f4169ab3640 1 -- 172.21.15.156:0/1944994909 shutdown_connections 2024-07-16T09:46:24.949 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:24.304+0000 7f4169ab3640 1 -- 172.21.15.156:0/1944994909 wait complete. 2024-07-16T09:46:24.949 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:24.305+0000 7f4169ab3640 1 Processor -- start 2024-07-16T09:46:24.949 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:24.305+0000 7f4169ab3640 1 -- start start 2024-07-16T09:46:24.949 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:24.305+0000 7f4169ab3640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4164075ba0 0x7f41641124e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:46:24.949 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:24.305+0000 7f4169ab3640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4164112a20 con 0x7f4164075ba0 2024-07-16T09:46:24.949 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:24.305+0000 7f41637fe640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4164075ba0 0x7f41641124e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:46:24.949 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:24.305+0000 7f41637fe640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4164075ba0 0x7f41641124e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:42716/0 (socket says 172.21.15.156:42716) 2024-07-16T09:46:24.949 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:24.305+0000 7f41637fe640 1 -- 172.21.15.156:0/3373666303 learned_addr learned my addr 172.21.15.156:0/3373666303 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T09:46:24.949 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:24.305+0000 7f41637fe640 1 -- 172.21.15.156:0/3373666303 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4154009450 con 0x7f4164075ba0 2024-07-16T09:46:24.949 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:24.306+0000 7f41637fe640 1 --2- 172.21.15.156:0/3373666303 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4164075ba0 0x7f41641124e0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f415402a700 tx=0x7f4154032000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:46:24.949 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:24.306+0000 7f414bfff640 1 -- 172.21.15.156:0/3373666303 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4154031be0 con 0x7f4164075ba0 2024-07-16T09:46:24.950 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:24.306+0000 7f414bfff640 1 -- 172.21.15.156:0/3373666303 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4154031d40 con 0x7f4164075ba0 2024-07-16T09:46:24.950 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:24.306+0000 7f4169ab3640 1 -- 172.21.15.156:0/3373666303 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4164112c20 con 0x7f4164075ba0 2024-07-16T09:46:24.950 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:24.306+0000 7f414bfff640 1 -- 172.21.15.156:0/3373666303 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f415403b5a0 con 0x7f4164075ba0 2024-07-16T09:46:24.950 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:24.306+0000 7f4169ab3640 1 -- 172.21.15.156:0/3373666303 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4164113030 con 0x7f4164075ba0 2024-07-16T09:46:24.950 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:24.306+0000 7f414bfff640 1 -- 172.21.15.156:0/3373666303 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 1) v1 ==== 529+0+0 (secure 0 0 0) 0x7f4154039040 con 0x7f4164075ba0 2024-07-16T09:46:24.950 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:24.306+0000 7f414bfff640 1 -- 172.21.15.156:0/3373666303 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f4154041030 con 0x7f4164075ba0 2024-07-16T09:46:24.950 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:24.307+0000 7f4169ab3640 1 -- 172.21.15.156:0/3373666303 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f416404b530 con 0x7f4164075ba0 2024-07-16T09:46:24.950 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:24.309+0000 7f414bfff640 1 -- 172.21.15.156:0/3373666303 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71547 (secure 0 0 0) 0x7f4154037070 con 0x7f4164075ba0 2024-07-16T09:46:24.950 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:24.370+0000 7f4169ab3640 1 -- 172.21.15.156:0/3373666303 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7f416410bdd0 con 0x7f4164075ba0 2024-07-16T09:46:24.950 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:24.370+0000 7f414bfff640 1 -- 172.21.15.156:0/3373666303 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+325 (secure 0 0 0) 0x7f415403bb80 con 0x7f4164075ba0 2024-07-16T09:46:24.950 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:24.371+0000 7f4169ab3640 1 -- 172.21.15.156:0/3373666303 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4164075ba0 msgr2=0x7f41641124e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:46:24.950 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:24.371+0000 7f4169ab3640 1 --2- 172.21.15.156:0/3373666303 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4164075ba0 0x7f41641124e0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f415402a700 tx=0x7f4154032000 comp rx=0 tx=0).stop 2024-07-16T09:46:24.951 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:24.371+0000 7f4169ab3640 1 -- 172.21.15.156:0/3373666303 shutdown_connections 2024-07-16T09:46:24.951 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:24.371+0000 7f4169ab3640 1 --2- 172.21.15.156:0/3373666303 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4164075ba0 0x7f41641124e0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:46:24.951 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:24.371+0000 7f4169ab3640 1 -- 172.21.15.156:0/3373666303 >> 172.21.15.156:0/3373666303 conn(0x7f41640fc520 msgr2=0x7f41640fe810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:46:24.951 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:24.371+0000 7f4169ab3640 1 -- 172.21.15.156:0/3373666303 shutdown_connections 2024-07-16T09:46:24.951 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:24.371+0000 7f4169ab3640 1 -- 172.21.15.156:0/3373666303 wait complete. 2024-07-16T09:46:24.951 INFO:teuthology.orchestra.run.smithi156.stdout:mon is available 2024-07-16T09:46:24.951 INFO:teuthology.orchestra.run.smithi156.stdout:Assimilating anything we can from ceph.conf... 2024-07-16T09:46:25.987 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout 2024-07-16T09:46:25.987 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout [global] 2024-07-16T09:46:25.987 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout fsid = 26c91c26-4358-11ef-bcad-c7b262605968 2024-07-16T09:46:25.987 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.156:3300,v1:172.21.15.156:6789] 2024-07-16T09:46:25.987 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-07-16T09:46:25.987 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-07-16T09:46:25.987 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-07-16T09:46:25.987 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-07-16T09:46:25.987 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout 2024-07-16T09:46:25.988 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout [mgr] 2024-07-16T09:46:25.988 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-07-16T09:46:25.988 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout 2024-07-16T09:46:25.988 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout [osd] 2024-07-16T09:46:25.988 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-07-16T09:46:25.988 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-07-16T09:46:25.988 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-07-16T09:46:25.988 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-07-16T09:46:25.988 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-07-16T09:46:25.988 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:25.329+0000 7f11fd7cb640 1 Processor -- start 2024-07-16T09:46:25.988 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:25.329+0000 7f11fd7cb640 1 -- start start 2024-07-16T09:46:25.988 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:25.329+0000 7f11fd7cb640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f11f8100ce0 0x7f11f81010f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:46:25.989 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:25.329+0000 7f11fd7cb640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11f8101630 con 0x7f11f8100ce0 2024-07-16T09:46:25.989 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:25.330+0000 7f11f6ffd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f11f8100ce0 0x7f11f81010f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:46:25.989 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:25.330+0000 7f11f6ffd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f11f8100ce0 0x7f11f81010f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:42722/0 (socket says 172.21.15.156:42722) 2024-07-16T09:46:25.989 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:25.330+0000 7f11f6ffd640 1 -- 172.21.15.156:0/2803767943 learned_addr learned my addr 172.21.15.156:0/2803767943 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T09:46:25.989 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:25.330+0000 7f11f6ffd640 1 -- 172.21.15.156:0/2803767943 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11f8101770 con 0x7f11f8100ce0 2024-07-16T09:46:25.989 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:25.331+0000 7f11f6ffd640 1 --2- 172.21.15.156:0/2803767943 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f11f8100ce0 0x7f11f81010f0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f11e8009840 tx=0x7f11e80091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79541b669c576579 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:46:25.989 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:25.331+0000 7f11f5ffb640 1 -- 172.21.15.156:0/2803767943 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f11e8032440 con 0x7f11f8100ce0 2024-07-16T09:46:25.989 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:25.331+0000 7f11f5ffb640 1 -- 172.21.15.156:0/2803767943 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11e80325a0 con 0x7f11f8100ce0 2024-07-16T09:46:25.989 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:25.331+0000 7f11f5ffb640 1 -- 172.21.15.156:0/2803767943 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f11e80328d0 con 0x7f11f8100ce0 2024-07-16T09:46:25.989 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:25.332+0000 7f11fd7cb640 1 -- 172.21.15.156:0/2803767943 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f11f8100ce0 msgr2=0x7f11f81010f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:46:25.989 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:25.332+0000 7f11fd7cb640 1 --2- 172.21.15.156:0/2803767943 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f11f8100ce0 0x7f11f81010f0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f11e8009840 tx=0x7f11e80091d0 comp rx=0 tx=0).stop 2024-07-16T09:46:25.989 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:25.332+0000 7f11fd7cb640 1 -- 172.21.15.156:0/2803767943 shutdown_connections 2024-07-16T09:46:25.989 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:25.332+0000 7f11fd7cb640 1 --2- 172.21.15.156:0/2803767943 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f11f8100ce0 0x7f11f81010f0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:46:25.990 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:25.332+0000 7f11fd7cb640 1 -- 172.21.15.156:0/2803767943 >> 172.21.15.156:0/2803767943 conn(0x7f11f80fc520 msgr2=0x7f11f80fe930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:46:25.990 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:25.332+0000 7f11fd7cb640 1 -- 172.21.15.156:0/2803767943 shutdown_connections 2024-07-16T09:46:25.990 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:25.332+0000 7f11fd7cb640 1 -- 172.21.15.156:0/2803767943 wait complete. 2024-07-16T09:46:25.990 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:25.332+0000 7f11fd7cb640 1 Processor -- start 2024-07-16T09:46:25.990 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:25.332+0000 7f11fd7cb640 1 -- start start 2024-07-16T09:46:25.990 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:25.332+0000 7f11fd7cb640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f11f8100ce0 0x7f11f8075b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:46:25.990 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:25.332+0000 7f11fd7cb640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11f8113030 con 0x7f11f8100ce0 2024-07-16T09:46:25.990 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:25.333+0000 7f11f6ffd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f11f8100ce0 0x7f11f8075b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:46:25.990 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:25.333+0000 7f11f6ffd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f11f8100ce0 0x7f11f8075b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:42732/0 (socket says 172.21.15.156:42732) 2024-07-16T09:46:25.990 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:25.333+0000 7f11f6ffd640 1 -- 172.21.15.156:0/3572993154 learned_addr learned my addr 172.21.15.156:0/3572993154 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T09:46:25.990 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:25.333+0000 7f11f6ffd640 1 -- 172.21.15.156:0/3572993154 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11e8009450 con 0x7f11f8100ce0 2024-07-16T09:46:25.990 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:25.333+0000 7f11f6ffd640 1 --2- 172.21.15.156:0/3572993154 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f11f8100ce0 0x7f11f8075b00 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f11e802a700 tx=0x7f11e8032000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:46:25.991 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:25.333+0000 7f11df7fe640 1 -- 172.21.15.156:0/3572993154 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f11e8031be0 con 0x7f11f8100ce0 2024-07-16T09:46:25.991 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:25.333+0000 7f11df7fe640 1 -- 172.21.15.156:0/3572993154 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11e8031d40 con 0x7f11f8100ce0 2024-07-16T09:46:25.991 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:25.333+0000 7f11fd7cb640 1 -- 172.21.15.156:0/3572993154 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11f8113170 con 0x7f11f8100ce0 2024-07-16T09:46:25.991 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:25.333+0000 7f11df7fe640 1 -- 172.21.15.156:0/3572993154 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f11e803b5a0 con 0x7f11f8100ce0 2024-07-16T09:46:25.991 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:25.333+0000 7f11fd7cb640 1 -- 172.21.15.156:0/3572993154 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11f8113520 con 0x7f11f8100ce0 2024-07-16T09:46:25.991 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:25.333+0000 7f11df7fe640 1 -- 172.21.15.156:0/3572993154 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 1) v1 ==== 529+0+0 (secure 0 0 0) 0x7f11e8039040 con 0x7f11f8100ce0 2024-07-16T09:46:25.991 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:25.333+0000 7f11df7fe640 1 -- 172.21.15.156:0/3572993154 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f11e8041030 con 0x7f11f8100ce0 2024-07-16T09:46:25.991 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:25.334+0000 7f11fd7cb640 1 -- 172.21.15.156:0/3572993154 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11f804b530 con 0x7f11f8100ce0 2024-07-16T09:46:25.991 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:25.336+0000 7f11df7fe640 1 -- 172.21.15.156:0/3572993154 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71547 (secure 0 0 0) 0x7f11e8037070 con 0x7f11f8100ce0 2024-07-16T09:46:25.991 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:25.395+0000 7f11fd7cb640 1 -- 172.21.15.156:0/3572993154 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7f11f8101fd0 con 0x7f11f8100ce0 2024-07-16T09:46:25.991 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:25.398+0000 7f11df7fe640 1 -- 172.21.15.156:0/3572993154 <== mon.0 v2:172.21.15.156:3300/0 7 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f11e8046d00 con 0x7f11f8100ce0 2024-07-16T09:46:25.991 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:25.398+0000 7f11df7fe640 1 -- 172.21.15.156:0/3572993154 <== mon.0 v2:172.21.15.156:3300/0 8 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) v1 ==== 70+0+431 (secure 0 0 0) 0x7f11e804eca0 con 0x7f11f8100ce0 2024-07-16T09:46:25.991 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:25.399+0000 7f11fd7cb640 1 -- 172.21.15.156:0/3572993154 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f11f8100ce0 msgr2=0x7f11f8075b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:46:25.992 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:25.399+0000 7f11fd7cb640 1 --2- 172.21.15.156:0/3572993154 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f11f8100ce0 0x7f11f8075b00 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f11e802a700 tx=0x7f11e8032000 comp rx=0 tx=0).stop 2024-07-16T09:46:25.992 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:25.400+0000 7f11fd7cb640 1 -- 172.21.15.156:0/3572993154 shutdown_connections 2024-07-16T09:46:25.992 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:25.400+0000 7f11fd7cb640 1 --2- 172.21.15.156:0/3572993154 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f11f8100ce0 0x7f11f8075b00 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:46:25.992 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:25.400+0000 7f11fd7cb640 1 -- 172.21.15.156:0/3572993154 >> 172.21.15.156:0/3572993154 conn(0x7f11f80fc520 msgr2=0x7f11f8103710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:46:25.992 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:25.400+0000 7f11fd7cb640 1 -- 172.21.15.156:0/3572993154 shutdown_connections 2024-07-16T09:46:25.992 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:25.400+0000 7f11fd7cb640 1 -- 172.21.15.156:0/3572993154 wait complete. 2024-07-16T09:46:25.992 INFO:teuthology.orchestra.run.smithi156.stdout:Generating new minimal ceph.conf... 2024-07-16T09:46:27.034 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:26 smithi156 ceph-mon[28933]: from='client.? 172.21.15.156:0/1446578789' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-16T09:46:27.042 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:26.412+0000 7f928714c640 1 Processor -- start 2024-07-16T09:46:27.042 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:26.412+0000 7f928714c640 1 -- start start 2024-07-16T09:46:27.042 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:26.412+0000 7f928714c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9280075bc0 0x7f9280104a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:46:27.042 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:26.412+0000 7f928714c640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9280104f50 con 0x7f9280075bc0 2024-07-16T09:46:27.042 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:26.412+0000 7f9284ee8640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9280075bc0 0x7f9280104a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:46:27.042 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:26.412+0000 7f9284ee8640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9280075bc0 0x7f9280104a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:42748/0 (socket says 172.21.15.156:42748) 2024-07-16T09:46:27.042 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:26.412+0000 7f9284ee8640 1 -- 172.21.15.156:0/254080221 learned_addr learned my addr 172.21.15.156:0/254080221 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T09:46:27.042 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:26.412+0000 7f9284ee8640 1 -- 172.21.15.156:0/254080221 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9280105090 con 0x7f9280075bc0 2024-07-16T09:46:27.042 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:26.413+0000 7f9284ee8640 1 --2- 172.21.15.156:0/254080221 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9280075bc0 0x7f9280104a10 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f9274009840 tx=0x7f92740091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8972630643bf0a43 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:46:27.042 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:26.413+0000 7f927f7fe640 1 -- 172.21.15.156:0/254080221 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9274032440 con 0x7f9280075bc0 2024-07-16T09:46:27.042 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:26.413+0000 7f927f7fe640 1 -- 172.21.15.156:0/254080221 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f9274032a30 con 0x7f9280075bc0 2024-07-16T09:46:27.043 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:26.413+0000 7f927f7fe640 1 -- 172.21.15.156:0/254080221 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9274031ad0 con 0x7f9280075bc0 2024-07-16T09:46:27.043 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:26.414+0000 7f928714c640 1 -- 172.21.15.156:0/254080221 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9280075bc0 msgr2=0x7f9280104a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:46:27.043 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:26.414+0000 7f928714c640 1 --2- 172.21.15.156:0/254080221 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9280075bc0 0x7f9280104a10 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f9274009840 tx=0x7f92740091d0 comp rx=0 tx=0).stop 2024-07-16T09:46:27.043 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:26.414+0000 7f928714c640 1 -- 172.21.15.156:0/254080221 shutdown_connections 2024-07-16T09:46:27.043 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:26.414+0000 7f928714c640 1 --2- 172.21.15.156:0/254080221 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9280075bc0 0x7f9280104a10 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:46:27.043 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:26.414+0000 7f928714c640 1 -- 172.21.15.156:0/254080221 >> 172.21.15.156:0/254080221 conn(0x7f92800fc520 msgr2=0x7f92800fe930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:46:27.043 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:26.414+0000 7f928714c640 1 -- 172.21.15.156:0/254080221 shutdown_connections 2024-07-16T09:46:27.043 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:26.414+0000 7f928714c640 1 -- 172.21.15.156:0/254080221 wait complete. 2024-07-16T09:46:27.043 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:26.414+0000 7f928714c640 1 Processor -- start 2024-07-16T09:46:27.043 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:26.414+0000 7f928714c640 1 -- start start 2024-07-16T09:46:27.043 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:26.415+0000 7f928714c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9280075bc0 0x7f9280110a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:46:27.043 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:26.415+0000 7f928714c640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9280110fb0 con 0x7f9280075bc0 2024-07-16T09:46:27.043 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:26.415+0000 7f9284ee8640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9280075bc0 0x7f9280110a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:46:27.044 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:26.415+0000 7f9284ee8640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9280075bc0 0x7f9280110a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:42756/0 (socket says 172.21.15.156:42756) 2024-07-16T09:46:27.044 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:26.415+0000 7f9284ee8640 1 -- 172.21.15.156:0/1446578789 learned_addr learned my addr 172.21.15.156:0/1446578789 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T09:46:27.044 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:26.415+0000 7f9284ee8640 1 -- 172.21.15.156:0/1446578789 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9274009450 con 0x7f9280075bc0 2024-07-16T09:46:27.044 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:26.415+0000 7f9284ee8640 1 --2- 172.21.15.156:0/1446578789 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9280075bc0 0x7f9280110a70 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f9274031b90 tx=0x7f9274008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:46:27.044 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:26.415+0000 7f927d7fa640 1 -- 172.21.15.156:0/1446578789 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f92740328b0 con 0x7f9280075bc0 2024-07-16T09:46:27.044 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:26.415+0000 7f927d7fa640 1 -- 172.21.15.156:0/1446578789 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f927403d8e0 con 0x7f9280075bc0 2024-07-16T09:46:27.044 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:26.415+0000 7f927d7fa640 1 -- 172.21.15.156:0/1446578789 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f927403ba20 con 0x7f9280075bc0 2024-07-16T09:46:27.044 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:26.415+0000 7f928714c640 1 -- 172.21.15.156:0/1446578789 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92801111b0 con 0x7f9280075bc0 2024-07-16T09:46:27.044 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:26.415+0000 7f928714c640 1 -- 172.21.15.156:0/1446578789 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9280111620 con 0x7f9280075bc0 2024-07-16T09:46:27.044 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:26.415+0000 7f927d7fa640 1 -- 172.21.15.156:0/1446578789 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 1) v1 ==== 529+0+0 (secure 0 0 0) 0x7f9274039070 con 0x7f9280075bc0 2024-07-16T09:46:27.044 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:26.415+0000 7f927d7fa640 1 -- 172.21.15.156:0/1446578789 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f9274046930 con 0x7f9280075bc0 2024-07-16T09:46:27.044 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:26.416+0000 7f928714c640 1 -- 172.21.15.156:0/1446578789 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f928004b530 con 0x7f9280075bc0 2024-07-16T09:46:27.045 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:26.418+0000 7f927d7fa640 1 -- 172.21.15.156:0/1446578789 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71547 (secure 0 0 0) 0x7f9274065790 con 0x7f9280075bc0 2024-07-16T09:46:27.045 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:26.476+0000 7f928714c640 1 -- 172.21.15.156:0/1446578789 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7f9280111d40 con 0x7f9280075bc0 2024-07-16T09:46:27.045 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:26.477+0000 7f927d7fa640 1 -- 172.21.15.156:0/1446578789 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) v1 ==== 76+0+177 (secure 0 0 0) 0x7f9274054020 con 0x7f9280075bc0 2024-07-16T09:46:27.045 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:26.477+0000 7f928714c640 1 -- 172.21.15.156:0/1446578789 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9280075bc0 msgr2=0x7f9280110a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:46:27.045 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:26.477+0000 7f928714c640 1 --2- 172.21.15.156:0/1446578789 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9280075bc0 0x7f9280110a70 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f9274031b90 tx=0x7f9274008040 comp rx=0 tx=0).stop 2024-07-16T09:46:27.045 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:26.477+0000 7f928714c640 1 -- 172.21.15.156:0/1446578789 shutdown_connections 2024-07-16T09:46:27.045 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:26.477+0000 7f928714c640 1 --2- 172.21.15.156:0/1446578789 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9280075bc0 0x7f9280110a70 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:46:27.045 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:26.477+0000 7f928714c640 1 -- 172.21.15.156:0/1446578789 >> 172.21.15.156:0/1446578789 conn(0x7f92800fc520 msgr2=0x7f92800fcfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:46:27.045 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:26.478+0000 7f928714c640 1 -- 172.21.15.156:0/1446578789 shutdown_connections 2024-07-16T09:46:27.045 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:26.478+0000 7f928714c640 1 -- 172.21.15.156:0/1446578789 wait complete. 2024-07-16T09:46:27.045 INFO:teuthology.orchestra.run.smithi156.stdout:Restarting the monitor... 2024-07-16T09:46:27.288 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:27 smithi156 systemd[1]: Stopping Ceph mon.smithi156 for 26c91c26-4358-11ef-bcad-c7b262605968... 2024-07-16T09:46:27.553 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:27 smithi156 ceph-26c91c26-4358-11ef-bcad-c7b262605968-mon-smithi156[28927]: 2024-07-16T09:46:27.285+0000 7f0f796c9640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi156 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-07-16T09:46:27.553 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:27 smithi156 ceph-26c91c26-4358-11ef-bcad-c7b262605968-mon-smithi156[28927]: 2024-07-16T09:46:27.285+0000 7f0f796c9640 -1 mon.smithi156@0(leader) e1 *** Got Signal Terminated *** 2024-07-16T09:46:27.856 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:27 smithi156 podman[29177]: 2024-07-16 09:46:27.552752666 +0000 UTC m=+0.346701307 container died 075ea8ae2b65a72f2a83e85abbd66d2c1430400d8343badc39d1a35e7368f18d (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-26c91c26-4358-11ef-bcad-c7b262605968-mon-smithi156, org.label-schema.license=GPLv2, RELEASE=quincy-e6231b0, org.label-schema.build-date=20240709, CEPH_POINT_RELEASE=, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-07-16T09:46:28.175 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:27 smithi156 podman[29177]: 2024-07-16 09:46:27.92825965 +0000 UTC m=+0.722208289 container cleanup 075ea8ae2b65a72f2a83e85abbd66d2c1430400d8343badc39d1a35e7368f18d (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-26c91c26-4358-11ef-bcad-c7b262605968-mon-smithi156, io.buildah.version=1.36.0, RELEASE=quincy-e6231b0, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240709, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-07-16T09:46:28.175 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:27 smithi156 bash[29177]: ceph-26c91c26-4358-11ef-bcad-c7b262605968-mon-smithi156 2024-07-16T09:46:28.521 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:28 smithi156 podman[29188]: 2024-07-16 09:46:28.179011958 +0000 UTC m=+0.629633357 container remove 075ea8ae2b65a72f2a83e85abbd66d2c1430400d8343badc39d1a35e7368f18d (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-26c91c26-4358-11ef-bcad-c7b262605968-mon-smithi156, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.36.0, RELEASE=quincy-e6231b0, org.label-schema.build-date=20240709, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-07-16T09:46:28.521 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:28 smithi156 systemd[1]: ceph-26c91c26-4358-11ef-bcad-c7b262605968@mon.smithi156.service: Deactivated successfully. 2024-07-16T09:46:28.521 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:28 smithi156 systemd[1]: Stopped Ceph mon.smithi156 for 26c91c26-4358-11ef-bcad-c7b262605968. 2024-07-16T09:46:28.521 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:28 smithi156 systemd[1]: Starting Ceph mon.smithi156 for 26c91c26-4358-11ef-bcad-c7b262605968... 2024-07-16T09:46:28.772 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:28 smithi156 podman[29282]: 2024-07-16 09:46:28.423146991 +0000 UTC m=+0.017523646 image pull d5de5f953b95b9d8fc22e2775f009f50c30fbc9c8f71f43c04eb71cd5289ebad quay.ceph.io/ceph-ci/ceph:quincy 2024-07-16T09:46:28.772 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:28 smithi156 podman[29282]: 2024-07-16 09:46:28.521493306 +0000 UTC m=+0.115869959 container create 6471c8acff4002d36c1d1a5a5b009315b53bee9c4642cd88cbfec151eed1ac2b (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-26c91c26-4358-11ef-bcad-c7b262605968-mon-smithi156, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=quincy-e6231b0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240709, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.36.0, GIT_CLEAN=True, ceph=True) 2024-07-16T09:46:28.772 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:28 smithi156 podman[29282]: 2024-07-16 09:46:28.740729919 +0000 UTC m=+0.335106567 container init 6471c8acff4002d36c1d1a5a5b009315b53bee9c4642cd88cbfec151eed1ac2b (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-26c91c26-4358-11ef-bcad-c7b262605968-mon-smithi156, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240709, RELEASE=quincy-e6231b0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.36.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-07-16T09:46:28.772 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:28 smithi156 podman[29282]: 2024-07-16 09:46:28.743853647 +0000 UTC m=+0.338230302 container start 6471c8acff4002d36c1d1a5a5b009315b53bee9c4642cd88cbfec151eed1ac2b (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-26c91c26-4358-11ef-bcad-c7b262605968-mon-smithi156, org.label-schema.build-date=20240709, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, RELEASE=quincy-e6231b0, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.36.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image) 2024-07-16T09:46:28.772 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:28 smithi156 ceph-mon[29302]: set uid:gid to 167:167 (ceph:ceph) 2024-07-16T09:46:28.772 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:28 smithi156 ceph-mon[29302]: ceph version 17.2.7-1104-ge6231b04 (e6231b04b52ebf3b9b31803282dd3074439e51ac) quincy (stable), process ceph-mon, pid 2 2024-07-16T09:46:28.772 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:28 smithi156 ceph-mon[29302]: pidfile_write: ignore empty --pid-file 2024-07-16T09:46:28.773 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:28 smithi156 ceph-mon[29302]: load: jerasure load: lrc 2024-07-16T09:46:28.773 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:28 smithi156 ceph-mon[29302]: rocksdb: RocksDB version: 6.15.5 2024-07-16T09:46:28.773 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:28 smithi156 ceph-mon[29302]: rocksdb: Git sha rocksdb_build_git_sha:@0@ 2024-07-16T09:46:28.773 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:28 smithi156 ceph-mon[29302]: rocksdb: Compile date Jul 15 2024 2024-07-16T09:46:28.773 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:28 smithi156 ceph-mon[29302]: rocksdb: DB SUMMARY 2024-07-16T09:46:28.773 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:28 smithi156 ceph-mon[29302]: rocksdb: DB Session ID: 718D3ZOX8YKGYD7KMYX8 2024-07-16T09:46:28.773 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:28 smithi156 ceph-mon[29302]: rocksdb: CURRENT file: CURRENT 2024-07-16T09:46:28.773 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:28 smithi156 ceph-mon[29302]: rocksdb: IDENTITY file: IDENTITY 2024-07-16T09:46:28.773 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:28 smithi156 ceph-mon[29302]: rocksdb: MANIFEST file: MANIFEST-000009 size: 131 Bytes 2024-07-16T09:46:28.773 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:28 smithi156 ceph-mon[29302]: rocksdb: SST files in /var/lib/ceph/mon/ceph-smithi156/store.db dir, Total Num: 1, files: 000008.sst 2024-07-16T09:46:28.773 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:28 smithi156 ceph-mon[29302]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-smithi156/store.db: 000010.log size: 98198 ; 2024-07-16T09:46:28.773 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:28 smithi156 ceph-mon[29302]: rocksdb: Options.error_if_exists: 0 2024-07-16T09:46:28.774 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:28 smithi156 ceph-mon[29302]: rocksdb: Options.create_if_missing: 0 2024-07-16T09:46:28.774 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:28 smithi156 ceph-mon[29302]: rocksdb: Options.paranoid_checks: 1 2024-07-16T09:46:28.774 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:28 smithi156 ceph-mon[29302]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-07-16T09:46:28.774 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:28 smithi156 ceph-mon[29302]: rocksdb: Options.env: 0x55fb2f3c8440 2024-07-16T09:46:28.774 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:28 smithi156 ceph-mon[29302]: rocksdb: Options.fs: Posix File System 2024-07-16T09:46:28.774 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:28 smithi156 ceph-mon[29302]: rocksdb: Options.info_log: 0x55fb31091120 2024-07-16T09:46:28.774 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:28 smithi156 ceph-mon[29302]: rocksdb: Options.max_file_opening_threads: 16 2024-07-16T09:46:28.774 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:28 smithi156 ceph-mon[29302]: rocksdb: Options.statistics: (nil) 2024-07-16T09:46:28.774 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:28 smithi156 ceph-mon[29302]: rocksdb: Options.use_fsync: 0 2024-07-16T09:46:28.774 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:28 smithi156 ceph-mon[29302]: rocksdb: Options.max_log_file_size: 0 2024-07-16T09:46:28.774 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:28 smithi156 ceph-mon[29302]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-07-16T09:46:28.774 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:28 smithi156 ceph-mon[29302]: rocksdb: Options.log_file_time_to_roll: 0 2024-07-16T09:46:28.775 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:28 smithi156 ceph-mon[29302]: rocksdb: Options.keep_log_file_num: 1000 2024-07-16T09:46:28.775 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:28 smithi156 ceph-mon[29302]: rocksdb: Options.recycle_log_file_num: 0 2024-07-16T09:46:28.775 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:28 smithi156 ceph-mon[29302]: rocksdb: Options.allow_fallocate: 1 2024-07-16T09:46:28.775 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:28 smithi156 ceph-mon[29302]: rocksdb: Options.allow_mmap_reads: 0 2024-07-16T09:46:28.775 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:28 smithi156 ceph-mon[29302]: rocksdb: Options.allow_mmap_writes: 0 2024-07-16T09:46:28.775 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:28 smithi156 ceph-mon[29302]: rocksdb: Options.use_direct_reads: 0 2024-07-16T09:46:28.775 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:28 smithi156 ceph-mon[29302]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-07-16T09:46:28.775 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:28 smithi156 ceph-mon[29302]: rocksdb: Options.create_missing_column_families: 0 2024-07-16T09:46:28.775 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:28 smithi156 ceph-mon[29302]: rocksdb: Options.db_log_dir: 2024-07-16T09:46:28.775 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:28 smithi156 ceph-mon[29302]: rocksdb: Options.wal_dir: /var/lib/ceph/mon/ceph-smithi156/store.db 2024-07-16T09:46:28.775 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:28 smithi156 ceph-mon[29302]: rocksdb: Options.table_cache_numshardbits: 6 2024-07-16T09:46:28.775 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:28 smithi156 ceph-mon[29302]: rocksdb: Options.WAL_ttl_seconds: 0 2024-07-16T09:46:28.775 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:28 smithi156 ceph-mon[29302]: rocksdb: Options.WAL_size_limit_MB: 0 2024-07-16T09:46:28.776 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:28 smithi156 ceph-mon[29302]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-07-16T09:46:28.776 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:28 smithi156 ceph-mon[29302]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-07-16T09:46:28.776 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:28 smithi156 ceph-mon[29302]: rocksdb: Options.is_fd_close_on_exec: 1 2024-07-16T09:46:28.776 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:28 smithi156 ceph-mon[29302]: rocksdb: Options.advise_random_on_open: 1 2024-07-16T09:46:28.776 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:28 smithi156 ceph-mon[29302]: rocksdb: Options.db_write_buffer_size: 0 2024-07-16T09:46:28.776 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:28 smithi156 ceph-mon[29302]: rocksdb: Options.write_buffer_manager: 0x55fb3114e7e0 2024-07-16T09:46:28.776 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:28 smithi156 ceph-mon[29302]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-07-16T09:46:28.776 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:28 smithi156 ceph-mon[29302]: rocksdb: Options.new_table_reader_for_compaction_inputs: 0 2024-07-16T09:46:28.776 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:28 smithi156 ceph-mon[29302]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-07-16T09:46:28.776 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:28 smithi156 ceph-mon[29302]: rocksdb: Options.use_adaptive_mutex: 0 2024-07-16T09:46:28.776 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:28 smithi156 ceph-mon[29302]: rocksdb: Options.rate_limiter: (nil) 2024-07-16T09:46:28.776 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:28 smithi156 ceph-mon[29302]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-07-16T09:46:28.776 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:28 smithi156 ceph-mon[29302]: rocksdb: Options.wal_recovery_mode: 2 2024-07-16T09:46:28.777 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:28 smithi156 ceph-mon[29302]: rocksdb: Options.enable_thread_tracking: 0 2024-07-16T09:46:28.777 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:28 smithi156 ceph-mon[29302]: rocksdb: Options.enable_pipelined_write: 0 2024-07-16T09:46:28.778 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:28 smithi156 ceph-mon[29302]: rocksdb: Options.unordered_write: 0 2024-07-16T09:46:28.778 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:28 smithi156 ceph-mon[29302]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-07-16T09:46:28.778 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:28 smithi156 ceph-mon[29302]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-07-16T09:46:28.778 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:28 smithi156 ceph-mon[29302]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-07-16T09:46:28.778 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:28 smithi156 ceph-mon[29302]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-07-16T09:46:28.778 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:28 smithi156 ceph-mon[29302]: rocksdb: Options.row_cache: None 2024-07-16T09:46:28.778 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:28 smithi156 ceph-mon[29302]: rocksdb: Options.wal_filter: None 2024-07-16T09:46:28.778 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:28 smithi156 ceph-mon[29302]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-07-16T09:46:28.778 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:28 smithi156 ceph-mon[29302]: rocksdb: Options.allow_ingest_behind: 0 2024-07-16T09:46:28.779 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:28 smithi156 ceph-mon[29302]: rocksdb: Options.preserve_deletes: 0 2024-07-16T09:46:28.779 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:28 smithi156 ceph-mon[29302]: rocksdb: Options.two_write_queues: 0 2024-07-16T09:46:28.779 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:28 smithi156 ceph-mon[29302]: rocksdb: Options.manual_wal_flush: 0 2024-07-16T09:46:28.779 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:28 smithi156 ceph-mon[29302]: rocksdb: Options.atomic_flush: 0 2024-07-16T09:46:28.779 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:28 smithi156 ceph-mon[29302]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-07-16T09:46:28.779 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:28 smithi156 ceph-mon[29302]: rocksdb: Options.persist_stats_to_disk: 0 2024-07-16T09:46:28.779 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:28 smithi156 ceph-mon[29302]: rocksdb: Options.write_dbid_to_manifest: 0 2024-07-16T09:46:28.779 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:28 smithi156 ceph-mon[29302]: rocksdb: Options.log_readahead_size: 0 2024-07-16T09:46:28.779 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:28 smithi156 ceph-mon[29302]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-07-16T09:46:28.779 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:28 smithi156 ceph-mon[29302]: rocksdb: Options.best_efforts_recovery: 0 2024-07-16T09:46:28.779 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:28 smithi156 ceph-mon[29302]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-07-16T09:46:28.780 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:28 smithi156 ceph-mon[29302]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-07-16T09:46:28.780 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:28 smithi156 ceph-mon[29302]: rocksdb: Options.allow_data_in_errors: 0 2024-07-16T09:46:28.780 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:28 smithi156 ceph-mon[29302]: rocksdb: Options.db_host_id: __hostname__ 2024-07-16T09:46:28.780 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:28 smithi156 ceph-mon[29302]: rocksdb: Options.max_background_jobs: 2 2024-07-16T09:46:28.780 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:28 smithi156 ceph-mon[29302]: rocksdb: Options.max_background_compactions: -1 2024-07-16T09:46:28.780 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:28 smithi156 ceph-mon[29302]: rocksdb: Options.max_subcompactions: 1 2024-07-16T09:46:28.780 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:28 smithi156 ceph-mon[29302]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-07-16T09:46:28.780 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:28 smithi156 ceph-mon[29302]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-07-16T09:46:28.780 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:28 smithi156 ceph-mon[29302]: rocksdb: Options.delayed_write_rate : 16777216 2024-07-16T09:46:28.780 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:28 smithi156 ceph-mon[29302]: rocksdb: Options.max_total_wal_size: 0 2024-07-16T09:46:28.780 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:28 smithi156 ceph-mon[29302]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-07-16T09:46:28.780 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:28 smithi156 ceph-mon[29302]: rocksdb: Options.stats_dump_period_sec: 600 2024-07-16T09:46:28.781 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:28 smithi156 ceph-mon[29302]: rocksdb: Options.stats_persist_period_sec: 600 2024-07-16T09:46:28.781 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:28 smithi156 ceph-mon[29302]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-07-16T09:46:28.781 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:28 smithi156 ceph-mon[29302]: rocksdb: Options.max_open_files: -1 2024-07-16T09:46:28.781 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:28 smithi156 ceph-mon[29302]: rocksdb: Options.bytes_per_sync: 0 2024-07-16T09:46:28.781 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:28 smithi156 ceph-mon[29302]: rocksdb: Options.wal_bytes_per_sync: 0 2024-07-16T09:46:28.781 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:28 smithi156 ceph-mon[29302]: rocksdb: Options.strict_bytes_per_sync: 0 2024-07-16T09:46:28.781 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:28 smithi156 ceph-mon[29302]: rocksdb: Options.compaction_readahead_size: 0 2024-07-16T09:46:28.781 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:28 smithi156 ceph-mon[29302]: rocksdb: Options.max_background_flushes: -1 2024-07-16T09:46:28.781 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:28 smithi156 ceph-mon[29302]: rocksdb: Compression algorithms supported: 2024-07-16T09:46:28.781 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:28 smithi156 ceph-mon[29302]: rocksdb: kZSTD supported: 0 2024-07-16T09:46:28.781 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:28 smithi156 ceph-mon[29302]: rocksdb: kXpressCompression supported: 0 2024-07-16T09:46:28.781 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:28 smithi156 ceph-mon[29302]: rocksdb: kBZip2Compression supported: 0 2024-07-16T09:46:28.782 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:28 smithi156 ceph-mon[29302]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-07-16T09:46:28.782 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:28 smithi156 ceph-mon[29302]: rocksdb: kLZ4Compression supported: 1 2024-07-16T09:46:28.782 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:28 smithi156 ceph-mon[29302]: rocksdb: kZlibCompression supported: 1 2024-07-16T09:46:28.782 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:28 smithi156 ceph-mon[29302]: rocksdb: kLZ4HCCompression supported: 1 2024-07-16T09:46:28.782 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:28 smithi156 ceph-mon[29302]: rocksdb: kSnappyCompression supported: 1 2024-07-16T09:46:28.782 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:28 smithi156 ceph-mon[29302]: rocksdb: Fast CRC32 supported: Supported on x86 2024-07-16T09:46:28.782 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:28 smithi156 ceph-mon[29302]: rocksdb: [db/version_set.cc:4724] Recovering from manifest file: /var/lib/ceph/mon/ceph-smithi156/store.db/MANIFEST-000009 2024-07-16T09:46:28.782 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:28 smithi156 ceph-mon[29302]: rocksdb: [db/column_family.cc:595] --------------- Options for column family [default]: 2024-07-16T09:46:28.782 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:28 smithi156 ceph-mon[29302]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-07-16T09:46:28.782 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:28 smithi156 ceph-mon[29302]: rocksdb: Options.merge_operator: 2024-07-16T09:46:28.782 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:28 smithi156 ceph-mon[29302]: rocksdb: Options.compaction_filter: None 2024-07-16T09:46:28.782 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:28 smithi156 ceph-mon[29302]: rocksdb: Options.compaction_filter_factory: None 2024-07-16T09:46:28.783 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:28 smithi156 ceph-mon[29302]: rocksdb: Options.sst_partitioner_factory: None 2024-07-16T09:46:28.783 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:28 smithi156 ceph-mon[29302]: rocksdb: Options.memtable_factory: SkipListFactory 2024-07-16T09:46:28.783 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:28 smithi156 ceph-mon[29302]: rocksdb: Options.table_factory: BlockBasedTable 2024-07-16T09:46:28.783 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:28 smithi156 ceph-mon[29302]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55fb31037a68) 2024-07-16T09:46:28.783 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: cache_index_and_filter_blocks: 1 2024-07-16T09:46:28.783 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-07-16T09:46:28.783 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-07-16T09:46:28.783 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: pin_top_level_index_and_filter: 1 2024-07-16T09:46:28.783 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: index_type: 0 2024-07-16T09:46:28.783 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: data_block_index_type: 0 2024-07-16T09:46:28.783 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: index_shortening: 1 2024-07-16T09:46:28.783 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: data_block_hash_table_util_ratio: 0.750000 2024-07-16T09:46:28.784 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: hash_index_allow_collision: 1 2024-07-16T09:46:28.784 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: checksum: 1 2024-07-16T09:46:28.784 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: no_block_cache: 0 2024-07-16T09:46:28.784 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: block_cache: 0x55fb3106d1f0 2024-07-16T09:46:28.784 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: block_cache_name: BinnedLRUCache 2024-07-16T09:46:28.784 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: block_cache_options: 2024-07-16T09:46:28.784 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: capacity : 536870912 2024-07-16T09:46:28.784 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: num_shard_bits : 4 2024-07-16T09:46:28.784 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: strict_capacity_limit : 0 2024-07-16T09:46:28.784 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: high_pri_pool_ratio: 0.000 2024-07-16T09:46:28.784 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: block_cache_compressed: (nil) 2024-07-16T09:46:28.784 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: persistent_cache: (nil) 2024-07-16T09:46:28.784 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: block_size: 4096 2024-07-16T09:46:28.784 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: block_size_deviation: 10 2024-07-16T09:46:28.785 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: block_restart_interval: 16 2024-07-16T09:46:28.785 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: index_block_restart_interval: 1 2024-07-16T09:46:28.785 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: metadata_block_size: 4096 2024-07-16T09:46:28.785 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: partition_filters: 0 2024-07-16T09:46:28.785 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: use_delta_encoding: 1 2024-07-16T09:46:28.785 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: filter_policy: rocksdb.BuiltinBloomFilter 2024-07-16T09:46:28.785 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: whole_key_filtering: 1 2024-07-16T09:46:28.785 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: verify_compression: 0 2024-07-16T09:46:28.785 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: read_amp_bytes_per_bit: 0 2024-07-16T09:46:28.785 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: format_version: 4 2024-07-16T09:46:28.785 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: enable_index_compression: 1 2024-07-16T09:46:28.785 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: block_align: 0 2024-07-16T09:46:28.786 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:28 smithi156 ceph-mon[29302]: rocksdb: Options.write_buffer_size: 33554432 2024-07-16T09:46:28.786 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:28 smithi156 ceph-mon[29302]: rocksdb: Options.max_write_buffer_number: 2 2024-07-16T09:46:28.786 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:28 smithi156 ceph-mon[29302]: rocksdb: Options.compression: NoCompression 2024-07-16T09:46:28.786 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:28 smithi156 ceph-mon[29302]: rocksdb: Options.bottommost_compression: Disabled 2024-07-16T09:46:28.786 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:28 smithi156 ceph-mon[29302]: rocksdb: Options.prefix_extractor: nullptr 2024-07-16T09:46:28.786 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:28 smithi156 ceph-mon[29302]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-07-16T09:46:28.786 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:28 smithi156 ceph-mon[29302]: rocksdb: Options.num_levels: 7 2024-07-16T09:46:28.786 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:28 smithi156 ceph-mon[29302]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-07-16T09:46:28.786 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:28 smithi156 ceph-mon[29302]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-07-16T09:46:28.786 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:28 smithi156 ceph-mon[29302]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-07-16T09:46:28.786 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:28 smithi156 ceph-mon[29302]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-07-16T09:46:28.787 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:28 smithi156 ceph-mon[29302]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-07-16T09:46:28.787 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:28 smithi156 ceph-mon[29302]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-07-16T09:46:28.787 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:28 smithi156 ceph-mon[29302]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-07-16T09:46:28.787 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:28 smithi156 ceph-mon[29302]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-07-16T09:46:28.787 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:28 smithi156 ceph-mon[29302]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-07-16T09:46:28.787 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:28 smithi156 ceph-mon[29302]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-07-16T09:46:28.787 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:28 smithi156 ceph-mon[29302]: rocksdb: Options.compression_opts.window_bits: -14 2024-07-16T09:46:28.787 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:28 smithi156 ceph-mon[29302]: rocksdb: Options.compression_opts.level: 32767 2024-07-16T09:46:28.787 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:28 smithi156 ceph-mon[29302]: rocksdb: Options.compression_opts.strategy: 0 2024-07-16T09:46:28.787 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:28 smithi156 ceph-mon[29302]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-07-16T09:46:28.787 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:28 smithi156 ceph-mon[29302]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-07-16T09:46:28.787 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:28 smithi156 ceph-mon[29302]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-07-16T09:46:28.787 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:28 smithi156 ceph-mon[29302]: rocksdb: Options.compression_opts.enabled: false 2024-07-16T09:46:28.787 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:28 smithi156 ceph-mon[29302]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-07-16T09:46:28.788 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:28 smithi156 ceph-mon[29302]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-07-16T09:46:28.788 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:28 smithi156 ceph-mon[29302]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-07-16T09:46:28.788 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:28 smithi156 ceph-mon[29302]: rocksdb: Options.target_file_size_base: 67108864 2024-07-16T09:46:28.788 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:28 smithi156 ceph-mon[29302]: rocksdb: Options.target_file_size_multiplier: 1 2024-07-16T09:46:28.788 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:28 smithi156 ceph-mon[29302]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-07-16T09:46:28.788 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:28 smithi156 ceph-mon[29302]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-07-16T09:46:28.788 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:28 smithi156 ceph-mon[29302]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-07-16T09:46:28.788 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:28 smithi156 ceph-mon[29302]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-07-16T09:46:28.788 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:28 smithi156 ceph-mon[29302]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-07-16T09:46:28.788 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:28 smithi156 ceph-mon[29302]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-07-16T09:46:28.788 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:28 smithi156 ceph-mon[29302]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-07-16T09:46:28.788 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:28 smithi156 ceph-mon[29302]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-07-16T09:46:28.788 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:28 smithi156 ceph-mon[29302]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-07-16T09:46:28.788 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:28 smithi156 ceph-mon[29302]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-07-16T09:46:28.789 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:28 smithi156 ceph-mon[29302]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-07-16T09:46:28.789 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:28 smithi156 ceph-mon[29302]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-07-16T09:46:28.789 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:28 smithi156 ceph-mon[29302]: rocksdb: Options.arena_block_size: 4194304 2024-07-16T09:46:28.789 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:28 smithi156 ceph-mon[29302]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-07-16T09:46:28.789 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:28 smithi156 ceph-mon[29302]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-07-16T09:46:28.789 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:28 smithi156 ceph-mon[29302]: rocksdb: Options.rate_limit_delay_max_milliseconds: 100 2024-07-16T09:46:28.789 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:28 smithi156 ceph-mon[29302]: rocksdb: Options.disable_auto_compactions: 0 2024-07-16T09:46:28.789 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:28 smithi156 ceph-mon[29302]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-07-16T09:46:28.789 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:28 smithi156 ceph-mon[29302]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-07-16T09:46:28.789 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:28 smithi156 ceph-mon[29302]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-07-16T09:46:28.789 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:28 smithi156 ceph-mon[29302]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-07-16T09:46:28.789 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:28 smithi156 ceph-mon[29302]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-07-16T09:46:28.789 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:28 smithi156 ceph-mon[29302]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-07-16T09:46:28.789 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:28 smithi156 ceph-mon[29302]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-07-16T09:46:28.790 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:28 smithi156 ceph-mon[29302]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-07-16T09:46:28.790 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:28 smithi156 ceph-mon[29302]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-07-16T09:46:28.790 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:28 smithi156 ceph-mon[29302]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-07-16T09:46:28.790 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:28 smithi156 ceph-mon[29302]: rocksdb: Options.table_properties_collectors: 2024-07-16T09:46:28.790 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:28 smithi156 ceph-mon[29302]: rocksdb: Options.inplace_update_support: 0 2024-07-16T09:46:28.790 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:28 smithi156 ceph-mon[29302]: rocksdb: Options.inplace_update_num_locks: 10000 2024-07-16T09:46:28.790 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:28 smithi156 ceph-mon[29302]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-07-16T09:46:28.790 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:28 smithi156 ceph-mon[29302]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-07-16T09:46:28.790 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:28 smithi156 ceph-mon[29302]: rocksdb: Options.memtable_huge_page_size: 0 2024-07-16T09:46:28.790 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:28 smithi156 ceph-mon[29302]: rocksdb: Options.bloom_locality: 0 2024-07-16T09:46:28.790 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:28 smithi156 ceph-mon[29302]: rocksdb: Options.max_successive_merges: 0 2024-07-16T09:46:28.790 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:28 smithi156 ceph-mon[29302]: rocksdb: Options.optimize_filters_for_hits: 0 2024-07-16T09:46:28.790 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:28 smithi156 ceph-mon[29302]: rocksdb: Options.paranoid_file_checks: 0 2024-07-16T09:46:28.791 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:28 smithi156 ceph-mon[29302]: rocksdb: Options.force_consistency_checks: 1 2024-07-16T09:46:28.791 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:28 smithi156 ceph-mon[29302]: rocksdb: Options.report_bg_io_stats: 0 2024-07-16T09:46:28.791 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:28 smithi156 ceph-mon[29302]: rocksdb: Options.ttl: 2592000 2024-07-16T09:46:28.791 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:28 smithi156 ceph-mon[29302]: rocksdb: Options.periodic_compaction_seconds: 0 2024-07-16T09:46:28.791 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:28 smithi156 ceph-mon[29302]: rocksdb: Options.enable_blob_files: false 2024-07-16T09:46:28.791 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:28 smithi156 ceph-mon[29302]: rocksdb: Options.min_blob_size: 0 2024-07-16T09:46:28.791 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:28 smithi156 ceph-mon[29302]: rocksdb: Options.blob_file_size: 268435456 2024-07-16T09:46:28.791 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:28 smithi156 ceph-mon[29302]: rocksdb: Options.blob_compression_type: NoCompression 2024-07-16T09:46:28.791 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:28 smithi156 ceph-mon[29302]: rocksdb: Options.enable_blob_garbage_collection: false 2024-07-16T09:46:28.791 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:28 smithi156 ceph-mon[29302]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-07-16T09:46:28.791 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:28 smithi156 ceph-mon[29302]: rocksdb: [db/version_set.cc:4764] Recovered from manifest file:/var/lib/ceph/mon/ceph-smithi156/store.db/MANIFEST-000009 succeeded,manifest_file_number is 9, next_file_number is 11, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-07-16T09:46:28.791 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:28 smithi156 ceph-mon[29302]: rocksdb: [db/version_set.cc:4779] Column family [default] (ID 0), log number is 5 2024-07-16T09:46:28.791 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:28 smithi156 ceph-mon[29302]: rocksdb: [db/version_set.cc:4082] Creating manifest 13 2024-07-16T09:46:28.792 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:28 smithi156 ceph-mon[29302]: rocksdb: EVENT_LOG_v1 {"time_micros": 1721123188767557, "job": 1, "event": "recovery_started", "wal_files": [10]} 2024-07-16T09:46:28.792 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:28 smithi156 ceph-mon[29302]: rocksdb: [db/db_impl/db_impl_open.cc:845] Recovering log #10 mode 2 2024-07-16T09:46:28.792 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:28 smithi156 ceph-mon[29302]: rocksdb: [table/block_based/filter_policy.cc:991] Using legacy Bloom filter with high (20) bits/key. Dramatic filter space and/or accuracy improvement is available with format_version>=5. 2024-07-16T09:46:28.792 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:28 smithi156 ceph-mon[29302]: rocksdb: EVENT_LOG_v1 {"time_micros": 1721123188768409, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 14, "file_size": 93404, "file_checksum": "", "file_checksum_func_name": "Unknown", "table_properties": {"data_size": 91516, "index_size": 248, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 709, "raw_key_size": 13576, "raw_average_key_size": 49, "raw_value_size": 84080, "raw_average_value_size": 307, "num_data_blocks": 11, "num_entries": 273, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "rocksdb.BuiltinBloomFilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; ", "creation_time": 1721123188, "oldest_key_time": 0, "file_creation_time": 0, "db_id": "6257f666-ec9f-4718-960d-4b6be1c33f87", "db_session_id": "718D3ZOX8YKGYD7KMYX8"}} 2024-07-16T09:46:28.792 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:28 smithi156 ceph-mon[29302]: rocksdb: [db/version_set.cc:4082] Creating manifest 15 2024-07-16T09:46:28.792 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:28 smithi156 ceph-mon[29302]: rocksdb: EVENT_LOG_v1 {"time_micros": 1721123188768767, "job": 1, "event": "recovery_finished"} 2024-07-16T09:46:28.792 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:28 smithi156 ceph-mon[29302]: rocksdb: [file/delete_scheduler.cc:69] Deleted file /var/lib/ceph/mon/ceph-smithi156/store.db/000010.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-07-16T09:46:28.792 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:28 smithi156 ceph-mon[29302]: rocksdb: [db/db_impl/db_impl_open.cc:1700] SstFileManager instance 0x55fb310a0700 2024-07-16T09:46:28.792 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:28 smithi156 ceph-mon[29302]: rocksdb: DB pointer 0x55fb31070000 2024-07-16T09:46:28.792 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:28 smithi156 ceph-mon[29302]: rocksdb: [db/db_impl/db_impl.cc:901] ------- DUMPING STATS ------- 2024-07-16T09:46:28.792 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:28 smithi156 ceph-mon[29302]: rocksdb: [db/db_impl/db_impl.cc:903] 2024-07-16T09:46:28.792 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: ** DB Stats ** 2024-07-16T09:46:28.792 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-07-16T09:46:28.793 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-07-16T09:46:28.793 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-07-16T09:46:28.793 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-07-16T09:46:28.793 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-07-16T09:46:28.793 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 MB, 0.00 MB/s 2024-07-16T09:46:28.793 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-07-16T09:46:28.793 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: 2024-07-16T09:46:28.793 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: ** Compaction Stats [default] ** 2024-07-16T09:46:28.793 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-07-16T09:46:28.793 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-07-16T09:46:28.793 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: L0 2/0 93.01 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 151.0 0.00 0.00 1 0.001 0 0 2024-07-16T09:46:28.793 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: Sum 2/0 93.01 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 151.0 0.00 0.00 1 0.001 0 0 2024-07-16T09:46:28.793 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 151.0 0.00 0.00 1 0.001 0 0 2024-07-16T09:46:28.794 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: 2024-07-16T09:46:28.794 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: ** Compaction Stats [default] ** 2024-07-16T09:46:28.794 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-07-16T09:46:28.794 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-07-16T09:46:28.794 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 151.0 0.00 0.00 1 0.001 0 0 2024-07-16T09:46:28.794 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-07-16T09:46:28.794 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-07-16T09:46:28.794 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-07-16T09:46:28.794 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-07-16T09:46:28.794 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-07-16T09:46:28.794 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: AddFile(Keys): cumulative 0, interval 0 2024-07-16T09:46:28.794 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: Cumulative compaction: 0.00 GB write, 27.50 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-07-16T09:46:28.794 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: Interval compaction: 0.00 GB write, 27.50 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-07-16T09:46:28.795 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-07-16T09:46:28.795 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: 2024-07-16T09:46:28.795 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: ** File Read Latency Histogram By Level [default] ** 2024-07-16T09:46:28.795 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: 2024-07-16T09:46:28.795 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: ** Compaction Stats [default] ** 2024-07-16T09:46:28.795 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-07-16T09:46:28.795 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-07-16T09:46:28.795 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: L0 2/0 93.01 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 151.0 0.00 0.00 1 0.001 0 0 2024-07-16T09:46:28.795 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: Sum 2/0 93.01 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 151.0 0.00 0.00 1 0.001 0 0 2024-07-16T09:46:28.795 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 2024-07-16T09:46:28.795 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: 2024-07-16T09:46:28.795 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: ** Compaction Stats [default] ** 2024-07-16T09:46:28.796 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-07-16T09:46:28.796 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-07-16T09:46:28.796 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 151.0 0.00 0.00 1 0.001 0 0 2024-07-16T09:46:28.796 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-07-16T09:46:28.796 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-07-16T09:46:28.796 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-07-16T09:46:28.796 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-07-16T09:46:28.796 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-07-16T09:46:28.796 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: AddFile(Keys): cumulative 0, interval 0 2024-07-16T09:46:28.796 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: Cumulative compaction: 0.00 GB write, 27.22 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-07-16T09:46:28.796 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-07-16T09:46:28.796 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-07-16T09:46:28.796 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: 2024-07-16T09:46:28.797 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: ** File Read Latency Histogram By Level [default] ** 2024-07-16T09:46:28.797 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:28 smithi156 ceph-mon[29302]: starting mon.smithi156 rank 0 at public addrs [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] at bind addrs [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] mon_data /var/lib/ceph/mon/ceph-smithi156 fsid 26c91c26-4358-11ef-bcad-c7b262605968 2024-07-16T09:46:28.797 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:28 smithi156 ceph-mon[29302]: mon.smithi156@-1(???) e1 preinit fsid 26c91c26-4358-11ef-bcad-c7b262605968 2024-07-16T09:46:28.797 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:28 smithi156 ceph-mon[29302]: mon.smithi156@-1(???).mds e1 new map 2024-07-16T09:46:28.797 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:28 smithi156 ceph-mon[29302]: mon.smithi156@-1(???).mds e1 print_map 2024-07-16T09:46:28.797 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: e1 2024-07-16T09:46:28.797 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-07-16T09:46:28.797 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-07-16T09:46:28.797 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: legacy client fscid: -1 2024-07-16T09:46:28.797 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: 2024-07-16T09:46:28.797 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: No filesystems configured 2024-07-16T09:46:28.797 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:28 smithi156 ceph-mon[29302]: mon.smithi156@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-07-16T09:46:28.797 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:28 smithi156 ceph-mon[29302]: mon.smithi156@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-07-16T09:46:28.798 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:28 smithi156 ceph-mon[29302]: mon.smithi156@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-07-16T09:46:28.798 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:28 smithi156 ceph-mon[29302]: mon.smithi156@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-07-16T09:46:28.798 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:28 smithi156 ceph-mon[29302]: mon.smithi156@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-07-16T09:46:28.848 INFO:teuthology.orchestra.run.smithi156.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-07-16T09:46:29.074 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:28 smithi156 ceph-mon[29302]: mon.smithi156 is new leader, mons smithi156 in quorum (ranks 0) 2024-07-16T09:46:29.075 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:28 smithi156 ceph-mon[29302]: monmap e1: 1 mons at {smithi156=[v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-07-16T09:46:29.075 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:28 smithi156 ceph-mon[29302]: fsmap 2024-07-16T09:46:29.075 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:28 smithi156 ceph-mon[29302]: osdmap e1: 0 total, 0 up, 0 in 2024-07-16T09:46:29.075 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:28 smithi156 ceph-mon[29302]: mgrmap e1: no daemons active 2024-07-16T09:46:29.075 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:28 smithi156 bash[29282]: 6471c8acff4002d36c1d1a5a5b009315b53bee9c4642cd88cbfec151eed1ac2b 2024-07-16T09:46:29.075 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:28 smithi156 systemd[1]: Started Ceph mon.smithi156 for 26c91c26-4358-11ef-bcad-c7b262605968. 2024-07-16T09:46:29.846 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:29.246+0000 7f08f6df4640 1 Processor -- start 2024-07-16T09:46:29.846 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:29.246+0000 7f08f6df4640 1 -- start start 2024-07-16T09:46:29.847 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:29.246+0000 7f08f6df4640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f08f0100ce0 0x7f08f01010f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:46:29.847 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:29.246+0000 7f08f6df4640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08f0101630 con 0x7f08f0100ce0 2024-07-16T09:46:29.847 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:29.247+0000 7f08f4b90640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f08f0100ce0 0x7f08f01010f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:46:29.847 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:29.247+0000 7f08f4b90640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f08f0100ce0 0x7f08f01010f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:42764/0 (socket says 172.21.15.156:42764) 2024-07-16T09:46:29.847 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:29.247+0000 7f08f4b90640 1 -- 172.21.15.156:0/1380427435 learned_addr learned my addr 172.21.15.156:0/1380427435 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T09:46:29.847 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:29.247+0000 7f08f4b90640 1 -- 172.21.15.156:0/1380427435 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08f0101770 con 0x7f08f0100ce0 2024-07-16T09:46:29.847 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:29.249+0000 7f08f4b90640 1 --2- 172.21.15.156:0/1380427435 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f08f0100ce0 0x7f08f01010f0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f08e000baf0 tx=0x7f08e000b460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2aee9fa284aafbcf server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:46:29.847 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:29.249+0000 7f08ef7fe640 1 -- 172.21.15.156:0/1380427435 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f08e00327a0 con 0x7f08f0100ce0 2024-07-16T09:46:29.847 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:29.249+0000 7f08ef7fe640 1 -- 172.21.15.156:0/1380427435 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f08e0032d90 con 0x7f08f0100ce0 2024-07-16T09:46:29.847 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:29.249+0000 7f08ef7fe640 1 -- 172.21.15.156:0/1380427435 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f08e0031e70 con 0x7f08f0100ce0 2024-07-16T09:46:29.847 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:29.250+0000 7f08f6df4640 1 -- 172.21.15.156:0/1380427435 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f08f0100ce0 msgr2=0x7f08f01010f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:46:29.847 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:29.250+0000 7f08f6df4640 1 --2- 172.21.15.156:0/1380427435 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f08f0100ce0 0x7f08f01010f0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f08e000baf0 tx=0x7f08e000b460 comp rx=0 tx=0).stop 2024-07-16T09:46:29.847 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:29.250+0000 7f08f6df4640 1 -- 172.21.15.156:0/1380427435 shutdown_connections 2024-07-16T09:46:29.848 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:29.250+0000 7f08f6df4640 1 --2- 172.21.15.156:0/1380427435 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f08f0100ce0 0x7f08f01010f0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:46:29.848 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:29.250+0000 7f08f6df4640 1 -- 172.21.15.156:0/1380427435 >> 172.21.15.156:0/1380427435 conn(0x7f08f00fc520 msgr2=0x7f08f00fe930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:46:29.848 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:29.250+0000 7f08f6df4640 1 -- 172.21.15.156:0/1380427435 shutdown_connections 2024-07-16T09:46:29.848 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:29.250+0000 7f08f6df4640 1 -- 172.21.15.156:0/1380427435 wait complete. 2024-07-16T09:46:29.848 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:29.250+0000 7f08f6df4640 1 Processor -- start 2024-07-16T09:46:29.848 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:29.250+0000 7f08f6df4640 1 -- start start 2024-07-16T09:46:29.848 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:29.250+0000 7f08f6df4640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f08f0100ce0 0x7f08f010e900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:46:29.848 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:29.250+0000 7f08f6df4640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08f010ee40 con 0x7f08f0100ce0 2024-07-16T09:46:29.848 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:29.251+0000 7f08f4b90640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f08f0100ce0 0x7f08f010e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:46:29.848 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:29.251+0000 7f08f4b90640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f08f0100ce0 0x7f08f010e900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:42768/0 (socket says 172.21.15.156:42768) 2024-07-16T09:46:29.848 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:29.251+0000 7f08f4b90640 1 -- 172.21.15.156:0/1027312682 learned_addr learned my addr 172.21.15.156:0/1027312682 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T09:46:29.848 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:29.251+0000 7f08f4b90640 1 -- 172.21.15.156:0/1027312682 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08e000b700 con 0x7f08f0100ce0 2024-07-16T09:46:29.848 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:29.251+0000 7f08f4b90640 1 --2- 172.21.15.156:0/1027312682 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f08f0100ce0 0x7f08f010e900 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f08e000b2c0 tx=0x7f08e000b460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:46:29.849 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:29.251+0000 7f08ed7fa640 1 -- 172.21.15.156:0/1027312682 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f08e0032a80 con 0x7f08f0100ce0 2024-07-16T09:46:29.849 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:29.251+0000 7f08ed7fa640 1 -- 172.21.15.156:0/1027312682 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f08e0032be0 con 0x7f08f0100ce0 2024-07-16T09:46:29.849 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:29.251+0000 7f08f6df4640 1 -- 172.21.15.156:0/1027312682 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08f010f040 con 0x7f08f0100ce0 2024-07-16T09:46:29.849 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:29.251+0000 7f08ed7fa640 1 -- 172.21.15.156:0/1027312682 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f08e0039750 con 0x7f08f0100ce0 2024-07-16T09:46:29.849 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:29.251+0000 7f08f6df4640 1 -- 172.21.15.156:0/1027312682 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08f010f450 con 0x7f08f0100ce0 2024-07-16T09:46:29.849 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:29.251+0000 7f08ed7fa640 1 -- 172.21.15.156:0/1027312682 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 1) v1 ==== 529+0+0 (secure 0 0 0) 0x7f08e0039ca0 con 0x7f08f0100ce0 2024-07-16T09:46:29.849 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:29.252+0000 7f08ed7fa640 1 -- 172.21.15.156:0/1027312682 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f08e00463f0 con 0x7f08f0100ce0 2024-07-16T09:46:29.849 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:29.252+0000 7f08f6df4640 1 -- 172.21.15.156:0/1027312682 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08f004b530 con 0x7f08f0100ce0 2024-07-16T09:46:29.849 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:29.254+0000 7f08ed7fa640 1 -- 172.21.15.156:0/1027312682 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71547 (secure 0 0 0) 0x7f08e0036020 con 0x7f08f0100ce0 2024-07-16T09:46:29.849 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:29.312+0000 7f08f6df4640 1 -- 172.21.15.156:0/1027312682 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) v1 -- 0x7f08f010fbd0 con 0x7f08f0100ce0 2024-07-16T09:46:29.849 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:29.314+0000 7f08ed7fa640 1 -- 172.21.15.156:0/1027312682 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) v1 ==== 124+0+0 (secure 0 0 0) 0x7f08e003bb20 con 0x7f08f0100ce0 2024-07-16T09:46:29.849 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:29.315+0000 7f08f6df4640 1 -- 172.21.15.156:0/1027312682 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f08f0100ce0 msgr2=0x7f08f010e900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:46:29.849 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:29.315+0000 7f08f6df4640 1 --2- 172.21.15.156:0/1027312682 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f08f0100ce0 0x7f08f010e900 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f08e000b2c0 tx=0x7f08e000b460 comp rx=0 tx=0).stop 2024-07-16T09:46:29.850 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:29.315+0000 7f08f6df4640 1 -- 172.21.15.156:0/1027312682 shutdown_connections 2024-07-16T09:46:29.850 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:29.315+0000 7f08f6df4640 1 --2- 172.21.15.156:0/1027312682 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f08f0100ce0 0x7f08f010e900 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:46:29.850 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:29.315+0000 7f08f6df4640 1 -- 172.21.15.156:0/1027312682 >> 172.21.15.156:0/1027312682 conn(0x7f08f00fc520 msgr2=0x7f08f01051a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:46:29.850 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:29.315+0000 7f08f6df4640 1 -- 172.21.15.156:0/1027312682 shutdown_connections 2024-07-16T09:46:29.850 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:29.315+0000 7f08f6df4640 1 -- 172.21.15.156:0/1027312682 wait complete. 2024-07-16T09:46:29.850 INFO:teuthology.orchestra.run.smithi156.stdout:Wrote config to /etc/ceph/ceph.conf 2024-07-16T09:46:29.850 INFO:teuthology.orchestra.run.smithi156.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-07-16T09:46:29.850 INFO:teuthology.orchestra.run.smithi156.stdout:Creating mgr... 2024-07-16T09:46:29.850 INFO:teuthology.orchestra.run.smithi156.stdout:Verifying port 9283 ... 2024-07-16T09:46:30.025 INFO:teuthology.orchestra.run.smithi156.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-26c91c26-4358-11ef-bcad-c7b262605968@mgr.smithi156.zkvylv 2024-07-16T09:46:30.025 INFO:teuthology.orchestra.run.smithi156.stdout:systemctl: stderr Failed to reset failed state of unit ceph-26c91c26-4358-11ef-bcad-c7b262605968@mgr.smithi156.zkvylv.service: Unit ceph-26c91c26-4358-11ef-bcad-c7b262605968@mgr.smithi156.zkvylv.service not loaded. 2024-07-16T09:46:30.171 INFO:teuthology.orchestra.run.smithi156.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-26c91c26-4358-11ef-bcad-c7b262605968.target.wants/ceph-26c91c26-4358-11ef-bcad-c7b262605968@mgr.smithi156.zkvylv.service → /etc/systemd/system/ceph-26c91c26-4358-11ef-bcad-c7b262605968@.service. 2024-07-16T09:46:30.611 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:30 smithi156 ceph-mon[29302]: from='client.? 172.21.15.156:0/1027312682' entity='client.admin' 2024-07-16T09:46:30.734 INFO:teuthology.orchestra.run.smithi156.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-07-16T09:46:30.734 INFO:teuthology.orchestra.run.smithi156.stdout:systemctl: stdout disabled 2024-07-16T09:46:30.742 INFO:teuthology.orchestra.run.smithi156.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-07-16T09:46:30.742 INFO:teuthology.orchestra.run.smithi156.stdout:systemctl: stdout inactive 2024-07-16T09:46:30.742 INFO:teuthology.orchestra.run.smithi156.stdout:firewalld.service is not enabled 2024-07-16T09:46:30.742 INFO:teuthology.orchestra.run.smithi156.stdout:Not possible to enable service . firewalld.service is not available 2024-07-16T09:46:30.751 INFO:teuthology.orchestra.run.smithi156.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-07-16T09:46:30.751 INFO:teuthology.orchestra.run.smithi156.stdout:systemctl: stdout disabled 2024-07-16T09:46:30.759 INFO:teuthology.orchestra.run.smithi156.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-07-16T09:46:30.759 INFO:teuthology.orchestra.run.smithi156.stdout:systemctl: stdout inactive 2024-07-16T09:46:30.759 INFO:teuthology.orchestra.run.smithi156.stdout:firewalld.service is not enabled 2024-07-16T09:46:30.759 INFO:teuthology.orchestra.run.smithi156.stdout:Not possible to open ports <[9283]>. firewalld.service is not available 2024-07-16T09:46:30.759 INFO:teuthology.orchestra.run.smithi156.stdout:Waiting for mgr to start... 2024-07-16T09:46:30.759 INFO:teuthology.orchestra.run.smithi156.stdout:Waiting for mgr... 2024-07-16T09:46:31.424 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:31 smithi156 ceph-mon[29302]: from='client.? 172.21.15.156:0/3306804047' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-07-16T09:46:31.781 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout 2024-07-16T09:46:31.784 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout { 2024-07-16T09:46:31.784 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout "fsid": "26c91c26-4358-11ef-bcad-c7b262605968", 2024-07-16T09:46:31.784 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout "health": { 2024-07-16T09:46:31.784 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-07-16T09:46:31.784 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-07-16T09:46:31.784 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-07-16T09:46:31.784 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout }, 2024-07-16T09:46:31.784 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-07-16T09:46:31.784 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-07-16T09:46:31.784 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout 0 2024-07-16T09:46:31.784 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout ], 2024-07-16T09:46:31.785 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-07-16T09:46:31.785 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout "smithi156" 2024-07-16T09:46:31.785 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout ], 2024-07-16T09:46:31.785 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-07-16T09:46:31.785 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout "monmap": { 2024-07-16T09:46:31.785 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-07-16T09:46:31.785 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "quincy", 2024-07-16T09:46:31.785 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-07-16T09:46:31.785 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout }, 2024-07-16T09:46:31.785 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-07-16T09:46:31.785 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-07-16T09:46:31.785 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-07-16T09:46:31.785 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-07-16T09:46:31.786 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-07-16T09:46:31.792 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-07-16T09:46:31.793 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-07-16T09:46:31.793 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-07-16T09:46:31.793 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout }, 2024-07-16T09:46:31.793 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-07-16T09:46:31.793 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-07-16T09:46:31.793 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-07-16T09:46:31.793 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-07-16T09:46:31.793 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-07-16T09:46:31.793 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-07-16T09:46:31.793 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-07-16T09:46:31.793 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-07-16T09:46:31.793 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-07-16T09:46:31.794 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout }, 2024-07-16T09:46:31.794 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-07-16T09:46:31.794 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-07-16T09:46:31.794 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-07-16T09:46:31.794 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-07-16T09:46:31.794 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout }, 2024-07-16T09:46:31.794 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-07-16T09:46:31.794 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout "available": false, 2024-07-16T09:46:31.794 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-07-16T09:46:31.794 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout "modules": [ 2024-07-16T09:46:31.794 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout "iostat", 2024-07-16T09:46:31.794 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout "nfs", 2024-07-16T09:46:31.794 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout "restful" 2024-07-16T09:46:31.795 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout ], 2024-07-16T09:46:31.795 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout "services": {} 2024-07-16T09:46:31.795 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout }, 2024-07-16T09:46:31.795 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-07-16T09:46:31.795 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-07-16T09:46:31.795 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout "modified": "2024-07-16T09:46:23.834404+0000", 2024-07-16T09:46:31.795 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout "services": {} 2024-07-16T09:46:31.795 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout }, 2024-07-16T09:46:31.795 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-07-16T09:46:31.795 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout } 2024-07-16T09:46:31.795 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:31.114+0000 7fb8e953b640 1 Processor -- start 2024-07-16T09:46:31.795 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:31.114+0000 7fb8e953b640 1 -- start start 2024-07-16T09:46:31.795 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:31.115+0000 7fb8e953b640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb8e4075ba0 0x7fb8e4104a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:46:31.795 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:31.115+0000 7fb8e953b640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8e4104f70 con 0x7fb8e4075ba0 2024-07-16T09:46:31.796 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:31.115+0000 7fb8e2ffd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb8e4075ba0 0x7fb8e4104a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:46:31.796 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:31.115+0000 7fb8e2ffd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb8e4075ba0 0x7fb8e4104a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:42804/0 (socket says 172.21.15.156:42804) 2024-07-16T09:46:31.796 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:31.115+0000 7fb8e2ffd640 1 -- 172.21.15.156:0/3809861563 learned_addr learned my addr 172.21.15.156:0/3809861563 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T09:46:31.796 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:31.115+0000 7fb8e2ffd640 1 -- 172.21.15.156:0/3809861563 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8e41050b0 con 0x7fb8e4075ba0 2024-07-16T09:46:31.796 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:31.116+0000 7fb8e2ffd640 1 --2- 172.21.15.156:0/3809861563 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb8e4075ba0 0x7fb8e4104a30 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fb8d4009840 tx=0x7fb8d40091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bca892f9cbf338f9 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:46:31.796 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:31.116+0000 7fb8e1ffb640 1 -- 172.21.15.156:0/3809861563 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb8d4032440 con 0x7fb8e4075ba0 2024-07-16T09:46:31.796 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:31.116+0000 7fb8e1ffb640 1 -- 172.21.15.156:0/3809861563 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7fb8d4032a30 con 0x7fb8e4075ba0 2024-07-16T09:46:31.796 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:31.116+0000 7fb8e1ffb640 1 -- 172.21.15.156:0/3809861563 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb8d4031ad0 con 0x7fb8e4075ba0 2024-07-16T09:46:31.796 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:31.117+0000 7fb8e953b640 1 -- 172.21.15.156:0/3809861563 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb8e4075ba0 msgr2=0x7fb8e4104a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:46:31.796 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:31.117+0000 7fb8e953b640 1 --2- 172.21.15.156:0/3809861563 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb8e4075ba0 0x7fb8e4104a30 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fb8d4009840 tx=0x7fb8d40091d0 comp rx=0 tx=0).stop 2024-07-16T09:46:31.796 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:31.117+0000 7fb8e953b640 1 -- 172.21.15.156:0/3809861563 shutdown_connections 2024-07-16T09:46:31.796 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:31.117+0000 7fb8e953b640 1 --2- 172.21.15.156:0/3809861563 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb8e4075ba0 0x7fb8e4104a30 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:46:31.796 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:31.117+0000 7fb8e953b640 1 -- 172.21.15.156:0/3809861563 >> 172.21.15.156:0/3809861563 conn(0x7fb8e40fc520 msgr2=0x7fb8e40fe930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:46:31.797 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:31.117+0000 7fb8e953b640 1 -- 172.21.15.156:0/3809861563 shutdown_connections 2024-07-16T09:46:31.797 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:31.117+0000 7fb8e953b640 1 -- 172.21.15.156:0/3809861563 wait complete. 2024-07-16T09:46:31.797 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:31.118+0000 7fb8e953b640 1 Processor -- start 2024-07-16T09:46:31.797 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:31.118+0000 7fb8e953b640 1 -- start start 2024-07-16T09:46:31.797 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:31.119+0000 7fb8e953b640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb8e4075ba0 0x7fb8e4110a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:46:31.797 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:31.119+0000 7fb8e953b640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8e4110fc0 con 0x7fb8e4075ba0 2024-07-16T09:46:31.797 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:31.119+0000 7fb8e2ffd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb8e4075ba0 0x7fb8e4110a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:46:31.797 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:31.119+0000 7fb8e2ffd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb8e4075ba0 0x7fb8e4110a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:42810/0 (socket says 172.21.15.156:42810) 2024-07-16T09:46:31.797 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:31.119+0000 7fb8e2ffd640 1 -- 172.21.15.156:0/3306804047 learned_addr learned my addr 172.21.15.156:0/3306804047 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T09:46:31.797 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:31.119+0000 7fb8e2ffd640 1 -- 172.21.15.156:0/3306804047 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8d4009450 con 0x7fb8e4075ba0 2024-07-16T09:46:31.797 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:31.119+0000 7fb8e2ffd640 1 --2- 172.21.15.156:0/3306804047 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb8e4075ba0 0x7fb8e4110a80 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fb8d4031b90 tx=0x7fb8d4008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:46:31.797 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:31.119+0000 7fb8cb7fe640 1 -- 172.21.15.156:0/3306804047 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb8d40328b0 con 0x7fb8e4075ba0 2024-07-16T09:46:31.797 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:31.119+0000 7fb8cb7fe640 1 -- 172.21.15.156:0/3306804047 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7fb8d403d8e0 con 0x7fb8e4075ba0 2024-07-16T09:46:31.798 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:31.119+0000 7fb8cb7fe640 1 -- 172.21.15.156:0/3306804047 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb8d403ba20 con 0x7fb8e4075ba0 2024-07-16T09:46:31.798 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:31.119+0000 7fb8e953b640 1 -- 172.21.15.156:0/3306804047 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8e41111c0 con 0x7fb8e4075ba0 2024-07-16T09:46:31.798 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:31.119+0000 7fb8e953b640 1 -- 172.21.15.156:0/3306804047 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8e4111630 con 0x7fb8e4075ba0 2024-07-16T09:46:31.798 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:31.120+0000 7fb8cb7fe640 1 -- 172.21.15.156:0/3306804047 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 1) v1 ==== 529+0+0 (secure 0 0 0) 0x7fb8d4039070 con 0x7fb8e4075ba0 2024-07-16T09:46:31.798 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:31.120+0000 7fb8cb7fe640 1 -- 172.21.15.156:0/3306804047 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7fb8d4046930 con 0x7fb8e4075ba0 2024-07-16T09:46:31.798 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:31.120+0000 7fb8e953b640 1 -- 172.21.15.156:0/3306804047 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8e404b530 con 0x7fb8e4075ba0 2024-07-16T09:46:31.798 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:31.122+0000 7fb8cb7fe640 1 -- 172.21.15.156:0/3306804047 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71547 (secure 0 0 0) 0x7fb8d4065790 con 0x7fb8e4075ba0 2024-07-16T09:46:31.798 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:31.185+0000 7fb8e953b640 1 -- 172.21.15.156:0/3306804047 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7fb8e4111db0 con 0x7fb8e4075ba0 2024-07-16T09:46:31.798 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:31.186+0000 7fb8cb7fe640 1 -- 172.21.15.156:0/3306804047 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1248 (secure 0 0 0) 0x7fb8d4054020 con 0x7fb8e4075ba0 2024-07-16T09:46:31.798 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:31.187+0000 7fb8e953b640 1 -- 172.21.15.156:0/3306804047 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb8e4075ba0 msgr2=0x7fb8e4110a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:46:31.798 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:31.187+0000 7fb8e953b640 1 --2- 172.21.15.156:0/3306804047 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb8e4075ba0 0x7fb8e4110a80 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fb8d4031b90 tx=0x7fb8d4008040 comp rx=0 tx=0).stop 2024-07-16T09:46:31.798 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:31.187+0000 7fb8e953b640 1 -- 172.21.15.156:0/3306804047 shutdown_connections 2024-07-16T09:46:31.798 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:31.187+0000 7fb8e953b640 1 --2- 172.21.15.156:0/3306804047 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb8e4075ba0 0x7fb8e4110a80 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:46:31.799 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:31.187+0000 7fb8e953b640 1 -- 172.21.15.156:0/3306804047 >> 172.21.15.156:0/3306804047 conn(0x7fb8e40fc520 msgr2=0x7fb8e40fcf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:46:31.799 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:31.187+0000 7fb8e953b640 1 -- 172.21.15.156:0/3306804047 shutdown_connections 2024-07-16T09:46:31.799 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:31.187+0000 7fb8e953b640 1 -- 172.21.15.156:0/3306804047 wait complete. 2024-07-16T09:46:31.799 INFO:teuthology.orchestra.run.smithi156.stdout:mgr not available, waiting (1/15)... 2024-07-16T09:46:34.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:34 smithi156 ceph-mon[29302]: from='client.? 172.21.15.156:0/2632400882' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-07-16T09:46:34.913 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout 2024-07-16T09:46:34.913 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout { 2024-07-16T09:46:34.914 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout "fsid": "26c91c26-4358-11ef-bcad-c7b262605968", 2024-07-16T09:46:34.914 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout "health": { 2024-07-16T09:46:34.914 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-07-16T09:46:34.914 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-07-16T09:46:34.914 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-07-16T09:46:34.914 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout }, 2024-07-16T09:46:34.914 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-07-16T09:46:34.914 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-07-16T09:46:34.914 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout 0 2024-07-16T09:46:34.914 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout ], 2024-07-16T09:46:34.914 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-07-16T09:46:34.914 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout "smithi156" 2024-07-16T09:46:34.915 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout ], 2024-07-16T09:46:34.915 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout "quorum_age": 5, 2024-07-16T09:46:34.915 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout "monmap": { 2024-07-16T09:46:34.915 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-07-16T09:46:34.915 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "quincy", 2024-07-16T09:46:34.915 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-07-16T09:46:34.915 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout }, 2024-07-16T09:46:34.915 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-07-16T09:46:34.915 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-07-16T09:46:34.915 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-07-16T09:46:34.915 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-07-16T09:46:34.915 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-07-16T09:46:34.916 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-07-16T09:46:34.916 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-07-16T09:46:34.916 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-07-16T09:46:34.916 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout }, 2024-07-16T09:46:34.916 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-07-16T09:46:34.916 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-07-16T09:46:34.916 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-07-16T09:46:34.916 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-07-16T09:46:34.916 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-07-16T09:46:34.916 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-07-16T09:46:34.916 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-07-16T09:46:34.916 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-07-16T09:46:34.916 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-07-16T09:46:34.917 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout }, 2024-07-16T09:46:34.917 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-07-16T09:46:34.917 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-07-16T09:46:34.917 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-07-16T09:46:34.917 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-07-16T09:46:34.917 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout }, 2024-07-16T09:46:34.917 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-07-16T09:46:34.917 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout "available": false, 2024-07-16T09:46:34.917 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-07-16T09:46:34.917 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout "modules": [ 2024-07-16T09:46:34.917 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout "iostat", 2024-07-16T09:46:34.918 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout "nfs", 2024-07-16T09:46:34.918 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout "restful" 2024-07-16T09:46:34.918 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout ], 2024-07-16T09:46:34.918 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout "services": {} 2024-07-16T09:46:34.918 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout }, 2024-07-16T09:46:34.918 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-07-16T09:46:34.918 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-07-16T09:46:34.918 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout "modified": "2024-07-16T09:46:23.834404+0000", 2024-07-16T09:46:34.918 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout "services": {} 2024-07-16T09:46:34.918 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout }, 2024-07-16T09:46:34.918 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-07-16T09:46:34.918 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout } 2024-07-16T09:46:34.918 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:34.221+0000 7f61811e3640 1 Processor -- start 2024-07-16T09:46:34.919 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:34.221+0000 7f61811e3640 1 -- start start 2024-07-16T09:46:34.919 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:34.221+0000 7f61811e3640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f617c075ba0 0x7f617c106e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:46:34.919 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:34.221+0000 7f61811e3640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f617c102820 con 0x7f617c075ba0 2024-07-16T09:46:34.919 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:34.221+0000 7f617ad9d640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f617c075ba0 0x7f617c106e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:46:34.919 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:34.221+0000 7f617ad9d640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f617c075ba0 0x7f617c106e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:34412/0 (socket says 172.21.15.156:34412) 2024-07-16T09:46:34.919 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:34.221+0000 7f617ad9d640 1 -- 172.21.15.156:0/1384672409 learned_addr learned my addr 172.21.15.156:0/1384672409 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T09:46:34.919 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:34.222+0000 7f617ad9d640 1 -- 172.21.15.156:0/1384672409 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f617c107370 con 0x7f617c075ba0 2024-07-16T09:46:34.919 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:34.222+0000 7f617ad9d640 1 --2- 172.21.15.156:0/1384672409 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f617c075ba0 0x7f617c106e30 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f616c009840 tx=0x7f616c0091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5a23eb8870258ed server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:46:34.919 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:34.223+0000 7f6179d9b640 1 -- 172.21.15.156:0/1384672409 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f616c032440 con 0x7f617c075ba0 2024-07-16T09:46:34.919 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:34.223+0000 7f6179d9b640 1 -- 172.21.15.156:0/1384672409 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f616c032a30 con 0x7f617c075ba0 2024-07-16T09:46:34.919 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:34.223+0000 7f6179d9b640 1 -- 172.21.15.156:0/1384672409 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f616c031ad0 con 0x7f617c075ba0 2024-07-16T09:46:34.919 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:34.223+0000 7f61811e3640 1 -- 172.21.15.156:0/1384672409 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f617c075ba0 msgr2=0x7f617c106e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:46:34.920 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:34.223+0000 7f61811e3640 1 --2- 172.21.15.156:0/1384672409 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f617c075ba0 0x7f617c106e30 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f616c009840 tx=0x7f616c0091d0 comp rx=0 tx=0).stop 2024-07-16T09:46:34.920 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:34.223+0000 7f61811e3640 1 -- 172.21.15.156:0/1384672409 shutdown_connections 2024-07-16T09:46:34.920 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:34.223+0000 7f61811e3640 1 --2- 172.21.15.156:0/1384672409 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f617c075ba0 0x7f617c106e30 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:46:34.920 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:34.223+0000 7f61811e3640 1 -- 172.21.15.156:0/1384672409 >> 172.21.15.156:0/1384672409 conn(0x7f617c0fc520 msgr2=0x7f617c0fe930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:46:34.920 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:34.223+0000 7f61811e3640 1 -- 172.21.15.156:0/1384672409 shutdown_connections 2024-07-16T09:46:34.920 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:34.223+0000 7f61811e3640 1 -- 172.21.15.156:0/1384672409 wait complete. 2024-07-16T09:46:34.920 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:34.224+0000 7f61811e3640 1 Processor -- start 2024-07-16T09:46:34.920 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:34.224+0000 7f61811e3640 1 -- start start 2024-07-16T09:46:34.920 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:34.224+0000 7f61811e3640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f617c075ba0 0x7f617c112c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:46:34.920 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:34.224+0000 7f61811e3640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f617c1131b0 con 0x7f617c075ba0 2024-07-16T09:46:34.920 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:34.224+0000 7f617ad9d640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f617c075ba0 0x7f617c112c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:46:34.920 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:34.224+0000 7f617ad9d640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f617c075ba0 0x7f617c112c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:34428/0 (socket says 172.21.15.156:34428) 2024-07-16T09:46:34.920 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:34.224+0000 7f617ad9d640 1 -- 172.21.15.156:0/2632400882 learned_addr learned my addr 172.21.15.156:0/2632400882 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T09:46:34.920 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:34.225+0000 7f617ad9d640 1 -- 172.21.15.156:0/2632400882 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f616c009450 con 0x7f617c075ba0 2024-07-16T09:46:34.921 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:34.225+0000 7f617ad9d640 1 --2- 172.21.15.156:0/2632400882 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f617c075ba0 0x7f617c112c70 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f616c006f00 tx=0x7f616c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:46:34.921 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:34.225+0000 7f61637fe640 1 -- 172.21.15.156:0/2632400882 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f616c03d450 con 0x7f617c075ba0 2024-07-16T09:46:34.921 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:34.225+0000 7f61637fe640 1 -- 172.21.15.156:0/2632400882 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f616c03da40 con 0x7f617c075ba0 2024-07-16T09:46:34.921 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:34.225+0000 7f61811e3640 1 -- 172.21.15.156:0/2632400882 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f617c1133b0 con 0x7f617c075ba0 2024-07-16T09:46:34.921 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:34.225+0000 7f61637fe640 1 -- 172.21.15.156:0/2632400882 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f616c03bb10 con 0x7f617c075ba0 2024-07-16T09:46:34.921 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:34.225+0000 7f61811e3640 1 -- 172.21.15.156:0/2632400882 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f617c113820 con 0x7f617c075ba0 2024-07-16T09:46:34.921 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:34.225+0000 7f61637fe640 1 -- 172.21.15.156:0/2632400882 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 1) v1 ==== 529+0+0 (secure 0 0 0) 0x7f616c039070 con 0x7f617c075ba0 2024-07-16T09:46:34.921 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:34.225+0000 7f61637fe640 1 -- 172.21.15.156:0/2632400882 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f616c046930 con 0x7f617c075ba0 2024-07-16T09:46:34.921 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:34.226+0000 7f61811e3640 1 -- 172.21.15.156:0/2632400882 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f617c04b530 con 0x7f617c075ba0 2024-07-16T09:46:34.921 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:34.228+0000 7f61637fe640 1 -- 172.21.15.156:0/2632400882 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71547 (secure 0 0 0) 0x7f617c113820 con 0x7f617c075ba0 2024-07-16T09:46:34.921 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:34.291+0000 7f61811e3640 1 -- 172.21.15.156:0/2632400882 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f617c113f40 con 0x7f617c075ba0 2024-07-16T09:46:34.921 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:34.292+0000 7f61637fe640 1 -- 172.21.15.156:0/2632400882 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1248 (secure 0 0 0) 0x7f616c041020 con 0x7f617c075ba0 2024-07-16T09:46:34.922 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:34.293+0000 7f61811e3640 1 -- 172.21.15.156:0/2632400882 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f617c075ba0 msgr2=0x7f617c112c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:46:34.922 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:34.293+0000 7f61811e3640 1 --2- 172.21.15.156:0/2632400882 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f617c075ba0 0x7f617c112c70 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f616c006f00 tx=0x7f616c008040 comp rx=0 tx=0).stop 2024-07-16T09:46:34.922 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:34.293+0000 7f61811e3640 1 -- 172.21.15.156:0/2632400882 shutdown_connections 2024-07-16T09:46:34.922 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:34.293+0000 7f61811e3640 1 --2- 172.21.15.156:0/2632400882 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f617c075ba0 0x7f617c112c70 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:46:34.922 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:34.293+0000 7f61811e3640 1 -- 172.21.15.156:0/2632400882 >> 172.21.15.156:0/2632400882 conn(0x7f617c0fc520 msgr2=0x7f617c0fcfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:46:34.922 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:34.293+0000 7f61811e3640 1 -- 172.21.15.156:0/2632400882 shutdown_connections 2024-07-16T09:46:34.922 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:34.293+0000 7f61811e3640 1 -- 172.21.15.156:0/2632400882 wait complete. 2024-07-16T09:46:34.922 INFO:teuthology.orchestra.run.smithi156.stdout:mgr not available, waiting (2/15)... 2024-07-16T09:46:35.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:35 smithi156 ceph-mon[29302]: Activating manager daemon smithi156.zkvylv 2024-07-16T09:46:35.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:35 smithi156 ceph-mon[29302]: mgrmap e2: smithi156.zkvylv(active, starting, since 0.00281585s) 2024-07-16T09:46:35.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:35 smithi156 ceph-mon[29302]: from='mgr.14100 172.21.15.156:0/652053769' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-07-16T09:46:35.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:35 smithi156 ceph-mon[29302]: from='mgr.14100 172.21.15.156:0/652053769' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-07-16T09:46:35.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:35 smithi156 ceph-mon[29302]: from='mgr.14100 172.21.15.156:0/652053769' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-07-16T09:46:35.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:35 smithi156 ceph-mon[29302]: from='mgr.14100 172.21.15.156:0/652053769' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "mon metadata", "id": "smithi156"}]: dispatch 2024-07-16T09:46:35.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:35 smithi156 ceph-mon[29302]: from='mgr.14100 172.21.15.156:0/652053769' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "mgr metadata", "who": "smithi156.zkvylv", "id": "smithi156.zkvylv"}]: dispatch 2024-07-16T09:46:35.676 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:35 smithi156 ceph-mon[29302]: Manager daemon smithi156.zkvylv is now available 2024-07-16T09:46:35.676 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:35 smithi156 ceph-mon[29302]: from='mgr.14100 172.21.15.156:0/652053769' entity='mgr.smithi156.zkvylv' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi156.zkvylv/mirror_snapshot_schedule"}]: dispatch 2024-07-16T09:46:35.676 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:35 smithi156 ceph-mon[29302]: from='mgr.14100 172.21.15.156:0/652053769' entity='mgr.smithi156.zkvylv' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi156.zkvylv/trash_purge_schedule"}]: dispatch 2024-07-16T09:46:35.676 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:35 smithi156 ceph-mon[29302]: from='mgr.14100 172.21.15.156:0/652053769' entity='mgr.smithi156.zkvylv' 2024-07-16T09:46:35.676 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:35 smithi156 ceph-mon[29302]: from='mgr.14100 172.21.15.156:0/652053769' entity='mgr.smithi156.zkvylv' 2024-07-16T09:46:35.676 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:35 smithi156 ceph-mon[29302]: from='mgr.14100 172.21.15.156:0/652053769' entity='mgr.smithi156.zkvylv' 2024-07-16T09:46:37.299 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:37 smithi156 ceph-mon[29302]: mgrmap e3: smithi156.zkvylv(active, since 1.00703s) 2024-07-16T09:46:38.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:38 smithi156 ceph-mon[29302]: mgrmap e4: smithi156.zkvylv(active, since 2s) 2024-07-16T09:46:38.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:38 smithi156 ceph-mon[29302]: from='client.? 172.21.15.156:0/3815911933' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-07-16T09:46:40.300 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout 2024-07-16T09:46:40.300 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout { 2024-07-16T09:46:40.300 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout "fsid": "26c91c26-4358-11ef-bcad-c7b262605968", 2024-07-16T09:46:40.301 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout "health": { 2024-07-16T09:46:40.301 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-07-16T09:46:40.301 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-07-16T09:46:40.301 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-07-16T09:46:40.301 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout }, 2024-07-16T09:46:40.301 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-07-16T09:46:40.301 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-07-16T09:46:40.301 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout 0 2024-07-16T09:46:40.302 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout ], 2024-07-16T09:46:40.302 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-07-16T09:46:40.302 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout "smithi156" 2024-07-16T09:46:40.302 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout ], 2024-07-16T09:46:40.303 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout "quorum_age": 8, 2024-07-16T09:46:40.303 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout "monmap": { 2024-07-16T09:46:40.303 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-07-16T09:46:40.303 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "quincy", 2024-07-16T09:46:40.303 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-07-16T09:46:40.303 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout }, 2024-07-16T09:46:40.303 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-07-16T09:46:40.303 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-07-16T09:46:40.303 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-07-16T09:46:40.303 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-07-16T09:46:40.303 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-07-16T09:46:40.312 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-07-16T09:46:40.313 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-07-16T09:46:40.313 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-07-16T09:46:40.313 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout }, 2024-07-16T09:46:40.313 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-07-16T09:46:40.313 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-07-16T09:46:40.313 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-07-16T09:46:40.313 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-07-16T09:46:40.313 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-07-16T09:46:40.313 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-07-16T09:46:40.313 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-07-16T09:46:40.314 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-07-16T09:46:40.314 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-07-16T09:46:40.314 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout }, 2024-07-16T09:46:40.314 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-07-16T09:46:40.314 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-07-16T09:46:40.314 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-07-16T09:46:40.314 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-07-16T09:46:40.314 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout }, 2024-07-16T09:46:40.314 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-07-16T09:46:40.315 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout "available": true, 2024-07-16T09:46:40.315 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-07-16T09:46:40.315 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout "modules": [ 2024-07-16T09:46:40.315 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout "iostat", 2024-07-16T09:46:40.315 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout "nfs", 2024-07-16T09:46:40.315 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout "restful" 2024-07-16T09:46:40.315 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout ], 2024-07-16T09:46:40.315 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout "services": {} 2024-07-16T09:46:40.316 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout }, 2024-07-16T09:46:40.316 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-07-16T09:46:40.316 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-07-16T09:46:40.316 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout "modified": "2024-07-16T09:46:23.834404+0000", 2024-07-16T09:46:40.316 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout "services": {} 2024-07-16T09:46:40.316 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout }, 2024-07-16T09:46:40.316 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-07-16T09:46:40.316 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout } 2024-07-16T09:46:40.316 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:37.297+0000 7f8a21e1c640 1 Processor -- start 2024-07-16T09:46:40.316 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:37.298+0000 7f8a21e1c640 1 -- start start 2024-07-16T09:46:40.317 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:37.298+0000 7f8a21e1c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8a1c100ce0 0x7f8a1c1010f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:46:40.317 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:37.298+0000 7f8a21e1c640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a1c101630 con 0x7f8a1c100ce0 2024-07-16T09:46:40.317 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:37.298+0000 7f8a1b7fe640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8a1c100ce0 0x7f8a1c1010f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:46:40.317 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:37.298+0000 7f8a1b7fe640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8a1c100ce0 0x7f8a1c1010f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:34524/0 (socket says 172.21.15.156:34524) 2024-07-16T09:46:40.317 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:37.298+0000 7f8a1b7fe640 1 -- 172.21.15.156:0/232101030 learned_addr learned my addr 172.21.15.156:0/232101030 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T09:46:40.317 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:37.298+0000 7f8a1b7fe640 1 -- 172.21.15.156:0/232101030 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a1c101770 con 0x7f8a1c100ce0 2024-07-16T09:46:40.317 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:37.299+0000 7f8a1b7fe640 1 --2- 172.21.15.156:0/232101030 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8a1c100ce0 0x7f8a1c1010f0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f8a0c009840 tx=0x7f8a0c0091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31a3882f87f66272 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:46:40.317 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:37.299+0000 7f8a1affd640 1 -- 172.21.15.156:0/232101030 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8a0c032440 con 0x7f8a1c100ce0 2024-07-16T09:46:40.317 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:37.299+0000 7f8a1affd640 1 -- 172.21.15.156:0/232101030 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f8a0c032a30 con 0x7f8a1c100ce0 2024-07-16T09:46:40.317 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:37.299+0000 7f8a1affd640 1 -- 172.21.15.156:0/232101030 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8a0c031ad0 con 0x7f8a1c100ce0 2024-07-16T09:46:40.317 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:37.300+0000 7f8a21e1c640 1 -- 172.21.15.156:0/232101030 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8a1c100ce0 msgr2=0x7f8a1c1010f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:46:40.318 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:37.300+0000 7f8a21e1c640 1 --2- 172.21.15.156:0/232101030 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8a1c100ce0 0x7f8a1c1010f0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f8a0c009840 tx=0x7f8a0c0091d0 comp rx=0 tx=0).stop 2024-07-16T09:46:40.318 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:37.300+0000 7f8a21e1c640 1 -- 172.21.15.156:0/232101030 shutdown_connections 2024-07-16T09:46:40.318 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:37.300+0000 7f8a21e1c640 1 --2- 172.21.15.156:0/232101030 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8a1c100ce0 0x7f8a1c1010f0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:46:40.318 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:37.300+0000 7f8a21e1c640 1 -- 172.21.15.156:0/232101030 >> 172.21.15.156:0/232101030 conn(0x7f8a1c0fc520 msgr2=0x7f8a1c0fe930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:46:40.318 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:37.300+0000 7f8a21e1c640 1 -- 172.21.15.156:0/232101030 shutdown_connections 2024-07-16T09:46:40.318 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:37.300+0000 7f8a21e1c640 1 -- 172.21.15.156:0/232101030 wait complete. 2024-07-16T09:46:40.318 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:37.300+0000 7f8a21e1c640 1 Processor -- start 2024-07-16T09:46:40.318 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:37.300+0000 7f8a21e1c640 1 -- start start 2024-07-16T09:46:40.318 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:37.301+0000 7f8a21e1c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8a1c100ce0 0x7f8a1c110ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:46:40.319 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:37.301+0000 7f8a21e1c640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a1c110ff0 con 0x7f8a1c100ce0 2024-07-16T09:46:40.319 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:37.301+0000 7f8a1b7fe640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8a1c100ce0 0x7f8a1c110ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:46:40.319 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:37.301+0000 7f8a1b7fe640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8a1c100ce0 0x7f8a1c110ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:34536/0 (socket says 172.21.15.156:34536) 2024-07-16T09:46:40.319 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:37.301+0000 7f8a1b7fe640 1 -- 172.21.15.156:0/3815911933 learned_addr learned my addr 172.21.15.156:0/3815911933 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T09:46:40.319 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:37.301+0000 7f8a1b7fe640 1 -- 172.21.15.156:0/3815911933 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a0c009450 con 0x7f8a1c100ce0 2024-07-16T09:46:40.319 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:37.301+0000 7f8a1b7fe640 1 --2- 172.21.15.156:0/3815911933 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8a1c100ce0 0x7f8a1c110ab0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f8a0c0317b0 tx=0x7f8a0c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:46:40.319 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:37.301+0000 7f8a18ff9640 1 -- 172.21.15.156:0/3815911933 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8a0c031e90 con 0x7f8a1c100ce0 2024-07-16T09:46:40.319 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:37.301+0000 7f8a21e1c640 1 -- 172.21.15.156:0/3815911933 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a1c1111f0 con 0x7f8a1c100ce0 2024-07-16T09:46:40.319 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:37.301+0000 7f8a18ff9640 1 -- 172.21.15.156:0/3815911933 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f8a0c03daf0 con 0x7f8a1c100ce0 2024-07-16T09:46:40.319 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:37.301+0000 7f8a18ff9640 1 -- 172.21.15.156:0/3815911933 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8a0c03baa0 con 0x7f8a1c100ce0 2024-07-16T09:46:40.319 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:37.301+0000 7f8a21e1c640 1 -- 172.21.15.156:0/3815911933 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a1c111660 con 0x7f8a1c100ce0 2024-07-16T09:46:40.319 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:37.302+0000 7f8a18ff9640 1 -- 172.21.15.156:0/3815911933 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 4) v1 ==== 42625+0+0 (secure 0 0 0) 0x7f8a0c039070 con 0x7f8a1c100ce0 2024-07-16T09:46:40.320 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:37.302+0000 7f8a18ff9640 1 --2- 172.21.15.156:0/3815911933 >> [v2:172.21.15.156:6800/2871070852,v1:172.21.15.156:6801/2871070852] conn(0x7f8a08035730 0x7f8a08037be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:46:40.320 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:37.302+0000 7f8a18ff9640 1 -- 172.21.15.156:0/3815911933 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f8a0c068680 con 0x7f8a1c100ce0 2024-07-16T09:46:40.320 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:37.302+0000 7f8a21e1c640 1 -- 172.21.15.156:0/3815911933 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a1c05eb80 con 0x7f8a1c100ce0 2024-07-16T09:46:40.320 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:37.302+0000 7f8a13fff640 1 --2- 172.21.15.156:0/3815911933 >> [v2:172.21.15.156:6800/2871070852,v1:172.21.15.156:6801/2871070852] conn(0x7f8a08035730 0x7f8a08037be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:46:40.320 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:37.303+0000 7f8a13fff640 1 --2- 172.21.15.156:0/3815911933 >> [v2:172.21.15.156:6800/2871070852,v1:172.21.15.156:6801/2871070852] conn(0x7f8a08035730 0x7f8a08037be0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f8a04006fd0 tx=0x7f8a04006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:46:40.320 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:37.307+0000 7f8a18ff9640 1 -- 172.21.15.156:0/3815911933 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f8a0c041070 con 0x7f8a1c100ce0 2024-07-16T09:46:40.320 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:37.531+0000 7f8a21e1c640 1 -- 172.21.15.156:0/3815911933 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f8a1c10a1d0 con 0x7f8a1c100ce0 2024-07-16T09:46:40.320 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:37.532+0000 7f8a18ff9640 1 -- 172.21.15.156:0/3815911933 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1247 (secure 0 0 0) 0x7f8a0c037030 con 0x7f8a1c100ce0 2024-07-16T09:46:40.321 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:37.534+0000 7f8a21e1c640 1 -- 172.21.15.156:0/3815911933 >> [v2:172.21.15.156:6800/2871070852,v1:172.21.15.156:6801/2871070852] conn(0x7f8a08035730 msgr2=0x7f8a08037be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:46:40.321 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:37.534+0000 7f8a21e1c640 1 --2- 172.21.15.156:0/3815911933 >> [v2:172.21.15.156:6800/2871070852,v1:172.21.15.156:6801/2871070852] conn(0x7f8a08035730 0x7f8a08037be0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f8a04006fd0 tx=0x7f8a04006db0 comp rx=0 tx=0).stop 2024-07-16T09:46:40.321 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:37.534+0000 7f8a21e1c640 1 -- 172.21.15.156:0/3815911933 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8a1c100ce0 msgr2=0x7f8a1c110ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:46:40.321 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:37.534+0000 7f8a21e1c640 1 --2- 172.21.15.156:0/3815911933 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8a1c100ce0 0x7f8a1c110ab0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f8a0c0317b0 tx=0x7f8a0c008040 comp rx=0 tx=0).stop 2024-07-16T09:46:40.321 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:37.534+0000 7f8a21e1c640 1 -- 172.21.15.156:0/3815911933 shutdown_connections 2024-07-16T09:46:40.321 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:37.534+0000 7f8a21e1c640 1 --2- 172.21.15.156:0/3815911933 >> [v2:172.21.15.156:6800/2871070852,v1:172.21.15.156:6801/2871070852] conn(0x7f8a08035730 0x7f8a08037be0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:46:40.321 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:37.534+0000 7f8a21e1c640 1 --2- 172.21.15.156:0/3815911933 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8a1c100ce0 0x7f8a1c110ab0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:46:40.321 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:37.534+0000 7f8a21e1c640 1 -- 172.21.15.156:0/3815911933 >> 172.21.15.156:0/3815911933 conn(0x7f8a1c0fc520 msgr2=0x7f8a1c107890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:46:40.321 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:37.534+0000 7f8a21e1c640 1 -- 172.21.15.156:0/3815911933 shutdown_connections 2024-07-16T09:46:40.321 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:37.534+0000 7f8a21e1c640 1 -- 172.21.15.156:0/3815911933 wait complete. 2024-07-16T09:46:40.322 INFO:teuthology.orchestra.run.smithi156.stdout:mgr is available 2024-07-16T09:46:43.743 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:43 smithi156 ceph-mon[29302]: from='client.? 172.21.15.156:0/620189653' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-07-16T09:46:48.509 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout 2024-07-16T09:46:48.509 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout [global] 2024-07-16T09:46:48.509 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout fsid = 26c91c26-4358-11ef-bcad-c7b262605968 2024-07-16T09:46:48.509 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-07-16T09:46:48.509 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-07-16T09:46:48.509 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-07-16T09:46:48.509 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-07-16T09:46:48.509 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout 2024-07-16T09:46:48.509 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout [mgr] 2024-07-16T09:46:48.510 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-07-16T09:46:48.510 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout 2024-07-16T09:46:48.510 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout [osd] 2024-07-16T09:46:48.510 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-07-16T09:46:48.510 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-07-16T09:46:48.510 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-07-16T09:46:48.510 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-07-16T09:46:48.510 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-07-16T09:46:48.510 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:43.306+0000 7f86f8930640 1 Processor -- start 2024-07-16T09:46:48.510 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:43.306+0000 7f86f8930640 1 -- start start 2024-07-16T09:46:48.510 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:43.306+0000 7f86f8930640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f86f4100260 0x7f86f4100670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:46:48.510 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:43.306+0000 7f86f8930640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86f4100bb0 con 0x7f86f4100260 2024-07-16T09:46:48.511 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:43.306+0000 7f86f259c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f86f4100260 0x7f86f4100670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:46:48.511 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:43.306+0000 7f86f259c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f86f4100260 0x7f86f4100670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:47742/0 (socket says 172.21.15.156:47742) 2024-07-16T09:46:48.511 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:43.306+0000 7f86f259c640 1 -- 172.21.15.156:0/3007381870 learned_addr learned my addr 172.21.15.156:0/3007381870 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T09:46:48.511 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:43.306+0000 7f86f259c640 1 -- 172.21.15.156:0/3007381870 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86f4100cf0 con 0x7f86f4100260 2024-07-16T09:46:48.511 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:43.307+0000 7f86f259c640 1 --2- 172.21.15.156:0/3007381870 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f86f4100260 0x7f86f4100670 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f86e4009840 tx=0x7f86e40091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdfbe919531d4488 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:46:48.511 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:43.307+0000 7f86f1d9b640 1 -- 172.21.15.156:0/3007381870 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f86e4032440 con 0x7f86f4100260 2024-07-16T09:46:48.511 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:43.307+0000 7f86f1d9b640 1 -- 172.21.15.156:0/3007381870 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f86e4032a30 con 0x7f86f4100260 2024-07-16T09:46:48.511 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:43.307+0000 7f86f1d9b640 1 -- 172.21.15.156:0/3007381870 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f86e4031ad0 con 0x7f86f4100260 2024-07-16T09:46:48.511 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:43.308+0000 7f86f8930640 1 -- 172.21.15.156:0/3007381870 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f86f4100260 msgr2=0x7f86f4100670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:46:48.511 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:43.308+0000 7f86f8930640 1 --2- 172.21.15.156:0/3007381870 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f86f4100260 0x7f86f4100670 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f86e4009840 tx=0x7f86e40091d0 comp rx=0 tx=0).stop 2024-07-16T09:46:48.511 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:43.308+0000 7f86f8930640 1 -- 172.21.15.156:0/3007381870 shutdown_connections 2024-07-16T09:46:48.511 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:43.308+0000 7f86f8930640 1 --2- 172.21.15.156:0/3007381870 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f86f4100260 0x7f86f4100670 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:46:48.511 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:43.308+0000 7f86f8930640 1 -- 172.21.15.156:0/3007381870 >> 172.21.15.156:0/3007381870 conn(0x7f86f40fbcc0 msgr2=0x7f86f40fe0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:46:48.511 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:43.308+0000 7f86f8930640 1 -- 172.21.15.156:0/3007381870 shutdown_connections 2024-07-16T09:46:48.512 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:43.308+0000 7f86f8930640 1 -- 172.21.15.156:0/3007381870 wait complete. 2024-07-16T09:46:48.512 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:43.309+0000 7f86f8930640 1 Processor -- start 2024-07-16T09:46:48.512 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:43.309+0000 7f86f8930640 1 -- start start 2024-07-16T09:46:48.512 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:43.309+0000 7f86f8930640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f86f4100260 0x7f86f4110a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:46:48.512 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:43.309+0000 7f86f8930640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86f4110fc0 con 0x7f86f4100260 2024-07-16T09:46:48.512 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:43.309+0000 7f86f259c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f86f4100260 0x7f86f4110a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:46:48.512 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:43.309+0000 7f86f259c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f86f4100260 0x7f86f4110a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:47748/0 (socket says 172.21.15.156:47748) 2024-07-16T09:46:48.512 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:43.309+0000 7f86f259c640 1 -- 172.21.15.156:0/620189653 learned_addr learned my addr 172.21.15.156:0/620189653 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T09:46:48.512 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:43.309+0000 7f86f259c640 1 -- 172.21.15.156:0/620189653 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86e4009450 con 0x7f86f4100260 2024-07-16T09:46:48.512 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:43.309+0000 7f86f259c640 1 --2- 172.21.15.156:0/620189653 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f86f4100260 0x7f86f4110a80 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f86e4009420 tx=0x7f86e4008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:46:48.512 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:43.310+0000 7f86eaffd640 1 -- 172.21.15.156:0/620189653 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f86e403d5b0 con 0x7f86f4100260 2024-07-16T09:46:48.513 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:43.310+0000 7f86eaffd640 1 -- 172.21.15.156:0/620189653 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f86e403dba0 con 0x7f86f4100260 2024-07-16T09:46:48.513 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:43.310+0000 7f86f8930640 1 -- 172.21.15.156:0/620189653 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86f41111c0 con 0x7f86f4100260 2024-07-16T09:46:48.513 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:43.310+0000 7f86eaffd640 1 -- 172.21.15.156:0/620189653 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f86e403bb40 con 0x7f86f4100260 2024-07-16T09:46:48.513 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:43.310+0000 7f86f8930640 1 -- 172.21.15.156:0/620189653 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86f4111630 con 0x7f86f4100260 2024-07-16T09:46:48.513 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:43.310+0000 7f86eaffd640 1 -- 172.21.15.156:0/620189653 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 4) v1 ==== 42625+0+0 (secure 0 0 0) 0x7f86e4039070 con 0x7f86f4100260 2024-07-16T09:46:48.513 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:43.311+0000 7f86eaffd640 1 --2- 172.21.15.156:0/620189653 >> [v2:172.21.15.156:6800/2871070852,v1:172.21.15.156:6801/2871070852] conn(0x7f86e0035780 0x7f86e0037c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:46:48.513 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:43.311+0000 7f86eaffd640 1 -- 172.21.15.156:0/620189653 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f86e4068570 con 0x7f86f4100260 2024-07-16T09:46:48.513 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:43.311+0000 7f86f8930640 1 -- 172.21.15.156:0/620189653 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86f405eb80 con 0x7f86f4100260 2024-07-16T09:46:48.513 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:43.311+0000 7f86ebfff640 1 --2- 172.21.15.156:0/620189653 >> [v2:172.21.15.156:6800/2871070852,v1:172.21.15.156:6801/2871070852] conn(0x7f86e0035780 0x7f86e0037c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:46:48.513 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:43.311+0000 7f86ebfff640 1 --2- 172.21.15.156:0/620189653 >> [v2:172.21.15.156:6800/2871070852,v1:172.21.15.156:6801/2871070852] conn(0x7f86e0035780 0x7f86e0037c30 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f86dc006fd0 tx=0x7f86dc006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:46:48.513 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:43.315+0000 7f86eaffd640 1 -- 172.21.15.156:0/620189653 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f86e4041070 con 0x7f86f4100260 2024-07-16T09:46:48.513 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:43.482+0000 7f86f8930640 1 -- 172.21.15.156:0/620189653 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7f86f404b530 con 0x7f86f4100260 2024-07-16T09:46:48.513 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:43.484+0000 7f86eaffd640 1 -- 172.21.15.156:0/620189653 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) v1 ==== 70+0+373 (secure 0 0 0) 0x7f86e4055020 con 0x7f86f4100260 2024-07-16T09:46:48.514 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:43.486+0000 7f86f8930640 1 -- 172.21.15.156:0/620189653 >> [v2:172.21.15.156:6800/2871070852,v1:172.21.15.156:6801/2871070852] conn(0x7f86e0035780 msgr2=0x7f86e0037c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:46:48.514 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:43.486+0000 7f86f8930640 1 --2- 172.21.15.156:0/620189653 >> [v2:172.21.15.156:6800/2871070852,v1:172.21.15.156:6801/2871070852] conn(0x7f86e0035780 0x7f86e0037c30 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f86dc006fd0 tx=0x7f86dc006db0 comp rx=0 tx=0).stop 2024-07-16T09:46:48.514 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:43.486+0000 7f86f8930640 1 -- 172.21.15.156:0/620189653 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f86f4100260 msgr2=0x7f86f4110a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:46:48.514 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:43.486+0000 7f86f8930640 1 --2- 172.21.15.156:0/620189653 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f86f4100260 0x7f86f4110a80 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f86e4009420 tx=0x7f86e4008040 comp rx=0 tx=0).stop 2024-07-16T09:46:48.514 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:43.486+0000 7f86f8930640 1 -- 172.21.15.156:0/620189653 shutdown_connections 2024-07-16T09:46:48.514 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:43.486+0000 7f86f8930640 1 --2- 172.21.15.156:0/620189653 >> [v2:172.21.15.156:6800/2871070852,v1:172.21.15.156:6801/2871070852] conn(0x7f86e0035780 0x7f86e0037c30 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:46:48.514 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:43.486+0000 7f86f8930640 1 --2- 172.21.15.156:0/620189653 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f86f4100260 0x7f86f4110a80 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:46:48.514 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:43.486+0000 7f86f8930640 1 -- 172.21.15.156:0/620189653 >> 172.21.15.156:0/620189653 conn(0x7f86f40fbcc0 msgr2=0x7f86f40fc700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:46:48.514 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:43.486+0000 7f86f8930640 1 -- 172.21.15.156:0/620189653 shutdown_connections 2024-07-16T09:46:48.514 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:43.486+0000 7f86f8930640 1 -- 172.21.15.156:0/620189653 wait complete. 2024-07-16T09:46:48.514 INFO:teuthology.orchestra.run.smithi156.stdout:Enabling cephadm module... 2024-07-16T09:46:49.411 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:49 smithi156 ceph-mon[29302]: from='client.? 172.21.15.156:0/2144872576' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-07-16T09:46:49.787 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:48.882+0000 7f0f7add2640 1 Processor -- start 2024-07-16T09:46:49.787 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:48.883+0000 7f0f7add2640 1 -- start start 2024-07-16T09:46:49.788 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:48.883+0000 7f0f7add2640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0f74077be0 0x7f0f74076250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:46:49.788 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:48.883+0000 7f0f7add2640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f74077ff0 con 0x7f0f74077be0 2024-07-16T09:46:49.788 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:48.883+0000 7f0f78b6e640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0f74077be0 0x7f0f74076250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:46:49.788 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:48.883+0000 7f0f78b6e640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0f74077be0 0x7f0f74076250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:47756/0 (socket says 172.21.15.156:47756) 2024-07-16T09:46:49.788 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:48.883+0000 7f0f78b6e640 1 -- 172.21.15.156:0/98876132 learned_addr learned my addr 172.21.15.156:0/98876132 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T09:46:49.788 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:48.884+0000 7f0f78b6e640 1 -- 172.21.15.156:0/98876132 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f74078130 con 0x7f0f74077be0 2024-07-16T09:46:49.788 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:48.884+0000 7f0f78b6e640 1 --2- 172.21.15.156:0/98876132 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0f74077be0 0x7f0f74076250 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f0f68009840 tx=0x7f0f680091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=890bb3381a19ce4e server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:46:49.788 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:48.884+0000 7f0f737fe640 1 -- 172.21.15.156:0/98876132 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0f68032440 con 0x7f0f74077be0 2024-07-16T09:46:49.788 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:48.884+0000 7f0f737fe640 1 -- 172.21.15.156:0/98876132 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f0f68032a30 con 0x7f0f74077be0 2024-07-16T09:46:49.788 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:48.884+0000 7f0f737fe640 1 -- 172.21.15.156:0/98876132 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0f68031ad0 con 0x7f0f74077be0 2024-07-16T09:46:49.788 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:48.885+0000 7f0f7add2640 1 -- 172.21.15.156:0/98876132 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0f74077be0 msgr2=0x7f0f74076250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:46:49.788 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:48.885+0000 7f0f7add2640 1 --2- 172.21.15.156:0/98876132 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0f74077be0 0x7f0f74076250 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f0f68009840 tx=0x7f0f680091d0 comp rx=0 tx=0).stop 2024-07-16T09:46:49.789 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:48.885+0000 7f0f7add2640 1 -- 172.21.15.156:0/98876132 shutdown_connections 2024-07-16T09:46:49.789 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:48.885+0000 7f0f7add2640 1 --2- 172.21.15.156:0/98876132 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0f74077be0 0x7f0f74076250 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:46:49.789 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:48.885+0000 7f0f7add2640 1 -- 172.21.15.156:0/98876132 >> 172.21.15.156:0/98876132 conn(0x7f0f740fd6a0 msgr2=0x7f0f740ffab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:46:49.789 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:48.885+0000 7f0f7add2640 1 -- 172.21.15.156:0/98876132 shutdown_connections 2024-07-16T09:46:49.789 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:48.885+0000 7f0f7add2640 1 -- 172.21.15.156:0/98876132 wait complete. 2024-07-16T09:46:49.789 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:48.886+0000 7f0f7add2640 1 Processor -- start 2024-07-16T09:46:49.789 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:48.886+0000 7f0f7add2640 1 -- start start 2024-07-16T09:46:49.789 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:48.886+0000 7f0f7add2640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0f74077be0 0x7f0f74112c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:46:49.789 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:48.886+0000 7f0f7add2640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f741131c0 con 0x7f0f74077be0 2024-07-16T09:46:49.789 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:48.886+0000 7f0f78b6e640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0f74077be0 0x7f0f74112c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:46:49.789 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:48.886+0000 7f0f78b6e640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0f74077be0 0x7f0f74112c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:47762/0 (socket says 172.21.15.156:47762) 2024-07-16T09:46:49.789 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:48.886+0000 7f0f78b6e640 1 -- 172.21.15.156:0/2144872576 learned_addr learned my addr 172.21.15.156:0/2144872576 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T09:46:49.789 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:48.886+0000 7f0f78b6e640 1 -- 172.21.15.156:0/2144872576 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f68009450 con 0x7f0f74077be0 2024-07-16T09:46:49.790 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:48.887+0000 7f0f78b6e640 1 --2- 172.21.15.156:0/2144872576 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0f74077be0 0x7f0f74112c80 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f0f68006f00 tx=0x7f0f68008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:46:49.790 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:48.887+0000 7f0f717fa640 1 -- 172.21.15.156:0/2144872576 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0f6803d450 con 0x7f0f74077be0 2024-07-16T09:46:49.790 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:48.887+0000 7f0f717fa640 1 -- 172.21.15.156:0/2144872576 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f0f6803da40 con 0x7f0f74077be0 2024-07-16T09:46:49.790 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:48.887+0000 7f0f717fa640 1 -- 172.21.15.156:0/2144872576 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0f6803bb10 con 0x7f0f74077be0 2024-07-16T09:46:49.790 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:48.887+0000 7f0f7add2640 1 -- 172.21.15.156:0/2144872576 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f741133c0 con 0x7f0f74077be0 2024-07-16T09:46:49.790 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:48.887+0000 7f0f7add2640 1 -- 172.21.15.156:0/2144872576 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f74113830 con 0x7f0f74077be0 2024-07-16T09:46:49.790 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:48.888+0000 7f0f717fa640 1 -- 172.21.15.156:0/2144872576 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 4) v1 ==== 42625+0+0 (secure 0 0 0) 0x7f0f68039070 con 0x7f0f74077be0 2024-07-16T09:46:49.790 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:48.888+0000 7f0f717fa640 1 --2- 172.21.15.156:0/2144872576 >> [v2:172.21.15.156:6800/2871070852,v1:172.21.15.156:6801/2871070852] conn(0x7f0f64035730 0x7f0f64037be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:46:49.790 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:48.888+0000 7f0f717fa640 1 -- 172.21.15.156:0/2144872576 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f0f68068390 con 0x7f0f74077be0 2024-07-16T09:46:49.790 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:48.888+0000 7f0f7add2640 1 -- 172.21.15.156:0/2144872576 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f74113830 con 0x7f0f74077be0 2024-07-16T09:46:49.790 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:48.888+0000 7f0f73fff640 1 --2- 172.21.15.156:0/2144872576 >> [v2:172.21.15.156:6800/2871070852,v1:172.21.15.156:6801/2871070852] conn(0x7f0f64035730 0x7f0f64037be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:46:49.790 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:48.889+0000 7f0f73fff640 1 --2- 172.21.15.156:0/2144872576 >> [v2:172.21.15.156:6800/2871070852,v1:172.21.15.156:6801/2871070852] conn(0x7f0f64035730 0x7f0f64037be0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f0f60006fd0 tx=0x7f0f60006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:46:49.791 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:48.893+0000 7f0f717fa640 1 -- 172.21.15.156:0/2144872576 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f0f74113830 con 0x7f0f74077be0 2024-07-16T09:46:49.791 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:49.097+0000 7f0f7add2640 1 -- 172.21.15.156:0/2144872576 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) v1 -- 0x7f0f74113830 con 0x7f0f74077be0 2024-07-16T09:46:49.791 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:49.149+0000 7f0f717fa640 1 -- 172.21.15.156:0/2144872576 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mgrmap(e 5) v1 ==== 42636+0+0 (secure 0 0 0) 0x7f0f68067230 con 0x7f0f74077be0 2024-07-16T09:46:49.791 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:49.151+0000 7f0f717fa640 1 -- 172.21.15.156:0/2144872576 <== mon.0 v2:172.21.15.156:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) v1 ==== 86+0+0 (secure 0 0 0) 0x7f0f6803d5b0 con 0x7f0f74077be0 2024-07-16T09:46:49.791 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:49.153+0000 7f0f7add2640 1 -- 172.21.15.156:0/2144872576 >> [v2:172.21.15.156:6800/2871070852,v1:172.21.15.156:6801/2871070852] conn(0x7f0f64035730 msgr2=0x7f0f64037be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:46:49.791 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:49.153+0000 7f0f7add2640 1 --2- 172.21.15.156:0/2144872576 >> [v2:172.21.15.156:6800/2871070852,v1:172.21.15.156:6801/2871070852] conn(0x7f0f64035730 0x7f0f64037be0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f0f60006fd0 tx=0x7f0f60006db0 comp rx=0 tx=0).stop 2024-07-16T09:46:49.791 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:49.153+0000 7f0f7add2640 1 -- 172.21.15.156:0/2144872576 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0f74077be0 msgr2=0x7f0f74112c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:46:49.791 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:49.153+0000 7f0f7add2640 1 --2- 172.21.15.156:0/2144872576 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0f74077be0 0x7f0f74112c80 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f0f68006f00 tx=0x7f0f68008040 comp rx=0 tx=0).stop 2024-07-16T09:46:49.791 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:49.153+0000 7f0f7add2640 1 -- 172.21.15.156:0/2144872576 shutdown_connections 2024-07-16T09:46:49.791 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:49.153+0000 7f0f7add2640 1 --2- 172.21.15.156:0/2144872576 >> [v2:172.21.15.156:6800/2871070852,v1:172.21.15.156:6801/2871070852] conn(0x7f0f64035730 0x7f0f64037be0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:46:49.791 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:49.153+0000 7f0f7add2640 1 --2- 172.21.15.156:0/2144872576 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0f74077be0 0x7f0f74112c80 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:46:49.791 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:49.153+0000 7f0f7add2640 1 -- 172.21.15.156:0/2144872576 >> 172.21.15.156:0/2144872576 conn(0x7f0f740fd6a0 msgr2=0x7f0f740fe0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:46:49.791 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:49.154+0000 7f0f7add2640 1 -- 172.21.15.156:0/2144872576 shutdown_connections 2024-07-16T09:46:49.792 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:49.154+0000 7f0f7add2640 1 -- 172.21.15.156:0/2144872576 wait complete. 2024-07-16T09:46:50.373 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:50 smithi156 ceph-mon[29302]: from='client.? 172.21.15.156:0/2144872576' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-07-16T09:46:50.374 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:50 smithi156 ceph-mon[29302]: mgrmap e5: smithi156.zkvylv(active, since 14s) 2024-07-16T09:46:50.971 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout { 2024-07-16T09:46:50.971 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-07-16T09:46:50.972 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout "available": true, 2024-07-16T09:46:50.972 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout "active_name": "smithi156.zkvylv", 2024-07-16T09:46:50.972 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-07-16T09:46:50.972 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout } 2024-07-16T09:46:50.972 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:50.166+0000 7f9e63b9d640 1 Processor -- start 2024-07-16T09:46:50.972 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:50.166+0000 7f9e63b9d640 1 -- start start 2024-07-16T09:46:50.972 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:50.166+0000 7f9e63b9d640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9e5c075b50 0x7f9e5c106e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:46:50.972 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:50.166+0000 7f9e63b9d640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e5c075f60 con 0x7f9e5c075b50 2024-07-16T09:46:50.972 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:50.167+0000 7f9e61939640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9e5c075b50 0x7f9e5c106e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:46:50.973 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:50.167+0000 7f9e61939640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9e5c075b50 0x7f9e5c106e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:47772/0 (socket says 172.21.15.156:47772) 2024-07-16T09:46:50.973 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:50.167+0000 7f9e61939640 1 -- 172.21.15.156:0/2848231358 learned_addr learned my addr 172.21.15.156:0/2848231358 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T09:46:50.973 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:50.167+0000 7f9e61939640 1 -- 172.21.15.156:0/2848231358 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e5c102860 con 0x7f9e5c075b50 2024-07-16T09:46:50.973 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:50.168+0000 7f9e61939640 1 --2- 172.21.15.156:0/2848231358 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9e5c075b50 0x7f9e5c106e70 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f9e4c009840 tx=0x7f9e4c0091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4c22a8c1be8b712 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:46:50.973 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:50.168+0000 7f9e60937640 1 -- 172.21.15.156:0/2848231358 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9e4c032440 con 0x7f9e5c075b50 2024-07-16T09:46:50.973 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:50.168+0000 7f9e60937640 1 -- 172.21.15.156:0/2848231358 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f9e4c032a30 con 0x7f9e5c075b50 2024-07-16T09:46:50.973 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:50.168+0000 7f9e60937640 1 -- 172.21.15.156:0/2848231358 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9e4c031ad0 con 0x7f9e5c075b50 2024-07-16T09:46:50.973 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:50.168+0000 7f9e63b9d640 1 -- 172.21.15.156:0/2848231358 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9e5c075b50 msgr2=0x7f9e5c106e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:46:50.973 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:50.168+0000 7f9e63b9d640 1 --2- 172.21.15.156:0/2848231358 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9e5c075b50 0x7f9e5c106e70 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f9e4c009840 tx=0x7f9e4c0091d0 comp rx=0 tx=0).stop 2024-07-16T09:46:50.973 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:50.169+0000 7f9e63b9d640 1 -- 172.21.15.156:0/2848231358 shutdown_connections 2024-07-16T09:46:50.973 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:50.169+0000 7f9e63b9d640 1 --2- 172.21.15.156:0/2848231358 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9e5c075b50 0x7f9e5c106e70 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:46:50.974 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:50.169+0000 7f9e63b9d640 1 -- 172.21.15.156:0/2848231358 >> 172.21.15.156:0/2848231358 conn(0x7f9e5c0fc560 msgr2=0x7f9e5c0fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:46:50.974 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:50.169+0000 7f9e63b9d640 1 -- 172.21.15.156:0/2848231358 shutdown_connections 2024-07-16T09:46:50.974 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:50.169+0000 7f9e63b9d640 1 -- 172.21.15.156:0/2848231358 wait complete. 2024-07-16T09:46:50.974 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:50.169+0000 7f9e63b9d640 1 Processor -- start 2024-07-16T09:46:50.974 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:50.169+0000 7f9e63b9d640 1 -- start start 2024-07-16T09:46:50.974 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:50.169+0000 7f9e63b9d640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9e5c075b50 0x7f9e5c112c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:46:50.974 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:50.169+0000 7f9e63b9d640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e5c1131a0 con 0x7f9e5c075b50 2024-07-16T09:46:50.974 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:50.169+0000 7f9e61939640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9e5c075b50 0x7f9e5c112c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:46:50.974 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:50.169+0000 7f9e61939640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9e5c075b50 0x7f9e5c112c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:47788/0 (socket says 172.21.15.156:47788) 2024-07-16T09:46:50.974 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:50.169+0000 7f9e61939640 1 -- 172.21.15.156:0/3464980334 learned_addr learned my addr 172.21.15.156:0/3464980334 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T09:46:50.974 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:50.170+0000 7f9e61939640 1 -- 172.21.15.156:0/3464980334 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e4c009450 con 0x7f9e5c075b50 2024-07-16T09:46:50.974 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:50.170+0000 7f9e61939640 1 --2- 172.21.15.156:0/3464980334 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9e5c075b50 0x7f9e5c112c60 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f9e4c031b90 tx=0x7f9e4c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:46:50.975 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:50.170+0000 7f9e527fc640 1 -- 172.21.15.156:0/3464980334 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9e4c0328b0 con 0x7f9e5c075b50 2024-07-16T09:46:50.975 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:50.170+0000 7f9e527fc640 1 -- 172.21.15.156:0/3464980334 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f9e4c03d8e0 con 0x7f9e5c075b50 2024-07-16T09:46:50.975 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:50.170+0000 7f9e63b9d640 1 -- 172.21.15.156:0/3464980334 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e5c113400 con 0x7f9e5c075b50 2024-07-16T09:46:50.975 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:50.170+0000 7f9e527fc640 1 -- 172.21.15.156:0/3464980334 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9e4c03ba20 con 0x7f9e5c075b50 2024-07-16T09:46:50.975 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:50.170+0000 7f9e63b9d640 1 -- 172.21.15.156:0/3464980334 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e5c113f30 con 0x7f9e5c075b50 2024-07-16T09:46:50.975 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:50.171+0000 7f9e527fc640 1 -- 172.21.15.156:0/3464980334 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 42636+0+0 (secure 0 0 0) 0x7f9e4c039070 con 0x7f9e5c075b50 2024-07-16T09:46:50.975 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:50.171+0000 7f9e527fc640 1 --2- 172.21.15.156:0/3464980334 >> [v2:172.21.15.156:6800/2871070852,v1:172.21.15.156:6801/2871070852] conn(0x7f9e48035780 0x7f9e48037c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:46:50.975 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:50.171+0000 7f9e63b9d640 1 -- 172.21.15.156:0/3464980334 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e5c05eb80 con 0x7f9e5c075b50 2024-07-16T09:46:50.975 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:50.171+0000 7f9e527fc640 1 -- 172.21.15.156:0/3464980334 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f9e4c068640 con 0x7f9e5c075b50 2024-07-16T09:46:50.975 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:50.171+0000 7f9e61138640 1 -- 172.21.15.156:0/3464980334 >> [v2:172.21.15.156:6800/2871070852,v1:172.21.15.156:6801/2871070852] conn(0x7f9e48035780 msgr2=0x7f9e48037c30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.156:6800/2871070852 2024-07-16T09:46:50.975 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:50.171+0000 7f9e61138640 1 --2- 172.21.15.156:0/3464980334 >> [v2:172.21.15.156:6800/2871070852,v1:172.21.15.156:6801/2871070852] conn(0x7f9e48035780 0x7f9e48037c30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-16T09:46:50.975 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:50.176+0000 7f9e527fc640 1 -- 172.21.15.156:0/3464980334 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f9e4c046400 con 0x7f9e5c075b50 2024-07-16T09:46:50.975 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:50.372+0000 7f9e61138640 1 -- 172.21.15.156:0/3464980334 >> [v2:172.21.15.156:6800/2871070852,v1:172.21.15.156:6801/2871070852] conn(0x7f9e48035780 msgr2=0x7f9e48037c30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.156:6800/2871070852 2024-07-16T09:46:50.976 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:50.372+0000 7f9e61138640 1 --2- 172.21.15.156:0/3464980334 >> [v2:172.21.15.156:6800/2871070852,v1:172.21.15.156:6801/2871070852] conn(0x7f9e48035780 0x7f9e48037c30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-16T09:46:50.976 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:50.384+0000 7f9e63b9d640 1 -- 172.21.15.156:0/3464980334 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7f9e5c04b530 con 0x7f9e5c075b50 2024-07-16T09:46:50.976 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:50.384+0000 7f9e527fc640 1 -- 172.21.15.156:0/3464980334 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) v1 ==== 56+0+103 (secure 0 0 0) 0x7f9e4c041020 con 0x7f9e5c075b50 2024-07-16T09:46:50.976 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:50.386+0000 7f9e63b9d640 1 -- 172.21.15.156:0/3464980334 >> [v2:172.21.15.156:6800/2871070852,v1:172.21.15.156:6801/2871070852] conn(0x7f9e48035780 msgr2=0x7f9e48037c30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-16T09:46:50.976 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:50.386+0000 7f9e63b9d640 1 --2- 172.21.15.156:0/3464980334 >> [v2:172.21.15.156:6800/2871070852,v1:172.21.15.156:6801/2871070852] conn(0x7f9e48035780 0x7f9e48037c30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:46:50.976 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:50.386+0000 7f9e63b9d640 1 -- 172.21.15.156:0/3464980334 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9e5c075b50 msgr2=0x7f9e5c112c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:46:50.976 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:50.386+0000 7f9e63b9d640 1 --2- 172.21.15.156:0/3464980334 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9e5c075b50 0x7f9e5c112c60 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f9e4c031b90 tx=0x7f9e4c008040 comp rx=0 tx=0).stop 2024-07-16T09:46:50.976 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:50.386+0000 7f9e63b9d640 1 -- 172.21.15.156:0/3464980334 shutdown_connections 2024-07-16T09:46:50.976 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:50.386+0000 7f9e63b9d640 1 --2- 172.21.15.156:0/3464980334 >> [v2:172.21.15.156:6800/2871070852,v1:172.21.15.156:6801/2871070852] conn(0x7f9e48035780 0x7f9e48037c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:46:50.976 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:50.386+0000 7f9e63b9d640 1 --2- 172.21.15.156:0/3464980334 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9e5c075b50 0x7f9e5c112c60 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:46:50.976 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:50.386+0000 7f9e63b9d640 1 -- 172.21.15.156:0/3464980334 >> 172.21.15.156:0/3464980334 conn(0x7f9e5c0fc560 msgr2=0x7f9e5c0fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:46:50.976 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:50.386+0000 7f9e63b9d640 1 -- 172.21.15.156:0/3464980334 shutdown_connections 2024-07-16T09:46:50.976 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:50.386+0000 7f9e63b9d640 1 -- 172.21.15.156:0/3464980334 wait complete. 2024-07-16T09:46:50.977 INFO:teuthology.orchestra.run.smithi156.stdout:Waiting for the mgr to restart... 2024-07-16T09:46:50.977 INFO:teuthology.orchestra.run.smithi156.stdout:Waiting for mgr epoch 5... 2024-07-16T09:46:51.348 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:51 smithi156 ceph-mon[29302]: from='client.? 172.21.15.156:0/3464980334' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-07-16T09:46:53.924 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:53 smithi156 ceph-mon[29302]: Active manager daemon smithi156.zkvylv restarted 2024-07-16T09:46:53.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:53 smithi156 ceph-mon[29302]: Activating manager daemon smithi156.zkvylv 2024-07-16T09:46:53.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:53 smithi156 ceph-mon[29302]: osdmap e2: 0 total, 0 up, 0 in 2024-07-16T09:46:53.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:53 smithi156 ceph-mon[29302]: mgrmap e6: smithi156.zkvylv(active, starting, since 0.00304064s) 2024-07-16T09:46:53.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:53 smithi156 ceph-mon[29302]: from='mgr.14118 172.21.15.156:0/3208530346' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "mon metadata", "id": "smithi156"}]: dispatch 2024-07-16T09:46:53.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:53 smithi156 ceph-mon[29302]: from='mgr.14118 172.21.15.156:0/3208530346' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "mgr metadata", "who": "smithi156.zkvylv", "id": "smithi156.zkvylv"}]: dispatch 2024-07-16T09:46:53.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:53 smithi156 ceph-mon[29302]: from='mgr.14118 172.21.15.156:0/3208530346' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-07-16T09:46:53.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:53 smithi156 ceph-mon[29302]: from='mgr.14118 172.21.15.156:0/3208530346' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-07-16T09:46:53.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:53 smithi156 ceph-mon[29302]: from='mgr.14118 172.21.15.156:0/3208530346' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-07-16T09:46:53.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:53 smithi156 ceph-mon[29302]: Manager daemon smithi156.zkvylv is now available 2024-07-16T09:46:53.926 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:53 smithi156 ceph-mon[29302]: from='mgr.14118 172.21.15.156:0/3208530346' entity='mgr.smithi156.zkvylv' 2024-07-16T09:46:53.926 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:53 smithi156 ceph-mon[29302]: from='mgr.14118 172.21.15.156:0/3208530346' entity='mgr.smithi156.zkvylv' 2024-07-16T09:46:53.926 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:53 smithi156 ceph-mon[29302]: from='mgr.14118 172.21.15.156:0/3208530346' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-16T09:46:53.926 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:53 smithi156 ceph-mon[29302]: from='mgr.14118 172.21.15.156:0/3208530346' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-16T09:46:53.926 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:53 smithi156 ceph-mon[29302]: from='mgr.14118 172.21.15.156:0/3208530346' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-16T09:46:53.926 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:53 smithi156 ceph-mon[29302]: from='mgr.14118 172.21.15.156:0/3208530346' entity='mgr.smithi156.zkvylv' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi156.zkvylv/mirror_snapshot_schedule"}]: dispatch 2024-07-16T09:46:53.926 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:53 smithi156 ceph-mon[29302]: from='mgr.14118 172.21.15.156:0/3208530346' entity='mgr.smithi156.zkvylv' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi156.zkvylv/trash_purge_schedule"}]: dispatch 2024-07-16T09:46:54.924 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:54 smithi156 ceph-mon[29302]: Found migration_current of "None". Setting to last migration. 2024-07-16T09:46:54.924 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:54 smithi156 ceph-mon[29302]: mgrmap e7: smithi156.zkvylv(active, since 1.00529s) 2024-07-16T09:46:55.185 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout { 2024-07-16T09:46:55.185 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-07-16T09:46:55.185 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout "initialized": true 2024-07-16T09:46:55.185 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout } 2024-07-16T09:46:55.185 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:51.424+0000 7fc444343640 1 Processor -- start 2024-07-16T09:46:55.185 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:51.424+0000 7fc444343640 1 -- start start 2024-07-16T09:46:55.186 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:51.425+0000 7fc444343640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc43c100270 0x7fc43c100680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:46:55.186 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:51.425+0000 7fc444343640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc43c100bc0 con 0x7fc43c100270 2024-07-16T09:46:55.186 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:51.425+0000 7fc4420df640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc43c100270 0x7fc43c100680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:46:55.186 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:51.425+0000 7fc4420df640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc43c100270 0x7fc43c100680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:47798/0 (socket says 172.21.15.156:47798) 2024-07-16T09:46:55.186 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:51.425+0000 7fc4420df640 1 -- 172.21.15.156:0/513145597 learned_addr learned my addr 172.21.15.156:0/513145597 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T09:46:55.186 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:51.425+0000 7fc4420df640 1 -- 172.21.15.156:0/513145597 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc43c100d00 con 0x7fc43c100270 2024-07-16T09:46:55.186 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:51.426+0000 7fc4420df640 1 --2- 172.21.15.156:0/513145597 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc43c100270 0x7fc43c100680 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fc42c009840 tx=0x7fc42c0091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f989318f5621b5a server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:46:55.186 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:51.426+0000 7fc4410dd640 1 -- 172.21.15.156:0/513145597 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc42c032440 con 0x7fc43c100270 2024-07-16T09:46:55.186 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:51.426+0000 7fc4410dd640 1 -- 172.21.15.156:0/513145597 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7fc42c032a30 con 0x7fc43c100270 2024-07-16T09:46:55.186 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:51.426+0000 7fc4410dd640 1 -- 172.21.15.156:0/513145597 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc42c031ad0 con 0x7fc43c100270 2024-07-16T09:46:55.186 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:51.426+0000 7fc444343640 1 -- 172.21.15.156:0/513145597 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc43c100270 msgr2=0x7fc43c100680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:46:55.186 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:51.426+0000 7fc444343640 1 --2- 172.21.15.156:0/513145597 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc43c100270 0x7fc43c100680 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fc42c009840 tx=0x7fc42c0091d0 comp rx=0 tx=0).stop 2024-07-16T09:46:55.187 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:51.427+0000 7fc444343640 1 -- 172.21.15.156:0/513145597 shutdown_connections 2024-07-16T09:46:55.187 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:51.427+0000 7fc444343640 1 --2- 172.21.15.156:0/513145597 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc43c100270 0x7fc43c100680 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:46:55.187 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:51.427+0000 7fc444343640 1 -- 172.21.15.156:0/513145597 >> 172.21.15.156:0/513145597 conn(0x7fc43c0fbc50 msgr2=0x7fc43c0fe0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:46:55.187 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:51.427+0000 7fc444343640 1 -- 172.21.15.156:0/513145597 shutdown_connections 2024-07-16T09:46:55.187 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:51.427+0000 7fc444343640 1 -- 172.21.15.156:0/513145597 wait complete. 2024-07-16T09:46:55.187 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:51.427+0000 7fc444343640 1 Processor -- start 2024-07-16T09:46:55.187 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:51.427+0000 7fc444343640 1 -- start start 2024-07-16T09:46:55.187 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:51.427+0000 7fc444343640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc43c100270 0x7fc43c110b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:46:55.187 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:51.427+0000 7fc444343640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc43c111050 con 0x7fc43c100270 2024-07-16T09:46:55.187 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:51.427+0000 7fc4420df640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc43c100270 0x7fc43c110b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:46:55.187 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:51.427+0000 7fc4420df640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc43c100270 0x7fc43c110b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:47810/0 (socket says 172.21.15.156:47810) 2024-07-16T09:46:55.187 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:51.427+0000 7fc4420df640 1 -- 172.21.15.156:0/3431881713 learned_addr learned my addr 172.21.15.156:0/3431881713 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T09:46:55.188 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:51.428+0000 7fc4420df640 1 -- 172.21.15.156:0/3431881713 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc42c009450 con 0x7fc43c100270 2024-07-16T09:46:55.188 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:51.428+0000 7fc4420df640 1 --2- 172.21.15.156:0/3431881713 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc43c100270 0x7fc43c110b10 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fc42c0317b0 tx=0x7fc42c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:46:55.188 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:51.428+0000 7fc432ffd640 1 -- 172.21.15.156:0/3431881713 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc42c031e90 con 0x7fc43c100270 2024-07-16T09:46:55.188 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:51.428+0000 7fc432ffd640 1 -- 172.21.15.156:0/3431881713 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7fc42c03daf0 con 0x7fc43c100270 2024-07-16T09:46:55.188 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:51.428+0000 7fc444343640 1 -- 172.21.15.156:0/3431881713 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc43c111250 con 0x7fc43c100270 2024-07-16T09:46:55.188 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:51.428+0000 7fc432ffd640 1 -- 172.21.15.156:0/3431881713 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc42c03baa0 con 0x7fc43c100270 2024-07-16T09:46:55.188 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:51.428+0000 7fc444343640 1 -- 172.21.15.156:0/3431881713 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc43c1116c0 con 0x7fc43c100270 2024-07-16T09:46:55.188 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:51.429+0000 7fc432ffd640 1 -- 172.21.15.156:0/3431881713 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 42636+0+0 (secure 0 0 0) 0x7fc42c039070 con 0x7fc43c100270 2024-07-16T09:46:55.188 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:51.429+0000 7fc432ffd640 1 --2- 172.21.15.156:0/3431881713 >> [v2:172.21.15.156:6800/2871070852,v1:172.21.15.156:6801/2871070852] conn(0x7fc4280357d0 0x7fc428037c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:46:55.188 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:51.429+0000 7fc444343640 1 -- 172.21.15.156:0/3431881713 --> [v2:172.21.15.156:6800/2871070852,v1:172.21.15.156:6801/2871070852] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc43c1116c0 con 0x7fc4280357d0 2024-07-16T09:46:55.188 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:51.429+0000 7fc432ffd640 1 -- 172.21.15.156:0/3431881713 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7fc42c0684e0 con 0x7fc43c100270 2024-07-16T09:46:55.188 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:51.429+0000 7fc4418de640 1 -- 172.21.15.156:0/3431881713 >> [v2:172.21.15.156:6800/2871070852,v1:172.21.15.156:6801/2871070852] conn(0x7fc4280357d0 msgr2=0x7fc428037c80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.156:6800/2871070852 2024-07-16T09:46:55.188 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:51.429+0000 7fc4418de640 1 --2- 172.21.15.156:0/3431881713 >> [v2:172.21.15.156:6800/2871070852,v1:172.21.15.156:6801/2871070852] conn(0x7fc4280357d0 0x7fc428037c80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-16T09:46:55.189 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:51.630+0000 7fc4418de640 1 -- 172.21.15.156:0/3431881713 >> [v2:172.21.15.156:6800/2871070852,v1:172.21.15.156:6801/2871070852] conn(0x7fc4280357d0 msgr2=0x7fc428037c80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.156:6800/2871070852 2024-07-16T09:46:55.189 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:51.630+0000 7fc4418de640 1 --2- 172.21.15.156:0/3431881713 >> [v2:172.21.15.156:6800/2871070852,v1:172.21.15.156:6801/2871070852] conn(0x7fc4280357d0 0x7fc428037c80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-16T09:46:55.189 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:52.031+0000 7fc4418de640 1 -- 172.21.15.156:0/3431881713 >> [v2:172.21.15.156:6800/2871070852,v1:172.21.15.156:6801/2871070852] conn(0x7fc4280357d0 msgr2=0x7fc428037c80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.156:6800/2871070852 2024-07-16T09:46:55.189 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:52.031+0000 7fc4418de640 1 --2- 172.21.15.156:0/3431881713 >> [v2:172.21.15.156:6800/2871070852,v1:172.21.15.156:6801/2871070852] conn(0x7fc4280357d0 0x7fc428037c80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-07-16T09:46:55.189 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:52.832+0000 7fc4418de640 1 -- 172.21.15.156:0/3431881713 >> [v2:172.21.15.156:6800/2871070852,v1:172.21.15.156:6801/2871070852] conn(0x7fc4280357d0 msgr2=0x7fc428037c80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.156:6800/2871070852 2024-07-16T09:46:55.189 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:52.832+0000 7fc4418de640 1 --2- 172.21.15.156:0/3431881713 >> [v2:172.21.15.156:6800/2871070852,v1:172.21.15.156:6801/2871070852] conn(0x7fc4280357d0 0x7fc428037c80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-07-16T09:46:55.189 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:53.643+0000 7fc432ffd640 1 -- 172.21.15.156:0/3431881713 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mgrmap(e 6) v1 ==== 42403+0+0 (secure 0 0 0) 0x7fc42c045cb0 con 0x7fc43c100270 2024-07-16T09:46:55.189 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:53.643+0000 7fc432ffd640 1 -- 172.21.15.156:0/3431881713 >> [v2:172.21.15.156:6800/2871070852,v1:172.21.15.156:6801/2871070852] conn(0x7fc4280357d0 msgr2=0x7fc428037c80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-16T09:46:55.189 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:53.643+0000 7fc432ffd640 1 --2- 172.21.15.156:0/3431881713 >> [v2:172.21.15.156:6800/2871070852,v1:172.21.15.156:6801/2871070852] conn(0x7fc4280357d0 0x7fc428037c80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:46:55.189 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:54.644+0000 7fc432ffd640 1 -- 172.21.15.156:0/3431881713 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mgrmap(e 7) v1 ==== 42530+0+0 (secure 0 0 0) 0x7fc42c0464f0 con 0x7fc43c100270 2024-07-16T09:46:55.189 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:54.644+0000 7fc432ffd640 1 --2- 172.21.15.156:0/3431881713 >> [v2:172.21.15.156:6800/1023000918,v1:172.21.15.156:6801/1023000918] conn(0x7fc428038f40 0x7fc42803b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:46:55.189 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:54.644+0000 7fc432ffd640 1 -- 172.21.15.156:0/3431881713 --> [v2:172.21.15.156:6800/1023000918,v1:172.21.15.156:6801/1023000918] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc43c1116c0 con 0x7fc428038f40 2024-07-16T09:46:55.189 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:54.644+0000 7fc4418de640 1 --2- 172.21.15.156:0/3431881713 >> [v2:172.21.15.156:6800/1023000918,v1:172.21.15.156:6801/1023000918] conn(0x7fc428038f40 0x7fc42803b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:46:55.190 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:54.645+0000 7fc4418de640 1 --2- 172.21.15.156:0/3431881713 >> [v2:172.21.15.156:6800/1023000918,v1:172.21.15.156:6801/1023000918] conn(0x7fc428038f40 0x7fc42803b320 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fc438003940 tx=0x7fc438009350 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:46:55.190 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:54.646+0000 7fc432ffd640 1 -- 172.21.15.156:0/3431881713 <== mgr.14118 v2:172.21.15.156:6800/1023000918 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+6549 (secure 0 0 0) 0x7fc43c1116c0 con 0x7fc428038f40 2024-07-16T09:46:55.190 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:54.651+0000 7fc444343640 1 -- 172.21.15.156:0/3431881713 --> [v2:172.21.15.156:6800/1023000918,v1:172.21.15.156:6801/1023000918] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7fc43c04b530 con 0x7fc428038f40 2024-07-16T09:46:55.190 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:54.651+0000 7fc432ffd640 1 -- 172.21.15.156:0/3431881713 <== mgr.14118 v2:172.21.15.156:6800/1023000918 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7fc43c04b530 con 0x7fc428038f40 2024-07-16T09:46:55.190 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:54.651+0000 7fc444343640 1 -- 172.21.15.156:0/3431881713 >> [v2:172.21.15.156:6800/1023000918,v1:172.21.15.156:6801/1023000918] conn(0x7fc428038f40 msgr2=0x7fc42803b320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:46:55.190 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:54.651+0000 7fc444343640 1 --2- 172.21.15.156:0/3431881713 >> [v2:172.21.15.156:6800/1023000918,v1:172.21.15.156:6801/1023000918] conn(0x7fc428038f40 0x7fc42803b320 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fc438003940 tx=0x7fc438009350 comp rx=0 tx=0).stop 2024-07-16T09:46:55.190 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:54.651+0000 7fc444343640 1 -- 172.21.15.156:0/3431881713 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc43c100270 msgr2=0x7fc43c110b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:46:55.190 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:54.651+0000 7fc444343640 1 --2- 172.21.15.156:0/3431881713 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc43c100270 0x7fc43c110b10 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fc42c0317b0 tx=0x7fc42c008040 comp rx=0 tx=0).stop 2024-07-16T09:46:55.190 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:54.652+0000 7fc444343640 1 -- 172.21.15.156:0/3431881713 shutdown_connections 2024-07-16T09:46:55.190 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:54.652+0000 7fc444343640 1 --2- 172.21.15.156:0/3431881713 >> [v2:172.21.15.156:6800/1023000918,v1:172.21.15.156:6801/1023000918] conn(0x7fc428038f40 0x7fc42803b320 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:46:55.190 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:54.652+0000 7fc444343640 1 --2- 172.21.15.156:0/3431881713 >> [v2:172.21.15.156:6800/2871070852,v1:172.21.15.156:6801/2871070852] conn(0x7fc4280357d0 0x7fc428037c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:46:55.190 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:54.652+0000 7fc444343640 1 --2- 172.21.15.156:0/3431881713 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc43c100270 0x7fc43c110b10 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:46:55.191 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:54.652+0000 7fc444343640 1 -- 172.21.15.156:0/3431881713 >> 172.21.15.156:0/3431881713 conn(0x7fc43c0fbc50 msgr2=0x7fc43c0fc710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:46:55.191 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:54.652+0000 7fc444343640 1 -- 172.21.15.156:0/3431881713 shutdown_connections 2024-07-16T09:46:55.191 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:54.652+0000 7fc444343640 1 -- 172.21.15.156:0/3431881713 wait complete. 2024-07-16T09:46:55.191 INFO:teuthology.orchestra.run.smithi156.stdout:mgr epoch 5 is available 2024-07-16T09:46:55.191 INFO:teuthology.orchestra.run.smithi156.stdout:Setting orchestrator backend to cephadm... 2024-07-16T09:46:55.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:55 smithi156 ceph-mon[29302]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-07-16T09:46:55.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:55 smithi156 ceph-mon[29302]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-07-16T09:46:56.282 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:55.520+0000 7fae3a127640 1 Processor -- start 2024-07-16T09:46:56.282 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:55.520+0000 7fae3a127640 1 -- start start 2024-07-16T09:46:56.282 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:55.521+0000 7fae3a127640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fae34075ba0 0x7fae34104a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:46:56.282 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:55.521+0000 7fae3a127640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae34104f70 con 0x7fae34075ba0 2024-07-16T09:46:56.282 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:55.521+0000 7fae337fe640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fae34075ba0 0x7fae34104a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:46:56.282 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:55.521+0000 7fae337fe640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fae34075ba0 0x7fae34104a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:45290/0 (socket says 172.21.15.156:45290) 2024-07-16T09:46:56.282 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:55.521+0000 7fae337fe640 1 -- 172.21.15.156:0/4222282793 learned_addr learned my addr 172.21.15.156:0/4222282793 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T09:46:56.282 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:55.521+0000 7fae337fe640 1 -- 172.21.15.156:0/4222282793 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae341050b0 con 0x7fae34075ba0 2024-07-16T09:46:56.282 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:55.522+0000 7fae337fe640 1 --2- 172.21.15.156:0/4222282793 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fae34075ba0 0x7fae34104a30 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fae24009840 tx=0x7fae240091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0ac98e0c422cafd server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:46:56.283 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:55.522+0000 7fae327fc640 1 -- 172.21.15.156:0/4222282793 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fae24032440 con 0x7fae34075ba0 2024-07-16T09:46:56.283 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:55.522+0000 7fae327fc640 1 -- 172.21.15.156:0/4222282793 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7fae24032a30 con 0x7fae34075ba0 2024-07-16T09:46:56.283 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:55.522+0000 7fae327fc640 1 -- 172.21.15.156:0/4222282793 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fae24031ad0 con 0x7fae34075ba0 2024-07-16T09:46:56.283 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:55.523+0000 7fae3a127640 1 -- 172.21.15.156:0/4222282793 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fae34075ba0 msgr2=0x7fae34104a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:46:56.283 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:55.523+0000 7fae3a127640 1 --2- 172.21.15.156:0/4222282793 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fae34075ba0 0x7fae34104a30 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fae24009840 tx=0x7fae240091d0 comp rx=0 tx=0).stop 2024-07-16T09:46:56.283 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:55.523+0000 7fae3a127640 1 -- 172.21.15.156:0/4222282793 shutdown_connections 2024-07-16T09:46:56.283 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:55.523+0000 7fae3a127640 1 --2- 172.21.15.156:0/4222282793 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fae34075ba0 0x7fae34104a30 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:46:56.283 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:55.523+0000 7fae3a127640 1 -- 172.21.15.156:0/4222282793 >> 172.21.15.156:0/4222282793 conn(0x7fae340fc520 msgr2=0x7fae340fe930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:46:56.283 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:55.523+0000 7fae3a127640 1 -- 172.21.15.156:0/4222282793 shutdown_connections 2024-07-16T09:46:56.283 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:55.523+0000 7fae3a127640 1 -- 172.21.15.156:0/4222282793 wait complete. 2024-07-16T09:46:56.283 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:55.524+0000 7fae3a127640 1 Processor -- start 2024-07-16T09:46:56.283 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:55.524+0000 7fae3a127640 1 -- start start 2024-07-16T09:46:56.284 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:55.524+0000 7fae3a127640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fae34075ba0 0x7fae34110a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:46:56.284 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:55.524+0000 7fae3a127640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae34110fc0 con 0x7fae34075ba0 2024-07-16T09:46:56.284 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:55.524+0000 7fae337fe640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fae34075ba0 0x7fae34110a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:46:56.284 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:55.524+0000 7fae337fe640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fae34075ba0 0x7fae34110a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:45300/0 (socket says 172.21.15.156:45300) 2024-07-16T09:46:56.284 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:55.524+0000 7fae337fe640 1 -- 172.21.15.156:0/3597619554 learned_addr learned my addr 172.21.15.156:0/3597619554 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T09:46:56.284 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:55.524+0000 7fae337fe640 1 -- 172.21.15.156:0/3597619554 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae24009450 con 0x7fae34075ba0 2024-07-16T09:46:56.284 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:55.524+0000 7fae337fe640 1 --2- 172.21.15.156:0/3597619554 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fae34075ba0 0x7fae34110a80 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fae24031b90 tx=0x7fae24008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:46:56.284 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:55.525+0000 7fae1bfff640 1 -- 172.21.15.156:0/3597619554 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fae240328b0 con 0x7fae34075ba0 2024-07-16T09:46:56.284 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:55.525+0000 7fae1bfff640 1 -- 172.21.15.156:0/3597619554 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7fae2403d8e0 con 0x7fae34075ba0 2024-07-16T09:46:56.284 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:55.525+0000 7fae3a127640 1 -- 172.21.15.156:0/3597619554 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae341111c0 con 0x7fae34075ba0 2024-07-16T09:46:56.284 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:55.525+0000 7fae1bfff640 1 -- 172.21.15.156:0/3597619554 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fae2403ba20 con 0x7fae34075ba0 2024-07-16T09:46:56.284 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:55.525+0000 7fae3a127640 1 -- 172.21.15.156:0/3597619554 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae34111630 con 0x7fae34075ba0 2024-07-16T09:46:56.284 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:55.526+0000 7fae1bfff640 1 -- 172.21.15.156:0/3597619554 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 7) v1 ==== 42530+0+0 (secure 0 0 0) 0x7fae24039070 con 0x7fae34075ba0 2024-07-16T09:46:56.285 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:55.526+0000 7fae1bfff640 1 --2- 172.21.15.156:0/3597619554 >> [v2:172.21.15.156:6800/1023000918,v1:172.21.15.156:6801/1023000918] conn(0x7fae1c0356a0 0x7fae1c037b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:46:56.285 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:55.526+0000 7fae1bfff640 1 -- 172.21.15.156:0/3597619554 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7fae24068730 con 0x7fae34075ba0 2024-07-16T09:46:56.285 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:55.526+0000 7fae32ffd640 1 --2- 172.21.15.156:0/3597619554 >> [v2:172.21.15.156:6800/1023000918,v1:172.21.15.156:6801/1023000918] conn(0x7fae1c0356a0 0x7fae1c037b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:46:56.285 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:55.526+0000 7fae3a127640 1 -- 172.21.15.156:0/3597619554 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae3405eb80 con 0x7fae34075ba0 2024-07-16T09:46:56.285 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:55.526+0000 7fae32ffd640 1 --2- 172.21.15.156:0/3597619554 >> [v2:172.21.15.156:6800/1023000918,v1:172.21.15.156:6801/1023000918] conn(0x7fae1c0356a0 0x7fae1c037b50 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fae28006fd0 tx=0x7fae28006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:46:56.285 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:55.533+0000 7fae1bfff640 1 -- 172.21.15.156:0/3597619554 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fae24041020 con 0x7fae34075ba0 2024-07-16T09:46:56.285 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:55.695+0000 7fae1bfff640 1 -- 172.21.15.156:0/3597619554 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mgrmap(e 8) v1 ==== 42636+0+0 (secure 0 0 0) 0x7fae2403da50 con 0x7fae34075ba0 2024-07-16T09:46:56.285 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:55.731+0000 7fae3a127640 1 -- 172.21.15.156:0/3597619554 --> [v2:172.21.15.156:6800/1023000918,v1:172.21.15.156:6801/1023000918] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) v1 -- 0x7fae340fd6a0 con 0x7fae1c0356a0 2024-07-16T09:46:56.285 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:55.738+0000 7fae1bfff640 1 -- 172.21.15.156:0/3597619554 <== mgr.14118 v2:172.21.15.156:6800/1023000918 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fae340fd6a0 con 0x7fae1c0356a0 2024-07-16T09:46:56.285 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:55.740+0000 7fae3a127640 1 -- 172.21.15.156:0/3597619554 >> [v2:172.21.15.156:6800/1023000918,v1:172.21.15.156:6801/1023000918] conn(0x7fae1c0356a0 msgr2=0x7fae1c037b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:46:56.285 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:55.740+0000 7fae3a127640 1 --2- 172.21.15.156:0/3597619554 >> [v2:172.21.15.156:6800/1023000918,v1:172.21.15.156:6801/1023000918] conn(0x7fae1c0356a0 0x7fae1c037b50 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fae28006fd0 tx=0x7fae28006db0 comp rx=0 tx=0).stop 2024-07-16T09:46:56.285 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:55.740+0000 7fae3a127640 1 -- 172.21.15.156:0/3597619554 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fae34075ba0 msgr2=0x7fae34110a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:46:56.285 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:55.740+0000 7fae3a127640 1 --2- 172.21.15.156:0/3597619554 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fae34075ba0 0x7fae34110a80 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fae24031b90 tx=0x7fae24008040 comp rx=0 tx=0).stop 2024-07-16T09:46:56.286 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:55.740+0000 7fae3a127640 1 -- 172.21.15.156:0/3597619554 shutdown_connections 2024-07-16T09:46:56.286 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:55.740+0000 7fae3a127640 1 --2- 172.21.15.156:0/3597619554 >> [v2:172.21.15.156:6800/1023000918,v1:172.21.15.156:6801/1023000918] conn(0x7fae1c0356a0 0x7fae1c037b50 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:46:56.286 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:55.740+0000 7fae3a127640 1 --2- 172.21.15.156:0/3597619554 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fae34075ba0 0x7fae34110a80 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:46:56.286 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:55.740+0000 7fae3a127640 1 -- 172.21.15.156:0/3597619554 >> 172.21.15.156:0/3597619554 conn(0x7fae340fc520 msgr2=0x7fae340fcf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:46:56.286 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:55.740+0000 7fae3a127640 1 -- 172.21.15.156:0/3597619554 shutdown_connections 2024-07-16T09:46:56.286 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:55.740+0000 7fae3a127640 1 -- 172.21.15.156:0/3597619554 wait complete. 2024-07-16T09:46:56.886 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:56 smithi156 ceph-mon[29302]: [16/Jul/2024:09:46:55] ENGINE Bus STARTING 2024-07-16T09:46:56.887 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:56 smithi156 ceph-mon[29302]: mgrmap e8: smithi156.zkvylv(active, since 2s) 2024-07-16T09:46:56.887 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:56 smithi156 ceph-mon[29302]: from='client.14130 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-07-16T09:46:56.887 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:56 smithi156 ceph-mon[29302]: from='mgr.14118 172.21.15.156:0/3208530346' entity='mgr.smithi156.zkvylv' 2024-07-16T09:46:56.887 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:56 smithi156 ceph-mon[29302]: from='mgr.14118 172.21.15.156:0/3208530346' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-16T09:46:56.887 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:56 smithi156 ceph-mon[29302]: [16/Jul/2024:09:46:55] ENGINE Serving on https://172.21.15.156:7150 2024-07-16T09:46:56.887 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:56 smithi156 ceph-mon[29302]: [16/Jul/2024:09:46:55] ENGINE Bus STARTED 2024-07-16T09:46:56.887 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:56 smithi156 ceph-mon[29302]: [16/Jul/2024:09:46:55] ENGINE Error in HTTPServer.serve 2024-07-16T09:46:56.887 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: Traceback (most recent call last): 2024-07-16T09:46:56.887 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-07-16T09:46:56.887 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: self._connections.run(self.expiration_interval) 2024-07-16T09:46:56.887 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-07-16T09:46:56.888 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: self._run(expiration_interval) 2024-07-16T09:46:56.888 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-07-16T09:46:56.888 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-07-16T09:46:56.888 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-07-16T09:46:56.888 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-07-16T09:46:56.888 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-07-16T09:46:56.888 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: s = self.context.wrap_socket( 2024-07-16T09:46:56.888 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-07-16T09:46:56.888 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: return self.sslsocket_class._create( 2024-07-16T09:46:56.888 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-07-16T09:46:56.888 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: self.do_handshake() 2024-07-16T09:46:56.888 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-07-16T09:46:56.888 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: self._sslobj.do_handshake() 2024-07-16T09:46:56.889 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-07-16T09:46:56.889 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:56 smithi156 ceph-mon[29302]: from='mgr.14118 172.21.15.156:0/3208530346' entity='mgr.smithi156.zkvylv' 2024-07-16T09:46:56.889 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:56 smithi156 ceph-mon[29302]: from='mgr.14118 172.21.15.156:0/3208530346' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-16T09:46:57.464 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout value unchanged 2024-07-16T09:46:57.464 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:56.709+0000 7ff7ef39c640 1 Processor -- start 2024-07-16T09:46:57.464 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:56.709+0000 7ff7ef39c640 1 -- start start 2024-07-16T09:46:57.464 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:56.710+0000 7ff7ef39c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff7e8100ce0 0x7ff7e81010f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:46:57.464 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:56.710+0000 7ff7ef39c640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7e8101630 con 0x7ff7e8100ce0 2024-07-16T09:46:57.464 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:56.710+0000 7ff7ed138640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff7e8100ce0 0x7ff7e81010f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:46:57.465 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:56.710+0000 7ff7ed138640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff7e8100ce0 0x7ff7e81010f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:45316/0 (socket says 172.21.15.156:45316) 2024-07-16T09:46:57.465 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:56.710+0000 7ff7ed138640 1 -- 172.21.15.156:0/3092268665 learned_addr learned my addr 172.21.15.156:0/3092268665 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T09:46:57.465 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:56.710+0000 7ff7ed138640 1 -- 172.21.15.156:0/3092268665 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7e8101770 con 0x7ff7e8100ce0 2024-07-16T09:46:57.465 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:56.711+0000 7ff7ed138640 1 --2- 172.21.15.156:0/3092268665 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff7e8100ce0 0x7ff7e81010f0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7ff7e400baf0 tx=0x7ff7e400b460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ab461efa6428cd5 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:46:57.465 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:56.711+0000 7ff7dffff640 1 -- 172.21.15.156:0/3092268665 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff7e40327a0 con 0x7ff7e8100ce0 2024-07-16T09:46:57.465 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:56.711+0000 7ff7dffff640 1 -- 172.21.15.156:0/3092268665 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7ff7e4032d90 con 0x7ff7e8100ce0 2024-07-16T09:46:57.465 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:56.711+0000 7ff7dffff640 1 -- 172.21.15.156:0/3092268665 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff7e4031e70 con 0x7ff7e8100ce0 2024-07-16T09:46:57.465 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:56.711+0000 7ff7ef39c640 1 -- 172.21.15.156:0/3092268665 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff7e8100ce0 msgr2=0x7ff7e81010f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:46:57.465 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:56.711+0000 7ff7ef39c640 1 --2- 172.21.15.156:0/3092268665 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff7e8100ce0 0x7ff7e81010f0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7ff7e400baf0 tx=0x7ff7e400b460 comp rx=0 tx=0).stop 2024-07-16T09:46:57.465 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:56.712+0000 7ff7ef39c640 1 -- 172.21.15.156:0/3092268665 shutdown_connections 2024-07-16T09:46:57.465 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:56.712+0000 7ff7ef39c640 1 --2- 172.21.15.156:0/3092268665 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff7e8100ce0 0x7ff7e81010f0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:46:57.466 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:56.712+0000 7ff7ef39c640 1 -- 172.21.15.156:0/3092268665 >> 172.21.15.156:0/3092268665 conn(0x7ff7e80fc4c0 msgr2=0x7ff7e80fe8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:46:57.466 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:56.712+0000 7ff7ef39c640 1 -- 172.21.15.156:0/3092268665 shutdown_connections 2024-07-16T09:46:57.466 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:56.712+0000 7ff7ef39c640 1 -- 172.21.15.156:0/3092268665 wait complete. 2024-07-16T09:46:57.466 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:56.712+0000 7ff7ef39c640 1 Processor -- start 2024-07-16T09:46:57.466 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:56.712+0000 7ff7ef39c640 1 -- start start 2024-07-16T09:46:57.466 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:56.712+0000 7ff7ef39c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff7e8100ce0 0x7ff7e810e980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:46:57.466 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:56.712+0000 7ff7ef39c640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7e810eec0 con 0x7ff7e8100ce0 2024-07-16T09:46:57.466 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:56.712+0000 7ff7ed138640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff7e8100ce0 0x7ff7e810e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:46:57.466 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:56.712+0000 7ff7ed138640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff7e8100ce0 0x7ff7e810e980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:45330/0 (socket says 172.21.15.156:45330) 2024-07-16T09:46:57.466 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:56.712+0000 7ff7ed138640 1 -- 172.21.15.156:0/4002193121 learned_addr learned my addr 172.21.15.156:0/4002193121 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T09:46:57.467 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:56.713+0000 7ff7ed138640 1 -- 172.21.15.156:0/4002193121 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7e400b700 con 0x7ff7e8100ce0 2024-07-16T09:46:57.467 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:56.713+0000 7ff7ed138640 1 --2- 172.21.15.156:0/4002193121 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff7e8100ce0 0x7ff7e810e980 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7ff7e4006f00 tx=0x7ff7e400b460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:46:57.467 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:56.713+0000 7ff7ddffb640 1 -- 172.21.15.156:0/4002193121 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff7e4032a30 con 0x7ff7e8100ce0 2024-07-16T09:46:57.467 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:56.713+0000 7ff7ef39c640 1 -- 172.21.15.156:0/4002193121 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7e810f0c0 con 0x7ff7e8100ce0 2024-07-16T09:46:57.467 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:56.713+0000 7ff7ddffb640 1 -- 172.21.15.156:0/4002193121 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7ff7e4032b90 con 0x7ff7e8100ce0 2024-07-16T09:46:57.467 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:56.713+0000 7ff7ddffb640 1 -- 172.21.15.156:0/4002193121 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff7e403a730 con 0x7ff7e8100ce0 2024-07-16T09:46:57.467 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:56.713+0000 7ff7ef39c640 1 -- 172.21.15.156:0/4002193121 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7e810f4d0 con 0x7ff7e8100ce0 2024-07-16T09:46:57.468 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:56.714+0000 7ff7ddffb640 1 -- 172.21.15.156:0/4002193121 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 8) v1 ==== 42636+0+0 (secure 0 0 0) 0x7ff7e403a970 con 0x7ff7e8100ce0 2024-07-16T09:46:57.468 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:56.714+0000 7ff7ddffb640 1 --2- 172.21.15.156:0/4002193121 >> [v2:172.21.15.156:6800/1023000918,v1:172.21.15.156:6801/1023000918] conn(0x7ff7d4035780 0x7ff7d4037c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:46:57.468 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:56.714+0000 7ff7ddffb640 1 -- 172.21.15.156:0/4002193121 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7ff7e406ac80 con 0x7ff7e8100ce0 2024-07-16T09:46:57.468 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:56.714+0000 7ff7ef39c640 1 -- 172.21.15.156:0/4002193121 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7e804b530 con 0x7ff7e8100ce0 2024-07-16T09:46:57.468 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:56.714+0000 7ff7ec937640 1 --2- 172.21.15.156:0/4002193121 >> [v2:172.21.15.156:6800/1023000918,v1:172.21.15.156:6801/1023000918] conn(0x7ff7d4035780 0x7ff7d4037c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:46:57.468 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:56.715+0000 7ff7ec937640 1 --2- 172.21.15.156:0/4002193121 >> [v2:172.21.15.156:6800/1023000918,v1:172.21.15.156:6801/1023000918] conn(0x7ff7d4035780 0x7ff7d4037c30 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ff7d8006fd0 tx=0x7ff7d8006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:46:57.468 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:56.719+0000 7ff7ddffb640 1 -- 172.21.15.156:0/4002193121 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7ff7e4036030 con 0x7ff7e8100ce0 2024-07-16T09:46:57.468 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:56.885+0000 7ff7ef39c640 1 -- 172.21.15.156:0/4002193121 --> [v2:172.21.15.156:6800/1023000918,v1:172.21.15.156:6801/1023000918] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) v1 -- 0x7ff7e810ff70 con 0x7ff7d4035780 2024-07-16T09:46:57.469 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:56.886+0000 7ff7ddffb640 1 -- 172.21.15.156:0/4002193121 <== mgr.14118 v2:172.21.15.156:6800/1023000918 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+16 (secure 0 0 0) 0x7ff7e810ff70 con 0x7ff7d4035780 2024-07-16T09:46:57.469 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:56.888+0000 7ff7ef39c640 1 -- 172.21.15.156:0/4002193121 >> [v2:172.21.15.156:6800/1023000918,v1:172.21.15.156:6801/1023000918] conn(0x7ff7d4035780 msgr2=0x7ff7d4037c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:46:57.469 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:56.888+0000 7ff7ef39c640 1 --2- 172.21.15.156:0/4002193121 >> [v2:172.21.15.156:6800/1023000918,v1:172.21.15.156:6801/1023000918] conn(0x7ff7d4035780 0x7ff7d4037c30 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ff7d8006fd0 tx=0x7ff7d8006db0 comp rx=0 tx=0).stop 2024-07-16T09:46:57.469 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:56.888+0000 7ff7ef39c640 1 -- 172.21.15.156:0/4002193121 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff7e8100ce0 msgr2=0x7ff7e810e980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:46:57.469 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:56.888+0000 7ff7ef39c640 1 --2- 172.21.15.156:0/4002193121 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff7e8100ce0 0x7ff7e810e980 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7ff7e4006f00 tx=0x7ff7e400b460 comp rx=0 tx=0).stop 2024-07-16T09:46:57.469 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:56.888+0000 7ff7ef39c640 1 -- 172.21.15.156:0/4002193121 shutdown_connections 2024-07-16T09:46:57.469 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:56.888+0000 7ff7ef39c640 1 --2- 172.21.15.156:0/4002193121 >> [v2:172.21.15.156:6800/1023000918,v1:172.21.15.156:6801/1023000918] conn(0x7ff7d4035780 0x7ff7d4037c30 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:46:57.469 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:56.888+0000 7ff7ef39c640 1 --2- 172.21.15.156:0/4002193121 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff7e8100ce0 0x7ff7e810e980 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:46:57.469 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:56.888+0000 7ff7ef39c640 1 -- 172.21.15.156:0/4002193121 >> 172.21.15.156:0/4002193121 conn(0x7ff7e80fc4c0 msgr2=0x7ff7e81051e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:46:57.469 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:56.888+0000 7ff7ef39c640 1 -- 172.21.15.156:0/4002193121 shutdown_connections 2024-07-16T09:46:57.469 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:56.888+0000 7ff7ef39c640 1 -- 172.21.15.156:0/4002193121 wait complete. 2024-07-16T09:46:57.470 INFO:teuthology.orchestra.run.smithi156.stdout:Generating ssh key... 2024-07-16T09:46:57.783 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:57 smithi156 ceph-mon[29302]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-07-16T09:46:58.702 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:57.855+0000 7f11be86e640 1 Processor -- start 2024-07-16T09:46:58.703 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:57.855+0000 7f11be86e640 1 -- start start 2024-07-16T09:46:58.703 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:57.855+0000 7f11be86e640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f11b81012d0 0x7f11b81016e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:46:58.703 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:57.855+0000 7f11be86e640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11b8101c20 con 0x7f11b81012d0 2024-07-16T09:46:58.703 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:57.855+0000 7f11b7fff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f11b81012d0 0x7f11b81016e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:46:58.703 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:57.855+0000 7f11b7fff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f11b81012d0 0x7f11b81016e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:45332/0 (socket says 172.21.15.156:45332) 2024-07-16T09:46:58.703 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:57.855+0000 7f11b7fff640 1 -- 172.21.15.156:0/3682637110 learned_addr learned my addr 172.21.15.156:0/3682637110 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T09:46:58.703 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:57.856+0000 7f11b7fff640 1 -- 172.21.15.156:0/3682637110 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11b8101d60 con 0x7f11b81012d0 2024-07-16T09:46:58.703 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:57.856+0000 7f11b7fff640 1 --2- 172.21.15.156:0/3682637110 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f11b81012d0 0x7f11b81016e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f11a800baf0 tx=0x7f11a800b460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eae05acc33546607 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:46:58.703 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:57.856+0000 7f11b6ffd640 1 -- 172.21.15.156:0/3682637110 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f11a80327a0 con 0x7f11b81012d0 2024-07-16T09:46:58.703 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:57.856+0000 7f11b6ffd640 1 -- 172.21.15.156:0/3682637110 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f11a8032d90 con 0x7f11b81012d0 2024-07-16T09:46:58.703 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:57.856+0000 7f11b6ffd640 1 -- 172.21.15.156:0/3682637110 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f11a8031e70 con 0x7f11b81012d0 2024-07-16T09:46:58.703 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:57.857+0000 7f11be86e640 1 -- 172.21.15.156:0/3682637110 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f11b81012d0 msgr2=0x7f11b81016e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:46:58.703 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:57.857+0000 7f11be86e640 1 --2- 172.21.15.156:0/3682637110 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f11b81012d0 0x7f11b81016e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f11a800baf0 tx=0x7f11a800b460 comp rx=0 tx=0).stop 2024-07-16T09:46:58.704 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:57.857+0000 7f11be86e640 1 -- 172.21.15.156:0/3682637110 shutdown_connections 2024-07-16T09:46:58.704 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:57.857+0000 7f11be86e640 1 --2- 172.21.15.156:0/3682637110 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f11b81012d0 0x7f11b81016e0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:46:58.704 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:57.857+0000 7f11be86e640 1 -- 172.21.15.156:0/3682637110 >> 172.21.15.156:0/3682637110 conn(0x7f11b80fbcc0 msgr2=0x7f11b80fe0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:46:58.704 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:57.857+0000 7f11be86e640 1 -- 172.21.15.156:0/3682637110 shutdown_connections 2024-07-16T09:46:58.704 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:57.857+0000 7f11be86e640 1 -- 172.21.15.156:0/3682637110 wait complete. 2024-07-16T09:46:58.704 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:57.857+0000 7f11be86e640 1 Processor -- start 2024-07-16T09:46:58.704 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:57.858+0000 7f11be86e640 1 -- start start 2024-07-16T09:46:58.704 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:57.858+0000 7f11be86e640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f11b81012d0 0x7f11b810e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:46:58.704 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:57.858+0000 7f11be86e640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11b810ee70 con 0x7f11b81012d0 2024-07-16T09:46:58.704 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:57.858+0000 7f11b7fff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f11b81012d0 0x7f11b810e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:46:58.704 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:57.858+0000 7f11b7fff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f11b81012d0 0x7f11b810e930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:45338/0 (socket says 172.21.15.156:45338) 2024-07-16T09:46:58.704 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:57.858+0000 7f11b7fff640 1 -- 172.21.15.156:0/2113324589 learned_addr learned my addr 172.21.15.156:0/2113324589 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T09:46:58.704 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:57.858+0000 7f11b7fff640 1 -- 172.21.15.156:0/2113324589 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11a800b700 con 0x7f11b81012d0 2024-07-16T09:46:58.705 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:57.858+0000 7f11b7fff640 1 --2- 172.21.15.156:0/2113324589 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f11b81012d0 0x7f11b810e930 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f11a800b2c0 tx=0x7f11a800b460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:46:58.705 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:57.858+0000 7f11b4ff9640 1 -- 172.21.15.156:0/2113324589 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f11a8032a80 con 0x7f11b81012d0 2024-07-16T09:46:58.705 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:57.858+0000 7f11b4ff9640 1 -- 172.21.15.156:0/2113324589 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f11a8032be0 con 0x7f11b81012d0 2024-07-16T09:46:58.705 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:57.858+0000 7f11be86e640 1 -- 172.21.15.156:0/2113324589 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11b810f070 con 0x7f11b81012d0 2024-07-16T09:46:58.705 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:57.859+0000 7f11b4ff9640 1 -- 172.21.15.156:0/2113324589 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f11a8039750 con 0x7f11b81012d0 2024-07-16T09:46:58.705 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:57.859+0000 7f11be86e640 1 -- 172.21.15.156:0/2113324589 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11b810f480 con 0x7f11b81012d0 2024-07-16T09:46:58.705 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:57.859+0000 7f11b4ff9640 1 -- 172.21.15.156:0/2113324589 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 8) v1 ==== 42636+0+0 (secure 0 0 0) 0x7f11a803b420 con 0x7f11b81012d0 2024-07-16T09:46:58.705 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:57.860+0000 7f11b4ff9640 1 --2- 172.21.15.156:0/2113324589 >> [v2:172.21.15.156:6800/1023000918,v1:172.21.15.156:6801/1023000918] conn(0x7f11a00357d0 0x7f11a0037c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:46:58.705 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:57.860+0000 7f11b4ff9640 1 -- 172.21.15.156:0/2113324589 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f11a806ad50 con 0x7f11b81012d0 2024-07-16T09:46:58.705 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:57.860+0000 7f11be86e640 1 -- 172.21.15.156:0/2113324589 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11b804b530 con 0x7f11b81012d0 2024-07-16T09:46:58.705 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:57.860+0000 7f11b77fe640 1 --2- 172.21.15.156:0/2113324589 >> [v2:172.21.15.156:6800/1023000918,v1:172.21.15.156:6801/1023000918] conn(0x7f11a00357d0 0x7f11a0037c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:46:58.705 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:57.860+0000 7f11b77fe640 1 --2- 172.21.15.156:0/2113324589 >> [v2:172.21.15.156:6800/1023000918,v1:172.21.15.156:6801/1023000918] conn(0x7f11a00357d0 0x7f11a0037c80 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f11ac006fd0 tx=0x7f11ac006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:46:58.705 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:57.864+0000 7f11b4ff9640 1 -- 172.21.15.156:0/2113324589 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f11a8036030 con 0x7f11b81012d0 2024-07-16T09:46:58.706 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:58.032+0000 7f11be86e640 1 -- 172.21.15.156:0/2113324589 --> [v2:172.21.15.156:6800/1023000918,v1:172.21.15.156:6801/1023000918] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) v1 -- 0x7f11b8107f30 con 0x7f11a00357d0 2024-07-16T09:46:58.706 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:58.115+0000 7f11b4ff9640 1 -- 172.21.15.156:0/2113324589 <== mgr.14118 v2:172.21.15.156:6800/1023000918 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f11b8107f30 con 0x7f11a00357d0 2024-07-16T09:46:58.706 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:58.117+0000 7f11be86e640 1 -- 172.21.15.156:0/2113324589 >> [v2:172.21.15.156:6800/1023000918,v1:172.21.15.156:6801/1023000918] conn(0x7f11a00357d0 msgr2=0x7f11a0037c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:46:58.706 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:58.117+0000 7f11be86e640 1 --2- 172.21.15.156:0/2113324589 >> [v2:172.21.15.156:6800/1023000918,v1:172.21.15.156:6801/1023000918] conn(0x7f11a00357d0 0x7f11a0037c80 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f11ac006fd0 tx=0x7f11ac006db0 comp rx=0 tx=0).stop 2024-07-16T09:46:58.706 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:58.117+0000 7f11be86e640 1 -- 172.21.15.156:0/2113324589 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f11b81012d0 msgr2=0x7f11b810e930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:46:58.706 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:58.117+0000 7f11be86e640 1 --2- 172.21.15.156:0/2113324589 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f11b81012d0 0x7f11b810e930 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f11a800b2c0 tx=0x7f11a800b460 comp rx=0 tx=0).stop 2024-07-16T09:46:58.706 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:58.117+0000 7f11be86e640 1 -- 172.21.15.156:0/2113324589 shutdown_connections 2024-07-16T09:46:58.706 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:58.117+0000 7f11be86e640 1 --2- 172.21.15.156:0/2113324589 >> [v2:172.21.15.156:6800/1023000918,v1:172.21.15.156:6801/1023000918] conn(0x7f11a00357d0 0x7f11a0037c80 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:46:58.706 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:58.117+0000 7f11be86e640 1 --2- 172.21.15.156:0/2113324589 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f11b81012d0 0x7f11b810e930 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:46:58.706 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:58.117+0000 7f11be86e640 1 -- 172.21.15.156:0/2113324589 >> 172.21.15.156:0/2113324589 conn(0x7f11b80fbcc0 msgr2=0x7f11b81052e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:46:58.706 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:58.117+0000 7f11be86e640 1 -- 172.21.15.156:0/2113324589 shutdown_connections 2024-07-16T09:46:58.707 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:58.117+0000 7f11be86e640 1 -- 172.21.15.156:0/2113324589 wait complete. 2024-07-16T09:46:59.248 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:59 smithi156 ceph-mon[29302]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-07-16T09:46:59.248 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:59 smithi156 ceph-mon[29302]: Generating ssh key... 2024-07-16T09:46:59.249 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:59 smithi156 ceph-mon[29302]: from='mgr.14118 172.21.15.156:0/3208530346' entity='mgr.smithi156.zkvylv' 2024-07-16T09:46:59.249 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:46:59 smithi156 ceph-mon[29302]: from='mgr.14118 172.21.15.156:0/3208530346' entity='mgr.smithi156.zkvylv' 2024-07-16T09:46:59.868 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCdOXvOWd5hoFBKp2rvbmYLWz2SrJ5kxPsJw/lBfxEHdug3ug6bK5VFvHKjiLdOxCeHD7kfze17gQVK4LwdupaKUBMhuR/k2VLCP0N6S7aNyd6Zgew4lPY50W1nHuRef5ejuYI9LKLwlh9IExIKrmHHKvmznV8FklOV0d8dGPx2LYT69dwfFZ+ITuuXYL4kMa61uDRMwg1lIU+DZJuTgeUxIQbAn8yjuyT7Rah9Rir+/SK0M4rWRIDunJYqHbe+xFVfvmnwTe0Bwp/kQwlEvdQnghYTkvg7AVioTM7cHRMX4aX8O6rKQBd/K2FTucZlLxJJIp7OSw94zn9HcY+wOXBbCwLxyrA9PGsWu4cf/ViT+BFWWDPjpO4CS+qyyp8HNiPGWCzJJ0GCUkjXgt1PUGWq2PXDKhNBlue01it6BCXAaiLcS9d5DlOROQ8Sn3psN6rYkl4AzSidJygFRhqwtZHNAeDG4x9MMXiXpNIz/G72qz3Qgl339SPZwELUzZGzq0k= ceph-26c91c26-4358-11ef-bcad-c7b262605968 2024-07-16T09:46:59.868 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:59.070+0000 7fdb610d3640 1 Processor -- start 2024-07-16T09:46:59.868 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:59.070+0000 7fdb610d3640 1 -- start start 2024-07-16T09:46:59.868 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:59.070+0000 7fdb610d3640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdb5c100200 0x7fdb5c100610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:46:59.868 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:59.070+0000 7fdb610d3640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb5c100b50 con 0x7fdb5c100200 2024-07-16T09:46:59.868 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:59.070+0000 7fdb5ad9d640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdb5c100200 0x7fdb5c100610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:46:59.868 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:59.070+0000 7fdb5ad9d640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdb5c100200 0x7fdb5c100610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:45350/0 (socket says 172.21.15.156:45350) 2024-07-16T09:46:59.869 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:59.070+0000 7fdb5ad9d640 1 -- 172.21.15.156:0/567072030 learned_addr learned my addr 172.21.15.156:0/567072030 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T09:46:59.869 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:59.071+0000 7fdb5ad9d640 1 -- 172.21.15.156:0/567072030 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb5c100c90 con 0x7fdb5c100200 2024-07-16T09:46:59.869 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:59.071+0000 7fdb5ad9d640 1 --2- 172.21.15.156:0/567072030 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdb5c100200 0x7fdb5c100610 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fdb4c009840 tx=0x7fdb4c0091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9dd4be0b7cf8201 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:46:59.869 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:59.071+0000 7fdb5a59c640 1 -- 172.21.15.156:0/567072030 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fdb4c032440 con 0x7fdb5c100200 2024-07-16T09:46:59.869 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:59.071+0000 7fdb5a59c640 1 -- 172.21.15.156:0/567072030 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7fdb4c032a30 con 0x7fdb5c100200 2024-07-16T09:46:59.869 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:59.071+0000 7fdb5a59c640 1 -- 172.21.15.156:0/567072030 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fdb4c031ad0 con 0x7fdb5c100200 2024-07-16T09:46:59.869 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:59.072+0000 7fdb610d3640 1 -- 172.21.15.156:0/567072030 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdb5c100200 msgr2=0x7fdb5c100610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:46:59.869 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:59.072+0000 7fdb610d3640 1 --2- 172.21.15.156:0/567072030 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdb5c100200 0x7fdb5c100610 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fdb4c009840 tx=0x7fdb4c0091d0 comp rx=0 tx=0).stop 2024-07-16T09:46:59.869 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:59.072+0000 7fdb610d3640 1 -- 172.21.15.156:0/567072030 shutdown_connections 2024-07-16T09:46:59.869 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:59.072+0000 7fdb610d3640 1 --2- 172.21.15.156:0/567072030 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdb5c100200 0x7fdb5c100610 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:46:59.869 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:59.072+0000 7fdb610d3640 1 -- 172.21.15.156:0/567072030 >> 172.21.15.156:0/567072030 conn(0x7fdb5c0fbcc0 msgr2=0x7fdb5c0fe0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:46:59.869 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:59.072+0000 7fdb610d3640 1 -- 172.21.15.156:0/567072030 shutdown_connections 2024-07-16T09:46:59.869 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:59.072+0000 7fdb610d3640 1 -- 172.21.15.156:0/567072030 wait complete. 2024-07-16T09:46:59.870 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:59.073+0000 7fdb610d3640 1 Processor -- start 2024-07-16T09:46:59.870 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:59.073+0000 7fdb610d3640 1 -- start start 2024-07-16T09:46:59.870 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:59.073+0000 7fdb610d3640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdb5c100200 0x7fdb5c110a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:46:59.870 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:59.073+0000 7fdb610d3640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb5c110fa0 con 0x7fdb5c100200 2024-07-16T09:46:59.870 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:59.073+0000 7fdb5ad9d640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdb5c100200 0x7fdb5c110a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:46:59.870 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:59.073+0000 7fdb5ad9d640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdb5c100200 0x7fdb5c110a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:45366/0 (socket says 172.21.15.156:45366) 2024-07-16T09:46:59.870 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:59.073+0000 7fdb5ad9d640 1 -- 172.21.15.156:0/3584419894 learned_addr learned my addr 172.21.15.156:0/3584419894 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T09:46:59.870 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:59.073+0000 7fdb5ad9d640 1 -- 172.21.15.156:0/3584419894 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb4c009450 con 0x7fdb5c100200 2024-07-16T09:46:59.870 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:59.073+0000 7fdb5ad9d640 1 --2- 172.21.15.156:0/3584419894 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdb5c100200 0x7fdb5c110a60 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fdb4c009420 tx=0x7fdb4c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:46:59.870 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:59.073+0000 7fdb537fe640 1 -- 172.21.15.156:0/3584419894 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fdb4c03d5b0 con 0x7fdb5c100200 2024-07-16T09:46:59.870 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:59.073+0000 7fdb537fe640 1 -- 172.21.15.156:0/3584419894 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7fdb4c03dba0 con 0x7fdb5c100200 2024-07-16T09:46:59.870 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:59.073+0000 7fdb610d3640 1 -- 172.21.15.156:0/3584419894 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb5c1111a0 con 0x7fdb5c100200 2024-07-16T09:46:59.870 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:59.074+0000 7fdb537fe640 1 -- 172.21.15.156:0/3584419894 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fdb4c03bb40 con 0x7fdb5c100200 2024-07-16T09:46:59.871 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:59.074+0000 7fdb610d3640 1 -- 172.21.15.156:0/3584419894 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb5c111610 con 0x7fdb5c100200 2024-07-16T09:46:59.871 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:59.074+0000 7fdb537fe640 1 -- 172.21.15.156:0/3584419894 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 8) v1 ==== 42636+0+0 (secure 0 0 0) 0x7fdb4c039070 con 0x7fdb5c100200 2024-07-16T09:46:59.871 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:59.074+0000 7fdb537fe640 1 --2- 172.21.15.156:0/3584419894 >> [v2:172.21.15.156:6800/1023000918,v1:172.21.15.156:6801/1023000918] conn(0x7fdb48035780 0x7fdb48037c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:46:59.871 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:59.075+0000 7fdb537fe640 1 -- 172.21.15.156:0/3584419894 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7fdb4c068840 con 0x7fdb5c100200 2024-07-16T09:46:59.871 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:59.075+0000 7fdb610d3640 1 -- 172.21.15.156:0/3584419894 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb5c111610 con 0x7fdb5c100200 2024-07-16T09:46:59.871 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:59.075+0000 7fdb53fff640 1 --2- 172.21.15.156:0/3584419894 >> [v2:172.21.15.156:6800/1023000918,v1:172.21.15.156:6801/1023000918] conn(0x7fdb48035780 0x7fdb48037c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:46:59.871 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:59.075+0000 7fdb53fff640 1 --2- 172.21.15.156:0/3584419894 >> [v2:172.21.15.156:6800/1023000918,v1:172.21.15.156:6801/1023000918] conn(0x7fdb48035780 0x7fdb48037c30 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fdb44006fd0 tx=0x7fdb44006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:46:59.871 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:59.079+0000 7fdb537fe640 1 -- 172.21.15.156:0/3584419894 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fdb5c111610 con 0x7fdb5c100200 2024-07-16T09:46:59.871 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:59.246+0000 7fdb610d3640 1 -- 172.21.15.156:0/3584419894 --> [v2:172.21.15.156:6800/1023000918,v1:172.21.15.156:6801/1023000918] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) v1 -- 0x7fdb5c111610 con 0x7fdb48035780 2024-07-16T09:46:59.872 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:59.247+0000 7fdb537fe640 1 -- 172.21.15.156:0/3584419894 <== mgr.14118 v2:172.21.15.156:6800/1023000918 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+595 (secure 0 0 0) 0x7fdb5c111610 con 0x7fdb48035780 2024-07-16T09:46:59.872 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:59.248+0000 7fdb610d3640 1 -- 172.21.15.156:0/3584419894 >> [v2:172.21.15.156:6800/1023000918,v1:172.21.15.156:6801/1023000918] conn(0x7fdb48035780 msgr2=0x7fdb48037c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:46:59.872 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:59.248+0000 7fdb610d3640 1 --2- 172.21.15.156:0/3584419894 >> [v2:172.21.15.156:6800/1023000918,v1:172.21.15.156:6801/1023000918] conn(0x7fdb48035780 0x7fdb48037c30 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fdb44006fd0 tx=0x7fdb44006db0 comp rx=0 tx=0).stop 2024-07-16T09:46:59.872 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:59.249+0000 7fdb610d3640 1 -- 172.21.15.156:0/3584419894 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdb5c100200 msgr2=0x7fdb5c110a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:46:59.872 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:59.249+0000 7fdb610d3640 1 --2- 172.21.15.156:0/3584419894 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdb5c100200 0x7fdb5c110a60 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fdb4c009420 tx=0x7fdb4c008040 comp rx=0 tx=0).stop 2024-07-16T09:46:59.872 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:59.249+0000 7fdb610d3640 1 -- 172.21.15.156:0/3584419894 shutdown_connections 2024-07-16T09:46:59.872 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:59.249+0000 7fdb610d3640 1 --2- 172.21.15.156:0/3584419894 >> [v2:172.21.15.156:6800/1023000918,v1:172.21.15.156:6801/1023000918] conn(0x7fdb48035780 0x7fdb48037c30 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:46:59.872 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:59.249+0000 7fdb610d3640 1 --2- 172.21.15.156:0/3584419894 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdb5c100200 0x7fdb5c110a60 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:46:59.872 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:59.249+0000 7fdb610d3640 1 -- 172.21.15.156:0/3584419894 >> 172.21.15.156:0/3584419894 conn(0x7fdb5c0fbcc0 msgr2=0x7fdb5c0fc6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:46:59.872 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:59.249+0000 7fdb610d3640 1 -- 172.21.15.156:0/3584419894 shutdown_connections 2024-07-16T09:46:59.872 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:46:59.249+0000 7fdb610d3640 1 -- 172.21.15.156:0/3584419894 wait complete. 2024-07-16T09:46:59.872 INFO:teuthology.orchestra.run.smithi156.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-07-16T09:46:59.873 INFO:teuthology.orchestra.run.smithi156.stdout:Adding key to root@localhost authorized_keys... 2024-07-16T09:46:59.873 INFO:teuthology.orchestra.run.smithi156.stdout:Adding host smithi156... 2024-07-16T09:47:00.411 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:47:00 smithi156 ceph-mon[29302]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-07-16T09:47:01.460 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:47:01 smithi156 ceph-mon[29302]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi156", "addr": "172.21.15.156", "target": ["mon-mgr", ""]}]: dispatch 2024-07-16T09:47:02.424 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:47:02 smithi156 ceph-mon[29302]: Deploying cephadm binary to smithi156 2024-07-16T09:47:03.717 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout Added host 'smithi156' with addr '172.21.15.156' 2024-07-16T09:47:03.718 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:00.231+0000 7f86feb74640 1 Processor -- start 2024-07-16T09:47:03.718 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:00.231+0000 7f86feb74640 1 -- start start 2024-07-16T09:47:03.718 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:00.231+0000 7f86feb74640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f86f8075bc0 0x7f86f8104a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:47:03.718 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:00.231+0000 7f86feb74640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86f8104f50 con 0x7f86f8075bc0 2024-07-16T09:47:03.718 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:00.232+0000 7f86fc910640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f86f8075bc0 0x7f86f8104a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:47:03.718 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:00.232+0000 7f86fc910640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f86f8075bc0 0x7f86f8104a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:45382/0 (socket says 172.21.15.156:45382) 2024-07-16T09:47:03.718 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:00.232+0000 7f86fc910640 1 -- 172.21.15.156:0/3444229833 learned_addr learned my addr 172.21.15.156:0/3444229833 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T09:47:03.718 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:00.232+0000 7f86fc910640 1 -- 172.21.15.156:0/3444229833 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86f8105090 con 0x7f86f8075bc0 2024-07-16T09:47:03.718 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:00.233+0000 7f86fc910640 1 --2- 172.21.15.156:0/3444229833 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f86f8075bc0 0x7f86f8104a10 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f86e8009840 tx=0x7f86e80091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b59cbcfd34072142 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:47:03.718 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:00.233+0000 7f86f77fe640 1 -- 172.21.15.156:0/3444229833 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f86e8032440 con 0x7f86f8075bc0 2024-07-16T09:47:03.718 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:00.233+0000 7f86f77fe640 1 -- 172.21.15.156:0/3444229833 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f86e8032a30 con 0x7f86f8075bc0 2024-07-16T09:47:03.719 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:00.233+0000 7f86f77fe640 1 -- 172.21.15.156:0/3444229833 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f86e8031ad0 con 0x7f86f8075bc0 2024-07-16T09:47:03.719 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:00.233+0000 7f86feb74640 1 -- 172.21.15.156:0/3444229833 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f86f8075bc0 msgr2=0x7f86f8104a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:47:03.719 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:00.233+0000 7f86feb74640 1 --2- 172.21.15.156:0/3444229833 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f86f8075bc0 0x7f86f8104a10 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f86e8009840 tx=0x7f86e80091d0 comp rx=0 tx=0).stop 2024-07-16T09:47:03.719 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:00.233+0000 7f86feb74640 1 -- 172.21.15.156:0/3444229833 shutdown_connections 2024-07-16T09:47:03.719 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:00.233+0000 7f86feb74640 1 --2- 172.21.15.156:0/3444229833 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f86f8075bc0 0x7f86f8104a10 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:47:03.719 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:00.233+0000 7f86feb74640 1 -- 172.21.15.156:0/3444229833 >> 172.21.15.156:0/3444229833 conn(0x7f86f80fc520 msgr2=0x7f86f80fe930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:47:03.719 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:00.234+0000 7f86feb74640 1 -- 172.21.15.156:0/3444229833 shutdown_connections 2024-07-16T09:47:03.719 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:00.234+0000 7f86feb74640 1 -- 172.21.15.156:0/3444229833 wait complete. 2024-07-16T09:47:03.719 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:00.234+0000 7f86feb74640 1 Processor -- start 2024-07-16T09:47:03.719 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:00.234+0000 7f86feb74640 1 -- start start 2024-07-16T09:47:03.719 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:00.234+0000 7f86feb74640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f86f8075bc0 0x7f86f8110a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:47:03.720 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:00.234+0000 7f86feb74640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86f8110fb0 con 0x7f86f8075bc0 2024-07-16T09:47:03.720 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:00.234+0000 7f86fc910640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f86f8075bc0 0x7f86f8110a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:47:03.720 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:00.235+0000 7f86fc910640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f86f8075bc0 0x7f86f8110a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:45394/0 (socket says 172.21.15.156:45394) 2024-07-16T09:47:03.720 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:00.235+0000 7f86fc910640 1 -- 172.21.15.156:0/3469713271 learned_addr learned my addr 172.21.15.156:0/3469713271 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T09:47:03.720 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:00.235+0000 7f86fc910640 1 -- 172.21.15.156:0/3469713271 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86e8009450 con 0x7f86f8075bc0 2024-07-16T09:47:03.720 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:00.235+0000 7f86fc910640 1 --2- 172.21.15.156:0/3469713271 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f86f8075bc0 0x7f86f8110a70 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f86e8006f00 tx=0x7f86e8008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:47:03.720 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:00.235+0000 7f86f57fa640 1 -- 172.21.15.156:0/3469713271 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f86e803d450 con 0x7f86f8075bc0 2024-07-16T09:47:03.720 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:00.235+0000 7f86f57fa640 1 -- 172.21.15.156:0/3469713271 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f86e803da40 con 0x7f86f8075bc0 2024-07-16T09:47:03.720 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:00.235+0000 7f86feb74640 1 -- 172.21.15.156:0/3469713271 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86f81111b0 con 0x7f86f8075bc0 2024-07-16T09:47:03.720 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:00.235+0000 7f86f57fa640 1 -- 172.21.15.156:0/3469713271 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f86e803bb10 con 0x7f86f8075bc0 2024-07-16T09:47:03.720 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:00.235+0000 7f86feb74640 1 -- 172.21.15.156:0/3469713271 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86f8111620 con 0x7f86f8075bc0 2024-07-16T09:47:03.720 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:00.236+0000 7f86f57fa640 1 -- 172.21.15.156:0/3469713271 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 8) v1 ==== 42636+0+0 (secure 0 0 0) 0x7f86e8039070 con 0x7f86f8075bc0 2024-07-16T09:47:03.721 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:00.236+0000 7f86f57fa640 1 --2- 172.21.15.156:0/3469713271 >> [v2:172.21.15.156:6800/1023000918,v1:172.21.15.156:6801/1023000918] conn(0x7f86e00357d0 0x7f86e0037c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:47:03.721 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:00.236+0000 7f86f57fa640 1 -- 172.21.15.156:0/3469713271 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f86e8068760 con 0x7f86f8075bc0 2024-07-16T09:47:03.721 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:00.236+0000 7f86feb74640 1 -- 172.21.15.156:0/3469713271 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86f805eb80 con 0x7f86f8075bc0 2024-07-16T09:47:03.721 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:00.236+0000 7f86f7fff640 1 --2- 172.21.15.156:0/3469713271 >> [v2:172.21.15.156:6800/1023000918,v1:172.21.15.156:6801/1023000918] conn(0x7f86e00357d0 0x7f86e0037c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:47:03.721 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:00.237+0000 7f86f7fff640 1 --2- 172.21.15.156:0/3469713271 >> [v2:172.21.15.156:6800/1023000918,v1:172.21.15.156:6801/1023000918] conn(0x7f86e00357d0 0x7f86e0037c80 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f86ec006fd0 tx=0x7f86ec006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:47:03.721 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:00.241+0000 7f86f57fa640 1 -- 172.21.15.156:0/3469713271 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f86e8041080 con 0x7f86f8075bc0 2024-07-16T09:47:03.721 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:00.409+0000 7f86feb74640 1 -- 172.21.15.156:0/3469713271 --> [v2:172.21.15.156:6800/1023000918,v1:172.21.15.156:6801/1023000918] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi156", "addr": "172.21.15.156", "target": ["mon-mgr", ""]}) v1 -- 0x7f86f80fd6c0 con 0x7f86e00357d0 2024-07-16T09:47:03.721 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:03.137+0000 7f86f57fa640 1 -- 172.21.15.156:0/3469713271 <== mgr.14118 v2:172.21.15.156:6800/1023000918 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f86f80fd6c0 con 0x7f86e00357d0 2024-07-16T09:47:03.721 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:03.139+0000 7f86feb74640 1 -- 172.21.15.156:0/3469713271 >> [v2:172.21.15.156:6800/1023000918,v1:172.21.15.156:6801/1023000918] conn(0x7f86e00357d0 msgr2=0x7f86e0037c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:47:03.721 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:03.139+0000 7f86feb74640 1 --2- 172.21.15.156:0/3469713271 >> [v2:172.21.15.156:6800/1023000918,v1:172.21.15.156:6801/1023000918] conn(0x7f86e00357d0 0x7f86e0037c80 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f86ec006fd0 tx=0x7f86ec006db0 comp rx=0 tx=0).stop 2024-07-16T09:47:03.721 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:03.139+0000 7f86feb74640 1 -- 172.21.15.156:0/3469713271 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f86f8075bc0 msgr2=0x7f86f8110a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:47:03.721 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:03.139+0000 7f86feb74640 1 --2- 172.21.15.156:0/3469713271 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f86f8075bc0 0x7f86f8110a70 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f86e8006f00 tx=0x7f86e8008040 comp rx=0 tx=0).stop 2024-07-16T09:47:03.721 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:03.139+0000 7f86feb74640 1 -- 172.21.15.156:0/3469713271 shutdown_connections 2024-07-16T09:47:03.722 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:03.139+0000 7f86feb74640 1 --2- 172.21.15.156:0/3469713271 >> [v2:172.21.15.156:6800/1023000918,v1:172.21.15.156:6801/1023000918] conn(0x7f86e00357d0 0x7f86e0037c80 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:47:03.722 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:03.139+0000 7f86feb74640 1 --2- 172.21.15.156:0/3469713271 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f86f8075bc0 0x7f86f8110a70 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:47:03.722 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:03.139+0000 7f86feb74640 1 -- 172.21.15.156:0/3469713271 >> 172.21.15.156:0/3469713271 conn(0x7f86f80fc520 msgr2=0x7f86f80fcfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:47:03.722 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:03.139+0000 7f86feb74640 1 -- 172.21.15.156:0/3469713271 shutdown_connections 2024-07-16T09:47:03.722 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:03.140+0000 7f86feb74640 1 -- 172.21.15.156:0/3469713271 wait complete. 2024-07-16T09:47:03.722 INFO:teuthology.orchestra.run.smithi156.stdout:Deploying mon service with default placement... 2024-07-16T09:47:04.342 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:47:04 smithi156 ceph-mon[29302]: from='mgr.14118 172.21.15.156:0/3208530346' entity='mgr.smithi156.zkvylv' 2024-07-16T09:47:04.342 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:47:04 smithi156 ceph-mon[29302]: Added host smithi156 2024-07-16T09:47:04.342 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:47:04 smithi156 ceph-mon[29302]: from='mgr.14118 172.21.15.156:0/3208530346' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-16T09:47:05.127 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-07-16T09:47:05.127 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:04.146+0000 7f9176cb6640 1 Processor -- start 2024-07-16T09:47:05.127 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:04.146+0000 7f9176cb6640 1 -- start start 2024-07-16T09:47:05.127 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:04.146+0000 7f9176cb6640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9170075b80 0x7f9170106e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:47:05.127 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:04.146+0000 7f9176cb6640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9170075f90 con 0x7f9170075b80 2024-07-16T09:47:05.127 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:04.146+0000 7f9174a52640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9170075b80 0x7f9170106e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:47:05.128 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:04.146+0000 7f9174a52640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9170075b80 0x7f9170106e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:59602/0 (socket says 172.21.15.156:59602) 2024-07-16T09:47:05.128 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:04.146+0000 7f9174a52640 1 -- 172.21.15.156:0/933793030 learned_addr learned my addr 172.21.15.156:0/933793030 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T09:47:05.128 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:04.147+0000 7f9174a52640 1 -- 172.21.15.156:0/933793030 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9170102840 con 0x7f9170075b80 2024-07-16T09:47:05.128 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:04.147+0000 7f9174a52640 1 --2- 172.21.15.156:0/933793030 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9170075b80 0x7f9170106e50 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f9164009840 tx=0x7f91640091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee5bc94b5e8d98f5 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:47:05.128 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:04.148+0000 7f916f7fe640 1 -- 172.21.15.156:0/933793030 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9164032440 con 0x7f9170075b80 2024-07-16T09:47:05.128 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:04.148+0000 7f916f7fe640 1 -- 172.21.15.156:0/933793030 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f9164032a30 con 0x7f9170075b80 2024-07-16T09:47:05.128 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:04.148+0000 7f916f7fe640 1 -- 172.21.15.156:0/933793030 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9164031ad0 con 0x7f9170075b80 2024-07-16T09:47:05.128 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:04.148+0000 7f9176cb6640 1 -- 172.21.15.156:0/933793030 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9170075b80 msgr2=0x7f9170106e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:47:05.128 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:04.148+0000 7f9176cb6640 1 --2- 172.21.15.156:0/933793030 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9170075b80 0x7f9170106e50 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f9164009840 tx=0x7f91640091d0 comp rx=0 tx=0).stop 2024-07-16T09:47:05.128 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:04.148+0000 7f9176cb6640 1 -- 172.21.15.156:0/933793030 shutdown_connections 2024-07-16T09:47:05.128 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:04.148+0000 7f9176cb6640 1 --2- 172.21.15.156:0/933793030 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9170075b80 0x7f9170106e50 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:47:05.128 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:04.148+0000 7f9176cb6640 1 -- 172.21.15.156:0/933793030 >> 172.21.15.156:0/933793030 conn(0x7f91700fc4c0 msgr2=0x7f91700fe8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:47:05.129 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:04.148+0000 7f9176cb6640 1 -- 172.21.15.156:0/933793030 shutdown_connections 2024-07-16T09:47:05.129 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:04.148+0000 7f9176cb6640 1 -- 172.21.15.156:0/933793030 wait complete. 2024-07-16T09:47:05.129 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:04.149+0000 7f9176cb6640 1 Processor -- start 2024-07-16T09:47:05.129 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:04.149+0000 7f9176cb6640 1 -- start start 2024-07-16T09:47:05.129 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:04.149+0000 7f9176cb6640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9170075b80 0x7f9170112c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:47:05.129 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:04.149+0000 7f9176cb6640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9170113180 con 0x7f9170075b80 2024-07-16T09:47:05.129 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:04.149+0000 7f9174a52640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9170075b80 0x7f9170112c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:47:05.129 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:04.149+0000 7f9174a52640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9170075b80 0x7f9170112c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:59614/0 (socket says 172.21.15.156:59614) 2024-07-16T09:47:05.129 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:04.149+0000 7f9174a52640 1 -- 172.21.15.156:0/1542219290 learned_addr learned my addr 172.21.15.156:0/1542219290 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T09:47:05.129 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:04.149+0000 7f9174a52640 1 -- 172.21.15.156:0/1542219290 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9164009450 con 0x7f9170075b80 2024-07-16T09:47:05.129 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:04.150+0000 7f9174a52640 1 --2- 172.21.15.156:0/1542219290 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9170075b80 0x7f9170112c40 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f9164006f00 tx=0x7f9164008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:47:05.129 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:04.150+0000 7f916d7fa640 1 -- 172.21.15.156:0/1542219290 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f916403d450 con 0x7f9170075b80 2024-07-16T09:47:05.130 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:04.150+0000 7f916d7fa640 1 -- 172.21.15.156:0/1542219290 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f916403da40 con 0x7f9170075b80 2024-07-16T09:47:05.130 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:04.150+0000 7f9176cb6640 1 -- 172.21.15.156:0/1542219290 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91701133e0 con 0x7f9170075b80 2024-07-16T09:47:05.130 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:04.150+0000 7f916d7fa640 1 -- 172.21.15.156:0/1542219290 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f916403bb10 con 0x7f9170075b80 2024-07-16T09:47:05.130 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:04.150+0000 7f9176cb6640 1 -- 172.21.15.156:0/1542219290 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9170113f10 con 0x7f9170075b80 2024-07-16T09:47:05.130 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:04.151+0000 7f916d7fa640 1 -- 172.21.15.156:0/1542219290 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 8) v1 ==== 42636+0+0 (secure 0 0 0) 0x7f9164039070 con 0x7f9170075b80 2024-07-16T09:47:05.130 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:04.151+0000 7f9176cb6640 1 -- 172.21.15.156:0/1542219290 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f917005eb80 con 0x7f9170075b80 2024-07-16T09:47:05.130 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:04.151+0000 7f916d7fa640 1 --2- 172.21.15.156:0/1542219290 >> [v2:172.21.15.156:6800/1023000918,v1:172.21.15.156:6801/1023000918] conn(0x7f9160035780 0x7f9160037c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:47:05.130 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:04.151+0000 7f916d7fa640 1 -- 172.21.15.156:0/1542219290 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f9164068b40 con 0x7f9170075b80 2024-07-16T09:47:05.130 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:04.151+0000 7f916ffff640 1 --2- 172.21.15.156:0/1542219290 >> [v2:172.21.15.156:6800/1023000918,v1:172.21.15.156:6801/1023000918] conn(0x7f9160035780 0x7f9160037c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:47:05.130 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:04.152+0000 7f916ffff640 1 --2- 172.21.15.156:0/1542219290 >> [v2:172.21.15.156:6800/1023000918,v1:172.21.15.156:6801/1023000918] conn(0x7f9160035780 0x7f9160037c30 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f915c006fd0 tx=0x7f915c006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:47:05.130 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:04.156+0000 7f916d7fa640 1 -- 172.21.15.156:0/1542219290 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f9164046400 con 0x7f9170075b80 2024-07-16T09:47:05.130 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:04.340+0000 7f9176cb6640 1 -- 172.21.15.156:0/1542219290 --> [v2:172.21.15.156:6800/1023000918,v1:172.21.15.156:6801/1023000918] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "target": ["mon-mgr", ""]}) v1 -- 0x7f91700fd5f0 con 0x7f9160035780 2024-07-16T09:47:05.130 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:04.344+0000 7f916d7fa640 1 -- 172.21.15.156:0/1542219290 <== mgr.14118 v2:172.21.15.156:6800/1023000918 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f91700fd5f0 con 0x7f9160035780 2024-07-16T09:47:05.131 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:04.346+0000 7f9176cb6640 1 -- 172.21.15.156:0/1542219290 >> [v2:172.21.15.156:6800/1023000918,v1:172.21.15.156:6801/1023000918] conn(0x7f9160035780 msgr2=0x7f9160037c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:47:05.131 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:04.347+0000 7f9176cb6640 1 --2- 172.21.15.156:0/1542219290 >> [v2:172.21.15.156:6800/1023000918,v1:172.21.15.156:6801/1023000918] conn(0x7f9160035780 0x7f9160037c30 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f915c006fd0 tx=0x7f915c006db0 comp rx=0 tx=0).stop 2024-07-16T09:47:05.131 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:04.347+0000 7f9176cb6640 1 -- 172.21.15.156:0/1542219290 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9170075b80 msgr2=0x7f9170112c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:47:05.131 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:04.347+0000 7f9176cb6640 1 --2- 172.21.15.156:0/1542219290 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9170075b80 0x7f9170112c40 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f9164006f00 tx=0x7f9164008040 comp rx=0 tx=0).stop 2024-07-16T09:47:05.131 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:04.347+0000 7f9176cb6640 1 -- 172.21.15.156:0/1542219290 shutdown_connections 2024-07-16T09:47:05.131 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:04.347+0000 7f9176cb6640 1 --2- 172.21.15.156:0/1542219290 >> [v2:172.21.15.156:6800/1023000918,v1:172.21.15.156:6801/1023000918] conn(0x7f9160035780 0x7f9160037c30 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:47:05.131 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:04.347+0000 7f9176cb6640 1 --2- 172.21.15.156:0/1542219290 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9170075b80 0x7f9170112c40 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:47:05.131 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:04.347+0000 7f9176cb6640 1 -- 172.21.15.156:0/1542219290 >> 172.21.15.156:0/1542219290 conn(0x7f91700fc4c0 msgr2=0x7f91700fcee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:47:05.131 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:04.347+0000 7f9176cb6640 1 -- 172.21.15.156:0/1542219290 shutdown_connections 2024-07-16T09:47:05.131 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:04.347+0000 7f9176cb6640 1 -- 172.21.15.156:0/1542219290 wait complete. 2024-07-16T09:47:05.131 INFO:teuthology.orchestra.run.smithi156.stdout:Deploying mgr service with default placement... 2024-07-16T09:47:05.385 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:47:05 smithi156 ceph-mon[29302]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "target": ["mon-mgr", ""]}]: dispatch 2024-07-16T09:47:05.385 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:47:05 smithi156 ceph-mon[29302]: Saving service mon spec with placement count:5 2024-07-16T09:47:05.385 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:47:05 smithi156 ceph-mon[29302]: from='mgr.14118 172.21.15.156:0/3208530346' entity='mgr.smithi156.zkvylv' 2024-07-16T09:47:06.284 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-07-16T09:47:06.285 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:05.563+0000 7f19d3d36640 1 Processor -- start 2024-07-16T09:47:06.285 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:05.563+0000 7f19d3d36640 1 -- start start 2024-07-16T09:47:06.285 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:05.563+0000 7f19d3d36640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f19cc101e80 0x7f19cc102290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:47:06.285 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:05.563+0000 7f19d3d36640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19cc1027d0 con 0x7f19cc101e80 2024-07-16T09:47:06.285 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:05.563+0000 7f19d1ad2640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f19cc101e80 0x7f19cc102290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:47:06.285 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:05.563+0000 7f19d1ad2640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f19cc101e80 0x7f19cc102290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:59622/0 (socket says 172.21.15.156:59622) 2024-07-16T09:47:06.285 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:05.563+0000 7f19d1ad2640 1 -- 172.21.15.156:0/2657825299 learned_addr learned my addr 172.21.15.156:0/2657825299 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T09:47:06.285 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:05.563+0000 7f19d1ad2640 1 -- 172.21.15.156:0/2657825299 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19cc102910 con 0x7f19cc101e80 2024-07-16T09:47:06.285 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:05.564+0000 7f19d1ad2640 1 --2- 172.21.15.156:0/2657825299 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f19cc101e80 0x7f19cc102290 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f19c8009840 tx=0x7f19c80091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6936fa2c8beaca88 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:47:06.285 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:05.564+0000 7f19d0ad0640 1 -- 172.21.15.156:0/2657825299 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f19c8032440 con 0x7f19cc101e80 2024-07-16T09:47:06.285 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:05.564+0000 7f19d0ad0640 1 -- 172.21.15.156:0/2657825299 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f19c8032a30 con 0x7f19cc101e80 2024-07-16T09:47:06.285 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:05.564+0000 7f19d0ad0640 1 -- 172.21.15.156:0/2657825299 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f19c8031ad0 con 0x7f19cc101e80 2024-07-16T09:47:06.286 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:05.565+0000 7f19d3d36640 1 -- 172.21.15.156:0/2657825299 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f19cc101e80 msgr2=0x7f19cc102290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:47:06.286 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:05.565+0000 7f19d3d36640 1 --2- 172.21.15.156:0/2657825299 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f19cc101e80 0x7f19cc102290 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f19c8009840 tx=0x7f19c80091d0 comp rx=0 tx=0).stop 2024-07-16T09:47:06.286 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:05.565+0000 7f19d3d36640 1 -- 172.21.15.156:0/2657825299 shutdown_connections 2024-07-16T09:47:06.286 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:05.565+0000 7f19d3d36640 1 --2- 172.21.15.156:0/2657825299 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f19cc101e80 0x7f19cc102290 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:47:06.286 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:05.565+0000 7f19d3d36640 1 -- 172.21.15.156:0/2657825299 >> 172.21.15.156:0/2657825299 conn(0x7f19cc0fd5e0 msgr2=0x7f19cc0ff9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:47:06.286 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:05.565+0000 7f19d3d36640 1 -- 172.21.15.156:0/2657825299 shutdown_connections 2024-07-16T09:47:06.286 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:05.565+0000 7f19d3d36640 1 -- 172.21.15.156:0/2657825299 wait complete. 2024-07-16T09:47:06.286 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:05.566+0000 7f19d3d36640 1 Processor -- start 2024-07-16T09:47:06.286 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:05.566+0000 7f19d3d36640 1 -- start start 2024-07-16T09:47:06.286 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:05.566+0000 7f19d3d36640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f19cc101e80 0x7f19cc077c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:47:06.286 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:05.566+0000 7f19d3d36640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19cc078160 con 0x7f19cc101e80 2024-07-16T09:47:06.286 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:05.566+0000 7f19d1ad2640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f19cc101e80 0x7f19cc077c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:47:06.286 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:05.566+0000 7f19d1ad2640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f19cc101e80 0x7f19cc077c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:59624/0 (socket says 172.21.15.156:59624) 2024-07-16T09:47:06.286 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:05.566+0000 7f19d1ad2640 1 -- 172.21.15.156:0/2265435680 learned_addr learned my addr 172.21.15.156:0/2265435680 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T09:47:06.287 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:05.566+0000 7f19d1ad2640 1 -- 172.21.15.156:0/2265435680 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19c8009450 con 0x7f19cc101e80 2024-07-16T09:47:06.287 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:05.566+0000 7f19d1ad2640 1 --2- 172.21.15.156:0/2265435680 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f19cc101e80 0x7f19cc077c20 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f19c80317b0 tx=0x7f19c8008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:47:06.287 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:05.567+0000 7f19c27fc640 1 -- 172.21.15.156:0/2265435680 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f19c8031e90 con 0x7f19cc101e80 2024-07-16T09:47:06.287 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:05.567+0000 7f19c27fc640 1 -- 172.21.15.156:0/2265435680 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f19c803daf0 con 0x7f19cc101e80 2024-07-16T09:47:06.287 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:05.567+0000 7f19d3d36640 1 -- 172.21.15.156:0/2265435680 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19cc0761f0 con 0x7f19cc101e80 2024-07-16T09:47:06.287 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:05.567+0000 7f19c27fc640 1 -- 172.21.15.156:0/2265435680 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f19c803baa0 con 0x7f19cc101e80 2024-07-16T09:47:06.287 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:05.567+0000 7f19d3d36640 1 -- 172.21.15.156:0/2265435680 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19cc076660 con 0x7f19cc101e80 2024-07-16T09:47:06.287 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:05.567+0000 7f19c27fc640 1 -- 172.21.15.156:0/2265435680 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 8) v1 ==== 42636+0+0 (secure 0 0 0) 0x7f19c8039070 con 0x7f19cc101e80 2024-07-16T09:47:06.287 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:05.568+0000 7f19c27fc640 1 --2- 172.21.15.156:0/2265435680 >> [v2:172.21.15.156:6800/1023000918,v1:172.21.15.156:6801/1023000918] conn(0x7f19b80357d0 0x7f19b8037c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:47:06.287 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:05.568+0000 7f19d3d36640 1 -- 172.21.15.156:0/2265435680 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19cc04b530 con 0x7f19cc101e80 2024-07-16T09:47:06.287 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:05.568+0000 7f19c27fc640 1 -- 172.21.15.156:0/2265435680 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f19c8068b60 con 0x7f19cc101e80 2024-07-16T09:47:06.287 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:05.568+0000 7f19d12d1640 1 --2- 172.21.15.156:0/2265435680 >> [v2:172.21.15.156:6800/1023000918,v1:172.21.15.156:6801/1023000918] conn(0x7f19b80357d0 0x7f19b8037c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:47:06.287 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:05.568+0000 7f19d12d1640 1 --2- 172.21.15.156:0/2265435680 >> [v2:172.21.15.156:6800/1023000918,v1:172.21.15.156:6801/1023000918] conn(0x7f19b80357d0 0x7f19b8037c80 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f19bc006fd0 tx=0x7f19bc006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:47:06.287 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:05.572+0000 7f19c27fc640 1 -- 172.21.15.156:0/2265435680 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f19c803d660 con 0x7f19cc101e80 2024-07-16T09:47:06.288 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:05.751+0000 7f19d3d36640 1 -- 172.21.15.156:0/2265435680 --> [v2:172.21.15.156:6800/1023000918,v1:172.21.15.156:6801/1023000918] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "target": ["mon-mgr", ""]}) v1 -- 0x7f19cc076e10 con 0x7f19b80357d0 2024-07-16T09:47:06.288 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:05.755+0000 7f19c27fc640 1 -- 172.21.15.156:0/2265435680 <== mgr.14118 v2:172.21.15.156:6800/1023000918 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f19cc076e10 con 0x7f19b80357d0 2024-07-16T09:47:06.288 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:05.757+0000 7f19d3d36640 1 -- 172.21.15.156:0/2265435680 >> [v2:172.21.15.156:6800/1023000918,v1:172.21.15.156:6801/1023000918] conn(0x7f19b80357d0 msgr2=0x7f19b8037c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:47:06.288 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:05.757+0000 7f19d3d36640 1 --2- 172.21.15.156:0/2265435680 >> [v2:172.21.15.156:6800/1023000918,v1:172.21.15.156:6801/1023000918] conn(0x7f19b80357d0 0x7f19b8037c80 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f19bc006fd0 tx=0x7f19bc006db0 comp rx=0 tx=0).stop 2024-07-16T09:47:06.288 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:05.757+0000 7f19d3d36640 1 -- 172.21.15.156:0/2265435680 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f19cc101e80 msgr2=0x7f19cc077c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:47:06.288 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:05.757+0000 7f19d3d36640 1 --2- 172.21.15.156:0/2265435680 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f19cc101e80 0x7f19cc077c20 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f19c80317b0 tx=0x7f19c8008040 comp rx=0 tx=0).stop 2024-07-16T09:47:06.288 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:05.757+0000 7f19d3d36640 1 -- 172.21.15.156:0/2265435680 shutdown_connections 2024-07-16T09:47:06.288 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:05.757+0000 7f19d3d36640 1 --2- 172.21.15.156:0/2265435680 >> [v2:172.21.15.156:6800/1023000918,v1:172.21.15.156:6801/1023000918] conn(0x7f19b80357d0 0x7f19b8037c80 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:47:06.288 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:05.757+0000 7f19d3d36640 1 --2- 172.21.15.156:0/2265435680 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f19cc101e80 0x7f19cc077c20 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:47:06.288 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:05.757+0000 7f19d3d36640 1 -- 172.21.15.156:0/2265435680 >> 172.21.15.156:0/2265435680 conn(0x7f19cc0fd5e0 msgr2=0x7f19cc0fdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:47:06.288 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:05.757+0000 7f19d3d36640 1 -- 172.21.15.156:0/2265435680 shutdown_connections 2024-07-16T09:47:06.288 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:05.757+0000 7f19d3d36640 1 -- 172.21.15.156:0/2265435680 wait complete. 2024-07-16T09:47:06.288 INFO:teuthology.orchestra.run.smithi156.stdout:Deploying crash service with default placement... 2024-07-16T09:47:06.865 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:47:06 smithi156 ceph-mon[29302]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "target": ["mon-mgr", ""]}]: dispatch 2024-07-16T09:47:06.866 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:47:06 smithi156 ceph-mon[29302]: Saving service mgr spec with placement count:2 2024-07-16T09:47:06.866 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:47:06 smithi156 ceph-mon[29302]: from='mgr.14118 172.21.15.156:0/3208530346' entity='mgr.smithi156.zkvylv' 2024-07-16T09:47:07.679 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout Scheduled crash update... 2024-07-16T09:47:07.680 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:06.678+0000 7ff673924640 1 Processor -- start 2024-07-16T09:47:07.680 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:06.679+0000 7ff673924640 1 -- start start 2024-07-16T09:47:07.680 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:06.679+0000 7ff673924640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff66c077c80 0x7ff66c076250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:47:07.680 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:06.679+0000 7ff673924640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff66c078090 con 0x7ff66c077c80 2024-07-16T09:47:07.681 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:06.679+0000 7ff6716c0640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff66c077c80 0x7ff66c076250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:47:07.681 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:06.679+0000 7ff6716c0640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff66c077c80 0x7ff66c076250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:59640/0 (socket says 172.21.15.156:59640) 2024-07-16T09:47:07.681 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:06.679+0000 7ff6716c0640 1 -- 172.21.15.156:0/651814072 learned_addr learned my addr 172.21.15.156:0/651814072 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T09:47:07.681 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:06.679+0000 7ff6716c0640 1 -- 172.21.15.156:0/651814072 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff66c076790 con 0x7ff66c077c80 2024-07-16T09:47:07.681 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:06.680+0000 7ff6716c0640 1 --2- 172.21.15.156:0/651814072 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff66c077c80 0x7ff66c076250 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7ff668009840 tx=0x7ff6680091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f470334c12b1bff server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:47:07.681 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:06.680+0000 7ff663fff640 1 -- 172.21.15.156:0/651814072 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff668032440 con 0x7ff66c077c80 2024-07-16T09:47:07.682 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:06.680+0000 7ff663fff640 1 -- 172.21.15.156:0/651814072 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7ff668032a30 con 0x7ff66c077c80 2024-07-16T09:47:07.682 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:06.680+0000 7ff663fff640 1 -- 172.21.15.156:0/651814072 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff668031ad0 con 0x7ff66c077c80 2024-07-16T09:47:07.682 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:06.681+0000 7ff673924640 1 -- 172.21.15.156:0/651814072 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff66c077c80 msgr2=0x7ff66c076250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:47:07.682 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:06.681+0000 7ff673924640 1 --2- 172.21.15.156:0/651814072 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff66c077c80 0x7ff66c076250 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7ff668009840 tx=0x7ff6680091d0 comp rx=0 tx=0).stop 2024-07-16T09:47:07.682 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:06.681+0000 7ff673924640 1 -- 172.21.15.156:0/651814072 shutdown_connections 2024-07-16T09:47:07.683 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:06.681+0000 7ff673924640 1 --2- 172.21.15.156:0/651814072 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff66c077c80 0x7ff66c076250 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:47:07.683 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:06.681+0000 7ff673924640 1 -- 172.21.15.156:0/651814072 >> 172.21.15.156:0/651814072 conn(0x7ff66c0fd5e0 msgr2=0x7ff66c0ff9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:47:07.683 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:06.681+0000 7ff673924640 1 -- 172.21.15.156:0/651814072 shutdown_connections 2024-07-16T09:47:07.683 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:06.681+0000 7ff673924640 1 -- 172.21.15.156:0/651814072 wait complete. 2024-07-16T09:47:07.683 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:06.681+0000 7ff673924640 1 Processor -- start 2024-07-16T09:47:07.683 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:06.681+0000 7ff673924640 1 -- start start 2024-07-16T09:47:07.684 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:06.681+0000 7ff673924640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff66c077c80 0x7ff66c112ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:47:07.684 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:06.681+0000 7ff673924640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff66c1131e0 con 0x7ff66c077c80 2024-07-16T09:47:07.684 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:06.682+0000 7ff6716c0640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff66c077c80 0x7ff66c112ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:47:07.684 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:06.682+0000 7ff6716c0640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff66c077c80 0x7ff66c112ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:59644/0 (socket says 172.21.15.156:59644) 2024-07-16T09:47:07.684 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:06.682+0000 7ff6716c0640 1 -- 172.21.15.156:0/1397822077 learned_addr learned my addr 172.21.15.156:0/1397822077 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T09:47:07.685 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:06.682+0000 7ff6716c0640 1 -- 172.21.15.156:0/1397822077 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff668009450 con 0x7ff66c077c80 2024-07-16T09:47:07.685 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:06.682+0000 7ff6716c0640 1 --2- 172.21.15.156:0/1397822077 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff66c077c80 0x7ff66c112ca0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7ff668031b90 tx=0x7ff668008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:47:07.685 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:06.682+0000 7ff661ffb640 1 -- 172.21.15.156:0/1397822077 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff6680328b0 con 0x7ff66c077c80 2024-07-16T09:47:07.685 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:06.682+0000 7ff661ffb640 1 -- 172.21.15.156:0/1397822077 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7ff66803d8e0 con 0x7ff66c077c80 2024-07-16T09:47:07.685 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:06.682+0000 7ff673924640 1 -- 172.21.15.156:0/1397822077 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff66c1133e0 con 0x7ff66c077c80 2024-07-16T09:47:07.685 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:06.682+0000 7ff661ffb640 1 -- 172.21.15.156:0/1397822077 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff66803ba20 con 0x7ff66c077c80 2024-07-16T09:47:07.685 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:06.682+0000 7ff673924640 1 -- 172.21.15.156:0/1397822077 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff66c113850 con 0x7ff66c077c80 2024-07-16T09:47:07.685 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:06.683+0000 7ff661ffb640 1 -- 172.21.15.156:0/1397822077 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 8) v1 ==== 42636+0+0 (secure 0 0 0) 0x7ff668039070 con 0x7ff66c077c80 2024-07-16T09:47:07.686 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:06.683+0000 7ff661ffb640 1 --2- 172.21.15.156:0/1397822077 >> [v2:172.21.15.156:6800/1023000918,v1:172.21.15.156:6801/1023000918] conn(0x7ff6580357d0 0x7ff658037c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:47:07.686 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:06.683+0000 7ff661ffb640 1 -- 172.21.15.156:0/1397822077 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7ff668068810 con 0x7ff66c077c80 2024-07-16T09:47:07.686 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:06.683+0000 7ff670ebf640 1 --2- 172.21.15.156:0/1397822077 >> [v2:172.21.15.156:6800/1023000918,v1:172.21.15.156:6801/1023000918] conn(0x7ff6580357d0 0x7ff658037c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:47:07.686 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:06.683+0000 7ff673924640 1 -- 172.21.15.156:0/1397822077 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff66c05eb80 con 0x7ff66c077c80 2024-07-16T09:47:07.686 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:06.684+0000 7ff670ebf640 1 --2- 172.21.15.156:0/1397822077 >> [v2:172.21.15.156:6800/1023000918,v1:172.21.15.156:6801/1023000918] conn(0x7ff6580357d0 0x7ff658037c80 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7ff65c006fd0 tx=0x7ff65c006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:47:07.686 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:06.688+0000 7ff661ffb640 1 -- 172.21.15.156:0/1397822077 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7ff668041020 con 0x7ff66c077c80 2024-07-16T09:47:07.686 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:06.864+0000 7ff673924640 1 -- 172.21.15.156:0/1397822077 --> [v2:172.21.15.156:6800/1023000918,v1:172.21.15.156:6801/1023000918] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "crash", "target": ["mon-mgr", ""]}) v1 -- 0x7ff66c0fe730 con 0x7ff6580357d0 2024-07-16T09:47:07.686 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:06.868+0000 7ff661ffb640 1 -- 172.21.15.156:0/1397822077 <== mgr.14118 v2:172.21.15.156:6800/1023000918 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+26 (secure 0 0 0) 0x7ff66c0fe730 con 0x7ff6580357d0 2024-07-16T09:47:07.686 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:06.870+0000 7ff673924640 1 -- 172.21.15.156:0/1397822077 >> [v2:172.21.15.156:6800/1023000918,v1:172.21.15.156:6801/1023000918] conn(0x7ff6580357d0 msgr2=0x7ff658037c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:47:07.686 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:06.870+0000 7ff673924640 1 --2- 172.21.15.156:0/1397822077 >> [v2:172.21.15.156:6800/1023000918,v1:172.21.15.156:6801/1023000918] conn(0x7ff6580357d0 0x7ff658037c80 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7ff65c006fd0 tx=0x7ff65c006db0 comp rx=0 tx=0).stop 2024-07-16T09:47:07.686 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:06.870+0000 7ff673924640 1 -- 172.21.15.156:0/1397822077 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff66c077c80 msgr2=0x7ff66c112ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:47:07.686 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:06.870+0000 7ff673924640 1 --2- 172.21.15.156:0/1397822077 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff66c077c80 0x7ff66c112ca0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7ff668031b90 tx=0x7ff668008040 comp rx=0 tx=0).stop 2024-07-16T09:47:07.687 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:06.870+0000 7ff673924640 1 -- 172.21.15.156:0/1397822077 shutdown_connections 2024-07-16T09:47:07.687 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:06.870+0000 7ff673924640 1 --2- 172.21.15.156:0/1397822077 >> [v2:172.21.15.156:6800/1023000918,v1:172.21.15.156:6801/1023000918] conn(0x7ff6580357d0 0x7ff658037c80 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:47:07.687 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:06.870+0000 7ff673924640 1 --2- 172.21.15.156:0/1397822077 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff66c077c80 0x7ff66c112ca0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:47:07.687 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:06.870+0000 7ff673924640 1 -- 172.21.15.156:0/1397822077 >> 172.21.15.156:0/1397822077 conn(0x7ff66c0fd5e0 msgr2=0x7ff66c0fe020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:47:07.687 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:06.871+0000 7ff673924640 1 -- 172.21.15.156:0/1397822077 shutdown_connections 2024-07-16T09:47:07.687 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:06.871+0000 7ff673924640 1 -- 172.21.15.156:0/1397822077 wait complete. 2024-07-16T09:47:07.687 INFO:teuthology.orchestra.run.smithi156.stdout:Deploying prometheus service with default placement... 2024-07-16T09:47:07.924 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:47:07 smithi156 ceph-mon[29302]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "crash", "target": ["mon-mgr", ""]}]: dispatch 2024-07-16T09:47:07.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:47:07 smithi156 ceph-mon[29302]: Saving service crash spec with placement * 2024-07-16T09:47:07.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:47:07 smithi156 ceph-mon[29302]: from='mgr.14118 172.21.15.156:0/3208530346' entity='mgr.smithi156.zkvylv' 2024-07-16T09:47:09.294 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:47:09 smithi156 ceph-mon[29302]: from='mgr.14118 172.21.15.156:0/3208530346' entity='mgr.smithi156.zkvylv' 2024-07-16T09:47:09.295 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:47:09 smithi156 ceph-mon[29302]: from='mgr.14118 172.21.15.156:0/3208530346' entity='mgr.smithi156.zkvylv' 2024-07-16T09:47:09.295 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:47:09 smithi156 ceph-mon[29302]: from='mgr.14118 172.21.15.156:0/3208530346' entity='mgr.smithi156.zkvylv' 2024-07-16T09:47:09.302 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout Scheduled prometheus update... 2024-07-16T09:47:09.302 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:08.552+0000 7f64f1bab640 1 Processor -- start 2024-07-16T09:47:09.302 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:08.552+0000 7f64f1bab640 1 -- start start 2024-07-16T09:47:09.302 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:08.552+0000 7f64f1bab640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f64ec103220 0x7f64ec103630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:47:09.302 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:08.552+0000 7f64f1bab640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64ec103c00 con 0x7f64ec103220 2024-07-16T09:47:09.302 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:08.552+0000 7f64eb7fe640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f64ec103220 0x7f64ec103630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:47:09.302 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:08.552+0000 7f64eb7fe640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f64ec103220 0x7f64ec103630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:59646/0 (socket says 172.21.15.156:59646) 2024-07-16T09:47:09.302 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:08.552+0000 7f64eb7fe640 1 -- 172.21.15.156:0/3957211893 learned_addr learned my addr 172.21.15.156:0/3957211893 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T09:47:09.302 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:08.552+0000 7f64eb7fe640 1 -- 172.21.15.156:0/3957211893 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64ec103d40 con 0x7f64ec103220 2024-07-16T09:47:09.302 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:08.553+0000 7f64eb7fe640 1 --2- 172.21.15.156:0/3957211893 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f64ec103220 0x7f64ec103630 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f64dc009840 tx=0x7f64dc0091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dba3b31b41fc4a3c server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:47:09.302 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:08.553+0000 7f64ea7fc640 1 -- 172.21.15.156:0/3957211893 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f64dc032440 con 0x7f64ec103220 2024-07-16T09:47:09.303 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:08.553+0000 7f64ea7fc640 1 -- 172.21.15.156:0/3957211893 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f64dc032a70 con 0x7f64ec103220 2024-07-16T09:47:09.303 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:08.553+0000 7f64ea7fc640 1 -- 172.21.15.156:0/3957211893 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f64dc031ad0 con 0x7f64ec103220 2024-07-16T09:47:09.303 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:08.554+0000 7f64f1bab640 1 -- 172.21.15.156:0/3957211893 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f64ec103220 msgr2=0x7f64ec103630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:47:09.303 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:08.554+0000 7f64f1bab640 1 --2- 172.21.15.156:0/3957211893 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f64ec103220 0x7f64ec103630 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f64dc009840 tx=0x7f64dc0091d0 comp rx=0 tx=0).stop 2024-07-16T09:47:09.303 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:08.554+0000 7f64f1bab640 1 -- 172.21.15.156:0/3957211893 shutdown_connections 2024-07-16T09:47:09.303 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:08.554+0000 7f64f1bab640 1 --2- 172.21.15.156:0/3957211893 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f64ec103220 0x7f64ec103630 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:47:09.303 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:08.554+0000 7f64f1bab640 1 -- 172.21.15.156:0/3957211893 >> 172.21.15.156:0/3957211893 conn(0x7f64ec0fbc50 msgr2=0x7f64ec0fe0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:47:09.303 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:08.554+0000 7f64f1bab640 1 -- 172.21.15.156:0/3957211893 shutdown_connections 2024-07-16T09:47:09.304 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:08.554+0000 7f64f1bab640 1 -- 172.21.15.156:0/3957211893 wait complete. 2024-07-16T09:47:09.304 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:08.555+0000 7f64f1bab640 1 Processor -- start 2024-07-16T09:47:09.304 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:08.555+0000 7f64f1bab640 1 -- start start 2024-07-16T09:47:09.304 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:08.555+0000 7f64f1bab640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f64ec103220 0x7f64ec110b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:47:09.304 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:08.555+0000 7f64f1bab640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64ec1110b0 con 0x7f64ec103220 2024-07-16T09:47:09.304 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:08.555+0000 7f64eb7fe640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f64ec103220 0x7f64ec110b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:47:09.304 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:08.555+0000 7f64eb7fe640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f64ec103220 0x7f64ec110b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:59654/0 (socket says 172.21.15.156:59654) 2024-07-16T09:47:09.305 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:08.555+0000 7f64eb7fe640 1 -- 172.21.15.156:0/198908115 learned_addr learned my addr 172.21.15.156:0/198908115 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T09:47:09.305 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:08.555+0000 7f64eb7fe640 1 -- 172.21.15.156:0/198908115 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64dc009450 con 0x7f64ec103220 2024-07-16T09:47:09.305 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:08.556+0000 7f64eb7fe640 1 --2- 172.21.15.156:0/198908115 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f64ec103220 0x7f64ec110b70 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f64dc009810 tx=0x7f64dc02c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:47:09.305 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:08.556+0000 7f64d3fff640 1 -- 172.21.15.156:0/198908115 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f64dc0328b0 con 0x7f64ec103220 2024-07-16T09:47:09.305 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:08.556+0000 7f64d3fff640 1 -- 172.21.15.156:0/198908115 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f64dc03b910 con 0x7f64ec103220 2024-07-16T09:47:09.305 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:08.556+0000 7f64f1bab640 1 -- 172.21.15.156:0/198908115 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64ec1112b0 con 0x7f64ec103220 2024-07-16T09:47:09.305 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:08.556+0000 7f64d3fff640 1 -- 172.21.15.156:0/198908115 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f64dc03a980 con 0x7f64ec103220 2024-07-16T09:47:09.305 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:08.556+0000 7f64f1bab640 1 -- 172.21.15.156:0/198908115 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64ec111720 con 0x7f64ec103220 2024-07-16T09:47:09.305 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:08.557+0000 7f64d3fff640 1 -- 172.21.15.156:0/198908115 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 8) v1 ==== 42636+0+0 (secure 0 0 0) 0x7f64dc039070 con 0x7f64ec103220 2024-07-16T09:47:09.305 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:08.557+0000 7f64d3fff640 1 --2- 172.21.15.156:0/198908115 >> [v2:172.21.15.156:6800/1023000918,v1:172.21.15.156:6801/1023000918] conn(0x7f64d40357d0 0x7f64d4037c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:47:09.306 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:08.557+0000 7f64d3fff640 1 -- 172.21.15.156:0/198908115 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f64dc068160 con 0x7f64ec103220 2024-07-16T09:47:09.306 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:08.557+0000 7f64f1bab640 1 -- 172.21.15.156:0/198908115 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64ec05eb80 con 0x7f64ec103220 2024-07-16T09:47:09.306 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:08.557+0000 7f64eaffd640 1 --2- 172.21.15.156:0/198908115 >> [v2:172.21.15.156:6800/1023000918,v1:172.21.15.156:6801/1023000918] conn(0x7f64d40357d0 0x7f64d4037c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:47:09.306 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:08.558+0000 7f64eaffd640 1 --2- 172.21.15.156:0/198908115 >> [v2:172.21.15.156:6800/1023000918,v1:172.21.15.156:6801/1023000918] conn(0x7f64d40357d0 0x7f64d4037c80 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f64e0006fd0 tx=0x7f64e0006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:47:09.306 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:08.562+0000 7f64d3fff640 1 -- 172.21.15.156:0/198908115 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f64dc037030 con 0x7f64ec103220 2024-07-16T09:47:09.306 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:08.757+0000 7f64f1bab640 1 -- 172.21.15.156:0/198908115 --> [v2:172.21.15.156:6800/1023000918,v1:172.21.15.156:6801/1023000918] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "prometheus", "target": ["mon-mgr", ""]}) v1 -- 0x7f64ec0fcee0 con 0x7f64d40357d0 2024-07-16T09:47:09.306 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:08.762+0000 7f64d3fff640 1 -- 172.21.15.156:0/198908115 <== mgr.14118 v2:172.21.15.156:6800/1023000918 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+31 (secure 0 0 0) 0x7f64ec0fcee0 con 0x7f64d40357d0 2024-07-16T09:47:09.306 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:08.764+0000 7f64f1bab640 1 -- 172.21.15.156:0/198908115 >> [v2:172.21.15.156:6800/1023000918,v1:172.21.15.156:6801/1023000918] conn(0x7f64d40357d0 msgr2=0x7f64d4037c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:47:09.306 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:08.764+0000 7f64f1bab640 1 --2- 172.21.15.156:0/198908115 >> [v2:172.21.15.156:6800/1023000918,v1:172.21.15.156:6801/1023000918] conn(0x7f64d40357d0 0x7f64d4037c80 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f64e0006fd0 tx=0x7f64e0006db0 comp rx=0 tx=0).stop 2024-07-16T09:47:09.306 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:08.764+0000 7f64f1bab640 1 -- 172.21.15.156:0/198908115 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f64ec103220 msgr2=0x7f64ec110b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:47:09.306 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:08.764+0000 7f64f1bab640 1 --2- 172.21.15.156:0/198908115 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f64ec103220 0x7f64ec110b70 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f64dc009810 tx=0x7f64dc02c040 comp rx=0 tx=0).stop 2024-07-16T09:47:09.306 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:08.764+0000 7f64f1bab640 1 -- 172.21.15.156:0/198908115 shutdown_connections 2024-07-16T09:47:09.307 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:08.764+0000 7f64f1bab640 1 --2- 172.21.15.156:0/198908115 >> [v2:172.21.15.156:6800/1023000918,v1:172.21.15.156:6801/1023000918] conn(0x7f64d40357d0 0x7f64d4037c80 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:47:09.307 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:08.764+0000 7f64f1bab640 1 --2- 172.21.15.156:0/198908115 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f64ec103220 0x7f64ec110b70 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:47:09.307 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:08.764+0000 7f64f1bab640 1 -- 172.21.15.156:0/198908115 >> 172.21.15.156:0/198908115 conn(0x7f64ec0fbc50 msgr2=0x7f64ec0fc7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:47:09.307 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:08.765+0000 7f64f1bab640 1 -- 172.21.15.156:0/198908115 shutdown_connections 2024-07-16T09:47:09.307 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:08.765+0000 7f64f1bab640 1 -- 172.21.15.156:0/198908115 wait complete. 2024-07-16T09:47:09.307 INFO:teuthology.orchestra.run.smithi156.stdout:Deploying grafana service with default placement... 2024-07-16T09:47:10.316 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:47:10 smithi156 ceph-mon[29302]: from='client.14146 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "target": ["mon-mgr", ""]}]: dispatch 2024-07-16T09:47:10.316 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:47:10 smithi156 ceph-mon[29302]: Saving service prometheus spec with placement count:1 2024-07-16T09:47:10.316 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:47:10 smithi156 ceph-mon[29302]: from='mgr.14118 172.21.15.156:0/3208530346' entity='mgr.smithi156.zkvylv' 2024-07-16T09:47:11.262 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout Scheduled grafana update... 2024-07-16T09:47:11.263 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:10.478+0000 7f4617321640 1 Processor -- start 2024-07-16T09:47:11.263 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:10.478+0000 7f4617321640 1 -- start start 2024-07-16T09:47:11.263 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:10.478+0000 7f4617321640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4610101dc0 0x7f46101021d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:47:11.263 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:10.478+0000 7f4617321640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4610102710 con 0x7f4610101dc0 2024-07-16T09:47:11.263 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:10.479+0000 7f46150bd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4610101dc0 0x7f46101021d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:47:11.263 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:10.479+0000 7f46150bd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4610101dc0 0x7f46101021d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:59656/0 (socket says 172.21.15.156:59656) 2024-07-16T09:47:11.263 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:10.479+0000 7f46150bd640 1 -- 172.21.15.156:0/3299374404 learned_addr learned my addr 172.21.15.156:0/3299374404 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T09:47:11.263 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:10.479+0000 7f46150bd640 1 -- 172.21.15.156:0/3299374404 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4610102850 con 0x7f4610101dc0 2024-07-16T09:47:11.263 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:10.480+0000 7f46150bd640 1 --2- 172.21.15.156:0/3299374404 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4610101dc0 0x7f46101021d0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f460c009840 tx=0x7f460c0091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=200f449e94879580 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:47:11.263 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:10.480+0000 7f4607fff640 1 -- 172.21.15.156:0/3299374404 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f460c032440 con 0x7f4610101dc0 2024-07-16T09:47:11.264 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:10.480+0000 7f4607fff640 1 -- 172.21.15.156:0/3299374404 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f460c032a70 con 0x7f4610101dc0 2024-07-16T09:47:11.264 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:10.480+0000 7f4607fff640 1 -- 172.21.15.156:0/3299374404 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f460c031ad0 con 0x7f4610101dc0 2024-07-16T09:47:11.264 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:10.481+0000 7f4617321640 1 -- 172.21.15.156:0/3299374404 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4610101dc0 msgr2=0x7f46101021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:47:11.264 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:10.481+0000 7f4617321640 1 --2- 172.21.15.156:0/3299374404 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4610101dc0 0x7f46101021d0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f460c009840 tx=0x7f460c0091d0 comp rx=0 tx=0).stop 2024-07-16T09:47:11.264 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:10.481+0000 7f4617321640 1 -- 172.21.15.156:0/3299374404 shutdown_connections 2024-07-16T09:47:11.264 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:10.481+0000 7f4617321640 1 --2- 172.21.15.156:0/3299374404 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4610101dc0 0x7f46101021d0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:47:11.264 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:10.481+0000 7f4617321640 1 -- 172.21.15.156:0/3299374404 >> 172.21.15.156:0/3299374404 conn(0x7f46100fd5e0 msgr2=0x7f46100ff9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:47:11.264 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:10.481+0000 7f4617321640 1 -- 172.21.15.156:0/3299374404 shutdown_connections 2024-07-16T09:47:11.264 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:10.481+0000 7f4617321640 1 -- 172.21.15.156:0/3299374404 wait complete. 2024-07-16T09:47:11.264 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:10.482+0000 7f4617321640 1 Processor -- start 2024-07-16T09:47:11.264 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:10.482+0000 7f4617321640 1 -- start start 2024-07-16T09:47:11.264 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:10.482+0000 7f4617321640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4610101dc0 0x7f4610077c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:47:11.264 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:10.482+0000 7f4617321640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4610078160 con 0x7f4610101dc0 2024-07-16T09:47:11.264 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:10.482+0000 7f46150bd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4610101dc0 0x7f4610077c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:47:11.265 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:10.482+0000 7f46150bd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4610101dc0 0x7f4610077c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:59660/0 (socket says 172.21.15.156:59660) 2024-07-16T09:47:11.265 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:10.482+0000 7f46150bd640 1 -- 172.21.15.156:0/2764302100 learned_addr learned my addr 172.21.15.156:0/2764302100 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T09:47:11.265 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:10.482+0000 7f46150bd640 1 -- 172.21.15.156:0/2764302100 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f460c009450 con 0x7f4610101dc0 2024-07-16T09:47:11.265 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:10.482+0000 7f46150bd640 1 --2- 172.21.15.156:0/2764302100 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4610101dc0 0x7f4610077c20 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f460c0098a0 tx=0x7f460c02c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:47:11.265 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:10.482+0000 7f4605ffb640 1 -- 172.21.15.156:0/2764302100 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f460c031e90 con 0x7f4610101dc0 2024-07-16T09:47:11.265 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:10.482+0000 7f4605ffb640 1 -- 172.21.15.156:0/2764302100 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f460c03b910 con 0x7f4610101dc0 2024-07-16T09:47:11.265 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:10.482+0000 7f4617321640 1 -- 172.21.15.156:0/2764302100 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46100761f0 con 0x7f4610101dc0 2024-07-16T09:47:11.265 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:10.483+0000 7f4605ffb640 1 -- 172.21.15.156:0/2764302100 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f460c03a8d0 con 0x7f4610101dc0 2024-07-16T09:47:11.265 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:10.483+0000 7f4617321640 1 -- 172.21.15.156:0/2764302100 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4610076660 con 0x7f4610101dc0 2024-07-16T09:47:11.265 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:10.484+0000 7f4605ffb640 1 -- 172.21.15.156:0/2764302100 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 8) v1 ==== 42636+0+0 (secure 0 0 0) 0x7f460c039070 con 0x7f4610101dc0 2024-07-16T09:47:11.265 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:10.484+0000 7f4605ffb640 1 --2- 172.21.15.156:0/2764302100 >> [v2:172.21.15.156:6800/1023000918,v1:172.21.15.156:6801/1023000918] conn(0x7f45fc035780 0x7f45fc037c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:47:11.265 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:10.484+0000 7f4605ffb640 1 -- 172.21.15.156:0/2764302100 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f460c067fc0 con 0x7f4610101dc0 2024-07-16T09:47:11.265 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:10.484+0000 7f4617321640 1 -- 172.21.15.156:0/2764302100 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f461004b530 con 0x7f4610101dc0 2024-07-16T09:47:11.266 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:10.484+0000 7f46148bc640 1 --2- 172.21.15.156:0/2764302100 >> [v2:172.21.15.156:6800/1023000918,v1:172.21.15.156:6801/1023000918] conn(0x7f45fc035780 0x7f45fc037c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:47:11.266 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:10.485+0000 7f46148bc640 1 --2- 172.21.15.156:0/2764302100 >> [v2:172.21.15.156:6800/1023000918,v1:172.21.15.156:6801/1023000918] conn(0x7f45fc035780 0x7f45fc037c30 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f4600006fd0 tx=0x7f4600006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:47:11.266 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:10.489+0000 7f4605ffb640 1 -- 172.21.15.156:0/2764302100 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f460c03f080 con 0x7f4610101dc0 2024-07-16T09:47:11.266 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:10.672+0000 7f4617321640 1 -- 172.21.15.156:0/2764302100 --> [v2:172.21.15.156:6800/1023000918,v1:172.21.15.156:6801/1023000918] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "grafana", "target": ["mon-mgr", ""]}) v1 -- 0x7f4610076c40 con 0x7f45fc035780 2024-07-16T09:47:11.266 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:10.677+0000 7f4605ffb640 1 -- 172.21.15.156:0/2764302100 <== mgr.14118 v2:172.21.15.156:6800/1023000918 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+28 (secure 0 0 0) 0x7f4610076c40 con 0x7f45fc035780 2024-07-16T09:47:11.266 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:10.680+0000 7f4617321640 1 -- 172.21.15.156:0/2764302100 >> [v2:172.21.15.156:6800/1023000918,v1:172.21.15.156:6801/1023000918] conn(0x7f45fc035780 msgr2=0x7f45fc037c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:47:11.266 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:10.680+0000 7f4617321640 1 --2- 172.21.15.156:0/2764302100 >> [v2:172.21.15.156:6800/1023000918,v1:172.21.15.156:6801/1023000918] conn(0x7f45fc035780 0x7f45fc037c30 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f4600006fd0 tx=0x7f4600006db0 comp rx=0 tx=0).stop 2024-07-16T09:47:11.266 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:10.680+0000 7f4617321640 1 -- 172.21.15.156:0/2764302100 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4610101dc0 msgr2=0x7f4610077c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:47:11.266 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:10.680+0000 7f4617321640 1 --2- 172.21.15.156:0/2764302100 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4610101dc0 0x7f4610077c20 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f460c0098a0 tx=0x7f460c02c040 comp rx=0 tx=0).stop 2024-07-16T09:47:11.266 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:10.680+0000 7f4617321640 1 -- 172.21.15.156:0/2764302100 shutdown_connections 2024-07-16T09:47:11.266 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:10.680+0000 7f4617321640 1 --2- 172.21.15.156:0/2764302100 >> [v2:172.21.15.156:6800/1023000918,v1:172.21.15.156:6801/1023000918] conn(0x7f45fc035780 0x7f45fc037c30 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:47:11.266 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:10.680+0000 7f4617321640 1 --2- 172.21.15.156:0/2764302100 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4610101dc0 0x7f4610077c20 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:47:11.266 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:10.680+0000 7f4617321640 1 -- 172.21.15.156:0/2764302100 >> 172.21.15.156:0/2764302100 conn(0x7f46100fd5e0 msgr2=0x7f46100fe0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:47:11.266 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:10.680+0000 7f4617321640 1 -- 172.21.15.156:0/2764302100 shutdown_connections 2024-07-16T09:47:11.267 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:10.680+0000 7f4617321640 1 -- 172.21.15.156:0/2764302100 wait complete. 2024-07-16T09:47:11.267 INFO:teuthology.orchestra.run.smithi156.stdout:Deploying node-exporter service with default placement... 2024-07-16T09:47:11.847 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:47:11 smithi156 ceph-mon[29302]: from='client.14148 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "target": ["mon-mgr", ""]}]: dispatch 2024-07-16T09:47:11.847 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:47:11 smithi156 ceph-mon[29302]: Saving service grafana spec with placement count:1 2024-07-16T09:47:11.847 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:47:11 smithi156 ceph-mon[29302]: from='mgr.14118 172.21.15.156:0/3208530346' entity='mgr.smithi156.zkvylv' 2024-07-16T09:47:11.848 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:47:11 smithi156 ceph-mon[29302]: from='mgr.14118 172.21.15.156:0/3208530346' entity='mgr.smithi156.zkvylv' 2024-07-16T09:47:13.108 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:47:12 smithi156 ceph-mon[29302]: from='client.14150 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "target": ["mon-mgr", ""]}]: dispatch 2024-07-16T09:47:13.108 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:47:12 smithi156 ceph-mon[29302]: Saving service node-exporter spec with placement * 2024-07-16T09:47:13.108 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:47:12 smithi156 ceph-mon[29302]: from='mgr.14118 172.21.15.156:0/3208530346' entity='mgr.smithi156.zkvylv' 2024-07-16T09:47:13.116 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout Scheduled node-exporter update... 2024-07-16T09:47:13.116 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:11.655+0000 7fb723e41640 1 Processor -- start 2024-07-16T09:47:13.116 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:11.656+0000 7fb723e41640 1 -- start start 2024-07-16T09:47:13.116 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:11.656+0000 7fb723e41640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb71c101e80 0x7fb71c102290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:47:13.116 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:11.656+0000 7fb723e41640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb71c1027d0 con 0x7fb71c101e80 2024-07-16T09:47:13.116 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:11.656+0000 7fb721bdd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb71c101e80 0x7fb71c102290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:47:13.116 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:11.656+0000 7fb721bdd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb71c101e80 0x7fb71c102290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:59664/0 (socket says 172.21.15.156:59664) 2024-07-16T09:47:13.117 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:11.656+0000 7fb721bdd640 1 -- 172.21.15.156:0/3144395436 learned_addr learned my addr 172.21.15.156:0/3144395436 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T09:47:13.117 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:11.656+0000 7fb721bdd640 1 -- 172.21.15.156:0/3144395436 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb71c102910 con 0x7fb71c101e80 2024-07-16T09:47:13.117 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:11.657+0000 7fb721bdd640 1 --2- 172.21.15.156:0/3144395436 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb71c101e80 0x7fb71c102290 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fb718009840 tx=0x7fb7180091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=245c55b5428d0dea server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:47:13.117 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:11.657+0000 7fb720bdb640 1 -- 172.21.15.156:0/3144395436 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb718032440 con 0x7fb71c101e80 2024-07-16T09:47:13.117 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:11.657+0000 7fb720bdb640 1 -- 172.21.15.156:0/3144395436 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb718032a70 con 0x7fb71c101e80 2024-07-16T09:47:13.117 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:11.657+0000 7fb720bdb640 1 -- 172.21.15.156:0/3144395436 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb718031ad0 con 0x7fb71c101e80 2024-07-16T09:47:13.117 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:11.658+0000 7fb723e41640 1 -- 172.21.15.156:0/3144395436 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb71c101e80 msgr2=0x7fb71c102290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:47:13.117 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:11.658+0000 7fb723e41640 1 --2- 172.21.15.156:0/3144395436 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb71c101e80 0x7fb71c102290 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fb718009840 tx=0x7fb7180091d0 comp rx=0 tx=0).stop 2024-07-16T09:47:13.117 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:11.658+0000 7fb723e41640 1 -- 172.21.15.156:0/3144395436 shutdown_connections 2024-07-16T09:47:13.117 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:11.658+0000 7fb723e41640 1 --2- 172.21.15.156:0/3144395436 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb71c101e80 0x7fb71c102290 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:47:13.117 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:11.658+0000 7fb723e41640 1 -- 172.21.15.156:0/3144395436 >> 172.21.15.156:0/3144395436 conn(0x7fb71c0fd5e0 msgr2=0x7fb71c0ff9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:47:13.117 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:11.658+0000 7fb723e41640 1 -- 172.21.15.156:0/3144395436 shutdown_connections 2024-07-16T09:47:13.118 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:11.658+0000 7fb723e41640 1 -- 172.21.15.156:0/3144395436 wait complete. 2024-07-16T09:47:13.118 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:11.658+0000 7fb723e41640 1 Processor -- start 2024-07-16T09:47:13.118 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:11.658+0000 7fb723e41640 1 -- start start 2024-07-16T09:47:13.118 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:11.659+0000 7fb723e41640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb71c101e80 0x7fb71c110b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:47:13.118 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:11.659+0000 7fb723e41640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb71c111070 con 0x7fb71c101e80 2024-07-16T09:47:13.118 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:11.659+0000 7fb721bdd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb71c101e80 0x7fb71c110b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:47:13.118 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:11.659+0000 7fb721bdd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb71c101e80 0x7fb71c110b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:59674/0 (socket says 172.21.15.156:59674) 2024-07-16T09:47:13.118 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:11.659+0000 7fb721bdd640 1 -- 172.21.15.156:0/988074667 learned_addr learned my addr 172.21.15.156:0/988074667 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T09:47:13.118 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:11.659+0000 7fb721bdd640 1 -- 172.21.15.156:0/988074667 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb718009450 con 0x7fb71c101e80 2024-07-16T09:47:13.118 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:11.659+0000 7fb721bdd640 1 --2- 172.21.15.156:0/988074667 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb71c101e80 0x7fb71c110b30 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fb718009810 tx=0x7fb71802c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:47:13.118 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:11.659+0000 7fb7127fc640 1 -- 172.21.15.156:0/988074667 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb7180328b0 con 0x7fb71c101e80 2024-07-16T09:47:13.118 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:11.659+0000 7fb7127fc640 1 -- 172.21.15.156:0/988074667 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb71803b910 con 0x7fb71c101e80 2024-07-16T09:47:13.118 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:11.659+0000 7fb723e41640 1 -- 172.21.15.156:0/988074667 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb71c111270 con 0x7fb71c101e80 2024-07-16T09:47:13.119 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:11.659+0000 7fb7127fc640 1 -- 172.21.15.156:0/988074667 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb71803a980 con 0x7fb71c101e80 2024-07-16T09:47:13.119 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:11.659+0000 7fb723e41640 1 -- 172.21.15.156:0/988074667 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb71c1116e0 con 0x7fb71c101e80 2024-07-16T09:47:13.119 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:11.660+0000 7fb7127fc640 1 -- 172.21.15.156:0/988074667 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 8) v1 ==== 42636+0+0 (secure 0 0 0) 0x7fb718039070 con 0x7fb71c101e80 2024-07-16T09:47:13.119 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:11.660+0000 7fb7127fc640 1 --2- 172.21.15.156:0/988074667 >> [v2:172.21.15.156:6800/1023000918,v1:172.21.15.156:6801/1023000918] conn(0x7fb708035780 0x7fb708037c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:47:13.119 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:11.660+0000 7fb7127fc640 1 -- 172.21.15.156:0/988074667 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7fb718067ee0 con 0x7fb71c101e80 2024-07-16T09:47:13.119 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:11.660+0000 7fb7213dc640 1 --2- 172.21.15.156:0/988074667 >> [v2:172.21.15.156:6800/1023000918,v1:172.21.15.156:6801/1023000918] conn(0x7fb708035780 0x7fb708037c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:47:13.119 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:11.660+0000 7fb723e41640 1 -- 172.21.15.156:0/988074667 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb71c05eb80 con 0x7fb71c101e80 2024-07-16T09:47:13.119 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:11.661+0000 7fb7213dc640 1 --2- 172.21.15.156:0/988074667 >> [v2:172.21.15.156:6800/1023000918,v1:172.21.15.156:6801/1023000918] conn(0x7fb708035780 0x7fb708037c30 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fb70c006fd0 tx=0x7fb70c006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:47:13.119 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:11.665+0000 7fb7127fc640 1 -- 172.21.15.156:0/988074667 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fb71803f080 con 0x7fb71c101e80 2024-07-16T09:47:13.119 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:11.846+0000 7fb723e41640 1 -- 172.21.15.156:0/988074667 --> [v2:172.21.15.156:6800/1023000918,v1:172.21.15.156:6801/1023000918] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "node-exporter", "target": ["mon-mgr", ""]}) v1 -- 0x7fb71c0fe770 con 0x7fb708035780 2024-07-16T09:47:13.119 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:11.850+0000 7fb7127fc640 1 -- 172.21.15.156:0/988074667 <== mgr.14118 v2:172.21.15.156:6800/1023000918 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+34 (secure 0 0 0) 0x7fb71c0fe770 con 0x7fb708035780 2024-07-16T09:47:13.119 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:11.852+0000 7fb723e41640 1 -- 172.21.15.156:0/988074667 >> [v2:172.21.15.156:6800/1023000918,v1:172.21.15.156:6801/1023000918] conn(0x7fb708035780 msgr2=0x7fb708037c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:47:13.120 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:11.852+0000 7fb723e41640 1 --2- 172.21.15.156:0/988074667 >> [v2:172.21.15.156:6800/1023000918,v1:172.21.15.156:6801/1023000918] conn(0x7fb708035780 0x7fb708037c30 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fb70c006fd0 tx=0x7fb70c006db0 comp rx=0 tx=0).stop 2024-07-16T09:47:13.120 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:11.852+0000 7fb723e41640 1 -- 172.21.15.156:0/988074667 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb71c101e80 msgr2=0x7fb71c110b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:47:13.120 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:11.852+0000 7fb723e41640 1 --2- 172.21.15.156:0/988074667 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb71c101e80 0x7fb71c110b30 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fb718009810 tx=0x7fb71802c040 comp rx=0 tx=0).stop 2024-07-16T09:47:13.120 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:11.852+0000 7fb723e41640 1 -- 172.21.15.156:0/988074667 shutdown_connections 2024-07-16T09:47:13.120 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:11.852+0000 7fb723e41640 1 --2- 172.21.15.156:0/988074667 >> [v2:172.21.15.156:6800/1023000918,v1:172.21.15.156:6801/1023000918] conn(0x7fb708035780 0x7fb708037c30 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:47:13.120 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:11.852+0000 7fb723e41640 1 --2- 172.21.15.156:0/988074667 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb71c101e80 0x7fb71c110b30 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:47:13.120 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:11.852+0000 7fb723e41640 1 -- 172.21.15.156:0/988074667 >> 172.21.15.156:0/988074667 conn(0x7fb71c0fd5e0 msgr2=0x7fb71c0fe060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:47:13.120 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:11.852+0000 7fb723e41640 1 -- 172.21.15.156:0/988074667 shutdown_connections 2024-07-16T09:47:13.120 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:11.852+0000 7fb723e41640 1 -- 172.21.15.156:0/988074667 wait complete. 2024-07-16T09:47:13.120 INFO:teuthology.orchestra.run.smithi156.stdout:Deploying alertmanager service with default placement... 2024-07-16T09:47:14.307 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout Scheduled alertmanager update... 2024-07-16T09:47:14.307 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:13.532+0000 7f4f93d1a640 1 Processor -- start 2024-07-16T09:47:14.307 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:13.532+0000 7f4f93d1a640 1 -- start start 2024-07-16T09:47:14.307 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:13.532+0000 7f4f93d1a640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4f8c100180 0x7f4f8c100590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:47:14.307 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:13.532+0000 7f4f93d1a640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f8c100ad0 con 0x7f4f8c100180 2024-07-16T09:47:14.307 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:13.532+0000 7f4f91ab6640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4f8c100180 0x7f4f8c100590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:47:14.307 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:13.532+0000 7f4f91ab6640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4f8c100180 0x7f4f8c100590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:43982/0 (socket says 172.21.15.156:43982) 2024-07-16T09:47:14.308 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:13.532+0000 7f4f91ab6640 1 -- 172.21.15.156:0/3258415848 learned_addr learned my addr 172.21.15.156:0/3258415848 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T09:47:14.308 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:13.533+0000 7f4f91ab6640 1 -- 172.21.15.156:0/3258415848 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f8c100c10 con 0x7f4f8c100180 2024-07-16T09:47:14.308 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:13.533+0000 7f4f91ab6640 1 --2- 172.21.15.156:0/3258415848 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4f8c100180 0x7f4f8c100590 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f4f88009840 tx=0x7f4f880091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9b4c019ec904551 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:47:14.308 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:13.533+0000 7f4f90ab4640 1 -- 172.21.15.156:0/3258415848 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4f88032440 con 0x7f4f8c100180 2024-07-16T09:47:14.308 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:13.533+0000 7f4f90ab4640 1 -- 172.21.15.156:0/3258415848 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4f88032a70 con 0x7f4f8c100180 2024-07-16T09:47:14.308 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:13.533+0000 7f4f90ab4640 1 -- 172.21.15.156:0/3258415848 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4f88031ad0 con 0x7f4f8c100180 2024-07-16T09:47:14.308 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:13.534+0000 7f4f93d1a640 1 -- 172.21.15.156:0/3258415848 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4f8c100180 msgr2=0x7f4f8c100590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:47:14.308 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:13.534+0000 7f4f93d1a640 1 --2- 172.21.15.156:0/3258415848 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4f8c100180 0x7f4f8c100590 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f4f88009840 tx=0x7f4f880091d0 comp rx=0 tx=0).stop 2024-07-16T09:47:14.308 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:13.534+0000 7f4f93d1a640 1 -- 172.21.15.156:0/3258415848 shutdown_connections 2024-07-16T09:47:14.308 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:13.534+0000 7f4f93d1a640 1 --2- 172.21.15.156:0/3258415848 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4f8c100180 0x7f4f8c100590 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:47:14.308 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:13.534+0000 7f4f93d1a640 1 -- 172.21.15.156:0/3258415848 >> 172.21.15.156:0/3258415848 conn(0x7f4f8c0fbc50 msgr2=0x7f4f8c0fe0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:47:14.308 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:13.534+0000 7f4f93d1a640 1 -- 172.21.15.156:0/3258415848 shutdown_connections 2024-07-16T09:47:14.308 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:13.534+0000 7f4f93d1a640 1 -- 172.21.15.156:0/3258415848 wait complete. 2024-07-16T09:47:14.309 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:13.535+0000 7f4f93d1a640 1 Processor -- start 2024-07-16T09:47:14.309 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:13.535+0000 7f4f93d1a640 1 -- start start 2024-07-16T09:47:14.309 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:13.535+0000 7f4f93d1a640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4f8c100180 0x7f4f8c110b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:47:14.309 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:13.535+0000 7f4f93d1a640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f8c1110a0 con 0x7f4f8c100180 2024-07-16T09:47:14.309 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:13.535+0000 7f4f91ab6640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4f8c100180 0x7f4f8c110b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:47:14.309 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:13.535+0000 7f4f91ab6640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4f8c100180 0x7f4f8c110b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:43998/0 (socket says 172.21.15.156:43998) 2024-07-16T09:47:14.309 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:13.535+0000 7f4f91ab6640 1 -- 172.21.15.156:0/203942033 learned_addr learned my addr 172.21.15.156:0/203942033 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T09:47:14.309 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:13.536+0000 7f4f91ab6640 1 -- 172.21.15.156:0/203942033 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f88009450 con 0x7f4f8c100180 2024-07-16T09:47:14.309 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:13.536+0000 7f4f91ab6640 1 --2- 172.21.15.156:0/203942033 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4f8c100180 0x7f4f8c110b60 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f4f88009810 tx=0x7f4f8802c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:47:14.309 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:13.536+0000 7f4f827fc640 1 -- 172.21.15.156:0/203942033 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4f880328b0 con 0x7f4f8c100180 2024-07-16T09:47:14.309 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:13.536+0000 7f4f827fc640 1 -- 172.21.15.156:0/203942033 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4f8803b910 con 0x7f4f8c100180 2024-07-16T09:47:14.309 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:13.536+0000 7f4f93d1a640 1 -- 172.21.15.156:0/203942033 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f8c1112a0 con 0x7f4f8c100180 2024-07-16T09:47:14.310 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:13.536+0000 7f4f827fc640 1 -- 172.21.15.156:0/203942033 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4f8803a980 con 0x7f4f8c100180 2024-07-16T09:47:14.310 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:13.536+0000 7f4f93d1a640 1 -- 172.21.15.156:0/203942033 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f8c111710 con 0x7f4f8c100180 2024-07-16T09:47:14.310 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:13.537+0000 7f4f827fc640 1 -- 172.21.15.156:0/203942033 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 8) v1 ==== 42636+0+0 (secure 0 0 0) 0x7f4f88039070 con 0x7f4f8c100180 2024-07-16T09:47:14.310 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:13.537+0000 7f4f827fc640 1 --2- 172.21.15.156:0/203942033 >> [v2:172.21.15.156:6800/1023000918,v1:172.21.15.156:6801/1023000918] conn(0x7f4f780357d0 0x7f4f78037c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:47:14.310 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:13.537+0000 7f4f827fc640 1 -- 172.21.15.156:0/203942033 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f4f88067d80 con 0x7f4f8c100180 2024-07-16T09:47:14.310 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:13.537+0000 7f4f912b5640 1 --2- 172.21.15.156:0/203942033 >> [v2:172.21.15.156:6800/1023000918,v1:172.21.15.156:6801/1023000918] conn(0x7f4f780357d0 0x7f4f78037c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:47:14.310 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:13.537+0000 7f4f93d1a640 1 -- 172.21.15.156:0/203942033 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f8c05eb80 con 0x7f4f8c100180 2024-07-16T09:47:14.310 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:13.538+0000 7f4f912b5640 1 --2- 172.21.15.156:0/203942033 >> [v2:172.21.15.156:6800/1023000918,v1:172.21.15.156:6801/1023000918] conn(0x7f4f780357d0 0x7f4f78037c80 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f4f7c006fd0 tx=0x7f4f7c006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:47:14.310 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:13.542+0000 7f4f827fc640 1 -- 172.21.15.156:0/203942033 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f4f8803f080 con 0x7f4f8c100180 2024-07-16T09:47:14.310 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:13.724+0000 7f4f93d1a640 1 -- 172.21.15.156:0/203942033 --> [v2:172.21.15.156:6800/1023000918,v1:172.21.15.156:6801/1023000918] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "alertmanager", "target": ["mon-mgr", ""]}) v1 -- 0x7f4f8c0fcef0 con 0x7f4f780357d0 2024-07-16T09:47:14.310 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:13.728+0000 7f4f827fc640 1 -- 172.21.15.156:0/203942033 <== mgr.14118 v2:172.21.15.156:6800/1023000918 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+33 (secure 0 0 0) 0x7f4f8c0fcef0 con 0x7f4f780357d0 2024-07-16T09:47:14.310 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:13.730+0000 7f4f93d1a640 1 -- 172.21.15.156:0/203942033 >> [v2:172.21.15.156:6800/1023000918,v1:172.21.15.156:6801/1023000918] conn(0x7f4f780357d0 msgr2=0x7f4f78037c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:47:14.311 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:13.730+0000 7f4f93d1a640 1 --2- 172.21.15.156:0/203942033 >> [v2:172.21.15.156:6800/1023000918,v1:172.21.15.156:6801/1023000918] conn(0x7f4f780357d0 0x7f4f78037c80 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f4f7c006fd0 tx=0x7f4f7c006db0 comp rx=0 tx=0).stop 2024-07-16T09:47:14.311 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:13.730+0000 7f4f93d1a640 1 -- 172.21.15.156:0/203942033 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4f8c100180 msgr2=0x7f4f8c110b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:47:14.311 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:13.730+0000 7f4f93d1a640 1 --2- 172.21.15.156:0/203942033 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4f8c100180 0x7f4f8c110b60 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f4f88009810 tx=0x7f4f8802c040 comp rx=0 tx=0).stop 2024-07-16T09:47:14.311 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:13.730+0000 7f4f93d1a640 1 -- 172.21.15.156:0/203942033 shutdown_connections 2024-07-16T09:47:14.311 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:13.730+0000 7f4f93d1a640 1 --2- 172.21.15.156:0/203942033 >> [v2:172.21.15.156:6800/1023000918,v1:172.21.15.156:6801/1023000918] conn(0x7f4f780357d0 0x7f4f78037c80 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:47:14.311 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:13.730+0000 7f4f93d1a640 1 --2- 172.21.15.156:0/203942033 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4f8c100180 0x7f4f8c110b60 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:47:14.311 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:13.730+0000 7f4f93d1a640 1 -- 172.21.15.156:0/203942033 >> 172.21.15.156:0/203942033 conn(0x7f4f8c0fbc50 msgr2=0x7f4f8c0fc7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:47:14.311 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:13.730+0000 7f4f93d1a640 1 -- 172.21.15.156:0/203942033 shutdown_connections 2024-07-16T09:47:14.311 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:13.730+0000 7f4f93d1a640 1 -- 172.21.15.156:0/203942033 wait complete. 2024-07-16T09:47:14.880 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:47:14 smithi156 ceph-mon[29302]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-16T09:47:14.880 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:47:14 smithi156 ceph-mon[29302]: from='client.14152 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "target": ["mon-mgr", ""]}]: dispatch 2024-07-16T09:47:14.880 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:47:14 smithi156 ceph-mon[29302]: Saving service alertmanager spec with placement count:1 2024-07-16T09:47:14.880 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:47:14 smithi156 ceph-mon[29302]: from='mgr.14118 172.21.15.156:0/3208530346' entity='mgr.smithi156.zkvylv' 2024-07-16T09:47:15.444 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:14.703+0000 7fd5ba64c640 1 Processor -- start 2024-07-16T09:47:15.445 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:14.703+0000 7fd5ba64c640 1 -- start start 2024-07-16T09:47:15.445 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:14.703+0000 7fd5ba64c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd5b4100ce0 0x7fd5b41010f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:47:15.445 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:14.703+0000 7fd5ba64c640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5b4101630 con 0x7fd5b4100ce0 2024-07-16T09:47:15.445 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:14.704+0000 7fd5b3fff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd5b4100ce0 0x7fd5b41010f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:47:15.445 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:14.704+0000 7fd5b3fff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd5b4100ce0 0x7fd5b41010f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:44010/0 (socket says 172.21.15.156:44010) 2024-07-16T09:47:15.445 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:14.704+0000 7fd5b3fff640 1 -- 172.21.15.156:0/2205341524 learned_addr learned my addr 172.21.15.156:0/2205341524 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T09:47:15.445 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:14.704+0000 7fd5b3fff640 1 -- 172.21.15.156:0/2205341524 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5b4101770 con 0x7fd5b4100ce0 2024-07-16T09:47:15.445 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:14.705+0000 7fd5b3fff640 1 --2- 172.21.15.156:0/2205341524 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd5b4100ce0 0x7fd5b41010f0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fd5a4009840 tx=0x7fd5a40091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34f25b6a52d6a5e5 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:47:15.445 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:14.705+0000 7fd5b37fe640 1 -- 172.21.15.156:0/2205341524 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd5a4032440 con 0x7fd5b4100ce0 2024-07-16T09:47:15.445 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:14.705+0000 7fd5b37fe640 1 -- 172.21.15.156:0/2205341524 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd5a4032a70 con 0x7fd5b4100ce0 2024-07-16T09:47:15.445 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:14.705+0000 7fd5b37fe640 1 -- 172.21.15.156:0/2205341524 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd5a4031ad0 con 0x7fd5b4100ce0 2024-07-16T09:47:15.445 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:14.706+0000 7fd5ba64c640 1 -- 172.21.15.156:0/2205341524 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd5b4100ce0 msgr2=0x7fd5b41010f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:47:15.446 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:14.706+0000 7fd5ba64c640 1 --2- 172.21.15.156:0/2205341524 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd5b4100ce0 0x7fd5b41010f0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fd5a4009840 tx=0x7fd5a40091d0 comp rx=0 tx=0).stop 2024-07-16T09:47:15.446 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:14.706+0000 7fd5ba64c640 1 -- 172.21.15.156:0/2205341524 shutdown_connections 2024-07-16T09:47:15.446 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:14.706+0000 7fd5ba64c640 1 --2- 172.21.15.156:0/2205341524 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd5b4100ce0 0x7fd5b41010f0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:47:15.446 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:14.706+0000 7fd5ba64c640 1 -- 172.21.15.156:0/2205341524 >> 172.21.15.156:0/2205341524 conn(0x7fd5b40fc520 msgr2=0x7fd5b40fe930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:47:15.446 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:14.706+0000 7fd5ba64c640 1 -- 172.21.15.156:0/2205341524 shutdown_connections 2024-07-16T09:47:15.446 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:14.706+0000 7fd5ba64c640 1 -- 172.21.15.156:0/2205341524 wait complete. 2024-07-16T09:47:15.446 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:14.706+0000 7fd5ba64c640 1 Processor -- start 2024-07-16T09:47:15.446 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:14.706+0000 7fd5ba64c640 1 -- start start 2024-07-16T09:47:15.446 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:14.706+0000 7fd5ba64c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd5b4100ce0 0x7fd5b4075b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:47:15.446 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:14.706+0000 7fd5ba64c640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5b4113840 con 0x7fd5b4100ce0 2024-07-16T09:47:15.446 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:14.706+0000 7fd5b3fff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd5b4100ce0 0x7fd5b4075b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:47:15.446 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:14.706+0000 7fd5b3fff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd5b4100ce0 0x7fd5b4075b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:44024/0 (socket says 172.21.15.156:44024) 2024-07-16T09:47:15.446 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:14.706+0000 7fd5b3fff640 1 -- 172.21.15.156:0/983602857 learned_addr learned my addr 172.21.15.156:0/983602857 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T09:47:15.447 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:14.707+0000 7fd5b3fff640 1 -- 172.21.15.156:0/983602857 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5a4009450 con 0x7fd5b4100ce0 2024-07-16T09:47:15.447 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:14.707+0000 7fd5b3fff640 1 --2- 172.21.15.156:0/983602857 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd5b4100ce0 0x7fd5b4075b00 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fd5a4009810 tx=0x7fd5a402c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:47:15.447 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:14.707+0000 7fd5b17fa640 1 -- 172.21.15.156:0/983602857 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd5a40328b0 con 0x7fd5b4100ce0 2024-07-16T09:47:15.447 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:14.707+0000 7fd5b17fa640 1 -- 172.21.15.156:0/983602857 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd5a403b910 con 0x7fd5b4100ce0 2024-07-16T09:47:15.447 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:14.707+0000 7fd5ba64c640 1 -- 172.21.15.156:0/983602857 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5b4113980 con 0x7fd5b4100ce0 2024-07-16T09:47:15.447 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:14.707+0000 7fd5b17fa640 1 -- 172.21.15.156:0/983602857 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd5a403a980 con 0x7fd5b4100ce0 2024-07-16T09:47:15.447 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:14.707+0000 7fd5ba64c640 1 -- 172.21.15.156:0/983602857 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5b4113d90 con 0x7fd5b4100ce0 2024-07-16T09:47:15.447 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:14.708+0000 7fd5b17fa640 1 -- 172.21.15.156:0/983602857 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 8) v1 ==== 42636+0+0 (secure 0 0 0) 0x7fd5a4039070 con 0x7fd5b4100ce0 2024-07-16T09:47:15.447 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:14.708+0000 7fd5b17fa640 1 --2- 172.21.15.156:0/983602857 >> [v2:172.21.15.156:6800/1023000918,v1:172.21.15.156:6801/1023000918] conn(0x7fd5a00357d0 0x7fd5a0037c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:47:15.447 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:14.708+0000 7fd5b17fa640 1 -- 172.21.15.156:0/983602857 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7fd5a4068070 con 0x7fd5b4100ce0 2024-07-16T09:47:15.447 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:14.708+0000 7fd5abfff640 1 --2- 172.21.15.156:0/983602857 >> [v2:172.21.15.156:6800/1023000918,v1:172.21.15.156:6801/1023000918] conn(0x7fd5a00357d0 0x7fd5a0037c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:47:15.447 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:14.708+0000 7fd5ba64c640 1 -- 172.21.15.156:0/983602857 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5b404b530 con 0x7fd5b4100ce0 2024-07-16T09:47:15.448 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:14.709+0000 7fd5abfff640 1 --2- 172.21.15.156:0/983602857 >> [v2:172.21.15.156:6800/1023000918,v1:172.21.15.156:6801/1023000918] conn(0x7fd5a00357d0 0x7fd5a0037c80 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fd59c006fd0 tx=0x7fd59c006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:47:15.448 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:14.713+0000 7fd5b17fa640 1 -- 172.21.15.156:0/983602857 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fd5a4037070 con 0x7fd5b4100ce0 2024-07-16T09:47:15.448 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:14.878+0000 7fd5ba64c640 1 -- 172.21.15.156:0/983602857 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) v1 -- 0x7fd5b4102b10 con 0x7fd5b4100ce0 2024-07-16T09:47:15.448 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:14.881+0000 7fd5b17fa640 1 -- 172.21.15.156:0/983602857 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) v1 ==== 142+0+0 (secure 0 0 0) 0x7fd5a4030340 con 0x7fd5b4100ce0 2024-07-16T09:47:15.448 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:14.883+0000 7fd5ba64c640 1 -- 172.21.15.156:0/983602857 >> [v2:172.21.15.156:6800/1023000918,v1:172.21.15.156:6801/1023000918] conn(0x7fd5a00357d0 msgr2=0x7fd5a0037c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:47:15.448 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:14.883+0000 7fd5ba64c640 1 --2- 172.21.15.156:0/983602857 >> [v2:172.21.15.156:6800/1023000918,v1:172.21.15.156:6801/1023000918] conn(0x7fd5a00357d0 0x7fd5a0037c80 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fd59c006fd0 tx=0x7fd59c006db0 comp rx=0 tx=0).stop 2024-07-16T09:47:15.448 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:14.883+0000 7fd5ba64c640 1 -- 172.21.15.156:0/983602857 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd5b4100ce0 msgr2=0x7fd5b4075b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:47:15.448 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:14.883+0000 7fd5ba64c640 1 --2- 172.21.15.156:0/983602857 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd5b4100ce0 0x7fd5b4075b00 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fd5a4009810 tx=0x7fd5a402c040 comp rx=0 tx=0).stop 2024-07-16T09:47:15.448 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:14.884+0000 7fd5ba64c640 1 -- 172.21.15.156:0/983602857 shutdown_connections 2024-07-16T09:47:15.448 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:14.884+0000 7fd5ba64c640 1 --2- 172.21.15.156:0/983602857 >> [v2:172.21.15.156:6800/1023000918,v1:172.21.15.156:6801/1023000918] conn(0x7fd5a00357d0 0x7fd5a0037c80 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:47:15.448 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:14.884+0000 7fd5ba64c640 1 --2- 172.21.15.156:0/983602857 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd5b4100ce0 0x7fd5b4075b00 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:47:15.448 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:14.884+0000 7fd5ba64c640 1 -- 172.21.15.156:0/983602857 >> 172.21.15.156:0/983602857 conn(0x7fd5b40fc520 msgr2=0x7fd5b40fcff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:47:15.448 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:14.884+0000 7fd5ba64c640 1 -- 172.21.15.156:0/983602857 shutdown_connections 2024-07-16T09:47:15.449 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:14.884+0000 7fd5ba64c640 1 -- 172.21.15.156:0/983602857 wait complete. 2024-07-16T09:47:16.015 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:47:15 smithi156 ceph-mon[29302]: from='client.? 172.21.15.156:0/983602857' entity='client.admin' 2024-07-16T09:47:16.543 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:15.842+0000 7f239b51a640 1 Processor -- start 2024-07-16T09:47:16.543 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:15.842+0000 7f239b51a640 1 -- start start 2024-07-16T09:47:16.543 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:15.842+0000 7f239b51a640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2394075ba0 0x7f2394104a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:47:16.543 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:15.842+0000 7f239b51a640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2394104f70 con 0x7f2394075ba0 2024-07-16T09:47:16.544 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:15.842+0000 7f23992b6640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2394075ba0 0x7f2394104a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:47:16.544 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:15.842+0000 7f23992b6640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2394075ba0 0x7f2394104a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:44038/0 (socket says 172.21.15.156:44038) 2024-07-16T09:47:16.544 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:15.842+0000 7f23992b6640 1 -- 172.21.15.156:0/2105606445 learned_addr learned my addr 172.21.15.156:0/2105606445 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T09:47:16.544 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:15.842+0000 7f23992b6640 1 -- 172.21.15.156:0/2105606445 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23941050b0 con 0x7f2394075ba0 2024-07-16T09:47:16.544 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:15.843+0000 7f23992b6640 1 --2- 172.21.15.156:0/2105606445 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2394075ba0 0x7f2394104a30 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f2390009840 tx=0x7f23900091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a91df2bd36070d0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:47:16.544 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:15.843+0000 7f238bfff640 1 -- 172.21.15.156:0/2105606445 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2390032440 con 0x7f2394075ba0 2024-07-16T09:47:16.544 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:15.843+0000 7f238bfff640 1 -- 172.21.15.156:0/2105606445 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2390032a70 con 0x7f2394075ba0 2024-07-16T09:47:16.544 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:15.843+0000 7f238bfff640 1 -- 172.21.15.156:0/2105606445 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2390031ad0 con 0x7f2394075ba0 2024-07-16T09:47:16.544 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:15.844+0000 7f239b51a640 1 -- 172.21.15.156:0/2105606445 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2394075ba0 msgr2=0x7f2394104a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:47:16.544 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:15.844+0000 7f239b51a640 1 --2- 172.21.15.156:0/2105606445 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2394075ba0 0x7f2394104a30 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f2390009840 tx=0x7f23900091d0 comp rx=0 tx=0).stop 2024-07-16T09:47:16.544 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:15.844+0000 7f239b51a640 1 -- 172.21.15.156:0/2105606445 shutdown_connections 2024-07-16T09:47:16.544 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:15.844+0000 7f239b51a640 1 --2- 172.21.15.156:0/2105606445 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2394075ba0 0x7f2394104a30 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:47:16.544 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:15.844+0000 7f239b51a640 1 -- 172.21.15.156:0/2105606445 >> 172.21.15.156:0/2105606445 conn(0x7f23940fc520 msgr2=0x7f23940fe930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:47:16.545 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:15.844+0000 7f239b51a640 1 -- 172.21.15.156:0/2105606445 shutdown_connections 2024-07-16T09:47:16.545 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:15.844+0000 7f239b51a640 1 -- 172.21.15.156:0/2105606445 wait complete. 2024-07-16T09:47:16.545 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:15.844+0000 7f239b51a640 1 Processor -- start 2024-07-16T09:47:16.545 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:15.844+0000 7f239b51a640 1 -- start start 2024-07-16T09:47:16.545 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:15.845+0000 7f239b51a640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2394075ba0 0x7f2394110b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:47:16.545 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:15.845+0000 7f239b51a640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23941110a0 con 0x7f2394075ba0 2024-07-16T09:47:16.545 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:15.845+0000 7f23992b6640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2394075ba0 0x7f2394110b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:47:16.545 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:15.845+0000 7f23992b6640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2394075ba0 0x7f2394110b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:44050/0 (socket says 172.21.15.156:44050) 2024-07-16T09:47:16.545 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:15.845+0000 7f23992b6640 1 -- 172.21.15.156:0/2197622130 learned_addr learned my addr 172.21.15.156:0/2197622130 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T09:47:16.545 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:15.845+0000 7f23992b6640 1 -- 172.21.15.156:0/2197622130 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2390009450 con 0x7f2394075ba0 2024-07-16T09:47:16.545 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:15.845+0000 7f23992b6640 1 --2- 172.21.15.156:0/2197622130 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2394075ba0 0x7f2394110b60 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f23900317f0 tx=0x7f239002c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:47:16.545 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:15.845+0000 7f2389ffb640 1 -- 172.21.15.156:0/2197622130 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2390032900 con 0x7f2394075ba0 2024-07-16T09:47:16.545 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:15.845+0000 7f239b51a640 1 -- 172.21.15.156:0/2197622130 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23941112a0 con 0x7f2394075ba0 2024-07-16T09:47:16.546 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:15.845+0000 7f2389ffb640 1 -- 172.21.15.156:0/2197622130 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f239003b910 con 0x7f2394075ba0 2024-07-16T09:47:16.546 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:15.845+0000 7f2389ffb640 1 -- 172.21.15.156:0/2197622130 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f239003a960 con 0x7f2394075ba0 2024-07-16T09:47:16.546 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:15.846+0000 7f239b51a640 1 -- 172.21.15.156:0/2197622130 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2394111710 con 0x7f2394075ba0 2024-07-16T09:47:16.546 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:15.846+0000 7f2389ffb640 1 -- 172.21.15.156:0/2197622130 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 8) v1 ==== 42636+0+0 (secure 0 0 0) 0x7f2390039070 con 0x7f2394075ba0 2024-07-16T09:47:16.546 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:15.846+0000 7f2389ffb640 1 --2- 172.21.15.156:0/2197622130 >> [v2:172.21.15.156:6800/1023000918,v1:172.21.15.156:6801/1023000918] conn(0x7f2380035780 0x7f2380037c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:47:16.546 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:15.847+0000 7f2389ffb640 1 -- 172.21.15.156:0/2197622130 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f2390067e60 con 0x7f2394075ba0 2024-07-16T09:47:16.546 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:15.847+0000 7f239b51a640 1 -- 172.21.15.156:0/2197622130 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f239405eb80 con 0x7f2394075ba0 2024-07-16T09:47:16.546 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:15.847+0000 7f2398ab5640 1 --2- 172.21.15.156:0/2197622130 >> [v2:172.21.15.156:6800/1023000918,v1:172.21.15.156:6801/1023000918] conn(0x7f2380035780 0x7f2380037c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:47:16.546 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:15.847+0000 7f2398ab5640 1 --2- 172.21.15.156:0/2197622130 >> [v2:172.21.15.156:6800/1023000918,v1:172.21.15.156:6801/1023000918] conn(0x7f2380035780 0x7f2380037c30 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f2384006fd0 tx=0x7f2384006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:47:16.546 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:15.851+0000 7f2389ffb640 1 -- 172.21.15.156:0/2197622130 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f2390094250 con 0x7f2394075ba0 2024-07-16T09:47:16.547 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:16.014+0000 7f239b51a640 1 -- 172.21.15.156:0/2197622130 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) v1 -- 0x7f239404b530 con 0x7f2394075ba0 2024-07-16T09:47:16.547 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:16.017+0000 7f2389ffb640 1 -- 172.21.15.156:0/2197622130 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) v1 ==== 130+0+0 (secure 0 0 0) 0x7f2390037070 con 0x7f2394075ba0 2024-07-16T09:47:16.547 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:16.019+0000 7f239b51a640 1 -- 172.21.15.156:0/2197622130 >> [v2:172.21.15.156:6800/1023000918,v1:172.21.15.156:6801/1023000918] conn(0x7f2380035780 msgr2=0x7f2380037c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:47:16.547 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:16.019+0000 7f239b51a640 1 --2- 172.21.15.156:0/2197622130 >> [v2:172.21.15.156:6800/1023000918,v1:172.21.15.156:6801/1023000918] conn(0x7f2380035780 0x7f2380037c30 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f2384006fd0 tx=0x7f2384006db0 comp rx=0 tx=0).stop 2024-07-16T09:47:16.547 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:16.019+0000 7f239b51a640 1 -- 172.21.15.156:0/2197622130 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2394075ba0 msgr2=0x7f2394110b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:47:16.547 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:16.019+0000 7f239b51a640 1 --2- 172.21.15.156:0/2197622130 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2394075ba0 0x7f2394110b60 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f23900317f0 tx=0x7f239002c040 comp rx=0 tx=0).stop 2024-07-16T09:47:16.547 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:16.019+0000 7f239b51a640 1 -- 172.21.15.156:0/2197622130 shutdown_connections 2024-07-16T09:47:16.547 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:16.019+0000 7f239b51a640 1 --2- 172.21.15.156:0/2197622130 >> [v2:172.21.15.156:6800/1023000918,v1:172.21.15.156:6801/1023000918] conn(0x7f2380035780 0x7f2380037c30 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:47:16.547 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:16.019+0000 7f239b51a640 1 --2- 172.21.15.156:0/2197622130 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2394075ba0 0x7f2394110b60 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:47:16.547 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:16.019+0000 7f239b51a640 1 -- 172.21.15.156:0/2197622130 >> 172.21.15.156:0/2197622130 conn(0x7f23940fc520 msgr2=0x7f23940fd010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:47:16.547 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:16.019+0000 7f239b51a640 1 -- 172.21.15.156:0/2197622130 shutdown_connections 2024-07-16T09:47:16.547 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:16.019+0000 7f239b51a640 1 -- 172.21.15.156:0/2197622130 wait complete. 2024-07-16T09:47:16.547 INFO:teuthology.orchestra.run.smithi156.stdout:Enabling the dashboard module... 2024-07-16T09:47:17.175 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:47:17 smithi156 ceph-mon[29302]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-16T09:47:17.175 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:47:17 smithi156 ceph-mon[29302]: from='client.? 172.21.15.156:0/2197622130' entity='client.admin' 2024-07-16T09:47:18.303 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:47:18 smithi156 ceph-mon[29302]: from='client.? 172.21.15.156:0/66806697' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-07-16T09:47:18.685 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:16.955+0000 7fc7bb2b8640 1 Processor -- start 2024-07-16T09:47:18.686 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:16.955+0000 7fc7bb2b8640 1 -- start start 2024-07-16T09:47:18.686 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:16.955+0000 7fc7bb2b8640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc7b4101e00 0x7fc7b4102210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:47:18.686 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:16.955+0000 7fc7bb2b8640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7b4102750 con 0x7fc7b4101e00 2024-07-16T09:47:18.686 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:16.956+0000 7fc7b9054640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc7b4101e00 0x7fc7b4102210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:47:18.686 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:16.956+0000 7fc7b9054640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc7b4101e00 0x7fc7b4102210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:44056/0 (socket says 172.21.15.156:44056) 2024-07-16T09:47:18.686 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:16.956+0000 7fc7b9054640 1 -- 172.21.15.156:0/955595255 learned_addr learned my addr 172.21.15.156:0/955595255 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T09:47:18.686 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:16.956+0000 7fc7b9054640 1 -- 172.21.15.156:0/955595255 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7b4102890 con 0x7fc7b4101e00 2024-07-16T09:47:18.686 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:16.957+0000 7fc7b9054640 1 --2- 172.21.15.156:0/955595255 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc7b4101e00 0x7fc7b4102210 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fc7ac009840 tx=0x7fc7ac0091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f2af94da000732d server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:47:18.686 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:16.957+0000 7fc7b3fff640 1 -- 172.21.15.156:0/955595255 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc7ac032440 con 0x7fc7b4101e00 2024-07-16T09:47:18.686 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:16.957+0000 7fc7b3fff640 1 -- 172.21.15.156:0/955595255 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc7ac032a70 con 0x7fc7b4101e00 2024-07-16T09:47:18.687 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:16.957+0000 7fc7b3fff640 1 -- 172.21.15.156:0/955595255 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc7ac031ad0 con 0x7fc7b4101e00 2024-07-16T09:47:18.687 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:16.957+0000 7fc7bb2b8640 1 -- 172.21.15.156:0/955595255 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc7b4101e00 msgr2=0x7fc7b4102210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:47:18.687 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:16.957+0000 7fc7bb2b8640 1 --2- 172.21.15.156:0/955595255 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc7b4101e00 0x7fc7b4102210 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fc7ac009840 tx=0x7fc7ac0091d0 comp rx=0 tx=0).stop 2024-07-16T09:47:18.687 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:16.957+0000 7fc7bb2b8640 1 -- 172.21.15.156:0/955595255 shutdown_connections 2024-07-16T09:47:18.687 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:16.957+0000 7fc7bb2b8640 1 --2- 172.21.15.156:0/955595255 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc7b4101e00 0x7fc7b4102210 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:47:18.687 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:16.957+0000 7fc7bb2b8640 1 -- 172.21.15.156:0/955595255 >> 172.21.15.156:0/955595255 conn(0x7fc7b40fd5a0 msgr2=0x7fc7b40ff9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:47:18.687 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:16.957+0000 7fc7bb2b8640 1 -- 172.21.15.156:0/955595255 shutdown_connections 2024-07-16T09:47:18.687 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:16.957+0000 7fc7bb2b8640 1 -- 172.21.15.156:0/955595255 wait complete. 2024-07-16T09:47:18.687 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:16.958+0000 7fc7bb2b8640 1 Processor -- start 2024-07-16T09:47:18.687 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:16.958+0000 7fc7bb2b8640 1 -- start start 2024-07-16T09:47:18.687 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:16.958+0000 7fc7bb2b8640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc7b4101e00 0x7fc7b4110bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:47:18.687 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:16.958+0000 7fc7bb2b8640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7b41110f0 con 0x7fc7b4101e00 2024-07-16T09:47:18.688 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:16.958+0000 7fc7b9054640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc7b4101e00 0x7fc7b4110bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:47:18.688 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:16.958+0000 7fc7b9054640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc7b4101e00 0x7fc7b4110bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:44070/0 (socket says 172.21.15.156:44070) 2024-07-16T09:47:18.688 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:16.958+0000 7fc7b9054640 1 -- 172.21.15.156:0/66806697 learned_addr learned my addr 172.21.15.156:0/66806697 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T09:47:18.688 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:16.958+0000 7fc7b9054640 1 -- 172.21.15.156:0/66806697 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7ac009450 con 0x7fc7b4101e00 2024-07-16T09:47:18.688 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:16.959+0000 7fc7b9054640 1 --2- 172.21.15.156:0/66806697 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc7b4101e00 0x7fc7b4110bb0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fc7ac031710 tx=0x7fc7ac02c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:47:18.688 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:16.959+0000 7fc7b1ffb640 1 -- 172.21.15.156:0/66806697 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc7ac032900 con 0x7fc7b4101e00 2024-07-16T09:47:18.688 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:16.959+0000 7fc7b1ffb640 1 -- 172.21.15.156:0/66806697 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc7ac03b910 con 0x7fc7b4101e00 2024-07-16T09:47:18.688 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:16.959+0000 7fc7bb2b8640 1 -- 172.21.15.156:0/66806697 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7b41112f0 con 0x7fc7b4101e00 2024-07-16T09:47:18.688 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:16.959+0000 7fc7b1ffb640 1 -- 172.21.15.156:0/66806697 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc7ac03a990 con 0x7fc7b4101e00 2024-07-16T09:47:18.688 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:16.959+0000 7fc7bb2b8640 1 -- 172.21.15.156:0/66806697 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7b4111760 con 0x7fc7b4101e00 2024-07-16T09:47:18.688 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:16.960+0000 7fc7b1ffb640 1 -- 172.21.15.156:0/66806697 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 8) v1 ==== 42636+0+0 (secure 0 0 0) 0x7fc7ac039070 con 0x7fc7b4101e00 2024-07-16T09:47:18.688 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:16.960+0000 7fc7b1ffb640 1 --2- 172.21.15.156:0/66806697 >> [v2:172.21.15.156:6800/1023000918,v1:172.21.15.156:6801/1023000918] conn(0x7fc79c0357d0 0x7fc79c037c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:47:18.688 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:16.960+0000 7fc7b1ffb640 1 -- 172.21.15.156:0/66806697 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7fc7ac067e80 con 0x7fc7b4101e00 2024-07-16T09:47:18.689 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:16.960+0000 7fc7b8853640 1 --2- 172.21.15.156:0/66806697 >> [v2:172.21.15.156:6800/1023000918,v1:172.21.15.156:6801/1023000918] conn(0x7fc79c0357d0 0x7fc79c037c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:47:18.689 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:16.960+0000 7fc7bb2b8640 1 -- 172.21.15.156:0/66806697 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7b405eb80 con 0x7fc7b4101e00 2024-07-16T09:47:18.689 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:16.960+0000 7fc7b8853640 1 --2- 172.21.15.156:0/66806697 >> [v2:172.21.15.156:6800/1023000918,v1:172.21.15.156:6801/1023000918] conn(0x7fc79c0357d0 0x7fc79c037c80 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fc7a8006fd0 tx=0x7fc7a8006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:47:18.689 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:16.965+0000 7fc7b1ffb640 1 -- 172.21.15.156:0/66806697 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fc7ac03f070 con 0x7fc7b4101e00 2024-07-16T09:47:18.689 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:17.175+0000 7fc7bb2b8640 1 -- 172.21.15.156:0/66806697 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) v1 -- 0x7fc7b404b530 con 0x7fc7b4101e00 2024-07-16T09:47:18.689 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:18.021+0000 7fc7b1ffb640 1 -- 172.21.15.156:0/66806697 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mgrmap(e 9) v1 ==== 42649+0+0 (secure 0 0 0) 0x7fc7ac03ad70 con 0x7fc7b4101e00 2024-07-16T09:47:18.689 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:18.023+0000 7fc7b1ffb640 1 -- 172.21.15.156:0/66806697 <== mon.0 v2:172.21.15.156:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) v1 ==== 88+0+0 (secure 0 0 0) 0x7fc7ac03a3a0 con 0x7fc7b4101e00 2024-07-16T09:47:18.689 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:18.025+0000 7fc7bb2b8640 1 -- 172.21.15.156:0/66806697 >> [v2:172.21.15.156:6800/1023000918,v1:172.21.15.156:6801/1023000918] conn(0x7fc79c0357d0 msgr2=0x7fc79c037c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:47:18.689 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:18.025+0000 7fc7bb2b8640 1 --2- 172.21.15.156:0/66806697 >> [v2:172.21.15.156:6800/1023000918,v1:172.21.15.156:6801/1023000918] conn(0x7fc79c0357d0 0x7fc79c037c80 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fc7a8006fd0 tx=0x7fc7a8006db0 comp rx=0 tx=0).stop 2024-07-16T09:47:18.689 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:18.025+0000 7fc7bb2b8640 1 -- 172.21.15.156:0/66806697 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc7b4101e00 msgr2=0x7fc7b4110bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:47:18.690 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:18.025+0000 7fc7bb2b8640 1 --2- 172.21.15.156:0/66806697 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc7b4101e00 0x7fc7b4110bb0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fc7ac031710 tx=0x7fc7ac02c040 comp rx=0 tx=0).stop 2024-07-16T09:47:18.690 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:18.026+0000 7fc7bb2b8640 1 -- 172.21.15.156:0/66806697 shutdown_connections 2024-07-16T09:47:18.690 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:18.026+0000 7fc7bb2b8640 1 --2- 172.21.15.156:0/66806697 >> [v2:172.21.15.156:6800/1023000918,v1:172.21.15.156:6801/1023000918] conn(0x7fc79c0357d0 0x7fc79c037c80 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:47:18.690 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:18.026+0000 7fc7bb2b8640 1 --2- 172.21.15.156:0/66806697 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc7b4101e00 0x7fc7b4110bb0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:47:18.690 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:18.026+0000 7fc7bb2b8640 1 -- 172.21.15.156:0/66806697 >> 172.21.15.156:0/66806697 conn(0x7fc7b40fd5a0 msgr2=0x7fc7b40fe100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:47:18.690 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:18.026+0000 7fc7bb2b8640 1 -- 172.21.15.156:0/66806697 shutdown_connections 2024-07-16T09:47:18.690 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:18.026+0000 7fc7bb2b8640 1 -- 172.21.15.156:0/66806697 wait complete. 2024-07-16T09:47:19.313 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:47:19 smithi156 ceph-mon[29302]: from='client.? 172.21.15.156:0/66806697' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-07-16T09:47:19.313 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:47:19 smithi156 ceph-mon[29302]: mgrmap e9: smithi156.zkvylv(active, since 24s) 2024-07-16T09:47:19.942 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout { 2024-07-16T09:47:19.942 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-07-16T09:47:19.942 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout "available": true, 2024-07-16T09:47:19.942 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout "active_name": "smithi156.zkvylv", 2024-07-16T09:47:19.942 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-07-16T09:47:19.942 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout } 2024-07-16T09:47:19.942 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:19.105+0000 7f216eeb0640 1 Processor -- start 2024-07-16T09:47:19.943 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:19.105+0000 7f216eeb0640 1 -- start start 2024-07-16T09:47:19.943 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:19.105+0000 7f216eeb0640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2168075bc0 0x7f2168104a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:47:19.943 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:19.105+0000 7f216eeb0640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2168104f50 con 0x7f2168075bc0 2024-07-16T09:47:19.943 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:19.105+0000 7f216cc4c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2168075bc0 0x7f2168104a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:47:19.943 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:19.105+0000 7f216cc4c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2168075bc0 0x7f2168104a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:44084/0 (socket says 172.21.15.156:44084) 2024-07-16T09:47:19.943 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:19.105+0000 7f216cc4c640 1 -- 172.21.15.156:0/2176961727 learned_addr learned my addr 172.21.15.156:0/2176961727 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T09:47:19.943 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:19.106+0000 7f216cc4c640 1 -- 172.21.15.156:0/2176961727 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2168105090 con 0x7f2168075bc0 2024-07-16T09:47:19.943 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:19.106+0000 7f216cc4c640 1 --2- 172.21.15.156:0/2176961727 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2168075bc0 0x7f2168104a10 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f215c009840 tx=0x7f215c0091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8a87385f3d1b0ed server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:47:19.943 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:19.106+0000 7f21677fe640 1 -- 172.21.15.156:0/2176961727 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f215c032440 con 0x7f2168075bc0 2024-07-16T09:47:19.943 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:19.106+0000 7f21677fe640 1 -- 172.21.15.156:0/2176961727 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f215c032a70 con 0x7f2168075bc0 2024-07-16T09:47:19.943 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:19.106+0000 7f21677fe640 1 -- 172.21.15.156:0/2176961727 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f215c031ad0 con 0x7f2168075bc0 2024-07-16T09:47:19.943 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:19.107+0000 7f216eeb0640 1 -- 172.21.15.156:0/2176961727 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2168075bc0 msgr2=0x7f2168104a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:47:19.944 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:19.107+0000 7f216eeb0640 1 --2- 172.21.15.156:0/2176961727 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2168075bc0 0x7f2168104a10 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f215c009840 tx=0x7f215c0091d0 comp rx=0 tx=0).stop 2024-07-16T09:47:19.944 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:19.107+0000 7f216eeb0640 1 -- 172.21.15.156:0/2176961727 shutdown_connections 2024-07-16T09:47:19.944 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:19.107+0000 7f216eeb0640 1 --2- 172.21.15.156:0/2176961727 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2168075bc0 0x7f2168104a10 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:47:19.944 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:19.107+0000 7f216eeb0640 1 -- 172.21.15.156:0/2176961727 >> 172.21.15.156:0/2176961727 conn(0x7f21680fc520 msgr2=0x7f21680fe930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:47:19.944 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:19.107+0000 7f216eeb0640 1 -- 172.21.15.156:0/2176961727 shutdown_connections 2024-07-16T09:47:19.944 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:19.107+0000 7f216eeb0640 1 -- 172.21.15.156:0/2176961727 wait complete. 2024-07-16T09:47:19.944 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:19.108+0000 7f216eeb0640 1 Processor -- start 2024-07-16T09:47:19.944 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:19.108+0000 7f216eeb0640 1 -- start start 2024-07-16T09:47:19.944 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:19.108+0000 7f216eeb0640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2168075bc0 0x7f2168110b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:47:19.944 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:19.108+0000 7f216eeb0640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2168111080 con 0x7f2168075bc0 2024-07-16T09:47:19.944 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:19.108+0000 7f216cc4c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2168075bc0 0x7f2168110b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:47:19.944 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:19.108+0000 7f216cc4c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2168075bc0 0x7f2168110b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:44096/0 (socket says 172.21.15.156:44096) 2024-07-16T09:47:19.945 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:19.108+0000 7f216cc4c640 1 -- 172.21.15.156:0/105680878 learned_addr learned my addr 172.21.15.156:0/105680878 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T09:47:19.945 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:19.108+0000 7f216cc4c640 1 -- 172.21.15.156:0/105680878 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f215c009450 con 0x7f2168075bc0 2024-07-16T09:47:19.945 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:19.109+0000 7f216cc4c640 1 --2- 172.21.15.156:0/105680878 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2168075bc0 0x7f2168110b40 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f215c0317f0 tx=0x7f215c02c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:47:19.945 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:19.109+0000 7f21657fa640 1 -- 172.21.15.156:0/105680878 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f215c032900 con 0x7f2168075bc0 2024-07-16T09:47:19.945 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:19.109+0000 7f21657fa640 1 -- 172.21.15.156:0/105680878 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f215c03b910 con 0x7f2168075bc0 2024-07-16T09:47:19.945 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:19.109+0000 7f216eeb0640 1 -- 172.21.15.156:0/105680878 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2168111280 con 0x7f2168075bc0 2024-07-16T09:47:19.945 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:19.109+0000 7f21657fa640 1 -- 172.21.15.156:0/105680878 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f215c03a960 con 0x7f2168075bc0 2024-07-16T09:47:19.945 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:19.109+0000 7f216eeb0640 1 -- 172.21.15.156:0/105680878 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21681116f0 con 0x7f2168075bc0 2024-07-16T09:47:19.945 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:19.110+0000 7f21657fa640 1 -- 172.21.15.156:0/105680878 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 9) v1 ==== 42649+0+0 (secure 0 0 0) 0x7f215c039070 con 0x7f2168075bc0 2024-07-16T09:47:19.945 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:19.110+0000 7f216eeb0640 1 -- 172.21.15.156:0/105680878 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f216804b530 con 0x7f2168075bc0 2024-07-16T09:47:19.945 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:19.110+0000 7f21657fa640 1 --2- 172.21.15.156:0/105680878 >> [v2:172.21.15.156:6800/1023000918,v1:172.21.15.156:6801/1023000918] conn(0x7f21580357d0 0x7f2158037c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:47:19.945 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:19.110+0000 7f21657fa640 1 -- 172.21.15.156:0/105680878 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f215c067e40 con 0x7f2168075bc0 2024-07-16T09:47:19.946 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:19.110+0000 7f2167fff640 1 -- 172.21.15.156:0/105680878 >> [v2:172.21.15.156:6800/1023000918,v1:172.21.15.156:6801/1023000918] conn(0x7f21580357d0 msgr2=0x7f2158037c80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.156:6800/1023000918 2024-07-16T09:47:19.946 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:19.110+0000 7f2167fff640 1 --2- 172.21.15.156:0/105680878 >> [v2:172.21.15.156:6800/1023000918,v1:172.21.15.156:6801/1023000918] conn(0x7f21580357d0 0x7f2158037c80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-16T09:47:19.946 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:19.114+0000 7f21657fa640 1 -- 172.21.15.156:0/105680878 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f215c039500 con 0x7f2168075bc0 2024-07-16T09:47:19.946 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:19.311+0000 7f2167fff640 1 -- 172.21.15.156:0/105680878 >> [v2:172.21.15.156:6800/1023000918,v1:172.21.15.156:6801/1023000918] conn(0x7f21580357d0 msgr2=0x7f2158037c80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.156:6800/1023000918 2024-07-16T09:47:19.946 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:19.311+0000 7f2167fff640 1 --2- 172.21.15.156:0/105680878 >> [v2:172.21.15.156:6800/1023000918,v1:172.21.15.156:6801/1023000918] conn(0x7f21580357d0 0x7f2158037c80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-16T09:47:19.946 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:19.330+0000 7f216eeb0640 1 -- 172.21.15.156:0/105680878 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7f2168105e20 con 0x7f2168075bc0 2024-07-16T09:47:19.946 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:19.331+0000 7f21657fa640 1 -- 172.21.15.156:0/105680878 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) v1 ==== 56+0+103 (secure 0 0 0) 0x7f215c03f020 con 0x7f2168075bc0 2024-07-16T09:47:19.946 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:19.332+0000 7f216eeb0640 1 -- 172.21.15.156:0/105680878 >> [v2:172.21.15.156:6800/1023000918,v1:172.21.15.156:6801/1023000918] conn(0x7f21580357d0 msgr2=0x7f2158037c80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-16T09:47:19.946 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:19.332+0000 7f216eeb0640 1 --2- 172.21.15.156:0/105680878 >> [v2:172.21.15.156:6800/1023000918,v1:172.21.15.156:6801/1023000918] conn(0x7f21580357d0 0x7f2158037c80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:47:19.946 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:19.332+0000 7f216eeb0640 1 -- 172.21.15.156:0/105680878 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2168075bc0 msgr2=0x7f2168110b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:47:19.946 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:19.332+0000 7f216eeb0640 1 --2- 172.21.15.156:0/105680878 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2168075bc0 0x7f2168110b40 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f215c0317f0 tx=0x7f215c02c040 comp rx=0 tx=0).stop 2024-07-16T09:47:19.946 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:19.333+0000 7f216eeb0640 1 -- 172.21.15.156:0/105680878 shutdown_connections 2024-07-16T09:47:19.947 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:19.333+0000 7f216eeb0640 1 --2- 172.21.15.156:0/105680878 >> [v2:172.21.15.156:6800/1023000918,v1:172.21.15.156:6801/1023000918] conn(0x7f21580357d0 0x7f2158037c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:47:19.947 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:19.333+0000 7f216eeb0640 1 --2- 172.21.15.156:0/105680878 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2168075bc0 0x7f2168110b40 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:47:19.947 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:19.333+0000 7f216eeb0640 1 -- 172.21.15.156:0/105680878 >> 172.21.15.156:0/105680878 conn(0x7f21680fc520 msgr2=0x7f21680fd070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:47:19.947 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:19.333+0000 7f216eeb0640 1 -- 172.21.15.156:0/105680878 shutdown_connections 2024-07-16T09:47:19.947 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:19.333+0000 7f216eeb0640 1 -- 172.21.15.156:0/105680878 wait complete. 2024-07-16T09:47:19.947 INFO:teuthology.orchestra.run.smithi156.stdout:Waiting for the mgr to restart... 2024-07-16T09:47:19.947 INFO:teuthology.orchestra.run.smithi156.stdout:Waiting for mgr epoch 9... 2024-07-16T09:47:20.260 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:47:20 smithi156 ceph-mon[29302]: from='client.? 172.21.15.156:0/105680878' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-07-16T09:47:22.721 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:47:22 smithi156 ceph-mon[29302]: Active manager daemon smithi156.zkvylv restarted 2024-07-16T09:47:22.721 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:47:22 smithi156 ceph-mon[29302]: Activating manager daemon smithi156.zkvylv 2024-07-16T09:47:22.721 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:47:22 smithi156 ceph-mon[29302]: osdmap e3: 0 total, 0 up, 0 in 2024-07-16T09:47:22.721 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:47:22 smithi156 ceph-mon[29302]: mgrmap e10: smithi156.zkvylv(active, starting, since 0.00325347s) 2024-07-16T09:47:22.721 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:47:22 smithi156 ceph-mon[29302]: from='mgr.14160 172.21.15.156:0/3789331119' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "mon metadata", "id": "smithi156"}]: dispatch 2024-07-16T09:47:22.721 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:47:22 smithi156 ceph-mon[29302]: from='mgr.14160 172.21.15.156:0/3789331119' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "mgr metadata", "who": "smithi156.zkvylv", "id": "smithi156.zkvylv"}]: dispatch 2024-07-16T09:47:22.721 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:47:22 smithi156 ceph-mon[29302]: from='mgr.14160 172.21.15.156:0/3789331119' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-07-16T09:47:22.721 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:47:22 smithi156 ceph-mon[29302]: from='mgr.14160 172.21.15.156:0/3789331119' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-07-16T09:47:22.722 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:47:22 smithi156 ceph-mon[29302]: from='mgr.14160 172.21.15.156:0/3789331119' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-07-16T09:47:22.722 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:47:22 smithi156 ceph-mon[29302]: Manager daemon smithi156.zkvylv is now available 2024-07-16T09:47:22.722 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:47:22 smithi156 ceph-mon[29302]: from='mgr.14160 172.21.15.156:0/3789331119' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-16T09:47:22.722 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:47:22 smithi156 ceph-mon[29302]: from='mgr.14160 172.21.15.156:0/3789331119' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-16T09:47:22.722 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:47:22 smithi156 ceph-mon[29302]: from='mgr.14160 172.21.15.156:0/3789331119' entity='mgr.smithi156.zkvylv' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi156.zkvylv/mirror_snapshot_schedule"}]: dispatch 2024-07-16T09:47:22.722 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:47:22 smithi156 ceph-mon[29302]: from='mgr.14160 172.21.15.156:0/3789331119' entity='mgr.smithi156.zkvylv' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi156.zkvylv/trash_purge_schedule"}]: dispatch 2024-07-16T09:47:24.218 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout { 2024-07-16T09:47:24.218 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-07-16T09:47:24.219 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout "initialized": true 2024-07-16T09:47:24.219 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout } 2024-07-16T09:47:24.219 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:20.335+0000 7fd705f33640 1 Processor -- start 2024-07-16T09:47:24.219 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:20.335+0000 7fd705f33640 1 -- start start 2024-07-16T09:47:24.219 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:20.335+0000 7fd705f33640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd700075b80 0x7fd700104a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:47:24.219 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:20.335+0000 7fd705f33640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd700075f90 con 0x7fd700075b80 2024-07-16T09:47:24.219 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:20.336+0000 7fd6ff7fe640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd700075b80 0x7fd700104a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:47:24.219 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:20.336+0000 7fd6ff7fe640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd700075b80 0x7fd700104a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:44102/0 (socket says 172.21.15.156:44102) 2024-07-16T09:47:24.219 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:20.336+0000 7fd6ff7fe640 1 -- 172.21.15.156:0/1893348800 learned_addr learned my addr 172.21.15.156:0/1893348800 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T09:47:24.219 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:20.336+0000 7fd6ff7fe640 1 -- 172.21.15.156:0/1893348800 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd700104f90 con 0x7fd700075b80 2024-07-16T09:47:24.219 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:20.337+0000 7fd6ff7fe640 1 --2- 172.21.15.156:0/1893348800 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd700075b80 0x7fd700104a50 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fd6f4009840 tx=0x7fd6f40091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64d55bde53b0d7d8 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:47:24.220 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:20.337+0000 7fd6fe7fc640 1 -- 172.21.15.156:0/1893348800 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd6f4032440 con 0x7fd700075b80 2024-07-16T09:47:24.220 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:20.337+0000 7fd6fe7fc640 1 -- 172.21.15.156:0/1893348800 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd6f4032a70 con 0x7fd700075b80 2024-07-16T09:47:24.220 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:20.337+0000 7fd6fe7fc640 1 -- 172.21.15.156:0/1893348800 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd6f4031ad0 con 0x7fd700075b80 2024-07-16T09:47:24.220 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:20.337+0000 7fd705f33640 1 -- 172.21.15.156:0/1893348800 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd700075b80 msgr2=0x7fd700104a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:47:24.220 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:20.337+0000 7fd705f33640 1 --2- 172.21.15.156:0/1893348800 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd700075b80 0x7fd700104a50 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fd6f4009840 tx=0x7fd6f40091d0 comp rx=0 tx=0).stop 2024-07-16T09:47:24.220 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:20.337+0000 7fd705f33640 1 -- 172.21.15.156:0/1893348800 shutdown_connections 2024-07-16T09:47:24.220 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:20.337+0000 7fd705f33640 1 --2- 172.21.15.156:0/1893348800 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd700075b80 0x7fd700104a50 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:47:24.220 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:20.337+0000 7fd705f33640 1 -- 172.21.15.156:0/1893348800 >> 172.21.15.156:0/1893348800 conn(0x7fd7000fc4c0 msgr2=0x7fd7000fe8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:47:24.220 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:20.338+0000 7fd705f33640 1 -- 172.21.15.156:0/1893348800 shutdown_connections 2024-07-16T09:47:24.220 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:20.338+0000 7fd705f33640 1 -- 172.21.15.156:0/1893348800 wait complete. 2024-07-16T09:47:24.220 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:20.338+0000 7fd705f33640 1 Processor -- start 2024-07-16T09:47:24.220 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:20.338+0000 7fd705f33640 1 -- start start 2024-07-16T09:47:24.220 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:20.338+0000 7fd705f33640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd700075b80 0x7fd700102380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:47:24.221 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:20.338+0000 7fd705f33640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7001028c0 con 0x7fd700075b80 2024-07-16T09:47:24.221 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:20.338+0000 7fd6ff7fe640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd700075b80 0x7fd700102380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:47:24.221 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:20.339+0000 7fd6ff7fe640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd700075b80 0x7fd700102380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:44112/0 (socket says 172.21.15.156:44112) 2024-07-16T09:47:24.221 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:20.339+0000 7fd6ff7fe640 1 -- 172.21.15.156:0/1540818861 learned_addr learned my addr 172.21.15.156:0/1540818861 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T09:47:24.221 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:20.339+0000 7fd6ff7fe640 1 -- 172.21.15.156:0/1540818861 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6f4009450 con 0x7fd700075b80 2024-07-16T09:47:24.221 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:20.339+0000 7fd6ff7fe640 1 --2- 172.21.15.156:0/1540818861 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd700075b80 0x7fd700102380 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fd6f40317f0 tx=0x7fd6f402c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:47:24.221 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:20.339+0000 7fd6e7fff640 1 -- 172.21.15.156:0/1540818861 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd6f4032900 con 0x7fd700075b80 2024-07-16T09:47:24.221 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:20.339+0000 7fd6e7fff640 1 -- 172.21.15.156:0/1540818861 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd6f403b910 con 0x7fd700075b80 2024-07-16T09:47:24.221 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:20.339+0000 7fd705f33640 1 -- 172.21.15.156:0/1540818861 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd700100950 con 0x7fd700075b80 2024-07-16T09:47:24.221 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:20.339+0000 7fd6e7fff640 1 -- 172.21.15.156:0/1540818861 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd6f403a960 con 0x7fd700075b80 2024-07-16T09:47:24.221 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:20.339+0000 7fd705f33640 1 -- 172.21.15.156:0/1540818861 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd700100dc0 con 0x7fd700075b80 2024-07-16T09:47:24.221 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:20.340+0000 7fd6e7fff640 1 -- 172.21.15.156:0/1540818861 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 9) v1 ==== 42649+0+0 (secure 0 0 0) 0x7fd6f4039070 con 0x7fd700075b80 2024-07-16T09:47:24.222 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:20.340+0000 7fd6e7fff640 1 --2- 172.21.15.156:0/1540818861 >> [v2:172.21.15.156:6800/1023000918,v1:172.21.15.156:6801/1023000918] conn(0x7fd6e80357d0 0x7fd6e8037c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:47:24.222 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:20.340+0000 7fd705f33640 1 -- 172.21.15.156:0/1540818861 --> [v2:172.21.15.156:6800/1023000918,v1:172.21.15.156:6801/1023000918] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd70004b530 con 0x7fd6e80357d0 2024-07-16T09:47:24.222 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:20.340+0000 7fd6e7fff640 1 -- 172.21.15.156:0/1540818861 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7fd6f4068130 con 0x7fd700075b80 2024-07-16T09:47:24.222 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:20.340+0000 7fd6feffd640 1 -- 172.21.15.156:0/1540818861 >> [v2:172.21.15.156:6800/1023000918,v1:172.21.15.156:6801/1023000918] conn(0x7fd6e80357d0 msgr2=0x7fd6e8037c80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.156:6800/1023000918 2024-07-16T09:47:24.222 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:20.340+0000 7fd6feffd640 1 --2- 172.21.15.156:0/1540818861 >> [v2:172.21.15.156:6800/1023000918,v1:172.21.15.156:6801/1023000918] conn(0x7fd6e80357d0 0x7fd6e8037c80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-16T09:47:24.222 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:20.541+0000 7fd6feffd640 1 -- 172.21.15.156:0/1540818861 >> [v2:172.21.15.156:6800/1023000918,v1:172.21.15.156:6801/1023000918] conn(0x7fd6e80357d0 msgr2=0x7fd6e8037c80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.156:6800/1023000918 2024-07-16T09:47:24.222 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:20.541+0000 7fd6feffd640 1 --2- 172.21.15.156:0/1540818861 >> [v2:172.21.15.156:6800/1023000918,v1:172.21.15.156:6801/1023000918] conn(0x7fd6e80357d0 0x7fd6e8037c80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-16T09:47:24.222 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:20.942+0000 7fd6feffd640 1 -- 172.21.15.156:0/1540818861 >> [v2:172.21.15.156:6800/1023000918,v1:172.21.15.156:6801/1023000918] conn(0x7fd6e80357d0 msgr2=0x7fd6e8037c80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.156:6800/1023000918 2024-07-16T09:47:24.222 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:20.942+0000 7fd6feffd640 1 --2- 172.21.15.156:0/1540818861 >> [v2:172.21.15.156:6800/1023000918,v1:172.21.15.156:6801/1023000918] conn(0x7fd6e80357d0 0x7fd6e8037c80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-07-16T09:47:24.222 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:21.743+0000 7fd6feffd640 1 -- 172.21.15.156:0/1540818861 >> [v2:172.21.15.156:6800/1023000918,v1:172.21.15.156:6801/1023000918] conn(0x7fd6e80357d0 msgr2=0x7fd6e8037c80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.156:6800/1023000918 2024-07-16T09:47:24.222 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:21.743+0000 7fd6feffd640 1 --2- 172.21.15.156:0/1540818861 >> [v2:172.21.15.156:6800/1023000918,v1:172.21.15.156:6801/1023000918] conn(0x7fd6e80357d0 0x7fd6e8037c80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-07-16T09:47:24.223 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:22.568+0000 7fd6e7fff640 1 -- 172.21.15.156:0/1540818861 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mgrmap(e 10) v1 ==== 42416+0+0 (secure 0 0 0) 0x7fd6f4067250 con 0x7fd700075b80 2024-07-16T09:47:24.223 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:22.568+0000 7fd6e7fff640 1 -- 172.21.15.156:0/1540818861 >> [v2:172.21.15.156:6800/1023000918,v1:172.21.15.156:6801/1023000918] conn(0x7fd6e80357d0 msgr2=0x7fd6e8037c80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-16T09:47:24.223 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:22.568+0000 7fd6e7fff640 1 --2- 172.21.15.156:0/1540818861 >> [v2:172.21.15.156:6800/1023000918,v1:172.21.15.156:6801/1023000918] conn(0x7fd6e80357d0 0x7fd6e8037c80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:47:24.223 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:23.569+0000 7fd6e7fff640 1 -- 172.21.15.156:0/1540818861 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mgrmap(e 11) v1 ==== 42543+0+0 (secure 0 0 0) 0x7fd6f404c350 con 0x7fd700075b80 2024-07-16T09:47:24.223 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:23.569+0000 7fd6e7fff640 1 --2- 172.21.15.156:0/1540818861 >> [v2:172.21.15.156:6800/3984987611,v1:172.21.15.156:6801/3984987611] conn(0x7fd6e8038fe0 0x7fd6e803b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:47:24.223 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:23.569+0000 7fd6e7fff640 1 -- 172.21.15.156:0/1540818861 --> [v2:172.21.15.156:6800/3984987611,v1:172.21.15.156:6801/3984987611] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd70004b530 con 0x7fd6e8038fe0 2024-07-16T09:47:24.223 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:23.570+0000 7fd6feffd640 1 --2- 172.21.15.156:0/1540818861 >> [v2:172.21.15.156:6800/3984987611,v1:172.21.15.156:6801/3984987611] conn(0x7fd6e8038fe0 0x7fd6e803b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:47:24.223 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:23.570+0000 7fd6feffd640 1 --2- 172.21.15.156:0/1540818861 >> [v2:172.21.15.156:6800/3984987611,v1:172.21.15.156:6801/3984987611] conn(0x7fd6e8038fe0 0x7fd6e803b3c0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fd6f0003940 tx=0x7fd6f0009350 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:47:24.223 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:23.571+0000 7fd6e7fff640 1 -- 172.21.15.156:0/1540818861 <== mgr.14160 v2:172.21.15.156:6800/3984987611 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+6549 (secure 0 0 0) 0x7fd70004b530 con 0x7fd6e8038fe0 2024-07-16T09:47:24.223 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:23.576+0000 7fd705f33640 1 -- 172.21.15.156:0/1540818861 --> [v2:172.21.15.156:6800/3984987611,v1:172.21.15.156:6801/3984987611] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7fd700105de0 con 0x7fd6e8038fe0 2024-07-16T09:47:24.223 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:23.576+0000 7fd6e7fff640 1 -- 172.21.15.156:0/1540818861 <== mgr.14160 v2:172.21.15.156:6800/3984987611 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+52 (secure 0 0 0) 0x7fd700105de0 con 0x7fd6e8038fe0 2024-07-16T09:47:24.223 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:23.577+0000 7fd705f33640 1 -- 172.21.15.156:0/1540818861 >> [v2:172.21.15.156:6800/3984987611,v1:172.21.15.156:6801/3984987611] conn(0x7fd6e8038fe0 msgr2=0x7fd6e803b3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:47:24.223 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:23.577+0000 7fd705f33640 1 --2- 172.21.15.156:0/1540818861 >> [v2:172.21.15.156:6800/3984987611,v1:172.21.15.156:6801/3984987611] conn(0x7fd6e8038fe0 0x7fd6e803b3c0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fd6f0003940 tx=0x7fd6f0009350 comp rx=0 tx=0).stop 2024-07-16T09:47:24.224 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:23.577+0000 7fd705f33640 1 -- 172.21.15.156:0/1540818861 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd700075b80 msgr2=0x7fd700102380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:47:24.224 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:23.577+0000 7fd705f33640 1 --2- 172.21.15.156:0/1540818861 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd700075b80 0x7fd700102380 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fd6f40317f0 tx=0x7fd6f402c040 comp rx=0 tx=0).stop 2024-07-16T09:47:24.224 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:23.577+0000 7fd705f33640 1 -- 172.21.15.156:0/1540818861 shutdown_connections 2024-07-16T09:47:24.224 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:23.577+0000 7fd705f33640 1 --2- 172.21.15.156:0/1540818861 >> [v2:172.21.15.156:6800/3984987611,v1:172.21.15.156:6801/3984987611] conn(0x7fd6e8038fe0 0x7fd6e803b3c0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:47:24.224 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:23.577+0000 7fd705f33640 1 --2- 172.21.15.156:0/1540818861 >> [v2:172.21.15.156:6800/1023000918,v1:172.21.15.156:6801/1023000918] conn(0x7fd6e80357d0 0x7fd6e8037c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:47:24.224 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:23.577+0000 7fd705f33640 1 --2- 172.21.15.156:0/1540818861 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd700075b80 0x7fd700102380 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:47:24.224 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:23.577+0000 7fd705f33640 1 -- 172.21.15.156:0/1540818861 >> 172.21.15.156:0/1540818861 conn(0x7fd7000fc4c0 msgr2=0x7fd7000fcfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:47:24.224 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:23.577+0000 7fd705f33640 1 -- 172.21.15.156:0/1540818861 shutdown_connections 2024-07-16T09:47:24.224 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:23.577+0000 7fd705f33640 1 -- 172.21.15.156:0/1540818861 wait complete. 2024-07-16T09:47:24.224 INFO:teuthology.orchestra.run.smithi156.stdout:mgr epoch 9 is available 2024-07-16T09:47:24.224 INFO:teuthology.orchestra.run.smithi156.stdout:Generating a dashboard self-signed certificate... 2024-07-16T09:47:24.654 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:47:24 smithi156 ceph-mon[29302]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-07-16T09:47:24.654 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:47:24 smithi156 ceph-mon[29302]: mgrmap e11: smithi156.zkvylv(active, since 1.00557s) 2024-07-16T09:47:24.654 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:47:24 smithi156 ceph-mon[29302]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-07-16T09:47:24.654 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:47:24 smithi156 ceph-mon[29302]: from='mgr.14160 172.21.15.156:0/3789331119' entity='mgr.smithi156.zkvylv' 2024-07-16T09:47:25.499 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-07-16T09:47:25.499 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:24.657+0000 7f54f1398640 1 Processor -- start 2024-07-16T09:47:25.499 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:24.657+0000 7f54f1398640 1 -- start start 2024-07-16T09:47:25.499 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:24.657+0000 7f54f1398640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f54ec101310 0x7f54ec101720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:47:25.499 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:24.657+0000 7f54f1398640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54ec101c60 con 0x7f54ec101310 2024-07-16T09:47:25.499 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:24.657+0000 7f54eaffd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f54ec101310 0x7f54ec101720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:47:25.499 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:24.657+0000 7f54eaffd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f54ec101310 0x7f54ec101720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:42218/0 (socket says 172.21.15.156:42218) 2024-07-16T09:47:25.499 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:24.657+0000 7f54eaffd640 1 -- 172.21.15.156:0/2794285357 learned_addr learned my addr 172.21.15.156:0/2794285357 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T09:47:25.499 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:24.657+0000 7f54eaffd640 1 -- 172.21.15.156:0/2794285357 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54ec101da0 con 0x7f54ec101310 2024-07-16T09:47:25.499 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:24.658+0000 7f54eaffd640 1 --2- 172.21.15.156:0/2794285357 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f54ec101310 0x7f54ec101720 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f54dc009840 tx=0x7f54dc0091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8ee5342e9832bd8 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:47:25.500 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:24.658+0000 7f54e9ffb640 1 -- 172.21.15.156:0/2794285357 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f54dc032440 con 0x7f54ec101310 2024-07-16T09:47:25.500 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:24.658+0000 7f54e9ffb640 1 -- 172.21.15.156:0/2794285357 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f54dc032a70 con 0x7f54ec101310 2024-07-16T09:47:25.500 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:24.658+0000 7f54e9ffb640 1 -- 172.21.15.156:0/2794285357 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f54dc031ad0 con 0x7f54ec101310 2024-07-16T09:47:25.500 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:24.659+0000 7f54f1398640 1 -- 172.21.15.156:0/2794285357 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f54ec101310 msgr2=0x7f54ec101720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:47:25.500 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:24.659+0000 7f54f1398640 1 --2- 172.21.15.156:0/2794285357 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f54ec101310 0x7f54ec101720 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f54dc009840 tx=0x7f54dc0091d0 comp rx=0 tx=0).stop 2024-07-16T09:47:25.500 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:24.659+0000 7f54f1398640 1 -- 172.21.15.156:0/2794285357 shutdown_connections 2024-07-16T09:47:25.500 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:24.659+0000 7f54f1398640 1 --2- 172.21.15.156:0/2794285357 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f54ec101310 0x7f54ec101720 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:47:25.500 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:24.659+0000 7f54f1398640 1 -- 172.21.15.156:0/2794285357 >> 172.21.15.156:0/2794285357 conn(0x7f54ec0fbcc0 msgr2=0x7f54ec0fe0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:47:25.500 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:24.659+0000 7f54f1398640 1 -- 172.21.15.156:0/2794285357 shutdown_connections 2024-07-16T09:47:25.500 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:24.659+0000 7f54f1398640 1 -- 172.21.15.156:0/2794285357 wait complete. 2024-07-16T09:47:25.500 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:24.660+0000 7f54f1398640 1 Processor -- start 2024-07-16T09:47:25.500 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:24.660+0000 7f54f1398640 1 -- start start 2024-07-16T09:47:25.500 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:24.660+0000 7f54f1398640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f54ec101310 0x7f54ec100b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:47:25.501 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:24.660+0000 7f54f1398640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54ec113860 con 0x7f54ec101310 2024-07-16T09:47:25.501 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:24.660+0000 7f54eaffd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f54ec101310 0x7f54ec100b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:47:25.501 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:24.660+0000 7f54eaffd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f54ec101310 0x7f54ec100b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:42222/0 (socket says 172.21.15.156:42222) 2024-07-16T09:47:25.501 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:24.660+0000 7f54eaffd640 1 -- 172.21.15.156:0/1596390367 learned_addr learned my addr 172.21.15.156:0/1596390367 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T09:47:25.501 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:24.660+0000 7f54eaffd640 1 -- 172.21.15.156:0/1596390367 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54dc009450 con 0x7f54ec101310 2024-07-16T09:47:25.501 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:24.661+0000 7f54eaffd640 1 --2- 172.21.15.156:0/1596390367 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f54ec101310 0x7f54ec100b10 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f54dc009810 tx=0x7f54dc02c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:47:25.501 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:24.661+0000 7f54d37fe640 1 -- 172.21.15.156:0/1596390367 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f54dc0328b0 con 0x7f54ec101310 2024-07-16T09:47:25.501 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:24.661+0000 7f54f1398640 1 -- 172.21.15.156:0/1596390367 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54ec1139a0 con 0x7f54ec101310 2024-07-16T09:47:25.501 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:24.661+0000 7f54d37fe640 1 -- 172.21.15.156:0/1596390367 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f54dc03b910 con 0x7f54ec101310 2024-07-16T09:47:25.501 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:24.661+0000 7f54d37fe640 1 -- 172.21.15.156:0/1596390367 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f54dc03a980 con 0x7f54ec101310 2024-07-16T09:47:25.501 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:24.661+0000 7f54f1398640 1 -- 172.21.15.156:0/1596390367 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54ec113db0 con 0x7f54ec101310 2024-07-16T09:47:25.501 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:24.662+0000 7f54f1398640 1 -- 172.21.15.156:0/1596390367 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54ec04b530 con 0x7f54ec101310 2024-07-16T09:47:25.502 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:24.662+0000 7f54d37fe640 1 -- 172.21.15.156:0/1596390367 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 11) v1 ==== 42543+0+0 (secure 0 0 0) 0x7f54dc039070 con 0x7f54ec101310 2024-07-16T09:47:25.502 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:24.662+0000 7f54d37fe640 1 --2- 172.21.15.156:0/1596390367 >> [v2:172.21.15.156:6800/3984987611,v1:172.21.15.156:6801/3984987611] conn(0x7f54d40356a0 0x7f54d4037b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:47:25.502 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:24.662+0000 7f54d37fe640 1 -- 172.21.15.156:0/1596390367 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f54dc068300 con 0x7f54ec101310 2024-07-16T09:47:25.502 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:24.663+0000 7f54ea7fc640 1 --2- 172.21.15.156:0/1596390367 >> [v2:172.21.15.156:6800/3984987611,v1:172.21.15.156:6801/3984987611] conn(0x7f54d40356a0 0x7f54d4037b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:47:25.502 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:24.663+0000 7f54ea7fc640 1 --2- 172.21.15.156:0/1596390367 >> [v2:172.21.15.156:6800/3984987611,v1:172.21.15.156:6801/3984987611] conn(0x7f54d40356a0 0x7f54d4037b50 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f54e0006fd0 tx=0x7f54e0006db0 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:47:25.502 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:24.666+0000 7f54d37fe640 1 -- 172.21.15.156:0/1596390367 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f54dc04c9a0 con 0x7f54ec101310 2024-07-16T09:47:25.502 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:24.856+0000 7f54f1398640 1 -- 172.21.15.156:0/1596390367 --> [v2:172.21.15.156:6800/3984987611,v1:172.21.15.156:6801/3984987611] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) v1 -- 0x7f54ec1140b0 con 0x7f54d40356a0 2024-07-16T09:47:25.502 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:24.899+0000 7f54d37fe640 1 -- 172.21.15.156:0/1596390367 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mgrmap(e 12) v1 ==== 42649+0+0 (secure 0 0 0) 0x7f54dc04ed10 con 0x7f54ec101310 2024-07-16T09:47:25.502 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:24.934+0000 7f54d37fe640 1 -- 172.21.15.156:0/1596390367 <== mgr.14160 v2:172.21.15.156:6800/3984987611 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+32 (secure 0 0 0) 0x7f54ec1140b0 con 0x7f54d40356a0 2024-07-16T09:47:25.502 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:24.937+0000 7f54f1398640 1 -- 172.21.15.156:0/1596390367 >> [v2:172.21.15.156:6800/3984987611,v1:172.21.15.156:6801/3984987611] conn(0x7f54d40356a0 msgr2=0x7f54d4037b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:47:25.502 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:24.937+0000 7f54f1398640 1 --2- 172.21.15.156:0/1596390367 >> [v2:172.21.15.156:6800/3984987611,v1:172.21.15.156:6801/3984987611] conn(0x7f54d40356a0 0x7f54d4037b50 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f54e0006fd0 tx=0x7f54e0006db0 comp rx=0 tx=0).stop 2024-07-16T09:47:25.502 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:24.937+0000 7f54f1398640 1 -- 172.21.15.156:0/1596390367 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f54ec101310 msgr2=0x7f54ec100b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:47:25.503 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:24.937+0000 7f54f1398640 1 --2- 172.21.15.156:0/1596390367 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f54ec101310 0x7f54ec100b10 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f54dc009810 tx=0x7f54dc02c040 comp rx=0 tx=0).stop 2024-07-16T09:47:25.503 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:24.937+0000 7f54f1398640 1 -- 172.21.15.156:0/1596390367 shutdown_connections 2024-07-16T09:47:25.503 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:24.937+0000 7f54f1398640 1 --2- 172.21.15.156:0/1596390367 >> [v2:172.21.15.156:6800/3984987611,v1:172.21.15.156:6801/3984987611] conn(0x7f54d40356a0 0x7f54d4037b50 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:47:25.503 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:24.937+0000 7f54f1398640 1 --2- 172.21.15.156:0/1596390367 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f54ec101310 0x7f54ec100b10 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:47:25.503 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:24.937+0000 7f54f1398640 1 -- 172.21.15.156:0/1596390367 >> 172.21.15.156:0/1596390367 conn(0x7f54ec0fbcc0 msgr2=0x7f54ec0fc7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:47:25.503 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:24.937+0000 7f54f1398640 1 -- 172.21.15.156:0/1596390367 shutdown_connections 2024-07-16T09:47:25.503 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:24.937+0000 7f54f1398640 1 -- 172.21.15.156:0/1596390367 wait complete. 2024-07-16T09:47:25.503 INFO:teuthology.orchestra.run.smithi156.stdout:Creating initial admin user... 2024-07-16T09:47:25.924 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:47:25 smithi156 ceph-mon[29302]: from='mgr.14160 172.21.15.156:0/3789331119' entity='mgr.smithi156.zkvylv' 2024-07-16T09:47:25.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:47:25 smithi156 ceph-mon[29302]: [16/Jul/2024:09:47:24] ENGINE Bus STARTING 2024-07-16T09:47:25.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:47:25 smithi156 ceph-mon[29302]: mgrmap e12: smithi156.zkvylv(active, since 2s) 2024-07-16T09:47:25.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:47:25 smithi156 ceph-mon[29302]: from='mgr.14160 172.21.15.156:0/3789331119' entity='mgr.smithi156.zkvylv' 2024-07-16T09:47:25.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:47:25 smithi156 ceph-mon[29302]: from='mgr.14160 172.21.15.156:0/3789331119' entity='mgr.smithi156.zkvylv' 2024-07-16T09:47:25.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:47:25 smithi156 ceph-mon[29302]: [16/Jul/2024:09:47:24] ENGINE Serving on https://172.21.15.156:7150 2024-07-16T09:47:25.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:47:25 smithi156 ceph-mon[29302]: [16/Jul/2024:09:47:24] ENGINE Bus STARTED 2024-07-16T09:47:25.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:47:25 smithi156 ceph-mon[29302]: [16/Jul/2024:09:47:24] ENGINE Error in HTTPServer.serve 2024-07-16T09:47:25.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: Traceback (most recent call last): 2024-07-16T09:47:25.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-07-16T09:47:25.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: self._connections.run(self.expiration_interval) 2024-07-16T09:47:25.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-07-16T09:47:25.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: self._run(expiration_interval) 2024-07-16T09:47:25.926 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-07-16T09:47:25.926 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-07-16T09:47:25.926 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-07-16T09:47:25.926 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-07-16T09:47:25.926 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-07-16T09:47:25.926 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: s = self.context.wrap_socket( 2024-07-16T09:47:25.926 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-07-16T09:47:25.926 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: return self.sslsocket_class._create( 2024-07-16T09:47:25.926 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-07-16T09:47:25.926 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: self.do_handshake() 2024-07-16T09:47:25.926 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-07-16T09:47:25.926 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: self._sslobj.do_handshake() 2024-07-16T09:47:25.926 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-07-16T09:47:25.927 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:47:25 smithi156 ceph-mon[29302]: from='mgr.14160 172.21.15.156:0/3789331119' entity='mgr.smithi156.zkvylv' 2024-07-16T09:47:27.176 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$m0/IZUDOG2TBsNLC0.VCAemCL35Y1.xE2hy7n6w.kJnVcuw988OCG", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1721123246, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-07-16T09:47:27.176 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:26.156+0000 7faf906df640 1 Processor -- start 2024-07-16T09:47:27.176 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:26.157+0000 7faf906df640 1 -- start start 2024-07-16T09:47:27.176 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:26.157+0000 7faf906df640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faf88101250 0x7faf88101660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:47:27.176 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:26.157+0000 7faf906df640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf88101ba0 con 0x7faf88101250 2024-07-16T09:47:27.176 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:26.157+0000 7faf8e47b640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faf88101250 0x7faf88101660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:47:27.176 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:26.157+0000 7faf8e47b640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faf88101250 0x7faf88101660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:42226/0 (socket says 172.21.15.156:42226) 2024-07-16T09:47:27.177 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:26.157+0000 7faf8e47b640 1 -- 172.21.15.156:0/2890993681 learned_addr learned my addr 172.21.15.156:0/2890993681 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T09:47:27.177 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:26.157+0000 7faf8e47b640 1 -- 172.21.15.156:0/2890993681 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf88101ce0 con 0x7faf88101250 2024-07-16T09:47:27.177 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:26.158+0000 7faf8e47b640 1 --2- 172.21.15.156:0/2890993681 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faf88101250 0x7faf88101660 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7faf7c00baf0 tx=0x7faf7c00b460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ccd944f0653b838 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:47:27.177 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:26.158+0000 7faf8d479640 1 -- 172.21.15.156:0/2890993681 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7faf7c0327a0 con 0x7faf88101250 2024-07-16T09:47:27.177 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:26.158+0000 7faf8d479640 1 -- 172.21.15.156:0/2890993681 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7faf7c032dd0 con 0x7faf88101250 2024-07-16T09:47:27.177 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:26.158+0000 7faf8d479640 1 -- 172.21.15.156:0/2890993681 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7faf7c039310 con 0x7faf88101250 2024-07-16T09:47:27.177 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:26.159+0000 7faf906df640 1 -- 172.21.15.156:0/2890993681 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faf88101250 msgr2=0x7faf88101660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:47:27.177 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:26.159+0000 7faf906df640 1 --2- 172.21.15.156:0/2890993681 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faf88101250 0x7faf88101660 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7faf7c00baf0 tx=0x7faf7c00b460 comp rx=0 tx=0).stop 2024-07-16T09:47:27.177 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:26.159+0000 7faf906df640 1 -- 172.21.15.156:0/2890993681 shutdown_connections 2024-07-16T09:47:27.177 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:26.159+0000 7faf906df640 1 --2- 172.21.15.156:0/2890993681 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faf88101250 0x7faf88101660 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:47:27.177 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:26.159+0000 7faf906df640 1 -- 172.21.15.156:0/2890993681 >> 172.21.15.156:0/2890993681 conn(0x7faf880fbcc0 msgr2=0x7faf880fe0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:47:27.177 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:26.159+0000 7faf906df640 1 -- 172.21.15.156:0/2890993681 shutdown_connections 2024-07-16T09:47:27.177 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:26.159+0000 7faf906df640 1 -- 172.21.15.156:0/2890993681 wait complete. 2024-07-16T09:47:27.178 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:26.160+0000 7faf906df640 1 Processor -- start 2024-07-16T09:47:27.178 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:26.160+0000 7faf906df640 1 -- start start 2024-07-16T09:47:27.178 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:26.160+0000 7faf906df640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faf88101250 0x7faf8810e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:47:27.178 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:26.160+0000 7faf906df640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf8810eee0 con 0x7faf88101250 2024-07-16T09:47:27.178 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:26.160+0000 7faf8e47b640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faf88101250 0x7faf8810e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:47:27.178 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:26.160+0000 7faf8e47b640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faf88101250 0x7faf8810e9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:42236/0 (socket says 172.21.15.156:42236) 2024-07-16T09:47:27.178 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:26.160+0000 7faf8e47b640 1 -- 172.21.15.156:0/3187191223 learned_addr learned my addr 172.21.15.156:0/3187191223 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T09:47:27.178 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:26.160+0000 7faf8e47b640 1 -- 172.21.15.156:0/3187191223 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf7c00b700 con 0x7faf88101250 2024-07-16T09:47:27.178 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:26.160+0000 7faf8e47b640 1 --2- 172.21.15.156:0/3187191223 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faf88101250 0x7faf8810e9a0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7faf7c00bb50 tx=0x7faf7c00b460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:47:27.178 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:26.160+0000 7faf7affd640 1 -- 172.21.15.156:0/3187191223 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7faf7c0329b0 con 0x7faf88101250 2024-07-16T09:47:27.178 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:26.160+0000 7faf7affd640 1 -- 172.21.15.156:0/3187191223 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7faf7c032b10 con 0x7faf88101250 2024-07-16T09:47:27.178 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:26.160+0000 7faf906df640 1 -- 172.21.15.156:0/3187191223 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf8810f0e0 con 0x7faf88101250 2024-07-16T09:47:27.179 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:26.161+0000 7faf7affd640 1 -- 172.21.15.156:0/3187191223 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7faf7c03b720 con 0x7faf88101250 2024-07-16T09:47:27.179 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:26.161+0000 7faf906df640 1 -- 172.21.15.156:0/3187191223 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf8810f4f0 con 0x7faf88101250 2024-07-16T09:47:27.179 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:26.161+0000 7faf7affd640 1 -- 172.21.15.156:0/3187191223 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 12) v1 ==== 42649+0+0 (secure 0 0 0) 0x7faf7c03b9e0 con 0x7faf88101250 2024-07-16T09:47:27.179 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:26.161+0000 7faf7affd640 1 --2- 172.21.15.156:0/3187191223 >> [v2:172.21.15.156:6800/3984987611,v1:172.21.15.156:6801/3984987611] conn(0x7faf740357d0 0x7faf74037c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:47:27.179 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:26.161+0000 7faf7affd640 1 -- 172.21.15.156:0/3187191223 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7faf7c06aca0 con 0x7faf88101250 2024-07-16T09:47:27.179 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:26.161+0000 7faf906df640 1 -- 172.21.15.156:0/3187191223 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf8804b530 con 0x7faf88101250 2024-07-16T09:47:27.179 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:26.162+0000 7faf8dc7a640 1 --2- 172.21.15.156:0/3187191223 >> [v2:172.21.15.156:6800/3984987611,v1:172.21.15.156:6801/3984987611] conn(0x7faf740357d0 0x7faf74037c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:47:27.179 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:26.162+0000 7faf8dc7a640 1 --2- 172.21.15.156:0/3187191223 >> [v2:172.21.15.156:6800/3984987611,v1:172.21.15.156:6801/3984987611] conn(0x7faf740357d0 0x7faf74037c80 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7faf84006fd0 tx=0x7faf84006db0 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:47:27.179 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:26.166+0000 7faf7affd640 1 -- 172.21.15.156:0/3187191223 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7faf7c036070 con 0x7faf88101250 2024-07-16T09:47:27.179 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:26.354+0000 7faf906df640 1 -- 172.21.15.156:0/3187191223 --> [v2:172.21.15.156:6800/3984987611,v1:172.21.15.156:6801/3984987611] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) v1 -- 0x7faf88100b20 con 0x7faf740357d0 2024-07-16T09:47:27.179 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:26.574+0000 7faf7affd640 1 -- 172.21.15.156:0/3187191223 <== mgr.14160 v2:172.21.15.156:6800/3984987611 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+252 (secure 0 0 0) 0x7faf88100b20 con 0x7faf740357d0 2024-07-16T09:47:27.179 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:26.576+0000 7faf906df640 1 -- 172.21.15.156:0/3187191223 >> [v2:172.21.15.156:6800/3984987611,v1:172.21.15.156:6801/3984987611] conn(0x7faf740357d0 msgr2=0x7faf74037c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:47:27.180 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:26.576+0000 7faf906df640 1 --2- 172.21.15.156:0/3187191223 >> [v2:172.21.15.156:6800/3984987611,v1:172.21.15.156:6801/3984987611] conn(0x7faf740357d0 0x7faf74037c80 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7faf84006fd0 tx=0x7faf84006db0 comp rx=0 tx=0).stop 2024-07-16T09:47:27.180 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:26.576+0000 7faf906df640 1 -- 172.21.15.156:0/3187191223 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faf88101250 msgr2=0x7faf8810e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:47:27.180 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:26.576+0000 7faf906df640 1 --2- 172.21.15.156:0/3187191223 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faf88101250 0x7faf8810e9a0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7faf7c00bb50 tx=0x7faf7c00b460 comp rx=0 tx=0).stop 2024-07-16T09:47:27.180 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:26.576+0000 7faf906df640 1 -- 172.21.15.156:0/3187191223 shutdown_connections 2024-07-16T09:47:27.180 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:26.576+0000 7faf906df640 1 --2- 172.21.15.156:0/3187191223 >> [v2:172.21.15.156:6800/3984987611,v1:172.21.15.156:6801/3984987611] conn(0x7faf740357d0 0x7faf74037c80 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:47:27.180 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:26.576+0000 7faf906df640 1 --2- 172.21.15.156:0/3187191223 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faf88101250 0x7faf8810e9a0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:47:27.180 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:26.576+0000 7faf906df640 1 -- 172.21.15.156:0/3187191223 >> 172.21.15.156:0/3187191223 conn(0x7faf880fbcc0 msgr2=0x7faf88105260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:47:27.180 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:26.576+0000 7faf906df640 1 -- 172.21.15.156:0/3187191223 shutdown_connections 2024-07-16T09:47:27.180 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:26.576+0000 7faf906df640 1 -- 172.21.15.156:0/3187191223 wait complete. 2024-07-16T09:47:27.180 INFO:teuthology.orchestra.run.smithi156.stdout:Fetching dashboard port number... 2024-07-16T09:47:27.740 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:47:27 smithi156 ceph-mon[29302]: from='client.14174 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-07-16T09:47:27.740 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:47:27 smithi156 ceph-mon[29302]: from='mgr.14160 172.21.15.156:0/3789331119' entity='mgr.smithi156.zkvylv' 2024-07-16T09:47:28.661 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:47:28 smithi156 ceph-mon[29302]: from='client.? 172.21.15.156:0/1586048773' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-07-16T09:47:28.668 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stdout 8443 2024-07-16T09:47:28.668 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:27.555+0000 7f7e98eee640 1 Processor -- start 2024-07-16T09:47:28.668 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:27.555+0000 7f7e98eee640 1 -- start start 2024-07-16T09:47:28.668 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:27.556+0000 7f7e98eee640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7e94075ba0 0x7f7e94104a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:47:28.668 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:27.556+0000 7f7e98eee640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e94104f70 con 0x7f7e94075ba0 2024-07-16T09:47:28.668 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:27.556+0000 7f7e9259c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7e94075ba0 0x7f7e94104a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:47:28.668 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:27.556+0000 7f7e9259c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7e94075ba0 0x7f7e94104a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:42238/0 (socket says 172.21.15.156:42238) 2024-07-16T09:47:28.668 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:27.556+0000 7f7e9259c640 1 -- 172.21.15.156:0/601178384 learned_addr learned my addr 172.21.15.156:0/601178384 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T09:47:28.669 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:27.556+0000 7f7e9259c640 1 -- 172.21.15.156:0/601178384 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e941050b0 con 0x7f7e94075ba0 2024-07-16T09:47:28.669 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:27.557+0000 7f7e9259c640 1 --2- 172.21.15.156:0/601178384 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7e94075ba0 0x7f7e94104a30 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f7e84009840 tx=0x7f7e840091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abe6f198de6768b server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:47:28.669 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:27.557+0000 7f7e91d9b640 1 -- 172.21.15.156:0/601178384 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7e84032440 con 0x7f7e94075ba0 2024-07-16T09:47:28.669 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:27.557+0000 7f7e91d9b640 1 -- 172.21.15.156:0/601178384 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7e84032a70 con 0x7f7e94075ba0 2024-07-16T09:47:28.669 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:27.557+0000 7f7e91d9b640 1 -- 172.21.15.156:0/601178384 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7e84031ad0 con 0x7f7e94075ba0 2024-07-16T09:47:28.669 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:27.558+0000 7f7e98eee640 1 -- 172.21.15.156:0/601178384 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7e94075ba0 msgr2=0x7f7e94104a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:47:28.669 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:27.558+0000 7f7e98eee640 1 --2- 172.21.15.156:0/601178384 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7e94075ba0 0x7f7e94104a30 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f7e84009840 tx=0x7f7e840091d0 comp rx=0 tx=0).stop 2024-07-16T09:47:28.669 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:27.558+0000 7f7e98eee640 1 -- 172.21.15.156:0/601178384 shutdown_connections 2024-07-16T09:47:28.669 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:27.558+0000 7f7e98eee640 1 --2- 172.21.15.156:0/601178384 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7e94075ba0 0x7f7e94104a30 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:47:28.669 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:27.558+0000 7f7e98eee640 1 -- 172.21.15.156:0/601178384 >> 172.21.15.156:0/601178384 conn(0x7f7e940fc520 msgr2=0x7f7e940fe930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:47:28.669 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:27.558+0000 7f7e98eee640 1 -- 172.21.15.156:0/601178384 shutdown_connections 2024-07-16T09:47:28.669 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:27.558+0000 7f7e98eee640 1 -- 172.21.15.156:0/601178384 wait complete. 2024-07-16T09:47:28.669 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:27.558+0000 7f7e98eee640 1 Processor -- start 2024-07-16T09:47:28.670 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:27.558+0000 7f7e98eee640 1 -- start start 2024-07-16T09:47:28.670 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:27.558+0000 7f7e98eee640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7e94075ba0 0x7f7e94110b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:47:28.670 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:27.558+0000 7f7e98eee640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e941110a0 con 0x7f7e94075ba0 2024-07-16T09:47:28.670 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:27.558+0000 7f7e9259c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7e94075ba0 0x7f7e94110b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:47:28.670 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:27.558+0000 7f7e9259c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7e94075ba0 0x7f7e94110b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:42244/0 (socket says 172.21.15.156:42244) 2024-07-16T09:47:28.670 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:27.558+0000 7f7e9259c640 1 -- 172.21.15.156:0/1586048773 learned_addr learned my addr 172.21.15.156:0/1586048773 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T09:47:28.670 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:27.559+0000 7f7e9259c640 1 -- 172.21.15.156:0/1586048773 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e84009450 con 0x7f7e94075ba0 2024-07-16T09:47:28.670 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:27.559+0000 7f7e9259c640 1 --2- 172.21.15.156:0/1586048773 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7e94075ba0 0x7f7e94110b60 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f7e840098a0 tx=0x7f7e8402c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:47:28.670 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:27.559+0000 7f7e8affd640 1 -- 172.21.15.156:0/1586048773 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7e84031e90 con 0x7f7e94075ba0 2024-07-16T09:47:28.670 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:27.559+0000 7f7e8affd640 1 -- 172.21.15.156:0/1586048773 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7e8403b910 con 0x7f7e94075ba0 2024-07-16T09:47:28.670 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:27.559+0000 7f7e98eee640 1 -- 172.21.15.156:0/1586048773 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e941112a0 con 0x7f7e94075ba0 2024-07-16T09:47:28.670 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:27.559+0000 7f7e8affd640 1 -- 172.21.15.156:0/1586048773 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7e8403a9b0 con 0x7f7e94075ba0 2024-07-16T09:47:28.671 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:27.559+0000 7f7e98eee640 1 -- 172.21.15.156:0/1586048773 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e94111710 con 0x7f7e94075ba0 2024-07-16T09:47:28.671 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:27.560+0000 7f7e8affd640 1 -- 172.21.15.156:0/1586048773 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 12) v1 ==== 42649+0+0 (secure 0 0 0) 0x7f7e84039070 con 0x7f7e94075ba0 2024-07-16T09:47:28.671 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:27.560+0000 7f7e98eee640 1 -- 172.21.15.156:0/1586048773 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e9404b530 con 0x7f7e94075ba0 2024-07-16T09:47:28.671 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:27.560+0000 7f7e8affd640 1 --2- 172.21.15.156:0/1586048773 >> [v2:172.21.15.156:6800/3984987611,v1:172.21.15.156:6801/3984987611] conn(0x7f7e80035820 0x7f7e80037cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:47:28.671 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:27.560+0000 7f7e8affd640 1 -- 172.21.15.156:0/1586048773 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f7e84067ef0 con 0x7f7e94075ba0 2024-07-16T09:47:28.671 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:27.560+0000 7f7e8bfff640 1 --2- 172.21.15.156:0/1586048773 >> [v2:172.21.15.156:6800/3984987611,v1:172.21.15.156:6801/3984987611] conn(0x7f7e80035820 0x7f7e80037cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:47:28.671 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:27.560+0000 7f7e8bfff640 1 --2- 172.21.15.156:0/1586048773 >> [v2:172.21.15.156:6800/3984987611,v1:172.21.15.156:6801/3984987611] conn(0x7f7e80035820 0x7f7e80037cd0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f7e7c006fd0 tx=0x7f7e7c006db0 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:47:28.671 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:27.564+0000 7f7e8affd640 1 -- 172.21.15.156:0/1586048773 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f7e84039500 con 0x7f7e94075ba0 2024-07-16T09:47:28.671 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:27.738+0000 7f7e98eee640 1 -- 172.21.15.156:0/1586048773 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) v1 -- 0x7f7e94112070 con 0x7f7e94075ba0 2024-07-16T09:47:28.671 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:27.738+0000 7f7e8affd640 1 -- 172.21.15.156:0/1586048773 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) v1 ==== 112+0+5 (secure 0 0 0) 0x7f7e8403f070 con 0x7f7e94075ba0 2024-07-16T09:47:28.671 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:27.740+0000 7f7e98eee640 1 -- 172.21.15.156:0/1586048773 >> [v2:172.21.15.156:6800/3984987611,v1:172.21.15.156:6801/3984987611] conn(0x7f7e80035820 msgr2=0x7f7e80037cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:47:28.671 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:27.740+0000 7f7e98eee640 1 --2- 172.21.15.156:0/1586048773 >> [v2:172.21.15.156:6800/3984987611,v1:172.21.15.156:6801/3984987611] conn(0x7f7e80035820 0x7f7e80037cd0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f7e7c006fd0 tx=0x7f7e7c006db0 comp rx=0 tx=0).stop 2024-07-16T09:47:28.671 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:27.741+0000 7f7e98eee640 1 -- 172.21.15.156:0/1586048773 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7e94075ba0 msgr2=0x7f7e94110b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:47:28.672 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:27.741+0000 7f7e98eee640 1 --2- 172.21.15.156:0/1586048773 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7e94075ba0 0x7f7e94110b60 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f7e840098a0 tx=0x7f7e8402c040 comp rx=0 tx=0).stop 2024-07-16T09:47:28.672 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:27.741+0000 7f7e98eee640 1 -- 172.21.15.156:0/1586048773 shutdown_connections 2024-07-16T09:47:28.672 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:27.741+0000 7f7e98eee640 1 --2- 172.21.15.156:0/1586048773 >> [v2:172.21.15.156:6800/3984987611,v1:172.21.15.156:6801/3984987611] conn(0x7f7e80035820 0x7f7e80037cd0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:47:28.672 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:27.741+0000 7f7e98eee640 1 --2- 172.21.15.156:0/1586048773 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7e94075ba0 0x7f7e94110b60 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:47:28.672 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:27.741+0000 7f7e98eee640 1 -- 172.21.15.156:0/1586048773 >> 172.21.15.156:0/1586048773 conn(0x7f7e940fc520 msgr2=0x7f7e940fd010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:47:28.672 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:27.741+0000 7f7e98eee640 1 -- 172.21.15.156:0/1586048773 shutdown_connections 2024-07-16T09:47:28.672 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:27.741+0000 7f7e98eee640 1 -- 172.21.15.156:0/1586048773 wait complete. 2024-07-16T09:47:28.677 INFO:teuthology.orchestra.run.smithi156.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-07-16T09:47:28.677 INFO:teuthology.orchestra.run.smithi156.stdout:systemctl: stdout disabled 2024-07-16T09:47:28.686 INFO:teuthology.orchestra.run.smithi156.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-07-16T09:47:28.686 INFO:teuthology.orchestra.run.smithi156.stdout:systemctl: stdout inactive 2024-07-16T09:47:28.686 INFO:teuthology.orchestra.run.smithi156.stdout:firewalld.service is not enabled 2024-07-16T09:47:28.686 INFO:teuthology.orchestra.run.smithi156.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-07-16T09:47:28.688 INFO:teuthology.orchestra.run.smithi156.stdout:Ceph Dashboard is now available at: 2024-07-16T09:47:28.688 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-07-16T09:47:28.688 INFO:teuthology.orchestra.run.smithi156.stdout: URL: https://smithi156.front.sepia.ceph.com:8443/ 2024-07-16T09:47:28.688 INFO:teuthology.orchestra.run.smithi156.stdout: User: admin 2024-07-16T09:47:28.688 INFO:teuthology.orchestra.run.smithi156.stdout: Password: s0287w8ds3 2024-07-16T09:47:28.688 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-07-16T09:47:28.689 INFO:teuthology.orchestra.run.smithi156.stdout:Saving cluster configuration to /var/lib/ceph/26c91c26-4358-11ef-bcad-c7b262605968/config directory 2024-07-16T09:47:28.689 INFO:teuthology.orchestra.run.smithi156.stdout:Enabling autotune for osd_memory_target 2024-07-16T09:47:29.833 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:47:29 smithi156 ceph-mon[29302]: mgrmap e13: smithi156.zkvylv(active, since 6s) 2024-07-16T09:47:29.844 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:29.093+0000 7fbd32e24640 1 Processor -- start 2024-07-16T09:47:29.844 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:29.094+0000 7fbd32e24640 1 -- start start 2024-07-16T09:47:29.844 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:29.094+0000 7fbd32e24640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbd2c075b40 0x7fbd2c104a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:47:29.844 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:29.094+0000 7fbd32e24640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd2c075f50 con 0x7fbd2c075b40 2024-07-16T09:47:29.844 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:29.094+0000 7fbd30bc0640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbd2c075b40 0x7fbd2c104a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:47:29.844 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:29.094+0000 7fbd30bc0640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbd2c075b40 0x7fbd2c104a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:42260/0 (socket says 172.21.15.156:42260) 2024-07-16T09:47:29.844 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:29.094+0000 7fbd30bc0640 1 -- 172.21.15.156:0/3568184083 learned_addr learned my addr 172.21.15.156:0/3568184083 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T09:47:29.844 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:29.094+0000 7fbd30bc0640 1 -- 172.21.15.156:0/3568184083 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd2c104fd0 con 0x7fbd2c075b40 2024-07-16T09:47:29.844 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:29.095+0000 7fbd30bc0640 1 --2- 172.21.15.156:0/3568184083 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbd2c075b40 0x7fbd2c104a90 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fbd20009840 tx=0x7fbd200091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=805ab5bf053b956a server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:47:29.844 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:29.095+0000 7fbd2b7fe640 1 -- 172.21.15.156:0/3568184083 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbd20032440 con 0x7fbd2c075b40 2024-07-16T09:47:29.844 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:29.095+0000 7fbd2b7fe640 1 -- 172.21.15.156:0/3568184083 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbd20032a70 con 0x7fbd2c075b40 2024-07-16T09:47:29.845 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:29.095+0000 7fbd2b7fe640 1 -- 172.21.15.156:0/3568184083 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbd20031ad0 con 0x7fbd2c075b40 2024-07-16T09:47:29.845 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:29.096+0000 7fbd32e24640 1 -- 172.21.15.156:0/3568184083 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbd2c075b40 msgr2=0x7fbd2c104a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:47:29.845 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:29.096+0000 7fbd32e24640 1 --2- 172.21.15.156:0/3568184083 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbd2c075b40 0x7fbd2c104a90 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fbd20009840 tx=0x7fbd200091d0 comp rx=0 tx=0).stop 2024-07-16T09:47:29.845 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:29.096+0000 7fbd32e24640 1 -- 172.21.15.156:0/3568184083 shutdown_connections 2024-07-16T09:47:29.845 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:29.096+0000 7fbd32e24640 1 --2- 172.21.15.156:0/3568184083 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbd2c075b40 0x7fbd2c104a90 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:47:29.845 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:29.096+0000 7fbd32e24640 1 -- 172.21.15.156:0/3568184083 >> 172.21.15.156:0/3568184083 conn(0x7fbd2c0fc520 msgr2=0x7fbd2c0fe930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:47:29.845 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:29.096+0000 7fbd32e24640 1 -- 172.21.15.156:0/3568184083 shutdown_connections 2024-07-16T09:47:29.845 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:29.096+0000 7fbd32e24640 1 -- 172.21.15.156:0/3568184083 wait complete. 2024-07-16T09:47:29.845 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:29.096+0000 7fbd32e24640 1 Processor -- start 2024-07-16T09:47:29.845 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:29.096+0000 7fbd32e24640 1 -- start start 2024-07-16T09:47:29.845 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:29.097+0000 7fbd32e24640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbd2c075b40 0x7fbd2c110b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:47:29.845 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:29.097+0000 7fbd32e24640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd2c111080 con 0x7fbd2c075b40 2024-07-16T09:47:29.845 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:29.097+0000 7fbd30bc0640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbd2c075b40 0x7fbd2c110b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:47:29.846 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:29.097+0000 7fbd30bc0640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbd2c075b40 0x7fbd2c110b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:42262/0 (socket says 172.21.15.156:42262) 2024-07-16T09:47:29.846 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:29.097+0000 7fbd30bc0640 1 -- 172.21.15.156:0/1281444355 learned_addr learned my addr 172.21.15.156:0/1281444355 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T09:47:29.846 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:29.097+0000 7fbd30bc0640 1 -- 172.21.15.156:0/1281444355 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd20009450 con 0x7fbd2c075b40 2024-07-16T09:47:29.846 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:29.097+0000 7fbd30bc0640 1 --2- 172.21.15.156:0/1281444355 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbd2c075b40 0x7fbd2c110b40 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fbd200317f0 tx=0x7fbd2002c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:47:29.846 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:29.097+0000 7fbd297fa640 1 -- 172.21.15.156:0/1281444355 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbd20032900 con 0x7fbd2c075b40 2024-07-16T09:47:29.846 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:29.098+0000 7fbd32e24640 1 -- 172.21.15.156:0/1281444355 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd2c111280 con 0x7fbd2c075b40 2024-07-16T09:47:29.846 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:29.098+0000 7fbd297fa640 1 -- 172.21.15.156:0/1281444355 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbd2003b910 con 0x7fbd2c075b40 2024-07-16T09:47:29.846 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:29.098+0000 7fbd297fa640 1 -- 172.21.15.156:0/1281444355 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbd2003a960 con 0x7fbd2c075b40 2024-07-16T09:47:29.846 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:29.098+0000 7fbd32e24640 1 -- 172.21.15.156:0/1281444355 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd2c1116f0 con 0x7fbd2c075b40 2024-07-16T09:47:29.846 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:29.099+0000 7fbd297fa640 1 -- 172.21.15.156:0/1281444355 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 13) v1 ==== 42693+0+0 (secure 0 0 0) 0x7fbd20039070 con 0x7fbd2c075b40 2024-07-16T09:47:29.846 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:29.099+0000 7fbd32e24640 1 -- 172.21.15.156:0/1281444355 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd2c04b530 con 0x7fbd2c075b40 2024-07-16T09:47:29.847 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:29.099+0000 7fbd297fa640 1 --2- 172.21.15.156:0/1281444355 >> [v2:172.21.15.156:6800/3984987611,v1:172.21.15.156:6801/3984987611] conn(0x7fbd1c035870 0x7fbd1c037d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:47:29.847 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:29.099+0000 7fbd2bfff640 1 --2- 172.21.15.156:0/1281444355 >> [v2:172.21.15.156:6800/3984987611,v1:172.21.15.156:6801/3984987611] conn(0x7fbd1c035870 0x7fbd1c037d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:47:29.847 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:29.099+0000 7fbd297fa640 1 -- 172.21.15.156:0/1281444355 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7fbd20068020 con 0x7fbd2c075b40 2024-07-16T09:47:29.847 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:29.099+0000 7fbd2bfff640 1 --2- 172.21.15.156:0/1281444355 >> [v2:172.21.15.156:6800/3984987611,v1:172.21.15.156:6801/3984987611] conn(0x7fbd1c035870 0x7fbd1c037d20 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fbd18006fd0 tx=0x7fbd18006db0 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:47:29.847 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:29.103+0000 7fbd297fa640 1 -- 172.21.15.156:0/1281444355 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fbd2003ba80 con 0x7fbd2c075b40 2024-07-16T09:47:29.847 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:29.270+0000 7fbd32e24640 1 -- 172.21.15.156:0/1281444355 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) v1 -- 0x7fbd2c112110 con 0x7fbd2c075b40 2024-07-16T09:47:29.847 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:29.270+0000 7fbd297fa640 1 -- 172.21.15.156:0/1281444355 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) v1 ==== 127+0+0 (secure 0 0 0) 0x7fbd2003f0c0 con 0x7fbd2c075b40 2024-07-16T09:47:29.847 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:29.272+0000 7fbd32e24640 1 -- 172.21.15.156:0/1281444355 >> [v2:172.21.15.156:6800/3984987611,v1:172.21.15.156:6801/3984987611] conn(0x7fbd1c035870 msgr2=0x7fbd1c037d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:47:29.847 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:29.272+0000 7fbd32e24640 1 --2- 172.21.15.156:0/1281444355 >> [v2:172.21.15.156:6800/3984987611,v1:172.21.15.156:6801/3984987611] conn(0x7fbd1c035870 0x7fbd1c037d20 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fbd18006fd0 tx=0x7fbd18006db0 comp rx=0 tx=0).stop 2024-07-16T09:47:29.847 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:29.272+0000 7fbd32e24640 1 -- 172.21.15.156:0/1281444355 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbd2c075b40 msgr2=0x7fbd2c110b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:47:29.847 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:29.272+0000 7fbd32e24640 1 --2- 172.21.15.156:0/1281444355 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbd2c075b40 0x7fbd2c110b40 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fbd200317f0 tx=0x7fbd2002c040 comp rx=0 tx=0).stop 2024-07-16T09:47:29.847 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:29.272+0000 7fbd32e24640 1 -- 172.21.15.156:0/1281444355 shutdown_connections 2024-07-16T09:47:29.848 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:29.272+0000 7fbd32e24640 1 --2- 172.21.15.156:0/1281444355 >> [v2:172.21.15.156:6800/3984987611,v1:172.21.15.156:6801/3984987611] conn(0x7fbd1c035870 0x7fbd1c037d20 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:47:29.848 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:29.272+0000 7fbd32e24640 1 --2- 172.21.15.156:0/1281444355 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbd2c075b40 0x7fbd2c110b40 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:47:29.848 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:29.272+0000 7fbd32e24640 1 -- 172.21.15.156:0/1281444355 >> 172.21.15.156:0/1281444355 conn(0x7fbd2c0fc520 msgr2=0x7fbd2c0fd0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:47:29.848 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:29.272+0000 7fbd32e24640 1 -- 172.21.15.156:0/1281444355 shutdown_connections 2024-07-16T09:47:29.848 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:29.272+0000 7fbd32e24640 1 -- 172.21.15.156:0/1281444355 wait complete. 2024-07-16T09:47:31.106 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:30.260+0000 7f3095287640 1 Processor -- start 2024-07-16T09:47:31.106 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:30.260+0000 7f3095287640 1 -- start start 2024-07-16T09:47:31.107 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:30.260+0000 7f3095287640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3090100d00 0x7f3090101110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:47:31.107 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:30.260+0000 7f3095287640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3090101650 con 0x7f3090100d00 2024-07-16T09:47:31.107 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:30.261+0000 7f308effd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3090100d00 0x7f3090101110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:47:31.107 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:30.261+0000 7f308effd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3090100d00 0x7f3090101110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:42264/0 (socket says 172.21.15.156:42264) 2024-07-16T09:47:31.107 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:30.261+0000 7f308effd640 1 -- 172.21.15.156:0/1079305259 learned_addr learned my addr 172.21.15.156:0/1079305259 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T09:47:31.107 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:30.261+0000 7f308effd640 1 -- 172.21.15.156:0/1079305259 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3090101790 con 0x7f3090100d00 2024-07-16T09:47:31.107 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:30.262+0000 7f308effd640 1 --2- 172.21.15.156:0/1079305259 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3090100d00 0x7f3090101110 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f308800baf0 tx=0x7f308800b460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b43a119fcff4a52 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:47:31.107 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:30.262+0000 7f308dffb640 1 -- 172.21.15.156:0/1079305259 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f30880327a0 con 0x7f3090100d00 2024-07-16T09:47:31.107 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:30.262+0000 7f308dffb640 1 -- 172.21.15.156:0/1079305259 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3088032dd0 con 0x7f3090100d00 2024-07-16T09:47:31.107 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:30.262+0000 7f308dffb640 1 -- 172.21.15.156:0/1079305259 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3088039310 con 0x7f3090100d00 2024-07-16T09:47:31.107 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:30.262+0000 7f3095287640 1 -- 172.21.15.156:0/1079305259 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3090100d00 msgr2=0x7f3090101110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:47:31.108 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:30.262+0000 7f3095287640 1 --2- 172.21.15.156:0/1079305259 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3090100d00 0x7f3090101110 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f308800baf0 tx=0x7f308800b460 comp rx=0 tx=0).stop 2024-07-16T09:47:31.108 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:30.263+0000 7f3095287640 1 -- 172.21.15.156:0/1079305259 shutdown_connections 2024-07-16T09:47:31.108 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:30.263+0000 7f3095287640 1 --2- 172.21.15.156:0/1079305259 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3090100d00 0x7f3090101110 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:47:31.108 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:30.263+0000 7f3095287640 1 -- 172.21.15.156:0/1079305259 >> 172.21.15.156:0/1079305259 conn(0x7f30900fc520 msgr2=0x7f30900fe930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:47:31.108 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:30.263+0000 7f3095287640 1 -- 172.21.15.156:0/1079305259 shutdown_connections 2024-07-16T09:47:31.108 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:30.263+0000 7f3095287640 1 -- 172.21.15.156:0/1079305259 wait complete. 2024-07-16T09:47:31.108 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:30.263+0000 7f3095287640 1 Processor -- start 2024-07-16T09:47:31.108 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:30.263+0000 7f3095287640 1 -- start start 2024-07-16T09:47:31.108 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:30.263+0000 7f3095287640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3090100d00 0x7f309010e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:47:31.108 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:30.263+0000 7f3095287640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f309010eee0 con 0x7f3090100d00 2024-07-16T09:47:31.108 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:30.264+0000 7f308effd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3090100d00 0x7f309010e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:47:31.108 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:30.264+0000 7f308effd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3090100d00 0x7f309010e9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:42270/0 (socket says 172.21.15.156:42270) 2024-07-16T09:47:31.109 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:30.264+0000 7f308effd640 1 -- 172.21.15.156:0/1069488964 learned_addr learned my addr 172.21.15.156:0/1069488964 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T09:47:31.109 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:30.264+0000 7f308effd640 1 -- 172.21.15.156:0/1069488964 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f308800b700 con 0x7f3090100d00 2024-07-16T09:47:31.109 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:30.264+0000 7f308effd640 1 --2- 172.21.15.156:0/1069488964 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3090100d00 0x7f309010e9a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f3088003080 tx=0x7f308803d040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:47:31.109 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:30.264+0000 7f30877fe640 1 -- 172.21.15.156:0/1069488964 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3088032b00 con 0x7f3090100d00 2024-07-16T09:47:31.109 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:30.264+0000 7f3095287640 1 -- 172.21.15.156:0/1069488964 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f309010f0e0 con 0x7f3090100d00 2024-07-16T09:47:31.109 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:30.264+0000 7f30877fe640 1 -- 172.21.15.156:0/1069488964 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3088032c60 con 0x7f3090100d00 2024-07-16T09:47:31.109 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:30.264+0000 7f30877fe640 1 -- 172.21.15.156:0/1069488964 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f30880457f0 con 0x7f3090100d00 2024-07-16T09:47:31.109 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:30.264+0000 7f3095287640 1 -- 172.21.15.156:0/1069488964 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f309010f550 con 0x7f3090100d00 2024-07-16T09:47:31.109 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:30.265+0000 7f30877fe640 1 -- 172.21.15.156:0/1069488964 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 13) v1 ==== 42693+0+0 (secure 0 0 0) 0x7f30880459e0 con 0x7f3090100d00 2024-07-16T09:47:31.109 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:30.265+0000 7f3095287640 1 -- 172.21.15.156:0/1069488964 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f309004b530 con 0x7f3090100d00 2024-07-16T09:47:31.109 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:30.265+0000 7f30877fe640 1 --2- 172.21.15.156:0/1069488964 >> [v2:172.21.15.156:6800/3984987611,v1:172.21.15.156:6801/3984987611] conn(0x7f307c035870 0x7f307c037d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:47:31.109 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:30.265+0000 7f30877fe640 1 -- 172.21.15.156:0/1069488964 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f308806b250 con 0x7f3090100d00 2024-07-16T09:47:31.110 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:30.265+0000 7f308e7fc640 1 --2- 172.21.15.156:0/1069488964 >> [v2:172.21.15.156:6800/3984987611,v1:172.21.15.156:6801/3984987611] conn(0x7f307c035870 0x7f307c037d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:47:31.110 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:30.266+0000 7f308e7fc640 1 --2- 172.21.15.156:0/1069488964 >> [v2:172.21.15.156:6800/3984987611,v1:172.21.15.156:6801/3984987611] conn(0x7f307c035870 0x7f307c037d20 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f3078006fd0 tx=0x7f3078006db0 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:47:31.110 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:30.270+0000 7f30877fe640 1 -- 172.21.15.156:0/1069488964 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f30880432f0 con 0x7f3090100d00 2024-07-16T09:47:31.110 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:30.525+0000 7f3095287640 1 -- 172.21.15.156:0/1069488964 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) v1 -- 0x7f309010f550 con 0x7f3090100d00 2024-07-16T09:47:31.110 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:30.527+0000 7f30877fe640 1 -- 172.21.15.156:0/1069488964 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v33)=0 set mgr/dashboard/cluster/status v33) v1 ==== 153+0+0 (secure 0 0 0) 0x7f308804caf0 con 0x7f3090100d00 2024-07-16T09:47:31.110 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-07-16T09:47:31.110 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:30.529+0000 7f3095287640 1 -- 172.21.15.156:0/1069488964 >> [v2:172.21.15.156:6800/3984987611,v1:172.21.15.156:6801/3984987611] conn(0x7f307c035870 msgr2=0x7f307c037d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:47:31.110 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:30.529+0000 7f3095287640 1 --2- 172.21.15.156:0/1069488964 >> [v2:172.21.15.156:6800/3984987611,v1:172.21.15.156:6801/3984987611] conn(0x7f307c035870 0x7f307c037d20 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f3078006fd0 tx=0x7f3078006db0 comp rx=0 tx=0).stop 2024-07-16T09:47:31.110 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:30.529+0000 7f3095287640 1 -- 172.21.15.156:0/1069488964 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3090100d00 msgr2=0x7f309010e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:47:31.110 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:30.529+0000 7f3095287640 1 --2- 172.21.15.156:0/1069488964 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3090100d00 0x7f309010e9a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f3088003080 tx=0x7f308803d040 comp rx=0 tx=0).stop 2024-07-16T09:47:31.110 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:30.529+0000 7f3095287640 1 -- 172.21.15.156:0/1069488964 shutdown_connections 2024-07-16T09:47:31.110 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:30.529+0000 7f3095287640 1 --2- 172.21.15.156:0/1069488964 >> [v2:172.21.15.156:6800/3984987611,v1:172.21.15.156:6801/3984987611] conn(0x7f307c035870 0x7f307c037d20 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:47:31.111 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:30.529+0000 7f3095287640 1 --2- 172.21.15.156:0/1069488964 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3090100d00 0x7f309010e9a0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:47:31.111 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:30.530+0000 7f3095287640 1 -- 172.21.15.156:0/1069488964 >> 172.21.15.156:0/1069488964 conn(0x7f30900fc520 msgr2=0x7f30901052a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:47:31.111 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:30.530+0000 7f3095287640 1 -- 172.21.15.156:0/1069488964 shutdown_connections 2024-07-16T09:47:31.111 INFO:teuthology.orchestra.run.smithi156.stdout:/usr/bin/ceph: stderr 2024-07-16T09:47:30.530+0000 7f3095287640 1 -- 172.21.15.156:0/1069488964 wait complete. 2024-07-16T09:47:31.111 INFO:teuthology.orchestra.run.smithi156.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-07-16T09:47:31.111 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-07-16T09:47:31.111 INFO:teuthology.orchestra.run.smithi156.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 26c91c26-4358-11ef-bcad-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-07-16T09:47:31.111 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-07-16T09:47:31.111 INFO:teuthology.orchestra.run.smithi156.stdout:Or, if you are only running a single cluster on this host: 2024-07-16T09:47:31.111 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-07-16T09:47:31.111 INFO:teuthology.orchestra.run.smithi156.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-07-16T09:47:31.111 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-07-16T09:47:31.111 INFO:teuthology.orchestra.run.smithi156.stdout:Please consider enabling telemetry to help improve Ceph: 2024-07-16T09:47:31.112 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-07-16T09:47:31.112 INFO:teuthology.orchestra.run.smithi156.stdout: ceph telemetry on 2024-07-16T09:47:31.112 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-07-16T09:47:31.112 INFO:teuthology.orchestra.run.smithi156.stdout:For more information see: 2024-07-16T09:47:31.112 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-07-16T09:47:31.112 INFO:teuthology.orchestra.run.smithi156.stdout: https://docs.ceph.com/docs/master/mgr/telemetry/ 2024-07-16T09:47:31.112 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-07-16T09:47:31.112 INFO:teuthology.orchestra.run.smithi156.stdout:Bootstrap complete. 2024-07-16T09:47:31.143 INFO:tasks.cephadm:Fetching config... 2024-07-16T09:47:31.143 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2024-07-16T09:47:31.143 DEBUG:teuthology.orchestra.run.smithi156:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-07-16T09:47:31.160 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-07-16T09:47:31.160 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2024-07-16T09:47:31.160 DEBUG:teuthology.orchestra.run.smithi156:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-07-16T09:47:31.216 INFO:tasks.cephadm:Fetching mon keyring... 2024-07-16T09:47:31.216 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2024-07-16T09:47:31.216 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd if=/var/lib/ceph/26c91c26-4358-11ef-bcad-c7b262605968/mon.smithi156/keyring of=/dev/stdout 2024-07-16T09:47:31.281 INFO:tasks.cephadm:Fetching pub ssh key... 2024-07-16T09:47:31.281 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2024-07-16T09:47:31.281 DEBUG:teuthology.orchestra.run.smithi156:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-07-16T09:47:31.337 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-07-16T09:47:31.337 DEBUG:teuthology.orchestra.run.smithi156:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCdOXvOWd5hoFBKp2rvbmYLWz2SrJ5kxPsJw/lBfxEHdug3ug6bK5VFvHKjiLdOxCeHD7kfze17gQVK4LwdupaKUBMhuR/k2VLCP0N6S7aNyd6Zgew4lPY50W1nHuRef5ejuYI9LKLwlh9IExIKrmHHKvmznV8FklOV0d8dGPx2LYT69dwfFZ+ITuuXYL4kMa61uDRMwg1lIU+DZJuTgeUxIQbAn8yjuyT7Rah9Rir+/SK0M4rWRIDunJYqHbe+xFVfvmnwTe0Bwp/kQwlEvdQnghYTkvg7AVioTM7cHRMX4aX8O6rKQBd/K2FTucZlLxJJIp7OSw94zn9HcY+wOXBbCwLxyrA9PGsWu4cf/ViT+BFWWDPjpO4CS+qyyp8HNiPGWCzJJ0GCUkjXgt1PUGWq2PXDKhNBlue01it6BCXAaiLcS9d5DlOROQ8Sn3psN6rYkl4AzSidJygFRhqwtZHNAeDG4x9MMXiXpNIz/G72qz3Qgl339SPZwELUzZGzq0k= ceph-26c91c26-4358-11ef-bcad-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-07-16T09:47:31.414 INFO:teuthology.orchestra.run.smithi156.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCdOXvOWd5hoFBKp2rvbmYLWz2SrJ5kxPsJw/lBfxEHdug3ug6bK5VFvHKjiLdOxCeHD7kfze17gQVK4LwdupaKUBMhuR/k2VLCP0N6S7aNyd6Zgew4lPY50W1nHuRef5ejuYI9LKLwlh9IExIKrmHHKvmznV8FklOV0d8dGPx2LYT69dwfFZ+ITuuXYL4kMa61uDRMwg1lIU+DZJuTgeUxIQbAn8yjuyT7Rah9Rir+/SK0M4rWRIDunJYqHbe+xFVfvmnwTe0Bwp/kQwlEvdQnghYTkvg7AVioTM7cHRMX4aX8O6rKQBd/K2FTucZlLxJJIp7OSw94zn9HcY+wOXBbCwLxyrA9PGsWu4cf/ViT+BFWWDPjpO4CS+qyyp8HNiPGWCzJJ0GCUkjXgt1PUGWq2PXDKhNBlue01it6BCXAaiLcS9d5DlOROQ8Sn3psN6rYkl4AzSidJygFRhqwtZHNAeDG4x9MMXiXpNIz/G72qz3Qgl339SPZwELUzZGzq0k= ceph-26c91c26-4358-11ef-bcad-c7b262605968 2024-07-16T09:47:31.422 DEBUG:teuthology.orchestra.run.smithi176:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCdOXvOWd5hoFBKp2rvbmYLWz2SrJ5kxPsJw/lBfxEHdug3ug6bK5VFvHKjiLdOxCeHD7kfze17gQVK4LwdupaKUBMhuR/k2VLCP0N6S7aNyd6Zgew4lPY50W1nHuRef5ejuYI9LKLwlh9IExIKrmHHKvmznV8FklOV0d8dGPx2LYT69dwfFZ+ITuuXYL4kMa61uDRMwg1lIU+DZJuTgeUxIQbAn8yjuyT7Rah9Rir+/SK0M4rWRIDunJYqHbe+xFVfvmnwTe0Bwp/kQwlEvdQnghYTkvg7AVioTM7cHRMX4aX8O6rKQBd/K2FTucZlLxJJIp7OSw94zn9HcY+wOXBbCwLxyrA9PGsWu4cf/ViT+BFWWDPjpO4CS+qyyp8HNiPGWCzJJ0GCUkjXgt1PUGWq2PXDKhNBlue01it6BCXAaiLcS9d5DlOROQ8Sn3psN6rYkl4AzSidJygFRhqwtZHNAeDG4x9MMXiXpNIz/G72qz3Qgl339SPZwELUzZGzq0k= ceph-26c91c26-4358-11ef-bcad-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-07-16T09:47:31.463 INFO:teuthology.orchestra.run.smithi176.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCdOXvOWd5hoFBKp2rvbmYLWz2SrJ5kxPsJw/lBfxEHdug3ug6bK5VFvHKjiLdOxCeHD7kfze17gQVK4LwdupaKUBMhuR/k2VLCP0N6S7aNyd6Zgew4lPY50W1nHuRef5ejuYI9LKLwlh9IExIKrmHHKvmznV8FklOV0d8dGPx2LYT69dwfFZ+ITuuXYL4kMa61uDRMwg1lIU+DZJuTgeUxIQbAn8yjuyT7Rah9Rir+/SK0M4rWRIDunJYqHbe+xFVfvmnwTe0Bwp/kQwlEvdQnghYTkvg7AVioTM7cHRMX4aX8O6rKQBd/K2FTucZlLxJJIp7OSw94zn9HcY+wOXBbCwLxyrA9PGsWu4cf/ViT+BFWWDPjpO4CS+qyyp8HNiPGWCzJJ0GCUkjXgt1PUGWq2PXDKhNBlue01it6BCXAaiLcS9d5DlOROQ8Sn3psN6rYkl4AzSidJygFRhqwtZHNAeDG4x9MMXiXpNIz/G72qz3Qgl339SPZwELUzZGzq0k= ceph-26c91c26-4358-11ef-bcad-c7b262605968 2024-07-16T09:47:31.476 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 26c91c26-4358-11ef-bcad-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-07-16T09:47:31.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:47:31 smithi156 ceph-mon[29302]: from='client.? 172.21.15.156:0/1069488964' entity='client.admin' 2024-07-16T09:47:31.747 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/26c91c26-4358-11ef-bcad-c7b262605968/mon.smithi156/config 2024-07-16T09:47:34.330 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:47:34.074+0000 7fd941e80640 1 -- 172.21.15.156:0/246710956 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd93c0fb0f0 msgr2=0x7fd93c0fb500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:47:34.330 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:47:34.074+0000 7fd941e80640 1 --2- 172.21.15.156:0/246710956 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd93c0fb0f0 0x7fd93c0fb500 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fd92c004850 tx=0x7fd92c009da0 comp rx=0 tx=0).stop 2024-07-16T09:47:34.330 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:47:34.074+0000 7fd941e80640 1 -- 172.21.15.156:0/246710956 shutdown_connections 2024-07-16T09:47:34.330 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:47:34.074+0000 7fd941e80640 1 --2- 172.21.15.156:0/246710956 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd93c0fb0f0 0x7fd93c0fb500 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:47:34.330 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:47:34.074+0000 7fd941e80640 1 -- 172.21.15.156:0/246710956 >> 172.21.15.156:0/246710956 conn(0x7fd93c0f68d0 msgr2=0x7fd93c0f8ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:47:34.330 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:47:34.074+0000 7fd941e80640 1 -- 172.21.15.156:0/246710956 shutdown_connections 2024-07-16T09:47:34.330 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:47:34.074+0000 7fd941e80640 1 -- 172.21.15.156:0/246710956 wait complete. 2024-07-16T09:47:34.330 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:47:34.074+0000 7fd941e80640 1 Processor -- start 2024-07-16T09:47:34.330 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:47:34.075+0000 7fd941e80640 1 -- start start 2024-07-16T09:47:34.330 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:47:34.075+0000 7fd941e80640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd93c0fb0f0 0x7fd93c065340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:47:34.331 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:47:34.075+0000 7fd941e80640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd93c10f4d0 con 0x7fd93c0fb0f0 2024-07-16T09:47:34.331 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:47:34.075+0000 7fd93b7fe640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd93c0fb0f0 0x7fd93c065340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:47:34.331 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:47:34.075+0000 7fd93b7fe640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd93c0fb0f0 0x7fd93c065340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:49170/0 (socket says 172.21.15.156:49170) 2024-07-16T09:47:34.331 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:47:34.075+0000 7fd93b7fe640 1 -- 172.21.15.156:0/3954315072 learned_addr learned my addr 172.21.15.156:0/3954315072 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T09:47:34.331 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:47:34.075+0000 7fd93b7fe640 1 -- 172.21.15.156:0/3954315072 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd92c005040 con 0x7fd93c0fb0f0 2024-07-16T09:47:34.331 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:47:34.075+0000 7fd93b7fe640 1 --2- 172.21.15.156:0/3954315072 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd93c0fb0f0 0x7fd93c065340 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fd92c004820 tx=0x7fd92c009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:47:34.331 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:47:34.075+0000 7fd938ff9640 1 -- 172.21.15.156:0/3954315072 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd92c032650 con 0x7fd93c0fb0f0 2024-07-16T09:47:34.331 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:47:34.075+0000 7fd938ff9640 1 -- 172.21.15.156:0/3954315072 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd92c032c80 con 0x7fd93c0fb0f0 2024-07-16T09:47:34.331 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:47:34.075+0000 7fd941e80640 1 -- 172.21.15.156:0/3954315072 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd93c10f610 con 0x7fd93c0fb0f0 2024-07-16T09:47:34.331 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:47:34.075+0000 7fd938ff9640 1 -- 172.21.15.156:0/3954315072 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd92c03eda0 con 0x7fd93c0fb0f0 2024-07-16T09:47:34.331 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:47:34.076+0000 7fd941e80640 1 -- 172.21.15.156:0/3954315072 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd93c10fae0 con 0x7fd93c0fb0f0 2024-07-16T09:47:34.331 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:47:34.076+0000 7fd938ff9640 1 -- 172.21.15.156:0/3954315072 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 13) v1 ==== 42693+0+0 (secure 0 0 0) 0x7fd92c030020 con 0x7fd93c0fb0f0 2024-07-16T09:47:34.331 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:47:34.077+0000 7fd938ff9640 1 --2- 172.21.15.156:0/3954315072 >> [v2:172.21.15.156:6800/3984987611,v1:172.21.15.156:6801/3984987611] conn(0x7fd9280357b0 0x7fd928037c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:47:34.332 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:47:34.077+0000 7fd938ff9640 1 -- 172.21.15.156:0/3954315072 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7fd92c067a80 con 0x7fd93c0fb0f0 2024-07-16T09:47:34.332 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:47:34.077+0000 7fd941e80640 1 -- 172.21.15.156:0/3954315072 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd93c05eb80 con 0x7fd93c0fb0f0 2024-07-16T09:47:34.332 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:47:34.077+0000 7fd933fff640 1 --2- 172.21.15.156:0/3954315072 >> [v2:172.21.15.156:6800/3984987611,v1:172.21.15.156:6801/3984987611] conn(0x7fd9280357b0 0x7fd928037c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:47:34.332 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:47:34.078+0000 7fd933fff640 1 --2- 172.21.15.156:0/3954315072 >> [v2:172.21.15.156:6800/3984987611,v1:172.21.15.156:6801/3984987611] conn(0x7fd9280357b0 0x7fd928037c60 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fd924006fd0 tx=0x7fd924006db0 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:47:34.332 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:47:34.082+0000 7fd938ff9640 1 -- 172.21.15.156:0/3954315072 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fd92c03c510 con 0x7fd93c0fb0f0 2024-07-16T09:47:34.332 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:47:34.255+0000 7fd941e80640 1 -- 172.21.15.156:0/3954315072 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) v1 -- 0x7fd93c04b530 con 0x7fd93c0fb0f0 2024-07-16T09:47:34.332 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:47:34.258+0000 7fd938ff9640 1 -- 172.21.15.156:0/3954315072 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) v1 ==== 125+0+0 (secure 0 0 0) 0x7fd92c03c330 con 0x7fd93c0fb0f0 2024-07-16T09:47:34.332 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:47:34.260+0000 7fd941e80640 1 -- 172.21.15.156:0/3954315072 >> [v2:172.21.15.156:6800/3984987611,v1:172.21.15.156:6801/3984987611] conn(0x7fd9280357b0 msgr2=0x7fd928037c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:47:34.332 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:47:34.260+0000 7fd941e80640 1 --2- 172.21.15.156:0/3954315072 >> [v2:172.21.15.156:6800/3984987611,v1:172.21.15.156:6801/3984987611] conn(0x7fd9280357b0 0x7fd928037c60 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fd924006fd0 tx=0x7fd924006db0 comp rx=0 tx=0).stop 2024-07-16T09:47:34.332 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:47:34.260+0000 7fd941e80640 1 -- 172.21.15.156:0/3954315072 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd93c0fb0f0 msgr2=0x7fd93c065340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:47:34.332 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:47:34.260+0000 7fd941e80640 1 --2- 172.21.15.156:0/3954315072 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd93c0fb0f0 0x7fd93c065340 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fd92c004820 tx=0x7fd92c009da0 comp rx=0 tx=0).stop 2024-07-16T09:47:34.332 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:47:34.260+0000 7fd941e80640 1 -- 172.21.15.156:0/3954315072 shutdown_connections 2024-07-16T09:47:34.332 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:47:34.260+0000 7fd941e80640 1 --2- 172.21.15.156:0/3954315072 >> [v2:172.21.15.156:6800/3984987611,v1:172.21.15.156:6801/3984987611] conn(0x7fd9280357b0 0x7fd928037c60 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:47:34.333 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:47:34.260+0000 7fd941e80640 1 --2- 172.21.15.156:0/3954315072 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd93c0fb0f0 0x7fd93c065340 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:47:34.333 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:47:34.260+0000 7fd941e80640 1 -- 172.21.15.156:0/3954315072 >> 172.21.15.156:0/3954315072 conn(0x7fd93c0f68d0 msgr2=0x7fd93c102290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:47:34.333 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:47:34.260+0000 7fd941e80640 1 -- 172.21.15.156:0/3954315072 shutdown_connections 2024-07-16T09:47:34.333 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:47:34.260+0000 7fd941e80640 1 -- 172.21.15.156:0/3954315072 wait complete. 2024-07-16T09:47:34.901 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-07-16T09:47:34.901 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 26c91c26-4358-11ef-bcad-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-07-16T09:47:35.485 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/26c91c26-4358-11ef-bcad-c7b262605968/mon.smithi156/config 2024-07-16T09:47:35.609 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:47:35 smithi156 ceph-mon[29302]: from='client.? 172.21.15.156:0/3954315072' entity='client.admin' 2024-07-16T09:47:36.763 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:47:36 smithi156 ceph-mon[29302]: from='mgr.14160 172.21.15.156:0/3789331119' entity='mgr.smithi156.zkvylv' 2024-07-16T09:47:36.763 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:47:36 smithi156 ceph-mon[29302]: from='mgr.14160 172.21.15.156:0/3789331119' entity='mgr.smithi156.zkvylv' 2024-07-16T09:47:36.763 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:47:36 smithi156 ceph-mon[29302]: from='mgr.14160 172.21.15.156:0/3789331119' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config rm", "who": "osd/host:smithi156", "name": "osd_memory_target"}]: dispatch 2024-07-16T09:47:36.763 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:47:36 smithi156 ceph-mon[29302]: from='mgr.14160 172.21.15.156:0/3789331119' entity='mgr.smithi156.zkvylv' 2024-07-16T09:47:36.763 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:47:36 smithi156 ceph-mon[29302]: from='mgr.14160 172.21.15.156:0/3789331119' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi156", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-07-16T09:47:36.763 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:47:36 smithi156 ceph-mon[29302]: from='mgr.14160 172.21.15.156:0/3789331119' entity='mgr.smithi156.zkvylv' cmd='[{"prefix": "auth get-or-create", "entity": "client.crash.smithi156", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]': finished 2024-07-16T09:47:36.763 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:47:36 smithi156 ceph-mon[29302]: from='mgr.14160 172.21.15.156:0/3789331119' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-16T09:47:36.763 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:47:36 smithi156 ceph-mon[29302]: Deploying daemon crash.smithi156 on smithi156 2024-07-16T09:47:37.491 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:47:37.488+0000 7f95678b4640 1 -- 172.21.15.156:0/2523277234 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f95600fd630 msgr2=0x7f95600fda40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:47:37.491 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:47:37.488+0000 7f95678b4640 1 --2- 172.21.15.156:0/2523277234 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f95600fd630 0x7f95600fda40 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f955c004850 tx=0x7f955c009da0 comp rx=0 tx=0).stop 2024-07-16T09:47:37.491 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:47:37.488+0000 7f95678b4640 1 -- 172.21.15.156:0/2523277234 shutdown_connections 2024-07-16T09:47:37.491 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:47:37.488+0000 7f95678b4640 1 --2- 172.21.15.156:0/2523277234 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f95600fd630 0x7f95600fda40 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:47:37.491 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:47:37.488+0000 7f95678b4640 1 -- 172.21.15.156:0/2523277234 >> 172.21.15.156:0/2523277234 conn(0x7f95600f5e50 msgr2=0x7f95600f8260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:47:37.492 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:47:37.488+0000 7f95678b4640 1 -- 172.21.15.156:0/2523277234 shutdown_connections 2024-07-16T09:47:37.492 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:47:37.489+0000 7f95678b4640 1 -- 172.21.15.156:0/2523277234 wait complete. 2024-07-16T09:47:37.492 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:47:37.489+0000 7f95678b4640 1 Processor -- start 2024-07-16T09:47:37.492 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:47:37.489+0000 7f95678b4640 1 -- start start 2024-07-16T09:47:37.493 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:47:37.489+0000 7f95678b4640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f95600fd630 0x7f95600fcd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:47:37.494 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:47:37.489+0000 7f95678b4640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95600fd280 con 0x7f95600fd630 2024-07-16T09:47:37.494 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:47:37.489+0000 7f9565650640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f95600fd630 0x7f95600fcd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:47:37.494 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:47:37.489+0000 7f9565650640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f95600fd630 0x7f95600fcd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:49190/0 (socket says 172.21.15.156:49190) 2024-07-16T09:47:37.494 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:47:37.489+0000 7f9565650640 1 -- 172.21.15.156:0/3403542451 learned_addr learned my addr 172.21.15.156:0/3403542451 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T09:47:37.494 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:47:37.490+0000 7f9565650640 1 -- 172.21.15.156:0/3403542451 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f955c005040 con 0x7f95600fd630 2024-07-16T09:47:37.494 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:47:37.490+0000 7f9565650640 1 --2- 172.21.15.156:0/3403542451 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f95600fd630 0x7f95600fcd40 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f955c004820 tx=0x7f955c009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:47:37.494 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:47:37.490+0000 7f9555ffb640 1 -- 172.21.15.156:0/3403542451 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f955c032650 con 0x7f95600fd630 2024-07-16T09:47:37.494 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:47:37.490+0000 7f9555ffb640 1 -- 172.21.15.156:0/3403542451 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f955c032c80 con 0x7f95600fd630 2024-07-16T09:47:37.494 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:47:37.490+0000 7f95678b4640 1 -- 172.21.15.156:0/3403542451 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95600fb310 con 0x7f95600fd630 2024-07-16T09:47:37.494 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:47:37.490+0000 7f9555ffb640 1 -- 172.21.15.156:0/3403542451 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f955c03eda0 con 0x7f95600fd630 2024-07-16T09:47:37.494 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:47:37.490+0000 7f95678b4640 1 -- 172.21.15.156:0/3403542451 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95600fb7e0 con 0x7f95600fd630 2024-07-16T09:47:37.495 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:47:37.491+0000 7f9555ffb640 1 -- 172.21.15.156:0/3403542451 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 13) v1 ==== 42693+0+0 (secure 0 0 0) 0x7f955c030020 con 0x7f95600fd630 2024-07-16T09:47:37.495 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:47:37.491+0000 7f9555ffb640 1 --2- 172.21.15.156:0/3403542451 >> [v2:172.21.15.156:6800/3984987611,v1:172.21.15.156:6801/3984987611] conn(0x7f954c035800 0x7f954c037cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:47:37.495 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:47:37.491+0000 7f95678b4640 1 -- 172.21.15.156:0/3403542451 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f956005eb80 con 0x7f95600fd630 2024-07-16T09:47:37.495 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:47:37.491+0000 7f9555ffb640 1 -- 172.21.15.156:0/3403542451 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f955c067a40 con 0x7f95600fd630 2024-07-16T09:47:37.495 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:47:37.491+0000 7f9564e4f640 1 --2- 172.21.15.156:0/3403542451 >> [v2:172.21.15.156:6800/3984987611,v1:172.21.15.156:6801/3984987611] conn(0x7f954c035800 0x7f954c037cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:47:37.495 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:47:37.492+0000 7f9564e4f640 1 --2- 172.21.15.156:0/3403542451 >> [v2:172.21.15.156:6800/3984987611,v1:172.21.15.156:6801/3984987611] conn(0x7f954c035800 0x7f954c037cb0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f9550006fd0 tx=0x7f9550006db0 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:47:37.498 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:47:37.495+0000 7f9555ffb640 1 -- 172.21.15.156:0/3403542451 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f955c033420 con 0x7f95600fd630 2024-07-16T09:47:37.673 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:47:37.671+0000 7f95678b4640 1 -- 172.21.15.156:0/3403542451 --> [v2:172.21.15.156:6800/3984987611,v1:172.21.15.156:6801/3984987611] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) v1 -- 0x7f9560105aa0 con 0x7f954c035800 2024-07-16T09:47:37.676 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:47:37.674+0000 7f9555ffb640 1 -- 172.21.15.156:0/3403542451 <== mgr.14160 v2:172.21.15.156:6800/3984987611 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f9560105aa0 con 0x7f954c035800 2024-07-16T09:47:37.678 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:47:37.676+0000 7f95678b4640 1 -- 172.21.15.156:0/3403542451 >> [v2:172.21.15.156:6800/3984987611,v1:172.21.15.156:6801/3984987611] conn(0x7f954c035800 msgr2=0x7f954c037cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:47:37.678 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:47:37.676+0000 7f95678b4640 1 --2- 172.21.15.156:0/3403542451 >> [v2:172.21.15.156:6800/3984987611,v1:172.21.15.156:6801/3984987611] conn(0x7f954c035800 0x7f954c037cb0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f9550006fd0 tx=0x7f9550006db0 comp rx=0 tx=0).stop 2024-07-16T09:47:37.679 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:47:37.676+0000 7f95678b4640 1 -- 172.21.15.156:0/3403542451 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f95600fd630 msgr2=0x7f95600fcd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:47:37.679 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:47:37.676+0000 7f95678b4640 1 --2- 172.21.15.156:0/3403542451 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f95600fd630 0x7f95600fcd40 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f955c004820 tx=0x7f955c009da0 comp rx=0 tx=0).stop 2024-07-16T09:47:37.679 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:47:37.676+0000 7f95678b4640 1 -- 172.21.15.156:0/3403542451 shutdown_connections 2024-07-16T09:47:37.679 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:47:37.676+0000 7f95678b4640 1 --2- 172.21.15.156:0/3403542451 >> [v2:172.21.15.156:6800/3984987611,v1:172.21.15.156:6801/3984987611] conn(0x7f954c035800 0x7f954c037cb0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:47:37.679 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:47:37.676+0000 7f95678b4640 1 --2- 172.21.15.156:0/3403542451 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f95600fd630 0x7f95600fcd40 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:47:37.679 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:47:37.676+0000 7f95678b4640 1 -- 172.21.15.156:0/3403542451 >> 172.21.15.156:0/3403542451 conn(0x7f95600f5e50 msgr2=0x7f9560103190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:47:37.679 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:47:37.676+0000 7f95678b4640 1 -- 172.21.15.156:0/3403542451 shutdown_connections 2024-07-16T09:47:37.679 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:47:37.676+0000 7f95678b4640 1 -- 172.21.15.156:0/3403542451 wait complete. 2024-07-16T09:47:38.683 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi176 2024-07-16T09:47:38.683 DEBUG:teuthology.orchestra.run.smithi176:> set -ex 2024-07-16T09:47:38.683 DEBUG:teuthology.orchestra.run.smithi176:> dd of=/etc/ceph/ceph.conf 2024-07-16T09:47:38.702 DEBUG:teuthology.orchestra.run.smithi176:> set -ex 2024-07-16T09:47:38.702 DEBUG:teuthology.orchestra.run.smithi176:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-07-16T09:47:38.760 INFO:tasks.cephadm:Adding host smithi176 to orchestrator... 2024-07-16T09:47:38.760 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 26c91c26-4358-11ef-bcad-c7b262605968 -- ceph orch host add smithi176 2024-07-16T09:47:38.924 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:47:38 smithi156 ceph-mon[29302]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-07-16T09:47:38.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:47:38 smithi156 ceph-mon[29302]: from='mgr.14160 172.21.15.156:0/3789331119' entity='mgr.smithi156.zkvylv' 2024-07-16T09:47:39.287 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/26c91c26-4358-11ef-bcad-c7b262605968/mon.smithi156/config 2024-07-16T09:47:40.500 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:47:40 smithi156 ceph-mon[29302]: from='mgr.14160 172.21.15.156:0/3789331119' entity='mgr.smithi156.zkvylv' 2024-07-16T09:47:40.501 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:47:40 smithi156 ceph-mon[29302]: from='mgr.14160 172.21.15.156:0/3789331119' entity='mgr.smithi156.zkvylv' 2024-07-16T09:47:40.501 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:47:40 smithi156 ceph-mon[29302]: from='mgr.14160 172.21.15.156:0/3789331119' entity='mgr.smithi156.zkvylv' 2024-07-16T09:47:40.501 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:47:40 smithi156 ceph-mon[29302]: from='mgr.14160 172.21.15.156:0/3789331119' entity='mgr.smithi156.zkvylv' 2024-07-16T09:47:40.501 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:47:40 smithi156 ceph-mon[29302]: Deploying daemon node-exporter.smithi156 on smithi156 2024-07-16T09:47:40.690 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:47:40.686+0000 7feb57202640 1 -- 172.21.15.156:0/4161179020 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7feb500fbfe0 msgr2=0x7feb500fc3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:47:40.690 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:47:40.686+0000 7feb57202640 1 --2- 172.21.15.156:0/4161179020 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7feb500fbfe0 0x7feb500fc3f0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7feb44004850 tx=0x7feb44009da0 comp rx=0 tx=0).stop 2024-07-16T09:47:40.690 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:47:40.686+0000 7feb57202640 1 -- 172.21.15.156:0/4161179020 shutdown_connections 2024-07-16T09:47:40.690 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:47:40.686+0000 7feb57202640 1 --2- 172.21.15.156:0/4161179020 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7feb500fbfe0 0x7feb500fc3f0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:47:40.690 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:47:40.686+0000 7feb57202640 1 -- 172.21.15.156:0/4161179020 >> 172.21.15.156:0/4161179020 conn(0x7feb500f77c0 msgr2=0x7feb500f9bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:47:40.690 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:47:40.686+0000 7feb57202640 1 -- 172.21.15.156:0/4161179020 shutdown_connections 2024-07-16T09:47:40.690 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:47:40.686+0000 7feb57202640 1 -- 172.21.15.156:0/4161179020 wait complete. 2024-07-16T09:47:40.690 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:47:40.686+0000 7feb57202640 1 Processor -- start 2024-07-16T09:47:40.691 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:47:40.686+0000 7feb57202640 1 -- start start 2024-07-16T09:47:40.691 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:47:40.687+0000 7feb57202640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7feb500fbfe0 0x7feb500712a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:47:40.691 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:47:40.687+0000 7feb57202640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb500717e0 con 0x7feb500fbfe0 2024-07-16T09:47:40.691 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:47:40.687+0000 7feb54f9e640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7feb500fbfe0 0x7feb500712a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:47:40.691 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:47:40.687+0000 7feb54f9e640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7feb500fbfe0 0x7feb500712a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:49224/0 (socket says 172.21.15.156:49224) 2024-07-16T09:47:40.691 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:47:40.687+0000 7feb54f9e640 1 -- 172.21.15.156:0/2353408700 learned_addr learned my addr 172.21.15.156:0/2353408700 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T09:47:40.691 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:47:40.687+0000 7feb54f9e640 1 -- 172.21.15.156:0/2353408700 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb44005040 con 0x7feb500fbfe0 2024-07-16T09:47:40.691 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:47:40.687+0000 7feb54f9e640 1 --2- 172.21.15.156:0/2353408700 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7feb500fbfe0 0x7feb500712a0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7feb44004820 tx=0x7feb44009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:47:40.691 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:47:40.687+0000 7feb4d7fa640 1 -- 172.21.15.156:0/2353408700 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7feb44032650 con 0x7feb500fbfe0 2024-07-16T09:47:40.691 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:47:40.688+0000 7feb4d7fa640 1 -- 172.21.15.156:0/2353408700 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7feb44032c80 con 0x7feb500fbfe0 2024-07-16T09:47:40.691 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:47:40.688+0000 7feb57202640 1 -- 172.21.15.156:0/2353408700 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb5006f870 con 0x7feb500fbfe0 2024-07-16T09:47:40.691 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:47:40.688+0000 7feb4d7fa640 1 -- 172.21.15.156:0/2353408700 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7feb4403eda0 con 0x7feb500fbfe0 2024-07-16T09:47:40.691 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:47:40.688+0000 7feb57202640 1 -- 172.21.15.156:0/2353408700 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb5006fd40 con 0x7feb500fbfe0 2024-07-16T09:47:40.692 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:47:40.688+0000 7feb4d7fa640 1 -- 172.21.15.156:0/2353408700 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 13) v1 ==== 42693+0+0 (secure 0 0 0) 0x7feb44030020 con 0x7feb500fbfe0 2024-07-16T09:47:40.692 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:47:40.689+0000 7feb57202640 1 -- 172.21.15.156:0/2353408700 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb5005eb80 con 0x7feb500fbfe0 2024-07-16T09:47:40.692 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:47:40.689+0000 7feb4d7fa640 1 --2- 172.21.15.156:0/2353408700 >> [v2:172.21.15.156:6800/3984987611,v1:172.21.15.156:6801/3984987611] conn(0x7feb400357b0 0x7feb40037c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:47:40.693 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:47:40.689+0000 7feb4d7fa640 1 -- 172.21.15.156:0/2353408700 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7feb44067b30 con 0x7feb500fbfe0 2024-07-16T09:47:40.693 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:47:40.689+0000 7feb4ffff640 1 --2- 172.21.15.156:0/2353408700 >> [v2:172.21.15.156:6800/3984987611,v1:172.21.15.156:6801/3984987611] conn(0x7feb400357b0 0x7feb40037c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:47:40.693 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:47:40.689+0000 7feb4ffff640 1 --2- 172.21.15.156:0/2353408700 >> [v2:172.21.15.156:6800/3984987611,v1:172.21.15.156:6801/3984987611] conn(0x7feb400357b0 0x7feb40037c60 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7feb3c006fd0 tx=0x7feb3c006db0 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:47:40.695 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:47:40.693+0000 7feb4d7fa640 1 -- 172.21.15.156:0/2353408700 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7feb4403c920 con 0x7feb500fbfe0 2024-07-16T09:47:40.880 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:47:40.877+0000 7feb57202640 1 -- 172.21.15.156:0/2353408700 --> [v2:172.21.15.156:6800/3984987611,v1:172.21.15.156:6801/3984987611] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi176", "target": ["mon-mgr", ""]}) v1 -- 0x7feb50105b30 con 0x7feb400357b0 2024-07-16T09:47:41.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:47:41 smithi156 ceph-mon[29302]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi176", "target": ["mon-mgr", ""]}]: dispatch 2024-07-16T09:47:42.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:47:42 smithi156 ceph-mon[29302]: Deploying cephadm binary to smithi176 2024-07-16T09:47:43.804 INFO:teuthology.orchestra.run.smithi156.stdout:Added host 'smithi176' with addr '172.21.15.176' 2024-07-16T09:47:43.804 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:47:43.801+0000 7feb4d7fa640 1 -- 172.21.15.156:0/2353408700 <== mgr.14160 v2:172.21.15.156:6800/3984987611 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7feb50105b30 con 0x7feb400357b0 2024-07-16T09:47:43.805 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:47:43.803+0000 7feb57202640 1 -- 172.21.15.156:0/2353408700 >> [v2:172.21.15.156:6800/3984987611,v1:172.21.15.156:6801/3984987611] conn(0x7feb400357b0 msgr2=0x7feb40037c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:47:43.805 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:47:43.803+0000 7feb57202640 1 --2- 172.21.15.156:0/2353408700 >> [v2:172.21.15.156:6800/3984987611,v1:172.21.15.156:6801/3984987611] conn(0x7feb400357b0 0x7feb40037c60 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7feb3c006fd0 tx=0x7feb3c006db0 comp rx=0 tx=0).stop 2024-07-16T09:47:43.805 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:47:43.803+0000 7feb57202640 1 -- 172.21.15.156:0/2353408700 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7feb500fbfe0 msgr2=0x7feb500712a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:47:43.805 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:47:43.803+0000 7feb57202640 1 --2- 172.21.15.156:0/2353408700 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7feb500fbfe0 0x7feb500712a0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7feb44004820 tx=0x7feb44009da0 comp rx=0 tx=0).stop 2024-07-16T09:47:43.805 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:47:43.803+0000 7feb57202640 1 -- 172.21.15.156:0/2353408700 shutdown_connections 2024-07-16T09:47:43.805 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:47:43.803+0000 7feb57202640 1 --2- 172.21.15.156:0/2353408700 >> [v2:172.21.15.156:6800/3984987611,v1:172.21.15.156:6801/3984987611] conn(0x7feb400357b0 0x7feb40037c60 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:47:43.805 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:47:43.803+0000 7feb57202640 1 --2- 172.21.15.156:0/2353408700 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7feb500fbfe0 0x7feb500712a0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:47:43.805 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:47:43.803+0000 7feb57202640 1 -- 172.21.15.156:0/2353408700 >> 172.21.15.156:0/2353408700 conn(0x7feb500f77c0 msgr2=0x7feb50103180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:47:43.805 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:47:43.803+0000 7feb57202640 1 -- 172.21.15.156:0/2353408700 shutdown_connections 2024-07-16T09:47:43.805 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:47:43.803+0000 7feb57202640 1 -- 172.21.15.156:0/2353408700 wait complete. 2024-07-16T09:47:43.924 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:47:43 smithi156 ceph-mon[29302]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-16T09:47:43.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:47:43 smithi156 ceph-mon[29302]: from='mgr.14160 172.21.15.156:0/3789331119' entity='mgr.smithi156.zkvylv' 2024-07-16T09:47:44.715 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 26c91c26-4358-11ef-bcad-c7b262605968 -- ceph orch host ls --format=json 2024-07-16T09:47:44.999 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:47:44 smithi156 ceph-mon[29302]: from='mgr.14160 172.21.15.156:0/3789331119' entity='mgr.smithi156.zkvylv' 2024-07-16T09:47:44.999 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:47:44 smithi156 ceph-mon[29302]: Added host smithi176 2024-07-16T09:47:45.232 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/26c91c26-4358-11ef-bcad-c7b262605968/mon.smithi156/config 2024-07-16T09:47:46.115 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:47:45 smithi156 ceph-mon[29302]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-16T09:47:46.115 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:47:45 smithi156 ceph-mon[29302]: from='mgr.14160 172.21.15.156:0/3789331119' entity='mgr.smithi156.zkvylv' 2024-07-16T09:47:46.115 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:47:45 smithi156 ceph-mon[29302]: from='mgr.14160 172.21.15.156:0/3789331119' entity='mgr.smithi156.zkvylv' 2024-07-16T09:47:46.115 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:47:45 smithi156 ceph-mon[29302]: from='mgr.14160 172.21.15.156:0/3789331119' entity='mgr.smithi156.zkvylv' 2024-07-16T09:47:46.115 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:47:45 smithi156 ceph-mon[29302]: from='mgr.14160 172.21.15.156:0/3789331119' entity='mgr.smithi156.zkvylv' 2024-07-16T09:47:46.835 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:47:46.830+0000 7fbf485d2640 1 -- 172.21.15.156:0/3676231451 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbf400653c0 msgr2=0x7fbf400feea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:47:46.835 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:47:46.830+0000 7fbf485d2640 1 --2- 172.21.15.156:0/3676231451 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbf400653c0 0x7fbf400feea0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fbf30009bd0 tx=0x7fbf30004d10 comp rx=0 tx=0).stop 2024-07-16T09:47:46.836 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:47:46.830+0000 7fbf485d2640 1 -- 172.21.15.156:0/3676231451 shutdown_connections 2024-07-16T09:47:46.836 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:47:46.830+0000 7fbf485d2640 1 --2- 172.21.15.156:0/3676231451 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbf400653c0 0x7fbf400feea0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:47:46.836 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:47:46.830+0000 7fbf485d2640 1 -- 172.21.15.156:0/3676231451 >> 172.21.15.156:0/3676231451 conn(0x7fbf400f68d0 msgr2=0x7fbf400f8ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:47:46.836 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:47:46.830+0000 7fbf485d2640 1 -- 172.21.15.156:0/3676231451 shutdown_connections 2024-07-16T09:47:46.836 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:47:46.830+0000 7fbf485d2640 1 -- 172.21.15.156:0/3676231451 wait complete. 2024-07-16T09:47:46.836 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:47:46.830+0000 7fbf485d2640 1 Processor -- start 2024-07-16T09:47:46.836 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:47:46.830+0000 7fbf485d2640 1 -- start start 2024-07-16T09:47:46.836 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:47:46.830+0000 7fbf485d2640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbf400653c0 0x7fbf4010c880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:47:46.836 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:47:46.830+0000 7fbf485d2640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf4010cdc0 con 0x7fbf400653c0 2024-07-16T09:47:46.836 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:47:46.830+0000 7fbf4636e640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbf400653c0 0x7fbf4010c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:47:46.836 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:47:46.830+0000 7fbf4636e640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbf400653c0 0x7fbf4010c880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:53564/0 (socket says 172.21.15.156:53564) 2024-07-16T09:47:46.836 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:47:46.830+0000 7fbf4636e640 1 -- 172.21.15.156:0/1518554955 learned_addr learned my addr 172.21.15.156:0/1518554955 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T09:47:46.836 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:47:46.831+0000 7fbf4636e640 1 -- 172.21.15.156:0/1518554955 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf300098b0 con 0x7fbf400653c0 2024-07-16T09:47:46.837 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:47:46.831+0000 7fbf4636e640 1 --2- 172.21.15.156:0/1518554955 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbf400653c0 0x7fbf4010c880 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fbf30009ba0 tx=0x7fbf30004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:47:46.837 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:47:46.831+0000 7fbf3effd640 1 -- 172.21.15.156:0/1518554955 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbf300329b0 con 0x7fbf400653c0 2024-07-16T09:47:46.837 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:47:46.831+0000 7fbf485d2640 1 -- 172.21.15.156:0/1518554955 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf4010cfc0 con 0x7fbf400653c0 2024-07-16T09:47:46.837 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:47:46.831+0000 7fbf3effd640 1 -- 172.21.15.156:0/1518554955 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbf3003d420 con 0x7fbf400653c0 2024-07-16T09:47:46.837 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:47:46.831+0000 7fbf3effd640 1 -- 172.21.15.156:0/1518554955 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbf30031590 con 0x7fbf400653c0 2024-07-16T09:47:46.837 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:47:46.831+0000 7fbf485d2640 1 -- 172.21.15.156:0/1518554955 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf4010d410 con 0x7fbf400653c0 2024-07-16T09:47:46.837 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:47:46.832+0000 7fbf3effd640 1 -- 172.21.15.156:0/1518554955 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 13) v1 ==== 42693+0+0 (secure 0 0 0) 0x7fbf3002f030 con 0x7fbf400653c0 2024-07-16T09:47:46.837 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:47:46.832+0000 7fbf485d2640 1 -- 172.21.15.156:0/1518554955 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf4005eb80 con 0x7fbf400653c0 2024-07-16T09:47:46.837 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:47:46.832+0000 7fbf3effd640 1 --2- 172.21.15.156:0/1518554955 >> [v2:172.21.15.156:6800/3984987611,v1:172.21.15.156:6801/3984987611] conn(0x7fbf28035760 0x7fbf28037c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:47:46.837 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:47:46.832+0000 7fbf3effd640 1 -- 172.21.15.156:0/1518554955 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7fbf30068e30 con 0x7fbf400653c0 2024-07-16T09:47:46.837 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:47:46.832+0000 7fbf45b6d640 1 --2- 172.21.15.156:0/1518554955 >> [v2:172.21.15.156:6800/3984987611,v1:172.21.15.156:6801/3984987611] conn(0x7fbf28035760 0x7fbf28037c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:47:46.837 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:47:46.832+0000 7fbf45b6d640 1 --2- 172.21.15.156:0/1518554955 >> [v2:172.21.15.156:6800/3984987611,v1:172.21.15.156:6801/3984987611] conn(0x7fbf28035760 0x7fbf28037c10 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fbf34006fd0 tx=0x7fbf34006db0 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:47:46.838 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:47:46.837+0000 7fbf3effd640 1 -- 172.21.15.156:0/1518554955 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fbf300307f0 con 0x7fbf400653c0 2024-07-16T09:47:46.924 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:47:46 smithi156 ceph-mon[29302]: Deploying daemon alertmanager.smithi156 on smithi156 2024-07-16T09:47:47.017 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:47:47.015+0000 7fbf485d2640 1 -- 172.21.15.156:0/1518554955 --> [v2:172.21.15.156:6800/3984987611,v1:172.21.15.156:6801/3984987611] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fbf4010db50 con 0x7fbf28035760 2024-07-16T09:47:47.018 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:47:47.017+0000 7fbf3effd640 1 -- 172.21.15.156:0/1518554955 <== mgr.14160 v2:172.21.15.156:6800/3984987611 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+161 (secure 0 0 0) 0x7fbf4010db50 con 0x7fbf28035760 2024-07-16T09:47:47.018 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-07-16T09:47:47.018 INFO:teuthology.orchestra.run.smithi156.stdout:[{"addr": "172.21.15.156", "hostname": "smithi156", "labels": [], "status": ""}, {"addr": "172.21.15.176", "hostname": "smithi176", "labels": [], "status": ""}] 2024-07-16T09:47:47.020 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:47:47.019+0000 7fbf485d2640 1 -- 172.21.15.156:0/1518554955 >> [v2:172.21.15.156:6800/3984987611,v1:172.21.15.156:6801/3984987611] conn(0x7fbf28035760 msgr2=0x7fbf28037c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:47:47.020 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:47:47.019+0000 7fbf485d2640 1 --2- 172.21.15.156:0/1518554955 >> [v2:172.21.15.156:6800/3984987611,v1:172.21.15.156:6801/3984987611] conn(0x7fbf28035760 0x7fbf28037c10 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fbf34006fd0 tx=0x7fbf34006db0 comp rx=0 tx=0).stop 2024-07-16T09:47:47.020 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:47:47.019+0000 7fbf485d2640 1 -- 172.21.15.156:0/1518554955 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbf400653c0 msgr2=0x7fbf4010c880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:47:47.020 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:47:47.019+0000 7fbf485d2640 1 --2- 172.21.15.156:0/1518554955 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbf400653c0 0x7fbf4010c880 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fbf30009ba0 tx=0x7fbf30004d10 comp rx=0 tx=0).stop 2024-07-16T09:47:47.020 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:47:47.019+0000 7fbf485d2640 1 -- 172.21.15.156:0/1518554955 shutdown_connections 2024-07-16T09:47:47.020 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:47:47.019+0000 7fbf485d2640 1 --2- 172.21.15.156:0/1518554955 >> [v2:172.21.15.156:6800/3984987611,v1:172.21.15.156:6801/3984987611] conn(0x7fbf28035760 0x7fbf28037c10 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:47:47.020 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:47:47.019+0000 7fbf485d2640 1 --2- 172.21.15.156:0/1518554955 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbf400653c0 0x7fbf4010c880 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:47:47.021 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:47:47.019+0000 7fbf485d2640 1 -- 172.21.15.156:0/1518554955 >> 172.21.15.156:0/1518554955 conn(0x7fbf400f68d0 msgr2=0x7fbf40103390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:47:47.021 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:47:47.019+0000 7fbf485d2640 1 -- 172.21.15.156:0/1518554955 shutdown_connections 2024-07-16T09:47:47.021 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:47:47.019+0000 7fbf485d2640 1 -- 172.21.15.156:0/1518554955 wait complete. 2024-07-16T09:47:47.884 INFO:tasks.cephadm:Setting crush tunables to default 2024-07-16T09:47:47.885 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 26c91c26-4358-11ef-bcad-c7b262605968 -- ceph osd crush tunables default 2024-07-16T09:47:48.139 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/26c91c26-4358-11ef-bcad-c7b262605968/mon.smithi156/config 2024-07-16T09:47:48.175 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:47:47 smithi156 ceph-mon[29302]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-16T09:47:48.175 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:47:47 smithi156 ceph-mon[29302]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-07-16T09:47:48.175 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:47:47 smithi156 ceph-mon[29302]: from='mgr.14160 172.21.15.156:0/3789331119' entity='mgr.smithi156.zkvylv' 2024-07-16T09:47:49.924 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:47:49 smithi156 ceph-mon[29302]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-16T09:47:50.949 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:47:50.877+0000 7fdb344f5640 1 -- 172.21.15.156:0/2075025702 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdb2c0fb0f0 msgr2=0x7fdb2c0fb500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:47:50.949 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:47:50.877+0000 7fdb344f5640 1 --2- 172.21.15.156:0/2075025702 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdb2c0fb0f0 0x7fdb2c0fb500 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fdb28004850 tx=0x7fdb28009da0 comp rx=0 tx=0).stop 2024-07-16T09:47:50.950 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:47:50.877+0000 7fdb344f5640 1 -- 172.21.15.156:0/2075025702 shutdown_connections 2024-07-16T09:47:50.950 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:47:50.877+0000 7fdb344f5640 1 --2- 172.21.15.156:0/2075025702 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdb2c0fb0f0 0x7fdb2c0fb500 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:47:50.950 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:47:50.877+0000 7fdb344f5640 1 -- 172.21.15.156:0/2075025702 >> 172.21.15.156:0/2075025702 conn(0x7fdb2c0f68d0 msgr2=0x7fdb2c0f8ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:47:50.950 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:47:50.877+0000 7fdb344f5640 1 -- 172.21.15.156:0/2075025702 shutdown_connections 2024-07-16T09:47:50.950 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:47:50.877+0000 7fdb344f5640 1 -- 172.21.15.156:0/2075025702 wait complete. 2024-07-16T09:47:50.950 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:47:50.877+0000 7fdb344f5640 1 Processor -- start 2024-07-16T09:47:50.950 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:47:50.877+0000 7fdb344f5640 1 -- start start 2024-07-16T09:47:50.950 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:47:50.877+0000 7fdb344f5640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdb2c0fb0f0 0x7fdb2c10c810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:47:50.950 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:47:50.877+0000 7fdb344f5640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb2c10cd50 con 0x7fdb2c0fb0f0 2024-07-16T09:47:50.950 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:47:50.878+0000 7fdb32291640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdb2c0fb0f0 0x7fdb2c10c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:47:50.950 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:47:50.878+0000 7fdb32291640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdb2c0fb0f0 0x7fdb2c10c810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:53580/0 (socket says 172.21.15.156:53580) 2024-07-16T09:47:50.951 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:47:50.878+0000 7fdb32291640 1 -- 172.21.15.156:0/1313131187 learned_addr learned my addr 172.21.15.156:0/1313131187 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T09:47:50.951 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:47:50.878+0000 7fdb32291640 1 -- 172.21.15.156:0/1313131187 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb28005040 con 0x7fdb2c0fb0f0 2024-07-16T09:47:50.951 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:47:50.878+0000 7fdb32291640 1 --2- 172.21.15.156:0/1313131187 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdb2c0fb0f0 0x7fdb2c10c810 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fdb28004820 tx=0x7fdb28009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:47:50.951 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:47:50.878+0000 7fdb22ffd640 1 -- 172.21.15.156:0/1313131187 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fdb28032650 con 0x7fdb2c0fb0f0 2024-07-16T09:47:50.951 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:47:50.878+0000 7fdb22ffd640 1 -- 172.21.15.156:0/1313131187 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdb28032c80 con 0x7fdb2c0fb0f0 2024-07-16T09:47:50.951 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:47:50.878+0000 7fdb344f5640 1 -- 172.21.15.156:0/1313131187 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb2c10cf50 con 0x7fdb2c0fb0f0 2024-07-16T09:47:50.951 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:47:50.878+0000 7fdb22ffd640 1 -- 172.21.15.156:0/1313131187 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fdb2803eda0 con 0x7fdb2c0fb0f0 2024-07-16T09:47:50.951 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:47:50.878+0000 7fdb344f5640 1 -- 172.21.15.156:0/1313131187 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb2c10dae0 con 0x7fdb2c0fb0f0 2024-07-16T09:47:50.951 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:47:50.879+0000 7fdb22ffd640 1 -- 172.21.15.156:0/1313131187 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 13) v1 ==== 42693+0+0 (secure 0 0 0) 0x7fdb28030020 con 0x7fdb2c0fb0f0 2024-07-16T09:47:50.951 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:47:50.879+0000 7fdb22ffd640 1 --2- 172.21.15.156:0/1313131187 >> [v2:172.21.15.156:6800/3984987611,v1:172.21.15.156:6801/3984987611] conn(0x7fdb180357b0 0x7fdb18037c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:47:50.951 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:47:50.879+0000 7fdb344f5640 1 -- 172.21.15.156:0/1313131187 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb2c101660 con 0x7fdb2c0fb0f0 2024-07-16T09:47:50.951 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:47:50.879+0000 7fdb22ffd640 1 -- 172.21.15.156:0/1313131187 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7fdb28067c40 con 0x7fdb2c0fb0f0 2024-07-16T09:47:50.951 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:47:50.879+0000 7fdb31a90640 1 --2- 172.21.15.156:0/1313131187 >> [v2:172.21.15.156:6800/3984987611,v1:172.21.15.156:6801/3984987611] conn(0x7fdb180357b0 0x7fdb18037c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:47:50.952 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:47:50.880+0000 7fdb31a90640 1 --2- 172.21.15.156:0/1313131187 >> [v2:172.21.15.156:6800/3984987611,v1:172.21.15.156:6801/3984987611] conn(0x7fdb180357b0 0x7fdb18037c60 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fdb1c006fd0 tx=0x7fdb1c006db0 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:47:50.952 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:47:50.884+0000 7fdb22ffd640 1 -- 172.21.15.156:0/1313131187 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fdb28033420 con 0x7fdb2c0fb0f0 2024-07-16T09:47:51.059 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:47:51.058+0000 7fdb344f5640 1 -- 172.21.15.156:0/1313131187 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7fdb2c04b530 con 0x7fdb2c0fb0f0 2024-07-16T09:47:51.812 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:47:51.810+0000 7fdb22ffd640 1 -- 172.21.15.156:0/1313131187 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) v1 ==== 124+0+0 (secure 0 0 0) 0x7fdb2803c330 con 0x7fdb2c0fb0f0 2024-07-16T09:47:51.812 INFO:teuthology.orchestra.run.smithi156.stderr:adjusted tunables profile to default 2024-07-16T09:47:51.814 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:47:51.812+0000 7fdb344f5640 1 -- 172.21.15.156:0/1313131187 >> [v2:172.21.15.156:6800/3984987611,v1:172.21.15.156:6801/3984987611] conn(0x7fdb180357b0 msgr2=0x7fdb18037c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:47:51.814 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:47:51.812+0000 7fdb344f5640 1 --2- 172.21.15.156:0/1313131187 >> [v2:172.21.15.156:6800/3984987611,v1:172.21.15.156:6801/3984987611] conn(0x7fdb180357b0 0x7fdb18037c60 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fdb1c006fd0 tx=0x7fdb1c006db0 comp rx=0 tx=0).stop 2024-07-16T09:47:51.814 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:47:51.812+0000 7fdb344f5640 1 -- 172.21.15.156:0/1313131187 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdb2c0fb0f0 msgr2=0x7fdb2c10c810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:47:51.814 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:47:51.812+0000 7fdb344f5640 1 --2- 172.21.15.156:0/1313131187 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdb2c0fb0f0 0x7fdb2c10c810 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fdb28004820 tx=0x7fdb28009da0 comp rx=0 tx=0).stop 2024-07-16T09:47:51.814 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:47:51.813+0000 7fdb344f5640 1 -- 172.21.15.156:0/1313131187 shutdown_connections 2024-07-16T09:47:51.814 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:47:51.813+0000 7fdb344f5640 1 --2- 172.21.15.156:0/1313131187 >> [v2:172.21.15.156:6800/3984987611,v1:172.21.15.156:6801/3984987611] conn(0x7fdb180357b0 0x7fdb18037c60 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:47:51.814 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:47:51.813+0000 7fdb344f5640 1 --2- 172.21.15.156:0/1313131187 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdb2c0fb0f0 0x7fdb2c10c810 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:47:51.815 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:47:51.813+0000 7fdb344f5640 1 -- 172.21.15.156:0/1313131187 >> 172.21.15.156:0/1313131187 conn(0x7fdb2c0f68d0 msgr2=0x7fdb2c0f8cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:47:51.815 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:47:51.813+0000 7fdb344f5640 1 -- 172.21.15.156:0/1313131187 shutdown_connections 2024-07-16T09:47:51.815 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:47:51.813+0000 7fdb344f5640 1 -- 172.21.15.156:0/1313131187 wait complete. 2024-07-16T09:47:52.153 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:47:51 smithi156 ceph-mon[29302]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-16T09:47:52.153 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:47:51 smithi156 ceph-mon[29302]: from='client.? 172.21.15.156:0/1313131187' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-07-16T09:47:52.743 INFO:tasks.cephadm:Adding mon.smithi156 on smithi156 2024-07-16T09:47:52.743 INFO:tasks.cephadm:Adding mon.smithi176 on smithi176 2024-07-16T09:47:52.744 DEBUG:teuthology.orchestra.run.smithi176:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 26c91c26-4358-11ef-bcad-c7b262605968 -- ceph orch apply mon '2;smithi156:172.21.15.156=smithi156;smithi176:172.21.15.176=smithi176' 2024-07-16T09:47:52.994 INFO:teuthology.orchestra.run.smithi176.stderr:Inferring config /etc/ceph/ceph.conf 2024-07-16T09:47:53.042 INFO:teuthology.orchestra.run.smithi176.stderr:Inferring config /etc/ceph/ceph.conf 2024-07-16T09:47:53.174 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:47:52 smithi156 ceph-mon[29302]: from='client.? 172.21.15.156:0/1313131187' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-07-16T09:47:53.175 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:47:52 smithi156 ceph-mon[29302]: osdmap e4: 0 total, 0 up, 0 in 2024-07-16T09:47:54.062 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:47:53 smithi156 ceph-mon[29302]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-16T09:47:54.619 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:47:54.615+0000 7f2d8f391640 1 -- 172.21.15.176:0/2118892842 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2d88065c90 msgr2=0x7f2d880fec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:47:54.620 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:47:54.615+0000 7f2d8f391640 1 --2- 172.21.15.176:0/2118892842 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2d88065c90 0x7f2d880fec40 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f2d84004850 tx=0x7f2d84009da0 comp rx=0 tx=0).stop 2024-07-16T09:47:54.620 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:47:54.615+0000 7f2d8f391640 1 -- 172.21.15.176:0/2118892842 shutdown_connections 2024-07-16T09:47:54.620 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:47:54.615+0000 7f2d8f391640 1 --2- 172.21.15.176:0/2118892842 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2d88065c90 0x7f2d880fec40 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:47:54.620 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:47:54.615+0000 7f2d8f391640 1 -- 172.21.15.176:0/2118892842 >> 172.21.15.176:0/2118892842 conn(0x7f2d880f6690 msgr2=0x7f2d880f8ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:47:54.620 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:47:54.615+0000 7f2d8f391640 1 -- 172.21.15.176:0/2118892842 shutdown_connections 2024-07-16T09:47:54.620 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:47:54.615+0000 7f2d8f391640 1 -- 172.21.15.176:0/2118892842 wait complete. 2024-07-16T09:47:54.620 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:47:54.616+0000 7f2d8f391640 1 Processor -- start 2024-07-16T09:47:54.620 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:47:54.616+0000 7f2d8f391640 1 -- start start 2024-07-16T09:47:54.620 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:47:54.616+0000 7f2d8f391640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2d88065c90 0x7f2d8810c650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:47:54.620 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:47:54.616+0000 7f2d8f391640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d8810cb90 con 0x7f2d88065c90 2024-07-16T09:47:54.621 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:47:54.616+0000 7f2d8d12d640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2d88065c90 0x7f2d8810c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:47:54.621 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:47:54.616+0000 7f2d8d12d640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2d88065c90 0x7f2d8810c650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.176:45162/0 (socket says 172.21.15.176:45162) 2024-07-16T09:47:54.621 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:47:54.616+0000 7f2d8d12d640 1 -- 172.21.15.176:0/568011770 learned_addr learned my addr 172.21.15.176:0/568011770 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-07-16T09:47:54.621 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:47:54.616+0000 7f2d8d12d640 1 -- 172.21.15.176:0/568011770 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d84005040 con 0x7f2d88065c90 2024-07-16T09:47:54.621 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:47:54.616+0000 7f2d8d12d640 1 --2- 172.21.15.176:0/568011770 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2d88065c90 0x7f2d8810c650 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f2d84003080 tx=0x7f2d84009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:47:54.621 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:47:54.617+0000 7f2d79ffb640 1 -- 172.21.15.176:0/568011770 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2d840326c0 con 0x7f2d88065c90 2024-07-16T09:47:54.621 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:47:54.617+0000 7f2d79ffb640 1 -- 172.21.15.176:0/568011770 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2d84032cf0 con 0x7f2d88065c90 2024-07-16T09:47:54.621 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:47:54.617+0000 7f2d8f391640 1 -- 172.21.15.176:0/568011770 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d8810cd90 con 0x7f2d88065c90 2024-07-16T09:47:54.621 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:47:54.617+0000 7f2d79ffb640 1 -- 172.21.15.176:0/568011770 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2d84033db0 con 0x7f2d88065c90 2024-07-16T09:47:54.621 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:47:54.617+0000 7f2d8f391640 1 -- 172.21.15.176:0/568011770 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d8810d920 con 0x7f2d88065c90 2024-07-16T09:47:54.622 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:47:54.618+0000 7f2d79ffb640 1 -- 172.21.15.176:0/568011770 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 13) v1 ==== 42693+0+0 (secure 0 0 0) 0x7f2d84030020 con 0x7f2d88065c90 2024-07-16T09:47:54.622 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:47:54.618+0000 7f2d79ffb640 1 --2- 172.21.15.176:0/568011770 >> [v2:172.21.15.156:6800/3984987611,v1:172.21.15.156:6801/3984987611] conn(0x7f2d740357b0 0x7f2d74037c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:47:54.622 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:47:54.618+0000 7f2d8f391640 1 -- 172.21.15.176:0/568011770 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d88102550 con 0x7f2d88065c90 2024-07-16T09:47:54.623 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:47:54.618+0000 7f2d79ffb640 1 -- 172.21.15.176:0/568011770 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f2d84067a60 con 0x7f2d88065c90 2024-07-16T09:47:54.623 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:47:54.618+0000 7f2d8c92c640 1 --2- 172.21.15.176:0/568011770 >> [v2:172.21.15.156:6800/3984987611,v1:172.21.15.156:6801/3984987611] conn(0x7f2d740357b0 0x7f2d74037c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:47:54.623 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:47:54.619+0000 7f2d8c92c640 1 --2- 172.21.15.176:0/568011770 >> [v2:172.21.15.156:6800/3984987611,v1:172.21.15.156:6801/3984987611] conn(0x7f2d740357b0 0x7f2d74037c60 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f2d7c006fd0 tx=0x7f2d7c006db0 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:47:54.625 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:47:54.623+0000 7f2d79ffb640 1 -- 172.21.15.176:0/568011770 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f2d8403c380 con 0x7f2d88065c90 2024-07-16T09:47:54.800 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:47:54.797+0000 7f2d8f391640 1 -- 172.21.15.176:0/568011770 --> [v2:172.21.15.156:6800/3984987611,v1:172.21.15.156:6801/3984987611] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "2;smithi156:172.21.15.156=smithi156;smithi176:172.21.15.176=smithi176", "target": ["mon-mgr", ""]}) v1 -- 0x7f2d8810dc10 con 0x7f2d740357b0 2024-07-16T09:47:54.805 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:47:54.803+0000 7f2d79ffb640 1 -- 172.21.15.176:0/568011770 <== mgr.14160 v2:172.21.15.156:6800/3984987611 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f2d8810dc10 con 0x7f2d740357b0 2024-07-16T09:47:54.805 INFO:teuthology.orchestra.run.smithi176.stdout:Scheduled mon update... 2024-07-16T09:47:54.807 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:47:54.804+0000 7f2d8f391640 1 -- 172.21.15.176:0/568011770 >> [v2:172.21.15.156:6800/3984987611,v1:172.21.15.156:6801/3984987611] conn(0x7f2d740357b0 msgr2=0x7f2d74037c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:47:54.807 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:47:54.804+0000 7f2d8f391640 1 --2- 172.21.15.176:0/568011770 >> [v2:172.21.15.156:6800/3984987611,v1:172.21.15.156:6801/3984987611] conn(0x7f2d740357b0 0x7f2d74037c60 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f2d7c006fd0 tx=0x7f2d7c006db0 comp rx=0 tx=0).stop 2024-07-16T09:47:54.807 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:47:54.804+0000 7f2d8f391640 1 -- 172.21.15.176:0/568011770 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2d88065c90 msgr2=0x7f2d8810c650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:47:54.807 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:47:54.804+0000 7f2d8f391640 1 --2- 172.21.15.176:0/568011770 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2d88065c90 0x7f2d8810c650 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f2d84003080 tx=0x7f2d84009da0 comp rx=0 tx=0).stop 2024-07-16T09:47:54.807 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:47:54.805+0000 7f2d8f391640 1 -- 172.21.15.176:0/568011770 shutdown_connections 2024-07-16T09:47:54.807 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:47:54.805+0000 7f2d8f391640 1 --2- 172.21.15.176:0/568011770 >> [v2:172.21.15.156:6800/3984987611,v1:172.21.15.156:6801/3984987611] conn(0x7f2d740357b0 0x7f2d74037c60 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:47:54.807 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:47:54.805+0000 7f2d8f391640 1 --2- 172.21.15.176:0/568011770 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2d88065c90 0x7f2d8810c650 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:47:54.808 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:47:54.805+0000 7f2d8f391640 1 -- 172.21.15.176:0/568011770 >> 172.21.15.176:0/568011770 conn(0x7f2d880f6690 msgr2=0x7f2d88103180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:47:54.808 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:47:54.805+0000 7f2d8f391640 1 -- 172.21.15.176:0/568011770 shutdown_connections 2024-07-16T09:47:54.808 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:47:54.805+0000 7f2d8f391640 1 -- 172.21.15.176:0/568011770 wait complete. 2024-07-16T09:47:55.397 DEBUG:teuthology.orchestra.run.smithi176:mon.smithi176> sudo journalctl -f -n 0 -u ceph-26c91c26-4358-11ef-bcad-c7b262605968@mon.smithi176.service 2024-07-16T09:47:55.400 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-07-16T09:47:55.400 DEBUG:teuthology.orchestra.run.smithi176:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 26c91c26-4358-11ef-bcad-c7b262605968 -- ceph mon dump -f json 2024-07-16T09:47:55.675 INFO:teuthology.orchestra.run.smithi176.stderr:Inferring config /etc/ceph/ceph.conf 2024-07-16T09:47:55.723 INFO:teuthology.orchestra.run.smithi176.stderr:Inferring config /etc/ceph/ceph.conf 2024-07-16T09:47:56.175 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:47:55 smithi156 ceph-mon[29302]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-16T09:47:56.175 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:47:55 smithi156 ceph-mon[29302]: from='client.14192 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "2;smithi156:172.21.15.156=smithi156;smithi176:172.21.15.176=smithi176", "target": ["mon-mgr", ""]}]: dispatch 2024-07-16T09:47:56.175 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:47:55 smithi156 ceph-mon[29302]: Saving service mon spec with placement smithi156:172.21.15.156=smithi156;smithi176:172.21.15.176=smithi176;count:2 2024-07-16T09:47:56.175 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:47:55 smithi156 ceph-mon[29302]: from='mgr.14160 172.21.15.156:0/3789331119' entity='mgr.smithi156.zkvylv' 2024-07-16T09:47:56.175 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:47:55 smithi156 ceph-mon[29302]: from='mgr.14160 172.21.15.156:0/3789331119' entity='mgr.smithi156.zkvylv' 2024-07-16T09:47:56.175 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:47:55 smithi156 ceph-mon[29302]: from='mgr.14160 172.21.15.156:0/3789331119' entity='mgr.smithi156.zkvylv' 2024-07-16T09:47:56.175 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:47:55 smithi156 ceph-mon[29302]: from='mgr.14160 172.21.15.156:0/3789331119' entity='mgr.smithi156.zkvylv' 2024-07-16T09:47:56.175 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:47:55 smithi156 ceph-mon[29302]: from='mgr.14160 172.21.15.156:0/3789331119' entity='mgr.smithi156.zkvylv' 2024-07-16T09:47:56.175 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:47:55 smithi156 ceph-mon[29302]: from='mgr.14160 172.21.15.156:0/3789331119' entity='mgr.smithi156.zkvylv' 2024-07-16T09:47:56.175 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:47:55 smithi156 ceph-mon[29302]: from='mgr.14160 172.21.15.156:0/3789331119' entity='mgr.smithi156.zkvylv' 2024-07-16T09:47:56.175 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:47:55 smithi156 ceph-mon[29302]: from='mgr.14160 172.21.15.156:0/3789331119' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-07-16T09:47:56.175 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:47:55 smithi156 ceph-mon[29302]: from='mgr.14160 172.21.15.156:0/3789331119' entity='mgr.smithi156.zkvylv' 2024-07-16T09:47:56.984 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:47:56.982+0000 7effa597e640 1 -- 172.21.15.176:0/3527111094 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7effa00fbfc0 msgr2=0x7effa00fc3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:47:56.985 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:47:56.982+0000 7effa597e640 1 --2- 172.21.15.176:0/3527111094 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7effa00fbfc0 0x7effa00fc3d0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7eff90004850 tx=0x7eff90009da0 comp rx=0 tx=0).stop 2024-07-16T09:47:56.985 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:47:56.982+0000 7effa597e640 1 -- 172.21.15.176:0/3527111094 shutdown_connections 2024-07-16T09:47:56.985 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:47:56.982+0000 7effa597e640 1 --2- 172.21.15.176:0/3527111094 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7effa00fbfc0 0x7effa00fc3d0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:47:56.985 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:47:56.982+0000 7effa597e640 1 -- 172.21.15.176:0/3527111094 >> 172.21.15.176:0/3527111094 conn(0x7effa00f77a0 msgr2=0x7effa00f9bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:47:56.985 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:47:56.982+0000 7effa597e640 1 -- 172.21.15.176:0/3527111094 shutdown_connections 2024-07-16T09:47:56.985 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:47:56.982+0000 7effa597e640 1 -- 172.21.15.176:0/3527111094 wait complete. 2024-07-16T09:47:56.985 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:47:56.983+0000 7effa597e640 1 Processor -- start 2024-07-16T09:47:56.985 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:47:56.983+0000 7effa597e640 1 -- start start 2024-07-16T09:47:56.985 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:47:56.983+0000 7effa597e640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7effa00fbfc0 0x7effa010c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:47:56.985 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:47:56.983+0000 7effa597e640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effa010cb00 con 0x7effa00fbfc0 2024-07-16T09:47:56.987 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:47:56.984+0000 7eff9effd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7effa00fbfc0 0x7effa010c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:47:56.987 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:47:56.984+0000 7eff9effd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7effa00fbfc0 0x7effa010c5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.176:52738/0 (socket says 172.21.15.176:52738) 2024-07-16T09:47:56.987 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:47:56.984+0000 7eff9effd640 1 -- 172.21.15.176:0/426384824 learned_addr learned my addr 172.21.15.176:0/426384824 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-07-16T09:47:56.987 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:47:56.984+0000 7eff9effd640 1 -- 172.21.15.176:0/426384824 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff90005040 con 0x7effa00fbfc0 2024-07-16T09:47:56.987 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:47:56.984+0000 7eff9effd640 1 --2- 172.21.15.176:0/426384824 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7effa00fbfc0 0x7effa010c5c0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7eff90004820 tx=0x7eff90009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:47:56.987 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:47:56.984+0000 7eff877fe640 1 -- 172.21.15.176:0/426384824 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7eff90032650 con 0x7effa00fbfc0 2024-07-16T09:47:56.987 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:47:56.984+0000 7eff877fe640 1 -- 172.21.15.176:0/426384824 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7eff90032c80 con 0x7effa00fbfc0 2024-07-16T09:47:56.987 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:47:56.984+0000 7effa597e640 1 -- 172.21.15.176:0/426384824 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7effa010cd00 con 0x7effa00fbfc0 2024-07-16T09:47:56.988 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:47:56.984+0000 7eff877fe640 1 -- 172.21.15.176:0/426384824 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7eff9003eda0 con 0x7effa00fbfc0 2024-07-16T09:47:56.988 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:47:56.984+0000 7effa597e640 1 -- 172.21.15.176:0/426384824 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effa010d890 con 0x7effa00fbfc0 2024-07-16T09:47:56.988 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:47:56.985+0000 7eff877fe640 1 -- 172.21.15.176:0/426384824 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 13) v1 ==== 42693+0+0 (secure 0 0 0) 0x7eff90030020 con 0x7effa00fbfc0 2024-07-16T09:47:56.988 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:47:56.985+0000 7eff877fe640 1 --2- 172.21.15.176:0/426384824 >> [v2:172.21.15.156:6800/3984987611,v1:172.21.15.156:6801/3984987611] conn(0x7eff880357b0 0x7eff88037c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:47:56.988 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:47:56.985+0000 7eff877fe640 1 -- 172.21.15.176:0/426384824 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7eff90067b70 con 0x7effa00fbfc0 2024-07-16T09:47:56.988 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:47:56.985+0000 7effa597e640 1 -- 172.21.15.176:0/426384824 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7effa0102530 con 0x7effa00fbfc0 2024-07-16T09:47:56.988 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:47:56.985+0000 7eff9e7fc640 1 --2- 172.21.15.176:0/426384824 >> [v2:172.21.15.156:6800/3984987611,v1:172.21.15.156:6801/3984987611] conn(0x7eff880357b0 0x7eff88037c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:47:56.988 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:47:56.986+0000 7eff9e7fc640 1 --2- 172.21.15.176:0/426384824 >> [v2:172.21.15.156:6800/3984987611,v1:172.21.15.156:6801/3984987611] conn(0x7eff880357b0 0x7eff88037c60 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7eff94006fd0 tx=0x7eff94006db0 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:47:56.991 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:47:56.990+0000 7eff877fe640 1 -- 172.21.15.176:0/426384824 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7eff90033420 con 0x7effa00fbfc0 2024-07-16T09:47:57.174 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:47:56 smithi156 ceph-mon[29302]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-07-16T09:47:57.175 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:47:56 smithi156 ceph-mon[29302]: Deploying daemon grafana.smithi156 on smithi156 2024-07-16T09:47:57.221 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:47:57.220+0000 7effa597e640 1 -- 172.21.15.176:0/426384824 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7effa005eb80 con 0x7effa00fbfc0 2024-07-16T09:47:57.222 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:47:57.220+0000 7eff877fe640 1 -- 172.21.15.176:0/426384824 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+743 (secure 0 0 0) 0x7effa005eb80 con 0x7effa00fbfc0 2024-07-16T09:47:57.222 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-07-16T09:47:57.222 INFO:teuthology.orchestra.run.smithi176.stdout:{"epoch":1,"fsid":"26c91c26-4358-11ef-bcad-c7b262605968","modified":"2024-07-16T09:46:20.852914Z","created":"2024-07-16T09:46:20.852914Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi156","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:3300","nonce":0},{"type":"v1","addr":"172.21.15.156:6789","nonce":0}]},"addr":"172.21.15.156:6789/0","public_addr":"172.21.15.156:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-07-16T09:47:57.222 INFO:teuthology.orchestra.run.smithi176.stderr:dumped monmap epoch 1 2024-07-16T09:47:57.224 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:47:57.222+0000 7effa597e640 1 -- 172.21.15.176:0/426384824 >> [v2:172.21.15.156:6800/3984987611,v1:172.21.15.156:6801/3984987611] conn(0x7eff880357b0 msgr2=0x7eff88037c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:47:57.224 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:47:57.222+0000 7effa597e640 1 --2- 172.21.15.176:0/426384824 >> [v2:172.21.15.156:6800/3984987611,v1:172.21.15.156:6801/3984987611] conn(0x7eff880357b0 0x7eff88037c60 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7eff94006fd0 tx=0x7eff94006db0 comp rx=0 tx=0).stop 2024-07-16T09:47:57.224 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:47:57.222+0000 7effa597e640 1 -- 172.21.15.176:0/426384824 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7effa00fbfc0 msgr2=0x7effa010c5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:47:57.224 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:47:57.222+0000 7effa597e640 1 --2- 172.21.15.176:0/426384824 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7effa00fbfc0 0x7effa010c5c0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7eff90004820 tx=0x7eff90009da0 comp rx=0 tx=0).stop 2024-07-16T09:47:57.224 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:47:57.222+0000 7effa597e640 1 -- 172.21.15.176:0/426384824 shutdown_connections 2024-07-16T09:47:57.224 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:47:57.222+0000 7effa597e640 1 --2- 172.21.15.176:0/426384824 >> [v2:172.21.15.156:6800/3984987611,v1:172.21.15.156:6801/3984987611] conn(0x7eff880357b0 0x7eff88037c60 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:47:57.224 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:47:57.222+0000 7effa597e640 1 --2- 172.21.15.176:0/426384824 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7effa00fbfc0 0x7effa010c5c0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:47:57.225 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:47:57.222+0000 7effa597e640 1 -- 172.21.15.176:0/426384824 >> 172.21.15.176:0/426384824 conn(0x7effa00f77a0 msgr2=0x7effa0103160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:47:57.225 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:47:57.222+0000 7effa597e640 1 -- 172.21.15.176:0/426384824 shutdown_connections 2024-07-16T09:47:57.225 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:47:57.222+0000 7effa597e640 1 -- 172.21.15.176:0/426384824 wait complete. 2024-07-16T09:47:58.174 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:47:57 smithi156 ceph-mon[29302]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-16T09:47:58.174 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:47:57 smithi156 ceph-mon[29302]: from='client.? 172.21.15.176:0/426384824' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-07-16T09:47:58.175 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:47:57 smithi156 ceph-mon[29302]: from='mgr.14160 172.21.15.156:0/3789331119' entity='mgr.smithi156.zkvylv' 2024-07-16T09:47:58.760 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-07-16T09:47:58.760 DEBUG:teuthology.orchestra.run.smithi176:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 26c91c26-4358-11ef-bcad-c7b262605968 -- ceph mon dump -f json 2024-07-16T09:47:59.011 INFO:teuthology.orchestra.run.smithi176.stderr:Inferring config /etc/ceph/ceph.conf 2024-07-16T09:47:59.058 INFO:teuthology.orchestra.run.smithi176.stderr:Inferring config /etc/ceph/ceph.conf 2024-07-16T09:48:00.174 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:47:59 smithi156 ceph-mon[29302]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-16T09:48:00.354 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:00.351+0000 7f43642ab640 1 -- 172.21.15.176:0/1764924668 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f435c0fbfc0 msgr2=0x7f435c0fc3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:48:00.354 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:00.351+0000 7f43642ab640 1 --2- 172.21.15.176:0/1764924668 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f435c0fbfc0 0x7f435c0fc3d0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f4358004850 tx=0x7f4358009da0 comp rx=0 tx=0).stop 2024-07-16T09:48:00.355 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:00.351+0000 7f43642ab640 1 -- 172.21.15.176:0/1764924668 shutdown_connections 2024-07-16T09:48:00.355 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:00.351+0000 7f43642ab640 1 --2- 172.21.15.176:0/1764924668 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f435c0fbfc0 0x7f435c0fc3d0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:48:00.355 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:00.351+0000 7f43642ab640 1 -- 172.21.15.176:0/1764924668 >> 172.21.15.176:0/1764924668 conn(0x7f435c0f77a0 msgr2=0x7f435c0f9bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:48:00.355 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:00.351+0000 7f43642ab640 1 -- 172.21.15.176:0/1764924668 shutdown_connections 2024-07-16T09:48:00.355 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:00.351+0000 7f43642ab640 1 -- 172.21.15.176:0/1764924668 wait complete. 2024-07-16T09:48:00.355 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:00.352+0000 7f43642ab640 1 Processor -- start 2024-07-16T09:48:00.355 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:00.352+0000 7f43642ab640 1 -- start start 2024-07-16T09:48:00.355 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:00.352+0000 7f43642ab640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f435c0fbfc0 0x7f435c10c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:48:00.355 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:00.352+0000 7f43642ab640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f435c10cb00 con 0x7f435c0fbfc0 2024-07-16T09:48:00.355 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:00.352+0000 7f4362047640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f435c0fbfc0 0x7f435c10c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:48:00.356 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:00.352+0000 7f4362047640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f435c0fbfc0 0x7f435c10c5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.176:52748/0 (socket says 172.21.15.176:52748) 2024-07-16T09:48:00.356 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:00.352+0000 7f4362047640 1 -- 172.21.15.176:0/2463778108 learned_addr learned my addr 172.21.15.176:0/2463778108 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-07-16T09:48:00.356 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:00.352+0000 7f4362047640 1 -- 172.21.15.176:0/2463778108 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4358005040 con 0x7f435c0fbfc0 2024-07-16T09:48:00.356 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:00.353+0000 7f4362047640 1 --2- 172.21.15.176:0/2463778108 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f435c0fbfc0 0x7f435c10c5c0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f4358004820 tx=0x7f4358009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:48:00.356 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:00.353+0000 7f4352ffd640 1 -- 172.21.15.176:0/2463778108 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4358032650 con 0x7f435c0fbfc0 2024-07-16T09:48:00.356 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:00.353+0000 7f4352ffd640 1 -- 172.21.15.176:0/2463778108 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4358032c80 con 0x7f435c0fbfc0 2024-07-16T09:48:00.356 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:00.353+0000 7f43642ab640 1 -- 172.21.15.176:0/2463778108 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f435c10cd00 con 0x7f435c0fbfc0 2024-07-16T09:48:00.356 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:00.353+0000 7f4352ffd640 1 -- 172.21.15.176:0/2463778108 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f435803eda0 con 0x7f435c0fbfc0 2024-07-16T09:48:00.356 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:00.353+0000 7f43642ab640 1 -- 172.21.15.176:0/2463778108 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f435c10d890 con 0x7f435c0fbfc0 2024-07-16T09:48:00.357 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:00.354+0000 7f4352ffd640 1 -- 172.21.15.176:0/2463778108 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 13) v1 ==== 42693+0+0 (secure 0 0 0) 0x7f4358030020 con 0x7f435c0fbfc0 2024-07-16T09:48:00.357 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:00.354+0000 7f43642ab640 1 -- 172.21.15.176:0/2463778108 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f435c102530 con 0x7f435c0fbfc0 2024-07-16T09:48:00.357 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:00.354+0000 7f4352ffd640 1 --2- 172.21.15.176:0/2463778108 >> [v2:172.21.15.156:6800/3984987611,v1:172.21.15.156:6801/3984987611] conn(0x7f43480357b0 0x7f4348037c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:48:00.357 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:00.354+0000 7f4352ffd640 1 -- 172.21.15.176:0/2463778108 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f4358067a60 con 0x7f435c0fbfc0 2024-07-16T09:48:00.357 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:00.354+0000 7f4361846640 1 --2- 172.21.15.176:0/2463778108 >> [v2:172.21.15.156:6800/3984987611,v1:172.21.15.156:6801/3984987611] conn(0x7f43480357b0 0x7f4348037c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:48:00.358 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:00.355+0000 7f4361846640 1 --2- 172.21.15.176:0/2463778108 >> [v2:172.21.15.156:6800/3984987611,v1:172.21.15.156:6801/3984987611] conn(0x7f43480357b0 0x7f4348037c60 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f434c006fd0 tx=0x7f434c006db0 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:48:00.360 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:00.359+0000 7f4352ffd640 1 -- 172.21.15.176:0/2463778108 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f43580327b0 con 0x7f435c0fbfc0 2024-07-16T09:48:00.590 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:00.588+0000 7f43642ab640 1 -- 172.21.15.176:0/2463778108 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f435c10e290 con 0x7f435c0fbfc0 2024-07-16T09:48:00.590 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:00.589+0000 7f4352ffd640 1 -- 172.21.15.176:0/2463778108 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+743 (secure 0 0 0) 0x7f435803c330 con 0x7f435c0fbfc0 2024-07-16T09:48:00.590 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-07-16T09:48:00.591 INFO:teuthology.orchestra.run.smithi176.stdout:{"epoch":1,"fsid":"26c91c26-4358-11ef-bcad-c7b262605968","modified":"2024-07-16T09:46:20.852914Z","created":"2024-07-16T09:46:20.852914Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi156","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:3300","nonce":0},{"type":"v1","addr":"172.21.15.156:6789","nonce":0}]},"addr":"172.21.15.156:6789/0","public_addr":"172.21.15.156:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-07-16T09:48:00.591 INFO:teuthology.orchestra.run.smithi176.stderr:dumped monmap epoch 1 2024-07-16T09:48:00.593 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:00.591+0000 7f43642ab640 1 -- 172.21.15.176:0/2463778108 >> [v2:172.21.15.156:6800/3984987611,v1:172.21.15.156:6801/3984987611] conn(0x7f43480357b0 msgr2=0x7f4348037c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:48:00.593 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:00.591+0000 7f43642ab640 1 --2- 172.21.15.176:0/2463778108 >> [v2:172.21.15.156:6800/3984987611,v1:172.21.15.156:6801/3984987611] conn(0x7f43480357b0 0x7f4348037c60 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f434c006fd0 tx=0x7f434c006db0 comp rx=0 tx=0).stop 2024-07-16T09:48:00.593 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:00.591+0000 7f43642ab640 1 -- 172.21.15.176:0/2463778108 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f435c0fbfc0 msgr2=0x7f435c10c5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:48:00.593 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:00.591+0000 7f43642ab640 1 --2- 172.21.15.176:0/2463778108 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f435c0fbfc0 0x7f435c10c5c0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f4358004820 tx=0x7f4358009da0 comp rx=0 tx=0).stop 2024-07-16T09:48:00.593 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:00.591+0000 7f43642ab640 1 -- 172.21.15.176:0/2463778108 shutdown_connections 2024-07-16T09:48:00.593 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:00.591+0000 7f43642ab640 1 --2- 172.21.15.176:0/2463778108 >> [v2:172.21.15.156:6800/3984987611,v1:172.21.15.156:6801/3984987611] conn(0x7f43480357b0 0x7f4348037c60 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:48:00.593 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:00.591+0000 7f43642ab640 1 --2- 172.21.15.176:0/2463778108 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f435c0fbfc0 0x7f435c10c5c0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:48:00.593 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:00.591+0000 7f43642ab640 1 -- 172.21.15.176:0/2463778108 >> 172.21.15.176:0/2463778108 conn(0x7f435c0f77a0 msgr2=0x7f435c103160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:48:00.594 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:00.591+0000 7f43642ab640 1 -- 172.21.15.176:0/2463778108 shutdown_connections 2024-07-16T09:48:00.594 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:00.591+0000 7f43642ab640 1 -- 172.21.15.176:0/2463778108 wait complete. 2024-07-16T09:48:01.174 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:48:00 smithi156 ceph-mon[29302]: from='client.? 172.21.15.176:0/2463778108' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-07-16T09:48:02.123 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-07-16T09:48:02.123 DEBUG:teuthology.orchestra.run.smithi176:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 26c91c26-4358-11ef-bcad-c7b262605968 -- ceph mon dump -f json 2024-07-16T09:48:02.174 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:48:01 smithi156 ceph-mon[29302]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-16T09:48:02.377 INFO:teuthology.orchestra.run.smithi176.stderr:Inferring config /etc/ceph/ceph.conf 2024-07-16T09:48:02.436 INFO:teuthology.orchestra.run.smithi176.stderr:Inferring config /etc/ceph/ceph.conf 2024-07-16T09:48:03.839 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:03.833+0000 7f263da12640 1 -- 172.21.15.176:0/1706480622 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f26380fc4c0 msgr2=0x7f26380fc8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:48:03.839 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:03.833+0000 7f263da12640 1 --2- 172.21.15.176:0/1706480622 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f26380fc4c0 0x7f26380fc8d0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f2620004850 tx=0x7f2620009da0 comp rx=0 tx=0).stop 2024-07-16T09:48:03.839 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:03.833+0000 7f263da12640 1 -- 172.21.15.176:0/1706480622 shutdown_connections 2024-07-16T09:48:03.839 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:03.833+0000 7f263da12640 1 --2- 172.21.15.176:0/1706480622 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f26380fc4c0 0x7f26380fc8d0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:48:03.840 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:03.833+0000 7f263da12640 1 -- 172.21.15.176:0/1706480622 >> 172.21.15.176:0/1706480622 conn(0x7f26380f5e50 msgr2=0x7f26380f8260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:48:03.840 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:03.833+0000 7f263da12640 1 -- 172.21.15.176:0/1706480622 shutdown_connections 2024-07-16T09:48:03.840 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:03.833+0000 7f263da12640 1 -- 172.21.15.176:0/1706480622 wait complete. 2024-07-16T09:48:03.840 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:03.834+0000 7f263da12640 1 Processor -- start 2024-07-16T09:48:03.840 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:03.834+0000 7f263da12640 1 -- start start 2024-07-16T09:48:03.840 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:03.834+0000 7f263da12640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f26380fc4c0 0x7f263810c610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:48:03.840 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:03.834+0000 7f263da12640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f263810cb50 con 0x7f26380fc4c0 2024-07-16T09:48:03.840 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:03.834+0000 7f2636ffd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f26380fc4c0 0x7f263810c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:48:03.840 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:03.834+0000 7f2636ffd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f26380fc4c0 0x7f263810c610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.176:52758/0 (socket says 172.21.15.176:52758) 2024-07-16T09:48:03.840 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:03.834+0000 7f2636ffd640 1 -- 172.21.15.176:0/3114464153 learned_addr learned my addr 172.21.15.176:0/3114464153 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-07-16T09:48:03.840 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:03.835+0000 7f2636ffd640 1 -- 172.21.15.176:0/3114464153 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2620005040 con 0x7f26380fc4c0 2024-07-16T09:48:03.840 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:03.835+0000 7f2636ffd640 1 --2- 172.21.15.176:0/3114464153 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f26380fc4c0 0x7f263810c610 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f2620004820 tx=0x7f2620009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:48:03.840 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:03.835+0000 7f262f7fe640 1 -- 172.21.15.176:0/3114464153 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2620032650 con 0x7f26380fc4c0 2024-07-16T09:48:03.840 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:03.835+0000 7f262f7fe640 1 -- 172.21.15.176:0/3114464153 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2620032c80 con 0x7f26380fc4c0 2024-07-16T09:48:03.841 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:03.835+0000 7f263da12640 1 -- 172.21.15.176:0/3114464153 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f263810cd50 con 0x7f26380fc4c0 2024-07-16T09:48:03.841 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:03.835+0000 7f262f7fe640 1 -- 172.21.15.176:0/3114464153 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2620033d60 con 0x7f26380fc4c0 2024-07-16T09:48:03.841 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:03.835+0000 7f263da12640 1 -- 172.21.15.176:0/3114464153 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f263810d8e0 con 0x7f26380fc4c0 2024-07-16T09:48:03.841 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:03.836+0000 7f262f7fe640 1 -- 172.21.15.176:0/3114464153 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 13) v1 ==== 42693+0+0 (secure 0 0 0) 0x7f2620030020 con 0x7f26380fc4c0 2024-07-16T09:48:03.841 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:03.836+0000 7f263da12640 1 -- 172.21.15.176:0/3114464153 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2638102560 con 0x7f26380fc4c0 2024-07-16T09:48:03.841 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:03.836+0000 7f262f7fe640 1 --2- 172.21.15.176:0/3114464153 >> [v2:172.21.15.156:6800/3984987611,v1:172.21.15.156:6801/3984987611] conn(0x7f26240357b0 0x7f2624037c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:48:03.841 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:03.836+0000 7f262f7fe640 1 -- 172.21.15.176:0/3114464153 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f2620067920 con 0x7f26380fc4c0 2024-07-16T09:48:03.841 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:03.836+0000 7f26367fc640 1 --2- 172.21.15.176:0/3114464153 >> [v2:172.21.15.156:6800/3984987611,v1:172.21.15.156:6801/3984987611] conn(0x7f26240357b0 0x7f2624037c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:48:03.841 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:03.837+0000 7f26367fc640 1 --2- 172.21.15.176:0/3114464153 >> [v2:172.21.15.156:6800/3984987611,v1:172.21.15.156:6801/3984987611] conn(0x7f26240357b0 0x7f2624037c60 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f2628006fd0 tx=0x7f2628006db0 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:48:03.842 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:03.841+0000 7f262f7fe640 1 -- 172.21.15.176:0/3114464153 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f262003c560 con 0x7f26380fc4c0 2024-07-16T09:48:04.068 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:04.067+0000 7f263da12640 1 -- 172.21.15.176:0/3114464153 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f263810e2e0 con 0x7f26380fc4c0 2024-07-16T09:48:04.068 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:04.067+0000 7f262f7fe640 1 -- 172.21.15.176:0/3114464153 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+743 (secure 0 0 0) 0x7f262003c380 con 0x7f26380fc4c0 2024-07-16T09:48:04.069 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-07-16T09:48:04.069 INFO:teuthology.orchestra.run.smithi176.stdout:{"epoch":1,"fsid":"26c91c26-4358-11ef-bcad-c7b262605968","modified":"2024-07-16T09:46:20.852914Z","created":"2024-07-16T09:46:20.852914Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi156","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:3300","nonce":0},{"type":"v1","addr":"172.21.15.156:6789","nonce":0}]},"addr":"172.21.15.156:6789/0","public_addr":"172.21.15.156:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-07-16T09:48:04.069 INFO:teuthology.orchestra.run.smithi176.stderr:dumped monmap epoch 1 2024-07-16T09:48:04.070 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:04.069+0000 7f263da12640 1 -- 172.21.15.176:0/3114464153 >> [v2:172.21.15.156:6800/3984987611,v1:172.21.15.156:6801/3984987611] conn(0x7f26240357b0 msgr2=0x7f2624037c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:48:04.071 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:04.069+0000 7f263da12640 1 --2- 172.21.15.176:0/3114464153 >> [v2:172.21.15.156:6800/3984987611,v1:172.21.15.156:6801/3984987611] conn(0x7f26240357b0 0x7f2624037c60 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f2628006fd0 tx=0x7f2628006db0 comp rx=0 tx=0).stop 2024-07-16T09:48:04.071 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:04.069+0000 7f263da12640 1 -- 172.21.15.176:0/3114464153 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f26380fc4c0 msgr2=0x7f263810c610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:48:04.071 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:04.069+0000 7f263da12640 1 --2- 172.21.15.176:0/3114464153 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f26380fc4c0 0x7f263810c610 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f2620004820 tx=0x7f2620009da0 comp rx=0 tx=0).stop 2024-07-16T09:48:04.071 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:04.069+0000 7f263da12640 1 -- 172.21.15.176:0/3114464153 shutdown_connections 2024-07-16T09:48:04.071 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:04.069+0000 7f263da12640 1 --2- 172.21.15.176:0/3114464153 >> [v2:172.21.15.156:6800/3984987611,v1:172.21.15.156:6801/3984987611] conn(0x7f26240357b0 0x7f2624037c60 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:48:04.071 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:04.069+0000 7f263da12640 1 --2- 172.21.15.176:0/3114464153 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f26380fc4c0 0x7f263810c610 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:48:04.071 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:04.069+0000 7f263da12640 1 -- 172.21.15.176:0/3114464153 >> 172.21.15.176:0/3114464153 conn(0x7f26380f5e50 msgr2=0x7f2638103190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:48:04.071 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:04.069+0000 7f263da12640 1 -- 172.21.15.176:0/3114464153 shutdown_connections 2024-07-16T09:48:04.071 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:04.069+0000 7f263da12640 1 -- 172.21.15.176:0/3114464153 wait complete. 2024-07-16T09:48:04.174 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:48:03 smithi156 ceph-mon[29302]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-16T09:48:05.174 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:48:04 smithi156 ceph-mon[29302]: from='client.? 172.21.15.176:0/3114464153' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-07-16T09:48:05.652 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-07-16T09:48:05.652 DEBUG:teuthology.orchestra.run.smithi176:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 26c91c26-4358-11ef-bcad-c7b262605968 -- ceph mon dump -f json 2024-07-16T09:48:05.906 INFO:teuthology.orchestra.run.smithi176.stderr:Inferring config /etc/ceph/ceph.conf 2024-07-16T09:48:05.953 INFO:teuthology.orchestra.run.smithi176.stderr:Inferring config /etc/ceph/ceph.conf 2024-07-16T09:48:06.128 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:48:05 smithi156 ceph-mon[29302]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-16T09:48:07.297 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:07.295+0000 7fd1cec65640 1 -- 172.21.15.176:0/2312378746 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd1c80fb0f0 msgr2=0x7fd1c80fb500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:48:07.297 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:07.295+0000 7fd1cec65640 1 --2- 172.21.15.176:0/2312378746 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd1c80fb0f0 0x7fd1c80fb500 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fd1bc004850 tx=0x7fd1bc009da0 comp rx=0 tx=0).stop 2024-07-16T09:48:07.297 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:07.295+0000 7fd1cec65640 1 -- 172.21.15.176:0/2312378746 shutdown_connections 2024-07-16T09:48:07.297 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:07.295+0000 7fd1cec65640 1 --2- 172.21.15.176:0/2312378746 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd1c80fb0f0 0x7fd1c80fb500 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:48:07.297 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:07.295+0000 7fd1cec65640 1 -- 172.21.15.176:0/2312378746 >> 172.21.15.176:0/2312378746 conn(0x7fd1c80f68d0 msgr2=0x7fd1c80f8ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:48:07.298 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:07.295+0000 7fd1cec65640 1 -- 172.21.15.176:0/2312378746 shutdown_connections 2024-07-16T09:48:07.298 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:07.295+0000 7fd1cec65640 1 -- 172.21.15.176:0/2312378746 wait complete. 2024-07-16T09:48:07.298 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:07.296+0000 7fd1cec65640 1 Processor -- start 2024-07-16T09:48:07.298 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:07.296+0000 7fd1cec65640 1 -- start start 2024-07-16T09:48:07.299 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:07.296+0000 7fd1cec65640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd1c80fb0f0 0x7fd1c810c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:48:07.299 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:07.296+0000 7fd1cec65640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1c810cd20 con 0x7fd1c80fb0f0 2024-07-16T09:48:07.300 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:07.296+0000 7fd1cca01640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd1c80fb0f0 0x7fd1c810c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:48:07.300 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:07.296+0000 7fd1cca01640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd1c80fb0f0 0x7fd1c810c7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.176:39670/0 (socket says 172.21.15.176:39670) 2024-07-16T09:48:07.300 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:07.296+0000 7fd1cca01640 1 -- 172.21.15.176:0/2306820649 learned_addr learned my addr 172.21.15.176:0/2306820649 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-07-16T09:48:07.300 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:07.297+0000 7fd1cca01640 1 -- 172.21.15.176:0/2306820649 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1bc005040 con 0x7fd1c80fb0f0 2024-07-16T09:48:07.300 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:07.297+0000 7fd1cca01640 1 --2- 172.21.15.176:0/2306820649 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd1c80fb0f0 0x7fd1c810c7e0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fd1bc004820 tx=0x7fd1bc009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:48:07.300 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:07.297+0000 7fd1c57fa640 1 -- 172.21.15.176:0/2306820649 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd1bc032650 con 0x7fd1c80fb0f0 2024-07-16T09:48:07.300 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:07.297+0000 7fd1c57fa640 1 -- 172.21.15.176:0/2306820649 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd1bc032c80 con 0x7fd1c80fb0f0 2024-07-16T09:48:07.300 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:07.297+0000 7fd1cec65640 1 -- 172.21.15.176:0/2306820649 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1c810cf20 con 0x7fd1c80fb0f0 2024-07-16T09:48:07.300 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:07.297+0000 7fd1c57fa640 1 -- 172.21.15.176:0/2306820649 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd1bc03eda0 con 0x7fd1c80fb0f0 2024-07-16T09:48:07.300 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:07.297+0000 7fd1cec65640 1 -- 172.21.15.176:0/2306820649 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1c810dab0 con 0x7fd1c80fb0f0 2024-07-16T09:48:07.300 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:07.298+0000 7fd1c57fa640 1 -- 172.21.15.176:0/2306820649 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 13) v1 ==== 42693+0+0 (secure 0 0 0) 0x7fd1bc030020 con 0x7fd1c80fb0f0 2024-07-16T09:48:07.300 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:07.298+0000 7fd1c57fa640 1 --2- 172.21.15.176:0/2306820649 >> [v2:172.21.15.156:6800/3984987611,v1:172.21.15.156:6801/3984987611] conn(0x7fd1b80357b0 0x7fd1b8037c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:48:07.300 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:07.298+0000 7fd1cec65640 1 -- 172.21.15.176:0/2306820649 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1c8101660 con 0x7fd1c80fb0f0 2024-07-16T09:48:07.301 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:07.298+0000 7fd1c57fa640 1 -- 172.21.15.176:0/2306820649 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7fd1bc067c90 con 0x7fd1c80fb0f0 2024-07-16T09:48:07.301 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:07.298+0000 7fd1c7fff640 1 --2- 172.21.15.176:0/2306820649 >> [v2:172.21.15.156:6800/3984987611,v1:172.21.15.156:6801/3984987611] conn(0x7fd1b80357b0 0x7fd1b8037c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:48:07.301 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:07.299+0000 7fd1c7fff640 1 --2- 172.21.15.176:0/2306820649 >> [v2:172.21.15.156:6800/3984987611,v1:172.21.15.156:6801/3984987611] conn(0x7fd1b80357b0 0x7fd1b8037c60 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fd1b4006fd0 tx=0x7fd1b4006db0 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:48:07.304 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:07.303+0000 7fd1c57fa640 1 -- 172.21.15.176:0/2306820649 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fd1bc033420 con 0x7fd1c80fb0f0 2024-07-16T09:48:07.529 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:07.528+0000 7fd1cec65640 1 -- 172.21.15.176:0/2306820649 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fd1c804b530 con 0x7fd1c80fb0f0 2024-07-16T09:48:07.530 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:07.528+0000 7fd1c57fa640 1 -- 172.21.15.176:0/2306820649 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+743 (secure 0 0 0) 0x7fd1bc03c330 con 0x7fd1c80fb0f0 2024-07-16T09:48:07.530 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-07-16T09:48:07.530 INFO:teuthology.orchestra.run.smithi176.stdout:{"epoch":1,"fsid":"26c91c26-4358-11ef-bcad-c7b262605968","modified":"2024-07-16T09:46:20.852914Z","created":"2024-07-16T09:46:20.852914Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi156","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:3300","nonce":0},{"type":"v1","addr":"172.21.15.156:6789","nonce":0}]},"addr":"172.21.15.156:6789/0","public_addr":"172.21.15.156:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-07-16T09:48:07.530 INFO:teuthology.orchestra.run.smithi176.stderr:dumped monmap epoch 1 2024-07-16T09:48:07.533 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:07.531+0000 7fd1cec65640 1 -- 172.21.15.176:0/2306820649 >> [v2:172.21.15.156:6800/3984987611,v1:172.21.15.156:6801/3984987611] conn(0x7fd1b80357b0 msgr2=0x7fd1b8037c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:48:07.533 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:07.531+0000 7fd1cec65640 1 --2- 172.21.15.176:0/2306820649 >> [v2:172.21.15.156:6800/3984987611,v1:172.21.15.156:6801/3984987611] conn(0x7fd1b80357b0 0x7fd1b8037c60 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fd1b4006fd0 tx=0x7fd1b4006db0 comp rx=0 tx=0).stop 2024-07-16T09:48:07.533 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:07.531+0000 7fd1cec65640 1 -- 172.21.15.176:0/2306820649 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd1c80fb0f0 msgr2=0x7fd1c810c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:48:07.533 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:07.531+0000 7fd1cec65640 1 --2- 172.21.15.176:0/2306820649 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd1c80fb0f0 0x7fd1c810c7e0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fd1bc004820 tx=0x7fd1bc009da0 comp rx=0 tx=0).stop 2024-07-16T09:48:07.533 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:07.531+0000 7fd1cec65640 1 -- 172.21.15.176:0/2306820649 shutdown_connections 2024-07-16T09:48:07.533 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:07.531+0000 7fd1cec65640 1 --2- 172.21.15.176:0/2306820649 >> [v2:172.21.15.156:6800/3984987611,v1:172.21.15.156:6801/3984987611] conn(0x7fd1b80357b0 0x7fd1b8037c60 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:48:07.534 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:07.531+0000 7fd1cec65640 1 --2- 172.21.15.176:0/2306820649 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd1c80fb0f0 0x7fd1c810c7e0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:48:07.534 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:07.531+0000 7fd1cec65640 1 -- 172.21.15.176:0/2306820649 >> 172.21.15.176:0/2306820649 conn(0x7fd1c80f68d0 msgr2=0x7fd1c80647a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:48:07.534 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:07.531+0000 7fd1cec65640 1 -- 172.21.15.176:0/2306820649 shutdown_connections 2024-07-16T09:48:07.534 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:07.531+0000 7fd1cec65640 1 -- 172.21.15.176:0/2306820649 wait complete. 2024-07-16T09:48:08.081 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:48:07 smithi156 ceph-mon[29302]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-16T09:48:08.081 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:48:07 smithi156 ceph-mon[29302]: from='client.? 172.21.15.176:0/2306820649' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-07-16T09:48:09.149 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-07-16T09:48:09.149 DEBUG:teuthology.orchestra.run.smithi176:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 26c91c26-4358-11ef-bcad-c7b262605968 -- ceph mon dump -f json 2024-07-16T09:48:09.318 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:48:09 smithi156 ceph-mon[29302]: from='mgr.14160 172.21.15.156:0/3789331119' entity='mgr.smithi156.zkvylv' 2024-07-16T09:48:09.319 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:48:09 smithi156 ceph-mon[29302]: from='mgr.14160 172.21.15.156:0/3789331119' entity='mgr.smithi156.zkvylv' 2024-07-16T09:48:09.319 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:48:09 smithi156 ceph-mon[29302]: from='mgr.14160 172.21.15.156:0/3789331119' entity='mgr.smithi156.zkvylv' 2024-07-16T09:48:09.319 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:48:09 smithi156 ceph-mon[29302]: from='mgr.14160 172.21.15.156:0/3789331119' entity='mgr.smithi156.zkvylv' 2024-07-16T09:48:09.319 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:48:09 smithi156 ceph-mon[29302]: from='mgr.14160 172.21.15.156:0/3789331119' entity='mgr.smithi156.zkvylv' 2024-07-16T09:48:09.319 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:48:09 smithi156 ceph-mon[29302]: from='mgr.14160 172.21.15.156:0/3789331119' entity='mgr.smithi156.zkvylv' 2024-07-16T09:48:09.319 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:48:09 smithi156 ceph-mon[29302]: Deploying daemon prometheus.smithi156 on smithi156 2024-07-16T09:48:09.402 INFO:teuthology.orchestra.run.smithi176.stderr:Inferring config /etc/ceph/ceph.conf 2024-07-16T09:48:09.451 INFO:teuthology.orchestra.run.smithi176.stderr:Inferring config /etc/ceph/ceph.conf 2024-07-16T09:48:10.424 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:48:10 smithi156 ceph-mon[29302]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-16T09:48:10.831 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:10.829+0000 7fba8adfc640 1 -- 172.21.15.176:0/3770187588 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fba84065fb0 msgr2=0x7fba84100ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:48:10.831 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:10.829+0000 7fba8adfc640 1 --2- 172.21.15.176:0/3770187588 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fba84065fb0 0x7fba84100ff0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fba78004850 tx=0x7fba78009da0 comp rx=0 tx=0).stop 2024-07-16T09:48:10.831 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:10.829+0000 7fba8adfc640 1 -- 172.21.15.176:0/3770187588 shutdown_connections 2024-07-16T09:48:10.832 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:10.829+0000 7fba8adfc640 1 --2- 172.21.15.176:0/3770187588 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fba84065fb0 0x7fba84100ff0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:48:10.832 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:10.829+0000 7fba8adfc640 1 -- 172.21.15.176:0/3770187588 >> 172.21.15.176:0/3770187588 conn(0x7fba840f66a0 msgr2=0x7fba840f8ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:48:10.832 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:10.829+0000 7fba8adfc640 1 -- 172.21.15.176:0/3770187588 shutdown_connections 2024-07-16T09:48:10.832 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:10.829+0000 7fba8adfc640 1 -- 172.21.15.176:0/3770187588 wait complete. 2024-07-16T09:48:10.832 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:10.830+0000 7fba8adfc640 1 Processor -- start 2024-07-16T09:48:10.832 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:10.830+0000 7fba8adfc640 1 -- start start 2024-07-16T09:48:10.834 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:10.830+0000 7fba8adfc640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fba84065fb0 0x7fba8410e800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:48:10.834 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:10.830+0000 7fba8adfc640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba8410ed40 con 0x7fba84065fb0 2024-07-16T09:48:10.834 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:10.831+0000 7fba88b98640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fba84065fb0 0x7fba8410e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:48:10.834 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:10.831+0000 7fba88b98640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fba84065fb0 0x7fba8410e800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.176:39680/0 (socket says 172.21.15.176:39680) 2024-07-16T09:48:10.834 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:10.831+0000 7fba88b98640 1 -- 172.21.15.176:0/1835661821 learned_addr learned my addr 172.21.15.176:0/1835661821 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-07-16T09:48:10.834 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:10.831+0000 7fba88b98640 1 -- 172.21.15.176:0/1835661821 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba78005040 con 0x7fba84065fb0 2024-07-16T09:48:10.834 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:10.831+0000 7fba88b98640 1 --2- 172.21.15.176:0/1835661821 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fba84065fb0 0x7fba8410e800 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fba78004820 tx=0x7fba78009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:48:10.835 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:10.831+0000 7fba817fa640 1 -- 172.21.15.176:0/1835661821 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fba78032650 con 0x7fba84065fb0 2024-07-16T09:48:10.835 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:10.831+0000 7fba817fa640 1 -- 172.21.15.176:0/1835661821 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fba78032c80 con 0x7fba84065fb0 2024-07-16T09:48:10.835 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:10.831+0000 7fba8adfc640 1 -- 172.21.15.176:0/1835661821 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba8410ef40 con 0x7fba84065fb0 2024-07-16T09:48:10.835 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:10.831+0000 7fba817fa640 1 -- 172.21.15.176:0/1835661821 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fba78033d60 con 0x7fba84065fb0 2024-07-16T09:48:10.835 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:10.831+0000 7fba8adfc640 1 -- 172.21.15.176:0/1835661821 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba8410fad0 con 0x7fba84065fb0 2024-07-16T09:48:10.835 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:10.832+0000 7fba817fa640 1 -- 172.21.15.176:0/1835661821 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 13) v1 ==== 42693+0+0 (secure 0 0 0) 0x7fba78030020 con 0x7fba84065fb0 2024-07-16T09:48:10.835 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:10.832+0000 7fba8adfc640 1 -- 172.21.15.176:0/1835661821 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba84104750 con 0x7fba84065fb0 2024-07-16T09:48:10.835 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:10.832+0000 7fba817fa640 1 --2- 172.21.15.176:0/1835661821 >> [v2:172.21.15.156:6800/3984987611,v1:172.21.15.156:6801/3984987611] conn(0x7fba74035800 0x7fba74037cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:48:10.835 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:10.832+0000 7fba817fa640 1 -- 172.21.15.176:0/1835661821 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7fba780679e0 con 0x7fba84065fb0 2024-07-16T09:48:10.835 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:10.832+0000 7fba83fff640 1 --2- 172.21.15.176:0/1835661821 >> [v2:172.21.15.156:6800/3984987611,v1:172.21.15.156:6801/3984987611] conn(0x7fba74035800 0x7fba74037cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:48:10.835 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:10.833+0000 7fba83fff640 1 --2- 172.21.15.176:0/1835661821 >> [v2:172.21.15.156:6800/3984987611,v1:172.21.15.156:6801/3984987611] conn(0x7fba74035800 0x7fba74037cb0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fba70006fd0 tx=0x7fba70006db0 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:48:10.838 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:10.837+0000 7fba817fa640 1 -- 172.21.15.176:0/1835661821 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fba780453a0 con 0x7fba84065fb0 2024-07-16T09:48:11.068 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:11.066+0000 7fba8adfc640 1 -- 172.21.15.176:0/1835661821 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fba841104d0 con 0x7fba84065fb0 2024-07-16T09:48:11.068 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:11.067+0000 7fba817fa640 1 -- 172.21.15.176:0/1835661821 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+743 (secure 0 0 0) 0x7fba7803b320 con 0x7fba84065fb0 2024-07-16T09:48:11.069 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-07-16T09:48:11.069 INFO:teuthology.orchestra.run.smithi176.stdout:{"epoch":1,"fsid":"26c91c26-4358-11ef-bcad-c7b262605968","modified":"2024-07-16T09:46:20.852914Z","created":"2024-07-16T09:46:20.852914Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi156","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:3300","nonce":0},{"type":"v1","addr":"172.21.15.156:6789","nonce":0}]},"addr":"172.21.15.156:6789/0","public_addr":"172.21.15.156:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-07-16T09:48:11.069 INFO:teuthology.orchestra.run.smithi176.stderr:dumped monmap epoch 1 2024-07-16T09:48:11.070 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:11.069+0000 7fba8adfc640 1 -- 172.21.15.176:0/1835661821 >> [v2:172.21.15.156:6800/3984987611,v1:172.21.15.156:6801/3984987611] conn(0x7fba74035800 msgr2=0x7fba74037cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:48:11.070 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:11.069+0000 7fba8adfc640 1 --2- 172.21.15.176:0/1835661821 >> [v2:172.21.15.156:6800/3984987611,v1:172.21.15.156:6801/3984987611] conn(0x7fba74035800 0x7fba74037cb0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fba70006fd0 tx=0x7fba70006db0 comp rx=0 tx=0).stop 2024-07-16T09:48:11.071 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:11.069+0000 7fba8adfc640 1 -- 172.21.15.176:0/1835661821 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fba84065fb0 msgr2=0x7fba8410e800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:48:11.071 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:11.069+0000 7fba8adfc640 1 --2- 172.21.15.176:0/1835661821 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fba84065fb0 0x7fba8410e800 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fba78004820 tx=0x7fba78009da0 comp rx=0 tx=0).stop 2024-07-16T09:48:11.071 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:11.069+0000 7fba8adfc640 1 -- 172.21.15.176:0/1835661821 shutdown_connections 2024-07-16T09:48:11.071 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:11.069+0000 7fba8adfc640 1 --2- 172.21.15.176:0/1835661821 >> [v2:172.21.15.156:6800/3984987611,v1:172.21.15.156:6801/3984987611] conn(0x7fba74035800 0x7fba74037cb0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:48:11.071 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:11.069+0000 7fba8adfc640 1 --2- 172.21.15.176:0/1835661821 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fba84065fb0 0x7fba8410e800 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:48:11.071 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:11.069+0000 7fba8adfc640 1 -- 172.21.15.176:0/1835661821 >> 172.21.15.176:0/1835661821 conn(0x7fba840f66a0 msgr2=0x7fba84105380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:48:11.071 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:11.069+0000 7fba8adfc640 1 -- 172.21.15.176:0/1835661821 shutdown_connections 2024-07-16T09:48:11.071 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:11.069+0000 7fba8adfc640 1 -- 172.21.15.176:0/1835661821 wait complete. 2024-07-16T09:48:11.424 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:48:11 smithi156 ceph-mon[29302]: from='client.? 172.21.15.176:0/1835661821' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-07-16T09:48:12.424 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:48:12 smithi156 ceph-mon[29302]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-16T09:48:12.726 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-07-16T09:48:12.727 DEBUG:teuthology.orchestra.run.smithi176:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 26c91c26-4358-11ef-bcad-c7b262605968 -- ceph mon dump -f json 2024-07-16T09:48:12.995 INFO:teuthology.orchestra.run.smithi176.stderr:Inferring config /etc/ceph/ceph.conf 2024-07-16T09:48:13.043 INFO:teuthology.orchestra.run.smithi176.stderr:Inferring config /etc/ceph/ceph.conf 2024-07-16T09:48:13.924 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:48:13 smithi156 ceph-mon[29302]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-16T09:48:13.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:48:13 smithi156 ceph-mon[29302]: from='mgr.14160 172.21.15.156:0/3789331119' entity='mgr.smithi156.zkvylv' 2024-07-16T09:48:14.358 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:14.341+0000 7f2b3f14c640 1 -- 172.21.15.176:0/3130230837 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2b380fbfc0 msgr2=0x7f2b380fc3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:48:14.359 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:14.341+0000 7f2b3f14c640 1 --2- 172.21.15.176:0/3130230837 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2b380fbfc0 0x7f2b380fc3d0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f2b2c004850 tx=0x7f2b2c009da0 comp rx=0 tx=0).stop 2024-07-16T09:48:14.359 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:14.341+0000 7f2b3f14c640 1 -- 172.21.15.176:0/3130230837 shutdown_connections 2024-07-16T09:48:14.359 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:14.341+0000 7f2b3f14c640 1 --2- 172.21.15.176:0/3130230837 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2b380fbfc0 0x7f2b380fc3d0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:48:14.359 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:14.341+0000 7f2b3f14c640 1 -- 172.21.15.176:0/3130230837 >> 172.21.15.176:0/3130230837 conn(0x7f2b380f77a0 msgr2=0x7f2b380f9bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:48:14.359 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:14.341+0000 7f2b3f14c640 1 -- 172.21.15.176:0/3130230837 shutdown_connections 2024-07-16T09:48:14.359 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:14.341+0000 7f2b3f14c640 1 -- 172.21.15.176:0/3130230837 wait complete. 2024-07-16T09:48:14.359 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:14.341+0000 7f2b3f14c640 1 Processor -- start 2024-07-16T09:48:14.359 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:14.342+0000 7f2b3f14c640 1 -- start start 2024-07-16T09:48:14.359 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:14.342+0000 7f2b3f14c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2b380fbfc0 0x7f2b3810c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:48:14.359 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:14.342+0000 7f2b3f14c640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b3810cb00 con 0x7f2b380fbfc0 2024-07-16T09:48:14.359 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:14.342+0000 7f2b3cee8640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2b380fbfc0 0x7f2b3810c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:48:14.359 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:14.342+0000 7f2b3cee8640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2b380fbfc0 0x7f2b3810c5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.176:39694/0 (socket says 172.21.15.176:39694) 2024-07-16T09:48:14.359 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:14.342+0000 7f2b3cee8640 1 -- 172.21.15.176:0/148068298 learned_addr learned my addr 172.21.15.176:0/148068298 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-07-16T09:48:14.359 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:14.342+0000 7f2b3cee8640 1 -- 172.21.15.176:0/148068298 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b2c005040 con 0x7f2b380fbfc0 2024-07-16T09:48:14.360 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:14.343+0000 7f2b3cee8640 1 --2- 172.21.15.176:0/148068298 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2b380fbfc0 0x7f2b3810c5c0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f2b2c004820 tx=0x7f2b2c009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:48:14.360 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:14.343+0000 7f2b357fa640 1 -- 172.21.15.176:0/148068298 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2b2c032650 con 0x7f2b380fbfc0 2024-07-16T09:48:14.360 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:14.343+0000 7f2b357fa640 1 -- 172.21.15.176:0/148068298 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2b2c032c80 con 0x7f2b380fbfc0 2024-07-16T09:48:14.360 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:14.343+0000 7f2b3f14c640 1 -- 172.21.15.176:0/148068298 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b3810cd00 con 0x7f2b380fbfc0 2024-07-16T09:48:14.360 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:14.343+0000 7f2b357fa640 1 -- 172.21.15.176:0/148068298 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2b2c03eda0 con 0x7f2b380fbfc0 2024-07-16T09:48:14.360 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:14.343+0000 7f2b3f14c640 1 -- 172.21.15.176:0/148068298 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b3810d890 con 0x7f2b380fbfc0 2024-07-16T09:48:14.360 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:14.344+0000 7f2b357fa640 1 -- 172.21.15.176:0/148068298 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 13) v1 ==== 42693+0+0 (secure 0 0 0) 0x7f2b2c030020 con 0x7f2b380fbfc0 2024-07-16T09:48:14.360 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:14.344+0000 7f2b3f14c640 1 -- 172.21.15.176:0/148068298 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b38102530 con 0x7f2b380fbfc0 2024-07-16T09:48:14.360 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:14.344+0000 7f2b357fa640 1 --2- 172.21.15.176:0/148068298 >> [v2:172.21.15.156:6800/3984987611,v1:172.21.15.156:6801/3984987611] conn(0x7f2b280357b0 0x7f2b28037c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:48:14.360 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:14.344+0000 7f2b357fa640 1 -- 172.21.15.176:0/148068298 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f2b2c067c30 con 0x7f2b380fbfc0 2024-07-16T09:48:14.360 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:14.344+0000 7f2b37fff640 1 --2- 172.21.15.176:0/148068298 >> [v2:172.21.15.156:6800/3984987611,v1:172.21.15.156:6801/3984987611] conn(0x7f2b280357b0 0x7f2b28037c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:48:14.361 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:14.344+0000 7f2b37fff640 1 --2- 172.21.15.176:0/148068298 >> [v2:172.21.15.156:6800/3984987611,v1:172.21.15.156:6801/3984987611] conn(0x7f2b280357b0 0x7f2b28037c60 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f2b24006fd0 tx=0x7f2b24006db0 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:48:14.361 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:14.349+0000 7f2b357fa640 1 -- 172.21.15.176:0/148068298 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f2b2c02c900 con 0x7f2b380fbfc0 2024-07-16T09:48:14.580 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:14.579+0000 7f2b3f14c640 1 -- 172.21.15.176:0/148068298 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f2b3810e290 con 0x7f2b380fbfc0 2024-07-16T09:48:14.581 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:14.579+0000 7f2b357fa640 1 -- 172.21.15.176:0/148068298 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+743 (secure 0 0 0) 0x7f2b2c04c300 con 0x7f2b380fbfc0 2024-07-16T09:48:14.581 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-07-16T09:48:14.581 INFO:teuthology.orchestra.run.smithi176.stdout:{"epoch":1,"fsid":"26c91c26-4358-11ef-bcad-c7b262605968","modified":"2024-07-16T09:46:20.852914Z","created":"2024-07-16T09:46:20.852914Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi156","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:3300","nonce":0},{"type":"v1","addr":"172.21.15.156:6789","nonce":0}]},"addr":"172.21.15.156:6789/0","public_addr":"172.21.15.156:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-07-16T09:48:14.581 INFO:teuthology.orchestra.run.smithi176.stderr:dumped monmap epoch 1 2024-07-16T09:48:14.583 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:14.582+0000 7f2b3f14c640 1 -- 172.21.15.176:0/148068298 >> [v2:172.21.15.156:6800/3984987611,v1:172.21.15.156:6801/3984987611] conn(0x7f2b280357b0 msgr2=0x7f2b28037c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:48:14.583 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:14.582+0000 7f2b3f14c640 1 --2- 172.21.15.176:0/148068298 >> [v2:172.21.15.156:6800/3984987611,v1:172.21.15.156:6801/3984987611] conn(0x7f2b280357b0 0x7f2b28037c60 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f2b24006fd0 tx=0x7f2b24006db0 comp rx=0 tx=0).stop 2024-07-16T09:48:14.583 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:14.582+0000 7f2b3f14c640 1 -- 172.21.15.176:0/148068298 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2b380fbfc0 msgr2=0x7f2b3810c5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:48:14.583 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:14.582+0000 7f2b3f14c640 1 --2- 172.21.15.176:0/148068298 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2b380fbfc0 0x7f2b3810c5c0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f2b2c004820 tx=0x7f2b2c009da0 comp rx=0 tx=0).stop 2024-07-16T09:48:14.583 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:14.582+0000 7f2b3f14c640 1 -- 172.21.15.176:0/148068298 shutdown_connections 2024-07-16T09:48:14.584 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:14.582+0000 7f2b3f14c640 1 --2- 172.21.15.176:0/148068298 >> [v2:172.21.15.156:6800/3984987611,v1:172.21.15.156:6801/3984987611] conn(0x7f2b280357b0 0x7f2b28037c60 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:48:14.584 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:14.582+0000 7f2b3f14c640 1 --2- 172.21.15.176:0/148068298 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2b380fbfc0 0x7f2b3810c5c0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:48:14.584 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:14.582+0000 7f2b3f14c640 1 -- 172.21.15.176:0/148068298 >> 172.21.15.176:0/148068298 conn(0x7f2b380f77a0 msgr2=0x7f2b38103160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:48:14.584 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:14.582+0000 7f2b3f14c640 1 -- 172.21.15.176:0/148068298 shutdown_connections 2024-07-16T09:48:14.584 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:14.582+0000 7f2b3f14c640 1 -- 172.21.15.176:0/148068298 wait complete. 2024-07-16T09:48:14.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:48:14 smithi156 ceph-mon[29302]: from='client.? 172.21.15.176:0/148068298' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-07-16T09:48:15.924 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:48:15 smithi156 ceph-mon[29302]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-16T09:48:16.208 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-07-16T09:48:16.208 DEBUG:teuthology.orchestra.run.smithi176:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 26c91c26-4358-11ef-bcad-c7b262605968 -- ceph mon dump -f json 2024-07-16T09:48:16.458 INFO:teuthology.orchestra.run.smithi176.stderr:Inferring config /etc/ceph/ceph.conf 2024-07-16T09:48:16.505 INFO:teuthology.orchestra.run.smithi176.stderr:Inferring config /etc/ceph/ceph.conf 2024-07-16T09:48:17.655 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:48:17 smithi156 ceph-mon[29302]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-16T09:48:17.905 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:17.900+0000 7fa49123c640 1 -- 172.21.15.176:0/3507183741 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa48c0fbfc0 msgr2=0x7fa48c0fc3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:48:17.905 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:17.900+0000 7fa49123c640 1 --2- 172.21.15.176:0/3507183741 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa48c0fbfc0 0x7fa48c0fc3d0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fa484004850 tx=0x7fa484009da0 comp rx=0 tx=0).stop 2024-07-16T09:48:17.905 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:17.900+0000 7fa49123c640 1 -- 172.21.15.176:0/3507183741 shutdown_connections 2024-07-16T09:48:17.906 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:17.900+0000 7fa49123c640 1 --2- 172.21.15.176:0/3507183741 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa48c0fbfc0 0x7fa48c0fc3d0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:48:17.906 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:17.900+0000 7fa49123c640 1 -- 172.21.15.176:0/3507183741 >> 172.21.15.176:0/3507183741 conn(0x7fa48c0f77a0 msgr2=0x7fa48c0f9bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:48:17.906 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:17.900+0000 7fa49123c640 1 -- 172.21.15.176:0/3507183741 shutdown_connections 2024-07-16T09:48:17.906 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:17.900+0000 7fa49123c640 1 -- 172.21.15.176:0/3507183741 wait complete. 2024-07-16T09:48:17.906 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:17.901+0000 7fa49123c640 1 Processor -- start 2024-07-16T09:48:17.906 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:17.901+0000 7fa49123c640 1 -- start start 2024-07-16T09:48:17.906 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:17.901+0000 7fa49123c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa48c0fbfc0 0x7fa48c10c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:48:17.906 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:17.901+0000 7fa49123c640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa48c10cb00 con 0x7fa48c0fbfc0 2024-07-16T09:48:17.906 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:17.901+0000 7fa48ad9d640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa48c0fbfc0 0x7fa48c10c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:48:17.906 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:17.902+0000 7fa48ad9d640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa48c0fbfc0 0x7fa48c10c5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.176:38306/0 (socket says 172.21.15.176:38306) 2024-07-16T09:48:17.906 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:17.902+0000 7fa48ad9d640 1 -- 172.21.15.176:0/2751800877 learned_addr learned my addr 172.21.15.176:0/2751800877 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-07-16T09:48:17.907 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:17.902+0000 7fa48ad9d640 1 -- 172.21.15.176:0/2751800877 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa484005040 con 0x7fa48c0fbfc0 2024-07-16T09:48:17.907 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:17.902+0000 7fa48ad9d640 1 --2- 172.21.15.176:0/2751800877 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa48c0fbfc0 0x7fa48c10c5c0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fa4840098d0 tx=0x7fa484009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:48:17.907 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:17.902+0000 7fa4737fe640 1 -- 172.21.15.176:0/2751800877 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa484032650 con 0x7fa48c0fbfc0 2024-07-16T09:48:17.907 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:17.902+0000 7fa49123c640 1 -- 172.21.15.176:0/2751800877 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa48c10cd00 con 0x7fa48c0fbfc0 2024-07-16T09:48:17.907 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:17.902+0000 7fa49123c640 1 -- 172.21.15.176:0/2751800877 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa48c10d170 con 0x7fa48c0fbfc0 2024-07-16T09:48:17.907 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:17.902+0000 7fa4737fe640 1 -- 172.21.15.176:0/2751800877 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa484032c80 con 0x7fa48c0fbfc0 2024-07-16T09:48:17.907 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:17.902+0000 7fa4737fe640 1 -- 172.21.15.176:0/2751800877 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa484033bb0 con 0x7fa48c0fbfc0 2024-07-16T09:48:17.907 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:17.903+0000 7fa49123c640 1 -- 172.21.15.176:0/2751800877 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa478005240 con 0x7fa48c0fbfc0 2024-07-16T09:48:17.907 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:17.903+0000 7fa4737fe640 1 -- 172.21.15.176:0/2751800877 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 13) v1 ==== 42693+0+0 (secure 0 0 0) 0x7fa484030020 con 0x7fa48c0fbfc0 2024-07-16T09:48:17.907 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:17.903+0000 7fa4737fe640 1 --2- 172.21.15.176:0/2751800877 >> [v2:172.21.15.156:6800/3984987611,v1:172.21.15.156:6801/3984987611] conn(0x7fa4740357b0 0x7fa474037c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:48:17.907 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:17.903+0000 7fa4737fe640 1 -- 172.21.15.176:0/2751800877 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7fa484067840 con 0x7fa48c0fbfc0 2024-07-16T09:48:17.907 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:17.904+0000 7fa48a59c640 1 --2- 172.21.15.176:0/2751800877 >> [v2:172.21.15.156:6800/3984987611,v1:172.21.15.156:6801/3984987611] conn(0x7fa4740357b0 0x7fa474037c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:48:17.908 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:17.904+0000 7fa48a59c640 1 --2- 172.21.15.176:0/2751800877 >> [v2:172.21.15.156:6800/3984987611,v1:172.21.15.156:6801/3984987611] conn(0x7fa4740357b0 0x7fa474037c60 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fa48000ad90 tx=0x7fa480009410 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:48:17.909 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:17.908+0000 7fa4737fe640 1 -- 172.21.15.176:0/2751800877 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fa484033d10 con 0x7fa48c0fbfc0 2024-07-16T09:48:18.137 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:18.136+0000 7fa49123c640 1 -- 172.21.15.176:0/2751800877 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fa4780054f0 con 0x7fa48c0fbfc0 2024-07-16T09:48:18.138 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:18.137+0000 7fa4737fe640 1 -- 172.21.15.176:0/2751800877 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+743 (secure 0 0 0) 0x7fa484044b40 con 0x7fa48c0fbfc0 2024-07-16T09:48:18.138 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-07-16T09:48:18.138 INFO:teuthology.orchestra.run.smithi176.stdout:{"epoch":1,"fsid":"26c91c26-4358-11ef-bcad-c7b262605968","modified":"2024-07-16T09:46:20.852914Z","created":"2024-07-16T09:46:20.852914Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi156","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:3300","nonce":0},{"type":"v1","addr":"172.21.15.156:6789","nonce":0}]},"addr":"172.21.15.156:6789/0","public_addr":"172.21.15.156:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-07-16T09:48:18.139 INFO:teuthology.orchestra.run.smithi176.stderr:dumped monmap epoch 1 2024-07-16T09:48:18.140 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:18.139+0000 7fa49123c640 1 -- 172.21.15.176:0/2751800877 >> [v2:172.21.15.156:6800/3984987611,v1:172.21.15.156:6801/3984987611] conn(0x7fa4740357b0 msgr2=0x7fa474037c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:48:18.140 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:18.139+0000 7fa49123c640 1 --2- 172.21.15.176:0/2751800877 >> [v2:172.21.15.156:6800/3984987611,v1:172.21.15.156:6801/3984987611] conn(0x7fa4740357b0 0x7fa474037c60 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fa48000ad90 tx=0x7fa480009410 comp rx=0 tx=0).stop 2024-07-16T09:48:18.140 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:18.139+0000 7fa49123c640 1 -- 172.21.15.176:0/2751800877 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa48c0fbfc0 msgr2=0x7fa48c10c5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:48:18.140 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:18.139+0000 7fa49123c640 1 --2- 172.21.15.176:0/2751800877 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa48c0fbfc0 0x7fa48c10c5c0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fa4840098d0 tx=0x7fa484009da0 comp rx=0 tx=0).stop 2024-07-16T09:48:18.140 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:18.139+0000 7fa49123c640 1 -- 172.21.15.176:0/2751800877 shutdown_connections 2024-07-16T09:48:18.140 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:18.139+0000 7fa49123c640 1 --2- 172.21.15.176:0/2751800877 >> [v2:172.21.15.156:6800/3984987611,v1:172.21.15.156:6801/3984987611] conn(0x7fa4740357b0 0x7fa474037c60 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:48:18.141 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:18.139+0000 7fa49123c640 1 --2- 172.21.15.176:0/2751800877 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa48c0fbfc0 0x7fa48c10c5c0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:48:18.141 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:18.139+0000 7fa49123c640 1 -- 172.21.15.176:0/2751800877 >> 172.21.15.176:0/2751800877 conn(0x7fa48c0f77a0 msgr2=0x7fa48c103160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:48:18.141 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:18.139+0000 7fa49123c640 1 -- 172.21.15.176:0/2751800877 shutdown_connections 2024-07-16T09:48:18.141 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:18.139+0000 7fa49123c640 1 -- 172.21.15.176:0/2751800877 wait complete. 2024-07-16T09:48:18.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:48:18 smithi156 ceph-mon[29302]: from='client.? 172.21.15.176:0/2751800877' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-07-16T09:48:19.709 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-07-16T09:48:19.709 DEBUG:teuthology.orchestra.run.smithi176:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 26c91c26-4358-11ef-bcad-c7b262605968 -- ceph mon dump -f json 2024-07-16T09:48:19.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:48:19 smithi156 ceph-mon[29302]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-16T09:48:19.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:48:19 smithi156 ceph-mon[29302]: from='mgr.14160 172.21.15.156:0/3789331119' entity='mgr.smithi156.zkvylv' 2024-07-16T09:48:19.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:48:19 smithi156 ceph-mon[29302]: from='mgr.14160 172.21.15.156:0/3789331119' entity='mgr.smithi156.zkvylv' 2024-07-16T09:48:19.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:48:19 smithi156 ceph-mon[29302]: from='mgr.14160 172.21.15.156:0/3789331119' entity='mgr.smithi156.zkvylv' 2024-07-16T09:48:19.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:48:19 smithi156 ceph-mon[29302]: from='mgr.14160 172.21.15.156:0/3789331119' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-07-16T09:48:19.962 INFO:teuthology.orchestra.run.smithi176.stderr:Inferring config /etc/ceph/ceph.conf 2024-07-16T09:48:20.009 INFO:teuthology.orchestra.run.smithi176.stderr:Inferring config /etc/ceph/ceph.conf 2024-07-16T09:48:21.492 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:21.451+0000 7f39f7467640 1 -- 172.21.15.176:0/3532821914 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f39f0065460 msgr2=0x7f39f00fee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:48:21.493 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:21.451+0000 7f39f7467640 1 --2- 172.21.15.176:0/3532821914 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f39f0065460 0x7f39f00fee00 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f39e0004850 tx=0x7f39e0009da0 comp rx=0 tx=0).stop 2024-07-16T09:48:21.493 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:21.451+0000 7f39f7467640 1 -- 172.21.15.176:0/3532821914 shutdown_connections 2024-07-16T09:48:21.493 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:21.451+0000 7f39f7467640 1 --2- 172.21.15.176:0/3532821914 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f39f0065460 0x7f39f00fee00 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:48:21.493 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:21.451+0000 7f39f7467640 1 -- 172.21.15.176:0/3532821914 >> 172.21.15.176:0/3532821914 conn(0x7f39f00f68d0 msgr2=0x7f39f00f8ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:48:21.493 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:21.451+0000 7f39f7467640 1 -- 172.21.15.176:0/3532821914 shutdown_connections 2024-07-16T09:48:21.493 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:21.451+0000 7f39f7467640 1 -- 172.21.15.176:0/3532821914 wait complete. 2024-07-16T09:48:21.493 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:21.452+0000 7f39f7467640 1 Processor -- start 2024-07-16T09:48:21.493 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:21.452+0000 7f39f7467640 1 -- start start 2024-07-16T09:48:21.493 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:21.452+0000 7f39f7467640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f39f0065460 0x7f39f010c860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:48:21.493 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:21.452+0000 7f39f7467640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39f010cda0 con 0x7f39f0065460 2024-07-16T09:48:21.494 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:21.452+0000 7f39f5203640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f39f0065460 0x7f39f010c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:48:21.494 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:21.452+0000 7f39f5203640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f39f0065460 0x7f39f010c860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.176:38330/0 (socket says 172.21.15.176:38330) 2024-07-16T09:48:21.494 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:21.452+0000 7f39f5203640 1 -- 172.21.15.176:0/540229878 learned_addr learned my addr 172.21.15.176:0/540229878 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-07-16T09:48:21.494 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:21.452+0000 7f39f5203640 1 -- 172.21.15.176:0/540229878 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39e0005040 con 0x7f39f0065460 2024-07-16T09:48:21.494 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:21.453+0000 7f39f5203640 1 --2- 172.21.15.176:0/540229878 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f39f0065460 0x7f39f010c860 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f39e0004820 tx=0x7f39e0009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:48:21.494 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:21.453+0000 7f39e5ffb640 1 -- 172.21.15.176:0/540229878 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f39e0032650 con 0x7f39f0065460 2024-07-16T09:48:21.494 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:21.453+0000 7f39e5ffb640 1 -- 172.21.15.176:0/540229878 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f39e0032c80 con 0x7f39f0065460 2024-07-16T09:48:21.494 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:21.453+0000 7f39f7467640 1 -- 172.21.15.176:0/540229878 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39f010cfa0 con 0x7f39f0065460 2024-07-16T09:48:21.494 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:21.453+0000 7f39e5ffb640 1 -- 172.21.15.176:0/540229878 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f39e0033d60 con 0x7f39f0065460 2024-07-16T09:48:21.494 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:21.453+0000 7f39f7467640 1 -- 172.21.15.176:0/540229878 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39f010db30 con 0x7f39f0065460 2024-07-16T09:48:21.494 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:21.454+0000 7f39e5ffb640 1 -- 172.21.15.176:0/540229878 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 14) v1 ==== 42707+0+0 (secure 0 0 0) 0x7f39e0030020 con 0x7f39f0065460 2024-07-16T09:48:21.495 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:21.454+0000 7f39e5ffb640 1 --2- 172.21.15.176:0/540229878 >> [v2:172.21.15.156:6800/3984987611,v1:172.21.15.156:6801/3984987611] conn(0x7f39dc035800 0x7f39dc037cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:48:21.495 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:21.454+0000 7f39e5ffb640 1 -- 172.21.15.176:0/540229878 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f39e00679d0 con 0x7f39f0065460 2024-07-16T09:48:21.495 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:21.454+0000 7f39f7467640 1 -- 172.21.15.176:0/540229878 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39f0102760 con 0x7f39f0065460 2024-07-16T09:48:21.495 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:21.454+0000 7f39f4a02640 1 -- 172.21.15.176:0/540229878 >> [v2:172.21.15.156:6800/3984987611,v1:172.21.15.156:6801/3984987611] conn(0x7f39dc035800 msgr2=0x7f39dc037cb0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.156:6800/3984987611 2024-07-16T09:48:21.495 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:21.454+0000 7f39f4a02640 1 --2- 172.21.15.176:0/540229878 >> [v2:172.21.15.156:6800/3984987611,v1:172.21.15.156:6801/3984987611] conn(0x7f39dc035800 0x7f39dc037cb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-16T09:48:21.495 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:21.458+0000 7f39e5ffb640 1 -- 172.21.15.176:0/540229878 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f39e0032df0 con 0x7f39f0065460 2024-07-16T09:48:21.656 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:21.655+0000 7f39f4a02640 1 -- 172.21.15.176:0/540229878 >> [v2:172.21.15.156:6800/3984987611,v1:172.21.15.156:6801/3984987611] conn(0x7f39dc035800 msgr2=0x7f39dc037cb0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.156:6800/3984987611 2024-07-16T09:48:21.656 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:21.655+0000 7f39f4a02640 1 --2- 172.21.15.176:0/540229878 >> [v2:172.21.15.156:6800/3984987611,v1:172.21.15.156:6801/3984987611] conn(0x7f39dc035800 0x7f39dc037cb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-16T09:48:21.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:48:21 smithi156 ceph-mon[29302]: from='mgr.14160 172.21.15.156:0/3789331119' entity='mgr.smithi156.zkvylv' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-07-16T09:48:21.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:48:21 smithi156 ceph-mon[29302]: mgrmap e14: smithi156.zkvylv(active, since 57s) 2024-07-16T09:48:21.687 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:21.686+0000 7f39f7467640 1 -- 172.21.15.176:0/540229878 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f39f005eb80 con 0x7f39f0065460 2024-07-16T09:48:21.687 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:21.686+0000 7f39e5ffb640 1 -- 172.21.15.176:0/540229878 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+743 (secure 0 0 0) 0x7f39f005eb80 con 0x7f39f0065460 2024-07-16T09:48:21.688 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-07-16T09:48:21.688 INFO:teuthology.orchestra.run.smithi176.stdout:{"epoch":1,"fsid":"26c91c26-4358-11ef-bcad-c7b262605968","modified":"2024-07-16T09:46:20.852914Z","created":"2024-07-16T09:46:20.852914Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi156","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:3300","nonce":0},{"type":"v1","addr":"172.21.15.156:6789","nonce":0}]},"addr":"172.21.15.156:6789/0","public_addr":"172.21.15.156:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-07-16T09:48:21.688 INFO:teuthology.orchestra.run.smithi176.stderr:dumped monmap epoch 1 2024-07-16T09:48:21.689 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:21.688+0000 7f39f7467640 1 -- 172.21.15.176:0/540229878 >> [v2:172.21.15.156:6800/3984987611,v1:172.21.15.156:6801/3984987611] conn(0x7f39dc035800 msgr2=0x7f39dc037cb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-16T09:48:21.689 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:21.688+0000 7f39f7467640 1 --2- 172.21.15.176:0/540229878 >> [v2:172.21.15.156:6800/3984987611,v1:172.21.15.156:6801/3984987611] conn(0x7f39dc035800 0x7f39dc037cb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:48:21.690 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:21.688+0000 7f39f7467640 1 -- 172.21.15.176:0/540229878 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f39f0065460 msgr2=0x7f39f010c860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:48:21.690 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:21.688+0000 7f39f7467640 1 --2- 172.21.15.176:0/540229878 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f39f0065460 0x7f39f010c860 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f39e0004820 tx=0x7f39e0009da0 comp rx=0 tx=0).stop 2024-07-16T09:48:21.690 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:21.688+0000 7f39f7467640 1 -- 172.21.15.176:0/540229878 shutdown_connections 2024-07-16T09:48:21.690 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:21.688+0000 7f39f7467640 1 --2- 172.21.15.176:0/540229878 >> [v2:172.21.15.156:6800/3984987611,v1:172.21.15.156:6801/3984987611] conn(0x7f39dc035800 0x7f39dc037cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:48:21.690 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:21.688+0000 7f39f7467640 1 --2- 172.21.15.176:0/540229878 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f39f0065460 0x7f39f010c860 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:48:21.690 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:21.688+0000 7f39f7467640 1 -- 172.21.15.176:0/540229878 >> 172.21.15.176:0/540229878 conn(0x7f39f00f68d0 msgr2=0x7f39f0103390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:48:21.690 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:21.688+0000 7f39f7467640 1 -- 172.21.15.176:0/540229878 shutdown_connections 2024-07-16T09:48:21.690 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:21.688+0000 7f39f7467640 1 -- 172.21.15.176:0/540229878 wait complete. 2024-07-16T09:48:22.450 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:48:22 smithi156 ceph-mon[29302]: from='client.? 172.21.15.176:0/540229878' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-07-16T09:48:23.213 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-07-16T09:48:23.213 DEBUG:teuthology.orchestra.run.smithi176:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 26c91c26-4358-11ef-bcad-c7b262605968 -- ceph mon dump -f json 2024-07-16T09:48:23.464 INFO:teuthology.orchestra.run.smithi176.stderr:Inferring config /etc/ceph/ceph.conf 2024-07-16T09:48:23.513 INFO:teuthology.orchestra.run.smithi176.stderr:Inferring config /etc/ceph/ceph.conf 2024-07-16T09:48:24.885 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:48:24 smithi156 ceph-mon[29302]: Active manager daemon smithi156.zkvylv restarted 2024-07-16T09:48:24.885 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:48:24 smithi156 ceph-mon[29302]: Activating manager daemon smithi156.zkvylv 2024-07-16T09:48:24.885 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:48:24 smithi156 ceph-mon[29302]: osdmap e5: 0 total, 0 up, 0 in 2024-07-16T09:48:24.885 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:48:24 smithi156 ceph-mon[29302]: mgrmap e15: smithi156.zkvylv(active, starting, since 0.00311774s) 2024-07-16T09:48:24.885 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:48:24 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "mon metadata", "id": "smithi156"}]: dispatch 2024-07-16T09:48:24.886 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:48:24 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "mgr metadata", "who": "smithi156.zkvylv", "id": "smithi156.zkvylv"}]: dispatch 2024-07-16T09:48:24.886 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:48:24 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-07-16T09:48:24.886 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:48:24 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-07-16T09:48:24.886 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:48:24 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-07-16T09:48:24.886 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:48:24 smithi156 ceph-mon[29302]: Manager daemon smithi156.zkvylv is now available 2024-07-16T09:48:24.886 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:48:24 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-16T09:48:24.886 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:48:24 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-16T09:48:24.886 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:48:24 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:48:24.886 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:48:24 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-16T09:48:24.886 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:48:24 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi156.zkvylv/mirror_snapshot_schedule"}]: dispatch 2024-07-16T09:48:25.214 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:25.183+0000 7f8d61cf1640 1 -- 172.21.15.176:0/1140025711 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8d5c0653e0 msgr2=0x7f8d5c0fee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:48:25.214 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:25.183+0000 7f8d61cf1640 1 --2- 172.21.15.176:0/1140025711 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8d5c0653e0 0x7f8d5c0fee80 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f8d4c004850 tx=0x7f8d4c009da0 comp rx=0 tx=0).stop 2024-07-16T09:48:25.214 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:25.183+0000 7f8d61cf1640 1 -- 172.21.15.176:0/1140025711 shutdown_connections 2024-07-16T09:48:25.214 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:25.183+0000 7f8d61cf1640 1 --2- 172.21.15.176:0/1140025711 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8d5c0653e0 0x7f8d5c0fee80 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:48:25.214 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:25.183+0000 7f8d61cf1640 1 -- 172.21.15.176:0/1140025711 >> 172.21.15.176:0/1140025711 conn(0x7f8d5c0f68d0 msgr2=0x7f8d5c0f8ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:48:25.214 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:25.183+0000 7f8d61cf1640 1 -- 172.21.15.176:0/1140025711 shutdown_connections 2024-07-16T09:48:25.214 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:25.183+0000 7f8d61cf1640 1 -- 172.21.15.176:0/1140025711 wait complete. 2024-07-16T09:48:25.214 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:25.184+0000 7f8d61cf1640 1 Processor -- start 2024-07-16T09:48:25.214 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:25.184+0000 7f8d61cf1640 1 -- start start 2024-07-16T09:48:25.214 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:25.184+0000 7f8d61cf1640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8d5c0653e0 0x7f8d5c0fc7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:48:25.215 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:25.184+0000 7f8d61cf1640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d5c0fccf0 con 0x7f8d5c0653e0 2024-07-16T09:48:25.215 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:25.184+0000 7f8d5b7fe640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8d5c0653e0 0x7f8d5c0fc7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:48:25.215 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:25.184+0000 7f8d5b7fe640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8d5c0653e0 0x7f8d5c0fc7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.176:38352/0 (socket says 172.21.15.176:38352) 2024-07-16T09:48:25.215 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:25.184+0000 7f8d5b7fe640 1 -- 172.21.15.176:0/2490149436 learned_addr learned my addr 172.21.15.176:0/2490149436 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-07-16T09:48:25.215 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:25.185+0000 7f8d5b7fe640 1 -- 172.21.15.176:0/2490149436 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d4c005040 con 0x7f8d5c0653e0 2024-07-16T09:48:25.215 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:25.185+0000 7f8d5b7fe640 1 --2- 172.21.15.176:0/2490149436 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8d5c0653e0 0x7f8d5c0fc7b0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f8d4c004820 tx=0x7f8d4c009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:48:25.215 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:25.185+0000 7f8d43fff640 1 -- 172.21.15.176:0/2490149436 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8d4c032650 con 0x7f8d5c0653e0 2024-07-16T09:48:25.215 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:25.185+0000 7f8d43fff640 1 -- 172.21.15.176:0/2490149436 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8d4c032c80 con 0x7f8d5c0653e0 2024-07-16T09:48:25.215 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:25.185+0000 7f8d61cf1640 1 -- 172.21.15.176:0/2490149436 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d5c0fad80 con 0x7f8d5c0653e0 2024-07-16T09:48:25.215 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:25.185+0000 7f8d43fff640 1 -- 172.21.15.176:0/2490149436 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8d4c033d60 con 0x7f8d5c0653e0 2024-07-16T09:48:25.215 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:25.185+0000 7f8d61cf1640 1 -- 172.21.15.176:0/2490149436 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d5c0fb250 con 0x7f8d5c0653e0 2024-07-16T09:48:25.215 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:25.186+0000 7f8d43fff640 1 -- 172.21.15.176:0/2490149436 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 15) v1 ==== 42430+0+0 (secure 0 0 0) 0x7f8d4c030020 con 0x7f8d5c0653e0 2024-07-16T09:48:25.216 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:25.186+0000 7f8d43fff640 1 -- 172.21.15.176:0/2490149436 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1366+0+0 (secure 0 0 0) 0x7f8d4c0675a0 con 0x7f8d5c0653e0 2024-07-16T09:48:25.216 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:25.186+0000 7f8d61cf1640 1 -- 172.21.15.176:0/2490149436 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d5c05eb80 con 0x7f8d5c0653e0 2024-07-16T09:48:25.216 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:25.191+0000 7f8d43fff640 1 -- 172.21.15.176:0/2490149436 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f8d4c03c380 con 0x7f8d5c0653e0 2024-07-16T09:48:25.435 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:25.434+0000 7f8d61cf1640 1 -- 172.21.15.176:0/2490149436 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f8d5c101fa0 con 0x7f8d5c0653e0 2024-07-16T09:48:25.436 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:25.435+0000 7f8d43fff640 1 -- 172.21.15.176:0/2490149436 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+743 (secure 0 0 0) 0x7f8d4c02c110 con 0x7f8d5c0653e0 2024-07-16T09:48:25.436 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-07-16T09:48:25.436 INFO:teuthology.orchestra.run.smithi176.stdout:{"epoch":1,"fsid":"26c91c26-4358-11ef-bcad-c7b262605968","modified":"2024-07-16T09:46:20.852914Z","created":"2024-07-16T09:46:20.852914Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi156","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:3300","nonce":0},{"type":"v1","addr":"172.21.15.156:6789","nonce":0}]},"addr":"172.21.15.156:6789/0","public_addr":"172.21.15.156:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-07-16T09:48:25.437 INFO:teuthology.orchestra.run.smithi176.stderr:dumped monmap epoch 1 2024-07-16T09:48:25.438 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:25.436+0000 7f8d61cf1640 1 -- 172.21.15.176:0/2490149436 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8d5c0653e0 msgr2=0x7f8d5c0fc7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:48:25.438 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:25.437+0000 7f8d61cf1640 1 --2- 172.21.15.176:0/2490149436 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8d5c0653e0 0x7f8d5c0fc7b0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f8d4c004820 tx=0x7f8d4c009da0 comp rx=0 tx=0).stop 2024-07-16T09:48:25.438 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:25.437+0000 7f8d61cf1640 1 -- 172.21.15.176:0/2490149436 shutdown_connections 2024-07-16T09:48:25.438 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:25.437+0000 7f8d61cf1640 1 --2- 172.21.15.176:0/2490149436 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8d5c0653e0 0x7f8d5c0fc7b0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:48:25.438 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:25.437+0000 7f8d61cf1640 1 -- 172.21.15.176:0/2490149436 >> 172.21.15.176:0/2490149436 conn(0x7f8d5c0f68d0 msgr2=0x7f8d5c1033a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:48:25.438 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:25.437+0000 7f8d61cf1640 1 -- 172.21.15.176:0/2490149436 shutdown_connections 2024-07-16T09:48:25.438 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:25.437+0000 7f8d61cf1640 1 -- 172.21.15.176:0/2490149436 wait complete. 2024-07-16T09:48:26.174 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:48:25 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi156.zkvylv/trash_purge_schedule"}]: dispatch 2024-07-16T09:48:26.175 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:48:25 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:48:26.175 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:48:25 smithi156 ceph-mon[29302]: from='client.? 172.21.15.176:0/2490149436' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-07-16T09:48:26.175 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:48:25 smithi156 ceph-mon[29302]: mgrmap e16: smithi156.zkvylv(active, since 1.00538s) 2024-07-16T09:48:26.962 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-07-16T09:48:26.962 DEBUG:teuthology.orchestra.run.smithi176:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 26c91c26-4358-11ef-bcad-c7b262605968 -- ceph mon dump -f json 2024-07-16T09:48:27.175 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:48:26 smithi156 ceph-mon[29302]: [16/Jul/2024:09:48:25] ENGINE Bus STARTING 2024-07-16T09:48:27.175 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:48:26 smithi156 ceph-mon[29302]: [16/Jul/2024:09:48:25] ENGINE Serving on https://172.21.15.156:7150 2024-07-16T09:48:27.175 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:48:26 smithi156 ceph-mon[29302]: [16/Jul/2024:09:48:25] ENGINE Bus STARTED 2024-07-16T09:48:27.175 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:48:26 smithi156 ceph-mon[29302]: [16/Jul/2024:09:48:25] ENGINE Error in HTTPServer.serve 2024-07-16T09:48:27.175 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: Traceback (most recent call last): 2024-07-16T09:48:27.175 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-07-16T09:48:27.175 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: self._connections.run(self.expiration_interval) 2024-07-16T09:48:27.175 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-07-16T09:48:27.175 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: self._run(expiration_interval) 2024-07-16T09:48:27.175 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-07-16T09:48:27.175 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-07-16T09:48:27.175 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-07-16T09:48:27.176 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-07-16T09:48:27.176 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-07-16T09:48:27.176 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: s = self.context.wrap_socket( 2024-07-16T09:48:27.176 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-07-16T09:48:27.176 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: return self.sslsocket_class._create( 2024-07-16T09:48:27.176 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-07-16T09:48:27.176 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: self.do_handshake() 2024-07-16T09:48:27.176 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-07-16T09:48:27.176 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: self._sslobj.do_handshake() 2024-07-16T09:48:27.176 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-07-16T09:48:27.176 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:48:26 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:48:27.176 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:48:26 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:48:27.176 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:48:26 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:48:27.809 INFO:teuthology.orchestra.run.smithi176.stderr:Inferring config /etc/ceph/ceph.conf 2024-07-16T09:48:27.859 INFO:teuthology.orchestra.run.smithi176.stderr:Inferring config /etc/ceph/ceph.conf 2024-07-16T09:48:28.924 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:48:28 smithi156 ceph-mon[29302]: mgrmap e17: smithi156.zkvylv(active, since 2s) 2024-07-16T09:48:30.175 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:48:29 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:48:30.175 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:48:29 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:48:30.175 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:48:29 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:48:30.175 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:48:29 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:48:30.175 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:48:29 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:48:30.175 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:48:29 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:48:30.175 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:48:29 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:48:30.175 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:48:29 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config rm", "who": "osd/host:smithi156", "name": "osd_memory_target"}]: dispatch 2024-07-16T09:48:30.175 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:48:29 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:48:30.175 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:48:29 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config rm", "who": "osd/host:smithi176", "name": "osd_memory_target"}]: dispatch 2024-07-16T09:48:30.176 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:48:29 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-16T09:48:30.176 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:48:29 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-16T09:48:30.454 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:30.453+0000 7ff371426640 1 -- 172.21.15.176:0/954753994 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff36c100ef0 msgr2=0x7ff36c1012c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:48:30.455 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:30.453+0000 7ff371426640 1 --2- 172.21.15.176:0/954753994 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff36c100ef0 0x7ff36c1012c0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7ff35c004850 tx=0x7ff35c009da0 comp rx=0 tx=0).stop 2024-07-16T09:48:30.455 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:30.453+0000 7ff371426640 1 -- 172.21.15.176:0/954753994 shutdown_connections 2024-07-16T09:48:30.455 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:30.453+0000 7ff371426640 1 --2- 172.21.15.176:0/954753994 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff36c100ef0 0x7ff36c1012c0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:48:30.455 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:30.453+0000 7ff371426640 1 -- 172.21.15.176:0/954753994 >> 172.21.15.176:0/954753994 conn(0x7ff36c070ec0 msgr2=0x7ff36c0712c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:48:30.455 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:30.453+0000 7ff371426640 1 -- 172.21.15.176:0/954753994 shutdown_connections 2024-07-16T09:48:30.455 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:30.453+0000 7ff371426640 1 -- 172.21.15.176:0/954753994 wait complete. 2024-07-16T09:48:30.455 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:30.453+0000 7ff371426640 1 Processor -- start 2024-07-16T09:48:30.455 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:30.453+0000 7ff371426640 1 -- start start 2024-07-16T09:48:30.457 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:30.454+0000 7ff371426640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff36c100ef0 0x7ff36c10b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:48:30.457 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:30.454+0000 7ff371426640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff36c065710 con 0x7ff36c100ef0 2024-07-16T09:48:30.457 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:30.454+0000 7ff36affd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff36c100ef0 0x7ff36c10b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:48:30.457 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:30.454+0000 7ff36affd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff36c100ef0 0x7ff36c10b3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.176:39230/0 (socket says 172.21.15.176:39230) 2024-07-16T09:48:30.457 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:30.454+0000 7ff36affd640 1 -- 172.21.15.176:0/924482336 learned_addr learned my addr 172.21.15.176:0/924482336 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-07-16T09:48:30.457 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:30.454+0000 7ff36affd640 1 -- 172.21.15.176:0/924482336 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff35c005040 con 0x7ff36c100ef0 2024-07-16T09:48:30.457 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:30.454+0000 7ff36affd640 1 --2- 172.21.15.176:0/924482336 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff36c100ef0 0x7ff36c10b3f0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7ff35c004820 tx=0x7ff35c009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:48:30.457 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:30.454+0000 7ff3537fe640 1 -- 172.21.15.176:0/924482336 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff35c02cf10 con 0x7ff36c100ef0 2024-07-16T09:48:30.457 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:30.454+0000 7ff3537fe640 1 -- 172.21.15.176:0/924482336 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff35c033b00 con 0x7ff36c100ef0 2024-07-16T09:48:30.458 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:30.454+0000 7ff371426640 1 -- 172.21.15.176:0/924482336 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff36c10b930 con 0x7ff36c100ef0 2024-07-16T09:48:30.458 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:30.454+0000 7ff3537fe640 1 -- 172.21.15.176:0/924482336 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff35c03db70 con 0x7ff36c100ef0 2024-07-16T09:48:30.458 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:30.455+0000 7ff371426640 1 -- 172.21.15.176:0/924482336 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff36c10be00 con 0x7ff36c100ef0 2024-07-16T09:48:30.458 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:30.455+0000 7ff3537fe640 1 -- 172.21.15.176:0/924482336 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 17) v1 ==== 42751+0+0 (secure 0 0 0) 0x7ff35c030020 con 0x7ff36c100ef0 2024-07-16T09:48:30.458 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:30.456+0000 7ff3537fe640 1 --2- 172.21.15.176:0/924482336 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7ff3540358a0 0x7ff354037d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:48:30.458 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:30.456+0000 7ff3537fe640 1 -- 172.21.15.176:0/924482336 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1366+0+0 (secure 0 0 0) 0x7ff35c068070 con 0x7ff36c100ef0 2024-07-16T09:48:30.459 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:30.456+0000 7ff36a7fc640 1 --2- 172.21.15.176:0/924482336 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7ff3540358a0 0x7ff354037d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:48:30.459 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:30.456+0000 7ff371426640 1 -- 172.21.15.176:0/924482336 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff36c1035a0 con 0x7ff36c100ef0 2024-07-16T09:48:30.459 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:30.456+0000 7ff36a7fc640 1 --2- 172.21.15.176:0/924482336 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7ff3540358a0 0x7ff354037d50 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7ff360006fd0 tx=0x7ff360006db0 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:48:30.462 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:30.461+0000 7ff3537fe640 1 -- 172.21.15.176:0/924482336 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7ff35c03b780 con 0x7ff36c100ef0 2024-07-16T09:48:30.702 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:30.700+0000 7ff371426640 1 -- 172.21.15.176:0/924482336 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7ff36c0fc660 con 0x7ff36c100ef0 2024-07-16T09:48:30.702 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:30.701+0000 7ff3537fe640 1 -- 172.21.15.176:0/924482336 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+743 (secure 0 0 0) 0x7ff35c033c70 con 0x7ff36c100ef0 2024-07-16T09:48:30.703 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-07-16T09:48:30.703 INFO:teuthology.orchestra.run.smithi176.stdout:{"epoch":1,"fsid":"26c91c26-4358-11ef-bcad-c7b262605968","modified":"2024-07-16T09:46:20.852914Z","created":"2024-07-16T09:46:20.852914Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi156","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:3300","nonce":0},{"type":"v1","addr":"172.21.15.156:6789","nonce":0}]},"addr":"172.21.15.156:6789/0","public_addr":"172.21.15.156:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-07-16T09:48:30.703 INFO:teuthology.orchestra.run.smithi176.stderr:dumped monmap epoch 1 2024-07-16T09:48:30.704 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:30.703+0000 7ff371426640 1 -- 172.21.15.176:0/924482336 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7ff3540358a0 msgr2=0x7ff354037d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:48:30.704 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:30.703+0000 7ff371426640 1 --2- 172.21.15.176:0/924482336 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7ff3540358a0 0x7ff354037d50 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7ff360006fd0 tx=0x7ff360006db0 comp rx=0 tx=0).stop 2024-07-16T09:48:30.704 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:30.703+0000 7ff371426640 1 -- 172.21.15.176:0/924482336 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff36c100ef0 msgr2=0x7ff36c10b3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:48:30.704 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:30.703+0000 7ff371426640 1 --2- 172.21.15.176:0/924482336 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff36c100ef0 0x7ff36c10b3f0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7ff35c004820 tx=0x7ff35c009da0 comp rx=0 tx=0).stop 2024-07-16T09:48:30.705 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:30.703+0000 7ff371426640 1 -- 172.21.15.176:0/924482336 shutdown_connections 2024-07-16T09:48:30.705 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:30.703+0000 7ff371426640 1 --2- 172.21.15.176:0/924482336 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7ff3540358a0 0x7ff354037d50 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:48:30.705 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:30.703+0000 7ff371426640 1 --2- 172.21.15.176:0/924482336 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff36c100ef0 0x7ff36c10b3f0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:48:30.705 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:30.703+0000 7ff371426640 1 -- 172.21.15.176:0/924482336 >> 172.21.15.176:0/924482336 conn(0x7ff36c070ec0 msgr2=0x7ff36c0f9ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:48:30.705 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:30.703+0000 7ff371426640 1 -- 172.21.15.176:0/924482336 shutdown_connections 2024-07-16T09:48:30.705 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:30.703+0000 7ff371426640 1 -- 172.21.15.176:0/924482336 wait complete. 2024-07-16T09:48:31.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:48:31 smithi156 ceph-mon[29302]: Updating smithi156:/etc/ceph/ceph.conf 2024-07-16T09:48:31.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:48:31 smithi156 ceph-mon[29302]: Updating smithi176:/etc/ceph/ceph.conf 2024-07-16T09:48:31.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:48:31 smithi156 ceph-mon[29302]: Updating smithi156:/var/lib/ceph/26c91c26-4358-11ef-bcad-c7b262605968/config/ceph.conf 2024-07-16T09:48:31.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:48:31 smithi156 ceph-mon[29302]: Updating smithi176:/var/lib/ceph/26c91c26-4358-11ef-bcad-c7b262605968/config/ceph.conf 2024-07-16T09:48:31.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:48:31 smithi156 ceph-mon[29302]: Updating smithi156:/etc/ceph/ceph.client.admin.keyring 2024-07-16T09:48:31.676 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:48:31 smithi156 ceph-mon[29302]: Updating smithi176:/etc/ceph/ceph.client.admin.keyring 2024-07-16T09:48:31.676 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:48:31 smithi156 ceph-mon[29302]: Updating smithi156:/var/lib/ceph/26c91c26-4358-11ef-bcad-c7b262605968/config/ceph.client.admin.keyring 2024-07-16T09:48:31.676 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:48:31 smithi156 ceph-mon[29302]: Updating smithi176:/var/lib/ceph/26c91c26-4358-11ef-bcad-c7b262605968/config/ceph.client.admin.keyring 2024-07-16T09:48:31.676 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:48:31 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:48:31.676 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:48:31 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:48:31.676 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:48:31 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:48:31.676 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:48:31 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:48:31.676 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:48:31 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:48:31.676 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:48:31 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi176", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-07-16T09:48:31.676 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:48:31 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd='[{"prefix": "auth get-or-create", "entity": "client.crash.smithi176", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]': finished 2024-07-16T09:48:31.676 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:48:31 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-16T09:48:31.676 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:48:31 smithi156 ceph-mon[29302]: Deploying daemon crash.smithi176 on smithi176 2024-07-16T09:48:31.676 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:48:31 smithi156 ceph-mon[29302]: from='client.? 172.21.15.176:0/924482336' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-07-16T09:48:32.734 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-07-16T09:48:32.734 DEBUG:teuthology.orchestra.run.smithi176:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 26c91c26-4358-11ef-bcad-c7b262605968 -- ceph mon dump -f json 2024-07-16T09:48:33.060 INFO:teuthology.orchestra.run.smithi176.stderr:Inferring config /var/lib/ceph/26c91c26-4358-11ef-bcad-c7b262605968/config/ceph.conf 2024-07-16T09:48:34.938 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:34.899+0000 7f2d597df640 1 -- 172.21.15.176:0/2881642895 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2d54102000 msgr2=0x7f2d541023d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:48:34.938 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:34.899+0000 7f2d597df640 1 --2- 172.21.15.176:0/2881642895 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2d54102000 0x7f2d541023d0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f2d4c009bd0 tx=0x7f2d4c004d10 comp rx=0 tx=0).stop 2024-07-16T09:48:34.938 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:34.899+0000 7f2d597df640 1 -- 172.21.15.176:0/2881642895 shutdown_connections 2024-07-16T09:48:34.939 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:34.899+0000 7f2d597df640 1 --2- 172.21.15.176:0/2881642895 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2d54102000 0x7f2d541023d0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:48:34.939 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:34.899+0000 7f2d597df640 1 -- 172.21.15.176:0/2881642895 >> 172.21.15.176:0/2881642895 conn(0x7f2d540715d0 msgr2=0x7f2d540719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:48:34.939 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:34.900+0000 7f2d597df640 1 -- 172.21.15.176:0/2881642895 shutdown_connections 2024-07-16T09:48:34.939 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:34.900+0000 7f2d597df640 1 -- 172.21.15.176:0/2881642895 wait complete. 2024-07-16T09:48:34.939 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:34.900+0000 7f2d597df640 1 Processor -- start 2024-07-16T09:48:34.939 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:34.900+0000 7f2d597df640 1 -- start start 2024-07-16T09:48:34.939 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:34.901+0000 7f2d597df640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2d54102000 0x7f2d54112630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:48:34.939 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:34.901+0000 7f2d597df640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d54112b70 con 0x7f2d54102000 2024-07-16T09:48:34.939 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:34.901+0000 7f2d52ffd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2d54102000 0x7f2d54112630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:48:34.939 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:34.901+0000 7f2d52ffd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2d54102000 0x7f2d54112630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.176:39264/0 (socket says 172.21.15.176:39264) 2024-07-16T09:48:34.939 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:34.901+0000 7f2d52ffd640 1 -- 172.21.15.176:0/1275329057 learned_addr learned my addr 172.21.15.176:0/1275329057 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-07-16T09:48:34.939 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:34.901+0000 7f2d52ffd640 1 -- 172.21.15.176:0/1275329057 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d4c0098b0 con 0x7f2d54102000 2024-07-16T09:48:34.939 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:34.901+0000 7f2d52ffd640 1 --2- 172.21.15.176:0/1275329057 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2d54102000 0x7f2d54112630 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f2d4c009ba0 tx=0x7f2d4c004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:48:34.939 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:34.902+0000 7f2d4b7fe640 1 -- 172.21.15.176:0/1275329057 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2d4c0328b0 con 0x7f2d54102000 2024-07-16T09:48:34.940 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:34.902+0000 7f2d4b7fe640 1 -- 172.21.15.176:0/1275329057 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2d4c03d420 con 0x7f2d54102000 2024-07-16T09:48:34.940 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:34.902+0000 7f2d597df640 1 -- 172.21.15.176:0/1275329057 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d54112d70 con 0x7f2d54102000 2024-07-16T09:48:34.940 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:34.902+0000 7f2d4b7fe640 1 -- 172.21.15.176:0/1275329057 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2d4c031590 con 0x7f2d54102000 2024-07-16T09:48:34.940 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:34.902+0000 7f2d597df640 1 -- 172.21.15.176:0/1275329057 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d541131c0 con 0x7f2d54102000 2024-07-16T09:48:34.940 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:34.902+0000 7f2d4b7fe640 1 -- 172.21.15.176:0/1275329057 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 17) v1 ==== 42751+0+0 (secure 0 0 0) 0x7f2d4c02f030 con 0x7f2d54102000 2024-07-16T09:48:34.940 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:34.903+0000 7f2d597df640 1 -- 172.21.15.176:0/1275329057 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d54065100 con 0x7f2d54102000 2024-07-16T09:48:34.940 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:34.903+0000 7f2d4b7fe640 1 --2- 172.21.15.176:0/1275329057 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f2d400358a0 0x7f2d40037d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:48:34.940 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:34.903+0000 7f2d4b7fe640 1 -- 172.21.15.176:0/1275329057 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1366+0+0 (secure 0 0 0) 0x7f2d4c068e10 con 0x7f2d54102000 2024-07-16T09:48:34.940 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:34.903+0000 7f2d527fc640 1 --2- 172.21.15.176:0/1275329057 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f2d400358a0 0x7f2d40037d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:48:34.940 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:34.903+0000 7f2d527fc640 1 --2- 172.21.15.176:0/1275329057 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f2d400358a0 0x7f2d40037d50 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f2d3c006fd0 tx=0x7f2d3c006db0 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:48:34.940 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:34.908+0000 7f2d4b7fe640 1 -- 172.21.15.176:0/1275329057 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f2d4c032a10 con 0x7f2d54102000 2024-07-16T09:48:35.164 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:35.161+0000 7f2d597df640 1 -- 172.21.15.176:0/1275329057 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f2d5410dca0 con 0x7f2d54102000 2024-07-16T09:48:35.164 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:35.162+0000 7f2d4b7fe640 1 -- 172.21.15.176:0/1275329057 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+743 (secure 0 0 0) 0x7f2d4c04e350 con 0x7f2d54102000 2024-07-16T09:48:35.164 INFO:teuthology.orchestra.run.smithi176.stderr:dumped monmap epoch 1 2024-07-16T09:48:35.164 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-07-16T09:48:35.164 INFO:teuthology.orchestra.run.smithi176.stdout:{"epoch":1,"fsid":"26c91c26-4358-11ef-bcad-c7b262605968","modified":"2024-07-16T09:46:20.852914Z","created":"2024-07-16T09:46:20.852914Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi156","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:3300","nonce":0},{"type":"v1","addr":"172.21.15.156:6789","nonce":0}]},"addr":"172.21.15.156:6789/0","public_addr":"172.21.15.156:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-07-16T09:48:35.166 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:35.164+0000 7f2d597df640 1 -- 172.21.15.176:0/1275329057 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f2d400358a0 msgr2=0x7f2d40037d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:48:35.166 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:35.164+0000 7f2d597df640 1 --2- 172.21.15.176:0/1275329057 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f2d400358a0 0x7f2d40037d50 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f2d3c006fd0 tx=0x7f2d3c006db0 comp rx=0 tx=0).stop 2024-07-16T09:48:35.166 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:35.164+0000 7f2d597df640 1 -- 172.21.15.176:0/1275329057 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2d54102000 msgr2=0x7f2d54112630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:48:35.166 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:35.164+0000 7f2d597df640 1 --2- 172.21.15.176:0/1275329057 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2d54102000 0x7f2d54112630 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f2d4c009ba0 tx=0x7f2d4c004d10 comp rx=0 tx=0).stop 2024-07-16T09:48:35.166 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:35.164+0000 7f2d597df640 1 -- 172.21.15.176:0/1275329057 shutdown_connections 2024-07-16T09:48:35.166 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:35.164+0000 7f2d597df640 1 --2- 172.21.15.176:0/1275329057 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f2d400358a0 0x7f2d40037d50 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:48:35.166 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:35.164+0000 7f2d597df640 1 --2- 172.21.15.176:0/1275329057 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2d54102000 0x7f2d54112630 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:48:35.166 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:35.164+0000 7f2d597df640 1 -- 172.21.15.176:0/1275329057 >> 172.21.15.176:0/1275329057 conn(0x7f2d540715d0 msgr2=0x7f2d540f93f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:48:35.166 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:35.164+0000 7f2d597df640 1 -- 172.21.15.176:0/1275329057 shutdown_connections 2024-07-16T09:48:35.166 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:35.164+0000 7f2d597df640 1 -- 172.21.15.176:0/1275329057 wait complete. 2024-07-16T09:48:35.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:48:35 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:48:35.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:48:35 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:48:35.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:48:35 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:48:35.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:48:35 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:48:35.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:48:35 smithi156 ceph-mon[29302]: Deploying daemon node-exporter.smithi176 on smithi176 2024-07-16T09:48:35.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:48:35 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:48:35.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:48:35 smithi156 ceph-mon[29302]: from='client.? 172.21.15.176:0/1275329057' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-07-16T09:48:36.792 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-07-16T09:48:36.792 DEBUG:teuthology.orchestra.run.smithi176:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 26c91c26-4358-11ef-bcad-c7b262605968 -- ceph mon dump -f json 2024-07-16T09:48:37.228 INFO:teuthology.orchestra.run.smithi176.stderr:Inferring config /var/lib/ceph/26c91c26-4358-11ef-bcad-c7b262605968/config/ceph.conf 2024-07-16T09:48:39.605 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:39.594+0000 7eff46430640 1 -- 172.21.15.176:0/465411026 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7eff40100ef0 msgr2=0x7eff401012c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:48:39.605 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:39.594+0000 7eff46430640 1 --2- 172.21.15.176:0/465411026 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7eff40100ef0 0x7eff401012c0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7eff30004850 tx=0x7eff30009da0 comp rx=0 tx=0).stop 2024-07-16T09:48:39.605 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:39.594+0000 7eff46430640 1 -- 172.21.15.176:0/465411026 shutdown_connections 2024-07-16T09:48:39.605 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:39.594+0000 7eff46430640 1 --2- 172.21.15.176:0/465411026 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7eff40100ef0 0x7eff401012c0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:48:39.605 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:39.594+0000 7eff46430640 1 -- 172.21.15.176:0/465411026 >> 172.21.15.176:0/465411026 conn(0x7eff40070ec0 msgr2=0x7eff400712c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:48:39.606 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:39.594+0000 7eff46430640 1 -- 172.21.15.176:0/465411026 shutdown_connections 2024-07-16T09:48:39.606 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:39.594+0000 7eff46430640 1 -- 172.21.15.176:0/465411026 wait complete. 2024-07-16T09:48:39.606 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:39.595+0000 7eff46430640 1 Processor -- start 2024-07-16T09:48:39.606 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:39.595+0000 7eff46430640 1 -- start start 2024-07-16T09:48:39.606 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:39.595+0000 7eff46430640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7eff40100ef0 0x7eff4010d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:48:39.606 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:39.595+0000 7eff46430640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff4010db40 con 0x7eff40100ef0 2024-07-16T09:48:39.606 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:39.596+0000 7eff3ffff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7eff40100ef0 0x7eff4010d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:48:39.606 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:39.596+0000 7eff3ffff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7eff40100ef0 0x7eff4010d600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.176:34896/0 (socket says 172.21.15.176:34896) 2024-07-16T09:48:39.606 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:39.596+0000 7eff3ffff640 1 -- 172.21.15.176:0/3121159375 learned_addr learned my addr 172.21.15.176:0/3121159375 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-07-16T09:48:39.606 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:39.596+0000 7eff3ffff640 1 -- 172.21.15.176:0/3121159375 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff30005040 con 0x7eff40100ef0 2024-07-16T09:48:39.606 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:39.596+0000 7eff3ffff640 1 --2- 172.21.15.176:0/3121159375 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7eff40100ef0 0x7eff4010d600 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7eff30004820 tx=0x7eff30009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:48:39.606 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:39.596+0000 7eff3cff9640 1 -- 172.21.15.176:0/3121159375 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7eff3002cf10 con 0x7eff40100ef0 2024-07-16T09:48:39.606 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:39.596+0000 7eff46430640 1 -- 172.21.15.176:0/3121159375 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff4010dd40 con 0x7eff40100ef0 2024-07-16T09:48:39.606 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:39.596+0000 7eff3cff9640 1 -- 172.21.15.176:0/3121159375 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7eff30033b00 con 0x7eff40100ef0 2024-07-16T09:48:39.607 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:39.596+0000 7eff3cff9640 1 -- 172.21.15.176:0/3121159375 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7eff3003db70 con 0x7eff40100ef0 2024-07-16T09:48:39.607 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:39.596+0000 7eff46430640 1 -- 172.21.15.176:0/3121159375 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff4010e130 con 0x7eff40100ef0 2024-07-16T09:48:39.607 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:39.597+0000 7eff3cff9640 1 -- 172.21.15.176:0/3121159375 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 17) v1 ==== 42751+0+0 (secure 0 0 0) 0x7eff30030020 con 0x7eff40100ef0 2024-07-16T09:48:39.607 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:39.597+0000 7eff46430640 1 -- 172.21.15.176:0/3121159375 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff400fc660 con 0x7eff40100ef0 2024-07-16T09:48:39.607 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:39.598+0000 7eff3cff9640 1 --2- 172.21.15.176:0/3121159375 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7eff280358a0 0x7eff28037d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:48:39.607 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:39.598+0000 7eff3cff9640 1 -- 172.21.15.176:0/3121159375 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1366+0+0 (secure 0 0 0) 0x7eff30067fb0 con 0x7eff40100ef0 2024-07-16T09:48:39.607 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:39.598+0000 7eff3f7fe640 1 --2- 172.21.15.176:0/3121159375 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7eff280358a0 0x7eff28037d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:48:39.607 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:39.598+0000 7eff3f7fe640 1 --2- 172.21.15.176:0/3121159375 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7eff280358a0 0x7eff28037d50 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7eff34006fd0 tx=0x7eff34006db0 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:48:39.607 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:39.603+0000 7eff3cff9640 1 -- 172.21.15.176:0/3121159375 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7eff3009a250 con 0x7eff40100ef0 2024-07-16T09:48:39.855 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:39.853+0000 7eff46430640 1 -- 172.21.15.176:0/3121159375 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7eff40105300 con 0x7eff40100ef0 2024-07-16T09:48:39.855 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:39.854+0000 7eff3cff9640 1 -- 172.21.15.176:0/3121159375 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+743 (secure 0 0 0) 0x7eff3003b780 con 0x7eff40100ef0 2024-07-16T09:48:39.856 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-07-16T09:48:39.856 INFO:teuthology.orchestra.run.smithi176.stdout:{"epoch":1,"fsid":"26c91c26-4358-11ef-bcad-c7b262605968","modified":"2024-07-16T09:46:20.852914Z","created":"2024-07-16T09:46:20.852914Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi156","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:3300","nonce":0},{"type":"v1","addr":"172.21.15.156:6789","nonce":0}]},"addr":"172.21.15.156:6789/0","public_addr":"172.21.15.156:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-07-16T09:48:39.856 INFO:teuthology.orchestra.run.smithi176.stderr:dumped monmap epoch 1 2024-07-16T09:48:39.857 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:39.856+0000 7eff46430640 1 -- 172.21.15.176:0/3121159375 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7eff280358a0 msgr2=0x7eff28037d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:48:39.858 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:39.856+0000 7eff46430640 1 --2- 172.21.15.176:0/3121159375 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7eff280358a0 0x7eff28037d50 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7eff34006fd0 tx=0x7eff34006db0 comp rx=0 tx=0).stop 2024-07-16T09:48:39.858 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:39.856+0000 7eff46430640 1 -- 172.21.15.176:0/3121159375 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7eff40100ef0 msgr2=0x7eff4010d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:48:39.858 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:39.856+0000 7eff46430640 1 --2- 172.21.15.176:0/3121159375 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7eff40100ef0 0x7eff4010d600 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7eff30004820 tx=0x7eff30009da0 comp rx=0 tx=0).stop 2024-07-16T09:48:39.858 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:39.856+0000 7eff46430640 1 -- 172.21.15.176:0/3121159375 shutdown_connections 2024-07-16T09:48:39.858 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:39.856+0000 7eff46430640 1 --2- 172.21.15.176:0/3121159375 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7eff280358a0 0x7eff28037d50 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:48:39.858 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:39.856+0000 7eff46430640 1 --2- 172.21.15.176:0/3121159375 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7eff40100ef0 0x7eff4010d600 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:48:39.858 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:39.856+0000 7eff46430640 1 -- 172.21.15.176:0/3121159375 >> 172.21.15.176:0/3121159375 conn(0x7eff40070ec0 msgr2=0x7eff400f9ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:48:39.858 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:39.856+0000 7eff46430640 1 -- 172.21.15.176:0/3121159375 shutdown_connections 2024-07-16T09:48:39.858 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:39.856+0000 7eff46430640 1 -- 172.21.15.176:0/3121159375 wait complete. 2024-07-16T09:48:40.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:48:40 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:48:40.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:48:40 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:48:40.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:48:40 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:48:40.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:48:40 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:48:40.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:48:40 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi176.dqbrvx", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-07-16T09:48:40.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:48:40 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.smithi176.dqbrvx", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-07-16T09:48:40.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:48:40 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "mgr services"}]: dispatch 2024-07-16T09:48:40.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:48:40 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-16T09:48:40.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:48:40 smithi156 ceph-mon[29302]: Deploying daemon mgr.smithi176.dqbrvx on smithi176 2024-07-16T09:48:40.676 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:48:40 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:48:40.676 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:48:40 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-16T09:48:40.676 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:48:40 smithi156 ceph-mon[29302]: from='client.? 172.21.15.176:0/3121159375' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-07-16T09:48:41.854 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-07-16T09:48:41.855 DEBUG:teuthology.orchestra.run.smithi176:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 26c91c26-4358-11ef-bcad-c7b262605968 -- ceph mon dump -f json 2024-07-16T09:48:42.218 INFO:teuthology.orchestra.run.smithi176.stderr:Inferring config /var/lib/ceph/26c91c26-4358-11ef-bcad-c7b262605968/config/ceph.conf 2024-07-16T09:48:43.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:48:43 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:48:43.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:48:43 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:48:43.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:48:43 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:48:43.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:48:43 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:48:43.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:48:43 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-07-16T09:48:43.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:48:43 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-16T09:48:43.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:48:43 smithi156 ceph-mon[29302]: Deploying daemon mon.smithi176 on smithi176 2024-07-16T09:48:43.776 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:43.774+0000 7f522f259640 1 -- 172.21.15.176:0/1896960457 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5228101ff0 msgr2=0x7f52281023c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:48:43.776 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:43.774+0000 7f522f259640 1 --2- 172.21.15.176:0/1896960457 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5228101ff0 0x7f52281023c0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f521c004850 tx=0x7f521c009da0 comp rx=0 tx=0).stop 2024-07-16T09:48:43.777 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:43.774+0000 7f522f259640 1 -- 172.21.15.176:0/1896960457 shutdown_connections 2024-07-16T09:48:43.777 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:43.774+0000 7f522f259640 1 --2- 172.21.15.176:0/1896960457 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5228101ff0 0x7f52281023c0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:48:43.777 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:43.774+0000 7f522f259640 1 -- 172.21.15.176:0/1896960457 >> 172.21.15.176:0/1896960457 conn(0x7f52280f7bc0 msgr2=0x7f52280f9fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:48:43.777 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:43.774+0000 7f522f259640 1 -- 172.21.15.176:0/1896960457 shutdown_connections 2024-07-16T09:48:43.777 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:43.774+0000 7f522f259640 1 -- 172.21.15.176:0/1896960457 wait complete. 2024-07-16T09:48:43.777 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:43.774+0000 7f522f259640 1 Processor -- start 2024-07-16T09:48:43.777 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:43.774+0000 7f522f259640 1 -- start start 2024-07-16T09:48:43.777 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:43.775+0000 7f522f259640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5228101ff0 0x7f5228112690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:48:43.777 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:43.775+0000 7f522f259640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5228112bd0 con 0x7f5228101ff0 2024-07-16T09:48:43.777 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:43.775+0000 7f522cff5640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5228101ff0 0x7f5228112690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:48:43.777 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:43.775+0000 7f522cff5640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5228101ff0 0x7f5228112690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.176:34934/0 (socket says 172.21.15.176:34934) 2024-07-16T09:48:43.777 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:43.775+0000 7f522cff5640 1 -- 172.21.15.176:0/77108646 learned_addr learned my addr 172.21.15.176:0/77108646 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-07-16T09:48:43.777 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:43.775+0000 7f522cff5640 1 -- 172.21.15.176:0/77108646 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f521c005040 con 0x7f5228101ff0 2024-07-16T09:48:43.779 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:43.775+0000 7f522cff5640 1 --2- 172.21.15.176:0/77108646 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5228101ff0 0x7f5228112690 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f521c004820 tx=0x7f521c009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:48:43.779 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:43.776+0000 7f52257fa640 1 -- 172.21.15.176:0/77108646 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f521c02cf10 con 0x7f5228101ff0 2024-07-16T09:48:43.779 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:43.776+0000 7f522f259640 1 -- 172.21.15.176:0/77108646 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5228112dd0 con 0x7f5228101ff0 2024-07-16T09:48:43.779 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:43.776+0000 7f52257fa640 1 -- 172.21.15.176:0/77108646 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f521c033b00 con 0x7f5228101ff0 2024-07-16T09:48:43.779 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:43.776+0000 7f52257fa640 1 -- 172.21.15.176:0/77108646 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f521c03db70 con 0x7f5228101ff0 2024-07-16T09:48:43.779 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:43.776+0000 7f522f259640 1 -- 172.21.15.176:0/77108646 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52281131c0 con 0x7f5228101ff0 2024-07-16T09:48:43.779 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:43.777+0000 7f522f259640 1 -- 172.21.15.176:0/77108646 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52280fd760 con 0x7f5228101ff0 2024-07-16T09:48:43.779 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:43.777+0000 7f52257fa640 1 -- 172.21.15.176:0/77108646 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 17) v1 ==== 42751+0+0 (secure 0 0 0) 0x7f521c030020 con 0x7f5228101ff0 2024-07-16T09:48:43.779 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:43.777+0000 7f52257fa640 1 --2- 172.21.15.176:0/77108646 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f52180358a0 0x7f5218037d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:48:43.780 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:43.777+0000 7f52257fa640 1 -- 172.21.15.176:0/77108646 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1366+0+0 (secure 0 0 0) 0x7f521c068160 con 0x7f5228101ff0 2024-07-16T09:48:43.780 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:43.777+0000 7f5227fff640 1 --2- 172.21.15.176:0/77108646 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f52180358a0 0x7f5218037d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:48:43.780 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:43.778+0000 7f5227fff640 1 --2- 172.21.15.176:0/77108646 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f52180358a0 0x7f5218037d50 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f5214006fd0 tx=0x7f5214006db0 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:48:43.783 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:43.782+0000 7f52257fa640 1 -- 172.21.15.176:0/77108646 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f521c045570 con 0x7f5228101ff0 2024-07-16T09:48:44.028 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:44.027+0000 7f522f259640 1 -- 172.21.15.176:0/77108646 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f522804b530 con 0x7f5228101ff0 2024-07-16T09:48:44.029 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:44.027+0000 7f52257fa640 1 -- 172.21.15.176:0/77108646 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+743 (secure 0 0 0) 0x7f521c045390 con 0x7f5228101ff0 2024-07-16T09:48:44.029 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-07-16T09:48:44.029 INFO:teuthology.orchestra.run.smithi176.stdout:{"epoch":1,"fsid":"26c91c26-4358-11ef-bcad-c7b262605968","modified":"2024-07-16T09:46:20.852914Z","created":"2024-07-16T09:46:20.852914Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi156","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:3300","nonce":0},{"type":"v1","addr":"172.21.15.156:6789","nonce":0}]},"addr":"172.21.15.156:6789/0","public_addr":"172.21.15.156:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-07-16T09:48:44.029 INFO:teuthology.orchestra.run.smithi176.stderr:dumped monmap epoch 1 2024-07-16T09:48:44.031 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:44.029+0000 7f522f259640 1 -- 172.21.15.176:0/77108646 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f52180358a0 msgr2=0x7f5218037d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:48:44.031 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:44.029+0000 7f522f259640 1 --2- 172.21.15.176:0/77108646 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f52180358a0 0x7f5218037d50 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f5214006fd0 tx=0x7f5214006db0 comp rx=0 tx=0).stop 2024-07-16T09:48:44.031 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:44.029+0000 7f522f259640 1 -- 172.21.15.176:0/77108646 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5228101ff0 msgr2=0x7f5228112690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:48:44.031 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:44.029+0000 7f522f259640 1 --2- 172.21.15.176:0/77108646 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5228101ff0 0x7f5228112690 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f521c004820 tx=0x7f521c009da0 comp rx=0 tx=0).stop 2024-07-16T09:48:44.031 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:44.030+0000 7f522f259640 1 -- 172.21.15.176:0/77108646 shutdown_connections 2024-07-16T09:48:44.031 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:44.030+0000 7f522f259640 1 --2- 172.21.15.176:0/77108646 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f52180358a0 0x7f5218037d50 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:48:44.031 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:44.030+0000 7f522f259640 1 --2- 172.21.15.176:0/77108646 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5228101ff0 0x7f5228112690 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:48:44.031 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:44.030+0000 7f522f259640 1 -- 172.21.15.176:0/77108646 >> 172.21.15.176:0/77108646 conn(0x7f52280f7bc0 msgr2=0x7f52280f91f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:48:44.032 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:44.030+0000 7f522f259640 1 -- 172.21.15.176:0/77108646 shutdown_connections 2024-07-16T09:48:44.032 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:44.030+0000 7f522f259640 1 -- 172.21.15.176:0/77108646 wait complete. 2024-07-16T09:48:44.651 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:48:44 smithi156 ceph-mon[29302]: from='client.? 172.21.15.176:0/77108646' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-07-16T09:48:45.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:48:45 smithi156 ceph-mon[29302]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-16T09:48:45.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:48:45 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:48:47.425 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-07-16T09:48:47.426 DEBUG:teuthology.orchestra.run.smithi176:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 26c91c26-4358-11ef-bcad-c7b262605968 -- ceph mon dump -f json 2024-07-16T09:48:47.677 INFO:teuthology.orchestra.run.smithi176.stderr:Inferring config /var/lib/ceph/26c91c26-4358-11ef-bcad-c7b262605968/mon.smithi176/config 2024-07-16T09:48:52.070 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:52.066+0000 7f9e77f1c640 1 -- 172.21.15.176:0/1858800921 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9e5c0034f0 msgr2=0x7f9e5c005970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:48:52.070 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:52.066+0000 7f9e77f1c640 1 --2- 172.21.15.176:0/1858800921 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9e5c0034f0 0x7f9e5c005970 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f9e700655f0 tx=0x7f9e6c00b040 comp rx=0 tx=0).stop 2024-07-16T09:48:52.070 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:52.066+0000 7f9e77f1c640 1 -- 172.21.15.176:0/1858800921 shutdown_connections 2024-07-16T09:48:52.070 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:52.066+0000 7f9e77f1c640 1 --2- 172.21.15.176:0/1858800921 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9e5c0034f0 0x7f9e5c005970 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:48:52.070 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:52.066+0000 7f9e77f1c640 1 -- 172.21.15.176:0/1858800921 >> 172.21.15.176:0/1858800921 conn(0x7f9e700f6c20 msgr2=0x7f9e700f9030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:48:52.070 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:52.066+0000 7f9e77f1c640 1 -- 172.21.15.176:0/1858800921 shutdown_connections 2024-07-16T09:48:52.070 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:52.066+0000 7f9e77f1c640 1 -- 172.21.15.176:0/1858800921 wait complete. 2024-07-16T09:48:52.071 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:52.067+0000 7f9e77f1c640 1 Processor -- start 2024-07-16T09:48:52.071 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:52.067+0000 7f9e77f1c640 1 -- start start 2024-07-16T09:48:52.071 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:52.067+0000 7f9e77f1c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9e5c0034f0 0x7f9e70112860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:48:52.071 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:52.067+0000 7f9e77f1c640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9e70112da0 0x7f9e7010c800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:48:52.071 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:52.067+0000 7f9e77f1c640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e70113360 con 0x7f9e5c0034f0 2024-07-16T09:48:52.071 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:52.067+0000 7f9e77f1c640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e701134a0 con 0x7f9e70112da0 2024-07-16T09:48:52.071 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:52.067+0000 7f9e754b7640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9e70112da0 0x7f9e7010c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:48:52.071 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:52.067+0000 7f9e754b7640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9e70112da0 0x7f9e7010c800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:50126/0 (socket says 172.21.15.176:50126) 2024-07-16T09:48:52.071 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:52.067+0000 7f9e754b7640 1 -- 172.21.15.176:0/3262870015 learned_addr learned my addr 172.21.15.176:0/3262870015 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-07-16T09:48:52.071 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:52.067+0000 7f9e75cb8640 1 --2- 172.21.15.176:0/3262870015 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9e5c0034f0 0x7f9e70112860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:48:52.071 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:52.067+0000 7f9e754b7640 1 -- 172.21.15.176:0/3262870015 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9e70112da0 msgr2=0x7f9e7010c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-07-16T09:48:52.071 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:52.067+0000 7f9e754b7640 1 -- 172.21.15.176:0/3262870015 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9e70112da0 msgr2=0x7f9e7010c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-07-16T09:48:52.072 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:52.067+0000 7f9e754b7640 1 --2- 172.21.15.176:0/3262870015 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9e70112da0 0x7f9e7010c800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-07-16T09:48:52.072 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:52.068+0000 7f9e754b7640 1 --2- 172.21.15.176:0/3262870015 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9e70112da0 0x7f9e7010c800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-16T09:48:52.072 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:52.068+0000 7f9e75cb8640 1 -- 172.21.15.176:0/3262870015 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9e70112da0 msgr2=0x7f9e7010c800 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-16T09:48:52.072 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:52.068+0000 7f9e75cb8640 1 --2- 172.21.15.176:0/3262870015 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9e70112da0 0x7f9e7010c800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:48:52.072 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:52.068+0000 7f9e75cb8640 1 -- 172.21.15.176:0/3262870015 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e6c005040 con 0x7f9e5c0034f0 2024-07-16T09:48:52.072 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:52.068+0000 7f9e75cb8640 1 --2- 172.21.15.176:0/3262870015 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9e5c0034f0 0x7f9e70112860 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f9e6c0043d0 tx=0x7f9e6c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:48:52.072 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:52.068+0000 7f9e667fc640 1 -- 172.21.15.176:0/3262870015 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9e6c030b60 con 0x7f9e5c0034f0 2024-07-16T09:48:52.072 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:52.068+0000 7f9e77f1c640 1 -- 172.21.15.176:0/3262870015 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e7010ce00 con 0x7f9e5c0034f0 2024-07-16T09:48:52.072 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:52.068+0000 7f9e667fc640 1 -- 172.21.15.176:0/3262870015 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9e6c00f040 con 0x7f9e5c0034f0 2024-07-16T09:48:52.072 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:52.068+0000 7f9e77f1c640 1 -- 172.21.15.176:0/3262870015 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e7010d320 con 0x7f9e5c0034f0 2024-07-16T09:48:52.072 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:52.068+0000 7f9e667fc640 1 -- 172.21.15.176:0/3262870015 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9e6c03c5f0 con 0x7f9e5c0034f0 2024-07-16T09:48:52.073 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:52.069+0000 7f9e77f1c640 1 -- 172.21.15.176:0/3262870015 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e70065df0 con 0x7f9e5c0034f0 2024-07-16T09:48:52.073 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:52.070+0000 7f9e667fc640 1 -- 172.21.15.176:0/3262870015 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 18) v1 ==== 85071+0+0 (secure 0 0 0) 0x7f9e6c03e440 con 0x7f9e5c0034f0 2024-07-16T09:48:52.073 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:52.070+0000 7f9e667fc640 1 --2- 172.21.15.176:0/3262870015 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f9e54067250 0x7f9e54069700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:48:52.074 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:52.070+0000 7f9e667fc640 1 -- 172.21.15.176:0/3262870015 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1366+0+0 (secure 0 0 0) 0x7f9e6c0a4fc0 con 0x7f9e5c0034f0 2024-07-16T09:48:52.074 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:52.070+0000 7f9e754b7640 1 --2- 172.21.15.176:0/3262870015 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f9e54067250 0x7f9e54069700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:48:52.074 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:52.071+0000 7f9e754b7640 1 --2- 172.21.15.176:0/3262870015 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f9e54067250 0x7f9e54069700 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f9e60006fd0 tx=0x7f9e60008040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:48:52.076 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:52.074+0000 7f9e667fc640 1 -- 172.21.15.176:0/3262870015 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f9e6c0732a0 con 0x7f9e5c0034f0 2024-07-16T09:48:52.175 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:48:51 smithi156 ceph-mon[29302]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-16T09:48:52.175 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:48:51 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "mon metadata", "id": "smithi156"}]: dispatch 2024-07-16T09:48:52.175 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:48:51 smithi156 ceph-mon[29302]: mon.smithi156 calling monitor election 2024-07-16T09:48:52.175 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:48:51 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "mon metadata", "id": "smithi176"}]: dispatch 2024-07-16T09:48:52.175 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:48:51 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "mon metadata", "id": "smithi176"}]: dispatch 2024-07-16T09:48:52.175 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:48:51 smithi156 ceph-mon[29302]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-16T09:48:52.175 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:48:51 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "mon metadata", "id": "smithi176"}]: dispatch 2024-07-16T09:48:52.175 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:48:51 smithi156 ceph-mon[29302]: mon.smithi176 calling monitor election 2024-07-16T09:48:52.176 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:48:51 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "mon metadata", "id": "smithi176"}]: dispatch 2024-07-16T09:48:52.176 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:48:51 smithi156 ceph-mon[29302]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-16T09:48:52.176 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:48:51 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "mon metadata", "id": "smithi176"}]: dispatch 2024-07-16T09:48:52.176 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:48:51 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "mon metadata", "id": "smithi176"}]: dispatch 2024-07-16T09:48:52.176 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:48:51 smithi156 ceph-mon[29302]: mon.smithi156 is new leader, mons smithi156,smithi176 in quorum (ranks 0,1) 2024-07-16T09:48:52.176 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:48:51 smithi156 ceph-mon[29302]: monmap e2: 2 mons at {smithi156=[v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0],smithi176=[v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-07-16T09:48:52.176 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:48:51 smithi156 ceph-mon[29302]: fsmap 2024-07-16T09:48:52.176 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:48:51 smithi156 ceph-mon[29302]: osdmap e5: 0 total, 0 up, 0 in 2024-07-16T09:48:52.176 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:48:51 smithi156 ceph-mon[29302]: mgrmap e17: smithi156.zkvylv(active, since 27s) 2024-07-16T09:48:52.176 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:48:51 smithi156 ceph-mon[29302]: overall HEALTH_OK 2024-07-16T09:48:52.176 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:48:51 smithi156 ceph-mon[29302]: Standby manager daemon smithi176.dqbrvx started 2024-07-16T09:48:52.176 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:48:51 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:48:52.177 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:48:51 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:48:52.177 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:48:51 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:48:52.177 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:48:51 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:48:52.177 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:48:51 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:48:52.357 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:52.355+0000 7f9e77f1c640 1 -- 172.21.15.176:0/3262870015 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f9e7004b530 con 0x7f9e5c0034f0 2024-07-16T09:48:52.358 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:52.356+0000 7f9e667fc640 1 -- 172.21.15.176:0/3262870015 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) v1 ==== 95+0+1021 (secure 0 0 0) 0x7f9e6c0768c0 con 0x7f9e5c0034f0 2024-07-16T09:48:52.358 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-07-16T09:48:52.358 INFO:teuthology.orchestra.run.smithi176.stdout:{"epoch":2,"fsid":"26c91c26-4358-11ef-bcad-c7b262605968","modified":"2024-07-16T09:48:46.842263Z","created":"2024-07-16T09:46:20.852914Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi156","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:3300","nonce":0},{"type":"v1","addr":"172.21.15.156:6789","nonce":0}]},"addr":"172.21.15.156:6789/0","public_addr":"172.21.15.156:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"smithi176","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:3300","nonce":0},{"type":"v1","addr":"172.21.15.176:6789","nonce":0}]},"addr":"172.21.15.176:6789/0","public_addr":"172.21.15.176:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-07-16T09:48:52.358 INFO:teuthology.orchestra.run.smithi176.stderr:dumped monmap epoch 2 2024-07-16T09:48:52.360 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:52.359+0000 7f9e77f1c640 1 -- 172.21.15.176:0/3262870015 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f9e54067250 msgr2=0x7f9e54069700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:48:52.360 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:52.359+0000 7f9e77f1c640 1 --2- 172.21.15.176:0/3262870015 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f9e54067250 0x7f9e54069700 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f9e60006fd0 tx=0x7f9e60008040 comp rx=0 tx=0).stop 2024-07-16T09:48:52.360 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:52.359+0000 7f9e77f1c640 1 -- 172.21.15.176:0/3262870015 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9e5c0034f0 msgr2=0x7f9e70112860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:48:52.360 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:52.359+0000 7f9e77f1c640 1 --2- 172.21.15.176:0/3262870015 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9e5c0034f0 0x7f9e70112860 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f9e6c0043d0 tx=0x7f9e6c00b040 comp rx=0 tx=0).stop 2024-07-16T09:48:52.361 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:52.359+0000 7f9e77f1c640 1 -- 172.21.15.176:0/3262870015 shutdown_connections 2024-07-16T09:48:52.361 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:52.359+0000 7f9e77f1c640 1 --2- 172.21.15.176:0/3262870015 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f9e54067250 0x7f9e54069700 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:48:52.361 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:52.359+0000 7f9e77f1c640 1 --2- 172.21.15.176:0/3262870015 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9e70112da0 0x7f9e7010c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:48:52.361 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:52.359+0000 7f9e77f1c640 1 --2- 172.21.15.176:0/3262870015 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9e5c0034f0 0x7f9e70112860 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:48:52.361 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:52.359+0000 7f9e77f1c640 1 -- 172.21.15.176:0/3262870015 >> 172.21.15.176:0/3262870015 conn(0x7f9e700f6c20 msgr2=0x7f9e70101a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:48:52.361 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:52.359+0000 7f9e77f1c640 1 -- 172.21.15.176:0/3262870015 shutdown_connections 2024-07-16T09:48:52.361 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:48:52.359+0000 7f9e77f1c640 1 -- 172.21.15.176:0/3262870015 wait complete. 2024-07-16T09:48:53.174 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:48:52 smithi156 ceph-mon[29302]: mgrmap e18: smithi156.zkvylv(active, since 27s), standbys: smithi176.dqbrvx 2024-07-16T09:48:53.175 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:48:52 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "mgr metadata", "who": "smithi176.dqbrvx", "id": "smithi176.dqbrvx"}]: dispatch 2024-07-16T09:48:53.175 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:48:52 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-16T09:48:53.175 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:48:52 smithi156 ceph-mon[29302]: from='client.? 172.21.15.176:0/3262870015' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-07-16T09:48:53.175 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:48:52 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "mon metadata", "id": "smithi176"}]: dispatch 2024-07-16T09:48:53.906 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-07-16T09:48:53.906 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 26c91c26-4358-11ef-bcad-c7b262605968 -- ceph config generate-minimal-conf 2024-07-16T09:48:54.169 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/26c91c26-4358-11ef-bcad-c7b262605968/mon.smithi156/config 2024-07-16T09:48:54.174 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:48:53 smithi156 ceph-mon[29302]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-16T09:48:55.415 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:48:55 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:48:55.415 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:48:55 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:48:55.415 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:48:55 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-16T09:48:55.415 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:48:55 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-16T09:48:55.415 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:48:55 smithi156 ceph-mon[29302]: Updating smithi156:/etc/ceph/ceph.conf 2024-07-16T09:48:55.416 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:48:55 smithi156 ceph-mon[29302]: Updating smithi176:/etc/ceph/ceph.conf 2024-07-16T09:48:55.416 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:48:55 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:48:55.416 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:48:55 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:48:55.416 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:48:55 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:48:55.416 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:48:55 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:48:55.416 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:48:55 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:48:55.416 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:48:55 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-16T09:48:55.416 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:48:55 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:48:55.416 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:48:55 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-07-16T09:48:55.416 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:48:55 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-07-16T09:48:55.416 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:48:55 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-16T09:48:55.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:48:55 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-16T09:48:55.779 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:48:55.775+0000 7f86c8cbf640 1 -- 172.21.15.156:0/178429825 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f86c4100ef0 msgr2=0x7f86c41012c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:48:55.779 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:48:55.775+0000 7f86c8cbf640 1 --2- 172.21.15.156:0/178429825 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f86c4100ef0 0x7f86c41012c0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f86b4004850 tx=0x7f86b4009da0 comp rx=0 tx=0).stop 2024-07-16T09:48:55.779 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:48:55.775+0000 7f86c8cbf640 1 -- 172.21.15.156:0/178429825 shutdown_connections 2024-07-16T09:48:55.779 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:48:55.775+0000 7f86c8cbf640 1 --2- 172.21.15.156:0/178429825 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f86c4100ef0 0x7f86c41012c0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:48:55.779 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:48:55.775+0000 7f86c8cbf640 1 -- 172.21.15.156:0/178429825 >> 172.21.15.156:0/178429825 conn(0x7f86c4070ec0 msgr2=0x7f86c40712c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:48:55.779 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:48:55.775+0000 7f86c8cbf640 1 -- 172.21.15.156:0/178429825 shutdown_connections 2024-07-16T09:48:55.779 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:48:55.775+0000 7f86c8cbf640 1 -- 172.21.15.156:0/178429825 wait complete. 2024-07-16T09:48:55.779 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:48:55.776+0000 7f86c8cbf640 1 Processor -- start 2024-07-16T09:48:55.780 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:48:55.776+0000 7f86c8cbf640 1 -- start start 2024-07-16T09:48:55.780 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:48:55.776+0000 7f86c8cbf640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f86c4100ef0 0x7f86c410d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:48:55.780 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:48:55.776+0000 7f86c8cbf640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f86c410dc20 0x7f86c410e070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:48:55.780 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:48:55.776+0000 7f86c8cbf640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86c41167e0 con 0x7f86c4100ef0 2024-07-16T09:48:55.780 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:48:55.776+0000 7f86c8cbf640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86c4116920 con 0x7f86c410dc20 2024-07-16T09:48:55.780 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:48:55.776+0000 7f86c259c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f86c4100ef0 0x7f86c410d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:48:55.780 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:48:55.776+0000 7f86c259c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f86c4100ef0 0x7f86c410d6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:60206/0 (socket says 172.21.15.156:60206) 2024-07-16T09:48:55.780 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:48:55.776+0000 7f86c259c640 1 -- 172.21.15.156:0/2355080580 learned_addr learned my addr 172.21.15.156:0/2355080580 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T09:48:55.780 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:48:55.776+0000 7f86c259c640 1 -- 172.21.15.156:0/2355080580 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f86c410dc20 msgr2=0x7f86c410e070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:48:55.780 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:48:55.776+0000 7f86c259c640 1 --2- 172.21.15.156:0/2355080580 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f86c410dc20 0x7f86c410e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:48:55.780 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:48:55.776+0000 7f86c259c640 1 -- 172.21.15.156:0/2355080580 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86b4005040 con 0x7f86c4100ef0 2024-07-16T09:48:55.780 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:48:55.777+0000 7f86c259c640 1 --2- 172.21.15.156:0/2355080580 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f86c4100ef0 0x7f86c410d6e0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f86b402d040 tx=0x7f86b4009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:48:55.780 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:48:55.777+0000 7f86baffd640 1 -- 172.21.15.156:0/2355080580 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f86b4038070 con 0x7f86c4100ef0 2024-07-16T09:48:55.781 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:48:55.777+0000 7f86baffd640 1 -- 172.21.15.156:0/2355080580 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f86b4032bf0 con 0x7f86c4100ef0 2024-07-16T09:48:55.781 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:48:55.777+0000 7f86c8cbf640 1 -- 172.21.15.156:0/2355080580 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86c4116b20 con 0x7f86c4100ef0 2024-07-16T09:48:55.781 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:48:55.777+0000 7f86baffd640 1 -- 172.21.15.156:0/2355080580 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f86b403dc00 con 0x7f86c4100ef0 2024-07-16T09:48:55.781 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:48:55.777+0000 7f86c8cbf640 1 -- 172.21.15.156:0/2355080580 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86c4116fb0 con 0x7f86c4100ef0 2024-07-16T09:48:55.782 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:48:55.778+0000 7f86c8cbf640 1 -- 172.21.15.156:0/2355080580 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86c40fc660 con 0x7f86c4100ef0 2024-07-16T09:48:55.782 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:48:55.778+0000 7f86baffd640 1 -- 172.21.15.156:0/2355080580 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 18) v1 ==== 85071+0+0 (secure 0 0 0) 0x7f86b4030020 con 0x7f86c4100ef0 2024-07-16T09:48:55.782 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:48:55.779+0000 7f86baffd640 1 --2- 172.21.15.156:0/2355080580 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f86b0067260 0x7f86b0069710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:48:55.782 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:48:55.779+0000 7f86baffd640 1 -- 172.21.15.156:0/2355080580 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1366+0+0 (secure 0 0 0) 0x7f86b40a3ee0 con 0x7f86c4100ef0 2024-07-16T09:48:55.782 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:48:55.779+0000 7f86bbfff640 1 --2- 172.21.15.156:0/2355080580 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f86b0067260 0x7f86b0069710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:48:55.782 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:48:55.779+0000 7f86bbfff640 1 --2- 172.21.15.156:0/2355080580 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f86b0067260 0x7f86b0069710 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f86ac006fd0 tx=0x7f86ac006d50 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:48:55.784 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:48:55.783+0000 7f86baffd640 1 -- 172.21.15.156:0/2355080580 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f86b40722e0 con 0x7f86c4100ef0 2024-07-16T09:48:55.951 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:48:55.950+0000 7f86c8cbf640 1 -- 172.21.15.156:0/2355080580 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7f86c405eb80 con 0x7f86c4100ef0 2024-07-16T09:48:55.951 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:48:55.950+0000 7f86baffd640 1 -- 172.21.15.156:0/2355080580 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v10) v1 ==== 76+0+227 (secure 0 0 0) 0x7f86b40a3690 con 0x7f86c4100ef0 2024-07-16T09:48:55.952 INFO:teuthology.orchestra.run.smithi156.stdout:# minimal ceph.conf for 26c91c26-4358-11ef-bcad-c7b262605968 2024-07-16T09:48:55.952 INFO:teuthology.orchestra.run.smithi156.stdout:[global] 2024-07-16T09:48:55.952 INFO:teuthology.orchestra.run.smithi156.stdout: fsid = 26c91c26-4358-11ef-bcad-c7b262605968 2024-07-16T09:48:55.952 INFO:teuthology.orchestra.run.smithi156.stdout: mon_host = [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] 2024-07-16T09:48:55.953 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:48:55.952+0000 7f86c8cbf640 1 -- 172.21.15.156:0/2355080580 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f86b0067260 msgr2=0x7f86b0069710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:48:55.954 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:48:55.952+0000 7f86c8cbf640 1 --2- 172.21.15.156:0/2355080580 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f86b0067260 0x7f86b0069710 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f86ac006fd0 tx=0x7f86ac006d50 comp rx=0 tx=0).stop 2024-07-16T09:48:55.954 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:48:55.952+0000 7f86c8cbf640 1 -- 172.21.15.156:0/2355080580 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f86c4100ef0 msgr2=0x7f86c410d6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:48:55.954 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:48:55.952+0000 7f86c8cbf640 1 --2- 172.21.15.156:0/2355080580 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f86c4100ef0 0x7f86c410d6e0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f86b402d040 tx=0x7f86b4009da0 comp rx=0 tx=0).stop 2024-07-16T09:48:55.954 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:48:55.952+0000 7f86c8cbf640 1 -- 172.21.15.156:0/2355080580 shutdown_connections 2024-07-16T09:48:55.954 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:48:55.952+0000 7f86c8cbf640 1 --2- 172.21.15.156:0/2355080580 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f86b0067260 0x7f86b0069710 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:48:55.954 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:48:55.952+0000 7f86c8cbf640 1 --2- 172.21.15.156:0/2355080580 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f86c410dc20 0x7f86c410e070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:48:55.954 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:48:55.952+0000 7f86c8cbf640 1 --2- 172.21.15.156:0/2355080580 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f86c4100ef0 0x7f86c410d6e0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:48:55.954 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:48:55.952+0000 7f86c8cbf640 1 -- 172.21.15.156:0/2355080580 >> 172.21.15.156:0/2355080580 conn(0x7f86c4070ec0 msgr2=0x7f86c40f9cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:48:55.954 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:48:55.952+0000 7f86c8cbf640 1 -- 172.21.15.156:0/2355080580 shutdown_connections 2024-07-16T09:48:55.954 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:48:55.952+0000 7f86c8cbf640 1 -- 172.21.15.156:0/2355080580 wait complete. 2024-07-16T09:48:56.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:48:56 smithi176 ceph-mon[36626]: Updating smithi176:/var/lib/ceph/26c91c26-4358-11ef-bcad-c7b262605968/config/ceph.conf 2024-07-16T09:48:56.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:48:56 smithi176 ceph-mon[36626]: Updating smithi156:/var/lib/ceph/26c91c26-4358-11ef-bcad-c7b262605968/config/ceph.conf 2024-07-16T09:48:56.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:48:56 smithi176 ceph-mon[36626]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-16T09:48:56.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:48:56 smithi176 ceph-mon[36626]: Reconfiguring mon.smithi156 (unknown last config time)... 2024-07-16T09:48:56.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:48:56 smithi176 ceph-mon[36626]: Reconfiguring daemon mon.smithi156 on smithi156 2024-07-16T09:48:56.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:48:56 smithi176 ceph-mon[36626]: from='client.? 172.21.15.156:0/2355080580' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-16T09:48:56.619 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:48:56 smithi156 ceph-mon[29302]: Updating smithi176:/var/lib/ceph/26c91c26-4358-11ef-bcad-c7b262605968/config/ceph.conf 2024-07-16T09:48:56.619 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:48:56 smithi156 ceph-mon[29302]: Updating smithi156:/var/lib/ceph/26c91c26-4358-11ef-bcad-c7b262605968/config/ceph.conf 2024-07-16T09:48:56.619 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:48:56 smithi156 ceph-mon[29302]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-16T09:48:56.619 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:48:56 smithi156 ceph-mon[29302]: Reconfiguring mon.smithi156 (unknown last config time)... 2024-07-16T09:48:56.619 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:48:56 smithi156 ceph-mon[29302]: Reconfiguring daemon mon.smithi156 on smithi156 2024-07-16T09:48:56.619 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:48:56 smithi156 ceph-mon[29302]: from='client.? 172.21.15.156:0/2355080580' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-16T09:48:56.644 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-07-16T09:48:56.644 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2024-07-16T09:48:56.644 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd of=/etc/ceph/ceph.conf 2024-07-16T09:48:56.672 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2024-07-16T09:48:56.672 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-07-16T09:48:56.737 DEBUG:teuthology.orchestra.run.smithi176:> set -ex 2024-07-16T09:48:56.737 DEBUG:teuthology.orchestra.run.smithi176:> sudo dd of=/etc/ceph/ceph.conf 2024-07-16T09:48:56.769 DEBUG:teuthology.orchestra.run.smithi176:> set -ex 2024-07-16T09:48:56.769 DEBUG:teuthology.orchestra.run.smithi176:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-07-16T09:48:56.838 INFO:tasks.cephadm:Deploying OSDs... 2024-07-16T09:48:56.838 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2024-07-16T09:48:56.838 DEBUG:teuthology.orchestra.run.smithi156:> dd if=/scratch_devs of=/dev/stdout 2024-07-16T09:48:56.854 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-07-16T09:48:56.854 DEBUG:teuthology.orchestra.run.smithi156:> stat /dev/vg_nvme/lv_1 2024-07-16T09:48:56.909 INFO:teuthology.orchestra.run.smithi156.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-07-16T09:48:56.909 INFO:teuthology.orchestra.run.smithi156.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-07-16T09:48:56.909 INFO:teuthology.orchestra.run.smithi156.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-07-16T09:48:56.909 INFO:teuthology.orchestra.run.smithi156.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-07-16T09:48:56.909 INFO:teuthology.orchestra.run.smithi156.stdout:Context: system_u:object_r:device_t:s0 2024-07-16T09:48:56.909 INFO:teuthology.orchestra.run.smithi156.stdout:Access: 2024-07-16 09:48:18.666391912 +0000 2024-07-16T09:48:56.910 INFO:teuthology.orchestra.run.smithi156.stdout:Modify: 2024-07-16 09:42:38.748443152 +0000 2024-07-16T09:48:56.910 INFO:teuthology.orchestra.run.smithi156.stdout:Change: 2024-07-16 09:42:38.748443152 +0000 2024-07-16T09:48:56.910 INFO:teuthology.orchestra.run.smithi156.stdout: Birth: 2024-07-16 09:42:38.748443152 +0000 2024-07-16T09:48:56.910 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-07-16T09:48:56.973 INFO:teuthology.orchestra.run.smithi156.stderr:1+0 records in 2024-07-16T09:48:56.973 INFO:teuthology.orchestra.run.smithi156.stderr:1+0 records out 2024-07-16T09:48:56.973 INFO:teuthology.orchestra.run.smithi156.stderr:512 bytes copied, 0.000110088 s, 4.7 MB/s 2024-07-16T09:48:56.974 DEBUG:teuthology.orchestra.run.smithi156:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-07-16T09:48:57.030 DEBUG:teuthology.orchestra.run.smithi156:> stat /dev/vg_nvme/lv_2 2024-07-16T09:48:57.085 INFO:teuthology.orchestra.run.smithi156.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-07-16T09:48:57.085 INFO:teuthology.orchestra.run.smithi156.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-07-16T09:48:57.085 INFO:teuthology.orchestra.run.smithi156.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-07-16T09:48:57.086 INFO:teuthology.orchestra.run.smithi156.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-07-16T09:48:57.086 INFO:teuthology.orchestra.run.smithi156.stdout:Context: system_u:object_r:device_t:s0 2024-07-16T09:48:57.086 INFO:teuthology.orchestra.run.smithi156.stdout:Access: 2024-07-16 09:48:18.666391912 +0000 2024-07-16T09:48:57.086 INFO:teuthology.orchestra.run.smithi156.stdout:Modify: 2024-07-16 09:42:39.075438512 +0000 2024-07-16T09:48:57.086 INFO:teuthology.orchestra.run.smithi156.stdout:Change: 2024-07-16 09:42:39.075438512 +0000 2024-07-16T09:48:57.086 INFO:teuthology.orchestra.run.smithi156.stdout: Birth: 2024-07-16 09:42:39.075438512 +0000 2024-07-16T09:48:57.086 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-07-16T09:48:57.148 INFO:teuthology.orchestra.run.smithi156.stderr:1+0 records in 2024-07-16T09:48:57.148 INFO:teuthology.orchestra.run.smithi156.stderr:1+0 records out 2024-07-16T09:48:57.149 INFO:teuthology.orchestra.run.smithi156.stderr:512 bytes copied, 8.7967e-05 s, 5.8 MB/s 2024-07-16T09:48:57.150 DEBUG:teuthology.orchestra.run.smithi156:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-07-16T09:48:57.205 DEBUG:teuthology.orchestra.run.smithi156:> stat /dev/vg_nvme/lv_3 2024-07-16T09:48:57.262 INFO:teuthology.orchestra.run.smithi156.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-07-16T09:48:57.262 INFO:teuthology.orchestra.run.smithi156.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-07-16T09:48:57.262 INFO:teuthology.orchestra.run.smithi156.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-07-16T09:48:57.262 INFO:teuthology.orchestra.run.smithi156.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-07-16T09:48:57.262 INFO:teuthology.orchestra.run.smithi156.stdout:Context: system_u:object_r:device_t:s0 2024-07-16T09:48:57.262 INFO:teuthology.orchestra.run.smithi156.stdout:Access: 2024-07-16 09:48:18.666391912 +0000 2024-07-16T09:48:57.262 INFO:teuthology.orchestra.run.smithi156.stdout:Modify: 2024-07-16 09:42:39.385434113 +0000 2024-07-16T09:48:57.263 INFO:teuthology.orchestra.run.smithi156.stdout:Change: 2024-07-16 09:42:39.385434113 +0000 2024-07-16T09:48:57.263 INFO:teuthology.orchestra.run.smithi156.stdout: Birth: 2024-07-16 09:42:39.385434113 +0000 2024-07-16T09:48:57.263 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-07-16T09:48:57.326 INFO:teuthology.orchestra.run.smithi156.stderr:1+0 records in 2024-07-16T09:48:57.326 INFO:teuthology.orchestra.run.smithi156.stderr:1+0 records out 2024-07-16T09:48:57.326 INFO:teuthology.orchestra.run.smithi156.stderr:512 bytes copied, 8.9946e-05 s, 5.7 MB/s 2024-07-16T09:48:57.327 DEBUG:teuthology.orchestra.run.smithi156:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-07-16T09:48:57.383 DEBUG:teuthology.orchestra.run.smithi156:> stat /dev/vg_nvme/lv_4 2024-07-16T09:48:57.438 INFO:teuthology.orchestra.run.smithi156.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-07-16T09:48:57.438 INFO:teuthology.orchestra.run.smithi156.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-07-16T09:48:57.438 INFO:teuthology.orchestra.run.smithi156.stdout:Device: 5h/5d Inode: 725 Links: 1 2024-07-16T09:48:57.438 INFO:teuthology.orchestra.run.smithi156.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-07-16T09:48:57.438 INFO:teuthology.orchestra.run.smithi156.stdout:Context: system_u:object_r:device_t:s0 2024-07-16T09:48:57.438 INFO:teuthology.orchestra.run.smithi156.stdout:Access: 2024-07-16 09:48:18.667391898 +0000 2024-07-16T09:48:57.439 INFO:teuthology.orchestra.run.smithi156.stdout:Modify: 2024-07-16 09:42:39.713429459 +0000 2024-07-16T09:48:57.439 INFO:teuthology.orchestra.run.smithi156.stdout:Change: 2024-07-16 09:42:39.713429459 +0000 2024-07-16T09:48:57.439 INFO:teuthology.orchestra.run.smithi156.stdout: Birth: 2024-07-16 09:42:39.713429459 +0000 2024-07-16T09:48:57.439 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-07-16T09:48:57.503 INFO:teuthology.orchestra.run.smithi156.stderr:1+0 records in 2024-07-16T09:48:57.503 INFO:teuthology.orchestra.run.smithi156.stderr:1+0 records out 2024-07-16T09:48:57.503 INFO:teuthology.orchestra.run.smithi156.stderr:512 bytes copied, 8.7477e-05 s, 5.9 MB/s 2024-07-16T09:48:57.505 DEBUG:teuthology.orchestra.run.smithi156:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-07-16T09:48:57.562 DEBUG:teuthology.orchestra.run.smithi176:> set -ex 2024-07-16T09:48:57.563 DEBUG:teuthology.orchestra.run.smithi176:> dd if=/scratch_devs of=/dev/stdout 2024-07-16T09:48:57.582 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-07-16T09:48:57.582 DEBUG:teuthology.orchestra.run.smithi176:> stat /dev/vg_nvme/lv_1 2024-07-16T09:48:57.640 INFO:teuthology.orchestra.run.smithi176.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-07-16T09:48:57.640 INFO:teuthology.orchestra.run.smithi176.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-07-16T09:48:57.640 INFO:teuthology.orchestra.run.smithi176.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-07-16T09:48:57.640 INFO:teuthology.orchestra.run.smithi176.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-07-16T09:48:57.640 INFO:teuthology.orchestra.run.smithi176.stdout:Context: system_u:object_r:device_t:s0 2024-07-16T09:48:57.640 INFO:teuthology.orchestra.run.smithi176.stdout:Access: 2024-07-16 09:48:46.275488454 +0000 2024-07-16T09:48:57.640 INFO:teuthology.orchestra.run.smithi176.stdout:Modify: 2024-07-16 09:42:38.756719429 +0000 2024-07-16T09:48:57.640 INFO:teuthology.orchestra.run.smithi176.stdout:Change: 2024-07-16 09:42:38.756719429 +0000 2024-07-16T09:48:57.640 INFO:teuthology.orchestra.run.smithi176.stdout: Birth: 2024-07-16 09:42:38.756719429 +0000 2024-07-16T09:48:57.641 DEBUG:teuthology.orchestra.run.smithi176:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-07-16T09:48:57.707 INFO:teuthology.orchestra.run.smithi176.stderr:1+0 records in 2024-07-16T09:48:57.707 INFO:teuthology.orchestra.run.smithi176.stderr:1+0 records out 2024-07-16T09:48:57.707 INFO:teuthology.orchestra.run.smithi176.stderr:512 bytes copied, 0.000300799 s, 1.7 MB/s 2024-07-16T09:48:57.708 DEBUG:teuthology.orchestra.run.smithi176:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-07-16T09:48:57.766 DEBUG:teuthology.orchestra.run.smithi176:> stat /dev/vg_nvme/lv_2 2024-07-16T09:48:57.823 INFO:teuthology.orchestra.run.smithi176.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-07-16T09:48:57.823 INFO:teuthology.orchestra.run.smithi176.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-07-16T09:48:57.823 INFO:teuthology.orchestra.run.smithi176.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-07-16T09:48:57.823 INFO:teuthology.orchestra.run.smithi176.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-07-16T09:48:57.823 INFO:teuthology.orchestra.run.smithi176.stdout:Context: system_u:object_r:device_t:s0 2024-07-16T09:48:57.823 INFO:teuthology.orchestra.run.smithi176.stdout:Access: 2024-07-16 09:48:46.275488454 +0000 2024-07-16T09:48:57.823 INFO:teuthology.orchestra.run.smithi176.stdout:Modify: 2024-07-16 09:42:39.092716574 +0000 2024-07-16T09:48:57.823 INFO:teuthology.orchestra.run.smithi176.stdout:Change: 2024-07-16 09:42:39.092716574 +0000 2024-07-16T09:48:57.823 INFO:teuthology.orchestra.run.smithi176.stdout: Birth: 2024-07-16 09:42:39.092716574 +0000 2024-07-16T09:48:57.824 DEBUG:teuthology.orchestra.run.smithi176:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-07-16T09:48:57.888 INFO:teuthology.orchestra.run.smithi176.stderr:1+0 records in 2024-07-16T09:48:57.888 INFO:teuthology.orchestra.run.smithi176.stderr:1+0 records out 2024-07-16T09:48:57.888 INFO:teuthology.orchestra.run.smithi176.stderr:512 bytes copied, 0.000224806 s, 2.3 MB/s 2024-07-16T09:48:57.890 DEBUG:teuthology.orchestra.run.smithi176:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-07-16T09:48:57.947 DEBUG:teuthology.orchestra.run.smithi176:> stat /dev/vg_nvme/lv_3 2024-07-16T09:48:58.003 INFO:teuthology.orchestra.run.smithi176.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-07-16T09:48:58.003 INFO:teuthology.orchestra.run.smithi176.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-07-16T09:48:58.003 INFO:teuthology.orchestra.run.smithi176.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-07-16T09:48:58.003 INFO:teuthology.orchestra.run.smithi176.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-07-16T09:48:58.003 INFO:teuthology.orchestra.run.smithi176.stdout:Context: system_u:object_r:device_t:s0 2024-07-16T09:48:58.003 INFO:teuthology.orchestra.run.smithi176.stdout:Access: 2024-07-16 09:48:46.275488454 +0000 2024-07-16T09:48:58.003 INFO:teuthology.orchestra.run.smithi176.stdout:Modify: 2024-07-16 09:42:39.430713702 +0000 2024-07-16T09:48:58.003 INFO:teuthology.orchestra.run.smithi176.stdout:Change: 2024-07-16 09:42:39.430713702 +0000 2024-07-16T09:48:58.004 INFO:teuthology.orchestra.run.smithi176.stdout: Birth: 2024-07-16 09:42:39.430713702 +0000 2024-07-16T09:48:58.004 DEBUG:teuthology.orchestra.run.smithi176:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-07-16T09:48:58.067 INFO:teuthology.orchestra.run.smithi176.stderr:1+0 records in 2024-07-16T09:48:58.067 INFO:teuthology.orchestra.run.smithi176.stderr:1+0 records out 2024-07-16T09:48:58.068 INFO:teuthology.orchestra.run.smithi176.stderr:512 bytes copied, 0.000227757 s, 2.2 MB/s 2024-07-16T09:48:58.069 DEBUG:teuthology.orchestra.run.smithi176:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-07-16T09:48:58.126 DEBUG:teuthology.orchestra.run.smithi176:> stat /dev/vg_nvme/lv_4 2024-07-16T09:48:58.181 INFO:teuthology.orchestra.run.smithi176.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-07-16T09:48:58.182 INFO:teuthology.orchestra.run.smithi176.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-07-16T09:48:58.182 INFO:teuthology.orchestra.run.smithi176.stdout:Device: 5h/5d Inode: 724 Links: 1 2024-07-16T09:48:58.182 INFO:teuthology.orchestra.run.smithi176.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-07-16T09:48:58.182 INFO:teuthology.orchestra.run.smithi176.stdout:Context: system_u:object_r:device_t:s0 2024-07-16T09:48:58.182 INFO:teuthology.orchestra.run.smithi176.stdout:Access: 2024-07-16 09:48:46.276488445 +0000 2024-07-16T09:48:58.182 INFO:teuthology.orchestra.run.smithi176.stdout:Modify: 2024-07-16 09:42:39.759710907 +0000 2024-07-16T09:48:58.182 INFO:teuthology.orchestra.run.smithi176.stdout:Change: 2024-07-16 09:42:39.759710907 +0000 2024-07-16T09:48:58.182 INFO:teuthology.orchestra.run.smithi176.stdout: Birth: 2024-07-16 09:42:39.759710907 +0000 2024-07-16T09:48:58.182 DEBUG:teuthology.orchestra.run.smithi176:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-07-16T09:48:58.245 INFO:teuthology.orchestra.run.smithi176.stderr:1+0 records in 2024-07-16T09:48:58.245 INFO:teuthology.orchestra.run.smithi176.stderr:1+0 records out 2024-07-16T09:48:58.246 INFO:teuthology.orchestra.run.smithi176.stderr:512 bytes copied, 0.000212216 s, 2.4 MB/s 2024-07-16T09:48:58.247 DEBUG:teuthology.orchestra.run.smithi176:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-07-16T09:48:58.303 INFO:tasks.cephadm:Deploying osd.0 on smithi156 with /dev/vg_nvme/lv_4... 2024-07-16T09:48:58.303 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 26c91c26-4358-11ef-bcad-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-07-16T09:48:58.343 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:48:58 smithi176 ceph-mon[36626]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-16T09:48:58.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:48:58 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:48:58.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:48:58 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:48:58.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:48:58 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi156.zkvylv", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-07-16T09:48:58.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:48:58 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "mgr services"}]: dispatch 2024-07-16T09:48:58.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:48:58 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-16T09:48:58.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:48:58 smithi156 ceph-mon[29302]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-16T09:48:58.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:48:58 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:48:58.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:48:58 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:48:58.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:48:58 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi156.zkvylv", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-07-16T09:48:58.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:48:58 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "mgr services"}]: dispatch 2024-07-16T09:48:58.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:48:58 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-16T09:48:58.891 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/26c91c26-4358-11ef-bcad-c7b262605968/mon.smithi156/config 2024-07-16T09:48:59.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:48:59 smithi176 ceph-mon[36626]: Reconfiguring mgr.smithi156.zkvylv (unknown last config time)... 2024-07-16T09:48:59.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:48:59 smithi176 ceph-mon[36626]: Reconfiguring daemon mgr.smithi156.zkvylv on smithi156 2024-07-16T09:48:59.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:48:59 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:48:59.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:48:59 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:48:59.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:48:59 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi156", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-07-16T09:48:59.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:48:59 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-16T09:48:59.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:48:59 smithi156 ceph-mon[29302]: Reconfiguring mgr.smithi156.zkvylv (unknown last config time)... 2024-07-16T09:48:59.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:48:59 smithi156 ceph-mon[29302]: Reconfiguring daemon mgr.smithi156.zkvylv on smithi156 2024-07-16T09:48:59.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:48:59 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:48:59.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:48:59 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:48:59.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:48:59 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi156", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-07-16T09:48:59.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:48:59 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-16T09:49:00.424 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:49:00 smithi156 ceph-mon[29302]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-16T09:49:00.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:49:00 smithi156 ceph-mon[29302]: Reconfiguring crash.smithi156 (monmap changed)... 2024-07-16T09:49:00.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:49:00 smithi156 ceph-mon[29302]: Reconfiguring daemon crash.smithi156 on smithi156 2024-07-16T09:49:00.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:49:00 smithi176 ceph-mon[36626]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-16T09:49:00.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:49:00 smithi176 ceph-mon[36626]: Reconfiguring crash.smithi156 (monmap changed)... 2024-07-16T09:49:00.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:49:00 smithi176 ceph-mon[36626]: Reconfiguring daemon crash.smithi156 on smithi156 2024-07-16T09:49:00.980 INFO:teuthology.orchestra.run.smithi156.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-07-16T09:49:01.073 INFO:teuthology.orchestra.run.smithi156.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-07-16T09:49:01.083 INFO:teuthology.orchestra.run.smithi156.stderr: stderr: 10+0 records in 2024-07-16T09:49:01.083 INFO:teuthology.orchestra.run.smithi156.stderr:10+0 records out 2024-07-16T09:49:01.084 INFO:teuthology.orchestra.run.smithi156.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00852415 s, 1.2 GB/s 2024-07-16T09:49:01.084 INFO:teuthology.orchestra.run.smithi156.stderr:--> Zapping successful for: 2024-07-16T09:49:01.906 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 26c91c26-4358-11ef-bcad-c7b262605968 -- ceph orch daemon add osd smithi156:vg_nvme/lv_4 2024-07-16T09:49:02.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:49:02 smithi176 ceph-mon[36626]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-16T09:49:02.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:49:02 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:49:02.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:49:02 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:49:02.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:49:02 smithi176 ceph-mon[36626]: Reconfiguring alertmanager.smithi156 (dependencies changed)... 2024-07-16T09:49:02.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:49:02 smithi176 ceph-mon[36626]: Reconfiguring daemon alertmanager.smithi156 on smithi156 2024-07-16T09:49:02.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:49:02 smithi156 ceph-mon[29302]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-16T09:49:02.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:49:02 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:49:02.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:49:02 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:49:02.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:49:02 smithi156 ceph-mon[29302]: Reconfiguring alertmanager.smithi156 (dependencies changed)... 2024-07-16T09:49:02.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:49:02 smithi156 ceph-mon[29302]: Reconfiguring daemon alertmanager.smithi156 on smithi156 2024-07-16T09:49:02.808 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/26c91c26-4358-11ef-bcad-c7b262605968/mon.smithi156/config 2024-07-16T09:49:04.459 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:49:04 smithi156 ceph-mon[29302]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-16T09:49:04.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:49:04 smithi176 ceph-mon[36626]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-16T09:49:04.652 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:49:04.649+0000 7fd40853b640 1 -- 172.21.15.156:0/4058003941 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd4000fd650 msgr2=0x7fd4000fdac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:49:04.652 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:49:04.649+0000 7fd40853b640 1 --2- 172.21.15.156:0/4058003941 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd4000fd650 0x7fd4000fdac0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fd3f00048d0 tx=0x7fd3f0009da0 comp rx=0 tx=0).stop 2024-07-16T09:49:04.653 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:49:04.649+0000 7fd40853b640 1 -- 172.21.15.156:0/4058003941 shutdown_connections 2024-07-16T09:49:04.653 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:49:04.649+0000 7fd40853b640 1 --2- 172.21.15.156:0/4058003941 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd4000fd650 0x7fd4000fdac0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:49:04.653 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:49:04.649+0000 7fd40853b640 1 --2- 172.21.15.156:0/4058003941 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd4000fc380 0x7fd4000fc790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:49:04.653 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:49:04.649+0000 7fd40853b640 1 -- 172.21.15.156:0/4058003941 >> 172.21.15.156:0/4058003941 conn(0x7fd4000f7b40 msgr2=0x7fd4000f9f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:49:04.653 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:49:04.649+0000 7fd40853b640 1 -- 172.21.15.156:0/4058003941 shutdown_connections 2024-07-16T09:49:04.653 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:49:04.649+0000 7fd40853b640 1 -- 172.21.15.156:0/4058003941 wait complete. 2024-07-16T09:49:04.653 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:49:04.650+0000 7fd40853b640 1 Processor -- start 2024-07-16T09:49:04.653 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:49:04.650+0000 7fd40853b640 1 -- start start 2024-07-16T09:49:04.653 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:49:04.650+0000 7fd40853b640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd4000fc380 0x7fd40010cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:49:04.653 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:49:04.650+0000 7fd40853b640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd4000fd650 0x7fd40010d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:49:04.653 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:49:04.650+0000 7fd40853b640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd40010e380 con 0x7fd4000fc380 2024-07-16T09:49:04.654 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:49:04.650+0000 7fd40853b640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd40010e4f0 con 0x7fd4000fd650 2024-07-16T09:49:04.654 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:49:04.650+0000 7fd4062d7640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd4000fc380 0x7fd40010cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:49:04.654 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:49:04.650+0000 7fd4062d7640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd4000fc380 0x7fd40010cfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:38034/0 (socket says 172.21.15.156:38034) 2024-07-16T09:49:04.654 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:49:04.650+0000 7fd4062d7640 1 -- 172.21.15.156:0/2616353314 learned_addr learned my addr 172.21.15.156:0/2616353314 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T09:49:04.654 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:49:04.650+0000 7fd405ad6640 1 --2- 172.21.15.156:0/2616353314 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd4000fd650 0x7fd40010d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:49:04.654 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:49:04.650+0000 7fd4062d7640 1 -- 172.21.15.156:0/2616353314 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd4000fd650 msgr2=0x7fd40010d510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:49:04.654 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:49:04.650+0000 7fd4062d7640 1 --2- 172.21.15.156:0/2616353314 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd4000fd650 0x7fd40010d510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:49:04.654 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:49:04.650+0000 7fd4062d7640 1 -- 172.21.15.156:0/2616353314 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3f0005040 con 0x7fd4000fc380 2024-07-16T09:49:04.654 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:49:04.650+0000 7fd405ad6640 1 --2- 172.21.15.156:0/2616353314 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd4000fd650 0x7fd40010d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-16T09:49:04.654 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:49:04.651+0000 7fd4062d7640 1 --2- 172.21.15.156:0/2616353314 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd4000fc380 0x7fd40010cfd0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fd3fc00b790 tx=0x7fd3fc0091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:49:04.655 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:49:04.651+0000 7fd3f6ffd640 1 -- 172.21.15.156:0/2616353314 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd3fc00c700 con 0x7fd4000fc380 2024-07-16T09:49:04.655 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:49:04.651+0000 7fd3f6ffd640 1 -- 172.21.15.156:0/2616353314 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd3fc00cd30 con 0x7fd4000fc380 2024-07-16T09:49:04.655 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:49:04.651+0000 7fd40853b640 1 -- 172.21.15.156:0/2616353314 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd40010e7d0 con 0x7fd4000fc380 2024-07-16T09:49:04.655 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:49:04.651+0000 7fd3f6ffd640 1 -- 172.21.15.156:0/2616353314 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd3fc013370 con 0x7fd4000fc380 2024-07-16T09:49:04.655 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:49:04.651+0000 7fd40853b640 1 -- 172.21.15.156:0/2616353314 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd40010ecf0 con 0x7fd4000fc380 2024-07-16T09:49:04.656 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:49:04.652+0000 7fd40853b640 1 -- 172.21.15.156:0/2616353314 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd400105760 con 0x7fd4000fc380 2024-07-16T09:49:04.656 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:49:04.652+0000 7fd3f6ffd640 1 -- 172.21.15.156:0/2616353314 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 18) v1 ==== 85071+0+0 (secure 0 0 0) 0x7fd3fc012070 con 0x7fd4000fc380 2024-07-16T09:49:04.656 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:49:04.653+0000 7fd3f6ffd640 1 --2- 172.21.15.156:0/2616353314 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7fd3ec067100 0x7fd3ec0695b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:49:04.656 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:49:04.653+0000 7fd405ad6640 1 --2- 172.21.15.156:0/2616353314 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7fd3ec067100 0x7fd3ec0695b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:49:04.656 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:49:04.653+0000 7fd3f6ffd640 1 -- 172.21.15.156:0/2616353314 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1366+0+0 (secure 0 0 0) 0x7fd3fc083420 con 0x7fd4000fc380 2024-07-16T09:49:04.656 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:49:04.653+0000 7fd405ad6640 1 --2- 172.21.15.156:0/2616353314 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7fd3ec067100 0x7fd3ec0695b0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fd3f0000c00 tx=0x7fd3f0035040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:49:04.658 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:49:04.657+0000 7fd3f6ffd640 1 -- 172.21.15.156:0/2616353314 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fd3fc0517a0 con 0x7fd4000fc380 2024-07-16T09:49:04.837 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:49:04.835+0000 7fd40853b640 1 -- 172.21.15.156:0/2616353314 --> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi156:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7fd40010efd0 con 0x7fd3ec067100 2024-07-16T09:49:05.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:49:05 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-16T09:49:05.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:49:05 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-16T09:49:05.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:49:05 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-16T09:49:05.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:49:05 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-16T09:49:05.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:49:05 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-16T09:49:05.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:49:05 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-16T09:49:06.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:49:06 smithi176 ceph-mon[36626]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-16T09:49:06.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:49:06 smithi176 ceph-mon[36626]: from='client.14240 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi156:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-07-16T09:49:06.594 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:49:06 smithi156 ceph-mon[29302]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-16T09:49:06.594 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:49:06 smithi156 ceph-mon[29302]: from='client.14240 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi156:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-07-16T09:49:08.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:49:08 smithi176 ceph-mon[36626]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-16T09:49:08.603 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:49:08 smithi156 ceph-mon[29302]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-16T09:49:09.721 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:49:09 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:49:09.721 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:49:09 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:49:09.721 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:49:09 smithi156 ceph-mon[29302]: Reconfiguring grafana.smithi156 (dependencies changed)... 2024-07-16T09:49:09.721 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:49:09 smithi156 ceph-mon[29302]: Reconfiguring daemon grafana.smithi156 on smithi156 2024-07-16T09:49:09.721 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:49:09 smithi156 ceph-mon[29302]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-16T09:49:09.721 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:49:09 smithi156 ceph-mon[29302]: from='client.? 172.21.15.156:0/3951179020' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "38938d0f-f41d-4d70-9f59-f977181e2943"}]: dispatch 2024-07-16T09:49:09.722 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:49:09 smithi156 ceph-mon[29302]: from='client.? 172.21.15.156:0/3951179020' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "38938d0f-f41d-4d70-9f59-f977181e2943"}]': finished 2024-07-16T09:49:09.722 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:49:09 smithi156 ceph-mon[29302]: osdmap e6: 1 total, 0 up, 1 in 2024-07-16T09:49:09.722 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:49:09 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-07-16T09:49:09.846 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:49:09 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:49:09.846 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:49:09 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:49:09.846 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:49:09 smithi176 ceph-mon[36626]: Reconfiguring grafana.smithi156 (dependencies changed)... 2024-07-16T09:49:09.846 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:49:09 smithi176 ceph-mon[36626]: Reconfiguring daemon grafana.smithi156 on smithi156 2024-07-16T09:49:09.846 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:49:09 smithi176 ceph-mon[36626]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-16T09:49:09.846 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:49:09 smithi176 ceph-mon[36626]: from='client.? 172.21.15.156:0/3951179020' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "38938d0f-f41d-4d70-9f59-f977181e2943"}]: dispatch 2024-07-16T09:49:09.847 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:49:09 smithi176 ceph-mon[36626]: from='client.? 172.21.15.156:0/3951179020' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "38938d0f-f41d-4d70-9f59-f977181e2943"}]': finished 2024-07-16T09:49:09.847 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:49:09 smithi176 ceph-mon[36626]: osdmap e6: 1 total, 0 up, 1 in 2024-07-16T09:49:09.847 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:49:09 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-07-16T09:49:10.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:49:10 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-16T09:49:10.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:49:10 smithi156 ceph-mon[29302]: from='client.? 172.21.15.156:0/4270139535' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-16T09:49:10.843 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:49:10 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-16T09:49:10.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:49:10 smithi176 ceph-mon[36626]: from='client.? 172.21.15.156:0/4270139535' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-16T09:49:11.843 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:49:11 smithi176 ceph-mon[36626]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-16T09:49:11.924 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:49:11 smithi156 ceph-mon[29302]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-16T09:49:14.093 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:49:13 smithi176 ceph-mon[36626]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-16T09:49:14.174 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:49:13 smithi156 ceph-mon[29302]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-16T09:49:16.051 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:49:15 smithi156 ceph-mon[29302]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-16T09:49:16.093 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:49:15 smithi176 ceph-mon[36626]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-16T09:49:17.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:49:17 smithi156 ceph-mon[29302]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-16T09:49:18.093 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:49:17 smithi176 ceph-mon[36626]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-16T09:49:19.424 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:49:19 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:49:19.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:49:19 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:49:19.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:49:19 smithi156 ceph-mon[29302]: Reconfiguring prometheus.smithi156 (dependencies changed)... 2024-07-16T09:49:19.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:49:19 smithi156 ceph-mon[29302]: Reconfiguring daemon prometheus.smithi156 on smithi156 2024-07-16T09:49:19.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:49:19 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:49:19.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:49:19 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:49:19.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:49:19 smithi176 ceph-mon[36626]: Reconfiguring prometheus.smithi156 (dependencies changed)... 2024-07-16T09:49:19.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:49:19 smithi176 ceph-mon[36626]: Reconfiguring daemon prometheus.smithi156 on smithi156 2024-07-16T09:49:20.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:49:20 smithi156 ceph-mon[29302]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-16T09:49:20.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:49:20 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-07-16T09:49:20.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:49:20 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-16T09:49:20.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:49:20 smithi176 ceph-mon[36626]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-16T09:49:20.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:49:20 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-07-16T09:49:20.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:49:20 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-16T09:49:21.424 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:49:21 smithi156 ceph-mon[29302]: Deploying daemon osd.0 on smithi156 2024-07-16T09:49:21.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:49:21 smithi176 ceph-mon[36626]: Deploying daemon osd.0 on smithi156 2024-07-16T09:49:22.525 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:49:22 smithi156 ceph-mon[29302]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-16T09:49:22.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:49:22 smithi176 ceph-mon[36626]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-16T09:49:24.389 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:49:24 smithi156 ceph-mon[29302]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-16T09:49:24.389 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:49:24 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:49:24.389 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:49:24 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:49:24.389 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:49:24 smithi156 ceph-mon[29302]: Reconfiguring crash.smithi176 (monmap changed)... 2024-07-16T09:49:24.389 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:49:24 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi176", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-07-16T09:49:24.389 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:49:24 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-16T09:49:24.389 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:49:24 smithi156 ceph-mon[29302]: Reconfiguring daemon crash.smithi176 on smithi176 2024-07-16T09:49:24.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:49:24 smithi176 ceph-mon[36626]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-16T09:49:24.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:49:24 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:49:24.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:49:24 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:49:24.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:49:24 smithi176 ceph-mon[36626]: Reconfiguring crash.smithi176 (monmap changed)... 2024-07-16T09:49:24.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:49:24 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi176", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-07-16T09:49:24.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:49:24 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-16T09:49:24.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:49:24 smithi176 ceph-mon[36626]: Reconfiguring daemon crash.smithi176 on smithi176 2024-07-16T09:49:25.754 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:49:25 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:49:25.755 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:49:25 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:49:25.755 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:49:25 smithi156 ceph-mon[29302]: Reconfiguring mgr.smithi176.dqbrvx (monmap changed)... 2024-07-16T09:49:25.755 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:49:25 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi176.dqbrvx", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-07-16T09:49:25.755 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:49:25 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "mgr services"}]: dispatch 2024-07-16T09:49:25.755 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:49:25 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-16T09:49:25.755 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:49:25 smithi156 ceph-mon[29302]: Reconfiguring daemon mgr.smithi176.dqbrvx on smithi176 2024-07-16T09:49:25.755 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:49:25 smithi156 ceph-mon[29302]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-16T09:49:25.756 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:49:25 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-16T09:49:25.843 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:49:25 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:49:25.843 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:49:25 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:49:25.843 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:49:25 smithi176 ceph-mon[36626]: Reconfiguring mgr.smithi176.dqbrvx (monmap changed)... 2024-07-16T09:49:25.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:49:25 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi176.dqbrvx", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-07-16T09:49:25.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:49:25 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "mgr services"}]: dispatch 2024-07-16T09:49:25.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:49:25 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-16T09:49:25.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:49:25 smithi176 ceph-mon[36626]: Reconfiguring daemon mgr.smithi176.dqbrvx on smithi176 2024-07-16T09:49:25.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:49:25 smithi176 ceph-mon[36626]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-16T09:49:25.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:49:25 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-16T09:49:27.159 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:49:26 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:49:27.159 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:49:26 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:49:27.159 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:49:26 smithi176 ceph-mon[36626]: Reconfiguring mon.smithi176 (monmap changed)... 2024-07-16T09:49:27.160 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:49:26 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-07-16T09:49:27.160 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:49:26 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-07-16T09:49:27.160 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:49:26 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-16T09:49:27.160 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:49:26 smithi176 ceph-mon[36626]: Reconfiguring daemon mon.smithi176 on smithi176 2024-07-16T09:49:27.168 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:49:26 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:49:27.168 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:49:26 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:49:27.168 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:49:26 smithi156 ceph-mon[29302]: Reconfiguring mon.smithi176 (monmap changed)... 2024-07-16T09:49:27.168 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:49:26 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-07-16T09:49:27.169 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:49:26 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-07-16T09:49:27.169 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:49:26 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-16T09:49:27.169 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:49:26 smithi156 ceph-mon[29302]: Reconfiguring daemon mon.smithi176 on smithi176 2024-07-16T09:49:28.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:49:28 smithi156 ceph-mon[29302]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-16T09:49:28.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:49:28 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:49:28.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:49:28 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:49:28.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:49:28 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-07-16T09:49:28.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:49:28 smithi156 ceph-mon[29302]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-07-16T09:49:28.426 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:49:28 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi156.front.sepia.ceph.com:9093"}]: dispatch 2024-07-16T09:49:28.426 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:49:28 smithi156 ceph-mon[29302]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi156.front.sepia.ceph.com:9093"}]: dispatch 2024-07-16T09:49:28.426 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:49:28 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:49:28.426 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:49:28 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-07-16T09:49:28.426 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:49:28 smithi156 ceph-mon[29302]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-07-16T09:49:28.426 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:49:28 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi156.front.sepia.ceph.com:3000"}]: dispatch 2024-07-16T09:49:28.426 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:49:28 smithi156 ceph-mon[29302]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi156.front.sepia.ceph.com:3000"}]: dispatch 2024-07-16T09:49:28.426 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:49:28 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:49:28.426 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:49:28 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-07-16T09:49:28.426 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:49:28 smithi156 ceph-mon[29302]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-07-16T09:49:28.426 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:49:28 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi156.front.sepia.ceph.com:9095"}]: dispatch 2024-07-16T09:49:28.426 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:49:28 smithi156 ceph-mon[29302]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi156.front.sepia.ceph.com:9095"}]: dispatch 2024-07-16T09:49:28.426 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:49:28 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:49:28.427 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:49:28 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-16T09:49:28.427 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:49:28 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:49:28.427 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:49:28 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:49:28.518 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:49:28 smithi176 ceph-mon[36626]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-16T09:49:28.519 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:49:28 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:49:28.519 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:49:28 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:49:28.519 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:49:28 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-07-16T09:49:28.519 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:49:28 smithi176 ceph-mon[36626]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-07-16T09:49:28.519 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:49:28 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi156.front.sepia.ceph.com:9093"}]: dispatch 2024-07-16T09:49:28.519 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:49:28 smithi176 ceph-mon[36626]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi156.front.sepia.ceph.com:9093"}]: dispatch 2024-07-16T09:49:28.519 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:49:28 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:49:28.519 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:49:28 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-07-16T09:49:28.519 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:49:28 smithi176 ceph-mon[36626]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-07-16T09:49:28.519 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:49:28 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi156.front.sepia.ceph.com:3000"}]: dispatch 2024-07-16T09:49:28.519 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:49:28 smithi176 ceph-mon[36626]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi156.front.sepia.ceph.com:3000"}]: dispatch 2024-07-16T09:49:28.519 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:49:28 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:49:28.520 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:49:28 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-07-16T09:49:28.520 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:49:28 smithi176 ceph-mon[36626]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-07-16T09:49:28.520 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:49:28 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi156.front.sepia.ceph.com:9095"}]: dispatch 2024-07-16T09:49:28.520 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:49:28 smithi176 ceph-mon[36626]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi156.front.sepia.ceph.com:9095"}]: dispatch 2024-07-16T09:49:28.520 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:49:28 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:49:28.520 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:49:28 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-16T09:49:28.520 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:49:28 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:49:28.520 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:49:28 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:49:30.146 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:49:30 smithi156 ceph-mon[29302]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-16T09:49:30.147 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:49:30 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:49:30.147 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:49:30 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:49:30.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:49:30 smithi176 ceph-mon[36626]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-16T09:49:30.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:49:30 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:49:30.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:49:30 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:49:31.593 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:49:31.591+0000 7fd3f6ffd640 1 -- 172.21.15.156:0/2616353314 <== mgr.14208 v2:172.21.15.156:6800/2615741717 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fd40010efd0 con 0x7fd3ec067100 2024-07-16T09:49:31.593 INFO:teuthology.orchestra.run.smithi156.stdout:Created osd(s) 0 on host 'smithi156' 2024-07-16T09:49:31.595 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:49:31.593+0000 7fd40853b640 1 -- 172.21.15.156:0/2616353314 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7fd3ec067100 msgr2=0x7fd3ec0695b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:49:31.595 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:49:31.593+0000 7fd40853b640 1 --2- 172.21.15.156:0/2616353314 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7fd3ec067100 0x7fd3ec0695b0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fd3f0000c00 tx=0x7fd3f0035040 comp rx=0 tx=0).stop 2024-07-16T09:49:31.595 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:49:31.593+0000 7fd40853b640 1 -- 172.21.15.156:0/2616353314 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd4000fc380 msgr2=0x7fd40010cfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:49:31.595 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:49:31.593+0000 7fd40853b640 1 --2- 172.21.15.156:0/2616353314 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd4000fc380 0x7fd40010cfd0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fd3fc00b790 tx=0x7fd3fc0091d0 comp rx=0 tx=0).stop 2024-07-16T09:49:31.595 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:49:31.593+0000 7fd40853b640 1 -- 172.21.15.156:0/2616353314 shutdown_connections 2024-07-16T09:49:31.595 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:49:31.593+0000 7fd40853b640 1 --2- 172.21.15.156:0/2616353314 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7fd3ec067100 0x7fd3ec0695b0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:49:31.595 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:49:31.593+0000 7fd40853b640 1 --2- 172.21.15.156:0/2616353314 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd4000fd650 0x7fd40010d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:49:31.595 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:49:31.593+0000 7fd40853b640 1 --2- 172.21.15.156:0/2616353314 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd4000fc380 0x7fd40010cfd0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:49:31.595 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:49:31.593+0000 7fd40853b640 1 -- 172.21.15.156:0/2616353314 >> 172.21.15.156:0/2616353314 conn(0x7fd4000f7b40 msgr2=0x7fd4000f9190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:49:31.595 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:49:31.593+0000 7fd40853b640 1 -- 172.21.15.156:0/2616353314 shutdown_connections 2024-07-16T09:49:31.595 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:49:31.593+0000 7fd40853b640 1 -- 172.21.15.156:0/2616353314 wait complete. 2024-07-16T09:49:32.174 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:49:32 smithi156 ceph-mon[29302]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-16T09:49:32.174 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:49:32 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:49:32.175 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:49:32 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:49:32.175 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:49:32 smithi156 ceph-mon[29302]: from='osd.0 [v2:172.21.15.156:6802/36114952,v1:172.21.15.156:6803/36114952]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-07-16T09:49:32.215 DEBUG:teuthology.orchestra.run.smithi156:osd.0> sudo journalctl -f -n 0 -u ceph-26c91c26-4358-11ef-bcad-c7b262605968@osd.0.service 2024-07-16T09:49:32.218 INFO:tasks.cephadm:Deploying osd.1 on smithi156 with /dev/vg_nvme/lv_3... 2024-07-16T09:49:32.218 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 26c91c26-4358-11ef-bcad-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-07-16T09:49:32.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:49:32 smithi176 ceph-mon[36626]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-16T09:49:32.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:49:32 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:49:32.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:49:32 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:49:32.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:49:32 smithi176 ceph-mon[36626]: from='osd.0 [v2:172.21.15.156:6802/36114952,v1:172.21.15.156:6803/36114952]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-07-16T09:49:32.924 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/26c91c26-4358-11ef-bcad-c7b262605968/mon.smithi156/config 2024-07-16T09:49:33.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:49:33 smithi176 ceph-mon[36626]: from='osd.0 [v2:172.21.15.156:6802/36114952,v1:172.21.15.156:6803/36114952]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-07-16T09:49:33.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:49:33 smithi176 ceph-mon[36626]: osdmap e7: 1 total, 0 up, 1 in 2024-07-16T09:49:33.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:49:33 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-07-16T09:49:33.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:49:33 smithi176 ceph-mon[36626]: from='osd.0 [v2:172.21.15.156:6802/36114952,v1:172.21.15.156:6803/36114952]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2024-07-16T09:49:33.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:49:33 smithi176 ceph-mon[36626]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-16T09:49:33.876 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:49:33 smithi156 ceph-mon[29302]: from='osd.0 [v2:172.21.15.156:6802/36114952,v1:172.21.15.156:6803/36114952]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-07-16T09:49:33.876 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:49:33 smithi156 ceph-mon[29302]: osdmap e7: 1 total, 0 up, 1 in 2024-07-16T09:49:33.876 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:49:33 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-07-16T09:49:33.876 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:49:33 smithi156 ceph-mon[29302]: from='osd.0 [v2:172.21.15.156:6802/36114952,v1:172.21.15.156:6803/36114952]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2024-07-16T09:49:33.876 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:49:33 smithi156 ceph-mon[29302]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-16T09:49:33.876 INFO:journalctl@ceph.osd.0.smithi156.stdout:Jul 16 09:49:33 smithi156 ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-0[47153]: 2024-07-16T09:49:33.597+0000 7f63838ba640 -1 osd.0 0 waiting for initial osdmap 2024-07-16T09:49:33.876 INFO:journalctl@ceph.osd.0.smithi156.stdout:Jul 16 09:49:33 smithi156 ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-0[47153]: 2024-07-16T09:49:33.599+0000 7f637def0640 -1 osd.0 8 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-07-16T09:49:34.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:49:34 smithi156 ceph-mon[29302]: from='osd.0 [v2:172.21.15.156:6802/36114952,v1:172.21.15.156:6803/36114952]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]': finished 2024-07-16T09:49:34.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:49:34 smithi156 ceph-mon[29302]: osdmap e8: 1 total, 0 up, 1 in 2024-07-16T09:49:34.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:49:34 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-07-16T09:49:34.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:49:34 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-07-16T09:49:34.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:49:34 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-07-16T09:49:35.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:49:34 smithi176 ceph-mon[36626]: from='osd.0 [v2:172.21.15.156:6802/36114952,v1:172.21.15.156:6803/36114952]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]': finished 2024-07-16T09:49:35.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:49:34 smithi176 ceph-mon[36626]: osdmap e8: 1 total, 0 up, 1 in 2024-07-16T09:49:35.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:49:34 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-07-16T09:49:35.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:49:34 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-07-16T09:49:35.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:49:34 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-07-16T09:49:35.674 INFO:teuthology.orchestra.run.smithi156.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-07-16T09:49:35.766 INFO:teuthology.orchestra.run.smithi156.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-07-16T09:49:35.766 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:49:35 smithi156 ceph-mon[29302]: purged_snaps scrub starts 2024-07-16T09:49:35.766 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:49:35 smithi156 ceph-mon[29302]: purged_snaps scrub ok 2024-07-16T09:49:35.766 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:49:35 smithi156 ceph-mon[29302]: osd.0 [v2:172.21.15.156:6802/36114952,v1:172.21.15.156:6803/36114952] boot 2024-07-16T09:49:35.766 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:49:35 smithi156 ceph-mon[29302]: osdmap e9: 1 total, 1 up, 1 in 2024-07-16T09:49:35.766 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:49:35 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-07-16T09:49:35.766 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:49:35 smithi156 ceph-mon[29302]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-16T09:49:35.767 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:49:35 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:49:35.767 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:49:35 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:49:35.778 INFO:teuthology.orchestra.run.smithi156.stderr: stderr: 10+0 records in 2024-07-16T09:49:35.778 INFO:teuthology.orchestra.run.smithi156.stderr:10+0 records out 2024-07-16T09:49:35.778 INFO:teuthology.orchestra.run.smithi156.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0099146 s, 1.1 GB/s 2024-07-16T09:49:35.778 INFO:teuthology.orchestra.run.smithi156.stderr:--> Zapping successful for: 2024-07-16T09:49:36.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:49:35 smithi176 ceph-mon[36626]: purged_snaps scrub starts 2024-07-16T09:49:36.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:49:35 smithi176 ceph-mon[36626]: purged_snaps scrub ok 2024-07-16T09:49:36.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:49:35 smithi176 ceph-mon[36626]: osd.0 [v2:172.21.15.156:6802/36114952,v1:172.21.15.156:6803/36114952] boot 2024-07-16T09:49:36.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:49:35 smithi176 ceph-mon[36626]: osdmap e9: 1 total, 1 up, 1 in 2024-07-16T09:49:36.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:49:35 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-07-16T09:49:36.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:49:35 smithi176 ceph-mon[36626]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-16T09:49:36.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:49:35 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:49:36.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:49:35 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:49:36.415 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 26c91c26-4358-11ef-bcad-c7b262605968 -- ceph orch daemon add osd smithi156:vg_nvme/lv_3 2024-07-16T09:49:36.647 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:49:36 smithi156 ceph-mon[29302]: osdmap e10: 1 total, 1 up, 1 in 2024-07-16T09:49:37.093 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:49:36 smithi176 ceph-mon[36626]: osdmap e10: 1 total, 1 up, 1 in 2024-07-16T09:49:37.924 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:49:37 smithi156 ceph-mon[29302]: pgmap v34: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-07-16T09:49:37.959 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/26c91c26-4358-11ef-bcad-c7b262605968/mon.smithi156/config 2024-07-16T09:49:38.093 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:49:37 smithi176 ceph-mon[36626]: pgmap v34: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-07-16T09:49:40.098 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:49:40.096+0000 7f0d8a272640 1 -- 172.21.15.156:0/1804987989 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0d840fc330 msgr2=0x7f0d840fc740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:49:40.098 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:49:40.096+0000 7f0d8a272640 1 --2- 172.21.15.156:0/1804987989 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0d840fc330 0x7f0d840fc740 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f0d740048d0 tx=0x7f0d74009da0 comp rx=0 tx=0).stop 2024-07-16T09:49:40.098 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:49:40.096+0000 7f0d8a272640 1 -- 172.21.15.156:0/1804987989 shutdown_connections 2024-07-16T09:49:40.098 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:49:40.096+0000 7f0d8a272640 1 --2- 172.21.15.156:0/1804987989 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0d840fd600 0x7f0d840fda70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:49:40.099 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:49:40.096+0000 7f0d8a272640 1 --2- 172.21.15.156:0/1804987989 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0d840fc330 0x7f0d840fc740 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:49:40.099 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:49:40.096+0000 7f0d8a272640 1 -- 172.21.15.156:0/1804987989 >> 172.21.15.156:0/1804987989 conn(0x7f0d840f7af0 msgr2=0x7f0d840f9f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:49:40.099 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:49:40.096+0000 7f0d8a272640 1 -- 172.21.15.156:0/1804987989 shutdown_connections 2024-07-16T09:49:40.099 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:49:40.096+0000 7f0d8a272640 1 -- 172.21.15.156:0/1804987989 wait complete. 2024-07-16T09:49:40.099 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:49:40.096+0000 7f0d8a272640 1 Processor -- start 2024-07-16T09:49:40.099 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:49:40.096+0000 7f0d8a272640 1 -- start start 2024-07-16T09:49:40.099 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:49:40.097+0000 7f0d8a272640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0d840fc330 0x7f0d84071380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:49:40.099 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:49:40.097+0000 7f0d8a272640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0d840fd600 0x7f0d8406f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:49:40.099 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:49:40.097+0000 7f0d8a272640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d8406fe00 con 0x7f0d840fc330 2024-07-16T09:49:40.099 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:49:40.097+0000 7f0d8a272640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d8406ff40 con 0x7f0d840fd600 2024-07-16T09:49:40.099 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:49:40.097+0000 7f0d83fff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0d840fc330 0x7f0d84071380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:49:40.099 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:49:40.097+0000 7f0d83fff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0d840fc330 0x7f0d84071380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:42060/0 (socket says 172.21.15.156:42060) 2024-07-16T09:49:40.099 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:49:40.097+0000 7f0d837fe640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0d840fd600 0x7f0d8406f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:49:40.100 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:49:40.097+0000 7f0d83fff640 1 -- 172.21.15.156:0/399565521 learned_addr learned my addr 172.21.15.156:0/399565521 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T09:49:40.100 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:49:40.097+0000 7f0d83fff640 1 -- 172.21.15.156:0/399565521 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0d840fd600 msgr2=0x7f0d8406f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:49:40.100 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:49:40.097+0000 7f0d83fff640 1 --2- 172.21.15.156:0/399565521 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0d840fd600 0x7f0d8406f870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:49:40.100 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:49:40.097+0000 7f0d83fff640 1 -- 172.21.15.156:0/399565521 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d74005040 con 0x7f0d840fc330 2024-07-16T09:49:40.100 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:49:40.097+0000 7f0d83fff640 1 --2- 172.21.15.156:0/399565521 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0d840fc330 0x7f0d84071380 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f0d74009f80 tx=0x7f0d74009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:49:40.101 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:49:40.097+0000 7f0d80ff9640 1 -- 172.21.15.156:0/399565521 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0d74038070 con 0x7f0d840fc330 2024-07-16T09:49:40.101 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:49:40.097+0000 7f0d80ff9640 1 -- 172.21.15.156:0/399565521 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0d74032970 con 0x7f0d840fc330 2024-07-16T09:49:40.101 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:49:40.097+0000 7f0d8a272640 1 -- 172.21.15.156:0/399565521 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d840701c0 con 0x7f0d840fc330 2024-07-16T09:49:40.101 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:49:40.097+0000 7f0d80ff9640 1 -- 172.21.15.156:0/399565521 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0d74033860 con 0x7f0d840fc330 2024-07-16T09:49:40.101 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:49:40.097+0000 7f0d8a272640 1 -- 172.21.15.156:0/399565521 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d84114f90 con 0x7f0d840fc330 2024-07-16T09:49:40.101 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:49:40.098+0000 7f0d8a272640 1 -- 172.21.15.156:0/399565521 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d840fb790 con 0x7f0d840fc330 2024-07-16T09:49:40.101 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:49:40.099+0000 7f0d80ff9640 1 -- 172.21.15.156:0/399565521 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 18) v1 ==== 85071+0+0 (secure 0 0 0) 0x7f0d74030020 con 0x7f0d840fc330 2024-07-16T09:49:40.102 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:49:40.099+0000 7f0d80ff9640 1 --2- 172.21.15.156:0/399565521 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f0d6c067100 0x7f0d6c0695b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:49:40.102 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:49:40.099+0000 7f0d80ff9640 1 -- 172.21.15.156:0/399565521 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1978+0+0 (secure 0 0 0) 0x7f0d740a3a30 con 0x7f0d840fc330 2024-07-16T09:49:40.102 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:49:40.099+0000 7f0d837fe640 1 --2- 172.21.15.156:0/399565521 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f0d6c067100 0x7f0d6c0695b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:49:40.102 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:49:40.100+0000 7f0d837fe640 1 --2- 172.21.15.156:0/399565521 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f0d6c067100 0x7f0d6c0695b0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f0d78004eb0 tx=0x7f0d7800b040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:49:40.105 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:49:40.104+0000 7f0d80ff9640 1 -- 172.21.15.156:0/399565521 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f0d74071c80 con 0x7f0d840fc330 2024-07-16T09:49:40.288 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:49:40.286+0000 7f0d8a272640 1 -- 172.21.15.156:0/399565521 --> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi156:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f0d84101930 con 0x7f0d6c067100 2024-07-16T09:49:40.445 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:49:40 smithi156 ceph-mon[29302]: pgmap v35: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-07-16T09:49:40.445 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:49:40 smithi156 ceph-mon[29302]: Detected new or changed devices on smithi156 2024-07-16T09:49:40.445 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:49:40 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:49:40.446 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:49:40 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:49:40.446 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:49:40 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-07-16T09:49:40.446 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:49:40 smithi156 ceph-mon[29302]: Adjusting osd_memory_target on smithi156 to 12864M 2024-07-16T09:49:40.446 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:49:40 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:49:40.446 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:49:40 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-16T09:49:40.446 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:49:40 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-16T09:49:40.446 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:49:40 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:49:40.446 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:49:40 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-16T09:49:40.446 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:49:40 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-16T09:49:40.461 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:49:40 smithi176 ceph-mon[36626]: pgmap v35: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-07-16T09:49:40.461 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:49:40 smithi176 ceph-mon[36626]: Detected new or changed devices on smithi156 2024-07-16T09:49:40.461 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:49:40 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:49:40.461 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:49:40 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:49:40.461 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:49:40 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-07-16T09:49:40.461 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:49:40 smithi176 ceph-mon[36626]: Adjusting osd_memory_target on smithi156 to 12864M 2024-07-16T09:49:40.462 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:49:40 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:49:40.462 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:49:40 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-16T09:49:40.462 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:49:40 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-16T09:49:40.462 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:49:40 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:49:40.462 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:49:40 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-16T09:49:40.462 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:49:40 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-16T09:49:41.206 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:49:41 smithi156 ceph-mon[29302]: from='client.14258 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi156:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-07-16T09:49:41.207 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:49:41 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-16T09:49:41.207 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:49:41 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-16T09:49:41.207 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:49:41 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-16T09:49:41.207 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:49:41 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:49:41.207 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:49:41 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:49:41.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:49:41 smithi176 ceph-mon[36626]: from='client.14258 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi156:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-07-16T09:49:41.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:49:41 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-16T09:49:41.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:49:41 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-16T09:49:41.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:49:41 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-16T09:49:41.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:49:41 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:49:41.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:49:41 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:49:42.430 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:49:42 smithi156 ceph-mon[29302]: pgmap v36: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-07-16T09:49:42.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:49:42 smithi176 ceph-mon[36626]: pgmap v36: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-07-16T09:49:44.349 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:49:44 smithi156 ceph-mon[29302]: pgmap v37: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-07-16T09:49:44.349 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:49:44 smithi156 ceph-mon[29302]: from='client.? 172.21.15.156:0/3957565155' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "04d30aff-25e2-4b87-927d-a2497d0123b8"}]: dispatch 2024-07-16T09:49:44.349 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:49:44 smithi156 ceph-mon[29302]: from='client.? 172.21.15.156:0/3957565155' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "04d30aff-25e2-4b87-927d-a2497d0123b8"}]': finished 2024-07-16T09:49:44.349 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:49:44 smithi156 ceph-mon[29302]: osdmap e11: 2 total, 1 up, 2 in 2024-07-16T09:49:44.350 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:49:44 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-07-16T09:49:44.350 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:49:44 smithi156 ceph-mon[29302]: from='client.? 172.21.15.156:0/3673261596' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-16T09:49:44.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:49:44 smithi176 ceph-mon[36626]: pgmap v37: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-07-16T09:49:44.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:49:44 smithi176 ceph-mon[36626]: from='client.? 172.21.15.156:0/3957565155' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "04d30aff-25e2-4b87-927d-a2497d0123b8"}]: dispatch 2024-07-16T09:49:44.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:49:44 smithi176 ceph-mon[36626]: from='client.? 172.21.15.156:0/3957565155' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "04d30aff-25e2-4b87-927d-a2497d0123b8"}]': finished 2024-07-16T09:49:44.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:49:44 smithi176 ceph-mon[36626]: osdmap e11: 2 total, 1 up, 2 in 2024-07-16T09:49:44.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:49:44 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-07-16T09:49:44.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:49:44 smithi176 ceph-mon[36626]: from='client.? 172.21.15.156:0/3673261596' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-16T09:49:46.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:49:46 smithi176 ceph-mon[36626]: pgmap v39: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-07-16T09:49:46.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:49:46 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:49:46.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:49:46 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:49:46.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:49:46 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-16T09:49:46.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:49:46 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-16T09:49:46.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:49:46 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:49:46.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:49:46 smithi156 ceph-mon[29302]: pgmap v39: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-07-16T09:49:46.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:49:46 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:49:46.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:49:46 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:49:46.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:49:46 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-16T09:49:46.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:49:46 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-16T09:49:46.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:49:46 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:49:48.424 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:49:48 smithi156 ceph-mon[29302]: pgmap v40: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-07-16T09:49:48.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:49:48 smithi176 ceph-mon[36626]: pgmap v40: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-07-16T09:49:50.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:49:50 smithi176 ceph-mon[36626]: pgmap v41: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-07-16T09:49:50.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:49:50 smithi156 ceph-mon[29302]: pgmap v41: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-07-16T09:49:51.390 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:49:51 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-07-16T09:49:51.390 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:49:51 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-16T09:49:51.390 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:49:51 smithi156 ceph-mon[29302]: Deploying daemon osd.1 on smithi156 2024-07-16T09:49:51.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:49:51 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-07-16T09:49:51.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:49:51 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-16T09:49:51.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:49:51 smithi176 ceph-mon[36626]: Deploying daemon osd.1 on smithi156 2024-07-16T09:49:52.390 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:49:52 smithi156 ceph-mon[29302]: pgmap v42: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-07-16T09:49:52.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:49:52 smithi176 ceph-mon[36626]: pgmap v42: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-07-16T09:49:54.419 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:49:54 smithi156 ceph-mon[29302]: pgmap v43: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-07-16T09:49:54.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:49:54 smithi176 ceph-mon[36626]: pgmap v43: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-07-16T09:49:55.474 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:49:55 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-16T09:49:55.474 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:49:55 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-16T09:49:55.474 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:49:55 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:49:55.474 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:49:55 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:49:55.555 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:49:55 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-16T09:49:55.555 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:49:55 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-16T09:49:55.555 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:49:55 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:49:55.555 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:49:55 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:49:56.546 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:49:56 smithi156 ceph-mon[29302]: pgmap v44: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-07-16T09:49:56.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:49:56 smithi176 ceph-mon[36626]: pgmap v44: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-07-16T09:49:57.842 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:49:57 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:49:57.842 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:49:57 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:49:57.842 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:49:57 smithi156 ceph-mon[29302]: pgmap v45: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-07-16T09:49:58.093 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:49:57 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:49:58.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:49:57 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:49:58.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:49:57 smithi176 ceph-mon[36626]: pgmap v45: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-07-16T09:49:58.453 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:49:58.449+0000 7f0d80ff9640 1 -- 172.21.15.156:0/399565521 <== mgr.14208 v2:172.21.15.156:6800/2615741717 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f0d84101930 con 0x7f0d6c067100 2024-07-16T09:49:58.453 INFO:teuthology.orchestra.run.smithi156.stdout:Created osd(s) 1 on host 'smithi156' 2024-07-16T09:49:58.454 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:49:58.452+0000 7f0d8a272640 1 -- 172.21.15.156:0/399565521 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f0d6c067100 msgr2=0x7f0d6c0695b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:49:58.454 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:49:58.452+0000 7f0d8a272640 1 --2- 172.21.15.156:0/399565521 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f0d6c067100 0x7f0d6c0695b0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f0d78004eb0 tx=0x7f0d7800b040 comp rx=0 tx=0).stop 2024-07-16T09:49:58.454 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:49:58.452+0000 7f0d8a272640 1 -- 172.21.15.156:0/399565521 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0d840fc330 msgr2=0x7f0d84071380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:49:58.454 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:49:58.452+0000 7f0d8a272640 1 --2- 172.21.15.156:0/399565521 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0d840fc330 0x7f0d84071380 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f0d74009f80 tx=0x7f0d74009da0 comp rx=0 tx=0).stop 2024-07-16T09:49:58.455 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:49:58.452+0000 7f0d8a272640 1 -- 172.21.15.156:0/399565521 shutdown_connections 2024-07-16T09:49:58.455 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:49:58.452+0000 7f0d8a272640 1 --2- 172.21.15.156:0/399565521 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f0d6c067100 0x7f0d6c0695b0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:49:58.455 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:49:58.452+0000 7f0d8a272640 1 --2- 172.21.15.156:0/399565521 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0d840fd600 0x7f0d8406f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:49:58.455 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:49:58.452+0000 7f0d8a272640 1 --2- 172.21.15.156:0/399565521 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0d840fc330 0x7f0d84071380 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:49:58.455 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:49:58.452+0000 7f0d8a272640 1 -- 172.21.15.156:0/399565521 >> 172.21.15.156:0/399565521 conn(0x7f0d840f7af0 msgr2=0x7f0d840f91d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:49:58.455 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:49:58.452+0000 7f0d8a272640 1 -- 172.21.15.156:0/399565521 shutdown_connections 2024-07-16T09:49:58.455 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:49:58.452+0000 7f0d8a272640 1 -- 172.21.15.156:0/399565521 wait complete. 2024-07-16T09:49:59.426 DEBUG:teuthology.orchestra.run.smithi156:osd.1> sudo journalctl -f -n 0 -u ceph-26c91c26-4358-11ef-bcad-c7b262605968@osd.1.service 2024-07-16T09:49:59.429 INFO:tasks.cephadm:Deploying osd.2 on smithi156 with /dev/vg_nvme/lv_2... 2024-07-16T09:49:59.429 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 26c91c26-4358-11ef-bcad-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-07-16T09:49:59.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:49:59 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:49:59.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:49:59 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:49:59.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:49:59 smithi156 ceph-mon[29302]: pgmap v46: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-07-16T09:49:59.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:49:59 smithi156 ceph-mon[29302]: from='osd.1 [v2:172.21.15.156:6810/1587604616,v1:172.21.15.156:6811/1587604616]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-07-16T09:49:59.843 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:49:59 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:49:59.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:49:59 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:49:59.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:49:59 smithi176 ceph-mon[36626]: pgmap v46: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-07-16T09:49:59.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:49:59 smithi176 ceph-mon[36626]: from='osd.1 [v2:172.21.15.156:6810/1587604616,v1:172.21.15.156:6811/1587604616]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-07-16T09:49:59.995 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/26c91c26-4358-11ef-bcad-c7b262605968/mon.smithi156/config 2024-07-16T09:50:00.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:50:00 smithi176 ceph-mon[36626]: from='osd.1 [v2:172.21.15.156:6810/1587604616,v1:172.21.15.156:6811/1587604616]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-07-16T09:50:00.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:50:00 smithi176 ceph-mon[36626]: osdmap e12: 2 total, 1 up, 2 in 2024-07-16T09:50:00.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:50:00 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-07-16T09:50:00.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:50:00 smithi176 ceph-mon[36626]: from='osd.1 [v2:172.21.15.156:6810/1587604616,v1:172.21.15.156:6811/1587604616]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2024-07-16T09:50:00.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:50:00 smithi176 ceph-mon[36626]: overall HEALTH_OK 2024-07-16T09:50:00.924 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:50:00 smithi156 ceph-mon[29302]: from='osd.1 [v2:172.21.15.156:6810/1587604616,v1:172.21.15.156:6811/1587604616]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-07-16T09:50:00.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:50:00 smithi156 ceph-mon[29302]: osdmap e12: 2 total, 1 up, 2 in 2024-07-16T09:50:00.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:50:00 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-07-16T09:50:00.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:50:00 smithi156 ceph-mon[29302]: from='osd.1 [v2:172.21.15.156:6810/1587604616,v1:172.21.15.156:6811/1587604616]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2024-07-16T09:50:00.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:50:00 smithi156 ceph-mon[29302]: overall HEALTH_OK 2024-07-16T09:50:00.925 INFO:journalctl@ceph.osd.1.smithi156.stdout:Jul 16 09:50:00 smithi156 ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-1[54640]: 2024-07-16T09:50:00.458+0000 7f0f2a2a2640 -1 osd.1 0 waiting for initial osdmap 2024-07-16T09:50:00.925 INFO:journalctl@ceph.osd.1.smithi156.stdout:Jul 16 09:50:00 smithi156 ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-1[54640]: 2024-07-16T09:50:00.463+0000 7f0f218b0640 -1 osd.1 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-07-16T09:50:01.559 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:50:01 smithi156 ceph-mon[29302]: from='osd.1 [v2:172.21.15.156:6810/1587604616,v1:172.21.15.156:6811/1587604616]' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]': finished 2024-07-16T09:50:01.559 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:50:01 smithi156 ceph-mon[29302]: osdmap e13: 2 total, 1 up, 2 in 2024-07-16T09:50:01.559 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:50:01 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-07-16T09:50:01.559 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:50:01 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-07-16T09:50:01.560 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:50:01 smithi156 ceph-mon[29302]: pgmap v49: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-07-16T09:50:01.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:50:01 smithi176 ceph-mon[36626]: from='osd.1 [v2:172.21.15.156:6810/1587604616,v1:172.21.15.156:6811/1587604616]' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]': finished 2024-07-16T09:50:01.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:50:01 smithi176 ceph-mon[36626]: osdmap e13: 2 total, 1 up, 2 in 2024-07-16T09:50:01.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:50:01 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-07-16T09:50:01.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:50:01 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-07-16T09:50:01.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:50:01 smithi176 ceph-mon[36626]: pgmap v49: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-07-16T09:50:02.057 INFO:teuthology.orchestra.run.smithi156.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-07-16T09:50:02.143 INFO:teuthology.orchestra.run.smithi156.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-07-16T09:50:02.158 INFO:teuthology.orchestra.run.smithi156.stderr: stderr: 10+0 records in 2024-07-16T09:50:02.158 INFO:teuthology.orchestra.run.smithi156.stderr:10+0 records out 2024-07-16T09:50:02.158 INFO:teuthology.orchestra.run.smithi156.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0132937 s, 789 MB/s 2024-07-16T09:50:02.158 INFO:teuthology.orchestra.run.smithi156.stderr:--> Zapping successful for: 2024-07-16T09:50:02.745 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:50:02 smithi176 ceph-mon[36626]: purged_snaps scrub starts 2024-07-16T09:50:02.745 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:50:02 smithi176 ceph-mon[36626]: purged_snaps scrub ok 2024-07-16T09:50:02.745 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:50:02 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-07-16T09:50:02.745 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:50:02 smithi176 ceph-mon[36626]: osd.1 [v2:172.21.15.156:6810/1587604616,v1:172.21.15.156:6811/1587604616] boot 2024-07-16T09:50:02.745 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:50:02 smithi176 ceph-mon[36626]: osdmap e14: 2 total, 2 up, 2 in 2024-07-16T09:50:02.745 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:50:02 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-07-16T09:50:02.850 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:50:02 smithi156 ceph-mon[29302]: purged_snaps scrub starts 2024-07-16T09:50:02.850 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:50:02 smithi156 ceph-mon[29302]: purged_snaps scrub ok 2024-07-16T09:50:02.850 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:50:02 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-07-16T09:50:02.850 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:50:02 smithi156 ceph-mon[29302]: osd.1 [v2:172.21.15.156:6810/1587604616,v1:172.21.15.156:6811/1587604616] boot 2024-07-16T09:50:02.851 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:50:02 smithi156 ceph-mon[29302]: osdmap e14: 2 total, 2 up, 2 in 2024-07-16T09:50:02.851 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:50:02 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-07-16T09:50:02.876 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 26c91c26-4358-11ef-bcad-c7b262605968 -- ceph orch daemon add osd smithi156:vg_nvme/lv_2 2024-07-16T09:50:03.201 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/26c91c26-4358-11ef-bcad-c7b262605968/mon.smithi156/config 2024-07-16T09:50:03.638 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:50:03 smithi156 ceph-mon[29302]: osdmap e15: 2 total, 2 up, 2 in 2024-07-16T09:50:03.638 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:50:03 smithi156 ceph-mon[29302]: pgmap v52: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-07-16T09:50:03.639 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:50:03 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:50:03.639 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:50:03 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:50:03.843 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:50:03 smithi176 ceph-mon[36626]: osdmap e15: 2 total, 2 up, 2 in 2024-07-16T09:50:03.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:50:03 smithi176 ceph-mon[36626]: pgmap v52: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-07-16T09:50:03.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:50:03 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:50:03.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:50:03 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:50:05.358 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:50:05.339+0000 7fc2a17e9640 1 -- 172.21.15.156:0/638327731 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc29c0fc360 msgr2=0x7fc29c0fc770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:50:05.358 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:50:05.339+0000 7fc2a17e9640 1 --2- 172.21.15.156:0/638327731 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc29c0fc360 0x7fc29c0fc770 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fc28c0048d0 tx=0x7fc28c009da0 comp rx=0 tx=0).stop 2024-07-16T09:50:05.358 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:50:05.339+0000 7fc2a17e9640 1 -- 172.21.15.156:0/638327731 shutdown_connections 2024-07-16T09:50:05.359 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:50:05.339+0000 7fc2a17e9640 1 --2- 172.21.15.156:0/638327731 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc29c0fd630 0x7fc29c0fdaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:50:05.359 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:50:05.339+0000 7fc2a17e9640 1 --2- 172.21.15.156:0/638327731 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc29c0fc360 0x7fc29c0fc770 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:50:05.359 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:50:05.339+0000 7fc2a17e9640 1 -- 172.21.15.156:0/638327731 >> 172.21.15.156:0/638327731 conn(0x7fc29c0f7b40 msgr2=0x7fc29c0f9f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:50:05.359 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:50:05.339+0000 7fc2a17e9640 1 -- 172.21.15.156:0/638327731 shutdown_connections 2024-07-16T09:50:05.359 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:50:05.339+0000 7fc2a17e9640 1 -- 172.21.15.156:0/638327731 wait complete. 2024-07-16T09:50:05.359 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:50:05.340+0000 7fc2a17e9640 1 Processor -- start 2024-07-16T09:50:05.359 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:50:05.340+0000 7fc2a17e9640 1 -- start start 2024-07-16T09:50:05.359 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:50:05.340+0000 7fc2a17e9640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc29c0fc360 0x7fc29c10cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:50:05.359 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:50:05.340+0000 7fc2a17e9640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc29c0fd630 0x7fc29c10d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:50:05.359 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:50:05.340+0000 7fc2a17e9640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc29c10e340 con 0x7fc29c0fc360 2024-07-16T09:50:05.359 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:50:05.340+0000 7fc2a17e9640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc29c10e4b0 con 0x7fc29c0fd630 2024-07-16T09:50:05.359 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:50:05.340+0000 7fc29affd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc29c0fc360 0x7fc29c10cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:50:05.359 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:50:05.340+0000 7fc29affd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc29c0fc360 0x7fc29c10cf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:60600/0 (socket says 172.21.15.156:60600) 2024-07-16T09:50:05.360 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:50:05.340+0000 7fc29affd640 1 -- 172.21.15.156:0/54568233 learned_addr learned my addr 172.21.15.156:0/54568233 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T09:50:05.360 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:50:05.340+0000 7fc29a7fc640 1 --2- 172.21.15.156:0/54568233 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc29c0fd630 0x7fc29c10d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:50:05.360 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:50:05.340+0000 7fc29affd640 1 -- 172.21.15.156:0/54568233 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc29c0fd630 msgr2=0x7fc29c10d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:50:05.360 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:50:05.340+0000 7fc29affd640 1 --2- 172.21.15.156:0/54568233 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc29c0fd630 0x7fc29c10d4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:50:05.360 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:50:05.340+0000 7fc29affd640 1 -- 172.21.15.156:0/54568233 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc28c005040 con 0x7fc29c0fc360 2024-07-16T09:50:05.360 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:50:05.340+0000 7fc29affd640 1 --2- 172.21.15.156:0/54568233 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc29c0fc360 0x7fc29c10cf90 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fc28c003080 tx=0x7fc28c009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:50:05.360 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:50:05.340+0000 7fc2937fe640 1 -- 172.21.15.156:0/54568233 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc28c038070 con 0x7fc29c0fc360 2024-07-16T09:50:05.360 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:50:05.340+0000 7fc2937fe640 1 -- 172.21.15.156:0/54568233 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc28c032970 con 0x7fc29c0fc360 2024-07-16T09:50:05.360 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:50:05.340+0000 7fc2a17e9640 1 -- 172.21.15.156:0/54568233 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc29c10e730 con 0x7fc29c0fc360 2024-07-16T09:50:05.360 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:50:05.341+0000 7fc2937fe640 1 -- 172.21.15.156:0/54568233 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc28c033900 con 0x7fc29c0fc360 2024-07-16T09:50:05.360 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:50:05.341+0000 7fc2a17e9640 1 -- 172.21.15.156:0/54568233 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc29c10ebf0 con 0x7fc29c0fc360 2024-07-16T09:50:05.360 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:50:05.341+0000 7fc2a17e9640 1 -- 172.21.15.156:0/54568233 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc29c105740 con 0x7fc29c0fc360 2024-07-16T09:50:05.360 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:50:05.342+0000 7fc2937fe640 1 -- 172.21.15.156:0/54568233 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 18) v1 ==== 85071+0+0 (secure 0 0 0) 0x7fc28c030020 con 0x7fc29c0fc360 2024-07-16T09:50:05.360 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:50:05.342+0000 7fc2937fe640 1 --2- 172.21.15.156:0/54568233 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7fc288067100 0x7fc2880695b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:50:05.361 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:50:05.342+0000 7fc2937fe640 1 -- 172.21.15.156:0/54568233 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 2410+0+0 (secure 0 0 0) 0x7fc28c0a3ed0 con 0x7fc29c0fc360 2024-07-16T09:50:05.361 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:50:05.342+0000 7fc29a7fc640 1 --2- 172.21.15.156:0/54568233 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7fc288067100 0x7fc2880695b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:50:05.361 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:50:05.343+0000 7fc29a7fc640 1 --2- 172.21.15.156:0/54568233 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7fc288067100 0x7fc2880695b0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fc284006fd0 tx=0x7fc284008040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:50:05.361 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:50:05.347+0000 7fc2937fe640 1 -- 172.21.15.156:0/54568233 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fc28c071e40 con 0x7fc29c0fc360 2024-07-16T09:50:05.526 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:50:05.524+0000 7fc2a17e9640 1 -- 172.21.15.156:0/54568233 --> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi156:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7fc29c10ee10 con 0x7fc288067100 2024-07-16T09:50:06.093 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:50:05 smithi176 ceph-mon[36626]: pgmap v53: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-07-16T09:50:06.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:50:05 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-16T09:50:06.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:50:05 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-16T09:50:06.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:50:05 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-16T09:50:06.106 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:50:05 smithi156 ceph-mon[29302]: pgmap v53: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-07-16T09:50:06.106 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:50:05 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-16T09:50:06.106 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:50:05 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-16T09:50:06.106 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:50:05 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-16T09:50:07.093 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:50:06 smithi176 ceph-mon[36626]: from='client.14276 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi156:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-07-16T09:50:07.175 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:50:06 smithi156 ceph-mon[29302]: from='client.14276 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi156:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-07-16T09:50:07.818 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:50:07 smithi156 ceph-mon[29302]: pgmap v54: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-07-16T09:50:08.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:50:07 smithi176 ceph-mon[36626]: pgmap v54: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-07-16T09:50:09.175 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:50:08 smithi156 ceph-mon[29302]: Detected new or changed devices on smithi156 2024-07-16T09:50:09.175 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:50:08 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:50:09.175 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:50:08 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:50:09.175 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:50:08 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-07-16T09:50:09.175 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:50:08 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-07-16T09:50:09.176 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:50:08 smithi156 ceph-mon[29302]: Adjusting osd_memory_target on smithi156 to 6432M 2024-07-16T09:50:09.176 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:50:08 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:50:09.176 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:50:08 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-16T09:50:09.176 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:50:08 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-16T09:50:09.176 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:50:08 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:50:09.176 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:50:08 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-16T09:50:09.176 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:50:08 smithi156 ceph-mon[29302]: from='client.? 172.21.15.156:0/61309089' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5b2cee4f-d3da-4dac-85b9-ee34914e548a"}]: dispatch 2024-07-16T09:50:09.176 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:50:08 smithi156 ceph-mon[29302]: from='client.? 172.21.15.156:0/61309089' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5b2cee4f-d3da-4dac-85b9-ee34914e548a"}]': finished 2024-07-16T09:50:09.176 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:50:08 smithi156 ceph-mon[29302]: osdmap e16: 3 total, 2 up, 3 in 2024-07-16T09:50:09.176 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:50:08 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-07-16T09:50:09.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:50:08 smithi176 ceph-mon[36626]: Detected new or changed devices on smithi156 2024-07-16T09:50:09.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:50:08 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:50:09.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:50:08 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:50:09.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:50:08 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-07-16T09:50:09.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:50:08 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-07-16T09:50:09.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:50:08 smithi176 ceph-mon[36626]: Adjusting osd_memory_target on smithi156 to 6432M 2024-07-16T09:50:09.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:50:08 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:50:09.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:50:08 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-16T09:50:09.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:50:08 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-16T09:50:09.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:50:08 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:50:09.345 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:50:08 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-16T09:50:09.345 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:50:08 smithi176 ceph-mon[36626]: from='client.? 172.21.15.156:0/61309089' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5b2cee4f-d3da-4dac-85b9-ee34914e548a"}]: dispatch 2024-07-16T09:50:09.345 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:50:08 smithi176 ceph-mon[36626]: from='client.? 172.21.15.156:0/61309089' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5b2cee4f-d3da-4dac-85b9-ee34914e548a"}]': finished 2024-07-16T09:50:09.345 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:50:08 smithi176 ceph-mon[36626]: osdmap e16: 3 total, 2 up, 3 in 2024-07-16T09:50:09.345 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:50:08 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-07-16T09:50:10.174 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:50:09 smithi156 ceph-mon[29302]: pgmap v55: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-07-16T09:50:10.175 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:50:09 smithi156 ceph-mon[29302]: from='client.? 172.21.15.156:0/2700050389' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-16T09:50:10.175 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:50:09 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-16T09:50:10.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:50:09 smithi176 ceph-mon[36626]: pgmap v55: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-07-16T09:50:10.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:50:09 smithi176 ceph-mon[36626]: from='client.? 172.21.15.156:0/2700050389' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-16T09:50:10.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:50:09 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-16T09:50:12.174 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:50:11 smithi156 ceph-mon[29302]: pgmap v57: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-07-16T09:50:12.343 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:50:11 smithi176 ceph-mon[36626]: pgmap v57: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-07-16T09:50:13.424 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:50:13 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:50:13.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:50:13 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:50:13.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:50:13 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-16T09:50:13.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:50:13 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-16T09:50:13.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:50:13 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:50:13.438 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:50:13 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:50:13.438 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:50:13 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:50:13.438 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:50:13 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-16T09:50:13.439 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:50:13 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-16T09:50:13.439 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:50:13 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:50:14.343 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:50:14 smithi176 ceph-mon[36626]: pgmap v58: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-07-16T09:50:14.424 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:50:14 smithi156 ceph-mon[29302]: pgmap v58: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-07-16T09:50:16.247 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:50:16 smithi156 ceph-mon[29302]: pgmap v59: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-07-16T09:50:16.247 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:50:16 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-07-16T09:50:16.248 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:50:16 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-16T09:50:16.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:50:16 smithi176 ceph-mon[36626]: pgmap v59: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-07-16T09:50:16.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:50:16 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-07-16T09:50:16.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:50:16 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-16T09:50:17.243 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:50:17 smithi156 ceph-mon[29302]: Deploying daemon osd.2 on smithi156 2024-07-16T09:50:17.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:50:17 smithi176 ceph-mon[36626]: Deploying daemon osd.2 on smithi156 2024-07-16T09:50:18.340 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:50:18 smithi156 ceph-mon[29302]: pgmap v60: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-07-16T09:50:18.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:50:18 smithi176 ceph-mon[36626]: pgmap v60: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-07-16T09:50:20.424 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:50:20 smithi156 ceph-mon[29302]: pgmap v61: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-07-16T09:50:20.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:50:20 smithi176 ceph-mon[36626]: pgmap v61: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-07-16T09:50:21.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:50:21 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-16T09:50:21.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:50:21 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:50:21.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:50:21 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:50:21.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:50:21 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-16T09:50:21.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:50:21 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:50:21.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:50:21 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:50:22.652 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:50:22 smithi156 ceph-mon[29302]: pgmap v62: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-07-16T09:50:22.843 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:50:22 smithi176 ceph-mon[36626]: pgmap v62: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-07-16T09:50:24.126 INFO:teuthology.orchestra.run.smithi156.stdout:Created osd(s) 2 on host 'smithi156' 2024-07-16T09:50:24.126 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:50:24.124+0000 7fc2937fe640 1 -- 172.21.15.156:0/54568233 <== mgr.14208 v2:172.21.15.156:6800/2615741717 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fc29c10ee10 con 0x7fc288067100 2024-07-16T09:50:24.128 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:50:24.126+0000 7fc2a17e9640 1 -- 172.21.15.156:0/54568233 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7fc288067100 msgr2=0x7fc2880695b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:50:24.128 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:50:24.126+0000 7fc2a17e9640 1 --2- 172.21.15.156:0/54568233 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7fc288067100 0x7fc2880695b0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fc284006fd0 tx=0x7fc284008040 comp rx=0 tx=0).stop 2024-07-16T09:50:24.128 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:50:24.126+0000 7fc2a17e9640 1 -- 172.21.15.156:0/54568233 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc29c0fc360 msgr2=0x7fc29c10cf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:50:24.128 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:50:24.126+0000 7fc2a17e9640 1 --2- 172.21.15.156:0/54568233 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc29c0fc360 0x7fc29c10cf90 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fc28c003080 tx=0x7fc28c009da0 comp rx=0 tx=0).stop 2024-07-16T09:50:24.128 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:50:24.126+0000 7fc2a17e9640 1 -- 172.21.15.156:0/54568233 shutdown_connections 2024-07-16T09:50:24.128 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:50:24.126+0000 7fc2a17e9640 1 --2- 172.21.15.156:0/54568233 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7fc288067100 0x7fc2880695b0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:50:24.128 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:50:24.126+0000 7fc2a17e9640 1 --2- 172.21.15.156:0/54568233 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc29c0fd630 0x7fc29c10d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:50:24.128 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:50:24.126+0000 7fc2a17e9640 1 --2- 172.21.15.156:0/54568233 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc29c0fc360 0x7fc29c10cf90 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:50:24.128 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:50:24.126+0000 7fc2a17e9640 1 -- 172.21.15.156:0/54568233 >> 172.21.15.156:0/54568233 conn(0x7fc29c0f7b40 msgr2=0x7fc29c0f91a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:50:24.128 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:50:24.126+0000 7fc2a17e9640 1 -- 172.21.15.156:0/54568233 shutdown_connections 2024-07-16T09:50:24.129 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:50:24.126+0000 7fc2a17e9640 1 -- 172.21.15.156:0/54568233 wait complete. 2024-07-16T09:50:24.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:50:24 smithi156 ceph-mon[29302]: pgmap v63: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-07-16T09:50:24.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:50:24 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:50:24.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:50:24 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:50:24.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:50:24 smithi156 ceph-mon[29302]: from='osd.2 [v2:172.21.15.156:6818/3710668239,v1:172.21.15.156:6819/3710668239]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-07-16T09:50:24.721 DEBUG:teuthology.orchestra.run.smithi156:osd.2> sudo journalctl -f -n 0 -u ceph-26c91c26-4358-11ef-bcad-c7b262605968@osd.2.service 2024-07-16T09:50:24.724 INFO:tasks.cephadm:Deploying osd.3 on smithi176 with /dev/vg_nvme/lv_4... 2024-07-16T09:50:24.724 DEBUG:teuthology.orchestra.run.smithi176:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 26c91c26-4358-11ef-bcad-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-07-16T09:50:24.753 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:50:24 smithi176 ceph-mon[36626]: pgmap v63: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-07-16T09:50:24.753 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:50:24 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:50:24.753 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:50:24 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:50:24.753 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:50:24 smithi176 ceph-mon[36626]: from='osd.2 [v2:172.21.15.156:6818/3710668239,v1:172.21.15.156:6819/3710668239]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-07-16T09:50:24.988 INFO:teuthology.orchestra.run.smithi176.stderr:Inferring config /var/lib/ceph/26c91c26-4358-11ef-bcad-c7b262605968/mon.smithi176/config 2024-07-16T09:50:25.652 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:50:25 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-16T09:50:25.652 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:50:25 smithi156 ceph-mon[29302]: from='osd.2 [v2:172.21.15.156:6818/3710668239,v1:172.21.15.156:6819/3710668239]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-07-16T09:50:25.652 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:50:25 smithi156 ceph-mon[29302]: osdmap e17: 3 total, 2 up, 3 in 2024-07-16T09:50:25.652 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:50:25 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-07-16T09:50:25.652 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:50:25 smithi156 ceph-mon[29302]: from='osd.2 [v2:172.21.15.156:6818/3710668239,v1:172.21.15.156:6819/3710668239]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2024-07-16T09:50:25.748 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:50:25 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-16T09:50:25.748 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:50:25 smithi176 ceph-mon[36626]: from='osd.2 [v2:172.21.15.156:6818/3710668239,v1:172.21.15.156:6819/3710668239]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-07-16T09:50:25.748 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:50:25 smithi176 ceph-mon[36626]: osdmap e17: 3 total, 2 up, 3 in 2024-07-16T09:50:25.749 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:50:25 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-07-16T09:50:25.749 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:50:25 smithi176 ceph-mon[36626]: from='osd.2 [v2:172.21.15.156:6818/3710668239,v1:172.21.15.156:6819/3710668239]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2024-07-16T09:50:26.362 INFO:journalctl@ceph.osd.2.smithi156.stdout:Jul 16 09:50:26 smithi156 ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-2[62327]: 2024-07-16T09:50:26.131+0000 7fbb8be15640 -1 osd.2 0 waiting for initial osdmap 2024-07-16T09:50:26.362 INFO:journalctl@ceph.osd.2.smithi156.stdout:Jul 16 09:50:26 smithi156 ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-2[62327]: 2024-07-16T09:50:26.136+0000 7fbb8644b640 -1 osd.2 18 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-07-16T09:50:26.431 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:50:26 smithi176 ceph-mon[36626]: pgmap v64: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-07-16T09:50:26.431 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:50:26 smithi176 ceph-mon[36626]: from='osd.2 [v2:172.21.15.156:6818/3710668239,v1:172.21.15.156:6819/3710668239]' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]': finished 2024-07-16T09:50:26.431 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:50:26 smithi176 ceph-mon[36626]: osdmap e18: 3 total, 2 up, 3 in 2024-07-16T09:50:26.431 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:50:26 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-07-16T09:50:26.432 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:50:26 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-07-16T09:50:26.568 INFO:teuthology.orchestra.run.smithi176.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-07-16T09:50:26.660 INFO:teuthology.orchestra.run.smithi176.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-07-16T09:50:26.673 INFO:teuthology.orchestra.run.smithi176.stderr: stderr: 10+0 records in 2024-07-16T09:50:26.673 INFO:teuthology.orchestra.run.smithi176.stderr:10+0 records out 2024-07-16T09:50:26.673 INFO:teuthology.orchestra.run.smithi176.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0112656 s, 931 MB/s 2024-07-16T09:50:26.673 INFO:teuthology.orchestra.run.smithi176.stderr:--> Zapping successful for: 2024-07-16T09:50:26.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:50:26 smithi156 ceph-mon[29302]: pgmap v64: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-07-16T09:50:26.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:50:26 smithi156 ceph-mon[29302]: from='osd.2 [v2:172.21.15.156:6818/3710668239,v1:172.21.15.156:6819/3710668239]' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]': finished 2024-07-16T09:50:26.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:50:26 smithi156 ceph-mon[29302]: osdmap e18: 3 total, 2 up, 3 in 2024-07-16T09:50:26.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:50:26 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-07-16T09:50:26.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:50:26 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-07-16T09:50:27.336 DEBUG:teuthology.orchestra.run.smithi176:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 26c91c26-4358-11ef-bcad-c7b262605968 -- ceph orch daemon add osd smithi176:vg_nvme/lv_4 2024-07-16T09:50:27.641 INFO:teuthology.orchestra.run.smithi176.stderr:Inferring config /var/lib/ceph/26c91c26-4358-11ef-bcad-c7b262605968/mon.smithi176/config 2024-07-16T09:50:28.046 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:50:27 smithi176 ceph-mon[36626]: purged_snaps scrub starts 2024-07-16T09:50:28.046 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:50:27 smithi176 ceph-mon[36626]: purged_snaps scrub ok 2024-07-16T09:50:28.046 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:50:27 smithi176 ceph-mon[36626]: pgmap v67: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-07-16T09:50:28.046 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:50:27 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:50:28.047 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:50:27 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:50:28.047 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:50:27 smithi176 ceph-mon[36626]: osd.2 [v2:172.21.15.156:6818/3710668239,v1:172.21.15.156:6819/3710668239] boot 2024-07-16T09:50:28.047 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:50:27 smithi176 ceph-mon[36626]: osdmap e19: 3 total, 3 up, 3 in 2024-07-16T09:50:28.047 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:50:27 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-07-16T09:50:28.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:50:27 smithi156 ceph-mon[29302]: purged_snaps scrub starts 2024-07-16T09:50:28.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:50:27 smithi156 ceph-mon[29302]: purged_snaps scrub ok 2024-07-16T09:50:28.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:50:27 smithi156 ceph-mon[29302]: pgmap v67: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-07-16T09:50:28.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:50:27 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:50:28.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:50:27 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:50:28.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:50:27 smithi156 ceph-mon[29302]: osd.2 [v2:172.21.15.156:6818/3710668239,v1:172.21.15.156:6819/3710668239] boot 2024-07-16T09:50:28.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:50:27 smithi156 ceph-mon[29302]: osdmap e19: 3 total, 3 up, 3 in 2024-07-16T09:50:28.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:50:27 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-07-16T09:50:29.210 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:50:29.197+0000 7fb278695640 1 -- 172.21.15.176:0/2818286137 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb2700fc380 msgr2=0x7fb2700fc790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:50:29.210 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:50:29.197+0000 7fb278695640 1 --2- 172.21.15.176:0/2818286137 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb2700fc380 0x7fb2700fc790 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fb2640048d0 tx=0x7fb264009da0 comp rx=0 tx=0).stop 2024-07-16T09:50:29.211 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:50:29.197+0000 7fb278695640 1 -- 172.21.15.176:0/2818286137 shutdown_connections 2024-07-16T09:50:29.211 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:50:29.197+0000 7fb278695640 1 --2- 172.21.15.176:0/2818286137 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb2700fd650 0x7fb2700fdac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:50:29.211 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:50:29.197+0000 7fb278695640 1 --2- 172.21.15.176:0/2818286137 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb2700fc380 0x7fb2700fc790 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:50:29.211 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:50:29.197+0000 7fb278695640 1 -- 172.21.15.176:0/2818286137 >> 172.21.15.176:0/2818286137 conn(0x7fb2700f7b40 msgr2=0x7fb2700f9f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:50:29.211 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:50:29.197+0000 7fb278695640 1 -- 172.21.15.176:0/2818286137 shutdown_connections 2024-07-16T09:50:29.211 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:50:29.197+0000 7fb278695640 1 -- 172.21.15.176:0/2818286137 wait complete. 2024-07-16T09:50:29.211 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:50:29.197+0000 7fb278695640 1 Processor -- start 2024-07-16T09:50:29.211 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:50:29.197+0000 7fb278695640 1 -- start start 2024-07-16T09:50:29.211 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:50:29.198+0000 7fb278695640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb2700fc380 0x7fb27010cf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:50:29.211 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:50:29.198+0000 7fb278695640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb2700fd650 0x7fb27010d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:50:29.211 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:50:29.198+0000 7fb278695640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb27010db00 con 0x7fb2700fd650 2024-07-16T09:50:29.211 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:50:29.198+0000 7fb278695640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb27010e320 con 0x7fb2700fc380 2024-07-16T09:50:29.212 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:50:29.198+0000 7fb276431640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb2700fc380 0x7fb27010cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:50:29.212 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:50:29.198+0000 7fb276431640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb2700fc380 0x7fb27010cf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:57614/0 (socket says 172.21.15.176:57614) 2024-07-16T09:50:29.212 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:50:29.198+0000 7fb276431640 1 -- 172.21.15.176:0/818126417 learned_addr learned my addr 172.21.15.176:0/818126417 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-07-16T09:50:29.212 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:50:29.198+0000 7fb275c30640 1 --2- 172.21.15.176:0/818126417 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb2700fd650 0x7fb27010d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:50:29.212 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:50:29.198+0000 7fb276431640 1 -- 172.21.15.176:0/818126417 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb2700fd650 msgr2=0x7fb27010d4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:50:29.212 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:50:29.198+0000 7fb276431640 1 --2- 172.21.15.176:0/818126417 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb2700fd650 0x7fb27010d4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:50:29.212 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:50:29.198+0000 7fb276431640 1 -- 172.21.15.176:0/818126417 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb264005040 con 0x7fb2700fc380 2024-07-16T09:50:29.212 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:50:29.198+0000 7fb276431640 1 --2- 172.21.15.176:0/818126417 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb2700fc380 0x7fb27010cf70 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fb2640048a0 tx=0x7fb264009da0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:50:29.212 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:50:29.198+0000 7fb262ffd640 1 -- 172.21.15.176:0/818126417 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb264038070 con 0x7fb2700fc380 2024-07-16T09:50:29.212 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:50:29.198+0000 7fb262ffd640 1 -- 172.21.15.176:0/818126417 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb2640092b0 con 0x7fb2700fc380 2024-07-16T09:50:29.212 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:50:29.198+0000 7fb278695640 1 -- 172.21.15.176:0/818126417 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb27010e550 con 0x7fb2700fc380 2024-07-16T09:50:29.212 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:50:29.198+0000 7fb262ffd640 1 -- 172.21.15.176:0/818126417 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb264033780 con 0x7fb2700fc380 2024-07-16T09:50:29.213 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:50:29.198+0000 7fb278695640 1 -- 172.21.15.176:0/818126417 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb27010ea90 con 0x7fb2700fc380 2024-07-16T09:50:29.213 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:50:29.199+0000 7fb278695640 1 -- 172.21.15.176:0/818126417 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2700fb7e0 con 0x7fb2700fc380 2024-07-16T09:50:29.213 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:50:29.200+0000 7fb262ffd640 1 -- 172.21.15.176:0/818126417 <== mon.1 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 18) v1 ==== 85071+0+0 (secure 0 0 0) 0x7fb264030020 con 0x7fb2700fc380 2024-07-16T09:50:29.213 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:50:29.200+0000 7fb262ffd640 1 --2- 172.21.15.176:0/818126417 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7fb25c067100 0x7fb25c0695b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:50:29.213 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:50:29.200+0000 7fb262ffd640 1 -- 172.21.15.176:0/818126417 <== mon.1 v2:172.21.15.176:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 3217+0+0 (secure 0 0 0) 0x7fb2640a3f10 con 0x7fb2700fc380 2024-07-16T09:50:29.213 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:50:29.201+0000 7fb275c30640 1 --2- 172.21.15.176:0/818126417 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7fb25c067100 0x7fb25c0695b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:50:29.213 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:50:29.201+0000 7fb275c30640 1 --2- 172.21.15.176:0/818126417 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7fb25c067100 0x7fb25c0695b0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fb27010cd20 tx=0x7fb26c00b040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:50:29.213 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:50:29.205+0000 7fb262ffd640 1 -- 172.21.15.176:0/818126417 <== mon.1 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fb264071c00 con 0x7fb2700fc380 2024-07-16T09:50:29.343 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:50:29 smithi176 ceph-mon[36626]: osdmap e20: 3 total, 3 up, 3 in 2024-07-16T09:50:29.343 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:50:29 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]: dispatch 2024-07-16T09:50:29.377 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:50:29.376+0000 7fb278695640 1 -- 172.21.15.176:0/818126417 --> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi176:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7fb27010edc0 con 0x7fb25c067100 2024-07-16T09:50:29.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:50:29 smithi156 ceph-mon[29302]: osdmap e20: 3 total, 3 up, 3 in 2024-07-16T09:50:29.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:50:29 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]: dispatch 2024-07-16T09:50:30.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:50:30 smithi176 ceph-mon[36626]: pgmap v70: 0 pgs: ; 0 B data, 872 MiB used, 267 GiB / 268 GiB avail 2024-07-16T09:50:30.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:50:30 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]': finished 2024-07-16T09:50:30.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:50:30 smithi176 ceph-mon[36626]: osdmap e21: 3 total, 3 up, 3 in 2024-07-16T09:50:30.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:50:30 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-07-16T09:50:30.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:50:30 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-16T09:50:30.345 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:50:30 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-16T09:50:30.345 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:50:30 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-16T09:50:30.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:50:30 smithi156 ceph-mon[29302]: pgmap v70: 0 pgs: ; 0 B data, 872 MiB used, 267 GiB / 268 GiB avail 2024-07-16T09:50:30.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:50:30 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]': finished 2024-07-16T09:50:30.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:50:30 smithi156 ceph-mon[29302]: osdmap e21: 3 total, 3 up, 3 in 2024-07-16T09:50:30.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:50:30 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-07-16T09:50:30.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:50:30 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-16T09:50:30.426 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:50:30 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-16T09:50:30.426 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:50:30 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-16T09:50:30.977 INFO:journalctl@ceph.osd.0.smithi156.stdout:Jul 16 09:50:30 smithi156 sudo[67035]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-07-16T09:50:30.977 INFO:journalctl@ceph.osd.0.smithi156.stdout:Jul 16 09:50:30 smithi156 sudo[67035]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-07-16T09:50:30.978 INFO:journalctl@ceph.osd.0.smithi156.stdout:Jul 16 09:50:30 smithi156 sudo[67035]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-07-16T09:50:30.978 INFO:journalctl@ceph.osd.0.smithi156.stdout:Jul 16 09:50:30 smithi156 sudo[67035]: pam_unix(sudo:session): session closed for user root 2024-07-16T09:50:31.183 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:50:31.181+0000 7fb262ffd640 1 -- 172.21.15.176:0/818126417 <== mon.1 v2:172.21.15.176:3300/0 7 ==== mgrmap(e 19) v1 ==== 85127+0+0 (secure 0 0 0) 0x7fb264032b90 con 0x7fb2700fc380 2024-07-16T09:50:31.255 INFO:journalctl@ceph.osd.0.smithi156.stdout:Jul 16 09:50:30 smithi156 sudo[67046]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-07-16T09:50:31.255 INFO:journalctl@ceph.osd.0.smithi156.stdout:Jul 16 09:50:30 smithi156 sudo[67046]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-07-16T09:50:31.255 INFO:journalctl@ceph.osd.0.smithi156.stdout:Jul 16 09:50:30 smithi156 sudo[67046]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-07-16T09:50:31.255 INFO:journalctl@ceph.osd.0.smithi156.stdout:Jul 16 09:50:30 smithi156 sudo[67046]: pam_unix(sudo:session): session closed for user root 2024-07-16T09:50:31.255 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:50:31 smithi156 ceph-mon[29302]: from='client.24127 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi176:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-07-16T09:50:31.255 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:50:31 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-07-16T09:50:31.255 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:50:31 smithi156 ceph-mon[29302]: osdmap e22: 3 total, 3 up, 3 in 2024-07-16T09:50:31.255 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:50:31 smithi156 ceph-mon[29302]: Detected new or changed devices on smithi156 2024-07-16T09:50:31.255 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:50:31 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:50:31.256 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:50:31 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:50:31.256 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:50:31 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-07-16T09:50:31.256 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:50:31 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-07-16T09:50:31.256 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:50:31 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-07-16T09:50:31.256 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:50:31 smithi156 ceph-mon[29302]: Adjusting osd_memory_target on smithi156 to 4288M 2024-07-16T09:50:31.256 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:50:31 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:50:31.256 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:50:31 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-16T09:50:31.256 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:50:31 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-16T09:50:31.256 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:50:31 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:50:31.256 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:50:31 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-16T09:50:31.256 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:50:31 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:50:31.343 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:50:31 smithi176 ceph-mon[36626]: from='client.24127 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi176:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-07-16T09:50:31.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:50:31 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-07-16T09:50:31.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:50:31 smithi176 ceph-mon[36626]: osdmap e22: 3 total, 3 up, 3 in 2024-07-16T09:50:31.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:50:31 smithi176 ceph-mon[36626]: Detected new or changed devices on smithi156 2024-07-16T09:50:31.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:50:31 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:50:31.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:50:31 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:50:31.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:50:31 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-07-16T09:50:31.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:50:31 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-07-16T09:50:31.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:50:31 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-07-16T09:50:31.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:50:31 smithi176 ceph-mon[36626]: Adjusting osd_memory_target on smithi156 to 4288M 2024-07-16T09:50:31.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:50:31 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:50:31.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:50:31 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-16T09:50:31.345 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:50:31 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-16T09:50:31.345 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:50:31 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:50:31.345 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:50:31 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-16T09:50:31.345 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:50:31 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:50:31.532 INFO:journalctl@ceph.osd.1.smithi156.stdout:Jul 16 09:50:31 smithi156 sudo[67062]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-07-16T09:50:31.532 INFO:journalctl@ceph.osd.1.smithi156.stdout:Jul 16 09:50:31 smithi156 sudo[67062]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-07-16T09:50:31.532 INFO:journalctl@ceph.osd.1.smithi156.stdout:Jul 16 09:50:31 smithi156 sudo[67062]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-07-16T09:50:31.532 INFO:journalctl@ceph.osd.1.smithi156.stdout:Jul 16 09:50:31 smithi156 sudo[67062]: pam_unix(sudo:session): session closed for user root 2024-07-16T09:50:31.782 INFO:journalctl@ceph.osd.2.smithi156.stdout:Jul 16 09:50:31 smithi156 sudo[67170]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-07-16T09:50:31.782 INFO:journalctl@ceph.osd.1.smithi156.stdout:Jul 16 09:50:31 smithi156 sudo[67148]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-07-16T09:50:31.782 INFO:journalctl@ceph.osd.1.smithi156.stdout:Jul 16 09:50:31 smithi156 sudo[67148]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-07-16T09:50:31.782 INFO:journalctl@ceph.osd.1.smithi156.stdout:Jul 16 09:50:31 smithi156 sudo[67148]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-07-16T09:50:31.783 INFO:journalctl@ceph.osd.1.smithi156.stdout:Jul 16 09:50:31 smithi156 sudo[67148]: pam_unix(sudo:session): session closed for user root 2024-07-16T09:50:32.038 INFO:journalctl@ceph.osd.2.smithi156.stdout:Jul 16 09:50:31 smithi156 sudo[67170]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-07-16T09:50:32.038 INFO:journalctl@ceph.osd.2.smithi156.stdout:Jul 16 09:50:31 smithi156 sudo[67170]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-07-16T09:50:32.039 INFO:journalctl@ceph.osd.2.smithi156.stdout:Jul 16 09:50:31 smithi156 sudo[67170]: pam_unix(sudo:session): session closed for user root 2024-07-16T09:50:32.302 INFO:journalctl@ceph.osd.2.smithi156.stdout:Jul 16 09:50:32 smithi156 sudo[67177]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-07-16T09:50:32.303 INFO:journalctl@ceph.osd.2.smithi156.stdout:Jul 16 09:50:32 smithi156 sudo[67177]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-07-16T09:50:32.303 INFO:journalctl@ceph.osd.2.smithi156.stdout:Jul 16 09:50:32 smithi156 sudo[67177]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-07-16T09:50:32.303 INFO:journalctl@ceph.osd.2.smithi156.stdout:Jul 16 09:50:32 smithi156 sudo[67177]: pam_unix(sudo:session): session closed for user root 2024-07-16T09:50:32.562 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:50:32 smithi156 sudo[67224]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-07-16T09:50:32.562 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:50:32 smithi156 sudo[67224]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-07-16T09:50:32.563 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:50:32 smithi156 sudo[67224]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-07-16T09:50:32.563 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:50:32 smithi156 sudo[67224]: pam_unix(sudo:session): session closed for user root 2024-07-16T09:50:32.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:50:32 smithi156 sudo[67253]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-07-16T09:50:32.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:50:32 smithi156 sudo[67253]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-07-16T09:50:32.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:50:32 smithi156 sudo[67253]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-07-16T09:50:32.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:50:32 smithi156 sudo[67253]: pam_unix(sudo:session): session closed for user root 2024-07-16T09:50:32.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:50:32 smithi156 ceph-mon[29302]: pgmap v73: 1 pgs: 1 unknown; 0 B data, 872 MiB used, 267 GiB / 268 GiB avail 2024-07-16T09:50:32.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:50:32 smithi156 ceph-mon[29302]: osdmap e23: 3 total, 3 up, 3 in 2024-07-16T09:50:32.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:50:32 smithi156 ceph-mon[29302]: mgrmap e19: smithi156.zkvylv(active, since 2m), standbys: smithi176.dqbrvx 2024-07-16T09:50:32.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:50:32 smithi156 ceph-mon[29302]: from='client.? 172.21.15.176:0/683089063' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8428be96-686f-4318-9585-4b8a19a7b9c5"}]: dispatch 2024-07-16T09:50:32.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:50:32 smithi156 ceph-mon[29302]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8428be96-686f-4318-9585-4b8a19a7b9c5"}]: dispatch 2024-07-16T09:50:32.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:50:32 smithi156 ceph-mon[29302]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8428be96-686f-4318-9585-4b8a19a7b9c5"}]': finished 2024-07-16T09:50:32.926 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:50:32 smithi156 ceph-mon[29302]: osdmap e24: 4 total, 3 up, 4 in 2024-07-16T09:50:32.926 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:50:32 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-07-16T09:50:32.926 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:50:32 smithi156 ceph-mon[29302]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-07-16T09:50:33.085 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:50:32 smithi176 sudo[40112]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-07-16T09:50:33.085 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:50:32 smithi176 sudo[40112]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-07-16T09:50:33.085 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:50:32 smithi176 sudo[40112]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-07-16T09:50:33.085 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:50:32 smithi176 sudo[40112]: pam_unix(sudo:session): session closed for user root 2024-07-16T09:50:33.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:50:33 smithi176 sudo[40303]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-07-16T09:50:33.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:50:33 smithi176 sudo[40303]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-07-16T09:50:33.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:50:33 smithi176 sudo[40303]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-07-16T09:50:33.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:50:33 smithi176 sudo[40303]: pam_unix(sudo:session): session closed for user root 2024-07-16T09:50:33.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:50:33 smithi176 ceph-mon[36626]: pgmap v73: 1 pgs: 1 unknown; 0 B data, 872 MiB used, 267 GiB / 268 GiB avail 2024-07-16T09:50:33.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:50:33 smithi176 ceph-mon[36626]: osdmap e23: 3 total, 3 up, 3 in 2024-07-16T09:50:33.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:50:33 smithi176 ceph-mon[36626]: mgrmap e19: smithi156.zkvylv(active, since 2m), standbys: smithi176.dqbrvx 2024-07-16T09:50:33.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:50:33 smithi176 ceph-mon[36626]: from='client.? 172.21.15.176:0/683089063' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8428be96-686f-4318-9585-4b8a19a7b9c5"}]: dispatch 2024-07-16T09:50:33.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:50:33 smithi176 ceph-mon[36626]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8428be96-686f-4318-9585-4b8a19a7b9c5"}]: dispatch 2024-07-16T09:50:33.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:50:33 smithi176 ceph-mon[36626]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8428be96-686f-4318-9585-4b8a19a7b9c5"}]': finished 2024-07-16T09:50:33.345 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:50:33 smithi176 ceph-mon[36626]: osdmap e24: 4 total, 3 up, 4 in 2024-07-16T09:50:33.345 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:50:33 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-07-16T09:50:33.345 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:50:33 smithi176 ceph-mon[36626]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-07-16T09:50:33.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:50:33 smithi176 ceph-mon[36626]: from='client.? 172.21.15.176:0/1702563077' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-16T09:50:33.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:50:33 smithi176 ceph-mon[36626]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-07-16T09:50:33.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:50:33 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "mon metadata", "id": "smithi156"}]: dispatch 2024-07-16T09:50:33.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:50:33 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "mon metadata", "id": "smithi176"}]: dispatch 2024-07-16T09:50:33.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:50:33 smithi176 ceph-mon[36626]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-07-16T09:50:33.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:50:33 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "mon metadata", "id": "smithi156"}]: dispatch 2024-07-16T09:50:33.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:50:33 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "mon metadata", "id": "smithi176"}]: dispatch 2024-07-16T09:50:33.845 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:50:33 smithi176 ceph-mon[36626]: pgmap v76: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-07-16T09:50:33.845 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:50:33 smithi176 ceph-mon[36626]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-07-16T09:50:33.845 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:50:33 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:50:33.858 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:50:33 smithi156 ceph-mon[29302]: from='client.? 172.21.15.176:0/1702563077' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-16T09:50:33.858 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:50:33 smithi156 ceph-mon[29302]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-07-16T09:50:33.858 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:50:33 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "mon metadata", "id": "smithi156"}]: dispatch 2024-07-16T09:50:33.859 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:50:33 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "mon metadata", "id": "smithi176"}]: dispatch 2024-07-16T09:50:33.859 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:50:33 smithi156 ceph-mon[29302]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-07-16T09:50:33.859 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:50:33 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "mon metadata", "id": "smithi156"}]: dispatch 2024-07-16T09:50:33.859 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:50:33 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "mon metadata", "id": "smithi176"}]: dispatch 2024-07-16T09:50:33.859 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:50:33 smithi156 ceph-mon[29302]: pgmap v76: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-07-16T09:50:33.859 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:50:33 smithi156 ceph-mon[29302]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-07-16T09:50:33.859 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:50:33 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:50:35.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:50:35 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:50:35.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:50:35 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:50:35.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:50:35 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-16T09:50:35.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:50:35 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-16T09:50:35.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:50:35 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:50:35.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:50:35 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:50:35.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:50:35 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:50:35.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:50:35 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-16T09:50:35.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:50:35 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-16T09:50:35.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:50:35 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:50:36.539 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:50:36 smithi176 ceph-mon[36626]: pgmap v77: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-07-16T09:50:36.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:50:36 smithi156 ceph-mon[29302]: pgmap v77: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-07-16T09:50:38.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:50:38 smithi176 ceph-mon[36626]: pgmap v78: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-07-16T09:50:38.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:50:38 smithi156 ceph-mon[29302]: pgmap v78: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-07-16T09:50:39.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:50:39 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-07-16T09:50:39.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:50:39 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-16T09:50:39.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:50:39 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-07-16T09:50:39.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:50:39 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-16T09:50:40.305 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:50:40 smithi176 ceph-mon[36626]: Deploying daemon osd.3 on smithi176 2024-07-16T09:50:40.305 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:50:40 smithi176 ceph-mon[36626]: pgmap v79: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 3.0 KiB/s rd, 131 KiB/s wr, 9 op/s 2024-07-16T09:50:40.305 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:50:40 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-16T09:50:40.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:50:40 smithi156 ceph-mon[29302]: Deploying daemon osd.3 on smithi176 2024-07-16T09:50:40.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:50:40 smithi156 ceph-mon[29302]: pgmap v79: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 3.0 KiB/s rd, 131 KiB/s wr, 9 op/s 2024-07-16T09:50:40.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:50:40 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-16T09:50:42.538 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:50:42 smithi176 ceph-mon[36626]: pgmap v80: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 2.2 KiB/s rd, 98 KiB/s wr, 6 op/s 2024-07-16T09:50:42.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:50:42 smithi156 ceph-mon[29302]: pgmap v80: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 2.2 KiB/s rd, 98 KiB/s wr, 6 op/s 2024-07-16T09:50:43.567 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:50:43 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-16T09:50:43.567 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:50:43 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:50:43.567 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:50:43 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:50:43.576 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:50:43 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-16T09:50:43.576 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:50:43 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:50:43.576 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:50:43 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:50:44.571 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:50:44 smithi176 ceph-mon[36626]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 1.8 KiB/s rd, 78 KiB/s wr, 5 op/s 2024-07-16T09:50:44.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:50:44 smithi156 ceph-mon[29302]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 1.8 KiB/s rd, 78 KiB/s wr, 5 op/s 2024-07-16T09:50:46.142 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:50:46.140+0000 7fb262ffd640 1 -- 172.21.15.176:0/818126417 <== mgr.14208 v2:172.21.15.156:6800/2615741717 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fb27010edc0 con 0x7fb25c067100 2024-07-16T09:50:46.142 INFO:teuthology.orchestra.run.smithi176.stdout:Created osd(s) 3 on host 'smithi176' 2024-07-16T09:50:46.144 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:50:46.142+0000 7fb278695640 1 -- 172.21.15.176:0/818126417 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7fb25c067100 msgr2=0x7fb25c0695b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:50:46.144 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:50:46.142+0000 7fb278695640 1 --2- 172.21.15.176:0/818126417 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7fb25c067100 0x7fb25c0695b0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fb27010cd20 tx=0x7fb26c00b040 comp rx=0 tx=0).stop 2024-07-16T09:50:46.144 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:50:46.142+0000 7fb278695640 1 -- 172.21.15.176:0/818126417 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb2700fc380 msgr2=0x7fb27010cf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:50:46.144 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:50:46.142+0000 7fb278695640 1 --2- 172.21.15.176:0/818126417 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb2700fc380 0x7fb27010cf70 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fb2640048a0 tx=0x7fb264009da0 comp rx=0 tx=0).stop 2024-07-16T09:50:46.144 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:50:46.142+0000 7fb278695640 1 -- 172.21.15.176:0/818126417 shutdown_connections 2024-07-16T09:50:46.144 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:50:46.142+0000 7fb278695640 1 --2- 172.21.15.176:0/818126417 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7fb25c067100 0x7fb25c0695b0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:50:46.144 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:50:46.142+0000 7fb278695640 1 --2- 172.21.15.176:0/818126417 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb2700fd650 0x7fb27010d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:50:46.144 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:50:46.142+0000 7fb278695640 1 --2- 172.21.15.176:0/818126417 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb2700fc380 0x7fb27010cf70 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:50:46.144 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:50:46.142+0000 7fb278695640 1 -- 172.21.15.176:0/818126417 >> 172.21.15.176:0/818126417 conn(0x7fb2700f7b40 msgr2=0x7fb2700f9190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:50:46.145 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:50:46.143+0000 7fb278695640 1 -- 172.21.15.176:0/818126417 shutdown_connections 2024-07-16T09:50:46.145 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:50:46.143+0000 7fb278695640 1 -- 172.21.15.176:0/818126417 wait complete. 2024-07-16T09:50:46.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:50:46 smithi176 ceph-mon[36626]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2024-07-16T09:50:46.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:50:46 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:50:46.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:50:46 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:50:46.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:50:46 smithi156 ceph-mon[29302]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2024-07-16T09:50:46.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:50:46 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:50:46.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:50:46 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:50:47.379 DEBUG:teuthology.orchestra.run.smithi176:osd.3> sudo journalctl -f -n 0 -u ceph-26c91c26-4358-11ef-bcad-c7b262605968@osd.3.service 2024-07-16T09:50:47.382 INFO:tasks.cephadm:Deploying osd.4 on smithi176 with /dev/vg_nvme/lv_3... 2024-07-16T09:50:47.382 DEBUG:teuthology.orchestra.run.smithi176:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 26c91c26-4358-11ef-bcad-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-07-16T09:50:47.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:50:47 smithi176 ceph-mon[36626]: from='osd.3 [v2:172.21.15.176:6800/1740426750,v1:172.21.15.176:6801/1740426750]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-07-16T09:50:47.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:50:47 smithi176 ceph-mon[36626]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-07-16T09:50:47.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:50:47 smithi156 ceph-mon[29302]: from='osd.3 [v2:172.21.15.176:6800/1740426750,v1:172.21.15.176:6801/1740426750]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-07-16T09:50:47.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:50:47 smithi156 ceph-mon[29302]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-07-16T09:50:47.688 INFO:teuthology.orchestra.run.smithi176.stderr:Inferring config /var/lib/ceph/26c91c26-4358-11ef-bcad-c7b262605968/mon.smithi176/config 2024-07-16T09:50:48.469 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:50:48 smithi176 ceph-mon[36626]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2024-07-16T09:50:48.469 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:50:48 smithi176 ceph-mon[36626]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-07-16T09:50:48.470 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:50:48 smithi176 ceph-mon[36626]: osdmap e25: 4 total, 3 up, 4 in 2024-07-16T09:50:48.470 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:50:48 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-07-16T09:50:48.470 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:50:48 smithi176 ceph-mon[36626]: from='osd.3 [v2:172.21.15.176:6800/1740426750,v1:172.21.15.176:6801/1740426750]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi176", "root=default"]}]: dispatch 2024-07-16T09:50:48.470 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:50:48 smithi176 ceph-mon[36626]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi176", "root=default"]}]: dispatch 2024-07-16T09:50:48.470 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:50:48 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:50:48.470 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:50:48 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:50:48.470 INFO:journalctl@ceph.osd.3.smithi176.stdout:Jul 16 09:50:48 smithi176 ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-3[42094]: 2024-07-16T09:50:48.327+0000 7fc83907d640 -1 osd.3 0 waiting for initial osdmap 2024-07-16T09:50:48.470 INFO:journalctl@ceph.osd.3.smithi176.stdout:Jul 16 09:50:48 smithi176 ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-3[42094]: 2024-07-16T09:50:48.333+0000 7fc83068b640 -1 osd.3 26 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-07-16T09:50:48.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:50:48 smithi156 ceph-mon[29302]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2024-07-16T09:50:48.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:50:48 smithi156 ceph-mon[29302]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-07-16T09:50:48.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:50:48 smithi156 ceph-mon[29302]: osdmap e25: 4 total, 3 up, 4 in 2024-07-16T09:50:48.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:50:48 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-07-16T09:50:48.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:50:48 smithi156 ceph-mon[29302]: from='osd.3 [v2:172.21.15.176:6800/1740426750,v1:172.21.15.176:6801/1740426750]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi176", "root=default"]}]: dispatch 2024-07-16T09:50:48.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:50:48 smithi156 ceph-mon[29302]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi176", "root=default"]}]: dispatch 2024-07-16T09:50:48.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:50:48 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:50:48.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:50:48 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:50:49.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:50:49 smithi176 ceph-mon[36626]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi176", "root=default"]}]': finished 2024-07-16T09:50:49.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:50:49 smithi176 ceph-mon[36626]: osdmap e26: 4 total, 3 up, 4 in 2024-07-16T09:50:49.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:50:49 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-07-16T09:50:49.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:50:49 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-07-16T09:50:49.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:50:49 smithi176 ceph-mon[36626]: osd.3 [v2:172.21.15.176:6800/1740426750,v1:172.21.15.176:6801/1740426750] boot 2024-07-16T09:50:49.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:50:49 smithi176 ceph-mon[36626]: osdmap e27: 4 total, 4 up, 4 in 2024-07-16T09:50:49.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:50:49 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-07-16T09:50:49.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:50:49 smithi156 ceph-mon[29302]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi176", "root=default"]}]': finished 2024-07-16T09:50:49.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:50:49 smithi156 ceph-mon[29302]: osdmap e26: 4 total, 3 up, 4 in 2024-07-16T09:50:49.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:50:49 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-07-16T09:50:49.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:50:49 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-07-16T09:50:49.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:50:49 smithi156 ceph-mon[29302]: osd.3 [v2:172.21.15.176:6800/1740426750,v1:172.21.15.176:6801/1740426750] boot 2024-07-16T09:50:49.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:50:49 smithi156 ceph-mon[29302]: osdmap e27: 4 total, 4 up, 4 in 2024-07-16T09:50:49.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:50:49 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-07-16T09:50:50.473 INFO:teuthology.orchestra.run.smithi176.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-07-16T09:50:50.574 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:50:50 smithi176 ceph-mon[36626]: purged_snaps scrub starts 2024-07-16T09:50:50.574 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:50:50 smithi176 ceph-mon[36626]: purged_snaps scrub ok 2024-07-16T09:50:50.574 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:50:50 smithi176 ceph-mon[36626]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-07-16T09:50:50.574 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:50:50 smithi176 ceph-mon[36626]: osdmap e28: 4 total, 4 up, 4 in 2024-07-16T09:50:50.590 INFO:teuthology.orchestra.run.smithi176.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-07-16T09:50:50.605 INFO:teuthology.orchestra.run.smithi176.stderr: stderr: 10+0 records in 2024-07-16T09:50:50.606 INFO:teuthology.orchestra.run.smithi176.stderr:10+0 records out 2024-07-16T09:50:50.606 INFO:teuthology.orchestra.run.smithi176.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0133203 s, 787 MB/s 2024-07-16T09:50:50.606 INFO:teuthology.orchestra.run.smithi176.stderr: stderr: 2024-07-16T09:50:50.606 INFO:teuthology.orchestra.run.smithi176.stderr:--> Zapping successful for: 2024-07-16T09:50:50.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:50:50 smithi156 ceph-mon[29302]: purged_snaps scrub starts 2024-07-16T09:50:50.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:50:50 smithi156 ceph-mon[29302]: purged_snaps scrub ok 2024-07-16T09:50:50.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:50:50 smithi156 ceph-mon[29302]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-07-16T09:50:50.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:50:50 smithi156 ceph-mon[29302]: osdmap e28: 4 total, 4 up, 4 in 2024-07-16T09:50:51.303 DEBUG:teuthology.orchestra.run.smithi176:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 26c91c26-4358-11ef-bcad-c7b262605968 -- ceph orch daemon add osd smithi176:vg_nvme/lv_3 2024-07-16T09:50:51.885 INFO:teuthology.orchestra.run.smithi176.stderr:Inferring config /var/lib/ceph/26c91c26-4358-11ef-bcad-c7b262605968/mon.smithi176/config 2024-07-16T09:50:52.093 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:50:52 smithi176 ceph-mon[36626]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-07-16T09:50:52.093 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:50:52 smithi176 ceph-mon[36626]: osdmap e29: 4 total, 4 up, 4 in 2024-07-16T09:50:52.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:50:52 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:50:52.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:50:52 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:50:52.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:50:52 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-07-16T09:50:52.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:50:52 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:50:52.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:50:52 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-16T09:50:52.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:50:52 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-16T09:50:52.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:50:52 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:50:52.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:50:52 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-16T09:50:52.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:50:52 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-16T09:50:52.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:50:52 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-16T09:50:52.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:50:52 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:50:52.424 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:50:52 smithi156 ceph-mon[29302]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-07-16T09:50:52.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:50:52 smithi156 ceph-mon[29302]: osdmap e29: 4 total, 4 up, 4 in 2024-07-16T09:50:52.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:50:52 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:50:52.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:50:52 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:50:52.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:50:52 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-07-16T09:50:52.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:50:52 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:50:52.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:50:52 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-16T09:50:52.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:50:52 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-16T09:50:52.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:50:52 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:50:52.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:50:52 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-16T09:50:52.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:50:52 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-16T09:50:52.426 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:50:52 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-16T09:50:52.426 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:50:52 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:50:53.093 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:50:53 smithi176 ceph-mon[36626]: Detected new or changed devices on smithi176 2024-07-16T09:50:53.093 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:50:53 smithi176 ceph-mon[36626]: Adjusting osd_memory_target on smithi176 to 15936M 2024-07-16T09:50:53.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:50:53 smithi176 ceph-mon[36626]: osdmap e30: 4 total, 4 up, 4 in 2024-07-16T09:50:53.424 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:50:53 smithi156 ceph-mon[29302]: Detected new or changed devices on smithi176 2024-07-16T09:50:53.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:50:53 smithi156 ceph-mon[29302]: Adjusting osd_memory_target on smithi176 to 15936M 2024-07-16T09:50:53.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:50:53 smithi156 ceph-mon[29302]: osdmap e30: 4 total, 4 up, 4 in 2024-07-16T09:50:53.430 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:50:53.427+0000 7f7d75472640 1 -- 172.21.15.176:0/1758471827 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7d700fd610 msgr2=0x7f7d700fda80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:50:53.430 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:50:53.427+0000 7f7d75472640 1 --2- 172.21.15.176:0/1758471827 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7d700fd610 0x7f7d700fda80 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f7d580048d0 tx=0x7f7d58009da0 comp rx=0 tx=0).stop 2024-07-16T09:50:53.430 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:50:53.427+0000 7f7d75472640 1 -- 172.21.15.176:0/1758471827 shutdown_connections 2024-07-16T09:50:53.430 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:50:53.427+0000 7f7d75472640 1 --2- 172.21.15.176:0/1758471827 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7d700fd610 0x7f7d700fda80 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:50:53.430 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:50:53.427+0000 7f7d75472640 1 --2- 172.21.15.176:0/1758471827 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7d700fc340 0x7f7d700fc750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:50:53.430 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:50:53.427+0000 7f7d75472640 1 -- 172.21.15.176:0/1758471827 >> 172.21.15.176:0/1758471827 conn(0x7f7d700f7b20 msgr2=0x7f7d700f9f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:50:53.430 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:50:53.427+0000 7f7d75472640 1 -- 172.21.15.176:0/1758471827 shutdown_connections 2024-07-16T09:50:53.431 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:50:53.427+0000 7f7d75472640 1 -- 172.21.15.176:0/1758471827 wait complete. 2024-07-16T09:50:53.431 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:50:53.428+0000 7f7d75472640 1 Processor -- start 2024-07-16T09:50:53.431 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:50:53.428+0000 7f7d75472640 1 -- start start 2024-07-16T09:50:53.431 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:50:53.428+0000 7f7d75472640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7d700fc340 0x7f7d7010cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:50:53.431 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:50:53.428+0000 7f7d75472640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7d700fd610 0x7f7d7010d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:50:53.431 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:50:53.428+0000 7f7d75472640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d7010e360 con 0x7f7d700fd610 2024-07-16T09:50:53.431 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:50:53.428+0000 7f7d75472640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d7010e4d0 con 0x7f7d700fc340 2024-07-16T09:50:53.431 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:50:53.428+0000 7f7d6effd640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7d700fc340 0x7f7d7010cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:50:53.431 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:50:53.428+0000 7f7d6effd640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7d700fc340 0x7f7d7010cfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:36144/0 (socket says 172.21.15.176:36144) 2024-07-16T09:50:53.431 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:50:53.428+0000 7f7d6effd640 1 -- 172.21.15.176:0/1359780463 learned_addr learned my addr 172.21.15.176:0/1359780463 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-07-16T09:50:53.431 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:50:53.428+0000 7f7d6effd640 1 -- 172.21.15.176:0/1359780463 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7d700fd610 msgr2=0x7f7d7010d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:50:53.431 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:50:53.428+0000 7f7d6effd640 1 --2- 172.21.15.176:0/1359780463 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7d700fd610 0x7f7d7010d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:50:53.432 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:50:53.428+0000 7f7d6effd640 1 -- 172.21.15.176:0/1359780463 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d58005040 con 0x7f7d700fc340 2024-07-16T09:50:53.432 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:50:53.428+0000 7f7d6effd640 1 --2- 172.21.15.176:0/1359780463 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7d700fc340 0x7f7d7010cfb0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f7d6000b790 tx=0x7f7d600091d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:50:53.432 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:50:53.429+0000 7f7d677fe640 1 -- 172.21.15.176:0/1359780463 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7d6000c700 con 0x7f7d700fc340 2024-07-16T09:50:53.432 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:50:53.429+0000 7f7d677fe640 1 -- 172.21.15.176:0/1359780463 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7d6000cd30 con 0x7f7d700fc340 2024-07-16T09:50:53.432 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:50:53.429+0000 7f7d677fe640 1 -- 172.21.15.176:0/1359780463 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7d60013370 con 0x7f7d700fc340 2024-07-16T09:50:53.432 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:50:53.429+0000 7f7d75472640 1 -- 172.21.15.176:0/1359780463 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d7010e7b0 con 0x7f7d700fc340 2024-07-16T09:50:53.432 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:50:53.429+0000 7f7d75472640 1 -- 172.21.15.176:0/1359780463 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d7010ecd0 con 0x7f7d700fc340 2024-07-16T09:50:53.433 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:50:53.430+0000 7f7d75472640 1 -- 172.21.15.176:0/1359780463 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d700fb7a0 con 0x7f7d700fc340 2024-07-16T09:50:53.433 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:50:53.430+0000 7f7d677fe640 1 -- 172.21.15.176:0/1359780463 <== mon.1 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 19) v1 ==== 85127+0+0 (secure 0 0 0) 0x7f7d60012070 con 0x7f7d700fc340 2024-07-16T09:50:53.433 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:50:53.431+0000 7f7d677fe640 1 --2- 172.21.15.176:0/1359780463 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f7d5c06b5a0 0x7f7d5c06da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:50:53.433 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:50:53.431+0000 7f7d677fe640 1 -- 172.21.15.176:0/1359780463 <== mon.1 v2:172.21.15.176:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 3770+0+0 (secure 0 0 0) 0x7f7d60084070 con 0x7f7d700fc340 2024-07-16T09:50:53.433 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:50:53.431+0000 7f7d67fff640 1 --2- 172.21.15.176:0/1359780463 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f7d5c06b5a0 0x7f7d5c06da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:50:53.433 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:50:53.431+0000 7f7d67fff640 1 --2- 172.21.15.176:0/1359780463 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f7d5c06b5a0 0x7f7d5c06da50 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f7d58009f80 tx=0x7f7d58009da0 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:50:53.436 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:50:53.435+0000 7f7d677fe640 1 -- 172.21.15.176:0/1359780463 <== mon.1 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f7d600519b0 con 0x7f7d700fc340 2024-07-16T09:50:53.608 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:50:53.607+0000 7f7d75472640 1 -- 172.21.15.176:0/1359780463 --> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi176:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f7d7010ef50 con 0x7f7d5c06b5a0 2024-07-16T09:50:54.326 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:50:54 smithi176 ceph-mon[36626]: pgmap v92: 1 pgs: 1 peering; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-07-16T09:50:54.326 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:50:54 smithi176 ceph-mon[36626]: osdmap e31: 4 total, 4 up, 4 in 2024-07-16T09:50:54.326 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:50:54 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-16T09:50:54.327 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:50:54 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-16T09:50:54.327 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:50:54 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-16T09:50:54.424 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:50:54 smithi156 ceph-mon[29302]: pgmap v92: 1 pgs: 1 peering; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-07-16T09:50:54.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:50:54 smithi156 ceph-mon[29302]: osdmap e31: 4 total, 4 up, 4 in 2024-07-16T09:50:54.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:50:54 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-16T09:50:54.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:50:54 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-16T09:50:54.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:50:54 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-16T09:50:55.175 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:50:55 smithi156 ceph-mon[29302]: from='client.24151 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi176:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-07-16T09:50:55.175 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:50:55 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-16T09:50:55.254 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:50:55 smithi176 ceph-mon[36626]: from='client.24151 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi176:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-07-16T09:50:55.254 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:50:55 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-16T09:50:56.343 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:50:56 smithi176 ceph-mon[36626]: pgmap v94: 1 pgs: 1 peering; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 122 KiB/s, 0 objects/s recovering 2024-07-16T09:50:56.343 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:50:56 smithi176 ceph-mon[36626]: from='client.? 172.21.15.176:0/4292014049' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fd311fe4-8853-4403-a3a8-863eb22a58ac"}]: dispatch 2024-07-16T09:50:56.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:50:56 smithi176 ceph-mon[36626]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fd311fe4-8853-4403-a3a8-863eb22a58ac"}]: dispatch 2024-07-16T09:50:56.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:50:56 smithi176 ceph-mon[36626]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fd311fe4-8853-4403-a3a8-863eb22a58ac"}]': finished 2024-07-16T09:50:56.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:50:56 smithi176 ceph-mon[36626]: osdmap e32: 5 total, 4 up, 5 in 2024-07-16T09:50:56.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:50:56 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-07-16T09:50:56.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:50:56 smithi156 ceph-mon[29302]: pgmap v94: 1 pgs: 1 peering; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 122 KiB/s, 0 objects/s recovering 2024-07-16T09:50:56.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:50:56 smithi156 ceph-mon[29302]: from='client.? 172.21.15.176:0/4292014049' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fd311fe4-8853-4403-a3a8-863eb22a58ac"}]: dispatch 2024-07-16T09:50:56.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:50:56 smithi156 ceph-mon[29302]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fd311fe4-8853-4403-a3a8-863eb22a58ac"}]: dispatch 2024-07-16T09:50:56.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:50:56 smithi156 ceph-mon[29302]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fd311fe4-8853-4403-a3a8-863eb22a58ac"}]': finished 2024-07-16T09:50:56.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:50:56 smithi156 ceph-mon[29302]: osdmap e32: 5 total, 4 up, 5 in 2024-07-16T09:50:56.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:50:56 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-07-16T09:50:57.343 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:50:57 smithi176 ceph-mon[36626]: from='client.? 172.21.15.176:0/1371004043' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-16T09:50:57.424 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:50:57 smithi156 ceph-mon[29302]: from='client.? 172.21.15.176:0/1371004043' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-16T09:50:58.343 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:50:58 smithi176 ceph-mon[36626]: pgmap v96: 1 pgs: 1 peering; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 101 KiB/s, 0 objects/s recovering 2024-07-16T09:50:58.424 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:50:58 smithi156 ceph-mon[29302]: pgmap v96: 1 pgs: 1 peering; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 101 KiB/s, 0 objects/s recovering 2024-07-16T09:51:00.338 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:51:00 smithi176 ceph-mon[36626]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 86 KiB/s, 0 objects/s recovering 2024-07-16T09:51:00.424 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:51:00 smithi156 ceph-mon[29302]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 86 KiB/s, 0 objects/s recovering 2024-07-16T09:51:02.321 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:51:02 smithi176 ceph-mon[36626]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-07-16T09:51:02.424 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:51:02 smithi156 ceph-mon[29302]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-07-16T09:51:03.343 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:51:03 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-07-16T09:51:03.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:51:03 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-16T09:51:03.424 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:51:03 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-07-16T09:51:03.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:51:03 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-16T09:51:04.343 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:51:04 smithi176 ceph-mon[36626]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-07-16T09:51:04.343 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:51:04 smithi176 ceph-mon[36626]: Deploying daemon osd.4 on smithi176 2024-07-16T09:51:04.424 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:51:04 smithi156 ceph-mon[29302]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-07-16T09:51:04.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:51:04 smithi156 ceph-mon[29302]: Deploying daemon osd.4 on smithi176 2024-07-16T09:51:06.321 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:51:06 smithi176 ceph-mon[36626]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-07-16T09:51:06.424 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:51:06 smithi156 ceph-mon[29302]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-07-16T09:51:08.141 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:51:08 smithi176 ceph-mon[36626]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-07-16T09:51:08.141 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:51:08 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-16T09:51:08.142 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:51:08 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:51:08.142 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:51:08 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:51:08.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:51:08 smithi156 ceph-mon[29302]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-07-16T09:51:08.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:51:08 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-16T09:51:08.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:51:08 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:51:08.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:51:08 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:51:10.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:51:10 smithi176 ceph-mon[36626]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-07-16T09:51:10.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:51:10 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-16T09:51:10.424 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:51:10 smithi156 ceph-mon[29302]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-07-16T09:51:10.424 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:51:10 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-16T09:51:11.496 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:51:11.494+0000 7f7d677fe640 1 -- 172.21.15.176:0/1359780463 <== mgr.14208 v2:172.21.15.156:6800/2615741717 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f7d7010ef50 con 0x7f7d5c06b5a0 2024-07-16T09:51:11.496 INFO:teuthology.orchestra.run.smithi176.stdout:Created osd(s) 4 on host 'smithi176' 2024-07-16T09:51:11.498 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:51:11.496+0000 7f7d75472640 1 -- 172.21.15.176:0/1359780463 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f7d5c06b5a0 msgr2=0x7f7d5c06da50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:51:11.498 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:51:11.496+0000 7f7d75472640 1 --2- 172.21.15.176:0/1359780463 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f7d5c06b5a0 0x7f7d5c06da50 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f7d58009f80 tx=0x7f7d58009da0 comp rx=0 tx=0).stop 2024-07-16T09:51:11.498 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:51:11.497+0000 7f7d75472640 1 -- 172.21.15.176:0/1359780463 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7d700fc340 msgr2=0x7f7d7010cfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:51:11.499 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:51:11.497+0000 7f7d75472640 1 --2- 172.21.15.176:0/1359780463 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7d700fc340 0x7f7d7010cfb0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f7d6000b790 tx=0x7f7d600091d0 comp rx=0 tx=0).stop 2024-07-16T09:51:11.499 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:51:11.497+0000 7f7d75472640 1 -- 172.21.15.176:0/1359780463 shutdown_connections 2024-07-16T09:51:11.499 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:51:11.497+0000 7f7d75472640 1 --2- 172.21.15.176:0/1359780463 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f7d5c06b5a0 0x7f7d5c06da50 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:51:11.499 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:51:11.497+0000 7f7d75472640 1 --2- 172.21.15.176:0/1359780463 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7d700fd610 0x7f7d7010d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:51:11.499 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:51:11.497+0000 7f7d75472640 1 --2- 172.21.15.176:0/1359780463 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7d700fc340 0x7f7d7010cfb0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:51:11.499 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:51:11.497+0000 7f7d75472640 1 -- 172.21.15.176:0/1359780463 >> 172.21.15.176:0/1359780463 conn(0x7f7d700f7b20 msgr2=0x7f7d700f9180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:51:11.499 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:51:11.497+0000 7f7d75472640 1 -- 172.21.15.176:0/1359780463 shutdown_connections 2024-07-16T09:51:11.499 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:51:11.497+0000 7f7d75472640 1 -- 172.21.15.176:0/1359780463 wait complete. 2024-07-16T09:51:12.057 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:51:11 smithi176 ceph-mon[36626]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-07-16T09:51:12.057 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:51:11 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:51:12.057 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:51:11 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:51:12.057 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:51:11 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-16T09:51:12.057 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:51:11 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-16T09:51:12.057 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:51:11 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:51:12.058 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:51:11 smithi176 ceph-mon[36626]: from='osd.4 [v2:172.21.15.176:6808/2702155727,v1:172.21.15.176:6809/2702155727]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-07-16T09:51:12.058 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:51:11 smithi176 ceph-mon[36626]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-07-16T09:51:12.058 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:51:11 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-16T09:51:12.058 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:51:11 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:51:12.058 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:51:11 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:51:12.080 DEBUG:teuthology.orchestra.run.smithi176:osd.4> sudo journalctl -f -n 0 -u ceph-26c91c26-4358-11ef-bcad-c7b262605968@osd.4.service 2024-07-16T09:51:12.082 INFO:tasks.cephadm:Deploying osd.5 on smithi176 with /dev/vg_nvme/lv_2... 2024-07-16T09:51:12.082 DEBUG:teuthology.orchestra.run.smithi176:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 26c91c26-4358-11ef-bcad-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-07-16T09:51:12.174 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:51:11 smithi156 ceph-mon[29302]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-07-16T09:51:12.175 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:51:11 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:51:12.175 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:51:11 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:51:12.175 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:51:11 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-16T09:51:12.175 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:51:11 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-16T09:51:12.175 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:51:11 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:51:12.175 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:51:11 smithi156 ceph-mon[29302]: from='osd.4 [v2:172.21.15.176:6808/2702155727,v1:172.21.15.176:6809/2702155727]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-07-16T09:51:12.175 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:51:11 smithi156 ceph-mon[29302]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-07-16T09:51:12.175 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:51:11 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-16T09:51:12.176 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:51:11 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:51:12.176 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:51:11 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:51:13.131 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:51:13 smithi176 ceph-mon[36626]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-07-16T09:51:13.131 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:51:13 smithi176 ceph-mon[36626]: osdmap e33: 5 total, 4 up, 5 in 2024-07-16T09:51:13.131 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:51:13 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-07-16T09:51:13.131 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:51:13 smithi176 ceph-mon[36626]: from='osd.4 [v2:172.21.15.176:6808/2702155727,v1:172.21.15.176:6809/2702155727]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi176", "root=default"]}]: dispatch 2024-07-16T09:51:13.132 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:51:13 smithi176 ceph-mon[36626]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi176", "root=default"]}]: dispatch 2024-07-16T09:51:13.132 INFO:journalctl@ceph.osd.4.smithi176.stdout:Jul 16 09:51:13 smithi176 ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-4[48461]: 2024-07-16T09:51:13.085+0000 7fb82d58c640 -1 osd.4 0 waiting for initial osdmap 2024-07-16T09:51:13.132 INFO:journalctl@ceph.osd.4.smithi176.stdout:Jul 16 09:51:13 smithi176 ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-4[48461]: 2024-07-16T09:51:13.092+0000 7fb826bc0640 -1 osd.4 34 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-07-16T09:51:13.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:51:13 smithi156 ceph-mon[29302]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-07-16T09:51:13.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:51:13 smithi156 ceph-mon[29302]: osdmap e33: 5 total, 4 up, 5 in 2024-07-16T09:51:13.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:51:13 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-07-16T09:51:13.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:51:13 smithi156 ceph-mon[29302]: from='osd.4 [v2:172.21.15.176:6808/2702155727,v1:172.21.15.176:6809/2702155727]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi176", "root=default"]}]: dispatch 2024-07-16T09:51:13.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:51:13 smithi156 ceph-mon[29302]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi176", "root=default"]}]: dispatch 2024-07-16T09:51:13.639 INFO:teuthology.orchestra.run.smithi176.stderr:Inferring config /var/lib/ceph/26c91c26-4358-11ef-bcad-c7b262605968/mon.smithi176/config 2024-07-16T09:51:14.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:51:14 smithi176 ceph-mon[36626]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-07-16T09:51:14.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:51:14 smithi176 ceph-mon[36626]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi176", "root=default"]}]': finished 2024-07-16T09:51:14.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:51:14 smithi176 ceph-mon[36626]: osdmap e34: 5 total, 4 up, 5 in 2024-07-16T09:51:14.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:51:14 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-07-16T09:51:14.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:51:14 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-07-16T09:51:14.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:51:14 smithi156 ceph-mon[29302]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-07-16T09:51:14.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:51:14 smithi156 ceph-mon[29302]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi176", "root=default"]}]': finished 2024-07-16T09:51:14.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:51:14 smithi156 ceph-mon[29302]: osdmap e34: 5 total, 4 up, 5 in 2024-07-16T09:51:14.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:51:14 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-07-16T09:51:14.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:51:14 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-07-16T09:51:15.249 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:51:15 smithi176 ceph-mon[36626]: purged_snaps scrub starts 2024-07-16T09:51:15.249 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:51:15 smithi176 ceph-mon[36626]: purged_snaps scrub ok 2024-07-16T09:51:15.249 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:51:15 smithi176 ceph-mon[36626]: osd.4 [v2:172.21.15.176:6808/2702155727,v1:172.21.15.176:6809/2702155727] boot 2024-07-16T09:51:15.249 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:51:15 smithi176 ceph-mon[36626]: osdmap e35: 5 total, 5 up, 5 in 2024-07-16T09:51:15.250 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:51:15 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-07-16T09:51:15.424 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:51:15 smithi156 ceph-mon[29302]: purged_snaps scrub starts 2024-07-16T09:51:15.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:51:15 smithi156 ceph-mon[29302]: purged_snaps scrub ok 2024-07-16T09:51:15.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:51:15 smithi156 ceph-mon[29302]: osd.4 [v2:172.21.15.176:6808/2702155727,v1:172.21.15.176:6809/2702155727] boot 2024-07-16T09:51:15.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:51:15 smithi156 ceph-mon[29302]: osdmap e35: 5 total, 5 up, 5 in 2024-07-16T09:51:15.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:51:15 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-07-16T09:51:15.811 INFO:teuthology.orchestra.run.smithi176.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-07-16T09:51:15.898 INFO:teuthology.orchestra.run.smithi176.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-07-16T09:51:15.913 INFO:teuthology.orchestra.run.smithi176.stderr: stderr: 10+0 records in 2024-07-16T09:51:15.913 INFO:teuthology.orchestra.run.smithi176.stderr:10+0 records out 2024-07-16T09:51:15.913 INFO:teuthology.orchestra.run.smithi176.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0120824 s, 868 MB/s 2024-07-16T09:51:15.913 INFO:teuthology.orchestra.run.smithi176.stderr:--> Zapping successful for: 2024-07-16T09:51:16.222 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:51:16 smithi176 ceph-mon[36626]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-07-16T09:51:16.222 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:51:16 smithi176 ceph-mon[36626]: osdmap e36: 5 total, 5 up, 5 in 2024-07-16T09:51:16.222 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:51:16 smithi176 ceph-mon[36626]: Detected new or changed devices on smithi176 2024-07-16T09:51:16.222 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:51:16 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:51:16.222 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:51:16 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:51:16.222 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:51:16 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-07-16T09:51:16.222 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:51:16 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-07-16T09:51:16.222 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:51:16 smithi176 ceph-mon[36626]: Adjusting osd_memory_target on smithi176 to 7968M 2024-07-16T09:51:16.223 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:51:16 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:51:16.223 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:51:16 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-16T09:51:16.223 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:51:16 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-16T09:51:16.223 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:51:16 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:51:16.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:51:16 smithi156 ceph-mon[29302]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-07-16T09:51:16.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:51:16 smithi156 ceph-mon[29302]: osdmap e36: 5 total, 5 up, 5 in 2024-07-16T09:51:16.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:51:16 smithi156 ceph-mon[29302]: Detected new or changed devices on smithi176 2024-07-16T09:51:16.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:51:16 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:51:16.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:51:16 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:51:16.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:51:16 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-07-16T09:51:16.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:51:16 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-07-16T09:51:16.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:51:16 smithi156 ceph-mon[29302]: Adjusting osd_memory_target on smithi176 to 7968M 2024-07-16T09:51:16.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:51:16 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:51:16.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:51:16 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-16T09:51:16.426 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:51:16 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-16T09:51:16.426 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:51:16 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:51:16.514 DEBUG:teuthology.orchestra.run.smithi176:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 26c91c26-4358-11ef-bcad-c7b262605968 -- ceph orch daemon add osd smithi176:vg_nvme/lv_2 2024-07-16T09:51:16.766 INFO:teuthology.orchestra.run.smithi176.stderr:Inferring config /var/lib/ceph/26c91c26-4358-11ef-bcad-c7b262605968/mon.smithi176/config 2024-07-16T09:51:18.211 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:51:18 smithi176 ceph-mon[36626]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-07-16T09:51:18.232 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:51:18.210+0000 7f2eb224d640 1 -- 172.21.15.176:0/3724616432 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2eac0fe310 msgr2=0x7f2eac0fa470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:51:18.232 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:51:18.210+0000 7f2eb224d640 1 --2- 172.21.15.176:0/3724616432 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2eac0fe310 0x7f2eac0fa470 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f2e940048d0 tx=0x7f2e94009da0 comp rx=0 tx=0).stop 2024-07-16T09:51:18.232 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:51:18.210+0000 7f2eb224d640 1 -- 172.21.15.176:0/3724616432 shutdown_connections 2024-07-16T09:51:18.232 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:51:18.210+0000 7f2eb224d640 1 --2- 172.21.15.176:0/3724616432 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2eac0fe310 0x7f2eac0fa470 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:51:18.232 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:51:18.210+0000 7f2eb224d640 1 --2- 172.21.15.176:0/3724616432 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2eac0fd9c0 0x7f2eac0fddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:51:18.232 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:51:18.210+0000 7f2eb224d640 1 -- 172.21.15.176:0/3724616432 >> 172.21.15.176:0/3724616432 conn(0x7f2eac0f61c0 msgr2=0x7f2eac0f85f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:51:18.232 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:51:18.210+0000 7f2eb224d640 1 -- 172.21.15.176:0/3724616432 shutdown_connections 2024-07-16T09:51:18.232 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:51:18.210+0000 7f2eb224d640 1 -- 172.21.15.176:0/3724616432 wait complete. 2024-07-16T09:51:18.232 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:51:18.210+0000 7f2eb224d640 1 Processor -- start 2024-07-16T09:51:18.233 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:51:18.211+0000 7f2eb224d640 1 -- start start 2024-07-16T09:51:18.233 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:51:18.211+0000 7f2eb224d640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2eac0fd9c0 0x7f2eac10cf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:51:18.233 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:51:18.211+0000 7f2eb224d640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2eac0fe310 0x7f2eac10d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:51:18.233 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:51:18.211+0000 7f2eb224d640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2eac10db10 con 0x7f2eac0fe310 2024-07-16T09:51:18.233 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:51:18.211+0000 7f2eb224d640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2eac10e330 con 0x7f2eac0fd9c0 2024-07-16T09:51:18.233 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:51:18.211+0000 7f2eab7fe640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2eac0fd9c0 0x7f2eac10cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:51:18.233 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:51:18.211+0000 7f2eab7fe640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2eac0fd9c0 0x7f2eac10cf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:37624/0 (socket says 172.21.15.176:37624) 2024-07-16T09:51:18.233 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:51:18.211+0000 7f2eab7fe640 1 -- 172.21.15.176:0/2302355632 learned_addr learned my addr 172.21.15.176:0/2302355632 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-07-16T09:51:18.233 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:51:18.211+0000 7f2eaaffd640 1 --2- 172.21.15.176:0/2302355632 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2eac0fe310 0x7f2eac10d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:51:18.233 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:51:18.211+0000 7f2eab7fe640 1 -- 172.21.15.176:0/2302355632 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2eac0fe310 msgr2=0x7f2eac10d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:51:18.233 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:51:18.211+0000 7f2eab7fe640 1 --2- 172.21.15.176:0/2302355632 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2eac0fe310 0x7f2eac10d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:51:18.233 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:51:18.211+0000 7f2eab7fe640 1 -- 172.21.15.176:0/2302355632 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e94005040 con 0x7f2eac0fd9c0 2024-07-16T09:51:18.233 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:51:18.211+0000 7f2eab7fe640 1 --2- 172.21.15.176:0/2302355632 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2eac0fd9c0 0x7f2eac10cf80 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f2e9c00cb10 tx=0x7f2e9c007ee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:51:18.234 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:51:18.212+0000 7f2ea3fff640 1 -- 172.21.15.176:0/2302355632 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2e9c004200 con 0x7f2eac0fd9c0 2024-07-16T09:51:18.234 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:51:18.212+0000 7f2ea3fff640 1 -- 172.21.15.176:0/2302355632 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2e9c00ac70 con 0x7f2eac0fd9c0 2024-07-16T09:51:18.234 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:51:18.212+0000 7f2eb224d640 1 -- 172.21.15.176:0/2302355632 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2eac10e5c0 con 0x7f2eac0fd9c0 2024-07-16T09:51:18.234 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:51:18.212+0000 7f2ea3fff640 1 -- 172.21.15.176:0/2302355632 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2e9c003db0 con 0x7f2eac0fd9c0 2024-07-16T09:51:18.234 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:51:18.212+0000 7f2eb224d640 1 -- 172.21.15.176:0/2302355632 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2eac10eb30 con 0x7f2eac0fd9c0 2024-07-16T09:51:18.234 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:51:18.213+0000 7f2eb224d640 1 -- 172.21.15.176:0/2302355632 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2eac105710 con 0x7f2eac0fd9c0 2024-07-16T09:51:18.234 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:51:18.213+0000 7f2ea3fff640 1 -- 172.21.15.176:0/2302355632 <== mon.1 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 19) v1 ==== 85127+0+0 (secure 0 0 0) 0x7f2e9c00a7a0 con 0x7f2eac0fd9c0 2024-07-16T09:51:18.234 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:51:18.214+0000 7f2ea3fff640 1 --2- 172.21.15.176:0/2302355632 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f2e98067190 0x7f2e98069640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:51:18.234 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:51:18.214+0000 7f2ea3fff640 1 -- 172.21.15.176:0/2302355632 <== mon.1 v2:172.21.15.176:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 4202+0+0 (secure 0 0 0) 0x7f2e9c084ba0 con 0x7f2eac0fd9c0 2024-07-16T09:51:18.234 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:51:18.214+0000 7f2eaaffd640 1 --2- 172.21.15.176:0/2302355632 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f2e98067190 0x7f2e98069640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:51:18.234 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:51:18.214+0000 7f2eaaffd640 1 --2- 172.21.15.176:0/2302355632 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f2e98067190 0x7f2e98069640 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f2e94003080 tx=0x7f2e94035040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:51:18.234 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:51:18.218+0000 7f2ea3fff640 1 -- 172.21.15.176:0/2302355632 <== mon.1 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f2e9c052330 con 0x7f2eac0fd9c0 2024-07-16T09:51:18.394 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:51:18.392+0000 7f2eb224d640 1 -- 172.21.15.176:0/2302355632 --> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi176:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f2eac10eec0 con 0x7f2e98067190 2024-07-16T09:51:18.424 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:51:18 smithi156 ceph-mon[29302]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-07-16T09:51:19.343 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:51:19 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-16T09:51:19.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:51:19 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-16T09:51:19.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:51:19 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-16T09:51:19.424 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:51:19 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-16T09:51:19.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:51:19 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-16T09:51:19.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:51:19 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-16T09:51:20.424 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:51:20 smithi156 ceph-mon[29302]: from='client.24169 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi176:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-07-16T09:51:20.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:51:20 smithi156 ceph-mon[29302]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-07-16T09:51:20.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:51:20 smithi176 ceph-mon[36626]: from='client.24169 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi176:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-07-16T09:51:20.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:51:20 smithi176 ceph-mon[36626]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-07-16T09:51:21.424 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:51:21 smithi156 ceph-mon[29302]: from='client.? 172.21.15.176:0/49163793' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "79a38eb4-c031-4a53-8588-a4e598eeb3ce"}]: dispatch 2024-07-16T09:51:21.424 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:51:21 smithi156 ceph-mon[29302]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "79a38eb4-c031-4a53-8588-a4e598eeb3ce"}]: dispatch 2024-07-16T09:51:21.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:51:21 smithi156 ceph-mon[29302]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "79a38eb4-c031-4a53-8588-a4e598eeb3ce"}]': finished 2024-07-16T09:51:21.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:51:21 smithi156 ceph-mon[29302]: osdmap e37: 6 total, 5 up, 6 in 2024-07-16T09:51:21.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:51:21 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-07-16T09:51:21.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:51:21 smithi176 ceph-mon[36626]: from='client.? 172.21.15.176:0/49163793' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "79a38eb4-c031-4a53-8588-a4e598eeb3ce"}]: dispatch 2024-07-16T09:51:21.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:51:21 smithi176 ceph-mon[36626]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "79a38eb4-c031-4a53-8588-a4e598eeb3ce"}]: dispatch 2024-07-16T09:51:21.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:51:21 smithi176 ceph-mon[36626]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "79a38eb4-c031-4a53-8588-a4e598eeb3ce"}]': finished 2024-07-16T09:51:21.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:51:21 smithi176 ceph-mon[36626]: osdmap e37: 6 total, 5 up, 6 in 2024-07-16T09:51:21.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:51:21 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-07-16T09:51:22.424 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:51:22 smithi156 ceph-mon[29302]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-07-16T09:51:22.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:51:22 smithi156 ceph-mon[29302]: from='client.? 172.21.15.176:0/1464057147' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-16T09:51:22.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:51:22 smithi176 ceph-mon[36626]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-07-16T09:51:22.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:51:22 smithi176 ceph-mon[36626]: from='client.? 172.21.15.176:0/1464057147' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-16T09:51:24.362 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:51:24 smithi176 ceph-mon[36626]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-07-16T09:51:24.424 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:51:24 smithi156 ceph-mon[29302]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-07-16T09:51:25.343 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:51:25 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-16T09:51:25.424 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:51:25 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-16T09:51:26.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:51:26 smithi176 ceph-mon[36626]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-07-16T09:51:26.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:51:26 smithi156 ceph-mon[29302]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-07-16T09:51:28.424 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:51:28 smithi156 ceph-mon[29302]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-07-16T09:51:28.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:51:28 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-07-16T09:51:28.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:51:28 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-16T09:51:28.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:51:28 smithi176 ceph-mon[36626]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-07-16T09:51:28.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:51:28 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-07-16T09:51:28.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:51:28 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-16T09:51:29.138 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:51:29 smithi176 ceph-mon[36626]: Deploying daemon osd.5 on smithi176 2024-07-16T09:51:29.424 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:51:29 smithi156 ceph-mon[29302]: Deploying daemon osd.5 on smithi176 2024-07-16T09:51:30.343 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:51:30 smithi176 ceph-mon[36626]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-07-16T09:51:30.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:51:30 smithi156 ceph-mon[29302]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-07-16T09:51:32.343 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:51:32 smithi176 ceph-mon[36626]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-07-16T09:51:32.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:51:32 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-16T09:51:32.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:51:32 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:51:32.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:51:32 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:51:32.426 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:51:32 smithi156 ceph-mon[29302]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-07-16T09:51:32.426 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:51:32 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-16T09:51:32.426 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:51:32 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:51:32.426 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:51:32 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:51:34.395 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:51:34 smithi176 ceph-mon[36626]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-07-16T09:51:34.424 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:51:34 smithi156 ceph-mon[29302]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-07-16T09:51:35.645 INFO:teuthology.orchestra.run.smithi176.stdout:Created osd(s) 5 on host 'smithi176' 2024-07-16T09:51:35.645 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:51:35.642+0000 7f2ea3fff640 1 -- 172.21.15.176:0/2302355632 <== mgr.14208 v2:172.21.15.156:6800/2615741717 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f2eac10eec0 con 0x7f2e98067190 2024-07-16T09:51:35.646 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:51:35.644+0000 7f2eb224d640 1 -- 172.21.15.176:0/2302355632 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f2e98067190 msgr2=0x7f2e98069640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:51:35.647 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:51:35.644+0000 7f2eb224d640 1 --2- 172.21.15.176:0/2302355632 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f2e98067190 0x7f2e98069640 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f2e94003080 tx=0x7f2e94035040 comp rx=0 tx=0).stop 2024-07-16T09:51:35.647 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:51:35.644+0000 7f2eb224d640 1 -- 172.21.15.176:0/2302355632 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2eac0fd9c0 msgr2=0x7f2eac10cf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:51:35.647 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:51:35.644+0000 7f2eb224d640 1 --2- 172.21.15.176:0/2302355632 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2eac0fd9c0 0x7f2eac10cf80 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f2e9c00cb10 tx=0x7f2e9c007ee0 comp rx=0 tx=0).stop 2024-07-16T09:51:35.647 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:51:35.644+0000 7f2eb224d640 1 -- 172.21.15.176:0/2302355632 shutdown_connections 2024-07-16T09:51:35.647 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:51:35.644+0000 7f2eb224d640 1 --2- 172.21.15.176:0/2302355632 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f2e98067190 0x7f2e98069640 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:51:35.647 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:51:35.644+0000 7f2eb224d640 1 --2- 172.21.15.176:0/2302355632 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2eac0fe310 0x7f2eac10d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:51:35.647 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:51:35.644+0000 7f2eb224d640 1 --2- 172.21.15.176:0/2302355632 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2eac0fd9c0 0x7f2eac10cf80 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:51:35.647 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:51:35.644+0000 7f2eb224d640 1 -- 172.21.15.176:0/2302355632 >> 172.21.15.176:0/2302355632 conn(0x7f2eac0f61c0 msgr2=0x7f2eac0f7840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:51:35.647 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:51:35.644+0000 7f2eb224d640 1 -- 172.21.15.176:0/2302355632 shutdown_connections 2024-07-16T09:51:35.647 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:51:35.644+0000 7f2eb224d640 1 -- 172.21.15.176:0/2302355632 wait complete. 2024-07-16T09:51:36.213 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:51:36 smithi176 ceph-mon[36626]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-07-16T09:51:36.214 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:51:36 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:51:36.214 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:51:36 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:51:36.214 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:51:36 smithi176 ceph-mon[36626]: from='osd.5 [v2:172.21.15.176:6816/2358804089,v1:172.21.15.176:6817/2358804089]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-07-16T09:51:36.214 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:51:36 smithi176 ceph-mon[36626]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-07-16T09:51:36.215 DEBUG:teuthology.orchestra.run.smithi176:osd.5> sudo journalctl -f -n 0 -u ceph-26c91c26-4358-11ef-bcad-c7b262605968@osd.5.service 2024-07-16T09:51:36.217 INFO:tasks.cephadm:Waiting for 6 OSDs to come up... 2024-07-16T09:51:36.217 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 26c91c26-4358-11ef-bcad-c7b262605968 -- ceph osd stat -f json 2024-07-16T09:51:36.424 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:51:36 smithi156 ceph-mon[29302]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-07-16T09:51:36.424 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:51:36 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:51:36.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:51:36 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:51:36.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:51:36 smithi156 ceph-mon[29302]: from='osd.5 [v2:172.21.15.176:6816/2358804089,v1:172.21.15.176:6817/2358804089]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-07-16T09:51:36.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:51:36 smithi156 ceph-mon[29302]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-07-16T09:51:36.504 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/26c91c26-4358-11ef-bcad-c7b262605968/mon.smithi156/config 2024-07-16T09:51:37.393 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:51:37 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:51:37.393 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:51:37 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:51:37.393 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:51:37 smithi176 ceph-mon[36626]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-07-16T09:51:37.393 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:51:37 smithi176 ceph-mon[36626]: osdmap e38: 6 total, 5 up, 6 in 2024-07-16T09:51:37.393 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:51:37 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-07-16T09:51:37.393 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:51:37 smithi176 ceph-mon[36626]: from='osd.5 [v2:172.21.15.176:6816/2358804089,v1:172.21.15.176:6817/2358804089]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi176", "root=default"]}]: dispatch 2024-07-16T09:51:37.393 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:51:37 smithi176 ceph-mon[36626]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi176", "root=default"]}]: dispatch 2024-07-16T09:51:37.616 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:51:37 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:51:37.616 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:51:37 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:51:37.616 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:51:37 smithi156 ceph-mon[29302]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-07-16T09:51:37.616 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:51:37 smithi156 ceph-mon[29302]: osdmap e38: 6 total, 5 up, 6 in 2024-07-16T09:51:37.616 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:51:37 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-07-16T09:51:37.616 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:51:37 smithi156 ceph-mon[29302]: from='osd.5 [v2:172.21.15.176:6816/2358804089,v1:172.21.15.176:6817/2358804089]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi176", "root=default"]}]: dispatch 2024-07-16T09:51:37.616 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:51:37 smithi156 ceph-mon[29302]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi176", "root=default"]}]: dispatch 2024-07-16T09:51:38.013 INFO:journalctl@ceph.osd.5.smithi176.stdout:Jul 16 09:51:37 smithi176 ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-5[54868]: 2024-07-16T09:51:37.651+0000 7fb034e66640 -1 osd.5 0 waiting for initial osdmap 2024-07-16T09:51:38.013 INFO:journalctl@ceph.osd.5.smithi176.stdout:Jul 16 09:51:37 smithi176 ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-5[54868]: 2024-07-16T09:51:37.659+0000 7fb03049e640 -1 osd.5 39 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-07-16T09:51:38.116 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:38.113+0000 7f65082ab640 1 -- 172.21.15.156:0/4025779754 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f65000fc520 msgr2=0x7f65000fc990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:51:38.116 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:38.113+0000 7f65082ab640 1 --2- 172.21.15.156:0/4025779754 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f65000fc520 0x7f65000fc990 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f64f00048d0 tx=0x7f64f0009da0 comp rx=0 tx=0).stop 2024-07-16T09:51:38.116 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:38.113+0000 7f65082ab640 1 -- 172.21.15.156:0/4025779754 shutdown_connections 2024-07-16T09:51:38.116 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:38.113+0000 7f65082ab640 1 --2- 172.21.15.156:0/4025779754 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f65000fc520 0x7f65000fc990 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:51:38.116 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:38.113+0000 7f65082ab640 1 --2- 172.21.15.156:0/4025779754 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f65000fb250 0x7f65000fb660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:51:38.116 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:38.113+0000 7f65082ab640 1 -- 172.21.15.156:0/4025779754 >> 172.21.15.156:0/4025779754 conn(0x7f65000f6a30 msgr2=0x7f65000f8e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:51:38.116 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:38.113+0000 7f65082ab640 1 -- 172.21.15.156:0/4025779754 shutdown_connections 2024-07-16T09:51:38.116 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:38.113+0000 7f65082ab640 1 -- 172.21.15.156:0/4025779754 wait complete. 2024-07-16T09:51:38.116 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:38.113+0000 7f65082ab640 1 Processor -- start 2024-07-16T09:51:38.117 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:38.113+0000 7f65082ab640 1 -- start start 2024-07-16T09:51:38.117 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:38.113+0000 7f65082ab640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f65000fb250 0x7f65000ff320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:51:38.117 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:38.113+0000 7f65082ab640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f65000fc520 0x7f65000fd860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:51:38.117 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:38.113+0000 7f65082ab640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65000fdda0 con 0x7f65000fb250 2024-07-16T09:51:38.117 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:38.113+0000 7f65082ab640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65000fdee0 con 0x7f65000fc520 2024-07-16T09:51:38.117 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:38.113+0000 7f6506047640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f65000fb250 0x7f65000ff320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:51:38.117 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:38.114+0000 7f6506047640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f65000fb250 0x7f65000ff320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:60750/0 (socket says 172.21.15.156:60750) 2024-07-16T09:51:38.117 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:38.114+0000 7f6506047640 1 -- 172.21.15.156:0/1805165402 learned_addr learned my addr 172.21.15.156:0/1805165402 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T09:51:38.117 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:38.114+0000 7f6505846640 1 --2- 172.21.15.156:0/1805165402 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f65000fc520 0x7f65000fd860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:51:38.117 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:38.114+0000 7f6506047640 1 -- 172.21.15.156:0/1805165402 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f65000fc520 msgr2=0x7f65000fd860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:51:38.118 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:38.114+0000 7f6506047640 1 --2- 172.21.15.156:0/1805165402 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f65000fc520 0x7f65000fd860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:51:38.118 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:38.114+0000 7f6506047640 1 -- 172.21.15.156:0/1805165402 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64f0005040 con 0x7f65000fb250 2024-07-16T09:51:38.118 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:38.114+0000 7f6506047640 1 --2- 172.21.15.156:0/1805165402 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f65000fb250 0x7f65000ff320 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f64fc002a50 tx=0x7f64fc004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:51:38.118 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:38.114+0000 7f64f6ffd640 1 -- 172.21.15.156:0/1805165402 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f64fc007c30 con 0x7f65000fb250 2024-07-16T09:51:38.118 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:38.114+0000 7f64f6ffd640 1 -- 172.21.15.156:0/1805165402 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f64fc00f040 con 0x7f65000fb250 2024-07-16T09:51:38.118 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:38.114+0000 7f65082ab640 1 -- 172.21.15.156:0/1805165402 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65000fe1c0 con 0x7f65000fb250 2024-07-16T09:51:38.118 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:38.114+0000 7f64f6ffd640 1 -- 172.21.15.156:0/1805165402 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f64fc00dad0 con 0x7f65000fb250 2024-07-16T09:51:38.118 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:38.114+0000 7f65082ab640 1 -- 172.21.15.156:0/1805165402 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65001150c0 con 0x7f65000fb250 2024-07-16T09:51:38.120 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:38.115+0000 7f65082ab640 1 -- 172.21.15.156:0/1805165402 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65000fa6b0 con 0x7f65000fb250 2024-07-16T09:51:38.120 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:38.116+0000 7f64f6ffd640 1 -- 172.21.15.156:0/1805165402 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 19) v1 ==== 85127+0+0 (secure 0 0 0) 0x7f64fc00dc70 con 0x7f65000fb250 2024-07-16T09:51:38.120 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:38.116+0000 7f64f6ffd640 1 --2- 172.21.15.156:0/1805165402 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f64ec067190 0x7f64ec069640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:51:38.120 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:38.116+0000 7f64f6ffd640 1 -- 172.21.15.156:0/1805165402 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4354+0+0 (secure 0 0 0) 0x7f64fc083dd0 con 0x7f65000fb250 2024-07-16T09:51:38.120 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:38.116+0000 7f6505846640 1 --2- 172.21.15.156:0/1805165402 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f64ec067190 0x7f64ec069640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:51:38.120 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:38.117+0000 7f6505846640 1 --2- 172.21.15.156:0/1805165402 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f64ec067190 0x7f64ec069640 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f64f00043d0 tx=0x7f64f0035040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:51:38.122 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:38.120+0000 7f64f6ffd640 1 -- 172.21.15.156:0/1805165402 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f64fc04d250 con 0x7f65000fb250 2024-07-16T09:51:38.295 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:38.293+0000 7f65082ab640 1 -- 172.21.15.156:0/1805165402 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7f650004b530 con 0x7f65000fb250 2024-07-16T09:51:38.295 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:38.294+0000 7f64f6ffd640 1 -- 172.21.15.156:0/1805165402 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v39) v1 ==== 74+0+130 (secure 0 0 0) 0x7f64fc0170a0 con 0x7f65000fb250 2024-07-16T09:51:38.296 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-07-16T09:51:38.297 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:38.296+0000 7f65082ab640 1 -- 172.21.15.156:0/1805165402 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f64ec067190 msgr2=0x7f64ec069640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:51:38.297 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:38.296+0000 7f65082ab640 1 --2- 172.21.15.156:0/1805165402 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f64ec067190 0x7f64ec069640 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f64f00043d0 tx=0x7f64f0035040 comp rx=0 tx=0).stop 2024-07-16T09:51:38.297 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:38.296+0000 7f65082ab640 1 -- 172.21.15.156:0/1805165402 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f65000fb250 msgr2=0x7f65000ff320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:51:38.297 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:38.296+0000 7f65082ab640 1 --2- 172.21.15.156:0/1805165402 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f65000fb250 0x7f65000ff320 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f64fc002a50 tx=0x7f64fc004600 comp rx=0 tx=0).stop 2024-07-16T09:51:38.297 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:38.296+0000 7f65082ab640 1 -- 172.21.15.156:0/1805165402 shutdown_connections 2024-07-16T09:51:38.297 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:38.296+0000 7f65082ab640 1 --2- 172.21.15.156:0/1805165402 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f64ec067190 0x7f64ec069640 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:51:38.298 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:38.296+0000 7f65082ab640 1 --2- 172.21.15.156:0/1805165402 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f65000fc520 0x7f65000fd860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:51:38.298 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:38.296+0000 7f65082ab640 1 --2- 172.21.15.156:0/1805165402 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f65000fb250 0x7f65000ff320 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:51:38.298 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:38.296+0000 7f65082ab640 1 -- 172.21.15.156:0/1805165402 >> 172.21.15.156:0/1805165402 conn(0x7f65000f6a30 msgr2=0x7f6500065e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:51:38.298 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:38.296+0000 7f65082ab640 1 -- 172.21.15.156:0/1805165402 shutdown_connections 2024-07-16T09:51:38.298 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:38.296+0000 7f65082ab640 1 -- 172.21.15.156:0/1805165402 wait complete. 2024-07-16T09:51:38.441 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:51:38 smithi156 ceph-mon[29302]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-07-16T09:51:38.441 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:51:38 smithi156 ceph-mon[29302]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi176", "root=default"]}]': finished 2024-07-16T09:51:38.441 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:51:38 smithi156 ceph-mon[29302]: osdmap e39: 6 total, 5 up, 6 in 2024-07-16T09:51:38.441 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:51:38 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-07-16T09:51:38.441 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:51:38 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-07-16T09:51:38.442 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:51:38 smithi156 ceph-mon[29302]: from='client.? 172.21.15.156:0/1805165402' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-07-16T09:51:38.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:51:38 smithi176 ceph-mon[36626]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-07-16T09:51:38.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:51:38 smithi176 ceph-mon[36626]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi176", "root=default"]}]': finished 2024-07-16T09:51:38.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:51:38 smithi176 ceph-mon[36626]: osdmap e39: 6 total, 5 up, 6 in 2024-07-16T09:51:38.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:51:38 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-07-16T09:51:38.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:51:38 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-07-16T09:51:38.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:51:38 smithi176 ceph-mon[36626]: from='client.? 172.21.15.156:0/1805165402' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-07-16T09:51:38.865 INFO:teuthology.orchestra.run.smithi156.stdout:{"epoch":39,"num_osds":6,"num_up_osds":5,"osd_up_since":1721123474,"num_in_osds":6,"osd_in_since":1721123480,"num_remapped_pgs":0} 2024-07-16T09:51:39.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:51:39 smithi176 ceph-mon[36626]: purged_snaps scrub starts 2024-07-16T09:51:39.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:51:39 smithi176 ceph-mon[36626]: purged_snaps scrub ok 2024-07-16T09:51:39.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:51:39 smithi176 ceph-mon[36626]: osd.5 [v2:172.21.15.176:6816/2358804089,v1:172.21.15.176:6817/2358804089] boot 2024-07-16T09:51:39.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:51:39 smithi176 ceph-mon[36626]: osdmap e40: 6 total, 6 up, 6 in 2024-07-16T09:51:39.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:51:39 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-07-16T09:51:39.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:51:39 smithi176 ceph-mon[36626]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-07-16T09:51:39.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:51:39 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:51:39.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:51:39 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:51:39.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:51:39 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-07-16T09:51:39.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:51:39 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-07-16T09:51:39.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:51:39 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-07-16T09:51:39.845 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:51:39 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:51:39.845 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:51:39 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-16T09:51:39.845 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:51:39 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-16T09:51:39.845 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:51:39 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:51:39.866 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 26c91c26-4358-11ef-bcad-c7b262605968 -- ceph osd stat -f json 2024-07-16T09:51:39.924 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:51:39 smithi156 ceph-mon[29302]: purged_snaps scrub starts 2024-07-16T09:51:39.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:51:39 smithi156 ceph-mon[29302]: purged_snaps scrub ok 2024-07-16T09:51:39.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:51:39 smithi156 ceph-mon[29302]: osd.5 [v2:172.21.15.176:6816/2358804089,v1:172.21.15.176:6817/2358804089] boot 2024-07-16T09:51:39.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:51:39 smithi156 ceph-mon[29302]: osdmap e40: 6 total, 6 up, 6 in 2024-07-16T09:51:39.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:51:39 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-07-16T09:51:39.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:51:39 smithi156 ceph-mon[29302]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-07-16T09:51:39.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:51:39 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:51:39.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:51:39 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:51:39.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:51:39 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-07-16T09:51:39.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:51:39 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-07-16T09:51:39.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:51:39 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-07-16T09:51:39.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:51:39 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:51:39.926 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:51:39 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-16T09:51:39.926 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:51:39 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-16T09:51:39.926 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:51:39 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:51:40.123 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/26c91c26-4358-11ef-bcad-c7b262605968/mon.smithi156/config 2024-07-16T09:51:40.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:51:40 smithi156 ceph-mon[29302]: Detected new or changed devices on smithi176 2024-07-16T09:51:40.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:51:40 smithi156 ceph-mon[29302]: Adjusting osd_memory_target on smithi176 to 5312M 2024-07-16T09:51:40.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:51:40 smithi156 ceph-mon[29302]: osdmap e41: 6 total, 6 up, 6 in 2024-07-16T09:51:40.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:51:40 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-16T09:51:40.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:51:40 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-16T09:51:40.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:51:40 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-16T09:51:40.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:51:40 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:51:40.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:51:40 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-16T09:51:41.093 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:51:40 smithi176 ceph-mon[36626]: Detected new or changed devices on smithi176 2024-07-16T09:51:41.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:51:40 smithi176 ceph-mon[36626]: Adjusting osd_memory_target on smithi176 to 5312M 2024-07-16T09:51:41.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:51:40 smithi176 ceph-mon[36626]: osdmap e41: 6 total, 6 up, 6 in 2024-07-16T09:51:41.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:51:40 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-16T09:51:41.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:51:40 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-16T09:51:41.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:51:40 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-16T09:51:41.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:51:40 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:51:41.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:51:40 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-16T09:51:41.583 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:41.568+0000 7f757ba5d640 1 -- 172.21.15.156:0/278285384 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f75740fc360 msgr2=0x7f75740fc770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:51:41.584 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:41.568+0000 7f757ba5d640 1 --2- 172.21.15.156:0/278285384 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f75740fc360 0x7f75740fc770 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f75700048d0 tx=0x7f7570009da0 comp rx=0 tx=0).stop 2024-07-16T09:51:41.584 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:41.568+0000 7f757ba5d640 1 -- 172.21.15.156:0/278285384 shutdown_connections 2024-07-16T09:51:41.584 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:41.568+0000 7f757ba5d640 1 --2- 172.21.15.156:0/278285384 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f75740fd630 0x7f75740fdaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:51:41.584 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:41.568+0000 7f757ba5d640 1 --2- 172.21.15.156:0/278285384 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f75740fc360 0x7f75740fc770 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:51:41.584 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:41.568+0000 7f757ba5d640 1 -- 172.21.15.156:0/278285384 >> 172.21.15.156:0/278285384 conn(0x7f75740f7b40 msgr2=0x7f75740f9f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:51:41.584 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:41.568+0000 7f757ba5d640 1 -- 172.21.15.156:0/278285384 shutdown_connections 2024-07-16T09:51:41.584 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:41.568+0000 7f757ba5d640 1 -- 172.21.15.156:0/278285384 wait complete. 2024-07-16T09:51:41.584 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:41.568+0000 7f757ba5d640 1 Processor -- start 2024-07-16T09:51:41.584 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:41.568+0000 7f757ba5d640 1 -- start start 2024-07-16T09:51:41.584 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:41.569+0000 7f757ba5d640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f75740fc360 0x7f757410cf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:51:41.584 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:41.569+0000 7f757ba5d640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f75740fd630 0x7f757410d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:51:41.584 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:41.569+0000 7f757ba5d640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f757410dac0 con 0x7f75740fc360 2024-07-16T09:51:41.585 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:41.569+0000 7f757ba5d640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f757410e2e0 con 0x7f75740fd630 2024-07-16T09:51:41.585 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:41.569+0000 7f75797f9640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f75740fc360 0x7f757410cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:51:41.585 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:41.569+0000 7f75797f9640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f75740fc360 0x7f757410cf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:60754/0 (socket says 172.21.15.156:60754) 2024-07-16T09:51:41.585 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:41.569+0000 7f75797f9640 1 -- 172.21.15.156:0/3186614961 learned_addr learned my addr 172.21.15.156:0/3186614961 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T09:51:41.585 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:41.569+0000 7f7578ff8640 1 --2- 172.21.15.156:0/3186614961 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f75740fd630 0x7f757410d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:51:41.585 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:41.569+0000 7f75797f9640 1 -- 172.21.15.156:0/3186614961 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f75740fd630 msgr2=0x7f757410d470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:51:41.585 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:41.569+0000 7f75797f9640 1 --2- 172.21.15.156:0/3186614961 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f75740fd630 0x7f757410d470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:51:41.585 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:41.569+0000 7f75797f9640 1 -- 172.21.15.156:0/3186614961 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7570005040 con 0x7f75740fc360 2024-07-16T09:51:41.585 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:41.569+0000 7f7578ff8640 1 --2- 172.21.15.156:0/3186614961 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f75740fd630 0x7f757410d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-16T09:51:41.585 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:41.569+0000 7f75797f9640 1 --2- 172.21.15.156:0/3186614961 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f75740fc360 0x7f757410cf30 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f7570003080 tx=0x7f7570009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:51:41.585 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:41.569+0000 7f7569ffb640 1 -- 172.21.15.156:0/3186614961 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7570038070 con 0x7f75740fc360 2024-07-16T09:51:41.585 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:41.569+0000 7f7569ffb640 1 -- 172.21.15.156:0/3186614961 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7570032970 con 0x7f75740fc360 2024-07-16T09:51:41.585 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:41.569+0000 7f7569ffb640 1 -- 172.21.15.156:0/3186614961 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7570033900 con 0x7f75740fc360 2024-07-16T09:51:41.586 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:41.569+0000 7f757ba5d640 1 -- 172.21.15.156:0/3186614961 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f757410e510 con 0x7f75740fc360 2024-07-16T09:51:41.586 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:41.570+0000 7f757ba5d640 1 -- 172.21.15.156:0/3186614961 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f757410ea50 con 0x7f75740fc360 2024-07-16T09:51:41.586 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:41.570+0000 7f757ba5d640 1 -- 172.21.15.156:0/3186614961 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75740fb7c0 con 0x7f75740fc360 2024-07-16T09:51:41.586 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:41.571+0000 7f7569ffb640 1 -- 172.21.15.156:0/3186614961 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 19) v1 ==== 85127+0+0 (secure 0 0 0) 0x7f7570030020 con 0x7f75740fc360 2024-07-16T09:51:41.586 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:41.571+0000 7f7569ffb640 1 --2- 172.21.15.156:0/3186614961 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f7560067190 0x7f7560069640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:51:41.586 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:41.571+0000 7f7569ffb640 1 -- 172.21.15.156:0/3186614961 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f75700a4880 con 0x7f75740fc360 2024-07-16T09:51:41.586 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:41.571+0000 7f7578ff8640 1 --2- 172.21.15.156:0/3186614961 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f7560067190 0x7f7560069640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:51:41.586 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:41.572+0000 7f7578ff8640 1 --2- 172.21.15.156:0/3186614961 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f7560067190 0x7f7560069640 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f757410cce0 tx=0x7f7564009040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:51:41.586 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:41.576+0000 7f7569ffb640 1 -- 172.21.15.156:0/3186614961 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f7570071e60 con 0x7f75740fc360 2024-07-16T09:51:41.746 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:41.744+0000 7f757ba5d640 1 -- 172.21.15.156:0/3186614961 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7f757404b530 con 0x7f75740fc360 2024-07-16T09:51:41.746 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:41.745+0000 7f7569ffb640 1 -- 172.21.15.156:0/3186614961 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v41) v1 ==== 74+0+130 (secure 0 0 0) 0x7f75700410a0 con 0x7f75740fc360 2024-07-16T09:51:41.746 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-07-16T09:51:41.748 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:41.747+0000 7f757ba5d640 1 -- 172.21.15.156:0/3186614961 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f7560067190 msgr2=0x7f7560069640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:51:41.748 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:41.747+0000 7f757ba5d640 1 --2- 172.21.15.156:0/3186614961 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f7560067190 0x7f7560069640 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f757410cce0 tx=0x7f7564009040 comp rx=0 tx=0).stop 2024-07-16T09:51:41.748 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:41.747+0000 7f757ba5d640 1 -- 172.21.15.156:0/3186614961 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f75740fc360 msgr2=0x7f757410cf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:51:41.748 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:41.747+0000 7f757ba5d640 1 --2- 172.21.15.156:0/3186614961 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f75740fc360 0x7f757410cf30 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f7570003080 tx=0x7f7570009da0 comp rx=0 tx=0).stop 2024-07-16T09:51:41.748 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:41.747+0000 7f757ba5d640 1 -- 172.21.15.156:0/3186614961 shutdown_connections 2024-07-16T09:51:41.749 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:41.747+0000 7f757ba5d640 1 --2- 172.21.15.156:0/3186614961 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f7560067190 0x7f7560069640 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:51:41.749 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:41.747+0000 7f757ba5d640 1 --2- 172.21.15.156:0/3186614961 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f75740fd630 0x7f757410d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:51:41.749 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:41.747+0000 7f757ba5d640 1 --2- 172.21.15.156:0/3186614961 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f75740fc360 0x7f757410cf30 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:51:41.749 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:41.747+0000 7f757ba5d640 1 -- 172.21.15.156:0/3186614961 >> 172.21.15.156:0/3186614961 conn(0x7f75740f7b40 msgr2=0x7f75740f91b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:51:41.749 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:41.747+0000 7f757ba5d640 1 -- 172.21.15.156:0/3186614961 shutdown_connections 2024-07-16T09:51:41.749 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:41.747+0000 7f757ba5d640 1 -- 172.21.15.156:0/3186614961 wait complete. 2024-07-16T09:51:41.924 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:51:41 smithi156 ceph-mon[29302]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-07-16T09:51:42.093 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:51:41 smithi176 ceph-mon[36626]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-07-16T09:51:42.378 INFO:teuthology.orchestra.run.smithi156.stdout:{"epoch":41,"num_osds":6,"num_up_osds":6,"osd_up_since":1721123498,"num_in_osds":6,"osd_in_since":1721123480,"num_remapped_pgs":0} 2024-07-16T09:51:42.379 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 26c91c26-4358-11ef-bcad-c7b262605968 -- ceph osd dump --format=json 2024-07-16T09:51:42.632 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/26c91c26-4358-11ef-bcad-c7b262605968/mon.smithi156/config 2024-07-16T09:51:42.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:51:42 smithi156 ceph-mon[29302]: from='client.? 172.21.15.156:0/3186614961' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-07-16T09:51:43.093 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:51:42 smithi176 ceph-mon[36626]: from='client.? 172.21.15.156:0/3186614961' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-07-16T09:51:43.676 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:51:43 smithi156 ceph-mon[29302]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-07-16T09:51:44.093 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:51:43 smithi176 ceph-mon[36626]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-07-16T09:51:44.141 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:44.138+0000 7f5963df3640 1 -- 172.21.15.156:0/262979772 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f595c0fdf50 msgr2=0x7f595c0fe3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:51:44.141 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:44.138+0000 7f5963df3640 1 --2- 172.21.15.156:0/262979772 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f595c0fdf50 0x7f595c0fe3c0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f594c0048d0 tx=0x7f594c009da0 comp rx=0 tx=0).stop 2024-07-16T09:51:44.142 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:44.138+0000 7f5963df3640 1 -- 172.21.15.156:0/262979772 shutdown_connections 2024-07-16T09:51:44.142 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:44.138+0000 7f5963df3640 1 --2- 172.21.15.156:0/262979772 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f595c0fdf50 0x7f595c0fe3c0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:51:44.142 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:44.138+0000 7f5963df3640 1 --2- 172.21.15.156:0/262979772 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f595c0650e0 0x7f595c0654b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:51:44.142 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:44.138+0000 7f5963df3640 1 -- 172.21.15.156:0/262979772 >> 172.21.15.156:0/262979772 conn(0x7f595c0721c0 msgr2=0x7f595c0725c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:51:44.142 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:44.138+0000 7f5963df3640 1 -- 172.21.15.156:0/262979772 shutdown_connections 2024-07-16T09:51:44.142 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:44.138+0000 7f5963df3640 1 -- 172.21.15.156:0/262979772 wait complete. 2024-07-16T09:51:44.142 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:44.139+0000 7f5963df3640 1 Processor -- start 2024-07-16T09:51:44.142 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:44.139+0000 7f5963df3640 1 -- start start 2024-07-16T09:51:44.142 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:44.139+0000 7f5963df3640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f595c0650e0 0x7f595c113120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:51:44.142 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:44.139+0000 7f5963df3640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f595c0fdf50 0x7f595c113660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:51:44.142 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:44.139+0000 7f5963df3640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f595c113c60 con 0x7f595c0fdf50 2024-07-16T09:51:44.143 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:44.139+0000 7f5963df3640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f595c10d0f0 con 0x7f595c0650e0 2024-07-16T09:51:44.143 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:44.139+0000 7f596138e640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f595c0fdf50 0x7f595c113660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:51:44.143 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:44.139+0000 7f596138e640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f595c0fdf50 0x7f595c113660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:37790/0 (socket says 172.21.15.156:37790) 2024-07-16T09:51:44.143 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:44.139+0000 7f596138e640 1 -- 172.21.15.156:0/3285579526 learned_addr learned my addr 172.21.15.156:0/3285579526 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T09:51:44.143 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:44.139+0000 7f5961b8f640 1 --2- 172.21.15.156:0/3285579526 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f595c0650e0 0x7f595c113120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:51:44.143 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:44.140+0000 7f596138e640 1 -- 172.21.15.156:0/3285579526 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f595c0650e0 msgr2=0x7f595c113120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:51:44.143 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:44.140+0000 7f596138e640 1 --2- 172.21.15.156:0/3285579526 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f595c0650e0 0x7f595c113120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:51:44.143 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:44.140+0000 7f596138e640 1 -- 172.21.15.156:0/3285579526 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f594c005040 con 0x7f595c0fdf50 2024-07-16T09:51:44.143 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:44.140+0000 7f5961b8f640 1 --2- 172.21.15.156:0/3285579526 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f595c0650e0 0x7f595c113120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-16T09:51:44.143 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:44.140+0000 7f596138e640 1 --2- 172.21.15.156:0/3285579526 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f595c0fdf50 0x7f595c113660 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f594c0043d0 tx=0x7f594c009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:51:44.143 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:44.140+0000 7f59527fc640 1 -- 172.21.15.156:0/3285579526 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f594c038070 con 0x7f595c0fdf50 2024-07-16T09:51:44.143 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:44.140+0000 7f5963df3640 1 -- 172.21.15.156:0/3285579526 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f595c10d370 con 0x7f595c0fdf50 2024-07-16T09:51:44.143 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:44.140+0000 7f59527fc640 1 -- 172.21.15.156:0/3285579526 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f594c032970 con 0x7f595c0fdf50 2024-07-16T09:51:44.144 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:44.140+0000 7f59527fc640 1 -- 172.21.15.156:0/3285579526 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f594c03da60 con 0x7f595c0fdf50 2024-07-16T09:51:44.144 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:44.140+0000 7f5963df3640 1 -- 172.21.15.156:0/3285579526 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f595c10d830 con 0x7f595c0fdf50 2024-07-16T09:51:44.144 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:44.141+0000 7f5963df3640 1 -- 172.21.15.156:0/3285579526 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f595c0fdaf0 con 0x7f595c0fdf50 2024-07-16T09:51:44.145 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:44.141+0000 7f59527fc640 1 -- 172.21.15.156:0/3285579526 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 19) v1 ==== 85127+0+0 (secure 0 0 0) 0x7f594c030020 con 0x7f595c0fdf50 2024-07-16T09:51:44.145 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:44.142+0000 7f59527fc640 1 --2- 172.21.15.156:0/3285579526 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f5948067190 0x7f5948069640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:51:44.145 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:44.142+0000 7f59527fc640 1 -- 172.21.15.156:0/3285579526 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f594c0a49d0 con 0x7f595c0fdf50 2024-07-16T09:51:44.145 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:44.142+0000 7f5961b8f640 1 --2- 172.21.15.156:0/3285579526 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f5948067190 0x7f5948069640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:51:44.145 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:44.142+0000 7f5961b8f640 1 --2- 172.21.15.156:0/3285579526 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f5948067190 0x7f5948069640 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f5958007fd0 tx=0x7f5958009e90 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:51:44.147 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:44.146+0000 7f59527fc640 1 -- 172.21.15.156:0/3285579526 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f594c071fb0 con 0x7f595c0fdf50 2024-07-16T09:51:44.313 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:44.312+0000 7f5963df3640 1 -- 172.21.15.156:0/3285579526 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f595c10e790 con 0x7f595c0fdf50 2024-07-16T09:51:44.314 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:44.312+0000 7f59527fc640 1 -- 172.21.15.156:0/3285579526 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v41) v1 ==== 74+0+11124 (secure 0 0 0) 0x7f594c0755d0 con 0x7f595c0fdf50 2024-07-16T09:51:44.314 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-07-16T09:51:44.314 INFO:teuthology.orchestra.run.smithi156.stdout:{"epoch":41,"fsid":"26c91c26-4358-11ef-bcad-c7b262605968","created":"2024-07-16T09:46:23.833980+0000","modified":"2024-07-16T09:51:39.649731+0000","last_up_change":"2024-07-16T09:51:38.648169+0000","last_in_change":"2024-07-16T09:51:20.573206+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":14,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-16T09:50:29.025633+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"38938d0f-f41d-4d70-9f59-f977181e2943","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":36114952},{"type":"v1","addr":"172.21.15.156:6803","nonce":36114952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":36114952},{"type":"v1","addr":"172.21.15.156:6805","nonce":36114952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":36114952},{"type":"v1","addr":"172.21.15.156:6809","nonce":36114952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":36114952},{"type":"v1","addr":"172.21.15.156:6807","nonce":36114952}]},"public_addr":"172.21.15.156:6803/36114952","cluster_addr":"172.21.15.156:6805/36114952","heartbeat_back_addr":"172.21.15.156:6809/36114952","heartbeat_front_addr":"172.21.15.156:6807/36114952","state":["exists","up"]},{"osd":1,"uuid":"04d30aff-25e2-4b87-927d-a2497d0123b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":1587604616},{"type":"v1","addr":"172.21.15.156:6811","nonce":1587604616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":1587604616},{"type":"v1","addr":"172.21.15.156:6813","nonce":1587604616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":1587604616},{"type":"v1","addr":"172.21.15.156:6817","nonce":1587604616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":1587604616},{"type":"v1","addr":"172.21.15.156:6815","nonce":1587604616}]},"public_addr":"172.21.15.156:6811/1587604616","cluster_addr":"172.21.15.156:6813/1587604616","heartbeat_back_addr":"172.21.15.156:6817/1587604616","heartbeat_front_addr":"172.21.15.156:6815/1587604616","state":["exists","up"]},{"osd":2,"uuid":"5b2cee4f-d3da-4dac-85b9-ee34914e548a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":3710668239},{"type":"v1","addr":"172.21.15.156:6819","nonce":3710668239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":3710668239},{"type":"v1","addr":"172.21.15.156:6821","nonce":3710668239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":3710668239},{"type":"v1","addr":"172.21.15.156:6825","nonce":3710668239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":3710668239},{"type":"v1","addr":"172.21.15.156:6823","nonce":3710668239}]},"public_addr":"172.21.15.156:6819/3710668239","cluster_addr":"172.21.15.156:6821/3710668239","heartbeat_back_addr":"172.21.15.156:6825/3710668239","heartbeat_front_addr":"172.21.15.156:6823/3710668239","state":["exists","up"]},{"osd":3,"uuid":"8428be96-686f-4318-9585-4b8a19a7b9c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6800","nonce":1740426750},{"type":"v1","addr":"172.21.15.176:6801","nonce":1740426750}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6802","nonce":1740426750},{"type":"v1","addr":"172.21.15.176:6803","nonce":1740426750}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6806","nonce":1740426750},{"type":"v1","addr":"172.21.15.176:6807","nonce":1740426750}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6804","nonce":1740426750},{"type":"v1","addr":"172.21.15.176:6805","nonce":1740426750}]},"public_addr":"172.21.15.176:6801/1740426750","cluster_addr":"172.21.15.176:6803/1740426750","heartbeat_back_addr":"172.21.15.176:6807/1740426750","heartbeat_front_addr":"172.21.15.176:6805/1740426750","state":["exists","up"]},{"osd":4,"uuid":"fd311fe4-8853-4403-a3a8-863eb22a58ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6808","nonce":2702155727},{"type":"v1","addr":"172.21.15.176:6809","nonce":2702155727}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6810","nonce":2702155727},{"type":"v1","addr":"172.21.15.176:6811","nonce":2702155727}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6814","nonce":2702155727},{"type":"v1","addr":"172.21.15.176:6815","nonce":2702155727}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6812","nonce":2702155727},{"type":"v1","addr":"172.21.15.176:6813","nonce":2702155727}]},"public_addr":"172.21.15.176:6809/2702155727","cluster_addr":"172.21.15.176:6811/2702155727","heartbeat_back_addr":"172.21.15.176:6815/2702155727","heartbeat_front_addr":"172.21.15.176:6813/2702155727","state":["exists","up"]},{"osd":5,"uuid":"79a38eb4-c031-4a53-8588-a4e598eeb3ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6816","nonce":2358804089},{"type":"v1","addr":"172.21.15.176:6817","nonce":2358804089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6818","nonce":2358804089},{"type":"v1","addr":"172.21.15.176:6819","nonce":2358804089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6822","nonce":2358804089},{"type":"v1","addr":"172.21.15.176:6823","nonce":2358804089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6820","nonce":2358804089},{"type":"v1","addr":"172.21.15.176:6821","nonce":2358804089}]},"public_addr":"172.21.15.176:6817/2358804089","cluster_addr":"172.21.15.176:6819/2358804089","heartbeat_back_addr":"172.21.15.176:6823/2358804089","heartbeat_front_addr":"172.21.15.176:6821/2358804089","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-07-16T09:49:32.966257+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-07-16T09:49:59.906579+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-07-16T09:50:25.246162+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-07-16T09:50:47.932851+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-07-16T09:51:12.299573+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-07-16T09:51:36.971019+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.156:6801/3984987611":"2024-07-17T09:48:24.722579+0000","172.21.15.156:0/3487646818":"2024-07-17T09:46:53.642082+0000","172.21.15.156:6801/2871070852":"2024-07-17T09:46:53.642082+0000","172.21.15.156:6800/2871070852":"2024-07-17T09:46:53.642082+0000","172.21.15.156:0/3390422489":"2024-07-17T09:46:53.642082+0000","172.21.15.156:0/4158213614":"2024-07-17T09:46:53.642082+0000","172.21.15.156:6800/1023000918":"2024-07-17T09:47:22.566842+0000","172.21.15.156:6800/3984987611":"2024-07-17T09:48:24.722579+0000","172.21.15.156:0/1864977086":"2024-07-17T09:47:22.566842+0000","172.21.15.156:0/2120399071":"2024-07-17T09:47:22.566842+0000","172.21.15.156:0/2320156657":"2024-07-17T09:48:24.722579+0000","172.21.15.156:6801/1023000918":"2024-07-17T09:47:22.566842+0000","172.21.15.156:0/2322505969":"2024-07-17T09:47:22.566842+0000","172.21.15.156:0/1325014720":"2024-07-17T09:48:24.722579+0000","172.21.15.156:0/3191234605":"2024-07-17T09:48:24.722579+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-16T09:51:44.316 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:44.314+0000 7f5963df3640 1 -- 172.21.15.156:0/3285579526 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f5948067190 msgr2=0x7f5948069640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:51:44.316 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:44.314+0000 7f5963df3640 1 --2- 172.21.15.156:0/3285579526 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f5948067190 0x7f5948069640 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f5958007fd0 tx=0x7f5958009e90 comp rx=0 tx=0).stop 2024-07-16T09:51:44.316 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:44.314+0000 7f5963df3640 1 -- 172.21.15.156:0/3285579526 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f595c0fdf50 msgr2=0x7f595c113660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:51:44.316 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:44.315+0000 7f5963df3640 1 --2- 172.21.15.156:0/3285579526 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f595c0fdf50 0x7f595c113660 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f594c0043d0 tx=0x7f594c009da0 comp rx=0 tx=0).stop 2024-07-16T09:51:44.317 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:44.315+0000 7f5963df3640 1 -- 172.21.15.156:0/3285579526 shutdown_connections 2024-07-16T09:51:44.317 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:44.315+0000 7f5963df3640 1 --2- 172.21.15.156:0/3285579526 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f5948067190 0x7f5948069640 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:51:44.317 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:44.315+0000 7f5963df3640 1 --2- 172.21.15.156:0/3285579526 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f595c0fdf50 0x7f595c113660 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:51:44.317 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:44.315+0000 7f5963df3640 1 --2- 172.21.15.156:0/3285579526 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f595c0650e0 0x7f595c113120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:51:44.317 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:44.315+0000 7f5963df3640 1 -- 172.21.15.156:0/3285579526 >> 172.21.15.156:0/3285579526 conn(0x7f595c0721c0 msgr2=0x7f595c1051d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:51:44.317 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:44.315+0000 7f5963df3640 1 -- 172.21.15.156:0/3285579526 shutdown_connections 2024-07-16T09:51:44.317 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:44.315+0000 7f5963df3640 1 -- 172.21.15.156:0/3285579526 wait complete. 2024-07-16T09:51:44.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:51:44 smithi156 ceph-mon[29302]: from='client.? 172.21.15.156:0/3285579526' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-16T09:51:45.051 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-07-16T09:50:29.025633+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2024-07-16T09:51:45.051 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 26c91c26-4358-11ef-bcad-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-07-16T09:51:45.093 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:51:44 smithi176 ceph-mon[36626]: from='client.? 172.21.15.156:0/3285579526' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-16T09:51:45.308 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/26c91c26-4358-11ef-bcad-c7b262605968/mon.smithi156/config 2024-07-16T09:51:45.682 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:51:45 smithi156 ceph-mon[29302]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-07-16T09:51:46.093 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:51:45 smithi176 ceph-mon[36626]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-07-16T09:51:46.843 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:46.840+0000 7f6563efa640 1 -- 172.21.15.156:0/1925906425 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f655c101c20 msgr2=0x7f655c0fb2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:51:46.843 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:46.840+0000 7f6563efa640 1 --2- 172.21.15.156:0/1925906425 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f655c101c20 0x7f655c0fb2b0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f65500048d0 tx=0x7f6550009da0 comp rx=0 tx=0).stop 2024-07-16T09:51:46.843 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:46.840+0000 7f6563efa640 1 -- 172.21.15.156:0/1925906425 shutdown_connections 2024-07-16T09:51:46.844 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:46.840+0000 7f6563efa640 1 --2- 172.21.15.156:0/1925906425 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f655c101c20 0x7f655c0fb2b0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:51:46.844 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:46.840+0000 7f6563efa640 1 --2- 172.21.15.156:0/1925906425 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f655c101280 0x7f655c101650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:51:46.844 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:46.840+0000 7f6563efa640 1 -- 172.21.15.156:0/1925906425 >> 172.21.15.156:0/1925906425 conn(0x7f655c0f6e70 msgr2=0x7f655c0f9280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:51:46.844 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:46.840+0000 7f6563efa640 1 -- 172.21.15.156:0/1925906425 shutdown_connections 2024-07-16T09:51:46.844 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:46.840+0000 7f6563efa640 1 -- 172.21.15.156:0/1925906425 wait complete. 2024-07-16T09:51:46.844 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:46.841+0000 7f6563efa640 1 Processor -- start 2024-07-16T09:51:46.844 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:46.841+0000 7f6563efa640 1 -- start start 2024-07-16T09:51:46.844 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:46.841+0000 7f6563efa640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f655c101280 0x7f655c0fcf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:51:46.844 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:46.841+0000 7f6563efa640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f655c101c20 0x7f655c0fd460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:51:46.845 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:46.841+0000 7f6563efa640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f655c0fd9a0 con 0x7f655c101c20 2024-07-16T09:51:46.845 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:46.841+0000 7f6563efa640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f655c0fdb10 con 0x7f655c101280 2024-07-16T09:51:46.845 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:46.841+0000 7f6561495640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f655c101c20 0x7f655c0fd460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:51:46.845 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:46.841+0000 7f6561495640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f655c101c20 0x7f655c0fd460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:37806/0 (socket says 172.21.15.156:37806) 2024-07-16T09:51:46.845 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:46.841+0000 7f6561495640 1 -- 172.21.15.156:0/3934676264 learned_addr learned my addr 172.21.15.156:0/3934676264 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T09:51:46.845 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:46.841+0000 7f6561c96640 1 --2- 172.21.15.156:0/3934676264 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f655c101280 0x7f655c0fcf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:51:46.845 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:46.842+0000 7f6561495640 1 -- 172.21.15.156:0/3934676264 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f655c101280 msgr2=0x7f655c0fcf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:51:46.845 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:46.842+0000 7f6561495640 1 --2- 172.21.15.156:0/3934676264 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f655c101280 0x7f655c0fcf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:51:46.845 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:46.842+0000 7f6561495640 1 -- 172.21.15.156:0/3934676264 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6550005040 con 0x7f655c101c20 2024-07-16T09:51:46.845 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:46.842+0000 7f6561c96640 1 --2- 172.21.15.156:0/3934676264 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f655c101280 0x7f655c0fcf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-16T09:51:46.845 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:46.842+0000 7f6561495640 1 --2- 172.21.15.156:0/3934676264 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f655c101c20 0x7f655c0fd460 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f65500043d0 tx=0x7f6550009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:51:46.845 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:46.842+0000 7f655a7fc640 1 -- 172.21.15.156:0/3934676264 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6550038070 con 0x7f655c101c20 2024-07-16T09:51:46.846 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:46.842+0000 7f655a7fc640 1 -- 172.21.15.156:0/3934676264 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6550032970 con 0x7f655c101c20 2024-07-16T09:51:46.846 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:46.842+0000 7f6563efa640 1 -- 172.21.15.156:0/3934676264 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f655c0fdd40 con 0x7f655c101c20 2024-07-16T09:51:46.846 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:46.842+0000 7f655a7fc640 1 -- 172.21.15.156:0/3934676264 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f655003da60 con 0x7f655c101c20 2024-07-16T09:51:46.846 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:46.842+0000 7f6563efa640 1 -- 172.21.15.156:0/3934676264 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f655c119730 con 0x7f655c101c20 2024-07-16T09:51:46.847 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:46.843+0000 7f6563efa640 1 -- 172.21.15.156:0/3934676264 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f655c0fc9f0 con 0x7f655c101c20 2024-07-16T09:51:46.847 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:46.843+0000 7f655a7fc640 1 -- 172.21.15.156:0/3934676264 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 19) v1 ==== 85127+0+0 (secure 0 0 0) 0x7f6550030020 con 0x7f655c101c20 2024-07-16T09:51:46.847 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:46.844+0000 7f655a7fc640 1 --2- 172.21.15.156:0/3934676264 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f6544067190 0x7f6544069640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:51:46.847 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:46.844+0000 7f655a7fc640 1 -- 172.21.15.156:0/3934676264 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f65500a48c0 con 0x7f655c101c20 2024-07-16T09:51:46.847 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:46.844+0000 7f6561c96640 1 --2- 172.21.15.156:0/3934676264 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f6544067190 0x7f6544069640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:51:46.847 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:46.844+0000 7f6561c96640 1 --2- 172.21.15.156:0/3934676264 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f6544067190 0x7f6544069640 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f654c007fc0 tx=0x7f654c004d00 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:51:46.849 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:46.848+0000 7f655a7fc640 1 -- 172.21.15.156:0/3934676264 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f6550071ea0 con 0x7f655c101c20 2024-07-16T09:51:47.012 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:47.011+0000 7f6563efa640 1 -- 172.21.15.156:0/3934676264 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f655c04b530 con 0x7f655c101c20 2024-07-16T09:51:47.012 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:47.011+0000 7f655a7fc640 1 -- 172.21.15.156:0/3934676264 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v41) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6550044300 con 0x7f655c101c20 2024-07-16T09:51:47.013 INFO:teuthology.orchestra.run.smithi156.stdout:pg_num: 1 2024-07-16T09:51:47.015 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:47.013+0000 7f6563efa640 1 -- 172.21.15.156:0/3934676264 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f6544067190 msgr2=0x7f6544069640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:51:47.015 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:47.013+0000 7f6563efa640 1 --2- 172.21.15.156:0/3934676264 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f6544067190 0x7f6544069640 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f654c007fc0 tx=0x7f654c004d00 comp rx=0 tx=0).stop 2024-07-16T09:51:47.015 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:47.013+0000 7f6563efa640 1 -- 172.21.15.156:0/3934676264 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f655c101c20 msgr2=0x7f655c0fd460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:51:47.015 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:47.013+0000 7f6563efa640 1 --2- 172.21.15.156:0/3934676264 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f655c101c20 0x7f655c0fd460 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f65500043d0 tx=0x7f6550009da0 comp rx=0 tx=0).stop 2024-07-16T09:51:47.015 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:47.013+0000 7f6563efa640 1 -- 172.21.15.156:0/3934676264 shutdown_connections 2024-07-16T09:51:47.015 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:47.013+0000 7f6563efa640 1 --2- 172.21.15.156:0/3934676264 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f6544067190 0x7f6544069640 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:51:47.015 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:47.013+0000 7f6563efa640 1 --2- 172.21.15.156:0/3934676264 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f655c101c20 0x7f655c0fd460 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:51:47.015 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:47.013+0000 7f6563efa640 1 --2- 172.21.15.156:0/3934676264 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f655c101280 0x7f655c0fcf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:51:47.015 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:47.013+0000 7f6563efa640 1 -- 172.21.15.156:0/3934676264 >> 172.21.15.156:0/3934676264 conn(0x7f655c0f6e70 msgr2=0x7f655c0f8570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:51:47.015 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:47.013+0000 7f6563efa640 1 -- 172.21.15.156:0/3934676264 shutdown_connections 2024-07-16T09:51:47.015 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:47.013+0000 7f6563efa640 1 -- 172.21.15.156:0/3934676264 wait complete. 2024-07-16T09:51:47.605 INFO:tasks.cephadm:Setting up client nodes... 2024-07-16T09:51:47.606 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 26c91c26-4358-11ef-bcad-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-07-16T09:51:47.860 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/26c91c26-4358-11ef-bcad-c7b262605968/mon.smithi156/config 2024-07-16T09:51:47.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:51:47 smithi156 ceph-mon[29302]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-07-16T09:51:47.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:51:47 smithi156 ceph-mon[29302]: from='client.? 172.21.15.156:0/3934676264' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-07-16T09:51:48.093 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:51:47 smithi176 ceph-mon[36626]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-07-16T09:51:48.093 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:51:47 smithi176 ceph-mon[36626]: from='client.? 172.21.15.156:0/3934676264' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-07-16T09:51:49.206 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:49.199+0000 7ff8c1c4b640 1 -- 172.21.15.156:0/258882082 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff8bc0fb0e0 msgr2=0x7ff8bc0fd960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:51:49.206 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:49.199+0000 7ff8c1c4b640 1 --2- 172.21.15.156:0/258882082 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff8bc0fb0e0 0x7ff8bc0fd960 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7ff8b00048d0 tx=0x7ff8b0009da0 comp rx=0 tx=0).stop 2024-07-16T09:51:49.207 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:49.199+0000 7ff8c1c4b640 1 -- 172.21.15.156:0/258882082 shutdown_connections 2024-07-16T09:51:49.207 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:49.199+0000 7ff8c1c4b640 1 --2- 172.21.15.156:0/258882082 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff8bc0fb0e0 0x7ff8bc0fd960 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:51:49.207 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:49.199+0000 7ff8c1c4b640 1 --2- 172.21.15.156:0/258882082 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff8bc0fa790 0x7ff8bc0faba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:51:49.207 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:49.199+0000 7ff8c1c4b640 1 -- 172.21.15.156:0/258882082 >> 172.21.15.156:0/258882082 conn(0x7ff8bc0f61b0 msgr2=0x7ff8bc0f8600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:51:49.207 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:49.199+0000 7ff8c1c4b640 1 -- 172.21.15.156:0/258882082 shutdown_connections 2024-07-16T09:51:49.207 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:49.199+0000 7ff8c1c4b640 1 -- 172.21.15.156:0/258882082 wait complete. 2024-07-16T09:51:49.207 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:49.200+0000 7ff8c1c4b640 1 Processor -- start 2024-07-16T09:51:49.207 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:49.200+0000 7ff8c1c4b640 1 -- start start 2024-07-16T09:51:49.207 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:49.200+0000 7ff8c1c4b640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff8bc0fa790 0x7ff8bc10d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:51:49.207 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:49.200+0000 7ff8c1c4b640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff8bc0fb0e0 0x7ff8bc10d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:51:49.207 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:49.200+0000 7ff8c1c4b640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8bc10e5d0 con 0x7ff8bc0fb0e0 2024-07-16T09:51:49.208 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:49.200+0000 7ff8c1c4b640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8bc10e740 con 0x7ff8bc0fa790 2024-07-16T09:51:49.208 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:49.200+0000 7ff8baffd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff8bc0fb0e0 0x7ff8bc10d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:51:49.208 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:49.200+0000 7ff8baffd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff8bc0fb0e0 0x7ff8bc10d760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:37818/0 (socket says 172.21.15.156:37818) 2024-07-16T09:51:49.208 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:49.200+0000 7ff8baffd640 1 -- 172.21.15.156:0/235972858 learned_addr learned my addr 172.21.15.156:0/235972858 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T09:51:49.208 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:49.201+0000 7ff8baffd640 1 -- 172.21.15.156:0/235972858 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff8bc0fa790 msgr2=0x7ff8bc10d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:51:49.208 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:49.201+0000 7ff8bb7fe640 1 --2- 172.21.15.156:0/235972858 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff8bc0fa790 0x7ff8bc10d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:51:49.208 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:49.201+0000 7ff8baffd640 1 --2- 172.21.15.156:0/235972858 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff8bc0fa790 0x7ff8bc10d220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:51:49.208 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:49.201+0000 7ff8baffd640 1 -- 172.21.15.156:0/235972858 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8b0005040 con 0x7ff8bc0fb0e0 2024-07-16T09:51:49.208 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:49.201+0000 7ff8bb7fe640 1 --2- 172.21.15.156:0/235972858 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff8bc0fa790 0x7ff8bc10d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-16T09:51:49.208 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:49.201+0000 7ff8baffd640 1 --2- 172.21.15.156:0/235972858 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff8bc0fb0e0 0x7ff8bc10d760 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7ff8b0000c00 tx=0x7ff8b0009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:51:49.208 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:49.201+0000 7ff8a3fff640 1 -- 172.21.15.156:0/235972858 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff8b0038070 con 0x7ff8bc0fb0e0 2024-07-16T09:51:49.209 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:49.201+0000 7ff8c1c4b640 1 -- 172.21.15.156:0/235972858 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8bc10e9c0 con 0x7ff8bc0fb0e0 2024-07-16T09:51:49.209 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:49.201+0000 7ff8a3fff640 1 -- 172.21.15.156:0/235972858 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff8b0032970 con 0x7ff8bc0fb0e0 2024-07-16T09:51:49.209 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:49.201+0000 7ff8a3fff640 1 -- 172.21.15.156:0/235972858 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff8b0033950 con 0x7ff8bc0fb0e0 2024-07-16T09:51:49.209 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:49.201+0000 7ff8c1c4b640 1 -- 172.21.15.156:0/235972858 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8bc10ee80 con 0x7ff8bc0fb0e0 2024-07-16T09:51:49.209 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:49.202+0000 7ff8c1c4b640 1 -- 172.21.15.156:0/235972858 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8bc0f9e70 con 0x7ff8bc0fb0e0 2024-07-16T09:51:49.209 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:49.202+0000 7ff8a3fff640 1 -- 172.21.15.156:0/235972858 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 19) v1 ==== 85127+0+0 (secure 0 0 0) 0x7ff8b0030020 con 0x7ff8bc0fb0e0 2024-07-16T09:51:49.209 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:49.203+0000 7ff8a3fff640 1 --2- 172.21.15.156:0/235972858 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7ff8a4067190 0x7ff8a4069640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:51:49.209 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:49.203+0000 7ff8bb7fe640 1 --2- 172.21.15.156:0/235972858 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7ff8a4067190 0x7ff8a4069640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:51:49.209 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:49.203+0000 7ff8a3fff640 1 -- 172.21.15.156:0/235972858 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7ff8b00a4860 con 0x7ff8bc0fb0e0 2024-07-16T09:51:49.209 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:49.203+0000 7ff8bb7fe640 1 --2- 172.21.15.156:0/235972858 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7ff8a4067190 0x7ff8a4069640 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7ff8ac006fd0 tx=0x7ff8ac008040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:51:49.210 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:49.207+0000 7ff8a3fff640 1 -- 172.21.15.156:0/235972858 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7ff8b0071e40 con 0x7ff8bc0fb0e0 2024-07-16T09:51:49.441 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:49.439+0000 7ff8c1c4b640 1 -- 172.21.15.156:0/235972858 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7ff8bc1051a0 con 0x7ff8bc0fb0e0 2024-07-16T09:51:49.445 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:49.443+0000 7ff8a3fff640 1 -- 172.21.15.156:0/235972858 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v14) v1 ==== 170+0+59 (secure 0 0 0) 0x7ff8b0041050 con 0x7ff8bc0fb0e0 2024-07-16T09:51:49.445 INFO:teuthology.orchestra.run.smithi156.stdout:[client.0] 2024-07-16T09:51:49.445 INFO:teuthology.orchestra.run.smithi156.stdout: key = AQC1QpZmTDBMGhAAAOIzc+9k/V2EbSOWnN/C/g== 2024-07-16T09:51:49.447 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:49.445+0000 7ff8c1c4b640 1 -- 172.21.15.156:0/235972858 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7ff8a4067190 msgr2=0x7ff8a4069640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:51:49.447 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:49.445+0000 7ff8c1c4b640 1 --2- 172.21.15.156:0/235972858 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7ff8a4067190 0x7ff8a4069640 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7ff8ac006fd0 tx=0x7ff8ac008040 comp rx=0 tx=0).stop 2024-07-16T09:51:49.447 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:49.445+0000 7ff8c1c4b640 1 -- 172.21.15.156:0/235972858 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff8bc0fb0e0 msgr2=0x7ff8bc10d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:51:49.447 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:49.445+0000 7ff8c1c4b640 1 --2- 172.21.15.156:0/235972858 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff8bc0fb0e0 0x7ff8bc10d760 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7ff8b0000c00 tx=0x7ff8b0009da0 comp rx=0 tx=0).stop 2024-07-16T09:51:49.447 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:49.446+0000 7ff8c1c4b640 1 -- 172.21.15.156:0/235972858 shutdown_connections 2024-07-16T09:51:49.447 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:49.446+0000 7ff8c1c4b640 1 --2- 172.21.15.156:0/235972858 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7ff8a4067190 0x7ff8a4069640 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:51:49.447 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:49.446+0000 7ff8c1c4b640 1 --2- 172.21.15.156:0/235972858 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff8bc0fb0e0 0x7ff8bc10d760 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:51:49.447 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:49.446+0000 7ff8c1c4b640 1 --2- 172.21.15.156:0/235972858 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff8bc0fa790 0x7ff8bc10d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:51:49.448 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:49.446+0000 7ff8c1c4b640 1 -- 172.21.15.156:0/235972858 >> 172.21.15.156:0/235972858 conn(0x7ff8bc0f61b0 msgr2=0x7ff8bc0f7890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:51:49.448 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:49.446+0000 7ff8c1c4b640 1 -- 172.21.15.156:0/235972858 shutdown_connections 2024-07-16T09:51:49.448 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:49.446+0000 7ff8c1c4b640 1 -- 172.21.15.156:0/235972858 wait complete. 2024-07-16T09:51:50.052 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:51:49 smithi156 ceph-mon[29302]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-07-16T09:51:50.052 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:51:49 smithi156 ceph-mon[29302]: from='client.? 172.21.15.156:0/235972858' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-07-16T09:51:50.053 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:51:49 smithi156 ceph-mon[29302]: from='client.? 172.21.15.156:0/235972858' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-07-16T09:51:50.078 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2024-07-16T09:51:50.078 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-07-16T09:51:50.078 DEBUG:teuthology.orchestra.run.smithi156:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-07-16T09:51:50.093 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:51:49 smithi176 ceph-mon[36626]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-07-16T09:51:50.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:51:49 smithi176 ceph-mon[36626]: from='client.? 172.21.15.156:0/235972858' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-07-16T09:51:50.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:51:49 smithi176 ceph-mon[36626]: from='client.? 172.21.15.156:0/235972858' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-07-16T09:51:50.116 DEBUG:teuthology.orchestra.run.smithi176:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 26c91c26-4358-11ef-bcad-c7b262605968 -- ceph auth get-or-create client.1 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-07-16T09:51:50.380 INFO:teuthology.orchestra.run.smithi176.stderr:Inferring config /var/lib/ceph/26c91c26-4358-11ef-bcad-c7b262605968/mon.smithi176/config 2024-07-16T09:51:51.789 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:51:51.782+0000 7efda1a54640 1 -- 172.21.15.176:0/3669065406 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7efd9c06fdf0 msgr2=0x7efd9c070260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:51:51.789 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:51:51.782+0000 7efda1a54640 1 --2- 172.21.15.176:0/3669065406 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7efd9c06fdf0 0x7efd9c070260 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7efd900048d0 tx=0x7efd90009da0 comp rx=0 tx=0).stop 2024-07-16T09:51:51.789 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:51:51.783+0000 7efda1a54640 1 -- 172.21.15.176:0/3669065406 shutdown_connections 2024-07-16T09:51:51.790 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:51:51.783+0000 7efda1a54640 1 --2- 172.21.15.176:0/3669065406 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7efd9c06fdf0 0x7efd9c070260 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:51:51.790 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:51:51.783+0000 7efda1a54640 1 --2- 172.21.15.176:0/3669065406 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efd9c071380 0x7efd9c06f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:51:51.790 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:51:51.783+0000 7efda1a54640 1 -- 172.21.15.176:0/3669065406 >> 172.21.15.176:0/3669065406 conn(0x7efd9c0f7b20 msgr2=0x7efd9c0f9f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:51:51.790 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:51:51.783+0000 7efda1a54640 1 -- 172.21.15.176:0/3669065406 shutdown_connections 2024-07-16T09:51:51.790 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:51:51.783+0000 7efda1a54640 1 -- 172.21.15.176:0/3669065406 wait complete. 2024-07-16T09:51:51.790 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:51:51.783+0000 7efda1a54640 1 Processor -- start 2024-07-16T09:51:51.790 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:51:51.783+0000 7efda1a54640 1 -- start start 2024-07-16T09:51:51.790 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:51:51.783+0000 7efda1a54640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7efd9c06fdf0 0x7efd9c10f150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:51:51.790 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:51:51.783+0000 7efda1a54640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efd9c071380 0x7efd9c10f690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:51:51.790 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:51:51.783+0000 7efda1a54640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd9c10fce0 con 0x7efd9c071380 2024-07-16T09:51:51.790 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:51:51.783+0000 7efda1a54640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd9c110500 con 0x7efd9c06fdf0 2024-07-16T09:51:51.790 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:51:51.784+0000 7efd9affd640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7efd9c06fdf0 0x7efd9c10f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:51:51.791 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:51:51.784+0000 7efd9affd640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7efd9c06fdf0 0x7efd9c10f150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:46514/0 (socket says 172.21.15.176:46514) 2024-07-16T09:51:51.791 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:51:51.784+0000 7efd9affd640 1 -- 172.21.15.176:0/1978263158 learned_addr learned my addr 172.21.15.176:0/1978263158 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-07-16T09:51:51.791 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:51:51.784+0000 7efd9a7fc640 1 --2- 172.21.15.176:0/1978263158 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efd9c071380 0x7efd9c10f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:51:51.791 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:51:51.784+0000 7efd9affd640 1 -- 172.21.15.176:0/1978263158 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efd9c071380 msgr2=0x7efd9c10f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:51:51.791 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:51:51.784+0000 7efd9affd640 1 --2- 172.21.15.176:0/1978263158 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efd9c071380 0x7efd9c10f690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:51:51.791 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:51:51.784+0000 7efd9affd640 1 -- 172.21.15.176:0/1978263158 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd90005040 con 0x7efd9c06fdf0 2024-07-16T09:51:51.791 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:51:51.784+0000 7efd9affd640 1 --2- 172.21.15.176:0/1978263158 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7efd9c06fdf0 0x7efd9c10f150 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7efd8c00c810 tx=0x7efd8c004430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:51:51.791 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:51:51.784+0000 7efd837fe640 1 -- 172.21.15.176:0/1978263158 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efd8c00a850 con 0x7efd9c06fdf0 2024-07-16T09:51:51.791 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:51:51.784+0000 7efda1a54640 1 -- 172.21.15.176:0/1978263158 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd9c110790 con 0x7efd9c06fdf0 2024-07-16T09:51:51.791 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:51:51.784+0000 7efd837fe640 1 -- 172.21.15.176:0/1978263158 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7efd8c00ae80 con 0x7efd9c06fdf0 2024-07-16T09:51:51.791 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:51:51.784+0000 7efd837fe640 1 -- 172.21.15.176:0/1978263158 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efd8c0115e0 con 0x7efd9c06fdf0 2024-07-16T09:51:51.791 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:51:51.784+0000 7efda1a54640 1 -- 172.21.15.176:0/1978263158 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd9c110d30 con 0x7efd9c06fdf0 2024-07-16T09:51:51.792 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:51:51.785+0000 7efda1a54640 1 -- 172.21.15.176:0/1978263158 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd9c0fb7a0 con 0x7efd9c06fdf0 2024-07-16T09:51:51.792 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:51:51.785+0000 7efd837fe640 1 -- 172.21.15.176:0/1978263158 <== mon.1 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 19) v1 ==== 85127+0+0 (secure 0 0 0) 0x7efd8c011740 con 0x7efd9c06fdf0 2024-07-16T09:51:51.792 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:51:51.786+0000 7efd837fe640 1 --2- 172.21.15.176:0/1978263158 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7efd84067190 0x7efd84069640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:51:51.792 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:51:51.786+0000 7efd837fe640 1 -- 172.21.15.176:0/1978263158 <== mon.1 v2:172.21.15.176:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7efd8c0849f0 con 0x7efd9c06fdf0 2024-07-16T09:51:51.792 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:51:51.786+0000 7efd9a7fc640 1 --2- 172.21.15.176:0/1978263158 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7efd84067190 0x7efd84069640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:51:51.792 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:51:51.787+0000 7efd9a7fc640 1 --2- 172.21.15.176:0/1978263158 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7efd84067190 0x7efd84069640 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7efd90009f80 tx=0x7efd90035040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:51:51.792 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:51:51.790+0000 7efd837fe640 1 -- 172.21.15.176:0/1978263158 <== mon.1 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7efd8c086e60 con 0x7efd9c06fdf0 2024-07-16T09:51:51.843 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:51:51 smithi176 ceph-mon[36626]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-07-16T09:51:52.019 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:51:52.018+0000 7efda1a54640 1 -- 172.21.15.176:0/1978263158 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7efd9c04b530 con 0x7efd9c06fdf0 2024-07-16T09:51:52.024 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:51:52.023+0000 7efd837fe640 1 -- 172.21.15.176:0/1978263158 <== mon.1 v2:172.21.15.176:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v15) v1 ==== 170+0+59 (secure 0 0 0) 0x7efd8c0521a0 con 0x7efd9c06fdf0 2024-07-16T09:51:52.024 INFO:teuthology.orchestra.run.smithi176.stdout:[client.1] 2024-07-16T09:51:52.024 INFO:teuthology.orchestra.run.smithi176.stdout: key = AQC4QpZmPTw3ARAAc253nzdP2sbFlwExx+tIig== 2024-07-16T09:51:52.026 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:51:52.025+0000 7efda1a54640 1 -- 172.21.15.176:0/1978263158 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7efd84067190 msgr2=0x7efd84069640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:51:52.026 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:51:52.025+0000 7efda1a54640 1 --2- 172.21.15.176:0/1978263158 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7efd84067190 0x7efd84069640 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7efd90009f80 tx=0x7efd90035040 comp rx=0 tx=0).stop 2024-07-16T09:51:52.026 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:51:52.025+0000 7efda1a54640 1 -- 172.21.15.176:0/1978263158 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7efd9c06fdf0 msgr2=0x7efd9c10f150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:51:52.026 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:51:52.025+0000 7efda1a54640 1 --2- 172.21.15.176:0/1978263158 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7efd9c06fdf0 0x7efd9c10f150 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7efd8c00c810 tx=0x7efd8c004430 comp rx=0 tx=0).stop 2024-07-16T09:51:52.026 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:51:52.025+0000 7efda1a54640 1 -- 172.21.15.176:0/1978263158 shutdown_connections 2024-07-16T09:51:52.027 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:51:52.025+0000 7efda1a54640 1 --2- 172.21.15.176:0/1978263158 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7efd84067190 0x7efd84069640 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:51:52.027 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:51:52.025+0000 7efda1a54640 1 --2- 172.21.15.176:0/1978263158 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efd9c071380 0x7efd9c10f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:51:52.027 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:51:52.025+0000 7efda1a54640 1 --2- 172.21.15.176:0/1978263158 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7efd9c06fdf0 0x7efd9c10f150 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:51:52.027 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:51:52.025+0000 7efda1a54640 1 -- 172.21.15.176:0/1978263158 >> 172.21.15.176:0/1978263158 conn(0x7efd9c0f7b20 msgr2=0x7efd9c0f9200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:51:52.027 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:51:52.025+0000 7efda1a54640 1 -- 172.21.15.176:0/1978263158 shutdown_connections 2024-07-16T09:51:52.027 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:51:52.025+0000 7efda1a54640 1 -- 172.21.15.176:0/1978263158 wait complete. 2024-07-16T09:51:52.174 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:51:51 smithi156 ceph-mon[29302]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-07-16T09:51:52.605 DEBUG:teuthology.orchestra.run.smithi176:> set -ex 2024-07-16T09:51:52.605 DEBUG:teuthology.orchestra.run.smithi176:> sudo dd of=/etc/ceph/ceph.client.1.keyring 2024-07-16T09:51:52.605 DEBUG:teuthology.orchestra.run.smithi176:> sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-07-16T09:51:52.645 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-07-16T09:51:52.645 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-07-16T09:51:52.645 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 26c91c26-4358-11ef-bcad-c7b262605968 -- ceph mgr dump --format=json 2024-07-16T09:51:52.913 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/26c91c26-4358-11ef-bcad-c7b262605968/mon.smithi156/config 2024-07-16T09:51:52.924 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:51:52 smithi156 ceph-mon[29302]: from='client.? 172.21.15.176:0/1978263158' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-07-16T09:51:52.924 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:51:52 smithi156 ceph-mon[29302]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-07-16T09:51:52.924 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:51:52 smithi156 ceph-mon[29302]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-07-16T09:51:53.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:51:52 smithi176 ceph-mon[36626]: from='client.? 172.21.15.176:0/1978263158' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-07-16T09:51:53.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:51:52 smithi176 ceph-mon[36626]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-07-16T09:51:53.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:51:52 smithi176 ceph-mon[36626]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-07-16T09:51:54.326 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:51:54 smithi156 ceph-mon[29302]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-07-16T09:51:54.343 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:51:54 smithi176 ceph-mon[36626]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-07-16T09:51:54.540 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:54.537+0000 7f852b89a640 1 -- 172.21.15.156:0/2607825680 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f85240faa70 msgr2=0x7f85240faec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:51:54.541 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:54.537+0000 7f852b89a640 1 --2- 172.21.15.156:0/2607825680 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f85240faa70 0x7f85240faec0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f85140048d0 tx=0x7f8514009da0 comp rx=0 tx=0).stop 2024-07-16T09:51:54.541 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:54.537+0000 7f852b89a640 1 -- 172.21.15.156:0/2607825680 shutdown_connections 2024-07-16T09:51:54.541 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:54.537+0000 7f852b89a640 1 --2- 172.21.15.156:0/2607825680 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f85240faa70 0x7f85240faec0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:51:54.541 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:54.537+0000 7f852b89a640 1 --2- 172.21.15.156:0/2607825680 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8524100a80 0x7f8524100e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:51:54.541 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:54.537+0000 7f852b89a640 1 -- 172.21.15.156:0/2607825680 >> 172.21.15.156:0/2607825680 conn(0x7f8524070a90 msgr2=0x7f8524070e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:51:54.541 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:54.537+0000 7f852b89a640 1 -- 172.21.15.156:0/2607825680 shutdown_connections 2024-07-16T09:51:54.541 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:54.537+0000 7f852b89a640 1 -- 172.21.15.156:0/2607825680 wait complete. 2024-07-16T09:51:54.541 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:54.538+0000 7f852b89a640 1 Processor -- start 2024-07-16T09:51:54.541 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:54.538+0000 7f852b89a640 1 -- start start 2024-07-16T09:51:54.542 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:54.538+0000 7f852b89a640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f85240faa70 0x7f85240fc630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:51:54.542 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:54.538+0000 7f852b89a640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8524100a80 0x7f85240fcb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:51:54.542 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:54.538+0000 7f852b89a640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85240fd0b0 con 0x7f85240faa70 2024-07-16T09:51:54.542 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:54.538+0000 7f852b89a640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85240fd220 con 0x7f8524100a80 2024-07-16T09:51:54.542 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:54.538+0000 7f8529636640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f85240faa70 0x7f85240fc630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:51:54.542 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:54.538+0000 7f8529636640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f85240faa70 0x7f85240fc630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:54534/0 (socket says 172.21.15.156:54534) 2024-07-16T09:51:54.542 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:54.538+0000 7f8529636640 1 -- 172.21.15.156:0/1880431952 learned_addr learned my addr 172.21.15.156:0/1880431952 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T09:51:54.542 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:54.538+0000 7f8528e35640 1 --2- 172.21.15.156:0/1880431952 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8524100a80 0x7f85240fcb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:51:54.542 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:54.538+0000 7f8529636640 1 -- 172.21.15.156:0/1880431952 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8524100a80 msgr2=0x7f85240fcb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:51:54.542 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:54.538+0000 7f8529636640 1 --2- 172.21.15.156:0/1880431952 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8524100a80 0x7f85240fcb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:51:54.542 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:54.538+0000 7f8529636640 1 -- 172.21.15.156:0/1880431952 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8514005040 con 0x7f85240faa70 2024-07-16T09:51:54.543 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:54.539+0000 7f8529636640 1 --2- 172.21.15.156:0/1880431952 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f85240faa70 0x7f85240fc630 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f851c00c830 tx=0x7f851c004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:51:54.543 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:54.539+0000 7f8519ffb640 1 -- 172.21.15.156:0/1880431952 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f851c00a7d0 con 0x7f85240faa70 2024-07-16T09:51:54.543 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:54.539+0000 7f8519ffb640 1 -- 172.21.15.156:0/1880431952 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f851c00ae00 con 0x7f85240faa70 2024-07-16T09:51:54.543 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:54.539+0000 7f852b89a640 1 -- 172.21.15.156:0/1880431952 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85240fd4b0 con 0x7f85240faa70 2024-07-16T09:51:54.543 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:54.539+0000 7f8519ffb640 1 -- 172.21.15.156:0/1880431952 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f851c0113a0 con 0x7f85240faa70 2024-07-16T09:51:54.543 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:54.539+0000 7f852b89a640 1 -- 172.21.15.156:0/1880431952 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85241197d0 con 0x7f85240faa70 2024-07-16T09:51:54.544 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:54.540+0000 7f852b89a640 1 -- 172.21.15.156:0/1880431952 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85240fc160 con 0x7f85240faa70 2024-07-16T09:51:54.544 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:54.540+0000 7f8519ffb640 1 -- 172.21.15.156:0/1880431952 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 19) v1 ==== 85127+0+0 (secure 0 0 0) 0x7f851c012400 con 0x7f85240faa70 2024-07-16T09:51:54.544 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:54.541+0000 7f8519ffb640 1 --2- 172.21.15.156:0/1880431952 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f8510067190 0x7f8510069640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:51:54.544 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:54.541+0000 7f8519ffb640 1 -- 172.21.15.156:0/1880431952 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f851c0848d0 con 0x7f85240faa70 2024-07-16T09:51:54.544 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:54.541+0000 7f8528e35640 1 --2- 172.21.15.156:0/1880431952 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f8510067190 0x7f8510069640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:51:54.544 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:54.541+0000 7f8528e35640 1 --2- 172.21.15.156:0/1880431952 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f8510067190 0x7f8510069640 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f8514009f80 tx=0x7f8514035040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:51:54.546 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:54.545+0000 7f8519ffb640 1 -- 172.21.15.156:0/1880431952 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f851c051f30 con 0x7f85240faa70 2024-07-16T09:51:54.754 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:54.752+0000 7f852b89a640 1 -- 172.21.15.156:0/1880431952 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f8524074a30 con 0x7f85240faa70 2024-07-16T09:51:54.756 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:54.755+0000 7f8519ffb640 1 -- 172.21.15.156:0/1880431952 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v19) v1 ==== 74+0+164114 (secure 0 0 0) 0x7f851c055550 con 0x7f85240faa70 2024-07-16T09:51:54.759 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-07-16T09:51:54.760 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:54.757+0000 7f852b89a640 1 -- 172.21.15.156:0/1880431952 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f8510067190 msgr2=0x7f8510069640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:51:54.760 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:54.757+0000 7f852b89a640 1 --2- 172.21.15.156:0/1880431952 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f8510067190 0x7f8510069640 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f8514009f80 tx=0x7f8514035040 comp rx=0 tx=0).stop 2024-07-16T09:51:54.760 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:54.757+0000 7f852b89a640 1 -- 172.21.15.156:0/1880431952 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f85240faa70 msgr2=0x7f85240fc630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:51:54.760 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:54.758+0000 7f852b89a640 1 --2- 172.21.15.156:0/1880431952 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f85240faa70 0x7f85240fc630 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f851c00c830 tx=0x7f851c004430 comp rx=0 tx=0).stop 2024-07-16T09:51:54.761 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:54.758+0000 7f852b89a640 1 -- 172.21.15.156:0/1880431952 shutdown_connections 2024-07-16T09:51:54.761 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:54.758+0000 7f852b89a640 1 --2- 172.21.15.156:0/1880431952 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f8510067190 0x7f8510069640 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:51:54.761 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:54.758+0000 7f852b89a640 1 --2- 172.21.15.156:0/1880431952 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8524100a80 0x7f85240fcb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:51:54.761 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:54.758+0000 7f852b89a640 1 --2- 172.21.15.156:0/1880431952 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f85240faa70 0x7f85240fc630 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:51:54.761 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:54.758+0000 7f852b89a640 1 -- 172.21.15.156:0/1880431952 >> 172.21.15.156:0/1880431952 conn(0x7f8524070a90 msgr2=0x7f85240f8f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:51:54.761 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:54.758+0000 7f852b89a640 1 -- 172.21.15.156:0/1880431952 shutdown_connections 2024-07-16T09:51:54.761 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:54.758+0000 7f852b89a640 1 -- 172.21.15.156:0/1880431952 wait complete. 2024-07-16T09:51:55.343 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:51:55 smithi176 ceph-mon[36626]: from='client.? 172.21.15.156:0/1880431952' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-07-16T09:51:55.345 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:51:55 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-16T09:51:55.424 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:51:55 smithi156 ceph-mon[29302]: from='client.? 172.21.15.156:0/1880431952' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-07-16T09:51:55.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:51:55 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-16T09:51:55.457 INFO:teuthology.orchestra.run.smithi156.stdout:{"epoch":19,"flags":0,"active_gid":14208,"active_name":"smithi156.zkvylv","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":2615741717},{"type":"v1","addr":"172.21.15.156:6801","nonce":2615741717}]},"active_addr":"172.21.15.156:6801/2615741717","active_change":"2024-07-16T09:48:24.722701+0000","active_mgr_features":4540138320759226367,"available":true,"standbys":[{"gid":14226,"name":"smithi176.dqbrvx","mgr_features":4540138320759226367,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","prometheus","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.156:8443/","prometheus":"http://172.21.15.156:9283/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":5,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.156:0","nonce":211214934}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.156:0","nonce":3148313406}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.156:0","nonce":590617513}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.156:0","nonce":3579914066}]}]} 2024-07-16T09:51:55.461 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-07-16T09:51:55.461 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-07-16T09:51:55.461 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 26c91c26-4358-11ef-bcad-c7b262605968 -- ceph osd dump --format=json 2024-07-16T09:51:55.713 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/26c91c26-4358-11ef-bcad-c7b262605968/mon.smithi156/config 2024-07-16T09:51:56.111 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:51:56 smithi156 ceph-mon[29302]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-07-16T09:51:56.343 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:51:56 smithi176 ceph-mon[36626]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-07-16T09:51:57.301 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:57.287+0000 7f58bc0dd640 1 -- 172.21.15.156:0/123348686 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f58b40fdf60 msgr2=0x7f58b40fe390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:51:57.301 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:57.287+0000 7f58bc0dd640 1 --2- 172.21.15.156:0/123348686 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f58b40fdf60 0x7f58b40fe390 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f58a80048d0 tx=0x7f58a8009da0 comp rx=0 tx=0).stop 2024-07-16T09:51:57.301 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:57.287+0000 7f58bc0dd640 1 -- 172.21.15.156:0/123348686 shutdown_connections 2024-07-16T09:51:57.302 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:57.287+0000 7f58bc0dd640 1 --2- 172.21.15.156:0/123348686 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f58b40fdf60 0x7f58b40fe390 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:51:57.302 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:57.287+0000 7f58bc0dd640 1 --2- 172.21.15.156:0/123348686 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f58b4065150 0x7f58b4065520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:51:57.302 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:57.287+0000 7f58bc0dd640 1 -- 172.21.15.156:0/123348686 >> 172.21.15.156:0/123348686 conn(0x7f58b40721c0 msgr2=0x7f58b40725c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:51:57.302 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:57.287+0000 7f58bc0dd640 1 -- 172.21.15.156:0/123348686 shutdown_connections 2024-07-16T09:51:57.302 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:57.287+0000 7f58bc0dd640 1 -- 172.21.15.156:0/123348686 wait complete. 2024-07-16T09:51:57.302 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:57.287+0000 7f58bc0dd640 1 Processor -- start 2024-07-16T09:51:57.302 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:57.287+0000 7f58bc0dd640 1 -- start start 2024-07-16T09:51:57.302 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:57.288+0000 7f58bc0dd640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f58b4065150 0x7f58b4115330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:51:57.302 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:57.288+0000 7f58bc0dd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f58b40fdf60 0x7f58b4115870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:51:57.302 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:57.288+0000 7f58bc0dd640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58b4115f00 con 0x7f58b40fdf60 2024-07-16T09:51:57.302 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:57.288+0000 7f58bc0dd640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58b410f2b0 con 0x7f58b4065150 2024-07-16T09:51:57.303 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:57.288+0000 7f58b9678640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f58b40fdf60 0x7f58b4115870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:51:57.303 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:57.288+0000 7f58b9678640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f58b40fdf60 0x7f58b4115870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:54560/0 (socket says 172.21.15.156:54560) 2024-07-16T09:51:57.303 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:57.288+0000 7f58b9678640 1 -- 172.21.15.156:0/2179276999 learned_addr learned my addr 172.21.15.156:0/2179276999 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T09:51:57.303 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:57.288+0000 7f58b9e79640 1 --2- 172.21.15.156:0/2179276999 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f58b4065150 0x7f58b4115330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:51:57.303 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:57.288+0000 7f58b9678640 1 -- 172.21.15.156:0/2179276999 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f58b4065150 msgr2=0x7f58b4115330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:51:57.303 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:57.288+0000 7f58b9678640 1 --2- 172.21.15.156:0/2179276999 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f58b4065150 0x7f58b4115330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:51:57.303 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:57.288+0000 7f58b9678640 1 -- 172.21.15.156:0/2179276999 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58a8005040 con 0x7f58b40fdf60 2024-07-16T09:51:57.303 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:57.288+0000 7f58b9e79640 1 --2- 172.21.15.156:0/2179276999 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f58b4065150 0x7f58b4115330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-16T09:51:57.303 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:57.288+0000 7f58b9678640 1 --2- 172.21.15.156:0/2179276999 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f58b40fdf60 0x7f58b4115870 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f58a8003080 tx=0x7f58a8009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:51:57.303 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:57.288+0000 7f58a67fc640 1 -- 172.21.15.156:0/2179276999 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f58a8038070 con 0x7f58b40fdf60 2024-07-16T09:51:57.303 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:57.288+0000 7f58a67fc640 1 -- 172.21.15.156:0/2179276999 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f58a8008ee0 con 0x7f58b40fdf60 2024-07-16T09:51:57.303 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:57.288+0000 7f58bc0dd640 1 -- 172.21.15.156:0/2179276999 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58b410f530 con 0x7f58b40fdf60 2024-07-16T09:51:57.303 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:57.288+0000 7f58a67fc640 1 -- 172.21.15.156:0/2179276999 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f58a803d870 con 0x7f58b40fdf60 2024-07-16T09:51:57.304 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:57.288+0000 7f58bc0dd640 1 -- 172.21.15.156:0/2179276999 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58b410f9f0 con 0x7f58b40fdf60 2024-07-16T09:51:57.304 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:57.289+0000 7f58bc0dd640 1 -- 172.21.15.156:0/2179276999 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58b40fdb00 con 0x7f58b40fdf60 2024-07-16T09:51:57.304 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:57.290+0000 7f58a67fc640 1 -- 172.21.15.156:0/2179276999 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 19) v1 ==== 85127+0+0 (secure 0 0 0) 0x7f58a8030020 con 0x7f58b40fdf60 2024-07-16T09:51:57.304 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:57.290+0000 7f58a67fc640 1 --2- 172.21.15.156:0/2179276999 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f58a0067190 0x7f58a0069640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:51:57.304 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:57.290+0000 7f58a67fc640 1 -- 172.21.15.156:0/2179276999 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f58a80a4550 con 0x7f58b40fdf60 2024-07-16T09:51:57.304 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:57.290+0000 7f58b9e79640 1 --2- 172.21.15.156:0/2179276999 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f58a0067190 0x7f58a0069640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:51:57.304 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:57.291+0000 7f58b9e79640 1 --2- 172.21.15.156:0/2179276999 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f58a0067190 0x7f58a0069640 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f58b00099f0 tx=0x7f58b0008040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:51:57.304 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:57.294+0000 7f58a67fc640 1 -- 172.21.15.156:0/2179276999 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f58a8071d00 con 0x7f58b40fdf60 2024-07-16T09:51:57.462 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:57.461+0000 7f58bc0dd640 1 -- 172.21.15.156:0/2179276999 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f58b41109d0 con 0x7f58b40fdf60 2024-07-16T09:51:57.463 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:57.462+0000 7f58a67fc640 1 -- 172.21.15.156:0/2179276999 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v41) v1 ==== 74+0+11124 (secure 0 0 0) 0x7f58a8075320 con 0x7f58b40fdf60 2024-07-16T09:51:57.463 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-07-16T09:51:57.463 INFO:teuthology.orchestra.run.smithi156.stdout:{"epoch":41,"fsid":"26c91c26-4358-11ef-bcad-c7b262605968","created":"2024-07-16T09:46:23.833980+0000","modified":"2024-07-16T09:51:39.649731+0000","last_up_change":"2024-07-16T09:51:38.648169+0000","last_in_change":"2024-07-16T09:51:20.573206+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":14,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-16T09:50:29.025633+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"38938d0f-f41d-4d70-9f59-f977181e2943","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":36114952},{"type":"v1","addr":"172.21.15.156:6803","nonce":36114952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":36114952},{"type":"v1","addr":"172.21.15.156:6805","nonce":36114952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":36114952},{"type":"v1","addr":"172.21.15.156:6809","nonce":36114952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":36114952},{"type":"v1","addr":"172.21.15.156:6807","nonce":36114952}]},"public_addr":"172.21.15.156:6803/36114952","cluster_addr":"172.21.15.156:6805/36114952","heartbeat_back_addr":"172.21.15.156:6809/36114952","heartbeat_front_addr":"172.21.15.156:6807/36114952","state":["exists","up"]},{"osd":1,"uuid":"04d30aff-25e2-4b87-927d-a2497d0123b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":1587604616},{"type":"v1","addr":"172.21.15.156:6811","nonce":1587604616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":1587604616},{"type":"v1","addr":"172.21.15.156:6813","nonce":1587604616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":1587604616},{"type":"v1","addr":"172.21.15.156:6817","nonce":1587604616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":1587604616},{"type":"v1","addr":"172.21.15.156:6815","nonce":1587604616}]},"public_addr":"172.21.15.156:6811/1587604616","cluster_addr":"172.21.15.156:6813/1587604616","heartbeat_back_addr":"172.21.15.156:6817/1587604616","heartbeat_front_addr":"172.21.15.156:6815/1587604616","state":["exists","up"]},{"osd":2,"uuid":"5b2cee4f-d3da-4dac-85b9-ee34914e548a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":3710668239},{"type":"v1","addr":"172.21.15.156:6819","nonce":3710668239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":3710668239},{"type":"v1","addr":"172.21.15.156:6821","nonce":3710668239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":3710668239},{"type":"v1","addr":"172.21.15.156:6825","nonce":3710668239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":3710668239},{"type":"v1","addr":"172.21.15.156:6823","nonce":3710668239}]},"public_addr":"172.21.15.156:6819/3710668239","cluster_addr":"172.21.15.156:6821/3710668239","heartbeat_back_addr":"172.21.15.156:6825/3710668239","heartbeat_front_addr":"172.21.15.156:6823/3710668239","state":["exists","up"]},{"osd":3,"uuid":"8428be96-686f-4318-9585-4b8a19a7b9c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6800","nonce":1740426750},{"type":"v1","addr":"172.21.15.176:6801","nonce":1740426750}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6802","nonce":1740426750},{"type":"v1","addr":"172.21.15.176:6803","nonce":1740426750}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6806","nonce":1740426750},{"type":"v1","addr":"172.21.15.176:6807","nonce":1740426750}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6804","nonce":1740426750},{"type":"v1","addr":"172.21.15.176:6805","nonce":1740426750}]},"public_addr":"172.21.15.176:6801/1740426750","cluster_addr":"172.21.15.176:6803/1740426750","heartbeat_back_addr":"172.21.15.176:6807/1740426750","heartbeat_front_addr":"172.21.15.176:6805/1740426750","state":["exists","up"]},{"osd":4,"uuid":"fd311fe4-8853-4403-a3a8-863eb22a58ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6808","nonce":2702155727},{"type":"v1","addr":"172.21.15.176:6809","nonce":2702155727}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6810","nonce":2702155727},{"type":"v1","addr":"172.21.15.176:6811","nonce":2702155727}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6814","nonce":2702155727},{"type":"v1","addr":"172.21.15.176:6815","nonce":2702155727}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6812","nonce":2702155727},{"type":"v1","addr":"172.21.15.176:6813","nonce":2702155727}]},"public_addr":"172.21.15.176:6809/2702155727","cluster_addr":"172.21.15.176:6811/2702155727","heartbeat_back_addr":"172.21.15.176:6815/2702155727","heartbeat_front_addr":"172.21.15.176:6813/2702155727","state":["exists","up"]},{"osd":5,"uuid":"79a38eb4-c031-4a53-8588-a4e598eeb3ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6816","nonce":2358804089},{"type":"v1","addr":"172.21.15.176:6817","nonce":2358804089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6818","nonce":2358804089},{"type":"v1","addr":"172.21.15.176:6819","nonce":2358804089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6822","nonce":2358804089},{"type":"v1","addr":"172.21.15.176:6823","nonce":2358804089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6820","nonce":2358804089},{"type":"v1","addr":"172.21.15.176:6821","nonce":2358804089}]},"public_addr":"172.21.15.176:6817/2358804089","cluster_addr":"172.21.15.176:6819/2358804089","heartbeat_back_addr":"172.21.15.176:6823/2358804089","heartbeat_front_addr":"172.21.15.176:6821/2358804089","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-07-16T09:49:32.966257+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-07-16T09:49:59.906579+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-07-16T09:50:25.246162+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-07-16T09:50:47.932851+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-07-16T09:51:12.299573+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-07-16T09:51:36.971019+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.156:6801/3984987611":"2024-07-17T09:48:24.722579+0000","172.21.15.156:0/3487646818":"2024-07-17T09:46:53.642082+0000","172.21.15.156:6801/2871070852":"2024-07-17T09:46:53.642082+0000","172.21.15.156:6800/2871070852":"2024-07-17T09:46:53.642082+0000","172.21.15.156:0/3390422489":"2024-07-17T09:46:53.642082+0000","172.21.15.156:0/4158213614":"2024-07-17T09:46:53.642082+0000","172.21.15.156:6800/1023000918":"2024-07-17T09:47:22.566842+0000","172.21.15.156:6800/3984987611":"2024-07-17T09:48:24.722579+0000","172.21.15.156:0/1864977086":"2024-07-17T09:47:22.566842+0000","172.21.15.156:0/2120399071":"2024-07-17T09:47:22.566842+0000","172.21.15.156:0/2320156657":"2024-07-17T09:48:24.722579+0000","172.21.15.156:6801/1023000918":"2024-07-17T09:47:22.566842+0000","172.21.15.156:0/2322505969":"2024-07-17T09:47:22.566842+0000","172.21.15.156:0/1325014720":"2024-07-17T09:48:24.722579+0000","172.21.15.156:0/3191234605":"2024-07-17T09:48:24.722579+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-16T09:51:57.465 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:57.464+0000 7f58bc0dd640 1 -- 172.21.15.156:0/2179276999 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f58a0067190 msgr2=0x7f58a0069640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:51:57.465 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:57.464+0000 7f58bc0dd640 1 --2- 172.21.15.156:0/2179276999 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f58a0067190 0x7f58a0069640 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f58b00099f0 tx=0x7f58b0008040 comp rx=0 tx=0).stop 2024-07-16T09:51:57.465 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:57.464+0000 7f58bc0dd640 1 -- 172.21.15.156:0/2179276999 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f58b40fdf60 msgr2=0x7f58b4115870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:51:57.465 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:57.464+0000 7f58bc0dd640 1 --2- 172.21.15.156:0/2179276999 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f58b40fdf60 0x7f58b4115870 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f58a8003080 tx=0x7f58a8009da0 comp rx=0 tx=0).stop 2024-07-16T09:51:57.465 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:57.464+0000 7f58bc0dd640 1 -- 172.21.15.156:0/2179276999 shutdown_connections 2024-07-16T09:51:57.466 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:57.464+0000 7f58bc0dd640 1 --2- 172.21.15.156:0/2179276999 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f58a0067190 0x7f58a0069640 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:51:57.466 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:57.464+0000 7f58bc0dd640 1 --2- 172.21.15.156:0/2179276999 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f58b40fdf60 0x7f58b4115870 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:51:57.466 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:57.464+0000 7f58bc0dd640 1 --2- 172.21.15.156:0/2179276999 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f58b4065150 0x7f58b4115330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:51:57.466 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:57.464+0000 7f58bc0dd640 1 -- 172.21.15.156:0/2179276999 >> 172.21.15.156:0/2179276999 conn(0x7f58b40721c0 msgr2=0x7f58b40f86f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:51:57.466 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:57.464+0000 7f58bc0dd640 1 -- 172.21.15.156:0/2179276999 shutdown_connections 2024-07-16T09:51:57.466 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:57.464+0000 7f58bc0dd640 1 -- 172.21.15.156:0/2179276999 wait complete. 2024-07-16T09:51:58.039 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-07-16T09:51:58.039 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 26c91c26-4358-11ef-bcad-c7b262605968 -- ceph osd dump --format=json 2024-07-16T09:51:58.295 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/26c91c26-4358-11ef-bcad-c7b262605968/mon.smithi156/config 2024-07-16T09:51:58.343 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:51:58 smithi176 ceph-mon[36626]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-07-16T09:51:58.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:51:58 smithi176 ceph-mon[36626]: from='client.? 172.21.15.156:0/2179276999' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-16T09:51:58.417 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:51:58 smithi156 ceph-mon[29302]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-07-16T09:51:58.418 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:51:58 smithi156 ceph-mon[29302]: from='client.? 172.21.15.156:0/2179276999' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-16T09:51:59.901 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:59.899+0000 7f54a95f3640 1 -- 172.21.15.156:0/4091705970 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f54a40fba30 msgr2=0x7f54a4108790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:51:59.901 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:59.899+0000 7f54a95f3640 1 --2- 172.21.15.156:0/4091705970 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f54a40fba30 0x7f54a4108790 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f54980048d0 tx=0x7f5498009da0 comp rx=0 tx=0).stop 2024-07-16T09:51:59.902 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:59.899+0000 7f54a95f3640 1 -- 172.21.15.156:0/4091705970 shutdown_connections 2024-07-16T09:51:59.902 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:59.899+0000 7f54a95f3640 1 --2- 172.21.15.156:0/4091705970 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f54a40fba30 0x7f54a4108790 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:51:59.902 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:59.899+0000 7f54a95f3640 1 --2- 172.21.15.156:0/4091705970 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f54a40fb120 0x7f54a40fb4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:51:59.902 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:59.899+0000 7f54a95f3640 1 -- 172.21.15.156:0/4091705970 >> 172.21.15.156:0/4091705970 conn(0x7f54a40f6e30 msgr2=0x7f54a40f9240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:51:59.902 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:59.899+0000 7f54a95f3640 1 -- 172.21.15.156:0/4091705970 shutdown_connections 2024-07-16T09:51:59.902 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:59.899+0000 7f54a95f3640 1 -- 172.21.15.156:0/4091705970 wait complete. 2024-07-16T09:51:59.902 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:59.899+0000 7f54a95f3640 1 Processor -- start 2024-07-16T09:51:59.902 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:59.899+0000 7f54a95f3640 1 -- start start 2024-07-16T09:51:59.902 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:59.900+0000 7f54a95f3640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f54a40fb120 0x7f54a41152e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:51:59.902 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:59.900+0000 7f54a95f3640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f54a40fba30 0x7f54a4115820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:51:59.902 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:59.900+0000 7f54a95f3640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54a4115e20 con 0x7f54a40fba30 2024-07-16T09:51:59.903 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:59.900+0000 7f54a95f3640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54a410f260 con 0x7f54a40fb120 2024-07-16T09:51:59.903 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:59.900+0000 7f54a27fc640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f54a40fba30 0x7f54a4115820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:51:59.903 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:59.900+0000 7f54a27fc640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f54a40fba30 0x7f54a4115820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:54584/0 (socket says 172.21.15.156:54584) 2024-07-16T09:51:59.903 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:59.900+0000 7f54a27fc640 1 -- 172.21.15.156:0/2880803012 learned_addr learned my addr 172.21.15.156:0/2880803012 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T09:51:59.903 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:59.900+0000 7f54a2ffd640 1 --2- 172.21.15.156:0/2880803012 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f54a40fb120 0x7f54a41152e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:51:59.903 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:59.900+0000 7f54a27fc640 1 -- 172.21.15.156:0/2880803012 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f54a40fb120 msgr2=0x7f54a41152e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:51:59.903 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:59.900+0000 7f54a27fc640 1 --2- 172.21.15.156:0/2880803012 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f54a40fb120 0x7f54a41152e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:51:59.903 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:59.900+0000 7f54a27fc640 1 -- 172.21.15.156:0/2880803012 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5498005040 con 0x7f54a40fba30 2024-07-16T09:51:59.903 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:59.900+0000 7f54a27fc640 1 --2- 172.21.15.156:0/2880803012 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f54a40fba30 0x7f54a4115820 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f54980048a0 tx=0x7f5498009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:51:59.904 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:59.900+0000 7f548b7fe640 1 -- 172.21.15.156:0/2880803012 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5498038070 con 0x7f54a40fba30 2024-07-16T09:51:59.904 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:59.900+0000 7f548b7fe640 1 -- 172.21.15.156:0/2880803012 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f549802abd0 con 0x7f54a40fba30 2024-07-16T09:51:59.904 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:59.900+0000 7f54a95f3640 1 -- 172.21.15.156:0/2880803012 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54a410f4e0 con 0x7f54a40fba30 2024-07-16T09:51:59.904 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:59.901+0000 7f548b7fe640 1 -- 172.21.15.156:0/2880803012 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5498033770 con 0x7f54a40fba30 2024-07-16T09:51:59.904 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:59.901+0000 7f54a95f3640 1 -- 172.21.15.156:0/2880803012 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54a410fa20 con 0x7f54a40fba30 2024-07-16T09:51:59.905 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:59.902+0000 7f54a95f3640 1 -- 172.21.15.156:0/2880803012 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54a4066140 con 0x7f54a40fba30 2024-07-16T09:51:59.905 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:59.902+0000 7f548b7fe640 1 -- 172.21.15.156:0/2880803012 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 19) v1 ==== 85127+0+0 (secure 0 0 0) 0x7f5498030020 con 0x7f54a40fba30 2024-07-16T09:51:59.905 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:59.902+0000 7f548b7fe640 1 --2- 172.21.15.156:0/2880803012 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f548c067190 0x7f548c069640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:51:59.905 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:59.902+0000 7f548b7fe640 1 -- 172.21.15.156:0/2880803012 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f54980a4460 con 0x7f54a40fba30 2024-07-16T09:51:59.905 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:59.902+0000 7f54a2ffd640 1 --2- 172.21.15.156:0/2880803012 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f548c067190 0x7f548c069640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:51:59.905 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:59.903+0000 7f54a2ffd640 1 --2- 172.21.15.156:0/2880803012 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f548c067190 0x7f548c069640 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f5494004ed0 tx=0x7f549400b040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:51:59.908 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:51:59.907+0000 7f548b7fe640 1 -- 172.21.15.156:0/2880803012 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f5498071af0 con 0x7f54a40fba30 2024-07-16T09:52:00.075 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:00.074+0000 7f54a95f3640 1 -- 172.21.15.156:0/2880803012 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f54a410b690 con 0x7f54a40fba30 2024-07-16T09:52:00.076 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:00.075+0000 7f548b7fe640 1 -- 172.21.15.156:0/2880803012 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v41) v1 ==== 74+0+11124 (secure 0 0 0) 0x7f5498075110 con 0x7f54a40fba30 2024-07-16T09:52:00.076 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-07-16T09:52:00.076 INFO:teuthology.orchestra.run.smithi156.stdout:{"epoch":41,"fsid":"26c91c26-4358-11ef-bcad-c7b262605968","created":"2024-07-16T09:46:23.833980+0000","modified":"2024-07-16T09:51:39.649731+0000","last_up_change":"2024-07-16T09:51:38.648169+0000","last_in_change":"2024-07-16T09:51:20.573206+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":14,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-16T09:50:29.025633+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"38938d0f-f41d-4d70-9f59-f977181e2943","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":36114952},{"type":"v1","addr":"172.21.15.156:6803","nonce":36114952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":36114952},{"type":"v1","addr":"172.21.15.156:6805","nonce":36114952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":36114952},{"type":"v1","addr":"172.21.15.156:6809","nonce":36114952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":36114952},{"type":"v1","addr":"172.21.15.156:6807","nonce":36114952}]},"public_addr":"172.21.15.156:6803/36114952","cluster_addr":"172.21.15.156:6805/36114952","heartbeat_back_addr":"172.21.15.156:6809/36114952","heartbeat_front_addr":"172.21.15.156:6807/36114952","state":["exists","up"]},{"osd":1,"uuid":"04d30aff-25e2-4b87-927d-a2497d0123b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":1587604616},{"type":"v1","addr":"172.21.15.156:6811","nonce":1587604616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":1587604616},{"type":"v1","addr":"172.21.15.156:6813","nonce":1587604616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":1587604616},{"type":"v1","addr":"172.21.15.156:6817","nonce":1587604616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":1587604616},{"type":"v1","addr":"172.21.15.156:6815","nonce":1587604616}]},"public_addr":"172.21.15.156:6811/1587604616","cluster_addr":"172.21.15.156:6813/1587604616","heartbeat_back_addr":"172.21.15.156:6817/1587604616","heartbeat_front_addr":"172.21.15.156:6815/1587604616","state":["exists","up"]},{"osd":2,"uuid":"5b2cee4f-d3da-4dac-85b9-ee34914e548a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":3710668239},{"type":"v1","addr":"172.21.15.156:6819","nonce":3710668239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":3710668239},{"type":"v1","addr":"172.21.15.156:6821","nonce":3710668239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":3710668239},{"type":"v1","addr":"172.21.15.156:6825","nonce":3710668239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":3710668239},{"type":"v1","addr":"172.21.15.156:6823","nonce":3710668239}]},"public_addr":"172.21.15.156:6819/3710668239","cluster_addr":"172.21.15.156:6821/3710668239","heartbeat_back_addr":"172.21.15.156:6825/3710668239","heartbeat_front_addr":"172.21.15.156:6823/3710668239","state":["exists","up"]},{"osd":3,"uuid":"8428be96-686f-4318-9585-4b8a19a7b9c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6800","nonce":1740426750},{"type":"v1","addr":"172.21.15.176:6801","nonce":1740426750}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6802","nonce":1740426750},{"type":"v1","addr":"172.21.15.176:6803","nonce":1740426750}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6806","nonce":1740426750},{"type":"v1","addr":"172.21.15.176:6807","nonce":1740426750}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6804","nonce":1740426750},{"type":"v1","addr":"172.21.15.176:6805","nonce":1740426750}]},"public_addr":"172.21.15.176:6801/1740426750","cluster_addr":"172.21.15.176:6803/1740426750","heartbeat_back_addr":"172.21.15.176:6807/1740426750","heartbeat_front_addr":"172.21.15.176:6805/1740426750","state":["exists","up"]},{"osd":4,"uuid":"fd311fe4-8853-4403-a3a8-863eb22a58ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6808","nonce":2702155727},{"type":"v1","addr":"172.21.15.176:6809","nonce":2702155727}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6810","nonce":2702155727},{"type":"v1","addr":"172.21.15.176:6811","nonce":2702155727}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6814","nonce":2702155727},{"type":"v1","addr":"172.21.15.176:6815","nonce":2702155727}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6812","nonce":2702155727},{"type":"v1","addr":"172.21.15.176:6813","nonce":2702155727}]},"public_addr":"172.21.15.176:6809/2702155727","cluster_addr":"172.21.15.176:6811/2702155727","heartbeat_back_addr":"172.21.15.176:6815/2702155727","heartbeat_front_addr":"172.21.15.176:6813/2702155727","state":["exists","up"]},{"osd":5,"uuid":"79a38eb4-c031-4a53-8588-a4e598eeb3ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6816","nonce":2358804089},{"type":"v1","addr":"172.21.15.176:6817","nonce":2358804089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6818","nonce":2358804089},{"type":"v1","addr":"172.21.15.176:6819","nonce":2358804089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6822","nonce":2358804089},{"type":"v1","addr":"172.21.15.176:6823","nonce":2358804089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6820","nonce":2358804089},{"type":"v1","addr":"172.21.15.176:6821","nonce":2358804089}]},"public_addr":"172.21.15.176:6817/2358804089","cluster_addr":"172.21.15.176:6819/2358804089","heartbeat_back_addr":"172.21.15.176:6823/2358804089","heartbeat_front_addr":"172.21.15.176:6821/2358804089","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-07-16T09:49:32.966257+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-07-16T09:49:59.906579+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-07-16T09:50:25.246162+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-07-16T09:50:47.932851+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-07-16T09:51:12.299573+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-07-16T09:51:36.971019+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.156:6801/3984987611":"2024-07-17T09:48:24.722579+0000","172.21.15.156:0/3487646818":"2024-07-17T09:46:53.642082+0000","172.21.15.156:6801/2871070852":"2024-07-17T09:46:53.642082+0000","172.21.15.156:6800/2871070852":"2024-07-17T09:46:53.642082+0000","172.21.15.156:0/3390422489":"2024-07-17T09:46:53.642082+0000","172.21.15.156:0/4158213614":"2024-07-17T09:46:53.642082+0000","172.21.15.156:6800/1023000918":"2024-07-17T09:47:22.566842+0000","172.21.15.156:6800/3984987611":"2024-07-17T09:48:24.722579+0000","172.21.15.156:0/1864977086":"2024-07-17T09:47:22.566842+0000","172.21.15.156:0/2120399071":"2024-07-17T09:47:22.566842+0000","172.21.15.156:0/2320156657":"2024-07-17T09:48:24.722579+0000","172.21.15.156:6801/1023000918":"2024-07-17T09:47:22.566842+0000","172.21.15.156:0/2322505969":"2024-07-17T09:47:22.566842+0000","172.21.15.156:0/1325014720":"2024-07-17T09:48:24.722579+0000","172.21.15.156:0/3191234605":"2024-07-17T09:48:24.722579+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-16T09:52:00.078 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:00.077+0000 7f54a95f3640 1 -- 172.21.15.156:0/2880803012 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f548c067190 msgr2=0x7f548c069640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:52:00.078 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:00.077+0000 7f54a95f3640 1 --2- 172.21.15.156:0/2880803012 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f548c067190 0x7f548c069640 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f5494004ed0 tx=0x7f549400b040 comp rx=0 tx=0).stop 2024-07-16T09:52:00.078 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:00.077+0000 7f54a95f3640 1 -- 172.21.15.156:0/2880803012 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f54a40fba30 msgr2=0x7f54a4115820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:52:00.078 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:00.077+0000 7f54a95f3640 1 --2- 172.21.15.156:0/2880803012 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f54a40fba30 0x7f54a4115820 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f54980048a0 tx=0x7f5498009da0 comp rx=0 tx=0).stop 2024-07-16T09:52:00.079 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:00.077+0000 7f54a95f3640 1 -- 172.21.15.156:0/2880803012 shutdown_connections 2024-07-16T09:52:00.079 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:00.077+0000 7f54a95f3640 1 --2- 172.21.15.156:0/2880803012 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f548c067190 0x7f548c069640 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:52:00.079 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:00.077+0000 7f54a95f3640 1 --2- 172.21.15.156:0/2880803012 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f54a40fba30 0x7f54a4115820 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:52:00.079 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:00.077+0000 7f54a95f3640 1 --2- 172.21.15.156:0/2880803012 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f54a40fb120 0x7f54a41152e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:52:00.079 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:00.077+0000 7f54a95f3640 1 -- 172.21.15.156:0/2880803012 >> 172.21.15.156:0/2880803012 conn(0x7f54a40f6e30 msgr2=0x7f54a40f7220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:52:00.079 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:00.077+0000 7f54a95f3640 1 -- 172.21.15.156:0/2880803012 shutdown_connections 2024-07-16T09:52:00.079 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:00.077+0000 7f54a95f3640 1 -- 172.21.15.156:0/2880803012 wait complete. 2024-07-16T09:52:00.343 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:52:00 smithi176 ceph-mon[36626]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-07-16T09:52:00.376 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:52:00 smithi156 ceph-mon[29302]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-07-16T09:52:00.643 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 26c91c26-4358-11ef-bcad-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-07-16T09:52:00.644 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 26c91c26-4358-11ef-bcad-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-07-16T09:52:00.644 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 26c91c26-4358-11ef-bcad-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-07-16T09:52:00.645 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 26c91c26-4358-11ef-bcad-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-07-16T09:52:00.645 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 26c91c26-4358-11ef-bcad-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-07-16T09:52:00.646 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 26c91c26-4358-11ef-bcad-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-07-16T09:52:00.961 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/26c91c26-4358-11ef-bcad-c7b262605968/mon.smithi156/config 2024-07-16T09:52:00.963 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/26c91c26-4358-11ef-bcad-c7b262605968/mon.smithi156/config 2024-07-16T09:52:01.343 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:52:01 smithi176 ceph-mon[36626]: from='client.? 172.21.15.156:0/2880803012' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-16T09:52:01.355 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:52:01 smithi156 ceph-mon[29302]: from='client.? 172.21.15.156:0/2880803012' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-16T09:52:02.343 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:52:02 smithi176 ceph-mon[36626]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-07-16T09:52:02.399 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/26c91c26-4358-11ef-bcad-c7b262605968/mon.smithi156/config 2024-07-16T09:52:02.400 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/26c91c26-4358-11ef-bcad-c7b262605968/mon.smithi156/config 2024-07-16T09:52:02.400 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/26c91c26-4358-11ef-bcad-c7b262605968/mon.smithi156/config 2024-07-16T09:52:02.424 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:52:02 smithi156 ceph-mon[29302]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-07-16T09:52:03.061 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/26c91c26-4358-11ef-bcad-c7b262605968/mon.smithi156/config 2024-07-16T09:52:04.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:52:04 smithi176 ceph-mon[36626]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-07-16T09:52:04.373 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:52:04 smithi156 ceph-mon[29302]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-07-16T09:52:05.766 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:05.747+0000 7f46b36ca640 1 -- 172.21.15.156:0/2928705196 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f46ac102cf0 msgr2=0x7f46ac0fc380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:52:05.766 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:05.747+0000 7f46b36ca640 1 --2- 172.21.15.156:0/2928705196 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f46ac102cf0 0x7f46ac0fc380 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f469c0048d0 tx=0x7f469c009da0 comp rx=0 tx=0).stop 2024-07-16T09:52:05.767 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:05.747+0000 7f46b36ca640 1 -- 172.21.15.156:0/2928705196 shutdown_connections 2024-07-16T09:52:05.767 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:05.747+0000 7f46b36ca640 1 --2- 172.21.15.156:0/2928705196 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f46ac102cf0 0x7f46ac0fc380 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:52:05.767 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:05.747+0000 7f46b36ca640 1 --2- 172.21.15.156:0/2928705196 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f46ac102350 0x7f46ac102720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:52:05.767 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:05.747+0000 7f46b36ca640 1 -- 172.21.15.156:0/2928705196 >> 172.21.15.156:0/2928705196 conn(0x7f46ac0f7f00 msgr2=0x7f46ac0fa310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:52:05.767 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:05.747+0000 7f46b36ca640 1 -- 172.21.15.156:0/2928705196 shutdown_connections 2024-07-16T09:52:05.767 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:05.747+0000 7f46b36ca640 1 -- 172.21.15.156:0/2928705196 wait complete. 2024-07-16T09:52:05.767 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:05.748+0000 7f46b36ca640 1 Processor -- start 2024-07-16T09:52:05.767 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:05.748+0000 7f46b36ca640 1 -- start start 2024-07-16T09:52:05.767 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:05.748+0000 7f46b36ca640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f46ac102350 0x7f46ac1130a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:52:05.767 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:05.748+0000 7f46b36ca640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f46ac102cf0 0x7f46ac1135e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:52:05.767 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:05.748+0000 7f46b36ca640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46ac113be0 con 0x7f46ac102350 2024-07-16T09:52:05.767 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:05.748+0000 7f46b36ca640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46ac10d020 con 0x7f46ac102cf0 2024-07-16T09:52:05.767 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:05.748+0000 7f46b1466640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f46ac102350 0x7f46ac1130a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:52:05.768 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:05.748+0000 7f46b1466640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f46ac102350 0x7f46ac1130a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:44598/0 (socket says 172.21.15.156:44598) 2024-07-16T09:52:05.768 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:05.748+0000 7f46b1466640 1 -- 172.21.15.156:0/3563477314 learned_addr learned my addr 172.21.15.156:0/3563477314 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T09:52:05.768 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:05.748+0000 7f46b0c65640 1 --2- 172.21.15.156:0/3563477314 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f46ac102cf0 0x7f46ac1135e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:52:05.768 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:05.748+0000 7f46b1466640 1 -- 172.21.15.156:0/3563477314 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f46ac102cf0 msgr2=0x7f46ac1135e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:52:05.768 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:05.748+0000 7f46b1466640 1 --2- 172.21.15.156:0/3563477314 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f46ac102cf0 0x7f46ac1135e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:52:05.768 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:05.748+0000 7f46b1466640 1 -- 172.21.15.156:0/3563477314 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f469c005040 con 0x7f46ac102350 2024-07-16T09:52:05.768 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:05.748+0000 7f46b0c65640 1 --2- 172.21.15.156:0/3563477314 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f46ac102cf0 0x7f46ac1135e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-16T09:52:05.768 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:05.749+0000 7f46b1466640 1 --2- 172.21.15.156:0/3563477314 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f46ac102350 0x7f46ac1130a0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f46a8002a50 tx=0x7f46a8004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:52:05.768 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:05.749+0000 7f46a1ffb640 1 -- 172.21.15.156:0/3563477314 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f46a8007c30 con 0x7f46ac102350 2024-07-16T09:52:05.768 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:05.749+0000 7f46b36ca640 1 -- 172.21.15.156:0/3563477314 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46ac10d300 con 0x7f46ac102350 2024-07-16T09:52:05.768 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:05.749+0000 7f46a1ffb640 1 -- 172.21.15.156:0/3563477314 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f46a800f040 con 0x7f46ac102350 2024-07-16T09:52:05.768 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:05.749+0000 7f46a1ffb640 1 -- 172.21.15.156:0/3563477314 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f46a800dbe0 con 0x7f46ac102350 2024-07-16T09:52:05.768 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:05.749+0000 7f46b36ca640 1 -- 172.21.15.156:0/3563477314 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46ac10d8a0 con 0x7f46ac102350 2024-07-16T09:52:05.769 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:05.750+0000 7f46b36ca640 1 -- 172.21.15.156:0/3563477314 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f46ac10dd80 con 0x7f46ac102350 2024-07-16T09:52:05.769 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:05.750+0000 7f46a1ffb640 1 -- 172.21.15.156:0/3563477314 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 19) v1 ==== 85127+0+0 (secure 0 0 0) 0x7f46a800dd40 con 0x7f46ac102350 2024-07-16T09:52:05.769 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:05.751+0000 7f46a1ffb640 1 --2- 172.21.15.156:0/3563477314 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f4698067190 0x7f4698069640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:52:05.769 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:05.751+0000 7f46b0c65640 1 --2- 172.21.15.156:0/3563477314 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f4698067190 0x7f4698069640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:52:05.769 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:05.751+0000 7f46a1ffb640 1 -- 172.21.15.156:0/3563477314 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f46a8084790 con 0x7f46ac102350 2024-07-16T09:52:05.769 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:05.751+0000 7f46a1ffb640 1 --2- 172.21.15.156:0/3563477314 >> [v2:172.21.15.156:6810/1587604616,v1:172.21.15.156:6811/1587604616] conn(0x7f469806cf10 0x7f469806f320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:52:05.769 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:05.751+0000 7f46a1ffb640 1 -- 172.21.15.156:0/3563477314 --> [v2:172.21.15.156:6810/1587604616,v1:172.21.15.156:6811/1587604616] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f469806f9d0 con 0x7f469806cf10 2024-07-16T09:52:05.769 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:05.751+0000 7f46a1ffb640 1 -- 172.21.15.156:0/3563477314 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f46a8089220 con 0x7f46ac102350 2024-07-16T09:52:05.769 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:05.751+0000 7f46b1c67640 1 --2- 172.21.15.156:0/3563477314 >> [v2:172.21.15.156:6810/1587604616,v1:172.21.15.156:6811/1587604616] conn(0x7f469806cf10 0x7f469806f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:52:05.769 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:05.751+0000 7f46b0c65640 1 --2- 172.21.15.156:0/3563477314 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f4698067190 0x7f4698069640 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f46ac10e960 tx=0x7f469c035040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:52:05.769 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:05.751+0000 7f46b1c67640 1 --2- 172.21.15.156:0/3563477314 >> [v2:172.21.15.156:6810/1587604616,v1:172.21.15.156:6811/1587604616] conn(0x7f469806cf10 0x7f469806f320 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:52:05.769 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:05.752+0000 7f46a1ffb640 1 -- 172.21.15.156:0/3563477314 <== osd.1 v2:172.21.15.156:6810/1587604616 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7f469806f9d0 con 0x7f469806cf10 2024-07-16T09:52:05.769 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:05.765+0000 7f46b36ca640 1 -- 172.21.15.156:0/3563477314 --> [v2:172.21.15.156:6810/1587604616,v1:172.21.15.156:6811/1587604616] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f46ac0fdac0 con 0x7f469806cf10 2024-07-16T09:52:05.770 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:05.765+0000 7f46a1ffb640 1 -- 172.21.15.156:0/3563477314 <== osd.1 v2:172.21.15.156:6810/1587604616 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f46ac0fdac0 con 0x7f469806cf10 2024-07-16T09:52:05.770 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:05.765+0000 7f46b36ca640 1 -- 172.21.15.156:0/3563477314 >> [v2:172.21.15.156:6810/1587604616,v1:172.21.15.156:6811/1587604616] conn(0x7f469806cf10 msgr2=0x7f469806f320 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:52:05.770 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:05.765+0000 7f46b36ca640 1 --2- 172.21.15.156:0/3563477314 >> [v2:172.21.15.156:6810/1587604616,v1:172.21.15.156:6811/1587604616] conn(0x7f469806cf10 0x7f469806f320 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:52:05.771 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:05.766+0000 7f46b36ca640 1 -- 172.21.15.156:0/3563477314 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f4698067190 msgr2=0x7f4698069640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:52:05.771 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:05.766+0000 7f46b36ca640 1 --2- 172.21.15.156:0/3563477314 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f4698067190 0x7f4698069640 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f46ac10e960 tx=0x7f469c035040 comp rx=0 tx=0).stop 2024-07-16T09:52:05.771 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:05.766+0000 7f46b36ca640 1 -- 172.21.15.156:0/3563477314 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f46ac102350 msgr2=0x7f46ac1130a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:52:05.771 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:05.766+0000 7f46b36ca640 1 --2- 172.21.15.156:0/3563477314 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f46ac102350 0x7f46ac1130a0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f46a8002a50 tx=0x7f46a8004600 comp rx=0 tx=0).stop 2024-07-16T09:52:05.771 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:05.766+0000 7f46b36ca640 1 -- 172.21.15.156:0/3563477314 shutdown_connections 2024-07-16T09:52:05.771 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:05.766+0000 7f46b36ca640 1 --2- 172.21.15.156:0/3563477314 >> [v2:172.21.15.156:6810/1587604616,v1:172.21.15.156:6811/1587604616] conn(0x7f469806cf10 0x7f469806f320 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:52:05.771 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:05.766+0000 7f46b36ca640 1 --2- 172.21.15.156:0/3563477314 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f4698067190 0x7f4698069640 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:52:05.771 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:05.766+0000 7f46b36ca640 1 --2- 172.21.15.156:0/3563477314 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f46ac102cf0 0x7f46ac1135e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:52:05.771 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:05.766+0000 7f46b36ca640 1 --2- 172.21.15.156:0/3563477314 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f46ac102350 0x7f46ac1130a0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:52:05.771 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:05.766+0000 7f46b36ca640 1 -- 172.21.15.156:0/3563477314 >> 172.21.15.156:0/3563477314 conn(0x7f46ac0f7f00 msgr2=0x7f46ac0f8e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:52:05.771 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:05.766+0000 7f46b36ca640 1 -- 172.21.15.156:0/3563477314 shutdown_connections 2024-07-16T09:52:05.771 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:05.766+0000 7f46b36ca640 1 -- 172.21.15.156:0/3563477314 wait complete. 2024-07-16T09:52:06.343 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:52:06 smithi176 ceph-mon[36626]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-07-16T09:52:06.420 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:52:06 smithi156 ceph-mon[29302]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-07-16T09:52:06.446 INFO:teuthology.orchestra.run.smithi156.stdout:60129542172 2024-07-16T09:52:06.446 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 26c91c26-4358-11ef-bcad-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-07-16T09:52:07.763 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:07.756+0000 7f78b37d6640 1 -- 172.21.15.156:0/1169830437 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f78ac102350 msgr2=0x7f78ac102720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:52:07.764 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:07.756+0000 7f78b37d6640 1 --2- 172.21.15.156:0/1169830437 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f78ac102350 0x7f78ac102720 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f78a8009c50 tx=0x7f78a8004d10 comp rx=0 tx=0).stop 2024-07-16T09:52:07.764 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:07.756+0000 7f78b37d6640 1 -- 172.21.15.156:0/1169830437 shutdown_connections 2024-07-16T09:52:07.764 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:07.756+0000 7f78b37d6640 1 --2- 172.21.15.156:0/1169830437 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f78ac102cf0 0x7f78ac0fc380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:52:07.764 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:07.756+0000 7f78b37d6640 1 --2- 172.21.15.156:0/1169830437 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f78ac102350 0x7f78ac102720 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:52:07.764 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:07.756+0000 7f78b37d6640 1 -- 172.21.15.156:0/1169830437 >> 172.21.15.156:0/1169830437 conn(0x7f78ac0f7f00 msgr2=0x7f78ac0fa310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:52:07.764 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:07.757+0000 7f78b37d6640 1 -- 172.21.15.156:0/1169830437 shutdown_connections 2024-07-16T09:52:07.765 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:07.757+0000 7f78b37d6640 1 -- 172.21.15.156:0/1169830437 wait complete. 2024-07-16T09:52:07.765 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:07.757+0000 7f78b37d6640 1 Processor -- start 2024-07-16T09:52:07.765 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:07.757+0000 7f78b37d6640 1 -- start start 2024-07-16T09:52:07.765 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:07.757+0000 7f78b37d6640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f78ac102350 0x7f78ac06f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:52:07.765 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:07.757+0000 7f78b37d6640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f78ac102cf0 0x7f78ac06fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:52:07.765 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:07.757+0000 7f78b37d6640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78ac0702a0 con 0x7f78ac102350 2024-07-16T09:52:07.766 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:07.757+0000 7f78b37d6640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78ac070410 con 0x7f78ac102cf0 2024-07-16T09:52:07.766 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:07.757+0000 7f78b1572640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f78ac102350 0x7f78ac06f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:52:07.766 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:07.757+0000 7f78b1572640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f78ac102350 0x7f78ac06f820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:44620/0 (socket says 172.21.15.156:44620) 2024-07-16T09:52:07.766 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:07.757+0000 7f78b1572640 1 -- 172.21.15.156:0/1566180425 learned_addr learned my addr 172.21.15.156:0/1566180425 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T09:52:07.767 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:07.758+0000 7f78b0d71640 1 --2- 172.21.15.156:0/1566180425 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f78ac102cf0 0x7f78ac06fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:52:07.767 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:07.758+0000 7f78b1572640 1 -- 172.21.15.156:0/1566180425 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f78ac102cf0 msgr2=0x7f78ac06fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:52:07.767 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:07.758+0000 7f78b1572640 1 --2- 172.21.15.156:0/1566180425 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f78ac102cf0 0x7f78ac06fd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:52:07.767 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:07.758+0000 7f78b1572640 1 -- 172.21.15.156:0/1566180425 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78a80098b0 con 0x7f78ac102350 2024-07-16T09:52:07.767 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:07.758+0000 7f78b0d71640 1 --2- 172.21.15.156:0/1566180425 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f78ac102cf0 0x7f78ac06fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-16T09:52:07.767 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:07.758+0000 7f78b1572640 1 --2- 172.21.15.156:0/1566180425 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f78ac102350 0x7f78ac06f820 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f78a8004f80 tx=0x7f78a8004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:52:07.768 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:07.758+0000 7f78a1ffb640 1 -- 172.21.15.156:0/1566180425 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f78a8007d50 con 0x7f78ac102350 2024-07-16T09:52:07.768 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:07.758+0000 7f78a1ffb640 1 -- 172.21.15.156:0/1566180425 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f78a8032c00 con 0x7f78ac102350 2024-07-16T09:52:07.768 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:07.758+0000 7f78a1ffb640 1 -- 172.21.15.156:0/1566180425 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f78a803db90 con 0x7f78ac102350 2024-07-16T09:52:07.768 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:07.758+0000 7f78b37d6640 1 -- 172.21.15.156:0/1566180425 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78ac070640 con 0x7f78ac102350 2024-07-16T09:52:07.768 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:07.758+0000 7f78b37d6640 1 -- 172.21.15.156:0/1566180425 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78ac1196b0 con 0x7f78ac102350 2024-07-16T09:52:07.769 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:07.759+0000 7f78b37d6640 1 -- 172.21.15.156:0/1566180425 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f78ac0f98d0 con 0x7f78ac102350 2024-07-16T09:52:07.769 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:07.760+0000 7f78a1ffb640 1 -- 172.21.15.156:0/1566180425 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 19) v1 ==== 85127+0+0 (secure 0 0 0) 0x7f78a802f030 con 0x7f78ac102350 2024-07-16T09:52:07.769 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:07.760+0000 7f78a1ffb640 1 --2- 172.21.15.156:0/1566180425 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f7898067190 0x7f7898069640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:52:07.769 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:07.760+0000 7f78a1ffb640 1 -- 172.21.15.156:0/1566180425 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f78a80a5900 con 0x7f78ac102350 2024-07-16T09:52:07.769 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:07.760+0000 7f78b0d71640 1 --2- 172.21.15.156:0/1566180425 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f7898067190 0x7f7898069640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:52:07.770 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:07.760+0000 7f78a1ffb640 1 --2- 172.21.15.156:0/1566180425 >> [v2:172.21.15.176:6816/2358804089,v1:172.21.15.176:6817/2358804089] conn(0x7f789806cf10 0x7f789806f320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:52:07.770 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:07.760+0000 7f78a1ffb640 1 -- 172.21.15.156:0/1566180425 --> [v2:172.21.15.176:6816/2358804089,v1:172.21.15.176:6817/2358804089] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f789806f9d0 con 0x7f789806cf10 2024-07-16T09:52:07.770 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:07.760+0000 7f78a1ffb640 1 -- 172.21.15.156:0/1566180425 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f78a80aa220 con 0x7f78ac102350 2024-07-16T09:52:07.770 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:07.760+0000 7f78b0d71640 1 --2- 172.21.15.156:0/1566180425 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f7898067190 0x7f7898069640 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f789c007fd0 tx=0x7f789c009e90 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:52:07.770 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:07.760+0000 7f78b1d73640 1 --2- 172.21.15.156:0/1566180425 >> [v2:172.21.15.176:6816/2358804089,v1:172.21.15.176:6817/2358804089] conn(0x7f789806cf10 0x7f789806f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:52:07.771 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:07.761+0000 7f78b1d73640 1 --2- 172.21.15.156:0/1566180425 >> [v2:172.21.15.176:6816/2358804089,v1:172.21.15.176:6817/2358804089] conn(0x7f789806cf10 0x7f789806f320 crc :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:52:07.771 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:07.762+0000 7f78a1ffb640 1 -- 172.21.15.156:0/1566180425 <== osd.5 v2:172.21.15.176:6816/2358804089 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7f789806f9d0 con 0x7f789806cf10 2024-07-16T09:52:07.777 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:07.775+0000 7f78b37d6640 1 -- 172.21.15.156:0/1566180425 --> [v2:172.21.15.176:6816/2358804089,v1:172.21.15.176:6817/2358804089] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f78ac0fdac0 con 0x7f789806cf10 2024-07-16T09:52:07.777 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:07.775+0000 7f78a1ffb640 1 -- 172.21.15.156:0/1566180425 <== osd.5 v2:172.21.15.176:6816/2358804089 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f78ac0fdac0 con 0x7f789806cf10 2024-07-16T09:52:07.779 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:07.776+0000 7f78b37d6640 1 -- 172.21.15.156:0/1566180425 >> [v2:172.21.15.176:6816/2358804089,v1:172.21.15.176:6817/2358804089] conn(0x7f789806cf10 msgr2=0x7f789806f320 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:52:07.779 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:07.776+0000 7f78b37d6640 1 --2- 172.21.15.156:0/1566180425 >> [v2:172.21.15.176:6816/2358804089,v1:172.21.15.176:6817/2358804089] conn(0x7f789806cf10 0x7f789806f320 crc :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:52:07.779 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:07.776+0000 7f78b37d6640 1 -- 172.21.15.156:0/1566180425 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f7898067190 msgr2=0x7f7898069640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:52:07.779 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:07.776+0000 7f78b37d6640 1 --2- 172.21.15.156:0/1566180425 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f7898067190 0x7f7898069640 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f789c007fd0 tx=0x7f789c009e90 comp rx=0 tx=0).stop 2024-07-16T09:52:07.779 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:07.776+0000 7f78b37d6640 1 -- 172.21.15.156:0/1566180425 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f78ac102350 msgr2=0x7f78ac06f820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:52:07.780 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:07.776+0000 7f78b37d6640 1 --2- 172.21.15.156:0/1566180425 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f78ac102350 0x7f78ac06f820 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f78a8004f80 tx=0x7f78a8004d10 comp rx=0 tx=0).stop 2024-07-16T09:52:07.780 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:07.776+0000 7f78b37d6640 1 -- 172.21.15.156:0/1566180425 shutdown_connections 2024-07-16T09:52:07.780 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:07.776+0000 7f78b37d6640 1 --2- 172.21.15.156:0/1566180425 >> [v2:172.21.15.176:6816/2358804089,v1:172.21.15.176:6817/2358804089] conn(0x7f789806cf10 0x7f789806f320 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:52:07.780 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:07.776+0000 7f78b37d6640 1 --2- 172.21.15.156:0/1566180425 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f7898067190 0x7f7898069640 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:52:07.780 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:07.776+0000 7f78b37d6640 1 --2- 172.21.15.156:0/1566180425 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f78ac102cf0 0x7f78ac06fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:52:07.781 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:07.776+0000 7f78b37d6640 1 --2- 172.21.15.156:0/1566180425 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f78ac102350 0x7f78ac06f820 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:52:07.781 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:07.776+0000 7f78b37d6640 1 -- 172.21.15.156:0/1566180425 >> 172.21.15.156:0/1566180425 conn(0x7f78ac0f7f00 msgr2=0x7f78ac0f8ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:52:07.781 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:07.776+0000 7f78b37d6640 1 -- 172.21.15.156:0/1566180425 shutdown_connections 2024-07-16T09:52:07.781 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:07.776+0000 7f78b37d6640 1 -- 172.21.15.156:0/1566180425 wait complete. 2024-07-16T09:52:08.174 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:52:08 smithi156 ceph-mon[29302]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-07-16T09:52:08.343 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:52:08 smithi176 ceph-mon[36626]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-07-16T09:52:09.626 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:09.531+0000 7f8bf03d1640 1 -- 172.21.15.156:0/85251686 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8be8065150 msgr2=0x7f8be8065520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:52:09.627 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:09.531+0000 7f8bf03d1640 1 --2- 172.21.15.156:0/85251686 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8be8065150 0x7f8be8065520 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f8be40048d0 tx=0x7f8be4009da0 comp rx=0 tx=0).stop 2024-07-16T09:52:09.627 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:09.531+0000 7f8bf03d1640 1 -- 172.21.15.156:0/85251686 shutdown_connections 2024-07-16T09:52:09.627 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:09.531+0000 7f8bf03d1640 1 --2- 172.21.15.156:0/85251686 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8be80fdf60 0x7f8be80fe390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:52:09.627 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:09.531+0000 7f8bf03d1640 1 --2- 172.21.15.156:0/85251686 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8be8065150 0x7f8be8065520 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:52:09.627 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:09.531+0000 7f8bf03d1640 1 -- 172.21.15.156:0/85251686 >> 172.21.15.156:0/85251686 conn(0x7f8be80721c0 msgr2=0x7f8be80725c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:52:09.627 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:09.531+0000 7f8bf03d1640 1 -- 172.21.15.156:0/85251686 shutdown_connections 2024-07-16T09:52:09.627 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:09.531+0000 7f8bf03d1640 1 -- 172.21.15.156:0/85251686 wait complete. 2024-07-16T09:52:09.627 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:09.532+0000 7f8bf03d1640 1 Processor -- start 2024-07-16T09:52:09.627 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:09.532+0000 7f8bf03d1640 1 -- start start 2024-07-16T09:52:09.627 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:09.532+0000 7f8bf03d1640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8be8065150 0x7f8be8113130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:52:09.628 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:09.532+0000 7f8bf03d1640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8be80fdf60 0x7f8be8113670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:52:09.628 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:09.532+0000 7f8bf03d1640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8be8113d00 con 0x7f8be80fdf60 2024-07-16T09:52:09.628 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:09.532+0000 7f8bf03d1640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8be810d0b0 con 0x7f8be8065150 2024-07-16T09:52:09.628 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:09.532+0000 7f8bed96c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8be80fdf60 0x7f8be8113670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:52:09.628 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:09.532+0000 7f8bed96c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8be80fdf60 0x7f8be8113670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:44632/0 (socket says 172.21.15.156:44632) 2024-07-16T09:52:09.628 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:09.532+0000 7f8bed96c640 1 -- 172.21.15.156:0/2811003738 learned_addr learned my addr 172.21.15.156:0/2811003738 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T09:52:09.628 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:09.532+0000 7f8bee16d640 1 --2- 172.21.15.156:0/2811003738 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8be8065150 0x7f8be8113130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:52:09.628 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:09.532+0000 7f8bed96c640 1 -- 172.21.15.156:0/2811003738 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8be8065150 msgr2=0x7f8be8113130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:52:09.628 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:09.532+0000 7f8bed96c640 1 --2- 172.21.15.156:0/2811003738 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8be8065150 0x7f8be8113130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:52:09.629 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:09.532+0000 7f8bed96c640 1 -- 172.21.15.156:0/2811003738 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8be4005040 con 0x7f8be80fdf60 2024-07-16T09:52:09.629 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:09.533+0000 7f8bed96c640 1 --2- 172.21.15.156:0/2811003738 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8be80fdf60 0x7f8be8113670 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f8bd800b790 tx=0x7f8bd80091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:52:09.629 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:09.533+0000 7f8bdeffd640 1 -- 172.21.15.156:0/2811003738 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8bd800c700 con 0x7f8be80fdf60 2024-07-16T09:52:09.629 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:09.533+0000 7f8bf03d1640 1 -- 172.21.15.156:0/2811003738 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8be810d390 con 0x7f8be80fdf60 2024-07-16T09:52:09.629 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:09.533+0000 7f8bdeffd640 1 -- 172.21.15.156:0/2811003738 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8bd800cd30 con 0x7f8be80fdf60 2024-07-16T09:52:09.629 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:09.533+0000 7f8bdeffd640 1 -- 172.21.15.156:0/2811003738 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8bd8013370 con 0x7f8be80fdf60 2024-07-16T09:52:09.629 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:09.533+0000 7f8bf03d1640 1 -- 172.21.15.156:0/2811003738 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8be810d8b0 con 0x7f8be80fdf60 2024-07-16T09:52:09.629 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:09.534+0000 7f8bf03d1640 1 -- 172.21.15.156:0/2811003738 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f8be810ddf0 con 0x7f8be80fdf60 2024-07-16T09:52:09.630 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:09.535+0000 7f8bdeffd640 1 -- 172.21.15.156:0/2811003738 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 19) v1 ==== 85127+0+0 (secure 0 0 0) 0x7f8bd8012070 con 0x7f8be80fdf60 2024-07-16T09:52:09.630 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:09.535+0000 7f8bdeffd640 1 --2- 172.21.15.156:0/2811003738 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f8bd4067190 0x7f8bd4069640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:52:09.630 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:09.535+0000 7f8bdeffd640 1 -- 172.21.15.156:0/2811003738 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f8bd8084910 con 0x7f8be80fdf60 2024-07-16T09:52:09.630 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:09.535+0000 7f8bee16d640 1 --2- 172.21.15.156:0/2811003738 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f8bd4067190 0x7f8bd4069640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:52:09.630 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:09.535+0000 7f8bdeffd640 1 --2- 172.21.15.156:0/2811003738 >> [v2:172.21.15.156:6802/36114952,v1:172.21.15.156:6803/36114952] conn(0x7f8bd406cf10 0x7f8bd406f320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:52:09.630 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:09.535+0000 7f8bdeffd640 1 -- 172.21.15.156:0/2811003738 --> [v2:172.21.15.156:6802/36114952,v1:172.21.15.156:6803/36114952] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8bd406f9d0 con 0x7f8bd406cf10 2024-07-16T09:52:09.630 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:09.535+0000 7f8bdeffd640 1 -- 172.21.15.156:0/2811003738 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8bd8084cf0 con 0x7f8be80fdf60 2024-07-16T09:52:09.630 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:09.536+0000 7f8bee96e640 1 --2- 172.21.15.156:0/2811003738 >> [v2:172.21.15.156:6802/36114952,v1:172.21.15.156:6803/36114952] conn(0x7f8bd406cf10 0x7f8bd406f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:52:09.630 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:09.536+0000 7f8bee16d640 1 --2- 172.21.15.156:0/2811003738 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f8bd4067190 0x7f8bd4069640 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f8be4009f80 tx=0x7f8be4035040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:52:09.630 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:09.536+0000 7f8bee96e640 1 --2- 172.21.15.156:0/2811003738 >> [v2:172.21.15.156:6802/36114952,v1:172.21.15.156:6803/36114952] conn(0x7f8bd406cf10 0x7f8bd406f320 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:52:09.630 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:09.537+0000 7f8bdeffd640 1 -- 172.21.15.156:0/2811003738 <== osd.0 v2:172.21.15.156:6802/36114952 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7f8bd406f9d0 con 0x7f8bd406cf10 2024-07-16T09:52:09.631 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:09.550+0000 7f8bf03d1640 1 -- 172.21.15.156:0/2811003738 --> [v2:172.21.15.156:6802/36114952,v1:172.21.15.156:6803/36114952] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f8be8101a90 con 0x7f8bd406cf10 2024-07-16T09:52:09.631 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:09.550+0000 7f8bdeffd640 1 -- 172.21.15.156:0/2811003738 <== osd.0 v2:172.21.15.156:6802/36114952 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f8be8101a90 con 0x7f8bd406cf10 2024-07-16T09:52:09.631 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:09.550+0000 7f8bf03d1640 1 -- 172.21.15.156:0/2811003738 >> [v2:172.21.15.156:6802/36114952,v1:172.21.15.156:6803/36114952] conn(0x7f8bd406cf10 msgr2=0x7f8bd406f320 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:52:09.631 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:09.550+0000 7f8bf03d1640 1 --2- 172.21.15.156:0/2811003738 >> [v2:172.21.15.156:6802/36114952,v1:172.21.15.156:6803/36114952] conn(0x7f8bd406cf10 0x7f8bd406f320 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:52:09.631 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:09.550+0000 7f8bf03d1640 1 -- 172.21.15.156:0/2811003738 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f8bd4067190 msgr2=0x7f8bd4069640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:52:09.631 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:09.550+0000 7f8bf03d1640 1 --2- 172.21.15.156:0/2811003738 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f8bd4067190 0x7f8bd4069640 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f8be4009f80 tx=0x7f8be4035040 comp rx=0 tx=0).stop 2024-07-16T09:52:09.631 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:09.550+0000 7f8bf03d1640 1 -- 172.21.15.156:0/2811003738 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8be80fdf60 msgr2=0x7f8be8113670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:52:09.631 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:09.550+0000 7f8bf03d1640 1 --2- 172.21.15.156:0/2811003738 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8be80fdf60 0x7f8be8113670 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f8bd800b790 tx=0x7f8bd80091d0 comp rx=0 tx=0).stop 2024-07-16T09:52:09.631 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:09.550+0000 7f8bf03d1640 1 -- 172.21.15.156:0/2811003738 shutdown_connections 2024-07-16T09:52:09.631 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:09.550+0000 7f8bf03d1640 1 --2- 172.21.15.156:0/2811003738 >> [v2:172.21.15.156:6802/36114952,v1:172.21.15.156:6803/36114952] conn(0x7f8bd406cf10 0x7f8bd406f320 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:52:09.631 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:09.550+0000 7f8bf03d1640 1 --2- 172.21.15.156:0/2811003738 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f8bd4067190 0x7f8bd4069640 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:52:09.631 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:09.550+0000 7f8bf03d1640 1 --2- 172.21.15.156:0/2811003738 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8be80fdf60 0x7f8be8113670 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:52:09.632 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:09.550+0000 7f8bf03d1640 1 --2- 172.21.15.156:0/2811003738 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8be8065150 0x7f8be8113130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:52:09.632 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:09.550+0000 7f8bf03d1640 1 -- 172.21.15.156:0/2811003738 >> 172.21.15.156:0/2811003738 conn(0x7f8be80721c0 msgr2=0x7f8be8101080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:52:09.632 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:09.551+0000 7f8bf03d1640 1 -- 172.21.15.156:0/2811003738 shutdown_connections 2024-07-16T09:52:09.632 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:09.551+0000 7f8bf03d1640 1 -- 172.21.15.156:0/2811003738 wait complete. 2024-07-16T09:52:10.019 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:10.016+0000 7f6c343d5640 1 -- 172.21.15.156:0/947760094 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6c2c102cf0 msgr2=0x7f6c2c0fc380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:52:10.020 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:10.016+0000 7f6c343d5640 1 --2- 172.21.15.156:0/947760094 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6c2c102cf0 0x7f6c2c0fc380 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f6c1c0048d0 tx=0x7f6c1c009da0 comp rx=0 tx=0).stop 2024-07-16T09:52:10.020 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:10.016+0000 7f6c343d5640 1 -- 172.21.15.156:0/947760094 shutdown_connections 2024-07-16T09:52:10.020 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:10.016+0000 7f6c343d5640 1 --2- 172.21.15.156:0/947760094 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6c2c102cf0 0x7f6c2c0fc380 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:52:10.020 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:10.016+0000 7f6c343d5640 1 --2- 172.21.15.156:0/947760094 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6c2c102350 0x7f6c2c102720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:52:10.020 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:10.016+0000 7f6c343d5640 1 -- 172.21.15.156:0/947760094 >> 172.21.15.156:0/947760094 conn(0x7f6c2c0f7f00 msgr2=0x7f6c2c0fa310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:52:10.020 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:10.016+0000 7f6c343d5640 1 -- 172.21.15.156:0/947760094 shutdown_connections 2024-07-16T09:52:10.020 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:10.016+0000 7f6c343d5640 1 -- 172.21.15.156:0/947760094 wait complete. 2024-07-16T09:52:10.020 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:10.016+0000 7f6c343d5640 1 Processor -- start 2024-07-16T09:52:10.020 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:10.017+0000 7f6c343d5640 1 -- start start 2024-07-16T09:52:10.021 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:10.017+0000 7f6c343d5640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6c2c102350 0x7f6c2c1152d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:52:10.021 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:10.017+0000 7f6c343d5640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6c2c102cf0 0x7f6c2c115810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:52:10.021 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:10.017+0000 7f6c343d5640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c2c115e10 con 0x7f6c2c102350 2024-07-16T09:52:10.021 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:10.017+0000 7f6c343d5640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c2c10f250 con 0x7f6c2c102cf0 2024-07-16T09:52:10.021 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:10.017+0000 7f6c32171640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6c2c102350 0x7f6c2c1152d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:52:10.021 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:10.017+0000 7f6c32171640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6c2c102350 0x7f6c2c1152d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:44654/0 (socket says 172.21.15.156:44654) 2024-07-16T09:52:10.021 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:10.017+0000 7f6c32171640 1 -- 172.21.15.156:0/3612508696 learned_addr learned my addr 172.21.15.156:0/3612508696 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T09:52:10.022 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:10.017+0000 7f6c31970640 1 --2- 172.21.15.156:0/3612508696 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6c2c102cf0 0x7f6c2c115810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:52:10.022 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:10.017+0000 7f6c32171640 1 -- 172.21.15.156:0/3612508696 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6c2c102cf0 msgr2=0x7f6c2c115810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:52:10.022 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:10.017+0000 7f6c32171640 1 --2- 172.21.15.156:0/3612508696 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6c2c102cf0 0x7f6c2c115810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:52:10.022 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:10.017+0000 7f6c32171640 1 -- 172.21.15.156:0/3612508696 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c1c005040 con 0x7f6c2c102350 2024-07-16T09:52:10.022 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:10.017+0000 7f6c32171640 1 --2- 172.21.15.156:0/3612508696 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6c2c102350 0x7f6c2c1152d0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f6c28002a50 tx=0x7f6c28004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:52:10.022 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:10.018+0000 7f6c22ffd640 1 -- 172.21.15.156:0/3612508696 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6c28007c30 con 0x7f6c2c102350 2024-07-16T09:52:10.022 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:10.018+0000 7f6c343d5640 1 -- 172.21.15.156:0/3612508696 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c2c10f530 con 0x7f6c2c102350 2024-07-16T09:52:10.022 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:10.018+0000 7f6c22ffd640 1 -- 172.21.15.156:0/3612508696 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6c2800f040 con 0x7f6c2c102350 2024-07-16T09:52:10.022 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:10.018+0000 7f6c22ffd640 1 -- 172.21.15.156:0/3612508696 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6c2800db50 con 0x7f6c2c102350 2024-07-16T09:52:10.022 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:10.018+0000 7f6c343d5640 1 -- 172.21.15.156:0/3612508696 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c2c10fad0 con 0x7f6c2c102350 2024-07-16T09:52:10.024 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:10.019+0000 7f6c343d5640 1 -- 172.21.15.156:0/3612508696 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f6c2c10ffb0 con 0x7f6c2c102350 2024-07-16T09:52:10.024 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:10.020+0000 7f6c22ffd640 1 -- 172.21.15.156:0/3612508696 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 19) v1 ==== 85127+0+0 (secure 0 0 0) 0x7f6c2801c400 con 0x7f6c2c102350 2024-07-16T09:52:10.024 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:10.020+0000 7f6c22ffd640 1 --2- 172.21.15.156:0/3612508696 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f6c18067190 0x7f6c18069640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:52:10.024 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:10.020+0000 7f6c22ffd640 1 -- 172.21.15.156:0/3612508696 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f6c28084120 con 0x7f6c2c102350 2024-07-16T09:52:10.024 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:10.020+0000 7f6c31970640 1 --2- 172.21.15.156:0/3612508696 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f6c18067190 0x7f6c18069640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:52:10.024 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:10.020+0000 7f6c22ffd640 1 --2- 172.21.15.156:0/3612508696 >> [v2:172.21.15.156:6818/3710668239,v1:172.21.15.156:6819/3710668239] conn(0x7f6c1806cf10 0x7f6c1806f320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:52:10.024 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:10.020+0000 7f6c22ffd640 1 -- 172.21.15.156:0/3612508696 --> [v2:172.21.15.156:6818/3710668239,v1:172.21.15.156:6819/3710668239] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6c1806f9d0 con 0x7f6c1806cf10 2024-07-16T09:52:10.025 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:10.020+0000 7f6c22ffd640 1 -- 172.21.15.156:0/3612508696 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6c28086770 con 0x7f6c2c102350 2024-07-16T09:52:10.025 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:10.021+0000 7f6c32972640 1 --2- 172.21.15.156:0/3612508696 >> [v2:172.21.15.156:6818/3710668239,v1:172.21.15.156:6819/3710668239] conn(0x7f6c1806cf10 0x7f6c1806f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:52:10.025 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:10.021+0000 7f6c31970640 1 --2- 172.21.15.156:0/3612508696 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f6c18067190 0x7f6c18069640 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f6c2c110b90 tx=0x7f6c1c035040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:52:10.025 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:10.021+0000 7f6c32972640 1 --2- 172.21.15.156:0/3612508696 >> [v2:172.21.15.156:6818/3710668239,v1:172.21.15.156:6819/3710668239] conn(0x7f6c1806cf10 0x7f6c1806f320 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:52:10.025 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:10.022+0000 7f6c22ffd640 1 -- 172.21.15.156:0/3612508696 <== osd.2 v2:172.21.15.156:6818/3710668239 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7f6c1806f9d0 con 0x7f6c1806cf10 2024-07-16T09:52:10.047 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:10.046+0000 7f6c343d5640 1 -- 172.21.15.156:0/3612508696 --> [v2:172.21.15.156:6818/3710668239,v1:172.21.15.156:6819/3710668239] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f6c2c0fdac0 con 0x7f6c1806cf10 2024-07-16T09:52:10.048 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:10.046+0000 7f6c22ffd640 1 -- 172.21.15.156:0/3612508696 <== osd.2 v2:172.21.15.156:6818/3710668239 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f6c2c0fdac0 con 0x7f6c1806cf10 2024-07-16T09:52:10.048 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:10.046+0000 7f6c343d5640 1 -- 172.21.15.156:0/3612508696 >> [v2:172.21.15.156:6818/3710668239,v1:172.21.15.156:6819/3710668239] conn(0x7f6c1806cf10 msgr2=0x7f6c1806f320 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:52:10.049 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:10.046+0000 7f6c343d5640 1 --2- 172.21.15.156:0/3612508696 >> [v2:172.21.15.156:6818/3710668239,v1:172.21.15.156:6819/3710668239] conn(0x7f6c1806cf10 0x7f6c1806f320 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:52:10.049 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:10.046+0000 7f6c343d5640 1 -- 172.21.15.156:0/3612508696 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f6c18067190 msgr2=0x7f6c18069640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:52:10.049 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:10.046+0000 7f6c343d5640 1 --2- 172.21.15.156:0/3612508696 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f6c18067190 0x7f6c18069640 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f6c2c110b90 tx=0x7f6c1c035040 comp rx=0 tx=0).stop 2024-07-16T09:52:10.049 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:10.046+0000 7f6c343d5640 1 -- 172.21.15.156:0/3612508696 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6c2c102350 msgr2=0x7f6c2c1152d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:52:10.049 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:10.046+0000 7f6c343d5640 1 --2- 172.21.15.156:0/3612508696 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6c2c102350 0x7f6c2c1152d0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f6c28002a50 tx=0x7f6c28004600 comp rx=0 tx=0).stop 2024-07-16T09:52:10.049 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:10.047+0000 7f6c343d5640 1 -- 172.21.15.156:0/3612508696 shutdown_connections 2024-07-16T09:52:10.049 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:10.047+0000 7f6c343d5640 1 --2- 172.21.15.156:0/3612508696 >> [v2:172.21.15.156:6818/3710668239,v1:172.21.15.156:6819/3710668239] conn(0x7f6c1806cf10 0x7f6c1806f320 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:52:10.049 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:10.047+0000 7f6c343d5640 1 --2- 172.21.15.156:0/3612508696 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f6c18067190 0x7f6c18069640 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:52:10.049 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:10.047+0000 7f6c343d5640 1 --2- 172.21.15.156:0/3612508696 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6c2c102cf0 0x7f6c2c115810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:52:10.050 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:10.047+0000 7f6c343d5640 1 --2- 172.21.15.156:0/3612508696 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6c2c102350 0x7f6c2c1152d0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:52:10.050 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:10.047+0000 7f6c343d5640 1 -- 172.21.15.156:0/3612508696 >> 172.21.15.156:0/3612508696 conn(0x7f6c2c0f7f00 msgr2=0x7f6c2c071250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:52:10.050 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:10.047+0000 7f6c343d5640 1 -- 172.21.15.156:0/3612508696 shutdown_connections 2024-07-16T09:52:10.050 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:10.047+0000 7f6c343d5640 1 -- 172.21.15.156:0/3612508696 wait complete. 2024-07-16T09:52:10.339 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:52:10 smithi156 ceph-mon[29302]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-07-16T09:52:10.339 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:52:10 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-16T09:52:10.343 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:52:10 smithi176 ceph-mon[36626]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-07-16T09:52:10.343 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:52:10 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-16T09:52:10.506 INFO:teuthology.orchestra.run.smithi156.stdout:38654705698 2024-07-16T09:52:10.509 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 26c91c26-4358-11ef-bcad-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-07-16T09:52:11.665 INFO:teuthology.orchestra.run.smithi156.stdout:171798691848 2024-07-16T09:52:11.666 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 26c91c26-4358-11ef-bcad-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-07-16T09:52:12.067 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:12.052+0000 7f9df2355640 1 -- 172.21.15.156:0/1337763680 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9dec102370 msgr2=0x7f9dec102740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:52:12.067 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:12.052+0000 7f9df2355640 1 --2- 172.21.15.156:0/1337763680 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9dec102370 0x7f9dec102740 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f9ddc0048d0 tx=0x7f9ddc009da0 comp rx=0 tx=0).stop 2024-07-16T09:52:12.067 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:12.052+0000 7f9df2355640 1 -- 172.21.15.156:0/1337763680 shutdown_connections 2024-07-16T09:52:12.067 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:12.052+0000 7f9df2355640 1 --2- 172.21.15.156:0/1337763680 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9dec102d10 0x7f9dec0fc3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:52:12.068 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:12.052+0000 7f9df2355640 1 --2- 172.21.15.156:0/1337763680 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9dec102370 0x7f9dec102740 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:52:12.068 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:12.052+0000 7f9df2355640 1 -- 172.21.15.156:0/1337763680 >> 172.21.15.156:0/1337763680 conn(0x7f9dec0f7f00 msgr2=0x7f9dec0fa310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:52:12.068 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:12.052+0000 7f9df2355640 1 -- 172.21.15.156:0/1337763680 shutdown_connections 2024-07-16T09:52:12.068 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:12.052+0000 7f9df2355640 1 -- 172.21.15.156:0/1337763680 wait complete. 2024-07-16T09:52:12.068 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:12.052+0000 7f9df2355640 1 Processor -- start 2024-07-16T09:52:12.068 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:12.053+0000 7f9df2355640 1 -- start start 2024-07-16T09:52:12.068 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:12.053+0000 7f9df2355640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9dec102370 0x7f9dec06f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:52:12.068 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:12.053+0000 7f9df2355640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9dec102d10 0x7f9dec06fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:52:12.068 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:12.053+0000 7f9df2355640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dec0702a0 con 0x7f9dec102370 2024-07-16T09:52:12.068 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:12.053+0000 7f9df2355640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dec070410 con 0x7f9dec102d10 2024-07-16T09:52:12.068 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:12.053+0000 7f9debfff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9dec102370 0x7f9dec06f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:52:12.068 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:12.053+0000 7f9debfff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9dec102370 0x7f9dec06f820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:44674/0 (socket says 172.21.15.156:44674) 2024-07-16T09:52:12.069 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:12.053+0000 7f9debfff640 1 -- 172.21.15.156:0/3233327534 learned_addr learned my addr 172.21.15.156:0/3233327534 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T09:52:12.069 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:12.053+0000 7f9deb7fe640 1 --2- 172.21.15.156:0/3233327534 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9dec102d10 0x7f9dec06fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:52:12.069 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:12.053+0000 7f9debfff640 1 -- 172.21.15.156:0/3233327534 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9dec102d10 msgr2=0x7f9dec06fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:52:12.069 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:12.053+0000 7f9debfff640 1 --2- 172.21.15.156:0/3233327534 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9dec102d10 0x7f9dec06fd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:52:12.069 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:12.053+0000 7f9debfff640 1 -- 172.21.15.156:0/3233327534 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ddc005040 con 0x7f9dec102370 2024-07-16T09:52:12.069 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:12.053+0000 7f9deb7fe640 1 --2- 172.21.15.156:0/3233327534 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9dec102d10 0x7f9dec06fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-16T09:52:12.069 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:12.053+0000 7f9debfff640 1 --2- 172.21.15.156:0/3233327534 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9dec102370 0x7f9dec06f820 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f9ddc003080 tx=0x7f9ddc009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:52:12.069 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:12.054+0000 7f9de8ff9640 1 -- 172.21.15.156:0/3233327534 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9ddc038070 con 0x7f9dec102370 2024-07-16T09:52:12.069 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:12.054+0000 7f9df2355640 1 -- 172.21.15.156:0/3233327534 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9dec070640 con 0x7f9dec102370 2024-07-16T09:52:12.069 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:12.054+0000 7f9de8ff9640 1 -- 172.21.15.156:0/3233327534 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9ddc008ee0 con 0x7f9dec102370 2024-07-16T09:52:12.069 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:12.054+0000 7f9de8ff9640 1 -- 172.21.15.156:0/3233327534 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9ddc03d870 con 0x7f9dec102370 2024-07-16T09:52:12.069 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:12.054+0000 7f9df2355640 1 -- 172.21.15.156:0/3233327534 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9dec119a60 con 0x7f9dec102370 2024-07-16T09:52:12.070 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:12.054+0000 7f9df2355640 1 -- 172.21.15.156:0/3233327534 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f9dec0fdae0 con 0x7f9dec102370 2024-07-16T09:52:12.070 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:12.055+0000 7f9de8ff9640 1 -- 172.21.15.156:0/3233327534 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 19) v1 ==== 85127+0+0 (secure 0 0 0) 0x7f9ddc030020 con 0x7f9dec102370 2024-07-16T09:52:12.070 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:12.055+0000 7f9de8ff9640 1 --2- 172.21.15.156:0/3233327534 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f9dd8067190 0x7f9dd8069640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:52:12.070 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:12.056+0000 7f9de8ff9640 1 -- 172.21.15.156:0/3233327534 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f9ddc0a4560 con 0x7f9dec102370 2024-07-16T09:52:12.070 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:12.056+0000 7f9deb7fe640 1 --2- 172.21.15.156:0/3233327534 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f9dd8067190 0x7f9dd8069640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:52:12.070 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:12.056+0000 7f9de8ff9640 1 --2- 172.21.15.156:0/3233327534 >> [v2:172.21.15.176:6808/2702155727,v1:172.21.15.176:6809/2702155727] conn(0x7f9dd806cf10 0x7f9dd806f320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:52:12.070 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:12.056+0000 7f9de8ff9640 1 -- 172.21.15.156:0/3233327534 --> [v2:172.21.15.176:6808/2702155727,v1:172.21.15.176:6809/2702155727] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9dd806f9d0 con 0x7f9dd806cf10 2024-07-16T09:52:12.070 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:12.056+0000 7f9de8ff9640 1 -- 172.21.15.156:0/3233327534 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9ddc0a6b50 con 0x7f9dec102370 2024-07-16T09:52:12.070 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:12.056+0000 7f9deb7fe640 1 --2- 172.21.15.156:0/3233327534 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f9dd8067190 0x7f9dd8069640 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f9dd4007fd0 tx=0x7f9dd4009040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:52:12.070 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:12.056+0000 7f9df08f2640 1 --2- 172.21.15.156:0/3233327534 >> [v2:172.21.15.176:6808/2702155727,v1:172.21.15.176:6809/2702155727] conn(0x7f9dd806cf10 0x7f9dd806f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:52:12.070 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:12.056+0000 7f9df08f2640 1 --2- 172.21.15.156:0/3233327534 >> [v2:172.21.15.176:6808/2702155727,v1:172.21.15.176:6809/2702155727] conn(0x7f9dd806cf10 0x7f9dd806f320 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:52:12.071 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:12.057+0000 7f9de8ff9640 1 -- 172.21.15.156:0/3233327534 <== osd.4 v2:172.21.15.176:6808/2702155727 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7f9dd806f9d0 con 0x7f9dd806cf10 2024-07-16T09:52:12.071 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:12.070+0000 7f9df2355640 1 -- 172.21.15.156:0/3233327534 --> [v2:172.21.15.176:6808/2702155727,v1:172.21.15.176:6809/2702155727] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f9dec0fbb60 con 0x7f9dd806cf10 2024-07-16T09:52:12.072 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:12.071+0000 7f9de8ff9640 1 -- 172.21.15.156:0/3233327534 <== osd.4 v2:172.21.15.176:6808/2702155727 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f9dec0fbb60 con 0x7f9dd806cf10 2024-07-16T09:52:12.073 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:12.071+0000 7f9df2355640 1 -- 172.21.15.156:0/3233327534 >> [v2:172.21.15.176:6808/2702155727,v1:172.21.15.176:6809/2702155727] conn(0x7f9dd806cf10 msgr2=0x7f9dd806f320 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:52:12.073 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:12.071+0000 7f9df2355640 1 --2- 172.21.15.156:0/3233327534 >> [v2:172.21.15.176:6808/2702155727,v1:172.21.15.176:6809/2702155727] conn(0x7f9dd806cf10 0x7f9dd806f320 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:52:12.073 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:12.071+0000 7f9df2355640 1 -- 172.21.15.156:0/3233327534 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f9dd8067190 msgr2=0x7f9dd8069640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:52:12.073 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:12.071+0000 7f9df2355640 1 --2- 172.21.15.156:0/3233327534 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f9dd8067190 0x7f9dd8069640 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f9dd4007fd0 tx=0x7f9dd4009040 comp rx=0 tx=0).stop 2024-07-16T09:52:12.073 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:12.071+0000 7f9df2355640 1 -- 172.21.15.156:0/3233327534 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9dec102370 msgr2=0x7f9dec06f820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:52:12.073 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:12.071+0000 7f9df2355640 1 --2- 172.21.15.156:0/3233327534 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9dec102370 0x7f9dec06f820 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f9ddc003080 tx=0x7f9ddc009da0 comp rx=0 tx=0).stop 2024-07-16T09:52:12.073 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:12.071+0000 7f9df2355640 1 -- 172.21.15.156:0/3233327534 shutdown_connections 2024-07-16T09:52:12.073 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:12.071+0000 7f9df2355640 1 --2- 172.21.15.156:0/3233327534 >> [v2:172.21.15.176:6808/2702155727,v1:172.21.15.176:6809/2702155727] conn(0x7f9dd806cf10 0x7f9dd806f320 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:52:12.073 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:12.071+0000 7f9df2355640 1 --2- 172.21.15.156:0/3233327534 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f9dd8067190 0x7f9dd8069640 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:52:12.073 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:12.071+0000 7f9df2355640 1 --2- 172.21.15.156:0/3233327534 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9dec102d10 0x7f9dec06fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:52:12.073 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:12.071+0000 7f9df2355640 1 --2- 172.21.15.156:0/3233327534 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9dec102370 0x7f9dec06f820 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:52:12.074 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:12.071+0000 7f9df2355640 1 -- 172.21.15.156:0/3233327534 >> 172.21.15.156:0/3233327534 conn(0x7f9dec0f7f00 msgr2=0x7f9dec0f8e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:52:12.074 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:12.071+0000 7f9df2355640 1 -- 172.21.15.156:0/3233327534 shutdown_connections 2024-07-16T09:52:12.074 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:12.071+0000 7f9df2355640 1 -- 172.21.15.156:0/3233327534 wait complete. 2024-07-16T09:52:12.245 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:52:12 smithi156 ceph-mon[29302]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-07-16T09:52:12.251 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:12.244+0000 7f11656c3640 1 -- 172.21.15.156:0/4219891870 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1160101c00 msgr2=0x7f11600fb290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:52:12.251 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:12.244+0000 7f11656c3640 1 --2- 172.21.15.156:0/4219891870 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1160101c00 0x7f11600fb290 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f11540048d0 tx=0x7f1154009da0 comp rx=0 tx=0).stop 2024-07-16T09:52:12.252 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:12.244+0000 7f11656c3640 1 -- 172.21.15.156:0/4219891870 shutdown_connections 2024-07-16T09:52:12.252 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:12.244+0000 7f11656c3640 1 --2- 172.21.15.156:0/4219891870 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1160101c00 0x7f11600fb290 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:52:12.252 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:12.244+0000 7f11656c3640 1 --2- 172.21.15.156:0/4219891870 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1160101260 0x7f1160101630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:52:12.252 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:12.244+0000 7f11656c3640 1 -- 172.21.15.156:0/4219891870 >> 172.21.15.156:0/4219891870 conn(0x7f11600f6e10 msgr2=0x7f11600f9220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:52:12.252 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:12.244+0000 7f11656c3640 1 -- 172.21.15.156:0/4219891870 shutdown_connections 2024-07-16T09:52:12.252 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:12.244+0000 7f11656c3640 1 -- 172.21.15.156:0/4219891870 wait complete. 2024-07-16T09:52:12.252 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:12.245+0000 7f11656c3640 1 Processor -- start 2024-07-16T09:52:12.252 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:12.245+0000 7f11656c3640 1 -- start start 2024-07-16T09:52:12.252 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:12.245+0000 7f11656c3640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1160101260 0x7f11601152f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:52:12.252 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:12.245+0000 7f11656c3640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1160101c00 0x7f1160115830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:52:12.252 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:12.245+0000 7f11656c3640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1160115ec0 con 0x7f1160101260 2024-07-16T09:52:12.252 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:12.245+0000 7f11656c3640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f116010f2b0 con 0x7f1160101c00 2024-07-16T09:52:12.253 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:12.245+0000 7f115effd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1160101260 0x7f11601152f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:52:12.253 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:12.245+0000 7f115effd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1160101260 0x7f11601152f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:44688/0 (socket says 172.21.15.156:44688) 2024-07-16T09:52:12.253 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:12.245+0000 7f115effd640 1 -- 172.21.15.156:0/4230255275 learned_addr learned my addr 172.21.15.156:0/4230255275 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T09:52:12.253 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:12.245+0000 7f115e7fc640 1 --2- 172.21.15.156:0/4230255275 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1160101c00 0x7f1160115830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:52:12.253 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:12.246+0000 7f115effd640 1 -- 172.21.15.156:0/4230255275 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1160101c00 msgr2=0x7f1160115830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:52:12.253 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:12.246+0000 7f115effd640 1 --2- 172.21.15.156:0/4230255275 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1160101c00 0x7f1160115830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:52:12.253 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:12.246+0000 7f115effd640 1 -- 172.21.15.156:0/4230255275 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1154005040 con 0x7f1160101260 2024-07-16T09:52:12.253 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:12.246+0000 7f115effd640 1 --2- 172.21.15.156:0/4230255275 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1160101260 0x7f11601152f0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f115000b790 tx=0x7f11500091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:52:12.253 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:12.246+0000 7f11477fe640 1 -- 172.21.15.156:0/4230255275 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f115000c700 con 0x7f1160101260 2024-07-16T09:52:12.253 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:12.246+0000 7f11656c3640 1 -- 172.21.15.156:0/4230255275 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f116010f590 con 0x7f1160101260 2024-07-16T09:52:12.253 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:12.246+0000 7f11477fe640 1 -- 172.21.15.156:0/4230255275 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f115000cd30 con 0x7f1160101260 2024-07-16T09:52:12.253 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:12.246+0000 7f11477fe640 1 -- 172.21.15.156:0/4230255275 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1150013370 con 0x7f1160101260 2024-07-16T09:52:12.254 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:12.246+0000 7f11656c3640 1 -- 172.21.15.156:0/4230255275 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f116010fab0 con 0x7f1160101260 2024-07-16T09:52:12.254 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:12.247+0000 7f11656c3640 1 -- 172.21.15.156:0/4230255275 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f1160110030 con 0x7f1160101260 2024-07-16T09:52:12.254 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:12.247+0000 7f11477fe640 1 -- 172.21.15.156:0/4230255275 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 19) v1 ==== 85127+0+0 (secure 0 0 0) 0x7f1150012070 con 0x7f1160101260 2024-07-16T09:52:12.254 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:12.248+0000 7f11477fe640 1 --2- 172.21.15.156:0/4230255275 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f1148067190 0x7f1148069640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:52:12.254 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:12.248+0000 7f11477fe640 1 -- 172.21.15.156:0/4230255275 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f11500843d0 con 0x7f1160101260 2024-07-16T09:52:12.254 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:12.248+0000 7f115e7fc640 1 --2- 172.21.15.156:0/4230255275 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f1148067190 0x7f1148069640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:52:12.254 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:12.248+0000 7f11477fe640 1 --2- 172.21.15.156:0/4230255275 >> [v2:172.21.15.176:6800/1740426750,v1:172.21.15.176:6801/1740426750] conn(0x7f114806cf10 0x7f114806f320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:52:12.254 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:12.248+0000 7f11477fe640 1 -- 172.21.15.156:0/4230255275 --> [v2:172.21.15.176:6800/1740426750,v1:172.21.15.176:6801/1740426750] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f114806f9d0 con 0x7f114806cf10 2024-07-16T09:52:12.254 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:12.248+0000 7f11477fe640 1 -- 172.21.15.156:0/4230255275 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1150086a20 con 0x7f1160101260 2024-07-16T09:52:12.254 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:12.248+0000 7f115f7fe640 1 --2- 172.21.15.156:0/4230255275 >> [v2:172.21.15.176:6800/1740426750,v1:172.21.15.176:6801/1740426750] conn(0x7f114806cf10 0x7f114806f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:52:12.254 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:12.248+0000 7f115e7fc640 1 --2- 172.21.15.156:0/4230255275 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f1148067190 0x7f1148069640 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f11540047f0 tx=0x7f1154009da0 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:52:12.254 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:12.248+0000 7f115f7fe640 1 --2- 172.21.15.156:0/4230255275 >> [v2:172.21.15.176:6800/1740426750,v1:172.21.15.176:6801/1740426750] conn(0x7f114806cf10 0x7f114806f320 crc :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:52:12.255 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:12.249+0000 7f11477fe640 1 -- 172.21.15.156:0/4230255275 <== osd.3 v2:172.21.15.176:6800/1740426750 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7f114806f9d0 con 0x7f114806cf10 2024-07-16T09:52:12.265 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:12.264+0000 7f11656c3640 1 -- 172.21.15.156:0/4230255275 --> [v2:172.21.15.176:6800/1740426750,v1:172.21.15.176:6801/1740426750] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f11600fc9d0 con 0x7f114806cf10 2024-07-16T09:52:12.265 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:12.264+0000 7f11477fe640 1 -- 172.21.15.156:0/4230255275 <== osd.3 v2:172.21.15.176:6800/1740426750 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f11600fc9d0 con 0x7f114806cf10 2024-07-16T09:52:12.266 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:12.265+0000 7f11656c3640 1 -- 172.21.15.156:0/4230255275 >> [v2:172.21.15.176:6800/1740426750,v1:172.21.15.176:6801/1740426750] conn(0x7f114806cf10 msgr2=0x7f114806f320 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:52:12.266 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:12.265+0000 7f11656c3640 1 --2- 172.21.15.156:0/4230255275 >> [v2:172.21.15.176:6800/1740426750,v1:172.21.15.176:6801/1740426750] conn(0x7f114806cf10 0x7f114806f320 crc :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:52:12.267 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:12.265+0000 7f11656c3640 1 -- 172.21.15.156:0/4230255275 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f1148067190 msgr2=0x7f1148069640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:52:12.267 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:12.265+0000 7f11656c3640 1 --2- 172.21.15.156:0/4230255275 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f1148067190 0x7f1148069640 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f11540047f0 tx=0x7f1154009da0 comp rx=0 tx=0).stop 2024-07-16T09:52:12.267 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:12.265+0000 7f11656c3640 1 -- 172.21.15.156:0/4230255275 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1160101260 msgr2=0x7f11601152f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:52:12.267 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:12.265+0000 7f11656c3640 1 --2- 172.21.15.156:0/4230255275 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1160101260 0x7f11601152f0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f115000b790 tx=0x7f11500091d0 comp rx=0 tx=0).stop 2024-07-16T09:52:12.267 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:12.265+0000 7f11656c3640 1 -- 172.21.15.156:0/4230255275 shutdown_connections 2024-07-16T09:52:12.267 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:12.265+0000 7f11656c3640 1 --2- 172.21.15.156:0/4230255275 >> [v2:172.21.15.176:6800/1740426750,v1:172.21.15.176:6801/1740426750] conn(0x7f114806cf10 0x7f114806f320 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:52:12.267 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:12.265+0000 7f11656c3640 1 --2- 172.21.15.156:0/4230255275 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f1148067190 0x7f1148069640 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:52:12.267 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:12.265+0000 7f11656c3640 1 --2- 172.21.15.156:0/4230255275 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1160101c00 0x7f1160115830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:52:12.267 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:12.265+0000 7f11656c3640 1 --2- 172.21.15.156:0/4230255275 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1160101260 0x7f11601152f0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:52:12.267 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:12.265+0000 7f11656c3640 1 -- 172.21.15.156:0/4230255275 >> 172.21.15.156:0/4230255275 conn(0x7f11600f6e10 msgr2=0x7f1160065860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:52:12.267 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:12.265+0000 7f11656c3640 1 -- 172.21.15.156:0/4230255275 shutdown_connections 2024-07-16T09:52:12.267 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:12.265+0000 7f11656c3640 1 -- 172.21.15.156:0/4230255275 wait complete. 2024-07-16T09:52:12.343 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:52:12 smithi176 ceph-mon[36626]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-07-16T09:52:13.067 INFO:teuthology.orchestra.run.smithi156.stdout:81604378646 2024-07-16T09:52:13.067 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 26c91c26-4358-11ef-bcad-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-07-16T09:52:13.304 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/26c91c26-4358-11ef-bcad-c7b262605968/mon.smithi156/config 2024-07-16T09:52:13.305 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/26c91c26-4358-11ef-bcad-c7b262605968/mon.smithi156/config 2024-07-16T09:52:13.810 INFO:teuthology.orchestra.run.smithi156.stdout:115964117011 2024-07-16T09:52:13.811 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 26c91c26-4358-11ef-bcad-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-07-16T09:52:14.343 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:52:14 smithi176 ceph-mon[36626]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-07-16T09:52:14.347 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:52:14 smithi156 ceph-mon[29302]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-07-16T09:52:14.370 INFO:teuthology.orchestra.run.smithi156.stdout:150323855374 2024-07-16T09:52:14.371 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 26c91c26-4358-11ef-bcad-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-07-16T09:52:14.828 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/26c91c26-4358-11ef-bcad-c7b262605968/mon.smithi156/config 2024-07-16T09:52:15.704 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/26c91c26-4358-11ef-bcad-c7b262605968/mon.smithi156/config 2024-07-16T09:52:15.708 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/26c91c26-4358-11ef-bcad-c7b262605968/mon.smithi156/config 2024-07-16T09:52:16.144 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:52:16 smithi156 ceph-mon[29302]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-07-16T09:52:16.343 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:52:16 smithi176 ceph-mon[36626]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-07-16T09:52:17.688 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:17.682+0000 7f48012dd640 1 -- 172.21.15.156:0/832576702 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f47fc102350 msgr2=0x7f47fc102720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:52:17.688 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:17.682+0000 7f48012dd640 1 --2- 172.21.15.156:0/832576702 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f47fc102350 0x7f47fc102720 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f47ec0048d0 tx=0x7f47ec009da0 comp rx=0 tx=0).stop 2024-07-16T09:52:17.688 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:17.682+0000 7f48012dd640 1 -- 172.21.15.156:0/832576702 shutdown_connections 2024-07-16T09:52:17.689 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:17.682+0000 7f48012dd640 1 --2- 172.21.15.156:0/832576702 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f47fc102cf0 0x7f47fc0fc380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:52:17.689 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:17.682+0000 7f48012dd640 1 --2- 172.21.15.156:0/832576702 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f47fc102350 0x7f47fc102720 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:52:17.689 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:17.682+0000 7f48012dd640 1 -- 172.21.15.156:0/832576702 >> 172.21.15.156:0/832576702 conn(0x7f47fc0f7f00 msgr2=0x7f47fc0fa310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:52:17.689 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:17.682+0000 7f48012dd640 1 -- 172.21.15.156:0/832576702 shutdown_connections 2024-07-16T09:52:17.689 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:17.682+0000 7f48012dd640 1 -- 172.21.15.156:0/832576702 wait complete. 2024-07-16T09:52:17.689 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:17.682+0000 7f48012dd640 1 Processor -- start 2024-07-16T09:52:17.689 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:17.682+0000 7f48012dd640 1 -- start start 2024-07-16T09:52:17.689 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:17.683+0000 7f48012dd640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f47fc102350 0x7f47fc115290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:52:17.689 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:17.683+0000 7f48012dd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f47fc102cf0 0x7f47fc1157d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:52:17.689 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:17.683+0000 7f48012dd640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47fc115dd0 con 0x7f47fc102cf0 2024-07-16T09:52:17.689 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:17.683+0000 7f48012dd640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47fc10f210 con 0x7f47fc102350 2024-07-16T09:52:17.689 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:17.683+0000 7f47fa7fc640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f47fc102cf0 0x7f47fc1157d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:52:17.689 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:17.683+0000 7f47fa7fc640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f47fc102cf0 0x7f47fc1157d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:33916/0 (socket says 172.21.15.156:33916) 2024-07-16T09:52:17.690 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:17.683+0000 7f47fa7fc640 1 -- 172.21.15.156:0/1359692668 learned_addr learned my addr 172.21.15.156:0/1359692668 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T09:52:17.690 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:17.683+0000 7f47faffd640 1 --2- 172.21.15.156:0/1359692668 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f47fc102350 0x7f47fc115290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:52:17.690 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:17.683+0000 7f47fa7fc640 1 -- 172.21.15.156:0/1359692668 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f47fc102350 msgr2=0x7f47fc115290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:52:17.690 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:17.683+0000 7f47fa7fc640 1 --2- 172.21.15.156:0/1359692668 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f47fc102350 0x7f47fc115290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:52:17.690 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:17.683+0000 7f47fa7fc640 1 -- 172.21.15.156:0/1359692668 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47ec005040 con 0x7f47fc102cf0 2024-07-16T09:52:17.690 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:17.683+0000 7f47fa7fc640 1 --2- 172.21.15.156:0/1359692668 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f47fc102cf0 0x7f47fc1157d0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f47f000c810 tx=0x7f47f00043c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:52:17.690 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:17.683+0000 7f47e37fe640 1 -- 172.21.15.156:0/1359692668 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f47f000a820 con 0x7f47fc102cf0 2024-07-16T09:52:17.690 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:17.683+0000 7f47e37fe640 1 -- 172.21.15.156:0/1359692668 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f47f000ae50 con 0x7f47fc102cf0 2024-07-16T09:52:17.690 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:17.683+0000 7f48012dd640 1 -- 172.21.15.156:0/1359692668 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47fc10f4f0 con 0x7f47fc102cf0 2024-07-16T09:52:17.690 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:17.683+0000 7f47e37fe640 1 -- 172.21.15.156:0/1359692668 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f47f00115c0 con 0x7f47fc102cf0 2024-07-16T09:52:17.690 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:17.683+0000 7f48012dd640 1 -- 172.21.15.156:0/1359692668 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47fc10fa90 con 0x7f47fc102cf0 2024-07-16T09:52:17.690 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:17.685+0000 7f48012dd640 1 -- 172.21.15.156:0/1359692668 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47fc0fdac0 con 0x7f47fc102cf0 2024-07-16T09:52:17.691 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:17.685+0000 7f47e37fe640 1 -- 172.21.15.156:0/1359692668 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 19) v1 ==== 85127+0+0 (secure 0 0 0) 0x7f47f000a980 con 0x7f47fc102cf0 2024-07-16T09:52:17.691 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:17.685+0000 7f47e37fe640 1 --2- 172.21.15.156:0/1359692668 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f47e4067190 0x7f47e4069640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:52:17.691 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:17.685+0000 7f47e37fe640 1 -- 172.21.15.156:0/1359692668 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f47f0084d80 con 0x7f47fc102cf0 2024-07-16T09:52:17.691 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:17.685+0000 7f47faffd640 1 --2- 172.21.15.156:0/1359692668 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f47e4067190 0x7f47e4069640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:52:17.691 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:17.686+0000 7f47faffd640 1 --2- 172.21.15.156:0/1359692668 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f47e4067190 0x7f47e4069640 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f47ec0047f0 tx=0x7f47ec009da0 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:52:17.691 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:17.689+0000 7f47e37fe640 1 -- 172.21.15.156:0/1359692668 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f47f0052360 con 0x7f47fc102cf0 2024-07-16T09:52:17.862 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:17.861+0000 7f48012dd640 1 -- 172.21.15.156:0/1359692668 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f47fc1094f0 con 0x7f47fc102cf0 2024-07-16T09:52:17.863 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:17.861+0000 7f47e37fe640 1 -- 172.21.15.156:0/1359692668 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f47f0055980 con 0x7f47fc102cf0 2024-07-16T09:52:17.863 INFO:teuthology.orchestra.run.smithi156.stdout:60129542174 2024-07-16T09:52:17.864 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:17.863+0000 7f48012dd640 1 -- 172.21.15.156:0/1359692668 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f47e4067190 msgr2=0x7f47e4069640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:52:17.865 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:17.863+0000 7f48012dd640 1 --2- 172.21.15.156:0/1359692668 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f47e4067190 0x7f47e4069640 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f47ec0047f0 tx=0x7f47ec009da0 comp rx=0 tx=0).stop 2024-07-16T09:52:17.865 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:17.863+0000 7f48012dd640 1 -- 172.21.15.156:0/1359692668 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f47fc102cf0 msgr2=0x7f47fc1157d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:52:17.865 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:17.863+0000 7f48012dd640 1 --2- 172.21.15.156:0/1359692668 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f47fc102cf0 0x7f47fc1157d0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f47f000c810 tx=0x7f47f00043c0 comp rx=0 tx=0).stop 2024-07-16T09:52:17.865 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:17.863+0000 7f48012dd640 1 -- 172.21.15.156:0/1359692668 shutdown_connections 2024-07-16T09:52:17.865 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:17.863+0000 7f48012dd640 1 --2- 172.21.15.156:0/1359692668 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f47e4067190 0x7f47e4069640 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:52:17.865 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:17.863+0000 7f48012dd640 1 --2- 172.21.15.156:0/1359692668 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f47fc102cf0 0x7f47fc1157d0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:52:17.865 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:17.863+0000 7f48012dd640 1 --2- 172.21.15.156:0/1359692668 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f47fc102350 0x7f47fc115290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:52:17.865 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:17.863+0000 7f48012dd640 1 -- 172.21.15.156:0/1359692668 >> 172.21.15.156:0/1359692668 conn(0x7f47fc0f7f00 msgr2=0x7f47fc0710c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:52:17.865 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:17.863+0000 7f48012dd640 1 -- 172.21.15.156:0/1359692668 shutdown_connections 2024-07-16T09:52:17.865 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:17.863+0000 7f48012dd640 1 -- 172.21.15.156:0/1359692668 wait complete. 2024-07-16T09:52:18.159 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:52:18 smithi156 ceph-mon[29302]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-07-16T09:52:18.159 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:52:18 smithi156 ceph-mon[29302]: from='client.? 172.21.15.156:0/1359692668' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-07-16T09:52:18.343 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:52:18 smithi176 ceph-mon[36626]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-07-16T09:52:18.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:52:18 smithi176 ceph-mon[36626]: from='client.? 172.21.15.156:0/1359692668' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-07-16T09:52:18.454 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:18.447+0000 7f15adb6c640 1 -- 172.21.15.156:0/1284018153 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f15a80fd370 msgr2=0x7f15a80fd740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:52:18.455 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:18.447+0000 7f15adb6c640 1 --2- 172.21.15.156:0/1284018153 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f15a80fd370 0x7f15a80fd740 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f1590004900 tx=0x7f1590009da0 comp rx=0 tx=0).stop 2024-07-16T09:52:18.455 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:18.447+0000 7f15adb6c640 1 -- 172.21.15.156:0/1284018153 shutdown_connections 2024-07-16T09:52:18.455 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:18.447+0000 7f15adb6c640 1 --2- 172.21.15.156:0/1284018153 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f15a80fdd10 0x7f15a81063e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:52:18.455 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:18.447+0000 7f15adb6c640 1 --2- 172.21.15.156:0/1284018153 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f15a80fd370 0x7f15a80fd740 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:52:18.455 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:18.447+0000 7f15adb6c640 1 -- 172.21.15.156:0/1284018153 >> 172.21.15.156:0/1284018153 conn(0x7f15a80f6e50 msgr2=0x7f15a80f9260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:52:18.455 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:18.447+0000 7f15adb6c640 1 -- 172.21.15.156:0/1284018153 shutdown_connections 2024-07-16T09:52:18.455 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:18.447+0000 7f15adb6c640 1 -- 172.21.15.156:0/1284018153 wait complete. 2024-07-16T09:52:18.455 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:18.447+0000 7f15adb6c640 1 Processor -- start 2024-07-16T09:52:18.455 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:18.448+0000 7f15adb6c640 1 -- start start 2024-07-16T09:52:18.455 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:18.448+0000 7f15adb6c640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f15a80fd370 0x7f15a8113100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:52:18.455 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:18.448+0000 7f15adb6c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f15a80fdd10 0x7f15a8113640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:52:18.455 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:18.448+0000 7f15adb6c640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15a8113c40 con 0x7f15a80fdd10 2024-07-16T09:52:18.456 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:18.448+0000 7f15adb6c640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15a810d080 con 0x7f15a80fd370 2024-07-16T09:52:18.456 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:18.448+0000 7f15a6ffd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f15a80fdd10 0x7f15a8113640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:52:18.456 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:18.448+0000 7f15a6ffd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f15a80fdd10 0x7f15a8113640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:33926/0 (socket says 172.21.15.156:33926) 2024-07-16T09:52:18.456 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:18.448+0000 7f15a6ffd640 1 -- 172.21.15.156:0/4107275544 learned_addr learned my addr 172.21.15.156:0/4107275544 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T09:52:18.456 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:18.448+0000 7f15a77fe640 1 --2- 172.21.15.156:0/4107275544 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f15a80fd370 0x7f15a8113100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:52:18.456 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:18.448+0000 7f15a6ffd640 1 -- 172.21.15.156:0/4107275544 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f15a80fd370 msgr2=0x7f15a8113100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:52:18.456 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:18.448+0000 7f15a6ffd640 1 --2- 172.21.15.156:0/4107275544 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f15a80fd370 0x7f15a8113100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:52:18.456 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:18.448+0000 7f15a6ffd640 1 -- 172.21.15.156:0/4107275544 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1590005040 con 0x7f15a80fdd10 2024-07-16T09:52:18.456 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:18.448+0000 7f15a6ffd640 1 --2- 172.21.15.156:0/4107275544 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f15a80fdd10 0x7f15a8113640 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f1598002a50 tx=0x7f1598004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:52:18.456 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:18.449+0000 7f159ffff640 1 -- 172.21.15.156:0/4107275544 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1598007c30 con 0x7f15a80fdd10 2024-07-16T09:52:18.456 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:18.449+0000 7f159ffff640 1 -- 172.21.15.156:0/4107275544 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f159800f040 con 0x7f15a80fdd10 2024-07-16T09:52:18.456 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:18.449+0000 7f159ffff640 1 -- 172.21.15.156:0/4107275544 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f159800db60 con 0x7f15a80fdd10 2024-07-16T09:52:18.456 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:18.449+0000 7f15adb6c640 1 -- 172.21.15.156:0/4107275544 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15a810d360 con 0x7f15a80fdd10 2024-07-16T09:52:18.457 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:18.449+0000 7f15adb6c640 1 -- 172.21.15.156:0/4107275544 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15a810d830 con 0x7f15a80fdd10 2024-07-16T09:52:18.457 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:18.450+0000 7f15adb6c640 1 -- 172.21.15.156:0/4107275544 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15a8066120 con 0x7f15a80fdd10 2024-07-16T09:52:18.457 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:18.450+0000 7f159ffff640 1 -- 172.21.15.156:0/4107275544 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 19) v1 ==== 85127+0+0 (secure 0 0 0) 0x7f159800dcc0 con 0x7f15a80fdd10 2024-07-16T09:52:18.457 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:18.451+0000 7f159ffff640 1 --2- 172.21.15.156:0/4107275544 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f1594067190 0x7f1594069640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:52:18.457 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:18.451+0000 7f159ffff640 1 -- 172.21.15.156:0/4107275544 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f1598084600 con 0x7f15a80fdd10 2024-07-16T09:52:18.457 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:18.451+0000 7f15a77fe640 1 --2- 172.21.15.156:0/4107275544 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f1594067190 0x7f1594069640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:52:18.457 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:18.451+0000 7f15a77fe640 1 --2- 172.21.15.156:0/4107275544 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f1594067190 0x7f1594069640 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f1590000c00 tx=0x7f1590035040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:52:18.457 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:18.455+0000 7f159ffff640 1 -- 172.21.15.156:0/4107275544 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f1598051be0 con 0x7f15a80fdd10 2024-07-16T09:52:18.623 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:18.621+0000 7f15adb6c640 1 -- 172.21.15.156:0/4107275544 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f15a80faa50 con 0x7f15a80fdd10 2024-07-16T09:52:18.623 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:18.622+0000 7f159ffff640 1 -- 172.21.15.156:0/4107275544 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f1598055200 con 0x7f15a80fdd10 2024-07-16T09:52:18.623 INFO:teuthology.orchestra.run.smithi156.stdout:38654705700 2024-07-16T09:52:18.625 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:18.624+0000 7f15adb6c640 1 -- 172.21.15.156:0/4107275544 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f1594067190 msgr2=0x7f1594069640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:52:18.626 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:18.624+0000 7f15adb6c640 1 --2- 172.21.15.156:0/4107275544 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f1594067190 0x7f1594069640 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f1590000c00 tx=0x7f1590035040 comp rx=0 tx=0).stop 2024-07-16T09:52:18.626 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:18.624+0000 7f15adb6c640 1 -- 172.21.15.156:0/4107275544 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f15a80fdd10 msgr2=0x7f15a8113640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:52:18.626 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:18.624+0000 7f15adb6c640 1 --2- 172.21.15.156:0/4107275544 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f15a80fdd10 0x7f15a8113640 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f1598002a50 tx=0x7f1598004600 comp rx=0 tx=0).stop 2024-07-16T09:52:18.626 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:18.624+0000 7f15adb6c640 1 -- 172.21.15.156:0/4107275544 shutdown_connections 2024-07-16T09:52:18.626 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:18.624+0000 7f15adb6c640 1 --2- 172.21.15.156:0/4107275544 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f1594067190 0x7f1594069640 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:52:18.626 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:18.624+0000 7f15adb6c640 1 --2- 172.21.15.156:0/4107275544 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f15a80fdd10 0x7f15a8113640 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:52:18.626 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:18.624+0000 7f15adb6c640 1 --2- 172.21.15.156:0/4107275544 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f15a80fd370 0x7f15a8113100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:52:18.626 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:18.624+0000 7f15adb6c640 1 -- 172.21.15.156:0/4107275544 >> 172.21.15.156:0/4107275544 conn(0x7f15a80f6e50 msgr2=0x7f15a80f82f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:52:18.626 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:18.624+0000 7f15adb6c640 1 -- 172.21.15.156:0/4107275544 shutdown_connections 2024-07-16T09:52:18.626 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:18.624+0000 7f15adb6c640 1 -- 172.21.15.156:0/4107275544 wait complete. 2024-07-16T09:52:19.343 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:52:19 smithi176 ceph-mon[36626]: from='client.? 172.21.15.156:0/4107275544' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-07-16T09:52:19.424 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:52:19 smithi156 ceph-mon[29302]: from='client.? 172.21.15.156:0/4107275544' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-07-16T09:52:20.346 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:52:20 smithi176 ceph-mon[36626]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-07-16T09:52:20.348 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:52:20 smithi156 ceph-mon[29302]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-07-16T09:52:20.426 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:20.425+0000 7f7fac0b8640 1 -- 172.21.15.156:0/2500472984 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7fa40714f0 msgr2=0x7f7fa406f8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:52:20.427 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:20.425+0000 7f7fac0b8640 1 --2- 172.21.15.156:0/2500472984 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7fa40714f0 0x7f7fa406f8c0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f7f980048d0 tx=0x7f7f98009da0 comp rx=0 tx=0).stop 2024-07-16T09:52:20.427 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:20.425+0000 7f7fac0b8640 1 -- 172.21.15.156:0/2500472984 shutdown_connections 2024-07-16T09:52:20.427 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:20.425+0000 7f7fac0b8640 1 --2- 172.21.15.156:0/2500472984 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7fa406fe00 0x7f7fa4070270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:52:20.427 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:20.425+0000 7f7fac0b8640 1 --2- 172.21.15.156:0/2500472984 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7fa40714f0 0x7f7fa406f8c0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:52:20.427 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:20.425+0000 7f7fac0b8640 1 -- 172.21.15.156:0/2500472984 >> 172.21.15.156:0/2500472984 conn(0x7f7fa40f7f00 msgr2=0x7f7fa40fa310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:52:20.427 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:20.425+0000 7f7fac0b8640 1 -- 172.21.15.156:0/2500472984 shutdown_connections 2024-07-16T09:52:20.427 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:20.425+0000 7f7fac0b8640 1 -- 172.21.15.156:0/2500472984 wait complete. 2024-07-16T09:52:20.429 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:20.426+0000 7f7fac0b8640 1 Processor -- start 2024-07-16T09:52:20.429 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:20.426+0000 7f7fac0b8640 1 -- start start 2024-07-16T09:52:20.429 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:20.426+0000 7f7fac0b8640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7fa406fe00 0x7f7fa4119a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:52:20.429 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:20.426+0000 7f7fac0b8640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7fa40714f0 0x7f7fa4119f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:52:20.429 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:20.426+0000 7f7fac0b8640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fa411a5b0 con 0x7f7fa40714f0 2024-07-16T09:52:20.429 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:20.426+0000 7f7fac0b8640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fa4113a10 con 0x7f7fa406fe00 2024-07-16T09:52:20.429 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:20.426+0000 7f7fa9653640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7fa40714f0 0x7f7fa4119f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:52:20.429 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:20.426+0000 7f7fa9653640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7fa40714f0 0x7f7fa4119f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:33944/0 (socket says 172.21.15.156:33944) 2024-07-16T09:52:20.429 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:20.426+0000 7f7fa9653640 1 -- 172.21.15.156:0/866805194 learned_addr learned my addr 172.21.15.156:0/866805194 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T09:52:20.429 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:20.426+0000 7f7fa9653640 1 -- 172.21.15.156:0/866805194 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7fa406fe00 msgr2=0x7f7fa4119a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:52:20.429 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:20.426+0000 7f7fa9e54640 1 --2- 172.21.15.156:0/866805194 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7fa406fe00 0x7f7fa4119a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:52:20.429 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:20.426+0000 7f7fa9653640 1 --2- 172.21.15.156:0/866805194 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7fa406fe00 0x7f7fa4119a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:52:20.430 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:20.426+0000 7f7fa9653640 1 -- 172.21.15.156:0/866805194 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f98005040 con 0x7f7fa40714f0 2024-07-16T09:52:20.430 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:20.426+0000 7f7fa9653640 1 --2- 172.21.15.156:0/866805194 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7fa40714f0 0x7f7fa4119f60 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f7fa000eb70 tx=0x7f7fa000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:52:20.430 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:20.427+0000 7f7f967fc640 1 -- 172.21.15.156:0/866805194 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7fa0007c30 con 0x7f7fa40714f0 2024-07-16T09:52:20.430 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:20.427+0000 7f7f967fc640 1 -- 172.21.15.156:0/866805194 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7fa000f040 con 0x7f7fa40714f0 2024-07-16T09:52:20.430 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:20.427+0000 7f7fac0b8640 1 -- 172.21.15.156:0/866805194 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7fa4113c90 con 0x7f7fa40714f0 2024-07-16T09:52:20.430 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:20.427+0000 7f7f967fc640 1 -- 172.21.15.156:0/866805194 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7fa0009680 con 0x7f7fa40714f0 2024-07-16T09:52:20.430 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:20.427+0000 7f7fac0b8640 1 -- 172.21.15.156:0/866805194 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fa41141b0 con 0x7f7fa40714f0 2024-07-16T09:52:20.431 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:20.428+0000 7f7fac0b8640 1 -- 172.21.15.156:0/866805194 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7fa4071000 con 0x7f7fa40714f0 2024-07-16T09:52:20.431 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:20.428+0000 7f7f967fc640 1 -- 172.21.15.156:0/866805194 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 19) v1 ==== 85127+0+0 (secure 0 0 0) 0x7f7fa000c770 con 0x7f7fa40714f0 2024-07-16T09:52:20.431 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:20.428+0000 7f7f967fc640 1 --2- 172.21.15.156:0/866805194 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f7f90067190 0x7f7f90069640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:52:20.431 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:20.429+0000 7f7f967fc640 1 -- 172.21.15.156:0/866805194 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f7fa00844c0 con 0x7f7fa40714f0 2024-07-16T09:52:20.431 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:20.429+0000 7f7fa9e54640 1 --2- 172.21.15.156:0/866805194 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f7f90067190 0x7f7f90069640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:52:20.431 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:20.429+0000 7f7fa9e54640 1 --2- 172.21.15.156:0/866805194 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f7f90067190 0x7f7f90069640 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f7f980043d0 tx=0x7f7f98009da0 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:52:20.434 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:20.433+0000 7f7f967fc640 1 -- 172.21.15.156:0/866805194 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f7fa0051b50 con 0x7f7fa40714f0 2024-07-16T09:52:20.603 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:20.601+0000 7f7fac0b8640 1 -- 172.21.15.156:0/866805194 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f7fa410fa70 con 0x7f7fa40714f0 2024-07-16T09:52:20.603 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:20.602+0000 7f7f967fc640 1 -- 172.21.15.156:0/866805194 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f7fa0055170 con 0x7f7fa40714f0 2024-07-16T09:52:20.604 INFO:teuthology.orchestra.run.smithi156.stdout:81604378648 2024-07-16T09:52:20.605 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:20.604+0000 7f7fac0b8640 1 -- 172.21.15.156:0/866805194 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f7f90067190 msgr2=0x7f7f90069640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:52:20.605 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:20.604+0000 7f7fac0b8640 1 --2- 172.21.15.156:0/866805194 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f7f90067190 0x7f7f90069640 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f7f980043d0 tx=0x7f7f98009da0 comp rx=0 tx=0).stop 2024-07-16T09:52:20.605 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:20.604+0000 7f7fac0b8640 1 -- 172.21.15.156:0/866805194 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7fa40714f0 msgr2=0x7f7fa4119f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:52:20.605 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:20.604+0000 7f7fac0b8640 1 --2- 172.21.15.156:0/866805194 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7fa40714f0 0x7f7fa4119f60 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f7fa000eb70 tx=0x7f7fa000b040 comp rx=0 tx=0).stop 2024-07-16T09:52:20.606 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:20.604+0000 7f7fac0b8640 1 -- 172.21.15.156:0/866805194 shutdown_connections 2024-07-16T09:52:20.606 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:20.604+0000 7f7fac0b8640 1 --2- 172.21.15.156:0/866805194 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f7f90067190 0x7f7f90069640 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:52:20.606 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:20.604+0000 7f7fac0b8640 1 --2- 172.21.15.156:0/866805194 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7fa40714f0 0x7f7fa4119f60 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:52:20.606 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:20.604+0000 7f7fac0b8640 1 --2- 172.21.15.156:0/866805194 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7fa406fe00 0x7f7fa4119a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:52:20.606 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:20.604+0000 7f7fac0b8640 1 -- 172.21.15.156:0/866805194 >> 172.21.15.156:0/866805194 conn(0x7f7fa40f7f00 msgr2=0x7f7fa40f9170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:52:20.606 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:20.604+0000 7f7fac0b8640 1 -- 172.21.15.156:0/866805194 shutdown_connections 2024-07-16T09:52:20.606 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:20.604+0000 7f7fac0b8640 1 -- 172.21.15.156:0/866805194 wait complete. 2024-07-16T09:52:20.860 INFO:tasks.cephadm.ceph_manager.ceph:need seq 60129542172 got 60129542174 for osd.1 2024-07-16T09:52:20.860 DEBUG:teuthology.parallel:result is None 2024-07-16T09:52:21.163 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:52:21 smithi156 ceph-mon[29302]: from='client.? 172.21.15.156:0/866805194' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-07-16T09:52:21.343 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:52:21 smithi176 ceph-mon[36626]: from='client.? 172.21.15.156:0/866805194' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-07-16T09:52:22.107 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:52:22 smithi156 ceph-mon[29302]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-07-16T09:52:22.245 INFO:tasks.cephadm.ceph_manager.ceph:need seq 81604378646 got 81604378648 for osd.2 2024-07-16T09:52:22.246 DEBUG:teuthology.parallel:result is None 2024-07-16T09:52:22.343 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:52:22 smithi176 ceph-mon[36626]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-07-16T09:52:22.619 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/26c91c26-4358-11ef-bcad-c7b262605968/mon.smithi156/config 2024-07-16T09:52:22.642 INFO:tasks.cephadm.ceph_manager.ceph:need seq 38654705698 got 38654705700 for osd.0 2024-07-16T09:52:22.643 DEBUG:teuthology.parallel:result is None 2024-07-16T09:52:23.499 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:23.463+0000 7fd578e06640 1 -- 172.21.15.156:0/3893553729 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd5740fb5d0 msgr2=0x7fd57410c9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:52:23.499 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:23.463+0000 7fd578e06640 1 --2- 172.21.15.156:0/3893553729 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd5740fb5d0 0x7fd57410c9e0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fd55c0048d0 tx=0x7fd55c009da0 comp rx=0 tx=0).stop 2024-07-16T09:52:23.499 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:23.463+0000 7fd578e06640 1 -- 172.21.15.156:0/3893553729 shutdown_connections 2024-07-16T09:52:23.499 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:23.463+0000 7fd578e06640 1 --2- 172.21.15.156:0/3893553729 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd5740fb5d0 0x7fd57410c9e0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:52:23.499 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:23.463+0000 7fd578e06640 1 --2- 172.21.15.156:0/3893553729 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd5740facc0 0x7fd5740fb090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:52:23.499 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:23.463+0000 7fd578e06640 1 -- 172.21.15.156:0/3893553729 >> 172.21.15.156:0/3893553729 conn(0x7fd574070af0 msgr2=0x7fd574070ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:52:23.499 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:23.463+0000 7fd578e06640 1 -- 172.21.15.156:0/3893553729 shutdown_connections 2024-07-16T09:52:23.499 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:23.463+0000 7fd578e06640 1 -- 172.21.15.156:0/3893553729 wait complete. 2024-07-16T09:52:23.499 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:23.464+0000 7fd578e06640 1 Processor -- start 2024-07-16T09:52:23.499 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:23.464+0000 7fd578e06640 1 -- start start 2024-07-16T09:52:23.500 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:23.464+0000 7fd578e06640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd5740facc0 0x7fd574119780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:52:23.500 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:23.464+0000 7fd578e06640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd5740fb5d0 0x7fd574119cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:52:23.500 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:23.464+0000 7fd578e06640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd57411a310 con 0x7fd5740facc0 2024-07-16T09:52:23.500 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:23.464+0000 7fd578e06640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd574113720 con 0x7fd5740fb5d0 2024-07-16T09:52:23.500 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:23.464+0000 7fd57259c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd5740facc0 0x7fd574119780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:52:23.500 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:23.464+0000 7fd57259c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd5740facc0 0x7fd574119780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:59156/0 (socket says 172.21.15.156:59156) 2024-07-16T09:52:23.500 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:23.464+0000 7fd57259c640 1 -- 172.21.15.156:0/2901435423 learned_addr learned my addr 172.21.15.156:0/2901435423 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T09:52:23.500 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:23.464+0000 7fd57259c640 1 -- 172.21.15.156:0/2901435423 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd5740fb5d0 msgr2=0x7fd574119cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:52:23.500 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:23.464+0000 7fd57259c640 1 --2- 172.21.15.156:0/2901435423 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd5740fb5d0 0x7fd574119cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:52:23.500 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:23.464+0000 7fd57259c640 1 -- 172.21.15.156:0/2901435423 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd55c005040 con 0x7fd5740facc0 2024-07-16T09:52:23.500 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:23.465+0000 7fd57259c640 1 --2- 172.21.15.156:0/2901435423 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd5740facc0 0x7fd574119780 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fd56400c790 tx=0x7fd5640043c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:52:23.500 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:23.465+0000 7fd56affd640 1 -- 172.21.15.156:0/2901435423 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd56400a820 con 0x7fd5740facc0 2024-07-16T09:52:23.501 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:23.465+0000 7fd56affd640 1 -- 172.21.15.156:0/2901435423 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd56400ae50 con 0x7fd5740facc0 2024-07-16T09:52:23.501 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:23.465+0000 7fd578e06640 1 -- 172.21.15.156:0/2901435423 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd574113a00 con 0x7fd5740facc0 2024-07-16T09:52:23.501 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:23.465+0000 7fd56affd640 1 -- 172.21.15.156:0/2901435423 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd5640115e0 con 0x7fd5740facc0 2024-07-16T09:52:23.501 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:23.465+0000 7fd578e06640 1 -- 172.21.15.156:0/2901435423 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd574113f20 con 0x7fd5740facc0 2024-07-16T09:52:23.501 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:23.466+0000 7fd578e06640 1 -- 172.21.15.156:0/2901435423 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5740fbaa0 con 0x7fd5740facc0 2024-07-16T09:52:23.501 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:23.466+0000 7fd56affd640 1 -- 172.21.15.156:0/2901435423 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 19) v1 ==== 85127+0+0 (secure 0 0 0) 0x7fd564012400 con 0x7fd5740facc0 2024-07-16T09:52:23.501 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:23.467+0000 7fd56affd640 1 --2- 172.21.15.156:0/2901435423 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7fd560067190 0x7fd560069640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:52:23.501 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:23.467+0000 7fd56affd640 1 -- 172.21.15.156:0/2901435423 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fd5640849a0 con 0x7fd5740facc0 2024-07-16T09:52:23.501 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:23.467+0000 7fd56bfff640 1 --2- 172.21.15.156:0/2901435423 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7fd560067190 0x7fd560069640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:52:23.501 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:23.467+0000 7fd56bfff640 1 --2- 172.21.15.156:0/2901435423 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7fd560067190 0x7fd560069640 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fd55c02a520 tx=0x7fd55c009da0 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:52:23.501 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:23.471+0000 7fd56affd640 1 -- 172.21.15.156:0/2901435423 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fd564052000 con 0x7fd5740facc0 2024-07-16T09:52:23.641 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:23.639+0000 7fd578e06640 1 -- 172.21.15.156:0/2901435423 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7fd57410fb00 con 0x7fd5740facc0 2024-07-16T09:52:23.641 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:23.639+0000 7fd56affd640 1 -- 172.21.15.156:0/2901435423 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd564055620 con 0x7fd5740facc0 2024-07-16T09:52:23.641 INFO:teuthology.orchestra.run.smithi156.stdout:171798691851 2024-07-16T09:52:23.644 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:23.641+0000 7fd578e06640 1 -- 172.21.15.156:0/2901435423 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7fd560067190 msgr2=0x7fd560069640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:52:23.644 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:23.641+0000 7fd578e06640 1 --2- 172.21.15.156:0/2901435423 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7fd560067190 0x7fd560069640 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fd55c02a520 tx=0x7fd55c009da0 comp rx=0 tx=0).stop 2024-07-16T09:52:23.644 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:23.641+0000 7fd578e06640 1 -- 172.21.15.156:0/2901435423 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd5740facc0 msgr2=0x7fd574119780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:52:23.644 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:23.641+0000 7fd578e06640 1 --2- 172.21.15.156:0/2901435423 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd5740facc0 0x7fd574119780 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fd56400c790 tx=0x7fd5640043c0 comp rx=0 tx=0).stop 2024-07-16T09:52:23.644 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:23.641+0000 7fd578e06640 1 -- 172.21.15.156:0/2901435423 shutdown_connections 2024-07-16T09:52:23.644 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:23.641+0000 7fd578e06640 1 --2- 172.21.15.156:0/2901435423 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7fd560067190 0x7fd560069640 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:52:23.644 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:23.641+0000 7fd578e06640 1 --2- 172.21.15.156:0/2901435423 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd5740fb5d0 0x7fd574119cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:52:23.644 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:23.641+0000 7fd578e06640 1 --2- 172.21.15.156:0/2901435423 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd5740facc0 0x7fd574119780 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:52:23.644 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:23.641+0000 7fd578e06640 1 -- 172.21.15.156:0/2901435423 >> 172.21.15.156:0/2901435423 conn(0x7fd574070af0 msgr2=0x7fd5740fbf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:52:23.644 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:23.641+0000 7fd578e06640 1 -- 172.21.15.156:0/2901435423 shutdown_connections 2024-07-16T09:52:23.644 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:23.641+0000 7fd578e06640 1 -- 172.21.15.156:0/2901435423 wait complete. 2024-07-16T09:52:23.722 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:23.709+0000 7f563f2df640 1 -- 172.21.15.156:0/4182629065 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f563806f860 msgr2=0x7f563806fc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:52:23.722 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:23.709+0000 7f563f2df640 1 --2- 172.21.15.156:0/4182629065 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f563806f860 0x7f563806fc30 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f562c0048d0 tx=0x7f562c009da0 comp rx=0 tx=0).stop 2024-07-16T09:52:23.722 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:23.709+0000 7f563f2df640 1 -- 172.21.15.156:0/4182629065 shutdown_connections 2024-07-16T09:52:23.723 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:23.709+0000 7f563f2df640 1 --2- 172.21.15.156:0/4182629065 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5638070170 0x7f56381086e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:52:23.723 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:23.709+0000 7f563f2df640 1 --2- 172.21.15.156:0/4182629065 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f563806f860 0x7f563806fc30 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:52:23.723 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:23.709+0000 7f563f2df640 1 -- 172.21.15.156:0/4182629065 >> 172.21.15.156:0/4182629065 conn(0x7f56380f7f20 msgr2=0x7f56380fa330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:52:23.723 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:23.709+0000 7f563f2df640 1 -- 172.21.15.156:0/4182629065 shutdown_connections 2024-07-16T09:52:23.723 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:23.709+0000 7f563f2df640 1 -- 172.21.15.156:0/4182629065 wait complete. 2024-07-16T09:52:23.723 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:23.709+0000 7f563f2df640 1 Processor -- start 2024-07-16T09:52:23.723 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:23.709+0000 7f563f2df640 1 -- start start 2024-07-16T09:52:23.723 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:23.710+0000 7f563f2df640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f563806f860 0x7f56381153a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:52:23.723 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:23.710+0000 7f563f2df640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5638070170 0x7f56381158e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:52:23.723 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:23.710+0000 7f563f2df640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5638115f70 con 0x7f5638070170 2024-07-16T09:52:23.723 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:23.710+0000 7f563f2df640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f563810f320 con 0x7f563806f860 2024-07-16T09:52:23.723 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:23.710+0000 7f563c87a640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5638070170 0x7f56381158e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:52:23.723 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:23.710+0000 7f563c87a640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5638070170 0x7f56381158e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:59166/0 (socket says 172.21.15.156:59166) 2024-07-16T09:52:23.724 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:23.710+0000 7f563c87a640 1 -- 172.21.15.156:0/412210388 learned_addr learned my addr 172.21.15.156:0/412210388 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T09:52:23.724 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:23.710+0000 7f563c87a640 1 -- 172.21.15.156:0/412210388 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f563806f860 msgr2=0x7f56381153a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:52:23.724 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:23.710+0000 7f563c87a640 1 --2- 172.21.15.156:0/412210388 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f563806f860 0x7f56381153a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:52:23.724 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:23.710+0000 7f563c87a640 1 -- 172.21.15.156:0/412210388 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f562c005040 con 0x7f5638070170 2024-07-16T09:52:23.724 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:23.710+0000 7f563c87a640 1 --2- 172.21.15.156:0/412210388 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5638070170 0x7f56381158e0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f563400b790 tx=0x7f56340091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:52:23.724 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:23.710+0000 7f5629ffb640 1 -- 172.21.15.156:0/412210388 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f563400c700 con 0x7f5638070170 2024-07-16T09:52:23.724 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:23.710+0000 7f5629ffb640 1 -- 172.21.15.156:0/412210388 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f563400cd30 con 0x7f5638070170 2024-07-16T09:52:23.724 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:23.710+0000 7f563f2df640 1 -- 172.21.15.156:0/412210388 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f563810f600 con 0x7f5638070170 2024-07-16T09:52:23.724 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:23.710+0000 7f5629ffb640 1 -- 172.21.15.156:0/412210388 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5634013370 con 0x7f5638070170 2024-07-16T09:52:23.724 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:23.710+0000 7f563f2df640 1 -- 172.21.15.156:0/412210388 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f563810fb20 con 0x7f5638070170 2024-07-16T09:52:23.724 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:23.711+0000 7f563f2df640 1 -- 172.21.15.156:0/412210388 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5638070fa0 con 0x7f5638070170 2024-07-16T09:52:23.724 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:23.712+0000 7f5629ffb640 1 -- 172.21.15.156:0/412210388 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 19) v1 ==== 85127+0+0 (secure 0 0 0) 0x7f5634012070 con 0x7f5638070170 2024-07-16T09:52:23.724 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:23.712+0000 7f5629ffb640 1 --2- 172.21.15.156:0/412210388 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f5624067190 0x7f5624069640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:52:23.725 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:23.712+0000 7f5629ffb640 1 -- 172.21.15.156:0/412210388 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f56340843d0 con 0x7f5638070170 2024-07-16T09:52:23.725 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:23.712+0000 7f563d07b640 1 --2- 172.21.15.156:0/412210388 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f5624067190 0x7f5624069640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:52:23.725 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:23.713+0000 7f563d07b640 1 --2- 172.21.15.156:0/412210388 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f5624067190 0x7f5624069640 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f562c02a530 tx=0x7f562c009da0 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:52:23.725 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:23.716+0000 7f5629ffb640 1 -- 172.21.15.156:0/412210388 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f56340519b0 con 0x7f5638070170 2024-07-16T09:52:23.887 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:23.884+0000 7f563f2df640 1 -- 172.21.15.156:0/412210388 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f563810b750 con 0x7f5638070170 2024-07-16T09:52:23.887 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:23.884+0000 7f5629ffb640 1 -- 172.21.15.156:0/412210388 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f5634054fd0 con 0x7f5638070170 2024-07-16T09:52:23.887 INFO:teuthology.orchestra.run.smithi156.stdout:150323855376 2024-07-16T09:52:23.889 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:23.886+0000 7f563f2df640 1 -- 172.21.15.156:0/412210388 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f5624067190 msgr2=0x7f5624069640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:52:23.889 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:23.886+0000 7f563f2df640 1 --2- 172.21.15.156:0/412210388 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f5624067190 0x7f5624069640 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f562c02a530 tx=0x7f562c009da0 comp rx=0 tx=0).stop 2024-07-16T09:52:23.889 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:23.886+0000 7f563f2df640 1 -- 172.21.15.156:0/412210388 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5638070170 msgr2=0x7f56381158e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:52:23.889 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:23.886+0000 7f563f2df640 1 --2- 172.21.15.156:0/412210388 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5638070170 0x7f56381158e0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f563400b790 tx=0x7f56340091d0 comp rx=0 tx=0).stop 2024-07-16T09:52:23.889 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:23.887+0000 7f563f2df640 1 -- 172.21.15.156:0/412210388 shutdown_connections 2024-07-16T09:52:23.889 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:23.887+0000 7f563f2df640 1 --2- 172.21.15.156:0/412210388 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f5624067190 0x7f5624069640 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:52:23.890 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:23.887+0000 7f563f2df640 1 --2- 172.21.15.156:0/412210388 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5638070170 0x7f56381158e0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:52:23.890 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:23.887+0000 7f563f2df640 1 --2- 172.21.15.156:0/412210388 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f563806f860 0x7f56381153a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:52:23.890 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:23.887+0000 7f563f2df640 1 -- 172.21.15.156:0/412210388 >> 172.21.15.156:0/412210388 conn(0x7f56380f7f20 msgr2=0x7f56380f93f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:52:23.890 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:23.887+0000 7f563f2df640 1 -- 172.21.15.156:0/412210388 shutdown_connections 2024-07-16T09:52:23.890 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:23.887+0000 7f563f2df640 1 -- 172.21.15.156:0/412210388 wait complete. 2024-07-16T09:52:24.343 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:52:24 smithi176 ceph-mon[36626]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-07-16T09:52:24.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:52:24 smithi176 ceph-mon[36626]: from='client.? 172.21.15.156:0/2901435423' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-07-16T09:52:24.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:52:24 smithi176 ceph-mon[36626]: from='client.? 172.21.15.156:0/412210388' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-07-16T09:52:24.409 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:52:24 smithi156 ceph-mon[29302]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-07-16T09:52:24.409 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:52:24 smithi156 ceph-mon[29302]: from='client.? 172.21.15.156:0/2901435423' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-07-16T09:52:24.410 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:52:24 smithi156 ceph-mon[29302]: from='client.? 172.21.15.156:0/412210388' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-07-16T09:52:25.020 INFO:tasks.cephadm.ceph_manager.ceph:need seq 150323855374 got 150323855376 for osd.4 2024-07-16T09:52:25.020 DEBUG:teuthology.parallel:result is None 2024-07-16T09:52:25.343 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:52:25 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-16T09:52:25.399 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:52:25 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-16T09:52:26.338 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:52:26 smithi156 ceph-mon[29302]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-07-16T09:52:26.338 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:52:26 smithi156 ceph-mon[29302]: from='client.? 172.21.15.156:0/2800422172' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-07-16T09:52:26.339 INFO:tasks.cephadm.ceph_manager.ceph:need seq 171798691848 got 171798691851 for osd.5 2024-07-16T09:52:26.339 DEBUG:teuthology.parallel:result is None 2024-07-16T09:52:26.343 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:52:26 smithi176 ceph-mon[36626]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-07-16T09:52:26.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:52:26 smithi176 ceph-mon[36626]: from='client.? 172.21.15.156:0/2800422172' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-07-16T09:52:26.448 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:25.895+0000 7f98dfde5640 1 -- 172.21.15.156:0/1084003586 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f98d80fdf60 msgr2=0x7f98d80fe390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:52:26.448 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:25.895+0000 7f98dfde5640 1 --2- 172.21.15.156:0/1084003586 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f98d80fdf60 0x7f98d80fe390 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f98c80048d0 tx=0x7f98c8009da0 comp rx=0 tx=0).stop 2024-07-16T09:52:26.448 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:25.895+0000 7f98dfde5640 1 -- 172.21.15.156:0/1084003586 shutdown_connections 2024-07-16T09:52:26.448 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:25.895+0000 7f98dfde5640 1 --2- 172.21.15.156:0/1084003586 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f98d80fdf60 0x7f98d80fe390 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:52:26.448 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:25.895+0000 7f98dfde5640 1 --2- 172.21.15.156:0/1084003586 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f98d8065150 0x7f98d8065520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:52:26.448 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:25.895+0000 7f98dfde5640 1 -- 172.21.15.156:0/1084003586 >> 172.21.15.156:0/1084003586 conn(0x7f98d80721c0 msgr2=0x7f98d80725c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:52:26.448 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:25.896+0000 7f98dfde5640 1 -- 172.21.15.156:0/1084003586 shutdown_connections 2024-07-16T09:52:26.448 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:25.896+0000 7f98dfde5640 1 -- 172.21.15.156:0/1084003586 wait complete. 2024-07-16T09:52:26.448 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:25.896+0000 7f98dfde5640 1 Processor -- start 2024-07-16T09:52:26.448 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:25.896+0000 7f98dfde5640 1 -- start start 2024-07-16T09:52:26.448 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:25.896+0000 7f98dfde5640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f98d8065150 0x7f98d81130c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:52:26.448 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:25.897+0000 7f98dfde5640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f98d80fdf60 0x7f98d8113600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:52:26.448 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:25.897+0000 7f98dfde5640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98d8113c00 con 0x7f98d8065150 2024-07-16T09:52:26.449 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:25.897+0000 7f98dfde5640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98d810d040 con 0x7f98d80fdf60 2024-07-16T09:52:26.449 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:25.897+0000 7f98ddb81640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f98d8065150 0x7f98d81130c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:52:26.449 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:25.897+0000 7f98ddb81640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f98d8065150 0x7f98d81130c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:59172/0 (socket says 172.21.15.156:59172) 2024-07-16T09:52:26.449 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:25.897+0000 7f98ddb81640 1 -- 172.21.15.156:0/2800422172 learned_addr learned my addr 172.21.15.156:0/2800422172 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T09:52:26.449 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:25.897+0000 7f98dd380640 1 --2- 172.21.15.156:0/2800422172 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f98d80fdf60 0x7f98d8113600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:52:26.449 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:25.897+0000 7f98ddb81640 1 -- 172.21.15.156:0/2800422172 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f98d80fdf60 msgr2=0x7f98d8113600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:52:26.449 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:25.897+0000 7f98ddb81640 1 --2- 172.21.15.156:0/2800422172 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f98d80fdf60 0x7f98d8113600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:52:26.449 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:25.897+0000 7f98ddb81640 1 -- 172.21.15.156:0/2800422172 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98c8005040 con 0x7f98d8065150 2024-07-16T09:52:26.449 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:25.897+0000 7f98ddb81640 1 --2- 172.21.15.156:0/2800422172 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f98d8065150 0x7f98d81130c0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f98d400db00 tx=0x7f98d4004e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:52:26.449 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:25.897+0000 7f98ce7fc640 1 -- 172.21.15.156:0/2800422172 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f98d4017070 con 0x7f98d8065150 2024-07-16T09:52:26.449 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:25.897+0000 7f98dfde5640 1 -- 172.21.15.156:0/2800422172 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98d810d320 con 0x7f98d8065150 2024-07-16T09:52:26.450 INFO:teuthology.orchestra.run.smithi156.stdout:115964117014 2024-07-16T09:52:26.450 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:25.897+0000 7f98ce7fc640 1 -- 172.21.15.156:0/2800422172 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f98d400f040 con 0x7f98d8065150 2024-07-16T09:52:26.450 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:25.897+0000 7f98ce7fc640 1 -- 172.21.15.156:0/2800422172 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f98d4003ae0 con 0x7f98d8065150 2024-07-16T09:52:26.450 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:25.897+0000 7f98dfde5640 1 -- 172.21.15.156:0/2800422172 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98d810d8c0 con 0x7f98d8065150 2024-07-16T09:52:26.450 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:25.898+0000 7f98dfde5640 1 -- 172.21.15.156:0/2800422172 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98d80fdb00 con 0x7f98d8065150 2024-07-16T09:52:26.450 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:25.899+0000 7f98ce7fc640 1 -- 172.21.15.156:0/2800422172 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 19) v1 ==== 85127+0+0 (secure 0 0 0) 0x7f98d4003c60 con 0x7f98d8065150 2024-07-16T09:52:26.450 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:25.899+0000 7f98ce7fc640 1 --2- 172.21.15.156:0/2800422172 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f98c4067190 0x7f98c4069640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:52:26.450 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:25.899+0000 7f98ce7fc640 1 -- 172.21.15.156:0/2800422172 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f98d4084800 con 0x7f98d8065150 2024-07-16T09:52:26.450 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:25.899+0000 7f98dd380640 1 --2- 172.21.15.156:0/2800422172 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f98c4067190 0x7f98c4069640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:52:26.450 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:25.899+0000 7f98dd380640 1 --2- 172.21.15.156:0/2800422172 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f98c4067190 0x7f98c4069640 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f98c8000c00 tx=0x7f98c8009da0 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:52:26.451 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:25.903+0000 7f98ce7fc640 1 -- 172.21.15.156:0/2800422172 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f98d4051e60 con 0x7f98d8065150 2024-07-16T09:52:26.451 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:26.072+0000 7f98dfde5640 1 -- 172.21.15.156:0/2800422172 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f98d804b560 con 0x7f98d8065150 2024-07-16T09:52:26.451 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:26.073+0000 7f98ce7fc640 1 -- 172.21.15.156:0/2800422172 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f98d4055480 con 0x7f98d8065150 2024-07-16T09:52:26.451 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:26.075+0000 7f98dfde5640 1 -- 172.21.15.156:0/2800422172 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f98c4067190 msgr2=0x7f98c4069640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:52:26.451 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:26.075+0000 7f98dfde5640 1 --2- 172.21.15.156:0/2800422172 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f98c4067190 0x7f98c4069640 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f98c8000c00 tx=0x7f98c8009da0 comp rx=0 tx=0).stop 2024-07-16T09:52:26.451 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:26.075+0000 7f98dfde5640 1 -- 172.21.15.156:0/2800422172 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f98d8065150 msgr2=0x7f98d81130c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:52:26.451 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:26.075+0000 7f98dfde5640 1 --2- 172.21.15.156:0/2800422172 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f98d8065150 0x7f98d81130c0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f98d400db00 tx=0x7f98d4004e60 comp rx=0 tx=0).stop 2024-07-16T09:52:26.451 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:26.075+0000 7f98dfde5640 1 -- 172.21.15.156:0/2800422172 shutdown_connections 2024-07-16T09:52:26.451 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:26.075+0000 7f98dfde5640 1 --2- 172.21.15.156:0/2800422172 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f98c4067190 0x7f98c4069640 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:52:26.451 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:26.075+0000 7f98dfde5640 1 --2- 172.21.15.156:0/2800422172 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f98d80fdf60 0x7f98d8113600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:52:26.451 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:26.075+0000 7f98dfde5640 1 --2- 172.21.15.156:0/2800422172 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f98d8065150 0x7f98d81130c0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:52:26.451 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:26.075+0000 7f98dfde5640 1 -- 172.21.15.156:0/2800422172 >> 172.21.15.156:0/2800422172 conn(0x7f98d80721c0 msgr2=0x7f98d8101080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:52:26.451 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:26.075+0000 7f98dfde5640 1 -- 172.21.15.156:0/2800422172 shutdown_connections 2024-07-16T09:52:26.452 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:26.075+0000 7f98dfde5640 1 -- 172.21.15.156:0/2800422172 wait complete. 2024-07-16T09:52:26.990 INFO:tasks.cephadm.ceph_manager.ceph:need seq 115964117011 got 115964117014 for osd.3 2024-07-16T09:52:26.991 DEBUG:teuthology.parallel:result is None 2024-07-16T09:52:26.991 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-07-16T09:52:26.991 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 26c91c26-4358-11ef-bcad-c7b262605968 -- ceph pg dump --format=json 2024-07-16T09:52:27.254 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/26c91c26-4358-11ef-bcad-c7b262605968/mon.smithi156/config 2024-07-16T09:52:28.380 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:52:28 smithi156 ceph-mon[29302]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-07-16T09:52:28.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:52:28 smithi176 ceph-mon[36626]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-07-16T09:52:28.895 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:28.871+0000 7f2f86987640 1 -- 172.21.15.156:0/1299442723 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2f80102370 msgr2=0x7f2f80102740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:52:28.895 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:28.871+0000 7f2f86987640 1 --2- 172.21.15.156:0/1299442723 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2f80102370 0x7f2f80102740 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f2f700048d0 tx=0x7f2f70009da0 comp rx=0 tx=0).stop 2024-07-16T09:52:28.895 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:28.871+0000 7f2f86987640 1 -- 172.21.15.156:0/1299442723 shutdown_connections 2024-07-16T09:52:28.895 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:28.871+0000 7f2f86987640 1 --2- 172.21.15.156:0/1299442723 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2f80102d10 0x7f2f800fc3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:52:28.895 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:28.871+0000 7f2f86987640 1 --2- 172.21.15.156:0/1299442723 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2f80102370 0x7f2f80102740 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:52:28.895 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:28.871+0000 7f2f86987640 1 -- 172.21.15.156:0/1299442723 >> 172.21.15.156:0/1299442723 conn(0x7f2f800f7f00 msgr2=0x7f2f800fa310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:52:28.895 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:28.871+0000 7f2f86987640 1 -- 172.21.15.156:0/1299442723 shutdown_connections 2024-07-16T09:52:28.896 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:28.871+0000 7f2f86987640 1 -- 172.21.15.156:0/1299442723 wait complete. 2024-07-16T09:52:28.896 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:28.871+0000 7f2f86987640 1 Processor -- start 2024-07-16T09:52:28.896 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:28.871+0000 7f2f86987640 1 -- start start 2024-07-16T09:52:28.896 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:28.872+0000 7f2f86987640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2f80102370 0x7f2f801133a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:52:28.896 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:28.872+0000 7f2f86987640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2f80102d10 0x7f2f801138e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:52:28.896 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:28.872+0000 7f2f86987640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f80113ee0 con 0x7f2f80102d10 2024-07-16T09:52:28.896 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:28.872+0000 7f2f86987640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f80114050 con 0x7f2f80102370 2024-07-16T09:52:28.896 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:28.872+0000 7f2f7f7fe640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2f80102d10 0x7f2f801138e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:52:28.896 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:28.872+0000 7f2f7f7fe640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2f80102d10 0x7f2f801138e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:59186/0 (socket says 172.21.15.156:59186) 2024-07-16T09:52:28.896 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:28.872+0000 7f2f7f7fe640 1 -- 172.21.15.156:0/3405819770 learned_addr learned my addr 172.21.15.156:0/3405819770 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T09:52:28.896 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:28.872+0000 7f2f7ffff640 1 --2- 172.21.15.156:0/3405819770 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2f80102370 0x7f2f801133a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:52:28.896 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:28.872+0000 7f2f7f7fe640 1 -- 172.21.15.156:0/3405819770 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2f80102370 msgr2=0x7f2f801133a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:52:28.896 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:28.872+0000 7f2f7f7fe640 1 --2- 172.21.15.156:0/3405819770 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2f80102370 0x7f2f801133a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:52:28.897 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:28.872+0000 7f2f7f7fe640 1 -- 172.21.15.156:0/3405819770 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f70005040 con 0x7f2f80102d10 2024-07-16T09:52:28.897 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:28.872+0000 7f2f7ffff640 1 --2- 172.21.15.156:0/3405819770 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2f80102370 0x7f2f801133a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-16T09:52:28.897 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:28.872+0000 7f2f7f7fe640 1 --2- 172.21.15.156:0/3405819770 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2f80102d10 0x7f2f801138e0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f2f74002a50 tx=0x7f2f74004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:52:28.897 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:28.872+0000 7f2f7cff9640 1 -- 172.21.15.156:0/3405819770 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2f74007c30 con 0x7f2f80102d10 2024-07-16T09:52:28.897 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:28.872+0000 7f2f86987640 1 -- 172.21.15.156:0/3405819770 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f8010d4d0 con 0x7f2f80102d10 2024-07-16T09:52:28.897 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:28.872+0000 7f2f7cff9640 1 -- 172.21.15.156:0/3405819770 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2f7400f040 con 0x7f2f80102d10 2024-07-16T09:52:28.897 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:28.872+0000 7f2f7cff9640 1 -- 172.21.15.156:0/3405819770 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2f7400db50 con 0x7f2f80102d10 2024-07-16T09:52:28.897 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:28.872+0000 7f2f86987640 1 -- 172.21.15.156:0/3405819770 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f8010da70 con 0x7f2f80102d10 2024-07-16T09:52:28.897 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:28.873+0000 7f2f86987640 1 -- 172.21.15.156:0/3405819770 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f800fdae0 con 0x7f2f80102d10 2024-07-16T09:52:28.897 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:28.874+0000 7f2f7cff9640 1 -- 172.21.15.156:0/3405819770 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 19) v1 ==== 85127+0+0 (secure 0 0 0) 0x7f2f7401c400 con 0x7f2f80102d10 2024-07-16T09:52:28.897 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:28.874+0000 7f2f7cff9640 1 --2- 172.21.15.156:0/3405819770 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f2f68067190 0x7f2f68069640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:52:28.897 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:28.874+0000 7f2f7cff9640 1 -- 172.21.15.156:0/3405819770 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f2f74084120 con 0x7f2f80102d10 2024-07-16T09:52:28.898 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:28.874+0000 7f2f7ffff640 1 --2- 172.21.15.156:0/3405819770 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f2f68067190 0x7f2f68069640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:52:28.898 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:28.875+0000 7f2f7ffff640 1 --2- 172.21.15.156:0/3405819770 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f2f68067190 0x7f2f68069640 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f2f70032430 tx=0x7f2f70035040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:52:28.898 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:28.878+0000 7f2f7cff9640 1 -- 172.21.15.156:0/3405819770 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f2f74051700 con 0x7f2f80102d10 2024-07-16T09:52:29.045 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:29.044+0000 7f2f86987640 1 -- 172.21.15.156:0/3405819770 --> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f2f8004b530 con 0x7f2f68067190 2024-07-16T09:52:29.047 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:29.045+0000 7f2f7cff9640 1 -- 172.21.15.156:0/3405819770 <== mgr.14208 v2:172.21.15.156:6800/2615741717 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+27944 (secure 0 0 0) 0x7f2f8004b530 con 0x7f2f68067190 2024-07-16T09:52:29.047 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-07-16T09:52:29.047 INFO:teuthology.orchestra.run.smithi156.stderr:dumped all 2024-07-16T09:52:29.049 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:29.048+0000 7f2f86987640 1 -- 172.21.15.156:0/3405819770 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f2f68067190 msgr2=0x7f2f68069640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:52:29.049 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:29.048+0000 7f2f86987640 1 --2- 172.21.15.156:0/3405819770 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f2f68067190 0x7f2f68069640 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f2f70032430 tx=0x7f2f70035040 comp rx=0 tx=0).stop 2024-07-16T09:52:29.049 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:29.048+0000 7f2f86987640 1 -- 172.21.15.156:0/3405819770 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2f80102d10 msgr2=0x7f2f801138e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:52:29.049 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:29.048+0000 7f2f86987640 1 --2- 172.21.15.156:0/3405819770 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2f80102d10 0x7f2f801138e0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f2f74002a50 tx=0x7f2f74004600 comp rx=0 tx=0).stop 2024-07-16T09:52:29.049 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:29.048+0000 7f2f86987640 1 -- 172.21.15.156:0/3405819770 shutdown_connections 2024-07-16T09:52:29.050 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:29.048+0000 7f2f86987640 1 --2- 172.21.15.156:0/3405819770 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f2f68067190 0x7f2f68069640 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:52:29.050 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:29.048+0000 7f2f86987640 1 --2- 172.21.15.156:0/3405819770 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2f80102d10 0x7f2f801138e0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:52:29.050 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:29.048+0000 7f2f86987640 1 --2- 172.21.15.156:0/3405819770 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2f80102370 0x7f2f801133a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:52:29.050 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:29.048+0000 7f2f86987640 1 -- 172.21.15.156:0/3405819770 >> 172.21.15.156:0/3405819770 conn(0x7f2f800f7f00 msgr2=0x7f2f800f8e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:52:29.050 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:29.048+0000 7f2f86987640 1 -- 172.21.15.156:0/3405819770 shutdown_connections 2024-07-16T09:52:29.050 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:29.048+0000 7f2f86987640 1 -- 172.21.15.156:0/3405819770 wait complete. 2024-07-16T09:52:29.685 INFO:teuthology.orchestra.run.smithi156.stdout:{"pg_ready":true,"pg_map":{"version":151,"stamp":"2024-07-16T09:52:28.766449+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":3,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":3,"kb":562470912,"kb_used":1788084,"kb_used_data":3924,"kb_used_omap":9,"kb_used_meta":1784054,"kb_avail":560682828,"statfs":{"total":575970213888,"available":574139215872,"internally_reserved":0,"allocated":4018176,"data_stored":2624466,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":10224,"internal_metadata":1826871312},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002049"},"pg_stats":[{"pgid":"1.0","version":"24'117","reported_seq":187,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-07-16T09:51:38.651141+0000","last_change":"2024-07-16T09:51:14.089347+0000","last_active":"2024-07-16T09:51:38.651141+0000","last_peered":"2024-07-16T09:51:38.651141+0000","last_clean":"2024-07-16T09:51:38.651141+0000","last_became_active":"2024-07-16T09:51:14.088992+0000","last_became_peered":"2024-07-16T09:51:14.088992+0000","last_unstale":"2024-07-16T09:51:38.651141+0000","last_undegraded":"2024-07-16T09:51:38.651141+0000","last_fullsized":"2024-07-16T09:51:38.651141+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-16T09:50:29.128733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-16T09:50:29.128733+0000","last_clean_scrub_stamp":"2024-07-16T09:50:29.128733+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-17T19:32:40.192972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":4}],"osd_stats":[{"osd":5,"up_from":40,"seq":171798691853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":297720,"kb_used_data":364,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447432,"statfs":{"total":95995035648,"available":95690170368,"internally_reserved":0,"allocated":372736,"data_stored":142227,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40999999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51400000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52500000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49299999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34799999999999998}]}]},{"osd":4,"up_from":35,"seq":150323855377,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":297724,"kb_used_data":364,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447428,"statfs":{"total":95995035648,"available":95690166272,"internally_reserved":0,"allocated":372736,"data_stored":142227,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 16 09:52:15 2024","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.629,"5min":0.629,"15min":0.629},"last":0.502}]},{"osd":1,"last update":"Tue Jul 16 09:52:15 2024","interfaces":[{"interface":"back","average":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":7.343,"5min":7.343,"15min":7.343},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"last":0.64500000000000002}]},{"osd":2,"last update":"Tue Jul 16 09:52:15 2024","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"last":0.59199999999999997}]},{"osd":3,"last update":"Tue Jul 16 09:52:15 2024","interfaces":[{"interface":"back","average":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"last":0.379},{"interface":"front","average":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"last":0.35999999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39300000000000002}]}]},{"osd":3,"up_from":27,"seq":115964117014,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298304,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446848,"statfs":{"total":95995035648,"available":95689572352,"internally_reserved":0,"allocated":966656,"data_stored":732595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 16 09:51:53 2024","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"last":0.41599999999999998}]},{"osd":1,"last update":"Tue Jul 16 09:51:53 2024","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.626,"5min":0.626,"15min":0.626},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.63,"5min":0.63,"15min":0.63},"last":0.51100000000000001}]},{"osd":2,"last update":"Tue Jul 16 09:51:53 2024","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"last":0.54600000000000004}]},{"osd":4,"last update":"Tue Jul 16 09:52:18 2024","interfaces":[{"interface":"back","average":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.376,"5min":0.376,"15min":0.376},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.498,"5min":0.498,"15min":0.498},"last":0.59099999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.624}]}]},{"osd":2,"up_from":19,"seq":81604378650,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":297720,"kb_used_data":364,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447432,"statfs":{"total":95995035648,"available":95690170368,"internally_reserved":0,"allocated":372736,"data_stored":142227,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 16 09:51:27 2024","interfaces":[{"interface":"back","average":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"min":{"1min":0.21199999999999999,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"last":0.23400000000000001},{"interface":"front","average":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"last":0.27100000000000002}]},{"osd":1,"last update":"Tue Jul 16 09:51:27 2024","interfaces":[{"interface":"back","average":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"last":0.25800000000000001},{"interface":"front","average":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"last":0.29199999999999998}]},{"osd":3,"last update":"Tue Jul 16 09:51:49 2024","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.495,"15min":0.495},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.502,"5min":0.502,"15min":0.502},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"last":0.371}]},{"osd":4,"last update":"Tue Jul 16 09:52:17 2024","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.46500000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45000000000000001}]}]},{"osd":1,"up_from":14,"seq":60129542177,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298308,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446844,"statfs":{"total":95995035648,"available":95689568256,"internally_reserved":0,"allocated":966656,"data_stored":732595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 16 09:52:08 2024","interfaces":[{"interface":"back","average":{"1min":0.32000000000000001,"5min":0.313,"15min":0.312},"min":{"1min":0.22500000000000001,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.47099999999999997,"5min":0.54400000000000004,"15min":0.54400000000000004},"last":0.35899999999999999},{"interface":"front","average":{"1min":0.33100000000000002,"5min":0.315,"15min":0.312},"min":{"1min":0.248,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.503,"5min":0.503,"15min":0.503},"last":0.32600000000000001}]},{"osd":2,"last update":"Tue Jul 16 09:51:29 2024","interfaces":[{"interface":"back","average":{"1min":0.37,"5min":0.37,"15min":0.37},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"last":0.39700000000000002}]},{"osd":3,"last update":"Tue Jul 16 09:51:52 2024","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.55900000000000005}]},{"osd":4,"last update":"Tue Jul 16 09:52:14 2024","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"last":0.54900000000000004},{"interface":"front","average":{"1min":1.2350000000000001,"5min":1.2350000000000001,"15min":1.2350000000000001},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":15.423,"5min":15.423,"15min":15.423},"last":0.49299999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57299999999999995}]}]},{"osd":0,"up_from":9,"seq":38654705702,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298308,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446844,"statfs":{"total":95995035648,"available":95689568256,"internally_reserved":0,"allocated":966656,"data_stored":732595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Jul 16 09:52:06 2024","interfaces":[{"interface":"back","average":{"1min":0.33100000000000002,"5min":0.30399999999999999,"15min":0.29999999999999999},"min":{"1min":0.221,"5min":0.217,"15min":0.217},"max":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"last":0.371},{"interface":"front","average":{"1min":0.374,"5min":0.32900000000000001,"15min":0.32100000000000001},"min":{"1min":0.254,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"last":0.33900000000000002}]},{"osd":2,"last update":"Tue Jul 16 09:51:28 2024","interfaces":[{"interface":"back","average":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.501,"5min":0.501,"15min":0.501},"last":0.35899999999999999}]},{"osd":3,"last update":"Tue Jul 16 09:51:50 2024","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"last":0.55900000000000005}]},{"osd":4,"last update":"Tue Jul 16 09:52:16 2024","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"last":0.53500000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52600000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-07-16T09:52:29.686 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 26c91c26-4358-11ef-bcad-c7b262605968 -- ceph pg dump --format=json 2024-07-16T09:52:29.945 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/26c91c26-4358-11ef-bcad-c7b262605968/mon.smithi156/config 2024-07-16T09:52:30.377 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:52:30 smithi156 ceph-mon[29302]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-07-16T09:52:30.377 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:52:30 smithi156 ceph-mon[29302]: from='client.14414 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-07-16T09:52:30.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:52:30 smithi176 ceph-mon[36626]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-07-16T09:52:30.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:52:30 smithi176 ceph-mon[36626]: from='client.14414 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-07-16T09:52:31.576 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:31.569+0000 7fe97aeca640 1 -- 172.21.15.156:0/2562020480 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe974101c00 msgr2=0x7fe9740fb290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:52:31.576 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:31.569+0000 7fe97aeca640 1 --2- 172.21.15.156:0/2562020480 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe974101c00 0x7fe9740fb290 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fe9600048d0 tx=0x7fe960009da0 comp rx=0 tx=0).stop 2024-07-16T09:52:31.576 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:31.569+0000 7fe97aeca640 1 -- 172.21.15.156:0/2562020480 shutdown_connections 2024-07-16T09:52:31.576 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:31.569+0000 7fe97aeca640 1 --2- 172.21.15.156:0/2562020480 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe974101c00 0x7fe9740fb290 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:52:31.576 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:31.569+0000 7fe97aeca640 1 --2- 172.21.15.156:0/2562020480 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe974101260 0x7fe974101630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:52:31.576 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:31.569+0000 7fe97aeca640 1 -- 172.21.15.156:0/2562020480 >> 172.21.15.156:0/2562020480 conn(0x7fe9740f6e10 msgr2=0x7fe9740f9220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:52:31.576 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:31.570+0000 7fe97aeca640 1 -- 172.21.15.156:0/2562020480 shutdown_connections 2024-07-16T09:52:31.576 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:31.570+0000 7fe97aeca640 1 -- 172.21.15.156:0/2562020480 wait complete. 2024-07-16T09:52:31.577 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:31.570+0000 7fe97aeca640 1 Processor -- start 2024-07-16T09:52:31.577 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:31.570+0000 7fe97aeca640 1 -- start start 2024-07-16T09:52:31.577 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:31.570+0000 7fe97aeca640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe974101260 0x7fe9741130e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:52:31.577 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:31.570+0000 7fe97aeca640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe974101c00 0x7fe974113620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:52:31.577 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:31.570+0000 7fe97aeca640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe974113c20 con 0x7fe974101260 2024-07-16T09:52:31.577 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:31.570+0000 7fe97aeca640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe974113d90 con 0x7fe974101c00 2024-07-16T09:52:31.577 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:31.571+0000 7fe978c66640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe974101260 0x7fe9741130e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:52:31.577 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:31.571+0000 7fe978c66640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe974101260 0x7fe9741130e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:59198/0 (socket says 172.21.15.156:59198) 2024-07-16T09:52:31.577 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:31.571+0000 7fe978c66640 1 -- 172.21.15.156:0/1556277179 learned_addr learned my addr 172.21.15.156:0/1556277179 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T09:52:31.577 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:31.571+0000 7fe973fff640 1 --2- 172.21.15.156:0/1556277179 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe974101c00 0x7fe974113620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:52:31.577 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:31.571+0000 7fe978c66640 1 -- 172.21.15.156:0/1556277179 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe974101c00 msgr2=0x7fe974113620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:52:31.577 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:31.571+0000 7fe978c66640 1 --2- 172.21.15.156:0/1556277179 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe974101c00 0x7fe974113620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:52:31.577 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:31.571+0000 7fe978c66640 1 -- 172.21.15.156:0/1556277179 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe960005040 con 0x7fe974101260 2024-07-16T09:52:31.577 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:31.571+0000 7fe978c66640 1 --2- 172.21.15.156:0/1556277179 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe974101260 0x7fe9741130e0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fe968002a50 tx=0x7fe968004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:52:31.578 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:31.571+0000 7fe9717fa640 1 -- 172.21.15.156:0/1556277179 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe968007c30 con 0x7fe974101260 2024-07-16T09:52:31.578 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:31.571+0000 7fe9717fa640 1 -- 172.21.15.156:0/1556277179 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe96800f040 con 0x7fe974101260 2024-07-16T09:52:31.578 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:31.571+0000 7fe97aeca640 1 -- 172.21.15.156:0/1556277179 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe97410d210 con 0x7fe974101260 2024-07-16T09:52:31.578 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:31.571+0000 7fe9717fa640 1 -- 172.21.15.156:0/1556277179 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe96800dad0 con 0x7fe974101260 2024-07-16T09:52:31.578 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:31.571+0000 7fe97aeca640 1 -- 172.21.15.156:0/1556277179 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe97410d780 con 0x7fe974101260 2024-07-16T09:52:31.578 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:31.572+0000 7fe97aeca640 1 -- 172.21.15.156:0/1556277179 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9740fc9d0 con 0x7fe974101260 2024-07-16T09:52:31.578 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:31.573+0000 7fe9717fa640 1 -- 172.21.15.156:0/1556277179 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 19) v1 ==== 85127+0+0 (secure 0 0 0) 0x7fe96800dd40 con 0x7fe974101260 2024-07-16T09:52:31.578 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:31.573+0000 7fe9717fa640 1 --2- 172.21.15.156:0/1556277179 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7fe964067190 0x7fe964069640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:52:31.578 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:31.573+0000 7fe9717fa640 1 -- 172.21.15.156:0/1556277179 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fe968083f90 con 0x7fe974101260 2024-07-16T09:52:31.578 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:31.573+0000 7fe973fff640 1 --2- 172.21.15.156:0/1556277179 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7fe964067190 0x7fe964069640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:52:31.578 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:31.574+0000 7fe973fff640 1 --2- 172.21.15.156:0/1556277179 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7fe964067190 0x7fe964069640 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fe960009f80 tx=0x7fe960035040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:52:31.579 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:31.577+0000 7fe9717fa640 1 -- 172.21.15.156:0/1556277179 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fe96804d250 con 0x7fe974101260 2024-07-16T09:52:31.743 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:31.742+0000 7fe97aeca640 1 -- 172.21.15.156:0/1556277179 --> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe97410e790 con 0x7fe964067190 2024-07-16T09:52:31.744 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:31.743+0000 7fe9717fa640 1 -- 172.21.15.156:0/1556277179 <== mgr.14208 v2:172.21.15.156:6800/2615741717 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+27902 (secure 0 0 0) 0x7fe97410e790 con 0x7fe964067190 2024-07-16T09:52:31.744 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-07-16T09:52:31.745 INFO:teuthology.orchestra.run.smithi156.stderr:dumped all 2024-07-16T09:52:31.746 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:31.745+0000 7fe97aeca640 1 -- 172.21.15.156:0/1556277179 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7fe964067190 msgr2=0x7fe964069640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:52:31.746 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:31.745+0000 7fe97aeca640 1 --2- 172.21.15.156:0/1556277179 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7fe964067190 0x7fe964069640 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fe960009f80 tx=0x7fe960035040 comp rx=0 tx=0).stop 2024-07-16T09:52:31.746 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:31.745+0000 7fe97aeca640 1 -- 172.21.15.156:0/1556277179 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe974101260 msgr2=0x7fe9741130e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:52:31.746 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:31.745+0000 7fe97aeca640 1 --2- 172.21.15.156:0/1556277179 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe974101260 0x7fe9741130e0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fe968002a50 tx=0x7fe968004600 comp rx=0 tx=0).stop 2024-07-16T09:52:31.747 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:31.745+0000 7fe97aeca640 1 -- 172.21.15.156:0/1556277179 shutdown_connections 2024-07-16T09:52:31.747 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:31.745+0000 7fe97aeca640 1 --2- 172.21.15.156:0/1556277179 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7fe964067190 0x7fe964069640 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:52:31.747 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:31.745+0000 7fe97aeca640 1 --2- 172.21.15.156:0/1556277179 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe974101c00 0x7fe974113620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:52:31.747 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:31.745+0000 7fe97aeca640 1 --2- 172.21.15.156:0/1556277179 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe974101260 0x7fe9741130e0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:52:31.747 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:31.745+0000 7fe97aeca640 1 -- 172.21.15.156:0/1556277179 >> 172.21.15.156:0/1556277179 conn(0x7fe9740f6e10 msgr2=0x7fe974065860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:52:31.747 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:31.745+0000 7fe97aeca640 1 -- 172.21.15.156:0/1556277179 shutdown_connections 2024-07-16T09:52:31.747 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:31.745+0000 7fe97aeca640 1 -- 172.21.15.156:0/1556277179 wait complete. 2024-07-16T09:52:32.396 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:52:32 smithi176 ceph-mon[36626]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-07-16T09:52:32.405 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:52:32 smithi156 ceph-mon[29302]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-07-16T09:52:32.430 INFO:teuthology.orchestra.run.smithi156.stdout:{"pg_ready":true,"pg_map":{"version":152,"stamp":"2024-07-16T09:52:30.766771+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":3,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":3,"kb":562470912,"kb_used":1788084,"kb_used_data":3924,"kb_used_omap":9,"kb_used_meta":1784054,"kb_avail":560682828,"statfs":{"total":575970213888,"available":574139215872,"internally_reserved":0,"allocated":4018176,"data_stored":2624466,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":10224,"internal_metadata":1826871312},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002006"},"pg_stats":[{"pgid":"1.0","version":"24'117","reported_seq":187,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-07-16T09:51:38.651141+0000","last_change":"2024-07-16T09:51:14.089347+0000","last_active":"2024-07-16T09:51:38.651141+0000","last_peered":"2024-07-16T09:51:38.651141+0000","last_clean":"2024-07-16T09:51:38.651141+0000","last_became_active":"2024-07-16T09:51:14.088992+0000","last_became_peered":"2024-07-16T09:51:14.088992+0000","last_unstale":"2024-07-16T09:51:38.651141+0000","last_undegraded":"2024-07-16T09:51:38.651141+0000","last_fullsized":"2024-07-16T09:51:38.651141+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-16T09:50:29.128733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-16T09:50:29.128733+0000","last_clean_scrub_stamp":"2024-07-16T09:50:29.128733+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-17T19:32:40.192972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":4}],"osd_stats":[{"osd":5,"up_from":40,"seq":171798691853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":297720,"kb_used_data":364,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447432,"statfs":{"total":95995035648,"available":95690170368,"internally_reserved":0,"allocated":372736,"data_stored":142227,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40999999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51400000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52500000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49299999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34799999999999998}]}]},{"osd":4,"up_from":35,"seq":150323855378,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":297724,"kb_used_data":364,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447428,"statfs":{"total":95995035648,"available":95690166272,"internally_reserved":0,"allocated":372736,"data_stored":142227,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 16 09:52:15 2024","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.629,"5min":0.629,"15min":0.629},"last":0.504}]},{"osd":1,"last update":"Tue Jul 16 09:52:15 2024","interfaces":[{"interface":"back","average":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":7.343,"5min":7.343,"15min":7.343},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"last":0.60299999999999998}]},{"osd":2,"last update":"Tue Jul 16 09:52:15 2024","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"last":0.54500000000000004}]},{"osd":3,"last update":"Tue Jul 16 09:52:15 2024","interfaces":[{"interface":"back","average":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"last":0.40400000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47399999999999998}]}]},{"osd":3,"up_from":27,"seq":115964117015,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298304,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446848,"statfs":{"total":95995035648,"available":95689572352,"internally_reserved":0,"allocated":966656,"data_stored":732595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 16 09:51:53 2024","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"last":0.372}]},{"osd":1,"last update":"Tue Jul 16 09:51:53 2024","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.626,"5min":0.626,"15min":0.626},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.63,"5min":0.63,"15min":0.63},"last":0.34499999999999997}]},{"osd":2,"last update":"Tue Jul 16 09:51:53 2024","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"last":0.433}]},{"osd":4,"last update":"Tue Jul 16 09:52:18 2024","interfaces":[{"interface":"back","average":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"last":0.311},{"interface":"front","average":{"1min":0.376,"5min":0.376,"15min":0.376},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.498,"5min":0.498,"15min":0.498},"last":0.35199999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.497}]}]},{"osd":2,"up_from":19,"seq":81604378650,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":297720,"kb_used_data":364,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447432,"statfs":{"total":95995035648,"available":95690170368,"internally_reserved":0,"allocated":372736,"data_stored":142227,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 16 09:51:27 2024","interfaces":[{"interface":"back","average":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"min":{"1min":0.21199999999999999,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"last":0.23400000000000001},{"interface":"front","average":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"last":0.27100000000000002}]},{"osd":1,"last update":"Tue Jul 16 09:51:27 2024","interfaces":[{"interface":"back","average":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"last":0.25800000000000001},{"interface":"front","average":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"last":0.29199999999999998}]},{"osd":3,"last update":"Tue Jul 16 09:51:49 2024","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.495,"15min":0.495},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.502,"5min":0.502,"15min":0.502},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"last":0.371}]},{"osd":4,"last update":"Tue Jul 16 09:52:17 2024","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.46500000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45000000000000001}]}]},{"osd":1,"up_from":14,"seq":60129542177,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298308,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446844,"statfs":{"total":95995035648,"available":95689568256,"internally_reserved":0,"allocated":966656,"data_stored":732595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 16 09:52:08 2024","interfaces":[{"interface":"back","average":{"1min":0.32000000000000001,"5min":0.313,"15min":0.312},"min":{"1min":0.22500000000000001,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.47099999999999997,"5min":0.54400000000000004,"15min":0.54400000000000004},"last":0.35899999999999999},{"interface":"front","average":{"1min":0.33100000000000002,"5min":0.315,"15min":0.312},"min":{"1min":0.248,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.503,"5min":0.503,"15min":0.503},"last":0.32600000000000001}]},{"osd":2,"last update":"Tue Jul 16 09:51:29 2024","interfaces":[{"interface":"back","average":{"1min":0.37,"5min":0.37,"15min":0.37},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"last":0.39700000000000002}]},{"osd":3,"last update":"Tue Jul 16 09:51:52 2024","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.55900000000000005}]},{"osd":4,"last update":"Tue Jul 16 09:52:14 2024","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"last":0.54900000000000004},{"interface":"front","average":{"1min":1.2350000000000001,"5min":1.2350000000000001,"15min":1.2350000000000001},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":15.423,"5min":15.423,"15min":15.423},"last":0.49299999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57299999999999995}]}]},{"osd":0,"up_from":9,"seq":38654705703,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298308,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446844,"statfs":{"total":95995035648,"available":95689568256,"internally_reserved":0,"allocated":966656,"data_stored":732595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Jul 16 09:52:06 2024","interfaces":[{"interface":"back","average":{"1min":0.33100000000000002,"5min":0.30399999999999999,"15min":0.29999999999999999},"min":{"1min":0.221,"5min":0.217,"15min":0.217},"max":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"last":0.30099999999999999},{"interface":"front","average":{"1min":0.374,"5min":0.32900000000000001,"15min":0.32100000000000001},"min":{"1min":0.254,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"last":0.25700000000000001}]},{"osd":2,"last update":"Tue Jul 16 09:52:29 2024","interfaces":[{"interface":"back","average":{"1min":0.32900000000000001,"5min":0.38100000000000001,"15min":0.39000000000000001},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.51900000000000002,"5min":0.53800000000000003,"15min":0.53800000000000003},"last":0.32200000000000001},{"interface":"front","average":{"1min":0.315,"5min":0.35799999999999998,"15min":0.36499999999999999},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.497,"5min":0.501,"15min":0.501},"last":0.28999999999999998}]},{"osd":3,"last update":"Tue Jul 16 09:51:50 2024","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"last":0.498},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"last":0.48799999999999999}]},{"osd":4,"last update":"Tue Jul 16 09:52:16 2024","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"last":0.43099999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47999999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-07-16T09:52:32.430 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-07-16T09:52:32.431 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-07-16T09:52:32.431 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-07-16T09:52:32.431 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 26c91c26-4358-11ef-bcad-c7b262605968 -- ceph health --format=json 2024-07-16T09:52:32.684 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/26c91c26-4358-11ef-bcad-c7b262605968/mon.smithi156/config 2024-07-16T09:52:33.424 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:52:33 smithi156 ceph-mon[29302]: from='client.14418 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-07-16T09:52:33.591 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:52:33 smithi176 ceph-mon[36626]: from='client.14418 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-07-16T09:52:34.190 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:34.174+0000 7f2d1e5f7640 1 -- 172.21.15.156:0/2477070512 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2d18102370 msgr2=0x7f2d18102740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:52:34.190 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:34.174+0000 7f2d1e5f7640 1 --2- 172.21.15.156:0/2477070512 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2d18102370 0x7f2d18102740 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f2d080048d0 tx=0x7f2d08009da0 comp rx=0 tx=0).stop 2024-07-16T09:52:34.190 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:34.174+0000 7f2d1e5f7640 1 -- 172.21.15.156:0/2477070512 shutdown_connections 2024-07-16T09:52:34.190 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:34.174+0000 7f2d1e5f7640 1 --2- 172.21.15.156:0/2477070512 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2d18102d10 0x7f2d180fc3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:52:34.191 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:34.174+0000 7f2d1e5f7640 1 --2- 172.21.15.156:0/2477070512 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2d18102370 0x7f2d18102740 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:52:34.191 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:34.174+0000 7f2d1e5f7640 1 -- 172.21.15.156:0/2477070512 >> 172.21.15.156:0/2477070512 conn(0x7f2d180f7f00 msgr2=0x7f2d180fa310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:52:34.191 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:34.174+0000 7f2d1e5f7640 1 -- 172.21.15.156:0/2477070512 shutdown_connections 2024-07-16T09:52:34.191 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:34.174+0000 7f2d1e5f7640 1 -- 172.21.15.156:0/2477070512 wait complete. 2024-07-16T09:52:34.191 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:34.175+0000 7f2d1e5f7640 1 Processor -- start 2024-07-16T09:52:34.191 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:34.175+0000 7f2d1e5f7640 1 -- start start 2024-07-16T09:52:34.191 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:34.175+0000 7f2d1e5f7640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2d18102370 0x7f2d181133a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:52:34.191 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:34.175+0000 7f2d1e5f7640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2d18102d10 0x7f2d181138e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:52:34.191 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:34.175+0000 7f2d1e5f7640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d18113ee0 con 0x7f2d18102370 2024-07-16T09:52:34.191 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:34.175+0000 7f2d1e5f7640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d18114050 con 0x7f2d18102d10 2024-07-16T09:52:34.191 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:34.175+0000 7f2d17fff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2d18102370 0x7f2d181133a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:52:34.191 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:34.175+0000 7f2d17fff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2d18102370 0x7f2d181133a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:47096/0 (socket says 172.21.15.156:47096) 2024-07-16T09:52:34.191 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:34.175+0000 7f2d17fff640 1 -- 172.21.15.156:0/3835285512 learned_addr learned my addr 172.21.15.156:0/3835285512 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T09:52:34.191 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:34.175+0000 7f2d177fe640 1 --2- 172.21.15.156:0/3835285512 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2d18102d10 0x7f2d181138e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:52:34.192 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:34.176+0000 7f2d17fff640 1 -- 172.21.15.156:0/3835285512 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2d18102d10 msgr2=0x7f2d181138e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:52:34.192 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:34.176+0000 7f2d17fff640 1 --2- 172.21.15.156:0/3835285512 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2d18102d10 0x7f2d181138e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:52:34.192 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:34.176+0000 7f2d17fff640 1 -- 172.21.15.156:0/3835285512 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d08005040 con 0x7f2d18102370 2024-07-16T09:52:34.192 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:34.176+0000 7f2d17fff640 1 --2- 172.21.15.156:0/3835285512 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2d18102370 0x7f2d181133a0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f2d080043d0 tx=0x7f2d08009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:52:34.192 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:34.176+0000 7f2d14ff9640 1 -- 172.21.15.156:0/3835285512 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2d08038070 con 0x7f2d18102370 2024-07-16T09:52:34.192 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:34.176+0000 7f2d1e5f7640 1 -- 172.21.15.156:0/3835285512 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d1810d470 con 0x7f2d18102370 2024-07-16T09:52:34.192 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:34.176+0000 7f2d14ff9640 1 -- 172.21.15.156:0/3835285512 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2d08032970 con 0x7f2d18102370 2024-07-16T09:52:34.192 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:34.176+0000 7f2d14ff9640 1 -- 172.21.15.156:0/3835285512 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2d0803da60 con 0x7f2d18102370 2024-07-16T09:52:34.192 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:34.176+0000 7f2d1e5f7640 1 -- 172.21.15.156:0/3835285512 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d1810d9b0 con 0x7f2d18102370 2024-07-16T09:52:34.192 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:34.177+0000 7f2d1e5f7640 1 -- 172.21.15.156:0/3835285512 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d180fdae0 con 0x7f2d18102370 2024-07-16T09:52:34.192 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:34.178+0000 7f2d14ff9640 1 -- 172.21.15.156:0/3835285512 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 19) v1 ==== 85127+0+0 (secure 0 0 0) 0x7f2d08030020 con 0x7f2d18102370 2024-07-16T09:52:34.192 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:34.178+0000 7f2d14ff9640 1 --2- 172.21.15.156:0/3835285512 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f2d04067190 0x7f2d04069640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:52:34.192 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:34.178+0000 7f2d14ff9640 1 -- 172.21.15.156:0/3835285512 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f2d080a4900 con 0x7f2d18102370 2024-07-16T09:52:34.192 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:34.178+0000 7f2d177fe640 1 --2- 172.21.15.156:0/3835285512 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f2d04067190 0x7f2d04069640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:52:34.193 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:34.179+0000 7f2d177fe640 1 --2- 172.21.15.156:0/3835285512 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f2d04067190 0x7f2d04069640 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f2d1810eca0 tx=0x7f2d0000b040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:52:34.193 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:34.182+0000 7f2d14ff9640 1 -- 172.21.15.156:0/3835285512 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f2d08071ee0 con 0x7f2d18102370 2024-07-16T09:52:34.383 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:34.381+0000 7f2d1e5f7640 1 -- 172.21.15.156:0/3835285512 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f2d180fbb60 con 0x7f2d18102370 2024-07-16T09:52:34.383 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:52:34 smithi156 ceph-mon[29302]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-07-16T09:52:34.383 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:34.382+0000 7f2d14ff9640 1 -- 172.21.15.156:0/3835285512 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f2d08075500 con 0x7f2d18102370 2024-07-16T09:52:34.384 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-07-16T09:52:34.384 INFO:teuthology.orchestra.run.smithi156.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-07-16T09:52:34.385 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:34.383+0000 7f2d1e5f7640 1 -- 172.21.15.156:0/3835285512 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f2d04067190 msgr2=0x7f2d04069640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:52:34.385 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:34.383+0000 7f2d1e5f7640 1 --2- 172.21.15.156:0/3835285512 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f2d04067190 0x7f2d04069640 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f2d1810eca0 tx=0x7f2d0000b040 comp rx=0 tx=0).stop 2024-07-16T09:52:34.385 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:34.384+0000 7f2d1e5f7640 1 -- 172.21.15.156:0/3835285512 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2d18102370 msgr2=0x7f2d181133a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:52:34.385 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:34.384+0000 7f2d1e5f7640 1 --2- 172.21.15.156:0/3835285512 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2d18102370 0x7f2d181133a0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f2d080043d0 tx=0x7f2d08009da0 comp rx=0 tx=0).stop 2024-07-16T09:52:34.385 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:34.384+0000 7f2d1e5f7640 1 -- 172.21.15.156:0/3835285512 shutdown_connections 2024-07-16T09:52:34.385 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:34.384+0000 7f2d1e5f7640 1 --2- 172.21.15.156:0/3835285512 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f2d04067190 0x7f2d04069640 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:52:34.385 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:34.384+0000 7f2d1e5f7640 1 --2- 172.21.15.156:0/3835285512 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2d18102d10 0x7f2d181138e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:52:34.386 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:34.384+0000 7f2d1e5f7640 1 --2- 172.21.15.156:0/3835285512 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2d18102370 0x7f2d181133a0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:52:34.386 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:34.384+0000 7f2d1e5f7640 1 -- 172.21.15.156:0/3835285512 >> 172.21.15.156:0/3835285512 conn(0x7f2d180f7f00 msgr2=0x7f2d180f8e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:52:34.386 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:34.384+0000 7f2d1e5f7640 1 -- 172.21.15.156:0/3835285512 shutdown_connections 2024-07-16T09:52:34.386 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:34.384+0000 7f2d1e5f7640 1 -- 172.21.15.156:0/3835285512 wait complete. 2024-07-16T09:52:34.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:52:34 smithi176 ceph-mon[36626]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-07-16T09:52:35.045 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-07-16T09:52:35.045 INFO:tasks.cephadm:Setup complete, yielding 2024-07-16T09:52:35.045 INFO:teuthology.run_tasks:Running task print... 2024-07-16T09:52:35.054 INFO:teuthology.task.print:**** done end installing quincy cephadm ... 2024-07-16T09:52:35.054 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-07-16T09:52:35.062 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi156.front.sepia.ceph.com 2024-07-16T09:52:35.063 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 26c91c26-4358-11ef-bcad-c7b262605968 -- bash -c 'ceph config set mgr mgr/cephadm/use_repo_digest true --force' 2024-07-16T09:52:35.317 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/26c91c26-4358-11ef-bcad-c7b262605968/mon.smithi156/config 2024-07-16T09:52:35.424 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:52:35 smithi156 ceph-mon[29302]: from='client.? 172.21.15.156:0/3835285512' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-07-16T09:52:35.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:52:35 smithi176 ceph-mon[36626]: from='client.? 172.21.15.156:0/3835285512' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-07-16T09:52:36.424 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:52:36 smithi156 ceph-mon[29302]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-07-16T09:52:36.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:52:36 smithi176 ceph-mon[36626]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-07-16T09:52:37.005 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:36.995+0000 7f985dea9640 1 -- 172.21.15.156:0/2467839010 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f98580fe300 msgr2=0x7f98580fa460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:52:37.005 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:36.995+0000 7f985dea9640 1 --2- 172.21.15.156:0/2467839010 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f98580fe300 0x7f98580fa460 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f984c0048d0 tx=0x7f984c009da0 comp rx=0 tx=0).stop 2024-07-16T09:52:37.005 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:36.995+0000 7f985dea9640 1 -- 172.21.15.156:0/2467839010 shutdown_connections 2024-07-16T09:52:37.005 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:36.995+0000 7f985dea9640 1 --2- 172.21.15.156:0/2467839010 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f98580fe300 0x7f98580fa460 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:52:37.006 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:36.996+0000 7f985dea9640 1 --2- 172.21.15.156:0/2467839010 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f98580fd9b0 0x7f98580fddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:52:37.006 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:36.996+0000 7f985dea9640 1 -- 172.21.15.156:0/2467839010 >> 172.21.15.156:0/2467839010 conn(0x7f98580f61b0 msgr2=0x7f98580f85e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:52:37.006 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:36.996+0000 7f985dea9640 1 -- 172.21.15.156:0/2467839010 shutdown_connections 2024-07-16T09:52:37.006 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:36.996+0000 7f985dea9640 1 -- 172.21.15.156:0/2467839010 wait complete. 2024-07-16T09:52:37.006 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:36.996+0000 7f985dea9640 1 Processor -- start 2024-07-16T09:52:37.006 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:36.996+0000 7f985dea9640 1 -- start start 2024-07-16T09:52:37.006 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:36.996+0000 7f985dea9640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f98580fd9b0 0x7f985810cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:52:37.006 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:36.996+0000 7f985dea9640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f98580fe300 0x7f985810d520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:52:37.006 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:36.996+0000 7f985dea9640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f985810e390 con 0x7f98580fe300 2024-07-16T09:52:37.007 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:36.996+0000 7f985dea9640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f985810e500 con 0x7f98580fd9b0 2024-07-16T09:52:37.007 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:36.997+0000 7f9856ffd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f98580fe300 0x7f985810d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:52:37.007 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:36.997+0000 7f9856ffd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f98580fe300 0x7f985810d520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:47122/0 (socket says 172.21.15.156:47122) 2024-07-16T09:52:37.007 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:36.997+0000 7f9856ffd640 1 -- 172.21.15.156:0/2120177485 learned_addr learned my addr 172.21.15.156:0/2120177485 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T09:52:37.007 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:36.997+0000 7f98577fe640 1 --2- 172.21.15.156:0/2120177485 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f98580fd9b0 0x7f985810cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:52:37.007 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:36.997+0000 7f9856ffd640 1 -- 172.21.15.156:0/2120177485 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f98580fd9b0 msgr2=0x7f985810cfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:52:37.007 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:36.997+0000 7f9856ffd640 1 --2- 172.21.15.156:0/2120177485 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f98580fd9b0 0x7f985810cfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:52:37.007 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:36.997+0000 7f9856ffd640 1 -- 172.21.15.156:0/2120177485 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f984c005040 con 0x7f98580fe300 2024-07-16T09:52:37.007 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:36.997+0000 7f9856ffd640 1 --2- 172.21.15.156:0/2120177485 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f98580fe300 0x7f985810d520 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f984c000c00 tx=0x7f984c009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:52:37.008 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:36.997+0000 7f983ffff640 1 -- 172.21.15.156:0/2120177485 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f984c038070 con 0x7f98580fe300 2024-07-16T09:52:37.008 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:36.997+0000 7f983ffff640 1 -- 172.21.15.156:0/2120177485 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f984c032970 con 0x7f98580fe300 2024-07-16T09:52:37.008 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:36.997+0000 7f985dea9640 1 -- 172.21.15.156:0/2120177485 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f985810e780 con 0x7f98580fe300 2024-07-16T09:52:37.008 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:36.997+0000 7f983ffff640 1 -- 172.21.15.156:0/2120177485 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f984c033950 con 0x7f98580fe300 2024-07-16T09:52:37.008 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:36.997+0000 7f985dea9640 1 -- 172.21.15.156:0/2120177485 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f985810ec40 con 0x7f98580fe300 2024-07-16T09:52:37.008 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:36.998+0000 7f985dea9640 1 -- 172.21.15.156:0/2120177485 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98580f9e50 con 0x7f98580fe300 2024-07-16T09:52:37.008 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:36.999+0000 7f983ffff640 1 -- 172.21.15.156:0/2120177485 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 19) v1 ==== 85127+0+0 (secure 0 0 0) 0x7f984c030020 con 0x7f98580fe300 2024-07-16T09:52:37.008 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:36.999+0000 7f983ffff640 1 --2- 172.21.15.156:0/2120177485 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f9840067190 0x7f9840069640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:52:37.008 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:36.999+0000 7f983ffff640 1 -- 172.21.15.156:0/2120177485 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f984c0a47f0 con 0x7f98580fe300 2024-07-16T09:52:37.008 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:36.999+0000 7f98577fe640 1 --2- 172.21.15.156:0/2120177485 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f9840067190 0x7f9840069640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:52:37.009 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:36.999+0000 7f98577fe640 1 --2- 172.21.15.156:0/2120177485 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f9840067190 0x7f9840069640 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f9848001250 tx=0x7f9848009620 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:52:37.009 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:37.003+0000 7f983ffff640 1 -- 172.21.15.156:0/2120177485 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f984c071dd0 con 0x7f98580fe300 2024-07-16T09:52:37.170 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:37.168+0000 7f985dea9640 1 -- 172.21.15.156:0/2120177485 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/use_repo_digest}] v 0) v1 -- 0x7f985804b530 con 0x7f98580fe300 2024-07-16T09:52:37.173 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:37.172+0000 7f983ffff640 1 -- 172.21.15.156:0/2120177485 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/use_repo_digest}]=0 v20)=0 v20) v1 ==== 143+0+0 (secure 0 0 0) 0x7f984c0753f0 con 0x7f98580fe300 2024-07-16T09:52:37.176 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:37.174+0000 7f985dea9640 1 -- 172.21.15.156:0/2120177485 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f9840067190 msgr2=0x7f9840069640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:52:37.176 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:37.174+0000 7f985dea9640 1 --2- 172.21.15.156:0/2120177485 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f9840067190 0x7f9840069640 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f9848001250 tx=0x7f9848009620 comp rx=0 tx=0).stop 2024-07-16T09:52:37.176 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:37.174+0000 7f985dea9640 1 -- 172.21.15.156:0/2120177485 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f98580fe300 msgr2=0x7f985810d520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:52:37.176 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:37.174+0000 7f985dea9640 1 --2- 172.21.15.156:0/2120177485 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f98580fe300 0x7f985810d520 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f984c000c00 tx=0x7f984c009da0 comp rx=0 tx=0).stop 2024-07-16T09:52:37.177 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:37.174+0000 7f985dea9640 1 -- 172.21.15.156:0/2120177485 shutdown_connections 2024-07-16T09:52:37.177 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:37.174+0000 7f985dea9640 1 --2- 172.21.15.156:0/2120177485 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f9840067190 0x7f9840069640 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:52:37.177 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:37.174+0000 7f985dea9640 1 --2- 172.21.15.156:0/2120177485 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f98580fe300 0x7f985810d520 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:52:37.177 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:37.174+0000 7f985dea9640 1 --2- 172.21.15.156:0/2120177485 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f98580fd9b0 0x7f985810cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:52:37.177 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:37.174+0000 7f985dea9640 1 -- 172.21.15.156:0/2120177485 >> 172.21.15.156:0/2120177485 conn(0x7f98580f61b0 msgr2=0x7f98580f7800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:52:37.177 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:37.175+0000 7f985dea9640 1 -- 172.21.15.156:0/2120177485 shutdown_connections 2024-07-16T09:52:37.177 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:37.175+0000 7f985dea9640 1 -- 172.21.15.156:0/2120177485 wait complete. 2024-07-16T09:52:37.825 INFO:teuthology.run_tasks:Running task print... 2024-07-16T09:52:37.836 INFO:teuthology.task.print:**** done cephadm.shell ceph config set mgr... 2024-07-16T09:52:37.836 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-07-16T09:52:37.843 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi156.front.sepia.ceph.com 2024-07-16T09:52:37.843 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 26c91c26-4358-11ef-bcad-c7b262605968 -- bash -c 'ceph orch status' 2024-07-16T09:52:38.108 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/26c91c26-4358-11ef-bcad-c7b262605968/mon.smithi156/config 2024-07-16T09:52:38.424 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:52:38 smithi156 ceph-mon[29302]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-07-16T09:52:38.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:52:38 smithi156 ceph-mon[29302]: from='client.? 172.21.15.156:0/2120177485' entity='client.admin' 2024-07-16T09:52:38.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:52:38 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-16T09:52:38.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:52:38 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-16T09:52:38.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:52:38 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-16T09:52:38.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:52:38 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:52:38.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:52:38 smithi176 ceph-mon[36626]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-07-16T09:52:38.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:52:38 smithi176 ceph-mon[36626]: from='client.? 172.21.15.156:0/2120177485' entity='client.admin' 2024-07-16T09:52:38.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:52:38 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-16T09:52:38.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:52:38 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-16T09:52:38.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:52:38 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-16T09:52:38.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:52:38 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:52:39.862 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:39.850+0000 7f35cddc7640 1 -- 172.21.15.156:0/3036343964 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f35c80665a0 msgr2=0x7f35c80fef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:52:39.862 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:39.850+0000 7f35cddc7640 1 --2- 172.21.15.156:0/3036343964 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f35c80665a0 0x7f35c80fef70 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f35b8009c50 tx=0x7f35b8004d10 comp rx=0 tx=0).stop 2024-07-16T09:52:39.862 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:39.851+0000 7f35cddc7640 1 -- 172.21.15.156:0/3036343964 shutdown_connections 2024-07-16T09:52:39.862 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:39.851+0000 7f35cddc7640 1 --2- 172.21.15.156:0/3036343964 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f35c80ff4b0 0x7f35c8101890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:52:39.863 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:39.851+0000 7f35cddc7640 1 --2- 172.21.15.156:0/3036343964 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f35c80665a0 0x7f35c80fef70 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:52:39.863 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:39.851+0000 7f35cddc7640 1 -- 172.21.15.156:0/3036343964 >> 172.21.15.156:0/3036343964 conn(0x7f35c80f6a20 msgr2=0x7f35c80f8e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:52:39.863 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:39.851+0000 7f35cddc7640 1 -- 172.21.15.156:0/3036343964 shutdown_connections 2024-07-16T09:52:39.863 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:39.851+0000 7f35cddc7640 1 -- 172.21.15.156:0/3036343964 wait complete. 2024-07-16T09:52:39.863 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:39.851+0000 7f35cddc7640 1 Processor -- start 2024-07-16T09:52:39.863 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:39.851+0000 7f35cddc7640 1 -- start start 2024-07-16T09:52:39.863 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:39.851+0000 7f35cddc7640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f35c80665a0 0x7f35c810d030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:52:39.863 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:39.851+0000 7f35cddc7640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f35c80ff4b0 0x7f35c810d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:52:39.863 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:39.851+0000 7f35cddc7640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35c810e3e0 con 0x7f35c80ff4b0 2024-07-16T09:52:39.863 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:39.851+0000 7f35cddc7640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35c810e550 con 0x7f35c80665a0 2024-07-16T09:52:39.863 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:39.851+0000 7f35c6ffd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f35c80ff4b0 0x7f35c810d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:52:39.863 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:39.852+0000 7f35c6ffd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f35c80ff4b0 0x7f35c810d570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:47138/0 (socket says 172.21.15.156:47138) 2024-07-16T09:52:39.864 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:39.852+0000 7f35c6ffd640 1 -- 172.21.15.156:0/2070304594 learned_addr learned my addr 172.21.15.156:0/2070304594 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T09:52:39.864 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:39.852+0000 7f35c77fe640 1 --2- 172.21.15.156:0/2070304594 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f35c80665a0 0x7f35c810d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:52:39.864 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:39.852+0000 7f35c6ffd640 1 -- 172.21.15.156:0/2070304594 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f35c80665a0 msgr2=0x7f35c810d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:52:39.864 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:39.852+0000 7f35c6ffd640 1 --2- 172.21.15.156:0/2070304594 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f35c80665a0 0x7f35c810d030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:52:39.864 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:39.852+0000 7f35c6ffd640 1 -- 172.21.15.156:0/2070304594 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35b80098b0 con 0x7f35c80ff4b0 2024-07-16T09:52:39.864 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:39.852+0000 7f35c77fe640 1 --2- 172.21.15.156:0/2070304594 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f35c80665a0 0x7f35c810d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-16T09:52:39.864 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:39.852+0000 7f35c6ffd640 1 --2- 172.21.15.156:0/2070304594 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f35c80ff4b0 0x7f35c810d570 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f35bc00b790 tx=0x7f35bc0091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:52:39.864 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:39.852+0000 7f35affff640 1 -- 172.21.15.156:0/2070304594 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f35bc00c700 con 0x7f35c80ff4b0 2024-07-16T09:52:39.864 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:39.852+0000 7f35cddc7640 1 -- 172.21.15.156:0/2070304594 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35c810e830 con 0x7f35c80ff4b0 2024-07-16T09:52:39.864 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:39.852+0000 7f35affff640 1 -- 172.21.15.156:0/2070304594 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f35bc00cd30 con 0x7f35c80ff4b0 2024-07-16T09:52:39.865 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:39.852+0000 7f35affff640 1 -- 172.21.15.156:0/2070304594 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f35bc013370 con 0x7f35c80ff4b0 2024-07-16T09:52:39.865 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:39.852+0000 7f35cddc7640 1 -- 172.21.15.156:0/2070304594 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35c810ed50 con 0x7f35c80ff4b0 2024-07-16T09:52:39.865 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:39.853+0000 7f35cddc7640 1 -- 172.21.15.156:0/2070304594 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35c80fa6c0 con 0x7f35c80ff4b0 2024-07-16T09:52:39.865 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:39.854+0000 7f35affff640 1 -- 172.21.15.156:0/2070304594 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 19) v1 ==== 85127+0+0 (secure 0 0 0) 0x7f35bc012070 con 0x7f35c80ff4b0 2024-07-16T09:52:39.865 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:39.854+0000 7f35affff640 1 --2- 172.21.15.156:0/2070304594 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f35b0067190 0x7f35b0069640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:52:39.865 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:39.854+0000 7f35affff640 1 -- 172.21.15.156:0/2070304594 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f35bc0843d0 con 0x7f35c80ff4b0 2024-07-16T09:52:39.865 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:39.854+0000 7f35c77fe640 1 --2- 172.21.15.156:0/2070304594 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f35b0067190 0x7f35b0069640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:52:39.865 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:39.854+0000 7f35c77fe640 1 --2- 172.21.15.156:0/2070304594 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f35b0067190 0x7f35b0069640 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f35b8000f80 tx=0x7f35b8004d10 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:52:39.865 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:39.858+0000 7f35affff640 1 -- 172.21.15.156:0/2070304594 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f35bc0519b0 con 0x7f35c80ff4b0 2024-07-16T09:52:40.037 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:40.035+0000 7f35cddc7640 1 -- 172.21.15.156:0/2070304594 --> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) v1 -- 0x7f35c810efd0 con 0x7f35b0067190 2024-07-16T09:52:40.038 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:40.036+0000 7f35affff640 1 -- 172.21.15.156:0/2070304594 <== mgr.14208 v2:172.21.15.156:6800/2615741717 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7f35c810efd0 con 0x7f35b0067190 2024-07-16T09:52:40.038 INFO:teuthology.orchestra.run.smithi156.stdout:Backend: cephadm 2024-07-16T09:52:40.038 INFO:teuthology.orchestra.run.smithi156.stdout:Available: Yes 2024-07-16T09:52:40.038 INFO:teuthology.orchestra.run.smithi156.stdout:Paused: No 2024-07-16T09:52:40.040 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:40.038+0000 7f35cddc7640 1 -- 172.21.15.156:0/2070304594 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f35b0067190 msgr2=0x7f35b0069640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:52:40.040 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:40.038+0000 7f35cddc7640 1 --2- 172.21.15.156:0/2070304594 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f35b0067190 0x7f35b0069640 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f35b8000f80 tx=0x7f35b8004d10 comp rx=0 tx=0).stop 2024-07-16T09:52:40.040 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:40.038+0000 7f35cddc7640 1 -- 172.21.15.156:0/2070304594 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f35c80ff4b0 msgr2=0x7f35c810d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:52:40.040 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:40.038+0000 7f35cddc7640 1 --2- 172.21.15.156:0/2070304594 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f35c80ff4b0 0x7f35c810d570 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f35bc00b790 tx=0x7f35bc0091d0 comp rx=0 tx=0).stop 2024-07-16T09:52:40.040 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:40.038+0000 7f35cddc7640 1 -- 172.21.15.156:0/2070304594 shutdown_connections 2024-07-16T09:52:40.040 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:40.038+0000 7f35cddc7640 1 --2- 172.21.15.156:0/2070304594 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f35b0067190 0x7f35b0069640 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:52:40.040 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:40.038+0000 7f35cddc7640 1 --2- 172.21.15.156:0/2070304594 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f35c80ff4b0 0x7f35c810d570 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:52:40.040 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:40.038+0000 7f35cddc7640 1 --2- 172.21.15.156:0/2070304594 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f35c80665a0 0x7f35c810d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:52:40.041 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:40.038+0000 7f35cddc7640 1 -- 172.21.15.156:0/2070304594 >> 172.21.15.156:0/2070304594 conn(0x7f35c80f6a20 msgr2=0x7f35c80f6e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:52:40.041 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:40.039+0000 7f35cddc7640 1 -- 172.21.15.156:0/2070304594 shutdown_connections 2024-07-16T09:52:40.041 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:40.039+0000 7f35cddc7640 1 -- 172.21.15.156:0/2070304594 wait complete. 2024-07-16T09:52:40.315 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:52:40 smithi156 ceph-mon[29302]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-07-16T09:52:40.315 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:52:40 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-16T09:52:40.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:52:40 smithi176 ceph-mon[36626]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-07-16T09:52:40.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:52:40 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-16T09:52:40.742 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 26c91c26-4358-11ef-bcad-c7b262605968 -- bash -c 'ceph orch ps' 2024-07-16T09:52:41.004 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/26c91c26-4358-11ef-bcad-c7b262605968/mon.smithi156/config 2024-07-16T09:52:41.386 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:52:41 smithi156 ceph-mon[29302]: from='client.14430 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-16T09:52:41.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:52:41 smithi176 ceph-mon[36626]: from='client.14430 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-16T09:52:42.350 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:52:42 smithi156 ceph-mon[29302]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-07-16T09:52:42.576 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:42.570+0000 7f334cee9640 1 -- 172.21.15.156:0/1637672542 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f33480fa7c0 msgr2=0x7f33480fabd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:52:42.576 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:42.570+0000 7f334cee9640 1 --2- 172.21.15.156:0/1637672542 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f33480fa7c0 0x7f33480fabd0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f33380048d0 tx=0x7f3338009da0 comp rx=0 tx=0).stop 2024-07-16T09:52:42.576 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:42.570+0000 7f334cee9640 1 -- 172.21.15.156:0/1637672542 shutdown_connections 2024-07-16T09:52:42.576 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:42.570+0000 7f334cee9640 1 --2- 172.21.15.156:0/1637672542 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f33480fb110 0x7f3348101960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:52:42.576 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:42.570+0000 7f334cee9640 1 --2- 172.21.15.156:0/1637672542 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f33480fa7c0 0x7f33480fabd0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:52:42.576 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:42.570+0000 7f334cee9640 1 -- 172.21.15.156:0/1637672542 >> 172.21.15.156:0/1637672542 conn(0x7f33480f61c0 msgr2=0x7f33480f85f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:52:42.576 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:42.570+0000 7f334cee9640 1 -- 172.21.15.156:0/1637672542 shutdown_connections 2024-07-16T09:52:42.576 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:42.570+0000 7f334cee9640 1 -- 172.21.15.156:0/1637672542 wait complete. 2024-07-16T09:52:42.576 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:42.571+0000 7f334cee9640 1 Processor -- start 2024-07-16T09:52:42.576 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:42.571+0000 7f334cee9640 1 -- start start 2024-07-16T09:52:42.577 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:42.571+0000 7f334cee9640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f33480fa7c0 0x7f334810d010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:52:42.577 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:42.571+0000 7f334cee9640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f33480fb110 0x7f334810d550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:52:42.577 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:42.571+0000 7f334cee9640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f334810dba0 con 0x7f33480fb110 2024-07-16T09:52:42.577 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:42.571+0000 7f334cee9640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f334810e3c0 con 0x7f33480fa7c0 2024-07-16T09:52:42.577 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:42.571+0000 7f3345d9b640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f33480fb110 0x7f334810d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:52:42.577 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:42.571+0000 7f3345d9b640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f33480fb110 0x7f334810d550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:47164/0 (socket says 172.21.15.156:47164) 2024-07-16T09:52:42.577 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:42.571+0000 7f3345d9b640 1 -- 172.21.15.156:0/936755013 learned_addr learned my addr 172.21.15.156:0/936755013 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T09:52:42.577 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:42.571+0000 7f3345d9b640 1 -- 172.21.15.156:0/936755013 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f33480fa7c0 msgr2=0x7f334810d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:52:42.577 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:42.571+0000 7f334659c640 1 --2- 172.21.15.156:0/936755013 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f33480fa7c0 0x7f334810d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:52:42.577 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:42.571+0000 7f3345d9b640 1 --2- 172.21.15.156:0/936755013 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f33480fa7c0 0x7f334810d010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:52:42.577 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:42.571+0000 7f3345d9b640 1 -- 172.21.15.156:0/936755013 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3338005040 con 0x7f33480fb110 2024-07-16T09:52:42.577 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:42.571+0000 7f3345d9b640 1 --2- 172.21.15.156:0/936755013 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f33480fb110 0x7f334810d550 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f333c00c7e0 tx=0x7f333c0043c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:52:42.578 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:42.571+0000 7f3336ffd640 1 -- 172.21.15.156:0/936755013 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f333c00a820 con 0x7f33480fb110 2024-07-16T09:52:42.578 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:42.572+0000 7f3336ffd640 1 -- 172.21.15.156:0/936755013 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f333c00ae50 con 0x7f33480fb110 2024-07-16T09:52:42.578 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:42.572+0000 7f3336ffd640 1 -- 172.21.15.156:0/936755013 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f333c0113a0 con 0x7f33480fb110 2024-07-16T09:52:42.578 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:42.572+0000 7f334cee9640 1 -- 172.21.15.156:0/936755013 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f334810e650 con 0x7f33480fb110 2024-07-16T09:52:42.578 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:42.572+0000 7f334cee9640 1 -- 172.21.15.156:0/936755013 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f334810ebf0 con 0x7f33480fb110 2024-07-16T09:52:42.578 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:42.573+0000 7f334cee9640 1 -- 172.21.15.156:0/936755013 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33480f9e60 con 0x7f33480fb110 2024-07-16T09:52:42.578 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:42.573+0000 7f3336ffd640 1 -- 172.21.15.156:0/936755013 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 19) v1 ==== 85127+0+0 (secure 0 0 0) 0x7f333c00a980 con 0x7f33480fb110 2024-07-16T09:52:42.578 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:42.573+0000 7f3336ffd640 1 --2- 172.21.15.156:0/936755013 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f3330067190 0x7f3330069640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:52:42.578 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:42.573+0000 7f3336ffd640 1 -- 172.21.15.156:0/936755013 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f333c084820 con 0x7f33480fb110 2024-07-16T09:52:42.578 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:42.573+0000 7f334659c640 1 --2- 172.21.15.156:0/936755013 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f3330067190 0x7f3330069640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:52:42.578 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:42.574+0000 7f334659c640 1 --2- 172.21.15.156:0/936755013 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f3330067190 0x7f3330069640 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f333802a530 tx=0x7f3338009da0 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:52:42.579 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:42.577+0000 7f3336ffd640 1 -- 172.21.15.156:0/936755013 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f333c051f20 con 0x7f33480fb110 2024-07-16T09:52:42.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:52:42 smithi176 ceph-mon[36626]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-07-16T09:52:42.751 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:42.750+0000 7f334cee9640 1 -- 172.21.15.156:0/936755013 --> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f334810f030 con 0x7f3330067190 2024-07-16T09:52:42.759 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:42.758+0000 7f3336ffd640 1 -- 172.21.15.156:0/936755013 <== mgr.14208 v2:172.21.15.156:6800/2615741717 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+2718 (secure 0 0 0) 0x7f334810f030 con 0x7f3330067190 2024-07-16T09:52:42.760 INFO:teuthology.orchestra.run.smithi156.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-07-16T09:52:42.760 INFO:teuthology.orchestra.run.smithi156.stdout:alertmanager.smithi156 smithi156 *:9093,9094 running (3m) 2m ago 4m 25.8M - 0.25.0 c8568f914cd2 83419a2a37f4 2024-07-16T09:52:42.760 INFO:teuthology.orchestra.run.smithi156.stdout:crash.smithi156 smithi156 running (5m) 2m ago 5m 6899k - 17.2.7-1104-ge6231b04 d5de5f953b95 bf6d49ea06b4 2024-07-16T09:52:42.760 INFO:teuthology.orchestra.run.smithi156.stdout:crash.smithi176 smithi176 running (4m) 66s ago 4m 6903k - 17.2.7-1104-ge6231b04 d5de5f953b95 907c9fa1e890 2024-07-16T09:52:42.760 INFO:teuthology.orchestra.run.smithi156.stdout:grafana.smithi156 smithi156 *:3000 running (3m) 2m ago 4m 78.1M - 9.4.7 954c08fa6188 184256cf055d 2024-07-16T09:52:42.760 INFO:teuthology.orchestra.run.smithi156.stdout:mgr.smithi156.zkvylv smithi156 *:9283 running (6m) 2m ago 6m 441M - 17.2.7-1104-ge6231b04 d5de5f953b95 a6937c84750a 2024-07-16T09:52:42.760 INFO:teuthology.orchestra.run.smithi156.stdout:mgr.smithi176.dqbrvx smithi176 *:8443,9283 running (4m) 66s ago 4m 401M - 17.2.7-1104-ge6231b04 d5de5f953b95 654c9bc9c82a 2024-07-16T09:52:42.760 INFO:teuthology.orchestra.run.smithi156.stdout:mon.smithi156 smithi156 running (6m) 2m ago 6m 42.7M 2048M 17.2.7-1104-ge6231b04 d5de5f953b95 6471c8acff40 2024-07-16T09:52:42.760 INFO:teuthology.orchestra.run.smithi156.stdout:mon.smithi176 smithi176 running (3m) 66s ago 3m 39.4M 2048M 17.2.7-1104-ge6231b04 d5de5f953b95 3a293ac6ef01 2024-07-16T09:52:42.760 INFO:teuthology.orchestra.run.smithi156.stdout:node-exporter.smithi156 smithi156 *:9100 running (4m) 2m ago 4m 15.8M - 1.5.0 0da6a335fe13 2c136840a7a9 2024-07-16T09:52:42.760 INFO:teuthology.orchestra.run.smithi156.stdout:node-exporter.smithi176 smithi176 *:9100 running (4m) 66s ago 4m 15.7M - 1.5.0 0da6a335fe13 536716db6afe 2024-07-16T09:52:42.760 INFO:teuthology.orchestra.run.smithi156.stdout:osd.0 smithi156 running (3m) 2m ago 3m 41.9M 4288M 17.2.7-1104-ge6231b04 d5de5f953b95 0471758ec349 2024-07-16T09:52:42.760 INFO:teuthology.orchestra.run.smithi156.stdout:osd.1 smithi156 running (2m) 2m ago 2m 40.8M 4288M 17.2.7-1104-ge6231b04 d5de5f953b95 149a0bda2f61 2024-07-16T09:52:42.761 INFO:teuthology.orchestra.run.smithi156.stdout:osd.2 smithi156 running (2m) 2m ago 2m 37.7M 4288M 17.2.7-1104-ge6231b04 d5de5f953b95 f28acec5cb79 2024-07-16T09:52:42.761 INFO:teuthology.orchestra.run.smithi156.stdout:osd.3 smithi176 running (2m) 66s ago 119s 39.4M 5312M 17.2.7-1104-ge6231b04 d5de5f953b95 733861ba84dc 2024-07-16T09:52:42.761 INFO:teuthology.orchestra.run.smithi156.stdout:osd.4 smithi176 running (95s) 66s ago 95s 35.9M 5312M 17.2.7-1104-ge6231b04 d5de5f953b95 ef0c67cec086 2024-07-16T09:52:42.761 INFO:teuthology.orchestra.run.smithi156.stdout:osd.5 smithi176 running (71s) 66s ago 70s 11.7M 5312M 17.2.7-1104-ge6231b04 d5de5f953b95 996e15448a52 2024-07-16T09:52:42.761 INFO:teuthology.orchestra.run.smithi156.stdout:prometheus.smithi156 smithi156 *:9095 running (3m) 2m ago 4m 35.8M - 2.43.0 a07b618ecd1d 1655b4227cd0 2024-07-16T09:52:42.762 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:42.760+0000 7f334cee9640 1 -- 172.21.15.156:0/936755013 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f3330067190 msgr2=0x7f3330069640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:52:42.762 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:42.760+0000 7f334cee9640 1 --2- 172.21.15.156:0/936755013 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f3330067190 0x7f3330069640 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f333802a530 tx=0x7f3338009da0 comp rx=0 tx=0).stop 2024-07-16T09:52:42.762 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:42.760+0000 7f334cee9640 1 -- 172.21.15.156:0/936755013 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f33480fb110 msgr2=0x7f334810d550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:52:42.762 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:42.760+0000 7f334cee9640 1 --2- 172.21.15.156:0/936755013 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f33480fb110 0x7f334810d550 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f333c00c7e0 tx=0x7f333c0043c0 comp rx=0 tx=0).stop 2024-07-16T09:52:42.762 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:42.760+0000 7f334cee9640 1 -- 172.21.15.156:0/936755013 shutdown_connections 2024-07-16T09:52:42.762 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:42.760+0000 7f334cee9640 1 --2- 172.21.15.156:0/936755013 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f3330067190 0x7f3330069640 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:52:42.762 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:42.760+0000 7f334cee9640 1 --2- 172.21.15.156:0/936755013 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f33480fb110 0x7f334810d550 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:52:42.762 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:42.760+0000 7f334cee9640 1 --2- 172.21.15.156:0/936755013 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f33480fa7c0 0x7f334810d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:52:42.762 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:42.760+0000 7f334cee9640 1 -- 172.21.15.156:0/936755013 >> 172.21.15.156:0/936755013 conn(0x7f33480f61c0 msgr2=0x7f33480ff9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:52:42.762 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:42.760+0000 7f334cee9640 1 -- 172.21.15.156:0/936755013 shutdown_connections 2024-07-16T09:52:42.762 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:42.760+0000 7f334cee9640 1 -- 172.21.15.156:0/936755013 wait complete. 2024-07-16T09:52:43.527 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 26c91c26-4358-11ef-bcad-c7b262605968 -- bash -c 'ceph orch ls' 2024-07-16T09:52:43.783 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/26c91c26-4358-11ef-bcad-c7b262605968/mon.smithi156/config 2024-07-16T09:52:44.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:52:44 smithi176 ceph-mon[36626]: from='client.14434 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-07-16T09:52:44.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:52:44 smithi176 ceph-mon[36626]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-07-16T09:52:44.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:52:44 smithi156 ceph-mon[29302]: from='client.14434 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-07-16T09:52:44.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:52:44 smithi156 ceph-mon[29302]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-07-16T09:52:45.263 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:45.260+0000 7fbced6b3640 1 -- 172.21.15.156:0/1115030040 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbce80fb240 msgr2=0x7fbce80fb650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:52:45.263 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:45.260+0000 7fbced6b3640 1 --2- 172.21.15.156:0/1115030040 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbce80fb240 0x7fbce80fb650 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fbcd00048d0 tx=0x7fbcd0009da0 comp rx=0 tx=0).stop 2024-07-16T09:52:45.263 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:45.260+0000 7fbced6b3640 1 -- 172.21.15.156:0/1115030040 shutdown_connections 2024-07-16T09:52:45.264 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:45.260+0000 7fbced6b3640 1 --2- 172.21.15.156:0/1115030040 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbce80fc510 0x7fbce80fc980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:52:45.264 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:45.260+0000 7fbced6b3640 1 --2- 172.21.15.156:0/1115030040 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbce80fb240 0x7fbce80fb650 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:52:45.264 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:45.260+0000 7fbced6b3640 1 -- 172.21.15.156:0/1115030040 >> 172.21.15.156:0/1115030040 conn(0x7fbce80f6a20 msgr2=0x7fbce80f8e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:52:45.264 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:45.260+0000 7fbced6b3640 1 -- 172.21.15.156:0/1115030040 shutdown_connections 2024-07-16T09:52:45.264 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:45.260+0000 7fbced6b3640 1 -- 172.21.15.156:0/1115030040 wait complete. 2024-07-16T09:52:45.264 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:45.261+0000 7fbced6b3640 1 Processor -- start 2024-07-16T09:52:45.264 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:45.261+0000 7fbced6b3640 1 -- start start 2024-07-16T09:52:45.264 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:45.261+0000 7fbced6b3640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbce80fb240 0x7fbce810cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:52:45.264 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:45.261+0000 7fbced6b3640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbce80fc510 0x7fbce810d520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:52:45.264 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:45.261+0000 7fbced6b3640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbce810db70 con 0x7fbce80fb240 2024-07-16T09:52:45.264 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:45.261+0000 7fbced6b3640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbce810e340 con 0x7fbce80fc510 2024-07-16T09:52:45.264 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:45.261+0000 7fbce6ffd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbce80fb240 0x7fbce810cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:52:45.265 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:45.261+0000 7fbce6ffd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbce80fb240 0x7fbce810cfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:46238/0 (socket says 172.21.15.156:46238) 2024-07-16T09:52:45.265 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:45.261+0000 7fbce6ffd640 1 -- 172.21.15.156:0/2833996040 learned_addr learned my addr 172.21.15.156:0/2833996040 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T09:52:45.265 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:45.261+0000 7fbce67fc640 1 --2- 172.21.15.156:0/2833996040 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbce80fc510 0x7fbce810d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:52:45.265 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:45.261+0000 7fbce6ffd640 1 -- 172.21.15.156:0/2833996040 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbce80fc510 msgr2=0x7fbce810d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:52:45.265 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:45.261+0000 7fbce6ffd640 1 --2- 172.21.15.156:0/2833996040 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbce80fc510 0x7fbce810d520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:52:45.265 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:45.261+0000 7fbce6ffd640 1 -- 172.21.15.156:0/2833996040 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcd0005040 con 0x7fbce80fb240 2024-07-16T09:52:45.265 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:45.261+0000 7fbce6ffd640 1 --2- 172.21.15.156:0/2833996040 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbce80fb240 0x7fbce810cfe0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fbcd0003080 tx=0x7fbcd0009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:52:45.265 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:45.262+0000 7fbcdf7fe640 1 -- 172.21.15.156:0/2833996040 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbcd0038070 con 0x7fbce80fb240 2024-07-16T09:52:45.265 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:45.262+0000 7fbcdf7fe640 1 -- 172.21.15.156:0/2833996040 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbcd0032970 con 0x7fbce80fb240 2024-07-16T09:52:45.265 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:45.262+0000 7fbced6b3640 1 -- 172.21.15.156:0/2833996040 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbce810e570 con 0x7fbce80fb240 2024-07-16T09:52:45.265 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:45.262+0000 7fbcdf7fe640 1 -- 172.21.15.156:0/2833996040 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbcd0033900 con 0x7fbce80fb240 2024-07-16T09:52:45.265 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:45.262+0000 7fbced6b3640 1 -- 172.21.15.156:0/2833996040 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbce810eab0 con 0x7fbce80fb240 2024-07-16T09:52:45.266 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:45.263+0000 7fbced6b3640 1 -- 172.21.15.156:0/2833996040 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbce80fa6a0 con 0x7fbce80fb240 2024-07-16T09:52:45.266 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:45.263+0000 7fbcdf7fe640 1 -- 172.21.15.156:0/2833996040 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 19) v1 ==== 85127+0+0 (secure 0 0 0) 0x7fbcd0030020 con 0x7fbce80fb240 2024-07-16T09:52:45.266 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:45.263+0000 7fbcdf7fe640 1 --2- 172.21.15.156:0/2833996040 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7fbcd4067190 0x7fbcd4069640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:52:45.267 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:45.264+0000 7fbcdf7fe640 1 -- 172.21.15.156:0/2833996040 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fbcd00a4880 con 0x7fbce80fb240 2024-07-16T09:52:45.267 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:45.264+0000 7fbce67fc640 1 --2- 172.21.15.156:0/2833996040 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7fbcd4067190 0x7fbcd4069640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:52:45.267 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:45.264+0000 7fbce67fc640 1 --2- 172.21.15.156:0/2833996040 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7fbcd4067190 0x7fbcd4069640 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fbce810cd90 tx=0x7fbcd8009040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:52:45.269 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:45.268+0000 7fbcdf7fe640 1 -- 172.21.15.156:0/2833996040 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fbcd0071e60 con 0x7fbce80fb240 2024-07-16T09:52:45.442 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:45.441+0000 7fbced6b3640 1 -- 172.21.15.156:0/2833996040 --> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fbce810ecd0 con 0x7fbcd4067190 2024-07-16T09:52:45.446 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:45.445+0000 7fbcdf7fe640 1 -- 172.21.15.156:0/2833996040 <== mgr.14208 v2:172.21.15.156:6800/2615741717 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+1179 (secure 0 0 0) 0x7fbce810ecd0 con 0x7fbcd4067190 2024-07-16T09:52:45.446 INFO:teuthology.orchestra.run.smithi156.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-07-16T09:52:45.446 INFO:teuthology.orchestra.run.smithi156.stdout:alertmanager ?:9093,9094 1/1 2m ago 5m count:1 2024-07-16T09:52:45.447 INFO:teuthology.orchestra.run.smithi156.stdout:crash 2/2 2m ago 5m * 2024-07-16T09:52:45.447 INFO:teuthology.orchestra.run.smithi156.stdout:grafana ?:3000 1/1 2m ago 5m count:1 2024-07-16T09:52:45.447 INFO:teuthology.orchestra.run.smithi156.stdout:mgr 2/2 2m ago 5m count:2 2024-07-16T09:52:45.447 INFO:teuthology.orchestra.run.smithi156.stdout:mon 2/2 2m ago 4m smithi156:172.21.15.156=smithi156;smithi176:172.21.15.176=smithi176;count:2 2024-07-16T09:52:45.447 INFO:teuthology.orchestra.run.smithi156.stdout:node-exporter ?:9100 2/2 2m ago 5m * 2024-07-16T09:52:45.447 INFO:teuthology.orchestra.run.smithi156.stdout:osd 6 2m ago - 2024-07-16T09:52:45.447 INFO:teuthology.orchestra.run.smithi156.stdout:prometheus ?:9095 1/1 2m ago 5m count:1 2024-07-16T09:52:45.448 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:45.447+0000 7fbced6b3640 1 -- 172.21.15.156:0/2833996040 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7fbcd4067190 msgr2=0x7fbcd4069640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:52:45.449 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:45.447+0000 7fbced6b3640 1 --2- 172.21.15.156:0/2833996040 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7fbcd4067190 0x7fbcd4069640 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fbce810cd90 tx=0x7fbcd8009040 comp rx=0 tx=0).stop 2024-07-16T09:52:45.449 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:45.447+0000 7fbced6b3640 1 -- 172.21.15.156:0/2833996040 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbce80fb240 msgr2=0x7fbce810cfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:52:45.449 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:45.447+0000 7fbced6b3640 1 --2- 172.21.15.156:0/2833996040 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbce80fb240 0x7fbce810cfe0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fbcd0003080 tx=0x7fbcd0009da0 comp rx=0 tx=0).stop 2024-07-16T09:52:45.449 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:45.447+0000 7fbced6b3640 1 -- 172.21.15.156:0/2833996040 shutdown_connections 2024-07-16T09:52:45.449 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:45.447+0000 7fbced6b3640 1 --2- 172.21.15.156:0/2833996040 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7fbcd4067190 0x7fbcd4069640 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:52:45.449 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:45.447+0000 7fbced6b3640 1 --2- 172.21.15.156:0/2833996040 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbce80fc510 0x7fbce810d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:52:45.449 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:45.447+0000 7fbced6b3640 1 --2- 172.21.15.156:0/2833996040 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbce80fb240 0x7fbce810cfe0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:52:45.449 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:45.447+0000 7fbced6b3640 1 -- 172.21.15.156:0/2833996040 >> 172.21.15.156:0/2833996040 conn(0x7fbce80f6a20 msgr2=0x7fbce80653c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:52:45.449 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:45.447+0000 7fbced6b3640 1 -- 172.21.15.156:0/2833996040 shutdown_connections 2024-07-16T09:52:45.449 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:45.447+0000 7fbced6b3640 1 -- 172.21.15.156:0/2833996040 wait complete. 2024-07-16T09:52:46.078 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 26c91c26-4358-11ef-bcad-c7b262605968 -- bash -c 'ceph orch host ls' 2024-07-16T09:52:46.334 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/26c91c26-4358-11ef-bcad-c7b262605968/mon.smithi156/config 2024-07-16T09:52:46.457 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:52:46 smithi156 ceph-mon[29302]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-07-16T09:52:46.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:52:46 smithi176 ceph-mon[36626]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-07-16T09:52:47.424 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:52:47 smithi156 ceph-mon[29302]: from='client.14438 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-07-16T09:52:47.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:52:47 smithi176 ceph-mon[36626]: from='client.14438 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-07-16T09:52:48.012 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:48.009+0000 7f7c7aebc640 1 -- 172.21.15.156:0/3142937081 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7c740fd630 msgr2=0x7f7c740fdaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:52:48.012 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:48.009+0000 7f7c7aebc640 1 --2- 172.21.15.156:0/3142937081 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7c740fd630 0x7f7c740fdaa0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f7c600048d0 tx=0x7f7c60009da0 comp rx=0 tx=0).stop 2024-07-16T09:52:48.012 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:48.009+0000 7f7c7aebc640 1 -- 172.21.15.156:0/3142937081 shutdown_connections 2024-07-16T09:52:48.012 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:48.009+0000 7f7c7aebc640 1 --2- 172.21.15.156:0/3142937081 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7c740fd630 0x7f7c740fdaa0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:52:48.013 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:48.009+0000 7f7c7aebc640 1 --2- 172.21.15.156:0/3142937081 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7c740fc360 0x7f7c740fc770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:52:48.013 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:48.009+0000 7f7c7aebc640 1 -- 172.21.15.156:0/3142937081 >> 172.21.15.156:0/3142937081 conn(0x7f7c740f7b40 msgr2=0x7f7c740f9f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:52:48.013 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:48.009+0000 7f7c7aebc640 1 -- 172.21.15.156:0/3142937081 shutdown_connections 2024-07-16T09:52:48.013 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:48.009+0000 7f7c7aebc640 1 -- 172.21.15.156:0/3142937081 wait complete. 2024-07-16T09:52:48.013 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:48.010+0000 7f7c7aebc640 1 Processor -- start 2024-07-16T09:52:48.013 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:48.010+0000 7f7c7aebc640 1 -- start start 2024-07-16T09:52:48.013 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:48.010+0000 7f7c7aebc640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7c740fc360 0x7f7c7410cf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:52:48.013 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:48.010+0000 7f7c7aebc640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7c740fd630 0x7f7c7410d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:52:48.013 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:48.010+0000 7f7c7aebc640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c7410db10 con 0x7f7c740fd630 2024-07-16T09:52:48.013 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:48.010+0000 7f7c7aebc640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c7410e330 con 0x7f7c740fc360 2024-07-16T09:52:48.013 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:48.010+0000 7f7c73fff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7c740fd630 0x7f7c7410d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:52:48.013 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:48.010+0000 7f7c73fff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7c740fd630 0x7f7c7410d4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:46256/0 (socket says 172.21.15.156:46256) 2024-07-16T09:52:48.013 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:48.010+0000 7f7c73fff640 1 -- 172.21.15.156:0/3480616699 learned_addr learned my addr 172.21.15.156:0/3480616699 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T09:52:48.014 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:48.010+0000 7f7c78c58640 1 --2- 172.21.15.156:0/3480616699 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7c740fc360 0x7f7c7410cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:52:48.014 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:48.010+0000 7f7c73fff640 1 -- 172.21.15.156:0/3480616699 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7c740fc360 msgr2=0x7f7c7410cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:52:48.014 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:48.010+0000 7f7c73fff640 1 --2- 172.21.15.156:0/3480616699 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7c740fc360 0x7f7c7410cf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:52:48.014 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:48.011+0000 7f7c73fff640 1 -- 172.21.15.156:0/3480616699 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c60005040 con 0x7f7c740fd630 2024-07-16T09:52:48.014 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:48.011+0000 7f7c73fff640 1 --2- 172.21.15.156:0/3480616699 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7c740fd630 0x7f7c7410d4c0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f7c60003080 tx=0x7f7c60009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:52:48.014 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:48.011+0000 7f7c717fa640 1 -- 172.21.15.156:0/3480616699 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7c60038070 con 0x7f7c740fd630 2024-07-16T09:52:48.014 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:48.011+0000 7f7c717fa640 1 -- 172.21.15.156:0/3480616699 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7c60032970 con 0x7f7c740fd630 2024-07-16T09:52:48.014 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:48.011+0000 7f7c7aebc640 1 -- 172.21.15.156:0/3480616699 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c7410e560 con 0x7f7c740fd630 2024-07-16T09:52:48.014 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:48.011+0000 7f7c717fa640 1 -- 172.21.15.156:0/3480616699 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7c60033900 con 0x7f7c740fd630 2024-07-16T09:52:48.014 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:48.011+0000 7f7c7aebc640 1 -- 172.21.15.156:0/3480616699 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c7410eaa0 con 0x7f7c740fd630 2024-07-16T09:52:48.015 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:48.012+0000 7f7c7aebc640 1 -- 172.21.15.156:0/3480616699 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c740fb7c0 con 0x7f7c740fd630 2024-07-16T09:52:48.015 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:48.013+0000 7f7c717fa640 1 -- 172.21.15.156:0/3480616699 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 19) v1 ==== 85127+0+0 (secure 0 0 0) 0x7f7c60030020 con 0x7f7c740fd630 2024-07-16T09:52:48.015 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:48.013+0000 7f7c717fa640 1 --2- 172.21.15.156:0/3480616699 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f7c64067190 0x7f7c64069640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:52:48.015 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:48.013+0000 7f7c717fa640 1 -- 172.21.15.156:0/3480616699 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f7c600a4800 con 0x7f7c740fd630 2024-07-16T09:52:48.015 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:48.013+0000 7f7c78c58640 1 --2- 172.21.15.156:0/3480616699 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f7c64067190 0x7f7c64069640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:52:48.016 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:48.014+0000 7f7c78c58640 1 --2- 172.21.15.156:0/3480616699 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f7c64067190 0x7f7c64069640 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f7c680041d0 tx=0x7f7c68009040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:52:48.018 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:48.017+0000 7f7c717fa640 1 -- 172.21.15.156:0/3480616699 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f7c60071e60 con 0x7f7c740fd630 2024-07-16T09:52:48.191 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:48.190+0000 7f7c7aebc640 1 -- 172.21.15.156:0/3480616699 --> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f7c7410ee40 con 0x7f7c64067190 2024-07-16T09:52:48.192 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:48.191+0000 7f7c717fa640 1 -- 172.21.15.156:0/3480616699 <== mgr.14208 v2:172.21.15.156:6800/2615741717 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+148 (secure 0 0 0) 0x7f7c7410ee40 con 0x7f7c64067190 2024-07-16T09:52:48.192 INFO:teuthology.orchestra.run.smithi156.stdout:HOST ADDR LABELS STATUS 2024-07-16T09:52:48.193 INFO:teuthology.orchestra.run.smithi156.stdout:smithi156 172.21.15.156 2024-07-16T09:52:48.193 INFO:teuthology.orchestra.run.smithi156.stdout:smithi176 172.21.15.176 2024-07-16T09:52:48.193 INFO:teuthology.orchestra.run.smithi156.stdout:2 hosts in cluster 2024-07-16T09:52:48.195 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:48.193+0000 7f7c7aebc640 1 -- 172.21.15.156:0/3480616699 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f7c64067190 msgr2=0x7f7c64069640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:52:48.195 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:48.193+0000 7f7c7aebc640 1 --2- 172.21.15.156:0/3480616699 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f7c64067190 0x7f7c64069640 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f7c680041d0 tx=0x7f7c68009040 comp rx=0 tx=0).stop 2024-07-16T09:52:48.195 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:48.193+0000 7f7c7aebc640 1 -- 172.21.15.156:0/3480616699 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7c740fd630 msgr2=0x7f7c7410d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:52:48.195 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:48.193+0000 7f7c7aebc640 1 --2- 172.21.15.156:0/3480616699 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7c740fd630 0x7f7c7410d4c0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f7c60003080 tx=0x7f7c60009da0 comp rx=0 tx=0).stop 2024-07-16T09:52:48.195 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:48.193+0000 7f7c7aebc640 1 -- 172.21.15.156:0/3480616699 shutdown_connections 2024-07-16T09:52:48.195 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:48.193+0000 7f7c7aebc640 1 --2- 172.21.15.156:0/3480616699 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f7c64067190 0x7f7c64069640 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:52:48.195 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:48.193+0000 7f7c7aebc640 1 --2- 172.21.15.156:0/3480616699 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7c740fd630 0x7f7c7410d4c0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:52:48.195 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:48.193+0000 7f7c7aebc640 1 --2- 172.21.15.156:0/3480616699 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7c740fc360 0x7f7c7410cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:52:48.195 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:48.193+0000 7f7c7aebc640 1 -- 172.21.15.156:0/3480616699 >> 172.21.15.156:0/3480616699 conn(0x7f7c740f7b40 msgr2=0x7f7c740f9230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:52:48.195 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:48.193+0000 7f7c7aebc640 1 -- 172.21.15.156:0/3480616699 shutdown_connections 2024-07-16T09:52:48.195 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:48.193+0000 7f7c7aebc640 1 -- 172.21.15.156:0/3480616699 wait complete. 2024-07-16T09:52:48.501 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:52:48 smithi156 ceph-mon[29302]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-07-16T09:52:48.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:52:48 smithi176 ceph-mon[36626]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-07-16T09:52:48.874 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 26c91c26-4358-11ef-bcad-c7b262605968 -- bash -c 'ceph orch device ls' 2024-07-16T09:52:49.133 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/26c91c26-4358-11ef-bcad-c7b262605968/mon.smithi156/config 2024-07-16T09:52:49.479 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:52:49 smithi156 ceph-mon[29302]: from='client.14442 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-07-16T09:52:49.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:52:49 smithi176 ceph-mon[36626]: from='client.14442 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-07-16T09:52:50.410 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:52:50 smithi156 ceph-mon[29302]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-07-16T09:52:50.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:52:50 smithi176 ceph-mon[36626]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-07-16T09:52:50.616 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:50.613+0000 7fa0643c2640 1 -- 172.21.15.156:0/164379508 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa05c0fc530 msgr2=0x7fa05c0fc9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:52:50.616 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:50.613+0000 7fa0643c2640 1 --2- 172.21.15.156:0/164379508 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa05c0fc530 0x7fa05c0fc9a0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fa04c0048d0 tx=0x7fa04c009da0 comp rx=0 tx=0).stop 2024-07-16T09:52:50.617 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:50.613+0000 7fa0643c2640 1 -- 172.21.15.156:0/164379508 shutdown_connections 2024-07-16T09:52:50.617 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:50.613+0000 7fa0643c2640 1 --2- 172.21.15.156:0/164379508 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa05c0fc530 0x7fa05c0fc9a0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:52:50.617 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:50.613+0000 7fa0643c2640 1 --2- 172.21.15.156:0/164379508 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa05c0fb260 0x7fa05c0fb670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:52:50.617 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:50.613+0000 7fa0643c2640 1 -- 172.21.15.156:0/164379508 >> 172.21.15.156:0/164379508 conn(0x7fa05c0f6a20 msgr2=0x7fa05c0f8e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:52:50.617 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:50.613+0000 7fa0643c2640 1 -- 172.21.15.156:0/164379508 shutdown_connections 2024-07-16T09:52:50.617 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:50.613+0000 7fa0643c2640 1 -- 172.21.15.156:0/164379508 wait complete. 2024-07-16T09:52:50.617 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:50.614+0000 7fa0643c2640 1 Processor -- start 2024-07-16T09:52:50.617 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:50.614+0000 7fa0643c2640 1 -- start start 2024-07-16T09:52:50.617 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:50.614+0000 7fa0643c2640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa05c0fb260 0x7fa05c10fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:52:50.617 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:50.614+0000 7fa0643c2640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa05c0fc530 0x7fa05c112230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:52:50.617 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:50.614+0000 7fa0643c2640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa05c066540 con 0x7fa05c0fc530 2024-07-16T09:52:50.617 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:50.614+0000 7fa0643c2640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa05c0666b0 con 0x7fa05c0fb260 2024-07-16T09:52:50.617 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:50.614+0000 7fa06195d640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa05c0fc530 0x7fa05c112230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:52:50.617 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:50.614+0000 7fa06195d640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa05c0fc530 0x7fa05c112230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:46268/0 (socket says 172.21.15.156:46268) 2024-07-16T09:52:50.618 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:50.614+0000 7fa06195d640 1 -- 172.21.15.156:0/1177997486 learned_addr learned my addr 172.21.15.156:0/1177997486 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T09:52:50.618 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:50.615+0000 7fa06215e640 1 --2- 172.21.15.156:0/1177997486 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa05c0fb260 0x7fa05c10fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:52:50.618 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:50.615+0000 7fa06195d640 1 -- 172.21.15.156:0/1177997486 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa05c0fb260 msgr2=0x7fa05c10fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:52:50.618 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:50.615+0000 7fa06195d640 1 --2- 172.21.15.156:0/1177997486 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa05c0fb260 0x7fa05c10fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:52:50.618 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:50.615+0000 7fa06195d640 1 -- 172.21.15.156:0/1177997486 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa04c005040 con 0x7fa05c0fc530 2024-07-16T09:52:50.618 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:50.615+0000 7fa06215e640 1 --2- 172.21.15.156:0/1177997486 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa05c0fb260 0x7fa05c10fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-16T09:52:50.618 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:50.615+0000 7fa06195d640 1 --2- 172.21.15.156:0/1177997486 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa05c0fc530 0x7fa05c112230 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fa04c003080 tx=0x7fa04c009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:52:50.618 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:50.615+0000 7fa052ffd640 1 -- 172.21.15.156:0/1177997486 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa04c038070 con 0x7fa05c0fc530 2024-07-16T09:52:50.618 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:50.615+0000 7fa052ffd640 1 -- 172.21.15.156:0/1177997486 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa04c032970 con 0x7fa05c0fc530 2024-07-16T09:52:50.618 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:50.615+0000 7fa0643c2640 1 -- 172.21.15.156:0/1177997486 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa05c112770 con 0x7fa05c0fc530 2024-07-16T09:52:50.618 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:50.615+0000 7fa052ffd640 1 -- 172.21.15.156:0/1177997486 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa04c033820 con 0x7fa05c0fc530 2024-07-16T09:52:50.618 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:50.615+0000 7fa0643c2640 1 -- 172.21.15.156:0/1177997486 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa05c112c80 con 0x7fa05c0fc530 2024-07-16T09:52:50.619 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:50.616+0000 7fa0643c2640 1 -- 172.21.15.156:0/1177997486 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa05c0fa6c0 con 0x7fa05c0fc530 2024-07-16T09:52:50.619 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:50.617+0000 7fa052ffd640 1 -- 172.21.15.156:0/1177997486 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 19) v1 ==== 85127+0+0 (secure 0 0 0) 0x7fa04c030020 con 0x7fa05c0fc530 2024-07-16T09:52:50.619 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:50.617+0000 7fa052ffd640 1 --2- 172.21.15.156:0/1177997486 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7fa048067190 0x7fa048069640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:52:50.619 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:50.617+0000 7fa052ffd640 1 -- 172.21.15.156:0/1177997486 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fa04c0a47a0 con 0x7fa05c0fc530 2024-07-16T09:52:50.619 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:50.617+0000 7fa06215e640 1 --2- 172.21.15.156:0/1177997486 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7fa048067190 0x7fa048069640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:52:50.620 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:50.617+0000 7fa06215e640 1 --2- 172.21.15.156:0/1177997486 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7fa048067190 0x7fa048069640 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fa05c066230 tx=0x7fa058009040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:52:50.622 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:50.621+0000 7fa052ffd640 1 -- 172.21.15.156:0/1177997486 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fa04c071d80 con 0x7fa05c0fc530 2024-07-16T09:52:50.800 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:50.798+0000 7fa0643c2640 1 -- 172.21.15.156:0/1177997486 --> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fa05c100840 con 0x7fa048067190 2024-07-16T09:52:50.801 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:50.800+0000 7fa052ffd640 1 -- 172.21.15.156:0/1177997486 <== mgr.14208 v2:172.21.15.156:6800/2615741717 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+399 (secure 0 0 0) 0x7fa05c100840 con 0x7fa048067190 2024-07-16T09:52:50.801 INFO:teuthology.orchestra.run.smithi156.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-07-16T09:52:50.801 INFO:teuthology.orchestra.run.smithi156.stdout:smithi156 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB74760040480DGN 447G No 2m ago Has a FileSystem, LVM detected 2024-07-16T09:52:50.801 INFO:teuthology.orchestra.run.smithi156.stdout:smithi176 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT62330099400BGN 372G No 71s ago Has a FileSystem, LVM detected 2024-07-16T09:52:50.803 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:50.802+0000 7fa0643c2640 1 -- 172.21.15.156:0/1177997486 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7fa048067190 msgr2=0x7fa048069640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:52:50.803 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:50.802+0000 7fa0643c2640 1 --2- 172.21.15.156:0/1177997486 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7fa048067190 0x7fa048069640 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fa05c066230 tx=0x7fa058009040 comp rx=0 tx=0).stop 2024-07-16T09:52:50.803 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:50.802+0000 7fa0643c2640 1 -- 172.21.15.156:0/1177997486 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa05c0fc530 msgr2=0x7fa05c112230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:52:50.803 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:50.802+0000 7fa0643c2640 1 --2- 172.21.15.156:0/1177997486 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa05c0fc530 0x7fa05c112230 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fa04c003080 tx=0x7fa04c009da0 comp rx=0 tx=0).stop 2024-07-16T09:52:50.804 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:50.802+0000 7fa0643c2640 1 -- 172.21.15.156:0/1177997486 shutdown_connections 2024-07-16T09:52:50.804 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:50.802+0000 7fa0643c2640 1 --2- 172.21.15.156:0/1177997486 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7fa048067190 0x7fa048069640 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:52:50.804 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:50.802+0000 7fa0643c2640 1 --2- 172.21.15.156:0/1177997486 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa05c0fc530 0x7fa05c112230 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:52:50.804 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:50.802+0000 7fa0643c2640 1 --2- 172.21.15.156:0/1177997486 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa05c0fb260 0x7fa05c10fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:52:50.804 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:50.802+0000 7fa0643c2640 1 -- 172.21.15.156:0/1177997486 >> 172.21.15.156:0/1177997486 conn(0x7fa05c0f6a20 msgr2=0x7fa05c0f8170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:52:50.804 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:50.802+0000 7fa0643c2640 1 -- 172.21.15.156:0/1177997486 shutdown_connections 2024-07-16T09:52:50.804 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:50.802+0000 7fa0643c2640 1 -- 172.21.15.156:0/1177997486 wait complete. 2024-07-16T09:52:51.460 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-07-16T09:52:51.471 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi156.front.sepia.ceph.com 2024-07-16T09:52:51.471 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 26c91c26-4358-11ef-bcad-c7b262605968 -- bash -c 'ceph fs volume create cephfs --placement=4' 2024-07-16T09:52:51.731 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/26c91c26-4358-11ef-bcad-c7b262605968/mon.smithi156/config 2024-07-16T09:52:52.424 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:52:52 smithi156 ceph-mon[29302]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-07-16T09:52:52.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:52:52 smithi156 ceph-mon[29302]: from='client.14446 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-07-16T09:52:52.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:52:52 smithi176 ceph-mon[36626]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-07-16T09:52:52.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:52:52 smithi176 ceph-mon[36626]: from='client.14446 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-07-16T09:52:53.301 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:53.296+0000 7fece7b60640 1 -- 172.21.15.156:0/1354456075 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fece00fc530 msgr2=0x7fece00fc9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:52:53.301 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:53.296+0000 7fece7b60640 1 --2- 172.21.15.156:0/1354456075 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fece00fc530 0x7fece00fc9a0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fecd00048d0 tx=0x7fecd0009da0 comp rx=0 tx=0).stop 2024-07-16T09:52:53.301 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:53.296+0000 7fece7b60640 1 -- 172.21.15.156:0/1354456075 shutdown_connections 2024-07-16T09:52:53.301 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:53.296+0000 7fece7b60640 1 --2- 172.21.15.156:0/1354456075 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fece00fc530 0x7fece00fc9a0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:52:53.301 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:53.296+0000 7fece7b60640 1 --2- 172.21.15.156:0/1354456075 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fece00fb260 0x7fece00fb670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:52:53.301 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:53.296+0000 7fece7b60640 1 -- 172.21.15.156:0/1354456075 >> 172.21.15.156:0/1354456075 conn(0x7fece00f6a20 msgr2=0x7fece00f8e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:52:53.301 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:53.296+0000 7fece7b60640 1 -- 172.21.15.156:0/1354456075 shutdown_connections 2024-07-16T09:52:53.301 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:53.296+0000 7fece7b60640 1 -- 172.21.15.156:0/1354456075 wait complete. 2024-07-16T09:52:53.301 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:53.296+0000 7fece7b60640 1 Processor -- start 2024-07-16T09:52:53.302 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:53.297+0000 7fece7b60640 1 -- start start 2024-07-16T09:52:53.302 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:53.297+0000 7fece7b60640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fece00fb260 0x7fece010bf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:52:53.302 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:53.297+0000 7fece7b60640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fece00fc530 0x7fece010c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:52:53.302 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:53.297+0000 7fece7b60640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fece010d2f0 con 0x7fece00fb260 2024-07-16T09:52:53.302 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:53.297+0000 7fece7b60640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fece010d460 con 0x7fece00fc530 2024-07-16T09:52:53.302 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:53.297+0000 7fece58fc640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fece00fb260 0x7fece010bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:52:53.302 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:53.297+0000 7fece58fc640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fece00fb260 0x7fece010bf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:34836/0 (socket says 172.21.15.156:34836) 2024-07-16T09:52:53.302 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:53.297+0000 7fece58fc640 1 -- 172.21.15.156:0/3517012456 learned_addr learned my addr 172.21.15.156:0/3517012456 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T09:52:53.302 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:53.297+0000 7fece58fc640 1 -- 172.21.15.156:0/3517012456 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fece00fc530 msgr2=0x7fece010c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:52:53.302 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:53.297+0000 7fece58fc640 1 --2- 172.21.15.156:0/3517012456 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fece00fc530 0x7fece010c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:52:53.302 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:53.297+0000 7fece58fc640 1 -- 172.21.15.156:0/3517012456 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecd0005040 con 0x7fece00fb260 2024-07-16T09:52:53.302 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:53.297+0000 7fece58fc640 1 --2- 172.21.15.156:0/3517012456 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fece00fb260 0x7fece010bf40 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fecdc00b790 tx=0x7fecdc0091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:52:53.303 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:53.297+0000 7fecd67fc640 1 -- 172.21.15.156:0/3517012456 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fecdc00c700 con 0x7fece00fb260 2024-07-16T09:52:53.303 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:53.297+0000 7fecd67fc640 1 -- 172.21.15.156:0/3517012456 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fecdc00cd30 con 0x7fece00fb260 2024-07-16T09:52:53.303 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:53.297+0000 7fecd67fc640 1 -- 172.21.15.156:0/3517012456 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fecdc013370 con 0x7fece00fb260 2024-07-16T09:52:53.303 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:53.297+0000 7fece7b60640 1 -- 172.21.15.156:0/3517012456 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fece010d740 con 0x7fece00fb260 2024-07-16T09:52:53.303 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:53.297+0000 7fece7b60640 1 -- 172.21.15.156:0/3517012456 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fece010dc10 con 0x7fece00fb260 2024-07-16T09:52:53.303 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:53.298+0000 7fece7b60640 1 -- 172.21.15.156:0/3517012456 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fece00fa6c0 con 0x7fece00fb260 2024-07-16T09:52:53.303 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:53.299+0000 7fecd67fc640 1 -- 172.21.15.156:0/3517012456 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 19) v1 ==== 85127+0+0 (secure 0 0 0) 0x7fecdc012070 con 0x7fece00fb260 2024-07-16T09:52:53.303 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:53.299+0000 7fecd67fc640 1 --2- 172.21.15.156:0/3517012456 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7feccc067190 0x7feccc069640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:52:53.303 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:53.299+0000 7fecd67fc640 1 -- 172.21.15.156:0/3517012456 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fecdc0843d0 con 0x7fece00fb260 2024-07-16T09:52:53.303 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:53.299+0000 7fece50fb640 1 --2- 172.21.15.156:0/3517012456 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7feccc067190 0x7feccc069640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:52:53.303 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:53.300+0000 7fece50fb640 1 --2- 172.21.15.156:0/3517012456 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7feccc067190 0x7feccc069640 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fecd0009f80 tx=0x7fecd0009da0 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:52:53.304 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:53.303+0000 7fecd67fc640 1 -- 172.21.15.156:0/3517012456 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fecdc0519b0 con 0x7fece00fb260 2024-07-16T09:52:53.495 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:53.494+0000 7fece7b60640 1 -- 172.21.15.156:0/3517012456 --> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] -- mgr_command(tid 0: {"prefix": "fs volume create", "name": "cephfs", "placement": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7fece010dee0 con 0x7feccc067190 2024-07-16T09:52:54.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:52:54 smithi156 ceph-mon[29302]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-07-16T09:52:54.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:52:54 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]: dispatch 2024-07-16T09:52:54.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:52:54 smithi176 ceph-mon[36626]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-07-16T09:52:54.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:52:54 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]: dispatch 2024-07-16T09:52:55.237 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:55.235+0000 7fecd67fc640 1 -- 172.21.15.156:0/3517012456 <== mgr.14208 v2:172.21.15.156:6800/2615741717 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fece010dee0 con 0x7feccc067190 2024-07-16T09:52:55.239 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:55.238+0000 7fece7b60640 1 -- 172.21.15.156:0/3517012456 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7feccc067190 msgr2=0x7feccc069640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:52:55.240 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:55.238+0000 7fece7b60640 1 --2- 172.21.15.156:0/3517012456 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7feccc067190 0x7feccc069640 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fecd0009f80 tx=0x7fecd0009da0 comp rx=0 tx=0).stop 2024-07-16T09:52:55.240 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:55.238+0000 7fece7b60640 1 -- 172.21.15.156:0/3517012456 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fece00fb260 msgr2=0x7fece010bf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:52:55.240 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:55.238+0000 7fece7b60640 1 --2- 172.21.15.156:0/3517012456 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fece00fb260 0x7fece010bf40 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fecdc00b790 tx=0x7fecdc0091d0 comp rx=0 tx=0).stop 2024-07-16T09:52:55.240 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:55.238+0000 7fece7b60640 1 -- 172.21.15.156:0/3517012456 shutdown_connections 2024-07-16T09:52:55.240 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:55.238+0000 7fece7b60640 1 --2- 172.21.15.156:0/3517012456 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7feccc067190 0x7feccc069640 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:52:55.240 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:55.238+0000 7fece7b60640 1 --2- 172.21.15.156:0/3517012456 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fece00fc530 0x7fece010c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:52:55.240 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:55.238+0000 7fece7b60640 1 --2- 172.21.15.156:0/3517012456 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fece00fb260 0x7fece010bf40 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:52:55.240 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:55.238+0000 7fece7b60640 1 -- 172.21.15.156:0/3517012456 >> 172.21.15.156:0/3517012456 conn(0x7fece00f6a20 msgr2=0x7fece00f8140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:52:55.240 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:55.238+0000 7fece7b60640 1 -- 172.21.15.156:0/3517012456 shutdown_connections 2024-07-16T09:52:55.240 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:55.238+0000 7fece7b60640 1 -- 172.21.15.156:0/3517012456 wait complete. 2024-07-16T09:52:55.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:52:55 smithi176 ceph-mon[36626]: from='client.14450 -' entity='client.admin' cmd=[{"prefix": "fs volume create", "name": "cephfs", "placement": "4", "target": ["mon-mgr", ""]}]: dispatch 2024-07-16T09:52:55.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:52:55 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd='[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]': finished 2024-07-16T09:52:55.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:52:55 smithi176 ceph-mon[36626]: osdmap e42: 6 total, 6 up, 6 in 2024-07-16T09:52:55.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:52:55 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"bulk": true, "prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]: dispatch 2024-07-16T09:52:55.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:52:55 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-16T09:52:55.596 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:52:55 smithi156 ceph-mon[29302]: from='client.14450 -' entity='client.admin' cmd=[{"prefix": "fs volume create", "name": "cephfs", "placement": "4", "target": ["mon-mgr", ""]}]: dispatch 2024-07-16T09:52:55.596 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:52:55 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd='[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]': finished 2024-07-16T09:52:55.596 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:52:55 smithi156 ceph-mon[29302]: osdmap e42: 6 total, 6 up, 6 in 2024-07-16T09:52:55.597 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:52:55 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"bulk": true, "prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]: dispatch 2024-07-16T09:52:55.597 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:52:55 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-16T09:52:55.597 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:52:55 smithi156 ceph-26c91c26-4358-11ef-bcad-c7b262605968-mon-smithi156[29298]: 2024-07-16T09:52:55.214+0000 7fdb82fe4640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-07-16T09:52:55.977 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 26c91c26-4358-11ef-bcad-c7b262605968 -- bash -c 'ceph fs dump' 2024-07-16T09:52:56.467 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:52:56 smithi156 ceph-mon[29302]: pgmap v165: 33 pgs: 9 creating+peering, 1 active+clean, 23 unknown; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-07-16T09:52:56.467 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:52:56 smithi156 ceph-mon[29302]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-07-16T09:52:56.467 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:52:56 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd='[{"bulk": true, "prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]': finished 2024-07-16T09:52:56.467 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:52:56 smithi156 ceph-mon[29302]: osdmap e43: 6 total, 6 up, 6 in 2024-07-16T09:52:56.467 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:52:56 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]: dispatch 2024-07-16T09:52:56.467 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:52:56 smithi156 ceph-mon[29302]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-07-16T09:52:56.467 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:52:56 smithi156 ceph-mon[29302]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-07-16T09:52:56.467 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:52:56 smithi156 ceph-mon[29302]: osdmap e44: 6 total, 6 up, 6 in 2024-07-16T09:52:56.467 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:52:56 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]': finished 2024-07-16T09:52:56.467 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:52:56 smithi156 ceph-mon[29302]: fsmap cephfs:0 2024-07-16T09:52:56.467 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:52:56 smithi156 ceph-mon[29302]: Saving service mds.cephfs spec with placement count:4 2024-07-16T09:52:56.468 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:52:56 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:52:56.468 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:52:56 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-16T09:52:56.468 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:52:56 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-16T09:52:56.468 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:52:56 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-16T09:52:56.468 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:52:56 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:52:56.468 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:52:56 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi156.rdibys", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-16T09:52:56.468 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:52:56 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi156.rdibys", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-07-16T09:52:56.468 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:52:56 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-16T09:52:56.468 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:52:56 smithi156 ceph-mon[29302]: Deploying daemon mds.cephfs.smithi156.rdibys on smithi156 2024-07-16T09:52:56.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:52:56 smithi176 ceph-mon[36626]: pgmap v165: 33 pgs: 9 creating+peering, 1 active+clean, 23 unknown; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-07-16T09:52:56.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:52:56 smithi176 ceph-mon[36626]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-07-16T09:52:56.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:52:56 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd='[{"bulk": true, "prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]': finished 2024-07-16T09:52:56.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:52:56 smithi176 ceph-mon[36626]: osdmap e43: 6 total, 6 up, 6 in 2024-07-16T09:52:56.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:52:56 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]: dispatch 2024-07-16T09:52:56.595 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:52:56 smithi176 ceph-mon[36626]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-07-16T09:52:56.595 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:52:56 smithi176 ceph-mon[36626]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-07-16T09:52:56.595 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:52:56 smithi176 ceph-mon[36626]: osdmap e44: 6 total, 6 up, 6 in 2024-07-16T09:52:56.595 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:52:56 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]': finished 2024-07-16T09:52:56.595 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:52:56 smithi176 ceph-mon[36626]: fsmap cephfs:0 2024-07-16T09:52:56.595 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:52:56 smithi176 ceph-mon[36626]: Saving service mds.cephfs spec with placement count:4 2024-07-16T09:52:56.595 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:52:56 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:52:56.595 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:52:56 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-16T09:52:56.595 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:52:56 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-16T09:52:56.595 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:52:56 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-16T09:52:56.595 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:52:56 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:52:56.595 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:52:56 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi156.rdibys", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-16T09:52:56.595 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:52:56 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi156.rdibys", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-07-16T09:52:56.595 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:52:56 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-16T09:52:56.596 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:52:56 smithi176 ceph-mon[36626]: Deploying daemon mds.cephfs.smithi156.rdibys on smithi156 2024-07-16T09:52:57.080 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/26c91c26-4358-11ef-bcad-c7b262605968/mon.smithi156/config 2024-07-16T09:52:57.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:52:57 smithi156 ceph-mon[29302]: osdmap e45: 6 total, 6 up, 6 in 2024-07-16T09:52:57.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:52:57 smithi176 ceph-mon[36626]: osdmap e45: 6 total, 6 up, 6 in 2024-07-16T09:52:58.411 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:52:58 smithi156 ceph-mon[29302]: pgmap v169: 65 pgs: 18 creating+peering, 4 active+clean, 43 unknown; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-07-16T09:52:58.412 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:52:58 smithi156 ceph-mon[29302]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-07-16T09:52:58.412 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:52:58 smithi156 ceph-mon[29302]: osdmap e46: 6 total, 6 up, 6 in 2024-07-16T09:52:58.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:52:58 smithi176 ceph-mon[36626]: pgmap v169: 65 pgs: 18 creating+peering, 4 active+clean, 43 unknown; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-07-16T09:52:58.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:52:58 smithi176 ceph-mon[36626]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-07-16T09:52:58.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:52:58 smithi176 ceph-mon[36626]: osdmap e46: 6 total, 6 up, 6 in 2024-07-16T09:52:58.722 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:58.716+0000 7ff8c9bef640 1 -- 172.21.15.156:0/710937281 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff8c40fb260 msgr2=0x7ff8c40fb670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:52:58.722 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:58.716+0000 7ff8c9bef640 1 --2- 172.21.15.156:0/710937281 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff8c40fb260 0x7ff8c40fb670 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7ff8b40048d0 tx=0x7ff8b4009da0 comp rx=0 tx=0).stop 2024-07-16T09:52:58.722 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:58.716+0000 7ff8c9bef640 1 -- 172.21.15.156:0/710937281 shutdown_connections 2024-07-16T09:52:58.722 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:58.716+0000 7ff8c9bef640 1 --2- 172.21.15.156:0/710937281 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff8c40fc530 0x7ff8c40fc9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:52:58.723 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:58.717+0000 7ff8c9bef640 1 --2- 172.21.15.156:0/710937281 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff8c40fb260 0x7ff8c40fb670 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:52:58.723 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:58.717+0000 7ff8c9bef640 1 -- 172.21.15.156:0/710937281 >> 172.21.15.156:0/710937281 conn(0x7ff8c40f6a40 msgr2=0x7ff8c40f8e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:52:58.723 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:58.717+0000 7ff8c9bef640 1 -- 172.21.15.156:0/710937281 shutdown_connections 2024-07-16T09:52:58.723 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:58.717+0000 7ff8c9bef640 1 -- 172.21.15.156:0/710937281 wait complete. 2024-07-16T09:52:58.723 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:58.717+0000 7ff8c9bef640 1 Processor -- start 2024-07-16T09:52:58.723 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:58.717+0000 7ff8c9bef640 1 -- start start 2024-07-16T09:52:58.723 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:58.717+0000 7ff8c9bef640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff8c40fb260 0x7ff8c410fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:52:58.723 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:58.717+0000 7ff8c9bef640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff8c40fc530 0x7ff8c41121a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:52:58.723 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:58.717+0000 7ff8c9bef640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8c4066580 con 0x7ff8c40fb260 2024-07-16T09:52:58.723 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:58.717+0000 7ff8c9bef640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8c40666f0 con 0x7ff8c40fc530 2024-07-16T09:52:58.723 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:58.717+0000 7ff8c37fe640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff8c40fb260 0x7ff8c410fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:52:58.723 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:58.717+0000 7ff8c37fe640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff8c40fb260 0x7ff8c410fc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:34850/0 (socket says 172.21.15.156:34850) 2024-07-16T09:52:58.724 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:58.717+0000 7ff8c37fe640 1 -- 172.21.15.156:0/3681673145 learned_addr learned my addr 172.21.15.156:0/3681673145 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T09:52:58.724 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:58.718+0000 7ff8c2ffd640 1 --2- 172.21.15.156:0/3681673145 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff8c40fc530 0x7ff8c41121a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:52:58.724 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:58.718+0000 7ff8c37fe640 1 -- 172.21.15.156:0/3681673145 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff8c40fc530 msgr2=0x7ff8c41121a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:52:58.724 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:58.718+0000 7ff8c37fe640 1 --2- 172.21.15.156:0/3681673145 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff8c40fc530 0x7ff8c41121a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:52:58.724 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:58.718+0000 7ff8c37fe640 1 -- 172.21.15.156:0/3681673145 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8b4005040 con 0x7ff8c40fb260 2024-07-16T09:52:58.724 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:58.718+0000 7ff8c2ffd640 1 --2- 172.21.15.156:0/3681673145 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff8c40fc530 0x7ff8c41121a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-16T09:52:58.724 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:58.718+0000 7ff8c37fe640 1 --2- 172.21.15.156:0/3681673145 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff8c40fb260 0x7ff8c410fc50 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7ff8b4009f80 tx=0x7ff8b4009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:52:58.724 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:58.718+0000 7ff8abfff640 1 -- 172.21.15.156:0/3681673145 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff8b4038070 con 0x7ff8c40fb260 2024-07-16T09:52:58.724 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:58.718+0000 7ff8abfff640 1 -- 172.21.15.156:0/3681673145 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff8b4032970 con 0x7ff8c40fb260 2024-07-16T09:52:58.724 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:58.718+0000 7ff8abfff640 1 -- 172.21.15.156:0/3681673145 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff8b4033860 con 0x7ff8c40fb260 2024-07-16T09:52:58.724 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:58.718+0000 7ff8c9bef640 1 -- 172.21.15.156:0/3681673145 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8c41126e0 con 0x7ff8c40fb260 2024-07-16T09:52:58.724 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:58.718+0000 7ff8c9bef640 1 -- 172.21.15.156:0/3681673145 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8c4112bf0 con 0x7ff8c40fb260 2024-07-16T09:52:58.724 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:58.719+0000 7ff8c9bef640 1 -- 172.21.15.156:0/3681673145 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8c40fa6c0 con 0x7ff8c40fb260 2024-07-16T09:52:58.724 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:58.720+0000 7ff8abfff640 1 -- 172.21.15.156:0/3681673145 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 19) v1 ==== 85127+0+0 (secure 0 0 0) 0x7ff8b4030020 con 0x7ff8c40fb260 2024-07-16T09:52:58.724 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:58.720+0000 7ff8abfff640 1 --2- 172.21.15.156:0/3681673145 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7ff8ac067190 0x7ff8ac069640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:52:58.725 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:58.720+0000 7ff8abfff640 1 -- 172.21.15.156:0/3681673145 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5464+0+0 (secure 0 0 0) 0x7ff8b40a4950 con 0x7ff8c40fb260 2024-07-16T09:52:58.725 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:58.720+0000 7ff8c2ffd640 1 --2- 172.21.15.156:0/3681673145 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7ff8ac067190 0x7ff8ac069640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:52:58.725 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:58.721+0000 7ff8c2ffd640 1 --2- 172.21.15.156:0/3681673145 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7ff8ac067190 0x7ff8ac069640 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7ff8b8004d60 tx=0x7ff8b800b040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:52:58.725 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:58.724+0000 7ff8abfff640 1 -- 172.21.15.156:0/3681673145 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7ff8b4071c70 con 0x7ff8c40fb260 2024-07-16T09:52:58.947 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:58.945+0000 7ff8c9bef640 1 -- 172.21.15.156:0/3681673145 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7ff8c4112f50 con 0x7ff8c40fb260 2024-07-16T09:52:58.947 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:58.945+0000 7ff8abfff640 1 -- 172.21.15.156:0/3681673145 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 2 v2) v1 ==== 75+0+1076 (secure 0 0 0) 0x7ff8b4041450 con 0x7ff8c40fb260 2024-07-16T09:52:58.947 INFO:teuthology.orchestra.run.smithi156.stdout:e2 2024-07-16T09:52:58.947 INFO:teuthology.orchestra.run.smithi156.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-07-16T09:52:58.947 INFO:teuthology.orchestra.run.smithi156.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-07-16T09:52:58.947 INFO:teuthology.orchestra.run.smithi156.stdout:legacy client fscid: 1 2024-07-16T09:52:58.947 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-07-16T09:52:58.947 INFO:teuthology.orchestra.run.smithi156.stdout:Filesystem 'cephfs' (1) 2024-07-16T09:52:58.947 INFO:teuthology.orchestra.run.smithi156.stdout:fs_name cephfs 2024-07-16T09:52:58.948 INFO:teuthology.orchestra.run.smithi156.stdout:epoch 2 2024-07-16T09:52:58.948 INFO:teuthology.orchestra.run.smithi156.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-07-16T09:52:58.948 INFO:teuthology.orchestra.run.smithi156.stdout:created 2024-07-16T09:52:55.214975+0000 2024-07-16T09:52:58.948 INFO:teuthology.orchestra.run.smithi156.stdout:modified 2024-07-16T09:52:55.215027+0000 2024-07-16T09:52:58.948 INFO:teuthology.orchestra.run.smithi156.stdout:tableserver 0 2024-07-16T09:52:58.948 INFO:teuthology.orchestra.run.smithi156.stdout:root 0 2024-07-16T09:52:58.948 INFO:teuthology.orchestra.run.smithi156.stdout:session_timeout 60 2024-07-16T09:52:58.948 INFO:teuthology.orchestra.run.smithi156.stdout:session_autoclose 300 2024-07-16T09:52:58.948 INFO:teuthology.orchestra.run.smithi156.stdout:max_file_size 1099511627776 2024-07-16T09:52:58.948 INFO:teuthology.orchestra.run.smithi156.stdout:required_client_features {} 2024-07-16T09:52:58.948 INFO:teuthology.orchestra.run.smithi156.stdout:last_failure 0 2024-07-16T09:52:58.948 INFO:teuthology.orchestra.run.smithi156.stdout:last_failure_osd_epoch 0 2024-07-16T09:52:58.949 INFO:teuthology.orchestra.run.smithi156.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-07-16T09:52:58.949 INFO:teuthology.orchestra.run.smithi156.stdout:max_mds 1 2024-07-16T09:52:58.949 INFO:teuthology.orchestra.run.smithi156.stdout:in 2024-07-16T09:52:58.949 INFO:teuthology.orchestra.run.smithi156.stdout:up {} 2024-07-16T09:52:58.949 INFO:teuthology.orchestra.run.smithi156.stdout:failed 2024-07-16T09:52:58.949 INFO:teuthology.orchestra.run.smithi156.stdout:damaged 2024-07-16T09:52:58.949 INFO:teuthology.orchestra.run.smithi156.stdout:stopped 2024-07-16T09:52:58.949 INFO:teuthology.orchestra.run.smithi156.stdout:data_pools [3] 2024-07-16T09:52:58.949 INFO:teuthology.orchestra.run.smithi156.stdout:metadata_pool 2 2024-07-16T09:52:58.949 INFO:teuthology.orchestra.run.smithi156.stdout:inline_data disabled 2024-07-16T09:52:58.949 INFO:teuthology.orchestra.run.smithi156.stdout:balancer 2024-07-16T09:52:58.949 INFO:teuthology.orchestra.run.smithi156.stdout:standby_count_wanted 0 2024-07-16T09:52:58.949 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-07-16T09:52:58.950 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-07-16T09:52:58.950 INFO:teuthology.orchestra.run.smithi156.stderr:dumped fsmap epoch 2 2024-07-16T09:52:58.950 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:58.947+0000 7ff8c9bef640 1 -- 172.21.15.156:0/3681673145 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7ff8ac067190 msgr2=0x7ff8ac069640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:52:58.950 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:58.947+0000 7ff8c9bef640 1 --2- 172.21.15.156:0/3681673145 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7ff8ac067190 0x7ff8ac069640 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7ff8b8004d60 tx=0x7ff8b800b040 comp rx=0 tx=0).stop 2024-07-16T09:52:58.950 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:58.947+0000 7ff8c9bef640 1 -- 172.21.15.156:0/3681673145 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff8c40fb260 msgr2=0x7ff8c410fc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:52:58.951 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:58.947+0000 7ff8c9bef640 1 --2- 172.21.15.156:0/3681673145 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff8c40fb260 0x7ff8c410fc50 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7ff8b4009f80 tx=0x7ff8b4009da0 comp rx=0 tx=0).stop 2024-07-16T09:52:58.951 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:58.947+0000 7ff8c9bef640 1 -- 172.21.15.156:0/3681673145 shutdown_connections 2024-07-16T09:52:58.951 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:58.947+0000 7ff8c9bef640 1 --2- 172.21.15.156:0/3681673145 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7ff8ac067190 0x7ff8ac069640 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:52:58.951 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:58.947+0000 7ff8c9bef640 1 --2- 172.21.15.156:0/3681673145 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff8c40fc530 0x7ff8c41121a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:52:58.951 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:58.947+0000 7ff8c9bef640 1 --2- 172.21.15.156:0/3681673145 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff8c40fb260 0x7ff8c410fc50 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:52:58.951 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:58.947+0000 7ff8c9bef640 1 -- 172.21.15.156:0/3681673145 >> 172.21.15.156:0/3681673145 conn(0x7ff8c40f6a40 msgr2=0x7ff8c40f80a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:52:58.951 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:58.948+0000 7ff8c9bef640 1 -- 172.21.15.156:0/3681673145 shutdown_connections 2024-07-16T09:52:58.951 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:52:58.948+0000 7ff8c9bef640 1 -- 172.21.15.156:0/3681673145 wait complete. 2024-07-16T09:52:59.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:52:59 smithi176 ceph-mon[36626]: from='client.? 172.21.15.156:0/3681673145' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-07-16T09:52:59.630 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:52:59 smithi156 ceph-mon[29302]: from='client.? 172.21.15.156:0/3681673145' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-07-16T09:52:59.935 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-07-16T09:52:59.945 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi156.front.sepia.ceph.com 2024-07-16T09:52:59.946 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 26c91c26-4358-11ef-bcad-c7b262605968 -- bash -c 'ceph fs set cephfs max_mds 2' 2024-07-16T09:53:00.205 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/26c91c26-4358-11ef-bcad-c7b262605968/mon.smithi156/config 2024-07-16T09:53:00.658 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:53:00 smithi156 ceph-mon[29302]: pgmap v171: 65 pgs: 18 creating+peering, 38 active+clean, 9 unknown; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-07-16T09:53:00.659 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:53:00 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:53:00.659 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:53:00 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:53:00.659 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:53:00 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:53:00.659 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:53:00 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi176.dvvbud", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-16T09:53:00.659 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:53:00 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi176.dvvbud", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-07-16T09:53:00.659 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:53:00 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-16T09:53:00.659 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:53:00 smithi156 ceph-mon[29302]: Deploying daemon mds.cephfs.smithi176.dvvbud on smithi176 2024-07-16T09:53:00.659 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:53:00 smithi156 ceph-mon[29302]: mds.? [v2:172.21.15.156:6826/1080176013,v1:172.21.15.156:6827/1080176013] up:boot 2024-07-16T09:53:00.659 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:53:00 smithi156 ceph-mon[29302]: daemon mds.cephfs.smithi156.rdibys assigned to filesystem cephfs as rank 0 (now has 1 ranks) 2024-07-16T09:53:00.659 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:53:00 smithi156 ceph-mon[29302]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-07-16T09:53:00.659 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:53:00 smithi156 ceph-mon[29302]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-07-16T09:53:00.660 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:53:00 smithi156 ceph-mon[29302]: Cluster is now healthy 2024-07-16T09:53:00.660 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:53:00 smithi156 ceph-mon[29302]: fsmap cephfs:0 1 up:standby 2024-07-16T09:53:00.660 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:53:00 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi156.rdibys"}]: dispatch 2024-07-16T09:53:00.660 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:53:00 smithi156 ceph-mon[29302]: fsmap cephfs:1 {0=cephfs.smithi156.rdibys=up:creating} 2024-07-16T09:53:00.660 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:53:00 smithi156 ceph-mon[29302]: daemon mds.cephfs.smithi156.rdibys is now active in filesystem cephfs as rank 0 2024-07-16T09:53:00.747 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:53:00 smithi176 ceph-mon[36626]: pgmap v171: 65 pgs: 18 creating+peering, 38 active+clean, 9 unknown; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-07-16T09:53:00.747 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:53:00 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:53:00.747 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:53:00 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:53:00.747 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:53:00 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:53:00.747 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:53:00 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi176.dvvbud", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-16T09:53:00.747 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:53:00 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi176.dvvbud", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-07-16T09:53:00.747 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:53:00 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-16T09:53:00.748 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:53:00 smithi176 ceph-mon[36626]: Deploying daemon mds.cephfs.smithi176.dvvbud on smithi176 2024-07-16T09:53:00.748 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:53:00 smithi176 ceph-mon[36626]: mds.? [v2:172.21.15.156:6826/1080176013,v1:172.21.15.156:6827/1080176013] up:boot 2024-07-16T09:53:00.748 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:53:00 smithi176 ceph-mon[36626]: daemon mds.cephfs.smithi156.rdibys assigned to filesystem cephfs as rank 0 (now has 1 ranks) 2024-07-16T09:53:00.748 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:53:00 smithi176 ceph-mon[36626]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-07-16T09:53:00.748 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:53:00 smithi176 ceph-mon[36626]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-07-16T09:53:00.748 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:53:00 smithi176 ceph-mon[36626]: Cluster is now healthy 2024-07-16T09:53:00.748 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:53:00 smithi176 ceph-mon[36626]: fsmap cephfs:0 1 up:standby 2024-07-16T09:53:00.748 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:53:00 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi156.rdibys"}]: dispatch 2024-07-16T09:53:00.748 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:53:00 smithi176 ceph-mon[36626]: fsmap cephfs:1 {0=cephfs.smithi156.rdibys=up:creating} 2024-07-16T09:53:00.748 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:53:00 smithi176 ceph-mon[36626]: daemon mds.cephfs.smithi156.rdibys is now active in filesystem cephfs as rank 0 2024-07-16T09:53:01.861 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:01.859+0000 7f3bb9065640 1 -- 172.21.15.156:0/2455435969 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3bb40fb240 msgr2=0x7f3bb40fb650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:53:01.861 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:01.859+0000 7f3bb9065640 1 --2- 172.21.15.156:0/2455435969 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3bb40fb240 0x7f3bb40fb650 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f3ba40048d0 tx=0x7f3ba4009da0 comp rx=0 tx=0).stop 2024-07-16T09:53:01.862 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:01.859+0000 7f3bb9065640 1 -- 172.21.15.156:0/2455435969 shutdown_connections 2024-07-16T09:53:01.862 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:01.859+0000 7f3bb9065640 1 --2- 172.21.15.156:0/2455435969 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3bb40fc510 0x7f3bb40fc980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:53:01.862 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:01.859+0000 7f3bb9065640 1 --2- 172.21.15.156:0/2455435969 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3bb40fb240 0x7f3bb40fb650 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:53:01.862 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:01.859+0000 7f3bb9065640 1 -- 172.21.15.156:0/2455435969 >> 172.21.15.156:0/2455435969 conn(0x7f3bb40f6a20 msgr2=0x7f3bb40f8e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:53:01.862 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:01.859+0000 7f3bb9065640 1 -- 172.21.15.156:0/2455435969 shutdown_connections 2024-07-16T09:53:01.862 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:01.859+0000 7f3bb9065640 1 -- 172.21.15.156:0/2455435969 wait complete. 2024-07-16T09:53:01.862 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:01.860+0000 7f3bb9065640 1 Processor -- start 2024-07-16T09:53:01.862 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:01.860+0000 7f3bb9065640 1 -- start start 2024-07-16T09:53:01.862 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:01.860+0000 7f3bb9065640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3bb40fb240 0x7f3bb410cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:53:01.862 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:01.860+0000 7f3bb9065640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3bb40fc510 0x7f3bb410d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:53:01.862 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:01.860+0000 7f3bb9065640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bb410e380 con 0x7f3bb40fc510 2024-07-16T09:53:01.862 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:01.860+0000 7f3bb9065640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bb410e4f0 con 0x7f3bb40fb240 2024-07-16T09:53:01.862 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:01.860+0000 7f3babfff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3bb40fc510 0x7f3bb410d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:53:01.863 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:01.860+0000 7f3babfff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3bb40fc510 0x7f3bb410d510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:34900/0 (socket says 172.21.15.156:34900) 2024-07-16T09:53:01.863 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:01.860+0000 7f3babfff640 1 -- 172.21.15.156:0/1114394581 learned_addr learned my addr 172.21.15.156:0/1114394581 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T09:53:01.863 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:01.860+0000 7f3bb2d9d640 1 --2- 172.21.15.156:0/1114394581 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3bb40fb240 0x7f3bb410cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:53:01.863 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:01.860+0000 7f3babfff640 1 -- 172.21.15.156:0/1114394581 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3bb40fb240 msgr2=0x7f3bb410cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:53:01.863 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:01.860+0000 7f3babfff640 1 --2- 172.21.15.156:0/1114394581 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3bb40fb240 0x7f3bb410cfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:53:01.863 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:01.860+0000 7f3babfff640 1 -- 172.21.15.156:0/1114394581 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ba4005040 con 0x7f3bb40fc510 2024-07-16T09:53:01.864 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:01.860+0000 7f3babfff640 1 --2- 172.21.15.156:0/1114394581 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3bb40fc510 0x7f3bb410d510 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f3b9c00b790 tx=0x7f3b9c0091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:53:01.864 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:01.861+0000 7f3bab7fe640 1 -- 172.21.15.156:0/1114394581 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3b9c00c700 con 0x7f3bb40fc510 2024-07-16T09:53:01.864 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:01.861+0000 7f3bb9065640 1 -- 172.21.15.156:0/1114394581 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3bb410e7d0 con 0x7f3bb40fc510 2024-07-16T09:53:01.864 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:01.861+0000 7f3bab7fe640 1 -- 172.21.15.156:0/1114394581 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3b9c00cd30 con 0x7f3bb40fc510 2024-07-16T09:53:01.864 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:01.861+0000 7f3bab7fe640 1 -- 172.21.15.156:0/1114394581 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3b9c013370 con 0x7f3bb40fc510 2024-07-16T09:53:01.864 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:01.861+0000 7f3bb9065640 1 -- 172.21.15.156:0/1114394581 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3bb410ecf0 con 0x7f3bb40fc510 2024-07-16T09:53:01.864 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:01.862+0000 7f3bb9065640 1 -- 172.21.15.156:0/1114394581 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3bb40fa6a0 con 0x7f3bb40fc510 2024-07-16T09:53:01.864 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:01.863+0000 7f3bab7fe640 1 -- 172.21.15.156:0/1114394581 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 19) v1 ==== 85127+0+0 (secure 0 0 0) 0x7f3b9c012070 con 0x7f3bb40fc510 2024-07-16T09:53:01.865 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:01.863+0000 7f3bab7fe640 1 --2- 172.21.15.156:0/1114394581 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f3ba0067190 0x7f3ba0069640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:53:01.865 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:01.863+0000 7f3bab7fe640 1 -- 172.21.15.156:0/1114394581 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f3b9c084380 con 0x7f3bb40fc510 2024-07-16T09:53:01.865 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:01.863+0000 7f3bb2d9d640 1 --2- 172.21.15.156:0/1114394581 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f3ba0067190 0x7f3ba0069640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:53:01.865 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:01.864+0000 7f3bb2d9d640 1 --2- 172.21.15.156:0/1114394581 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f3ba0067190 0x7f3ba0069640 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f3ba4000c00 tx=0x7f3ba4035040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:53:01.868 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:01.867+0000 7f3bab7fe640 1 -- 172.21.15.156:0/1114394581 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f3b9c0516a0 con 0x7f3bb40fc510 2024-07-16T09:53:02.080 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:02.079+0000 7f3bb9065640 1 -- 172.21.15.156:0/1114394581 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"} v 0) v1 -- 0x7f3bb4103e10 con 0x7f3bb40fc510 2024-07-16T09:53:02.346 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:53:02 smithi156 ceph-mon[29302]: pgmap v172: 65 pgs: 65 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.3 KiB/s wr, 1 op/s 2024-07-16T09:53:02.346 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:53:02 smithi156 ceph-mon[29302]: mds.? [v2:172.21.15.156:6826/1080176013,v1:172.21.15.156:6827/1080176013] up:active 2024-07-16T09:53:02.346 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:53:02 smithi156 ceph-mon[29302]: fsmap cephfs:1 {0=cephfs.smithi156.rdibys=up:active} 2024-07-16T09:53:02.346 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:53:02 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:53:02.346 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:53:02 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:53:02.346 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:53:02 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:53:02.346 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:53:02 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi156.evwwaz", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-16T09:53:02.346 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:53:02 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi156.evwwaz", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-07-16T09:53:02.346 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:53:02 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-16T09:53:02.346 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:53:02 smithi156 ceph-mon[29302]: from='client.? 172.21.15.156:0/1114394581' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"}]: dispatch 2024-07-16T09:53:02.369 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:02.368+0000 7f3bab7fe640 1 -- 172.21.15.156:0/1114394581 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"}]=0 v6) v1 ==== 105+0+0 (secure 0 0 0) 0x7f3b9c054cc0 con 0x7f3bb40fc510 2024-07-16T09:53:02.372 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:02.370+0000 7f3bb9065640 1 -- 172.21.15.156:0/1114394581 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f3ba0067190 msgr2=0x7f3ba0069640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:53:02.372 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:02.370+0000 7f3bb9065640 1 --2- 172.21.15.156:0/1114394581 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f3ba0067190 0x7f3ba0069640 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f3ba4000c00 tx=0x7f3ba4035040 comp rx=0 tx=0).stop 2024-07-16T09:53:02.372 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:02.370+0000 7f3bb9065640 1 -- 172.21.15.156:0/1114394581 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3bb40fc510 msgr2=0x7f3bb410d510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:53:02.372 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:02.370+0000 7f3bb9065640 1 --2- 172.21.15.156:0/1114394581 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3bb40fc510 0x7f3bb410d510 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f3b9c00b790 tx=0x7f3b9c0091d0 comp rx=0 tx=0).stop 2024-07-16T09:53:02.372 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:02.371+0000 7f3bb9065640 1 -- 172.21.15.156:0/1114394581 shutdown_connections 2024-07-16T09:53:02.372 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:02.371+0000 7f3bb9065640 1 --2- 172.21.15.156:0/1114394581 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f3ba0067190 0x7f3ba0069640 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:53:02.373 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:02.371+0000 7f3bb9065640 1 --2- 172.21.15.156:0/1114394581 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3bb40fc510 0x7f3bb410d510 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:53:02.373 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:02.371+0000 7f3bb9065640 1 --2- 172.21.15.156:0/1114394581 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3bb40fb240 0x7f3bb410cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:53:02.373 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:02.371+0000 7f3bb9065640 1 -- 172.21.15.156:0/1114394581 >> 172.21.15.156:0/1114394581 conn(0x7f3bb40f6a20 msgr2=0x7f3bb40f80d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:53:02.373 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:02.371+0000 7f3bb9065640 1 -- 172.21.15.156:0/1114394581 shutdown_connections 2024-07-16T09:53:02.373 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:02.371+0000 7f3bb9065640 1 -- 172.21.15.156:0/1114394581 wait complete. 2024-07-16T09:53:02.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:53:02 smithi176 ceph-mon[36626]: pgmap v172: 65 pgs: 65 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.3 KiB/s wr, 1 op/s 2024-07-16T09:53:02.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:53:02 smithi176 ceph-mon[36626]: mds.? [v2:172.21.15.156:6826/1080176013,v1:172.21.15.156:6827/1080176013] up:active 2024-07-16T09:53:02.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:53:02 smithi176 ceph-mon[36626]: fsmap cephfs:1 {0=cephfs.smithi156.rdibys=up:active} 2024-07-16T09:53:02.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:53:02 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:53:02.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:53:02 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:53:02.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:53:02 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:53:02.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:53:02 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi156.evwwaz", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-16T09:53:02.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:53:02 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi156.evwwaz", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-07-16T09:53:02.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:53:02 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-16T09:53:02.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:53:02 smithi176 ceph-mon[36626]: from='client.? 172.21.15.156:0/1114394581' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"}]: dispatch 2024-07-16T09:53:03.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:53:03 smithi156 ceph-mon[29302]: Deploying daemon mds.cephfs.smithi156.evwwaz on smithi156 2024-07-16T09:53:03.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:53:03 smithi156 ceph-mon[29302]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-07-16T09:53:03.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:53:03 smithi156 ceph-mon[29302]: mds.? [v2:172.21.15.176:6824/1577019287,v1:172.21.15.176:6825/1577019287] up:boot 2024-07-16T09:53:03.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:53:03 smithi156 ceph-mon[29302]: from='client.? 172.21.15.156:0/1114394581' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"}]': finished 2024-07-16T09:53:03.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:53:03 smithi156 ceph-mon[29302]: daemon mds.cephfs.smithi176.dvvbud assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-07-16T09:53:03.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:53:03 smithi156 ceph-mon[29302]: Health check failed: insufficient standby MDS daemons available (MDS_INSUFFICIENT_STANDBY) 2024-07-16T09:53:03.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:53:03 smithi156 ceph-mon[29302]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-07-16T09:53:03.426 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:53:03 smithi156 ceph-mon[29302]: fsmap cephfs:1 {0=cephfs.smithi156.rdibys=up:active} 1 up:standby 2024-07-16T09:53:03.426 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:53:03 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi176.dvvbud"}]: dispatch 2024-07-16T09:53:03.426 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:53:03 smithi156 ceph-mon[29302]: fsmap cephfs:2 {0=cephfs.smithi156.rdibys=up:active,1=cephfs.smithi176.dvvbud=up:creating} 2024-07-16T09:53:03.426 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:53:03 smithi156 ceph-mon[29302]: daemon mds.cephfs.smithi176.dvvbud is now active in filesystem cephfs as rank 1 2024-07-16T09:53:03.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:53:03 smithi176 ceph-mon[36626]: Deploying daemon mds.cephfs.smithi156.evwwaz on smithi156 2024-07-16T09:53:03.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:53:03 smithi176 ceph-mon[36626]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-07-16T09:53:03.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:53:03 smithi176 ceph-mon[36626]: mds.? [v2:172.21.15.176:6824/1577019287,v1:172.21.15.176:6825/1577019287] up:boot 2024-07-16T09:53:03.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:53:03 smithi176 ceph-mon[36626]: from='client.? 172.21.15.156:0/1114394581' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"}]': finished 2024-07-16T09:53:03.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:53:03 smithi176 ceph-mon[36626]: daemon mds.cephfs.smithi176.dvvbud assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-07-16T09:53:03.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:53:03 smithi176 ceph-mon[36626]: Health check failed: insufficient standby MDS daemons available (MDS_INSUFFICIENT_STANDBY) 2024-07-16T09:53:03.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:53:03 smithi176 ceph-mon[36626]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-07-16T09:53:03.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:53:03 smithi176 ceph-mon[36626]: fsmap cephfs:1 {0=cephfs.smithi156.rdibys=up:active} 1 up:standby 2024-07-16T09:53:03.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:53:03 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi176.dvvbud"}]: dispatch 2024-07-16T09:53:03.595 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:53:03 smithi176 ceph-mon[36626]: fsmap cephfs:2 {0=cephfs.smithi156.rdibys=up:active,1=cephfs.smithi176.dvvbud=up:creating} 2024-07-16T09:53:03.595 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:53:03 smithi176 ceph-mon[36626]: daemon mds.cephfs.smithi176.dvvbud is now active in filesystem cephfs as rank 1 2024-07-16T09:53:03.688 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-07-16T09:53:03.698 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi156.front.sepia.ceph.com 2024-07-16T09:53:03.698 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 26c91c26-4358-11ef-bcad-c7b262605968 -- bash -c 'ceph fs set cephfs allow_standby_replay true' 2024-07-16T09:53:04.291 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/26c91c26-4358-11ef-bcad-c7b262605968/mon.smithi156/config 2024-07-16T09:53:04.416 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:53:04 smithi156 ceph-mon[29302]: pgmap v173: 65 pgs: 65 active+clean; 578 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s wr, 4 op/s 2024-07-16T09:53:04.416 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:53:04 smithi156 ceph-mon[29302]: mds.? [v2:172.21.15.176:6824/1577019287,v1:172.21.15.176:6825/1577019287] up:active 2024-07-16T09:53:04.416 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:53:04 smithi156 ceph-mon[29302]: fsmap cephfs:2 {0=cephfs.smithi156.rdibys=up:active,1=cephfs.smithi176.dvvbud=up:active} 2024-07-16T09:53:04.416 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:53:04 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:53:04.416 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:53:04 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:53:04.416 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:53:04 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:53:04.417 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:53:04 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi176.wahbsi", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-16T09:53:04.417 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:53:04 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi176.wahbsi", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-07-16T09:53:04.417 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:53:04 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-16T09:53:04.742 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:53:04 smithi176 ceph-mon[36626]: pgmap v173: 65 pgs: 65 active+clean; 578 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s wr, 4 op/s 2024-07-16T09:53:04.742 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:53:04 smithi176 ceph-mon[36626]: mds.? [v2:172.21.15.176:6824/1577019287,v1:172.21.15.176:6825/1577019287] up:active 2024-07-16T09:53:04.743 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:53:04 smithi176 ceph-mon[36626]: fsmap cephfs:2 {0=cephfs.smithi156.rdibys=up:active,1=cephfs.smithi176.dvvbud=up:active} 2024-07-16T09:53:04.743 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:53:04 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:53:04.743 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:53:04 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:53:04.743 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:53:04 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:53:04.743 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:53:04 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi176.wahbsi", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-16T09:53:04.743 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:53:04 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi176.wahbsi", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-07-16T09:53:04.743 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:53:04 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-16T09:53:05.525 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:53:05 smithi156 ceph-mon[29302]: Deploying daemon mds.cephfs.smithi176.wahbsi on smithi176 2024-07-16T09:53:05.525 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:53:05 smithi156 ceph-mon[29302]: Health check cleared: MDS_INSUFFICIENT_STANDBY (was: insufficient standby MDS daemons available) 2024-07-16T09:53:05.525 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:53:05 smithi156 ceph-mon[29302]: Cluster is now healthy 2024-07-16T09:53:05.525 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:53:05 smithi156 ceph-mon[29302]: mds.? [v2:172.21.15.156:6828/560329502,v1:172.21.15.156:6829/560329502] up:boot 2024-07-16T09:53:05.525 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:53:05 smithi156 ceph-mon[29302]: fsmap cephfs:2 {0=cephfs.smithi156.rdibys=up:active,1=cephfs.smithi176.dvvbud=up:active} 1 up:standby 2024-07-16T09:53:05.525 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:53:05 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi156.evwwaz"}]: dispatch 2024-07-16T09:53:05.748 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:53:05 smithi176 ceph-mon[36626]: Deploying daemon mds.cephfs.smithi176.wahbsi on smithi176 2024-07-16T09:53:05.748 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:53:05 smithi176 ceph-mon[36626]: Health check cleared: MDS_INSUFFICIENT_STANDBY (was: insufficient standby MDS daemons available) 2024-07-16T09:53:05.748 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:53:05 smithi176 ceph-mon[36626]: Cluster is now healthy 2024-07-16T09:53:05.748 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:53:05 smithi176 ceph-mon[36626]: mds.? [v2:172.21.15.156:6828/560329502,v1:172.21.15.156:6829/560329502] up:boot 2024-07-16T09:53:05.748 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:53:05 smithi176 ceph-mon[36626]: fsmap cephfs:2 {0=cephfs.smithi156.rdibys=up:active,1=cephfs.smithi176.dvvbud=up:active} 1 up:standby 2024-07-16T09:53:05.748 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:53:05 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi156.evwwaz"}]: dispatch 2024-07-16T09:53:05.977 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:05.974+0000 7fb8edb9f640 1 -- 172.21.15.156:0/868430511 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb8e80fb240 msgr2=0x7fb8e80fb650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:53:05.977 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:05.974+0000 7fb8edb9f640 1 --2- 172.21.15.156:0/868430511 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb8e80fb240 0x7fb8e80fb650 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fb8d80048d0 tx=0x7fb8d8009da0 comp rx=0 tx=0).stop 2024-07-16T09:53:05.977 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:05.974+0000 7fb8edb9f640 1 -- 172.21.15.156:0/868430511 shutdown_connections 2024-07-16T09:53:05.977 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:05.974+0000 7fb8edb9f640 1 --2- 172.21.15.156:0/868430511 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb8e80fc510 0x7fb8e80fc980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:53:05.977 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:05.974+0000 7fb8edb9f640 1 --2- 172.21.15.156:0/868430511 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb8e80fb240 0x7fb8e80fb650 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:53:05.977 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:05.974+0000 7fb8edb9f640 1 -- 172.21.15.156:0/868430511 >> 172.21.15.156:0/868430511 conn(0x7fb8e80f6a20 msgr2=0x7fb8e80f8e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:53:05.977 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:05.974+0000 7fb8edb9f640 1 -- 172.21.15.156:0/868430511 shutdown_connections 2024-07-16T09:53:05.977 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:05.974+0000 7fb8edb9f640 1 -- 172.21.15.156:0/868430511 wait complete. 2024-07-16T09:53:05.977 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:05.974+0000 7fb8edb9f640 1 Processor -- start 2024-07-16T09:53:05.977 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:05.974+0000 7fb8edb9f640 1 -- start start 2024-07-16T09:53:05.978 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:05.975+0000 7fb8edb9f640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb8e80fb240 0x7fb8e810fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:53:05.978 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:05.975+0000 7fb8edb9f640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb8e80fc510 0x7fb8e8112250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:53:05.978 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:05.975+0000 7fb8edb9f640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8e8065b00 con 0x7fb8e80fc510 2024-07-16T09:53:05.978 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:05.975+0000 7fb8edb9f640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8e8065c70 con 0x7fb8e80fb240 2024-07-16T09:53:05.978 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:05.975+0000 7fb8dffff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb8e80fc510 0x7fb8e8112250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:53:05.978 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:05.975+0000 7fb8dffff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb8e80fc510 0x7fb8e8112250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:60586/0 (socket says 172.21.15.156:60586) 2024-07-16T09:53:05.978 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:05.975+0000 7fb8dffff640 1 -- 172.21.15.156:0/1147873859 learned_addr learned my addr 172.21.15.156:0/1147873859 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T09:53:05.978 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:05.975+0000 7fb8e77fe640 1 --2- 172.21.15.156:0/1147873859 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb8e80fb240 0x7fb8e810fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:53:05.978 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:05.975+0000 7fb8dffff640 1 -- 172.21.15.156:0/1147873859 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb8e80fb240 msgr2=0x7fb8e810fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:53:05.978 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:05.975+0000 7fb8dffff640 1 --2- 172.21.15.156:0/1147873859 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb8e80fb240 0x7fb8e810fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:53:05.978 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:05.975+0000 7fb8dffff640 1 -- 172.21.15.156:0/1147873859 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8d8005040 con 0x7fb8e80fc510 2024-07-16T09:53:05.979 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:05.975+0000 7fb8dffff640 1 --2- 172.21.15.156:0/1147873859 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb8e80fc510 0x7fb8e8112250 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fb8d000b790 tx=0x7fb8d00091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:53:05.979 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:05.975+0000 7fb8e4ff9640 1 -- 172.21.15.156:0/1147873859 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb8d000c700 con 0x7fb8e80fc510 2024-07-16T09:53:05.979 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:05.975+0000 7fb8e4ff9640 1 -- 172.21.15.156:0/1147873859 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb8d000cd30 con 0x7fb8e80fc510 2024-07-16T09:53:05.979 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:05.975+0000 7fb8e4ff9640 1 -- 172.21.15.156:0/1147873859 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb8d0013370 con 0x7fb8e80fc510 2024-07-16T09:53:05.979 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:05.975+0000 7fb8edb9f640 1 -- 172.21.15.156:0/1147873859 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8e8112850 con 0x7fb8e80fc510 2024-07-16T09:53:05.979 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:05.975+0000 7fb8edb9f640 1 -- 172.21.15.156:0/1147873859 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8e8112d40 con 0x7fb8e80fc510 2024-07-16T09:53:05.980 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:05.976+0000 7fb8edb9f640 1 -- 172.21.15.156:0/1147873859 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8e80fa6a0 con 0x7fb8e80fc510 2024-07-16T09:53:05.980 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:05.977+0000 7fb8e4ff9640 1 -- 172.21.15.156:0/1147873859 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 19) v1 ==== 85127+0+0 (secure 0 0 0) 0x7fb8d0012070 con 0x7fb8e80fc510 2024-07-16T09:53:05.980 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:05.977+0000 7fb8e4ff9640 1 --2- 172.21.15.156:0/1147873859 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7fb8d4067190 0x7fb8d4069640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:53:05.980 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:05.977+0000 7fb8e4ff9640 1 -- 172.21.15.156:0/1147873859 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5464+0+0 (secure 0 0 0) 0x7fb8d0084450 con 0x7fb8e80fc510 2024-07-16T09:53:05.980 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:05.977+0000 7fb8e77fe640 1 --2- 172.21.15.156:0/1147873859 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7fb8d4067190 0x7fb8d4069640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:53:05.980 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:05.978+0000 7fb8e77fe640 1 --2- 172.21.15.156:0/1147873859 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7fb8d4067190 0x7fb8d4069640 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fb8d8000c00 tx=0x7fb8d8035040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:53:05.982 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:05.981+0000 7fb8e4ff9640 1 -- 172.21.15.156:0/1147873859 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fb8d00516f0 con 0x7fb8e80fc510 2024-07-16T09:53:06.184 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:06.182+0000 7fb8edb9f640 1 -- 172.21.15.156:0/1147873859 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) v1 -- 0x7fb8e8103e10 con 0x7fb8e80fc510 2024-07-16T09:53:06.392 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:06.390+0000 7fb8e4ff9640 1 -- 172.21.15.156:0/1147873859 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v10) v1 ==== 121+0+0 (secure 0 0 0) 0x7fb8d0054d10 con 0x7fb8e80fc510 2024-07-16T09:53:06.394 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:06.392+0000 7fb8edb9f640 1 -- 172.21.15.156:0/1147873859 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7fb8d4067190 msgr2=0x7fb8d4069640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:53:06.394 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:06.392+0000 7fb8edb9f640 1 --2- 172.21.15.156:0/1147873859 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7fb8d4067190 0x7fb8d4069640 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fb8d8000c00 tx=0x7fb8d8035040 comp rx=0 tx=0).stop 2024-07-16T09:53:06.394 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:06.392+0000 7fb8edb9f640 1 -- 172.21.15.156:0/1147873859 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb8e80fc510 msgr2=0x7fb8e8112250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:53:06.394 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:06.392+0000 7fb8edb9f640 1 --2- 172.21.15.156:0/1147873859 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb8e80fc510 0x7fb8e8112250 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fb8d000b790 tx=0x7fb8d00091d0 comp rx=0 tx=0).stop 2024-07-16T09:53:06.394 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:06.392+0000 7fb8edb9f640 1 -- 172.21.15.156:0/1147873859 shutdown_connections 2024-07-16T09:53:06.395 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:06.392+0000 7fb8edb9f640 1 --2- 172.21.15.156:0/1147873859 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7fb8d4067190 0x7fb8d4069640 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:53:06.395 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:06.392+0000 7fb8edb9f640 1 --2- 172.21.15.156:0/1147873859 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb8e80fc510 0x7fb8e8112250 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:53:06.395 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:06.392+0000 7fb8edb9f640 1 --2- 172.21.15.156:0/1147873859 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb8e80fb240 0x7fb8e810fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:53:06.395 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:06.392+0000 7fb8edb9f640 1 -- 172.21.15.156:0/1147873859 >> 172.21.15.156:0/1147873859 conn(0x7fb8e80f6a20 msgr2=0x7fb8e80f80d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:53:06.395 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:06.393+0000 7fb8edb9f640 1 -- 172.21.15.156:0/1147873859 shutdown_connections 2024-07-16T09:53:06.395 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:06.393+0000 7fb8edb9f640 1 -- 172.21.15.156:0/1147873859 wait complete. 2024-07-16T09:53:06.499 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:53:06 smithi176 ceph-mon[36626]: pgmap v174: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.7 KiB/s wr, 8 op/s 2024-07-16T09:53:06.500 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:53:06 smithi176 ceph-mon[36626]: from='client.? 172.21.15.156:0/1147873859' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-07-16T09:53:06.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:53:06 smithi156 ceph-mon[29302]: pgmap v174: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.7 KiB/s wr, 8 op/s 2024-07-16T09:53:06.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:53:06 smithi156 ceph-mon[29302]: from='client.? 172.21.15.156:0/1147873859' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-07-16T09:53:07.118 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-07-16T09:53:07.128 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi156.front.sepia.ceph.com 2024-07-16T09:53:07.128 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 26c91c26-4358-11ef-bcad-c7b262605968 -- bash -c 'ceph fs set cephfs inline_data false' 2024-07-16T09:53:07.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:53:07 smithi156 ceph-mon[29302]: from='client.? 172.21.15.156:0/1147873859' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-07-16T09:53:07.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:53:07 smithi156 ceph-mon[29302]: fsmap cephfs:2 {0=cephfs.smithi156.rdibys=up:active,1=cephfs.smithi176.dvvbud=up:active} 1 up:standby 2024-07-16T09:53:07.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:53:07 smithi156 ceph-mon[29302]: fsmap cephfs:2 {0=cephfs.smithi156.rdibys=up:active,1=cephfs.smithi176.dvvbud=up:active} 1 up:standby-replay 2024-07-16T09:53:07.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:53:07 smithi156 ceph-mon[29302]: pgmap v175: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.4 KiB/s wr, 7 op/s 2024-07-16T09:53:07.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:53:07 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:53:07.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:53:07 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:53:07.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:53:07 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:53:07.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:53:07 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:53:07.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:53:07 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:53:07.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:53:07 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-16T09:53:07.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:53:07 smithi176 ceph-mon[36626]: from='client.? 172.21.15.156:0/1147873859' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-07-16T09:53:07.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:53:07 smithi176 ceph-mon[36626]: fsmap cephfs:2 {0=cephfs.smithi156.rdibys=up:active,1=cephfs.smithi176.dvvbud=up:active} 1 up:standby 2024-07-16T09:53:07.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:53:07 smithi176 ceph-mon[36626]: fsmap cephfs:2 {0=cephfs.smithi156.rdibys=up:active,1=cephfs.smithi176.dvvbud=up:active} 1 up:standby-replay 2024-07-16T09:53:07.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:53:07 smithi176 ceph-mon[36626]: pgmap v175: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.4 KiB/s wr, 7 op/s 2024-07-16T09:53:07.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:53:07 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:53:07.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:53:07 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:53:07.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:53:07 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:53:07.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:53:07 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:53:07.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:53:07 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:53:07.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:53:07 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-16T09:53:08.044 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/26c91c26-4358-11ef-bcad-c7b262605968/mon.smithi156/config 2024-07-16T09:53:08.501 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:53:08 smithi156 ceph-mon[29302]: mds.? [v2:172.21.15.176:6826/2196775412,v1:172.21.15.176:6827/2196775412] up:boot 2024-07-16T09:53:08.502 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:53:08 smithi156 ceph-mon[29302]: fsmap cephfs:2 {0=cephfs.smithi156.rdibys=up:active,1=cephfs.smithi176.dvvbud=up:active} 1 up:standby-replay 1 up:standby 2024-07-16T09:53:08.502 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:53:08 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi176.wahbsi"}]: dispatch 2024-07-16T09:53:08.502 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:53:08 smithi156 ceph-mon[29302]: fsmap cephfs:2 {0=cephfs.smithi156.rdibys=up:active,1=cephfs.smithi176.dvvbud=up:active} 2 up:standby-replay 2024-07-16T09:53:08.843 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:53:08 smithi176 ceph-mon[36626]: mds.? [v2:172.21.15.176:6826/2196775412,v1:172.21.15.176:6827/2196775412] up:boot 2024-07-16T09:53:08.843 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:53:08 smithi176 ceph-mon[36626]: fsmap cephfs:2 {0=cephfs.smithi156.rdibys=up:active,1=cephfs.smithi176.dvvbud=up:active} 1 up:standby-replay 1 up:standby 2024-07-16T09:53:08.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:53:08 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi176.wahbsi"}]: dispatch 2024-07-16T09:53:08.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:53:08 smithi176 ceph-mon[36626]: fsmap cephfs:2 {0=cephfs.smithi156.rdibys=up:active,1=cephfs.smithi176.dvvbud=up:active} 2 up:standby-replay 2024-07-16T09:53:09.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:53:09 smithi156 ceph-mon[29302]: mds.? [v2:172.21.15.156:6828/560329502,v1:172.21.15.156:6829/560329502] up:standby-replay 2024-07-16T09:53:09.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:53:09 smithi156 ceph-mon[29302]: mds.? [v2:172.21.15.156:6826/1080176013,v1:172.21.15.156:6827/1080176013] up:active 2024-07-16T09:53:09.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:53:09 smithi156 ceph-mon[29302]: fsmap cephfs:2 {0=cephfs.smithi156.rdibys=up:active,1=cephfs.smithi176.dvvbud=up:active} 2 up:standby-replay 2024-07-16T09:53:09.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:53:09 smithi156 ceph-mon[29302]: pgmap v176: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 709 B/s rd, 2.1 KiB/s wr, 8 op/s 2024-07-16T09:53:09.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:53:09 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:53:09.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:53:09 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:53:09.843 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:53:09 smithi176 ceph-mon[36626]: mds.? [v2:172.21.15.156:6828/560329502,v1:172.21.15.156:6829/560329502] up:standby-replay 2024-07-16T09:53:09.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:53:09 smithi176 ceph-mon[36626]: mds.? [v2:172.21.15.156:6826/1080176013,v1:172.21.15.156:6827/1080176013] up:active 2024-07-16T09:53:09.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:53:09 smithi176 ceph-mon[36626]: fsmap cephfs:2 {0=cephfs.smithi156.rdibys=up:active,1=cephfs.smithi176.dvvbud=up:active} 2 up:standby-replay 2024-07-16T09:53:09.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:53:09 smithi176 ceph-mon[36626]: pgmap v176: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 709 B/s rd, 2.1 KiB/s wr, 8 op/s 2024-07-16T09:53:09.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:53:09 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:53:09.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:53:09 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:53:10.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:53:10 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-16T09:53:10.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:53:10 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:53:10.843 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:53:10 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-16T09:53:10.843 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:53:10 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:53:11.012 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:11.001+0000 7fb4492e8640 1 -- 172.21.15.156:0/2557670443 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb4440ff570 msgr2=0x7fb444101950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:53:11.013 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:11.001+0000 7fb4492e8640 1 --2- 172.21.15.156:0/2557670443 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb4440ff570 0x7fb444101950 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fb42c0048d0 tx=0x7fb42c009da0 comp rx=0 tx=0).stop 2024-07-16T09:53:11.013 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:11.001+0000 7fb4492e8640 1 -- 172.21.15.156:0/2557670443 shutdown_connections 2024-07-16T09:53:11.013 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:11.001+0000 7fb4492e8640 1 --2- 172.21.15.156:0/2557670443 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb4440ff570 0x7fb444101950 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:53:11.013 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:11.001+0000 7fb4492e8640 1 --2- 172.21.15.156:0/2557670443 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb444066480 0x7fb4440ff030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:53:11.013 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:11.001+0000 7fb4492e8640 1 -- 172.21.15.156:0/2557670443 >> 172.21.15.156:0/2557670443 conn(0x7fb4440f6a20 msgr2=0x7fb4440f8e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:53:11.013 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:11.001+0000 7fb4492e8640 1 -- 172.21.15.156:0/2557670443 shutdown_connections 2024-07-16T09:53:11.013 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:11.001+0000 7fb4492e8640 1 -- 172.21.15.156:0/2557670443 wait complete. 2024-07-16T09:53:11.013 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:11.002+0000 7fb4492e8640 1 Processor -- start 2024-07-16T09:53:11.013 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:11.002+0000 7fb4492e8640 1 -- start start 2024-07-16T09:53:11.013 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:11.002+0000 7fb4492e8640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb444066480 0x7fb44410cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:53:11.013 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:11.002+0000 7fb4492e8640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb4440ff570 0x7fb44410d520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:53:11.013 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:11.002+0000 7fb4492e8640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb44410db70 con 0x7fb4440ff570 2024-07-16T09:53:11.013 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:11.002+0000 7fb4492e8640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb44410e390 con 0x7fb444066480 2024-07-16T09:53:11.014 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:11.003+0000 7fb43a7fc640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb4440ff570 0x7fb44410d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:53:11.014 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:11.003+0000 7fb43a7fc640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb4440ff570 0x7fb44410d520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:60604/0 (socket says 172.21.15.156:60604) 2024-07-16T09:53:11.014 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:11.003+0000 7fb43a7fc640 1 -- 172.21.15.156:0/4047818562 learned_addr learned my addr 172.21.15.156:0/4047818562 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T09:53:11.014 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:11.003+0000 7fb442ffd640 1 --2- 172.21.15.156:0/4047818562 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb444066480 0x7fb44410cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:53:11.014 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:11.003+0000 7fb43a7fc640 1 -- 172.21.15.156:0/4047818562 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb444066480 msgr2=0x7fb44410cfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:53:11.014 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:11.003+0000 7fb43a7fc640 1 --2- 172.21.15.156:0/4047818562 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb444066480 0x7fb44410cfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:53:11.014 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:11.003+0000 7fb43a7fc640 1 -- 172.21.15.156:0/4047818562 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb42c005040 con 0x7fb4440ff570 2024-07-16T09:53:11.014 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:11.003+0000 7fb43a7fc640 1 --2- 172.21.15.156:0/4047818562 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb4440ff570 0x7fb44410d520 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fb42c009f80 tx=0x7fb42c009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:53:11.014 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:11.003+0000 7fb43bfff640 1 -- 172.21.15.156:0/4047818562 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb42c038070 con 0x7fb4440ff570 2024-07-16T09:53:11.014 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:11.003+0000 7fb43bfff640 1 -- 172.21.15.156:0/4047818562 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb42c032970 con 0x7fb4440ff570 2024-07-16T09:53:11.015 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:11.003+0000 7fb4492e8640 1 -- 172.21.15.156:0/4047818562 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb44410e5c0 con 0x7fb4440ff570 2024-07-16T09:53:11.015 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:11.003+0000 7fb43bfff640 1 -- 172.21.15.156:0/4047818562 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb42c033860 con 0x7fb4440ff570 2024-07-16T09:53:11.015 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:11.003+0000 7fb4492e8640 1 -- 172.21.15.156:0/4047818562 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb44410eb00 con 0x7fb4440ff570 2024-07-16T09:53:11.015 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:11.004+0000 7fb4492e8640 1 -- 172.21.15.156:0/4047818562 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4440fa6c0 con 0x7fb4440ff570 2024-07-16T09:53:11.015 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:11.005+0000 7fb43bfff640 1 -- 172.21.15.156:0/4047818562 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 19) v1 ==== 85127+0+0 (secure 0 0 0) 0x7fb42c030020 con 0x7fb4440ff570 2024-07-16T09:53:11.015 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:11.005+0000 7fb43bfff640 1 --2- 172.21.15.156:0/4047818562 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7fb430067190 0x7fb430069640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:53:11.015 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:11.005+0000 7fb43bfff640 1 -- 172.21.15.156:0/4047818562 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5464+0+0 (secure 0 0 0) 0x7fb42c0a49c0 con 0x7fb4440ff570 2024-07-16T09:53:11.015 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:11.005+0000 7fb442ffd640 1 --2- 172.21.15.156:0/4047818562 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7fb430067190 0x7fb430069640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:53:11.015 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:11.006+0000 7fb442ffd640 1 --2- 172.21.15.156:0/4047818562 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7fb430067190 0x7fb430069640 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fb4440661e0 tx=0x7fb43400b040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:53:11.015 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:11.009+0000 7fb43bfff640 1 -- 172.21.15.156:0/4047818562 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fb42c071d10 con 0x7fb4440ff570 2024-07-16T09:53:11.223 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:11.222+0000 7fb4492e8640 1 -- 172.21.15.156:0/4047818562 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "false"} v 0) v1 -- 0x7fb44404b530 con 0x7fb4440ff570 2024-07-16T09:53:11.806 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:11.804+0000 7fb43bfff640 1 -- 172.21.15.156:0/4047818562 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "false"}]=0 inline data disabled v16) v1 ==== 133+0+0 (secure 0 0 0) 0x7fb42c0410b0 con 0x7fb4440ff570 2024-07-16T09:53:11.806 INFO:teuthology.orchestra.run.smithi156.stderr:inline data disabled 2024-07-16T09:53:11.808 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:11.806+0000 7fb4492e8640 1 -- 172.21.15.156:0/4047818562 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7fb430067190 msgr2=0x7fb430069640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:53:11.808 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:11.806+0000 7fb4492e8640 1 --2- 172.21.15.156:0/4047818562 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7fb430067190 0x7fb430069640 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fb4440661e0 tx=0x7fb43400b040 comp rx=0 tx=0).stop 2024-07-16T09:53:11.808 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:11.806+0000 7fb4492e8640 1 -- 172.21.15.156:0/4047818562 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb4440ff570 msgr2=0x7fb44410d520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:53:11.808 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:11.806+0000 7fb4492e8640 1 --2- 172.21.15.156:0/4047818562 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb4440ff570 0x7fb44410d520 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fb42c009f80 tx=0x7fb42c009da0 comp rx=0 tx=0).stop 2024-07-16T09:53:11.808 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:11.806+0000 7fb4492e8640 1 -- 172.21.15.156:0/4047818562 shutdown_connections 2024-07-16T09:53:11.808 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:11.806+0000 7fb4492e8640 1 --2- 172.21.15.156:0/4047818562 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7fb430067190 0x7fb430069640 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:53:11.808 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:11.806+0000 7fb4492e8640 1 --2- 172.21.15.156:0/4047818562 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb4440ff570 0x7fb44410d520 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:53:11.809 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:11.806+0000 7fb4492e8640 1 --2- 172.21.15.156:0/4047818562 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb444066480 0x7fb44410cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:53:11.809 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:11.806+0000 7fb4492e8640 1 -- 172.21.15.156:0/4047818562 >> 172.21.15.156:0/4047818562 conn(0x7fb4440f6a20 msgr2=0x7fb4440f6e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:53:11.809 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:11.807+0000 7fb4492e8640 1 -- 172.21.15.156:0/4047818562 shutdown_connections 2024-07-16T09:53:11.809 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:11.807+0000 7fb4492e8640 1 -- 172.21.15.156:0/4047818562 wait complete. 2024-07-16T09:53:12.093 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:53:11 smithi176 ceph-mon[36626]: pgmap v177: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.6 KiB/s rd, 2.0 KiB/s wr, 8 op/s 2024-07-16T09:53:12.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:53:11 smithi176 ceph-mon[36626]: mds.? [v2:172.21.15.176:6824/1577019287,v1:172.21.15.176:6825/1577019287] up:active 2024-07-16T09:53:12.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:53:11 smithi176 ceph-mon[36626]: mds.? [v2:172.21.15.176:6826/2196775412,v1:172.21.15.176:6827/2196775412] up:standby-replay 2024-07-16T09:53:12.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:53:11 smithi176 ceph-mon[36626]: fsmap cephfs:2 {0=cephfs.smithi156.rdibys=up:active,1=cephfs.smithi176.dvvbud=up:active} 2 up:standby-replay 2024-07-16T09:53:12.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:53:11 smithi176 ceph-mon[36626]: from='client.? 172.21.15.156:0/4047818562' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "false"}]: dispatch 2024-07-16T09:53:12.112 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:53:11 smithi156 ceph-mon[29302]: pgmap v177: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.6 KiB/s rd, 2.0 KiB/s wr, 8 op/s 2024-07-16T09:53:12.112 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:53:11 smithi156 ceph-mon[29302]: mds.? [v2:172.21.15.176:6824/1577019287,v1:172.21.15.176:6825/1577019287] up:active 2024-07-16T09:53:12.113 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:53:11 smithi156 ceph-mon[29302]: mds.? [v2:172.21.15.176:6826/2196775412,v1:172.21.15.176:6827/2196775412] up:standby-replay 2024-07-16T09:53:12.113 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:53:11 smithi156 ceph-mon[29302]: fsmap cephfs:2 {0=cephfs.smithi156.rdibys=up:active,1=cephfs.smithi176.dvvbud=up:active} 2 up:standby-replay 2024-07-16T09:53:12.113 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:53:11 smithi156 ceph-mon[29302]: from='client.? 172.21.15.156:0/4047818562' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "false"}]: dispatch 2024-07-16T09:53:12.444 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-07-16T09:53:12.455 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi156.front.sepia.ceph.com 2024-07-16T09:53:12.455 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 26c91c26-4358-11ef-bcad-c7b262605968 -- bash -c 'ceph fs dump' 2024-07-16T09:53:12.924 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:53:12 smithi156 ceph-mon[29302]: from='client.? 172.21.15.156:0/4047818562' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "false"}]': finished 2024-07-16T09:53:12.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:53:12 smithi156 ceph-mon[29302]: fsmap cephfs:2 {0=cephfs.smithi156.rdibys=up:active,1=cephfs.smithi176.dvvbud=up:active} 2 up:standby-replay 2024-07-16T09:53:13.093 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:53:12 smithi176 ceph-mon[36626]: from='client.? 172.21.15.156:0/4047818562' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "false"}]': finished 2024-07-16T09:53:13.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:53:12 smithi176 ceph-mon[36626]: fsmap cephfs:2 {0=cephfs.smithi156.rdibys=up:active,1=cephfs.smithi176.dvvbud=up:active} 2 up:standby-replay 2024-07-16T09:53:13.210 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/26c91c26-4358-11ef-bcad-c7b262605968/mon.smithi156/config 2024-07-16T09:53:13.924 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:53:13 smithi156 ceph-mon[29302]: pgmap v178: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.6 KiB/s rd, 1.4 KiB/s wr, 8 op/s 2024-07-16T09:53:14.093 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:53:13 smithi176 ceph-mon[36626]: pgmap v178: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.6 KiB/s rd, 1.4 KiB/s wr, 8 op/s 2024-07-16T09:53:14.998 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:14.995+0000 7f814e7ec640 1 -- 172.21.15.156:0/3039015858 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8148104520 msgr2=0x7f8148106900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:53:14.998 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:14.995+0000 7f814e7ec640 1 --2- 172.21.15.156:0/3039015858 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8148104520 0x7f8148106900 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f81380048d0 tx=0x7f8138009da0 comp rx=0 tx=0).stop 2024-07-16T09:53:14.998 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:14.995+0000 7f814e7ec640 1 -- 172.21.15.156:0/3039015858 shutdown_connections 2024-07-16T09:53:14.998 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:14.995+0000 7f814e7ec640 1 --2- 172.21.15.156:0/3039015858 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8148106e40 0x7f8148109220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:53:14.998 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:14.995+0000 7f814e7ec640 1 --2- 172.21.15.156:0/3039015858 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8148104520 0x7f8148106900 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:53:14.998 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:14.995+0000 7f814e7ec640 1 -- 172.21.15.156:0/3039015858 >> 172.21.15.156:0/3039015858 conn(0x7f8148069910 msgr2=0x7f814806bd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:53:14.998 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:14.995+0000 7f814e7ec640 1 -- 172.21.15.156:0/3039015858 shutdown_connections 2024-07-16T09:53:14.998 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:14.995+0000 7f814e7ec640 1 -- 172.21.15.156:0/3039015858 wait complete. 2024-07-16T09:53:14.998 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:14.995+0000 7f814e7ec640 1 Processor -- start 2024-07-16T09:53:14.998 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:14.995+0000 7f814e7ec640 1 -- start start 2024-07-16T09:53:14.998 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:14.996+0000 7f814e7ec640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8148104520 0x7f81481157c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:53:14.999 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:14.996+0000 7f814e7ec640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8148106e40 0x7f8148115d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:53:14.999 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:14.996+0000 7f814e7ec640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8148116350 con 0x7f8148104520 2024-07-16T09:53:14.999 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:14.996+0000 7f814e7ec640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8148116b70 con 0x7f8148106e40 2024-07-16T09:53:14.999 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:14.996+0000 7f8147fff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8148104520 0x7f81481157c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:53:14.999 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:14.996+0000 7f8147fff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8148104520 0x7f81481157c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:54390/0 (socket says 172.21.15.156:54390) 2024-07-16T09:53:14.999 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:14.996+0000 7f8147fff640 1 -- 172.21.15.156:0/2589490578 learned_addr learned my addr 172.21.15.156:0/2589490578 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T09:53:14.999 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:14.996+0000 7f81477fe640 1 --2- 172.21.15.156:0/2589490578 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8148106e40 0x7f8148115d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:53:14.999 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:14.996+0000 7f8147fff640 1 -- 172.21.15.156:0/2589490578 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8148106e40 msgr2=0x7f8148115d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:53:14.999 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:14.996+0000 7f8147fff640 1 --2- 172.21.15.156:0/2589490578 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8148106e40 0x7f8148115d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:53:14.999 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:14.996+0000 7f8147fff640 1 -- 172.21.15.156:0/2589490578 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8138005040 con 0x7f8148104520 2024-07-16T09:53:14.999 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:14.996+0000 7f8147fff640 1 --2- 172.21.15.156:0/2589490578 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8148104520 0x7f81481157c0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f81380043d0 tx=0x7f8138009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:53:14.999 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:14.996+0000 7f8144ff9640 1 -- 172.21.15.156:0/2589490578 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8138038070 con 0x7f8148104520 2024-07-16T09:53:14.999 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:14.996+0000 7f8144ff9640 1 -- 172.21.15.156:0/2589490578 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8138032970 con 0x7f8148104520 2024-07-16T09:53:15.000 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:14.996+0000 7f814e7ec640 1 -- 172.21.15.156:0/2589490578 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8148116da0 con 0x7f8148104520 2024-07-16T09:53:15.000 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:14.996+0000 7f814e7ec640 1 -- 172.21.15.156:0/2589490578 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8148117260 con 0x7f8148104520 2024-07-16T09:53:15.000 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:14.996+0000 7f8144ff9640 1 -- 172.21.15.156:0/2589490578 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f81380337f0 con 0x7f8148104520 2024-07-16T09:53:15.001 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:14.997+0000 7f814e7ec640 1 -- 172.21.15.156:0/2589490578 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f814806d460 con 0x7f8148104520 2024-07-16T09:53:15.001 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:14.998+0000 7f8144ff9640 1 -- 172.21.15.156:0/2589490578 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 19) v1 ==== 85127+0+0 (secure 0 0 0) 0x7f8138030020 con 0x7f8148104520 2024-07-16T09:53:15.001 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:14.998+0000 7f8144ff9640 1 --2- 172.21.15.156:0/2589490578 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f8130067190 0x7f8130069640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:53:15.001 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:14.998+0000 7f8144ff9640 1 -- 172.21.15.156:0/2589490578 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f81380a4920 con 0x7f8148104520 2024-07-16T09:53:15.001 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:14.998+0000 7f81477fe640 1 --2- 172.21.15.156:0/2589490578 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f8130067190 0x7f8130069640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:53:15.001 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:14.999+0000 7f81477fe640 1 --2- 172.21.15.156:0/2589490578 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f8130067190 0x7f8130069640 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f8148115570 tx=0x7f813c00b040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:53:15.004 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:15.002+0000 7f8144ff9640 1 -- 172.21.15.156:0/2589490578 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f8138071cf0 con 0x7f8148104520 2024-07-16T09:53:15.246 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:15.244+0000 7f814e7ec640 1 -- 172.21.15.156:0/2589490578 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f814810c5c0 con 0x7f8148104520 2024-07-16T09:53:15.246 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:15.245+0000 7f8144ff9640 1 -- 172.21.15.156:0/2589490578 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 16 v16) v1 ==== 76+0+1835 (secure 0 0 0) 0x7f8138041070 con 0x7f8148104520 2024-07-16T09:53:15.246 INFO:teuthology.orchestra.run.smithi156.stdout:e16 2024-07-16T09:53:15.247 INFO:teuthology.orchestra.run.smithi156.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-07-16T09:53:15.247 INFO:teuthology.orchestra.run.smithi156.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-07-16T09:53:15.247 INFO:teuthology.orchestra.run.smithi156.stdout:legacy client fscid: 1 2024-07-16T09:53:15.247 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-07-16T09:53:15.247 INFO:teuthology.orchestra.run.smithi156.stdout:Filesystem 'cephfs' (1) 2024-07-16T09:53:15.247 INFO:teuthology.orchestra.run.smithi156.stdout:fs_name cephfs 2024-07-16T09:53:15.247 INFO:teuthology.orchestra.run.smithi156.stdout:epoch 16 2024-07-16T09:53:15.247 INFO:teuthology.orchestra.run.smithi156.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-07-16T09:53:15.247 INFO:teuthology.orchestra.run.smithi156.stdout:created 2024-07-16T09:52:55.214975+0000 2024-07-16T09:53:15.247 INFO:teuthology.orchestra.run.smithi156.stdout:modified 2024-07-16T09:53:11.798619+0000 2024-07-16T09:53:15.247 INFO:teuthology.orchestra.run.smithi156.stdout:tableserver 0 2024-07-16T09:53:15.248 INFO:teuthology.orchestra.run.smithi156.stdout:root 0 2024-07-16T09:53:15.248 INFO:teuthology.orchestra.run.smithi156.stdout:session_timeout 60 2024-07-16T09:53:15.248 INFO:teuthology.orchestra.run.smithi156.stdout:session_autoclose 300 2024-07-16T09:53:15.248 INFO:teuthology.orchestra.run.smithi156.stdout:max_file_size 1099511627776 2024-07-16T09:53:15.248 INFO:teuthology.orchestra.run.smithi156.stdout:required_client_features {} 2024-07-16T09:53:15.248 INFO:teuthology.orchestra.run.smithi156.stdout:last_failure 0 2024-07-16T09:53:15.248 INFO:teuthology.orchestra.run.smithi156.stdout:last_failure_osd_epoch 0 2024-07-16T09:53:15.248 INFO:teuthology.orchestra.run.smithi156.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-07-16T09:53:15.248 INFO:teuthology.orchestra.run.smithi156.stdout:max_mds 2 2024-07-16T09:53:15.248 INFO:teuthology.orchestra.run.smithi156.stdout:in 0,1 2024-07-16T09:53:15.248 INFO:teuthology.orchestra.run.smithi156.stdout:up {0=14458,1=24261} 2024-07-16T09:53:15.248 INFO:teuthology.orchestra.run.smithi156.stdout:failed 2024-07-16T09:53:15.248 INFO:teuthology.orchestra.run.smithi156.stdout:damaged 2024-07-16T09:53:15.249 INFO:teuthology.orchestra.run.smithi156.stdout:stopped 2024-07-16T09:53:15.249 INFO:teuthology.orchestra.run.smithi156.stdout:data_pools [3] 2024-07-16T09:53:15.249 INFO:teuthology.orchestra.run.smithi156.stdout:metadata_pool 2 2024-07-16T09:53:15.249 INFO:teuthology.orchestra.run.smithi156.stdout:inline_data disabled 2024-07-16T09:53:15.249 INFO:teuthology.orchestra.run.smithi156.stdout:balancer 2024-07-16T09:53:15.249 INFO:teuthology.orchestra.run.smithi156.stdout:standby_count_wanted 1 2024-07-16T09:53:15.249 INFO:teuthology.orchestra.run.smithi156.stdout:[mds.cephfs.smithi156.rdibys{0:14458} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.156:6826/1080176013,v1:172.21.15.156:6827/1080176013] compat {c=[1],r=[1],i=[7ff]}] 2024-07-16T09:53:15.249 INFO:teuthology.orchestra.run.smithi156.stdout:[mds.cephfs.smithi156.evwwaz{0:14468} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.156:6828/560329502,v1:172.21.15.156:6829/560329502] compat {c=[1],r=[1],i=[7ff]}] 2024-07-16T09:53:15.249 INFO:teuthology.orchestra.run.smithi156.stdout:[mds.cephfs.smithi176.dvvbud{1:24261} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.176:6824/1577019287,v1:172.21.15.176:6825/1577019287] compat {c=[1],r=[1],i=[7ff]}] 2024-07-16T09:53:15.249 INFO:teuthology.orchestra.run.smithi156.stdout:[mds.cephfs.smithi176.wahbsi{1:24273} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.176:6826/2196775412,v1:172.21.15.176:6827/2196775412] compat {c=[1],r=[1],i=[7ff]}] 2024-07-16T09:53:15.249 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-07-16T09:53:15.249 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-07-16T09:53:15.250 INFO:teuthology.orchestra.run.smithi156.stderr:dumped fsmap epoch 16 2024-07-16T09:53:15.250 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:15.247+0000 7f814e7ec640 1 -- 172.21.15.156:0/2589490578 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f8130067190 msgr2=0x7f8130069640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:53:15.250 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:15.247+0000 7f814e7ec640 1 --2- 172.21.15.156:0/2589490578 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f8130067190 0x7f8130069640 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f8148115570 tx=0x7f813c00b040 comp rx=0 tx=0).stop 2024-07-16T09:53:15.250 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:15.247+0000 7f814e7ec640 1 -- 172.21.15.156:0/2589490578 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8148104520 msgr2=0x7f81481157c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:53:15.251 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:15.247+0000 7f814e7ec640 1 --2- 172.21.15.156:0/2589490578 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8148104520 0x7f81481157c0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f81380043d0 tx=0x7f8138009da0 comp rx=0 tx=0).stop 2024-07-16T09:53:15.251 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:15.247+0000 7f814e7ec640 1 -- 172.21.15.156:0/2589490578 shutdown_connections 2024-07-16T09:53:15.251 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:15.247+0000 7f814e7ec640 1 --2- 172.21.15.156:0/2589490578 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f8130067190 0x7f8130069640 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:53:15.251 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:15.247+0000 7f814e7ec640 1 --2- 172.21.15.156:0/2589490578 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8148106e40 0x7f8148115d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:53:15.251 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:15.247+0000 7f814e7ec640 1 --2- 172.21.15.156:0/2589490578 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8148104520 0x7f81481157c0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:53:15.251 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:15.247+0000 7f814e7ec640 1 -- 172.21.15.156:0/2589490578 >> 172.21.15.156:0/2589490578 conn(0x7f8148069910 msgr2=0x7f814806c070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:53:15.251 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:15.247+0000 7f814e7ec640 1 -- 172.21.15.156:0/2589490578 shutdown_connections 2024-07-16T09:53:15.251 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:15.247+0000 7f814e7ec640 1 -- 172.21.15.156:0/2589490578 wait complete. 2024-07-16T09:53:15.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:53:15 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:53:15.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:53:15 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:53:15.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:53:15 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-16T09:53:15.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:53:15 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-16T09:53:15.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:53:15 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:53:15.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:53:15 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-16T09:53:15.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:53:15 smithi176 ceph-mon[36626]: pgmap v179: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.4 KiB/s rd, 938 B/s wr, 7 op/s 2024-07-16T09:53:15.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:53:15 smithi176 ceph-mon[36626]: from='client.? 172.21.15.156:0/2589490578' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-07-16T09:53:15.887 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:53:15 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:53:15.887 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:53:15 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:53:15.887 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:53:15 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-16T09:53:15.887 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:53:15 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-16T09:53:15.888 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:53:15 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:53:15.888 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:53:15 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-16T09:53:15.888 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:53:15 smithi156 ceph-mon[29302]: pgmap v179: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.4 KiB/s rd, 938 B/s wr, 7 op/s 2024-07-16T09:53:15.888 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:53:15 smithi156 ceph-mon[29302]: from='client.? 172.21.15.156:0/2589490578' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-07-16T09:53:16.541 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 26c91c26-4358-11ef-bcad-c7b262605968 -- bash -c 'ceph --format=json fs dump | jq -e ".filesystems | length == 1"' 2024-07-16T09:53:17.450 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/26c91c26-4358-11ef-bcad-c7b262605968/mon.smithi156/config 2024-07-16T09:53:17.843 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:53:17 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:53:17.843 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:53:17 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:53:17.843 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:53:17 smithi176 ceph-mon[36626]: pgmap v180: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.4 KiB/s rd, 85 B/s wr, 4 op/s 2024-07-16T09:53:17.916 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:53:17 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:53:17.916 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:53:17 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:53:17.916 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:53:17 smithi156 ceph-mon[29302]: pgmap v180: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.4 KiB/s rd, 85 B/s wr, 4 op/s 2024-07-16T09:53:20.041 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:53:19 smithi156 ceph-mon[29302]: pgmap v181: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.8 KiB/s rd, 0 B/s wr, 4 op/s 2024-07-16T09:53:20.093 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:53:19 smithi176 ceph-mon[36626]: pgmap v181: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.8 KiB/s rd, 0 B/s wr, 4 op/s 2024-07-16T09:53:20.857 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:20.854+0000 7f86577ae640 1 -- 172.21.15.156:0/864335104 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8650066520 msgr2=0x7f86500fef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:53:20.857 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:20.854+0000 7f86577ae640 1 --2- 172.21.15.156:0/864335104 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8650066520 0x7f86500fef80 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f864c0048d0 tx=0x7f864c009da0 comp rx=0 tx=0).stop 2024-07-16T09:53:20.857 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:20.854+0000 7f86577ae640 1 -- 172.21.15.156:0/864335104 shutdown_connections 2024-07-16T09:53:20.857 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:20.854+0000 7f86577ae640 1 --2- 172.21.15.156:0/864335104 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f86500ff4c0 0x7f86501018a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:53:20.857 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:20.854+0000 7f86577ae640 1 --2- 172.21.15.156:0/864335104 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8650066520 0x7f86500fef80 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:53:20.857 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:20.854+0000 7f86577ae640 1 -- 172.21.15.156:0/864335104 >> 172.21.15.156:0/864335104 conn(0x7f86500f6a50 msgr2=0x7f86500f8e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:53:20.857 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:20.854+0000 7f86577ae640 1 -- 172.21.15.156:0/864335104 shutdown_connections 2024-07-16T09:53:20.857 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:20.854+0000 7f86577ae640 1 -- 172.21.15.156:0/864335104 wait complete. 2024-07-16T09:53:20.857 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:20.854+0000 7f86577ae640 1 Processor -- start 2024-07-16T09:53:20.858 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:20.854+0000 7f86577ae640 1 -- start start 2024-07-16T09:53:20.858 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:20.855+0000 7f86577ae640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8650066520 0x7f865010cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:53:20.858 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:20.855+0000 7f86577ae640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f86500ff4c0 0x7f865010d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:53:20.858 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:20.855+0000 7f86577ae640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f865010db30 con 0x7f8650066520 2024-07-16T09:53:20.858 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:20.855+0000 7f86577ae640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f865010e350 con 0x7f86500ff4c0 2024-07-16T09:53:20.858 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:20.855+0000 7f865554a640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8650066520 0x7f865010cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:53:20.858 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:20.855+0000 7f865554a640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8650066520 0x7f865010cfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:54422/0 (socket says 172.21.15.156:54422) 2024-07-16T09:53:20.858 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:20.855+0000 7f865554a640 1 -- 172.21.15.156:0/491274681 learned_addr learned my addr 172.21.15.156:0/491274681 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T09:53:20.858 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:20.855+0000 7f8654d49640 1 --2- 172.21.15.156:0/491274681 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f86500ff4c0 0x7f865010d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:53:20.858 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:20.855+0000 7f865554a640 1 -- 172.21.15.156:0/491274681 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f86500ff4c0 msgr2=0x7f865010d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:53:20.858 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:20.855+0000 7f865554a640 1 --2- 172.21.15.156:0/491274681 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f86500ff4c0 0x7f865010d4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:53:20.859 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:20.855+0000 7f865554a640 1 -- 172.21.15.156:0/491274681 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f864c005040 con 0x7f8650066520 2024-07-16T09:53:20.859 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:20.855+0000 7f865554a640 1 --2- 172.21.15.156:0/491274681 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8650066520 0x7f865010cfa0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f864c009f80 tx=0x7f864c009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:53:20.861 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:20.855+0000 7f8645ffb640 1 -- 172.21.15.156:0/491274681 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f864c038070 con 0x7f8650066520 2024-07-16T09:53:20.861 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:20.855+0000 7f86577ae640 1 -- 172.21.15.156:0/491274681 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f865010e580 con 0x7f8650066520 2024-07-16T09:53:20.861 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:20.856+0000 7f86577ae640 1 -- 172.21.15.156:0/491274681 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f865010eac0 con 0x7f8650066520 2024-07-16T09:53:20.861 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:20.856+0000 7f8645ffb640 1 -- 172.21.15.156:0/491274681 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f864c008ee0 con 0x7f8650066520 2024-07-16T09:53:20.862 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:20.856+0000 7f8645ffb640 1 -- 172.21.15.156:0/491274681 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f864c033660 con 0x7f8650066520 2024-07-16T09:53:20.863 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:20.857+0000 7f86577ae640 1 -- 172.21.15.156:0/491274681 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8634005240 con 0x7f8650066520 2024-07-16T09:53:20.863 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:20.857+0000 7f8645ffb640 1 -- 172.21.15.156:0/491274681 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 19) v1 ==== 85127+0+0 (secure 0 0 0) 0x7f864c030020 con 0x7f8650066520 2024-07-16T09:53:20.863 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:20.857+0000 7f8645ffb640 1 --2- 172.21.15.156:0/491274681 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f863c067190 0x7f863c069640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:53:20.864 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:20.857+0000 7f8645ffb640 1 -- 172.21.15.156:0/491274681 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f864c0a4860 con 0x7f8650066520 2024-07-16T09:53:20.864 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:20.857+0000 7f8654d49640 1 --2- 172.21.15.156:0/491274681 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f863c067190 0x7f863c069640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:53:20.864 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:20.858+0000 7f8654d49640 1 --2- 172.21.15.156:0/491274681 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f863c067190 0x7f863c069640 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f865010cd50 tx=0x7f864000b040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:53:20.864 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:20.862+0000 7f8645ffb640 1 -- 172.21.15.156:0/491274681 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f864c071bb0 con 0x7f8650066520 2024-07-16T09:53:21.077 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:21.075+0000 7f86577ae640 1 -- 172.21.15.156:0/491274681 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8634005a70 con 0x7f8650066520 2024-07-16T09:53:21.077 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:21.076+0000 7f8645ffb640 1 -- 172.21.15.156:0/491274681 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4726 (secure 0 0 0) 0x7f864c0751d0 con 0x7f8650066520 2024-07-16T09:53:21.077 INFO:teuthology.orchestra.run.smithi156.stderr:dumped fsmap epoch 16 2024-07-16T09:53:21.080 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:21.078+0000 7f86577ae640 1 -- 172.21.15.156:0/491274681 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f863c067190 msgr2=0x7f863c069640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:53:21.080 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:21.078+0000 7f86577ae640 1 --2- 172.21.15.156:0/491274681 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f863c067190 0x7f863c069640 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f865010cd50 tx=0x7f864000b040 comp rx=0 tx=0).stop 2024-07-16T09:53:21.080 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:21.078+0000 7f86577ae640 1 -- 172.21.15.156:0/491274681 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8650066520 msgr2=0x7f865010cfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:53:21.080 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:21.078+0000 7f86577ae640 1 --2- 172.21.15.156:0/491274681 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8650066520 0x7f865010cfa0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f864c009f80 tx=0x7f864c009da0 comp rx=0 tx=0).stop 2024-07-16T09:53:21.080 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:21.078+0000 7f86577ae640 1 -- 172.21.15.156:0/491274681 shutdown_connections 2024-07-16T09:53:21.080 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:21.078+0000 7f86577ae640 1 --2- 172.21.15.156:0/491274681 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f863c067190 0x7f863c069640 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:53:21.080 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:21.078+0000 7f86577ae640 1 --2- 172.21.15.156:0/491274681 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f86500ff4c0 0x7f865010d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:53:21.080 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:21.078+0000 7f86577ae640 1 --2- 172.21.15.156:0/491274681 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8650066520 0x7f865010cfa0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:53:21.080 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:21.078+0000 7f86577ae640 1 -- 172.21.15.156:0/491274681 >> 172.21.15.156:0/491274681 conn(0x7f86500f6a50 msgr2=0x7f86500f6e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:53:21.080 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:21.078+0000 7f86577ae640 1 -- 172.21.15.156:0/491274681 shutdown_connections 2024-07-16T09:53:21.081 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:21.078+0000 7f86577ae640 1 -- 172.21.15.156:0/491274681 wait complete. 2024-07-16T09:53:21.089 INFO:teuthology.orchestra.run.smithi156.stdout:true 2024-07-16T09:53:21.671 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 26c91c26-4358-11ef-bcad-c7b262605968 -- bash -c 'while ! ceph --format=json mds versions | jq -e ". | add == 4"; do sleep 1; done' 2024-07-16T09:53:21.926 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/26c91c26-4358-11ef-bcad-c7b262605968/mon.smithi156/config 2024-07-16T09:53:22.332 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:53:22 smithi156 ceph-mon[29302]: pgmap v182: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.6 KiB/s rd, 4 op/s 2024-07-16T09:53:22.333 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:53:22 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:53:22.333 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:53:22 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:53:22.333 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:53:22 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-16T09:53:22.333 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:53:22 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-16T09:53:22.333 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:53:22 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:53:22.333 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:53:22 smithi156 ceph-mon[29302]: from='client.? 172.21.15.156:0/491274681' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-16T09:53:22.343 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:53:22 smithi176 ceph-mon[36626]: pgmap v182: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.6 KiB/s rd, 4 op/s 2024-07-16T09:53:22.343 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:53:22 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:53:22.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:53:22 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:53:22.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:53:22 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-16T09:53:22.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:53:22 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-16T09:53:22.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:53:22 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:53:22.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:53:22 smithi176 ceph-mon[36626]: from='client.? 172.21.15.156:0/491274681' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-16T09:53:23.428 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:23.425+0000 7f84c772d640 1 -- 172.21.15.156:0/1267122106 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f84c00ff510 msgr2=0x7f84c01018f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:53:23.428 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:23.425+0000 7f84c772d640 1 --2- 172.21.15.156:0/1267122106 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f84c00ff510 0x7f84c01018f0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f84b00048d0 tx=0x7f84b0009da0 comp rx=0 tx=0).stop 2024-07-16T09:53:23.428 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:23.425+0000 7f84c772d640 1 -- 172.21.15.156:0/1267122106 shutdown_connections 2024-07-16T09:53:23.429 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:23.425+0000 7f84c772d640 1 --2- 172.21.15.156:0/1267122106 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f84c00ff510 0x7f84c01018f0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:53:23.429 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:23.425+0000 7f84c772d640 1 --2- 172.21.15.156:0/1267122106 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f84c00659d0 0x7f84c00fefd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:53:23.429 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:23.425+0000 7f84c772d640 1 -- 172.21.15.156:0/1267122106 >> 172.21.15.156:0/1267122106 conn(0x7f84c00f6a20 msgr2=0x7f84c00f8e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:53:23.429 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:23.425+0000 7f84c772d640 1 -- 172.21.15.156:0/1267122106 shutdown_connections 2024-07-16T09:53:23.429 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:23.425+0000 7f84c772d640 1 -- 172.21.15.156:0/1267122106 wait complete. 2024-07-16T09:53:23.429 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:23.426+0000 7f84c772d640 1 Processor -- start 2024-07-16T09:53:23.429 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:23.426+0000 7f84c772d640 1 -- start start 2024-07-16T09:53:23.429 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:23.426+0000 7f84c772d640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f84c00659d0 0x7f84c00fc9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:53:23.429 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:23.426+0000 7f84c772d640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f84c00ff510 0x7f84c00faed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:53:23.429 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:23.426+0000 7f84c772d640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84c00fb460 con 0x7f84c00ff510 2024-07-16T09:53:23.429 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:23.426+0000 7f84c772d640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84c00fb5a0 con 0x7f84c00659d0 2024-07-16T09:53:23.429 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:23.426+0000 7f84c4cc8640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f84c00ff510 0x7f84c00faed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:53:23.430 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:23.426+0000 7f84c4cc8640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f84c00ff510 0x7f84c00faed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:35868/0 (socket says 172.21.15.156:35868) 2024-07-16T09:53:23.430 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:23.426+0000 7f84c4cc8640 1 -- 172.21.15.156:0/4075415193 learned_addr learned my addr 172.21.15.156:0/4075415193 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T09:53:23.430 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:23.426+0000 7f84c54c9640 1 --2- 172.21.15.156:0/4075415193 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f84c00659d0 0x7f84c00fc9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:53:23.430 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:23.426+0000 7f84c4cc8640 1 -- 172.21.15.156:0/4075415193 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f84c00659d0 msgr2=0x7f84c00fc9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:53:23.430 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:23.426+0000 7f84c4cc8640 1 --2- 172.21.15.156:0/4075415193 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f84c00659d0 0x7f84c00fc9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:53:23.430 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:23.426+0000 7f84c4cc8640 1 -- 172.21.15.156:0/4075415193 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84b0005040 con 0x7f84c00ff510 2024-07-16T09:53:23.430 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:23.426+0000 7f84c4cc8640 1 --2- 172.21.15.156:0/4075415193 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f84c00ff510 0x7f84c00faed0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f84b0000c00 tx=0x7f84b0009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:53:23.430 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:23.427+0000 7f84b5ffb640 1 -- 172.21.15.156:0/4075415193 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f84b0038070 con 0x7f84c00ff510 2024-07-16T09:53:23.430 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:23.427+0000 7f84b5ffb640 1 -- 172.21.15.156:0/4075415193 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f84b0032970 con 0x7f84c00ff510 2024-07-16T09:53:23.430 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:23.427+0000 7f84c772d640 1 -- 172.21.15.156:0/4075415193 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84c00fb820 con 0x7f84c00ff510 2024-07-16T09:53:23.430 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:23.427+0000 7f84b5ffb640 1 -- 172.21.15.156:0/4075415193 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f84b0033950 con 0x7f84c00ff510 2024-07-16T09:53:23.430 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:23.427+0000 7f84c772d640 1 -- 172.21.15.156:0/4075415193 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84c0114ff0 con 0x7f84c00ff510 2024-07-16T09:53:23.431 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:23.428+0000 7f84c772d640 1 -- 172.21.15.156:0/4075415193 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84c00fa6a0 con 0x7f84c00ff510 2024-07-16T09:53:23.432 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:23.428+0000 7f84b5ffb640 1 -- 172.21.15.156:0/4075415193 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 19) v1 ==== 85127+0+0 (secure 0 0 0) 0x7f84b0030020 con 0x7f84c00ff510 2024-07-16T09:53:23.432 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:23.429+0000 7f84b5ffb640 1 --2- 172.21.15.156:0/4075415193 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f84ac067190 0x7f84ac069640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:53:23.432 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:23.429+0000 7f84b5ffb640 1 -- 172.21.15.156:0/4075415193 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f84b00a4ba0 con 0x7f84c00ff510 2024-07-16T09:53:23.432 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:23.429+0000 7f84c54c9640 1 --2- 172.21.15.156:0/4075415193 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f84ac067190 0x7f84ac069640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:53:23.432 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:23.429+0000 7f84c54c9640 1 --2- 172.21.15.156:0/4075415193 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f84ac067190 0x7f84ac069640 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f84c00fc790 tx=0x7f84bc009040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:53:23.434 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:23.433+0000 7f84b5ffb640 1 -- 172.21.15.156:0/4075415193 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f84b0071e40 con 0x7f84c00ff510 2024-07-16T09:53:23.662 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:23.661+0000 7f84c772d640 1 -- 172.21.15.156:0/4075415193 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "mds versions", "format": "json"} v 0) v1 -- 0x7f84c004b530 con 0x7f84c00ff510 2024-07-16T09:53:23.663 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:23.662+0000 7f84b5ffb640 1 -- 172.21.15.156:0/4075415193 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "mds versions", "format": "json"}]=0 v16) v1 ==== 78+0+100 (secure 0 0 0) 0x7f84b0075460 con 0x7f84c00ff510 2024-07-16T09:53:23.665 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:23.664+0000 7f84c772d640 1 -- 172.21.15.156:0/4075415193 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f84ac067190 msgr2=0x7f84ac069640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:53:23.665 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:23.664+0000 7f84c772d640 1 --2- 172.21.15.156:0/4075415193 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f84ac067190 0x7f84ac069640 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f84c00fc790 tx=0x7f84bc009040 comp rx=0 tx=0).stop 2024-07-16T09:53:23.665 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:23.664+0000 7f84c772d640 1 -- 172.21.15.156:0/4075415193 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f84c00ff510 msgr2=0x7f84c00faed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:53:23.665 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:23.664+0000 7f84c772d640 1 --2- 172.21.15.156:0/4075415193 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f84c00ff510 0x7f84c00faed0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f84b0000c00 tx=0x7f84b0009da0 comp rx=0 tx=0).stop 2024-07-16T09:53:23.665 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:23.664+0000 7f84c772d640 1 -- 172.21.15.156:0/4075415193 shutdown_connections 2024-07-16T09:53:23.666 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:23.664+0000 7f84c772d640 1 --2- 172.21.15.156:0/4075415193 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f84ac067190 0x7f84ac069640 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:53:23.666 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:23.664+0000 7f84c772d640 1 --2- 172.21.15.156:0/4075415193 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f84c00ff510 0x7f84c00faed0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:53:23.666 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:23.664+0000 7f84c772d640 1 --2- 172.21.15.156:0/4075415193 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f84c00659d0 0x7f84c00fc9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:53:23.666 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:23.664+0000 7f84c772d640 1 -- 172.21.15.156:0/4075415193 >> 172.21.15.156:0/4075415193 conn(0x7f84c00f6a20 msgr2=0x7f84c00f6df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:53:23.666 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:23.664+0000 7f84c772d640 1 -- 172.21.15.156:0/4075415193 shutdown_connections 2024-07-16T09:53:23.666 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:23.664+0000 7f84c772d640 1 -- 172.21.15.156:0/4075415193 wait complete. 2024-07-16T09:53:23.674 INFO:teuthology.orchestra.run.smithi156.stdout:true 2024-07-16T09:53:24.258 INFO:teuthology.run_tasks:Running task fs.pre_upgrade_save... 2024-07-16T09:53:24.272 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 26c91c26-4358-11ef-bcad-c7b262605968 -- ceph fs dump --format=json 2024-07-16T09:53:24.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:53:24 smithi176 ceph-mon[36626]: pgmap v183: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-07-16T09:53:24.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:53:24 smithi176 ceph-mon[36626]: from='client.? 172.21.15.156:0/4075415193' entity='client.admin' cmd=[{"prefix": "mds versions", "format": "json"}]: dispatch 2024-07-16T09:53:24.424 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:53:24 smithi156 ceph-mon[29302]: pgmap v183: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-07-16T09:53:24.424 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:53:24 smithi156 ceph-mon[29302]: from='client.? 172.21.15.156:0/4075415193' entity='client.admin' cmd=[{"prefix": "mds versions", "format": "json"}]: dispatch 2024-07-16T09:53:24.527 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/26c91c26-4358-11ef-bcad-c7b262605968/mon.smithi156/config 2024-07-16T09:53:25.310 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:53:25 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-16T09:53:25.343 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:53:25 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-16T09:53:26.092 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:26.089+0000 7f2e0c1a5640 1 -- 172.21.15.156:0/2826882290 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2e04102cf0 msgr2=0x7f2e040fc380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:53:26.092 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:26.089+0000 7f2e0c1a5640 1 --2- 172.21.15.156:0/2826882290 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2e04102cf0 0x7f2e040fc380 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f2df40048d0 tx=0x7f2df4009da0 comp rx=0 tx=0).stop 2024-07-16T09:53:26.092 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:26.089+0000 7f2e0c1a5640 1 -- 172.21.15.156:0/2826882290 shutdown_connections 2024-07-16T09:53:26.092 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:26.089+0000 7f2e0c1a5640 1 --2- 172.21.15.156:0/2826882290 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2e04102cf0 0x7f2e040fc380 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:53:26.092 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:26.089+0000 7f2e0c1a5640 1 --2- 172.21.15.156:0/2826882290 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2e04102350 0x7f2e04102720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:53:26.092 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:26.089+0000 7f2e0c1a5640 1 -- 172.21.15.156:0/2826882290 >> 172.21.15.156:0/2826882290 conn(0x7f2e040f7f20 msgr2=0x7f2e040fa330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:53:26.092 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:26.089+0000 7f2e0c1a5640 1 -- 172.21.15.156:0/2826882290 shutdown_connections 2024-07-16T09:53:26.092 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:26.089+0000 7f2e0c1a5640 1 -- 172.21.15.156:0/2826882290 wait complete. 2024-07-16T09:53:26.092 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:26.089+0000 7f2e0c1a5640 1 Processor -- start 2024-07-16T09:53:26.092 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:26.089+0000 7f2e0c1a5640 1 -- start start 2024-07-16T09:53:26.093 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:26.089+0000 7f2e0c1a5640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2e04102350 0x7f2e04113100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:53:26.093 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:26.089+0000 7f2e0c1a5640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2e04102cf0 0x7f2e04113640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:53:26.093 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:26.089+0000 7f2e0c1a5640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e04113c40 con 0x7f2e04102350 2024-07-16T09:53:26.093 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:26.089+0000 7f2e0c1a5640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e0410d080 con 0x7f2e04102cf0 2024-07-16T09:53:26.093 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:26.090+0000 7f2e09f41640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2e04102350 0x7f2e04113100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:53:26.093 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:26.090+0000 7f2e09f41640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2e04102350 0x7f2e04113100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:35882/0 (socket says 172.21.15.156:35882) 2024-07-16T09:53:26.093 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:26.090+0000 7f2e09f41640 1 -- 172.21.15.156:0/2167350940 learned_addr learned my addr 172.21.15.156:0/2167350940 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T09:53:26.093 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:26.090+0000 7f2e09740640 1 --2- 172.21.15.156:0/2167350940 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2e04102cf0 0x7f2e04113640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:53:26.093 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:26.090+0000 7f2e09f41640 1 -- 172.21.15.156:0/2167350940 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2e04102cf0 msgr2=0x7f2e04113640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:53:26.093 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:26.090+0000 7f2e09f41640 1 --2- 172.21.15.156:0/2167350940 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2e04102cf0 0x7f2e04113640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:53:26.093 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:26.090+0000 7f2e09f41640 1 -- 172.21.15.156:0/2167350940 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2df4005040 con 0x7f2e04102350 2024-07-16T09:53:26.094 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:26.090+0000 7f2e09f41640 1 --2- 172.21.15.156:0/2167350940 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2e04102350 0x7f2e04113100 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f2e00002a50 tx=0x7f2e00004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:53:26.094 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:26.090+0000 7f2dfa7fc640 1 -- 172.21.15.156:0/2167350940 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2e00007c30 con 0x7f2e04102350 2024-07-16T09:53:26.094 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:26.090+0000 7f2e0c1a5640 1 -- 172.21.15.156:0/2167350940 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e0410d360 con 0x7f2e04102350 2024-07-16T09:53:26.094 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:26.090+0000 7f2dfa7fc640 1 -- 172.21.15.156:0/2167350940 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2e0000f040 con 0x7f2e04102350 2024-07-16T09:53:26.094 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:26.090+0000 7f2dfa7fc640 1 -- 172.21.15.156:0/2167350940 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2e0000dad0 con 0x7f2e04102350 2024-07-16T09:53:26.094 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:26.090+0000 7f2e0c1a5640 1 -- 172.21.15.156:0/2167350940 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e0410d8d0 con 0x7f2e04102350 2024-07-16T09:53:26.095 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:26.091+0000 7f2e0c1a5640 1 -- 172.21.15.156:0/2167350940 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e040fdac0 con 0x7f2e04102350 2024-07-16T09:53:26.095 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:26.092+0000 7f2dfa7fc640 1 -- 172.21.15.156:0/2167350940 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 19) v1 ==== 85127+0+0 (secure 0 0 0) 0x7f2e0000dd40 con 0x7f2e04102350 2024-07-16T09:53:26.095 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:26.092+0000 7f2dfa7fc640 1 --2- 172.21.15.156:0/2167350940 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f2df0067190 0x7f2df0069640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:53:26.095 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:26.092+0000 7f2dfa7fc640 1 -- 172.21.15.156:0/2167350940 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f2e000842d0 con 0x7f2e04102350 2024-07-16T09:53:26.095 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:26.092+0000 7f2e09740640 1 --2- 172.21.15.156:0/2167350940 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f2df0067190 0x7f2df0069640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:53:26.096 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:26.093+0000 7f2e09740640 1 --2- 172.21.15.156:0/2167350940 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f2df0067190 0x7f2df0069640 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f2df4009f80 tx=0x7f2df4035040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:53:26.097 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:26.096+0000 7f2dfa7fc640 1 -- 172.21.15.156:0/2167350940 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f2e0004d250 con 0x7f2e04102350 2024-07-16T09:53:26.174 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:53:26 smithi156 ceph-mon[29302]: pgmap v184: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-07-16T09:53:26.299 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:26.298+0000 7f2e0c1a5640 1 -- 172.21.15.156:0/2167350940 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2e041094b0 con 0x7f2e04102350 2024-07-16T09:53:26.299 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:26.298+0000 7f2dfa7fc640 1 -- 172.21.15.156:0/2167350940 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4726 (secure 0 0 0) 0x7f2e00051620 con 0x7f2e04102350 2024-07-16T09:53:26.300 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-07-16T09:53:26.300 INFO:teuthology.orchestra.run.smithi156.stdout:{"epoch":16,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":16,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-16T09:52:55.214975+0000","modified":"2024-07-16T09:53:11.798619+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":14458,"mds_1":24261},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14458":{"gid":14458,"name":"cephfs.smithi156.rdibys","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.156:6827/1080176013","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":1080176013},{"type":"v1","addr":"172.21.15.156:6827","nonce":1080176013}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14468":{"gid":14468,"name":"cephfs.smithi156.evwwaz","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":2,"addr":"172.21.15.156:6829/560329502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":560329502},{"type":"v1","addr":"172.21.15.156:6829","nonce":560329502}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24261":{"gid":24261,"name":"cephfs.smithi176.dvvbud","rank":1,"incarnation":7,"state":"up:active","state_seq":4,"addr":"172.21.15.176:6825/1577019287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6824","nonce":1577019287},{"type":"v1","addr":"172.21.15.176:6825","nonce":1577019287}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24273":{"gid":24273,"name":"cephfs.smithi176.wahbsi","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":2,"addr":"172.21.15.176:6827/2196775412","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6826","nonce":2196775412},{"type":"v1","addr":"172.21.15.176:6827","nonce":2196775412}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":1},"id":1}]} 2024-07-16T09:53:26.300 INFO:teuthology.orchestra.run.smithi156.stderr:dumped fsmap epoch 16 2024-07-16T09:53:26.302 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:26.300+0000 7f2e0c1a5640 1 -- 172.21.15.156:0/2167350940 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f2df0067190 msgr2=0x7f2df0069640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:53:26.302 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:26.300+0000 7f2e0c1a5640 1 --2- 172.21.15.156:0/2167350940 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f2df0067190 0x7f2df0069640 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f2df4009f80 tx=0x7f2df4035040 comp rx=0 tx=0).stop 2024-07-16T09:53:26.302 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:26.300+0000 7f2e0c1a5640 1 -- 172.21.15.156:0/2167350940 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2e04102350 msgr2=0x7f2e04113100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:53:26.302 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:26.300+0000 7f2e0c1a5640 1 --2- 172.21.15.156:0/2167350940 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2e04102350 0x7f2e04113100 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f2e00002a50 tx=0x7f2e00004600 comp rx=0 tx=0).stop 2024-07-16T09:53:26.303 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:26.300+0000 7f2e0c1a5640 1 -- 172.21.15.156:0/2167350940 shutdown_connections 2024-07-16T09:53:26.303 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:26.300+0000 7f2e0c1a5640 1 --2- 172.21.15.156:0/2167350940 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f2df0067190 0x7f2df0069640 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:53:26.303 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:26.300+0000 7f2e0c1a5640 1 --2- 172.21.15.156:0/2167350940 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2e04102cf0 0x7f2e04113640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:53:26.303 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:26.300+0000 7f2e0c1a5640 1 --2- 172.21.15.156:0/2167350940 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2e04102350 0x7f2e04113100 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:53:26.303 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:26.300+0000 7f2e0c1a5640 1 -- 172.21.15.156:0/2167350940 >> 172.21.15.156:0/2167350940 conn(0x7f2e040f7f20 msgr2=0x7f2e040f8e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:53:26.303 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:26.300+0000 7f2e0c1a5640 1 -- 172.21.15.156:0/2167350940 shutdown_connections 2024-07-16T09:53:26.303 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:26.300+0000 7f2e0c1a5640 1 -- 172.21.15.156:0/2167350940 wait complete. 2024-07-16T09:53:26.343 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:53:26 smithi176 ceph-mon[36626]: pgmap v184: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-07-16T09:53:26.917 DEBUG:tasks.fs:fs fscid=1,name=cephfs state = {'epoch': 16, 'max_mds': 2, 'flags': 50} 2024-07-16T09:53:26.917 INFO:teuthology.run_tasks:Running task ceph-fuse... 2024-07-16T09:53:26.950 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2024-07-16T09:53:26.950 INFO:tasks.ceph_fuse:config is {'client.0': {}, 'client.1': {}} 2024-07-16T09:53:26.950 INFO:tasks.ceph_fuse:client.0 config is {} 2024-07-16T09:53:26.950 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-07-16T09:53:26.950 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-07-16T09:53:26.951 INFO:tasks.ceph_fuse:client.1 config is {} 2024-07-16T09:53:26.951 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-07-16T09:53:26.951 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-07-16T09:53:26.951 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-16T09:53:26.951 DEBUG:teuthology.orchestra.run.smithi156:> ip netns list 2024-07-16T09:53:26.968 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-16T09:53:26.968 DEBUG:teuthology.orchestra.run.smithi156:> sudo ip link delete ceph-brx 2024-07-16T09:53:27.036 INFO:teuthology.orchestra.run.smithi156.stderr:Cannot find device "ceph-brx" 2024-07-16T09:53:27.038 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-07-16T09:53:27.038 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-16T09:53:27.038 DEBUG:teuthology.orchestra.run.smithi176:> ip netns list 2024-07-16T09:53:27.059 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-16T09:53:27.059 DEBUG:teuthology.orchestra.run.smithi176:> sudo ip link delete ceph-brx 2024-07-16T09:53:27.133 INFO:teuthology.orchestra.run.smithi176.stderr:Cannot find device "ceph-brx" 2024-07-16T09:53:27.134 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-07-16T09:53:27.135 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2024-07-16T09:53:27.135 DEBUG:tasks.ceph_fuse:passing mntargs=[] 2024-07-16T09:53:27.135 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 26c91c26-4358-11ef-bcad-c7b262605968 -- ceph fs ls 2024-07-16T09:53:27.156 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:53:27 smithi156 ceph-mon[29302]: from='client.? 172.21.15.156:0/2167350940' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-16T09:53:27.343 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:53:27 smithi176 ceph-mon[36626]: from='client.? 172.21.15.156:0/2167350940' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-16T09:53:27.393 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/26c91c26-4358-11ef-bcad-c7b262605968/mon.smithi156/config 2024-07-16T09:53:28.343 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:53:28 smithi176 ceph-mon[36626]: pgmap v185: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-07-16T09:53:28.424 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:53:28 smithi156 ceph-mon[29302]: pgmap v185: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-07-16T09:53:28.965 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:28.962+0000 7f079b8d2640 1 -- 172.21.15.156:0/1463764687 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0794102cf0 msgr2=0x7f07940fc380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:53:28.965 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:28.962+0000 7f079b8d2640 1 --2- 172.21.15.156:0/1463764687 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0794102cf0 0x7f07940fc380 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f07840048d0 tx=0x7f0784009da0 comp rx=0 tx=0).stop 2024-07-16T09:53:28.965 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:28.962+0000 7f079b8d2640 1 -- 172.21.15.156:0/1463764687 shutdown_connections 2024-07-16T09:53:28.965 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:28.962+0000 7f079b8d2640 1 --2- 172.21.15.156:0/1463764687 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0794102cf0 0x7f07940fc380 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:53:28.966 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:28.962+0000 7f079b8d2640 1 --2- 172.21.15.156:0/1463764687 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0794102350 0x7f0794102720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:53:28.966 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:28.962+0000 7f079b8d2640 1 -- 172.21.15.156:0/1463764687 >> 172.21.15.156:0/1463764687 conn(0x7f07940f7f20 msgr2=0x7f07940fa330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:53:28.966 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:28.962+0000 7f079b8d2640 1 -- 172.21.15.156:0/1463764687 shutdown_connections 2024-07-16T09:53:28.966 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:28.962+0000 7f079b8d2640 1 -- 172.21.15.156:0/1463764687 wait complete. 2024-07-16T09:53:28.966 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:28.962+0000 7f079b8d2640 1 Processor -- start 2024-07-16T09:53:28.966 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:28.962+0000 7f079b8d2640 1 -- start start 2024-07-16T09:53:28.966 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:28.963+0000 7f079b8d2640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0794102350 0x7f0794113170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:53:28.966 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:28.963+0000 7f079b8d2640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0794102cf0 0x7f07941136b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:53:28.966 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:28.963+0000 7f079b8d2640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0794113d40 con 0x7f0794102350 2024-07-16T09:53:28.966 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:28.963+0000 7f079b8d2640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f079410d0f0 con 0x7f0794102cf0 2024-07-16T09:53:28.967 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:28.963+0000 7f079966e640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0794102350 0x7f0794113170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:53:28.967 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:28.963+0000 7f079966e640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0794102350 0x7f0794113170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:35896/0 (socket says 172.21.15.156:35896) 2024-07-16T09:53:28.967 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:28.963+0000 7f079966e640 1 -- 172.21.15.156:0/2443619649 learned_addr learned my addr 172.21.15.156:0/2443619649 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T09:53:28.967 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:28.963+0000 7f0798e6d640 1 --2- 172.21.15.156:0/2443619649 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0794102cf0 0x7f07941136b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:53:28.967 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:28.963+0000 7f079966e640 1 -- 172.21.15.156:0/2443619649 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0794102cf0 msgr2=0x7f07941136b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:53:28.967 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:28.963+0000 7f079966e640 1 --2- 172.21.15.156:0/2443619649 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0794102cf0 0x7f07941136b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:53:28.967 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:28.963+0000 7f079966e640 1 -- 172.21.15.156:0/2443619649 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0784005040 con 0x7f0794102350 2024-07-16T09:53:28.967 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:28.963+0000 7f0798e6d640 1 --2- 172.21.15.156:0/2443619649 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0794102cf0 0x7f07941136b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-16T09:53:28.967 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:28.963+0000 7f079966e640 1 --2- 172.21.15.156:0/2443619649 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0794102350 0x7f0794113170 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f079000b790 tx=0x7f07900091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:53:28.967 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:28.963+0000 7f0789ffb640 1 -- 172.21.15.156:0/2443619649 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f079000c700 con 0x7f0794102350 2024-07-16T09:53:28.968 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:28.963+0000 7f0789ffb640 1 -- 172.21.15.156:0/2443619649 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f079000cd30 con 0x7f0794102350 2024-07-16T09:53:28.968 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:28.963+0000 7f079b8d2640 1 -- 172.21.15.156:0/2443619649 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f079410d3d0 con 0x7f0794102350 2024-07-16T09:53:28.968 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:28.963+0000 7f0789ffb640 1 -- 172.21.15.156:0/2443619649 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0790013370 con 0x7f0794102350 2024-07-16T09:53:28.968 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:28.964+0000 7f079b8d2640 1 -- 172.21.15.156:0/2443619649 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f079410d8f0 con 0x7f0794102350 2024-07-16T09:53:28.969 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:28.965+0000 7f079b8d2640 1 -- 172.21.15.156:0/2443619649 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07940fdac0 con 0x7f0794102350 2024-07-16T09:53:28.969 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:28.965+0000 7f0789ffb640 1 -- 172.21.15.156:0/2443619649 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 19) v1 ==== 85127+0+0 (secure 0 0 0) 0x7f0790012070 con 0x7f0794102350 2024-07-16T09:53:28.969 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:28.965+0000 7f0789ffb640 1 --2- 172.21.15.156:0/2443619649 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f0780067190 0x7f0780069640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:53:28.969 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:28.965+0000 7f0789ffb640 1 -- 172.21.15.156:0/2443619649 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f0790084710 con 0x7f0794102350 2024-07-16T09:53:28.969 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:28.965+0000 7f0798e6d640 1 --2- 172.21.15.156:0/2443619649 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f0780067190 0x7f0780069640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:53:28.969 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:28.966+0000 7f0798e6d640 1 --2- 172.21.15.156:0/2443619649 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f0780067190 0x7f0780069640 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f079410ea30 tx=0x7f0784035040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:53:28.971 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:28.970+0000 7f0789ffb640 1 -- 172.21.15.156:0/2443619649 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f07900519b0 con 0x7f0794102350 2024-07-16T09:53:29.173 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:29.171+0000 7f079b8d2640 1 -- 172.21.15.156:0/2443619649 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f079404b560 con 0x7f0794102350 2024-07-16T09:53:29.173 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:29.171+0000 7f0789ffb640 1 -- 172.21.15.156:0/2443619649 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v16) v1 ==== 53+0+83 (secure 0 0 0) 0x7f0790054fd0 con 0x7f0794102350 2024-07-16T09:53:29.173 INFO:teuthology.orchestra.run.smithi156.stdout:name: cephfs, metadata pool: cephfs.cephfs.meta, data pools: [cephfs.cephfs.data ] 2024-07-16T09:53:29.175 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:29.173+0000 7f079b8d2640 1 -- 172.21.15.156:0/2443619649 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f0780067190 msgr2=0x7f0780069640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:53:29.175 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:29.173+0000 7f079b8d2640 1 --2- 172.21.15.156:0/2443619649 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f0780067190 0x7f0780069640 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f079410ea30 tx=0x7f0784035040 comp rx=0 tx=0).stop 2024-07-16T09:53:29.175 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:29.173+0000 7f079b8d2640 1 -- 172.21.15.156:0/2443619649 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0794102350 msgr2=0x7f0794113170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:53:29.176 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:29.173+0000 7f079b8d2640 1 --2- 172.21.15.156:0/2443619649 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0794102350 0x7f0794113170 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f079000b790 tx=0x7f07900091d0 comp rx=0 tx=0).stop 2024-07-16T09:53:29.176 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:29.174+0000 7f079b8d2640 1 -- 172.21.15.156:0/2443619649 shutdown_connections 2024-07-16T09:53:29.176 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:29.174+0000 7f079b8d2640 1 --2- 172.21.15.156:0/2443619649 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f0780067190 0x7f0780069640 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:53:29.176 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:29.174+0000 7f079b8d2640 1 --2- 172.21.15.156:0/2443619649 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0794102cf0 0x7f07941136b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:53:29.176 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:29.174+0000 7f079b8d2640 1 --2- 172.21.15.156:0/2443619649 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0794102350 0x7f0794113170 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:53:29.176 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:29.174+0000 7f079b8d2640 1 -- 172.21.15.156:0/2443619649 >> 172.21.15.156:0/2443619649 conn(0x7f07940f7f20 msgr2=0x7f07940f8e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:53:29.176 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:29.174+0000 7f079b8d2640 1 -- 172.21.15.156:0/2443619649 shutdown_connections 2024-07-16T09:53:29.176 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:29.174+0000 7f079b8d2640 1 -- 172.21.15.156:0/2443619649 wait complete. 2024-07-16T09:53:29.782 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-07-16T09:53:29.782 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-07-16T09:53:29.782 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi156.front.sepia.ceph.com 2024-07-16T09:53:29.783 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-07-16T09:53:29.783 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-07-16T09:53:29.783 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-07-16T09:53:29.783 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-07-16T09:53:29.783 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-07-16T09:53:29.783 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-07-16T09:53:29.783 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-16T09:53:29.783 DEBUG:teuthology.orchestra.run.smithi156:> ip addr 2024-07-16T09:53:29.839 INFO:teuthology.orchestra.run.smithi156.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-07-16T09:53:29.840 INFO:teuthology.orchestra.run.smithi156.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-07-16T09:53:29.840 INFO:teuthology.orchestra.run.smithi156.stdout: inet 127.0.0.1/8 scope host lo 2024-07-16T09:53:29.840 INFO:teuthology.orchestra.run.smithi156.stdout: valid_lft forever preferred_lft forever 2024-07-16T09:53:29.840 INFO:teuthology.orchestra.run.smithi156.stdout: inet6 ::1/128 scope host 2024-07-16T09:53:29.840 INFO:teuthology.orchestra.run.smithi156.stdout: valid_lft forever preferred_lft forever 2024-07-16T09:53:29.840 INFO:teuthology.orchestra.run.smithi156.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-07-16T09:53:29.840 INFO:teuthology.orchestra.run.smithi156.stdout: link/ether 0c:c4:7a:da:6a:c0 brd ff:ff:ff:ff:ff:ff 2024-07-16T09:53:29.840 INFO:teuthology.orchestra.run.smithi156.stdout: altname enp7s0f0 2024-07-16T09:53:29.840 INFO:teuthology.orchestra.run.smithi156.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-07-16T09:53:29.840 INFO:teuthology.orchestra.run.smithi156.stdout: link/ether 0c:c4:7a:da:6a:c1 brd ff:ff:ff:ff:ff:ff 2024-07-16T09:53:29.840 INFO:teuthology.orchestra.run.smithi156.stdout: altname enp7s0f1 2024-07-16T09:53:29.840 INFO:teuthology.orchestra.run.smithi156.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-07-16T09:53:29.840 INFO:teuthology.orchestra.run.smithi156.stdout: link/ether 0c:c4:7a:88:6d:fc brd ff:ff:ff:ff:ff:ff 2024-07-16T09:53:29.841 INFO:teuthology.orchestra.run.smithi156.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-07-16T09:53:29.841 INFO:teuthology.orchestra.run.smithi156.stdout: link/ether 0c:c4:7a:88:6d:fd brd ff:ff:ff:ff:ff:ff 2024-07-16T09:53:29.841 INFO:teuthology.orchestra.run.smithi156.stdout: inet 172.21.15.156/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-07-16T09:53:29.841 INFO:teuthology.orchestra.run.smithi156.stdout: valid_lft 42319sec preferred_lft 42319sec 2024-07-16T09:53:29.841 INFO:teuthology.orchestra.run.smithi156.stdout: inet6 fe80::ec4:7aff:fe88:6dfd/64 scope link 2024-07-16T09:53:29.841 INFO:teuthology.orchestra.run.smithi156.stdout: valid_lft forever preferred_lft forever 2024-07-16T09:53:29.841 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-07-16T09:53:29.842 DEBUG:teuthology.orchestra.run.smithi156:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-07-16T09:53:29.842 DEBUG:teuthology.orchestra.run.smithi156:> set -e 2024-07-16T09:53:29.842 DEBUG:teuthology.orchestra.run.smithi156:> sudo ip link add name ceph-brx type bridge 2024-07-16T09:53:29.842 DEBUG:teuthology.orchestra.run.smithi156:> sudo ip addr flush dev ceph-brx 2024-07-16T09:53:29.842 DEBUG:teuthology.orchestra.run.smithi156:> sudo ip link set ceph-brx up 2024-07-16T09:53:29.842 DEBUG:teuthology.orchestra.run.smithi156:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-07-16T09:53:29.842 DEBUG:teuthology.orchestra.run.smithi156:> ') 2024-07-16T09:53:29.929 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:29 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-07-16T09:53:30.050 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-07-16T09:53:30.056 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-16T09:53:30.056 DEBUG:teuthology.orchestra.run.smithi156:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-07-16T09:53:30.081 INFO:teuthology.orchestra.run.smithi156.stdout:1 2024-07-16T09:53:30.083 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-16T09:53:30.083 DEBUG:teuthology.orchestra.run.smithi156:> route 2024-07-16T09:53:30.157 INFO:teuthology.orchestra.run.smithi156.stdout:Kernel IP routing table 2024-07-16T09:53:30.157 INFO:teuthology.orchestra.run.smithi156.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-07-16T09:53:30.157 INFO:teuthology.orchestra.run.smithi156.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-07-16T09:53:30.157 INFO:teuthology.orchestra.run.smithi156.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-07-16T09:53:30.157 INFO:teuthology.orchestra.run.smithi156.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-07-16T09:53:30.157 DEBUG:teuthology.orchestra.run.smithi156:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-07-16T09:53:30.157 DEBUG:teuthology.orchestra.run.smithi156:> set -e 2024-07-16T09:53:30.158 DEBUG:teuthology.orchestra.run.smithi156:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-07-16T09:53:30.158 DEBUG:teuthology.orchestra.run.smithi156:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-07-16T09:53:30.158 DEBUG:teuthology.orchestra.run.smithi156:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-07-16T09:53:30.158 DEBUG:teuthology.orchestra.run.smithi156:> ') 2024-07-16T09:53:30.246 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:30 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-07-16T09:53:30.343 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:53:30 smithi176 ceph-mon[36626]: pgmap v186: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T09:53:30.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:53:30 smithi176 ceph-mon[36626]: from='client.? 172.21.15.156:0/2443619649' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-07-16T09:53:30.363 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:53:30 smithi156 ceph-mon[29302]: pgmap v186: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T09:53:30.363 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:53:30 smithi156 ceph-mon[29302]: from='client.? 172.21.15.156:0/2443619649' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-07-16T09:53:30.470 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-07-16T09:53:30.477 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-16T09:53:30.477 DEBUG:teuthology.orchestra.run.smithi156:> ip netns list 2024-07-16T09:53:30.493 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-16T09:53:30.493 DEBUG:teuthology.orchestra.run.smithi156:> ip netns list-id 2024-07-16T09:53:30.548 DEBUG:teuthology.orchestra.run.smithi156:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-07-16T09:53:30.549 DEBUG:teuthology.orchestra.run.smithi156:> set -e 2024-07-16T09:53:30.549 DEBUG:teuthology.orchestra.run.smithi156:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-07-16T09:53:30.549 DEBUG:teuthology.orchestra.run.smithi156:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-07-16T09:53:30.549 DEBUG:teuthology.orchestra.run.smithi156:> ') 2024-07-16T09:53:30.637 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:30 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-07-16T09:53:30.661 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-07-16T09:53:30.667 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-07-16T09:53:30.667 DEBUG:teuthology.orchestra.run.smithi156:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-07-16T09:53:30.667 DEBUG:teuthology.orchestra.run.smithi156:> set -e 2024-07-16T09:53:30.667 DEBUG:teuthology.orchestra.run.smithi156:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-07-16T09:53:30.667 DEBUG:teuthology.orchestra.run.smithi156:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-07-16T09:53:30.667 DEBUG:teuthology.orchestra.run.smithi156:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-07-16T09:53:30.667 DEBUG:teuthology.orchestra.run.smithi156:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-07-16T09:53:30.668 DEBUG:teuthology.orchestra.run.smithi156:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-07-16T09:53:30.668 DEBUG:teuthology.orchestra.run.smithi156:> ') 2024-07-16T09:53:30.716 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:30 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-07-16T09:53:30.787 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-07-16T09:53:30.793 DEBUG:teuthology.orchestra.run.smithi156:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-07-16T09:53:30.794 DEBUG:teuthology.orchestra.run.smithi156:> set -e 2024-07-16T09:53:30.794 DEBUG:teuthology.orchestra.run.smithi156:> sudo ip link set brx.0 up 2024-07-16T09:53:30.794 DEBUG:teuthology.orchestra.run.smithi156:> sudo ip link set dev brx.0 master ceph-brx 2024-07-16T09:53:30.794 DEBUG:teuthology.orchestra.run.smithi156:> ') 2024-07-16T09:53:30.842 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:30 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-07-16T09:53:30.865 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-07-16T09:53:30.871 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-07-16T09:53:30.871 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-07-16T09:53:30.871 DEBUG:teuthology.orchestra.run.smithi156:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-07-16T09:53:30.886 INFO:teuthology.orchestra.run.smithi156.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-07-16T09:53:30.887 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-07-16T09:53:30.887 DEBUG:teuthology.orchestra.run.smithi156:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-07-16T09:53:30.949 DEBUG:teuthology.orchestra.run.smithi156:> sudo modprobe fuse 2024-07-16T09:53:31.015 DEBUG:teuthology.orchestra.run.smithi156:> cat /proc/self/mounts | awk '{print $2}' 2024-07-16T09:53:31.071 INFO:teuthology.orchestra.run.smithi156.stdout:/proc 2024-07-16T09:53:31.071 INFO:teuthology.orchestra.run.smithi156.stdout:/sys 2024-07-16T09:53:31.072 INFO:teuthology.orchestra.run.smithi156.stdout:/dev 2024-07-16T09:53:31.072 INFO:teuthology.orchestra.run.smithi156.stdout:/sys/kernel/security 2024-07-16T09:53:31.072 INFO:teuthology.orchestra.run.smithi156.stdout:/dev/shm 2024-07-16T09:53:31.072 INFO:teuthology.orchestra.run.smithi156.stdout:/dev/pts 2024-07-16T09:53:31.072 INFO:teuthology.orchestra.run.smithi156.stdout:/run 2024-07-16T09:53:31.072 INFO:teuthology.orchestra.run.smithi156.stdout:/sys/fs/cgroup 2024-07-16T09:53:31.072 INFO:teuthology.orchestra.run.smithi156.stdout:/sys/fs/pstore 2024-07-16T09:53:31.072 INFO:teuthology.orchestra.run.smithi156.stdout:/sys/fs/bpf 2024-07-16T09:53:31.072 INFO:teuthology.orchestra.run.smithi156.stdout:/ 2024-07-16T09:53:31.072 INFO:teuthology.orchestra.run.smithi156.stdout:/sys/fs/selinux 2024-07-16T09:53:31.072 INFO:teuthology.orchestra.run.smithi156.stdout:/proc/sys/fs/binfmt_misc 2024-07-16T09:53:31.072 INFO:teuthology.orchestra.run.smithi156.stdout:/dev/mqueue 2024-07-16T09:53:31.072 INFO:teuthology.orchestra.run.smithi156.stdout:/sys/kernel/debug 2024-07-16T09:53:31.073 INFO:teuthology.orchestra.run.smithi156.stdout:/dev/hugepages 2024-07-16T09:53:31.073 INFO:teuthology.orchestra.run.smithi156.stdout:/sys/kernel/tracing 2024-07-16T09:53:31.073 INFO:teuthology.orchestra.run.smithi156.stdout:/sys/fs/fuse/connections 2024-07-16T09:53:31.073 INFO:teuthology.orchestra.run.smithi156.stdout:/run/credentials/systemd-sysctl.service 2024-07-16T09:53:31.073 INFO:teuthology.orchestra.run.smithi156.stdout:/sys/kernel/config 2024-07-16T09:53:31.073 INFO:teuthology.orchestra.run.smithi156.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-07-16T09:53:31.073 INFO:teuthology.orchestra.run.smithi156.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-07-16T09:53:31.073 INFO:teuthology.orchestra.run.smithi156.stdout:/var/lib/nfs/rpc_pipefs 2024-07-16T09:53:31.073 INFO:teuthology.orchestra.run.smithi156.stdout:/run/user/1000 2024-07-16T09:53:31.073 INFO:teuthology.orchestra.run.smithi156.stdout:/var/lib/ceph 2024-07-16T09:53:31.073 INFO:teuthology.orchestra.run.smithi156.stdout:/var/lib/containers/storage/overlay 2024-07-16T09:53:31.073 INFO:teuthology.orchestra.run.smithi156.stdout:/var/lib/containers/storage/overlay/4fb18c686d9a695dedce12a4d3b751a681577cc6109e15351f54d87b06cfa186/merged 2024-07-16T09:53:31.073 INFO:teuthology.orchestra.run.smithi156.stdout:/var/lib/containers/storage/overlay/9f0e2751b07fd96875d655b84546e6f731454910514617b48e933a23ab8623d6/merged 2024-07-16T09:53:31.073 INFO:teuthology.orchestra.run.smithi156.stdout:/run/user/0 2024-07-16T09:53:31.074 INFO:teuthology.orchestra.run.smithi156.stdout:/proc/sys/fs/binfmt_misc 2024-07-16T09:53:31.074 INFO:teuthology.orchestra.run.smithi156.stdout:/var/lib/containers/storage/overlay/a201aea9d77b219bb98591a63ec23f6c7c30c7adc4396248d2eabdd960a994b7/merged 2024-07-16T09:53:31.074 INFO:teuthology.orchestra.run.smithi156.stdout:/var/lib/containers/storage/overlay/c05530c347dc484ff55f0253e505f0f61e9e8f738c918607fab79638d750d05c/merged 2024-07-16T09:53:31.074 INFO:teuthology.orchestra.run.smithi156.stdout:/var/lib/containers/storage/overlay/d6bb9e68a0172f9a49f095808efc341f0e2b0d8e14f70c26c6f39eb0b8db2489/merged 2024-07-16T09:53:31.074 INFO:teuthology.orchestra.run.smithi156.stdout:/var/lib/containers/storage/overlay/676c434e92ebd87ef0535c2e78983370fb919047621e1abb53d51265732e5de1/merged 2024-07-16T09:53:31.074 INFO:teuthology.orchestra.run.smithi156.stdout:/var/lib/containers/storage/overlay/7875a7bd272e118625aee96ad43fb5946e755ef9b5d67bceb206c000068e04be/merged 2024-07-16T09:53:31.074 INFO:teuthology.orchestra.run.smithi156.stdout:/var/lib/containers/storage/overlay/e873da0b0ab3cb0e96645ad36ae064e39aa85ce33414849bdca699714f82f765/merged 2024-07-16T09:53:31.074 INFO:teuthology.orchestra.run.smithi156.stdout:/var/lib/containers/storage/overlay/1ec79ba9622759fb96e6e9b377f3cffe6fb81a15ae707e8cce7a628af1367774/merged 2024-07-16T09:53:31.074 INFO:teuthology.orchestra.run.smithi156.stdout:/var/lib/containers/storage/overlay/b2881cb8b76d2cfbacbb5ad566e6069252481d50e135df30232054c0a62a0def/merged 2024-07-16T09:53:31.074 INFO:teuthology.orchestra.run.smithi156.stdout:/var/lib/containers/storage/overlay/7bdcab8b28ddc3d8932dd2a33c7328cf15642a8cd4f43f2cd1db5771c07bb3ad/merged 2024-07-16T09:53:31.074 INFO:teuthology.orchestra.run.smithi156.stdout:/var/lib/containers/storage/overlay/9a5c1804bf3a1b330b73fc709106dcc9927f80e5b13b037fc0348497d794c333/merged 2024-07-16T09:53:31.074 INFO:teuthology.orchestra.run.smithi156.stdout:/run/netns 2024-07-16T09:53:31.074 INFO:teuthology.orchestra.run.smithi156.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-07-16T09:53:31.075 INFO:teuthology.orchestra.run.smithi156.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-07-16T09:53:31.075 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-16T09:53:31.075 DEBUG:teuthology.orchestra.run.smithi156:> ls /sys/fs/fuse/connections 2024-07-16T09:53:31.128 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-07-16T09:53:31.128 DEBUG:teuthology.orchestra.run.smithi156:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-07-16T09:53:31.170 DEBUG:teuthology.orchestra.run.smithi156:> sudo modprobe fuse 2024-07-16T09:53:31.194 DEBUG:teuthology.orchestra.run.smithi156:> cat /proc/self/mounts | awk '{print $2}' 2024-07-16T09:53:31.251 INFO:teuthology.orchestra.run.smithi156.stdout:/proc 2024-07-16T09:53:31.251 INFO:teuthology.orchestra.run.smithi156.stdout:/sys 2024-07-16T09:53:31.251 INFO:teuthology.orchestra.run.smithi156.stdout:/dev 2024-07-16T09:53:31.251 INFO:teuthology.orchestra.run.smithi156.stdout:/sys/kernel/security 2024-07-16T09:53:31.251 INFO:teuthology.orchestra.run.smithi156.stdout:/dev/shm 2024-07-16T09:53:31.251 INFO:teuthology.orchestra.run.smithi156.stdout:/dev/pts 2024-07-16T09:53:31.251 INFO:teuthology.orchestra.run.smithi156.stdout:/run 2024-07-16T09:53:31.251 INFO:teuthology.orchestra.run.smithi156.stdout:/sys/fs/cgroup 2024-07-16T09:53:31.251 INFO:teuthology.orchestra.run.smithi156.stdout:/sys/fs/pstore 2024-07-16T09:53:31.252 INFO:teuthology.orchestra.run.smithi156.stdout:/sys/fs/bpf 2024-07-16T09:53:31.252 INFO:teuthology.orchestra.run.smithi156.stdout:/ 2024-07-16T09:53:31.252 INFO:teuthology.orchestra.run.smithi156.stdout:/sys/fs/selinux 2024-07-16T09:53:31.252 INFO:teuthology.orchestra.run.smithi156.stdout:/proc/sys/fs/binfmt_misc 2024-07-16T09:53:31.252 INFO:teuthology.orchestra.run.smithi156.stdout:/dev/mqueue 2024-07-16T09:53:31.252 INFO:teuthology.orchestra.run.smithi156.stdout:/sys/kernel/debug 2024-07-16T09:53:31.252 INFO:teuthology.orchestra.run.smithi156.stdout:/dev/hugepages 2024-07-16T09:53:31.252 INFO:teuthology.orchestra.run.smithi156.stdout:/sys/kernel/tracing 2024-07-16T09:53:31.252 INFO:teuthology.orchestra.run.smithi156.stdout:/sys/fs/fuse/connections 2024-07-16T09:53:31.252 INFO:teuthology.orchestra.run.smithi156.stdout:/run/credentials/systemd-sysctl.service 2024-07-16T09:53:31.252 INFO:teuthology.orchestra.run.smithi156.stdout:/sys/kernel/config 2024-07-16T09:53:31.252 INFO:teuthology.orchestra.run.smithi156.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-07-16T09:53:31.253 INFO:teuthology.orchestra.run.smithi156.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-07-16T09:53:31.253 INFO:teuthology.orchestra.run.smithi156.stdout:/var/lib/nfs/rpc_pipefs 2024-07-16T09:53:31.253 INFO:teuthology.orchestra.run.smithi156.stdout:/run/user/1000 2024-07-16T09:53:31.253 INFO:teuthology.orchestra.run.smithi156.stdout:/var/lib/ceph 2024-07-16T09:53:31.253 INFO:teuthology.orchestra.run.smithi156.stdout:/var/lib/containers/storage/overlay 2024-07-16T09:53:31.253 INFO:teuthology.orchestra.run.smithi156.stdout:/var/lib/containers/storage/overlay/4fb18c686d9a695dedce12a4d3b751a681577cc6109e15351f54d87b06cfa186/merged 2024-07-16T09:53:31.253 INFO:teuthology.orchestra.run.smithi156.stdout:/var/lib/containers/storage/overlay/9f0e2751b07fd96875d655b84546e6f731454910514617b48e933a23ab8623d6/merged 2024-07-16T09:53:31.253 INFO:teuthology.orchestra.run.smithi156.stdout:/run/user/0 2024-07-16T09:53:31.253 INFO:teuthology.orchestra.run.smithi156.stdout:/proc/sys/fs/binfmt_misc 2024-07-16T09:53:31.253 INFO:teuthology.orchestra.run.smithi156.stdout:/var/lib/containers/storage/overlay/a201aea9d77b219bb98591a63ec23f6c7c30c7adc4396248d2eabdd960a994b7/merged 2024-07-16T09:53:31.253 INFO:teuthology.orchestra.run.smithi156.stdout:/var/lib/containers/storage/overlay/c05530c347dc484ff55f0253e505f0f61e9e8f738c918607fab79638d750d05c/merged 2024-07-16T09:53:31.253 INFO:teuthology.orchestra.run.smithi156.stdout:/var/lib/containers/storage/overlay/d6bb9e68a0172f9a49f095808efc341f0e2b0d8e14f70c26c6f39eb0b8db2489/merged 2024-07-16T09:53:31.253 INFO:teuthology.orchestra.run.smithi156.stdout:/var/lib/containers/storage/overlay/676c434e92ebd87ef0535c2e78983370fb919047621e1abb53d51265732e5de1/merged 2024-07-16T09:53:31.254 INFO:teuthology.orchestra.run.smithi156.stdout:/var/lib/containers/storage/overlay/7875a7bd272e118625aee96ad43fb5946e755ef9b5d67bceb206c000068e04be/merged 2024-07-16T09:53:31.254 INFO:teuthology.orchestra.run.smithi156.stdout:/var/lib/containers/storage/overlay/e873da0b0ab3cb0e96645ad36ae064e39aa85ce33414849bdca699714f82f765/merged 2024-07-16T09:53:31.254 INFO:teuthology.orchestra.run.smithi156.stdout:/var/lib/containers/storage/overlay/1ec79ba9622759fb96e6e9b377f3cffe6fb81a15ae707e8cce7a628af1367774/merged 2024-07-16T09:53:31.254 INFO:teuthology.orchestra.run.smithi156.stdout:/var/lib/containers/storage/overlay/b2881cb8b76d2cfbacbb5ad566e6069252481d50e135df30232054c0a62a0def/merged 2024-07-16T09:53:31.254 INFO:teuthology.orchestra.run.smithi156.stdout:/var/lib/containers/storage/overlay/7bdcab8b28ddc3d8932dd2a33c7328cf15642a8cd4f43f2cd1db5771c07bb3ad/merged 2024-07-16T09:53:31.254 INFO:teuthology.orchestra.run.smithi156.stdout:/var/lib/containers/storage/overlay/9a5c1804bf3a1b330b73fc709106dcc9927f80e5b13b037fc0348497d794c333/merged 2024-07-16T09:53:31.254 INFO:teuthology.orchestra.run.smithi156.stdout:/run/netns 2024-07-16T09:53:31.254 INFO:teuthology.orchestra.run.smithi156.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-07-16T09:53:31.254 INFO:teuthology.orchestra.run.smithi156.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-07-16T09:53:31.254 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-16T09:53:31.254 DEBUG:teuthology.orchestra.run.smithi156:> ls /sys/fs/fuse/connections 2024-07-16T09:53:31.282 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi156.stderr:2024-07-16T09:53:31.280+0000 7fe5340c5300 -1 init, newargv = 0x561a2ae57700 newargc=15 2024-07-16T09:53:31.282 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi156.stderr:ceph-fuse[76406]: starting ceph client 2024-07-16T09:53:31.289 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi156.stderr:ceph-fuse[76406]: starting fuse 2024-07-16T09:53:31.298 INFO:teuthology.orchestra.run.smithi156.stdout:152 2024-07-16T09:53:31.298 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [152] 2024-07-16T09:53:31.298 DEBUG:teuthology.orchestra.run.smithi156:> sudo stdin-killer -- python3 -c ' 2024-07-16T09:53:31.298 DEBUG:teuthology.orchestra.run.smithi156:> import glob 2024-07-16T09:53:31.298 DEBUG:teuthology.orchestra.run.smithi156:> import re 2024-07-16T09:53:31.299 DEBUG:teuthology.orchestra.run.smithi156:> import os 2024-07-16T09:53:31.299 DEBUG:teuthology.orchestra.run.smithi156:> import subprocess 2024-07-16T09:53:31.299 DEBUG:teuthology.orchestra.run.smithi156:> 2024-07-16T09:53:31.299 DEBUG:teuthology.orchestra.run.smithi156:> def _find_admin_socket(client_name): 2024-07-16T09:53:31.299 DEBUG:teuthology.orchestra.run.smithi156:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-07-16T09:53:31.299 DEBUG:teuthology.orchestra.run.smithi156:> files = glob.glob(asok_path) 2024-07-16T09:53:31.299 DEBUG:teuthology.orchestra.run.smithi156:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-07-16T09:53:31.299 DEBUG:teuthology.orchestra.run.smithi156:> 2024-07-16T09:53:31.299 DEBUG:teuthology.orchestra.run.smithi156:> # Given a non-glob path, it better be there 2024-07-16T09:53:31.299 DEBUG:teuthology.orchestra.run.smithi156:> if "*" not in asok_path: 2024-07-16T09:53:31.299 DEBUG:teuthology.orchestra.run.smithi156:> assert(len(files) == 1) 2024-07-16T09:53:31.299 DEBUG:teuthology.orchestra.run.smithi156:> return files[0] 2024-07-16T09:53:31.299 DEBUG:teuthology.orchestra.run.smithi156:> 2024-07-16T09:53:31.299 DEBUG:teuthology.orchestra.run.smithi156:> for f in files: 2024-07-16T09:53:31.299 DEBUG:teuthology.orchestra.run.smithi156:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-07-16T09:53:31.300 DEBUG:teuthology.orchestra.run.smithi156:> if os.path.exists("/proc/{0}".format(pid)): 2024-07-16T09:53:31.300 DEBUG:teuthology.orchestra.run.smithi156:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-07-16T09:53:31.300 DEBUG:teuthology.orchestra.run.smithi156:> contents = proc_f.read() 2024-07-16T09:53:31.300 DEBUG:teuthology.orchestra.run.smithi156:> if mountpoint in contents: 2024-07-16T09:53:31.300 DEBUG:teuthology.orchestra.run.smithi156:> return f 2024-07-16T09:53:31.300 DEBUG:teuthology.orchestra.run.smithi156:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-07-16T09:53:31.300 DEBUG:teuthology.orchestra.run.smithi156:> 2024-07-16T09:53:31.300 DEBUG:teuthology.orchestra.run.smithi156:> print(_find_admin_socket("client.0")) 2024-07-16T09:53:31.300 DEBUG:teuthology.orchestra.run.smithi156:> ' 2024-07-16T09:53:31.416 INFO:teuthology.orchestra.run.smithi156.stdout:/var/run/ceph/ceph-client.0.76406.asok 2024-07-16T09:53:31.419 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:31 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-07-16T09:53:31.426 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.76406.asok 2024-07-16T09:53:31.427 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-16T09:53:31.427 DEBUG:teuthology.orchestra.run.smithi156:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.76406.asok status 2024-07-16T09:53:31.558 INFO:teuthology.orchestra.run.smithi156.stdout:{ 2024-07-16T09:53:31.559 INFO:teuthology.orchestra.run.smithi156.stdout: "metadata": { 2024-07-16T09:53:31.559 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph_sha1": "e6231b04b52ebf3b9b31803282dd3074439e51ac", 2024-07-16T09:53:31.559 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph_version": "ceph version 17.2.7-1104-ge6231b04 (e6231b04b52ebf3b9b31803282dd3074439e51ac) quincy (stable)", 2024-07-16T09:53:31.559 INFO:teuthology.orchestra.run.smithi156.stdout: "entity_id": "0", 2024-07-16T09:53:31.559 INFO:teuthology.orchestra.run.smithi156.stdout: "hostname": "smithi156", 2024-07-16T09:53:31.559 INFO:teuthology.orchestra.run.smithi156.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-07-16T09:53:31.559 INFO:teuthology.orchestra.run.smithi156.stdout: "pid": "76406", 2024-07-16T09:53:31.559 INFO:teuthology.orchestra.run.smithi156.stdout: "root": "/" 2024-07-16T09:53:31.559 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-07-16T09:53:31.559 INFO:teuthology.orchestra.run.smithi156.stdout: "dentry_count": 0, 2024-07-16T09:53:31.559 INFO:teuthology.orchestra.run.smithi156.stdout: "dentry_pinned_count": 0, 2024-07-16T09:53:31.560 INFO:teuthology.orchestra.run.smithi156.stdout: "id": 14500, 2024-07-16T09:53:31.560 INFO:teuthology.orchestra.run.smithi156.stdout: "inst": { 2024-07-16T09:53:31.560 INFO:teuthology.orchestra.run.smithi156.stdout: "name": { 2024-07-16T09:53:31.560 INFO:teuthology.orchestra.run.smithi156.stdout: "type": "client", 2024-07-16T09:53:31.560 INFO:teuthology.orchestra.run.smithi156.stdout: "num": 14500 2024-07-16T09:53:31.560 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-07-16T09:53:31.560 INFO:teuthology.orchestra.run.smithi156.stdout: "addr": { 2024-07-16T09:53:31.560 INFO:teuthology.orchestra.run.smithi156.stdout: "type": "v1", 2024-07-16T09:53:31.560 INFO:teuthology.orchestra.run.smithi156.stdout: "addr": "192.168.0.1:0", 2024-07-16T09:53:31.560 INFO:teuthology.orchestra.run.smithi156.stdout: "nonce": 893267894 2024-07-16T09:53:31.560 INFO:teuthology.orchestra.run.smithi156.stdout: } 2024-07-16T09:53:31.560 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-07-16T09:53:31.561 INFO:teuthology.orchestra.run.smithi156.stdout: "addr": { 2024-07-16T09:53:31.561 INFO:teuthology.orchestra.run.smithi156.stdout: "type": "v1", 2024-07-16T09:53:31.561 INFO:teuthology.orchestra.run.smithi156.stdout: "addr": "192.168.0.1:0", 2024-07-16T09:53:31.561 INFO:teuthology.orchestra.run.smithi156.stdout: "nonce": 893267894 2024-07-16T09:53:31.561 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-07-16T09:53:31.561 INFO:teuthology.orchestra.run.smithi156.stdout: "inst_str": "client.14500 192.168.0.1:0/893267894", 2024-07-16T09:53:31.561 INFO:teuthology.orchestra.run.smithi156.stdout: "addr_str": "192.168.0.1:0/893267894", 2024-07-16T09:53:31.561 INFO:teuthology.orchestra.run.smithi156.stdout: "inode_count": 1, 2024-07-16T09:53:31.561 INFO:teuthology.orchestra.run.smithi156.stdout: "mds_epoch": 16, 2024-07-16T09:53:31.561 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_epoch": 46, 2024-07-16T09:53:31.561 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_epoch_barrier": 0, 2024-07-16T09:53:31.561 INFO:teuthology.orchestra.run.smithi156.stdout: "blocklisted": false, 2024-07-16T09:53:31.561 INFO:teuthology.orchestra.run.smithi156.stdout: "fs_name": "cephfs" 2024-07-16T09:53:31.561 INFO:teuthology.orchestra.run.smithi156.stdout:} 2024-07-16T09:53:31.567 DEBUG:tasks.ceph_fuse:passing mntargs=[] 2024-07-16T09:53:31.567 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 26c91c26-4358-11ef-bcad-c7b262605968 -- ceph fs ls 2024-07-16T09:53:31.823 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/26c91c26-4358-11ef-bcad-c7b262605968/mon.smithi156/config 2024-07-16T09:53:32.205 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:53:32 smithi156 ceph-mon[29302]: pgmap v187: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T09:53:32.343 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:53:32 smithi176 ceph-mon[36626]: pgmap v187: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T09:53:33.257 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:33.253+0000 7f4a59766640 1 -- 172.21.15.156:0/3383329343 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4a540fbf30 msgr2=0x7f4a540fce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:53:33.258 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:33.253+0000 7f4a59766640 1 --2- 172.21.15.156:0/3383329343 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4a540fbf30 0x7f4a540fce30 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f4a3c0048d0 tx=0x7f4a3c009da0 comp rx=0 tx=0).stop 2024-07-16T09:53:33.258 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:33.253+0000 7f4a59766640 1 -- 172.21.15.156:0/3383329343 shutdown_connections 2024-07-16T09:53:33.258 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:33.253+0000 7f4a59766640 1 --2- 172.21.15.156:0/3383329343 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4a540fbf30 0x7f4a540fce30 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:53:33.258 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:33.253+0000 7f4a59766640 1 --2- 172.21.15.156:0/3383329343 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4a540fb620 0x7f4a540fb9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:53:33.258 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:33.253+0000 7f4a59766640 1 -- 172.21.15.156:0/3383329343 >> 172.21.15.156:0/3383329343 conn(0x7f4a540721c0 msgr2=0x7f4a540725c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:53:33.258 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:33.253+0000 7f4a59766640 1 -- 172.21.15.156:0/3383329343 shutdown_connections 2024-07-16T09:53:33.258 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:33.254+0000 7f4a59766640 1 -- 172.21.15.156:0/3383329343 wait complete. 2024-07-16T09:53:33.258 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:33.254+0000 7f4a59766640 1 Processor -- start 2024-07-16T09:53:33.258 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:33.254+0000 7f4a59766640 1 -- start start 2024-07-16T09:53:33.259 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:33.254+0000 7f4a59766640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4a540fb620 0x7f4a54113080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:53:33.259 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:33.254+0000 7f4a59766640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4a540fbf30 0x7f4a541135c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:53:33.259 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:33.254+0000 7f4a59766640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a54113bc0 con 0x7f4a540fbf30 2024-07-16T09:53:33.259 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:33.254+0000 7f4a59766640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a5410d000 con 0x7f4a540fb620 2024-07-16T09:53:33.259 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:33.254+0000 7f4a4bfff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4a540fbf30 0x7f4a541135c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:53:33.259 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:33.254+0000 7f4a52ffd640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4a540fb620 0x7f4a54113080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:53:33.259 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:33.254+0000 7f4a4bfff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4a540fbf30 0x7f4a541135c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:41690/0 (socket says 172.21.15.156:41690) 2024-07-16T09:53:33.259 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:33.254+0000 7f4a4bfff640 1 -- 172.21.15.156:0/3813486761 learned_addr learned my addr 172.21.15.156:0/3813486761 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T09:53:33.259 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:33.255+0000 7f4a52ffd640 1 -- 172.21.15.156:0/3813486761 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4a540fbf30 msgr2=0x7f4a541135c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:53:33.259 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:33.255+0000 7f4a52ffd640 1 --2- 172.21.15.156:0/3813486761 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4a540fbf30 0x7f4a541135c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:53:33.259 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:33.255+0000 7f4a52ffd640 1 -- 172.21.15.156:0/3813486761 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a3c005040 con 0x7f4a540fb620 2024-07-16T09:53:33.260 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:33.255+0000 7f4a52ffd640 1 --2- 172.21.15.156:0/3813486761 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4a540fb620 0x7f4a54113080 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f4a4400eb40 tx=0x7f4a4400b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:53:33.260 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:33.255+0000 7f4a4b7fe640 1 -- 172.21.15.156:0/3813486761 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4a44007c30 con 0x7f4a540fb620 2024-07-16T09:53:33.260 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:33.255+0000 7f4a4b7fe640 1 -- 172.21.15.156:0/3813486761 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4a4400f040 con 0x7f4a540fb620 2024-07-16T09:53:33.260 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:33.255+0000 7f4a59766640 1 -- 172.21.15.156:0/3813486761 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a5410d280 con 0x7f4a540fb620 2024-07-16T09:53:33.260 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:33.255+0000 7f4a4b7fe640 1 -- 172.21.15.156:0/3813486761 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4a4400cb50 con 0x7f4a540fb620 2024-07-16T09:53:33.260 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:33.255+0000 7f4a59766640 1 -- 172.21.15.156:0/3813486761 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a5410d820 con 0x7f4a540fb620 2024-07-16T09:53:33.261 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:33.256+0000 7f4a59766640 1 -- 172.21.15.156:0/3813486761 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a540fc9d0 con 0x7f4a540fb620 2024-07-16T09:53:33.261 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:33.257+0000 7f4a4b7fe640 1 -- 172.21.15.156:0/3813486761 <== mon.1 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 19) v1 ==== 85127+0+0 (secure 0 0 0) 0x7f4a4400ccb0 con 0x7f4a540fb620 2024-07-16T09:53:33.261 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:33.257+0000 7f4a4b7fe640 1 --2- 172.21.15.156:0/3813486761 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f4a40067190 0x7f4a40069640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:53:33.261 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:33.257+0000 7f4a4b7fe640 1 -- 172.21.15.156:0/3813486761 <== mon.1 v2:172.21.15.176:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f4a440848a0 con 0x7f4a540fb620 2024-07-16T09:53:33.261 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:33.257+0000 7f4a4bfff640 1 --2- 172.21.15.156:0/3813486761 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f4a40067190 0x7f4a40069640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:53:33.261 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:33.258+0000 7f4a4bfff640 1 --2- 172.21.15.156:0/3813486761 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f4a40067190 0x7f4a40069640 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f4a5410e940 tx=0x7f4a3c035040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:53:33.263 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:33.261+0000 7f4a4b7fe640 1 -- 172.21.15.156:0/3813486761 <== mon.1 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f4a44051b90 con 0x7f4a540fb620 2024-07-16T09:53:33.466 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:33.463+0000 7f4a59766640 1 -- 172.21.15.156:0/3813486761 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f4a5404b560 con 0x7f4a540fb620 2024-07-16T09:53:33.466 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:33.464+0000 7f4a4b7fe640 1 -- 172.21.15.156:0/3813486761 <== mon.1 v2:172.21.15.176:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v16) v1 ==== 53+0+83 (secure 0 0 0) 0x7f4a440551b0 con 0x7f4a540fb620 2024-07-16T09:53:33.466 INFO:teuthology.orchestra.run.smithi156.stdout:name: cephfs, metadata pool: cephfs.cephfs.meta, data pools: [cephfs.cephfs.data ] 2024-07-16T09:53:33.468 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:33.466+0000 7f4a59766640 1 -- 172.21.15.156:0/3813486761 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f4a40067190 msgr2=0x7f4a40069640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:53:33.468 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:33.466+0000 7f4a59766640 1 --2- 172.21.15.156:0/3813486761 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f4a40067190 0x7f4a40069640 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f4a5410e940 tx=0x7f4a3c035040 comp rx=0 tx=0).stop 2024-07-16T09:53:33.469 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:33.466+0000 7f4a59766640 1 -- 172.21.15.156:0/3813486761 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4a540fb620 msgr2=0x7f4a54113080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:53:33.469 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:33.466+0000 7f4a59766640 1 --2- 172.21.15.156:0/3813486761 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4a540fb620 0x7f4a54113080 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f4a4400eb40 tx=0x7f4a4400b040 comp rx=0 tx=0).stop 2024-07-16T09:53:33.469 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:33.466+0000 7f4a59766640 1 -- 172.21.15.156:0/3813486761 shutdown_connections 2024-07-16T09:53:33.469 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:33.466+0000 7f4a59766640 1 --2- 172.21.15.156:0/3813486761 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f4a40067190 0x7f4a40069640 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:53:33.469 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:33.466+0000 7f4a59766640 1 --2- 172.21.15.156:0/3813486761 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4a540fbf30 0x7f4a541135c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:53:33.469 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:33.466+0000 7f4a59766640 1 --2- 172.21.15.156:0/3813486761 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4a540fb620 0x7f4a54113080 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:53:33.469 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:33.466+0000 7f4a59766640 1 -- 172.21.15.156:0/3813486761 >> 172.21.15.156:0/3813486761 conn(0x7f4a540721c0 msgr2=0x7f4a54100150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:53:33.469 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:33.466+0000 7f4a59766640 1 -- 172.21.15.156:0/3813486761 shutdown_connections 2024-07-16T09:53:33.469 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:33.466+0000 7f4a59766640 1 -- 172.21.15.156:0/3813486761 wait complete. 2024-07-16T09:53:34.064 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-07-16T09:53:34.065 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-07-16T09:53:34.065 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi176.front.sepia.ceph.com 2024-07-16T09:53:34.065 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-07-16T09:53:34.065 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-07-16T09:53:34.065 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-07-16T09:53:34.065 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-07-16T09:53:34.065 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-07-16T09:53:34.065 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-07-16T09:53:34.065 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-16T09:53:34.065 DEBUG:teuthology.orchestra.run.smithi176:> ip addr 2024-07-16T09:53:34.085 INFO:teuthology.orchestra.run.smithi176.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-07-16T09:53:34.085 INFO:teuthology.orchestra.run.smithi176.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-07-16T09:53:34.085 INFO:teuthology.orchestra.run.smithi176.stdout: inet 127.0.0.1/8 scope host lo 2024-07-16T09:53:34.085 INFO:teuthology.orchestra.run.smithi176.stdout: valid_lft forever preferred_lft forever 2024-07-16T09:53:34.085 INFO:teuthology.orchestra.run.smithi176.stdout: inet6 ::1/128 scope host 2024-07-16T09:53:34.085 INFO:teuthology.orchestra.run.smithi176.stdout: valid_lft forever preferred_lft forever 2024-07-16T09:53:34.085 INFO:teuthology.orchestra.run.smithi176.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-07-16T09:53:34.085 INFO:teuthology.orchestra.run.smithi176.stdout: link/ether 0c:c4:7a:d9:36:80 brd ff:ff:ff:ff:ff:ff 2024-07-16T09:53:34.085 INFO:teuthology.orchestra.run.smithi176.stdout: altname enp7s0f0 2024-07-16T09:53:34.086 INFO:teuthology.orchestra.run.smithi176.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-07-16T09:53:34.086 INFO:teuthology.orchestra.run.smithi176.stdout: link/ether 0c:c4:7a:d9:36:81 brd ff:ff:ff:ff:ff:ff 2024-07-16T09:53:34.086 INFO:teuthology.orchestra.run.smithi176.stdout: altname enp7s0f1 2024-07-16T09:53:34.086 INFO:teuthology.orchestra.run.smithi176.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-07-16T09:53:34.086 INFO:teuthology.orchestra.run.smithi176.stdout: link/ether 0c:c4:7a:88:6f:60 brd ff:ff:ff:ff:ff:ff 2024-07-16T09:53:34.086 INFO:teuthology.orchestra.run.smithi176.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-07-16T09:53:34.086 INFO:teuthology.orchestra.run.smithi176.stdout: link/ether 0c:c4:7a:88:6f:61 brd ff:ff:ff:ff:ff:ff 2024-07-16T09:53:34.086 INFO:teuthology.orchestra.run.smithi176.stdout: inet 172.21.15.176/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-07-16T09:53:34.086 INFO:teuthology.orchestra.run.smithi176.stdout: valid_lft 42309sec preferred_lft 42309sec 2024-07-16T09:53:34.086 INFO:teuthology.orchestra.run.smithi176.stdout: inet6 fe80::ec4:7aff:fe88:6f61/64 scope link 2024-07-16T09:53:34.086 INFO:teuthology.orchestra.run.smithi176.stdout: valid_lft forever preferred_lft forever 2024-07-16T09:53:34.087 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-07-16T09:53:34.087 DEBUG:teuthology.orchestra.run.smithi176:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-07-16T09:53:34.087 DEBUG:teuthology.orchestra.run.smithi176:> set -e 2024-07-16T09:53:34.087 DEBUG:teuthology.orchestra.run.smithi176:> sudo ip link add name ceph-brx type bridge 2024-07-16T09:53:34.087 DEBUG:teuthology.orchestra.run.smithi176:> sudo ip addr flush dev ceph-brx 2024-07-16T09:53:34.087 DEBUG:teuthology.orchestra.run.smithi176:> sudo ip link set ceph-brx up 2024-07-16T09:53:34.087 DEBUG:teuthology.orchestra.run.smithi176:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-07-16T09:53:34.087 DEBUG:teuthology.orchestra.run.smithi176:> ') 2024-07-16T09:53:34.177 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:53:34 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-07-16T09:53:34.302 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:53:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-07-16T09:53:34.309 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-16T09:53:34.309 DEBUG:teuthology.orchestra.run.smithi176:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-07-16T09:53:34.335 INFO:teuthology.orchestra.run.smithi176.stdout:1 2024-07-16T09:53:34.335 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-16T09:53:34.335 DEBUG:teuthology.orchestra.run.smithi176:> route 2024-07-16T09:53:34.343 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:53:34 smithi176 ceph-mon[36626]: pgmap v188: 65 pgs: 65 active+clean; 581 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 85 B/s wr, 3 op/s 2024-07-16T09:53:34.343 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:53:34 smithi176 ceph-mon[36626]: from='client.? 172.21.15.156:0/3813486761' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-07-16T09:53:34.414 INFO:teuthology.orchestra.run.smithi176.stdout:Kernel IP routing table 2024-07-16T09:53:34.415 INFO:teuthology.orchestra.run.smithi176.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-07-16T09:53:34.415 INFO:teuthology.orchestra.run.smithi176.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-07-16T09:53:34.415 INFO:teuthology.orchestra.run.smithi176.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-07-16T09:53:34.415 INFO:teuthology.orchestra.run.smithi176.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-07-16T09:53:34.415 DEBUG:teuthology.orchestra.run.smithi176:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-07-16T09:53:34.415 DEBUG:teuthology.orchestra.run.smithi176:> set -e 2024-07-16T09:53:34.415 DEBUG:teuthology.orchestra.run.smithi176:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-07-16T09:53:34.415 DEBUG:teuthology.orchestra.run.smithi176:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-07-16T09:53:34.415 DEBUG:teuthology.orchestra.run.smithi176:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-07-16T09:53:34.416 DEBUG:teuthology.orchestra.run.smithi176:> ') 2024-07-16T09:53:34.424 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:53:34 smithi156 ceph-mon[29302]: pgmap v188: 65 pgs: 65 active+clean; 581 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 85 B/s wr, 3 op/s 2024-07-16T09:53:34.424 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:53:34 smithi156 ceph-mon[29302]: from='client.? 172.21.15.156:0/3813486761' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-07-16T09:53:34.464 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:53:34 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-07-16T09:53:34.682 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:53:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-07-16T09:53:34.688 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-16T09:53:34.688 DEBUG:teuthology.orchestra.run.smithi176:> ip netns list 2024-07-16T09:53:34.704 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-16T09:53:34.704 DEBUG:teuthology.orchestra.run.smithi176:> ip netns list-id 2024-07-16T09:53:34.759 DEBUG:teuthology.orchestra.run.smithi176:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-07-16T09:53:34.759 DEBUG:teuthology.orchestra.run.smithi176:> set -e 2024-07-16T09:53:34.759 DEBUG:teuthology.orchestra.run.smithi176:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-07-16T09:53:34.760 DEBUG:teuthology.orchestra.run.smithi176:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-07-16T09:53:34.760 DEBUG:teuthology.orchestra.run.smithi176:> ') 2024-07-16T09:53:34.847 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:53:34 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-07-16T09:53:34.871 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:53:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-07-16T09:53:34.877 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-07-16T09:53:34.878 DEBUG:teuthology.orchestra.run.smithi176:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-07-16T09:53:34.878 DEBUG:teuthology.orchestra.run.smithi176:> set -e 2024-07-16T09:53:34.878 DEBUG:teuthology.orchestra.run.smithi176:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-07-16T09:53:34.878 DEBUG:teuthology.orchestra.run.smithi176:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-07-16T09:53:34.878 DEBUG:teuthology.orchestra.run.smithi176:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-07-16T09:53:34.878 DEBUG:teuthology.orchestra.run.smithi176:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-07-16T09:53:34.878 DEBUG:teuthology.orchestra.run.smithi176:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-07-16T09:53:34.878 DEBUG:teuthology.orchestra.run.smithi176:> ') 2024-07-16T09:53:34.926 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:53:34 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-07-16T09:53:35.003 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:53:35 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-07-16T09:53:35.009 DEBUG:teuthology.orchestra.run.smithi176:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-07-16T09:53:35.009 DEBUG:teuthology.orchestra.run.smithi176:> set -e 2024-07-16T09:53:35.010 DEBUG:teuthology.orchestra.run.smithi176:> sudo ip link set brx.0 up 2024-07-16T09:53:35.010 DEBUG:teuthology.orchestra.run.smithi176:> sudo ip link set dev brx.0 master ceph-brx 2024-07-16T09:53:35.010 DEBUG:teuthology.orchestra.run.smithi176:> ') 2024-07-16T09:53:35.058 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:53:35 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-07-16T09:53:35.080 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:53:35 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-07-16T09:53:35.086 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-07-16T09:53:35.087 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-07-16T09:53:35.087 DEBUG:teuthology.orchestra.run.smithi176:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-07-16T09:53:35.102 INFO:teuthology.orchestra.run.smithi176.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-07-16T09:53:35.103 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-07-16T09:53:35.103 DEBUG:teuthology.orchestra.run.smithi176:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-07-16T09:53:35.158 DEBUG:teuthology.orchestra.run.smithi176:> sudo modprobe fuse 2024-07-16T09:53:35.223 DEBUG:teuthology.orchestra.run.smithi176:> cat /proc/self/mounts | awk '{print $2}' 2024-07-16T09:53:35.279 INFO:teuthology.orchestra.run.smithi176.stdout:/proc 2024-07-16T09:53:35.279 INFO:teuthology.orchestra.run.smithi176.stdout:/sys 2024-07-16T09:53:35.279 INFO:teuthology.orchestra.run.smithi176.stdout:/dev 2024-07-16T09:53:35.279 INFO:teuthology.orchestra.run.smithi176.stdout:/sys/kernel/security 2024-07-16T09:53:35.279 INFO:teuthology.orchestra.run.smithi176.stdout:/dev/shm 2024-07-16T09:53:35.280 INFO:teuthology.orchestra.run.smithi176.stdout:/dev/pts 2024-07-16T09:53:35.280 INFO:teuthology.orchestra.run.smithi176.stdout:/run 2024-07-16T09:53:35.280 INFO:teuthology.orchestra.run.smithi176.stdout:/sys/fs/cgroup 2024-07-16T09:53:35.280 INFO:teuthology.orchestra.run.smithi176.stdout:/sys/fs/pstore 2024-07-16T09:53:35.280 INFO:teuthology.orchestra.run.smithi176.stdout:/sys/fs/bpf 2024-07-16T09:53:35.280 INFO:teuthology.orchestra.run.smithi176.stdout:/ 2024-07-16T09:53:35.280 INFO:teuthology.orchestra.run.smithi176.stdout:/sys/fs/selinux 2024-07-16T09:53:35.280 INFO:teuthology.orchestra.run.smithi176.stdout:/proc/sys/fs/binfmt_misc 2024-07-16T09:53:35.280 INFO:teuthology.orchestra.run.smithi176.stdout:/sys/kernel/tracing 2024-07-16T09:53:35.280 INFO:teuthology.orchestra.run.smithi176.stdout:/dev/mqueue 2024-07-16T09:53:35.280 INFO:teuthology.orchestra.run.smithi176.stdout:/dev/hugepages 2024-07-16T09:53:35.280 INFO:teuthology.orchestra.run.smithi176.stdout:/sys/kernel/debug 2024-07-16T09:53:35.280 INFO:teuthology.orchestra.run.smithi176.stdout:/sys/fs/fuse/connections 2024-07-16T09:53:35.281 INFO:teuthology.orchestra.run.smithi176.stdout:/sys/kernel/config 2024-07-16T09:53:35.281 INFO:teuthology.orchestra.run.smithi176.stdout:/run/credentials/systemd-sysctl.service 2024-07-16T09:53:35.281 INFO:teuthology.orchestra.run.smithi176.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-07-16T09:53:35.281 INFO:teuthology.orchestra.run.smithi176.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-07-16T09:53:35.281 INFO:teuthology.orchestra.run.smithi176.stdout:/var/lib/nfs/rpc_pipefs 2024-07-16T09:53:35.281 INFO:teuthology.orchestra.run.smithi176.stdout:/run/user/1000 2024-07-16T09:53:35.281 INFO:teuthology.orchestra.run.smithi176.stdout:/var/lib/ceph 2024-07-16T09:53:35.281 INFO:teuthology.orchestra.run.smithi176.stdout:/run/user/0 2024-07-16T09:53:35.281 INFO:teuthology.orchestra.run.smithi176.stdout:/proc/sys/fs/binfmt_misc 2024-07-16T09:53:35.281 INFO:teuthology.orchestra.run.smithi176.stdout:/var/lib/containers/storage/overlay 2024-07-16T09:53:35.281 INFO:teuthology.orchestra.run.smithi176.stdout:/var/lib/containers/storage/overlay/854dc0349a573f4670f16809141b5b41481f29512fac27d6a85a1454a83fb817/merged 2024-07-16T09:53:35.281 INFO:teuthology.orchestra.run.smithi176.stdout:/var/lib/containers/storage/overlay/1821a013e1af89fe7996ad2f4735e5bca9995110e50abe2eb5d5db2dcd4d7962/merged 2024-07-16T09:53:35.281 INFO:teuthology.orchestra.run.smithi176.stdout:/var/lib/containers/storage/overlay/ff6dcfca4371f4a8947f5ea1912a387ade56f391879444a2d98997a51ce05c62/merged 2024-07-16T09:53:35.282 INFO:teuthology.orchestra.run.smithi176.stdout:/var/lib/containers/storage/overlay/b4de9bfc9e42e445d5c93b5865fdd4fbedb933dcc4965fd35334b0704aa8b5a0/merged 2024-07-16T09:53:35.282 INFO:teuthology.orchestra.run.smithi176.stdout:/var/lib/containers/storage/overlay/6458a18356bdae0ce0097aeec9821976a561c276f3b0307b1a9ad3cb80d5e629/merged 2024-07-16T09:53:35.282 INFO:teuthology.orchestra.run.smithi176.stdout:/var/lib/containers/storage/overlay/f6504a100e3487713a2d36d9946fcfe877231af35dfb82bba1e0d9a6f9061287/merged 2024-07-16T09:53:35.282 INFO:teuthology.orchestra.run.smithi176.stdout:/var/lib/containers/storage/overlay/5360b32321e86964e0aa9dfda99ca7da54fde123e348f6b699c9d8837ae20a46/merged 2024-07-16T09:53:35.282 INFO:teuthology.orchestra.run.smithi176.stdout:/var/lib/containers/storage/overlay/1d08283545362cd1d35611e4f677f97566c059e75c3f25c914d93771f70ddf1f/merged 2024-07-16T09:53:35.282 INFO:teuthology.orchestra.run.smithi176.stdout:/var/lib/containers/storage/overlay/fad73029089e6b8cd556bbfd45623e2f420fefa3715fbf9b0b8fb97a5f6cf203/merged 2024-07-16T09:53:35.282 INFO:teuthology.orchestra.run.smithi176.stdout:/run/netns 2024-07-16T09:53:35.282 INFO:teuthology.orchestra.run.smithi176.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-07-16T09:53:35.282 INFO:teuthology.orchestra.run.smithi176.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-07-16T09:53:35.283 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-16T09:53:35.283 DEBUG:teuthology.orchestra.run.smithi176:> ls /sys/fs/fuse/connections 2024-07-16T09:53:35.335 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-07-16T09:53:35.335 DEBUG:teuthology.orchestra.run.smithi176:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-07-16T09:53:35.378 DEBUG:teuthology.orchestra.run.smithi176:> sudo modprobe fuse 2024-07-16T09:53:35.402 DEBUG:teuthology.orchestra.run.smithi176:> cat /proc/self/mounts | awk '{print $2}' 2024-07-16T09:53:35.459 INFO:teuthology.orchestra.run.smithi176.stdout:/proc 2024-07-16T09:53:35.459 INFO:teuthology.orchestra.run.smithi176.stdout:/sys 2024-07-16T09:53:35.459 INFO:teuthology.orchestra.run.smithi176.stdout:/dev 2024-07-16T09:53:35.460 INFO:teuthology.orchestra.run.smithi176.stdout:/sys/kernel/security 2024-07-16T09:53:35.460 INFO:teuthology.orchestra.run.smithi176.stdout:/dev/shm 2024-07-16T09:53:35.460 INFO:teuthology.orchestra.run.smithi176.stdout:/dev/pts 2024-07-16T09:53:35.460 INFO:teuthology.orchestra.run.smithi176.stdout:/run 2024-07-16T09:53:35.460 INFO:teuthology.orchestra.run.smithi176.stdout:/sys/fs/cgroup 2024-07-16T09:53:35.460 INFO:teuthology.orchestra.run.smithi176.stdout:/sys/fs/pstore 2024-07-16T09:53:35.460 INFO:teuthology.orchestra.run.smithi176.stdout:/sys/fs/bpf 2024-07-16T09:53:35.460 INFO:teuthology.orchestra.run.smithi176.stdout:/ 2024-07-16T09:53:35.460 INFO:teuthology.orchestra.run.smithi176.stdout:/sys/fs/selinux 2024-07-16T09:53:35.460 INFO:teuthology.orchestra.run.smithi176.stdout:/proc/sys/fs/binfmt_misc 2024-07-16T09:53:35.460 INFO:teuthology.orchestra.run.smithi176.stdout:/sys/kernel/tracing 2024-07-16T09:53:35.460 INFO:teuthology.orchestra.run.smithi176.stdout:/dev/mqueue 2024-07-16T09:53:35.461 INFO:teuthology.orchestra.run.smithi176.stdout:/dev/hugepages 2024-07-16T09:53:35.461 INFO:teuthology.orchestra.run.smithi176.stdout:/sys/kernel/debug 2024-07-16T09:53:35.461 INFO:teuthology.orchestra.run.smithi176.stdout:/sys/fs/fuse/connections 2024-07-16T09:53:35.461 INFO:teuthology.orchestra.run.smithi176.stdout:/sys/kernel/config 2024-07-16T09:53:35.461 INFO:teuthology.orchestra.run.smithi176.stdout:/run/credentials/systemd-sysctl.service 2024-07-16T09:53:35.461 INFO:teuthology.orchestra.run.smithi176.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-07-16T09:53:35.461 INFO:teuthology.orchestra.run.smithi176.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-07-16T09:53:35.461 INFO:teuthology.orchestra.run.smithi176.stdout:/var/lib/nfs/rpc_pipefs 2024-07-16T09:53:35.461 INFO:teuthology.orchestra.run.smithi176.stdout:/run/user/1000 2024-07-16T09:53:35.461 INFO:teuthology.orchestra.run.smithi176.stdout:/var/lib/ceph 2024-07-16T09:53:35.461 INFO:teuthology.orchestra.run.smithi176.stdout:/run/user/0 2024-07-16T09:53:35.461 INFO:teuthology.orchestra.run.smithi176.stdout:/proc/sys/fs/binfmt_misc 2024-07-16T09:53:35.462 INFO:teuthology.orchestra.run.smithi176.stdout:/var/lib/containers/storage/overlay 2024-07-16T09:53:35.462 INFO:teuthology.orchestra.run.smithi176.stdout:/var/lib/containers/storage/overlay/854dc0349a573f4670f16809141b5b41481f29512fac27d6a85a1454a83fb817/merged 2024-07-16T09:53:35.462 INFO:teuthology.orchestra.run.smithi176.stdout:/var/lib/containers/storage/overlay/1821a013e1af89fe7996ad2f4735e5bca9995110e50abe2eb5d5db2dcd4d7962/merged 2024-07-16T09:53:35.462 INFO:teuthology.orchestra.run.smithi176.stdout:/var/lib/containers/storage/overlay/ff6dcfca4371f4a8947f5ea1912a387ade56f391879444a2d98997a51ce05c62/merged 2024-07-16T09:53:35.462 INFO:teuthology.orchestra.run.smithi176.stdout:/var/lib/containers/storage/overlay/b4de9bfc9e42e445d5c93b5865fdd4fbedb933dcc4965fd35334b0704aa8b5a0/merged 2024-07-16T09:53:35.462 INFO:teuthology.orchestra.run.smithi176.stdout:/var/lib/containers/storage/overlay/6458a18356bdae0ce0097aeec9821976a561c276f3b0307b1a9ad3cb80d5e629/merged 2024-07-16T09:53:35.462 INFO:teuthology.orchestra.run.smithi176.stdout:/var/lib/containers/storage/overlay/f6504a100e3487713a2d36d9946fcfe877231af35dfb82bba1e0d9a6f9061287/merged 2024-07-16T09:53:35.462 INFO:teuthology.orchestra.run.smithi176.stdout:/var/lib/containers/storage/overlay/5360b32321e86964e0aa9dfda99ca7da54fde123e348f6b699c9d8837ae20a46/merged 2024-07-16T09:53:35.462 INFO:teuthology.orchestra.run.smithi176.stdout:/var/lib/containers/storage/overlay/1d08283545362cd1d35611e4f677f97566c059e75c3f25c914d93771f70ddf1f/merged 2024-07-16T09:53:35.462 INFO:teuthology.orchestra.run.smithi176.stdout:/var/lib/containers/storage/overlay/fad73029089e6b8cd556bbfd45623e2f420fefa3715fbf9b0b8fb97a5f6cf203/merged 2024-07-16T09:53:35.462 INFO:teuthology.orchestra.run.smithi176.stdout:/run/netns 2024-07-16T09:53:35.463 INFO:teuthology.orchestra.run.smithi176.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-07-16T09:53:35.463 INFO:teuthology.orchestra.run.smithi176.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-07-16T09:53:35.463 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-16T09:53:35.463 DEBUG:teuthology.orchestra.run.smithi176:> ls /sys/fs/fuse/connections 2024-07-16T09:53:35.478 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi176.stderr:2024-07-16T09:53:35.477+0000 7fb638453300 -1 init, newargv = 0x56034dc7e730 newargc=15 2024-07-16T09:53:35.478 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi176.stderr:ceph-fuse[61357]: starting ceph client 2024-07-16T09:53:35.485 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi176.stderr:ceph-fuse[61357]: starting fuse 2024-07-16T09:53:35.494 INFO:teuthology.orchestra.run.smithi176.stdout:116 2024-07-16T09:53:35.495 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [116] 2024-07-16T09:53:35.495 DEBUG:teuthology.orchestra.run.smithi176:> sudo stdin-killer -- python3 -c ' 2024-07-16T09:53:35.495 DEBUG:teuthology.orchestra.run.smithi176:> import glob 2024-07-16T09:53:35.495 DEBUG:teuthology.orchestra.run.smithi176:> import re 2024-07-16T09:53:35.495 DEBUG:teuthology.orchestra.run.smithi176:> import os 2024-07-16T09:53:35.495 DEBUG:teuthology.orchestra.run.smithi176:> import subprocess 2024-07-16T09:53:35.495 DEBUG:teuthology.orchestra.run.smithi176:> 2024-07-16T09:53:35.495 DEBUG:teuthology.orchestra.run.smithi176:> def _find_admin_socket(client_name): 2024-07-16T09:53:35.495 DEBUG:teuthology.orchestra.run.smithi176:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-07-16T09:53:35.496 DEBUG:teuthology.orchestra.run.smithi176:> files = glob.glob(asok_path) 2024-07-16T09:53:35.496 DEBUG:teuthology.orchestra.run.smithi176:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-07-16T09:53:35.496 DEBUG:teuthology.orchestra.run.smithi176:> 2024-07-16T09:53:35.496 DEBUG:teuthology.orchestra.run.smithi176:> # Given a non-glob path, it better be there 2024-07-16T09:53:35.496 DEBUG:teuthology.orchestra.run.smithi176:> if "*" not in asok_path: 2024-07-16T09:53:35.496 DEBUG:teuthology.orchestra.run.smithi176:> assert(len(files) == 1) 2024-07-16T09:53:35.496 DEBUG:teuthology.orchestra.run.smithi176:> return files[0] 2024-07-16T09:53:35.496 DEBUG:teuthology.orchestra.run.smithi176:> 2024-07-16T09:53:35.496 DEBUG:teuthology.orchestra.run.smithi176:> for f in files: 2024-07-16T09:53:35.496 DEBUG:teuthology.orchestra.run.smithi176:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-07-16T09:53:35.496 DEBUG:teuthology.orchestra.run.smithi176:> if os.path.exists("/proc/{0}".format(pid)): 2024-07-16T09:53:35.496 DEBUG:teuthology.orchestra.run.smithi176:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-07-16T09:53:35.496 DEBUG:teuthology.orchestra.run.smithi176:> contents = proc_f.read() 2024-07-16T09:53:35.496 DEBUG:teuthology.orchestra.run.smithi176:> if mountpoint in contents: 2024-07-16T09:53:35.497 DEBUG:teuthology.orchestra.run.smithi176:> return f 2024-07-16T09:53:35.497 DEBUG:teuthology.orchestra.run.smithi176:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-07-16T09:53:35.497 DEBUG:teuthology.orchestra.run.smithi176:> 2024-07-16T09:53:35.497 DEBUG:teuthology.orchestra.run.smithi176:> print(_find_admin_socket("client.1")) 2024-07-16T09:53:35.497 DEBUG:teuthology.orchestra.run.smithi176:> ' 2024-07-16T09:53:35.612 INFO:teuthology.orchestra.run.smithi176.stdout:/var/run/ceph/ceph-client.1.61357.asok 2024-07-16T09:53:35.615 INFO:teuthology.orchestra.run.smithi176.stderr:2024-07-16T09:53:35 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-07-16T09:53:35.622 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.61357.asok 2024-07-16T09:53:35.622 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-16T09:53:35.622 DEBUG:teuthology.orchestra.run.smithi176:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.61357.asok status 2024-07-16T09:53:35.752 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-07-16T09:53:35.752 INFO:teuthology.orchestra.run.smithi176.stdout: "metadata": { 2024-07-16T09:53:35.752 INFO:teuthology.orchestra.run.smithi176.stdout: "ceph_sha1": "e6231b04b52ebf3b9b31803282dd3074439e51ac", 2024-07-16T09:53:35.752 INFO:teuthology.orchestra.run.smithi176.stdout: "ceph_version": "ceph version 17.2.7-1104-ge6231b04 (e6231b04b52ebf3b9b31803282dd3074439e51ac) quincy (stable)", 2024-07-16T09:53:35.752 INFO:teuthology.orchestra.run.smithi176.stdout: "entity_id": "1", 2024-07-16T09:53:35.752 INFO:teuthology.orchestra.run.smithi176.stdout: "hostname": "smithi176", 2024-07-16T09:53:35.752 INFO:teuthology.orchestra.run.smithi176.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-07-16T09:53:35.752 INFO:teuthology.orchestra.run.smithi176.stdout: "pid": "61357", 2024-07-16T09:53:35.752 INFO:teuthology.orchestra.run.smithi176.stdout: "root": "/" 2024-07-16T09:53:35.752 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-07-16T09:53:35.753 INFO:teuthology.orchestra.run.smithi176.stdout: "dentry_count": 0, 2024-07-16T09:53:35.753 INFO:teuthology.orchestra.run.smithi176.stdout: "dentry_pinned_count": 0, 2024-07-16T09:53:35.753 INFO:teuthology.orchestra.run.smithi176.stdout: "id": 24301, 2024-07-16T09:53:35.753 INFO:teuthology.orchestra.run.smithi176.stdout: "inst": { 2024-07-16T09:53:35.753 INFO:teuthology.orchestra.run.smithi176.stdout: "name": { 2024-07-16T09:53:35.753 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "client", 2024-07-16T09:53:35.753 INFO:teuthology.orchestra.run.smithi176.stdout: "num": 24301 2024-07-16T09:53:35.753 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-07-16T09:53:35.753 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": { 2024-07-16T09:53:35.753 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v1", 2024-07-16T09:53:35.753 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "192.168.0.1:0", 2024-07-16T09:53:35.753 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 3627935956 2024-07-16T09:53:35.753 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-07-16T09:53:35.754 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-07-16T09:53:35.754 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": { 2024-07-16T09:53:35.754 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v1", 2024-07-16T09:53:35.754 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "192.168.0.1:0", 2024-07-16T09:53:35.754 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 3627935956 2024-07-16T09:53:35.754 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-07-16T09:53:35.754 INFO:teuthology.orchestra.run.smithi176.stdout: "inst_str": "client.24301 192.168.0.1:0/3627935956", 2024-07-16T09:53:35.754 INFO:teuthology.orchestra.run.smithi176.stdout: "addr_str": "192.168.0.1:0/3627935956", 2024-07-16T09:53:35.754 INFO:teuthology.orchestra.run.smithi176.stdout: "inode_count": 1, 2024-07-16T09:53:35.754 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_epoch": 16, 2024-07-16T09:53:35.754 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_epoch": 46, 2024-07-16T09:53:35.754 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_epoch_barrier": 0, 2024-07-16T09:53:35.755 INFO:teuthology.orchestra.run.smithi176.stdout: "blocklisted": false, 2024-07-16T09:53:35.755 INFO:teuthology.orchestra.run.smithi176.stdout: "fs_name": "cephfs" 2024-07-16T09:53:35.755 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-07-16T09:53:35.760 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-16T09:53:35.761 DEBUG:teuthology.orchestra.run.smithi156:> stat --file-system '--printf=%T 2024-07-16T09:53:35.761 DEBUG:teuthology.orchestra.run.smithi156:> ' -- /home/ubuntu/cephtest/mnt.0 2024-07-16T09:53:35.782 INFO:teuthology.orchestra.run.smithi156.stdout:fuseblk 2024-07-16T09:53:35.782 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-07-16T09:53:35.782 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-16T09:53:35.783 DEBUG:teuthology.orchestra.run.smithi156:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-07-16T09:53:35.856 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-16T09:53:35.856 DEBUG:teuthology.orchestra.run.smithi176:> stat --file-system '--printf=%T 2024-07-16T09:53:35.856 DEBUG:teuthology.orchestra.run.smithi176:> ' -- /home/ubuntu/cephtest/mnt.1 2024-07-16T09:53:35.872 INFO:teuthology.orchestra.run.smithi176.stdout:fuseblk 2024-07-16T09:53:35.872 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-07-16T09:53:35.872 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-16T09:53:35.872 DEBUG:teuthology.orchestra.run.smithi176:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-07-16T09:53:36.174 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:53:36 smithi156 ceph-mon[29302]: pgmap v189: 65 pgs: 65 active+clean; 581 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.7 KiB/s rd, 341 B/s wr, 5 op/s 2024-07-16T09:53:36.343 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:53:36 smithi176 ceph-mon[36626]: pgmap v189: 65 pgs: 65 active+clean; 581 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.7 KiB/s rd, 341 B/s wr, 5 op/s 2024-07-16T09:53:38.343 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:53:38 smithi176 ceph-mon[36626]: pgmap v190: 65 pgs: 65 active+clean; 581 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.8 KiB/s rd, 341 B/s wr, 3 op/s 2024-07-16T09:53:38.424 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:53:38 smithi156 ceph-mon[29302]: pgmap v190: 65 pgs: 65 active+clean; 581 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.8 KiB/s rd, 341 B/s wr, 3 op/s 2024-07-16T09:53:39.258 INFO:teuthology.run_tasks:Running task print... 2024-07-16T09:53:39.268 INFO:teuthology.task.print:**** done client 2024-07-16T09:53:39.268 INFO:teuthology.run_tasks:Running task parallel... 2024-07-16T09:53:39.280 INFO:teuthology.task.parallel:starting parallel... 2024-07-16T09:53:39.280 INFO:teuthology.task.parallel:In parallel, running task sequential... 2024-07-16T09:53:39.281 INFO:teuthology.task.sequential:In sequential, running task cephadm.shell... 2024-07-16T09:53:39.281 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi156.front.sepia.ceph.com 2024-07-16T09:53:39.282 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 26c91c26-4358-11ef-bcad-c7b262605968 -e sha1=a6ca461be7cc1aff3658c2c6df4cff975a24449a -- bash -c 'ceph config set mgr mgr/orchestrator/fail_fs false || true' 2024-07-16T09:53:39.283 INFO:teuthology.task.parallel:In parallel, running task sequential... 2024-07-16T09:53:39.283 INFO:teuthology.task.sequential:In sequential, running task workunit... 2024-07-16T09:53:39.287 INFO:tasks.workunit:Pulling workunits from ref a6ca461be7cc1aff3658c2c6df4cff975a24449a 2024-07-16T09:53:39.287 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-07-16T09:53:39.288 INFO:tasks.workunit:timeout=3h 2024-07-16T09:53:39.288 INFO:tasks.workunit:cleanup=True 2024-07-16T09:53:39.288 DEBUG:teuthology.orchestra.run.smithi156:> stat -- /home/ubuntu/cephtest/mnt.0 2024-07-16T09:53:39.309 INFO:teuthology.orchestra.run.smithi156.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-07-16T09:53:39.309 INFO:teuthology.orchestra.run.smithi156.stdout: Size: 0 Blocks: 1 IO Block: 4194304 directory 2024-07-16T09:53:39.309 INFO:teuthology.orchestra.run.smithi156.stdout:Device: 98h/152d Inode: 1 Links: 2 2024-07-16T09:53:39.309 INFO:teuthology.orchestra.run.smithi156.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-07-16T09:53:39.309 INFO:teuthology.orchestra.run.smithi156.stdout:Context: system_u:object_r:fusefs_t:s0 2024-07-16T09:53:39.309 INFO:teuthology.orchestra.run.smithi156.stdout:Access: 1970-01-01 00:00:00.000000000 +0000 2024-07-16T09:53:39.309 INFO:teuthology.orchestra.run.smithi156.stdout:Modify: 2024-07-16 09:53:00.237337020 +0000 2024-07-16T09:53:39.309 INFO:teuthology.orchestra.run.smithi156.stdout:Change: 2024-07-16 09:53:35.852630974 +0000 2024-07-16T09:53:39.309 INFO:teuthology.orchestra.run.smithi156.stdout: Birth: - 2024-07-16T09:53:39.309 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-07-16T09:53:39.309 DEBUG:teuthology.orchestra.run.smithi156:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-07-16T09:53:39.382 DEBUG:teuthology.orchestra.run.smithi176:> stat -- /home/ubuntu/cephtest/mnt.1 2024-07-16T09:53:39.403 INFO:teuthology.orchestra.run.smithi176.stdout: File: /home/ubuntu/cephtest/mnt.1 2024-07-16T09:53:39.403 INFO:teuthology.orchestra.run.smithi176.stdout: Size: 0 Blocks: 1 IO Block: 4194304 directory 2024-07-16T09:53:39.403 INFO:teuthology.orchestra.run.smithi176.stdout:Device: 74h/116d Inode: 1 Links: 3 2024-07-16T09:53:39.403 INFO:teuthology.orchestra.run.smithi176.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-07-16T09:53:39.403 INFO:teuthology.orchestra.run.smithi176.stdout:Context: system_u:object_r:fusefs_t:s0 2024-07-16T09:53:39.403 INFO:teuthology.orchestra.run.smithi176.stdout:Access: 1970-01-01 00:00:00.000000000 +0000 2024-07-16T09:53:39.403 INFO:teuthology.orchestra.run.smithi176.stdout:Modify: 2024-07-16 09:53:39.374872891 +0000 2024-07-16T09:53:39.403 INFO:teuthology.orchestra.run.smithi176.stdout:Change: 2024-07-16 09:53:39.374872891 +0000 2024-07-16T09:53:39.403 INFO:teuthology.orchestra.run.smithi176.stdout: Birth: - 2024-07-16T09:53:39.404 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.1 2024-07-16T09:53:39.404 DEBUG:teuthology.orchestra.run.smithi176:> cd -- /home/ubuntu/cephtest/mnt.1 && sudo install -d -m 0755 --owner=ubuntu -- client.1 2024-07-16T09:53:39.478 DEBUG:teuthology.orchestra.run.smithi156:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout a6ca461be7cc1aff3658c2c6df4cff975a24449a 2024-07-16T09:53:39.478 DEBUG:teuthology.orchestra.run.smithi176:> rm -rf /home/ubuntu/cephtest/clone.client.1 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.1 && cd /home/ubuntu/cephtest/clone.client.1 && git checkout a6ca461be7cc1aff3658c2c6df4cff975a24449a 2024-07-16T09:53:39.495 INFO:tasks.workunit.client.0.smithi156.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-07-16T09:53:39.536 INFO:tasks.workunit.client.1.smithi176.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.1'... 2024-07-16T09:53:39.569 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/26c91c26-4358-11ef-bcad-c7b262605968/mon.smithi156/config 2024-07-16T09:53:40.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:53:40 smithi156 ceph-mon[29302]: pgmap v191: 65 pgs: 65 active+clean; 581 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.3 KiB/s rd, 511 B/s wr, 4 op/s 2024-07-16T09:53:40.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:53:40 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-16T09:53:40.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:53:40 smithi176 ceph-mon[36626]: pgmap v191: 65 pgs: 65 active+clean; 581 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.3 KiB/s rd, 511 B/s wr, 4 op/s 2024-07-16T09:53:40.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:53:40 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-16T09:53:41.329 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:41.314+0000 7f6a6e02c640 1 -- 172.21.15.156:0/2097507028 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6a680fa790 msgr2=0x7f6a680faba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:53:41.330 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:41.314+0000 7f6a6e02c640 1 --2- 172.21.15.156:0/2097507028 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6a680fa790 0x7f6a680faba0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f6a580048d0 tx=0x7f6a58009da0 comp rx=0 tx=0).stop 2024-07-16T09:53:41.330 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:41.314+0000 7f6a6e02c640 1 -- 172.21.15.156:0/2097507028 shutdown_connections 2024-07-16T09:53:41.330 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:41.314+0000 7f6a6e02c640 1 --2- 172.21.15.156:0/2097507028 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6a680fb0e0 0x7f6a680fd960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:53:41.330 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:41.314+0000 7f6a6e02c640 1 --2- 172.21.15.156:0/2097507028 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6a680fa790 0x7f6a680faba0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:53:41.330 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:41.314+0000 7f6a6e02c640 1 -- 172.21.15.156:0/2097507028 >> 172.21.15.156:0/2097507028 conn(0x7f6a680f61d0 msgr2=0x7f6a680f8600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:53:41.330 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:41.314+0000 7f6a6e02c640 1 -- 172.21.15.156:0/2097507028 shutdown_connections 2024-07-16T09:53:41.330 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:41.314+0000 7f6a6e02c640 1 -- 172.21.15.156:0/2097507028 wait complete. 2024-07-16T09:53:41.330 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:41.315+0000 7f6a6e02c640 1 Processor -- start 2024-07-16T09:53:41.330 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:41.315+0000 7f6a6e02c640 1 -- start start 2024-07-16T09:53:41.331 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:41.315+0000 7f6a6e02c640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6a680fa790 0x7f6a6810d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:53:41.331 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:41.315+0000 7f6a6e02c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6a680fb0e0 0x7f6a6810d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:53:41.331 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:41.315+0000 7f6a6e02c640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a6810ddf0 con 0x7f6a680fb0e0 2024-07-16T09:53:41.331 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:41.315+0000 7f6a6e02c640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a6810e610 con 0x7f6a680fa790 2024-07-16T09:53:41.331 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:41.315+0000 7f6a66ffd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6a680fb0e0 0x7f6a6810d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:53:41.331 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:41.315+0000 7f6a66ffd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6a680fb0e0 0x7f6a6810d7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:41706/0 (socket says 172.21.15.156:41706) 2024-07-16T09:53:41.331 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:41.315+0000 7f6a66ffd640 1 -- 172.21.15.156:0/2404778115 learned_addr learned my addr 172.21.15.156:0/2404778115 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T09:53:41.331 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:41.316+0000 7f6a677fe640 1 --2- 172.21.15.156:0/2404778115 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6a680fa790 0x7f6a6810d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:53:41.331 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:41.316+0000 7f6a66ffd640 1 -- 172.21.15.156:0/2404778115 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6a680fa790 msgr2=0x7f6a6810d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:53:41.331 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:41.316+0000 7f6a66ffd640 1 --2- 172.21.15.156:0/2404778115 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6a680fa790 0x7f6a6810d260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:53:41.331 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:41.316+0000 7f6a66ffd640 1 -- 172.21.15.156:0/2404778115 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a58005040 con 0x7f6a680fb0e0 2024-07-16T09:53:41.331 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:41.316+0000 7f6a66ffd640 1 --2- 172.21.15.156:0/2404778115 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6a680fb0e0 0x7f6a6810d7a0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f6a5c00c760 tx=0x7f6a5c0043c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:53:41.332 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:41.316+0000 7f6a4ffff640 1 -- 172.21.15.156:0/2404778115 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6a5c00a790 con 0x7f6a680fb0e0 2024-07-16T09:53:41.332 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:41.316+0000 7f6a6e02c640 1 -- 172.21.15.156:0/2404778115 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a6810e840 con 0x7f6a680fb0e0 2024-07-16T09:53:41.332 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:41.316+0000 7f6a4ffff640 1 -- 172.21.15.156:0/2404778115 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6a5c00adc0 con 0x7f6a680fb0e0 2024-07-16T09:53:41.332 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:41.316+0000 7f6a4ffff640 1 -- 172.21.15.156:0/2404778115 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6a5c0115e0 con 0x7f6a680fb0e0 2024-07-16T09:53:41.332 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:41.316+0000 7f6a6e02c640 1 -- 172.21.15.156:0/2404778115 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a6810ede0 con 0x7f6a680fb0e0 2024-07-16T09:53:41.332 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:41.317+0000 7f6a6e02c640 1 -- 172.21.15.156:0/2404778115 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a680f9e70 con 0x7f6a680fb0e0 2024-07-16T09:53:41.332 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:41.318+0000 7f6a4ffff640 1 -- 172.21.15.156:0/2404778115 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 19) v1 ==== 85127+0+0 (secure 0 0 0) 0x7f6a5c012400 con 0x7f6a680fb0e0 2024-07-16T09:53:41.332 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:41.318+0000 7f6a4ffff640 1 --2- 172.21.15.156:0/2404778115 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f6a50067190 0x7f6a50069640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:53:41.332 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:41.318+0000 7f6a4ffff640 1 -- 172.21.15.156:0/2404778115 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f6a5c084ca0 con 0x7f6a680fb0e0 2024-07-16T09:53:41.332 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:41.318+0000 7f6a677fe640 1 --2- 172.21.15.156:0/2404778115 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f6a50067190 0x7f6a50069640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:53:41.332 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:41.319+0000 7f6a677fe640 1 --2- 172.21.15.156:0/2404778115 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f6a50067190 0x7f6a50069640 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f6a58000c00 tx=0x7f6a58035040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:53:41.332 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:41.322+0000 7f6a4ffff640 1 -- 172.21.15.156:0/2404778115 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f6a5c089070 con 0x7f6a680fb0e0 2024-07-16T09:53:41.491 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:41.489+0000 7f6a6e02c640 1 -- 172.21.15.156:0/2404778115 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command([{prefix=config set, name=mgr/orchestrator/fail_fs}] v 0) v1 -- 0x7f6a6804b530 con 0x7f6a680fb0e0 2024-07-16T09:53:41.491 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:41.490+0000 7f6a4ffff640 1 -- 172.21.15.156:0/2404778115 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/orchestrator/fail_fs}]=-22 unrecognized config option 'mgr/orchestrator/fail_fs' v21)=-22 unrecognized config option 'mgr/orchestrator/fail_fs' v21) v1 ==== 179+0+0 (secure 0 0 0) 0x7f6a5c051fc0 con 0x7f6a680fb0e0 2024-07-16T09:53:41.491 INFO:teuthology.orchestra.run.smithi156.stderr:Error EINVAL: unrecognized config option 'mgr/orchestrator/fail_fs' 2024-07-16T09:53:42.071 INFO:teuthology.task.sequential:In sequential, running task cephadm.shell... 2024-07-16T09:53:42.072 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi156.front.sepia.ceph.com 2024-07-16T09:53:42.072 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 26c91c26-4358-11ef-bcad-c7b262605968 -e sha1=a6ca461be7cc1aff3658c2c6df4cff975a24449a -- bash -c 'ceph config set mon mon_warn_on_insecure_global_id_reclaim false --force' 2024-07-16T09:53:42.342 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:53:42 smithi156 ceph-mon[29302]: pgmap v192: 65 pgs: 65 active+clean; 583 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.4 KiB/s rd, 682 B/s wr, 5 op/s 2024-07-16T09:53:42.380 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/26c91c26-4358-11ef-bcad-c7b262605968/mon.smithi156/config 2024-07-16T09:53:42.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:53:42 smithi176 ceph-mon[36626]: pgmap v192: 65 pgs: 65 active+clean; 583 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.4 KiB/s rd, 682 B/s wr, 5 op/s 2024-07-16T09:53:43.957 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:43.929+0000 7feefec76640 1 -- 172.21.15.156:0/2590814082 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7feef80fd630 msgr2=0x7feef80fdaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:53:43.957 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:43.929+0000 7feefec76640 1 --2- 172.21.15.156:0/2590814082 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7feef80fd630 0x7feef80fdaa0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7feee40048d0 tx=0x7feee4009da0 comp rx=0 tx=0).stop 2024-07-16T09:53:43.957 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:43.929+0000 7feefec76640 1 -- 172.21.15.156:0/2590814082 shutdown_connections 2024-07-16T09:53:43.957 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:43.929+0000 7feefec76640 1 --2- 172.21.15.156:0/2590814082 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7feef80fd630 0x7feef80fdaa0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:53:43.957 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:43.929+0000 7feefec76640 1 --2- 172.21.15.156:0/2590814082 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7feef80fc360 0x7feef80fc770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:53:43.958 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:43.929+0000 7feefec76640 1 -- 172.21.15.156:0/2590814082 >> 172.21.15.156:0/2590814082 conn(0x7feef80f7b40 msgr2=0x7feef80f9f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:53:43.958 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:43.930+0000 7feefec76640 1 -- 172.21.15.156:0/2590814082 shutdown_connections 2024-07-16T09:53:43.958 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:43.930+0000 7feefec76640 1 -- 172.21.15.156:0/2590814082 wait complete. 2024-07-16T09:53:43.958 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:43.930+0000 7feefec76640 1 Processor -- start 2024-07-16T09:53:43.958 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:43.930+0000 7feefec76640 1 -- start start 2024-07-16T09:53:43.958 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:43.931+0000 7feefec76640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7feef80fc360 0x7feef810cff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:53:43.958 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:43.931+0000 7feefec76640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7feef80fd630 0x7feef810d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:53:43.958 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:43.931+0000 7feefec76640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feef810db80 con 0x7feef80fc360 2024-07-16T09:53:43.958 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:43.931+0000 7feefec76640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feef810e350 con 0x7feef80fd630 2024-07-16T09:53:43.958 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:43.931+0000 7feefca12640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7feef80fc360 0x7feef810cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:53:43.959 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:43.931+0000 7feefca12640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7feef80fc360 0x7feef810cff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:46320/0 (socket says 172.21.15.156:46320) 2024-07-16T09:53:43.959 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:43.931+0000 7feefca12640 1 -- 172.21.15.156:0/829604272 learned_addr learned my addr 172.21.15.156:0/829604272 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T09:53:43.959 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:43.931+0000 7feef7fff640 1 --2- 172.21.15.156:0/829604272 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7feef80fd630 0x7feef810d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:53:43.959 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:43.931+0000 7feefca12640 1 -- 172.21.15.156:0/829604272 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7feef80fd630 msgr2=0x7feef810d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:53:43.959 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:43.931+0000 7feefca12640 1 --2- 172.21.15.156:0/829604272 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7feef80fd630 0x7feef810d530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:53:43.959 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:43.931+0000 7feefca12640 1 -- 172.21.15.156:0/829604272 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feee4005040 con 0x7feef80fc360 2024-07-16T09:53:43.959 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:43.931+0000 7feef7fff640 1 --2- 172.21.15.156:0/829604272 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7feef80fd630 0x7feef810d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-16T09:53:43.959 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:43.931+0000 7feefca12640 1 --2- 172.21.15.156:0/829604272 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7feef80fc360 0x7feef810cff0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7feeec002a50 tx=0x7feeec004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:53:43.959 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:43.931+0000 7feef57fa640 1 -- 172.21.15.156:0/829604272 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feeec007c30 con 0x7feef80fc360 2024-07-16T09:53:43.959 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:43.931+0000 7feef57fa640 1 -- 172.21.15.156:0/829604272 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7feeec00f040 con 0x7feef80fc360 2024-07-16T09:53:43.959 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:43.931+0000 7feefec76640 1 -- 172.21.15.156:0/829604272 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feef810e5e0 con 0x7feef80fc360 2024-07-16T09:53:43.959 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:43.931+0000 7feef57fa640 1 -- 172.21.15.156:0/829604272 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feeec00db50 con 0x7feef80fc360 2024-07-16T09:53:43.959 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:43.931+0000 7feefec76640 1 -- 172.21.15.156:0/829604272 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feef810eb80 con 0x7feef80fc360 2024-07-16T09:53:43.960 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:43.933+0000 7feefec76640 1 -- 172.21.15.156:0/829604272 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feef80fb7c0 con 0x7feef80fc360 2024-07-16T09:53:43.960 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:43.933+0000 7feef57fa640 1 -- 172.21.15.156:0/829604272 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 19) v1 ==== 85127+0+0 (secure 0 0 0) 0x7feeec01c400 con 0x7feef80fc360 2024-07-16T09:53:43.960 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:43.933+0000 7feef57fa640 1 --2- 172.21.15.156:0/829604272 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7feee8067190 0x7feee8069640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:53:43.960 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:43.933+0000 7feef57fa640 1 -- 172.21.15.156:0/829604272 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5464+0+0 (secure 0 0 0) 0x7feeec084460 con 0x7feef80fc360 2024-07-16T09:53:43.960 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:43.933+0000 7feef7fff640 1 --2- 172.21.15.156:0/829604272 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7feee8067190 0x7feee8069640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:53:43.960 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:43.934+0000 7feef7fff640 1 --2- 172.21.15.156:0/829604272 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7feee8067190 0x7feee8069640 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7feee4000c00 tx=0x7feee4035040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:53:43.960 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:43.938+0000 7feef57fa640 1 -- 172.21.15.156:0/829604272 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7feeec051700 con 0x7feef80fc360 2024-07-16T09:53:44.105 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:44.104+0000 7feefec76640 1 -- 172.21.15.156:0/829604272 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim}] v 0) v1 -- 0x7feef8104ed0 con 0x7feef80fc360 2024-07-16T09:53:44.105 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:44.104+0000 7feef57fa640 1 -- 172.21.15.156:0/829604272 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim}]=0 v21)=0 v21) v1 ==== 155+0+0 (secure 0 0 0) 0x7feeec054d20 con 0x7feef80fc360 2024-07-16T09:53:44.107 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:44.106+0000 7feefec76640 1 -- 172.21.15.156:0/829604272 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7feee8067190 msgr2=0x7feee8069640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:53:44.107 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:44.106+0000 7feefec76640 1 --2- 172.21.15.156:0/829604272 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7feee8067190 0x7feee8069640 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7feee4000c00 tx=0x7feee4035040 comp rx=0 tx=0).stop 2024-07-16T09:53:44.108 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:44.106+0000 7feefec76640 1 -- 172.21.15.156:0/829604272 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7feef80fc360 msgr2=0x7feef810cff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:53:44.108 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:44.106+0000 7feefec76640 1 --2- 172.21.15.156:0/829604272 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7feef80fc360 0x7feef810cff0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7feeec002a50 tx=0x7feeec004600 comp rx=0 tx=0).stop 2024-07-16T09:53:44.108 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:44.106+0000 7feefec76640 1 -- 172.21.15.156:0/829604272 shutdown_connections 2024-07-16T09:53:44.108 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:44.106+0000 7feefec76640 1 --2- 172.21.15.156:0/829604272 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7feee8067190 0x7feee8069640 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:53:44.108 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:44.106+0000 7feefec76640 1 --2- 172.21.15.156:0/829604272 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7feef80fd630 0x7feef810d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:53:44.108 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:44.106+0000 7feefec76640 1 --2- 172.21.15.156:0/829604272 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7feef80fc360 0x7feef810cff0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:53:44.108 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:44.106+0000 7feefec76640 1 -- 172.21.15.156:0/829604272 >> 172.21.15.156:0/829604272 conn(0x7feef80f7b40 msgr2=0x7feef80f91a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:53:44.108 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:44.106+0000 7feefec76640 1 -- 172.21.15.156:0/829604272 shutdown_connections 2024-07-16T09:53:44.108 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:44.106+0000 7feefec76640 1 -- 172.21.15.156:0/829604272 wait complete. 2024-07-16T09:53:44.406 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:53:44 smithi176 ceph-mon[36626]: pgmap v193: 65 pgs: 65 active+clean; 583 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 767 B/s wr, 5 op/s 2024-07-16T09:53:44.424 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:53:44 smithi156 ceph-mon[29302]: pgmap v193: 65 pgs: 65 active+clean; 583 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 767 B/s wr, 5 op/s 2024-07-16T09:53:44.710 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 26c91c26-4358-11ef-bcad-c7b262605968 -e sha1=a6ca461be7cc1aff3658c2c6df4cff975a24449a -- bash -c 'ceph config set mon mon_warn_on_insecure_global_id_reclaim_allowed false --force' 2024-07-16T09:53:44.969 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/26c91c26-4358-11ef-bcad-c7b262605968/mon.smithi156/config 2024-07-16T09:53:46.344 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:53:46 smithi156 ceph-mon[29302]: pgmap v194: 65 pgs: 65 active+clean; 587 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 3.2 KiB/s rd, 1023 B/s wr, 6 op/s 2024-07-16T09:53:46.579 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:46.569+0000 7f06431bd640 1 -- 172.21.15.156:0/2517807144 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f063c0fd650 msgr2=0x7f063c0fdac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:53:46.579 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:46.569+0000 7f06431bd640 1 --2- 172.21.15.156:0/2517807144 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f063c0fd650 0x7f063c0fdac0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f06280048d0 tx=0x7f0628009da0 comp rx=0 tx=0).stop 2024-07-16T09:53:46.580 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:46.569+0000 7f06431bd640 1 -- 172.21.15.156:0/2517807144 shutdown_connections 2024-07-16T09:53:46.580 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:46.569+0000 7f06431bd640 1 --2- 172.21.15.156:0/2517807144 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f063c0fd650 0x7f063c0fdac0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:53:46.580 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:46.569+0000 7f06431bd640 1 --2- 172.21.15.156:0/2517807144 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f063c0fc380 0x7f063c0fc790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:53:46.580 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:46.569+0000 7f06431bd640 1 -- 172.21.15.156:0/2517807144 >> 172.21.15.156:0/2517807144 conn(0x7f063c0f7b40 msgr2=0x7f063c0f9f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:53:46.580 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:46.569+0000 7f06431bd640 1 -- 172.21.15.156:0/2517807144 shutdown_connections 2024-07-16T09:53:46.580 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:46.569+0000 7f06431bd640 1 -- 172.21.15.156:0/2517807144 wait complete. 2024-07-16T09:53:46.580 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:46.569+0000 7f06431bd640 1 Processor -- start 2024-07-16T09:53:46.580 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:46.569+0000 7f06431bd640 1 -- start start 2024-07-16T09:53:46.580 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:46.570+0000 7f06431bd640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f063c0fc380 0x7f063c10cf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:53:46.580 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:46.570+0000 7f06431bd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f063c0fd650 0x7f063c10d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:53:46.581 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:46.570+0000 7f06431bd640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f063c10db10 con 0x7f063c0fd650 2024-07-16T09:53:46.581 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:46.570+0000 7f06431bd640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f063c10e330 con 0x7f063c0fc380 2024-07-16T09:53:46.581 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:46.570+0000 7f063bfff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f063c0fd650 0x7f063c10d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:53:46.581 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:46.570+0000 7f063bfff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f063c0fd650 0x7f063c10d4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:46352/0 (socket says 172.21.15.156:46352) 2024-07-16T09:53:46.581 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:46.570+0000 7f063bfff640 1 -- 172.21.15.156:0/3140177497 learned_addr learned my addr 172.21.15.156:0/3140177497 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T09:53:46.581 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:46.570+0000 7f0640f59640 1 --2- 172.21.15.156:0/3140177497 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f063c0fc380 0x7f063c10cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:53:46.581 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:46.570+0000 7f063bfff640 1 -- 172.21.15.156:0/3140177497 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f063c0fc380 msgr2=0x7f063c10cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:53:46.581 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:46.570+0000 7f063bfff640 1 --2- 172.21.15.156:0/3140177497 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f063c0fc380 0x7f063c10cf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:53:46.581 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:46.570+0000 7f063bfff640 1 -- 172.21.15.156:0/3140177497 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0628005040 con 0x7f063c0fd650 2024-07-16T09:53:46.581 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:46.570+0000 7f063bfff640 1 --2- 172.21.15.156:0/3140177497 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f063c0fd650 0x7f063c10d4c0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f0628009f80 tx=0x7f0628009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:53:46.581 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:46.571+0000 7f06397fa640 1 -- 172.21.15.156:0/3140177497 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0628038070 con 0x7f063c0fd650 2024-07-16T09:53:46.581 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:46.571+0000 7f06431bd640 1 -- 172.21.15.156:0/3140177497 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f063c10e560 con 0x7f063c0fd650 2024-07-16T09:53:46.582 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:46.571+0000 7f06397fa640 1 -- 172.21.15.156:0/3140177497 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0628032970 con 0x7f063c0fd650 2024-07-16T09:53:46.582 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:46.571+0000 7f06397fa640 1 -- 172.21.15.156:0/3140177497 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0628033860 con 0x7f063c0fd650 2024-07-16T09:53:46.582 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:46.571+0000 7f06431bd640 1 -- 172.21.15.156:0/3140177497 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f063c10eaa0 con 0x7f063c0fd650 2024-07-16T09:53:46.582 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:46.572+0000 7f06431bd640 1 -- 172.21.15.156:0/3140177497 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f063c0fb7e0 con 0x7f063c0fd650 2024-07-16T09:53:46.582 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:46.572+0000 7f06397fa640 1 -- 172.21.15.156:0/3140177497 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 19) v1 ==== 85127+0+0 (secure 0 0 0) 0x7f0628030020 con 0x7f063c0fd650 2024-07-16T09:53:46.582 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:46.573+0000 7f06397fa640 1 --2- 172.21.15.156:0/3140177497 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f062c067190 0x7f062c069640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:53:46.582 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:46.573+0000 7f06397fa640 1 -- 172.21.15.156:0/3140177497 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f06280a49c0 con 0x7f063c0fd650 2024-07-16T09:53:46.582 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:46.573+0000 7f0640f59640 1 --2- 172.21.15.156:0/3140177497 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f062c067190 0x7f062c069640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:53:46.582 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:46.573+0000 7f0640f59640 1 --2- 172.21.15.156:0/3140177497 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f062c067190 0x7f062c069640 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f0630004ed0 tx=0x7f063000b040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:53:46.582 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:46.577+0000 7f06397fa640 1 -- 172.21.15.156:0/3140177497 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f0628071d10 con 0x7f063c0fd650 2024-07-16T09:53:46.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:53:46 smithi176 ceph-mon[36626]: pgmap v194: 65 pgs: 65 active+clean; 587 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 3.2 KiB/s rd, 1023 B/s wr, 6 op/s 2024-07-16T09:53:46.745 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:46.744+0000 7f06431bd640 1 -- 172.21.15.156:0/3140177497 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim_allowed}] v 0) v1 -- 0x7f063c10ed80 con 0x7f063c0fd650 2024-07-16T09:53:46.745 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:46.744+0000 7f06397fa640 1 -- 172.21.15.156:0/3140177497 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim_allowed}]=0 v21)=0 v21) v1 ==== 163+0+0 (secure 0 0 0) 0x7f0628075330 con 0x7f063c0fd650 2024-07-16T09:53:46.747 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:46.746+0000 7f06431bd640 1 -- 172.21.15.156:0/3140177497 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f062c067190 msgr2=0x7f062c069640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:53:46.747 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:46.746+0000 7f06431bd640 1 --2- 172.21.15.156:0/3140177497 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f062c067190 0x7f062c069640 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f0630004ed0 tx=0x7f063000b040 comp rx=0 tx=0).stop 2024-07-16T09:53:46.747 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:46.746+0000 7f06431bd640 1 -- 172.21.15.156:0/3140177497 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f063c0fd650 msgr2=0x7f063c10d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:53:46.747 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:46.746+0000 7f06431bd640 1 --2- 172.21.15.156:0/3140177497 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f063c0fd650 0x7f063c10d4c0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f0628009f80 tx=0x7f0628009da0 comp rx=0 tx=0).stop 2024-07-16T09:53:46.747 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:46.746+0000 7f06431bd640 1 -- 172.21.15.156:0/3140177497 shutdown_connections 2024-07-16T09:53:46.747 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:46.746+0000 7f06431bd640 1 --2- 172.21.15.156:0/3140177497 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f062c067190 0x7f062c069640 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:53:46.748 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:46.746+0000 7f06431bd640 1 --2- 172.21.15.156:0/3140177497 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f063c0fd650 0x7f063c10d4c0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:53:46.748 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:46.746+0000 7f06431bd640 1 --2- 172.21.15.156:0/3140177497 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f063c0fc380 0x7f063c10cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:53:46.748 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:46.746+0000 7f06431bd640 1 -- 172.21.15.156:0/3140177497 >> 172.21.15.156:0/3140177497 conn(0x7f063c0f7b40 msgr2=0x7f063c0f91d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:53:46.748 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:46.746+0000 7f06431bd640 1 -- 172.21.15.156:0/3140177497 shutdown_connections 2024-07-16T09:53:46.748 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:46.746+0000 7f06431bd640 1 -- 172.21.15.156:0/3140177497 wait complete. 2024-07-16T09:53:47.391 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 26c91c26-4358-11ef-bcad-c7b262605968 -e sha1=a6ca461be7cc1aff3658c2c6df4cff975a24449a -- bash -c 'ceph config set global log_to_journald false --force' 2024-07-16T09:53:47.648 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/26c91c26-4358-11ef-bcad-c7b262605968/mon.smithi156/config 2024-07-16T09:53:48.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:53:48 smithi156 ceph-mon[29302]: pgmap v195: 65 pgs: 65 active+clean; 587 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.2 KiB/s rd, 767 B/s wr, 4 op/s 2024-07-16T09:53:48.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:53:48 smithi176 ceph-mon[36626]: pgmap v195: 65 pgs: 65 active+clean; 587 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.2 KiB/s rd, 767 B/s wr, 4 op/s 2024-07-16T09:53:49.185 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:49.176+0000 7fe779920640 1 -- 172.21.15.156:0/2308467525 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe7740fc330 msgr2=0x7fe7740fc740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:53:49.186 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:49.176+0000 7fe779920640 1 --2- 172.21.15.156:0/2308467525 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe7740fc330 0x7fe7740fc740 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fe7640048d0 tx=0x7fe764009da0 comp rx=0 tx=0).stop 2024-07-16T09:53:49.186 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:49.176+0000 7fe779920640 1 -- 172.21.15.156:0/2308467525 shutdown_connections 2024-07-16T09:53:49.186 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:49.176+0000 7fe779920640 1 --2- 172.21.15.156:0/2308467525 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe7740fd600 0x7fe7740fda70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:53:49.186 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:49.176+0000 7fe779920640 1 --2- 172.21.15.156:0/2308467525 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe7740fc330 0x7fe7740fc740 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:53:49.186 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:49.176+0000 7fe779920640 1 -- 172.21.15.156:0/2308467525 >> 172.21.15.156:0/2308467525 conn(0x7fe7740f7af0 msgr2=0x7fe7740f9f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:53:49.186 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:49.176+0000 7fe779920640 1 -- 172.21.15.156:0/2308467525 shutdown_connections 2024-07-16T09:53:49.186 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:49.176+0000 7fe779920640 1 -- 172.21.15.156:0/2308467525 wait complete. 2024-07-16T09:53:49.186 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:49.177+0000 7fe779920640 1 Processor -- start 2024-07-16T09:53:49.186 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:49.177+0000 7fe779920640 1 -- start start 2024-07-16T09:53:49.186 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:49.177+0000 7fe779920640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe7740fc330 0x7fe77410cf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:53:49.187 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:49.177+0000 7fe779920640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe7740fd600 0x7fe77410d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:53:49.187 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:49.177+0000 7fe779920640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe77410dad0 con 0x7fe7740fd600 2024-07-16T09:53:49.187 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:49.177+0000 7fe779920640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe77410e2f0 con 0x7fe7740fc330 2024-07-16T09:53:49.187 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:49.177+0000 7fe7727fc640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe7740fd600 0x7fe77410d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:53:49.187 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:49.177+0000 7fe7727fc640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe7740fd600 0x7fe77410d480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:46368/0 (socket says 172.21.15.156:46368) 2024-07-16T09:53:49.187 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:49.177+0000 7fe7727fc640 1 -- 172.21.15.156:0/3061628541 learned_addr learned my addr 172.21.15.156:0/3061628541 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T09:53:49.187 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:49.178+0000 7fe7727fc640 1 -- 172.21.15.156:0/3061628541 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe7740fc330 msgr2=0x7fe77410cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:53:49.187 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:49.178+0000 7fe7727fc640 1 --2- 172.21.15.156:0/3061628541 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe7740fc330 0x7fe77410cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:53:49.187 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:49.178+0000 7fe7727fc640 1 -- 172.21.15.156:0/3061628541 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe764005040 con 0x7fe7740fd600 2024-07-16T09:53:49.187 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:49.178+0000 7fe7727fc640 1 --2- 172.21.15.156:0/3061628541 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe7740fd600 0x7fe77410d480 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fe768002a50 tx=0x7fe768004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:53:49.187 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:49.178+0000 7fe75b7fe640 1 -- 172.21.15.156:0/3061628541 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe768007c30 con 0x7fe7740fd600 2024-07-16T09:53:49.187 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:49.178+0000 7fe75b7fe640 1 -- 172.21.15.156:0/3061628541 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe76800f040 con 0x7fe7740fd600 2024-07-16T09:53:49.188 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:49.178+0000 7fe779920640 1 -- 172.21.15.156:0/3061628541 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe77410e580 con 0x7fe7740fd600 2024-07-16T09:53:49.188 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:49.178+0000 7fe75b7fe640 1 -- 172.21.15.156:0/3061628541 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe76800db50 con 0x7fe7740fd600 2024-07-16T09:53:49.188 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:49.178+0000 7fe779920640 1 -- 172.21.15.156:0/3061628541 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe77410eb20 con 0x7fe7740fd600 2024-07-16T09:53:49.188 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:49.179+0000 7fe779920640 1 -- 172.21.15.156:0/3061628541 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7740fb790 con 0x7fe7740fd600 2024-07-16T09:53:49.188 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:49.179+0000 7fe75b7fe640 1 -- 172.21.15.156:0/3061628541 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 19) v1 ==== 85127+0+0 (secure 0 0 0) 0x7fe768009070 con 0x7fe7740fd600 2024-07-16T09:53:49.188 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:49.180+0000 7fe75b7fe640 1 --2- 172.21.15.156:0/3061628541 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7fe75c067190 0x7fe75c069640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:53:49.188 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:49.180+0000 7fe75b7fe640 1 -- 172.21.15.156:0/3061628541 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5464+0+0 (secure 0 0 0) 0x7fe7680844d0 con 0x7fe7740fd600 2024-07-16T09:53:49.188 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:49.180+0000 7fe772ffd640 1 --2- 172.21.15.156:0/3061628541 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7fe75c067190 0x7fe75c069640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:53:49.188 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:49.180+0000 7fe772ffd640 1 --2- 172.21.15.156:0/3061628541 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7fe75c067190 0x7fe75c069640 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fe764009f80 tx=0x7fe764009da0 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:53:49.188 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:49.184+0000 7fe75b7fe640 1 -- 172.21.15.156:0/3061628541 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fe76804d250 con 0x7fe7740fd600 2024-07-16T09:53:49.355 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:49.353+0000 7fe779920640 1 -- 172.21.15.156:0/3061628541 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command([{prefix=config set, name=log_to_journald}] v 0) v1 -- 0x7fe77404b530 con 0x7fe7740fd600 2024-07-16T09:53:49.355 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:49.353+0000 7fe75b7fe640 1 -- 172.21.15.156:0/3061628541 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{prefix=config set, name=log_to_journald}]=0 v21)=0 v21) v1 ==== 135+0+0 (secure 0 0 0) 0x7fe768082e60 con 0x7fe7740fd600 2024-07-16T09:53:49.357 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:49.355+0000 7fe779920640 1 -- 172.21.15.156:0/3061628541 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7fe75c067190 msgr2=0x7fe75c069640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:53:49.357 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:49.355+0000 7fe779920640 1 --2- 172.21.15.156:0/3061628541 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7fe75c067190 0x7fe75c069640 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fe764009f80 tx=0x7fe764009da0 comp rx=0 tx=0).stop 2024-07-16T09:53:49.358 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:49.355+0000 7fe779920640 1 -- 172.21.15.156:0/3061628541 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe7740fd600 msgr2=0x7fe77410d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:53:49.358 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:49.355+0000 7fe779920640 1 --2- 172.21.15.156:0/3061628541 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe7740fd600 0x7fe77410d480 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fe768002a50 tx=0x7fe768004600 comp rx=0 tx=0).stop 2024-07-16T09:53:49.358 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:49.355+0000 7fe779920640 1 -- 172.21.15.156:0/3061628541 shutdown_connections 2024-07-16T09:53:49.358 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:49.355+0000 7fe779920640 1 --2- 172.21.15.156:0/3061628541 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7fe75c067190 0x7fe75c069640 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:53:49.358 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:49.355+0000 7fe779920640 1 --2- 172.21.15.156:0/3061628541 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe7740fd600 0x7fe77410d480 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:53:49.358 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:49.355+0000 7fe779920640 1 --2- 172.21.15.156:0/3061628541 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe7740fc330 0x7fe77410cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:53:49.358 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:49.355+0000 7fe779920640 1 -- 172.21.15.156:0/3061628541 >> 172.21.15.156:0/3061628541 conn(0x7fe7740f7af0 msgr2=0x7fe7740f91b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:53:49.359 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:49.356+0000 7fe779920640 1 -- 172.21.15.156:0/3061628541 shutdown_connections 2024-07-16T09:53:49.359 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:49.356+0000 7fe779920640 1 -- 172.21.15.156:0/3061628541 wait complete. 2024-07-16T09:53:50.032 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 26c91c26-4358-11ef-bcad-c7b262605968 -e sha1=a6ca461be7cc1aff3658c2c6df4cff975a24449a -- bash -c 'ceph orch upgrade start --image quay.ceph.io/ceph-ci/ceph:$sha1' 2024-07-16T09:53:50.290 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/26c91c26-4358-11ef-bcad-c7b262605968/mon.smithi156/config 2024-07-16T09:53:50.413 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:53:50 smithi156 ceph-mon[29302]: pgmap v196: 65 pgs: 65 active+clean; 587 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.7 KiB/s rd, 853 B/s wr, 5 op/s 2024-07-16T09:53:50.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:53:50 smithi176 ceph-mon[36626]: pgmap v196: 65 pgs: 65 active+clean; 587 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.7 KiB/s rd, 853 B/s wr, 5 op/s 2024-07-16T09:53:51.826 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:51.823+0000 7f518cf9c640 1 -- 172.21.15.156:0/3714859755 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f51880fb260 msgr2=0x7f51880fb670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:53:51.826 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:51.823+0000 7f518cf9c640 1 --2- 172.21.15.156:0/3714859755 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f51880fb260 0x7f51880fb670 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f5180009c50 tx=0x7f5180004d10 comp rx=0 tx=0).stop 2024-07-16T09:53:51.827 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:51.823+0000 7f518cf9c640 1 -- 172.21.15.156:0/3714859755 shutdown_connections 2024-07-16T09:53:51.827 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:51.823+0000 7f518cf9c640 1 --2- 172.21.15.156:0/3714859755 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f51880fc530 0x7f51880fc9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:53:51.827 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:51.823+0000 7f518cf9c640 1 --2- 172.21.15.156:0/3714859755 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f51880fb260 0x7f51880fb670 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:53:51.827 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:51.823+0000 7f518cf9c640 1 -- 172.21.15.156:0/3714859755 >> 172.21.15.156:0/3714859755 conn(0x7f51880f6a20 msgr2=0x7f51880f8e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:53:51.827 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:51.823+0000 7f518cf9c640 1 -- 172.21.15.156:0/3714859755 shutdown_connections 2024-07-16T09:53:51.827 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:51.823+0000 7f518cf9c640 1 -- 172.21.15.156:0/3714859755 wait complete. 2024-07-16T09:53:51.827 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:51.824+0000 7f518cf9c640 1 Processor -- start 2024-07-16T09:53:51.827 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:51.824+0000 7f518cf9c640 1 -- start start 2024-07-16T09:53:51.828 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:51.824+0000 7f518cf9c640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f51880fb260 0x7f51880ff330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:53:51.828 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:51.824+0000 7f518cf9c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f51880fc530 0x7f51880fd870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:53:51.828 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:51.824+0000 7f518cf9c640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51880fde40 con 0x7f51880fc530 2024-07-16T09:53:51.828 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:51.824+0000 7f518cf9c640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51880fdf80 con 0x7f51880fb260 2024-07-16T09:53:51.828 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:51.824+0000 7f5185d9b640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f51880fc530 0x7f51880fd870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:53:51.828 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:51.824+0000 7f5185d9b640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f51880fc530 0x7f51880fd870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:46388/0 (socket says 172.21.15.156:46388) 2024-07-16T09:53:51.828 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:51.824+0000 7f5185d9b640 1 -- 172.21.15.156:0/2284312519 learned_addr learned my addr 172.21.15.156:0/2284312519 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T09:53:51.828 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:51.824+0000 7f518659c640 1 --2- 172.21.15.156:0/2284312519 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f51880fb260 0x7f51880ff330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:53:51.828 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:51.825+0000 7f5185d9b640 1 -- 172.21.15.156:0/2284312519 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f51880fb260 msgr2=0x7f51880ff330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:53:51.829 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:51.825+0000 7f5185d9b640 1 --2- 172.21.15.156:0/2284312519 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f51880fb260 0x7f51880ff330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:53:51.829 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:51.825+0000 7f5185d9b640 1 -- 172.21.15.156:0/2284312519 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51800098b0 con 0x7f51880fc530 2024-07-16T09:53:51.829 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:51.825+0000 7f5185d9b640 1 --2- 172.21.15.156:0/2284312519 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f51880fc530 0x7f51880fd870 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f517c00b790 tx=0x7f517c0091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:53:51.829 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:51.825+0000 7f5176ffd640 1 -- 172.21.15.156:0/2284312519 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f517c00c700 con 0x7f51880fc530 2024-07-16T09:53:51.829 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:51.825+0000 7f5176ffd640 1 -- 172.21.15.156:0/2284312519 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f517c00cd30 con 0x7f51880fc530 2024-07-16T09:53:51.829 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:51.825+0000 7f518cf9c640 1 -- 172.21.15.156:0/2284312519 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51880fe260 con 0x7f51880fc530 2024-07-16T09:53:51.829 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:51.825+0000 7f5176ffd640 1 -- 172.21.15.156:0/2284312519 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f517c013370 con 0x7f51880fc530 2024-07-16T09:53:51.829 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:51.825+0000 7f518cf9c640 1 -- 172.21.15.156:0/2284312519 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5188115080 con 0x7f51880fc530 2024-07-16T09:53:51.830 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:51.826+0000 7f518cf9c640 1 -- 172.21.15.156:0/2284312519 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51880fa6c0 con 0x7f51880fc530 2024-07-16T09:53:51.830 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:51.827+0000 7f5176ffd640 1 -- 172.21.15.156:0/2284312519 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 19) v1 ==== 85127+0+0 (secure 0 0 0) 0x7f517c012070 con 0x7f51880fc530 2024-07-16T09:53:51.830 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:51.827+0000 7f5176ffd640 1 --2- 172.21.15.156:0/2284312519 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f5170067190 0x7f5170069640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:53:51.830 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:51.827+0000 7f518659c640 1 --2- 172.21.15.156:0/2284312519 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f5170067190 0x7f5170069640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:53:51.830 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:51.827+0000 7f5176ffd640 1 -- 172.21.15.156:0/2284312519 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f517c084560 con 0x7f51880fc530 2024-07-16T09:53:51.830 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:51.827+0000 7f518659c640 1 --2- 172.21.15.156:0/2284312519 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f5170067190 0x7f5170069640 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f5180000f80 tx=0x7f5180004d10 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:53:51.832 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:51.831+0000 7f5176ffd640 1 -- 172.21.15.156:0/2284312519 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f517c051800 con 0x7f51880fc530 2024-07-16T09:53:52.012 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:52.011+0000 7f518cf9c640 1 -- 172.21.15.156:0/2284312519 --> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] -- mgr_command(tid 0: {"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:a6ca461be7cc1aff3658c2c6df4cff975a24449a", "target": ["mon-mgr", ""]}) v1 -- 0x7f518805d960 con 0x7f5170067190 2024-07-16T09:53:52.017 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:52.016+0000 7f5176ffd640 1 -- 172.21.15.156:0/2284312519 <== mgr.14208 v2:172.21.15.156:6800/2615741717 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+89 (secure 0 0 0) 0x7f518805d960 con 0x7f5170067190 2024-07-16T09:53:52.018 INFO:teuthology.orchestra.run.smithi156.stdout:Initiating upgrade to quay.ceph.io/ceph-ci/ceph:a6ca461be7cc1aff3658c2c6df4cff975a24449a 2024-07-16T09:53:52.020 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:52.019+0000 7f518cf9c640 1 -- 172.21.15.156:0/2284312519 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f5170067190 msgr2=0x7f5170069640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:53:52.020 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:52.019+0000 7f518cf9c640 1 --2- 172.21.15.156:0/2284312519 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f5170067190 0x7f5170069640 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f5180000f80 tx=0x7f5180004d10 comp rx=0 tx=0).stop 2024-07-16T09:53:52.020 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:52.019+0000 7f518cf9c640 1 -- 172.21.15.156:0/2284312519 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f51880fc530 msgr2=0x7f51880fd870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:53:52.020 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:52.019+0000 7f518cf9c640 1 --2- 172.21.15.156:0/2284312519 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f51880fc530 0x7f51880fd870 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f517c00b790 tx=0x7f517c0091d0 comp rx=0 tx=0).stop 2024-07-16T09:53:52.021 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:52.019+0000 7f518cf9c640 1 -- 172.21.15.156:0/2284312519 shutdown_connections 2024-07-16T09:53:52.021 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:52.019+0000 7f518cf9c640 1 --2- 172.21.15.156:0/2284312519 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f5170067190 0x7f5170069640 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:53:52.021 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:52.019+0000 7f518cf9c640 1 --2- 172.21.15.156:0/2284312519 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f51880fc530 0x7f51880fd870 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:53:52.021 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:52.019+0000 7f518cf9c640 1 --2- 172.21.15.156:0/2284312519 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f51880fb260 0x7f51880ff330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:53:52.021 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:52.019+0000 7f518cf9c640 1 -- 172.21.15.156:0/2284312519 >> 172.21.15.156:0/2284312519 conn(0x7f51880f6a20 msgr2=0x7f5188065e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:53:52.021 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:52.019+0000 7f518cf9c640 1 -- 172.21.15.156:0/2284312519 shutdown_connections 2024-07-16T09:53:52.021 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:52.019+0000 7f518cf9c640 1 -- 172.21.15.156:0/2284312519 wait complete. 2024-07-16T09:53:52.372 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:53:52 smithi156 ceph-mon[29302]: pgmap v197: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.6 KiB/s rd, 1.1 KiB/s wr, 5 op/s 2024-07-16T09:53:52.373 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:53:52 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:53:52.373 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:53:52 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-16T09:53:52.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:53:52 smithi176 ceph-mon[36626]: pgmap v197: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.6 KiB/s rd, 1.1 KiB/s wr, 5 op/s 2024-07-16T09:53:52.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:53:52 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:53:52.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:53:52 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-16T09:53:52.744 INFO:teuthology.task.sequential:In sequential, running task cephadm.shell... 2024-07-16T09:53:52.745 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi156.front.sepia.ceph.com 2024-07-16T09:53:52.745 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 26c91c26-4358-11ef-bcad-c7b262605968 -e sha1=a6ca461be7cc1aff3658c2c6df4cff975a24449a -- bash -c 'while ceph orch upgrade status | jq '"'"'.in_progress'"'"' | grep true && ! ceph orch upgrade status | jq '"'"'.message'"'"' | grep Error ; do ceph orch ps ; ceph versions ; ceph fs dump; ceph orch upgrade status ; ceph health detail ; sleep 30 ; done' 2024-07-16T09:53:53.015 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/26c91c26-4358-11ef-bcad-c7b262605968/mon.smithi156/config 2024-07-16T09:53:53.402 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:53:53 smithi156 ceph-mon[29302]: from='client.14526 -' entity='client.admin' cmd=[{"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:a6ca461be7cc1aff3658c2c6df4cff975a24449a", "target": ["mon-mgr", ""]}]: dispatch 2024-07-16T09:53:53.402 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:53:53 smithi156 ceph-mon[29302]: Upgrade: Started with target quay.ceph.io/ceph-ci/ceph:a6ca461be7cc1aff3658c2c6df4cff975a24449a 2024-07-16T09:53:53.402 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:53:53 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-16T09:53:53.402 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:53:53 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-16T09:53:53.402 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:53:53 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:53:53.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:53:53 smithi176 ceph-mon[36626]: from='client.14526 -' entity='client.admin' cmd=[{"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:a6ca461be7cc1aff3658c2c6df4cff975a24449a", "target": ["mon-mgr", ""]}]: dispatch 2024-07-16T09:53:53.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:53:53 smithi176 ceph-mon[36626]: Upgrade: Started with target quay.ceph.io/ceph-ci/ceph:a6ca461be7cc1aff3658c2c6df4cff975a24449a 2024-07-16T09:53:53.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:53:53 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-16T09:53:53.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:53:53 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-16T09:53:53.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:53:53 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:53:54.372 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:53:54 smithi156 ceph-mon[29302]: Upgrade: First pull of quay.ceph.io/ceph-ci/ceph:a6ca461be7cc1aff3658c2c6df4cff975a24449a 2024-07-16T09:53:54.373 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:53:54 smithi156 ceph-mon[29302]: pgmap v198: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 1023 B/s wr, 4 op/s 2024-07-16T09:53:54.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:53:54 smithi176 ceph-mon[36626]: Upgrade: First pull of quay.ceph.io/ceph-ci/ceph:a6ca461be7cc1aff3658c2c6df4cff975a24449a 2024-07-16T09:53:54.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:53:54 smithi176 ceph-mon[36626]: pgmap v198: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 1023 B/s wr, 4 op/s 2024-07-16T09:53:54.769 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:54.732+0000 7fe4b39ed640 1 -- 172.21.15.156:0/2328716571 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe4ac0662e0 msgr2=0x7fe4ac0fef40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:53:54.769 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:54.732+0000 7fe4b39ed640 1 --2- 172.21.15.156:0/2328716571 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe4ac0662e0 0x7fe4ac0fef40 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fe4a80048d0 tx=0x7fe4a8009da0 comp rx=0 tx=0).stop 2024-07-16T09:53:54.769 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:54.732+0000 7fe4b39ed640 1 -- 172.21.15.156:0/2328716571 shutdown_connections 2024-07-16T09:53:54.769 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:54.733+0000 7fe4b39ed640 1 --2- 172.21.15.156:0/2328716571 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe4ac0ff480 0x7fe4ac101860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:53:54.769 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:54.733+0000 7fe4b39ed640 1 --2- 172.21.15.156:0/2328716571 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe4ac0662e0 0x7fe4ac0fef40 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:53:54.769 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:54.733+0000 7fe4b39ed640 1 -- 172.21.15.156:0/2328716571 >> 172.21.15.156:0/2328716571 conn(0x7fe4ac0f6a10 msgr2=0x7fe4ac0f8e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:53:54.770 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:54.733+0000 7fe4b39ed640 1 -- 172.21.15.156:0/2328716571 shutdown_connections 2024-07-16T09:53:54.770 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:54.733+0000 7fe4b39ed640 1 -- 172.21.15.156:0/2328716571 wait complete. 2024-07-16T09:53:54.770 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:54.733+0000 7fe4b39ed640 1 Processor -- start 2024-07-16T09:53:54.770 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:54.733+0000 7fe4b39ed640 1 -- start start 2024-07-16T09:53:54.770 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:54.733+0000 7fe4b39ed640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe4ac0662e0 0x7fe4ac0fc950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:53:54.770 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:54.733+0000 7fe4b39ed640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe4ac0ff480 0x7fe4ac0fae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:53:54.770 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:54.733+0000 7fe4b39ed640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4ac0fb3d0 con 0x7fe4ac0662e0 2024-07-16T09:53:54.770 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:54.733+0000 7fe4b39ed640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4ac0fb510 con 0x7fe4ac0ff480 2024-07-16T09:53:54.770 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:54.733+0000 7fe4b1789640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe4ac0662e0 0x7fe4ac0fc950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:53:54.770 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:54.734+0000 7fe4b1789640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe4ac0662e0 0x7fe4ac0fc950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:36440/0 (socket says 172.21.15.156:36440) 2024-07-16T09:53:54.770 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:54.734+0000 7fe4b1789640 1 -- 172.21.15.156:0/3260393343 learned_addr learned my addr 172.21.15.156:0/3260393343 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T09:53:54.770 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:54.734+0000 7fe4b0f88640 1 --2- 172.21.15.156:0/3260393343 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe4ac0ff480 0x7fe4ac0fae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:53:54.771 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:54.734+0000 7fe4b1789640 1 -- 172.21.15.156:0/3260393343 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe4ac0ff480 msgr2=0x7fe4ac0fae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:53:54.771 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:54.734+0000 7fe4b1789640 1 --2- 172.21.15.156:0/3260393343 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe4ac0ff480 0x7fe4ac0fae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:53:54.771 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:54.734+0000 7fe4b1789640 1 -- 172.21.15.156:0/3260393343 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4a8005040 con 0x7fe4ac0662e0 2024-07-16T09:53:54.771 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:54.734+0000 7fe4b1789640 1 --2- 172.21.15.156:0/3260393343 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe4ac0662e0 0x7fe4ac0fc950 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fe4a8009f80 tx=0x7fe4a8009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:53:54.771 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:54.734+0000 7fe4a1ffb640 1 -- 172.21.15.156:0/3260393343 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe4a8038070 con 0x7fe4ac0662e0 2024-07-16T09:53:54.771 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:54.734+0000 7fe4b39ed640 1 -- 172.21.15.156:0/3260393343 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4ac0fb790 con 0x7fe4ac0662e0 2024-07-16T09:53:54.771 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:54.734+0000 7fe4a1ffb640 1 -- 172.21.15.156:0/3260393343 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe4a8008ee0 con 0x7fe4ac0662e0 2024-07-16T09:53:54.771 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:54.734+0000 7fe4a1ffb640 1 -- 172.21.15.156:0/3260393343 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe4a8033780 con 0x7fe4ac0662e0 2024-07-16T09:53:54.771 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:54.735+0000 7fe4b39ed640 1 -- 172.21.15.156:0/3260393343 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4ac114ff0 con 0x7fe4ac0662e0 2024-07-16T09:53:54.771 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:54.736+0000 7fe4b39ed640 1 -- 172.21.15.156:0/3260393343 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4ac0fa690 con 0x7fe4ac0662e0 2024-07-16T09:53:54.771 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:54.736+0000 7fe4a1ffb640 1 -- 172.21.15.156:0/3260393343 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 19) v1 ==== 85127+0+0 (secure 0 0 0) 0x7fe4a8030020 con 0x7fe4ac0662e0 2024-07-16T09:53:54.771 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:54.737+0000 7fe4a1ffb640 1 --2- 172.21.15.156:0/3260393343 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7fe498067190 0x7fe498069640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:53:54.771 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:54.737+0000 7fe4a1ffb640 1 -- 172.21.15.156:0/3260393343 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5464+0+0 (secure 0 0 0) 0x7fe4a80a47a0 con 0x7fe4ac0662e0 2024-07-16T09:53:54.772 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:54.737+0000 7fe4b0f88640 1 --2- 172.21.15.156:0/3260393343 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7fe498067190 0x7fe498069640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:53:54.772 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:54.737+0000 7fe4b0f88640 1 --2- 172.21.15.156:0/3260393343 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7fe498067190 0x7fe498069640 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fe49c009af0 tx=0x7fe49c00b040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:53:54.772 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:54.742+0000 7fe4a1ffb640 1 -- 172.21.15.156:0/3260393343 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fe4a8071b70 con 0x7fe4ac0662e0 2024-07-16T09:53:54.970 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:54.968+0000 7fe4b39ed640 1 -- 172.21.15.156:0/3260393343 --> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe4ac0f8500 con 0x7fe498067190 2024-07-16T09:53:54.970 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:54.969+0000 7fe4a1ffb640 1 -- 172.21.15.156:0/3260393343 <== mgr.14208 v2:172.21.15.156:6800/2615741717 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+358 (secure 0 0 0) 0x7fe4ac0f8500 con 0x7fe498067190 2024-07-16T09:53:54.973 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:54.972+0000 7fe4b39ed640 1 -- 172.21.15.156:0/3260393343 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7fe498067190 msgr2=0x7fe498069640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:53:54.973 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:54.972+0000 7fe4b39ed640 1 --2- 172.21.15.156:0/3260393343 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7fe498067190 0x7fe498069640 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fe49c009af0 tx=0x7fe49c00b040 comp rx=0 tx=0).stop 2024-07-16T09:53:54.973 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:54.972+0000 7fe4b39ed640 1 -- 172.21.15.156:0/3260393343 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe4ac0662e0 msgr2=0x7fe4ac0fc950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:53:54.973 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:54.972+0000 7fe4b39ed640 1 --2- 172.21.15.156:0/3260393343 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe4ac0662e0 0x7fe4ac0fc950 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fe4a8009f80 tx=0x7fe4a8009da0 comp rx=0 tx=0).stop 2024-07-16T09:53:54.973 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:54.972+0000 7fe4b39ed640 1 -- 172.21.15.156:0/3260393343 shutdown_connections 2024-07-16T09:53:54.973 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:54.972+0000 7fe4b39ed640 1 --2- 172.21.15.156:0/3260393343 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7fe498067190 0x7fe498069640 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:53:54.974 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:54.972+0000 7fe4b39ed640 1 --2- 172.21.15.156:0/3260393343 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe4ac0ff480 0x7fe4ac0fae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:53:54.974 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:54.972+0000 7fe4b39ed640 1 --2- 172.21.15.156:0/3260393343 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe4ac0662e0 0x7fe4ac0fc950 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:53:54.974 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:54.972+0000 7fe4b39ed640 1 -- 172.21.15.156:0/3260393343 >> 172.21.15.156:0/3260393343 conn(0x7fe4ac0f6a10 msgr2=0x7fe4ac0f6de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:53:54.974 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:54.972+0000 7fe4b39ed640 1 -- 172.21.15.156:0/3260393343 shutdown_connections 2024-07-16T09:53:54.974 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:54.972+0000 7fe4b39ed640 1 -- 172.21.15.156:0/3260393343 wait complete. 2024-07-16T09:53:54.984 INFO:teuthology.orchestra.run.smithi156.stdout:true 2024-07-16T09:53:55.073 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:55.071+0000 7f9664e65640 1 -- 172.21.15.156:0/1706790831 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f96600fc330 msgr2=0x7f96600fc740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:53:55.073 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:55.071+0000 7f9664e65640 1 --2- 172.21.15.156:0/1706790831 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f96600fc330 0x7f96600fc740 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f96500048d0 tx=0x7f9650009da0 comp rx=0 tx=0).stop 2024-07-16T09:53:55.074 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:55.071+0000 7f9664e65640 1 -- 172.21.15.156:0/1706790831 shutdown_connections 2024-07-16T09:53:55.074 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:55.071+0000 7f9664e65640 1 --2- 172.21.15.156:0/1706790831 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f96600fd600 0x7f96600fda70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:53:55.074 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:55.071+0000 7f9664e65640 1 --2- 172.21.15.156:0/1706790831 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f96600fc330 0x7f96600fc740 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:53:55.074 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:55.071+0000 7f9664e65640 1 -- 172.21.15.156:0/1706790831 >> 172.21.15.156:0/1706790831 conn(0x7f96600f7af0 msgr2=0x7f96600f9f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:53:55.074 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:55.071+0000 7f9664e65640 1 -- 172.21.15.156:0/1706790831 shutdown_connections 2024-07-16T09:53:55.074 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:55.071+0000 7f9664e65640 1 -- 172.21.15.156:0/1706790831 wait complete. 2024-07-16T09:53:55.074 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:55.072+0000 7f9664e65640 1 Processor -- start 2024-07-16T09:53:55.074 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:55.072+0000 7f9664e65640 1 -- start start 2024-07-16T09:53:55.074 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:55.072+0000 7f9664e65640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f96600fc330 0x7f966010cf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:53:55.074 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:55.072+0000 7f9664e65640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f96600fd600 0x7f966010d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:53:55.074 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:55.072+0000 7f9664e65640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f966010dad0 con 0x7f96600fc330 2024-07-16T09:53:55.074 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:55.072+0000 7f9664e65640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f966010e2f0 con 0x7f96600fd600 2024-07-16T09:53:55.075 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:55.072+0000 7f965e59c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f96600fc330 0x7f966010cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:53:55.076 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:55.072+0000 7f965e59c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f96600fc330 0x7f966010cf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:36462/0 (socket says 172.21.15.156:36462) 2024-07-16T09:53:55.076 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:55.072+0000 7f965e59c640 1 -- 172.21.15.156:0/1451027156 learned_addr learned my addr 172.21.15.156:0/1451027156 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T09:53:55.076 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:55.072+0000 7f9657fff640 1 --2- 172.21.15.156:0/1451027156 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f96600fd600 0x7f966010d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:53:55.076 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:55.073+0000 7f965e59c640 1 -- 172.21.15.156:0/1451027156 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f96600fd600 msgr2=0x7f966010d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:53:55.076 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:55.073+0000 7f965e59c640 1 --2- 172.21.15.156:0/1451027156 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f96600fd600 0x7f966010d480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:53:55.076 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:55.073+0000 7f965e59c640 1 -- 172.21.15.156:0/1451027156 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9650005040 con 0x7f96600fc330 2024-07-16T09:53:55.076 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:55.073+0000 7f9657fff640 1 --2- 172.21.15.156:0/1451027156 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f96600fd600 0x7f966010d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-16T09:53:55.076 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:55.073+0000 7f965e59c640 1 --2- 172.21.15.156:0/1451027156 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f96600fc330 0x7f966010cf40 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f9650009f80 tx=0x7f9650009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:53:55.076 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:55.073+0000 7f9656ffd640 1 -- 172.21.15.156:0/1451027156 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9650038070 con 0x7f96600fc330 2024-07-16T09:53:55.076 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:55.073+0000 7f9656ffd640 1 -- 172.21.15.156:0/1451027156 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9650032970 con 0x7f96600fc330 2024-07-16T09:53:55.076 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:55.073+0000 7f9656ffd640 1 -- 172.21.15.156:0/1451027156 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9650033860 con 0x7f96600fc330 2024-07-16T09:53:55.076 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:55.073+0000 7f9664e65640 1 -- 172.21.15.156:0/1451027156 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f966010e520 con 0x7f96600fc330 2024-07-16T09:53:55.077 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:55.073+0000 7f9664e65640 1 -- 172.21.15.156:0/1451027156 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f966010e9e0 con 0x7f96600fc330 2024-07-16T09:53:55.077 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:55.074+0000 7f9664e65640 1 -- 172.21.15.156:0/1451027156 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96600fb790 con 0x7f96600fc330 2024-07-16T09:53:55.077 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:55.075+0000 7f9656ffd640 1 -- 172.21.15.156:0/1451027156 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 19) v1 ==== 85127+0+0 (secure 0 0 0) 0x7f9650030020 con 0x7f96600fc330 2024-07-16T09:53:55.077 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:55.075+0000 7f9656ffd640 1 --2- 172.21.15.156:0/1451027156 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f964c067250 0x7f964c069700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:53:55.077 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:55.076+0000 7f9656ffd640 1 -- 172.21.15.156:0/1451027156 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f96500a4800 con 0x7f96600fc330 2024-07-16T09:53:55.078 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:55.076+0000 7f9657fff640 1 --2- 172.21.15.156:0/1451027156 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f964c067250 0x7f964c069700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:53:55.078 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:55.076+0000 7f9657fff640 1 --2- 172.21.15.156:0/1451027156 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f964c067250 0x7f964c069700 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f966010ccf0 tx=0x7f964800b040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:53:55.081 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:55.079+0000 7f9656ffd640 1 -- 172.21.15.156:0/1451027156 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f9650071c70 con 0x7f96600fc330 2024-07-16T09:53:55.331 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:55.329+0000 7f9664e65640 1 -- 172.21.15.156:0/1451027156 --> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f966010ece0 con 0x7f964c067250 2024-07-16T09:53:55.331 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:53:55 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-16T09:53:55.332 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:55.331+0000 7f9656ffd640 1 -- 172.21.15.156:0/1451027156 <== mgr.14208 v2:172.21.15.156:6800/2615741717 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+358 (secure 0 0 0) 0x7f966010ece0 con 0x7f964c067250 2024-07-16T09:53:55.335 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:55.334+0000 7f9664e65640 1 -- 172.21.15.156:0/1451027156 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f964c067250 msgr2=0x7f964c069700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:53:55.335 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:55.334+0000 7f9664e65640 1 --2- 172.21.15.156:0/1451027156 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f964c067250 0x7f964c069700 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f966010ccf0 tx=0x7f964800b040 comp rx=0 tx=0).stop 2024-07-16T09:53:55.335 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:55.334+0000 7f9664e65640 1 -- 172.21.15.156:0/1451027156 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f96600fc330 msgr2=0x7f966010cf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:53:55.335 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:55.334+0000 7f9664e65640 1 --2- 172.21.15.156:0/1451027156 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f96600fc330 0x7f966010cf40 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f9650009f80 tx=0x7f9650009da0 comp rx=0 tx=0).stop 2024-07-16T09:53:55.335 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:55.334+0000 7f9664e65640 1 -- 172.21.15.156:0/1451027156 shutdown_connections 2024-07-16T09:53:55.335 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:55.334+0000 7f9664e65640 1 --2- 172.21.15.156:0/1451027156 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f964c067250 0x7f964c069700 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:53:55.335 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:55.334+0000 7f9664e65640 1 --2- 172.21.15.156:0/1451027156 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f96600fd600 0x7f966010d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:53:55.336 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:55.334+0000 7f9664e65640 1 --2- 172.21.15.156:0/1451027156 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f96600fc330 0x7f966010cf40 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:53:55.336 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:55.334+0000 7f9664e65640 1 -- 172.21.15.156:0/1451027156 >> 172.21.15.156:0/1451027156 conn(0x7f96600f7af0 msgr2=0x7f96600f91d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:53:55.336 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:55.334+0000 7f9664e65640 1 -- 172.21.15.156:0/1451027156 shutdown_connections 2024-07-16T09:53:55.336 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:55.334+0000 7f9664e65640 1 -- 172.21.15.156:0/1451027156 wait complete. 2024-07-16T09:53:55.428 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:55.426+0000 7f5a43ffd640 1 -- 172.21.15.156:0/3910433654 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5a3c0fb240 msgr2=0x7f5a3c0fb650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:53:55.428 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:55.426+0000 7f5a43ffd640 1 --2- 172.21.15.156:0/3910433654 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5a3c0fb240 0x7f5a3c0fb650 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f5a380048d0 tx=0x7f5a38009da0 comp rx=0 tx=0).stop 2024-07-16T09:53:55.428 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:55.426+0000 7f5a43ffd640 1 -- 172.21.15.156:0/3910433654 shutdown_connections 2024-07-16T09:53:55.428 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:55.426+0000 7f5a43ffd640 1 --2- 172.21.15.156:0/3910433654 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5a3c0fc510 0x7f5a3c0fc980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:53:55.428 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:55.426+0000 7f5a43ffd640 1 --2- 172.21.15.156:0/3910433654 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5a3c0fb240 0x7f5a3c0fb650 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:53:55.428 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:55.426+0000 7f5a43ffd640 1 -- 172.21.15.156:0/3910433654 >> 172.21.15.156:0/3910433654 conn(0x7f5a3c0f6a20 msgr2=0x7f5a3c0f8e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:53:55.429 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:55.426+0000 7f5a43ffd640 1 -- 172.21.15.156:0/3910433654 shutdown_connections 2024-07-16T09:53:55.429 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:55.426+0000 7f5a43ffd640 1 -- 172.21.15.156:0/3910433654 wait complete. 2024-07-16T09:53:55.429 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:55.427+0000 7f5a43ffd640 1 Processor -- start 2024-07-16T09:53:55.429 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:55.427+0000 7f5a43ffd640 1 -- start start 2024-07-16T09:53:55.430 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:55.427+0000 7f5a43ffd640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5a3c0fb240 0x7f5a3c10bf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:53:55.430 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:55.427+0000 7f5a43ffd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5a3c0fc510 0x7f5a3c10c440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:53:55.430 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:55.427+0000 7f5a43ffd640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a3c10d170 con 0x7f5a3c0fc510 2024-07-16T09:53:55.430 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:55.427+0000 7f5a43ffd640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a3c10d2e0 con 0x7f5a3c0fb240 2024-07-16T09:53:55.430 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:55.427+0000 7f5a41598640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5a3c0fc510 0x7f5a3c10c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:53:55.430 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:55.427+0000 7f5a41598640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5a3c0fc510 0x7f5a3c10c440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:36480/0 (socket says 172.21.15.156:36480) 2024-07-16T09:53:55.430 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:55.427+0000 7f5a41598640 1 -- 172.21.15.156:0/2286457589 learned_addr learned my addr 172.21.15.156:0/2286457589 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T09:53:55.430 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:55.427+0000 7f5a41d99640 1 --2- 172.21.15.156:0/2286457589 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5a3c0fb240 0x7f5a3c10bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:53:55.431 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:55.428+0000 7f5a41598640 1 -- 172.21.15.156:0/2286457589 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5a3c0fb240 msgr2=0x7f5a3c10bf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:53:55.431 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:55.428+0000 7f5a41598640 1 --2- 172.21.15.156:0/2286457589 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5a3c0fb240 0x7f5a3c10bf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:53:55.431 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:55.428+0000 7f5a41598640 1 -- 172.21.15.156:0/2286457589 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a38005040 con 0x7f5a3c0fc510 2024-07-16T09:53:55.431 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:55.428+0000 7f5a41d99640 1 --2- 172.21.15.156:0/2286457589 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5a3c0fb240 0x7f5a3c10bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-16T09:53:55.431 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:55.428+0000 7f5a41598640 1 --2- 172.21.15.156:0/2286457589 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5a3c0fc510 0x7f5a3c10c440 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f5a2c010d80 tx=0x7f5a2c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:53:55.431 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:55.428+0000 7f5a327fc640 1 -- 172.21.15.156:0/2286457589 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5a2c00cd10 con 0x7f5a3c0fc510 2024-07-16T09:53:55.431 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:55.428+0000 7f5a43ffd640 1 -- 172.21.15.156:0/2286457589 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a3c10d560 con 0x7f5a3c0fc510 2024-07-16T09:53:55.431 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:55.428+0000 7f5a327fc640 1 -- 172.21.15.156:0/2286457589 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5a2c00ce70 con 0x7f5a3c0fc510 2024-07-16T09:53:55.431 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:55.428+0000 7f5a327fc640 1 -- 172.21.15.156:0/2286457589 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5a2c012430 con 0x7f5a3c0fc510 2024-07-16T09:53:55.431 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:55.428+0000 7f5a43ffd640 1 -- 172.21.15.156:0/2286457589 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a3c10db00 con 0x7f5a3c0fc510 2024-07-16T09:53:55.432 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:55.429+0000 7f5a43ffd640 1 -- 172.21.15.156:0/2286457589 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a3c0fa6a0 con 0x7f5a3c0fc510 2024-07-16T09:53:55.432 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:55.430+0000 7f5a327fc640 1 -- 172.21.15.156:0/2286457589 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 19) v1 ==== 85127+0+0 (secure 0 0 0) 0x7f5a2c018030 con 0x7f5a3c0fc510 2024-07-16T09:53:55.432 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:55.430+0000 7f5a327fc640 1 --2- 172.21.15.156:0/2286457589 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f5a28067190 0x7f5a28069640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:53:55.432 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:55.431+0000 7f5a327fc640 1 -- 172.21.15.156:0/2286457589 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f5a2c016070 con 0x7f5a3c0fc510 2024-07-16T09:53:55.432 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:55.431+0000 7f5a41d99640 1 --2- 172.21.15.156:0/2286457589 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f5a28067190 0x7f5a28069640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:53:55.433 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:55.431+0000 7f5a41d99640 1 --2- 172.21.15.156:0/2286457589 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f5a28067190 0x7f5a28069640 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f5a38000c00 tx=0x7f5a38035040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:53:55.436 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:55.435+0000 7f5a327fc640 1 -- 172.21.15.156:0/2286457589 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f5a2c0547a0 con 0x7f5a3c0fc510 2024-07-16T09:53:55.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:53:55 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-16T09:53:55.651 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:55.650+0000 7f5a43ffd640 1 -- 172.21.15.156:0/2286457589 --> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f5a3c10dd90 con 0x7f5a28067190 2024-07-16T09:53:55.662 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:55.661+0000 7f5a327fc640 1 -- 172.21.15.156:0/2286457589 <== mgr.14208 v2:172.21.15.156:6800/2615741717 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3388 (secure 0 0 0) 0x7f5a3c10dd90 con 0x7f5a28067190 2024-07-16T09:53:55.662 INFO:teuthology.orchestra.run.smithi156.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-07-16T09:53:55.662 INFO:teuthology.orchestra.run.smithi156.stdout:alertmanager.smithi156 smithi156 *:9093,9094 running (4m) 34s ago 6m 24.6M - 0.25.0 c8568f914cd2 83419a2a37f4 2024-07-16T09:53:55.662 INFO:teuthology.orchestra.run.smithi156.stdout:crash.smithi156 smithi156 running (6m) 34s ago 6m 6899k - 17.2.7-1104-ge6231b04 d5de5f953b95 bf6d49ea06b4 2024-07-16T09:53:55.663 INFO:teuthology.orchestra.run.smithi156.stdout:crash.smithi176 smithi176 running (5m) 39s ago 5m 6903k - 17.2.7-1104-ge6231b04 d5de5f953b95 907c9fa1e890 2024-07-16T09:53:55.663 INFO:teuthology.orchestra.run.smithi156.stdout:grafana.smithi156 smithi156 *:3000 running (4m) 34s ago 5m 85.0M - 9.4.7 954c08fa6188 184256cf055d 2024-07-16T09:53:55.663 INFO:teuthology.orchestra.run.smithi156.stdout:mds.cephfs.smithi156.evwwaz smithi156 running (51s) 34s ago 51s 14.9M - 17.2.7-1104-ge6231b04 d5de5f953b95 7c0ff9ae43f8 2024-07-16T09:53:55.663 INFO:teuthology.orchestra.run.smithi156.stdout:mds.cephfs.smithi156.rdibys smithi156 running (56s) 34s ago 56s 15.8M - 17.2.7-1104-ge6231b04 d5de5f953b95 b985cd1e6924 2024-07-16T09:53:55.663 INFO:teuthology.orchestra.run.smithi156.stdout:mds.cephfs.smithi176.dvvbud smithi176 running (54s) 39s ago 54s 14.6M - 17.2.7-1104-ge6231b04 d5de5f953b95 1b00147950f9 2024-07-16T09:53:55.663 INFO:teuthology.orchestra.run.smithi156.stdout:mds.cephfs.smithi176.wahbsi smithi176 running (49s) 39s ago 48s 14.7M - 17.2.7-1104-ge6231b04 d5de5f953b95 866664af2311 2024-07-16T09:53:55.663 INFO:teuthology.orchestra.run.smithi156.stdout:mgr.smithi156.zkvylv smithi156 *:9283 running (7m) 34s ago 7m 450M - 17.2.7-1104-ge6231b04 d5de5f953b95 a6937c84750a 2024-07-16T09:53:55.663 INFO:teuthology.orchestra.run.smithi156.stdout:mgr.smithi176.dqbrvx smithi176 *:8443,9283 running (5m) 39s ago 5m 402M - 17.2.7-1104-ge6231b04 d5de5f953b95 654c9bc9c82a 2024-07-16T09:53:55.663 INFO:teuthology.orchestra.run.smithi156.stdout:mon.smithi156 smithi156 running (7m) 34s ago 7m 47.6M 2048M 17.2.7-1104-ge6231b04 d5de5f953b95 6471c8acff40 2024-07-16T09:53:55.663 INFO:teuthology.orchestra.run.smithi156.stdout:mon.smithi176 smithi176 running (5m) 39s ago 5m 45.1M 2048M 17.2.7-1104-ge6231b04 d5de5f953b95 3a293ac6ef01 2024-07-16T09:53:55.663 INFO:teuthology.orchestra.run.smithi156.stdout:node-exporter.smithi156 smithi156 *:9100 running (6m) 34s ago 6m 15.9M - 1.5.0 0da6a335fe13 2c136840a7a9 2024-07-16T09:53:55.663 INFO:teuthology.orchestra.run.smithi156.stdout:node-exporter.smithi176 smithi176 *:9100 running (5m) 39s ago 5m 15.6M - 1.5.0 0da6a335fe13 536716db6afe 2024-07-16T09:53:55.663 INFO:teuthology.orchestra.run.smithi156.stdout:osd.0 smithi156 running (4m) 34s ago 4m 48.5M 4288M 17.2.7-1104-ge6231b04 d5de5f953b95 0471758ec349 2024-07-16T09:53:55.664 INFO:teuthology.orchestra.run.smithi156.stdout:osd.1 smithi156 running (4m) 34s ago 4m 49.0M 4288M 17.2.7-1104-ge6231b04 d5de5f953b95 149a0bda2f61 2024-07-16T09:53:55.664 INFO:teuthology.orchestra.run.smithi156.stdout:osd.2 smithi156 running (3m) 34s ago 3m 44.5M 4288M 17.2.7-1104-ge6231b04 d5de5f953b95 f28acec5cb79 2024-07-16T09:53:55.664 INFO:teuthology.orchestra.run.smithi156.stdout:osd.3 smithi176 running (3m) 39s ago 3m 44.8M 5312M 17.2.7-1104-ge6231b04 d5de5f953b95 733861ba84dc 2024-07-16T09:53:55.664 INFO:teuthology.orchestra.run.smithi156.stdout:osd.4 smithi176 running (2m) 39s ago 2m 41.3M 5312M 17.2.7-1104-ge6231b04 d5de5f953b95 ef0c67cec086 2024-07-16T09:53:55.664 INFO:teuthology.orchestra.run.smithi156.stdout:osd.5 smithi176 running (2m) 39s ago 2m 42.4M 5312M 17.2.7-1104-ge6231b04 d5de5f953b95 996e15448a52 2024-07-16T09:53:55.664 INFO:teuthology.orchestra.run.smithi156.stdout:prometheus.smithi156 smithi156 *:9095 running (4m) 34s ago 5m 55.5M - 2.43.0 a07b618ecd1d 1655b4227cd0 2024-07-16T09:53:55.665 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:55.663+0000 7f5a43ffd640 1 -- 172.21.15.156:0/2286457589 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f5a28067190 msgr2=0x7f5a28069640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:53:55.665 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:55.663+0000 7f5a43ffd640 1 --2- 172.21.15.156:0/2286457589 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f5a28067190 0x7f5a28069640 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f5a38000c00 tx=0x7f5a38035040 comp rx=0 tx=0).stop 2024-07-16T09:53:55.665 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:55.663+0000 7f5a43ffd640 1 -- 172.21.15.156:0/2286457589 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5a3c0fc510 msgr2=0x7f5a3c10c440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:53:55.665 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:55.663+0000 7f5a43ffd640 1 --2- 172.21.15.156:0/2286457589 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5a3c0fc510 0x7f5a3c10c440 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f5a2c010d80 tx=0x7f5a2c00b040 comp rx=0 tx=0).stop 2024-07-16T09:53:55.665 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:55.663+0000 7f5a43ffd640 1 -- 172.21.15.156:0/2286457589 shutdown_connections 2024-07-16T09:53:55.665 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:55.663+0000 7f5a43ffd640 1 --2- 172.21.15.156:0/2286457589 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f5a28067190 0x7f5a28069640 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:53:55.665 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:55.663+0000 7f5a43ffd640 1 --2- 172.21.15.156:0/2286457589 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5a3c0fc510 0x7f5a3c10c440 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:53:55.665 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:55.663+0000 7f5a43ffd640 1 --2- 172.21.15.156:0/2286457589 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5a3c0fb240 0x7f5a3c10bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:53:55.665 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:55.663+0000 7f5a43ffd640 1 -- 172.21.15.156:0/2286457589 >> 172.21.15.156:0/2286457589 conn(0x7f5a3c0f6a20 msgr2=0x7f5a3c0f8110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:53:55.665 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:55.663+0000 7f5a43ffd640 1 -- 172.21.15.156:0/2286457589 shutdown_connections 2024-07-16T09:53:55.665 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:55.663+0000 7f5a43ffd640 1 -- 172.21.15.156:0/2286457589 wait complete. 2024-07-16T09:53:55.756 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:55.754+0000 7ff8571b7640 1 -- 172.21.15.156:0/276483714 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff8500fc330 msgr2=0x7ff8500fc740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:53:55.756 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:55.754+0000 7ff8571b7640 1 --2- 172.21.15.156:0/276483714 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff8500fc330 0x7ff8500fc740 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7ff8440048d0 tx=0x7ff844009da0 comp rx=0 tx=0).stop 2024-07-16T09:53:55.756 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:55.754+0000 7ff8571b7640 1 -- 172.21.15.156:0/276483714 shutdown_connections 2024-07-16T09:53:55.756 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:55.754+0000 7ff8571b7640 1 --2- 172.21.15.156:0/276483714 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff8500fd600 0x7ff8500fda70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:53:55.756 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:55.754+0000 7ff8571b7640 1 --2- 172.21.15.156:0/276483714 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff8500fc330 0x7ff8500fc740 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:53:55.756 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:55.754+0000 7ff8571b7640 1 -- 172.21.15.156:0/276483714 >> 172.21.15.156:0/276483714 conn(0x7ff8500f7af0 msgr2=0x7ff8500f9f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:53:55.756 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:55.754+0000 7ff8571b7640 1 -- 172.21.15.156:0/276483714 shutdown_connections 2024-07-16T09:53:55.756 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:55.754+0000 7ff8571b7640 1 -- 172.21.15.156:0/276483714 wait complete. 2024-07-16T09:53:55.756 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:55.755+0000 7ff8571b7640 1 Processor -- start 2024-07-16T09:53:55.757 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:55.755+0000 7ff8571b7640 1 -- start start 2024-07-16T09:53:55.758 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:55.755+0000 7ff8571b7640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff8500fc330 0x7ff85010cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:53:55.758 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:55.755+0000 7ff8571b7640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff8500fd600 0x7ff85010d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:53:55.758 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:55.755+0000 7ff8571b7640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff85010daf0 con 0x7ff8500fd600 2024-07-16T09:53:55.758 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:55.755+0000 7ff8571b7640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff85010e310 con 0x7ff8500fc330 2024-07-16T09:53:55.758 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:55.755+0000 7ff854f53640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff8500fc330 0x7ff85010cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:53:55.758 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:55.755+0000 7ff84ffff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff8500fd600 0x7ff85010d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:53:55.758 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:55.755+0000 7ff854f53640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff8500fc330 0x7ff85010cf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.156:35770/0 (socket says 172.21.15.156:35770) 2024-07-16T09:53:55.758 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:55.755+0000 7ff84ffff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff8500fd600 0x7ff85010d4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:36498/0 (socket says 172.21.15.156:36498) 2024-07-16T09:53:55.758 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:55.755+0000 7ff854f53640 1 -- 172.21.15.156:0/44287145 learned_addr learned my addr 172.21.15.156:0/44287145 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T09:53:55.758 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:55.756+0000 7ff854f53640 1 -- 172.21.15.156:0/44287145 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff8500fd600 msgr2=0x7ff85010d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:53:55.758 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:55.756+0000 7ff854f53640 1 --2- 172.21.15.156:0/44287145 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff8500fd600 0x7ff85010d4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:53:55.759 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:55.756+0000 7ff854f53640 1 -- 172.21.15.156:0/44287145 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff844005040 con 0x7ff8500fc330 2024-07-16T09:53:55.759 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:55.756+0000 7ff854f53640 1 --2- 172.21.15.156:0/44287145 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff8500fc330 0x7ff85010cf60 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7ff8440048a0 tx=0x7ff844009da0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:53:55.759 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:55.756+0000 7ff84d7fa640 1 -- 172.21.15.156:0/44287145 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff844038070 con 0x7ff8500fc330 2024-07-16T09:53:55.759 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:55.756+0000 7ff84d7fa640 1 -- 172.21.15.156:0/44287145 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff8440092b0 con 0x7ff8500fc330 2024-07-16T09:53:55.759 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:55.756+0000 7ff84d7fa640 1 -- 172.21.15.156:0/44287145 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff844033780 con 0x7ff8500fc330 2024-07-16T09:53:55.759 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:55.756+0000 7ff8571b7640 1 -- 172.21.15.156:0/44287145 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff85010e540 con 0x7ff8500fc330 2024-07-16T09:53:55.759 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:55.756+0000 7ff8571b7640 1 -- 172.21.15.156:0/44287145 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff85010ea80 con 0x7ff8500fc330 2024-07-16T09:53:55.760 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:55.758+0000 7ff8571b7640 1 -- 172.21.15.156:0/44287145 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8500fb790 con 0x7ff8500fc330 2024-07-16T09:53:55.760 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:55.758+0000 7ff84d7fa640 1 -- 172.21.15.156:0/44287145 <== mon.1 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 19) v1 ==== 85127+0+0 (secure 0 0 0) 0x7ff844030020 con 0x7ff8500fc330 2024-07-16T09:53:55.760 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:55.758+0000 7ff84d7fa640 1 --2- 172.21.15.156:0/44287145 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7ff840067190 0x7ff840069640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:53:55.760 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:55.758+0000 7ff84d7fa640 1 -- 172.21.15.156:0/44287145 <== mon.1 v2:172.21.15.176:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5464+0+0 (secure 0 0 0) 0x7ff8440a4840 con 0x7ff8500fc330 2024-07-16T09:53:55.760 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:55.759+0000 7ff84ffff640 1 --2- 172.21.15.156:0/44287145 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7ff840067190 0x7ff840069640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:53:55.760 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:55.759+0000 7ff84ffff640 1 --2- 172.21.15.156:0/44287145 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7ff840067190 0x7ff840069640 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7ff85010cd10 tx=0x7ff83c00b040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:53:55.764 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:55.763+0000 7ff84d7fa640 1 -- 172.21.15.156:0/44287145 <== mon.1 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7ff844071c10 con 0x7ff8500fc330 2024-07-16T09:53:56.059 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:56.057+0000 7ff8571b7640 1 -- 172.21.15.156:0/44287145 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7ff85010ed90 con 0x7ff8500fc330 2024-07-16T09:53:56.059 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:56.058+0000 7ff84d7fa640 1 -- 172.21.15.156:0/44287145 <== mon.1 v2:172.21.15.176:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+643 (secure 0 0 0) 0x7ff844075230 con 0x7ff8500fc330 2024-07-16T09:53:56.059 INFO:teuthology.orchestra.run.smithi156.stdout:{ 2024-07-16T09:53:56.059 INFO:teuthology.orchestra.run.smithi156.stdout: "mon": { 2024-07-16T09:53:56.060 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph version 17.2.7-1104-ge6231b04 (e6231b04b52ebf3b9b31803282dd3074439e51ac) quincy (stable)": 2 2024-07-16T09:53:56.060 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-07-16T09:53:56.060 INFO:teuthology.orchestra.run.smithi156.stdout: "mgr": { 2024-07-16T09:53:56.060 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph version 17.2.7-1104-ge6231b04 (e6231b04b52ebf3b9b31803282dd3074439e51ac) quincy (stable)": 2 2024-07-16T09:53:56.060 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-07-16T09:53:56.060 INFO:teuthology.orchestra.run.smithi156.stdout: "osd": { 2024-07-16T09:53:56.060 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph version 17.2.7-1104-ge6231b04 (e6231b04b52ebf3b9b31803282dd3074439e51ac) quincy (stable)": 6 2024-07-16T09:53:56.060 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-07-16T09:53:56.060 INFO:teuthology.orchestra.run.smithi156.stdout: "mds": { 2024-07-16T09:53:56.060 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph version 17.2.7-1104-ge6231b04 (e6231b04b52ebf3b9b31803282dd3074439e51ac) quincy (stable)": 4 2024-07-16T09:53:56.060 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-07-16T09:53:56.060 INFO:teuthology.orchestra.run.smithi156.stdout: "overall": { 2024-07-16T09:53:56.061 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph version 17.2.7-1104-ge6231b04 (e6231b04b52ebf3b9b31803282dd3074439e51ac) quincy (stable)": 14 2024-07-16T09:53:56.061 INFO:teuthology.orchestra.run.smithi156.stdout: } 2024-07-16T09:53:56.061 INFO:teuthology.orchestra.run.smithi156.stdout:} 2024-07-16T09:53:56.062 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:56.060+0000 7ff8571b7640 1 -- 172.21.15.156:0/44287145 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7ff840067190 msgr2=0x7ff840069640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:53:56.062 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:56.060+0000 7ff8571b7640 1 --2- 172.21.15.156:0/44287145 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7ff840067190 0x7ff840069640 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7ff85010cd10 tx=0x7ff83c00b040 comp rx=0 tx=0).stop 2024-07-16T09:53:56.062 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:56.060+0000 7ff8571b7640 1 -- 172.21.15.156:0/44287145 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff8500fc330 msgr2=0x7ff85010cf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:53:56.062 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:56.060+0000 7ff8571b7640 1 --2- 172.21.15.156:0/44287145 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff8500fc330 0x7ff85010cf60 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7ff8440048a0 tx=0x7ff844009da0 comp rx=0 tx=0).stop 2024-07-16T09:53:56.062 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:56.060+0000 7ff8571b7640 1 -- 172.21.15.156:0/44287145 shutdown_connections 2024-07-16T09:53:56.062 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:56.060+0000 7ff8571b7640 1 --2- 172.21.15.156:0/44287145 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7ff840067190 0x7ff840069640 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:53:56.062 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:56.060+0000 7ff8571b7640 1 --2- 172.21.15.156:0/44287145 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff8500fd600 0x7ff85010d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:53:56.062 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:56.060+0000 7ff8571b7640 1 --2- 172.21.15.156:0/44287145 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff8500fc330 0x7ff85010cf60 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:53:56.062 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:56.060+0000 7ff8571b7640 1 -- 172.21.15.156:0/44287145 >> 172.21.15.156:0/44287145 conn(0x7ff8500f7af0 msgr2=0x7ff8500f91d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:53:56.062 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:56.060+0000 7ff8571b7640 1 -- 172.21.15.156:0/44287145 shutdown_connections 2024-07-16T09:53:56.062 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:56.060+0000 7ff8571b7640 1 -- 172.21.15.156:0/44287145 wait complete. 2024-07-16T09:53:56.151 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:56.150+0000 7f1371bef640 1 -- 172.21.15.156:0/2189519813 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f136c0fb240 msgr2=0x7f136c0fb650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:53:56.151 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:56.150+0000 7f1371bef640 1 --2- 172.21.15.156:0/2189519813 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f136c0fb240 0x7f136c0fb650 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f135c0048d0 tx=0x7f135c009da0 comp rx=0 tx=0).stop 2024-07-16T09:53:56.152 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:56.150+0000 7f1371bef640 1 -- 172.21.15.156:0/2189519813 shutdown_connections 2024-07-16T09:53:56.152 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:56.150+0000 7f1371bef640 1 --2- 172.21.15.156:0/2189519813 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f136c0fc510 0x7f136c0fc980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:53:56.152 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:56.150+0000 7f1371bef640 1 --2- 172.21.15.156:0/2189519813 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f136c0fb240 0x7f136c0fb650 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:53:56.152 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:56.150+0000 7f1371bef640 1 -- 172.21.15.156:0/2189519813 >> 172.21.15.156:0/2189519813 conn(0x7f136c0f6a20 msgr2=0x7f136c0f8e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:53:56.152 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:56.150+0000 7f1371bef640 1 -- 172.21.15.156:0/2189519813 shutdown_connections 2024-07-16T09:53:56.152 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:56.150+0000 7f1371bef640 1 -- 172.21.15.156:0/2189519813 wait complete. 2024-07-16T09:53:56.152 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:56.150+0000 7f1371bef640 1 Processor -- start 2024-07-16T09:53:56.153 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:56.151+0000 7f1371bef640 1 -- start start 2024-07-16T09:53:56.153 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:56.151+0000 7f1371bef640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f136c0fb240 0x7f136c0ff360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:53:56.153 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:56.151+0000 7f1371bef640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f136c0fc510 0x7f136c0fd850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:53:56.153 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:56.151+0000 7f1371bef640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f136c0fdde0 con 0x7f136c0fc510 2024-07-16T09:53:56.153 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:56.151+0000 7f1371bef640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f136c0fdf20 con 0x7f136c0fb240 2024-07-16T09:53:56.153 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:56.151+0000 7f136affd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f136c0fc510 0x7f136c0fd850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:53:56.154 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:56.151+0000 7f136b7fe640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f136c0fb240 0x7f136c0ff360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:53:56.154 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:56.151+0000 7f136affd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f136c0fc510 0x7f136c0fd850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:36528/0 (socket says 172.21.15.156:36528) 2024-07-16T09:53:56.154 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:56.151+0000 7f136affd640 1 -- 172.21.15.156:0/473825032 learned_addr learned my addr 172.21.15.156:0/473825032 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T09:53:56.154 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:56.151+0000 7f136affd640 1 -- 172.21.15.156:0/473825032 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f136c0fb240 msgr2=0x7f136c0ff360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:53:56.154 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:56.151+0000 7f136affd640 1 --2- 172.21.15.156:0/473825032 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f136c0fb240 0x7f136c0ff360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:53:56.154 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:56.151+0000 7f136affd640 1 -- 172.21.15.156:0/473825032 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f135c005040 con 0x7f136c0fc510 2024-07-16T09:53:56.154 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:56.151+0000 7f136b7fe640 1 --2- 172.21.15.156:0/473825032 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f136c0fb240 0x7f136c0ff360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-16T09:53:56.154 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:56.151+0000 7f136affd640 1 --2- 172.21.15.156:0/473825032 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f136c0fc510 0x7f136c0fd850 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f136000eb70 tx=0x7f136000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:53:56.154 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:56.152+0000 7f1353fff640 1 -- 172.21.15.156:0/473825032 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1360007c30 con 0x7f136c0fc510 2024-07-16T09:53:56.154 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:56.152+0000 7f1353fff640 1 -- 172.21.15.156:0/473825032 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f136000f040 con 0x7f136c0fc510 2024-07-16T09:53:56.154 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:56.152+0000 7f1371bef640 1 -- 172.21.15.156:0/473825032 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f136c0fe1a0 con 0x7f136c0fc510 2024-07-16T09:53:56.154 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:56.152+0000 7f1353fff640 1 -- 172.21.15.156:0/473825032 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1360009550 con 0x7f136c0fc510 2024-07-16T09:53:56.154 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:56.152+0000 7f1371bef640 1 -- 172.21.15.156:0/473825032 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f136c114fc0 con 0x7f136c0fc510 2024-07-16T09:53:56.155 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:56.153+0000 7f1371bef640 1 -- 172.21.15.156:0/473825032 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f136c0fa6a0 con 0x7f136c0fc510 2024-07-16T09:53:56.155 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:56.153+0000 7f1353fff640 1 -- 172.21.15.156:0/473825032 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 19) v1 ==== 85127+0+0 (secure 0 0 0) 0x7f136000c6a0 con 0x7f136c0fc510 2024-07-16T09:53:56.155 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:56.154+0000 7f1353fff640 1 --2- 172.21.15.156:0/473825032 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f1354067190 0x7f1354069640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:53:56.155 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:56.154+0000 7f1353fff640 1 -- 172.21.15.156:0/473825032 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f13600847b0 con 0x7f136c0fc510 2024-07-16T09:53:56.156 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:56.154+0000 7f136b7fe640 1 --2- 172.21.15.156:0/473825032 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f1354067190 0x7f1354069640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:53:56.156 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:56.154+0000 7f136b7fe640 1 --2- 172.21.15.156:0/473825032 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f1354067190 0x7f1354069640 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f135c009f80 tx=0x7f135c035040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:53:56.159 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:56.158+0000 7f1353fff640 1 -- 172.21.15.156:0/473825032 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f1360051b00 con 0x7f136c0fc510 2024-07-16T09:53:56.415 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:56.414+0000 7f1371bef640 1 -- 172.21.15.156:0/473825032 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f136c04b530 con 0x7f136c0fc510 2024-07-16T09:53:56.416 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:53:56 smithi156 ceph-mon[29302]: pgmap v199: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 3.2 KiB/s rd, 938 B/s wr, 5 op/s 2024-07-16T09:53:56.416 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:53:56 smithi156 ceph-mon[29302]: from='client.14530 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-16T09:53:56.416 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:53:56 smithi156 ceph-mon[29302]: from='client.14534 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-16T09:53:56.416 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:53:56 smithi156 ceph-mon[29302]: from='client.? 172.21.15.156:0/44287145' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T09:53:56.416 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:56.415+0000 7f1353fff640 1 -- 172.21.15.156:0/473825032 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 16 v16) v1 ==== 76+0+1835 (secure 0 0 0) 0x7f1360019070 con 0x7f136c0fc510 2024-07-16T09:53:56.417 INFO:teuthology.orchestra.run.smithi156.stdout:e16 2024-07-16T09:53:56.417 INFO:teuthology.orchestra.run.smithi156.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-07-16T09:53:56.417 INFO:teuthology.orchestra.run.smithi156.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-07-16T09:53:56.417 INFO:teuthology.orchestra.run.smithi156.stdout:legacy client fscid: 1 2024-07-16T09:53:56.417 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-07-16T09:53:56.417 INFO:teuthology.orchestra.run.smithi156.stdout:Filesystem 'cephfs' (1) 2024-07-16T09:53:56.417 INFO:teuthology.orchestra.run.smithi156.stdout:fs_name cephfs 2024-07-16T09:53:56.417 INFO:teuthology.orchestra.run.smithi156.stdout:epoch 16 2024-07-16T09:53:56.417 INFO:teuthology.orchestra.run.smithi156.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-07-16T09:53:56.417 INFO:teuthology.orchestra.run.smithi156.stdout:created 2024-07-16T09:52:55.214975+0000 2024-07-16T09:53:56.418 INFO:teuthology.orchestra.run.smithi156.stdout:modified 2024-07-16T09:53:11.798619+0000 2024-07-16T09:53:56.418 INFO:teuthology.orchestra.run.smithi156.stdout:tableserver 0 2024-07-16T09:53:56.418 INFO:teuthology.orchestra.run.smithi156.stdout:root 0 2024-07-16T09:53:56.418 INFO:teuthology.orchestra.run.smithi156.stdout:session_timeout 60 2024-07-16T09:53:56.418 INFO:teuthology.orchestra.run.smithi156.stdout:session_autoclose 300 2024-07-16T09:53:56.418 INFO:teuthology.orchestra.run.smithi156.stdout:max_file_size 1099511627776 2024-07-16T09:53:56.418 INFO:teuthology.orchestra.run.smithi156.stdout:required_client_features {} 2024-07-16T09:53:56.418 INFO:teuthology.orchestra.run.smithi156.stdout:last_failure 0 2024-07-16T09:53:56.418 INFO:teuthology.orchestra.run.smithi156.stdout:last_failure_osd_epoch 0 2024-07-16T09:53:56.418 INFO:teuthology.orchestra.run.smithi156.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-07-16T09:53:56.419 INFO:teuthology.orchestra.run.smithi156.stdout:max_mds 2 2024-07-16T09:53:56.419 INFO:teuthology.orchestra.run.smithi156.stdout:in 0,1 2024-07-16T09:53:56.419 INFO:teuthology.orchestra.run.smithi156.stdout:up {0=14458,1=24261} 2024-07-16T09:53:56.419 INFO:teuthology.orchestra.run.smithi156.stdout:failed 2024-07-16T09:53:56.419 INFO:teuthology.orchestra.run.smithi156.stdout:damaged 2024-07-16T09:53:56.419 INFO:teuthology.orchestra.run.smithi156.stdout:stopped 2024-07-16T09:53:56.419 INFO:teuthology.orchestra.run.smithi156.stdout:data_pools [3] 2024-07-16T09:53:56.419 INFO:teuthology.orchestra.run.smithi156.stdout:metadata_pool 2 2024-07-16T09:53:56.419 INFO:teuthology.orchestra.run.smithi156.stdout:inline_data disabled 2024-07-16T09:53:56.419 INFO:teuthology.orchestra.run.smithi156.stdout:balancer 2024-07-16T09:53:56.419 INFO:teuthology.orchestra.run.smithi156.stdout:standby_count_wanted 1 2024-07-16T09:53:56.419 INFO:teuthology.orchestra.run.smithi156.stdout:[mds.cephfs.smithi156.rdibys{0:14458} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.156:6826/1080176013,v1:172.21.15.156:6827/1080176013] compat {c=[1],r=[1],i=[7ff]}] 2024-07-16T09:53:56.419 INFO:teuthology.orchestra.run.smithi156.stdout:[mds.cephfs.smithi156.evwwaz{0:14468} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.156:6828/560329502,v1:172.21.15.156:6829/560329502] compat {c=[1],r=[1],i=[7ff]}] 2024-07-16T09:53:56.419 INFO:teuthology.orchestra.run.smithi156.stdout:[mds.cephfs.smithi176.dvvbud{1:24261} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.176:6824/1577019287,v1:172.21.15.176:6825/1577019287] compat {c=[1],r=[1],i=[7ff]}] 2024-07-16T09:53:56.420 INFO:teuthology.orchestra.run.smithi156.stdout:[mds.cephfs.smithi176.wahbsi{1:24273} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.176:6826/2196775412,v1:172.21.15.176:6827/2196775412] compat {c=[1],r=[1],i=[7ff]}] 2024-07-16T09:53:56.420 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-07-16T09:53:56.420 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-07-16T09:53:56.421 INFO:teuthology.orchestra.run.smithi156.stderr:dumped fsmap epoch 16 2024-07-16T09:53:56.421 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:56.417+0000 7f1371bef640 1 -- 172.21.15.156:0/473825032 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f1354067190 msgr2=0x7f1354069640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:53:56.421 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:56.417+0000 7f1371bef640 1 --2- 172.21.15.156:0/473825032 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f1354067190 0x7f1354069640 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f135c009f80 tx=0x7f135c035040 comp rx=0 tx=0).stop 2024-07-16T09:53:56.421 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:56.417+0000 7f1371bef640 1 -- 172.21.15.156:0/473825032 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f136c0fc510 msgr2=0x7f136c0fd850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:53:56.421 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:56.417+0000 7f1371bef640 1 --2- 172.21.15.156:0/473825032 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f136c0fc510 0x7f136c0fd850 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f136000eb70 tx=0x7f136000b040 comp rx=0 tx=0).stop 2024-07-16T09:53:56.421 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:56.417+0000 7f1371bef640 1 -- 172.21.15.156:0/473825032 shutdown_connections 2024-07-16T09:53:56.421 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:56.417+0000 7f1371bef640 1 --2- 172.21.15.156:0/473825032 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f1354067190 0x7f1354069640 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:53:56.421 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:56.417+0000 7f1371bef640 1 --2- 172.21.15.156:0/473825032 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f136c0fc510 0x7f136c0fd850 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:53:56.421 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:56.417+0000 7f1371bef640 1 --2- 172.21.15.156:0/473825032 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f136c0fb240 0x7f136c0ff360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:53:56.421 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:56.417+0000 7f1371bef640 1 -- 172.21.15.156:0/473825032 >> 172.21.15.156:0/473825032 conn(0x7f136c0f6a20 msgr2=0x7f136c065300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:53:56.421 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:56.418+0000 7f1371bef640 1 -- 172.21.15.156:0/473825032 shutdown_connections 2024-07-16T09:53:56.421 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:56.418+0000 7f1371bef640 1 -- 172.21.15.156:0/473825032 wait complete. 2024-07-16T09:53:56.508 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:56.506+0000 7f558ace4640 1 -- 172.21.15.156:0/3766661002 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f55840fc510 msgr2=0x7f55840fc980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:53:56.508 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:56.506+0000 7f558ace4640 1 --2- 172.21.15.156:0/3766661002 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f55840fc510 0x7f55840fc980 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f55700048d0 tx=0x7f5570009da0 comp rx=0 tx=0).stop 2024-07-16T09:53:56.508 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:56.506+0000 7f558ace4640 1 -- 172.21.15.156:0/3766661002 shutdown_connections 2024-07-16T09:53:56.508 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:56.506+0000 7f558ace4640 1 --2- 172.21.15.156:0/3766661002 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f55840fc510 0x7f55840fc980 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:53:56.508 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:56.506+0000 7f558ace4640 1 --2- 172.21.15.156:0/3766661002 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f55840fb240 0x7f55840fb650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:53:56.508 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:56.506+0000 7f558ace4640 1 -- 172.21.15.156:0/3766661002 >> 172.21.15.156:0/3766661002 conn(0x7f55840f6a20 msgr2=0x7f55840f8e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:53:56.509 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:56.506+0000 7f558ace4640 1 -- 172.21.15.156:0/3766661002 shutdown_connections 2024-07-16T09:53:56.509 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:56.506+0000 7f558ace4640 1 -- 172.21.15.156:0/3766661002 wait complete. 2024-07-16T09:53:56.509 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:56.507+0000 7f558ace4640 1 Processor -- start 2024-07-16T09:53:56.509 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:56.507+0000 7f558ace4640 1 -- start start 2024-07-16T09:53:56.510 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:56.507+0000 7f558ace4640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f55840fb240 0x7f55840ff310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:53:56.510 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:56.507+0000 7f558ace4640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f55840fc510 0x7f55840fd850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:53:56.510 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:56.507+0000 7f558ace4640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55840fdd90 con 0x7f55840fc510 2024-07-16T09:53:56.510 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:56.507+0000 7f558ace4640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55840fded0 con 0x7f55840fb240 2024-07-16T09:53:56.510 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:56.507+0000 7f5583fff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f55840fc510 0x7f55840fd850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:53:56.510 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:56.507+0000 7f5583fff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f55840fc510 0x7f55840fd850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:36552/0 (socket says 172.21.15.156:36552) 2024-07-16T09:53:56.510 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:56.507+0000 7f5583fff640 1 -- 172.21.15.156:0/4136035343 learned_addr learned my addr 172.21.15.156:0/4136035343 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T09:53:56.510 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:56.507+0000 7f5588a80640 1 --2- 172.21.15.156:0/4136035343 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f55840fb240 0x7f55840ff310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:53:56.510 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:56.508+0000 7f5583fff640 1 -- 172.21.15.156:0/4136035343 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f55840fb240 msgr2=0x7f55840ff310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:53:56.511 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:56.508+0000 7f5583fff640 1 --2- 172.21.15.156:0/4136035343 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f55840fb240 0x7f55840ff310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:53:56.511 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:56.508+0000 7f5583fff640 1 -- 172.21.15.156:0/4136035343 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5570005040 con 0x7f55840fc510 2024-07-16T09:53:56.511 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:56.508+0000 7f5588a80640 1 --2- 172.21.15.156:0/4136035343 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f55840fb240 0x7f55840ff310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-16T09:53:56.511 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:56.508+0000 7f5583fff640 1 --2- 172.21.15.156:0/4136035343 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f55840fc510 0x7f55840fd850 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f5570009f80 tx=0x7f5570009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:53:56.511 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:56.508+0000 7f55817fa640 1 -- 172.21.15.156:0/4136035343 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5570038070 con 0x7f55840fc510 2024-07-16T09:53:56.511 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:56.508+0000 7f558ace4640 1 -- 172.21.15.156:0/4136035343 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55840fe150 con 0x7f55840fc510 2024-07-16T09:53:56.511 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:56.508+0000 7f55817fa640 1 -- 172.21.15.156:0/4136035343 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5570032970 con 0x7f55840fc510 2024-07-16T09:53:56.511 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:56.508+0000 7f55817fa640 1 -- 172.21.15.156:0/4136035343 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5570033860 con 0x7f55840fc510 2024-07-16T09:53:56.511 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:56.508+0000 7f558ace4640 1 -- 172.21.15.156:0/4136035343 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55840fe690 con 0x7f55840fc510 2024-07-16T09:53:56.512 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:56.509+0000 7f558ace4640 1 -- 172.21.15.156:0/4136035343 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55840fa6a0 con 0x7f55840fc510 2024-07-16T09:53:56.512 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:56.510+0000 7f55817fa640 1 -- 172.21.15.156:0/4136035343 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 19) v1 ==== 85127+0+0 (secure 0 0 0) 0x7f5570030020 con 0x7f55840fc510 2024-07-16T09:53:56.512 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:56.510+0000 7f55817fa640 1 --2- 172.21.15.156:0/4136035343 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f5574067190 0x7f5574069640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:53:56.512 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:56.510+0000 7f55817fa640 1 -- 172.21.15.156:0/4136035343 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f55700a4970 con 0x7f55840fc510 2024-07-16T09:53:56.512 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:56.510+0000 7f5588a80640 1 --2- 172.21.15.156:0/4136035343 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f5574067190 0x7f5574069640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:53:56.513 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:56.511+0000 7f5588a80640 1 --2- 172.21.15.156:0/4136035343 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f5574067190 0x7f5574069640 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f55840ff0c0 tx=0x7f557800b040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:53:56.516 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:56.515+0000 7f55817fa640 1 -- 172.21.15.156:0/4136035343 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f5570071cc0 con 0x7f55840fc510 2024-07-16T09:53:56.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:53:56 smithi176 ceph-mon[36626]: pgmap v199: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 3.2 KiB/s rd, 938 B/s wr, 5 op/s 2024-07-16T09:53:56.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:53:56 smithi176 ceph-mon[36626]: from='client.14530 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-16T09:53:56.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:53:56 smithi176 ceph-mon[36626]: from='client.14534 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-16T09:53:56.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:53:56 smithi176 ceph-mon[36626]: from='client.? 172.21.15.156:0/44287145' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T09:53:56.751 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:56.750+0000 7f558ace4640 1 -- 172.21.15.156:0/4136035343 --> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f5584103ec0 con 0x7f5574067190 2024-07-16T09:53:56.752 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:56.751+0000 7f55817fa640 1 -- 172.21.15.156:0/4136035343 <== mgr.14208 v2:172.21.15.156:6800/2615741717 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+358 (secure 0 0 0) 0x7f5584103ec0 con 0x7f5574067190 2024-07-16T09:53:56.753 INFO:teuthology.orchestra.run.smithi156.stdout:{ 2024-07-16T09:53:56.753 INFO:teuthology.orchestra.run.smithi156.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph:a6ca461be7cc1aff3658c2c6df4cff975a24449a", 2024-07-16T09:53:56.753 INFO:teuthology.orchestra.run.smithi156.stdout: "in_progress": true, 2024-07-16T09:53:56.753 INFO:teuthology.orchestra.run.smithi156.stdout: "which": "Upgrading all daemon types on all hosts", 2024-07-16T09:53:56.753 INFO:teuthology.orchestra.run.smithi156.stdout: "services_complete": [], 2024-07-16T09:53:56.753 INFO:teuthology.orchestra.run.smithi156.stdout: "progress": "", 2024-07-16T09:53:56.753 INFO:teuthology.orchestra.run.smithi156.stdout: "message": "Doing first pull of quay.ceph.io/ceph-ci/ceph:a6ca461be7cc1aff3658c2c6df4cff975a24449a image", 2024-07-16T09:53:56.753 INFO:teuthology.orchestra.run.smithi156.stdout: "is_paused": false 2024-07-16T09:53:56.753 INFO:teuthology.orchestra.run.smithi156.stdout:} 2024-07-16T09:53:56.755 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:56.753+0000 7f558ace4640 1 -- 172.21.15.156:0/4136035343 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f5574067190 msgr2=0x7f5574069640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:53:56.755 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:56.753+0000 7f558ace4640 1 --2- 172.21.15.156:0/4136035343 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f5574067190 0x7f5574069640 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f55840ff0c0 tx=0x7f557800b040 comp rx=0 tx=0).stop 2024-07-16T09:53:56.755 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:56.754+0000 7f558ace4640 1 -- 172.21.15.156:0/4136035343 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f55840fc510 msgr2=0x7f55840fd850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:53:56.755 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:56.754+0000 7f558ace4640 1 --2- 172.21.15.156:0/4136035343 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f55840fc510 0x7f55840fd850 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f5570009f80 tx=0x7f5570009da0 comp rx=0 tx=0).stop 2024-07-16T09:53:56.755 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:56.754+0000 7f558ace4640 1 -- 172.21.15.156:0/4136035343 shutdown_connections 2024-07-16T09:53:56.755 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:56.754+0000 7f558ace4640 1 --2- 172.21.15.156:0/4136035343 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f5574067190 0x7f5574069640 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:53:56.756 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:56.754+0000 7f558ace4640 1 --2- 172.21.15.156:0/4136035343 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f55840fc510 0x7f55840fd850 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:53:56.756 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:56.754+0000 7f558ace4640 1 --2- 172.21.15.156:0/4136035343 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f55840fb240 0x7f55840ff310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:53:56.756 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:56.754+0000 7f558ace4640 1 -- 172.21.15.156:0/4136035343 >> 172.21.15.156:0/4136035343 conn(0x7f55840f6a20 msgr2=0x7f5584065390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:53:56.756 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:56.754+0000 7f558ace4640 1 -- 172.21.15.156:0/4136035343 shutdown_connections 2024-07-16T09:53:56.756 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:56.754+0000 7f558ace4640 1 -- 172.21.15.156:0/4136035343 wait complete. 2024-07-16T09:53:56.850 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:56.848+0000 7f571e99e640 1 -- 172.21.15.156:0/3389254073 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f57180ff2e0 msgr2=0x7f57181016c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:53:56.850 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:56.848+0000 7f571e99e640 1 --2- 172.21.15.156:0/3389254073 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f57180ff2e0 0x7f57181016c0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f57080048d0 tx=0x7f5708009da0 comp rx=0 tx=0).stop 2024-07-16T09:53:56.850 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:56.848+0000 7f571e99e640 1 -- 172.21.15.156:0/3389254073 shutdown_connections 2024-07-16T09:53:56.850 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:56.848+0000 7f571e99e640 1 --2- 172.21.15.156:0/3389254073 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f57180ff2e0 0x7f57181016c0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:53:56.850 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:56.848+0000 7f571e99e640 1 --2- 172.21.15.156:0/3389254073 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f57180662f0 0x7f57180feda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:53:56.850 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:56.848+0000 7f571e99e640 1 -- 172.21.15.156:0/3389254073 >> 172.21.15.156:0/3389254073 conn(0x7f57180f6870 msgr2=0x7f57180f8c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:53:56.851 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:56.849+0000 7f571e99e640 1 -- 172.21.15.156:0/3389254073 shutdown_connections 2024-07-16T09:53:56.851 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:56.849+0000 7f571e99e640 1 -- 172.21.15.156:0/3389254073 wait complete. 2024-07-16T09:53:56.852 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:56.849+0000 7f571e99e640 1 Processor -- start 2024-07-16T09:53:56.852 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:56.849+0000 7f571e99e640 1 -- start start 2024-07-16T09:53:56.852 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:56.849+0000 7f571e99e640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f57180662f0 0x7f57180fc760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:53:56.852 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:56.849+0000 7f571e99e640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f57180ff2e0 0x7f57180faca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:53:56.852 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:56.849+0000 7f571e99e640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57180fb1e0 con 0x7f57180662f0 2024-07-16T09:53:56.852 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:56.849+0000 7f571e99e640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57180fb320 con 0x7f57180ff2e0 2024-07-16T09:53:56.853 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:56.849+0000 7f571d99c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f57180662f0 0x7f57180fc760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:53:56.853 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:56.849+0000 7f571d99c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f57180662f0 0x7f57180fc760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:36582/0 (socket says 172.21.15.156:36582) 2024-07-16T09:53:56.853 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:56.849+0000 7f571d99c640 1 -- 172.21.15.156:0/588282629 learned_addr learned my addr 172.21.15.156:0/588282629 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T09:53:56.853 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:56.849+0000 7f571d19b640 1 --2- 172.21.15.156:0/588282629 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f57180ff2e0 0x7f57180faca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:53:56.853 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:56.850+0000 7f571d99c640 1 -- 172.21.15.156:0/588282629 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f57180ff2e0 msgr2=0x7f57180faca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:53:56.853 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:56.850+0000 7f571d99c640 1 --2- 172.21.15.156:0/588282629 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f57180ff2e0 0x7f57180faca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:53:56.853 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:56.850+0000 7f571d99c640 1 -- 172.21.15.156:0/588282629 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5708005040 con 0x7f57180662f0 2024-07-16T09:53:56.853 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:56.850+0000 7f571d19b640 1 --2- 172.21.15.156:0/588282629 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f57180ff2e0 0x7f57180faca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-16T09:53:56.853 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:56.850+0000 7f571d99c640 1 --2- 172.21.15.156:0/588282629 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f57180662f0 0x7f57180fc760 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f5714002840 tx=0x7f5714004570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:53:56.853 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:56.850+0000 7f570e7fc640 1 -- 172.21.15.156:0/588282629 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5714002dd0 con 0x7f57180662f0 2024-07-16T09:53:56.853 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:56.850+0000 7f570e7fc640 1 -- 172.21.15.156:0/588282629 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f571400f040 con 0x7f57180662f0 2024-07-16T09:53:56.854 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:56.850+0000 7f571e99e640 1 -- 172.21.15.156:0/588282629 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57180fb600 con 0x7f57180662f0 2024-07-16T09:53:56.854 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:56.850+0000 7f570e7fc640 1 -- 172.21.15.156:0/588282629 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5714009460 con 0x7f57180662f0 2024-07-16T09:53:56.854 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:56.850+0000 7f571e99e640 1 -- 172.21.15.156:0/588282629 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5718114df0 con 0x7f57180662f0 2024-07-16T09:53:56.855 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:56.851+0000 7f571e99e640 1 -- 172.21.15.156:0/588282629 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57180fa4f0 con 0x7f57180662f0 2024-07-16T09:53:56.855 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:56.852+0000 7f570e7fc640 1 -- 172.21.15.156:0/588282629 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 19) v1 ==== 85127+0+0 (secure 0 0 0) 0x7f5714007c30 con 0x7f57180662f0 2024-07-16T09:53:56.855 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:56.852+0000 7f570e7fc640 1 --2- 172.21.15.156:0/588282629 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f5704067190 0x7f5704069640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:53:56.855 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:56.852+0000 7f570e7fc640 1 -- 172.21.15.156:0/588282629 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f5714084ab0 con 0x7f57180662f0 2024-07-16T09:53:56.855 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:56.852+0000 7f571d19b640 1 --2- 172.21.15.156:0/588282629 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f5704067190 0x7f5704069640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:53:56.855 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:56.853+0000 7f571d19b640 1 --2- 172.21.15.156:0/588282629 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f5704067190 0x7f5704069640 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f57080043d0 tx=0x7f5708035040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:53:56.858 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:56.857+0000 7f570e7fc640 1 -- 172.21.15.156:0/588282629 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f5714051d50 con 0x7f57180662f0 2024-07-16T09:53:57.152 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:57.151+0000 7f571e99e640 1 -- 172.21.15.156:0/588282629 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f5718115040 con 0x7f57180662f0 2024-07-16T09:53:57.153 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:57.151+0000 7f570e7fc640 1 -- 172.21.15.156:0/588282629 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7f5714055370 con 0x7f57180662f0 2024-07-16T09:53:57.153 INFO:teuthology.orchestra.run.smithi156.stdout:HEALTH_OK 2024-07-16T09:53:57.155 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:57.153+0000 7f571e99e640 1 -- 172.21.15.156:0/588282629 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f5704067190 msgr2=0x7f5704069640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:53:57.155 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:57.153+0000 7f571e99e640 1 --2- 172.21.15.156:0/588282629 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f5704067190 0x7f5704069640 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f57080043d0 tx=0x7f5708035040 comp rx=0 tx=0).stop 2024-07-16T09:53:57.155 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:57.153+0000 7f571e99e640 1 -- 172.21.15.156:0/588282629 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f57180662f0 msgr2=0x7f57180fc760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:53:57.155 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:57.153+0000 7f571e99e640 1 --2- 172.21.15.156:0/588282629 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f57180662f0 0x7f57180fc760 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f5714002840 tx=0x7f5714004570 comp rx=0 tx=0).stop 2024-07-16T09:53:57.155 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:57.153+0000 7f571e99e640 1 -- 172.21.15.156:0/588282629 shutdown_connections 2024-07-16T09:53:57.155 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:57.153+0000 7f571e99e640 1 --2- 172.21.15.156:0/588282629 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f5704067190 0x7f5704069640 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:53:57.155 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:57.154+0000 7f571e99e640 1 --2- 172.21.15.156:0/588282629 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f57180ff2e0 0x7f57180faca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:53:57.156 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:57.154+0000 7f571e99e640 1 --2- 172.21.15.156:0/588282629 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f57180662f0 0x7f57180fc760 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:53:57.156 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:57.154+0000 7f571e99e640 1 -- 172.21.15.156:0/588282629 >> 172.21.15.156:0/588282629 conn(0x7f57180f6870 msgr2=0x7f57180f6c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:53:57.156 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:57.154+0000 7f571e99e640 1 -- 172.21.15.156:0/588282629 shutdown_connections 2024-07-16T09:53:57.156 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:53:57.154+0000 7f571e99e640 1 -- 172.21.15.156:0/588282629 wait complete. 2024-07-16T09:53:57.424 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:53:57 smithi156 ceph-mon[29302]: from='client.14538 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-07-16T09:53:57.424 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:53:57 smithi156 ceph-mon[29302]: from='client.? 172.21.15.156:0/473825032' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-07-16T09:53:57.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:53:57 smithi176 ceph-mon[36626]: from='client.14538 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-07-16T09:53:57.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:53:57 smithi176 ceph-mon[36626]: from='client.? 172.21.15.156:0/473825032' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-07-16T09:53:58.424 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:53:58 smithi156 ceph-mon[29302]: from='client.14550 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-16T09:53:58.424 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:53:58 smithi156 ceph-mon[29302]: pgmap v200: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 597 B/s wr, 3 op/s 2024-07-16T09:53:58.424 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:53:58 smithi156 ceph-mon[29302]: from='client.? 172.21.15.156:0/588282629' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-07-16T09:53:58.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:53:58 smithi176 ceph-mon[36626]: from='client.14550 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-16T09:53:58.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:53:58 smithi176 ceph-mon[36626]: pgmap v200: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 597 B/s wr, 3 op/s 2024-07-16T09:53:58.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:53:58 smithi176 ceph-mon[36626]: from='client.? 172.21.15.156:0/588282629' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-07-16T09:54:00.424 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:54:00 smithi156 ceph-mon[29302]: pgmap v201: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.4 KiB/s rd, 597 B/s wr, 4 op/s 2024-07-16T09:54:00.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:54:00 smithi176 ceph-mon[36626]: pgmap v201: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.4 KiB/s rd, 597 B/s wr, 4 op/s 2024-07-16T09:54:02.395 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:54:02 smithi176 ceph-mon[36626]: pgmap v202: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.4 KiB/s rd, 511 B/s wr, 4 op/s 2024-07-16T09:54:02.424 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:54:02 smithi156 ceph-mon[29302]: pgmap v202: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.4 KiB/s rd, 511 B/s wr, 4 op/s 2024-07-16T09:54:04.424 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:54:04 smithi156 ceph-mon[29302]: pgmap v203: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 85 B/s wr, 3 op/s 2024-07-16T09:54:04.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:54:04 smithi176 ceph-mon[36626]: pgmap v203: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 85 B/s wr, 3 op/s 2024-07-16T09:54:06.424 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:54:06 smithi156 ceph-mon[29302]: pgmap v204: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.8 KiB/s rd, 4 op/s 2024-07-16T09:54:06.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:54:06 smithi176 ceph-mon[36626]: pgmap v204: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.8 KiB/s rd, 4 op/s 2024-07-16T09:54:08.424 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:54:08 smithi156 ceph-mon[29302]: pgmap v205: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-07-16T09:54:08.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:54:08 smithi176 ceph-mon[36626]: pgmap v205: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-07-16T09:54:10.424 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:54:10 smithi156 ceph-mon[29302]: pgmap v206: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T09:54:10.424 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:54:10 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-16T09:54:10.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:54:10 smithi176 ceph-mon[36626]: pgmap v206: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T09:54:10.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:54:10 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-16T09:54:12.424 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:54:12 smithi156 ceph-mon[29302]: pgmap v207: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T09:54:12.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:54:12 smithi176 ceph-mon[36626]: pgmap v207: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T09:54:14.136 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:54:14 smithi176 ceph-mon[36626]: pgmap v208: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-07-16T09:54:14.426 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:54:14 smithi156 ceph-mon[29302]: pgmap v208: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-07-16T09:54:16.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:54:16 smithi156 ceph-mon[29302]: pgmap v209: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-07-16T09:54:16.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:54:16 smithi176 ceph-mon[36626]: pgmap v209: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-07-16T09:54:18.424 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:54:18 smithi156 ceph-mon[29302]: pgmap v210: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-07-16T09:54:18.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:54:18 smithi176 ceph-mon[36626]: pgmap v210: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-07-16T09:54:20.424 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:54:20 smithi156 ceph-mon[29302]: pgmap v211: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T09:54:20.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:54:20 smithi176 ceph-mon[36626]: pgmap v211: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T09:54:22.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:54:22 smithi156 ceph-mon[29302]: pgmap v212: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T09:54:22.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:54:22 smithi176 ceph-mon[36626]: pgmap v212: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T09:54:24.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:54:23 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:54:24.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:54:23 smithi176 ceph-mon[36626]: Upgrade: Target is version 19.0.0-3495-ga6ca461b (unknown) 2024-07-16T09:54:24.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:54:23 smithi176 ceph-mon[36626]: Upgrade: Target container is quay.ceph.io/ceph-ci/ceph@sha256:c15ec3aa8e6c0a9c12ee24ff79f3e7dd4d01b3d1a702dfeb7d7fff9b0811a393, digests ['quay.ceph.io/ceph-ci/ceph@sha256:c15ec3aa8e6c0a9c12ee24ff79f3e7dd4d01b3d1a702dfeb7d7fff9b0811a393'] 2024-07-16T09:54:24.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:54:23 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-16T09:54:24.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:54:23 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:54:24.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:54:23 smithi176 ceph-mon[36626]: Upgrade: Need to upgrade myself (mgr.smithi156.zkvylv) 2024-07-16T09:54:24.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:54:23 smithi176 ceph-mon[36626]: pgmap v213: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-07-16T09:54:24.095 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:54:23 smithi176 ceph-mon[36626]: Upgrade: Pulling quay.ceph.io/ceph-ci/ceph@sha256:c15ec3aa8e6c0a9c12ee24ff79f3e7dd4d01b3d1a702dfeb7d7fff9b0811a393 on smithi176 2024-07-16T09:54:24.174 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:54:23 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:54:24.175 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:54:23 smithi156 ceph-mon[29302]: Upgrade: Target is version 19.0.0-3495-ga6ca461b (unknown) 2024-07-16T09:54:24.175 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:54:23 smithi156 ceph-mon[29302]: Upgrade: Target container is quay.ceph.io/ceph-ci/ceph@sha256:c15ec3aa8e6c0a9c12ee24ff79f3e7dd4d01b3d1a702dfeb7d7fff9b0811a393, digests ['quay.ceph.io/ceph-ci/ceph@sha256:c15ec3aa8e6c0a9c12ee24ff79f3e7dd4d01b3d1a702dfeb7d7fff9b0811a393'] 2024-07-16T09:54:24.175 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:54:23 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-16T09:54:24.175 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:54:23 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:54:24.175 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:54:23 smithi156 ceph-mon[29302]: Upgrade: Need to upgrade myself (mgr.smithi156.zkvylv) 2024-07-16T09:54:24.175 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:54:23 smithi156 ceph-mon[29302]: pgmap v213: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-07-16T09:54:24.175 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:54:23 smithi156 ceph-mon[29302]: Upgrade: Pulling quay.ceph.io/ceph-ci/ceph@sha256:c15ec3aa8e6c0a9c12ee24ff79f3e7dd4d01b3d1a702dfeb7d7fff9b0811a393 on smithi176 2024-07-16T09:54:25.343 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:54:25 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-16T09:54:25.424 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:54:25 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-16T09:54:26.072 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:54:26 smithi176 ceph-mon[36626]: pgmap v214: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-07-16T09:54:26.424 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:54:26 smithi156 ceph-mon[29302]: pgmap v214: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-07-16T09:54:27.280 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:27.278+0000 7f5c661c9640 1 -- 172.21.15.156:0/2873042383 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5c600fb230 msgr2=0x7f5c600fb640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:54:27.280 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:27.278+0000 7f5c661c9640 1 --2- 172.21.15.156:0/2873042383 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5c600fb230 0x7f5c600fb640 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f5c500048d0 tx=0x7f5c50009da0 comp rx=0 tx=0).stop 2024-07-16T09:54:27.280 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:27.278+0000 7f5c661c9640 1 -- 172.21.15.156:0/2873042383 shutdown_connections 2024-07-16T09:54:27.280 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:27.278+0000 7f5c661c9640 1 --2- 172.21.15.156:0/2873042383 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5c600fc500 0x7f5c600fc970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:54:27.280 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:27.278+0000 7f5c661c9640 1 --2- 172.21.15.156:0/2873042383 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5c600fb230 0x7f5c600fb640 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:54:27.280 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:27.278+0000 7f5c661c9640 1 -- 172.21.15.156:0/2873042383 >> 172.21.15.156:0/2873042383 conn(0x7f5c600f6a10 msgr2=0x7f5c600f8e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:54:27.281 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:27.279+0000 7f5c661c9640 1 -- 172.21.15.156:0/2873042383 shutdown_connections 2024-07-16T09:54:27.281 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:27.279+0000 7f5c661c9640 1 -- 172.21.15.156:0/2873042383 wait complete. 2024-07-16T09:54:27.281 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:27.279+0000 7f5c661c9640 1 Processor -- start 2024-07-16T09:54:27.282 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:27.279+0000 7f5c661c9640 1 -- start start 2024-07-16T09:54:27.282 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:27.280+0000 7f5c661c9640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5c600fb230 0x7f5c600ff300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:54:27.282 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:27.280+0000 7f5c661c9640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5c600fc500 0x7f5c600fd840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:54:27.282 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:27.280+0000 7f5c661c9640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c600fdd80 con 0x7f5c600fc500 2024-07-16T09:54:27.282 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:27.280+0000 7f5c661c9640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c600fdec0 con 0x7f5c600fb230 2024-07-16T09:54:27.282 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:27.280+0000 7f5c5f7fe640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5c600fb230 0x7f5c600ff300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:54:27.282 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:27.280+0000 7f5c5effd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5c600fc500 0x7f5c600fd840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:54:27.282 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:27.280+0000 7f5c5f7fe640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5c600fb230 0x7f5c600ff300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.156:51860/0 (socket says 172.21.15.156:51860) 2024-07-16T09:54:27.282 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:27.280+0000 7f5c5f7fe640 1 -- 172.21.15.156:0/4146034655 learned_addr learned my addr 172.21.15.156:0/4146034655 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T09:54:27.282 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:27.280+0000 7f5c5effd640 1 --2- 172.21.15.156:0/4146034655 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5c600fc500 0x7f5c600fd840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:40258/0 (socket says 172.21.15.156:40258) 2024-07-16T09:54:27.282 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:27.280+0000 7f5c5f7fe640 1 -- 172.21.15.156:0/4146034655 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5c600fc500 msgr2=0x7f5c600fd840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:54:27.283 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:27.280+0000 7f5c5f7fe640 1 --2- 172.21.15.156:0/4146034655 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5c600fc500 0x7f5c600fd840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:54:27.283 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:27.280+0000 7f5c5f7fe640 1 -- 172.21.15.156:0/4146034655 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c50005040 con 0x7f5c600fb230 2024-07-16T09:54:27.283 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:27.280+0000 7f5c5effd640 1 --2- 172.21.15.156:0/4146034655 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5c600fc500 0x7f5c600fd840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-16T09:54:27.283 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:27.281+0000 7f5c5f7fe640 1 --2- 172.21.15.156:0/4146034655 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5c600fb230 0x7f5c600ff300 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f5c500048a0 tx=0x7f5c50009da0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:54:27.283 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:27.281+0000 7f5c47fff640 1 -- 172.21.15.156:0/4146034655 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5c50038070 con 0x7f5c600fb230 2024-07-16T09:54:27.283 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:27.281+0000 7f5c47fff640 1 -- 172.21.15.156:0/4146034655 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5c500092b0 con 0x7f5c600fb230 2024-07-16T09:54:27.284 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:27.281+0000 7f5c47fff640 1 -- 172.21.15.156:0/4146034655 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5c50033780 con 0x7f5c600fb230 2024-07-16T09:54:27.284 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:27.281+0000 7f5c661c9640 1 -- 172.21.15.156:0/4146034655 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c600fe140 con 0x7f5c600fb230 2024-07-16T09:54:27.284 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:27.281+0000 7f5c661c9640 1 -- 172.21.15.156:0/4146034655 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c600fe630 con 0x7f5c600fb230 2024-07-16T09:54:27.284 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:27.282+0000 7f5c661c9640 1 -- 172.21.15.156:0/4146034655 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c600fa690 con 0x7f5c600fb230 2024-07-16T09:54:27.287 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:27.283+0000 7f5c47fff640 1 -- 172.21.15.156:0/4146034655 <== mon.1 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 19) v1 ==== 85127+0+0 (secure 0 0 0) 0x7f5c50030020 con 0x7f5c600fb230 2024-07-16T09:54:27.287 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:27.283+0000 7f5c47fff640 1 --2- 172.21.15.156:0/4146034655 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f5c48067190 0x7f5c48069640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:54:27.287 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:27.283+0000 7f5c47fff640 1 -- 172.21.15.156:0/4146034655 <== mon.1 v2:172.21.15.176:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f5c500a4950 con 0x7f5c600fb230 2024-07-16T09:54:27.287 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:27.284+0000 7f5c5effd640 1 --2- 172.21.15.156:0/4146034655 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f5c48067190 0x7f5c48069640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:54:27.287 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:27.284+0000 7f5c5effd640 1 --2- 172.21.15.156:0/4146034655 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f5c48067190 0x7f5c48069640 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f5c54004fd0 tx=0x7f5c5400b040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:54:27.293 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:27.291+0000 7f5c47fff640 1 -- 172.21.15.156:0/4146034655 <== mon.1 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f5c50071c40 con 0x7f5c600fb230 2024-07-16T09:54:27.666 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:27.664+0000 7f5c661c9640 1 -- 172.21.15.156:0/4146034655 --> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f5c6005d960 con 0x7f5c48067190 2024-07-16T09:54:27.667 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:27.666+0000 7f5c47fff640 1 -- 172.21.15.156:0/4146034655 <== mgr.14208 v2:172.21.15.156:6800/2615741717 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+447 (secure 0 0 0) 0x7f5c6005d960 con 0x7f5c48067190 2024-07-16T09:54:27.670 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:27.668+0000 7f5c661c9640 1 -- 172.21.15.156:0/4146034655 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f5c48067190 msgr2=0x7f5c48069640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:54:27.670 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:27.668+0000 7f5c661c9640 1 --2- 172.21.15.156:0/4146034655 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f5c48067190 0x7f5c48069640 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f5c54004fd0 tx=0x7f5c5400b040 comp rx=0 tx=0).stop 2024-07-16T09:54:27.670 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:27.668+0000 7f5c661c9640 1 -- 172.21.15.156:0/4146034655 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5c600fb230 msgr2=0x7f5c600ff300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:54:27.671 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:27.668+0000 7f5c661c9640 1 --2- 172.21.15.156:0/4146034655 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5c600fb230 0x7f5c600ff300 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f5c500048a0 tx=0x7f5c50009da0 comp rx=0 tx=0).stop 2024-07-16T09:54:27.671 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:27.669+0000 7f5c661c9640 1 -- 172.21.15.156:0/4146034655 shutdown_connections 2024-07-16T09:54:27.671 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:27.669+0000 7f5c661c9640 1 --2- 172.21.15.156:0/4146034655 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f5c48067190 0x7f5c48069640 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:54:27.671 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:27.669+0000 7f5c661c9640 1 --2- 172.21.15.156:0/4146034655 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5c600fc500 0x7f5c600fd840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:54:27.671 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:27.669+0000 7f5c661c9640 1 --2- 172.21.15.156:0/4146034655 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5c600fb230 0x7f5c600ff300 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:54:27.671 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:27.669+0000 7f5c661c9640 1 -- 172.21.15.156:0/4146034655 >> 172.21.15.156:0/4146034655 conn(0x7f5c600f6a10 msgr2=0x7f5c600f8df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:54:27.671 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:27.669+0000 7f5c661c9640 1 -- 172.21.15.156:0/4146034655 shutdown_connections 2024-07-16T09:54:27.671 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:27.669+0000 7f5c661c9640 1 -- 172.21.15.156:0/4146034655 wait complete. 2024-07-16T09:54:27.682 INFO:teuthology.orchestra.run.smithi156.stdout:true 2024-07-16T09:54:27.790 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:27.789+0000 7fa53260f640 1 -- 172.21.15.156:0/3659633321 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa52c065a50 msgr2=0x7fa52c0fef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:54:27.790 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:27.789+0000 7fa53260f640 1 --2- 172.21.15.156:0/3659633321 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa52c065a50 0x7fa52c0fef50 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fa51c0048d0 tx=0x7fa51c009da0 comp rx=0 tx=0).stop 2024-07-16T09:54:27.791 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:27.789+0000 7fa53260f640 1 -- 172.21.15.156:0/3659633321 shutdown_connections 2024-07-16T09:54:27.791 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:27.789+0000 7fa53260f640 1 --2- 172.21.15.156:0/3659633321 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa52c0ff490 0x7fa52c101870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:54:27.791 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:27.789+0000 7fa53260f640 1 --2- 172.21.15.156:0/3659633321 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa52c065a50 0x7fa52c0fef50 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:54:27.791 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:27.789+0000 7fa53260f640 1 -- 172.21.15.156:0/3659633321 >> 172.21.15.156:0/3659633321 conn(0x7fa52c0f6a20 msgr2=0x7fa52c0f8e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:54:27.791 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:27.789+0000 7fa53260f640 1 -- 172.21.15.156:0/3659633321 shutdown_connections 2024-07-16T09:54:27.791 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:27.789+0000 7fa53260f640 1 -- 172.21.15.156:0/3659633321 wait complete. 2024-07-16T09:54:27.792 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:27.790+0000 7fa53260f640 1 Processor -- start 2024-07-16T09:54:27.792 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:27.790+0000 7fa53260f640 1 -- start start 2024-07-16T09:54:27.792 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:27.790+0000 7fa53260f640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa52c065a50 0x7fa52c0fc960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:54:27.792 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:27.790+0000 7fa53260f640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa52c0ff490 0x7fa52c0fae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:54:27.792 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:27.790+0000 7fa53260f640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa52c0fb3e0 con 0x7fa52c065a50 2024-07-16T09:54:27.792 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:27.790+0000 7fa53260f640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa52c0fb520 con 0x7fa52c0ff490 2024-07-16T09:54:27.793 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:27.790+0000 7fa52bfff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa52c065a50 0x7fa52c0fc960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:54:27.793 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:27.790+0000 7fa52b7fe640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa52c0ff490 0x7fa52c0fae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:54:27.793 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:27.790+0000 7fa52b7fe640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa52c0ff490 0x7fa52c0fae50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.156:51876/0 (socket says 172.21.15.156:51876) 2024-07-16T09:54:27.793 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:27.790+0000 7fa52b7fe640 1 -- 172.21.15.156:0/3965149237 learned_addr learned my addr 172.21.15.156:0/3965149237 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T09:54:27.793 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:27.791+0000 7fa52b7fe640 1 -- 172.21.15.156:0/3965149237 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa52c065a50 msgr2=0x7fa52c0fc960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:54:27.793 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:27.791+0000 7fa52b7fe640 1 --2- 172.21.15.156:0/3965149237 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa52c065a50 0x7fa52c0fc960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:54:27.793 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:27.791+0000 7fa52b7fe640 1 -- 172.21.15.156:0/3965149237 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa51c005040 con 0x7fa52c0ff490 2024-07-16T09:54:27.793 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:27.791+0000 7fa52bfff640 1 --2- 172.21.15.156:0/3965149237 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa52c065a50 0x7fa52c0fc960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-16T09:54:27.793 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:27.791+0000 7fa52b7fe640 1 --2- 172.21.15.156:0/3965149237 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa52c0ff490 0x7fa52c0fae50 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fa520002a50 tx=0x7fa520004600 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:54:27.794 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:27.791+0000 7fa528ff9640 1 -- 172.21.15.156:0/3965149237 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa520007c30 con 0x7fa52c0ff490 2024-07-16T09:54:27.794 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:27.791+0000 7fa528ff9640 1 -- 172.21.15.156:0/3965149237 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa52000f040 con 0x7fa52c0ff490 2024-07-16T09:54:27.794 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:27.791+0000 7fa528ff9640 1 -- 172.21.15.156:0/3965149237 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa520009570 con 0x7fa52c0ff490 2024-07-16T09:54:27.794 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:27.792+0000 7fa53260f640 1 -- 172.21.15.156:0/3965149237 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa52c0fb800 con 0x7fa52c0ff490 2024-07-16T09:54:27.794 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:27.792+0000 7fa53260f640 1 -- 172.21.15.156:0/3965149237 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa52c114fe0 con 0x7fa52c0ff490 2024-07-16T09:54:27.795 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:27.793+0000 7fa53260f640 1 -- 172.21.15.156:0/3965149237 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa52c0fa6a0 con 0x7fa52c0ff490 2024-07-16T09:54:27.795 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:27.794+0000 7fa528ff9640 1 -- 172.21.15.156:0/3965149237 <== mon.1 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 19) v1 ==== 85127+0+0 (secure 0 0 0) 0x7fa5200096d0 con 0x7fa52c0ff490 2024-07-16T09:54:27.796 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:27.795+0000 7fa528ff9640 1 --2- 172.21.15.156:0/3965149237 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7fa514067190 0x7fa514069640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:54:27.796 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:27.795+0000 7fa52bfff640 1 --2- 172.21.15.156:0/3965149237 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7fa514067190 0x7fa514069640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:54:27.796 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:27.795+0000 7fa528ff9640 1 -- 172.21.15.156:0/3965149237 <== mon.1 v2:172.21.15.176:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5464+0+0 (secure 0 0 0) 0x7fa5200848a0 con 0x7fa52c0ff490 2024-07-16T09:54:27.797 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:27.796+0000 7fa52bfff640 1 --2- 172.21.15.156:0/3965149237 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7fa514067190 0x7fa514069640 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fa51c000c00 tx=0x7fa51c035040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:54:27.804 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:27.803+0000 7fa528ff9640 1 -- 172.21.15.156:0/3965149237 <== mon.1 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fa520051aa0 con 0x7fa52c0ff490 2024-07-16T09:54:27.991 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:27.989+0000 7fa53260f640 1 -- 172.21.15.156:0/3965149237 --> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa52c0f8510 con 0x7fa514067190 2024-07-16T09:54:27.992 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:27.991+0000 7fa528ff9640 1 -- 172.21.15.156:0/3965149237 <== mgr.14208 v2:172.21.15.156:6800/2615741717 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+447 (secure 0 0 0) 0x7fa52c0f8510 con 0x7fa514067190 2024-07-16T09:54:27.995 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:27.993+0000 7fa53260f640 1 -- 172.21.15.156:0/3965149237 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7fa514067190 msgr2=0x7fa514069640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:54:27.995 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:27.993+0000 7fa53260f640 1 --2- 172.21.15.156:0/3965149237 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7fa514067190 0x7fa514069640 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fa51c000c00 tx=0x7fa51c035040 comp rx=0 tx=0).stop 2024-07-16T09:54:27.995 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:27.993+0000 7fa53260f640 1 -- 172.21.15.156:0/3965149237 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa52c0ff490 msgr2=0x7fa52c0fae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:54:27.995 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:27.993+0000 7fa53260f640 1 --2- 172.21.15.156:0/3965149237 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa52c0ff490 0x7fa52c0fae50 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fa520002a50 tx=0x7fa520004600 comp rx=0 tx=0).stop 2024-07-16T09:54:27.995 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:27.994+0000 7fa53260f640 1 -- 172.21.15.156:0/3965149237 shutdown_connections 2024-07-16T09:54:27.995 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:27.994+0000 7fa53260f640 1 --2- 172.21.15.156:0/3965149237 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7fa514067190 0x7fa514069640 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:54:27.995 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:27.994+0000 7fa53260f640 1 --2- 172.21.15.156:0/3965149237 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa52c0ff490 0x7fa52c0fae50 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:54:27.995 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:27.994+0000 7fa53260f640 1 --2- 172.21.15.156:0/3965149237 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa52c065a50 0x7fa52c0fc960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:54:27.995 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:27.994+0000 7fa53260f640 1 -- 172.21.15.156:0/3965149237 >> 172.21.15.156:0/3965149237 conn(0x7fa52c0f6a20 msgr2=0x7fa52c0f6df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:54:27.996 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:27.994+0000 7fa53260f640 1 -- 172.21.15.156:0/3965149237 shutdown_connections 2024-07-16T09:54:27.996 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:27.994+0000 7fa53260f640 1 -- 172.21.15.156:0/3965149237 wait complete. 2024-07-16T09:54:28.081 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:28.080+0000 7f5ac525f640 1 -- 172.21.15.156:0/3783333598 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5ab0032720 con 0x7f5ac00fc510 2024-07-16T09:54:28.082 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:28.080+0000 7f5ac84c5640 1 -- 172.21.15.156:0/3783333598 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5ac00fc510 msgr2=0x7f5ac00fc980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:54:28.082 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:28.080+0000 7f5ac84c5640 1 --2- 172.21.15.156:0/3783333598 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5ac00fc510 0x7f5ac00fc980 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f5ab00048d0 tx=0x7f5ab0009da0 comp rx=0 tx=0).stop 2024-07-16T09:54:28.082 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:28.081+0000 7f5ac84c5640 1 -- 172.21.15.156:0/3783333598 shutdown_connections 2024-07-16T09:54:28.082 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:28.081+0000 7f5ac84c5640 1 --2- 172.21.15.156:0/3783333598 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5ac00fc510 0x7f5ac00fc980 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:54:28.082 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:28.081+0000 7f5ac84c5640 1 --2- 172.21.15.156:0/3783333598 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5ac00fb240 0x7f5ac00fb650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:54:28.082 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:28.081+0000 7f5ac84c5640 1 -- 172.21.15.156:0/3783333598 >> 172.21.15.156:0/3783333598 conn(0x7f5ac00f6a20 msgr2=0x7f5ac00f8e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:54:28.083 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:28.081+0000 7f5ac84c5640 1 -- 172.21.15.156:0/3783333598 shutdown_connections 2024-07-16T09:54:28.083 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:28.081+0000 7f5ac84c5640 1 -- 172.21.15.156:0/3783333598 wait complete. 2024-07-16T09:54:28.084 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:28.081+0000 7f5ac84c5640 1 Processor -- start 2024-07-16T09:54:28.084 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:28.082+0000 7f5ac84c5640 1 -- start start 2024-07-16T09:54:28.084 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:28.082+0000 7f5ac84c5640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5ac00fb240 0x7f5ac0113540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:54:28.084 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:28.082+0000 7f5ac84c5640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5ac00fc510 0x7f5ac0113a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:54:28.084 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:28.082+0000 7f5ac84c5640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ac01140d0 con 0x7f5ac00fb240 2024-07-16T09:54:28.084 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:28.082+0000 7f5ac84c5640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ac01148f0 con 0x7f5ac00fc510 2024-07-16T09:54:28.084 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:28.082+0000 7f5ac5a60640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5ac00fc510 0x7f5ac0113a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:54:28.085 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:28.082+0000 7f5ac5a60640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5ac00fc510 0x7f5ac0113a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.156:51888/0 (socket says 172.21.15.156:51888) 2024-07-16T09:54:28.085 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:28.082+0000 7f5ac6261640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5ac00fb240 0x7f5ac0113540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:54:28.085 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:28.082+0000 7f5ac5a60640 1 -- 172.21.15.156:0/2573542610 learned_addr learned my addr 172.21.15.156:0/2573542610 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T09:54:28.085 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:28.082+0000 7f5ac5a60640 1 -- 172.21.15.156:0/2573542610 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5ac00fb240 msgr2=0x7f5ac0113540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:54:28.085 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:28.082+0000 7f5ac5a60640 1 --2- 172.21.15.156:0/2573542610 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5ac00fb240 0x7f5ac0113540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:54:28.085 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:28.082+0000 7f5ac5a60640 1 -- 172.21.15.156:0/2573542610 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ab0005040 con 0x7f5ac00fc510 2024-07-16T09:54:28.085 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:28.083+0000 7f5ac6261640 1 --2- 172.21.15.156:0/2573542610 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5ac00fb240 0x7f5ac0113540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-16T09:54:28.085 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:28.083+0000 7f5ac5a60640 1 --2- 172.21.15.156:0/2573542610 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5ac00fc510 0x7f5ac0113a80 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f5ab00048a0 tx=0x7f5ab0009da0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:54:28.085 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:28.083+0000 7f5ab6ffd640 1 -- 172.21.15.156:0/2573542610 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5ab0038070 con 0x7f5ac00fc510 2024-07-16T09:54:28.085 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:28.083+0000 7f5ab6ffd640 1 -- 172.21.15.156:0/2573542610 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5ab00092b0 con 0x7f5ac00fc510 2024-07-16T09:54:28.086 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:28.083+0000 7f5ac84c5640 1 -- 172.21.15.156:0/2573542610 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ac0114b20 con 0x7f5ac00fc510 2024-07-16T09:54:28.086 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:28.083+0000 7f5ab6ffd640 1 -- 172.21.15.156:0/2573542610 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5ab0033780 con 0x7f5ac00fc510 2024-07-16T09:54:28.086 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:28.083+0000 7f5ac84c5640 1 -- 172.21.15.156:0/2573542610 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ac0114fa0 con 0x7f5ac00fc510 2024-07-16T09:54:28.086 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:28.085+0000 7f5ac84c5640 1 -- 172.21.15.156:0/2573542610 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ac00fa6a0 con 0x7f5ac00fc510 2024-07-16T09:54:28.087 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:28.085+0000 7f5ab6ffd640 1 -- 172.21.15.156:0/2573542610 <== mon.1 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 19) v1 ==== 85127+0+0 (secure 0 0 0) 0x7f5ab0030020 con 0x7f5ac00fc510 2024-07-16T09:54:28.087 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:28.085+0000 7f5ab6ffd640 1 --2- 172.21.15.156:0/2573542610 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f5aac067190 0x7f5aac069640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:54:28.087 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:28.085+0000 7f5ab6ffd640 1 -- 172.21.15.156:0/2573542610 <== mon.1 v2:172.21.15.176:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f5ab00a48d0 con 0x7f5ac00fc510 2024-07-16T09:54:28.087 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:28.086+0000 7f5ac6261640 1 --2- 172.21.15.156:0/2573542610 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f5aac067190 0x7f5aac069640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:54:28.087 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:28.086+0000 7f5ac6261640 1 --2- 172.21.15.156:0/2573542610 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f5aac067190 0x7f5aac069640 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f5abc004fd0 tx=0x7f5abc00b040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:54:28.095 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:28.093+0000 7f5ab6ffd640 1 -- 172.21.15.156:0/2573542610 <== mon.1 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f5ab0071c20 con 0x7f5ac00fc510 2024-07-16T09:54:28.307 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:28.305+0000 7f5ac84c5640 1 -- 172.21.15.156:0/2573542610 --> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f5ac0115390 con 0x7f5aac067190 2024-07-16T09:54:28.307 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:54:28 smithi156 ceph-mon[29302]: pgmap v215: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-07-16T09:54:28.316 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:28.315+0000 7f5ab6ffd640 1 -- 172.21.15.156:0/2573542610 <== mgr.14208 v2:172.21.15.156:6800/2615741717 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3388 (secure 0 0 0) 0x7f5ac0115390 con 0x7f5aac067190 2024-07-16T09:54:28.316 INFO:teuthology.orchestra.run.smithi156.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-07-16T09:54:28.316 INFO:teuthology.orchestra.run.smithi156.stdout:alertmanager.smithi156 smithi156 *:9093,9094 running (5m) 67s ago 6m 24.6M - 0.25.0 c8568f914cd2 83419a2a37f4 2024-07-16T09:54:28.317 INFO:teuthology.orchestra.run.smithi156.stdout:crash.smithi156 smithi156 running (6m) 67s ago 6m 6899k - 17.2.7-1104-ge6231b04 d5de5f953b95 bf6d49ea06b4 2024-07-16T09:54:28.317 INFO:teuthology.orchestra.run.smithi156.stdout:crash.smithi176 smithi176 running (5m) 71s ago 5m 6903k - 17.2.7-1104-ge6231b04 d5de5f953b95 907c9fa1e890 2024-07-16T09:54:28.317 INFO:teuthology.orchestra.run.smithi156.stdout:grafana.smithi156 smithi156 *:3000 running (5m) 67s ago 6m 85.0M - 9.4.7 954c08fa6188 184256cf055d 2024-07-16T09:54:28.317 INFO:teuthology.orchestra.run.smithi156.stdout:mds.cephfs.smithi156.evwwaz smithi156 running (84s) 67s ago 84s 14.9M - 17.2.7-1104-ge6231b04 d5de5f953b95 7c0ff9ae43f8 2024-07-16T09:54:28.317 INFO:teuthology.orchestra.run.smithi156.stdout:mds.cephfs.smithi156.rdibys smithi156 running (89s) 67s ago 88s 15.8M - 17.2.7-1104-ge6231b04 d5de5f953b95 b985cd1e6924 2024-07-16T09:54:28.317 INFO:teuthology.orchestra.run.smithi156.stdout:mds.cephfs.smithi176.dvvbud smithi176 running (87s) 71s ago 86s 14.6M - 17.2.7-1104-ge6231b04 d5de5f953b95 1b00147950f9 2024-07-16T09:54:28.317 INFO:teuthology.orchestra.run.smithi156.stdout:mds.cephfs.smithi176.wahbsi smithi176 running (82s) 71s ago 81s 14.7M - 17.2.7-1104-ge6231b04 d5de5f953b95 866664af2311 2024-07-16T09:54:28.317 INFO:teuthology.orchestra.run.smithi156.stdout:mgr.smithi156.zkvylv smithi156 *:9283 running (7m) 67s ago 7m 450M - 17.2.7-1104-ge6231b04 d5de5f953b95 a6937c84750a 2024-07-16T09:54:28.317 INFO:teuthology.orchestra.run.smithi156.stdout:mgr.smithi176.dqbrvx smithi176 *:8443,9283 running (5m) 71s ago 5m 402M - 17.2.7-1104-ge6231b04 d5de5f953b95 654c9bc9c82a 2024-07-16T09:54:28.317 INFO:teuthology.orchestra.run.smithi156.stdout:mon.smithi156 smithi156 running (7m) 67s ago 8m 47.6M 2048M 17.2.7-1104-ge6231b04 d5de5f953b95 6471c8acff40 2024-07-16T09:54:28.317 INFO:teuthology.orchestra.run.smithi156.stdout:mon.smithi176 smithi176 running (5m) 71s ago 5m 45.1M 2048M 17.2.7-1104-ge6231b04 d5de5f953b95 3a293ac6ef01 2024-07-16T09:54:28.317 INFO:teuthology.orchestra.run.smithi156.stdout:node-exporter.smithi156 smithi156 *:9100 running (6m) 67s ago 6m 15.9M - 1.5.0 0da6a335fe13 2c136840a7a9 2024-07-16T09:54:28.318 INFO:teuthology.orchestra.run.smithi156.stdout:node-exporter.smithi176 smithi176 *:9100 running (5m) 71s ago 5m 15.6M - 1.5.0 0da6a335fe13 536716db6afe 2024-07-16T09:54:28.318 INFO:teuthology.orchestra.run.smithi156.stdout:osd.0 smithi156 running (5m) 67s ago 5m 48.5M 4288M 17.2.7-1104-ge6231b04 d5de5f953b95 0471758ec349 2024-07-16T09:54:28.318 INFO:teuthology.orchestra.run.smithi156.stdout:osd.1 smithi156 running (4m) 67s ago 4m 49.0M 4288M 17.2.7-1104-ge6231b04 d5de5f953b95 149a0bda2f61 2024-07-16T09:54:28.318 INFO:teuthology.orchestra.run.smithi156.stdout:osd.2 smithi156 running (4m) 67s ago 4m 44.5M 4288M 17.2.7-1104-ge6231b04 d5de5f953b95 f28acec5cb79 2024-07-16T09:54:28.318 INFO:teuthology.orchestra.run.smithi156.stdout:osd.3 smithi176 running (3m) 71s ago 3m 44.8M 5312M 17.2.7-1104-ge6231b04 d5de5f953b95 733861ba84dc 2024-07-16T09:54:28.318 INFO:teuthology.orchestra.run.smithi156.stdout:osd.4 smithi176 running (3m) 71s ago 3m 41.3M 5312M 17.2.7-1104-ge6231b04 d5de5f953b95 ef0c67cec086 2024-07-16T09:54:28.318 INFO:teuthology.orchestra.run.smithi156.stdout:osd.5 smithi176 running (2m) 71s ago 2m 42.4M 5312M 17.2.7-1104-ge6231b04 d5de5f953b95 996e15448a52 2024-07-16T09:54:28.318 INFO:teuthology.orchestra.run.smithi156.stdout:prometheus.smithi156 smithi156 *:9095 running (5m) 67s ago 6m 55.5M - 2.43.0 a07b618ecd1d 1655b4227cd0 2024-07-16T09:54:28.319 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:28.317+0000 7f5ac84c5640 1 -- 172.21.15.156:0/2573542610 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f5aac067190 msgr2=0x7f5aac069640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:54:28.319 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:28.317+0000 7f5ac84c5640 1 --2- 172.21.15.156:0/2573542610 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f5aac067190 0x7f5aac069640 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f5abc004fd0 tx=0x7f5abc00b040 comp rx=0 tx=0).stop 2024-07-16T09:54:28.319 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:28.317+0000 7f5ac84c5640 1 -- 172.21.15.156:0/2573542610 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5ac00fc510 msgr2=0x7f5ac0113a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:54:28.319 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:28.317+0000 7f5ac84c5640 1 --2- 172.21.15.156:0/2573542610 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5ac00fc510 0x7f5ac0113a80 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f5ab00048a0 tx=0x7f5ab0009da0 comp rx=0 tx=0).stop 2024-07-16T09:54:28.320 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:28.317+0000 7f5ac84c5640 1 -- 172.21.15.156:0/2573542610 shutdown_connections 2024-07-16T09:54:28.320 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:28.317+0000 7f5ac84c5640 1 --2- 172.21.15.156:0/2573542610 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f5aac067190 0x7f5aac069640 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:54:28.320 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:28.317+0000 7f5ac84c5640 1 --2- 172.21.15.156:0/2573542610 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5ac00fc510 0x7f5ac0113a80 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:54:28.320 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:28.317+0000 7f5ac84c5640 1 --2- 172.21.15.156:0/2573542610 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5ac00fb240 0x7f5ac0113540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:54:28.320 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:28.317+0000 7f5ac84c5640 1 -- 172.21.15.156:0/2573542610 >> 172.21.15.156:0/2573542610 conn(0x7f5ac00f6a20 msgr2=0x7f5ac00f8510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:54:28.320 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:28.318+0000 7f5ac84c5640 1 -- 172.21.15.156:0/2573542610 shutdown_connections 2024-07-16T09:54:28.320 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:28.318+0000 7f5ac84c5640 1 -- 172.21.15.156:0/2573542610 wait complete. 2024-07-16T09:54:28.343 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:54:28 smithi176 ceph-mon[36626]: pgmap v215: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-07-16T09:54:28.404 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:28.402+0000 7f29eb99d640 1 -- 172.21.15.156:0/2889090692 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f29e40fd600 msgr2=0x7f29e40fda70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:54:28.404 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:28.402+0000 7f29eb99d640 1 --2- 172.21.15.156:0/2889090692 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f29e40fd600 0x7f29e40fda70 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f29d40048d0 tx=0x7f29d4009da0 comp rx=0 tx=0).stop 2024-07-16T09:54:28.404 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:28.402+0000 7f29eb99d640 1 -- 172.21.15.156:0/2889090692 shutdown_connections 2024-07-16T09:54:28.404 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:28.402+0000 7f29eb99d640 1 --2- 172.21.15.156:0/2889090692 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f29e40fd600 0x7f29e40fda70 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:54:28.404 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:28.402+0000 7f29eb99d640 1 --2- 172.21.15.156:0/2889090692 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f29e40fc330 0x7f29e40fc740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:54:28.404 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:28.402+0000 7f29eb99d640 1 -- 172.21.15.156:0/2889090692 >> 172.21.15.156:0/2889090692 conn(0x7f29e40f7af0 msgr2=0x7f29e40f9f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:54:28.404 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:28.402+0000 7f29eb99d640 1 -- 172.21.15.156:0/2889090692 shutdown_connections 2024-07-16T09:54:28.404 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:28.402+0000 7f29eb99d640 1 -- 172.21.15.156:0/2889090692 wait complete. 2024-07-16T09:54:28.406 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:28.403+0000 7f29eb99d640 1 Processor -- start 2024-07-16T09:54:28.406 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:28.403+0000 7f29eb99d640 1 -- start start 2024-07-16T09:54:28.406 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:28.404+0000 7f29eb99d640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f29e40fc330 0x7f29e410cf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:54:28.406 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:28.404+0000 7f29eb99d640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f29e40fd600 0x7f29e410d490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:54:28.406 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:28.404+0000 7f29eb99d640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29e410dae0 con 0x7f29e40fd600 2024-07-16T09:54:28.406 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:28.404+0000 7f29e9739640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f29e40fc330 0x7f29e410cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:54:28.406 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:28.404+0000 7f29e9739640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f29e40fc330 0x7f29e410cf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.156:51900/0 (socket says 172.21.15.156:51900) 2024-07-16T09:54:28.406 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:28.404+0000 7f29e9739640 1 -- 172.21.15.156:0/3534112926 learned_addr learned my addr 172.21.15.156:0/3534112926 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T09:54:28.406 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:28.404+0000 7f29e8f38640 1 --2- 172.21.15.156:0/3534112926 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f29e40fd600 0x7f29e410d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:54:28.407 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:28.404+0000 7f29eb99d640 1 -- 172.21.15.156:0/3534112926 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29e410e300 con 0x7f29e40fc330 2024-07-16T09:54:28.407 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:28.405+0000 7f29e8f38640 1 -- 172.21.15.156:0/3534112926 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f29e40fc330 msgr2=0x7f29e410cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:54:28.407 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:28.405+0000 7f29e8f38640 1 --2- 172.21.15.156:0/3534112926 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f29e40fc330 0x7f29e410cf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:54:28.407 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:28.405+0000 7f29e8f38640 1 -- 172.21.15.156:0/3534112926 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29d4005040 con 0x7f29e40fd600 2024-07-16T09:54:28.408 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:28.405+0000 7f29e8f38640 1 --2- 172.21.15.156:0/3534112926 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f29e40fd600 0x7f29e410d490 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f29d40048a0 tx=0x7f29d4009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:54:28.408 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:28.405+0000 7f29d9ffb640 1 -- 172.21.15.156:0/3534112926 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f29d4038070 con 0x7f29e40fd600 2024-07-16T09:54:28.408 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:28.405+0000 7f29d9ffb640 1 -- 172.21.15.156:0/3534112926 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f29d40092b0 con 0x7f29e40fd600 2024-07-16T09:54:28.408 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:28.405+0000 7f29eb99d640 1 -- 172.21.15.156:0/3534112926 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29e410e530 con 0x7f29e40fd600 2024-07-16T09:54:28.408 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:28.405+0000 7f29d9ffb640 1 -- 172.21.15.156:0/3534112926 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f29d4033660 con 0x7f29e40fd600 2024-07-16T09:54:28.408 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:28.406+0000 7f29eb99d640 1 -- 172.21.15.156:0/3534112926 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29e410ea40 con 0x7f29e40fd600 2024-07-16T09:54:28.408 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:28.407+0000 7f29eb99d640 1 -- 172.21.15.156:0/3534112926 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29e40fb790 con 0x7f29e40fd600 2024-07-16T09:54:28.409 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:28.408+0000 7f29d9ffb640 1 -- 172.21.15.156:0/3534112926 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 19) v1 ==== 85127+0+0 (secure 0 0 0) 0x7f29d4030020 con 0x7f29e40fd600 2024-07-16T09:54:28.410 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:28.408+0000 7f29d9ffb640 1 --2- 172.21.15.156:0/3534112926 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f29d0067190 0x7f29d0069640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:54:28.410 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:28.409+0000 7f29d9ffb640 1 -- 172.21.15.156:0/3534112926 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f29d40a4970 con 0x7f29e40fd600 2024-07-16T09:54:28.410 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:28.409+0000 7f29e9739640 1 --2- 172.21.15.156:0/3534112926 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f29d0067190 0x7f29d0069640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:54:28.410 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:28.409+0000 7f29e9739640 1 --2- 172.21.15.156:0/3534112926 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f29d0067190 0x7f29d0069640 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f29e0004aa0 tx=0x7f29e000b040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:54:28.414 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:28.413+0000 7f29d9ffb640 1 -- 172.21.15.156:0/3534112926 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f29d4071c30 con 0x7f29e40fd600 2024-07-16T09:54:28.757 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:28.755+0000 7f29eb99d640 1 -- 172.21.15.156:0/3534112926 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f29e404b530 con 0x7f29e40fd600 2024-07-16T09:54:28.757 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:28.756+0000 7f29d9ffb640 1 -- 172.21.15.156:0/3534112926 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+643 (secure 0 0 0) 0x7f29d40412a0 con 0x7f29e40fd600 2024-07-16T09:54:28.757 INFO:teuthology.orchestra.run.smithi156.stdout:{ 2024-07-16T09:54:28.757 INFO:teuthology.orchestra.run.smithi156.stdout: "mon": { 2024-07-16T09:54:28.757 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph version 17.2.7-1104-ge6231b04 (e6231b04b52ebf3b9b31803282dd3074439e51ac) quincy (stable)": 2 2024-07-16T09:54:28.758 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-07-16T09:54:28.758 INFO:teuthology.orchestra.run.smithi156.stdout: "mgr": { 2024-07-16T09:54:28.758 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph version 17.2.7-1104-ge6231b04 (e6231b04b52ebf3b9b31803282dd3074439e51ac) quincy (stable)": 2 2024-07-16T09:54:28.758 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-07-16T09:54:28.758 INFO:teuthology.orchestra.run.smithi156.stdout: "osd": { 2024-07-16T09:54:28.758 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph version 17.2.7-1104-ge6231b04 (e6231b04b52ebf3b9b31803282dd3074439e51ac) quincy (stable)": 6 2024-07-16T09:54:28.758 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-07-16T09:54:28.758 INFO:teuthology.orchestra.run.smithi156.stdout: "mds": { 2024-07-16T09:54:28.758 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph version 17.2.7-1104-ge6231b04 (e6231b04b52ebf3b9b31803282dd3074439e51ac) quincy (stable)": 4 2024-07-16T09:54:28.758 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-07-16T09:54:28.758 INFO:teuthology.orchestra.run.smithi156.stdout: "overall": { 2024-07-16T09:54:28.758 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph version 17.2.7-1104-ge6231b04 (e6231b04b52ebf3b9b31803282dd3074439e51ac) quincy (stable)": 14 2024-07-16T09:54:28.758 INFO:teuthology.orchestra.run.smithi156.stdout: } 2024-07-16T09:54:28.759 INFO:teuthology.orchestra.run.smithi156.stdout:} 2024-07-16T09:54:28.760 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:28.759+0000 7f29eb99d640 1 -- 172.21.15.156:0/3534112926 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f29d0067190 msgr2=0x7f29d0069640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:54:28.760 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:28.759+0000 7f29eb99d640 1 --2- 172.21.15.156:0/3534112926 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f29d0067190 0x7f29d0069640 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f29e0004aa0 tx=0x7f29e000b040 comp rx=0 tx=0).stop 2024-07-16T09:54:28.761 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:28.759+0000 7f29eb99d640 1 -- 172.21.15.156:0/3534112926 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f29e40fd600 msgr2=0x7f29e410d490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:54:28.761 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:28.759+0000 7f29eb99d640 1 --2- 172.21.15.156:0/3534112926 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f29e40fd600 0x7f29e410d490 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f29d40048a0 tx=0x7f29d4009da0 comp rx=0 tx=0).stop 2024-07-16T09:54:28.761 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:28.759+0000 7f29eb99d640 1 -- 172.21.15.156:0/3534112926 shutdown_connections 2024-07-16T09:54:28.761 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:28.759+0000 7f29eb99d640 1 --2- 172.21.15.156:0/3534112926 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f29d0067190 0x7f29d0069640 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:54:28.761 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:28.759+0000 7f29eb99d640 1 --2- 172.21.15.156:0/3534112926 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f29e40fd600 0x7f29e410d490 secure :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f29d40048a0 tx=0x7f29d4009da0 comp rx=0 tx=0).stop 2024-07-16T09:54:28.761 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:28.759+0000 7f29eb99d640 1 --2- 172.21.15.156:0/3534112926 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f29e40fc330 0x7f29e410cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:54:28.761 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:28.759+0000 7f29eb99d640 1 -- 172.21.15.156:0/3534112926 >> 172.21.15.156:0/3534112926 conn(0x7f29e40f7af0 msgr2=0x7f29e40f91c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:54:28.761 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:28.759+0000 7f29eb99d640 1 -- 172.21.15.156:0/3534112926 shutdown_connections 2024-07-16T09:54:28.761 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:28.759+0000 7f29eb99d640 1 -- 172.21.15.156:0/3534112926 wait complete. 2024-07-16T09:54:28.878 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:28.876+0000 7fa390e6f640 1 -- 172.21.15.156:0/2107857695 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa38c0659f0 msgr2=0x7fa38c1013b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:54:28.878 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:28.876+0000 7fa390e6f640 1 --2- 172.21.15.156:0/2107857695 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa38c0659f0 0x7fa38c1013b0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fa37c0048d0 tx=0x7fa37c009da0 comp rx=0 tx=0).stop 2024-07-16T09:54:28.879 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:28.877+0000 7fa390e6f640 1 -- 172.21.15.156:0/2107857695 shutdown_connections 2024-07-16T09:54:28.879 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:28.877+0000 7fa390e6f640 1 --2- 172.21.15.156:0/2107857695 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa38c1018f0 0x7fa38c103cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:54:28.879 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:28.877+0000 7fa390e6f640 1 --2- 172.21.15.156:0/2107857695 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa38c0659f0 0x7fa38c1013b0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:54:28.879 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:28.877+0000 7fa390e6f640 1 -- 172.21.15.156:0/2107857695 >> 172.21.15.156:0/2107857695 conn(0x7fa38c0f6a20 msgr2=0x7fa38c0f8e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:54:28.879 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:28.877+0000 7fa390e6f640 1 -- 172.21.15.156:0/2107857695 shutdown_connections 2024-07-16T09:54:28.879 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:28.877+0000 7fa390e6f640 1 -- 172.21.15.156:0/2107857695 wait complete. 2024-07-16T09:54:28.880 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:28.877+0000 7fa390e6f640 1 Processor -- start 2024-07-16T09:54:28.881 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:28.878+0000 7fa390e6f640 1 -- start start 2024-07-16T09:54:28.881 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:28.878+0000 7fa390e6f640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa38c0659f0 0x7fa38c0fd330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:54:28.881 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:28.878+0000 7fa390e6f640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa38c1018f0 0x7fa38c0fd870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:54:28.881 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:28.878+0000 7fa390e6f640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa38c0fde70 con 0x7fa38c0659f0 2024-07-16T09:54:28.881 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:28.878+0000 7fa390e6f640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa38c0fe730 con 0x7fa38c1018f0 2024-07-16T09:54:28.881 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:28.878+0000 7fa389d9b640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa38c1018f0 0x7fa38c0fd870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:54:28.881 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:28.878+0000 7fa389d9b640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa38c1018f0 0x7fa38c0fd870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.156:51920/0 (socket says 172.21.15.156:51920) 2024-07-16T09:54:28.881 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:28.878+0000 7fa389d9b640 1 -- 172.21.15.156:0/511365572 learned_addr learned my addr 172.21.15.156:0/511365572 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T09:54:28.881 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:28.878+0000 7fa38a59c640 1 --2- 172.21.15.156:0/511365572 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa38c0659f0 0x7fa38c0fd330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:54:28.881 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:28.878+0000 7fa389d9b640 1 -- 172.21.15.156:0/511365572 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa38c0659f0 msgr2=0x7fa38c0fd330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:54:28.881 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:28.878+0000 7fa389d9b640 1 --2- 172.21.15.156:0/511365572 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa38c0659f0 0x7fa38c0fd330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:54:28.881 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:28.878+0000 7fa389d9b640 1 -- 172.21.15.156:0/511365572 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa37c005040 con 0x7fa38c1018f0 2024-07-16T09:54:28.881 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:28.879+0000 7fa38a59c640 1 --2- 172.21.15.156:0/511365572 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa38c0659f0 0x7fa38c0fd330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-16T09:54:28.882 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:28.879+0000 7fa389d9b640 1 --2- 172.21.15.156:0/511365572 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa38c1018f0 0x7fa38c0fd870 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fa38000eb70 tx=0x7fa38000b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:54:28.882 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:28.879+0000 7fa37affd640 1 -- 172.21.15.156:0/511365572 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa380007c30 con 0x7fa38c1018f0 2024-07-16T09:54:28.882 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:28.879+0000 7fa390e6f640 1 -- 172.21.15.156:0/511365572 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa38c0fe9b0 con 0x7fa38c1018f0 2024-07-16T09:54:28.882 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:28.879+0000 7fa390e6f640 1 -- 172.21.15.156:0/511365572 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa38c0fef50 con 0x7fa38c1018f0 2024-07-16T09:54:28.882 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:28.879+0000 7fa37affd640 1 -- 172.21.15.156:0/511365572 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa38000f040 con 0x7fa38c1018f0 2024-07-16T09:54:28.882 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:28.879+0000 7fa37affd640 1 -- 172.21.15.156:0/511365572 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa380009580 con 0x7fa38c1018f0 2024-07-16T09:54:28.883 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:28.880+0000 7fa390e6f640 1 -- 172.21.15.156:0/511365572 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa38c0fa6a0 con 0x7fa38c1018f0 2024-07-16T09:54:28.883 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:28.881+0000 7fa37affd640 1 -- 172.21.15.156:0/511365572 <== mon.1 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 19) v1 ==== 85127+0+0 (secure 0 0 0) 0x7fa38000c6a0 con 0x7fa38c1018f0 2024-07-16T09:54:28.883 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:28.881+0000 7fa37affd640 1 --2- 172.21.15.156:0/511365572 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7fa374067190 0x7fa374069640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:54:28.883 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:28.881+0000 7fa37affd640 1 -- 172.21.15.156:0/511365572 <== mon.1 v2:172.21.15.176:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5464+0+0 (secure 0 0 0) 0x7fa380084830 con 0x7fa38c1018f0 2024-07-16T09:54:28.883 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:28.881+0000 7fa38a59c640 1 --2- 172.21.15.156:0/511365572 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7fa374067190 0x7fa374069640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:54:28.883 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:28.882+0000 7fa38a59c640 1 --2- 172.21.15.156:0/511365572 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7fa374067190 0x7fa374069640 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fa37c009fa0 tx=0x7fa37c035040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:54:28.887 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:28.886+0000 7fa37affd640 1 -- 172.21.15.156:0/511365572 <== mon.1 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fa380051af0 con 0x7fa38c1018f0 2024-07-16T09:54:29.100 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:29.099+0000 7fa390e6f640 1 -- 172.21.15.156:0/511365572 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fa38c04b530 con 0x7fa38c1018f0 2024-07-16T09:54:29.103 INFO:teuthology.orchestra.run.smithi156.stdout:e16 2024-07-16T09:54:29.103 INFO:teuthology.orchestra.run.smithi156.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-07-16T09:54:29.103 INFO:teuthology.orchestra.run.smithi156.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-07-16T09:54:29.103 INFO:teuthology.orchestra.run.smithi156.stdout:legacy client fscid: 1 2024-07-16T09:54:29.103 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-07-16T09:54:29.103 INFO:teuthology.orchestra.run.smithi156.stdout:Filesystem 'cephfs' (1) 2024-07-16T09:54:29.103 INFO:teuthology.orchestra.run.smithi156.stdout:fs_name cephfs 2024-07-16T09:54:29.104 INFO:teuthology.orchestra.run.smithi156.stdout:epoch 16 2024-07-16T09:54:29.104 INFO:teuthology.orchestra.run.smithi156.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-07-16T09:54:29.104 INFO:teuthology.orchestra.run.smithi156.stdout:created 2024-07-16T09:52:55.214975+0000 2024-07-16T09:54:29.104 INFO:teuthology.orchestra.run.smithi156.stdout:modified 2024-07-16T09:53:11.798619+0000 2024-07-16T09:54:29.104 INFO:teuthology.orchestra.run.smithi156.stdout:tableserver 0 2024-07-16T09:54:29.104 INFO:teuthology.orchestra.run.smithi156.stdout:root 0 2024-07-16T09:54:29.104 INFO:teuthology.orchestra.run.smithi156.stdout:session_timeout 60 2024-07-16T09:54:29.104 INFO:teuthology.orchestra.run.smithi156.stdout:session_autoclose 300 2024-07-16T09:54:29.104 INFO:teuthology.orchestra.run.smithi156.stdout:max_file_size 1099511627776 2024-07-16T09:54:29.104 INFO:teuthology.orchestra.run.smithi156.stdout:required_client_features {} 2024-07-16T09:54:29.105 INFO:teuthology.orchestra.run.smithi156.stdout:last_failure 0 2024-07-16T09:54:29.105 INFO:teuthology.orchestra.run.smithi156.stdout:last_failure_osd_epoch 0 2024-07-16T09:54:29.105 INFO:teuthology.orchestra.run.smithi156.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-07-16T09:54:29.105 INFO:teuthology.orchestra.run.smithi156.stdout:max_mds 2 2024-07-16T09:54:29.105 INFO:teuthology.orchestra.run.smithi156.stdout:in 0,1 2024-07-16T09:54:29.105 INFO:teuthology.orchestra.run.smithi156.stdout:up {0=14458,1=24261} 2024-07-16T09:54:29.105 INFO:teuthology.orchestra.run.smithi156.stdout:failed 2024-07-16T09:54:29.105 INFO:teuthology.orchestra.run.smithi156.stdout:damaged 2024-07-16T09:54:29.105 INFO:teuthology.orchestra.run.smithi156.stdout:stopped 2024-07-16T09:54:29.105 INFO:teuthology.orchestra.run.smithi156.stdout:data_pools [3] 2024-07-16T09:54:29.105 INFO:teuthology.orchestra.run.smithi156.stdout:metadata_pool 2 2024-07-16T09:54:29.105 INFO:teuthology.orchestra.run.smithi156.stdout:inline_data disabled 2024-07-16T09:54:29.105 INFO:teuthology.orchestra.run.smithi156.stdout:balancer 2024-07-16T09:54:29.106 INFO:teuthology.orchestra.run.smithi156.stdout:standby_count_wanted 1 2024-07-16T09:54:29.106 INFO:teuthology.orchestra.run.smithi156.stdout:[mds.cephfs.smithi156.rdibys{0:14458} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.156:6826/1080176013,v1:172.21.15.156:6827/1080176013] compat {c=[1],r=[1],i=[7ff]}] 2024-07-16T09:54:29.106 INFO:teuthology.orchestra.run.smithi156.stdout:[mds.cephfs.smithi156.evwwaz{0:14468} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.156:6828/560329502,v1:172.21.15.156:6829/560329502] compat {c=[1],r=[1],i=[7ff]}] 2024-07-16T09:54:29.106 INFO:teuthology.orchestra.run.smithi156.stdout:[mds.cephfs.smithi176.dvvbud{1:24261} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.176:6824/1577019287,v1:172.21.15.176:6825/1577019287] compat {c=[1],r=[1],i=[7ff]}] 2024-07-16T09:54:29.106 INFO:teuthology.orchestra.run.smithi156.stdout:[mds.cephfs.smithi176.wahbsi{1:24273} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.176:6826/2196775412,v1:172.21.15.176:6827/2196775412] compat {c=[1],r=[1],i=[7ff]}] 2024-07-16T09:54:29.106 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-07-16T09:54:29.106 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-07-16T09:54:29.106 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:29.100+0000 7fa37affd640 1 -- 172.21.15.156:0/511365572 <== mon.1 v2:172.21.15.176:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 16 v16) v1 ==== 76+0+1835 (secure 0 0 0) 0x7fa380019070 con 0x7fa38c1018f0 2024-07-16T09:54:29.107 INFO:teuthology.orchestra.run.smithi156.stderr:dumped fsmap epoch 16 2024-07-16T09:54:29.107 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:29.102+0000 7fa390e6f640 1 -- 172.21.15.156:0/511365572 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7fa374067190 msgr2=0x7fa374069640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:54:29.107 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:29.102+0000 7fa390e6f640 1 --2- 172.21.15.156:0/511365572 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7fa374067190 0x7fa374069640 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fa37c009fa0 tx=0x7fa37c035040 comp rx=0 tx=0).stop 2024-07-16T09:54:29.107 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:29.102+0000 7fa390e6f640 1 -- 172.21.15.156:0/511365572 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa38c1018f0 msgr2=0x7fa38c0fd870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:54:29.107 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:29.102+0000 7fa390e6f640 1 --2- 172.21.15.156:0/511365572 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa38c1018f0 0x7fa38c0fd870 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fa38000eb70 tx=0x7fa38000b040 comp rx=0 tx=0).stop 2024-07-16T09:54:29.107 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:29.102+0000 7fa390e6f640 1 -- 172.21.15.156:0/511365572 shutdown_connections 2024-07-16T09:54:29.107 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:29.102+0000 7fa390e6f640 1 --2- 172.21.15.156:0/511365572 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7fa374067190 0x7fa374069640 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:54:29.107 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:29.102+0000 7fa390e6f640 1 --2- 172.21.15.156:0/511365572 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa38c1018f0 0x7fa38c0fd870 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:54:29.107 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:29.102+0000 7fa390e6f640 1 --2- 172.21.15.156:0/511365572 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa38c0659f0 0x7fa38c0fd330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:54:29.107 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:29.102+0000 7fa390e6f640 1 -- 172.21.15.156:0/511365572 >> 172.21.15.156:0/511365572 conn(0x7fa38c0f6a20 msgr2=0x7fa38c0f6df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:54:29.107 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:29.102+0000 7fa390e6f640 1 -- 172.21.15.156:0/511365572 shutdown_connections 2024-07-16T09:54:29.107 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:29.102+0000 7fa390e6f640 1 -- 172.21.15.156:0/511365572 wait complete. 2024-07-16T09:54:29.191 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:29.189+0000 7f0903fff640 1 -- 172.21.15.156:0/990312128 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f090802a9f0 con 0x7f090c0fd980 2024-07-16T09:54:29.191 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:29.190+0000 7f0913277640 1 -- 172.21.15.156:0/990312128 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f090c0fd980 msgr2=0x7f090c0fdd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:54:29.192 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:29.190+0000 7f0913277640 1 --2- 172.21.15.156:0/990312128 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f090c0fd980 0x7f090c0fdd90 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f09080048d0 tx=0x7f0908009da0 comp rx=0 tx=0).stop 2024-07-16T09:54:29.192 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:29.190+0000 7f0913277640 1 -- 172.21.15.156:0/990312128 shutdown_connections 2024-07-16T09:54:29.192 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:29.190+0000 7f0913277640 1 --2- 172.21.15.156:0/990312128 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f090c0fe2d0 0x7f090c0fa430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:54:29.192 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:29.190+0000 7f0913277640 1 --2- 172.21.15.156:0/990312128 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f090c0fd980 0x7f090c0fdd90 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:54:29.192 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:29.190+0000 7f0913277640 1 -- 172.21.15.156:0/990312128 >> 172.21.15.156:0/990312128 conn(0x7f090c0f61a0 msgr2=0x7f090c0f85b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:54:29.194 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:29.190+0000 7f0913277640 1 -- 172.21.15.156:0/990312128 shutdown_connections 2024-07-16T09:54:29.194 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:29.190+0000 7f0913277640 1 -- 172.21.15.156:0/990312128 wait complete. 2024-07-16T09:54:29.194 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:29.190+0000 7f0913277640 1 Processor -- start 2024-07-16T09:54:29.194 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:29.191+0000 7f0913277640 1 -- start start 2024-07-16T09:54:29.194 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:29.191+0000 7f0913277640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f090c0fe2d0 0x7f090c10aed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:54:29.194 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:29.191+0000 7f0913277640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f090c10b410 0x7f090c10c280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:54:29.194 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:29.191+0000 7f0913277640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f090c10b880 con 0x7f090c0fe2d0 2024-07-16T09:54:29.194 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:29.191+0000 7f0913277640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f090c10b9f0 con 0x7f090c10b410 2024-07-16T09:54:29.194 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:29.191+0000 7f0910812640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f090c10b410 0x7f090c10c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:54:29.195 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:29.191+0000 7f0910812640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f090c10b410 0x7f090c10c280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.156:51936/0 (socket says 172.21.15.156:51936) 2024-07-16T09:54:29.195 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:29.191+0000 7f0910812640 1 -- 172.21.15.156:0/2233121745 learned_addr learned my addr 172.21.15.156:0/2233121745 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T09:54:29.195 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:29.191+0000 7f0911013640 1 --2- 172.21.15.156:0/2233121745 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f090c0fe2d0 0x7f090c10aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:54:29.195 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:29.191+0000 7f0910812640 1 -- 172.21.15.156:0/2233121745 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f090c0fe2d0 msgr2=0x7f090c10aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:54:29.195 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:29.191+0000 7f0910812640 1 --2- 172.21.15.156:0/2233121745 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f090c0fe2d0 0x7f090c10aed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:54:29.195 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:29.191+0000 7f0910812640 1 -- 172.21.15.156:0/2233121745 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0908005040 con 0x7f090c10b410 2024-07-16T09:54:29.195 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:29.191+0000 7f0911013640 1 --2- 172.21.15.156:0/2233121745 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f090c0fe2d0 0x7f090c10aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-16T09:54:29.195 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:29.192+0000 7f0910812640 1 --2- 172.21.15.156:0/2233121745 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f090c10b410 0x7f090c10c280 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f08fc002f40 tx=0x7f08fc00b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:54:29.195 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:29.192+0000 7f0901ffb640 1 -- 172.21.15.156:0/2233121745 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f08fc007c30 con 0x7f090c10b410 2024-07-16T09:54:29.195 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:29.192+0000 7f0913277640 1 -- 172.21.15.156:0/2233121745 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f090c10c7c0 con 0x7f090c10b410 2024-07-16T09:54:29.195 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:29.192+0000 7f0901ffb640 1 -- 172.21.15.156:0/2233121745 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f08fc00f040 con 0x7f090c10b410 2024-07-16T09:54:29.195 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:29.192+0000 7f0901ffb640 1 -- 172.21.15.156:0/2233121745 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f08fc00ccb0 con 0x7f090c10b410 2024-07-16T09:54:29.196 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:29.192+0000 7f0913277640 1 -- 172.21.15.156:0/2233121745 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f090c10ccb0 con 0x7f090c10b410 2024-07-16T09:54:29.196 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:29.194+0000 7f0913277640 1 -- 172.21.15.156:0/2233121745 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f090c0f9e20 con 0x7f090c10b410 2024-07-16T09:54:29.196 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:29.194+0000 7f0901ffb640 1 -- 172.21.15.156:0/2233121745 <== mon.1 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 19) v1 ==== 85127+0+0 (secure 0 0 0) 0x7f08fc0090d0 con 0x7f090c10b410 2024-07-16T09:54:29.196 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:29.194+0000 7f0901ffb640 1 --2- 172.21.15.156:0/2233121745 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f08f8067260 0x7f08f8069710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:54:29.197 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:29.194+0000 7f0901ffb640 1 -- 172.21.15.156:0/2233121745 <== mon.1 v2:172.21.15.176:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f08fc084810 con 0x7f090c10b410 2024-07-16T09:54:29.197 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:29.194+0000 7f0911013640 1 --2- 172.21.15.156:0/2233121745 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f08f8067260 0x7f08f8069710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:54:29.197 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:29.195+0000 7f0911013640 1 --2- 172.21.15.156:0/2233121745 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f08f8067260 0x7f08f8069710 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f0908009040 tx=0x7f0908035040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:54:29.204 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:29.202+0000 7f0901ffb640 1 -- 172.21.15.156:0/2233121745 <== mon.1 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f08fc051a90 con 0x7f090c10b410 2024-07-16T09:54:29.343 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:54:29 smithi176 ceph-mon[36626]: from='client.24347 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-16T09:54:29.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:54:29 smithi176 ceph-mon[36626]: from='client.24351 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-16T09:54:29.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:54:29 smithi176 ceph-mon[36626]: from='client.24355 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-07-16T09:54:29.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:54:29 smithi176 ceph-mon[36626]: from='client.? 172.21.15.156:0/3534112926' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T09:54:29.401 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:54:29 smithi156 ceph-mon[29302]: from='client.24347 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-16T09:54:29.401 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:54:29 smithi156 ceph-mon[29302]: from='client.24351 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-16T09:54:29.401 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:54:29 smithi156 ceph-mon[29302]: from='client.24355 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-07-16T09:54:29.402 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:54:29 smithi156 ceph-mon[29302]: from='client.? 172.21.15.156:0/3534112926' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T09:54:29.402 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:29.399+0000 7f0913277640 1 -- 172.21.15.156:0/2233121745 --> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f090c10cf30 con 0x7f08f8067260 2024-07-16T09:54:29.403 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:29.401+0000 7f0901ffb640 1 -- 172.21.15.156:0/2233121745 <== mgr.14208 v2:172.21.15.156:6800/2615741717 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+447 (secure 0 0 0) 0x7f090c10cf30 con 0x7f08f8067260 2024-07-16T09:54:29.403 INFO:teuthology.orchestra.run.smithi156.stdout:{ 2024-07-16T09:54:29.403 INFO:teuthology.orchestra.run.smithi156.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:c15ec3aa8e6c0a9c12ee24ff79f3e7dd4d01b3d1a702dfeb7d7fff9b0811a393", 2024-07-16T09:54:29.403 INFO:teuthology.orchestra.run.smithi156.stdout: "in_progress": true, 2024-07-16T09:54:29.403 INFO:teuthology.orchestra.run.smithi156.stdout: "which": "Upgrading all daemon types on all hosts", 2024-07-16T09:54:29.403 INFO:teuthology.orchestra.run.smithi156.stdout: "services_complete": [], 2024-07-16T09:54:29.403 INFO:teuthology.orchestra.run.smithi156.stdout: "progress": "0/21 daemons upgraded", 2024-07-16T09:54:29.403 INFO:teuthology.orchestra.run.smithi156.stdout: "message": "Pulling quay.ceph.io/ceph-ci/ceph@sha256:c15ec3aa8e6c0a9c12ee24ff79f3e7dd4d01b3d1a702dfeb7d7fff9b0811a393 image on host smithi176", 2024-07-16T09:54:29.403 INFO:teuthology.orchestra.run.smithi156.stdout: "is_paused": false 2024-07-16T09:54:29.403 INFO:teuthology.orchestra.run.smithi156.stdout:} 2024-07-16T09:54:29.405 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:29.404+0000 7f0913277640 1 -- 172.21.15.156:0/2233121745 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f08f8067260 msgr2=0x7f08f8069710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:54:29.405 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:29.404+0000 7f0913277640 1 --2- 172.21.15.156:0/2233121745 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f08f8067260 0x7f08f8069710 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f0908009040 tx=0x7f0908035040 comp rx=0 tx=0).stop 2024-07-16T09:54:29.405 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:29.404+0000 7f0913277640 1 -- 172.21.15.156:0/2233121745 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f090c10b410 msgr2=0x7f090c10c280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:54:29.405 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:29.404+0000 7f0913277640 1 --2- 172.21.15.156:0/2233121745 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f090c10b410 0x7f090c10c280 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f08fc002f40 tx=0x7f08fc00b040 comp rx=0 tx=0).stop 2024-07-16T09:54:29.405 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:29.404+0000 7f0913277640 1 -- 172.21.15.156:0/2233121745 shutdown_connections 2024-07-16T09:54:29.405 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:29.404+0000 7f0913277640 1 --2- 172.21.15.156:0/2233121745 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f08f8067260 0x7f08f8069710 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:54:29.405 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:29.404+0000 7f0913277640 1 --2- 172.21.15.156:0/2233121745 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f090c10b410 0x7f090c10c280 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:54:29.406 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:29.404+0000 7f0913277640 1 --2- 172.21.15.156:0/2233121745 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f090c0fe2d0 0x7f090c10aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:54:29.406 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:29.404+0000 7f0913277640 1 -- 172.21.15.156:0/2233121745 >> 172.21.15.156:0/2233121745 conn(0x7f090c0f61a0 msgr2=0x7f090c0f7900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:54:29.406 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:29.404+0000 7f0913277640 1 -- 172.21.15.156:0/2233121745 shutdown_connections 2024-07-16T09:54:29.406 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:29.404+0000 7f0913277640 1 -- 172.21.15.156:0/2233121745 wait complete. 2024-07-16T09:54:29.489 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:29.487+0000 7f4a19a2e640 1 -- 172.21.15.156:0/913392697 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4a140fe300 msgr2=0x7f4a140fa460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:54:29.489 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:29.487+0000 7f4a19a2e640 1 --2- 172.21.15.156:0/913392697 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4a140fe300 0x7f4a140fa460 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f4a080048d0 tx=0x7f4a08009da0 comp rx=0 tx=0).stop 2024-07-16T09:54:29.489 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:29.487+0000 7f4a19a2e640 1 -- 172.21.15.156:0/913392697 shutdown_connections 2024-07-16T09:54:29.489 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:29.487+0000 7f4a19a2e640 1 --2- 172.21.15.156:0/913392697 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4a140fe300 0x7f4a140fa460 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:54:29.490 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:29.487+0000 7f4a19a2e640 1 --2- 172.21.15.156:0/913392697 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4a140fd9b0 0x7f4a140fddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:54:29.490 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:29.487+0000 7f4a19a2e640 1 -- 172.21.15.156:0/913392697 >> 172.21.15.156:0/913392697 conn(0x7f4a140f6190 msgr2=0x7f4a140f85e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:54:29.490 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:29.488+0000 7f4a19a2e640 1 -- 172.21.15.156:0/913392697 shutdown_connections 2024-07-16T09:54:29.490 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:29.488+0000 7f4a19a2e640 1 -- 172.21.15.156:0/913392697 wait complete. 2024-07-16T09:54:29.490 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:29.489+0000 7f4a19a2e640 1 Processor -- start 2024-07-16T09:54:29.490 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:29.489+0000 7f4a19a2e640 1 -- start start 2024-07-16T09:54:29.491 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:29.489+0000 7f4a19a2e640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4a140fd9b0 0x7f4a140fd1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:54:29.491 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:29.489+0000 7f4a19a2e640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4a140fb690 0x7f4a140fbb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:54:29.491 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:29.489+0000 7f4a19a2e640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a140fc040 con 0x7f4a140fb690 2024-07-16T09:54:29.491 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:29.489+0000 7f4a19a2e640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a140fc1b0 con 0x7f4a140fd9b0 2024-07-16T09:54:29.491 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:29.489+0000 7f4a127fc640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4a140fb690 0x7f4a140fbb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:54:29.491 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:29.489+0000 7f4a127fc640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4a140fb690 0x7f4a140fbb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:40340/0 (socket says 172.21.15.156:40340) 2024-07-16T09:54:29.491 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:29.489+0000 7f4a127fc640 1 -- 172.21.15.156:0/1280039231 learned_addr learned my addr 172.21.15.156:0/1280039231 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T09:54:29.492 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:29.489+0000 7f4a12ffd640 1 --2- 172.21.15.156:0/1280039231 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4a140fd9b0 0x7f4a140fd1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:54:29.492 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:29.490+0000 7f4a127fc640 1 -- 172.21.15.156:0/1280039231 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4a140fd9b0 msgr2=0x7f4a140fd1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:54:29.492 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:29.490+0000 7f4a127fc640 1 --2- 172.21.15.156:0/1280039231 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4a140fd9b0 0x7f4a140fd1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:54:29.492 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:29.490+0000 7f4a127fc640 1 -- 172.21.15.156:0/1280039231 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a08005040 con 0x7f4a140fb690 2024-07-16T09:54:29.492 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:29.490+0000 7f4a12ffd640 1 --2- 172.21.15.156:0/1280039231 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4a140fd9b0 0x7f4a140fd1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-16T09:54:29.493 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:29.491+0000 7f4a127fc640 1 --2- 172.21.15.156:0/1280039231 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4a140fb690 0x7f4a140fbb00 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f4a080048a0 tx=0x7f4a08009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:54:29.493 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:29.491+0000 7f49fb7fe640 1 -- 172.21.15.156:0/1280039231 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4a08038070 con 0x7f4a140fb690 2024-07-16T09:54:29.493 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:29.491+0000 7f49fb7fe640 1 -- 172.21.15.156:0/1280039231 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4a08032970 con 0x7f4a140fb690 2024-07-16T09:54:29.493 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:29.491+0000 7f49fb7fe640 1 -- 172.21.15.156:0/1280039231 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4a08033780 con 0x7f4a140fb690 2024-07-16T09:54:29.493 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:29.491+0000 7f4a19a2e640 1 -- 172.21.15.156:0/1280039231 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a140fc430 con 0x7f4a140fb690 2024-07-16T09:54:29.493 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:29.491+0000 7f4a19a2e640 1 -- 172.21.15.156:0/1280039231 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a14115360 con 0x7f4a140fb690 2024-07-16T09:54:29.494 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:29.493+0000 7f4a19a2e640 1 -- 172.21.15.156:0/1280039231 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a140f9e50 con 0x7f4a140fb690 2024-07-16T09:54:29.495 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:29.493+0000 7f49fb7fe640 1 -- 172.21.15.156:0/1280039231 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 19) v1 ==== 85127+0+0 (secure 0 0 0) 0x7f4a08030020 con 0x7f4a140fb690 2024-07-16T09:54:29.495 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:29.494+0000 7f49fb7fe640 1 --2- 172.21.15.156:0/1280039231 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f49fc067190 0x7f49fc069640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:54:29.495 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:29.494+0000 7f4a12ffd640 1 --2- 172.21.15.156:0/1280039231 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f49fc067190 0x7f49fc069640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:54:29.496 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:29.495+0000 7f49fb7fe640 1 -- 172.21.15.156:0/1280039231 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f4a080a4790 con 0x7f4a140fb690 2024-07-16T09:54:29.496 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:29.495+0000 7f4a12ffd640 1 --2- 172.21.15.156:0/1280039231 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f49fc067190 0x7f49fc069640 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f4a140fcf50 tx=0x7f4a0400b040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:54:29.499 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:29.498+0000 7f49fb7fe640 1 -- 172.21.15.156:0/1280039231 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f4a08071a10 con 0x7f4a140fb690 2024-07-16T09:54:29.748 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:29.746+0000 7f4a19a2e640 1 -- 172.21.15.156:0/1280039231 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f4a1404b530 con 0x7f4a140fb690 2024-07-16T09:54:29.748 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:29.747+0000 7f49fb7fe640 1 -- 172.21.15.156:0/1280039231 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7f4a08075030 con 0x7f4a140fb690 2024-07-16T09:54:29.748 INFO:teuthology.orchestra.run.smithi156.stdout:HEALTH_OK 2024-07-16T09:54:29.750 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:29.749+0000 7f4a19a2e640 1 -- 172.21.15.156:0/1280039231 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f49fc067190 msgr2=0x7f49fc069640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:54:29.750 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:29.749+0000 7f4a19a2e640 1 --2- 172.21.15.156:0/1280039231 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f49fc067190 0x7f49fc069640 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f4a140fcf50 tx=0x7f4a0400b040 comp rx=0 tx=0).stop 2024-07-16T09:54:29.751 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:29.749+0000 7f4a19a2e640 1 -- 172.21.15.156:0/1280039231 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4a140fb690 msgr2=0x7f4a140fbb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:54:29.751 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:29.749+0000 7f4a19a2e640 1 --2- 172.21.15.156:0/1280039231 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4a140fb690 0x7f4a140fbb00 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f4a080048a0 tx=0x7f4a08009da0 comp rx=0 tx=0).stop 2024-07-16T09:54:29.751 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:29.749+0000 7f4a19a2e640 1 -- 172.21.15.156:0/1280039231 shutdown_connections 2024-07-16T09:54:29.751 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:29.749+0000 7f4a19a2e640 1 --2- 172.21.15.156:0/1280039231 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f49fc067190 0x7f49fc069640 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:54:29.751 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:29.749+0000 7f4a19a2e640 1 --2- 172.21.15.156:0/1280039231 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4a140fb690 0x7f4a140fbb00 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:54:29.751 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:29.749+0000 7f4a19a2e640 1 --2- 172.21.15.156:0/1280039231 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4a140fd9b0 0x7f4a140fd1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:54:29.751 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:29.749+0000 7f4a19a2e640 1 -- 172.21.15.156:0/1280039231 >> 172.21.15.156:0/1280039231 conn(0x7f4a140f6190 msgr2=0x7f4a140f7920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:54:29.752 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:29.750+0000 7f4a19a2e640 1 -- 172.21.15.156:0/1280039231 shutdown_connections 2024-07-16T09:54:29.752 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:29.750+0000 7f4a19a2e640 1 -- 172.21.15.156:0/1280039231 wait complete. 2024-07-16T09:54:30.343 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:54:30 smithi176 ceph-mon[36626]: pgmap v216: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T09:54:30.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:54:30 smithi176 ceph-mon[36626]: from='client.? 172.21.15.156:0/511365572' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-07-16T09:54:30.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:54:30 smithi176 ceph-mon[36626]: from='client.? 172.21.15.156:0/1280039231' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-07-16T09:54:30.424 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:54:30 smithi156 ceph-mon[29302]: pgmap v216: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T09:54:30.424 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:54:30 smithi156 ceph-mon[29302]: from='client.? 172.21.15.156:0/511365572' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-07-16T09:54:30.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:54:30 smithi156 ceph-mon[29302]: from='client.? 172.21.15.156:0/1280039231' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-07-16T09:54:31.343 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:54:31 smithi176 ceph-mon[36626]: from='client.24367 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-16T09:54:31.424 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:54:31 smithi156 ceph-mon[29302]: from='client.24367 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-16T09:54:32.343 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:54:32 smithi176 ceph-mon[36626]: pgmap v217: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T09:54:32.424 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:54:32 smithi156 ceph-mon[29302]: pgmap v217: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T09:54:34.343 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:54:34 smithi176 ceph-mon[36626]: pgmap v218: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-07-16T09:54:34.424 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:54:34 smithi156 ceph-mon[29302]: pgmap v218: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-07-16T09:54:36.343 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:54:36 smithi176 ceph-mon[36626]: pgmap v219: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-07-16T09:54:36.424 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:54:36 smithi156 ceph-mon[29302]: pgmap v219: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-07-16T09:54:36.803 INFO:tasks.workunit.client.1.smithi176.stderr:Updating files: 88% (11260/12697) Updating files: 89% (11301/12697) Updating files: 90% (11428/12697) Updating files: 91% (11555/12697) Updating files: 92% (11682/12697) Updating files: 93% (11809/12697) Updating files: 94% (11936/12697) Updating files: 95% (12063/12697) Updating files: 96% (12190/12697) Updating files: 97% (12317/12697) Updating files: 98% (12444/12697) Updating files: 99% (12571/12697) Updating files: 100% (12697/12697) Updating files: 100% (12697/12697), done. 2024-07-16T09:54:37.649 INFO:tasks.workunit.client.1.smithi176.stderr:Note: switching to 'a6ca461be7cc1aff3658c2c6df4cff975a24449a'. 2024-07-16T09:54:37.650 INFO:tasks.workunit.client.1.smithi176.stderr: 2024-07-16T09:54:37.650 INFO:tasks.workunit.client.1.smithi176.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-07-16T09:54:37.650 INFO:tasks.workunit.client.1.smithi176.stderr:changes and commit them, and you can discard any commits you make in this 2024-07-16T09:54:37.650 INFO:tasks.workunit.client.1.smithi176.stderr:state without impacting any branches by switching back to a branch. 2024-07-16T09:54:37.650 INFO:tasks.workunit.client.1.smithi176.stderr: 2024-07-16T09:54:37.650 INFO:tasks.workunit.client.1.smithi176.stderr:If you want to create a new branch to retain commits you create, you may 2024-07-16T09:54:37.650 INFO:tasks.workunit.client.1.smithi176.stderr:do so (now or later) by using -c with the switch command. Example: 2024-07-16T09:54:37.650 INFO:tasks.workunit.client.1.smithi176.stderr: 2024-07-16T09:54:37.650 INFO:tasks.workunit.client.1.smithi176.stderr: git switch -c 2024-07-16T09:54:37.650 INFO:tasks.workunit.client.1.smithi176.stderr: 2024-07-16T09:54:37.650 INFO:tasks.workunit.client.1.smithi176.stderr:Or undo this operation with: 2024-07-16T09:54:37.651 INFO:tasks.workunit.client.1.smithi176.stderr: 2024-07-16T09:54:37.651 INFO:tasks.workunit.client.1.smithi176.stderr: git switch - 2024-07-16T09:54:37.651 INFO:tasks.workunit.client.1.smithi176.stderr: 2024-07-16T09:54:37.651 INFO:tasks.workunit.client.1.smithi176.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-07-16T09:54:37.651 INFO:tasks.workunit.client.1.smithi176.stderr: 2024-07-16T09:54:37.651 INFO:tasks.workunit.client.1.smithi176.stderr:HEAD is now at a6ca461be7c Merge branch 'wip-66958-squid' of https://github.com/adk3798/ceph into wip-adk4-testing-2024-07-15-1832-squid 2024-07-16T09:54:37.655 DEBUG:teuthology.orchestra.run.smithi176:> cd -- /home/ubuntu/cephtest/clone.client.1/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.1 2024-07-16T09:54:37.720 INFO:tasks.workunit.client.1.smithi176.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-07-16T09:54:37.723 INFO:tasks.workunit.client.1.smithi176.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/direct_io' 2024-07-16T09:54:37.723 INFO:tasks.workunit.client.1.smithi176.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-07-16T09:54:38.343 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:54:38 smithi176 ceph-mon[36626]: pgmap v220: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-07-16T09:54:38.408 INFO:tasks.workunit.client.1.smithi176.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-07-16T09:54:38.426 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:54:38 smithi156 ceph-mon[29302]: pgmap v220: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-07-16T09:54:38.461 INFO:tasks.workunit.client.1.smithi176.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-07-16T09:54:38.500 INFO:tasks.workunit.client.1.smithi176.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/direct_io' 2024-07-16T09:54:38.502 INFO:tasks.workunit.client.1.smithi176.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/fs' 2024-07-16T09:54:38.502 INFO:tasks.workunit.client.1.smithi176.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-07-16T09:54:38.543 INFO:tasks.workunit.client.1.smithi176.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/fs' 2024-07-16T09:54:38.546 DEBUG:teuthology.orchestra.run.smithi176:> set -ex 2024-07-16T09:54:38.546 DEBUG:teuthology.orchestra.run.smithi176:> dd if=/home/ubuntu/cephtest/workunits.list.client.1 of=/dev/stdout 2024-07-16T09:54:38.602 INFO:tasks.workunit:Running workunits matching suites/fsstress.sh on client.1... 2024-07-16T09:54:38.604 INFO:tasks.workunit:Running workunit suites/fsstress.sh... 2024-07-16T09:54:38.604 DEBUG:teuthology.orchestra.run.smithi176:workunit test suites/fsstress.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && cd -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=a6ca461be7cc1aff3658c2c6df4cff975a24449a TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="1" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.1 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.1 CEPH_MNT=/home/ubuntu/cephtest/mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.1/qa/workunits/suites/fsstress.sh 2024-07-16T09:54:38.670 INFO:tasks.workunit.client.1.smithi176.stderr:+ mkdir -p fsstress 2024-07-16T09:54:38.672 INFO:tasks.workunit.client.1.smithi176.stderr:+ pushd fsstress 2024-07-16T09:54:38.673 INFO:tasks.workunit.client.1.smithi176.stdout:~/cephtest/mnt.1/client.1/tmp/fsstress ~/cephtest/mnt.1/client.1/tmp 2024-07-16T09:54:38.673 INFO:tasks.workunit.client.1.smithi176.stderr:+ wget -q -O ltp-full.tgz http://download.ceph.com/qa/ltp-full-20091231.tgz 2024-07-16T09:54:39.225 INFO:tasks.workunit.client.1.smithi176.stderr:+ tar xzf ltp-full.tgz 2024-07-16T09:54:40.424 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:54:40 smithi156 ceph-mon[29302]: pgmap v221: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T09:54:40.424 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:54:40 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-16T09:54:40.478 INFO:tasks.workunit.client.0.smithi156.stderr:Updating files: 86% (10965/12697) Updating files: 87% (11047/12697) Updating files: 88% (11174/12697) Updating files: 89% (11301/12697) Updating files: 90% (11428/12697) Updating files: 91% (11555/12697) Updating files: 92% (11682/12697) Updating files: 93% (11809/12697) Updating files: 94% (11936/12697) Updating files: 95% (12063/12697) Updating files: 96% (12190/12697) Updating files: 97% (12317/12697) Updating files: 98% (12444/12697) Updating files: 99% (12571/12697) Updating files: 100% (12697/12697) Updating files: 100% (12697/12697), done. 2024-07-16T09:54:40.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:54:40 smithi176 ceph-mon[36626]: pgmap v221: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T09:54:40.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:54:40 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-16T09:54:41.201 INFO:tasks.workunit.client.0.smithi156.stderr:Note: switching to 'a6ca461be7cc1aff3658c2c6df4cff975a24449a'. 2024-07-16T09:54:41.201 INFO:tasks.workunit.client.0.smithi156.stderr: 2024-07-16T09:54:41.201 INFO:tasks.workunit.client.0.smithi156.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-07-16T09:54:41.201 INFO:tasks.workunit.client.0.smithi156.stderr:changes and commit them, and you can discard any commits you make in this 2024-07-16T09:54:41.201 INFO:tasks.workunit.client.0.smithi156.stderr:state without impacting any branches by switching back to a branch. 2024-07-16T09:54:41.201 INFO:tasks.workunit.client.0.smithi156.stderr: 2024-07-16T09:54:41.201 INFO:tasks.workunit.client.0.smithi156.stderr:If you want to create a new branch to retain commits you create, you may 2024-07-16T09:54:41.201 INFO:tasks.workunit.client.0.smithi156.stderr:do so (now or later) by using -c with the switch command. Example: 2024-07-16T09:54:41.201 INFO:tasks.workunit.client.0.smithi156.stderr: 2024-07-16T09:54:41.202 INFO:tasks.workunit.client.0.smithi156.stderr: git switch -c 2024-07-16T09:54:41.202 INFO:tasks.workunit.client.0.smithi156.stderr: 2024-07-16T09:54:41.202 INFO:tasks.workunit.client.0.smithi156.stderr:Or undo this operation with: 2024-07-16T09:54:41.202 INFO:tasks.workunit.client.0.smithi156.stderr: 2024-07-16T09:54:41.202 INFO:tasks.workunit.client.0.smithi156.stderr: git switch - 2024-07-16T09:54:41.202 INFO:tasks.workunit.client.0.smithi156.stderr: 2024-07-16T09:54:41.202 INFO:tasks.workunit.client.0.smithi156.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-07-16T09:54:41.202 INFO:tasks.workunit.client.0.smithi156.stderr: 2024-07-16T09:54:41.202 INFO:tasks.workunit.client.0.smithi156.stderr:HEAD is now at a6ca461be7c Merge branch 'wip-66958-squid' of https://github.com/adk3798/ceph into wip-adk4-testing-2024-07-15-1832-squid 2024-07-16T09:54:41.207 DEBUG:teuthology.orchestra.run.smithi156:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-07-16T09:54:41.232 INFO:tasks.workunit.client.0.smithi156.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-07-16T09:54:41.234 INFO:tasks.workunit.client.0.smithi156.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-07-16T09:54:41.234 INFO:tasks.workunit.client.0.smithi156.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-07-16T09:54:41.968 INFO:tasks.workunit.client.0.smithi156.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-07-16T09:54:42.022 INFO:tasks.workunit.client.0.smithi156.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-07-16T09:54:42.063 INFO:tasks.workunit.client.0.smithi156.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-07-16T09:54:42.065 INFO:tasks.workunit.client.0.smithi156.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-07-16T09:54:42.065 INFO:tasks.workunit.client.0.smithi156.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-07-16T09:54:42.106 INFO:tasks.workunit.client.0.smithi156.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-07-16T09:54:42.110 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2024-07-16T09:54:42.111 DEBUG:teuthology.orchestra.run.smithi156:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-07-16T09:54:42.167 INFO:tasks.workunit:Running workunits matching suites/fsstress.sh on client.0... 2024-07-16T09:54:42.168 INFO:tasks.workunit:Running workunit suites/fsstress.sh... 2024-07-16T09:54:42.168 DEBUG:teuthology.orchestra.run.smithi156:workunit test suites/fsstress.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=a6ca461be7cc1aff3658c2c6df4cff975a24449a TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsstress.sh 2024-07-16T09:54:42.232 INFO:tasks.workunit.client.0.smithi156.stderr:+ mkdir -p fsstress 2024-07-16T09:54:42.234 INFO:tasks.workunit.client.0.smithi156.stderr:+ pushd fsstress 2024-07-16T09:54:42.235 INFO:tasks.workunit.client.0.smithi156.stdout:~/cephtest/mnt.0/client.0/tmp/fsstress ~/cephtest/mnt.0/client.0/tmp 2024-07-16T09:54:42.235 INFO:tasks.workunit.client.0.smithi156.stderr:+ wget -q -O ltp-full.tgz http://download.ceph.com/qa/ltp-full-20091231.tgz 2024-07-16T09:54:42.424 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:54:42 smithi156 ceph-mon[29302]: pgmap v222: 65 pgs: 65 active+clean; 1.4 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 78 KiB/s wr, 16 op/s 2024-07-16T09:54:42.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:54:42 smithi176 ceph-mon[36626]: pgmap v222: 65 pgs: 65 active+clean; 1.4 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 78 KiB/s wr, 16 op/s 2024-07-16T09:54:42.862 INFO:tasks.workunit.client.0.smithi156.stderr:+ tar xzf ltp-full.tgz 2024-07-16T09:54:44.424 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:54:44 smithi156 ceph-mon[29302]: pgmap v223: 65 pgs: 65 active+clean; 8.7 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 717 KiB/s wr, 42 op/s 2024-07-16T09:54:44.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:54:44 smithi176 ceph-mon[36626]: pgmap v223: 65 pgs: 65 active+clean; 8.7 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 717 KiB/s wr, 42 op/s 2024-07-16T09:54:46.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:54:46 smithi156 ceph-mon[29302]: pgmap v224: 65 pgs: 65 active+clean; 26 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 343 KiB/s rd, 2.2 MiB/s wr, 114 op/s 2024-07-16T09:54:46.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:54:46 smithi176 ceph-mon[36626]: pgmap v224: 65 pgs: 65 active+clean; 26 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 343 KiB/s rd, 2.2 MiB/s wr, 114 op/s 2024-07-16T09:54:48.424 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:54:48 smithi156 ceph-mon[29302]: pgmap v225: 65 pgs: 65 active+clean; 32 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 342 KiB/s rd, 2.7 MiB/s wr, 170 op/s 2024-07-16T09:54:48.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:54:48 smithi176 ceph-mon[36626]: pgmap v225: 65 pgs: 65 active+clean; 32 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 342 KiB/s rd, 2.7 MiB/s wr, 170 op/s 2024-07-16T09:54:50.424 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:54:50 smithi156 ceph-mon[29302]: pgmap v226: 65 pgs: 65 active+clean; 41 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail; 684 KiB/s rd, 3.6 MiB/s wr, 290 op/s 2024-07-16T09:54:50.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:54:50 smithi176 ceph-mon[36626]: pgmap v226: 65 pgs: 65 active+clean; 41 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail; 684 KiB/s rd, 3.6 MiB/s wr, 290 op/s 2024-07-16T09:54:52.424 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:54:52 smithi156 ceph-mon[29302]: pgmap v227: 65 pgs: 65 active+clean; 53 MiB data, 1.9 GiB used, 534 GiB / 536 GiB avail; 1.3 MiB/s rd, 4.6 MiB/s wr, 392 op/s 2024-07-16T09:54:52.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:54:52 smithi176 ceph-mon[36626]: pgmap v227: 65 pgs: 65 active+clean; 53 MiB data, 1.9 GiB used, 534 GiB / 536 GiB avail; 1.3 MiB/s rd, 4.6 MiB/s wr, 392 op/s 2024-07-16T09:54:54.343 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:54:54 smithi176 ceph-mon[36626]: pgmap v228: 65 pgs: 65 active+clean; 58 MiB data, 1.9 GiB used, 534 GiB / 536 GiB avail; 1.3 MiB/s rd, 4.9 MiB/s wr, 436 op/s 2024-07-16T09:54:54.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:54:54 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:54:54.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:54:54 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi176.dqbrvx", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-07-16T09:54:54.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:54:54 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "mgr services"}]: dispatch 2024-07-16T09:54:54.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:54:54 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-16T09:54:54.424 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:54:54 smithi156 ceph-mon[29302]: pgmap v228: 65 pgs: 65 active+clean; 58 MiB data, 1.9 GiB used, 534 GiB / 536 GiB avail; 1.3 MiB/s rd, 4.9 MiB/s wr, 436 op/s 2024-07-16T09:54:54.424 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:54:54 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:54:54.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:54:54 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi176.dqbrvx", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-07-16T09:54:54.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:54:54 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "mgr services"}]: dispatch 2024-07-16T09:54:54.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:54:54 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-16T09:54:55.424 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:54:55 smithi156 ceph-mon[29302]: Upgrade: Updating mgr.smithi176.dqbrvx 2024-07-16T09:54:55.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:54:55 smithi156 ceph-mon[29302]: Deploying daemon mgr.smithi176.dqbrvx on smithi176 2024-07-16T09:54:55.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:54:55 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-16T09:54:55.485 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:54:55 smithi176 ceph-mon[36626]: Upgrade: Updating mgr.smithi176.dqbrvx 2024-07-16T09:54:55.486 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:54:55 smithi176 ceph-mon[36626]: Deploying daemon mgr.smithi176.dqbrvx on smithi176 2024-07-16T09:54:55.486 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:54:55 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-16T09:54:56.298 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:54:56 smithi176 ceph-mon[36626]: pgmap v229: 65 pgs: 65 active+clean; 71 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 1.7 MiB/s rd, 5.5 MiB/s wr, 509 op/s 2024-07-16T09:54:56.299 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:54:56 smithi176 ceph-mon[36626]: fsmap cephfs:2 {0=cephfs.smithi156.rdibys=up:active,1=cephfs.smithi176.dvvbud=up:active} 2 up:standby-replay 2024-07-16T09:54:56.424 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:54:56 smithi156 ceph-mon[29302]: pgmap v229: 65 pgs: 65 active+clean; 71 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 1.7 MiB/s rd, 5.5 MiB/s wr, 509 op/s 2024-07-16T09:54:56.424 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:54:56 smithi156 ceph-mon[29302]: fsmap cephfs:2 {0=cephfs.smithi156.rdibys=up:active,1=cephfs.smithi176.dvvbud=up:active} 2 up:standby-replay 2024-07-16T09:54:58.385 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:54:58 smithi176 ceph-mon[36626]: pgmap v230: 65 pgs: 65 active+clean; 75 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 1.3 MiB/s rd, 4.4 MiB/s wr, 498 op/s 2024-07-16T09:54:58.424 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:54:58 smithi156 ceph-mon[29302]: pgmap v230: 65 pgs: 65 active+clean; 75 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 1.3 MiB/s rd, 4.4 MiB/s wr, 498 op/s 2024-07-16T09:54:59.870 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:59.868+0000 7f6a8548a640 1 -- 172.21.15.156:0/1754835441 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6a800ff820 msgr2=0x7f6a800ffc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:54:59.871 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:59.868+0000 7f6a8548a640 1 --2- 172.21.15.156:0/1754835441 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6a800ff820 0x7f6a800ffc90 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f6a740048d0 tx=0x7f6a74009da0 comp rx=0 tx=0).stop 2024-07-16T09:54:59.871 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:59.869+0000 7f6a8548a640 1 -- 172.21.15.156:0/1754835441 shutdown_connections 2024-07-16T09:54:59.871 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:59.869+0000 7f6a8548a640 1 --2- 172.21.15.156:0/1754835441 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6a800ff820 0x7f6a800ffc90 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:54:59.871 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:59.869+0000 7f6a8548a640 1 --2- 172.21.15.156:0/1754835441 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6a800fe550 0x7f6a800fe960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:54:59.871 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:59.869+0000 7f6a8548a640 1 -- 172.21.15.156:0/1754835441 >> 172.21.15.156:0/1754835441 conn(0x7f6a800f9e70 msgr2=0x7f6a800fc250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:54:59.871 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:59.869+0000 7f6a8548a640 1 -- 172.21.15.156:0/1754835441 shutdown_connections 2024-07-16T09:54:59.871 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:59.869+0000 7f6a8548a640 1 -- 172.21.15.156:0/1754835441 wait complete. 2024-07-16T09:54:59.872 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:59.870+0000 7f6a8548a640 1 Processor -- start 2024-07-16T09:54:59.872 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:59.870+0000 7f6a8548a640 1 -- start start 2024-07-16T09:54:59.872 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:59.870+0000 7f6a8548a640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6a800fe550 0x7f6a80113510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:54:59.872 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:59.870+0000 7f6a8548a640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6a800ff820 0x7f6a80113a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:54:59.872 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:59.870+0000 7f6a8548a640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a801140a0 con 0x7f6a800fe550 2024-07-16T09:54:59.872 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:59.870+0000 7f6a8548a640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a801148c0 con 0x7f6a800ff820 2024-07-16T09:54:59.873 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:59.870+0000 7f6a7e7fc640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6a800ff820 0x7f6a80113a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:54:59.873 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:59.870+0000 7f6a7e7fc640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6a800ff820 0x7f6a80113a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.156:49638/0 (socket says 172.21.15.156:49638) 2024-07-16T09:54:59.873 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:59.870+0000 7f6a7e7fc640 1 -- 172.21.15.156:0/2196933996 learned_addr learned my addr 172.21.15.156:0/2196933996 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T09:54:59.873 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:59.870+0000 7f6a7effd640 1 --2- 172.21.15.156:0/2196933996 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6a800fe550 0x7f6a80113510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:54:59.873 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:59.871+0000 7f6a7e7fc640 1 -- 172.21.15.156:0/2196933996 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6a800fe550 msgr2=0x7f6a80113510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:54:59.873 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:59.871+0000 7f6a7e7fc640 1 --2- 172.21.15.156:0/2196933996 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6a800fe550 0x7f6a80113510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:54:59.873 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:59.871+0000 7f6a7e7fc640 1 -- 172.21.15.156:0/2196933996 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a74005040 con 0x7f6a800ff820 2024-07-16T09:54:59.873 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:59.871+0000 7f6a7e7fc640 1 --2- 172.21.15.156:0/2196933996 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6a800ff820 0x7f6a80113a50 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f6a740048a0 tx=0x7f6a74009da0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:54:59.874 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:59.871+0000 7f6a677fe640 1 -- 172.21.15.156:0/2196933996 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6a74038070 con 0x7f6a800ff820 2024-07-16T09:54:59.874 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:59.871+0000 7f6a677fe640 1 -- 172.21.15.156:0/2196933996 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6a740092b0 con 0x7f6a800ff820 2024-07-16T09:54:59.874 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:59.871+0000 7f6a677fe640 1 -- 172.21.15.156:0/2196933996 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6a74033780 con 0x7f6a800ff820 2024-07-16T09:54:59.874 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:59.871+0000 7f6a8548a640 1 -- 172.21.15.156:0/2196933996 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a80114af0 con 0x7f6a800ff820 2024-07-16T09:54:59.874 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:59.871+0000 7f6a8548a640 1 -- 172.21.15.156:0/2196933996 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a80115030 con 0x7f6a800ff820 2024-07-16T09:54:59.874 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:59.873+0000 7f6a8548a640 1 -- 172.21.15.156:0/2196933996 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a800fd9b0 con 0x7f6a800ff820 2024-07-16T09:54:59.875 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:59.874+0000 7f6a677fe640 1 -- 172.21.15.156:0/2196933996 <== mon.1 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 19) v1 ==== 85127+0+0 (secure 0 0 0) 0x7f6a74030020 con 0x7f6a800ff820 2024-07-16T09:54:59.875 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:59.874+0000 7f6a677fe640 1 --2- 172.21.15.156:0/2196933996 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f6a68067190 0x7f6a68069640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:54:59.875 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:59.874+0000 7f6a677fe640 1 -- 172.21.15.156:0/2196933996 <== mon.1 v2:172.21.15.176:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f6a740a48c0 con 0x7f6a800ff820 2024-07-16T09:54:59.876 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:59.874+0000 7f6a7effd640 1 --2- 172.21.15.156:0/2196933996 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f6a68067190 0x7f6a68069640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:54:59.876 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:59.875+0000 7f6a7effd640 1 --2- 172.21.15.156:0/2196933996 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f6a68067190 0x7f6a68069640 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f6a70009af0 tx=0x7f6a7000b040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:54:59.880 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:54:59.879+0000 7f6a677fe640 1 -- 172.21.15.156:0/2196933996 <== mon.1 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f6a74071c10 con 0x7f6a800ff820 2024-07-16T09:55:00.146 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:00.145+0000 7f6a8548a640 1 -- 172.21.15.156:0/2196933996 --> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f6a80115380 con 0x7f6a68067190 2024-07-16T09:55:00.147 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:55:00 smithi156 ceph-mon[29302]: pgmap v231: 65 pgs: 65 active+clean; 89 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 1.8 MiB/s rd, 5.1 MiB/s wr, 534 op/s 2024-07-16T09:55:00.148 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:00.147+0000 7f6a677fe640 1 -- 172.21.15.156:0/2196933996 <== mgr.14208 v2:172.21.15.156:6800/2615741717 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+349 (secure 0 0 0) 0x7f6a80115380 con 0x7f6a68067190 2024-07-16T09:55:00.151 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:00.149+0000 7f6a8548a640 1 -- 172.21.15.156:0/2196933996 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f6a68067190 msgr2=0x7f6a68069640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:55:00.151 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:00.149+0000 7f6a8548a640 1 --2- 172.21.15.156:0/2196933996 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f6a68067190 0x7f6a68069640 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f6a70009af0 tx=0x7f6a7000b040 comp rx=0 tx=0).stop 2024-07-16T09:55:00.151 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:00.149+0000 7f6a8548a640 1 -- 172.21.15.156:0/2196933996 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6a800ff820 msgr2=0x7f6a80113a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:55:00.151 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:00.149+0000 7f6a8548a640 1 --2- 172.21.15.156:0/2196933996 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6a800ff820 0x7f6a80113a50 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f6a740048a0 tx=0x7f6a74009da0 comp rx=0 tx=0).stop 2024-07-16T09:55:00.151 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:00.150+0000 7f6a8548a640 1 -- 172.21.15.156:0/2196933996 shutdown_connections 2024-07-16T09:55:00.151 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:00.150+0000 7f6a8548a640 1 --2- 172.21.15.156:0/2196933996 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f6a68067190 0x7f6a68069640 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:55:00.151 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:00.150+0000 7f6a8548a640 1 --2- 172.21.15.156:0/2196933996 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6a800ff820 0x7f6a80113a50 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:55:00.152 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:00.150+0000 7f6a8548a640 1 --2- 172.21.15.156:0/2196933996 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6a800fe550 0x7f6a80113510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:55:00.152 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:00.150+0000 7f6a8548a640 1 -- 172.21.15.156:0/2196933996 >> 172.21.15.156:0/2196933996 conn(0x7f6a800f9e70 msgr2=0x7f6a800fc410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:55:00.152 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:00.150+0000 7f6a8548a640 1 -- 172.21.15.156:0/2196933996 shutdown_connections 2024-07-16T09:55:00.152 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:00.150+0000 7f6a8548a640 1 -- 172.21.15.156:0/2196933996 wait complete. 2024-07-16T09:55:00.162 INFO:teuthology.orchestra.run.smithi156.stdout:true 2024-07-16T09:55:00.267 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:00.265+0000 7f3821fbd640 1 -- 172.21.15.156:0/983127086 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f381c0fb240 msgr2=0x7f381c0fb650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:55:00.267 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:00.265+0000 7f3821fbd640 1 --2- 172.21.15.156:0/983127086 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f381c0fb240 0x7f381c0fb650 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f380c0048d0 tx=0x7f380c009da0 comp rx=0 tx=0).stop 2024-07-16T09:55:00.268 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:00.266+0000 7f3821fbd640 1 -- 172.21.15.156:0/983127086 shutdown_connections 2024-07-16T09:55:00.268 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:00.266+0000 7f3821fbd640 1 --2- 172.21.15.156:0/983127086 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f381c0fc510 0x7f381c0fc980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:55:00.268 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:00.266+0000 7f3821fbd640 1 --2- 172.21.15.156:0/983127086 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f381c0fb240 0x7f381c0fb650 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:55:00.268 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:00.266+0000 7f3821fbd640 1 -- 172.21.15.156:0/983127086 >> 172.21.15.156:0/983127086 conn(0x7f381c0f6a20 msgr2=0x7f381c0f8e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:55:00.268 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:00.266+0000 7f3821fbd640 1 -- 172.21.15.156:0/983127086 shutdown_connections 2024-07-16T09:55:00.268 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:00.266+0000 7f3821fbd640 1 -- 172.21.15.156:0/983127086 wait complete. 2024-07-16T09:55:00.269 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:00.267+0000 7f3821fbd640 1 Processor -- start 2024-07-16T09:55:00.269 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:00.267+0000 7f3821fbd640 1 -- start start 2024-07-16T09:55:00.269 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:00.267+0000 7f3821fbd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f381c0fb240 0x7f381c10efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:55:00.269 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:00.267+0000 7f3821fbd640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f381c0fc510 0x7f381c10f510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:55:00.270 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:00.268+0000 7f3821fbd640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f381c110380 con 0x7f381c0fb240 2024-07-16T09:55:00.270 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:00.268+0000 7f3821fbd640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f381c1104f0 con 0x7f381c0fc510 2024-07-16T09:55:00.270 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:00.268+0000 7f381b7fe640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f381c0fb240 0x7f381c10efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:55:00.270 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:00.268+0000 7f381b7fe640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f381c0fb240 0x7f381c10efd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:49022/0 (socket says 172.21.15.156:49022) 2024-07-16T09:55:00.270 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:00.268+0000 7f381b7fe640 1 -- 172.21.15.156:0/663310838 learned_addr learned my addr 172.21.15.156:0/663310838 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T09:55:00.271 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:00.268+0000 7f381b7fe640 1 -- 172.21.15.156:0/663310838 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f381c0fc510 msgr2=0x7f381c10f510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:55:00.271 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:00.268+0000 7f381b7fe640 1 --2- 172.21.15.156:0/663310838 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f381c0fc510 0x7f381c10f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:55:00.271 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:00.268+0000 7f381b7fe640 1 -- 172.21.15.156:0/663310838 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f380c005040 con 0x7f381c0fb240 2024-07-16T09:55:00.271 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:00.269+0000 7f381b7fe640 1 --2- 172.21.15.156:0/663310838 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f381c0fb240 0x7f381c10efd0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f380c0048a0 tx=0x7f380c009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:55:00.271 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:00.269+0000 7f3803fff640 1 -- 172.21.15.156:0/663310838 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f380c038070 con 0x7f381c0fb240 2024-07-16T09:55:00.271 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:00.269+0000 7f3821fbd640 1 -- 172.21.15.156:0/663310838 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f381c110770 con 0x7f381c0fb240 2024-07-16T09:55:00.271 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:00.269+0000 7f3803fff640 1 -- 172.21.15.156:0/663310838 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f380c032b80 con 0x7f381c0fb240 2024-07-16T09:55:00.271 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:00.269+0000 7f3803fff640 1 -- 172.21.15.156:0/663310838 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f380c033a90 con 0x7f381c0fb240 2024-07-16T09:55:00.272 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:00.269+0000 7f3821fbd640 1 -- 172.21.15.156:0/663310838 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f381c110c30 con 0x7f381c0fb240 2024-07-16T09:55:00.272 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:00.270+0000 7f3821fbd640 1 -- 172.21.15.156:0/663310838 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f381c0fa6a0 con 0x7f381c0fb240 2024-07-16T09:55:00.273 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:00.272+0000 7f3803fff640 1 -- 172.21.15.156:0/663310838 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 19) v1 ==== 85127+0+0 (secure 0 0 0) 0x7f380c030020 con 0x7f381c0fb240 2024-07-16T09:55:00.273 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:00.272+0000 7f3803fff640 1 --2- 172.21.15.156:0/663310838 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f3804067230 0x7f38040696e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:55:00.274 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:00.272+0000 7f3803fff640 1 -- 172.21.15.156:0/663310838 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f380c0a5140 con 0x7f381c0fb240 2024-07-16T09:55:00.274 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:00.272+0000 7f381affd640 1 --2- 172.21.15.156:0/663310838 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f3804067230 0x7f38040696e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:55:00.274 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:00.273+0000 7f381affd640 1 --2- 172.21.15.156:0/663310838 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f3804067230 0x7f38040696e0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f381c10ed80 tx=0x7f3810006c50 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:55:00.281 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:00.280+0000 7f3803fff640 1 -- 172.21.15.156:0/663310838 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f380c072340 con 0x7f381c0fb240 2024-07-16T09:55:00.548 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:00.546+0000 7f3821fbd640 1 -- 172.21.15.156:0/663310838 --> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f381c110f60 con 0x7f3804067230 2024-07-16T09:55:00.549 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:00.548+0000 7f3803fff640 1 -- 172.21.15.156:0/663310838 <== mgr.14208 v2:172.21.15.156:6800/2615741717 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+349 (secure 0 0 0) 0x7f381c110f60 con 0x7f3804067230 2024-07-16T09:55:00.552 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:00.550+0000 7f3821fbd640 1 -- 172.21.15.156:0/663310838 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f3804067230 msgr2=0x7f38040696e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:55:00.552 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:00.550+0000 7f3821fbd640 1 --2- 172.21.15.156:0/663310838 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f3804067230 0x7f38040696e0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f381c10ed80 tx=0x7f3810006c50 comp rx=0 tx=0).stop 2024-07-16T09:55:00.552 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:00.550+0000 7f3821fbd640 1 -- 172.21.15.156:0/663310838 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f381c0fb240 msgr2=0x7f381c10efd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:55:00.552 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:00.550+0000 7f3821fbd640 1 --2- 172.21.15.156:0/663310838 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f381c0fb240 0x7f381c10efd0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f380c0048a0 tx=0x7f380c009da0 comp rx=0 tx=0).stop 2024-07-16T09:55:00.552 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:00.551+0000 7f3821fbd640 1 -- 172.21.15.156:0/663310838 shutdown_connections 2024-07-16T09:55:00.553 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:00.551+0000 7f3821fbd640 1 --2- 172.21.15.156:0/663310838 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f3804067230 0x7f38040696e0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:55:00.553 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:00.551+0000 7f3821fbd640 1 --2- 172.21.15.156:0/663310838 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f381c0fc510 0x7f381c10f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:55:00.553 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:00.551+0000 7f3821fbd640 1 --2- 172.21.15.156:0/663310838 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f381c0fb240 0x7f381c10efd0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:55:00.553 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:00.551+0000 7f3821fbd640 1 -- 172.21.15.156:0/663310838 >> 172.21.15.156:0/663310838 conn(0x7f381c0f6a20 msgr2=0x7f381c0f80e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:55:00.553 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:00.551+0000 7f3821fbd640 1 -- 172.21.15.156:0/663310838 shutdown_connections 2024-07-16T09:55:00.553 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:00.551+0000 7f3821fbd640 1 -- 172.21.15.156:0/663310838 wait complete. 2024-07-16T09:55:00.577 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:55:00 smithi176 ceph-mon[36626]: pgmap v231: 65 pgs: 65 active+clean; 89 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 1.8 MiB/s rd, 5.1 MiB/s wr, 534 op/s 2024-07-16T09:55:00.666 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:00.664+0000 7fc530563640 1 -- 172.21.15.156:0/1573333348 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc5280ff490 msgr2=0x7fc528101870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:55:00.667 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:00.664+0000 7fc530563640 1 --2- 172.21.15.156:0/1573333348 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc5280ff490 0x7fc528101870 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fc5180048d0 tx=0x7fc518009da0 comp rx=0 tx=0).stop 2024-07-16T09:55:00.667 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:00.665+0000 7fc530563640 1 -- 172.21.15.156:0/1573333348 shutdown_connections 2024-07-16T09:55:00.667 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:00.665+0000 7fc530563640 1 --2- 172.21.15.156:0/1573333348 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc5280ff490 0x7fc528101870 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:55:00.667 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:00.665+0000 7fc530563640 1 --2- 172.21.15.156:0/1573333348 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc528065a50 0x7fc5280fef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:55:00.667 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:00.665+0000 7fc530563640 1 -- 172.21.15.156:0/1573333348 >> 172.21.15.156:0/1573333348 conn(0x7fc5280f6a20 msgr2=0x7fc5280f8e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:55:00.667 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:00.665+0000 7fc530563640 1 -- 172.21.15.156:0/1573333348 shutdown_connections 2024-07-16T09:55:00.667 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:00.665+0000 7fc530563640 1 -- 172.21.15.156:0/1573333348 wait complete. 2024-07-16T09:55:00.667 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:00.665+0000 7fc530563640 1 Processor -- start 2024-07-16T09:55:00.667 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:00.665+0000 7fc530563640 1 -- start start 2024-07-16T09:55:00.668 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:00.666+0000 7fc530563640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc528065a50 0x7fc52810cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:55:00.669 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:00.666+0000 7fc530563640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc5280ff490 0x7fc52810d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:55:00.669 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:00.666+0000 7fc530563640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc52810db20 con 0x7fc528065a50 2024-07-16T09:55:00.669 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:00.666+0000 7fc530563640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc52810e2f0 con 0x7fc5280ff490 2024-07-16T09:55:00.669 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:00.666+0000 7fc52e2ff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc528065a50 0x7fc52810cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:55:00.669 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:00.666+0000 7fc52e2ff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc528065a50 0x7fc52810cf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:49046/0 (socket says 172.21.15.156:49046) 2024-07-16T09:55:00.669 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:00.666+0000 7fc52e2ff640 1 -- 172.21.15.156:0/1101360934 learned_addr learned my addr 172.21.15.156:0/1101360934 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T09:55:00.669 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:00.666+0000 7fc52dafe640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc5280ff490 0x7fc52810d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:55:00.669 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:00.666+0000 7fc52e2ff640 1 -- 172.21.15.156:0/1101360934 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc5280ff490 msgr2=0x7fc52810d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:55:00.669 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:00.666+0000 7fc52e2ff640 1 --2- 172.21.15.156:0/1101360934 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc5280ff490 0x7fc52810d4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:55:00.669 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:00.666+0000 7fc52e2ff640 1 -- 172.21.15.156:0/1101360934 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc518005040 con 0x7fc528065a50 2024-07-16T09:55:00.669 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:00.667+0000 7fc52dafe640 1 --2- 172.21.15.156:0/1101360934 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc5280ff490 0x7fc52810d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-16T09:55:00.670 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:00.667+0000 7fc52e2ff640 1 --2- 172.21.15.156:0/1101360934 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc528065a50 0x7fc52810cf90 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fc52400eb70 tx=0x7fc52400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:55:00.670 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:00.667+0000 7fc51effd640 1 -- 172.21.15.156:0/1101360934 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc524007c30 con 0x7fc528065a50 2024-07-16T09:55:00.670 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:00.667+0000 7fc530563640 1 -- 172.21.15.156:0/1101360934 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc52810e580 con 0x7fc528065a50 2024-07-16T09:55:00.670 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:00.667+0000 7fc530563640 1 -- 172.21.15.156:0/1101360934 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc52810eaf0 con 0x7fc528065a50 2024-07-16T09:55:00.670 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:00.667+0000 7fc51effd640 1 -- 172.21.15.156:0/1101360934 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc52400f040 con 0x7fc528065a50 2024-07-16T09:55:00.670 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:00.668+0000 7fc51effd640 1 -- 172.21.15.156:0/1101360934 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc524009510 con 0x7fc528065a50 2024-07-16T09:55:00.671 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:00.669+0000 7fc530563640 1 -- 172.21.15.156:0/1101360934 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc50c005240 con 0x7fc528065a50 2024-07-16T09:55:00.671 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:00.670+0000 7fc51effd640 1 -- 172.21.15.156:0/1101360934 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 19) v1 ==== 85127+0+0 (secure 0 0 0) 0x7fc524009670 con 0x7fc528065a50 2024-07-16T09:55:00.672 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:00.670+0000 7fc51effd640 1 --2- 172.21.15.156:0/1101360934 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7fc514067190 0x7fc514069640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:55:00.672 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:00.671+0000 7fc51effd640 1 -- 172.21.15.156:0/1101360934 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5464+0+0 (secure 0 0 0) 0x7fc524084bc0 con 0x7fc528065a50 2024-07-16T09:55:00.672 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:00.671+0000 7fc52dafe640 1 --2- 172.21.15.156:0/1101360934 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7fc514067190 0x7fc514069640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:55:00.672 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:00.671+0000 7fc52dafe640 1 --2- 172.21.15.156:0/1101360934 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7fc514067190 0x7fc514069640 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fc518009f80 tx=0x7fc518035040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:55:00.678 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:00.676+0000 7fc51effd640 1 -- 172.21.15.156:0/1101360934 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fc524051dc0 con 0x7fc528065a50 2024-07-16T09:55:00.943 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:00.941+0000 7fc530563640 1 -- 172.21.15.156:0/1101360934 --> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fc50c000bf0 con 0x7fc514067190 2024-07-16T09:55:00.955 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:00.954+0000 7fc51effd640 1 -- 172.21.15.156:0/1101360934 <== mgr.14208 v2:172.21.15.156:6800/2615741717 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3432 (secure 0 0 0) 0x7fc50c000bf0 con 0x7fc514067190 2024-07-16T09:55:00.956 INFO:teuthology.orchestra.run.smithi156.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-07-16T09:55:00.956 INFO:teuthology.orchestra.run.smithi156.stdout:alertmanager.smithi156 smithi156 *:9093,9094 running (5m) 99s ago 7m 24.6M - 0.25.0 c8568f914cd2 83419a2a37f4 2024-07-16T09:55:00.956 INFO:teuthology.orchestra.run.smithi156.stdout:crash.smithi156 smithi156 running (7m) 99s ago 7m 6899k - 17.2.7-1104-ge6231b04 d5de5f953b95 bf6d49ea06b4 2024-07-16T09:55:00.956 INFO:teuthology.orchestra.run.smithi156.stdout:crash.smithi176 smithi176 running (6m) 104s ago 6m 6903k - 17.2.7-1104-ge6231b04 d5de5f953b95 907c9fa1e890 2024-07-16T09:55:00.956 INFO:teuthology.orchestra.run.smithi156.stdout:grafana.smithi156 smithi156 *:3000 running (5m) 99s ago 6m 85.0M - 9.4.7 954c08fa6188 184256cf055d 2024-07-16T09:55:00.956 INFO:teuthology.orchestra.run.smithi156.stdout:mds.cephfs.smithi156.evwwaz smithi156 running (117s) 99s ago 116s 14.9M - 17.2.7-1104-ge6231b04 d5de5f953b95 7c0ff9ae43f8 2024-07-16T09:55:00.956 INFO:teuthology.orchestra.run.smithi156.stdout:mds.cephfs.smithi156.rdibys smithi156 running (2m) 99s ago 2m 15.8M - 17.2.7-1104-ge6231b04 d5de5f953b95 b985cd1e6924 2024-07-16T09:55:00.956 INFO:teuthology.orchestra.run.smithi156.stdout:mds.cephfs.smithi176.dvvbud smithi176 running (119s) 104s ago 119s 14.6M - 17.2.7-1104-ge6231b04 d5de5f953b95 1b00147950f9 2024-07-16T09:55:00.956 INFO:teuthology.orchestra.run.smithi156.stdout:mds.cephfs.smithi176.wahbsi smithi176 running (114s) 104s ago 114s 14.7M - 17.2.7-1104-ge6231b04 d5de5f953b95 866664af2311 2024-07-16T09:55:00.956 INFO:teuthology.orchestra.run.smithi156.stdout:mgr.smithi156.zkvylv smithi156 *:9283 running (8m) 99s ago 8m 450M - 17.2.7-1104-ge6231b04 d5de5f953b95 a6937c84750a 2024-07-16T09:55:00.956 INFO:teuthology.orchestra.run.smithi156.stdout:mgr.smithi176.dqbrvx smithi176 *:8443,9283 running (6m) 104s ago 6m 402M - 17.2.7-1104-ge6231b04 d5de5f953b95 654c9bc9c82a 2024-07-16T09:55:00.957 INFO:teuthology.orchestra.run.smithi156.stdout:mon.smithi156 smithi156 running (8m) 99s ago 8m 47.6M 2048M 17.2.7-1104-ge6231b04 d5de5f953b95 6471c8acff40 2024-07-16T09:55:00.957 INFO:teuthology.orchestra.run.smithi156.stdout:mon.smithi176 smithi176 running (6m) 104s ago 6m 45.1M 2048M 17.2.7-1104-ge6231b04 d5de5f953b95 3a293ac6ef01 2024-07-16T09:55:00.957 INFO:teuthology.orchestra.run.smithi156.stdout:node-exporter.smithi156 smithi156 *:9100 running (7m) 99s ago 7m 15.9M - 1.5.0 0da6a335fe13 2c136840a7a9 2024-07-16T09:55:00.957 INFO:teuthology.orchestra.run.smithi156.stdout:node-exporter.smithi176 smithi176 *:9100 running (6m) 104s ago 6m 15.6M - 1.5.0 0da6a335fe13 536716db6afe 2024-07-16T09:55:00.957 INFO:teuthology.orchestra.run.smithi156.stdout:osd.0 smithi156 running (5m) 99s ago 5m 48.5M 4288M 17.2.7-1104-ge6231b04 d5de5f953b95 0471758ec349 2024-07-16T09:55:00.957 INFO:teuthology.orchestra.run.smithi156.stdout:osd.1 smithi156 running (5m) 99s ago 5m 49.0M 4288M 17.2.7-1104-ge6231b04 d5de5f953b95 149a0bda2f61 2024-07-16T09:55:00.957 INFO:teuthology.orchestra.run.smithi156.stdout:osd.2 smithi156 running (4m) 99s ago 4m 44.5M 4288M 17.2.7-1104-ge6231b04 d5de5f953b95 f28acec5cb79 2024-07-16T09:55:00.957 INFO:teuthology.orchestra.run.smithi156.stdout:osd.3 smithi176 running (4m) 104s ago 4m 44.8M 5312M 17.2.7-1104-ge6231b04 d5de5f953b95 733861ba84dc 2024-07-16T09:55:00.957 INFO:teuthology.orchestra.run.smithi156.stdout:osd.4 smithi176 running (3m) 104s ago 3m 41.3M 5312M 17.2.7-1104-ge6231b04 d5de5f953b95 ef0c67cec086 2024-07-16T09:55:00.957 INFO:teuthology.orchestra.run.smithi156.stdout:osd.5 smithi176 running (3m) 104s ago 3m 42.4M 5312M 17.2.7-1104-ge6231b04 d5de5f953b95 996e15448a52 2024-07-16T09:55:00.957 INFO:teuthology.orchestra.run.smithi156.stdout:prometheus.smithi156 smithi156 *:9095 running (5m) 99s ago 6m 55.5M - 2.43.0 a07b618ecd1d 1655b4227cd0 2024-07-16T09:55:00.958 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:00.957+0000 7fc530563640 1 -- 172.21.15.156:0/1101360934 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7fc514067190 msgr2=0x7fc514069640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:55:00.958 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:00.957+0000 7fc530563640 1 --2- 172.21.15.156:0/1101360934 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7fc514067190 0x7fc514069640 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fc518009f80 tx=0x7fc518035040 comp rx=0 tx=0).stop 2024-07-16T09:55:00.958 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:00.957+0000 7fc530563640 1 -- 172.21.15.156:0/1101360934 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc528065a50 msgr2=0x7fc52810cf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:55:00.958 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:00.957+0000 7fc530563640 1 --2- 172.21.15.156:0/1101360934 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc528065a50 0x7fc52810cf90 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fc52400eb70 tx=0x7fc52400b040 comp rx=0 tx=0).stop 2024-07-16T09:55:00.958 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:00.957+0000 7fc530563640 1 -- 172.21.15.156:0/1101360934 shutdown_connections 2024-07-16T09:55:00.959 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:00.957+0000 7fc530563640 1 --2- 172.21.15.156:0/1101360934 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7fc514067190 0x7fc514069640 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:55:00.959 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:00.957+0000 7fc530563640 1 --2- 172.21.15.156:0/1101360934 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc5280ff490 0x7fc52810d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:55:00.959 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:00.957+0000 7fc530563640 1 --2- 172.21.15.156:0/1101360934 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc528065a50 0x7fc52810cf90 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:55:00.959 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:00.957+0000 7fc530563640 1 -- 172.21.15.156:0/1101360934 >> 172.21.15.156:0/1101360934 conn(0x7fc5280f6a20 msgr2=0x7fc5280f6df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:55:00.959 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:00.957+0000 7fc530563640 1 -- 172.21.15.156:0/1101360934 shutdown_connections 2024-07-16T09:55:00.959 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:00.957+0000 7fc530563640 1 -- 172.21.15.156:0/1101360934 wait complete. 2024-07-16T09:55:01.053 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:01.051+0000 7f134af4a640 1 -- 172.21.15.156:0/3359084600 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f13440fc330 msgr2=0x7f13440fc740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:55:01.053 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:01.051+0000 7f134af4a640 1 --2- 172.21.15.156:0/3359084600 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f13440fc330 0x7f13440fc740 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f133c0048d0 tx=0x7f133c009da0 comp rx=0 tx=0).stop 2024-07-16T09:55:01.053 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:01.052+0000 7f134af4a640 1 -- 172.21.15.156:0/3359084600 shutdown_connections 2024-07-16T09:55:01.053 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:01.052+0000 7f134af4a640 1 --2- 172.21.15.156:0/3359084600 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f13440fd600 0x7f13440fda70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:55:01.053 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:01.052+0000 7f134af4a640 1 --2- 172.21.15.156:0/3359084600 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f13440fc330 0x7f13440fc740 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:55:01.053 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:01.052+0000 7f134af4a640 1 -- 172.21.15.156:0/3359084600 >> 172.21.15.156:0/3359084600 conn(0x7f13440f7af0 msgr2=0x7f13440f9f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:55:01.053 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:01.052+0000 7f134af4a640 1 -- 172.21.15.156:0/3359084600 shutdown_connections 2024-07-16T09:55:01.054 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:01.052+0000 7f134af4a640 1 -- 172.21.15.156:0/3359084600 wait complete. 2024-07-16T09:55:01.055 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:01.052+0000 7f134af4a640 1 Processor -- start 2024-07-16T09:55:01.055 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:01.052+0000 7f134af4a640 1 -- start start 2024-07-16T09:55:01.055 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:01.053+0000 7f134af4a640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f13440fc330 0x7f1344071380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:55:01.056 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:01.053+0000 7f134af4a640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f13440fd600 0x7f134406f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:55:01.056 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:01.053+0000 7f134af4a640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f134406fe00 con 0x7f13440fc330 2024-07-16T09:55:01.056 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:01.053+0000 7f134af4a640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f134406ff40 con 0x7f13440fd600 2024-07-16T09:55:01.056 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:01.053+0000 7f1348ce6640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f13440fc330 0x7f1344071380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:55:01.056 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:01.053+0000 7f1348ce6640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f13440fc330 0x7f1344071380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:49068/0 (socket says 172.21.15.156:49068) 2024-07-16T09:55:01.056 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:01.053+0000 7f1348ce6640 1 -- 172.21.15.156:0/2473290775 learned_addr learned my addr 172.21.15.156:0/2473290775 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T09:55:01.056 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:01.053+0000 7f1343fff640 1 --2- 172.21.15.156:0/2473290775 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f13440fd600 0x7f134406f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:55:01.056 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:01.053+0000 7f1348ce6640 1 -- 172.21.15.156:0/2473290775 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f13440fd600 msgr2=0x7f134406f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:55:01.056 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:01.053+0000 7f1348ce6640 1 --2- 172.21.15.156:0/2473290775 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f13440fd600 0x7f134406f870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:55:01.056 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:01.053+0000 7f1348ce6640 1 -- 172.21.15.156:0/2473290775 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f133c005040 con 0x7f13440fc330 2024-07-16T09:55:01.056 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:01.053+0000 7f1348ce6640 1 --2- 172.21.15.156:0/2473290775 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f13440fc330 0x7f1344071380 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f133c0048a0 tx=0x7f133c009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:55:01.056 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:01.053+0000 7f13417fa640 1 -- 172.21.15.156:0/2473290775 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f133c038070 con 0x7f13440fc330 2024-07-16T09:55:01.057 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:01.053+0000 7f13417fa640 1 -- 172.21.15.156:0/2473290775 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f133c0092b0 con 0x7f13440fc330 2024-07-16T09:55:01.057 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:01.053+0000 7f134af4a640 1 -- 172.21.15.156:0/2473290775 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13440701c0 con 0x7f13440fc330 2024-07-16T09:55:01.057 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:01.053+0000 7f13417fa640 1 -- 172.21.15.156:0/2473290775 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f133c033780 con 0x7f13440fc330 2024-07-16T09:55:01.057 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:01.053+0000 7f134af4a640 1 -- 172.21.15.156:0/2473290775 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1344114fb0 con 0x7f13440fc330 2024-07-16T09:55:01.058 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:01.054+0000 7f134af4a640 1 -- 172.21.15.156:0/2473290775 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13440fb790 con 0x7f13440fc330 2024-07-16T09:55:01.058 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:01.055+0000 7f13417fa640 1 -- 172.21.15.156:0/2473290775 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 19) v1 ==== 85127+0+0 (secure 0 0 0) 0x7f133c030020 con 0x7f13440fc330 2024-07-16T09:55:01.058 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:01.055+0000 7f13417fa640 1 --2- 172.21.15.156:0/2473290775 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f1330067190 0x7f1330069640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:55:01.058 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:01.055+0000 7f13417fa640 1 -- 172.21.15.156:0/2473290775 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f133c0a48f0 con 0x7f13440fc330 2024-07-16T09:55:01.058 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:01.055+0000 7f1343fff640 1 --2- 172.21.15.156:0/2473290775 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f1330067190 0x7f1330069640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:55:01.058 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:01.056+0000 7f1343fff640 1 --2- 172.21.15.156:0/2473290775 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f1330067190 0x7f1330069640 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f1334004fd0 tx=0x7f133400b040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:55:01.060 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:01.059+0000 7f13417fa640 1 -- 172.21.15.156:0/2473290775 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f133c071c40 con 0x7f13440fc330 2024-07-16T09:55:01.324 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:55:01 smithi176 ceph-mon[36626]: from='client.24373 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-16T09:55:01.374 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:01.373+0000 7f134af4a640 1 -- 172.21.15.156:0/2473290775 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f1344115140 con 0x7f13440fc330 2024-07-16T09:55:01.375 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:55:01 smithi156 ceph-mon[29302]: from='client.24373 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-16T09:55:01.375 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:01.374+0000 7f13417fa640 1 -- 172.21.15.156:0/2473290775 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+643 (secure 0 0 0) 0x7f133c0412a0 con 0x7f13440fc330 2024-07-16T09:55:01.375 INFO:teuthology.orchestra.run.smithi156.stdout:{ 2024-07-16T09:55:01.375 INFO:teuthology.orchestra.run.smithi156.stdout: "mon": { 2024-07-16T09:55:01.376 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph version 17.2.7-1104-ge6231b04 (e6231b04b52ebf3b9b31803282dd3074439e51ac) quincy (stable)": 2 2024-07-16T09:55:01.376 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-07-16T09:55:01.376 INFO:teuthology.orchestra.run.smithi156.stdout: "mgr": { 2024-07-16T09:55:01.376 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph version 17.2.7-1104-ge6231b04 (e6231b04b52ebf3b9b31803282dd3074439e51ac) quincy (stable)": 2 2024-07-16T09:55:01.376 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-07-16T09:55:01.376 INFO:teuthology.orchestra.run.smithi156.stdout: "osd": { 2024-07-16T09:55:01.376 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph version 17.2.7-1104-ge6231b04 (e6231b04b52ebf3b9b31803282dd3074439e51ac) quincy (stable)": 6 2024-07-16T09:55:01.376 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-07-16T09:55:01.376 INFO:teuthology.orchestra.run.smithi156.stdout: "mds": { 2024-07-16T09:55:01.376 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph version 17.2.7-1104-ge6231b04 (e6231b04b52ebf3b9b31803282dd3074439e51ac) quincy (stable)": 4 2024-07-16T09:55:01.376 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-07-16T09:55:01.376 INFO:teuthology.orchestra.run.smithi156.stdout: "overall": { 2024-07-16T09:55:01.377 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph version 17.2.7-1104-ge6231b04 (e6231b04b52ebf3b9b31803282dd3074439e51ac) quincy (stable)": 14 2024-07-16T09:55:01.377 INFO:teuthology.orchestra.run.smithi156.stdout: } 2024-07-16T09:55:01.377 INFO:teuthology.orchestra.run.smithi156.stdout:} 2024-07-16T09:55:01.378 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:01.376+0000 7f134af4a640 1 -- 172.21.15.156:0/2473290775 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f1330067190 msgr2=0x7f1330069640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:55:01.378 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:01.376+0000 7f134af4a640 1 --2- 172.21.15.156:0/2473290775 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f1330067190 0x7f1330069640 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f1334004fd0 tx=0x7f133400b040 comp rx=0 tx=0).stop 2024-07-16T09:55:01.378 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:01.376+0000 7f134af4a640 1 -- 172.21.15.156:0/2473290775 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f13440fc330 msgr2=0x7f1344071380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:55:01.378 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:01.376+0000 7f134af4a640 1 --2- 172.21.15.156:0/2473290775 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f13440fc330 0x7f1344071380 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f133c0048a0 tx=0x7f133c009da0 comp rx=0 tx=0).stop 2024-07-16T09:55:01.378 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:01.376+0000 7f134af4a640 1 -- 172.21.15.156:0/2473290775 shutdown_connections 2024-07-16T09:55:01.378 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:01.376+0000 7f134af4a640 1 --2- 172.21.15.156:0/2473290775 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f1330067190 0x7f1330069640 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:55:01.378 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:01.376+0000 7f134af4a640 1 --2- 172.21.15.156:0/2473290775 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f13440fd600 0x7f134406f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:55:01.378 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:01.376+0000 7f134af4a640 1 --2- 172.21.15.156:0/2473290775 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f13440fc330 0x7f1344071380 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:55:01.378 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:01.376+0000 7f134af4a640 1 -- 172.21.15.156:0/2473290775 >> 172.21.15.156:0/2473290775 conn(0x7f13440f7af0 msgr2=0x7f13440f91d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:55:01.378 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:01.377+0000 7f134af4a640 1 -- 172.21.15.156:0/2473290775 shutdown_connections 2024-07-16T09:55:01.379 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:01.377+0000 7f134af4a640 1 -- 172.21.15.156:0/2473290775 wait complete. 2024-07-16T09:55:01.478 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:01.477+0000 7fb8e2e2b640 1 -- 172.21.15.156:0/1781955683 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb8dc0fb640 msgr2=0x7fb8dc0fba50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:55:01.479 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:01.477+0000 7fb8e2e2b640 1 --2- 172.21.15.156:0/1781955683 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb8dc0fb640 0x7fb8dc0fba50 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fb8d00048d0 tx=0x7fb8d0009da0 comp rx=0 tx=0).stop 2024-07-16T09:55:01.479 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:01.477+0000 7fb8e2e2b640 1 -- 172.21.15.156:0/1781955683 shutdown_connections 2024-07-16T09:55:01.479 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:01.477+0000 7fb8e2e2b640 1 --2- 172.21.15.156:0/1781955683 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb8dc0fc020 0x7fb8dc0fa130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:55:01.479 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:01.477+0000 7fb8e2e2b640 1 --2- 172.21.15.156:0/1781955683 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb8dc0fb640 0x7fb8dc0fba50 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:55:01.479 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:01.477+0000 7fb8e2e2b640 1 -- 172.21.15.156:0/1781955683 >> 172.21.15.156:0/1781955683 conn(0x7fb8dc0f5d30 msgr2=0x7fb8dc0f8160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:55:01.479 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:01.477+0000 7fb8e2e2b640 1 -- 172.21.15.156:0/1781955683 shutdown_connections 2024-07-16T09:55:01.479 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:01.477+0000 7fb8e2e2b640 1 -- 172.21.15.156:0/1781955683 wait complete. 2024-07-16T09:55:01.479 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:01.477+0000 7fb8e2e2b640 1 Processor -- start 2024-07-16T09:55:01.479 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:01.477+0000 7fb8e2e2b640 1 -- start start 2024-07-16T09:55:01.480 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:01.478+0000 7fb8e2e2b640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb8dc0fb640 0x7fb8dc10f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:55:01.480 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:01.478+0000 7fb8e2e2b640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb8dc0fc020 0x7fb8dc10f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:55:01.480 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:01.478+0000 7fb8e2e2b640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8dc10fd80 con 0x7fb8dc0fb640 2024-07-16T09:55:01.481 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:01.478+0000 7fb8e2e2b640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8dc1105a0 con 0x7fb8dc0fc020 2024-07-16T09:55:01.481 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:01.478+0000 7fb8e0bc7640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb8dc0fb640 0x7fb8dc10f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:55:01.481 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:01.478+0000 7fb8dbfff640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb8dc0fc020 0x7fb8dc10f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:55:01.481 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:01.478+0000 7fb8e0bc7640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb8dc0fb640 0x7fb8dc10f1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:49084/0 (socket says 172.21.15.156:49084) 2024-07-16T09:55:01.481 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:01.478+0000 7fb8dbfff640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb8dc0fc020 0x7fb8dc10f730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.156:49710/0 (socket says 172.21.15.156:49710) 2024-07-16T09:55:01.481 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:01.478+0000 7fb8e0bc7640 1 -- 172.21.15.156:0/485807471 learned_addr learned my addr 172.21.15.156:0/485807471 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T09:55:01.481 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:01.478+0000 7fb8dbfff640 1 -- 172.21.15.156:0/485807471 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb8dc0fb640 msgr2=0x7fb8dc10f1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:55:01.481 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:01.478+0000 7fb8dbfff640 1 --2- 172.21.15.156:0/485807471 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb8dc0fb640 0x7fb8dc10f1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:55:01.481 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:01.478+0000 7fb8dbfff640 1 -- 172.21.15.156:0/485807471 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8d0005040 con 0x7fb8dc0fc020 2024-07-16T09:55:01.481 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:01.478+0000 7fb8e0bc7640 1 --2- 172.21.15.156:0/485807471 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb8dc0fb640 0x7fb8dc10f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-16T09:55:01.481 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:01.479+0000 7fb8dbfff640 1 --2- 172.21.15.156:0/485807471 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb8dc0fc020 0x7fb8dc10f730 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fb8dc0fc3f0 tx=0x7fb8c8009f70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:55:01.481 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:01.479+0000 7fb8d97fa640 1 -- 172.21.15.156:0/485807471 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb8c8007a20 con 0x7fb8dc0fc020 2024-07-16T09:55:01.481 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:01.479+0000 7fb8e2e2b640 1 -- 172.21.15.156:0/485807471 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8dc110830 con 0x7fb8dc0fc020 2024-07-16T09:55:01.481 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:01.479+0000 7fb8d97fa640 1 -- 172.21.15.156:0/485807471 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb8c800f040 con 0x7fb8dc0fc020 2024-07-16T09:55:01.482 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:01.479+0000 7fb8d97fa640 1 -- 172.21.15.156:0/485807471 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb8c8009490 con 0x7fb8dc0fc020 2024-07-16T09:55:01.482 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:01.479+0000 7fb8e2e2b640 1 -- 172.21.15.156:0/485807471 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8dc110dd0 con 0x7fb8dc0fc020 2024-07-16T09:55:01.482 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:01.480+0000 7fb8e2e2b640 1 -- 172.21.15.156:0/485807471 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8dc0f99d0 con 0x7fb8dc0fc020 2024-07-16T09:55:01.483 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:01.482+0000 7fb8d97fa640 1 -- 172.21.15.156:0/485807471 <== mon.1 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 19) v1 ==== 85127+0+0 (secure 0 0 0) 0x7fb8c8009700 con 0x7fb8dc0fc020 2024-07-16T09:55:01.483 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:01.482+0000 7fb8d97fa640 1 --2- 172.21.15.156:0/485807471 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7fb8cc067190 0x7fb8cc069640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:55:01.484 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:01.482+0000 7fb8d97fa640 1 -- 172.21.15.156:0/485807471 <== mon.1 v2:172.21.15.176:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5464+0+0 (secure 0 0 0) 0x7fb8c8084210 con 0x7fb8dc0fc020 2024-07-16T09:55:01.484 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:01.483+0000 7fb8e0bc7640 1 --2- 172.21.15.156:0/485807471 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7fb8cc067190 0x7fb8cc069640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:55:01.484 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:01.483+0000 7fb8e0bc7640 1 --2- 172.21.15.156:0/485807471 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7fb8cc067190 0x7fb8cc069640 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fb8d0009f80 tx=0x7fb8d0035040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:55:01.490 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:01.489+0000 7fb8d97fa640 1 -- 172.21.15.156:0/485807471 <== mon.1 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fb8c804d250 con 0x7fb8dc0fc020 2024-07-16T09:55:01.806 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:01.804+0000 7fb8e2e2b640 1 -- 172.21.15.156:0/485807471 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fb8dc04b530 con 0x7fb8dc0fc020 2024-07-16T09:55:01.806 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:01.804+0000 7fb8d97fa640 1 -- 172.21.15.156:0/485807471 <== mon.1 v2:172.21.15.176:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 17 v17) v1 ==== 76+0+1852 (secure 0 0 0) 0x7fb8c8051540 con 0x7fb8dc0fc020 2024-07-16T09:55:01.806 INFO:teuthology.orchestra.run.smithi156.stderr:dumped fsmap epoch 17 2024-07-16T09:55:01.807 INFO:teuthology.orchestra.run.smithi156.stdout:e17 2024-07-16T09:55:01.807 INFO:teuthology.orchestra.run.smithi156.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-07-16T09:55:01.807 INFO:teuthology.orchestra.run.smithi156.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-07-16T09:55:01.807 INFO:teuthology.orchestra.run.smithi156.stdout:legacy client fscid: 1 2024-07-16T09:55:01.807 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-07-16T09:55:01.807 INFO:teuthology.orchestra.run.smithi156.stdout:Filesystem 'cephfs' (1) 2024-07-16T09:55:01.807 INFO:teuthology.orchestra.run.smithi156.stdout:fs_name cephfs 2024-07-16T09:55:01.807 INFO:teuthology.orchestra.run.smithi156.stdout:epoch 17 2024-07-16T09:55:01.807 INFO:teuthology.orchestra.run.smithi156.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-07-16T09:55:01.808 INFO:teuthology.orchestra.run.smithi156.stdout:created 2024-07-16T09:52:55.214975+0000 2024-07-16T09:55:01.808 INFO:teuthology.orchestra.run.smithi156.stdout:modified 2024-07-16T09:54:55.108622+0000 2024-07-16T09:55:01.808 INFO:teuthology.orchestra.run.smithi156.stdout:tableserver 0 2024-07-16T09:55:01.808 INFO:teuthology.orchestra.run.smithi156.stdout:root 0 2024-07-16T09:55:01.808 INFO:teuthology.orchestra.run.smithi156.stdout:session_timeout 60 2024-07-16T09:55:01.808 INFO:teuthology.orchestra.run.smithi156.stdout:session_autoclose 300 2024-07-16T09:55:01.808 INFO:teuthology.orchestra.run.smithi156.stdout:max_file_size 1099511627776 2024-07-16T09:55:01.808 INFO:teuthology.orchestra.run.smithi156.stdout:required_client_features {} 2024-07-16T09:55:01.808 INFO:teuthology.orchestra.run.smithi156.stdout:last_failure 0 2024-07-16T09:55:01.808 INFO:teuthology.orchestra.run.smithi156.stdout:last_failure_osd_epoch 0 2024-07-16T09:55:01.808 INFO:teuthology.orchestra.run.smithi156.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-07-16T09:55:01.808 INFO:teuthology.orchestra.run.smithi156.stdout:max_mds 2 2024-07-16T09:55:01.809 INFO:teuthology.orchestra.run.smithi156.stdout:in 0,1 2024-07-16T09:55:01.809 INFO:teuthology.orchestra.run.smithi156.stdout:up {0=14458,1=24261} 2024-07-16T09:55:01.809 INFO:teuthology.orchestra.run.smithi156.stdout:failed 2024-07-16T09:55:01.809 INFO:teuthology.orchestra.run.smithi156.stdout:damaged 2024-07-16T09:55:01.809 INFO:teuthology.orchestra.run.smithi156.stdout:stopped 2024-07-16T09:55:01.809 INFO:teuthology.orchestra.run.smithi156.stdout:data_pools [3] 2024-07-16T09:55:01.809 INFO:teuthology.orchestra.run.smithi156.stdout:metadata_pool 2 2024-07-16T09:55:01.809 INFO:teuthology.orchestra.run.smithi156.stdout:inline_data disabled 2024-07-16T09:55:01.809 INFO:teuthology.orchestra.run.smithi156.stdout:balancer 2024-07-16T09:55:01.809 INFO:teuthology.orchestra.run.smithi156.stdout:standby_count_wanted 1 2024-07-16T09:55:01.809 INFO:teuthology.orchestra.run.smithi156.stdout:[mds.cephfs.smithi156.rdibys{0:14458} state up:active seq 4 export targets 1 join_fscid=1 addr [v2:172.21.15.156:6826/1080176013,v1:172.21.15.156:6827/1080176013] compat {c=[1],r=[1],i=[7ff]}] 2024-07-16T09:55:01.809 INFO:teuthology.orchestra.run.smithi156.stdout:[mds.cephfs.smithi156.evwwaz{0:14468} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.156:6828/560329502,v1:172.21.15.156:6829/560329502] compat {c=[1],r=[1],i=[7ff]}] 2024-07-16T09:55:01.809 INFO:teuthology.orchestra.run.smithi156.stdout:[mds.cephfs.smithi176.dvvbud{1:24261} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.176:6824/1577019287,v1:172.21.15.176:6825/1577019287] compat {c=[1],r=[1],i=[7ff]}] 2024-07-16T09:55:01.810 INFO:teuthology.orchestra.run.smithi156.stdout:[mds.cephfs.smithi176.wahbsi{1:24273} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.176:6826/2196775412,v1:172.21.15.176:6827/2196775412] compat {c=[1],r=[1],i=[7ff]}] 2024-07-16T09:55:01.810 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-07-16T09:55:01.810 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-07-16T09:55:01.811 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:01.808+0000 7fb8e2e2b640 1 -- 172.21.15.156:0/485807471 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7fb8cc067190 msgr2=0x7fb8cc069640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:55:01.811 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:01.808+0000 7fb8e2e2b640 1 --2- 172.21.15.156:0/485807471 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7fb8cc067190 0x7fb8cc069640 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fb8d0009f80 tx=0x7fb8d0035040 comp rx=0 tx=0).stop 2024-07-16T09:55:01.811 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:01.808+0000 7fb8e2e2b640 1 -- 172.21.15.156:0/485807471 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb8dc0fc020 msgr2=0x7fb8dc10f730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:55:01.811 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:01.808+0000 7fb8e2e2b640 1 --2- 172.21.15.156:0/485807471 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb8dc0fc020 0x7fb8dc10f730 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fb8dc0fc3f0 tx=0x7fb8c8009f70 comp rx=0 tx=0).stop 2024-07-16T09:55:01.811 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:01.808+0000 7fb8e2e2b640 1 -- 172.21.15.156:0/485807471 shutdown_connections 2024-07-16T09:55:01.811 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:01.808+0000 7fb8e2e2b640 1 --2- 172.21.15.156:0/485807471 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7fb8cc067190 0x7fb8cc069640 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:55:01.811 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:01.808+0000 7fb8e2e2b640 1 --2- 172.21.15.156:0/485807471 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb8dc0fc020 0x7fb8dc10f730 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:55:01.811 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:01.808+0000 7fb8e2e2b640 1 --2- 172.21.15.156:0/485807471 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb8dc0fb640 0x7fb8dc10f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:55:01.811 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:01.808+0000 7fb8e2e2b640 1 -- 172.21.15.156:0/485807471 >> 172.21.15.156:0/485807471 conn(0x7fb8dc0f5d30 msgr2=0x7fb8dc0f7440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:55:01.811 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:01.808+0000 7fb8e2e2b640 1 -- 172.21.15.156:0/485807471 shutdown_connections 2024-07-16T09:55:01.811 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:01.808+0000 7fb8e2e2b640 1 -- 172.21.15.156:0/485807471 wait complete. 2024-07-16T09:55:01.931 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:01.929+0000 7f3335916640 1 -- 172.21.15.156:0/1529396607 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3330071380 msgr2=0x7f333006f8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:55:01.931 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:01.929+0000 7f3335916640 1 --2- 172.21.15.156:0/1529396607 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3330071380 0x7f333006f8b0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f33200048d0 tx=0x7f3320009da0 comp rx=0 tx=0).stop 2024-07-16T09:55:01.931 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:01.930+0000 7f3335916640 1 -- 172.21.15.156:0/1529396607 shutdown_connections 2024-07-16T09:55:01.931 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:01.930+0000 7f3335916640 1 --2- 172.21.15.156:0/1529396607 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f333006fdf0 0x7f3330070260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:55:01.931 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:01.930+0000 7f3335916640 1 --2- 172.21.15.156:0/1529396607 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3330071380 0x7f333006f8b0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:55:01.931 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:01.930+0000 7f3335916640 1 -- 172.21.15.156:0/1529396607 >> 172.21.15.156:0/1529396607 conn(0x7f33300f7b40 msgr2=0x7f33300f9f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:55:01.932 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:01.930+0000 7f3335916640 1 -- 172.21.15.156:0/1529396607 shutdown_connections 2024-07-16T09:55:01.932 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:01.930+0000 7f3335916640 1 -- 172.21.15.156:0/1529396607 wait complete. 2024-07-16T09:55:01.933 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:01.930+0000 7f3335916640 1 Processor -- start 2024-07-16T09:55:01.933 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:01.931+0000 7f3335916640 1 -- start start 2024-07-16T09:55:01.933 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:01.931+0000 7f3335916640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f333006fdf0 0x7f333010f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:55:01.933 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:01.931+0000 7f3335916640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3330071380 0x7f333010f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:55:01.934 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:01.931+0000 7f3335916640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f333010fd10 con 0x7f3330071380 2024-07-16T09:55:01.934 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:01.931+0000 7f3335916640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33301104c0 con 0x7f333006fdf0 2024-07-16T09:55:01.934 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:01.931+0000 7f332e7fc640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3330071380 0x7f333010f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:55:01.934 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:01.931+0000 7f332e7fc640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3330071380 0x7f333010f6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:49108/0 (socket says 172.21.15.156:49108) 2024-07-16T09:55:01.934 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:01.931+0000 7f332e7fc640 1 -- 172.21.15.156:0/2519763737 learned_addr learned my addr 172.21.15.156:0/2519763737 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T09:55:01.934 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:01.931+0000 7f332effd640 1 --2- 172.21.15.156:0/2519763737 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f333006fdf0 0x7f333010f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:55:01.934 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:01.931+0000 7f332e7fc640 1 -- 172.21.15.156:0/2519763737 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f333006fdf0 msgr2=0x7f333010f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:55:01.934 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:01.931+0000 7f332e7fc640 1 --2- 172.21.15.156:0/2519763737 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f333006fdf0 0x7f333010f180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:55:01.934 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:01.931+0000 7f332e7fc640 1 -- 172.21.15.156:0/2519763737 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3320005040 con 0x7f3330071380 2024-07-16T09:55:01.934 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:01.932+0000 7f332e7fc640 1 --2- 172.21.15.156:0/2519763737 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3330071380 0x7f333010f6c0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f332400eb70 tx=0x7f332400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:55:01.934 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:01.932+0000 7f33177fe640 1 -- 172.21.15.156:0/2519763737 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3324007c30 con 0x7f3330071380 2024-07-16T09:55:01.934 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:01.932+0000 7f3335916640 1 -- 172.21.15.156:0/2519763737 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33301106f0 con 0x7f3330071380 2024-07-16T09:55:01.935 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:01.932+0000 7f33177fe640 1 -- 172.21.15.156:0/2519763737 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f332400f040 con 0x7f3330071380 2024-07-16T09:55:01.935 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:01.932+0000 7f33177fe640 1 -- 172.21.15.156:0/2519763737 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f332400cad0 con 0x7f3330071380 2024-07-16T09:55:01.935 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:01.932+0000 7f3335916640 1 -- 172.21.15.156:0/2519763737 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3330110c90 con 0x7f3330071380 2024-07-16T09:55:01.935 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:01.934+0000 7f3335916640 1 -- 172.21.15.156:0/2519763737 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33300fb7c0 con 0x7f3330071380 2024-07-16T09:55:01.935 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:01.934+0000 7f33177fe640 1 -- 172.21.15.156:0/2519763737 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 19) v1 ==== 85127+0+0 (secure 0 0 0) 0x7f332400cc30 con 0x7f3330071380 2024-07-16T09:55:01.936 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:01.935+0000 7f33177fe640 1 --2- 172.21.15.156:0/2519763737 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f3318067190 0x7f3318069640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:55:01.936 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:01.935+0000 7f33177fe640 1 -- 172.21.15.156:0/2519763737 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f3324084680 con 0x7f3330071380 2024-07-16T09:55:01.936 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:01.935+0000 7f332effd640 1 --2- 172.21.15.156:0/2519763737 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f3318067190 0x7f3318069640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:55:01.937 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:01.936+0000 7f332effd640 1 --2- 172.21.15.156:0/2519763737 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f3318067190 0x7f3318069640 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f3320007ac0 tx=0x7f3320035040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:55:01.942 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:01.941+0000 7f33177fe640 1 -- 172.21.15.156:0/2519763737 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f3324051a50 con 0x7f3330071380 2024-07-16T09:55:02.205 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:02.203+0000 7f3335916640 1 -- 172.21.15.156:0/2519763737 --> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f3330110fc0 con 0x7f3318067190 2024-07-16T09:55:02.206 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:02.205+0000 7f33177fe640 1 -- 172.21.15.156:0/2519763737 <== mgr.14208 v2:172.21.15.156:6800/2615741717 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+328 (secure 0 0 0) 0x7f3330110fc0 con 0x7f3318067190 2024-07-16T09:55:02.206 INFO:teuthology.orchestra.run.smithi156.stdout:{ 2024-07-16T09:55:02.206 INFO:teuthology.orchestra.run.smithi156.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:c15ec3aa8e6c0a9c12ee24ff79f3e7dd4d01b3d1a702dfeb7d7fff9b0811a393", 2024-07-16T09:55:02.206 INFO:teuthology.orchestra.run.smithi156.stdout: "in_progress": true, 2024-07-16T09:55:02.206 INFO:teuthology.orchestra.run.smithi156.stdout: "which": "Upgrading all daemon types on all hosts", 2024-07-16T09:55:02.206 INFO:teuthology.orchestra.run.smithi156.stdout: "services_complete": [], 2024-07-16T09:55:02.206 INFO:teuthology.orchestra.run.smithi156.stdout: "progress": "", 2024-07-16T09:55:02.207 INFO:teuthology.orchestra.run.smithi156.stdout: "message": "Currently upgrading mgr daemons", 2024-07-16T09:55:02.207 INFO:teuthology.orchestra.run.smithi156.stdout: "is_paused": false 2024-07-16T09:55:02.207 INFO:teuthology.orchestra.run.smithi156.stdout:} 2024-07-16T09:55:02.209 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:02.207+0000 7f3335916640 1 -- 172.21.15.156:0/2519763737 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f3318067190 msgr2=0x7f3318069640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:55:02.209 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:02.207+0000 7f3335916640 1 --2- 172.21.15.156:0/2519763737 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f3318067190 0x7f3318069640 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f3320007ac0 tx=0x7f3320035040 comp rx=0 tx=0).stop 2024-07-16T09:55:02.209 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:02.207+0000 7f3335916640 1 -- 172.21.15.156:0/2519763737 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3330071380 msgr2=0x7f333010f6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:55:02.209 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:02.207+0000 7f3335916640 1 --2- 172.21.15.156:0/2519763737 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3330071380 0x7f333010f6c0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f332400eb70 tx=0x7f332400b040 comp rx=0 tx=0).stop 2024-07-16T09:55:02.209 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:02.207+0000 7f3335916640 1 -- 172.21.15.156:0/2519763737 shutdown_connections 2024-07-16T09:55:02.209 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:02.207+0000 7f3335916640 1 --2- 172.21.15.156:0/2519763737 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f3318067190 0x7f3318069640 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:55:02.209 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:02.207+0000 7f3335916640 1 --2- 172.21.15.156:0/2519763737 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3330071380 0x7f333010f6c0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:55:02.209 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:02.207+0000 7f3335916640 1 --2- 172.21.15.156:0/2519763737 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f333006fdf0 0x7f333010f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:55:02.209 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:02.207+0000 7f3335916640 1 -- 172.21.15.156:0/2519763737 >> 172.21.15.156:0/2519763737 conn(0x7f33300f7b40 msgr2=0x7f33300f9210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:55:02.209 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:02.207+0000 7f3335916640 1 -- 172.21.15.156:0/2519763737 shutdown_connections 2024-07-16T09:55:02.209 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:02.207+0000 7f3335916640 1 -- 172.21.15.156:0/2519763737 wait complete. 2024-07-16T09:55:02.314 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:02.310+0000 7fc4bf273640 1 -- 172.21.15.156:0/178261084 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc4b80fa060 msgr2=0x7fc4b80fa470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:55:02.314 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:02.310+0000 7fc4bf273640 1 --2- 172.21.15.156:0/178261084 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc4b80fa060 0x7fc4b80fa470 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fc4b40048d0 tx=0x7fc4b4009da0 comp rx=0 tx=0).stop 2024-07-16T09:55:02.314 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:02.310+0000 7fc4bf273640 1 -- 172.21.15.156:0/178261084 shutdown_connections 2024-07-16T09:55:02.314 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:02.310+0000 7fc4bf273640 1 --2- 172.21.15.156:0/178261084 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc4b80fa9b0 0x7fc4b8101c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:55:02.314 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:02.310+0000 7fc4bf273640 1 --2- 172.21.15.156:0/178261084 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc4b80fa060 0x7fc4b80fa470 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:55:02.314 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:02.310+0000 7fc4bf273640 1 -- 172.21.15.156:0/178261084 >> 172.21.15.156:0/178261084 conn(0x7fc4b80f5d30 msgr2=0x7fc4b80f8160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:55:02.314 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:02.310+0000 7fc4bf273640 1 -- 172.21.15.156:0/178261084 shutdown_connections 2024-07-16T09:55:02.314 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:02.310+0000 7fc4bf273640 1 -- 172.21.15.156:0/178261084 wait complete. 2024-07-16T09:55:02.315 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:02.311+0000 7fc4bf273640 1 Processor -- start 2024-07-16T09:55:02.315 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:02.311+0000 7fc4bf273640 1 -- start start 2024-07-16T09:55:02.315 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:02.311+0000 7fc4bf273640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc4b80fa060 0x7fc4b810cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:55:02.315 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:02.311+0000 7fc4bf273640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc4b80fa9b0 0x7fc4b810d500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:55:02.315 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:02.311+0000 7fc4bf273640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4b810db50 con 0x7fc4b80fa9b0 2024-07-16T09:55:02.315 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:02.311+0000 7fc4bf273640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4b810e320 con 0x7fc4b80fa060 2024-07-16T09:55:02.315 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:02.311+0000 7fc4bc80e640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc4b80fa9b0 0x7fc4b810d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:55:02.315 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:02.311+0000 7fc4bc80e640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc4b80fa9b0 0x7fc4b810d500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:49136/0 (socket says 172.21.15.156:49136) 2024-07-16T09:55:02.315 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:02.311+0000 7fc4bc80e640 1 -- 172.21.15.156:0/2003830461 learned_addr learned my addr 172.21.15.156:0/2003830461 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T09:55:02.316 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:02.311+0000 7fc4bd00f640 1 --2- 172.21.15.156:0/2003830461 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc4b80fa060 0x7fc4b810cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:55:02.316 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:02.312+0000 7fc4bc80e640 1 -- 172.21.15.156:0/2003830461 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc4b80fa060 msgr2=0x7fc4b810cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:55:02.316 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:02.312+0000 7fc4bc80e640 1 --2- 172.21.15.156:0/2003830461 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc4b80fa060 0x7fc4b810cfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:55:02.316 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:02.312+0000 7fc4bc80e640 1 -- 172.21.15.156:0/2003830461 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4b4005040 con 0x7fc4b80fa9b0 2024-07-16T09:55:02.316 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:02.312+0000 7fc4bc80e640 1 --2- 172.21.15.156:0/2003830461 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc4b80fa9b0 0x7fc4b810d500 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fc4a800eb70 tx=0x7fc4a800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:55:02.316 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:02.312+0000 7fc4adffb640 1 -- 172.21.15.156:0/2003830461 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc4a8007c30 con 0x7fc4b80fa9b0 2024-07-16T09:55:02.316 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:02.312+0000 7fc4adffb640 1 -- 172.21.15.156:0/2003830461 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc4a800f040 con 0x7fc4b80fa9b0 2024-07-16T09:55:02.316 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:02.312+0000 7fc4adffb640 1 -- 172.21.15.156:0/2003830461 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc4a800cad0 con 0x7fc4b80fa9b0 2024-07-16T09:55:02.316 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:02.312+0000 7fc4bf273640 1 -- 172.21.15.156:0/2003830461 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4b810e550 con 0x7fc4b80fa9b0 2024-07-16T09:55:02.316 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:02.312+0000 7fc4bf273640 1 -- 172.21.15.156:0/2003830461 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4b810eaf0 con 0x7fc4b80fa9b0 2024-07-16T09:55:02.317 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:02.314+0000 7fc4bf273640 1 -- 172.21.15.156:0/2003830461 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4b80f99d0 con 0x7fc4b80fa9b0 2024-07-16T09:55:02.317 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:02.314+0000 7fc4adffb640 1 -- 172.21.15.156:0/2003830461 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 19) v1 ==== 85127+0+0 (secure 0 0 0) 0x7fc4a800cc30 con 0x7fc4b80fa9b0 2024-07-16T09:55:02.317 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:02.315+0000 7fc4adffb640 1 --2- 172.21.15.156:0/2003830461 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7fc4a4067260 0x7fc4a4069710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:55:02.317 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:02.315+0000 7fc4adffb640 1 -- 172.21.15.156:0/2003830461 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5464+0+0 (secure 0 0 0) 0x7fc4a80847b0 con 0x7fc4b80fa9b0 2024-07-16T09:55:02.317 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:02.315+0000 7fc4bd00f640 1 --2- 172.21.15.156:0/2003830461 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7fc4a4067260 0x7fc4a4069710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:55:02.318 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:02.316+0000 7fc4bd00f640 1 --2- 172.21.15.156:0/2003830461 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7fc4a4067260 0x7fc4a4069710 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fc4b402af90 tx=0x7fc4b4009da0 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:55:02.322 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:02.321+0000 7fc4adffb640 1 -- 172.21.15.156:0/2003830461 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fc4a8051a30 con 0x7fc4b80fa9b0 2024-07-16T09:55:02.424 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:55:02 smithi156 ceph-mon[29302]: from='client.14590 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-16T09:55:02.424 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:55:02 smithi156 ceph-mon[29302]: pgmap v232: 65 pgs: 65 active+clean; 101 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 2.0 MiB/s rd, 5.2 MiB/s wr, 505 op/s 2024-07-16T09:55:02.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:55:02 smithi156 ceph-mon[29302]: from='client.14598 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-07-16T09:55:02.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:55:02 smithi156 ceph-mon[29302]: from='client.? 172.21.15.156:0/2473290775' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T09:55:02.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:55:02 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:55:02.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:55:02 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:55:02.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:55:02 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-16T09:55:02.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:55:02 smithi156 ceph-mon[29302]: from='client.? 172.21.15.156:0/485807471' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-07-16T09:55:02.477 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:55:02 smithi176 ceph-mon[36626]: from='client.14590 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-16T09:55:02.477 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:55:02 smithi176 ceph-mon[36626]: pgmap v232: 65 pgs: 65 active+clean; 101 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 2.0 MiB/s rd, 5.2 MiB/s wr, 505 op/s 2024-07-16T09:55:02.477 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:55:02 smithi176 ceph-mon[36626]: from='client.14598 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-07-16T09:55:02.477 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:55:02 smithi176 ceph-mon[36626]: from='client.? 172.21.15.156:0/2473290775' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T09:55:02.477 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:55:02 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:55:02.477 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:55:02 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:55:02.477 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:55:02 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-16T09:55:02.478 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:55:02 smithi176 ceph-mon[36626]: from='client.? 172.21.15.156:0/485807471' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-07-16T09:55:02.696 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:02.695+0000 7fc4bf273640 1 -- 172.21.15.156:0/2003830461 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fc4b804b530 con 0x7fc4b80fa9b0 2024-07-16T09:55:02.696 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:02.695+0000 7fc4adffb640 1 -- 172.21.15.156:0/2003830461 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7fc4a8055050 con 0x7fc4b80fa9b0 2024-07-16T09:55:02.697 INFO:teuthology.orchestra.run.smithi156.stdout:HEALTH_OK 2024-07-16T09:55:02.700 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:02.698+0000 7fc4bf273640 1 -- 172.21.15.156:0/2003830461 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7fc4a4067260 msgr2=0x7fc4a4069710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:55:02.700 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:02.698+0000 7fc4bf273640 1 --2- 172.21.15.156:0/2003830461 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7fc4a4067260 0x7fc4a4069710 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fc4b402af90 tx=0x7fc4b4009da0 comp rx=0 tx=0).stop 2024-07-16T09:55:02.700 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:02.698+0000 7fc4bf273640 1 -- 172.21.15.156:0/2003830461 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc4b80fa9b0 msgr2=0x7fc4b810d500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:55:02.700 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:02.698+0000 7fc4bf273640 1 --2- 172.21.15.156:0/2003830461 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc4b80fa9b0 0x7fc4b810d500 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fc4a800eb70 tx=0x7fc4a800b040 comp rx=0 tx=0).stop 2024-07-16T09:55:02.700 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:02.698+0000 7fc4bf273640 1 -- 172.21.15.156:0/2003830461 shutdown_connections 2024-07-16T09:55:02.700 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:02.698+0000 7fc4bf273640 1 --2- 172.21.15.156:0/2003830461 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7fc4a4067260 0x7fc4a4069710 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:55:02.700 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:02.698+0000 7fc4bf273640 1 --2- 172.21.15.156:0/2003830461 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc4b80fa9b0 0x7fc4b810d500 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:55:02.700 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:02.698+0000 7fc4bf273640 1 --2- 172.21.15.156:0/2003830461 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc4b80fa060 0x7fc4b810cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:55:02.700 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:02.698+0000 7fc4bf273640 1 -- 172.21.15.156:0/2003830461 >> 172.21.15.156:0/2003830461 conn(0x7fc4b80f5d30 msgr2=0x7fc4b80ffcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:55:02.701 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:02.698+0000 7fc4bf273640 1 -- 172.21.15.156:0/2003830461 shutdown_connections 2024-07-16T09:55:02.701 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:02.698+0000 7fc4bf273640 1 -- 172.21.15.156:0/2003830461 wait complete. 2024-07-16T09:55:03.343 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:55:03 smithi176 ceph-mon[36626]: from='client.14610 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-16T09:55:03.343 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:55:03 smithi176 ceph-mon[36626]: from='client.? 172.21.15.156:0/2003830461' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-07-16T09:55:03.424 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:55:03 smithi156 ceph-mon[29302]: from='client.14610 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-16T09:55:03.424 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:55:03 smithi156 ceph-mon[29302]: from='client.? 172.21.15.156:0/2003830461' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-07-16T09:55:04.343 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:55:04 smithi176 ceph-mon[36626]: pgmap v233: 65 pgs: 65 active+clean; 107 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 1.7 MiB/s rd, 4.7 MiB/s wr, 473 op/s 2024-07-16T09:55:04.424 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:55:04 smithi156 ceph-mon[29302]: pgmap v233: 65 pgs: 65 active+clean; 107 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 1.7 MiB/s rd, 4.7 MiB/s wr, 473 op/s 2024-07-16T09:55:06.214 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:55:06 smithi176 ceph-mon[36626]: pgmap v234: 65 pgs: 65 active+clean; 114 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 2.2 MiB/s rd, 5.0 MiB/s wr, 517 op/s 2024-07-16T09:55:06.424 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:55:06 smithi156 ceph-mon[29302]: pgmap v234: 65 pgs: 65 active+clean; 114 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 2.2 MiB/s rd, 5.0 MiB/s wr, 517 op/s 2024-07-16T09:55:08.175 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:55:08 smithi156 ceph-mon[29302]: pgmap v235: 65 pgs: 65 active+clean; 118 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 2.1 MiB/s rd, 4.2 MiB/s wr, 479 op/s 2024-07-16T09:55:08.175 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:55:08 smithi156 ceph-mon[29302]: Standby manager daemon smithi176.dqbrvx restarted 2024-07-16T09:55:08.175 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:55:08 smithi156 ceph-mon[29302]: Standby manager daemon smithi176.dqbrvx started 2024-07-16T09:55:08.176 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:55:08 smithi156 ceph-mon[29302]: from='mgr.? 172.21.15.176:0/1082993331' entity='mgr.smithi176.dqbrvx' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi176.dqbrvx/crt"}]: dispatch 2024-07-16T09:55:08.176 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:55:08 smithi156 ceph-mon[29302]: from='mgr.? 172.21.15.176:0/1082993331' entity='mgr.smithi176.dqbrvx' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-07-16T09:55:08.176 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:55:08 smithi156 ceph-mon[29302]: from='mgr.? 172.21.15.176:0/1082993331' entity='mgr.smithi176.dqbrvx' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi176.dqbrvx/key"}]: dispatch 2024-07-16T09:55:08.176 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:55:08 smithi156 ceph-mon[29302]: from='mgr.? 172.21.15.176:0/1082993331' entity='mgr.smithi176.dqbrvx' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-07-16T09:55:08.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:55:08 smithi176 ceph-mon[36626]: pgmap v235: 65 pgs: 65 active+clean; 118 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 2.1 MiB/s rd, 4.2 MiB/s wr, 479 op/s 2024-07-16T09:55:08.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:55:08 smithi176 ceph-mon[36626]: Standby manager daemon smithi176.dqbrvx restarted 2024-07-16T09:55:08.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:55:08 smithi176 ceph-mon[36626]: Standby manager daemon smithi176.dqbrvx started 2024-07-16T09:55:08.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:55:08 smithi176 ceph-mon[36626]: from='mgr.? 172.21.15.176:0/1082993331' entity='mgr.smithi176.dqbrvx' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi176.dqbrvx/crt"}]: dispatch 2024-07-16T09:55:08.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:55:08 smithi176 ceph-mon[36626]: from='mgr.? 172.21.15.176:0/1082993331' entity='mgr.smithi176.dqbrvx' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-07-16T09:55:08.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:55:08 smithi176 ceph-mon[36626]: from='mgr.? 172.21.15.176:0/1082993331' entity='mgr.smithi176.dqbrvx' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi176.dqbrvx/key"}]: dispatch 2024-07-16T09:55:08.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:55:08 smithi176 ceph-mon[36626]: from='mgr.? 172.21.15.176:0/1082993331' entity='mgr.smithi176.dqbrvx' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-07-16T09:55:09.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:55:09 smithi176 ceph-mon[36626]: mgrmap e20: smithi156.zkvylv(active, since 6m), standbys: smithi176.dqbrvx 2024-07-16T09:55:09.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:55:09 smithi156 ceph-mon[29302]: mgrmap e20: smithi156.zkvylv(active, since 6m), standbys: smithi176.dqbrvx 2024-07-16T09:55:10.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:55:10 smithi176 ceph-mon[36626]: pgmap v236: 65 pgs: 65 active+clean; 125 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 2.3 MiB/s rd, 4.4 MiB/s wr, 513 op/s 2024-07-16T09:55:10.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:55:10 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-16T09:55:10.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:55:10 smithi156 ceph-mon[29302]: pgmap v236: 65 pgs: 65 active+clean; 125 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 2.3 MiB/s rd, 4.4 MiB/s wr, 513 op/s 2024-07-16T09:55:10.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:55:10 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-16T09:55:12.570 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:55:12 smithi176 ceph-mon[36626]: pgmap v237: 65 pgs: 65 active+clean; 136 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 2.2 MiB/s rd, 4.1 MiB/s wr, 508 op/s 2024-07-16T09:55:12.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:55:12 smithi156 ceph-mon[29302]: pgmap v237: 65 pgs: 65 active+clean; 136 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 2.2 MiB/s rd, 4.1 MiB/s wr, 508 op/s 2024-07-16T09:55:14.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:55:14 smithi176 ceph-mon[36626]: pgmap v238: 65 pgs: 65 active+clean; 142 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 2.0 MiB/s rd, 3.7 MiB/s wr, 469 op/s 2024-07-16T09:55:14.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:55:14 smithi156 ceph-mon[29302]: pgmap v238: 65 pgs: 65 active+clean; 142 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 2.0 MiB/s rd, 3.7 MiB/s wr, 469 op/s 2024-07-16T09:55:16.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:55:16 smithi176 ceph-mon[36626]: pgmap v239: 65 pgs: 65 active+clean; 155 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 2.2 MiB/s rd, 4.3 MiB/s wr, 484 op/s 2024-07-16T09:55:16.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:55:16 smithi156 ceph-mon[29302]: pgmap v239: 65 pgs: 65 active+clean; 155 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 2.2 MiB/s rd, 4.3 MiB/s wr, 484 op/s 2024-07-16T09:55:18.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:55:18 smithi176 ceph-mon[36626]: pgmap v240: 65 pgs: 65 active+clean; 158 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.8 MiB/s rd, 3.9 MiB/s wr, 430 op/s 2024-07-16T09:55:18.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:55:18 smithi156 ceph-mon[29302]: pgmap v240: 65 pgs: 65 active+clean; 158 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.8 MiB/s rd, 3.9 MiB/s wr, 430 op/s 2024-07-16T09:55:20.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:55:20 smithi176 ceph-mon[36626]: pgmap v241: 65 pgs: 65 active+clean; 168 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1.7 MiB/s rd, 4.5 MiB/s wr, 475 op/s 2024-07-16T09:55:20.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:55:20 smithi156 ceph-mon[29302]: pgmap v241: 65 pgs: 65 active+clean; 168 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1.7 MiB/s rd, 4.5 MiB/s wr, 475 op/s 2024-07-16T09:55:21.417 INFO:tasks.workunit.client.1.smithi176.stderr:+ pushd ltp-full-20091231/testcases/kernel/fs/fsstress 2024-07-16T09:55:21.424 INFO:tasks.workunit.client.1.smithi176.stdout:~/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress ~/cephtest/mnt.1/client.1/tmp/fsstress ~/cephtest/mnt.1/client.1/tmp 2024-07-16T09:55:21.424 INFO:tasks.workunit.client.1.smithi176.stderr:+ make 2024-07-16T09:55:21.768 INFO:tasks.workunit.client.1.smithi176.stdout:cc -DNO_XFS -I/home/ubuntu/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress -D_LARGEFILE64_SOURCE -D_GNU_SOURCE -I../../../../include -I../../../../include -L../../../../lib fsstress.c -o fsstress 2024-07-16T09:55:22.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:55:22 smithi176 ceph-mon[36626]: pgmap v242: 65 pgs: 65 active+clean; 185 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 2.1 MiB/s rd, 5.2 MiB/s wr, 480 op/s 2024-07-16T09:55:22.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:55:22 smithi156 ceph-mon[29302]: pgmap v242: 65 pgs: 65 active+clean; 185 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 2.1 MiB/s rd, 5.2 MiB/s wr, 480 op/s 2024-07-16T09:55:23.503 INFO:tasks.workunit.client.1.smithi176.stderr:++ readlink -f fsstress 2024-07-16T09:55:23.505 INFO:tasks.workunit.client.1.smithi176.stderr:+ BIN=/home/ubuntu/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress 2024-07-16T09:55:23.505 INFO:tasks.workunit.client.1.smithi176.stderr:+ popd 2024-07-16T09:55:23.506 INFO:tasks.workunit.client.1.smithi176.stdout:~/cephtest/mnt.1/client.1/tmp/fsstress ~/cephtest/mnt.1/client.1/tmp 2024-07-16T09:55:23.507 INFO:tasks.workunit.client.1.smithi176.stderr:+ popd 2024-07-16T09:55:23.507 INFO:tasks.workunit.client.1.smithi176.stdout:~/cephtest/mnt.1/client.1/tmp 2024-07-16T09:55:23.508 INFO:tasks.workunit.client.1.smithi176.stderr:++ mktemp -d -p . 2024-07-16T09:55:23.510 INFO:tasks.workunit.client.1.smithi176.stderr:+ T=./tmp.IB6AWFslDW 2024-07-16T09:55:23.510 INFO:tasks.workunit.client.1.smithi176.stderr:+ /home/ubuntu/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress -d ./tmp.IB6AWFslDW -l 1 -n 1000 -p 10 -v 2024-07-16T09:55:23.513 INFO:tasks.workunit.client.1.smithi176.stdout:seed = 1720817363 2024-07-16T09:55:23.516 INFO:tasks.workunit.client.1.smithi176.stdout:0/0: rmdir - no directory 2024-07-16T09:55:23.516 INFO:tasks.workunit.client.1.smithi176.stdout:0/1: dwrite - no filename 2024-07-16T09:55:23.517 INFO:tasks.workunit.client.1.smithi176.stdout:0/2: rename - no filename 2024-07-16T09:55:23.518 INFO:tasks.workunit.client.1.smithi176.stdout:0/3: mknod c0 0 2024-07-16T09:55:23.520 INFO:tasks.workunit.client.1.smithi176.stdout:2/0: link - no file 2024-07-16T09:55:23.520 INFO:tasks.workunit.client.1.smithi176.stdout:0/4: unlink c0 0 2024-07-16T09:55:23.520 INFO:tasks.workunit.client.1.smithi176.stdout:0/5: chown . 880858567 1 2024-07-16T09:55:23.521 INFO:tasks.workunit.client.1.smithi176.stdout:0/6: readlink - no filename 2024-07-16T09:55:23.521 INFO:tasks.workunit.client.1.smithi176.stdout:0/7: dread - no filename 2024-07-16T09:55:23.521 INFO:tasks.workunit.client.1.smithi176.stdout:0/8: read - no filename 2024-07-16T09:55:23.521 INFO:tasks.workunit.client.1.smithi176.stdout:1/0: mknod c0 0 2024-07-16T09:55:23.521 INFO:tasks.workunit.client.1.smithi176.stdout:1/1: read - no filename 2024-07-16T09:55:23.521 INFO:tasks.workunit.client.1.smithi176.stdout:1/2: truncate - no filename 2024-07-16T09:55:23.522 INFO:tasks.workunit.client.1.smithi176.stdout:3/0: dread - no filename 2024-07-16T09:55:23.522 INFO:tasks.workunit.client.1.smithi176.stdout:3/1: stat - no entries 2024-07-16T09:55:23.523 INFO:tasks.workunit.client.1.smithi176.stdout:0/9: creat f1 x:0 0 0 2024-07-16T09:55:23.523 INFO:tasks.workunit.client.1.smithi176.stdout:1/3: creat f1 x:0 0 0 2024-07-16T09:55:23.523 INFO:tasks.workunit.client.1.smithi176.stdout:2/1: symlink l0 0 2024-07-16T09:55:23.523 INFO:tasks.workunit.client.1.smithi176.stdout:2/2: fdatasync - no filename 2024-07-16T09:55:23.524 INFO:tasks.workunit.client.1.smithi176.stdout:0/10: chown f1 1 1 2024-07-16T09:55:23.524 INFO:tasks.workunit.client.1.smithi176.stdout:0/11: readlink - no filename 2024-07-16T09:55:23.524 INFO:tasks.workunit.client.1.smithi176.stdout:1/4: truncate f1 550786 0 2024-07-16T09:55:23.524 INFO:tasks.workunit.client.1.smithi176.stdout:0/12: chown f1 78240130 1 2024-07-16T09:55:23.524 INFO:tasks.workunit.client.1.smithi176.stdout:2/3: mknod c1 0 2024-07-16T09:55:23.526 INFO:tasks.workunit.client.1.smithi176.stdout:1/5: write f1 [1479947,121566] 0 2024-07-16T09:55:23.526 INFO:tasks.workunit.client.1.smithi176.stdout:0/13: symlink l2 0 2024-07-16T09:55:23.527 INFO:tasks.workunit.client.1.smithi176.stdout:4/0: creat f0 x:0 0 0 2024-07-16T09:55:23.528 INFO:tasks.workunit.client.1.smithi176.stdout:2/4: creat f2 x:0 0 0 2024-07-16T09:55:23.530 INFO:tasks.workunit.client.1.smithi176.stdout:6/0: unlink - no file 2024-07-16T09:55:23.531 INFO:tasks.workunit.client.1.smithi176.stdout:0/14: creat f3 x:0 0 0 2024-07-16T09:55:23.531 INFO:tasks.workunit.client.1.smithi176.stdout:4/1: creat f1 x:0 0 0 2024-07-16T09:55:23.531 INFO:tasks.workunit.client.1.smithi176.stdout:2/5: creat f3 x:0 0 0 2024-07-16T09:55:23.532 INFO:tasks.workunit.client.1.smithi176.stdout:5/0: getdents . 0 2024-07-16T09:55:23.532 INFO:tasks.workunit.client.1.smithi176.stdout:0/15: chown f3 2853 1 2024-07-16T09:55:23.532 INFO:tasks.workunit.client.1.smithi176.stdout:5/1: unlink - no file 2024-07-16T09:55:23.532 INFO:tasks.workunit.client.1.smithi176.stdout:5/2: truncate - no filename 2024-07-16T09:55:23.532 INFO:tasks.workunit.client.1.smithi176.stdout:5/3: fsync - no filename 2024-07-16T09:55:23.532 INFO:tasks.workunit.client.1.smithi176.stdout:5/4: chown . 29016952 1 2024-07-16T09:55:23.533 INFO:tasks.workunit.client.1.smithi176.stdout:0/16: chown l2 3684211 1 2024-07-16T09:55:23.533 INFO:tasks.workunit.client.1.smithi176.stdout:4/2: creat f2 x:0 0 0 2024-07-16T09:55:23.534 INFO:tasks.workunit.client.1.smithi176.stdout:4/3: fdatasync f0 0 2024-07-16T09:55:23.536 INFO:tasks.workunit.client.1.smithi176.stdout:6/1: creat f0 x:0 0 0 2024-07-16T09:55:23.536 INFO:tasks.workunit.client.1.smithi176.stdout:6/2: fdatasync f0 0 2024-07-16T09:55:23.538 INFO:tasks.workunit.client.1.smithi176.stdout:5/5: mknod c0 0 2024-07-16T09:55:23.538 INFO:tasks.workunit.client.1.smithi176.stdout:5/6: dread - no filename 2024-07-16T09:55:23.538 INFO:tasks.workunit.client.1.smithi176.stdout:5/7: dwrite - no filename 2024-07-16T09:55:23.538 INFO:tasks.workunit.client.1.smithi176.stdout:5/8: dwrite - no filename 2024-07-16T09:55:23.540 INFO:tasks.workunit.client.1.smithi176.stdout:0/17: link f3 f4 0 2024-07-16T09:55:23.541 INFO:tasks.workunit.client.1.smithi176.stdout:7/0: creat f0 x:0 0 0 2024-07-16T09:55:23.541 INFO:tasks.workunit.client.1.smithi176.stdout:6/3: truncate f0 826030 0 2024-07-16T09:55:23.541 INFO:tasks.workunit.client.1.smithi176.stdout:9/0: dwrite - no filename 2024-07-16T09:55:23.541 INFO:tasks.workunit.client.1.smithi176.stdout:9/1: truncate - no filename 2024-07-16T09:55:23.541 INFO:tasks.workunit.client.1.smithi176.stdout:9/2: stat - no entries 2024-07-16T09:55:23.543 INFO:tasks.workunit.client.1.smithi176.stdout:5/9: creat f1 x:0 0 0 2024-07-16T09:55:23.544 INFO:tasks.workunit.client.1.smithi176.stdout:8/0: creat f0 x:0 0 0 2024-07-16T09:55:23.544 INFO:tasks.workunit.client.1.smithi176.stdout:0/18: mkdir d5 0 2024-07-16T09:55:23.545 INFO:tasks.workunit.client.1.smithi176.stdout:7/1: write f0 [686818,70570] 0 2024-07-16T09:55:23.545 INFO:tasks.workunit.client.1.smithi176.stdout:6/4: mkdir d1 0 2024-07-16T09:55:23.546 INFO:tasks.workunit.client.1.smithi176.stdout:8/1: write f0 [731893,7524] 0 2024-07-16T09:55:23.548 INFO:tasks.workunit.client.1.smithi176.stdout:7/2: fdatasync f0 0 2024-07-16T09:55:23.549 INFO:tasks.workunit.client.1.smithi176.stdout:9/3: creat f0 x:0 0 0 2024-07-16T09:55:23.551 INFO:tasks.workunit.client.1.smithi176.stdout:9/4: truncate f0 200337 0 2024-07-16T09:55:23.552 INFO:tasks.workunit.client.1.smithi176.stdout:8/2: rename f0 to f1 0 2024-07-16T09:55:23.554 INFO:tasks.workunit.client.1.smithi176.stdout:7/3: write f0 [1148805,83634] 0 2024-07-16T09:55:23.554 INFO:tasks.workunit.client.1.smithi176.stdout:0/19: link f1 d5/f6 0 2024-07-16T09:55:23.556 INFO:tasks.workunit.client.1.smithi176.stdout:6/5: dread f0 [0,4194304] 0 2024-07-16T09:55:23.557 INFO:tasks.workunit.client.1.smithi176.stdout:9/5: rename f0 to f1 0 2024-07-16T09:55:23.557 INFO:tasks.workunit.client.1.smithi176.stdout:0/20: dread - d5/f6 zero size 2024-07-16T09:55:23.559 INFO:tasks.workunit.client.1.smithi176.stdout:7/4: symlink l1 0 2024-07-16T09:55:23.560 INFO:tasks.workunit.client.1.smithi176.stdout:4/4: dwrite f0 [0,4194304] 0 2024-07-16T09:55:23.560 INFO:tasks.workunit.client.1.smithi176.stdout:9/6: chown f1 871229974 1 2024-07-16T09:55:23.561 INFO:tasks.workunit.client.1.smithi176.stdout:7/5: chown f0 75096 1 2024-07-16T09:55:23.563 INFO:tasks.workunit.client.1.smithi176.stdout:8/3: dread f1 [0,4194304] 0 2024-07-16T09:55:23.564 INFO:tasks.workunit.client.1.smithi176.stdout:0/21: creat d5/f7 x:0 0 0 2024-07-16T09:55:23.564 INFO:tasks.workunit.client.1.smithi176.stdout:8/4: chown f1 15821070 1 2024-07-16T09:55:23.567 INFO:tasks.workunit.client.1.smithi176.stdout:4/5: link f1 f3 0 2024-07-16T09:55:23.568 INFO:tasks.workunit.client.1.smithi176.stdout:0/22: creat d5/f8 x:0 0 0 2024-07-16T09:55:23.571 INFO:tasks.workunit.client.1.smithi176.stdout:8/5: mknod c2 0 2024-07-16T09:55:23.574 INFO:tasks.workunit.client.1.smithi176.stdout:4/6: symlink l4 0 2024-07-16T09:55:23.574 INFO:tasks.workunit.client.1.smithi176.stdout:4/7: read - f3 zero size 2024-07-16T09:55:23.576 INFO:tasks.workunit.client.1.smithi176.stdout:4/8: readlink l4 0 2024-07-16T09:55:23.579 INFO:tasks.workunit.client.1.smithi176.stdout:9/7: dwrite f1 [0,4194304] 0 2024-07-16T09:55:23.580 INFO:tasks.workunit.client.1.smithi176.stdout:4/9: mknod c5 0 2024-07-16T09:55:23.581 INFO:tasks.workunit.client.1.smithi176.stdout:4/10: stat f1 0 2024-07-16T09:55:23.584 INFO:tasks.workunit.client.1.smithi176.stdout:9/8: creat f2 x:0 0 0 2024-07-16T09:55:23.587 INFO:tasks.workunit.client.1.smithi176.stdout:0/23: dwrite d5/f8 [0,4194304] 0 2024-07-16T09:55:23.596 INFO:tasks.workunit.client.1.smithi176.stdout:4/11: dwrite f1 [0,4194304] 0 2024-07-16T09:55:23.600 INFO:tasks.workunit.client.1.smithi176.stdout:4/12: link l4 l6 0 2024-07-16T09:55:23.649 INFO:tasks.workunit.client.1.smithi176.stdout:1/6: getdents . 0 2024-07-16T09:55:23.654 INFO:tasks.workunit.client.1.smithi176.stdout:2/6: getdents . 0 2024-07-16T09:55:23.655 INFO:tasks.workunit.client.1.smithi176.stdout:1/7: dwrite f1 [0,4194304] 0 2024-07-16T09:55:23.656 INFO:tasks.workunit.client.1.smithi176.stdout:2/7: unlink f2 0 2024-07-16T09:55:23.659 INFO:tasks.workunit.client.1.smithi176.stdout:1/8: mknod c2 0 2024-07-16T09:55:23.659 INFO:tasks.workunit.client.1.smithi176.stdout:1/9: stat f1 0 2024-07-16T09:55:23.660 INFO:tasks.workunit.client.1.smithi176.stdout:2/8: creat f4 x:0 0 0 2024-07-16T09:55:23.661 INFO:tasks.workunit.client.1.smithi176.stdout:2/9: dread - f3 zero size 2024-07-16T09:55:23.662 INFO:tasks.workunit.client.1.smithi176.stdout:2/10: write f3 [407314,75382] 0 2024-07-16T09:55:23.662 INFO:tasks.workunit.client.1.smithi176.stdout:1/10: creat f3 x:0 0 0 2024-07-16T09:55:23.662 INFO:tasks.workunit.client.1.smithi176.stdout:1/11: fdatasync f3 0 2024-07-16T09:55:23.663 INFO:tasks.workunit.client.1.smithi176.stdout:1/12: dread - f3 zero size 2024-07-16T09:55:23.663 INFO:tasks.workunit.client.1.smithi176.stdout:2/11: write f3 [462347,95811] 0 2024-07-16T09:55:23.665 INFO:tasks.workunit.client.1.smithi176.stdout:5/10: getdents . 0 2024-07-16T09:55:23.665 INFO:tasks.workunit.client.1.smithi176.stdout:5/11: readlink - no filename 2024-07-16T09:55:23.667 INFO:tasks.workunit.client.1.smithi176.stdout:1/13: mkdir d4 0 2024-07-16T09:55:23.669 INFO:tasks.workunit.client.1.smithi176.stdout:5/12: creat f2 x:0 0 0 2024-07-16T09:55:23.669 INFO:tasks.workunit.client.1.smithi176.stdout:5/13: dread - f2 zero size 2024-07-16T09:55:23.670 INFO:tasks.workunit.client.1.smithi176.stdout:1/14: unlink f3 0 2024-07-16T09:55:23.673 INFO:tasks.workunit.client.1.smithi176.stdout:5/14: unlink c0 0 2024-07-16T09:55:23.673 INFO:tasks.workunit.client.1.smithi176.stdout:8/6: fsync f1 0 2024-07-16T09:55:23.673 INFO:tasks.workunit.client.1.smithi176.stdout:5/15: read - f2 zero size 2024-07-16T09:55:23.674 INFO:tasks.workunit.client.1.smithi176.stdout:5/16: write f2 [1002840,11440] 0 2024-07-16T09:55:23.674 INFO:tasks.workunit.client.1.smithi176.stdout:1/15: mkdir d4/d5 0 2024-07-16T09:55:23.676 INFO:tasks.workunit.client.1.smithi176.stdout:5/17: write f1 [70669,130491] 0 2024-07-16T09:55:23.677 INFO:tasks.workunit.client.1.smithi176.stdout:1/16: symlink d4/l6 0 2024-07-16T09:55:23.678 INFO:tasks.workunit.client.1.smithi176.stdout:6/6: truncate f0 16027 0 2024-07-16T09:55:23.678 INFO:tasks.workunit.client.1.smithi176.stdout:5/18: write f1 [904000,37034] 0 2024-07-16T09:55:23.681 INFO:tasks.workunit.client.1.smithi176.stdout:4/13: dwrite f0 [4194304,4194304] 0 2024-07-16T09:55:23.681 INFO:tasks.workunit.client.1.smithi176.stdout:5/19: mknod c3 0 2024-07-16T09:55:23.682 INFO:tasks.workunit.client.1.smithi176.stdout:4/14: read f0 [4694205,127254] 0 2024-07-16T09:55:23.683 INFO:tasks.workunit.client.1.smithi176.stdout:0/24: getdents d5 0 2024-07-16T09:55:23.683 INFO:tasks.workunit.client.1.smithi176.stdout:0/25: readlink l2 0 2024-07-16T09:55:23.683 INFO:tasks.workunit.client.1.smithi176.stdout:7/6: truncate f0 640605 0 2024-07-16T09:55:23.683 INFO:tasks.workunit.client.1.smithi176.stdout:0/26: dread - f3 zero size 2024-07-16T09:55:23.686 INFO:tasks.workunit.client.1.smithi176.stdout:5/20: mkdir d4 0 2024-07-16T09:55:23.687 INFO:tasks.workunit.client.1.smithi176.stdout:4/15: symlink l7 0 2024-07-16T09:55:23.687 INFO:tasks.workunit.client.1.smithi176.stdout:1/17: link c2 d4/c7 0 2024-07-16T09:55:23.691 INFO:tasks.workunit.client.1.smithi176.stdout:7/7: symlink l2 0 2024-07-16T09:55:23.691 INFO:tasks.workunit.client.1.smithi176.stdout:4/16: dread f1 [0,4194304] 0 2024-07-16T09:55:23.692 INFO:tasks.workunit.client.1.smithi176.stdout:0/27: symlink d5/l9 0 2024-07-16T09:55:23.693 INFO:tasks.workunit.client.1.smithi176.stdout:0/28: read - f1 zero size 2024-07-16T09:55:23.695 INFO:tasks.workunit.client.1.smithi176.stdout:4/17: dread f3 [0,4194304] 0 2024-07-16T09:55:23.696 INFO:tasks.workunit.client.1.smithi176.stdout:1/18: rename d4/d5 to d4/d8 0 2024-07-16T09:55:23.698 INFO:tasks.workunit.client.1.smithi176.stdout:7/8: mknod c3 0 2024-07-16T09:55:23.699 INFO:tasks.workunit.client.1.smithi176.stdout:5/21: creat d4/f5 x:0 0 0 2024-07-16T09:55:23.700 INFO:tasks.workunit.client.1.smithi176.stdout:0/29: mknod d5/ca 0 2024-07-16T09:55:23.701 INFO:tasks.workunit.client.1.smithi176.stdout:0/30: dread - f1 zero size 2024-07-16T09:55:23.703 INFO:tasks.workunit.client.1.smithi176.stdout:0/31: chown f3 249 1 2024-07-16T09:55:23.704 INFO:tasks.workunit.client.1.smithi176.stdout:4/18: dwrite f2 [0,4194304] 0 2024-07-16T09:55:23.709 INFO:tasks.workunit.client.1.smithi176.stdout:0/32: dread d5/f8 [0,4194304] 0 2024-07-16T09:55:23.709 INFO:tasks.workunit.client.1.smithi176.stdout:4/19: dread f2 [0,4194304] 0 2024-07-16T09:55:23.710 INFO:tasks.workunit.client.1.smithi176.stdout:0/33: chown f1 10722817 1 2024-07-16T09:55:23.712 INFO:tasks.workunit.client.1.smithi176.stdout:1/19: link c2 d4/d8/c9 0 2024-07-16T09:55:23.716 INFO:tasks.workunit.client.1.smithi176.stdout:1/20: dread f1 [0,4194304] 0 2024-07-16T09:55:23.725 INFO:tasks.workunit.client.1.smithi176.stdout:1/21: creat d4/d8/fa x:0 0 0 2024-07-16T09:55:23.726 INFO:tasks.workunit.client.1.smithi176.stdout:4/20: link f2 f8 0 2024-07-16T09:55:23.728 INFO:tasks.workunit.client.1.smithi176.stdout:5/22: getdents d4 0 2024-07-16T09:55:23.729 INFO:tasks.workunit.client.1.smithi176.stdout:5/23: write d4/f5 [649387,29101] 0 2024-07-16T09:55:23.730 INFO:tasks.workunit.client.1.smithi176.stdout:4/21: symlink l9 0 2024-07-16T09:55:23.731 INFO:tasks.workunit.client.1.smithi176.stdout:5/24: creat d4/f6 x:0 0 0 2024-07-16T09:55:23.732 INFO:tasks.workunit.client.1.smithi176.stdout:4/22: mkdir da 0 2024-07-16T09:55:23.732 INFO:tasks.workunit.client.1.smithi176.stdout:4/23: read f2 [2935735,114629] 0 2024-07-16T09:55:23.733 INFO:tasks.workunit.client.1.smithi176.stdout:5/25: creat d4/f7 x:0 0 0 2024-07-16T09:55:23.733 INFO:tasks.workunit.client.1.smithi176.stdout:6/7: link f0 d1/f2 0 2024-07-16T09:55:23.734 INFO:tasks.workunit.client.1.smithi176.stdout:4/24: read f0 [26283,76865] 0 2024-07-16T09:55:23.734 INFO:tasks.workunit.client.1.smithi176.stdout:7/9: write f0 [367156,119746] 0 2024-07-16T09:55:23.741 INFO:tasks.workunit.client.1.smithi176.stdout:1/22: truncate f1 1853980 0 2024-07-16T09:55:23.742 INFO:tasks.workunit.client.1.smithi176.stdout:7/10: dread f0 [0,4194304] 0 2024-07-16T09:55:23.745 INFO:tasks.workunit.client.1.smithi176.stdout:1/23: fdatasync d4/d8/fa 0 2024-07-16T09:55:23.746 INFO:tasks.workunit.client.1.smithi176.stdout:6/8: write d1/f2 [22727,31518] 0 2024-07-16T09:55:23.747 INFO:tasks.workunit.client.1.smithi176.stdout:0/34: dwrite f1 [0,4194304] 0 2024-07-16T09:55:23.747 INFO:tasks.workunit.client.1.smithi176.stdout:5/26: truncate f2 536468 0 2024-07-16T09:55:23.749 INFO:tasks.workunit.client.1.smithi176.stdout:7/11: write f0 [649945,57196] 0 2024-07-16T09:55:23.749 INFO:tasks.workunit.client.1.smithi176.stdout:4/25: dread f2 [0,4194304] 0 2024-07-16T09:55:23.750 INFO:tasks.workunit.client.1.smithi176.stdout:0/35: creat d5/fb x:0 0 0 2024-07-16T09:55:23.751 INFO:tasks.workunit.client.1.smithi176.stdout:0/36: read f1 [2928152,68082] 0 2024-07-16T09:55:23.752 INFO:tasks.workunit.client.1.smithi176.stdout:5/27: symlink d4/l8 0 2024-07-16T09:55:23.752 INFO:tasks.workunit.client.1.smithi176.stdout:7/12: mkdir d4 0 2024-07-16T09:55:23.752 INFO:tasks.workunit.client.1.smithi176.stdout:0/37: dread - d5/f7 zero size 2024-07-16T09:55:23.753 INFO:tasks.workunit.client.1.smithi176.stdout:6/9: symlink d1/l3 0 2024-07-16T09:55:23.755 INFO:tasks.workunit.client.1.smithi176.stdout:5/28: symlink d4/l9 0 2024-07-16T09:55:23.757 INFO:tasks.workunit.client.1.smithi176.stdout:0/38: symlink d5/lc 0 2024-07-16T09:55:23.760 INFO:tasks.workunit.client.1.smithi176.stdout:5/29: symlink d4/la 0 2024-07-16T09:55:23.763 INFO:tasks.workunit.client.1.smithi176.stdout:6/10: getdents d1 0 2024-07-16T09:55:23.928 INFO:tasks.workunit.client.1.smithi176.stdout:5/30: creat d4/fb x:0 0 0 2024-07-16T09:55:23.934 INFO:tasks.workunit.client.1.smithi176.stdout:0/39: dwrite d5/fb [0,4194304] 0 2024-07-16T09:55:23.944 INFO:tasks.workunit.client.1.smithi176.stdout:6/11: dwrite d1/f2 [0,4194304] 0 2024-07-16T09:55:23.946 INFO:tasks.workunit.client.1.smithi176.stdout:6/12: chown d1/l3 102004 1 2024-07-16T09:55:23.949 INFO:tasks.workunit.client.1.smithi176.stdout:4/26: fdatasync f8 0 2024-07-16T09:55:23.950 INFO:tasks.workunit.client.1.smithi176.stdout:6/13: link d1/l3 d1/l4 0 2024-07-16T09:55:23.953 INFO:tasks.workunit.client.1.smithi176.stdout:7/13: dread f0 [0,4194304] 0 2024-07-16T09:55:23.953 INFO:tasks.workunit.client.1.smithi176.stdout:5/31: getdents d4 0 2024-07-16T09:55:23.953 INFO:tasks.workunit.client.1.smithi176.stdout:6/14: read d1/f2 [1229449,119042] 0 2024-07-16T09:55:23.953 INFO:tasks.workunit.client.1.smithi176.stdout:7/14: read f0 [634658,30339] 0 2024-07-16T09:55:23.954 INFO:tasks.workunit.client.1.smithi176.stdout:0/40: write f4 [221938,126004] 0 2024-07-16T09:55:23.955 INFO:tasks.workunit.client.1.smithi176.stdout:7/15: chown d4 3 1 2024-07-16T09:55:23.958 INFO:tasks.workunit.client.1.smithi176.stdout:0/41: rmdir d5 39 2024-07-16T09:55:23.963 INFO:tasks.workunit.client.1.smithi176.stdout:7/16: dread f0 [0,4194304] 0 2024-07-16T09:55:23.964 INFO:tasks.workunit.client.1.smithi176.stdout:4/27: dread f3 [0,4194304] 0 2024-07-16T09:55:23.965 INFO:tasks.workunit.client.1.smithi176.stdout:0/42: chown d5/lc 4304220 1 2024-07-16T09:55:23.968 INFO:tasks.workunit.client.1.smithi176.stdout:7/17: mkdir d4/d5 0 2024-07-16T09:55:23.969 INFO:tasks.workunit.client.1.smithi176.stdout:5/32: dwrite d4/f6 [0,4194304] 0 2024-07-16T09:55:23.970 INFO:tasks.workunit.client.1.smithi176.stdout:0/43: dread f3 [0,4194304] 0 2024-07-16T09:55:23.970 INFO:tasks.workunit.client.1.smithi176.stdout:7/18: fdatasync f0 0 2024-07-16T09:55:23.972 INFO:tasks.workunit.client.1.smithi176.stdout:0/44: rmdir d5 39 2024-07-16T09:55:23.975 INFO:tasks.workunit.client.1.smithi176.stdout:5/33: creat d4/fc x:0 0 0 2024-07-16T09:55:23.976 INFO:tasks.workunit.client.1.smithi176.stdout:6/15: dwrite f0 [0,4194304] 0 2024-07-16T09:55:23.979 INFO:tasks.workunit.client.1.smithi176.stdout:7/19: dread f0 [0,4194304] 0 2024-07-16T09:55:23.981 INFO:tasks.workunit.client.1.smithi176.stdout:6/16: mkdir d1/d5 0 2024-07-16T09:55:23.982 INFO:tasks.workunit.client.1.smithi176.stdout:5/34: creat d4/fd x:0 0 0 2024-07-16T09:55:23.983 INFO:tasks.workunit.client.1.smithi176.stdout:4/28: dwrite f1 [4194304,4194304] 0 2024-07-16T09:55:23.986 INFO:tasks.workunit.client.1.smithi176.stdout:5/35: stat d4/fd 0 2024-07-16T09:55:23.986 INFO:tasks.workunit.client.1.smithi176.stdout:7/20: creat d4/f6 x:0 0 0 2024-07-16T09:55:23.987 INFO:tasks.workunit.client.1.smithi176.stdout:4/29: chown l9 151 1 2024-07-16T09:55:23.989 INFO:tasks.workunit.client.1.smithi176.stdout:7/21: truncate d4/f6 754328 0 2024-07-16T09:55:23.990 INFO:tasks.workunit.client.1.smithi176.stdout:5/36: chown d4/l9 236529539 1 2024-07-16T09:55:23.992 INFO:tasks.workunit.client.1.smithi176.stdout:0/45: dwrite f3 [0,4194304] 0 2024-07-16T09:55:23.995 INFO:tasks.workunit.client.1.smithi176.stdout:6/17: link d1/f2 d1/d5/f6 0 2024-07-16T09:55:23.996 INFO:tasks.workunit.client.1.smithi176.stdout:6/18: chown f0 5201 1 2024-07-16T09:55:23.997 INFO:tasks.workunit.client.1.smithi176.stdout:5/37: rename d4/f7 to d4/fe 0 2024-07-16T09:55:23.999 INFO:tasks.workunit.client.1.smithi176.stdout:7/22: symlink d4/d5/l7 0 2024-07-16T09:55:24.002 INFO:tasks.workunit.client.1.smithi176.stdout:4/30: truncate f0 8386188 0 2024-07-16T09:55:24.003 INFO:tasks.workunit.client.1.smithi176.stdout:5/38: creat d4/ff x:0 0 0 2024-07-16T09:55:24.004 INFO:tasks.workunit.client.1.smithi176.stdout:6/19: dread f0 [0,4194304] 0 2024-07-16T09:55:24.009 INFO:tasks.workunit.client.1.smithi176.stdout:5/39: truncate d4/ff 29211 0 2024-07-16T09:55:24.018 INFO:tasks.workunit.client.1.smithi176.stdout:6/20: truncate f0 2183769 0 2024-07-16T09:55:24.018 INFO:tasks.workunit.client.1.smithi176.stdout:7/23: dwrite d4/f6 [0,4194304] 0 2024-07-16T09:55:24.020 INFO:tasks.workunit.client.1.smithi176.stdout:0/46: dwrite d5/fb [4194304,4194304] 0 2024-07-16T09:55:24.027 INFO:tasks.workunit.client.1.smithi176.stdout:6/21: mkdir d1/d5/d7 0 2024-07-16T09:55:24.030 INFO:tasks.workunit.client.1.smithi176.stdout:0/47: mknod d5/cd 0 2024-07-16T09:55:24.030 INFO:tasks.workunit.client.1.smithi176.stdout:4/31: dwrite f8 [0,4194304] 0 2024-07-16T09:55:24.030 INFO:tasks.workunit.client.1.smithi176.stdout:5/40: dwrite d4/f6 [0,4194304] 0 2024-07-16T09:55:24.033 INFO:tasks.workunit.client.1.smithi176.stdout:4/32: chown l9 2808 1 2024-07-16T09:55:24.035 INFO:tasks.workunit.client.1.smithi176.stdout:0/48: unlink d5/cd 0 2024-07-16T09:55:24.036 INFO:tasks.workunit.client.1.smithi176.stdout:0/49: chown d5/fb 133043897 1 2024-07-16T09:55:24.037 INFO:tasks.workunit.client.1.smithi176.stdout:0/50: dread - d5/f7 zero size 2024-07-16T09:55:24.038 INFO:tasks.workunit.client.1.smithi176.stdout:7/24: dwrite d4/f6 [4194304,4194304] 0 2024-07-16T09:55:24.038 INFO:tasks.workunit.client.1.smithi176.stdout:5/41: unlink d4/l9 0 2024-07-16T09:55:24.042 INFO:tasks.workunit.client.1.smithi176.stdout:0/51: mknod d5/ce 0 2024-07-16T09:55:24.042 INFO:tasks.workunit.client.1.smithi176.stdout:5/42: write d4/f5 [844033,58489] 0 2024-07-16T09:55:24.044 INFO:tasks.workunit.client.1.smithi176.stdout:4/33: creat da/fb x:0 0 0 2024-07-16T09:55:24.044 INFO:tasks.workunit.client.1.smithi176.stdout:5/43: readlink d4/l8 0 2024-07-16T09:55:24.055 INFO:tasks.workunit.client.1.smithi176.stdout:5/44: dwrite d4/f6 [4194304,4194304] 0 2024-07-16T09:55:24.057 INFO:tasks.workunit.client.1.smithi176.stdout:6/22: dread f0 [0,4194304] 0 2024-07-16T09:55:24.058 INFO:tasks.workunit.client.1.smithi176.stdout:5/45: mkdir d4/d10 0 2024-07-16T09:55:24.060 INFO:tasks.workunit.client.1.smithi176.stdout:6/23: dread f0 [0,4194304] 0 2024-07-16T09:55:24.062 INFO:tasks.workunit.client.1.smithi176.stdout:6/24: readlink d1/l4 0 2024-07-16T09:55:24.063 INFO:tasks.workunit.client.1.smithi176.stdout:6/25: readlink d1/l4 0 2024-07-16T09:55:24.063 INFO:tasks.workunit.client.1.smithi176.stdout:0/52: rmdir d5 39 2024-07-16T09:55:24.065 INFO:tasks.workunit.client.1.smithi176.stdout:6/26: unlink f0 0 2024-07-16T09:55:24.067 INFO:tasks.workunit.client.1.smithi176.stdout:4/34: getdents da 0 2024-07-16T09:55:24.068 INFO:tasks.workunit.client.1.smithi176.stdout:4/35: fsync f2 0 2024-07-16T09:55:24.068 INFO:tasks.workunit.client.1.smithi176.stdout:5/46: dread d4/f5 [0,4194304] 0 2024-07-16T09:55:24.069 INFO:tasks.workunit.client.1.smithi176.stdout:4/36: rmdir da 39 2024-07-16T09:55:24.070 INFO:tasks.workunit.client.1.smithi176.stdout:5/47: write d4/f6 [2316764,130351] 0 2024-07-16T09:55:24.071 INFO:tasks.workunit.client.1.smithi176.stdout:5/48: stat d4/fe 0 2024-07-16T09:55:24.072 INFO:tasks.workunit.client.1.smithi176.stdout:4/37: mkdir da/dc 0 2024-07-16T09:55:24.073 INFO:tasks.workunit.client.1.smithi176.stdout:5/49: mknod d4/d10/c11 0 2024-07-16T09:55:24.076 INFO:tasks.workunit.client.1.smithi176.stdout:0/53: write d5/f8 [4689822,28034] 0 2024-07-16T09:55:24.078 INFO:tasks.workunit.client.1.smithi176.stdout:4/38: dread f8 [0,4194304] 0 2024-07-16T09:55:24.079 INFO:tasks.workunit.client.1.smithi176.stdout:0/54: fdatasync d5/f8 0 2024-07-16T09:55:24.079 INFO:tasks.workunit.client.1.smithi176.stdout:5/50: link d4/fb d4/d10/f12 0 2024-07-16T09:55:24.080 INFO:tasks.workunit.client.1.smithi176.stdout:0/55: chown d5/ce 256791 1 2024-07-16T09:55:24.080 INFO:tasks.workunit.client.1.smithi176.stdout:6/27: truncate d1/f2 1598044 0 2024-07-16T09:55:24.081 INFO:tasks.workunit.client.1.smithi176.stdout:4/39: fdatasync da/fb 0 2024-07-16T09:55:24.082 INFO:tasks.workunit.client.1.smithi176.stdout:0/56: chown f1 993372 1 2024-07-16T09:55:24.084 INFO:tasks.workunit.client.1.smithi176.stdout:4/40: getdents da/dc 0 2024-07-16T09:55:24.089 INFO:tasks.workunit.client.1.smithi176.stdout:5/51: getdents d4 0 2024-07-16T09:55:24.089 INFO:tasks.workunit.client.1.smithi176.stdout:4/41: mknod da/cd 0 2024-07-16T09:55:24.090 INFO:tasks.workunit.client.1.smithi176.stdout:4/42: chown da 4526728 1 2024-07-16T09:55:24.090 INFO:tasks.workunit.client.1.smithi176.stdout:5/52: creat d4/f13 x:0 0 0 2024-07-16T09:55:24.091 INFO:tasks.workunit.client.1.smithi176.stdout:0/57: dwrite d5/fb [0,4194304] 0 2024-07-16T09:55:24.095 INFO:tasks.workunit.client.1.smithi176.stdout:5/53: mkdir d4/d14 0 2024-07-16T09:55:24.097 INFO:tasks.workunit.client.1.smithi176.stdout:0/58: creat d5/ff x:0 0 0 2024-07-16T09:55:24.097 INFO:tasks.workunit.client.1.smithi176.stdout:4/43: dread f0 [4194304,4194304] 0 2024-07-16T09:55:24.098 INFO:tasks.workunit.client.1.smithi176.stdout:0/59: read d5/f6 [3965372,111386] 0 2024-07-16T09:55:24.098 INFO:tasks.workunit.client.1.smithi176.stdout:5/54: fdatasync d4/fb 0 2024-07-16T09:55:24.099 INFO:tasks.workunit.client.1.smithi176.stdout:0/60: readlink d5/lc 0 2024-07-16T09:55:24.100 INFO:tasks.workunit.client.1.smithi176.stdout:5/55: write d4/f6 [5460406,20551] 0 2024-07-16T09:55:24.102 INFO:tasks.workunit.client.1.smithi176.stdout:4/44: truncate f8 2094230 0 2024-07-16T09:55:24.103 INFO:tasks.workunit.client.1.smithi176.stdout:0/61: creat d5/f10 x:0 0 0 2024-07-16T09:55:24.116 INFO:tasks.workunit.client.1.smithi176.stdout:5/56: dwrite f1 [0,4194304] 0 2024-07-16T09:55:24.117 INFO:tasks.workunit.client.1.smithi176.stdout:4/45: dwrite f3 [4194304,4194304] 0 2024-07-16T09:55:24.120 INFO:tasks.workunit.client.1.smithi176.stdout:6/28: dwrite d1/f2 [0,4194304] 0 2024-07-16T09:55:24.122 INFO:tasks.workunit.client.1.smithi176.stdout:0/62: dread d5/f8 [0,4194304] 0 2024-07-16T09:55:24.122 INFO:tasks.workunit.client.1.smithi176.stdout:0/63: stat d5/f8 0 2024-07-16T09:55:24.123 INFO:tasks.workunit.client.1.smithi176.stdout:4/46: chown f8 207 1 2024-07-16T09:55:24.125 INFO:tasks.workunit.client.1.smithi176.stdout:4/47: write f1 [1562351,37843] 0 2024-07-16T09:55:24.128 INFO:tasks.workunit.client.1.smithi176.stdout:5/57: link d4/fc d4/d10/f15 0 2024-07-16T09:55:24.135 INFO:tasks.workunit.client.1.smithi176.stdout:0/64: dwrite d5/f10 [0,4194304] 0 2024-07-16T09:55:24.136 INFO:tasks.workunit.client.1.smithi176.stdout:5/58: truncate d4/fb 824880 0 2024-07-16T09:55:24.141 INFO:tasks.workunit.client.1.smithi176.stdout:4/48: write f2 [3095404,48811] 0 2024-07-16T09:55:24.141 INFO:tasks.workunit.client.1.smithi176.stdout:5/59: mknod d4/d10/c16 0 2024-07-16T09:55:24.144 INFO:tasks.workunit.client.1.smithi176.stdout:6/29: dwrite d1/d5/f6 [4194304,4194304] 0 2024-07-16T09:55:24.146 INFO:tasks.workunit.client.1.smithi176.stdout:4/49: chown l4 1307618 1 2024-07-16T09:55:24.146 INFO:tasks.workunit.client.1.smithi176.stdout:4/50: chown da 0 1 2024-07-16T09:55:24.149 INFO:tasks.workunit.client.1.smithi176.stdout:5/60: mkdir d4/d10/d17 0 2024-07-16T09:55:24.154 INFO:tasks.workunit.client.1.smithi176.stdout:0/65: getdents d5 0 2024-07-16T09:55:24.157 INFO:tasks.workunit.client.1.smithi176.stdout:0/66: chown d5/f6 81968347 1 2024-07-16T09:55:24.157 INFO:tasks.workunit.client.1.smithi176.stdout:0/67: read f3 [1366406,120926] 0 2024-07-16T09:55:24.157 INFO:tasks.workunit.client.1.smithi176.stdout:4/51: dread f8 [0,4194304] 0 2024-07-16T09:55:24.158 INFO:tasks.workunit.client.1.smithi176.stdout:0/68: read f3 [2092644,126502] 0 2024-07-16T09:55:24.161 INFO:tasks.workunit.client.1.smithi176.stdout:4/52: link f0 da/dc/fe 0 2024-07-16T09:55:24.162 INFO:tasks.workunit.client.1.smithi176.stdout:5/61: truncate d4/f6 5563786 0 2024-07-16T09:55:24.163 INFO:tasks.workunit.client.1.smithi176.stdout:4/53: unlink l6 0 2024-07-16T09:55:24.163 INFO:tasks.workunit.client.1.smithi176.stdout:6/30: fsync d1/f2 0 2024-07-16T09:55:24.164 INFO:tasks.workunit.client.1.smithi176.stdout:0/69: dread d5/f8 [0,4194304] 0 2024-07-16T09:55:24.165 INFO:tasks.workunit.client.1.smithi176.stdout:5/62: creat d4/d10/d17/f18 x:0 0 0 2024-07-16T09:55:24.165 INFO:tasks.workunit.client.1.smithi176.stdout:4/54: fdatasync f8 0 2024-07-16T09:55:24.166 INFO:tasks.workunit.client.1.smithi176.stdout:5/63: chown f1 153763 1 2024-07-16T09:55:24.166 INFO:tasks.workunit.client.1.smithi176.stdout:0/70: mknod d5/c11 0 2024-07-16T09:55:24.168 INFO:tasks.workunit.client.1.smithi176.stdout:0/71: chown d5/l9 8113 1 2024-07-16T09:55:24.168 INFO:tasks.workunit.client.1.smithi176.stdout:6/31: symlink d1/d5/l8 0 2024-07-16T09:55:24.170 INFO:tasks.workunit.client.1.smithi176.stdout:5/64: mkdir d4/d14/d19 0 2024-07-16T09:55:24.171 INFO:tasks.workunit.client.1.smithi176.stdout:5/65: stat d4/d10 0 2024-07-16T09:55:24.171 INFO:tasks.workunit.client.1.smithi176.stdout:4/55: link da/cd da/cf 0 2024-07-16T09:55:24.174 INFO:tasks.workunit.client.1.smithi176.stdout:4/56: stat da/dc 0 2024-07-16T09:55:24.177 INFO:tasks.workunit.client.1.smithi176.stdout:6/32: creat d1/d5/d7/f9 x:0 0 0 2024-07-16T09:55:24.179 INFO:tasks.workunit.client.1.smithi176.stdout:5/66: getdents d4/d10 0 2024-07-16T09:55:24.179 INFO:tasks.workunit.client.1.smithi176.stdout:0/72: dwrite d5/f7 [0,4194304] 0 2024-07-16T09:55:24.182 INFO:tasks.workunit.client.1.smithi176.stdout:6/33: write d1/d5/d7/f9 [989490,16695] 0 2024-07-16T09:55:24.182 INFO:tasks.workunit.client.1.smithi176.stdout:5/67: mknod d4/c1a 0 2024-07-16T09:55:24.187 INFO:tasks.workunit.client.1.smithi176.stdout:0/73: chown d5/ff 4411389 1 2024-07-16T09:55:24.188 INFO:tasks.workunit.client.1.smithi176.stdout:4/57: dread da/dc/fe [4194304,4194304] 0 2024-07-16T09:55:24.188 INFO:tasks.workunit.client.1.smithi176.stdout:5/68: mkdir d4/d14/d19/d1b 0 2024-07-16T09:55:24.188 INFO:tasks.workunit.client.1.smithi176.stdout:5/69: chown d4/f13 22768075 1 2024-07-16T09:55:24.190 INFO:tasks.workunit.client.1.smithi176.stdout:6/34: symlink d1/d5/la 0 2024-07-16T09:55:24.190 INFO:tasks.workunit.client.1.smithi176.stdout:0/74: creat d5/f12 x:0 0 0 2024-07-16T09:55:24.191 INFO:tasks.workunit.client.1.smithi176.stdout:4/58: unlink f2 0 2024-07-16T09:55:24.192 INFO:tasks.workunit.client.1.smithi176.stdout:4/59: write f1 [8999374,80939] 0 2024-07-16T09:55:24.193 INFO:tasks.workunit.client.1.smithi176.stdout:6/35: readlink d1/l4 0 2024-07-16T09:55:24.195 INFO:tasks.workunit.client.1.smithi176.stdout:4/60: truncate f0 7304763 0 2024-07-16T09:55:24.198 INFO:tasks.workunit.client.1.smithi176.stdout:6/36: symlink d1/d5/d7/lb 0 2024-07-16T09:55:24.200 INFO:tasks.workunit.client.1.smithi176.stdout:4/61: mkdir da/dc/d10 0 2024-07-16T09:55:24.201 INFO:tasks.workunit.client.1.smithi176.stdout:6/37: fsync d1/f2 0 2024-07-16T09:55:24.204 INFO:tasks.workunit.client.1.smithi176.stdout:5/70: dwrite d4/f5 [0,4194304] 0 2024-07-16T09:55:24.206 INFO:tasks.workunit.client.1.smithi176.stdout:5/71: write d4/f5 [2251126,120450] 0 2024-07-16T09:55:24.207 INFO:tasks.workunit.client.1.smithi176.stdout:6/38: truncate d1/d5/f6 3403365 0 2024-07-16T09:55:24.209 INFO:tasks.workunit.client.1.smithi176.stdout:6/39: chown d1/l4 0 1 2024-07-16T09:55:24.209 INFO:tasks.workunit.client.1.smithi176.stdout:5/72: dread - d4/d10/f15 zero size 2024-07-16T09:55:24.210 INFO:tasks.workunit.client.1.smithi176.stdout:0/75: getdents d5 0 2024-07-16T09:55:24.216 INFO:tasks.workunit.client.1.smithi176.stdout:0/76: creat d5/f13 x:0 0 0 2024-07-16T09:55:24.221 INFO:tasks.workunit.client.1.smithi176.stdout:6/40: mkdir d1/dc 0 2024-07-16T09:55:24.222 INFO:tasks.workunit.client.1.smithi176.stdout:0/77: creat d5/f14 x:0 0 0 2024-07-16T09:55:24.222 INFO:tasks.workunit.client.1.smithi176.stdout:6/41: chown d1/l4 72047393 1 2024-07-16T09:55:24.223 INFO:tasks.workunit.client.1.smithi176.stdout:5/73: getdents d4/d14/d19/d1b 0 2024-07-16T09:55:24.223 INFO:tasks.workunit.client.1.smithi176.stdout:0/78: fdatasync d5/f13 0 2024-07-16T09:55:24.226 INFO:tasks.workunit.client.1.smithi176.stdout:0/79: write d5/ff [159290,36078] 0 2024-07-16T09:55:24.226 INFO:tasks.workunit.client.1.smithi176.stdout:6/42: creat d1/dc/fd x:0 0 0 2024-07-16T09:55:24.228 INFO:tasks.workunit.client.1.smithi176.stdout:0/80: read d5/f8 [3788482,23025] 0 2024-07-16T09:55:24.230 INFO:tasks.workunit.client.1.smithi176.stdout:5/74: dread f2 [0,4194304] 0 2024-07-16T09:55:24.231 INFO:tasks.workunit.client.1.smithi176.stdout:5/75: chown c3 173555 1 2024-07-16T09:55:24.233 INFO:tasks.workunit.client.1.smithi176.stdout:4/62: dwrite da/dc/fe [0,4194304] 0 2024-07-16T09:55:24.234 INFO:tasks.workunit.client.1.smithi176.stdout:6/43: write d1/d5/f6 [1685248,23172] 0 2024-07-16T09:55:24.234 INFO:tasks.workunit.client.1.smithi176.stdout:4/63: readlink l4 0 2024-07-16T09:55:24.236 INFO:tasks.workunit.client.1.smithi176.stdout:6/44: readlink d1/l3 0 2024-07-16T09:55:24.238 INFO:tasks.workunit.client.1.smithi176.stdout:0/81: dread d5/f6 [0,4194304] 0 2024-07-16T09:55:24.242 INFO:tasks.workunit.client.1.smithi176.stdout:6/45: creat d1/d5/d7/fe x:0 0 0 2024-07-16T09:55:24.242 INFO:tasks.workunit.client.1.smithi176.stdout:5/76: truncate d4/f5 1359327 0 2024-07-16T09:55:24.247 INFO:tasks.workunit.client.1.smithi176.stdout:0/82: dwrite d5/f12 [0,4194304] 0 2024-07-16T09:55:24.247 INFO:tasks.workunit.client.1.smithi176.stdout:5/77: symlink d4/d14/l1c 0 2024-07-16T09:55:24.248 INFO:tasks.workunit.client.1.smithi176.stdout:0/83: chown f1 5179857 1 2024-07-16T09:55:24.249 INFO:tasks.workunit.client.1.smithi176.stdout:5/78: fdatasync d4/f13 0 2024-07-16T09:55:24.250 INFO:tasks.workunit.client.1.smithi176.stdout:5/79: stat d4/d14/l1c 0 2024-07-16T09:55:24.251 INFO:tasks.workunit.client.1.smithi176.stdout:5/80: dread - d4/fc zero size 2024-07-16T09:55:24.253 INFO:tasks.workunit.client.1.smithi176.stdout:5/81: dread - d4/d10/f15 zero size 2024-07-16T09:55:24.254 INFO:tasks.workunit.client.1.smithi176.stdout:6/46: rmdir d1/d5/d7 39 2024-07-16T09:55:24.254 INFO:tasks.workunit.client.1.smithi176.stdout:4/64: truncate f0 1403497 0 2024-07-16T09:55:24.255 INFO:tasks.workunit.client.1.smithi176.stdout:0/84: dread d5/f12 [0,4194304] 0 2024-07-16T09:55:24.255 INFO:tasks.workunit.client.1.smithi176.stdout:6/47: fsync d1/dc/fd 0 2024-07-16T09:55:24.256 INFO:tasks.workunit.client.1.smithi176.stdout:5/82: truncate d4/d10/f15 642816 0 2024-07-16T09:55:24.256 INFO:tasks.workunit.client.1.smithi176.stdout:0/85: chown d5/f10 199730033 1 2024-07-16T09:55:24.283 INFO:tasks.workunit.client.1.smithi176.stdout:0/86: creat d5/f15 x:0 0 0 2024-07-16T09:55:24.290 INFO:tasks.workunit.client.1.smithi176.stdout:6/48: dwrite d1/d5/d7/f9 [0,4194304] 0 2024-07-16T09:55:24.439 INFO:tasks.workunit.client.1.smithi176.stdout:4/65: mknod da/dc/d10/c11 0 2024-07-16T09:55:24.442 INFO:tasks.workunit.client.1.smithi176.stdout:6/49: symlink d1/lf 0 2024-07-16T09:55:24.444 INFO:tasks.workunit.client.1.smithi176.stdout:6/50: write d1/d5/d7/fe [792041,72642] 0 2024-07-16T09:55:24.448 INFO:tasks.workunit.client.1.smithi176.stdout:0/87: dwrite f4 [0,4194304] 0 2024-07-16T09:55:24.452 INFO:tasks.workunit.client.1.smithi176.stdout:0/88: link f3 d5/f16 0 2024-07-16T09:55:24.454 INFO:tasks.workunit.client.1.smithi176.stdout:0/89: mknod d5/c17 0 2024-07-16T09:55:24.454 INFO:tasks.workunit.client.1.smithi176.stdout:0/90: fdatasync d5/f8 0 2024-07-16T09:55:24.515 INFO:tasks.workunit.client.1.smithi176.stdout:4/66: truncate f3 5117242 0 2024-07-16T09:55:24.515 INFO:tasks.workunit.client.1.smithi176.stdout:6/51: write d1/d5/d7/f9 [4193702,47455] 0 2024-07-16T09:55:24.518 INFO:tasks.workunit.client.1.smithi176.stdout:6/52: creat d1/dc/f10 x:0 0 0 2024-07-16T09:55:24.522 INFO:tasks.workunit.client.1.smithi176.stdout:4/67: rename f1 to da/dc/d10/f12 0 2024-07-16T09:55:24.524 INFO:tasks.workunit.client.1.smithi176.stdout:5/83: dwrite d4/f5 [0,4194304] 0 2024-07-16T09:55:24.526 INFO:tasks.workunit.client.1.smithi176.stdout:6/53: getdents d1/d5/d7 0 2024-07-16T09:55:24.528 INFO:tasks.workunit.client.1.smithi176.stdout:4/68: rename da/cf to da/c13 0 2024-07-16T09:55:24.539 INFO:tasks.workunit.client.1.smithi176.stdout:6/54: truncate d1/d5/f6 3316315 0 2024-07-16T09:55:24.542 INFO:tasks.workunit.client.1.smithi176.stdout:4/69: dwrite da/dc/fe [0,4194304] 0 2024-07-16T09:55:24.543 INFO:tasks.workunit.client.1.smithi176.stdout:6/55: chown d1/d5/l8 322 1 2024-07-16T09:55:24.545 INFO:tasks.workunit.client.1.smithi176.stdout:5/84: dwrite d4/d10/f15 [0,4194304] 0 2024-07-16T09:55:24.548 INFO:tasks.workunit.client.1.smithi176.stdout:4/70: mkdir da/d14 0 2024-07-16T09:55:24.551 INFO:tasks.workunit.client.1.smithi176.stdout:5/85: write d4/fc [741105,115842] 0 2024-07-16T09:55:24.553 INFO:tasks.workunit.client.1.smithi176.stdout:4/71: rmdir da/dc 39 2024-07-16T09:55:24.553 INFO:tasks.workunit.client.1.smithi176.stdout:5/86: stat d4/d10/d17/f18 0 2024-07-16T09:55:24.556 INFO:tasks.workunit.client.1.smithi176.stdout:6/56: dwrite d1/dc/f10 [0,4194304] 0 2024-07-16T09:55:24.557 INFO:tasks.workunit.client.1.smithi176.stdout:4/72: chown l4 0 1 2024-07-16T09:55:24.561 INFO:tasks.workunit.client.1.smithi176.stdout:5/87: symlink d4/d10/d17/l1d 0 2024-07-16T09:55:24.561 INFO:tasks.workunit.client.1.smithi176.stdout:6/57: rmdir d1/d5/d7 39 2024-07-16T09:55:24.565 INFO:tasks.workunit.client.1.smithi176.stdout:5/88: fdatasync d4/fb 0 2024-07-16T09:55:24.568 INFO:tasks.workunit.client.1.smithi176.stdout:5/89: symlink d4/l1e 0 2024-07-16T09:55:24.569 INFO:tasks.workunit.client.1.smithi176.stdout:5/90: write d4/d10/d17/f18 [359799,94894] 0 2024-07-16T09:55:24.571 INFO:tasks.workunit.client.1.smithi176.stdout:5/91: write d4/f13 [553937,21450] 0 2024-07-16T09:55:24.571 INFO:tasks.workunit.client.1.smithi176.stdout:5/92: fdatasync d4/f5 0 2024-07-16T09:55:24.572 INFO:tasks.workunit.client.1.smithi176.stdout:5/93: fsync d4/ff 0 2024-07-16T09:55:24.572 INFO:tasks.workunit.client.1.smithi176.stdout:4/73: fdatasync f0 0 2024-07-16T09:55:24.575 INFO:tasks.workunit.client.1.smithi176.stdout:5/94: symlink d4/d14/l1f 0 2024-07-16T09:55:24.575 INFO:tasks.workunit.client.1.smithi176.stdout:5/95: readlink d4/l1e 0 2024-07-16T09:55:24.577 INFO:tasks.workunit.client.1.smithi176.stdout:6/58: fdatasync d1/dc/f10 0 2024-07-16T09:55:24.577 INFO:tasks.workunit.client.1.smithi176.stdout:5/96: symlink d4/d10/d17/l20 0 2024-07-16T09:55:24.578 INFO:tasks.workunit.client.1.smithi176.stdout:5/97: readlink d4/d10/d17/l1d 0 2024-07-16T09:55:24.580 INFO:tasks.workunit.client.1.smithi176.stdout:6/59: symlink d1/d5/l11 0 2024-07-16T09:55:24.580 INFO:tasks.workunit.client.1.smithi176.stdout:4/74: dread f8 [0,4194304] 0 2024-07-16T09:55:24.581 INFO:tasks.workunit.client.1.smithi176.stdout:5/98: creat d4/d14/d19/f21 x:0 0 0 2024-07-16T09:55:24.582 INFO:tasks.workunit.client.1.smithi176.stdout:6/60: creat d1/dc/f12 x:0 0 0 2024-07-16T09:55:24.587 INFO:tasks.workunit.client.1.smithi176.stdout:4/75: truncate da/dc/fe 3264146 0 2024-07-16T09:55:24.592 INFO:tasks.workunit.client.1.smithi176.stdout:6/61: getdents d1/dc 0 2024-07-16T09:55:24.592 INFO:tasks.workunit.client.1.smithi176.stdout:5/99: write f1 [4108795,96818] 0 2024-07-16T09:55:24.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:55:24 smithi176 ceph-mon[36626]: pgmap v243: 65 pgs: 65 active+clean; 190 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1.6 MiB/s rd, 4.8 MiB/s wr, 450 op/s 2024-07-16T09:55:24.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:55:24 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:55:24.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:55:24 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:55:24.595 INFO:tasks.workunit.client.1.smithi176.stdout:6/62: write d1/d5/d7/fe [1111937,62452] 0 2024-07-16T09:55:24.597 INFO:tasks.workunit.client.1.smithi176.stdout:5/100: rename f1 to d4/d10/f22 0 2024-07-16T09:55:24.602 INFO:tasks.workunit.client.1.smithi176.stdout:6/63: link d1/d5/d7/fe d1/d5/f13 0 2024-07-16T09:55:24.607 INFO:tasks.workunit.client.1.smithi176.stdout:6/64: dread d1/dc/f10 [0,4194304] 0 2024-07-16T09:55:24.609 INFO:tasks.workunit.client.1.smithi176.stdout:6/65: write d1/dc/fd [355618,47491] 0 2024-07-16T09:55:24.611 INFO:tasks.workunit.client.1.smithi176.stdout:6/66: write d1/dc/f12 [647033,54268] 0 2024-07-16T09:55:24.612 INFO:tasks.workunit.client.1.smithi176.stdout:5/101: dwrite d4/f6 [4194304,4194304] 0 2024-07-16T09:55:24.615 INFO:tasks.workunit.client.1.smithi176.stdout:5/102: unlink c3 0 2024-07-16T09:55:24.616 INFO:tasks.workunit.client.1.smithi176.stdout:6/67: mknod d1/c14 0 2024-07-16T09:55:24.617 INFO:tasks.workunit.client.1.smithi176.stdout:5/103: mknod d4/d10/d17/c23 0 2024-07-16T09:55:24.619 INFO:tasks.workunit.client.1.smithi176.stdout:5/104: dread - d4/fd zero size 2024-07-16T09:55:24.624 INFO:tasks.workunit.client.1.smithi176.stdout:6/68: dwrite d1/d5/d7/f9 [0,4194304] 0 2024-07-16T09:55:24.625 INFO:tasks.workunit.client.1.smithi176.stdout:5/105: link d4/d14/l1c d4/d14/l24 0 2024-07-16T09:55:24.627 INFO:tasks.workunit.client.1.smithi176.stdout:5/106: creat d4/d14/f25 x:0 0 0 2024-07-16T09:55:24.628 INFO:tasks.workunit.client.1.smithi176.stdout:5/107: fdatasync d4/fd 0 2024-07-16T09:55:24.629 INFO:tasks.workunit.client.1.smithi176.stdout:6/69: creat d1/f15 x:0 0 0 2024-07-16T09:55:24.636 INFO:tasks.workunit.client.1.smithi176.stdout:6/70: rename d1/f2 to d1/d5/d7/f16 0 2024-07-16T09:55:24.638 INFO:tasks.workunit.client.1.smithi176.stdout:5/108: dwrite d4/fe [0,4194304] 0 2024-07-16T09:55:24.640 INFO:tasks.workunit.client.1.smithi176.stdout:6/71: creat d1/dc/f17 x:0 0 0 2024-07-16T09:55:24.640 INFO:tasks.workunit.client.1.smithi176.stdout:5/109: chown d4/d14 7996890 1 2024-07-16T09:55:24.641 INFO:tasks.workunit.client.1.smithi176.stdout:5/110: readlink d4/la 0 2024-07-16T09:55:24.642 INFO:tasks.workunit.client.1.smithi176.stdout:6/72: read d1/d5/f13 [560993,76216] 0 2024-07-16T09:55:24.643 INFO:tasks.workunit.client.1.smithi176.stdout:5/111: dread - d4/d14/f25 zero size 2024-07-16T09:55:24.643 INFO:tasks.workunit.client.1.smithi176.stdout:6/73: chown d1/l4 24 1 2024-07-16T09:55:24.646 INFO:tasks.workunit.client.1.smithi176.stdout:6/74: rmdir d1/d5/d7 39 2024-07-16T09:55:24.648 INFO:tasks.workunit.client.1.smithi176.stdout:5/112: dread d4/d10/f15 [0,4194304] 0 2024-07-16T09:55:24.650 INFO:tasks.workunit.client.1.smithi176.stdout:5/113: fdatasync d4/d10/d17/f18 0 2024-07-16T09:55:24.654 INFO:tasks.workunit.client.1.smithi176.stdout:6/75: rename d1/d5/d7/f16 to d1/f18 0 2024-07-16T09:55:24.655 INFO:tasks.workunit.client.1.smithi176.stdout:5/114: getdents d4/d10/d17 0 2024-07-16T09:55:24.656 INFO:tasks.workunit.client.1.smithi176.stdout:6/76: creat d1/d5/d7/f19 x:0 0 0 2024-07-16T09:55:24.658 INFO:tasks.workunit.client.1.smithi176.stdout:5/115: creat d4/d14/d19/d1b/f26 x:0 0 0 2024-07-16T09:55:24.659 INFO:tasks.workunit.client.1.smithi176.stdout:6/77: symlink d1/d5/l1a 0 2024-07-16T09:55:24.661 INFO:tasks.workunit.client.1.smithi176.stdout:6/78: symlink d1/dc/l1b 0 2024-07-16T09:55:24.662 INFO:tasks.workunit.client.1.smithi176.stdout:6/79: mkdir d1/dc/d1c 0 2024-07-16T09:55:24.664 INFO:tasks.workunit.client.1.smithi176.stdout:6/80: chown d1/d5/f6 727565168 1 2024-07-16T09:55:24.665 INFO:tasks.workunit.client.1.smithi176.stdout:5/116: truncate d4/f13 278833 0 2024-07-16T09:55:24.666 INFO:tasks.workunit.client.1.smithi176.stdout:6/81: mkdir d1/d5/d7/d1d 0 2024-07-16T09:55:24.668 INFO:tasks.workunit.client.1.smithi176.stdout:6/82: write d1/dc/f17 [375389,52359] 0 2024-07-16T09:55:24.670 INFO:tasks.workunit.client.1.smithi176.stdout:5/117: dread d4/f6 [4194304,4194304] 0 2024-07-16T09:55:24.673 INFO:tasks.workunit.client.1.smithi176.stdout:5/118: write d4/fc [2457485,75161] 0 2024-07-16T09:55:24.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:55:24 smithi156 ceph-mon[29302]: pgmap v243: 65 pgs: 65 active+clean; 190 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1.6 MiB/s rd, 4.8 MiB/s wr, 450 op/s 2024-07-16T09:55:24.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:55:24 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:55:24.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:55:24 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:55:24.675 INFO:tasks.workunit.client.1.smithi176.stdout:5/119: creat d4/f27 x:0 0 0 2024-07-16T09:55:24.678 INFO:tasks.workunit.client.1.smithi176.stdout:5/120: getdents d4/d10/d17 0 2024-07-16T09:55:24.681 INFO:tasks.workunit.client.1.smithi176.stdout:5/121: dread d4/ff [0,4194304] 0 2024-07-16T09:55:24.682 INFO:tasks.workunit.client.1.smithi176.stdout:6/83: dwrite d1/d5/d7/fe [0,4194304] 0 2024-07-16T09:55:24.682 INFO:tasks.workunit.client.1.smithi176.stdout:6/84: read - d1/f15 zero size 2024-07-16T09:55:24.684 INFO:tasks.workunit.client.1.smithi176.stdout:5/122: truncate d4/d14/d19/f21 581233 0 2024-07-16T09:55:24.692 INFO:tasks.workunit.client.1.smithi176.stdout:6/85: dwrite d1/d5/f13 [0,4194304] 0 2024-07-16T09:55:24.697 INFO:tasks.workunit.client.1.smithi176.stdout:6/86: rename d1/d5/d7/d1d to d1/d5/d7/d1e 0 2024-07-16T09:55:24.697 INFO:tasks.workunit.client.1.smithi176.stdout:5/123: write d4/f13 [273756,37764] 0 2024-07-16T09:55:24.698 INFO:tasks.workunit.client.1.smithi176.stdout:6/87: fsync d1/dc/f12 0 2024-07-16T09:55:24.699 INFO:tasks.workunit.client.1.smithi176.stdout:6/88: read d1/d5/f13 [4030005,26403] 0 2024-07-16T09:55:24.699 INFO:tasks.workunit.client.1.smithi176.stdout:6/89: readlink d1/lf 0 2024-07-16T09:55:24.703 INFO:tasks.workunit.client.1.smithi176.stdout:5/124: rename d4/fb to d4/d10/d17/f28 0 2024-07-16T09:55:24.707 INFO:tasks.workunit.client.1.smithi176.stdout:5/125: write d4/f6 [7805475,65280] 0 2024-07-16T09:55:24.708 INFO:tasks.workunit.client.1.smithi176.stdout:5/126: chown d4/d14/d19 18 1 2024-07-16T09:55:24.709 INFO:tasks.workunit.client.1.smithi176.stdout:5/127: dread - d4/fd zero size 2024-07-16T09:55:24.711 INFO:tasks.workunit.client.1.smithi176.stdout:6/90: dwrite d1/d5/d7/f19 [0,4194304] 0 2024-07-16T09:55:24.713 INFO:tasks.workunit.client.1.smithi176.stdout:5/128: symlink d4/d10/d17/l29 0 2024-07-16T09:55:24.715 INFO:tasks.workunit.client.1.smithi176.stdout:5/129: write d4/fd [243938,3397] 0 2024-07-16T09:55:24.720 INFO:tasks.workunit.client.1.smithi176.stdout:6/91: dwrite d1/d5/d7/f19 [0,4194304] 0 2024-07-16T09:55:24.721 INFO:tasks.workunit.client.1.smithi176.stdout:6/92: readlink d1/d5/l8 0 2024-07-16T09:55:24.721 INFO:tasks.workunit.client.1.smithi176.stdout:5/130: link d4/l1e d4/d14/l2a 0 2024-07-16T09:55:24.723 INFO:tasks.workunit.client.1.smithi176.stdout:5/131: dread - d4/d14/f25 zero size 2024-07-16T09:55:24.723 INFO:tasks.workunit.client.1.smithi176.stdout:6/93: rmdir d1/d5 39 2024-07-16T09:55:24.726 INFO:tasks.workunit.client.1.smithi176.stdout:6/94: unlink d1/dc/f10 0 2024-07-16T09:55:24.732 INFO:tasks.workunit.client.1.smithi176.stdout:5/132: link d4/d10/c11 d4/d10/d17/c2b 0 2024-07-16T09:55:24.733 INFO:tasks.workunit.client.1.smithi176.stdout:5/133: write d4/d14/f25 [611486,89706] 0 2024-07-16T09:55:24.734 INFO:tasks.workunit.client.1.smithi176.stdout:6/95: rename d1/d5/f13 to d1/dc/d1c/f1f 0 2024-07-16T09:55:24.736 INFO:tasks.workunit.client.1.smithi176.stdout:5/134: readlink d4/d14/l2a 0 2024-07-16T09:55:24.736 INFO:tasks.workunit.client.1.smithi176.stdout:6/96: chown d1/dc/fd 49922406 1 2024-07-16T09:55:24.737 INFO:tasks.workunit.client.1.smithi176.stdout:6/97: readlink d1/d5/l1a 0 2024-07-16T09:55:24.738 INFO:tasks.workunit.client.1.smithi176.stdout:5/135: creat d4/d14/f2c x:0 0 0 2024-07-16T09:55:24.745 INFO:tasks.workunit.client.1.smithi176.stdout:5/136: write d4/d10/f12 [1356583,39672] 0 2024-07-16T09:55:24.751 INFO:tasks.workunit.client.1.smithi176.stdout:5/137: dwrite d4/d10/f12 [0,4194304] 0 2024-07-16T09:55:24.752 INFO:tasks.workunit.client.1.smithi176.stdout:5/138: stat d4/d10/d17/l29 0 2024-07-16T09:55:24.754 INFO:tasks.workunit.client.1.smithi176.stdout:5/139: symlink d4/l2d 0 2024-07-16T09:55:24.757 INFO:tasks.workunit.client.1.smithi176.stdout:5/140: mknod d4/d10/d17/c2e 0 2024-07-16T09:55:24.760 INFO:tasks.workunit.client.1.smithi176.stdout:5/141: dread d4/f13 [0,4194304] 0 2024-07-16T09:55:24.762 INFO:tasks.workunit.client.1.smithi176.stdout:5/142: creat d4/d10/d17/f2f x:0 0 0 2024-07-16T09:55:24.764 INFO:tasks.workunit.client.1.smithi176.stdout:5/143: symlink d4/d14/d19/l30 0 2024-07-16T09:55:24.766 INFO:tasks.workunit.client.1.smithi176.stdout:5/144: creat d4/d10/f31 x:0 0 0 2024-07-16T09:55:24.768 INFO:tasks.workunit.client.1.smithi176.stdout:5/145: creat d4/d10/d17/f32 x:0 0 0 2024-07-16T09:55:24.770 INFO:tasks.workunit.client.1.smithi176.stdout:5/146: creat d4/d10/f33 x:0 0 0 2024-07-16T09:55:24.777 INFO:tasks.workunit.client.1.smithi176.stdout:5/147: dwrite d4/d14/d19/f21 [0,4194304] 0 2024-07-16T09:55:24.779 INFO:tasks.workunit.client.1.smithi176.stdout:5/148: dread - d4/d10/f33 zero size 2024-07-16T09:55:24.781 INFO:tasks.workunit.client.1.smithi176.stdout:5/149: mknod d4/c34 0 2024-07-16T09:55:24.783 INFO:tasks.workunit.client.1.smithi176.stdout:5/150: creat d4/d14/d19/f35 x:0 0 0 2024-07-16T09:55:24.783 INFO:tasks.workunit.client.1.smithi176.stdout:5/151: fsync d4/ff 0 2024-07-16T09:55:24.784 INFO:tasks.workunit.client.1.smithi176.stdout:5/152: read - d4/d14/f2c zero size 2024-07-16T09:55:24.785 INFO:tasks.workunit.client.1.smithi176.stdout:5/153: fsync d4/d10/d17/f2f 0 2024-07-16T09:55:24.788 INFO:tasks.workunit.client.1.smithi176.stdout:5/154: mkdir d4/d14/d19/d1b/d36 0 2024-07-16T09:55:24.796 INFO:tasks.workunit.client.1.smithi176.stdout:5/155: dwrite d4/d14/d19/f21 [0,4194304] 0 2024-07-16T09:55:24.799 INFO:tasks.workunit.client.1.smithi176.stdout:5/156: mkdir d4/d37 0 2024-07-16T09:55:24.802 INFO:tasks.workunit.client.1.smithi176.stdout:5/157: mkdir d4/d14/d19/d1b/d36/d38 0 2024-07-16T09:55:24.802 INFO:tasks.workunit.client.1.smithi176.stdout:5/158: chown d4/d10/f12 18742956 1 2024-07-16T09:55:24.803 INFO:tasks.workunit.client.1.smithi176.stdout:5/159: truncate d4/d14/f2c 852410 0 2024-07-16T09:55:25.924 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:55:25 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:55:25.924 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:55:25 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:55:25.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:55:25 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-16T09:55:25.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:55:25 smithi156 ceph-mon[29302]: pgmap v244: 65 pgs: 65 active+clean; 223 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.0 MiB/s rd, 7.3 MiB/s wr, 490 op/s 2024-07-16T09:55:26.093 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:55:25 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:55:26.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:55:25 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:55:26.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:55:25 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-16T09:55:26.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:55:25 smithi176 ceph-mon[36626]: pgmap v244: 65 pgs: 65 active+clean; 223 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.0 MiB/s rd, 7.3 MiB/s wr, 490 op/s 2024-07-16T09:55:27.326 INFO:tasks.workunit.client.0.smithi156.stderr:+ pushd ltp-full-20091231/testcases/kernel/fs/fsstress 2024-07-16T09:55:27.334 INFO:tasks.workunit.client.0.smithi156.stdout:~/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress ~/cephtest/mnt.0/client.0/tmp/fsstress ~/cephtest/mnt.0/client.0/tmp 2024-07-16T09:55:27.335 INFO:tasks.workunit.client.0.smithi156.stderr:+ make 2024-07-16T09:55:27.372 INFO:tasks.workunit.client.0.smithi156.stdout:cc -DNO_XFS -I/home/ubuntu/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress -D_LARGEFILE64_SOURCE -D_GNU_SOURCE -I../../../../include -I../../../../include -L../../../../lib fsstress.c -o fsstress 2024-07-16T09:55:27.824 INFO:tasks.workunit.client.0.smithi156.stderr:++ readlink -f fsstress 2024-07-16T09:55:27.826 INFO:tasks.workunit.client.0.smithi156.stderr:+ BIN=/home/ubuntu/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress 2024-07-16T09:55:27.826 INFO:tasks.workunit.client.0.smithi156.stderr:+ popd 2024-07-16T09:55:27.826 INFO:tasks.workunit.client.0.smithi156.stdout:~/cephtest/mnt.0/client.0/tmp/fsstress ~/cephtest/mnt.0/client.0/tmp 2024-07-16T09:55:27.827 INFO:tasks.workunit.client.0.smithi156.stderr:+ popd 2024-07-16T09:55:27.827 INFO:tasks.workunit.client.0.smithi156.stdout:~/cephtest/mnt.0/client.0/tmp 2024-07-16T09:55:27.827 INFO:tasks.workunit.client.0.smithi156.stderr:++ mktemp -d -p . 2024-07-16T09:55:27.829 INFO:tasks.workunit.client.0.smithi156.stderr:+ T=./tmp.6LAWRNGYoj 2024-07-16T09:55:27.829 INFO:tasks.workunit.client.0.smithi156.stderr:+ /home/ubuntu/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress -d ./tmp.6LAWRNGYoj -l 1 -n 1000 -p 10 -v 2024-07-16T09:55:27.832 INFO:tasks.workunit.client.0.smithi156.stdout:seed = 1721430375 2024-07-16T09:55:27.837 INFO:tasks.workunit.client.0.smithi156.stdout:0/0: getdents . 0 2024-07-16T09:55:27.837 INFO:tasks.workunit.client.0.smithi156.stdout:0/1: dwrite - no filename 2024-07-16T09:55:27.837 INFO:tasks.workunit.client.0.smithi156.stdout:0/2: dwrite - no filename 2024-07-16T09:55:27.837 INFO:tasks.workunit.client.0.smithi156.stdout:0/3: dread - no filename 2024-07-16T09:55:27.838 INFO:tasks.workunit.client.0.smithi156.stdout:1/0: symlink l0 0 2024-07-16T09:55:27.838 INFO:tasks.workunit.client.0.smithi156.stdout:0/4: mkdir d0 0 2024-07-16T09:55:27.838 INFO:tasks.workunit.client.0.smithi156.stdout:0/5: dwrite - no filename 2024-07-16T09:55:27.838 INFO:tasks.workunit.client.0.smithi156.stdout:0/6: truncate - no filename 2024-07-16T09:55:27.838 INFO:tasks.workunit.client.0.smithi156.stdout:0/7: chown d0 495483776 1 2024-07-16T09:55:27.838 INFO:tasks.workunit.client.0.smithi156.stdout:0/8: dwrite - no filename 2024-07-16T09:55:27.840 INFO:tasks.workunit.client.0.smithi156.stdout:1/1: rename l0 to l1 0 2024-07-16T09:55:27.840 INFO:tasks.workunit.client.0.smithi156.stdout:2/0: mkdir d0 0 2024-07-16T09:55:27.842 INFO:tasks.workunit.client.0.smithi156.stdout:0/9: creat d0/f1 x:0 0 0 2024-07-16T09:55:27.843 INFO:tasks.workunit.client.0.smithi156.stdout:3/0: creat f0 x:0 0 0 2024-07-16T09:55:27.844 INFO:tasks.workunit.client.0.smithi156.stdout:2/1: creat d0/f1 x:0 0 0 2024-07-16T09:55:27.844 INFO:tasks.workunit.client.0.smithi156.stdout:0/10: mknod d0/c2 0 2024-07-16T09:55:27.845 INFO:tasks.workunit.client.0.smithi156.stdout:4/0: mknod c0 0 2024-07-16T09:55:27.846 INFO:tasks.workunit.client.0.smithi156.stdout:6/0: rename - no filename 2024-07-16T09:55:27.847 INFO:tasks.workunit.client.0.smithi156.stdout:2/2: symlink d0/l2 0 2024-07-16T09:55:27.848 INFO:tasks.workunit.client.0.smithi156.stdout:2/3: write d0/f1 [892196,110606] 0 2024-07-16T09:55:27.848 INFO:tasks.workunit.client.0.smithi156.stdout:0/11: rename d0/f1 to d0/f3 0 2024-07-16T09:55:27.848 INFO:tasks.workunit.client.0.smithi156.stdout:0/12: stat d0 0 2024-07-16T09:55:27.849 INFO:tasks.workunit.client.0.smithi156.stdout:0/13: dread - d0/f3 zero size 2024-07-16T09:55:27.850 INFO:tasks.workunit.client.0.smithi156.stdout:4/1: unlink c0 0 2024-07-16T09:55:27.850 INFO:tasks.workunit.client.0.smithi156.stdout:4/2: dread - no filename 2024-07-16T09:55:27.850 INFO:tasks.workunit.client.0.smithi156.stdout:4/3: chown . 53 1 2024-07-16T09:55:27.850 INFO:tasks.workunit.client.0.smithi156.stdout:4/4: dwrite - no filename 2024-07-16T09:55:27.850 INFO:tasks.workunit.client.0.smithi156.stdout:4/5: chown . 467670 1 2024-07-16T09:55:27.851 INFO:tasks.workunit.client.0.smithi156.stdout:5/0: mknod c0 0 2024-07-16T09:55:27.854 INFO:tasks.workunit.client.0.smithi156.stdout:2/4: symlink d0/l3 0 2024-07-16T09:55:27.856 INFO:tasks.workunit.client.0.smithi156.stdout:0/14: write d0/f3 [679887,69050] 0 2024-07-16T09:55:27.856 INFO:tasks.workunit.client.0.smithi156.stdout:4/6: creat f1 x:0 0 0 2024-07-16T09:55:27.857 INFO:tasks.workunit.client.0.smithi156.stdout:5/1: mkdir d1 0 2024-07-16T09:55:27.857 INFO:tasks.workunit.client.0.smithi156.stdout:5/2: fsync - no filename 2024-07-16T09:55:27.858 INFO:tasks.workunit.client.0.smithi156.stdout:6/1: creat f0 x:0 0 0 2024-07-16T09:55:27.859 INFO:tasks.workunit.client.0.smithi156.stdout:6/2: chown f0 387217948 1 2024-07-16T09:55:27.859 INFO:tasks.workunit.client.0.smithi156.stdout:8/0: dread - no filename 2024-07-16T09:55:27.860 INFO:tasks.workunit.client.0.smithi156.stdout:8/1: readlink - no filename 2024-07-16T09:55:27.860 INFO:tasks.workunit.client.0.smithi156.stdout:8/2: write - no filename 2024-07-16T09:55:27.860 INFO:tasks.workunit.client.0.smithi156.stdout:8/3: write - no filename 2024-07-16T09:55:27.860 INFO:tasks.workunit.client.0.smithi156.stdout:8/4: rename - no filename 2024-07-16T09:55:27.861 INFO:tasks.workunit.client.0.smithi156.stdout:0/15: dread d0/f3 [0,4194304] 0 2024-07-16T09:55:27.861 INFO:tasks.workunit.client.0.smithi156.stdout:2/5: unlink d0/f1 0 2024-07-16T09:55:27.861 INFO:tasks.workunit.client.0.smithi156.stdout:3/1: fsync f0 0 2024-07-16T09:55:27.862 INFO:tasks.workunit.client.0.smithi156.stdout:2/6: stat d0 0 2024-07-16T09:55:27.862 INFO:tasks.workunit.client.0.smithi156.stdout:2/7: fsync - no filename 2024-07-16T09:55:27.863 INFO:tasks.workunit.client.0.smithi156.stdout:4/7: creat f2 x:0 0 0 2024-07-16T09:55:27.863 INFO:tasks.workunit.client.0.smithi156.stdout:4/8: write f2 [822138,86523] 0 2024-07-16T09:55:27.865 INFO:tasks.workunit.client.0.smithi156.stdout:6/3: unlink f0 0 2024-07-16T09:55:27.865 INFO:tasks.workunit.client.0.smithi156.stdout:6/4: rename - no filename 2024-07-16T09:55:27.867 INFO:tasks.workunit.client.0.smithi156.stdout:7/0: creat f0 x:0 0 0 2024-07-16T09:55:27.867 INFO:tasks.workunit.client.0.smithi156.stdout:9/0: write - no filename 2024-07-16T09:55:27.868 INFO:tasks.workunit.client.0.smithi156.stdout:3/2: symlink l1 0 2024-07-16T09:55:27.868 INFO:tasks.workunit.client.0.smithi156.stdout:7/1: write f0 [990094,83164] 0 2024-07-16T09:55:27.869 INFO:tasks.workunit.client.0.smithi156.stdout:5/3: getdents d1 0 2024-07-16T09:55:27.870 INFO:tasks.workunit.client.0.smithi156.stdout:5/4: truncate - no filename 2024-07-16T09:55:27.870 INFO:tasks.workunit.client.0.smithi156.stdout:5/5: dread - no filename 2024-07-16T09:55:27.870 INFO:tasks.workunit.client.0.smithi156.stdout:1/2: link l1 l2 0 2024-07-16T09:55:27.870 INFO:tasks.workunit.client.0.smithi156.stdout:1/3: rmdir - no directory 2024-07-16T09:55:27.871 INFO:tasks.workunit.client.0.smithi156.stdout:6/5: creat f1 x:0 0 0 2024-07-16T09:55:27.871 INFO:tasks.workunit.client.0.smithi156.stdout:8/5: mkdir d0 0 2024-07-16T09:55:27.871 INFO:tasks.workunit.client.0.smithi156.stdout:8/6: fdatasync - no filename 2024-07-16T09:55:27.871 INFO:tasks.workunit.client.0.smithi156.stdout:8/7: dwrite - no filename 2024-07-16T09:55:27.871 INFO:tasks.workunit.client.0.smithi156.stdout:8/8: read - no filename 2024-07-16T09:55:27.872 INFO:tasks.workunit.client.0.smithi156.stdout:8/9: link - no file 2024-07-16T09:55:27.872 INFO:tasks.workunit.client.0.smithi156.stdout:8/10: read - no filename 2024-07-16T09:55:27.872 INFO:tasks.workunit.client.0.smithi156.stdout:3/3: symlink l2 0 2024-07-16T09:55:27.873 INFO:tasks.workunit.client.0.smithi156.stdout:5/6: unlink c0 0 2024-07-16T09:55:27.873 INFO:tasks.workunit.client.0.smithi156.stdout:4/9: dwrite f1 [0,4194304] 0 2024-07-16T09:55:27.873 INFO:tasks.workunit.client.0.smithi156.stdout:6/6: truncate f1 755538 0 2024-07-16T09:55:27.874 INFO:tasks.workunit.client.0.smithi156.stdout:3/4: write f0 [758346,126489] 0 2024-07-16T09:55:27.875 INFO:tasks.workunit.client.0.smithi156.stdout:1/4: mknod c3 0 2024-07-16T09:55:27.875 INFO:tasks.workunit.client.0.smithi156.stdout:1/5: write - no filename 2024-07-16T09:55:27.875 INFO:tasks.workunit.client.0.smithi156.stdout:1/6: dread - no filename 2024-07-16T09:55:27.876 INFO:tasks.workunit.client.0.smithi156.stdout:1/7: stat c3 0 2024-07-16T09:55:27.876 INFO:tasks.workunit.client.0.smithi156.stdout:1/8: fdatasync - no filename 2024-07-16T09:55:27.876 INFO:tasks.workunit.client.0.smithi156.stdout:1/9: write - no filename 2024-07-16T09:55:27.876 INFO:tasks.workunit.client.0.smithi156.stdout:1/10: dwrite - no filename 2024-07-16T09:55:27.877 INFO:tasks.workunit.client.0.smithi156.stdout:9/1: mknod c0 0 2024-07-16T09:55:27.878 INFO:tasks.workunit.client.0.smithi156.stdout:9/2: stat c0 0 2024-07-16T09:55:27.878 INFO:tasks.workunit.client.0.smithi156.stdout:9/3: dwrite - no filename 2024-07-16T09:55:27.878 INFO:tasks.workunit.client.0.smithi156.stdout:9/4: dwrite - no filename 2024-07-16T09:55:27.881 INFO:tasks.workunit.client.0.smithi156.stdout:6/7: mknod c2 0 2024-07-16T09:55:27.882 INFO:tasks.workunit.client.0.smithi156.stdout:3/5: dwrite f0 [0,4194304] 0 2024-07-16T09:55:27.883 INFO:tasks.workunit.client.0.smithi156.stdout:4/10: symlink l3 0 2024-07-16T09:55:27.884 INFO:tasks.workunit.client.0.smithi156.stdout:0/16: getdents d0 0 2024-07-16T09:55:27.885 INFO:tasks.workunit.client.0.smithi156.stdout:0/17: readlink - no filename 2024-07-16T09:55:27.889 INFO:tasks.workunit.client.0.smithi156.stdout:1/11: creat f4 x:0 0 0 2024-07-16T09:55:27.890 INFO:tasks.workunit.client.0.smithi156.stdout:9/5: creat f1 x:0 0 0 2024-07-16T09:55:27.891 INFO:tasks.workunit.client.0.smithi156.stdout:2/8: getdents d0 0 2024-07-16T09:55:27.892 INFO:tasks.workunit.client.0.smithi156.stdout:8/11: creat d0/f1 x:0 0 0 2024-07-16T09:55:27.893 INFO:tasks.workunit.client.0.smithi156.stdout:4/11: dwrite f2 [0,4194304] 0 2024-07-16T09:55:27.895 INFO:tasks.workunit.client.0.smithi156.stdout:6/8: mkdir d3 0 2024-07-16T09:55:27.896 INFO:tasks.workunit.client.0.smithi156.stdout:3/6: creat f3 x:0 0 0 2024-07-16T09:55:27.896 INFO:tasks.workunit.client.0.smithi156.stdout:3/7: rmdir - no directory 2024-07-16T09:55:27.899 INFO:tasks.workunit.client.0.smithi156.stdout:1/12: creat f5 x:0 0 0 2024-07-16T09:55:27.900 INFO:tasks.workunit.client.0.smithi156.stdout:1/13: chown f5 222579 1 2024-07-16T09:55:27.900 INFO:tasks.workunit.client.0.smithi156.stdout:9/6: mkdir d2 0 2024-07-16T09:55:27.901 INFO:tasks.workunit.client.0.smithi156.stdout:4/12: dwrite f2 [0,4194304] 0 2024-07-16T09:55:27.901 INFO:tasks.workunit.client.0.smithi156.stdout:1/14: write f4 [685995,128853] 0 2024-07-16T09:55:27.902 INFO:tasks.workunit.client.0.smithi156.stdout:4/13: chown f1 52411 1 2024-07-16T09:55:27.903 INFO:tasks.workunit.client.0.smithi156.stdout:2/9: rmdir d0 39 2024-07-16T09:55:27.903 INFO:tasks.workunit.client.0.smithi156.stdout:2/10: dwrite - no filename 2024-07-16T09:55:27.904 INFO:tasks.workunit.client.0.smithi156.stdout:5/7: rmdir d1 0 2024-07-16T09:55:27.908 INFO:tasks.workunit.client.0.smithi156.stdout:3/8: symlink l4 0 2024-07-16T09:55:27.910 INFO:tasks.workunit.client.0.smithi156.stdout:9/7: rename f1 to d2/f3 0 2024-07-16T09:55:27.911 INFO:tasks.workunit.client.0.smithi156.stdout:7/2: truncate f0 511454 0 2024-07-16T09:55:27.912 INFO:tasks.workunit.client.0.smithi156.stdout:1/15: creat f6 x:0 0 0 2024-07-16T09:55:27.913 INFO:tasks.workunit.client.0.smithi156.stdout:4/14: mkdir d4 0 2024-07-16T09:55:27.916 INFO:tasks.workunit.client.0.smithi156.stdout:5/8: creat f2 x:0 0 0 2024-07-16T09:55:27.917 INFO:tasks.workunit.client.0.smithi156.stdout:8/12: link d0/f1 d0/f2 0 2024-07-16T09:55:27.918 INFO:tasks.workunit.client.0.smithi156.stdout:6/9: mkdir d3/d4 0 2024-07-16T09:55:27.918 INFO:tasks.workunit.client.0.smithi156.stdout:6/10: chown f1 97291152 1 2024-07-16T09:55:27.919 INFO:tasks.workunit.client.0.smithi156.stdout:3/9: creat f5 x:0 0 0 2024-07-16T09:55:27.919 INFO:tasks.workunit.client.0.smithi156.stdout:0/18: write d0/f3 [1581416,8465] 0 2024-07-16T09:55:27.921 INFO:tasks.workunit.client.0.smithi156.stdout:9/8: write d2/f3 [765450,113886] 0 2024-07-16T09:55:27.922 INFO:tasks.workunit.client.0.smithi156.stdout:1/16: creat f7 x:0 0 0 2024-07-16T09:55:27.923 INFO:tasks.workunit.client.0.smithi156.stdout:4/15: unlink l3 0 2024-07-16T09:55:27.923 INFO:tasks.workunit.client.0.smithi156.stdout:2/11: rmdir d0 39 2024-07-16T09:55:27.924 INFO:tasks.workunit.client.0.smithi156.stdout:2/12: truncate - no filename 2024-07-16T09:55:27.924 INFO:tasks.workunit.client.0.smithi156.stdout:2/13: dwrite - no filename 2024-07-16T09:55:27.924 INFO:tasks.workunit.client.0.smithi156.stdout:9/9: fdatasync d2/f3 0 2024-07-16T09:55:27.924 INFO:tasks.workunit.client.0.smithi156.stdout:5/9: rename f2 to f3 0 2024-07-16T09:55:27.925 INFO:tasks.workunit.client.0.smithi156.stdout:0/19: dread d0/f3 [0,4194304] 0 2024-07-16T09:55:27.925 INFO:tasks.workunit.client.0.smithi156.stdout:1/17: write f5 [986618,11612] 0 2024-07-16T09:55:27.925 INFO:tasks.workunit.client.0.smithi156.stdout:8/13: mknod d0/c3 0 2024-07-16T09:55:27.926 INFO:tasks.workunit.client.0.smithi156.stdout:6/11: unlink f1 0 2024-07-16T09:55:27.926 INFO:tasks.workunit.client.0.smithi156.stdout:6/12: dread - no filename 2024-07-16T09:55:27.926 INFO:tasks.workunit.client.0.smithi156.stdout:6/13: dread - no filename 2024-07-16T09:55:27.928 INFO:tasks.workunit.client.0.smithi156.stdout:0/20: write d0/f3 [2032512,106034] 0 2024-07-16T09:55:27.929 INFO:tasks.workunit.client.0.smithi156.stdout:3/10: dwrite f3 [0,4194304] 0 2024-07-16T09:55:27.930 INFO:tasks.workunit.client.0.smithi156.stdout:9/10: creat d2/f4 x:0 0 0 2024-07-16T09:55:27.931 INFO:tasks.workunit.client.0.smithi156.stdout:9/11: dread - d2/f4 zero size 2024-07-16T09:55:27.932 INFO:tasks.workunit.client.0.smithi156.stdout:3/11: write f3 [2273539,104129] 0 2024-07-16T09:55:27.932 INFO:tasks.workunit.client.0.smithi156.stdout:1/18: mknod c8 0 2024-07-16T09:55:27.934 INFO:tasks.workunit.client.0.smithi156.stdout:4/16: creat d4/f5 x:0 0 0 2024-07-16T09:55:27.934 INFO:tasks.workunit.client.0.smithi156.stdout:8/14: mknod d0/c4 0 2024-07-16T09:55:27.935 INFO:tasks.workunit.client.0.smithi156.stdout:8/15: rename d0 to d0/d5 22 2024-07-16T09:55:27.936 INFO:tasks.workunit.client.0.smithi156.stdout:2/14: creat d0/f4 x:0 0 0 2024-07-16T09:55:27.937 INFO:tasks.workunit.client.0.smithi156.stdout:0/21: rmdir d0 39 2024-07-16T09:55:27.940 INFO:tasks.workunit.client.0.smithi156.stdout:1/19: mknod c9 0 2024-07-16T09:55:27.941 INFO:tasks.workunit.client.0.smithi156.stdout:3/12: rename l2 to l6 0 2024-07-16T09:55:27.943 INFO:tasks.workunit.client.0.smithi156.stdout:5/10: rename f3 to f4 0 2024-07-16T09:55:27.945 INFO:tasks.workunit.client.0.smithi156.stdout:2/15: mkdir d0/d5 0 2024-07-16T09:55:27.946 INFO:tasks.workunit.client.0.smithi156.stdout:9/12: dwrite d2/f4 [0,4194304] 0 2024-07-16T09:55:27.950 INFO:tasks.workunit.client.0.smithi156.stdout:4/17: dwrite f1 [4194304,4194304] 0 2024-07-16T09:55:27.952 INFO:tasks.workunit.client.0.smithi156.stdout:2/16: truncate d0/f4 1026422 0 2024-07-16T09:55:27.953 INFO:tasks.workunit.client.0.smithi156.stdout:6/14: getdents d3 0 2024-07-16T09:55:27.960 INFO:tasks.workunit.client.0.smithi156.stdout:5/11: fsync f4 0 2024-07-16T09:55:27.963 INFO:tasks.workunit.client.0.smithi156.stdout:4/18: creat d4/f6 x:0 0 0 2024-07-16T09:55:27.963 INFO:tasks.workunit.client.0.smithi156.stdout:3/13: dread f0 [0,4194304] 0 2024-07-16T09:55:27.979 INFO:tasks.workunit.client.0.smithi156.stdout:5/12: dread - f4 zero size 2024-07-16T09:55:27.980 INFO:tasks.workunit.client.0.smithi156.stdout:2/17: symlink d0/l6 0 2024-07-16T09:55:27.980 INFO:tasks.workunit.client.0.smithi156.stdout:0/22: dread d0/f3 [0,4194304] 0 2024-07-16T09:55:27.980 INFO:tasks.workunit.client.0.smithi156.stdout:1/20: dwrite f4 [0,4194304] 0 2024-07-16T09:55:27.981 INFO:tasks.workunit.client.0.smithi156.stdout:6/15: creat d3/d4/f5 x:0 0 0 2024-07-16T09:55:27.983 INFO:tasks.workunit.client.0.smithi156.stdout:6/16: dread - d3/d4/f5 zero size 2024-07-16T09:55:27.983 INFO:tasks.workunit.client.0.smithi156.stdout:1/21: write f7 [897009,80983] 0 2024-07-16T09:55:27.987 INFO:tasks.workunit.client.0.smithi156.stdout:9/13: dwrite d2/f4 [0,4194304] 0 2024-07-16T09:55:27.994 INFO:tasks.workunit.client.0.smithi156.stdout:9/14: dread d2/f3 [0,4194304] 0 2024-07-16T09:55:27.995 INFO:tasks.workunit.client.0.smithi156.stdout:6/17: dwrite d3/d4/f5 [0,4194304] 0 2024-07-16T09:55:27.995 INFO:tasks.workunit.client.0.smithi156.stdout:1/22: fdatasync f4 0 2024-07-16T09:55:27.996 INFO:tasks.workunit.client.0.smithi156.stdout:9/15: truncate d2/f4 5208037 0 2024-07-16T09:55:28.000 INFO:tasks.workunit.client.0.smithi156.stdout:6/18: dread d3/d4/f5 [0,4194304] 0 2024-07-16T09:55:28.003 INFO:tasks.workunit.client.0.smithi156.stdout:6/19: dread d3/d4/f5 [0,4194304] 0 2024-07-16T09:55:28.010 INFO:tasks.workunit.client.0.smithi156.stdout:6/20: dwrite d3/d4/f5 [0,4194304] 0 2024-07-16T09:55:28.013 INFO:tasks.workunit.client.0.smithi156.stdout:6/21: dread d3/d4/f5 [0,4194304] 0 2024-07-16T09:55:28.014 INFO:tasks.workunit.client.0.smithi156.stdout:6/22: write d3/d4/f5 [2160329,126246] 0 2024-07-16T09:55:28.146 INFO:tasks.workunit.client.0.smithi156.stdout:4/19: symlink d4/l7 0 2024-07-16T09:55:28.146 INFO:tasks.workunit.client.0.smithi156.stdout:2/18: creat d0/f7 x:0 0 0 2024-07-16T09:55:28.148 INFO:tasks.workunit.client.0.smithi156.stdout:4/20: read - d4/f5 zero size 2024-07-16T09:55:28.148 INFO:tasks.workunit.client.0.smithi156.stdout:5/13: rename f4 to f5 0 2024-07-16T09:55:28.148 INFO:tasks.workunit.client.0.smithi156.stdout:2/19: write d0/f4 [1180676,55333] 0 2024-07-16T09:55:28.148 INFO:tasks.workunit.client.0.smithi156.stdout:5/14: dread - f5 zero size 2024-07-16T09:55:28.148 INFO:tasks.workunit.client.0.smithi156.stdout:5/15: dread - f5 zero size 2024-07-16T09:55:28.149 INFO:tasks.workunit.client.0.smithi156.stdout:9/16: mkdir d2/d5 0 2024-07-16T09:55:28.151 INFO:tasks.workunit.client.0.smithi156.stdout:6/23: rename c2 to d3/d4/c6 0 2024-07-16T09:55:28.151 INFO:tasks.workunit.client.0.smithi156.stdout:9/17: dread d2/f3 [0,4194304] 0 2024-07-16T09:55:28.152 INFO:tasks.workunit.client.0.smithi156.stdout:4/21: creat d4/f8 x:0 0 0 2024-07-16T09:55:28.152 INFO:tasks.workunit.client.0.smithi156.stdout:2/20: creat d0/f8 x:0 0 0 2024-07-16T09:55:28.153 INFO:tasks.workunit.client.0.smithi156.stdout:5/16: mknod c6 0 2024-07-16T09:55:28.153 INFO:tasks.workunit.client.0.smithi156.stdout:4/22: write d4/f8 [420700,57619] 0 2024-07-16T09:55:28.154 INFO:tasks.workunit.client.0.smithi156.stdout:2/21: write d0/f8 [841348,70300] 0 2024-07-16T09:55:28.154 INFO:tasks.workunit.client.0.smithi156.stdout:1/23: rename l2 to la 0 2024-07-16T09:55:28.155 INFO:tasks.workunit.client.0.smithi156.stdout:1/24: read f5 [204354,63904] 0 2024-07-16T09:55:28.157 INFO:tasks.workunit.client.0.smithi156.stdout:1/25: write f4 [1260573,67792] 0 2024-07-16T09:55:28.158 INFO:tasks.workunit.client.0.smithi156.stdout:1/26: chown c3 116957586 1 2024-07-16T09:55:28.158 INFO:tasks.workunit.client.0.smithi156.stdout:9/18: mknod d2/c6 0 2024-07-16T09:55:28.159 INFO:tasks.workunit.client.0.smithi156.stdout:5/17: unlink c6 0 2024-07-16T09:55:28.160 INFO:tasks.workunit.client.0.smithi156.stdout:4/23: creat d4/f9 x:0 0 0 2024-07-16T09:55:28.160 INFO:tasks.workunit.client.0.smithi156.stdout:9/19: write d2/f4 [1714150,51257] 0 2024-07-16T09:55:28.162 INFO:tasks.workunit.client.0.smithi156.stdout:3/14: getdents . 0 2024-07-16T09:55:28.163 INFO:tasks.workunit.client.0.smithi156.stdout:5/18: symlink l7 0 2024-07-16T09:55:28.164 INFO:tasks.workunit.client.0.smithi156.stdout:5/19: readlink l7 0 2024-07-16T09:55:28.164 INFO:tasks.workunit.client.0.smithi156.stdout:3/15: chown f0 136 1 2024-07-16T09:55:28.164 INFO:tasks.workunit.client.0.smithi156.stdout:3/16: rmdir - no directory 2024-07-16T09:55:28.165 INFO:tasks.workunit.client.0.smithi156.stdout:4/24: mknod d4/ca 0 2024-07-16T09:55:28.165 INFO:tasks.workunit.client.0.smithi156.stdout:9/20: rename d2/c6 to d2/c7 0 2024-07-16T09:55:28.167 INFO:tasks.workunit.client.0.smithi156.stdout:9/21: stat d2 0 2024-07-16T09:55:28.167 INFO:tasks.workunit.client.0.smithi156.stdout:5/20: fdatasync f5 0 2024-07-16T09:55:28.168 INFO:tasks.workunit.client.0.smithi156.stdout:3/17: symlink l7 0 2024-07-16T09:55:28.169 INFO:tasks.workunit.client.0.smithi156.stdout:6/24: dwrite d3/d4/f5 [0,4194304] 0 2024-07-16T09:55:28.170 INFO:tasks.workunit.client.0.smithi156.stdout:5/21: fdatasync f5 0 2024-07-16T09:55:28.170 INFO:tasks.workunit.client.0.smithi156.stdout:4/25: mkdir d4/db 0 2024-07-16T09:55:28.170 INFO:tasks.workunit.client.0.smithi156.stdout:3/18: readlink l1 0 2024-07-16T09:55:28.171 INFO:tasks.workunit.client.0.smithi156.stdout:3/19: fsync f5 0 2024-07-16T09:55:28.172 INFO:tasks.workunit.client.0.smithi156.stdout:4/26: dread - d4/f9 zero size 2024-07-16T09:55:28.172 INFO:tasks.workunit.client.0.smithi156.stdout:1/27: dwrite f7 [0,4194304] 0 2024-07-16T09:55:28.173 INFO:tasks.workunit.client.0.smithi156.stdout:9/22: write d2/f4 [154265,15736] 0 2024-07-16T09:55:28.173 INFO:tasks.workunit.client.0.smithi156.stdout:3/20: stat f5 0 2024-07-16T09:55:28.174 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:55:27 smithi156 ceph-mon[29302]: pgmap v245: 65 pgs: 65 active+clean; 257 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.8 MiB/s rd, 11 MiB/s wr, 451 op/s 2024-07-16T09:55:28.178 INFO:tasks.workunit.client.0.smithi156.stdout:1/28: symlink lb 0 2024-07-16T09:55:28.179 INFO:tasks.workunit.client.0.smithi156.stdout:6/25: creat d3/d4/f7 x:0 0 0 2024-07-16T09:55:28.179 INFO:tasks.workunit.client.0.smithi156.stdout:4/27: dread d4/f8 [0,4194304] 0 2024-07-16T09:55:28.180 INFO:tasks.workunit.client.0.smithi156.stdout:1/29: stat c8 0 2024-07-16T09:55:28.184 INFO:tasks.workunit.client.0.smithi156.stdout:6/26: mkdir d3/d8 0 2024-07-16T09:55:28.185 INFO:tasks.workunit.client.0.smithi156.stdout:1/30: readlink l1 0 2024-07-16T09:55:28.188 INFO:tasks.workunit.client.0.smithi156.stdout:1/31: mkdir dc 0 2024-07-16T09:55:28.188 INFO:tasks.workunit.client.0.smithi156.stdout:9/23: dread d2/f4 [0,4194304] 0 2024-07-16T09:55:28.190 INFO:tasks.workunit.client.0.smithi156.stdout:5/22: dwrite f5 [0,4194304] 0 2024-07-16T09:55:28.194 INFO:tasks.workunit.client.0.smithi156.stdout:5/23: mknod c8 0 2024-07-16T09:55:28.196 INFO:tasks.workunit.client.0.smithi156.stdout:5/24: mknod c9 0 2024-07-16T09:55:28.204 INFO:tasks.workunit.client.0.smithi156.stdout:9/24: dwrite d2/f3 [0,4194304] 0 2024-07-16T09:55:28.205 INFO:tasks.workunit.client.0.smithi156.stdout:1/32: dwrite f5 [0,4194304] 0 2024-07-16T09:55:28.210 INFO:tasks.workunit.client.0.smithi156.stdout:1/33: creat dc/fd x:0 0 0 2024-07-16T09:55:28.211 INFO:tasks.workunit.client.0.smithi156.stdout:1/34: dread - dc/fd zero size 2024-07-16T09:55:28.211 INFO:tasks.workunit.client.0.smithi156.stdout:9/25: creat d2/d5/f8 x:0 0 0 2024-07-16T09:55:28.213 INFO:tasks.workunit.client.0.smithi156.stdout:1/35: symlink dc/le 0 2024-07-16T09:55:28.213 INFO:tasks.workunit.client.0.smithi156.stdout:9/26: write d2/f4 [2347014,11247] 0 2024-07-16T09:55:28.215 INFO:tasks.workunit.client.0.smithi156.stdout:1/36: rename c9 to dc/cf 0 2024-07-16T09:55:28.216 INFO:tasks.workunit.client.0.smithi156.stdout:9/27: dread d2/f4 [4194304,4194304] 0 2024-07-16T09:55:28.217 INFO:tasks.workunit.client.0.smithi156.stdout:1/37: creat dc/f10 x:0 0 0 2024-07-16T09:55:28.218 INFO:tasks.workunit.client.0.smithi156.stdout:9/28: symlink d2/d5/l9 0 2024-07-16T09:55:28.219 INFO:tasks.workunit.client.0.smithi156.stdout:9/29: readlink d2/d5/l9 0 2024-07-16T09:55:28.221 INFO:tasks.workunit.client.0.smithi156.stdout:1/38: link c3 dc/c11 0 2024-07-16T09:55:28.222 INFO:tasks.workunit.client.0.smithi156.stdout:1/39: chown f5 4 1 2024-07-16T09:55:28.222 INFO:tasks.workunit.client.0.smithi156.stdout:1/40: truncate dc/f10 923073 0 2024-07-16T09:55:28.224 INFO:tasks.workunit.client.0.smithi156.stdout:1/41: symlink dc/l12 0 2024-07-16T09:55:28.225 INFO:tasks.workunit.client.0.smithi156.stdout:1/42: creat dc/f13 x:0 0 0 2024-07-16T09:55:28.231 INFO:tasks.workunit.client.0.smithi156.stdout:1/43: dwrite f7 [0,4194304] 0 2024-07-16T09:55:28.232 INFO:tasks.workunit.client.0.smithi156.stdout:1/44: chown f6 3296 1 2024-07-16T09:55:28.271 INFO:tasks.workunit.client.0.smithi156.stdout:5/25: rename f5 to fa 0 2024-07-16T09:55:28.271 INFO:tasks.workunit.client.0.smithi156.stdout:2/22: getdents d0 0 2024-07-16T09:55:28.272 INFO:tasks.workunit.client.0.smithi156.stdout:5/26: creat fb x:0 0 0 2024-07-16T09:55:28.272 INFO:tasks.workunit.client.0.smithi156.stdout:5/27: readlink l7 0 2024-07-16T09:55:28.273 INFO:tasks.workunit.client.0.smithi156.stdout:2/23: mknod d0/c9 0 2024-07-16T09:55:28.273 INFO:tasks.workunit.client.0.smithi156.stdout:2/24: fdatasync d0/f8 0 2024-07-16T09:55:28.277 INFO:tasks.workunit.client.0.smithi156.stdout:4/28: truncate f2 3474340 0 2024-07-16T09:55:28.278 INFO:tasks.workunit.client.0.smithi156.stdout:6/27: truncate d3/d4/f5 2976547 0 2024-07-16T09:55:28.279 INFO:tasks.workunit.client.0.smithi156.stdout:4/29: mkdir d4/db/dc 0 2024-07-16T09:55:28.281 INFO:tasks.workunit.client.0.smithi156.stdout:4/30: unlink d4/ca 0 2024-07-16T09:55:28.283 INFO:tasks.workunit.client.0.smithi156.stdout:6/28: getdents d3 0 2024-07-16T09:55:28.284 INFO:tasks.workunit.client.0.smithi156.stdout:9/30: truncate d2/f4 2409643 0 2024-07-16T09:55:28.286 INFO:tasks.workunit.client.0.smithi156.stdout:6/29: unlink d3/d4/c6 0 2024-07-16T09:55:28.288 INFO:tasks.workunit.client.0.smithi156.stdout:4/31: dwrite d4/f8 [0,4194304] 0 2024-07-16T09:55:28.289 INFO:tasks.workunit.client.0.smithi156.stdout:5/28: unlink fa 0 2024-07-16T09:55:28.290 INFO:tasks.workunit.client.0.smithi156.stdout:1/45: write f5 [4977011,61008] 0 2024-07-16T09:55:28.291 INFO:tasks.workunit.client.0.smithi156.stdout:1/46: readlink dc/le 0 2024-07-16T09:55:28.291 INFO:tasks.workunit.client.0.smithi156.stdout:4/32: mknod d4/db/cd 0 2024-07-16T09:55:28.293 INFO:tasks.workunit.client.0.smithi156.stdout:9/31: getdents d2 0 2024-07-16T09:55:28.293 INFO:tasks.workunit.client.0.smithi156.stdout:2/25: truncate d0/f8 92563 0 2024-07-16T09:55:28.295 INFO:tasks.workunit.client.0.smithi156.stdout:5/29: unlink fb 0 2024-07-16T09:55:28.297 INFO:tasks.workunit.client.0.smithi156.stdout:2/26: fsync d0/f4 0 2024-07-16T09:55:28.297 INFO:tasks.workunit.client.0.smithi156.stdout:5/30: readlink l7 0 2024-07-16T09:55:28.297 INFO:tasks.workunit.client.0.smithi156.stdout:5/31: dwrite - no filename 2024-07-16T09:55:28.297 INFO:tasks.workunit.client.0.smithi156.stdout:5/32: read - no filename 2024-07-16T09:55:28.300 INFO:tasks.workunit.client.0.smithi156.stdout:9/32: symlink d2/la 0 2024-07-16T09:55:28.303 INFO:tasks.workunit.client.0.smithi156.stdout:6/30: dwrite d3/d4/f7 [0,4194304] 0 2024-07-16T09:55:28.304 INFO:tasks.workunit.client.0.smithi156.stdout:5/33: creat fc x:0 0 0 2024-07-16T09:55:28.304 INFO:tasks.workunit.client.0.smithi156.stdout:5/34: rmdir - no directory 2024-07-16T09:55:28.305 INFO:tasks.workunit.client.0.smithi156.stdout:5/35: readlink l7 0 2024-07-16T09:55:28.305 INFO:tasks.workunit.client.0.smithi156.stdout:6/31: chown d3/d4 97 1 2024-07-16T09:55:28.307 INFO:tasks.workunit.client.0.smithi156.stdout:4/33: dwrite d4/f9 [0,4194304] 0 2024-07-16T09:55:28.310 INFO:tasks.workunit.client.0.smithi156.stdout:2/27: dread d0/f8 [0,4194304] 0 2024-07-16T09:55:28.311 INFO:tasks.workunit.client.0.smithi156.stdout:4/34: chown d4/f5 1941181 1 2024-07-16T09:55:28.312 INFO:tasks.workunit.client.0.smithi156.stdout:9/33: chown d2/f4 24 1 2024-07-16T09:55:28.312 INFO:tasks.workunit.client.0.smithi156.stdout:1/47: truncate f5 5016904 0 2024-07-16T09:55:28.313 INFO:tasks.workunit.client.0.smithi156.stdout:4/35: chown d4/db/dc 60 1 2024-07-16T09:55:28.315 INFO:tasks.workunit.client.0.smithi156.stdout:9/34: stat d2/c7 0 2024-07-16T09:55:28.317 INFO:tasks.workunit.client.0.smithi156.stdout:9/35: dread - d2/d5/f8 zero size 2024-07-16T09:55:28.319 INFO:tasks.workunit.client.0.smithi156.stdout:2/28: link d0/l3 d0/d5/la 0 2024-07-16T09:55:28.321 INFO:tasks.workunit.client.0.smithi156.stdout:5/36: dwrite fc [0,4194304] 0 2024-07-16T09:55:28.321 INFO:tasks.workunit.client.0.smithi156.stdout:5/37: rmdir - no directory 2024-07-16T09:55:28.322 INFO:tasks.workunit.client.0.smithi156.stdout:5/38: write fc [3777937,13898] 0 2024-07-16T09:55:28.324 INFO:tasks.workunit.client.0.smithi156.stdout:4/36: getdents d4 0 2024-07-16T09:55:28.325 INFO:tasks.workunit.client.0.smithi156.stdout:9/36: creat d2/d5/fb x:0 0 0 2024-07-16T09:55:28.325 INFO:tasks.workunit.client.0.smithi156.stdout:2/29: rmdir d0/d5 39 2024-07-16T09:55:28.327 INFO:tasks.workunit.client.0.smithi156.stdout:1/48: truncate dc/f10 690450 0 2024-07-16T09:55:28.327 INFO:tasks.workunit.client.0.smithi156.stdout:2/30: fdatasync d0/f7 0 2024-07-16T09:55:28.327 INFO:tasks.workunit.client.0.smithi156.stdout:9/37: chown d2/d5 143 1 2024-07-16T09:55:28.328 INFO:tasks.workunit.client.0.smithi156.stdout:6/32: dwrite d3/d4/f5 [0,4194304] 0 2024-07-16T09:55:28.330 INFO:tasks.workunit.client.0.smithi156.stdout:9/38: fsync d2/d5/fb 0 2024-07-16T09:55:28.330 INFO:tasks.workunit.client.0.smithi156.stdout:1/49: mkdir dc/d14 0 2024-07-16T09:55:28.331 INFO:tasks.workunit.client.0.smithi156.stdout:4/37: symlink d4/db/dc/le 0 2024-07-16T09:55:28.331 INFO:tasks.workunit.client.0.smithi156.stdout:9/39: rename d2 to d2/dc 22 2024-07-16T09:55:28.332 INFO:tasks.workunit.client.0.smithi156.stdout:6/33: mkdir d3/d4/d9 0 2024-07-16T09:55:28.334 INFO:tasks.workunit.client.0.smithi156.stdout:4/38: write f1 [3581606,66168] 0 2024-07-16T09:55:28.334 INFO:tasks.workunit.client.0.smithi156.stdout:9/40: symlink d2/d5/ld 0 2024-07-16T09:55:28.337 INFO:tasks.workunit.client.0.smithi156.stdout:5/39: dread fc [0,4194304] 0 2024-07-16T09:55:28.337 INFO:tasks.workunit.client.0.smithi156.stdout:9/41: read d2/f3 [760199,117216] 0 2024-07-16T09:55:28.337 INFO:tasks.workunit.client.0.smithi156.stdout:4/39: write d4/f5 [968347,124700] 0 2024-07-16T09:55:28.338 INFO:tasks.workunit.client.0.smithi156.stdout:1/50: rename la to dc/l15 0 2024-07-16T09:55:28.339 INFO:tasks.workunit.client.0.smithi156.stdout:6/34: creat d3/d8/fa x:0 0 0 2024-07-16T09:55:28.341 INFO:tasks.workunit.client.0.smithi156.stdout:5/40: creat fd x:0 0 0 2024-07-16T09:55:28.342 INFO:tasks.workunit.client.0.smithi156.stdout:6/35: dread - d3/d8/fa zero size 2024-07-16T09:55:28.343 INFO:tasks.workunit.client.0.smithi156.stdout:9/42: mkdir d2/de 0 2024-07-16T09:55:28.343 INFO:tasks.workunit.client.0.smithi156.stdout:2/31: dwrite d0/f7 [0,4194304] 0 2024-07-16T09:55:28.343 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:55:27 smithi176 ceph-mon[36626]: pgmap v245: 65 pgs: 65 active+clean; 257 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.8 MiB/s rd, 11 MiB/s wr, 451 op/s 2024-07-16T09:55:28.344 INFO:tasks.workunit.client.0.smithi156.stdout:4/40: symlink d4/lf 0 2024-07-16T09:55:28.345 INFO:tasks.workunit.client.0.smithi156.stdout:6/36: dread - d3/d8/fa zero size 2024-07-16T09:55:28.347 INFO:tasks.workunit.client.0.smithi156.stdout:9/43: creat d2/d5/ff x:0 0 0 2024-07-16T09:55:28.349 INFO:tasks.workunit.client.0.smithi156.stdout:4/41: symlink d4/db/dc/l10 0 2024-07-16T09:55:28.349 INFO:tasks.workunit.client.0.smithi156.stdout:2/32: write d0/f8 [784835,24490] 0 2024-07-16T09:55:28.349 INFO:tasks.workunit.client.0.smithi156.stdout:9/44: readlink d2/la 0 2024-07-16T09:55:28.351 INFO:tasks.workunit.client.0.smithi156.stdout:2/33: fdatasync d0/f4 0 2024-07-16T09:55:28.354 INFO:tasks.workunit.client.0.smithi156.stdout:9/45: symlink d2/l10 0 2024-07-16T09:55:28.354 INFO:tasks.workunit.client.0.smithi156.stdout:1/51: truncate dc/f10 1179432 0 2024-07-16T09:55:28.355 INFO:tasks.workunit.client.0.smithi156.stdout:2/34: fdatasync d0/f4 0 2024-07-16T09:55:28.356 INFO:tasks.workunit.client.0.smithi156.stdout:5/41: dwrite fd [0,4194304] 0 2024-07-16T09:55:28.358 INFO:tasks.workunit.client.0.smithi156.stdout:1/52: write f4 [1119269,83463] 0 2024-07-16T09:55:28.358 INFO:tasks.workunit.client.0.smithi156.stdout:5/42: chown c8 3 1 2024-07-16T09:55:28.360 INFO:tasks.workunit.client.0.smithi156.stdout:9/46: mknod d2/de/c11 0 2024-07-16T09:55:28.361 INFO:tasks.workunit.client.0.smithi156.stdout:6/37: dwrite d3/d4/f5 [0,4194304] 0 2024-07-16T09:55:28.362 INFO:tasks.workunit.client.0.smithi156.stdout:4/42: dread f2 [0,4194304] 0 2024-07-16T09:55:28.362 INFO:tasks.workunit.client.0.smithi156.stdout:2/35: symlink d0/lb 0 2024-07-16T09:55:28.364 INFO:tasks.workunit.client.0.smithi156.stdout:6/38: write d3/d4/f5 [187676,86027] 0 2024-07-16T09:55:28.364 INFO:tasks.workunit.client.0.smithi156.stdout:9/47: fdatasync d2/f3 0 2024-07-16T09:55:28.367 INFO:tasks.workunit.client.0.smithi156.stdout:1/53: rename dc/l15 to dc/l16 0 2024-07-16T09:55:28.369 INFO:tasks.workunit.client.0.smithi156.stdout:5/43: dread fd [0,4194304] 0 2024-07-16T09:55:28.371 INFO:tasks.workunit.client.0.smithi156.stdout:2/36: dread d0/f8 [0,4194304] 0 2024-07-16T09:55:28.372 INFO:tasks.workunit.client.0.smithi156.stdout:2/37: rename d0 to d0/d5/dc 22 2024-07-16T09:55:28.372 INFO:tasks.workunit.client.0.smithi156.stdout:1/54: creat dc/d14/f17 x:0 0 0 2024-07-16T09:55:28.372 INFO:tasks.workunit.client.0.smithi156.stdout:9/48: dread d2/f4 [0,4194304] 0 2024-07-16T09:55:28.373 INFO:tasks.workunit.client.0.smithi156.stdout:5/44: link fc fe 0 2024-07-16T09:55:28.374 INFO:tasks.workunit.client.0.smithi156.stdout:9/49: dread - d2/d5/fb zero size 2024-07-16T09:55:28.374 INFO:tasks.workunit.client.0.smithi156.stdout:6/39: dread d3/d4/f7 [0,4194304] 0 2024-07-16T09:55:28.375 INFO:tasks.workunit.client.0.smithi156.stdout:2/38: chown d0/d5/la 32 1 2024-07-16T09:55:28.376 INFO:tasks.workunit.client.0.smithi156.stdout:5/45: unlink fe 0 2024-07-16T09:55:28.376 INFO:tasks.workunit.client.0.smithi156.stdout:9/50: mknod d2/d5/c12 0 2024-07-16T09:55:28.377 INFO:tasks.workunit.client.0.smithi156.stdout:9/51: chown d2/de 8896 1 2024-07-16T09:55:28.378 INFO:tasks.workunit.client.0.smithi156.stdout:9/52: read - d2/d5/fb zero size 2024-07-16T09:55:28.380 INFO:tasks.workunit.client.0.smithi156.stdout:5/46: creat ff x:0 0 0 2024-07-16T09:55:28.380 INFO:tasks.workunit.client.0.smithi156.stdout:1/55: dread f5 [0,4194304] 0 2024-07-16T09:55:28.380 INFO:tasks.workunit.client.0.smithi156.stdout:1/56: dread - f6 zero size 2024-07-16T09:55:28.380 INFO:tasks.workunit.client.0.smithi156.stdout:2/39: rename d0/l3 to d0/d5/ld 0 2024-07-16T09:55:28.380 INFO:tasks.workunit.client.0.smithi156.stdout:1/57: dread f5 [4194304,4194304] 0 2024-07-16T09:55:28.381 INFO:tasks.workunit.client.0.smithi156.stdout:1/58: dread - dc/d14/f17 zero size 2024-07-16T09:55:28.382 INFO:tasks.workunit.client.1.smithi176.stdout:0/91: sync 2024-07-16T09:55:28.382 INFO:tasks.workunit.client.1.smithi176.stdout:4/76: sync 2024-07-16T09:55:28.382 INFO:tasks.workunit.client.1.smithi176.stdout:5/160: sync 2024-07-16T09:55:28.382 INFO:tasks.workunit.client.1.smithi176.stdout:7/25: sync 2024-07-16T09:55:28.382 INFO:tasks.workunit.client.1.smithi176.stdout:1/24: sync 2024-07-16T09:55:28.382 INFO:tasks.workunit.client.1.smithi176.stdout:6/98: sync 2024-07-16T09:55:28.382 INFO:tasks.workunit.client.1.smithi176.stdout:3/2: sync 2024-07-16T09:55:28.382 INFO:tasks.workunit.client.1.smithi176.stdout:3/3: write - no filename 2024-07-16T09:55:28.382 INFO:tasks.workunit.client.1.smithi176.stdout:3/4: write - no filename 2024-07-16T09:55:28.382 INFO:tasks.workunit.client.1.smithi176.stdout:3/5: rename - no filename 2024-07-16T09:55:28.382 INFO:tasks.workunit.client.1.smithi176.stdout:8/7: sync 2024-07-16T09:55:28.382 INFO:tasks.workunit.client.1.smithi176.stdout:9/9: sync 2024-07-16T09:55:28.383 INFO:tasks.workunit.client.1.smithi176.stdout:2/12: sync 2024-07-16T09:55:28.383 INFO:tasks.workunit.client.1.smithi176.stdout:4/77: fdatasync da/fb 0 2024-07-16T09:55:28.383 INFO:tasks.workunit.client.0.smithi156.stdout:9/53: creat d2/f13 x:0 0 0 2024-07-16T09:55:28.384 INFO:tasks.workunit.client.0.smithi156.stdout:5/47: mknod c10 0 2024-07-16T09:55:28.384 INFO:tasks.workunit.client.0.smithi156.stdout:5/48: truncate ff 678680 0 2024-07-16T09:55:28.384 INFO:tasks.workunit.client.1.smithi176.stdout:3/6: symlink l0 0 2024-07-16T09:55:28.384 INFO:tasks.workunit.client.1.smithi176.stdout:6/99: chown d1/d5/d7/f9 2039 1 2024-07-16T09:55:28.384 INFO:tasks.workunit.client.1.smithi176.stdout:5/161: truncate d4/ff 862058 0 2024-07-16T09:55:28.385 INFO:tasks.workunit.client.1.smithi176.stdout:9/10: write f2 [400643,81548] 0 2024-07-16T09:55:28.385 INFO:tasks.workunit.client.1.smithi176.stdout:1/25: unlink d4/l6 0 2024-07-16T09:55:28.385 INFO:tasks.workunit.client.0.smithi156.stdout:6/40: rmdir d3/d4/d9 0 2024-07-16T09:55:28.386 INFO:tasks.workunit.client.1.smithi176.stdout:4/78: stat da 0 2024-07-16T09:55:28.386 INFO:tasks.workunit.client.0.smithi156.stdout:1/59: creat dc/f18 x:0 0 0 2024-07-16T09:55:28.387 INFO:tasks.workunit.client.1.smithi176.stdout:5/162: readlink d4/l2d 0 2024-07-16T09:55:28.389 INFO:tasks.workunit.client.1.smithi176.stdout:9/11: write f2 [785479,34668] 0 2024-07-16T09:55:28.389 INFO:tasks.workunit.client.1.smithi176.stdout:7/26: creat d4/d5/f8 x:0 0 0 2024-07-16T09:55:28.389 INFO:tasks.workunit.client.0.smithi156.stdout:5/49: mknod c11 0 2024-07-16T09:55:28.390 INFO:tasks.workunit.client.0.smithi156.stdout:5/50: truncate ff 1320742 0 2024-07-16T09:55:28.390 INFO:tasks.workunit.client.0.smithi156.stdout:6/41: symlink d3/d4/lb 0 2024-07-16T09:55:28.391 INFO:tasks.workunit.client.1.smithi176.stdout:6/100: mknod d1/dc/c20 0 2024-07-16T09:55:28.391 INFO:tasks.workunit.client.1.smithi176.stdout:1/26: rmdir d4/d8 39 2024-07-16T09:55:28.393 INFO:tasks.workunit.client.1.smithi176.stdout:8/8: link c2 c3 0 2024-07-16T09:55:28.393 INFO:tasks.workunit.client.1.smithi176.stdout:7/27: read - d4/d5/f8 zero size 2024-07-16T09:55:28.393 INFO:tasks.workunit.client.1.smithi176.stdout:6/101: fsync d1/f15 0 2024-07-16T09:55:28.394 INFO:tasks.workunit.client.0.smithi156.stdout:2/40: creat d0/fe x:0 0 0 2024-07-16T09:55:28.395 INFO:tasks.workunit.client.1.smithi176.stdout:4/79: mkdir da/dc/d10/d15 0 2024-07-16T09:55:28.396 INFO:tasks.workunit.client.0.smithi156.stdout:6/42: unlink d3/d4/f7 0 2024-07-16T09:55:28.397 INFO:tasks.workunit.client.0.smithi156.stdout:4/43: dwrite f2 [0,4194304] 0 2024-07-16T09:55:28.397 INFO:tasks.workunit.client.1.smithi176.stdout:9/12: creat f3 x:0 0 0 2024-07-16T09:55:28.398 INFO:tasks.workunit.client.1.smithi176.stdout:2/13: dwrite f3 [0,4194304] 0 2024-07-16T09:55:28.398 INFO:tasks.workunit.client.1.smithi176.stdout:8/9: mknod c4 0 2024-07-16T09:55:28.399 INFO:tasks.workunit.client.1.smithi176.stdout:0/92: dread d5/f12 [0,4194304] 0 2024-07-16T09:55:28.400 INFO:tasks.workunit.client.1.smithi176.stdout:9/13: write f2 [1104501,30743] 0 2024-07-16T09:55:28.401 INFO:tasks.workunit.client.0.smithi156.stdout:1/60: getdents dc/d14 0 2024-07-16T09:55:28.401 INFO:tasks.workunit.client.1.smithi176.stdout:2/14: write f4 [662607,49617] 0 2024-07-16T09:55:28.402 INFO:tasks.workunit.client.1.smithi176.stdout:7/28: mknod d4/c9 0 2024-07-16T09:55:28.402 INFO:tasks.workunit.client.0.smithi156.stdout:6/43: creat d3/fc x:0 0 0 2024-07-16T09:55:28.403 INFO:tasks.workunit.client.1.smithi176.stdout:1/27: rmdir d4/d8 39 2024-07-16T09:55:28.403 INFO:tasks.workunit.client.1.smithi176.stdout:9/14: write f1 [1283020,68300] 0 2024-07-16T09:55:28.406 INFO:tasks.workunit.client.1.smithi176.stdout:5/163: dread d4/d10/d17/f28 [0,4194304] 0 2024-07-16T09:55:28.406 INFO:tasks.workunit.client.1.smithi176.stdout:8/10: rename f1 to f5 0 2024-07-16T09:55:28.406 INFO:tasks.workunit.client.1.smithi176.stdout:8/11: readlink - no filename 2024-07-16T09:55:28.406 INFO:tasks.workunit.client.1.smithi176.stdout:7/29: dread f0 [0,4194304] 0 2024-07-16T09:55:28.406 INFO:tasks.workunit.client.0.smithi156.stdout:1/61: creat dc/f19 x:0 0 0 2024-07-16T09:55:28.407 INFO:tasks.workunit.client.0.smithi156.stdout:6/44: stat d3/d8/fa 0 2024-07-16T09:55:28.407 INFO:tasks.workunit.client.0.smithi156.stdout:5/51: dwrite fd [0,4194304] 0 2024-07-16T09:55:28.409 INFO:tasks.workunit.client.0.smithi156.stdout:2/41: dwrite d0/fe [0,4194304] 0 2024-07-16T09:55:28.411 INFO:tasks.workunit.client.1.smithi176.stdout:5/164: fdatasync d4/d14/f25 0 2024-07-16T09:55:28.411 INFO:tasks.workunit.client.1.smithi176.stdout:4/80: unlink f3 0 2024-07-16T09:55:28.411 INFO:tasks.workunit.client.1.smithi176.stdout:2/15: symlink l5 0 2024-07-16T09:55:28.411 INFO:tasks.workunit.client.1.smithi176.stdout:7/30: chown d4/d5 1592 1 2024-07-16T09:55:28.412 INFO:tasks.workunit.client.1.smithi176.stdout:8/12: chown f5 796601 1 2024-07-16T09:55:28.417 INFO:tasks.workunit.client.1.smithi176.stdout:9/15: mknod c4 0 2024-07-16T09:55:28.417 INFO:tasks.workunit.client.1.smithi176.stdout:1/28: stat d4/d8/fa 0 2024-07-16T09:55:28.417 INFO:tasks.workunit.client.0.smithi156.stdout:2/42: rename d0/c9 to d0/d5/cf 0 2024-07-16T09:55:28.418 INFO:tasks.workunit.client.0.smithi156.stdout:4/44: getdents d4/db/dc 0 2024-07-16T09:55:28.421 INFO:tasks.workunit.client.1.smithi176.stdout:8/13: dread f5 [0,4194304] 0 2024-07-16T09:55:28.421 INFO:tasks.workunit.client.1.smithi176.stdout:2/16: rename l5 to l6 0 2024-07-16T09:55:28.421 INFO:tasks.workunit.client.0.smithi156.stdout:1/62: dwrite dc/d14/f17 [0,4194304] 0 2024-07-16T09:55:28.422 INFO:tasks.workunit.client.0.smithi156.stdout:6/45: dread d3/d4/f5 [0,4194304] 0 2024-07-16T09:55:28.426 INFO:tasks.workunit.client.1.smithi176.stdout:4/81: dwrite da/fb [0,4194304] 0 2024-07-16T09:55:28.427 INFO:tasks.workunit.client.0.smithi156.stdout:5/52: dread fc [0,4194304] 0 2024-07-16T09:55:28.428 INFO:tasks.workunit.client.0.smithi156.stdout:5/53: stat fc 0 2024-07-16T09:55:28.429 INFO:tasks.workunit.client.0.smithi156.stdout:4/45: unlink d4/db/dc/le 0 2024-07-16T09:55:28.431 INFO:tasks.workunit.client.0.smithi156.stdout:4/46: write d4/f5 [1548317,55929] 0 2024-07-16T09:55:28.432 INFO:tasks.workunit.client.1.smithi176.stdout:6/102: link d1/d5/l1a d1/d5/d7/l21 0 2024-07-16T09:55:28.432 INFO:tasks.workunit.client.1.smithi176.stdout:9/16: dread f2 [0,4194304] 0 2024-07-16T09:55:28.433 INFO:tasks.workunit.client.1.smithi176.stdout:7/31: dwrite d4/f6 [4194304,4194304] 0 2024-07-16T09:55:28.433 INFO:tasks.workunit.client.1.smithi176.stdout:5/165: mkdir d4/d14/d19/d1b/d36/d38/d39 0 2024-07-16T09:55:28.435 INFO:tasks.workunit.client.0.smithi156.stdout:5/54: fsync fc 0 2024-07-16T09:55:28.435 INFO:tasks.workunit.client.1.smithi176.stdout:2/17: rename f4 to f7 0 2024-07-16T09:55:28.436 INFO:tasks.workunit.client.0.smithi156.stdout:6/46: dwrite d3/d8/fa [0,4194304] 0 2024-07-16T09:55:28.436 INFO:tasks.workunit.client.1.smithi176.stdout:5/166: dread - d4/d10/f31 zero size 2024-07-16T09:55:28.436 INFO:tasks.workunit.client.0.smithi156.stdout:5/55: chown c10 1 1 2024-07-16T09:55:28.437 INFO:tasks.workunit.client.0.smithi156.stdout:6/47: rename d3/d4 to d3/d4/dd 22 2024-07-16T09:55:28.437 INFO:tasks.workunit.client.1.smithi176.stdout:1/29: mkdir d4/db 0 2024-07-16T09:55:28.438 INFO:tasks.workunit.client.1.smithi176.stdout:5/167: chown d4/d10/d17/l1d 325 1 2024-07-16T09:55:28.439 INFO:tasks.workunit.client.1.smithi176.stdout:1/30: truncate d4/d8/fa 332145 0 2024-07-16T09:55:28.439 INFO:tasks.workunit.client.0.smithi156.stdout:4/47: mkdir d4/db/d11 0 2024-07-16T09:55:28.440 INFO:tasks.workunit.client.0.smithi156.stdout:1/63: link c8 dc/d14/c1a 0 2024-07-16T09:55:28.441 INFO:tasks.workunit.client.1.smithi176.stdout:9/17: mkdir d5 0 2024-07-16T09:55:28.442 INFO:tasks.workunit.client.0.smithi156.stdout:1/64: write dc/f18 [416295,8804] 0 2024-07-16T09:55:28.444 INFO:tasks.workunit.client.1.smithi176.stdout:8/14: stat c2 0 2024-07-16T09:55:28.444 INFO:tasks.workunit.client.0.smithi156.stdout:5/56: dread fc [0,4194304] 0 2024-07-16T09:55:28.444 INFO:tasks.workunit.client.0.smithi156.stdout:5/57: chown fc 8 1 2024-07-16T09:55:28.445 INFO:tasks.workunit.client.1.smithi176.stdout:6/103: mknod d1/d5/d7/d1e/c22 0 2024-07-16T09:55:28.446 INFO:tasks.workunit.client.1.smithi176.stdout:6/104: stat d1/d5/f6 0 2024-07-16T09:55:28.446 INFO:tasks.workunit.client.1.smithi176.stdout:6/105: chown d1/lf 0 1 2024-07-16T09:55:28.447 INFO:tasks.workunit.client.0.smithi156.stdout:4/48: rename d4/l7 to d4/db/l12 0 2024-07-16T09:55:28.447 INFO:tasks.workunit.client.0.smithi156.stdout:4/49: write d4/f5 [1391417,67800] 0 2024-07-16T09:55:28.448 INFO:tasks.workunit.client.1.smithi176.stdout:6/106: truncate d1/d5/d7/f9 5112026 0 2024-07-16T09:55:28.448 INFO:tasks.workunit.client.1.smithi176.stdout:2/18: mknod c8 0 2024-07-16T09:55:28.451 INFO:tasks.workunit.client.1.smithi176.stdout:4/82: mknod da/dc/d10/d15/c16 0 2024-07-16T09:55:28.451 INFO:tasks.workunit.client.1.smithi176.stdout:5/168: rmdir d4/d10 39 2024-07-16T09:55:28.452 INFO:tasks.workunit.client.0.smithi156.stdout:4/50: rename d4/lf to d4/db/dc/l13 0 2024-07-16T09:55:28.452 INFO:tasks.workunit.client.1.smithi176.stdout:2/19: dread f3 [0,4194304] 0 2024-07-16T09:55:28.452 INFO:tasks.workunit.client.0.smithi156.stdout:4/51: fsync f2 0 2024-07-16T09:55:28.453 INFO:tasks.workunit.client.1.smithi176.stdout:2/20: chown f7 89 1 2024-07-16T09:55:28.454 INFO:tasks.workunit.client.1.smithi176.stdout:9/18: unlink f1 0 2024-07-16T09:55:28.455 INFO:tasks.workunit.client.1.smithi176.stdout:9/19: write f3 [891070,121930] 0 2024-07-16T09:55:28.456 INFO:tasks.workunit.client.0.smithi156.stdout:4/52: mknod d4/db/c14 0 2024-07-16T09:55:28.456 INFO:tasks.workunit.client.1.smithi176.stdout:7/32: rename d4/d5/f8 to d4/fa 0 2024-07-16T09:55:28.458 INFO:tasks.workunit.client.0.smithi156.stdout:4/53: symlink d4/db/dc/l15 0 2024-07-16T09:55:28.459 INFO:tasks.workunit.client.1.smithi176.stdout:7/33: chown d4/f6 0 1 2024-07-16T09:55:28.459 INFO:tasks.workunit.client.0.smithi156.stdout:4/54: chown d4/db 39 1 2024-07-16T09:55:28.459 INFO:tasks.workunit.client.1.smithi176.stdout:2/21: creat f9 x:0 0 0 2024-07-16T09:55:28.461 INFO:tasks.workunit.client.1.smithi176.stdout:9/20: symlink d5/l6 0 2024-07-16T09:55:28.461 INFO:tasks.workunit.client.0.smithi156.stdout:4/55: creat d4/db/f16 x:0 0 0 2024-07-16T09:55:28.463 INFO:tasks.workunit.client.1.smithi176.stdout:7/34: creat d4/fb x:0 0 0 2024-07-16T09:55:28.463 INFO:tasks.workunit.client.1.smithi176.stdout:5/169: dwrite d4/f27 [0,4194304] 0 2024-07-16T09:55:28.463 INFO:tasks.workunit.client.0.smithi156.stdout:4/56: creat d4/db/d11/f17 x:0 0 0 2024-07-16T09:55:28.464 INFO:tasks.workunit.client.1.smithi176.stdout:4/83: truncate da/dc/fe 536488 0 2024-07-16T09:55:28.464 INFO:tasks.workunit.client.1.smithi176.stdout:9/21: write f3 [451653,87759] 0 2024-07-16T09:55:28.465 INFO:tasks.workunit.client.0.smithi156.stdout:4/57: creat d4/db/f18 x:0 0 0 2024-07-16T09:55:28.468 INFO:tasks.workunit.client.0.smithi156.stdout:5/58: fdatasync fd 0 2024-07-16T09:55:28.468 INFO:tasks.workunit.client.1.smithi176.stdout:7/35: symlink d4/d5/lc 0 2024-07-16T09:55:28.469 INFO:tasks.workunit.client.1.smithi176.stdout:9/22: symlink d5/l7 0 2024-07-16T09:55:28.469 INFO:tasks.workunit.client.0.smithi156.stdout:5/59: fsync fd 0 2024-07-16T09:55:28.469 INFO:tasks.workunit.client.0.smithi156.stdout:4/58: rename d4/f6 to d4/db/d11/f19 0 2024-07-16T09:55:28.469 INFO:tasks.workunit.client.0.smithi156.stdout:4/59: chown f2 0 1 2024-07-16T09:55:28.470 INFO:tasks.workunit.client.0.smithi156.stdout:6/48: rmdir d3 39 2024-07-16T09:55:28.470 INFO:tasks.workunit.client.1.smithi176.stdout:4/84: chown da/dc/d10/d15/c16 11 1 2024-07-16T09:55:28.471 INFO:tasks.workunit.client.1.smithi176.stdout:8/15: dwrite f5 [0,4194304] 0 2024-07-16T09:55:28.471 INFO:tasks.workunit.client.0.smithi156.stdout:5/60: unlink c10 0 2024-07-16T09:55:28.475 INFO:tasks.workunit.client.1.smithi176.stdout:9/23: unlink d5/l6 0 2024-07-16T09:55:28.476 INFO:tasks.workunit.client.1.smithi176.stdout:7/36: mkdir d4/dd 0 2024-07-16T09:55:28.477 INFO:tasks.workunit.client.0.smithi156.stdout:6/49: creat d3/d8/fe x:0 0 0 2024-07-16T09:55:28.477 INFO:tasks.workunit.client.1.smithi176.stdout:9/24: write f3 [1738831,50500] 0 2024-07-16T09:55:28.478 INFO:tasks.workunit.client.0.smithi156.stdout:5/61: link fc f12 0 2024-07-16T09:55:28.478 INFO:tasks.workunit.client.1.smithi176.stdout:7/37: dread - d4/fb zero size 2024-07-16T09:55:28.479 INFO:tasks.workunit.client.1.smithi176.stdout:2/22: dwrite f7 [0,4194304] 0 2024-07-16T09:55:28.479 INFO:tasks.workunit.client.0.smithi156.stdout:5/62: stat fd 0 2024-07-16T09:55:28.479 INFO:tasks.workunit.client.0.smithi156.stdout:5/63: fdatasync fd 0 2024-07-16T09:55:28.480 INFO:tasks.workunit.client.1.smithi176.stdout:4/85: mkdir da/dc/d10/d15/d17 0 2024-07-16T09:55:28.481 INFO:tasks.workunit.client.0.smithi156.stdout:6/50: creat d3/ff x:0 0 0 2024-07-16T09:55:28.481 INFO:tasks.workunit.client.1.smithi176.stdout:0/93: truncate d5/f6 3663191 0 2024-07-16T09:55:28.482 INFO:tasks.workunit.client.0.smithi156.stdout:5/64: rename fd to f13 0 2024-07-16T09:55:28.483 INFO:tasks.workunit.client.1.smithi176.stdout:5/170: link d4/d14/f25 d4/d14/d19/d1b/d36/d38/d39/f3a 0 2024-07-16T09:55:28.483 INFO:tasks.workunit.client.0.smithi156.stdout:5/65: chown c8 948 1 2024-07-16T09:55:28.484 INFO:tasks.workunit.client.0.smithi156.stdout:2/43: truncate d0/f7 1438272 0 2024-07-16T09:55:28.484 INFO:tasks.workunit.client.1.smithi176.stdout:9/25: symlink d5/l8 0 2024-07-16T09:55:28.484 INFO:tasks.workunit.client.1.smithi176.stdout:5/171: chown d4/d14/d19/f21 0 1 2024-07-16T09:55:28.485 INFO:tasks.workunit.client.0.smithi156.stdout:6/51: creat d3/d8/f10 x:0 0 0 2024-07-16T09:55:28.487 INFO:tasks.workunit.client.1.smithi176.stdout:2/23: creat fa x:0 0 0 2024-07-16T09:55:28.488 INFO:tasks.workunit.client.0.smithi156.stdout:6/52: mknod d3/c11 0 2024-07-16T09:55:28.488 INFO:tasks.workunit.client.1.smithi176.stdout:0/94: creat d5/f18 x:0 0 0 2024-07-16T09:55:28.489 INFO:tasks.workunit.client.1.smithi176.stdout:0/95: dread - d5/f14 zero size 2024-07-16T09:55:28.489 INFO:tasks.workunit.client.1.smithi176.stdout:0/96: chown f3 7 1 2024-07-16T09:55:28.490 INFO:tasks.workunit.client.0.smithi156.stdout:4/60: readlink d4/db/l12 0 2024-07-16T09:55:28.491 INFO:tasks.workunit.client.0.smithi156.stdout:6/53: mknod d3/d4/c12 0 2024-07-16T09:55:28.492 INFO:tasks.workunit.client.0.smithi156.stdout:4/61: mknod d4/db/c1a 0 2024-07-16T09:55:28.492 INFO:tasks.workunit.client.1.smithi176.stdout:7/38: symlink d4/dd/le 0 2024-07-16T09:55:28.493 INFO:tasks.workunit.client.1.smithi176.stdout:7/39: chown d4/c9 2 1 2024-07-16T09:55:28.495 INFO:tasks.workunit.client.1.smithi176.stdout:4/86: creat da/dc/d10/d15/d17/f18 x:0 0 0 2024-07-16T09:55:28.495 INFO:tasks.workunit.client.1.smithi176.stdout:4/87: chown da/d14 66 1 2024-07-16T09:55:28.496 INFO:tasks.workunit.client.0.smithi156.stdout:4/62: write d4/db/d11/f19 [549098,73542] 0 2024-07-16T09:55:28.497 INFO:tasks.workunit.client.1.smithi176.stdout:5/172: mkdir d4/d10/d17/d3b 0 2024-07-16T09:55:28.499 INFO:tasks.workunit.client.1.smithi176.stdout:8/16: fdatasync f5 0 2024-07-16T09:55:28.499 INFO:tasks.workunit.client.0.smithi156.stdout:1/65: dwrite dc/f10 [0,4194304] 0 2024-07-16T09:55:28.500 INFO:tasks.workunit.client.1.smithi176.stdout:8/17: chown c4 1 1 2024-07-16T09:55:28.500 INFO:tasks.workunit.client.0.smithi156.stdout:1/66: readlink lb 0 2024-07-16T09:55:28.501 INFO:tasks.workunit.client.1.smithi176.stdout:5/173: symlink d4/d14/d19/d1b/d36/l3c 0 2024-07-16T09:55:28.501 INFO:tasks.workunit.client.0.smithi156.stdout:4/63: rename d4/db/d11 to d4/d1b 0 2024-07-16T09:55:28.502 INFO:tasks.workunit.client.1.smithi176.stdout:2/24: link c1 cb 0 2024-07-16T09:55:28.503 INFO:tasks.workunit.client.0.smithi156.stdout:1/67: mknod dc/d14/c1b 0 2024-07-16T09:55:28.504 INFO:tasks.workunit.client.0.smithi156.stdout:4/64: symlink d4/l1c 0 2024-07-16T09:55:28.505 INFO:tasks.workunit.client.0.smithi156.stdout:1/68: creat dc/f1c x:0 0 0 2024-07-16T09:55:28.507 INFO:tasks.workunit.client.0.smithi156.stdout:4/65: write d4/f9 [1310230,128661] 0 2024-07-16T09:55:28.507 INFO:tasks.workunit.client.1.smithi176.stdout:5/174: mkdir d4/d14/d3d 0 2024-07-16T09:55:28.508 INFO:tasks.workunit.client.1.smithi176.stdout:8/18: dwrite f5 [0,4194304] 0 2024-07-16T09:55:28.510 INFO:tasks.workunit.client.1.smithi176.stdout:2/25: unlink f7 0 2024-07-16T09:55:28.510 INFO:tasks.workunit.client.1.smithi176.stdout:5/175: dread - d4/d14/d19/f35 zero size 2024-07-16T09:55:28.510 INFO:tasks.workunit.client.1.smithi176.stdout:7/40: link l1 d4/lf 0 2024-07-16T09:55:28.514 INFO:tasks.workunit.client.0.smithi156.stdout:2/44: dread d0/f7 [0,4194304] 0 2024-07-16T09:55:28.514 INFO:tasks.workunit.client.0.smithi156.stdout:6/54: getdents d3/d4 0 2024-07-16T09:55:28.515 INFO:tasks.workunit.client.1.smithi176.stdout:8/19: creat f6 x:0 0 0 2024-07-16T09:55:28.517 INFO:tasks.workunit.client.1.smithi176.stdout:7/41: mkdir d4/d5/d10 0 2024-07-16T09:55:28.518 INFO:tasks.workunit.client.0.smithi156.stdout:6/55: chown d3/d8/f10 27696033 1 2024-07-16T09:55:28.518 INFO:tasks.workunit.client.0.smithi156.stdout:1/69: truncate dc/d14/f17 1789585 0 2024-07-16T09:55:28.518 INFO:tasks.workunit.client.0.smithi156.stdout:4/66: dwrite d4/d1b/f19 [0,4194304] 0 2024-07-16T09:55:28.519 INFO:tasks.workunit.client.0.smithi156.stdout:6/56: stat d3/d8/fe 0 2024-07-16T09:55:28.521 INFO:tasks.workunit.client.1.smithi176.stdout:4/88: rename da/dc/d10/f12 to da/f19 0 2024-07-16T09:55:28.521 INFO:tasks.workunit.client.0.smithi156.stdout:6/57: write d3/ff [157753,109937] 0 2024-07-16T09:55:28.522 INFO:tasks.workunit.client.1.smithi176.stdout:7/42: unlink d4/fa 0 2024-07-16T09:55:28.523 INFO:tasks.workunit.client.1.smithi176.stdout:8/20: dwrite f6 [0,4194304] 0 2024-07-16T09:55:28.524 INFO:tasks.workunit.client.0.smithi156.stdout:6/58: mkdir d3/d13 0 2024-07-16T09:55:28.525 INFO:tasks.workunit.client.1.smithi176.stdout:9/26: rmdir d5 39 2024-07-16T09:55:28.527 INFO:tasks.workunit.client.0.smithi156.stdout:6/59: write d3/d8/fa [2780419,61371] 0 2024-07-16T09:55:28.527 INFO:tasks.workunit.client.0.smithi156.stdout:4/67: dread d4/f9 [0,4194304] 0 2024-07-16T09:55:28.528 INFO:tasks.workunit.client.1.smithi176.stdout:2/26: fsync fa 0 2024-07-16T09:55:28.530 INFO:tasks.workunit.client.1.smithi176.stdout:7/43: rename d4/f6 to d4/f11 0 2024-07-16T09:55:28.531 INFO:tasks.workunit.client.0.smithi156.stdout:4/68: creat d4/db/f1d x:0 0 0 2024-07-16T09:55:28.531 INFO:tasks.workunit.client.0.smithi156.stdout:2/45: truncate d0/fe 2352180 0 2024-07-16T09:55:28.532 INFO:tasks.workunit.client.1.smithi176.stdout:0/97: rmdir d5 39 2024-07-16T09:55:28.533 INFO:tasks.workunit.client.0.smithi156.stdout:4/69: mknod d4/db/dc/c1e 0 2024-07-16T09:55:28.535 INFO:tasks.workunit.client.0.smithi156.stdout:4/70: mknod d4/d1b/c1f 0 2024-07-16T09:55:28.537 INFO:tasks.workunit.client.1.smithi176.stdout:8/21: dwrite f5 [0,4194304] 0 2024-07-16T09:55:28.538 INFO:tasks.workunit.client.0.smithi156.stdout:4/71: creat d4/db/dc/f20 x:0 0 0 2024-07-16T09:55:28.539 INFO:tasks.workunit.client.1.smithi176.stdout:4/89: creat da/f1a x:0 0 0 2024-07-16T09:55:28.541 INFO:tasks.workunit.client.1.smithi176.stdout:7/44: creat d4/d5/f12 x:0 0 0 2024-07-16T09:55:28.541 INFO:tasks.workunit.client.0.smithi156.stdout:4/72: write d4/db/f18 [7369,77761] 0 2024-07-16T09:55:28.542 INFO:tasks.workunit.client.0.smithi156.stdout:6/60: truncate d3/d4/f5 2335749 0 2024-07-16T09:55:28.543 INFO:tasks.workunit.client.1.smithi176.stdout:9/27: dwrite f3 [0,4194304] 0 2024-07-16T09:55:28.543 INFO:tasks.workunit.client.1.smithi176.stdout:5/176: rename d4/d14/d19/l30 to d4/l3e 0 2024-07-16T09:55:28.544 INFO:tasks.workunit.client.0.smithi156.stdout:2/46: dwrite d0/f8 [0,4194304] 0 2024-07-16T09:55:28.545 INFO:tasks.workunit.client.1.smithi176.stdout:0/98: rmdir d5 39 2024-07-16T09:55:28.550 INFO:tasks.workunit.client.1.smithi176.stdout:9/28: dread f2 [0,4194304] 0 2024-07-16T09:55:28.550 INFO:tasks.workunit.client.0.smithi156.stdout:4/73: dread d4/f9 [0,4194304] 0 2024-07-16T09:55:28.550 INFO:tasks.workunit.client.0.smithi156.stdout:6/61: chown d3/d4/f5 15329337 1 2024-07-16T09:55:28.551 INFO:tasks.workunit.client.0.smithi156.stdout:4/74: chown d4/f9 28708724 1 2024-07-16T09:55:28.551 INFO:tasks.workunit.client.1.smithi176.stdout:2/27: dwrite f3 [0,4194304] 0 2024-07-16T09:55:28.551 INFO:tasks.workunit.client.1.smithi176.stdout:0/99: creat d5/f19 x:0 0 0 2024-07-16T09:55:28.552 INFO:tasks.workunit.client.1.smithi176.stdout:2/28: readlink l0 0 2024-07-16T09:55:28.554 INFO:tasks.workunit.client.1.smithi176.stdout:2/29: chown fa 87 1 2024-07-16T09:55:28.555 INFO:tasks.workunit.client.0.smithi156.stdout:2/47: rename d0/lb to d0/l10 0 2024-07-16T09:55:28.559 INFO:tasks.workunit.client.0.smithi156.stdout:6/62: dwrite d3/d8/fe [0,4194304] 0 2024-07-16T09:55:28.562 INFO:tasks.workunit.client.1.smithi176.stdout:4/90: dwrite da/dc/d10/d15/d17/f18 [0,4194304] 0 2024-07-16T09:55:28.563 INFO:tasks.workunit.client.1.smithi176.stdout:7/45: link d4/dd/le d4/dd/l13 0 2024-07-16T09:55:28.564 INFO:tasks.workunit.client.1.smithi176.stdout:2/30: symlink lc 0 2024-07-16T09:55:28.564 INFO:tasks.workunit.client.1.smithi176.stdout:5/177: creat d4/d37/f3f x:0 0 0 2024-07-16T09:55:28.565 INFO:tasks.workunit.client.1.smithi176.stdout:0/100: fsync d5/f16 0 2024-07-16T09:55:28.565 INFO:tasks.workunit.client.1.smithi176.stdout:9/29: creat d5/f9 x:0 0 0 2024-07-16T09:55:28.566 INFO:tasks.workunit.client.1.smithi176.stdout:7/46: readlink d4/d5/lc 0 2024-07-16T09:55:28.567 INFO:tasks.workunit.client.0.smithi156.stdout:6/63: dread d3/d8/fa [0,4194304] 0 2024-07-16T09:55:28.568 INFO:tasks.workunit.client.1.smithi176.stdout:2/31: chown lc 1006 1 2024-07-16T09:55:28.569 INFO:tasks.workunit.client.0.smithi156.stdout:2/48: mknod d0/c11 0 2024-07-16T09:55:28.570 INFO:tasks.workunit.client.0.smithi156.stdout:4/75: link d4/db/dc/c1e d4/db/c21 0 2024-07-16T09:55:28.571 INFO:tasks.workunit.client.0.smithi156.stdout:2/49: read d0/f4 [371428,96171] 0 2024-07-16T09:55:28.571 INFO:tasks.workunit.client.1.smithi176.stdout:7/47: write d4/f11 [6941560,105453] 0 2024-07-16T09:55:28.572 INFO:tasks.workunit.client.0.smithi156.stdout:6/64: creat d3/d4/f14 x:0 0 0 2024-07-16T09:55:28.574 INFO:tasks.workunit.client.1.smithi176.stdout:7/48: dread f0 [0,4194304] 0 2024-07-16T09:55:28.575 INFO:tasks.workunit.client.1.smithi176.stdout:5/178: mknod d4/d14/d19/d1b/d36/d38/c40 0 2024-07-16T09:55:28.575 INFO:tasks.workunit.client.1.smithi176.stdout:4/91: link da/f1a da/dc/d10/d15/f1b 0 2024-07-16T09:55:28.576 INFO:tasks.workunit.client.0.smithi156.stdout:2/50: creat d0/d5/f12 x:0 0 0 2024-07-16T09:55:28.577 INFO:tasks.workunit.client.1.smithi176.stdout:7/49: creat d4/f14 x:0 0 0 2024-07-16T09:55:28.578 INFO:tasks.workunit.client.1.smithi176.stdout:2/32: dwrite f3 [0,4194304] 0 2024-07-16T09:55:28.579 INFO:tasks.workunit.client.1.smithi176.stdout:4/92: chown da/c13 250 1 2024-07-16T09:55:28.580 INFO:tasks.workunit.client.1.smithi176.stdout:2/33: write f9 [694759,119225] 0 2024-07-16T09:55:28.580 INFO:tasks.workunit.client.1.smithi176.stdout:4/93: readlink l4 0 2024-07-16T09:55:28.583 INFO:tasks.workunit.client.0.smithi156.stdout:4/76: dwrite d4/db/f18 [0,4194304] 0 2024-07-16T09:55:28.583 INFO:tasks.workunit.client.1.smithi176.stdout:4/94: unlink f8 0 2024-07-16T09:55:28.585 INFO:tasks.workunit.client.1.smithi176.stdout:7/50: dread d4/f11 [4194304,4194304] 0 2024-07-16T09:55:28.586 INFO:tasks.workunit.client.1.smithi176.stdout:5/179: rmdir d4/d10/d17/d3b 0 2024-07-16T09:55:28.586 INFO:tasks.workunit.client.1.smithi176.stdout:2/34: mkdir dd 0 2024-07-16T09:55:28.588 INFO:tasks.workunit.client.1.smithi176.stdout:7/51: creat d4/dd/f15 x:0 0 0 2024-07-16T09:55:28.589 INFO:tasks.workunit.client.0.smithi156.stdout:4/77: getdents d4/db/dc 0 2024-07-16T09:55:28.589 INFO:tasks.workunit.client.1.smithi176.stdout:2/35: write f9 [1612715,56259] 0 2024-07-16T09:55:28.590 INFO:tasks.workunit.client.1.smithi176.stdout:5/180: chown d4/d10/d17/f18 3240909 1 2024-07-16T09:55:28.590 INFO:tasks.workunit.client.0.smithi156.stdout:4/78: write f1 [8684689,25806] 0 2024-07-16T09:55:28.591 INFO:tasks.workunit.client.0.smithi156.stdout:4/79: fdatasync d4/d1b/f19 0 2024-07-16T09:55:28.592 INFO:tasks.workunit.client.1.smithi176.stdout:7/52: creat d4/dd/f16 x:0 0 0 2024-07-16T09:55:28.594 INFO:tasks.workunit.client.0.smithi156.stdout:4/80: mkdir d4/db/dc/d22 0 2024-07-16T09:55:28.594 INFO:tasks.workunit.client.1.smithi176.stdout:7/53: write d4/f14 [1012318,108671] 0 2024-07-16T09:55:28.595 INFO:tasks.workunit.client.1.smithi176.stdout:2/36: mkdir dd/de 0 2024-07-16T09:55:28.596 INFO:tasks.workunit.client.1.smithi176.stdout:5/181: rename d4/d10/f31 to d4/d14/d3d/f41 0 2024-07-16T09:55:28.598 INFO:tasks.workunit.client.1.smithi176.stdout:7/54: mkdir d4/d5/d10/d17 0 2024-07-16T09:55:28.599 INFO:tasks.workunit.client.1.smithi176.stdout:7/55: chown d4/d5/d10 3993 1 2024-07-16T09:55:28.601 INFO:tasks.workunit.client.0.smithi156.stdout:4/81: dwrite d4/f5 [0,4194304] 0 2024-07-16T09:55:28.601 INFO:tasks.workunit.client.1.smithi176.stdout:7/56: chown d4/d5/f12 385093 1 2024-07-16T09:55:28.602 INFO:tasks.workunit.client.0.smithi156.stdout:4/82: dread - d4/d1b/f17 zero size 2024-07-16T09:55:28.602 INFO:tasks.workunit.client.0.smithi156.stdout:4/83: fsync f2 0 2024-07-16T09:55:28.603 INFO:tasks.workunit.client.0.smithi156.stdout:4/84: chown d4/f8 2440 1 2024-07-16T09:55:28.604 INFO:tasks.workunit.client.0.smithi156.stdout:4/85: write d4/db/f16 [444641,113881] 0 2024-07-16T09:55:28.606 INFO:tasks.workunit.client.1.smithi176.stdout:2/37: dwrite fa [0,4194304] 0 2024-07-16T09:55:28.608 INFO:tasks.workunit.client.0.smithi156.stdout:6/65: rename d3/d4/f5 to d3/d8/f15 0 2024-07-16T09:55:28.609 INFO:tasks.workunit.client.0.smithi156.stdout:4/86: unlink d4/d1b/f19 0 2024-07-16T09:55:28.610 INFO:tasks.workunit.client.0.smithi156.stdout:6/66: truncate d3/d8/f10 823040 0 2024-07-16T09:55:28.610 INFO:tasks.workunit.client.1.smithi176.stdout:5/182: dwrite d4/d14/f2c [0,4194304] 0 2024-07-16T09:55:28.615 INFO:tasks.workunit.client.1.smithi176.stdout:7/57: rename l1 to d4/d5/d10/l18 0 2024-07-16T09:55:28.618 INFO:tasks.workunit.client.1.smithi176.stdout:7/58: creat d4/f19 x:0 0 0 2024-07-16T09:55:28.619 INFO:tasks.workunit.client.1.smithi176.stdout:2/38: dread f3 [0,4194304] 0 2024-07-16T09:55:28.619 INFO:tasks.workunit.client.1.smithi176.stdout:5/183: symlink d4/d10/d17/l42 0 2024-07-16T09:55:28.622 INFO:tasks.workunit.client.0.smithi156.stdout:4/87: dwrite f2 [0,4194304] 0 2024-07-16T09:55:28.622 INFO:tasks.workunit.client.1.smithi176.stdout:0/101: getdents d5 0 2024-07-16T09:55:28.623 INFO:tasks.workunit.client.0.smithi156.stdout:2/51: dread d0/fe [0,4194304] 0 2024-07-16T09:55:28.624 INFO:tasks.workunit.client.1.smithi176.stdout:7/59: creat d4/dd/f1a x:0 0 0 2024-07-16T09:55:28.624 INFO:tasks.workunit.client.0.smithi156.stdout:6/67: dwrite d3/d8/f10 [0,4194304] 0 2024-07-16T09:55:28.625 INFO:tasks.workunit.client.0.smithi156.stdout:2/52: dread - d0/d5/f12 zero size 2024-07-16T09:55:28.629 INFO:tasks.workunit.client.1.smithi176.stdout:0/102: rename d5/c17 to d5/c1a 0 2024-07-16T09:55:28.630 INFO:tasks.workunit.client.1.smithi176.stdout:0/103: chown f3 755399204 1 2024-07-16T09:55:28.631 INFO:tasks.workunit.client.1.smithi176.stdout:2/39: dwrite fa [0,4194304] 0 2024-07-16T09:55:28.635 INFO:tasks.workunit.client.0.smithi156.stdout:6/68: dwrite d3/d4/f14 [0,4194304] 0 2024-07-16T09:55:28.635 INFO:tasks.workunit.client.1.smithi176.stdout:5/184: dread d4/d10/f15 [0,4194304] 0 2024-07-16T09:55:28.636 INFO:tasks.workunit.client.1.smithi176.stdout:9/30: getdents d5 0 2024-07-16T09:55:28.637 INFO:tasks.workunit.client.1.smithi176.stdout:2/40: write f9 [1404074,40326] 0 2024-07-16T09:55:28.639 INFO:tasks.workunit.client.0.smithi156.stdout:6/69: unlink d3/ff 0 2024-07-16T09:55:28.640 INFO:tasks.workunit.client.1.smithi176.stdout:5/185: fdatasync d4/d10/d17/f2f 0 2024-07-16T09:55:28.642 INFO:tasks.workunit.client.1.smithi176.stdout:0/104: truncate d5/f13 984508 0 2024-07-16T09:55:28.643 INFO:tasks.workunit.client.1.smithi176.stdout:4/95: write da/f19 [1533794,129937] 0 2024-07-16T09:55:28.644 INFO:tasks.workunit.client.1.smithi176.stdout:2/41: rmdir dd 39 2024-07-16T09:55:28.644 INFO:tasks.workunit.client.0.smithi156.stdout:6/70: symlink d3/d4/l16 0 2024-07-16T09:55:28.645 INFO:tasks.workunit.client.1.smithi176.stdout:2/42: stat f9 0 2024-07-16T09:55:28.645 INFO:tasks.workunit.client.1.smithi176.stdout:5/186: dread - d4/d14/d19/f35 zero size 2024-07-16T09:55:28.645 INFO:tasks.workunit.client.1.smithi176.stdout:2/43: chown lc 56 1 2024-07-16T09:55:28.646 INFO:tasks.workunit.client.0.smithi156.stdout:6/71: write d3/d8/f10 [3864481,106167] 0 2024-07-16T09:55:28.647 INFO:tasks.workunit.client.1.smithi176.stdout:9/31: getdents d5 0 2024-07-16T09:55:28.647 INFO:tasks.workunit.client.1.smithi176.stdout:9/32: chown f2 3560013 1 2024-07-16T09:55:28.648 INFO:tasks.workunit.client.1.smithi176.stdout:9/33: dread - d5/f9 zero size 2024-07-16T09:55:28.649 INFO:tasks.workunit.client.0.smithi156.stdout:2/53: dwrite d0/fe [0,4194304] 0 2024-07-16T09:55:28.650 INFO:tasks.workunit.client.1.smithi176.stdout:2/44: dread f3 [0,4194304] 0 2024-07-16T09:55:28.650 INFO:tasks.workunit.client.1.smithi176.stdout:0/105: creat d5/f1b x:0 0 0 2024-07-16T09:55:28.651 INFO:tasks.workunit.client.1.smithi176.stdout:7/60: link d4/d5/lc d4/d5/d10/l1b 0 2024-07-16T09:55:28.652 INFO:tasks.workunit.client.1.smithi176.stdout:0/106: truncate d5/ff 1235615 0 2024-07-16T09:55:28.652 INFO:tasks.workunit.client.1.smithi176.stdout:9/34: symlink d5/la 0 2024-07-16T09:55:28.653 INFO:tasks.workunit.client.1.smithi176.stdout:7/61: truncate d4/f19 901755 0 2024-07-16T09:55:28.653 INFO:tasks.workunit.client.1.smithi176.stdout:7/62: chown d4 1041631360 1 2024-07-16T09:55:28.655 INFO:tasks.workunit.client.1.smithi176.stdout:0/107: mkdir d5/d1c 0 2024-07-16T09:55:28.659 INFO:tasks.workunit.client.1.smithi176.stdout:2/45: unlink cb 0 2024-07-16T09:55:28.659 INFO:tasks.workunit.client.1.smithi176.stdout:0/108: write d5/f13 [701113,77113] 0 2024-07-16T09:55:28.659 INFO:tasks.workunit.client.1.smithi176.stdout:9/35: creat d5/fb x:0 0 0 2024-07-16T09:55:28.660 INFO:tasks.workunit.client.0.smithi156.stdout:6/72: dwrite d3/fc [0,4194304] 0 2024-07-16T09:55:28.660 INFO:tasks.workunit.client.1.smithi176.stdout:7/63: mkdir d4/dd/d1c 0 2024-07-16T09:55:28.661 INFO:tasks.workunit.client.1.smithi176.stdout:4/96: getdents da/dc/d10/d15 0 2024-07-16T09:55:28.661 INFO:tasks.workunit.client.1.smithi176.stdout:7/64: write d4/dd/f1a [513560,62589] 0 2024-07-16T09:55:28.662 INFO:tasks.workunit.client.1.smithi176.stdout:5/187: link d4/d14/l1c d4/d14/d19/d1b/l43 0 2024-07-16T09:55:28.663 INFO:tasks.workunit.client.1.smithi176.stdout:2/46: chown l6 2 1 2024-07-16T09:55:28.664 INFO:tasks.workunit.client.0.smithi156.stdout:2/54: dwrite d0/f4 [0,4194304] 0 2024-07-16T09:55:28.664 INFO:tasks.workunit.client.0.smithi156.stdout:2/55: readlink d0/l2 0 2024-07-16T09:55:28.664 INFO:tasks.workunit.client.1.smithi176.stdout:9/36: symlink d5/lc 0 2024-07-16T09:55:28.665 INFO:tasks.workunit.client.1.smithi176.stdout:4/97: mknod da/dc/c1c 0 2024-07-16T09:55:28.667 INFO:tasks.workunit.client.1.smithi176.stdout:7/65: write d4/f11 [4902858,109354] 0 2024-07-16T09:55:28.668 INFO:tasks.workunit.client.1.smithi176.stdout:5/188: symlink d4/d14/d19/d1b/d36/l44 0 2024-07-16T09:55:28.668 INFO:tasks.workunit.client.1.smithi176.stdout:2/47: dread f3 [0,4194304] 0 2024-07-16T09:55:28.670 INFO:tasks.workunit.client.1.smithi176.stdout:2/48: write f9 [1640641,25925] 0 2024-07-16T09:55:28.672 INFO:tasks.workunit.client.0.smithi156.stdout:2/56: mkdir d0/d13 0 2024-07-16T09:55:28.672 INFO:tasks.workunit.client.1.smithi176.stdout:9/37: link d5/fb d5/fd 0 2024-07-16T09:55:28.673 INFO:tasks.workunit.client.1.smithi176.stdout:4/98: symlink da/d14/l1d 0 2024-07-16T09:55:28.675 INFO:tasks.workunit.client.1.smithi176.stdout:5/189: rmdir d4/d14/d3d 39 2024-07-16T09:55:28.676 INFO:tasks.workunit.client.1.smithi176.stdout:7/66: mkdir d4/d5/d10/d17/d1d 0 2024-07-16T09:55:28.678 INFO:tasks.workunit.client.0.smithi156.stdout:6/73: truncate d3/d8/fe 2335751 0 2024-07-16T09:55:28.679 INFO:tasks.workunit.client.1.smithi176.stdout:7/67: mknod d4/d5/d10/c1e 0 2024-07-16T09:55:28.680 INFO:tasks.workunit.client.0.smithi156.stdout:6/74: mkdir d3/d13/d17 0 2024-07-16T09:55:28.683 INFO:tasks.workunit.client.0.smithi156.stdout:6/75: chown d3/d8/fe 1687 1 2024-07-16T09:55:28.683 INFO:tasks.workunit.client.1.smithi176.stdout:5/190: rename d4/fc to d4/d14/f45 0 2024-07-16T09:55:28.687 INFO:tasks.workunit.client.1.smithi176.stdout:2/49: dwrite f3 [4194304,4194304] 0 2024-07-16T09:55:28.687 INFO:tasks.workunit.client.1.smithi176.stdout:0/109: getdents d5 0 2024-07-16T09:55:28.691 INFO:tasks.workunit.client.0.smithi156.stdout:2/57: fsync d0/f4 0 2024-07-16T09:55:28.692 INFO:tasks.workunit.client.1.smithi176.stdout:7/68: dwrite f0 [0,4194304] 0 2024-07-16T09:55:28.692 INFO:tasks.workunit.client.1.smithi176.stdout:2/50: mknod dd/cf 0 2024-07-16T09:55:28.692 INFO:tasks.workunit.client.0.smithi156.stdout:2/58: write d0/f4 [3155328,13228] 0 2024-07-16T09:55:28.693 INFO:tasks.workunit.client.1.smithi176.stdout:0/110: getdents d5 0 2024-07-16T09:55:28.694 INFO:tasks.workunit.client.1.smithi176.stdout:0/111: fsync f3 0 2024-07-16T09:55:28.695 INFO:tasks.workunit.client.0.smithi156.stdout:2/59: fsync d0/f4 0 2024-07-16T09:55:28.696 INFO:tasks.workunit.client.1.smithi176.stdout:9/38: truncate f2 1029931 0 2024-07-16T09:55:28.697 INFO:tasks.workunit.client.0.smithi156.stdout:2/60: truncate d0/d5/f12 1043372 0 2024-07-16T09:55:28.698 INFO:tasks.workunit.client.0.smithi156.stdout:2/61: write d0/f8 [4760583,66393] 0 2024-07-16T09:55:28.700 INFO:tasks.workunit.client.1.smithi176.stdout:0/112: dread - d5/f14 zero size 2024-07-16T09:55:28.702 INFO:tasks.workunit.client.0.smithi156.stdout:2/62: creat d0/f14 x:0 0 0 2024-07-16T09:55:28.703 INFO:tasks.workunit.client.1.smithi176.stdout:9/39: stat d5/la 0 2024-07-16T09:55:28.706 INFO:tasks.workunit.client.0.smithi156.stdout:2/63: symlink d0/l15 0 2024-07-16T09:55:28.706 INFO:tasks.workunit.client.0.smithi156.stdout:2/64: readlink d0/l15 0 2024-07-16T09:55:28.708 INFO:tasks.workunit.client.0.smithi156.stdout:2/65: write d0/fe [1290348,49403] 0 2024-07-16T09:55:28.709 INFO:tasks.workunit.client.1.smithi176.stdout:5/191: dwrite d4/f13 [0,4194304] 0 2024-07-16T09:55:28.709 INFO:tasks.workunit.client.1.smithi176.stdout:0/113: fdatasync d5/f15 0 2024-07-16T09:55:28.710 INFO:tasks.workunit.client.0.smithi156.stdout:2/66: stat d0/l15 0 2024-07-16T09:55:28.710 INFO:tasks.workunit.client.1.smithi176.stdout:9/40: mkdir d5/de 0 2024-07-16T09:55:28.712 INFO:tasks.workunit.client.0.smithi156.stdout:6/76: dwrite d3/d8/fe [0,4194304] 0 2024-07-16T09:55:28.715 INFO:tasks.workunit.client.0.smithi156.stdout:2/67: fsync d0/f7 0 2024-07-16T09:55:28.717 INFO:tasks.workunit.client.0.smithi156.stdout:6/77: symlink d3/l18 0 2024-07-16T09:55:28.718 INFO:tasks.workunit.client.1.smithi176.stdout:2/51: dwrite fa [0,4194304] 0 2024-07-16T09:55:28.719 INFO:tasks.workunit.client.1.smithi176.stdout:9/41: mknod d5/de/cf 0 2024-07-16T09:55:28.720 INFO:tasks.workunit.client.1.smithi176.stdout:7/69: dwrite d4/dd/f16 [0,4194304] 0 2024-07-16T09:55:28.721 INFO:tasks.workunit.client.1.smithi176.stdout:4/99: dwrite da/f1a [0,4194304] 0 2024-07-16T09:55:28.721 INFO:tasks.workunit.client.1.smithi176.stdout:2/52: write f9 [979215,96549] 0 2024-07-16T09:55:28.722 INFO:tasks.workunit.client.1.smithi176.stdout:5/192: truncate d4/f5 953829 0 2024-07-16T09:55:28.723 INFO:tasks.workunit.client.1.smithi176.stdout:7/70: write d4/d5/f12 [909803,59815] 0 2024-07-16T09:55:28.725 INFO:tasks.workunit.client.0.smithi156.stdout:6/78: dread d3/d8/f15 [0,4194304] 0 2024-07-16T09:55:28.727 INFO:tasks.workunit.client.1.smithi176.stdout:4/100: symlink da/dc/l1e 0 2024-07-16T09:55:28.729 INFO:tasks.workunit.client.0.smithi156.stdout:2/68: dwrite d0/d5/f12 [0,4194304] 0 2024-07-16T09:55:28.729 INFO:tasks.workunit.client.1.smithi176.stdout:5/193: symlink d4/d10/d17/l46 0 2024-07-16T09:55:28.732 INFO:tasks.workunit.client.0.smithi156.stdout:2/69: truncate d0/f14 156879 0 2024-07-16T09:55:28.733 INFO:tasks.workunit.client.1.smithi176.stdout:2/53: creat dd/de/f10 x:0 0 0 2024-07-16T09:55:28.734 INFO:tasks.workunit.client.1.smithi176.stdout:7/71: dwrite d4/dd/f16 [0,4194304] 0 2024-07-16T09:55:28.739 INFO:tasks.workunit.client.0.smithi156.stdout:6/79: dwrite d3/d8/fe [0,4194304] 0 2024-07-16T09:55:28.743 INFO:tasks.workunit.client.0.smithi156.stdout:2/70: rename d0/d5/f12 to d0/f16 0 2024-07-16T09:55:28.744 INFO:tasks.workunit.client.1.smithi176.stdout:7/72: chown d4/d5/l7 752807424 1 2024-07-16T09:55:28.744 INFO:tasks.workunit.client.1.smithi176.stdout:7/73: dread - d4/fb zero size 2024-07-16T09:55:28.744 INFO:tasks.workunit.client.1.smithi176.stdout:5/194: dread d4/d14/f45 [0,4194304] 0 2024-07-16T09:55:28.746 INFO:tasks.workunit.client.1.smithi176.stdout:4/101: dread da/f1a [0,4194304] 0 2024-07-16T09:55:28.746 INFO:tasks.workunit.client.1.smithi176.stdout:0/114: truncate d5/f12 2850159 0 2024-07-16T09:55:28.748 INFO:tasks.workunit.client.1.smithi176.stdout:0/115: write d5/f18 [49816,41373] 0 2024-07-16T09:55:28.749 INFO:tasks.workunit.client.0.smithi156.stdout:2/71: mkdir d0/d13/d17 0 2024-07-16T09:55:28.749 INFO:tasks.workunit.client.1.smithi176.stdout:7/74: rename d4/dd/d1c to d4/d1f 0 2024-07-16T09:55:28.749 INFO:tasks.workunit.client.1.smithi176.stdout:2/54: dread f3 [4194304,4194304] 0 2024-07-16T09:55:28.749 INFO:tasks.workunit.client.1.smithi176.stdout:4/102: chown da/dc/d10/d15/f1b 2550558 1 2024-07-16T09:55:28.749 INFO:tasks.workunit.client.0.smithi156.stdout:2/72: rename d0/d13/d17 to d0/d13/d17/d18 22 2024-07-16T09:55:28.750 INFO:tasks.workunit.client.0.smithi156.stdout:2/73: fsync d0/fe 0 2024-07-16T09:55:28.750 INFO:tasks.workunit.client.1.smithi176.stdout:7/75: write d4/dd/f1a [784531,103609] 0 2024-07-16T09:55:28.751 INFO:tasks.workunit.client.1.smithi176.stdout:5/195: dread d4/d14/d19/d1b/d36/d38/d39/f3a [0,4194304] 0 2024-07-16T09:55:28.752 INFO:tasks.workunit.client.0.smithi156.stdout:2/74: creat d0/d5/f19 x:0 0 0 2024-07-16T09:55:28.753 INFO:tasks.workunit.client.1.smithi176.stdout:7/76: read - d4/dd/f15 zero size 2024-07-16T09:55:28.755 INFO:tasks.workunit.client.0.smithi156.stdout:2/75: creat d0/d13/f1a x:0 0 0 2024-07-16T09:55:28.756 INFO:tasks.workunit.client.1.smithi176.stdout:4/103: symlink da/dc/d10/d15/d17/l1f 0 2024-07-16T09:55:28.757 INFO:tasks.workunit.client.0.smithi156.stdout:2/76: write d0/d5/f19 [613616,56044] 0 2024-07-16T09:55:28.757 INFO:tasks.workunit.client.1.smithi176.stdout:2/55: rename l0 to dd/de/l11 0 2024-07-16T09:55:28.758 INFO:tasks.workunit.client.1.smithi176.stdout:4/104: write da/f19 [3706959,68364] 0 2024-07-16T09:55:28.758 INFO:tasks.workunit.client.0.smithi156.stdout:6/80: write d3/d8/fa [1913403,3881] 0 2024-07-16T09:55:28.759 INFO:tasks.workunit.client.1.smithi176.stdout:5/196: dwrite d4/fe [4194304,4194304] 0 2024-07-16T09:55:28.759 INFO:tasks.workunit.client.1.smithi176.stdout:4/105: truncate da/f19 5866483 0 2024-07-16T09:55:28.760 INFO:tasks.workunit.client.1.smithi176.stdout:0/116: symlink d5/d1c/l1d 0 2024-07-16T09:55:28.761 INFO:tasks.workunit.client.1.smithi176.stdout:7/77: mknod d4/d5/c20 0 2024-07-16T09:55:28.762 INFO:tasks.workunit.client.1.smithi176.stdout:2/56: mknod dd/de/c12 0 2024-07-16T09:55:28.763 INFO:tasks.workunit.client.1.smithi176.stdout:2/57: fsync fa 0 2024-07-16T09:55:28.764 INFO:tasks.workunit.client.1.smithi176.stdout:0/117: creat d5/f1e x:0 0 0 2024-07-16T09:55:28.766 INFO:tasks.workunit.client.1.smithi176.stdout:0/118: chown d5/c1a 332 1 2024-07-16T09:55:28.767 INFO:tasks.workunit.client.1.smithi176.stdout:7/78: mkdir d4/d5/d21 0 2024-07-16T09:55:28.767 INFO:tasks.workunit.client.1.smithi176.stdout:9/42: write f2 [1017050,91057] 0 2024-07-16T09:55:28.768 INFO:tasks.workunit.client.1.smithi176.stdout:7/79: chown d4/f19 73614 1 2024-07-16T09:55:28.768 INFO:tasks.workunit.client.0.smithi156.stdout:2/77: dwrite d0/f14 [0,4194304] 0 2024-07-16T09:55:28.768 INFO:tasks.workunit.client.1.smithi176.stdout:7/80: write d4/f14 [1267220,61559] 0 2024-07-16T09:55:28.769 INFO:tasks.workunit.client.1.smithi176.stdout:2/58: mknod dd/de/c13 0 2024-07-16T09:55:28.770 INFO:tasks.workunit.client.1.smithi176.stdout:2/59: write fa [2407100,55974] 0 2024-07-16T09:55:28.770 INFO:tasks.workunit.client.1.smithi176.stdout:2/60: fsync f9 0 2024-07-16T09:55:28.771 INFO:tasks.workunit.client.0.smithi156.stdout:2/78: creat d0/d5/f1b x:0 0 0 2024-07-16T09:55:28.772 INFO:tasks.workunit.client.1.smithi176.stdout:7/81: mkdir d4/d1f/d22 0 2024-07-16T09:55:28.774 INFO:tasks.workunit.client.0.smithi156.stdout:2/79: mkdir d0/d13/d17/d1c 0 2024-07-16T09:55:28.775 INFO:tasks.workunit.client.1.smithi176.stdout:0/119: link f4 d5/d1c/f1f 0 2024-07-16T09:55:28.776 INFO:tasks.workunit.client.1.smithi176.stdout:0/120: readlink l2 0 2024-07-16T09:55:28.779 INFO:tasks.workunit.client.0.smithi156.stdout:6/81: truncate d3/d8/fe 404616 0 2024-07-16T09:55:28.779 INFO:tasks.workunit.client.0.smithi156.stdout:2/80: dread d0/f8 [0,4194304] 0 2024-07-16T09:55:28.783 INFO:tasks.workunit.client.0.smithi156.stdout:2/81: symlink d0/d13/d17/d1c/l1d 0 2024-07-16T09:55:28.783 INFO:tasks.workunit.client.1.smithi176.stdout:4/106: rmdir da 39 2024-07-16T09:55:28.784 INFO:tasks.workunit.client.1.smithi176.stdout:7/82: creat d4/dd/f23 x:0 0 0 2024-07-16T09:55:28.784 INFO:tasks.workunit.client.0.smithi156.stdout:2/82: fsync d0/f8 0 2024-07-16T09:55:28.784 INFO:tasks.workunit.client.1.smithi176.stdout:5/197: dwrite d4/d14/d3d/f41 [0,4194304] 0 2024-07-16T09:55:28.786 INFO:tasks.workunit.client.1.smithi176.stdout:7/83: dread - d4/dd/f23 zero size 2024-07-16T09:55:28.789 INFO:tasks.workunit.client.1.smithi176.stdout:2/61: rename dd/de/l11 to dd/de/l14 0 2024-07-16T09:55:28.789 INFO:tasks.workunit.client.1.smithi176.stdout:0/121: dwrite d5/f7 [0,4194304] 0 2024-07-16T09:55:28.791 INFO:tasks.workunit.client.1.smithi176.stdout:0/122: fdatasync d5/f14 0 2024-07-16T09:55:28.791 INFO:tasks.workunit.client.1.smithi176.stdout:7/84: readlink d4/dd/l13 0 2024-07-16T09:55:28.792 INFO:tasks.workunit.client.1.smithi176.stdout:2/62: mknod dd/c15 0 2024-07-16T09:55:28.795 INFO:tasks.workunit.client.1.smithi176.stdout:2/63: truncate dd/de/f10 833370 0 2024-07-16T09:55:28.796 INFO:tasks.workunit.client.1.smithi176.stdout:0/123: creat d5/d1c/f20 x:0 0 0 2024-07-16T09:55:28.798 INFO:tasks.workunit.client.1.smithi176.stdout:7/85: dread d4/f14 [0,4194304] 0 2024-07-16T09:55:28.798 INFO:tasks.workunit.client.0.smithi156.stdout:6/82: dwrite d3/d8/f15 [0,4194304] 0 2024-07-16T09:55:28.799 INFO:tasks.workunit.client.1.smithi176.stdout:7/86: chown d4/dd/f1a 510 1 2024-07-16T09:55:28.799 INFO:tasks.workunit.client.1.smithi176.stdout:7/87: chown d4/d1f 1127 1 2024-07-16T09:55:28.799 INFO:tasks.workunit.client.1.smithi176.stdout:2/64: mkdir dd/d16 0 2024-07-16T09:55:28.800 INFO:tasks.workunit.client.0.smithi156.stdout:2/83: dwrite d0/f7 [0,4194304] 0 2024-07-16T09:55:28.802 INFO:tasks.workunit.client.1.smithi176.stdout:0/124: creat d5/d1c/f21 x:0 0 0 2024-07-16T09:55:28.802 INFO:tasks.workunit.client.0.smithi156.stdout:6/83: mkdir d3/d8/d19 0 2024-07-16T09:55:28.803 INFO:tasks.workunit.client.1.smithi176.stdout:4/107: rename da/dc/d10/d15/c16 to da/c20 0 2024-07-16T09:55:28.803 INFO:tasks.workunit.client.0.smithi156.stdout:2/84: creat d0/d5/f1e x:0 0 0 2024-07-16T09:55:28.803 INFO:tasks.workunit.client.0.smithi156.stdout:6/84: dread d3/d8/fe [0,4194304] 0 2024-07-16T09:55:28.804 INFO:tasks.workunit.client.0.smithi156.stdout:2/85: stat d0/d13/d17/d1c 0 2024-07-16T09:55:28.804 INFO:tasks.workunit.client.1.smithi176.stdout:9/43: truncate f2 966579 0 2024-07-16T09:55:28.805 INFO:tasks.workunit.client.1.smithi176.stdout:4/108: chown da/dc/d10/d15/f1b 871 1 2024-07-16T09:55:28.805 INFO:tasks.workunit.client.1.smithi176.stdout:2/65: fdatasync f3 0 2024-07-16T09:55:28.807 INFO:tasks.workunit.client.1.smithi176.stdout:2/66: write f9 [667166,84322] 0 2024-07-16T09:55:28.809 INFO:tasks.workunit.client.1.smithi176.stdout:7/88: dwrite d4/dd/f16 [0,4194304] 0 2024-07-16T09:55:28.810 INFO:tasks.workunit.client.1.smithi176.stdout:4/109: symlink da/dc/d10/d15/d17/l21 0 2024-07-16T09:55:28.811 INFO:tasks.workunit.client.0.smithi156.stdout:6/85: dwrite d3/d8/fa [0,4194304] 0 2024-07-16T09:55:28.812 INFO:tasks.workunit.client.1.smithi176.stdout:2/67: symlink dd/d16/l17 0 2024-07-16T09:55:28.812 INFO:tasks.workunit.client.0.smithi156.stdout:6/86: write d3/d8/f10 [3963819,6218] 0 2024-07-16T09:55:28.813 INFO:tasks.workunit.client.1.smithi176.stdout:0/125: dread d5/f6 [0,4194304] 0 2024-07-16T09:55:28.813 INFO:tasks.workunit.client.1.smithi176.stdout:7/89: mknod d4/d5/d21/c24 0 2024-07-16T09:55:28.814 INFO:tasks.workunit.client.1.smithi176.stdout:7/90: chown d4/d5/d10 115866 1 2024-07-16T09:55:28.816 INFO:tasks.workunit.client.1.smithi176.stdout:7/91: dread - d4/dd/f15 zero size 2024-07-16T09:55:28.817 INFO:tasks.workunit.client.0.smithi156.stdout:2/86: rename d0/d5/f19 to d0/d13/d17/d1c/f1f 0 2024-07-16T09:55:28.817 INFO:tasks.workunit.client.1.smithi176.stdout:2/68: rmdir dd 39 2024-07-16T09:55:28.819 INFO:tasks.workunit.client.1.smithi176.stdout:2/69: write fa [2868615,49862] 0 2024-07-16T09:55:28.819 INFO:tasks.workunit.client.1.smithi176.stdout:7/92: write d4/dd/f23 [3746,38096] 0 2024-07-16T09:55:28.821 INFO:tasks.workunit.client.0.smithi156.stdout:2/87: write d0/f16 [3673947,47346] 0 2024-07-16T09:55:28.823 INFO:tasks.workunit.client.0.smithi156.stdout:6/87: dwrite d3/fc [0,4194304] 0 2024-07-16T09:55:28.824 INFO:tasks.workunit.client.1.smithi176.stdout:0/126: rename f1 to d5/f22 0 2024-07-16T09:55:28.824 INFO:tasks.workunit.client.0.smithi156.stdout:6/88: chown d3/d13 41 1 2024-07-16T09:55:28.825 INFO:tasks.workunit.client.0.smithi156.stdout:2/88: read - d0/d5/f1b zero size 2024-07-16T09:55:28.826 INFO:tasks.workunit.client.1.smithi176.stdout:2/70: creat dd/de/f18 x:0 0 0 2024-07-16T09:55:28.826 INFO:tasks.workunit.client.0.smithi156.stdout:2/89: write d0/f4 [2483467,76471] 0 2024-07-16T09:55:28.827 INFO:tasks.workunit.client.1.smithi176.stdout:0/127: creat d5/f23 x:0 0 0 2024-07-16T09:55:28.829 INFO:tasks.workunit.client.0.smithi156.stdout:6/89: symlink d3/d13/l1a 0 2024-07-16T09:55:28.829 INFO:tasks.workunit.client.0.smithi156.stdout:2/90: write d0/d5/f1b [374797,39743] 0 2024-07-16T09:55:28.830 INFO:tasks.workunit.client.1.smithi176.stdout:2/71: dread f3 [0,4194304] 0 2024-07-16T09:55:28.831 INFO:tasks.workunit.client.0.smithi156.stdout:2/91: dread - d0/d13/f1a zero size 2024-07-16T09:55:28.831 INFO:tasks.workunit.client.0.smithi156.stdout:2/92: stat d0/f16 0 2024-07-16T09:55:28.833 INFO:tasks.workunit.client.1.smithi176.stdout:7/93: getdents d4/d5/d10/d17 0 2024-07-16T09:55:28.833 INFO:tasks.workunit.client.0.smithi156.stdout:6/90: symlink d3/d13/d17/l1b 0 2024-07-16T09:55:28.834 INFO:tasks.workunit.client.0.smithi156.stdout:2/93: mknod d0/c20 0 2024-07-16T09:55:28.834 INFO:tasks.workunit.client.1.smithi176.stdout:2/72: unlink l6 0 2024-07-16T09:55:28.835 INFO:tasks.workunit.client.0.smithi156.stdout:2/94: dread - d0/d13/f1a zero size 2024-07-16T09:55:28.835 INFO:tasks.workunit.client.0.smithi156.stdout:6/91: creat d3/f1c x:0 0 0 2024-07-16T09:55:28.835 INFO:tasks.workunit.client.1.smithi176.stdout:2/73: truncate dd/de/f18 1012862 0 2024-07-16T09:55:28.836 INFO:tasks.workunit.client.1.smithi176.stdout:7/94: rename d4/d5/c20 to d4/c25 0 2024-07-16T09:55:28.838 INFO:tasks.workunit.client.0.smithi156.stdout:6/92: rename d3/fc to d3/f1d 0 2024-07-16T09:55:28.838 INFO:tasks.workunit.client.0.smithi156.stdout:2/95: dread d0/f4 [0,4194304] 0 2024-07-16T09:55:28.839 INFO:tasks.workunit.client.1.smithi176.stdout:7/95: rmdir d4/dd 39 2024-07-16T09:55:28.841 INFO:tasks.workunit.client.0.smithi156.stdout:2/96: creat d0/d13/d17/f21 x:0 0 0 2024-07-16T09:55:28.842 INFO:tasks.workunit.client.1.smithi176.stdout:2/74: dwrite fa [0,4194304] 0 2024-07-16T09:55:28.842 INFO:tasks.workunit.client.1.smithi176.stdout:7/96: mkdir d4/dd/d26 0 2024-07-16T09:55:28.844 INFO:tasks.workunit.client.0.smithi156.stdout:2/97: mknod d0/d13/d17/d1c/c22 0 2024-07-16T09:55:28.845 INFO:tasks.workunit.client.0.smithi156.stdout:2/98: readlink d0/l6 0 2024-07-16T09:55:28.846 INFO:tasks.workunit.client.0.smithi156.stdout:6/93: dwrite d3/f1c [0,4194304] 0 2024-07-16T09:55:28.850 INFO:tasks.workunit.client.0.smithi156.stdout:2/99: creat d0/d13/d17/f23 x:0 0 0 2024-07-16T09:55:28.852 INFO:tasks.workunit.client.1.smithi176.stdout:4/110: getdents da 0 2024-07-16T09:55:28.852 INFO:tasks.workunit.client.0.smithi156.stdout:2/100: chown d0/l6 7147 1 2024-07-16T09:55:28.852 INFO:tasks.workunit.client.1.smithi176.stdout:4/111: readlink l4 0 2024-07-16T09:55:28.854 INFO:tasks.workunit.client.1.smithi176.stdout:7/97: dwrite d4/f11 [4194304,4194304] 0 2024-07-16T09:55:28.855 INFO:tasks.workunit.client.1.smithi176.stdout:2/75: dwrite dd/de/f10 [0,4194304] 0 2024-07-16T09:55:28.856 INFO:tasks.workunit.client.0.smithi156.stdout:6/94: dwrite d3/d8/f10 [0,4194304] 0 2024-07-16T09:55:28.857 INFO:tasks.workunit.client.1.smithi176.stdout:2/76: stat dd 0 2024-07-16T09:55:28.858 INFO:tasks.workunit.client.1.smithi176.stdout:2/77: rename dd to dd/d19 22 2024-07-16T09:55:28.858 INFO:tasks.workunit.client.0.smithi156.stdout:2/101: creat d0/d13/d17/d1c/f24 x:0 0 0 2024-07-16T09:55:28.859 INFO:tasks.workunit.client.0.smithi156.stdout:6/95: dread d3/d8/fe [0,4194304] 0 2024-07-16T09:55:28.860 INFO:tasks.workunit.client.0.smithi156.stdout:2/102: mknod d0/d13/c25 0 2024-07-16T09:55:28.861 INFO:tasks.workunit.client.0.smithi156.stdout:2/103: chown d0/d13/d17 16564 1 2024-07-16T09:55:28.862 INFO:tasks.workunit.client.0.smithi156.stdout:6/96: mkdir d3/d4/d1e 0 2024-07-16T09:55:28.863 INFO:tasks.workunit.client.1.smithi176.stdout:7/98: write d4/f14 [1415126,10037] 0 2024-07-16T09:55:28.869 INFO:tasks.workunit.client.0.smithi156.stdout:6/97: rename d3/d8 to d3/d4/d1f 0 2024-07-16T09:55:28.870 INFO:tasks.workunit.client.1.smithi176.stdout:4/112: dread da/dc/d10/d15/f1b [0,4194304] 0 2024-07-16T09:55:28.871 INFO:tasks.workunit.client.1.smithi176.stdout:2/78: rename dd/d16/l17 to dd/d16/l1a 0 2024-07-16T09:55:28.871 INFO:tasks.workunit.client.1.smithi176.stdout:9/44: dwrite d5/fb [0,4194304] 0 2024-07-16T09:55:28.873 INFO:tasks.workunit.client.1.smithi176.stdout:4/113: write da/f19 [4263056,59964] 0 2024-07-16T09:55:28.873 INFO:tasks.workunit.client.0.smithi156.stdout:6/98: creat d3/d4/d1e/f20 x:0 0 0 2024-07-16T09:55:28.874 INFO:tasks.workunit.client.0.smithi156.stdout:6/99: chown d3/d13/d17/l1b 216 1 2024-07-16T09:55:28.876 INFO:tasks.workunit.client.1.smithi176.stdout:7/99: creat d4/f27 x:0 0 0 2024-07-16T09:55:28.879 INFO:tasks.workunit.client.0.smithi156.stdout:6/100: symlink d3/d13/d17/l21 0 2024-07-16T09:55:28.880 INFO:tasks.workunit.client.0.smithi156.stdout:6/101: stat d3/d4/lb 0 2024-07-16T09:55:28.880 INFO:tasks.workunit.client.1.smithi176.stdout:7/100: chown d4/d5/l7 4947 1 2024-07-16T09:55:28.882 INFO:tasks.workunit.client.1.smithi176.stdout:2/79: link c8 dd/de/c1b 0 2024-07-16T09:55:28.884 INFO:tasks.workunit.client.1.smithi176.stdout:2/80: write f3 [723837,94323] 0 2024-07-16T09:55:28.886 INFO:tasks.workunit.client.1.smithi176.stdout:7/101: mknod d4/d5/d10/d17/d1d/c28 0 2024-07-16T09:55:28.887 INFO:tasks.workunit.client.1.smithi176.stdout:2/81: rmdir dd/d16 39 2024-07-16T09:55:28.889 INFO:tasks.workunit.client.1.smithi176.stdout:4/114: dwrite da/f19 [4194304,4194304] 0 2024-07-16T09:55:28.890 INFO:tasks.workunit.client.1.smithi176.stdout:9/45: dwrite f2 [0,4194304] 0 2024-07-16T09:55:28.894 INFO:tasks.workunit.client.0.smithi156.stdout:2/104: getdents d0/d13 0 2024-07-16T09:55:28.895 INFO:tasks.workunit.client.1.smithi176.stdout:7/102: getdents d4/d1f 0 2024-07-16T09:55:28.896 INFO:tasks.workunit.client.1.smithi176.stdout:4/115: write da/f1a [589447,100445] 0 2024-07-16T09:55:28.897 INFO:tasks.workunit.client.1.smithi176.stdout:9/46: rename d5/f9 to d5/f10 0 2024-07-16T09:55:28.898 INFO:tasks.workunit.client.1.smithi176.stdout:4/116: read da/fb [152284,17933] 0 2024-07-16T09:55:28.899 INFO:tasks.workunit.client.0.smithi156.stdout:2/105: dread d0/f8 [0,4194304] 0 2024-07-16T09:55:28.900 INFO:tasks.workunit.client.1.smithi176.stdout:2/82: rename dd/de/f18 to dd/d16/f1c 0 2024-07-16T09:55:28.901 INFO:tasks.workunit.client.1.smithi176.stdout:0/128: dwrite d5/f12 [0,4194304] 0 2024-07-16T09:55:28.901 INFO:tasks.workunit.client.1.smithi176.stdout:2/83: chown dd/cf 16 1 2024-07-16T09:55:28.905 INFO:tasks.workunit.client.1.smithi176.stdout:0/129: dread d5/f13 [0,4194304] 0 2024-07-16T09:55:28.910 INFO:tasks.workunit.client.1.smithi176.stdout:9/47: mknod d5/de/c11 0 2024-07-16T09:55:28.910 INFO:tasks.workunit.client.1.smithi176.stdout:4/117: rmdir da 39 2024-07-16T09:55:28.911 INFO:tasks.workunit.client.0.smithi156.stdout:2/106: dwrite d0/d13/d17/f21 [0,4194304] 0 2024-07-16T09:55:28.913 INFO:tasks.workunit.client.1.smithi176.stdout:9/48: dread f3 [0,4194304] 0 2024-07-16T09:55:28.913 INFO:tasks.workunit.client.1.smithi176.stdout:7/103: symlink d4/d1f/d22/l29 0 2024-07-16T09:55:28.913 INFO:tasks.workunit.client.0.smithi156.stdout:2/107: creat d0/d13/f26 x:0 0 0 2024-07-16T09:55:28.915 INFO:tasks.workunit.client.0.smithi156.stdout:2/108: write d0/d13/d17/f23 [959011,5774] 0 2024-07-16T09:55:28.915 INFO:tasks.workunit.client.1.smithi176.stdout:0/130: mkdir d5/d1c/d24 0 2024-07-16T09:55:28.921 INFO:tasks.workunit.client.1.smithi176.stdout:2/84: unlink dd/d16/l1a 0 2024-07-16T09:55:28.923 INFO:tasks.workunit.client.1.smithi176.stdout:2/85: fsync dd/de/f10 0 2024-07-16T09:55:28.923 INFO:tasks.workunit.client.1.smithi176.stdout:7/104: creat d4/d1f/f2a x:0 0 0 2024-07-16T09:55:28.925 INFO:tasks.workunit.client.1.smithi176.stdout:7/105: write d4/dd/f1a [1424331,107077] 0 2024-07-16T09:55:28.926 INFO:tasks.workunit.client.0.smithi156.stdout:2/109: dwrite d0/d13/d17/d1c/f1f [0,4194304] 0 2024-07-16T09:55:28.928 INFO:tasks.workunit.client.1.smithi176.stdout:2/86: chown c8 2802 1 2024-07-16T09:55:28.931 INFO:tasks.workunit.client.1.smithi176.stdout:9/49: unlink f3 0 2024-07-16T09:55:28.931 INFO:tasks.workunit.client.1.smithi176.stdout:0/131: dwrite d5/f14 [0,4194304] 0 2024-07-16T09:55:28.932 INFO:tasks.workunit.client.0.smithi156.stdout:2/110: link d0/f14 d0/d13/d17/d1c/f27 0 2024-07-16T09:55:28.933 INFO:tasks.workunit.client.1.smithi176.stdout:7/106: dread d4/d5/f12 [0,4194304] 0 2024-07-16T09:55:28.934 INFO:tasks.workunit.client.1.smithi176.stdout:4/118: rename l7 to da/dc/d10/d15/l22 0 2024-07-16T09:55:28.937 INFO:tasks.workunit.client.1.smithi176.stdout:0/132: mkdir d5/d25 0 2024-07-16T09:55:28.938 INFO:tasks.workunit.client.0.smithi156.stdout:2/111: dread d0/d13/d17/d1c/f27 [0,4194304] 0 2024-07-16T09:55:28.938 INFO:tasks.workunit.client.1.smithi176.stdout:7/107: truncate d4/dd/f16 4473769 0 2024-07-16T09:55:28.939 INFO:tasks.workunit.client.1.smithi176.stdout:2/87: dread fa [0,4194304] 0 2024-07-16T09:55:28.939 INFO:tasks.workunit.client.1.smithi176.stdout:0/133: chown d5/c11 0 1 2024-07-16T09:55:28.939 INFO:tasks.workunit.client.1.smithi176.stdout:7/108: dread - d4/fb zero size 2024-07-16T09:55:28.939 INFO:tasks.workunit.client.1.smithi176.stdout:9/50: dread d5/fd [0,4194304] 0 2024-07-16T09:55:28.940 INFO:tasks.workunit.client.1.smithi176.stdout:4/119: mknod da/c23 0 2024-07-16T09:55:28.941 INFO:tasks.workunit.client.1.smithi176.stdout:9/51: chown d5/la 11558593 1 2024-07-16T09:55:28.943 INFO:tasks.workunit.client.0.smithi156.stdout:2/112: rename d0/d5/f1b to d0/f28 0 2024-07-16T09:55:28.946 INFO:tasks.workunit.client.1.smithi176.stdout:9/52: write d5/fd [4829905,15846] 0 2024-07-16T09:55:28.946 INFO:tasks.workunit.client.1.smithi176.stdout:0/134: mkdir d5/d1c/d24/d26 0 2024-07-16T09:55:28.947 INFO:tasks.workunit.client.0.smithi156.stdout:2/113: write d0/d13/d17/d1c/f27 [1667767,9751] 0 2024-07-16T09:55:28.947 INFO:tasks.workunit.client.1.smithi176.stdout:7/109: creat d4/f2b x:0 0 0 2024-07-16T09:55:28.948 INFO:tasks.workunit.client.0.smithi156.stdout:2/114: dread - d0/d5/f1e zero size 2024-07-16T09:55:28.949 INFO:tasks.workunit.client.1.smithi176.stdout:0/135: write d5/d1c/f20 [224009,122520] 0 2024-07-16T09:55:28.949 INFO:tasks.workunit.client.1.smithi176.stdout:9/53: write d5/f10 [513694,44931] 0 2024-07-16T09:55:28.950 INFO:tasks.workunit.client.1.smithi176.stdout:7/110: symlink d4/d1f/d22/l2c 0 2024-07-16T09:55:28.951 INFO:tasks.workunit.client.0.smithi156.stdout:2/115: mkdir d0/d5/d29 0 2024-07-16T09:55:28.951 INFO:tasks.workunit.client.1.smithi176.stdout:2/88: dwrite f9 [0,4194304] 0 2024-07-16T09:55:28.952 INFO:tasks.workunit.client.1.smithi176.stdout:4/120: rmdir da 39 2024-07-16T09:55:28.954 INFO:tasks.workunit.client.1.smithi176.stdout:2/89: read dd/d16/f1c [369075,128036] 0 2024-07-16T09:55:28.954 INFO:tasks.workunit.client.1.smithi176.stdout:7/111: symlink d4/d1f/d22/l2d 0 2024-07-16T09:55:28.955 INFO:tasks.workunit.client.1.smithi176.stdout:7/112: write d4/f19 [749934,66561] 0 2024-07-16T09:55:28.955 INFO:tasks.workunit.client.1.smithi176.stdout:7/113: stat d4/c9 0 2024-07-16T09:55:28.955 INFO:tasks.workunit.client.1.smithi176.stdout:7/114: fsync d4/dd/f23 0 2024-07-16T09:55:28.956 INFO:tasks.workunit.client.1.smithi176.stdout:7/115: chown d4/d1f/f2a 2230575 1 2024-07-16T09:55:28.957 INFO:tasks.workunit.client.1.smithi176.stdout:2/90: creat dd/d16/f1d x:0 0 0 2024-07-16T09:55:28.960 INFO:tasks.workunit.client.0.smithi156.stdout:2/116: dwrite d0/f14 [0,4194304] 0 2024-07-16T09:55:28.961 INFO:tasks.workunit.client.0.smithi156.stdout:2/117: dread - d0/d13/f26 zero size 2024-07-16T09:55:28.962 INFO:tasks.workunit.client.1.smithi176.stdout:7/116: creat d4/dd/f2e x:0 0 0 2024-07-16T09:55:28.962 INFO:tasks.workunit.client.1.smithi176.stdout:7/117: fsync d4/d1f/f2a 0 2024-07-16T09:55:28.962 INFO:tasks.workunit.client.1.smithi176.stdout:4/121: mknod da/dc/c24 0 2024-07-16T09:55:28.963 INFO:tasks.workunit.client.1.smithi176.stdout:7/118: chown d4/d5/d10/d17 2130732 1 2024-07-16T09:55:28.963 INFO:tasks.workunit.client.1.smithi176.stdout:7/119: stat d4/d1f 0 2024-07-16T09:55:28.971 INFO:tasks.workunit.client.1.smithi176.stdout:7/120: link d4/d1f/f2a d4/d5/f2f 0 2024-07-16T09:55:28.971 INFO:tasks.workunit.client.0.smithi156.stdout:2/118: dwrite d0/f28 [0,4194304] 0 2024-07-16T09:55:28.972 INFO:tasks.workunit.client.1.smithi176.stdout:4/122: getdents da/dc/d10 0 2024-07-16T09:55:28.974 INFO:tasks.workunit.client.0.smithi156.stdout:2/119: creat d0/d5/f2a x:0 0 0 2024-07-16T09:55:28.976 INFO:tasks.workunit.client.1.smithi176.stdout:4/123: mknod da/dc/c25 0 2024-07-16T09:55:28.976 INFO:tasks.workunit.client.0.smithi156.stdout:2/120: mknod d0/c2b 0 2024-07-16T09:55:28.976 INFO:tasks.workunit.client.1.smithi176.stdout:9/54: truncate d5/fb 3545542 0 2024-07-16T09:55:28.977 INFO:tasks.workunit.client.1.smithi176.stdout:4/124: readlink l4 0 2024-07-16T09:55:28.977 INFO:tasks.workunit.client.1.smithi176.stdout:0/136: truncate d5/d1c/f20 130649 0 2024-07-16T09:55:28.981 INFO:tasks.workunit.client.1.smithi176.stdout:4/125: read f0 [438934,101218] 0 2024-07-16T09:55:28.981 INFO:tasks.workunit.client.1.smithi176.stdout:7/121: dwrite f0 [4194304,4194304] 0 2024-07-16T09:55:28.981 INFO:tasks.workunit.client.1.smithi176.stdout:2/91: truncate f9 1716617 0 2024-07-16T09:55:28.984 INFO:tasks.workunit.client.1.smithi176.stdout:2/92: rename dd/de to dd/de/d1e 22 2024-07-16T09:55:28.986 INFO:tasks.workunit.client.1.smithi176.stdout:7/122: mknod d4/c30 0 2024-07-16T09:55:28.988 INFO:tasks.workunit.client.1.smithi176.stdout:7/123: chown d4/d1f/d22/l2d 5 1 2024-07-16T09:55:28.988 INFO:tasks.workunit.client.1.smithi176.stdout:2/93: mknod dd/c1f 0 2024-07-16T09:55:28.990 INFO:tasks.workunit.client.1.smithi176.stdout:0/137: dwrite d5/f18 [0,4194304] 0 2024-07-16T09:55:28.995 INFO:tasks.workunit.client.1.smithi176.stdout:7/124: creat d4/d5/f31 x:0 0 0 2024-07-16T09:55:28.996 INFO:tasks.workunit.client.1.smithi176.stdout:9/55: dread d5/fb [0,4194304] 0 2024-07-16T09:55:28.997 INFO:tasks.workunit.client.1.smithi176.stdout:2/94: write dd/d16/f1c [1925518,32580] 0 2024-07-16T09:55:28.998 INFO:tasks.workunit.client.1.smithi176.stdout:0/138: creat d5/d1c/f27 x:0 0 0 2024-07-16T09:55:28.998 INFO:tasks.workunit.client.1.smithi176.stdout:9/56: chown d5/l7 0 1 2024-07-16T09:55:28.998 INFO:tasks.workunit.client.0.smithi156.stdout:2/121: creat d0/f2c x:0 0 0 2024-07-16T09:55:28.999 INFO:tasks.workunit.client.1.smithi176.stdout:0/139: read - d5/d1c/f27 zero size 2024-07-16T09:55:28.999 INFO:tasks.workunit.client.1.smithi176.stdout:2/95: read dd/de/f10 [2646324,96082] 0 2024-07-16T09:55:29.000 INFO:tasks.workunit.client.1.smithi176.stdout:7/125: fdatasync d4/d1f/f2a 0 2024-07-16T09:55:29.000 INFO:tasks.workunit.client.0.smithi156.stdout:2/122: write d0/d13/d17/f21 [4864486,89550] 0 2024-07-16T09:55:29.000 INFO:tasks.workunit.client.1.smithi176.stdout:0/140: chown d5 880440680 1 2024-07-16T09:55:29.001 INFO:tasks.workunit.client.1.smithi176.stdout:9/57: read f2 [1046880,126222] 0 2024-07-16T09:55:29.002 INFO:tasks.workunit.client.1.smithi176.stdout:4/126: truncate da/dc/d10/d15/d17/f18 1696173 0 2024-07-16T09:55:29.004 INFO:tasks.workunit.client.1.smithi176.stdout:0/141: fdatasync f4 0 2024-07-16T09:55:29.006 INFO:tasks.workunit.client.1.smithi176.stdout:9/58: creat d5/de/f12 x:0 0 0 2024-07-16T09:55:29.007 INFO:tasks.workunit.client.1.smithi176.stdout:7/126: mknod d4/dd/d26/c32 0 2024-07-16T09:55:29.008 INFO:tasks.workunit.client.1.smithi176.stdout:0/142: mknod d5/d1c/d24/c28 0 2024-07-16T09:55:29.009 INFO:tasks.workunit.client.1.smithi176.stdout:9/59: mknod d5/c13 0 2024-07-16T09:55:29.009 INFO:tasks.workunit.client.1.smithi176.stdout:7/127: read - d4/dd/f15 zero size 2024-07-16T09:55:29.011 INFO:tasks.workunit.client.1.smithi176.stdout:4/127: getdents da/d14 0 2024-07-16T09:55:29.013 INFO:tasks.workunit.client.1.smithi176.stdout:2/96: dwrite dd/d16/f1d [0,4194304] 0 2024-07-16T09:55:29.013 INFO:tasks.workunit.client.1.smithi176.stdout:4/128: fdatasync da/dc/d10/d15/f1b 0 2024-07-16T09:55:29.014 INFO:tasks.workunit.client.1.smithi176.stdout:9/60: symlink d5/l14 0 2024-07-16T09:55:29.014 INFO:tasks.workunit.client.1.smithi176.stdout:4/129: read f0 [72506,33488] 0 2024-07-16T09:55:29.018 INFO:tasks.workunit.client.1.smithi176.stdout:2/97: rmdir dd/d16 39 2024-07-16T09:55:29.019 INFO:tasks.workunit.client.1.smithi176.stdout:2/98: fdatasync f3 0 2024-07-16T09:55:29.019 INFO:tasks.workunit.client.1.smithi176.stdout:0/143: getdents d5 0 2024-07-16T09:55:29.020 INFO:tasks.workunit.client.1.smithi176.stdout:9/61: creat d5/de/f15 x:0 0 0 2024-07-16T09:55:29.020 INFO:tasks.workunit.client.1.smithi176.stdout:4/130: rmdir da/d14 39 2024-07-16T09:55:29.021 INFO:tasks.workunit.client.1.smithi176.stdout:2/99: read dd/de/f10 [2988062,24362] 0 2024-07-16T09:55:29.021 INFO:tasks.workunit.client.1.smithi176.stdout:0/144: symlink d5/l29 0 2024-07-16T09:55:29.022 INFO:tasks.workunit.client.1.smithi176.stdout:0/145: stat d5/fb 0 2024-07-16T09:55:29.022 INFO:tasks.workunit.client.1.smithi176.stdout:9/62: creat d5/f16 x:0 0 0 2024-07-16T09:55:29.023 INFO:tasks.workunit.client.1.smithi176.stdout:9/63: readlink d5/la 0 2024-07-16T09:55:29.023 INFO:tasks.workunit.client.1.smithi176.stdout:0/146: chown d5/c11 16101788 1 2024-07-16T09:55:29.024 INFO:tasks.workunit.client.1.smithi176.stdout:4/131: creat da/d14/f26 x:0 0 0 2024-07-16T09:55:29.025 INFO:tasks.workunit.client.1.smithi176.stdout:0/147: creat d5/f2a x:0 0 0 2024-07-16T09:55:29.025 INFO:tasks.workunit.client.1.smithi176.stdout:0/148: dread - d5/f2a zero size 2024-07-16T09:55:29.025 INFO:tasks.workunit.client.1.smithi176.stdout:9/64: creat d5/de/f17 x:0 0 0 2024-07-16T09:55:29.026 INFO:tasks.workunit.client.1.smithi176.stdout:2/100: rename dd/c15 to dd/d16/c20 0 2024-07-16T09:55:29.027 INFO:tasks.workunit.client.1.smithi176.stdout:9/65: write d5/de/f17 [553180,13908] 0 2024-07-16T09:55:29.027 INFO:tasks.workunit.client.1.smithi176.stdout:4/132: creat da/dc/d10/f27 x:0 0 0 2024-07-16T09:55:29.031 INFO:tasks.workunit.client.1.smithi176.stdout:4/133: chown l4 2204 1 2024-07-16T09:55:29.033 INFO:tasks.workunit.client.1.smithi176.stdout:9/66: dread d5/de/f17 [0,4194304] 0 2024-07-16T09:55:29.034 INFO:tasks.workunit.client.1.smithi176.stdout:4/134: mkdir da/dc/d10/d28 0 2024-07-16T09:55:29.034 INFO:tasks.workunit.client.1.smithi176.stdout:0/149: dread d5/f10 [0,4194304] 0 2024-07-16T09:55:29.035 INFO:tasks.workunit.client.0.smithi156.stdout:2/123: dwrite d0/d5/f1e [0,4194304] 0 2024-07-16T09:55:29.036 INFO:tasks.workunit.client.1.smithi176.stdout:9/67: write d5/fb [734584,19725] 0 2024-07-16T09:55:29.037 INFO:tasks.workunit.client.1.smithi176.stdout:2/101: dwrite dd/d16/f1d [0,4194304] 0 2024-07-16T09:55:29.038 INFO:tasks.workunit.client.1.smithi176.stdout:9/68: write d5/fb [4414761,90165] 0 2024-07-16T09:55:29.038 INFO:tasks.workunit.client.1.smithi176.stdout:0/150: symlink d5/l2b 0 2024-07-16T09:55:29.039 INFO:tasks.workunit.client.1.smithi176.stdout:2/102: read dd/d16/f1d [1114644,115321] 0 2024-07-16T09:55:29.040 INFO:tasks.workunit.client.1.smithi176.stdout:4/135: write da/dc/fe [334251,65501] 0 2024-07-16T09:55:29.044 INFO:tasks.workunit.client.1.smithi176.stdout:9/69: rmdir d5/de 39 2024-07-16T09:55:29.045 INFO:tasks.workunit.client.1.smithi176.stdout:2/103: dread f9 [0,4194304] 0 2024-07-16T09:55:29.045 INFO:tasks.workunit.client.1.smithi176.stdout:0/151: mkdir d5/d2c 0 2024-07-16T09:55:29.045 INFO:tasks.workunit.client.1.smithi176.stdout:4/136: chown da/c20 569 1 2024-07-16T09:55:29.053 INFO:tasks.workunit.client.1.smithi176.stdout:2/104: write f9 [184199,87338] 0 2024-07-16T09:55:29.053 INFO:tasks.workunit.client.1.smithi176.stdout:9/70: symlink d5/de/l18 0 2024-07-16T09:55:29.056 INFO:tasks.workunit.client.1.smithi176.stdout:9/71: write d5/fd [1063455,18629] 0 2024-07-16T09:55:29.056 INFO:tasks.workunit.client.1.smithi176.stdout:4/137: truncate da/f1a 2683547 0 2024-07-16T09:55:29.056 INFO:tasks.workunit.client.1.smithi176.stdout:9/72: write d5/de/f12 [1024982,104401] 0 2024-07-16T09:55:29.057 INFO:tasks.workunit.client.1.smithi176.stdout:2/105: mkdir dd/d21 0 2024-07-16T09:55:29.057 INFO:tasks.workunit.client.1.smithi176.stdout:0/152: dwrite d5/fb [0,4194304] 0 2024-07-16T09:55:29.065 INFO:tasks.workunit.client.1.smithi176.stdout:3/7: sync 2024-07-16T09:55:29.070 INFO:tasks.workunit.client.1.smithi176.stdout:3/8: creat f1 x:0 0 0 2024-07-16T09:55:29.072 INFO:tasks.workunit.client.1.smithi176.stdout:2/106: mknod dd/de/c22 0 2024-07-16T09:55:29.074 INFO:tasks.workunit.client.1.smithi176.stdout:3/9: mknod c2 0 2024-07-16T09:55:29.074 INFO:tasks.workunit.client.1.smithi176.stdout:0/153: creat d5/d2c/f2d x:0 0 0 2024-07-16T09:55:29.075 INFO:tasks.workunit.client.1.smithi176.stdout:3/10: chown l0 17409708 1 2024-07-16T09:55:29.075 INFO:tasks.workunit.client.1.smithi176.stdout:2/107: stat dd/d16/c20 0 2024-07-16T09:55:29.078 INFO:tasks.workunit.client.1.smithi176.stdout:3/11: mkdir d3 0 2024-07-16T09:55:29.079 INFO:tasks.workunit.client.1.smithi176.stdout:4/138: dread da/fb [0,4194304] 0 2024-07-16T09:55:29.080 INFO:tasks.workunit.client.1.smithi176.stdout:2/108: write dd/de/f10 [4256410,14510] 0 2024-07-16T09:55:29.080 INFO:tasks.workunit.client.1.smithi176.stdout:9/73: dwrite d5/de/f12 [0,4194304] 0 2024-07-16T09:55:29.081 INFO:tasks.workunit.client.1.smithi176.stdout:0/154: truncate d5/f8 956413 0 2024-07-16T09:55:29.083 INFO:tasks.workunit.client.1.smithi176.stdout:9/74: rmdir d5 39 2024-07-16T09:55:29.084 INFO:tasks.workunit.client.1.smithi176.stdout:3/12: symlink d3/l4 0 2024-07-16T09:55:29.085 INFO:tasks.workunit.client.1.smithi176.stdout:2/109: unlink dd/de/c12 0 2024-07-16T09:55:29.089 INFO:tasks.workunit.client.1.smithi176.stdout:0/155: dwrite d5/f14 [0,4194304] 0 2024-07-16T09:55:29.092 INFO:tasks.workunit.client.1.smithi176.stdout:9/75: write d5/f16 [78957,119037] 0 2024-07-16T09:55:29.092 INFO:tasks.workunit.client.1.smithi176.stdout:3/13: mknod d3/c5 0 2024-07-16T09:55:29.093 INFO:tasks.workunit.client.1.smithi176.stdout:9/76: chown d5/l14 2300711 1 2024-07-16T09:55:29.095 INFO:tasks.workunit.client.1.smithi176.stdout:9/77: truncate d5/f16 1113535 0 2024-07-16T09:55:29.095 INFO:tasks.workunit.client.1.smithi176.stdout:9/78: write d5/f16 [1670570,115250] 0 2024-07-16T09:55:29.096 INFO:tasks.workunit.client.1.smithi176.stdout:2/110: dread f3 [4194304,4194304] 0 2024-07-16T09:55:29.099 INFO:tasks.workunit.client.1.smithi176.stdout:0/156: mknod d5/d1c/d24/d26/c2e 0 2024-07-16T09:55:29.099 INFO:tasks.workunit.client.1.smithi176.stdout:4/139: write da/dc/d10/d15/f1b [2773023,31492] 0 2024-07-16T09:55:29.100 INFO:tasks.workunit.client.1.smithi176.stdout:3/14: unlink f1 0 2024-07-16T09:55:29.101 INFO:tasks.workunit.client.1.smithi176.stdout:3/15: chown d3 6 1 2024-07-16T09:55:29.102 INFO:tasks.workunit.client.1.smithi176.stdout:2/111: unlink dd/c1f 0 2024-07-16T09:55:29.104 INFO:tasks.workunit.client.1.smithi176.stdout:0/157: rmdir d5/d1c/d24/d26 39 2024-07-16T09:55:29.105 INFO:tasks.workunit.client.1.smithi176.stdout:3/16: creat d3/f6 x:0 0 0 2024-07-16T09:55:29.105 INFO:tasks.workunit.client.1.smithi176.stdout:0/158: chown d5 5984317 1 2024-07-16T09:55:29.107 INFO:tasks.workunit.client.1.smithi176.stdout:9/79: dwrite d5/de/f12 [0,4194304] 0 2024-07-16T09:55:29.110 INFO:tasks.workunit.client.1.smithi176.stdout:9/80: chown d5 2438 1 2024-07-16T09:55:29.110 INFO:tasks.workunit.client.1.smithi176.stdout:0/159: write d5/f1b [463351,75542] 0 2024-07-16T09:55:29.112 INFO:tasks.workunit.client.1.smithi176.stdout:3/17: rmdir d3 39 2024-07-16T09:55:29.114 INFO:tasks.workunit.client.1.smithi176.stdout:0/160: creat d5/f2f x:0 0 0 2024-07-16T09:55:29.116 INFO:tasks.workunit.client.1.smithi176.stdout:4/140: dwrite da/f19 [8388608,4194304] 0 2024-07-16T09:55:29.118 INFO:tasks.workunit.client.1.smithi176.stdout:2/112: dwrite f9 [0,4194304] 0 2024-07-16T09:55:29.128 INFO:tasks.workunit.client.1.smithi176.stdout:3/18: dwrite d3/f6 [0,4194304] 0 2024-07-16T09:55:29.150 INFO:tasks.workunit.client.0.smithi156.stdout:2/124: mkdir d0/d5/d29/d2d 0 2024-07-16T09:55:29.151 INFO:tasks.workunit.client.0.smithi156.stdout:2/125: write d0/fe [2049062,103897] 0 2024-07-16T09:55:29.152 INFO:tasks.workunit.client.0.smithi156.stdout:2/126: mknod d0/d5/d29/c2e 0 2024-07-16T09:55:29.155 INFO:tasks.workunit.client.0.smithi156.stdout:2/127: dread d0/fe [0,4194304] 0 2024-07-16T09:55:29.158 INFO:tasks.workunit.client.0.smithi156.stdout:2/128: unlink d0/d13/d17/d1c/c22 0 2024-07-16T09:55:29.158 INFO:tasks.workunit.client.0.smithi156.stdout:2/129: write d0/d5/f2a [289363,116235] 0 2024-07-16T09:55:29.161 INFO:tasks.workunit.client.0.smithi156.stdout:2/130: unlink d0/d5/la 0 2024-07-16T09:55:29.167 INFO:tasks.workunit.client.0.smithi156.stdout:2/131: dwrite d0/f16 [0,4194304] 0 2024-07-16T09:55:29.169 INFO:tasks.workunit.client.0.smithi156.stdout:2/132: creat d0/d5/f2f x:0 0 0 2024-07-16T09:55:29.170 INFO:tasks.workunit.client.1.smithi176.stdout:0/161: stat d5/d1c/d24/d26 0 2024-07-16T09:55:29.171 INFO:tasks.workunit.client.0.smithi156.stdout:2/133: unlink d0/f28 0 2024-07-16T09:55:29.172 INFO:tasks.workunit.client.1.smithi176.stdout:3/19: symlink d3/l7 0 2024-07-16T09:55:29.172 INFO:tasks.workunit.client.0.smithi156.stdout:2/134: dread - d0/d5/f2f zero size 2024-07-16T09:55:29.172 INFO:tasks.workunit.client.1.smithi176.stdout:2/113: symlink dd/d21/l23 0 2024-07-16T09:55:29.173 INFO:tasks.workunit.client.0.smithi156.stdout:2/135: fsync d0/f2c 0 2024-07-16T09:55:29.173 INFO:tasks.workunit.client.1.smithi176.stdout:3/20: chown d3/l7 55885 1 2024-07-16T09:55:29.174 INFO:tasks.workunit.client.1.smithi176.stdout:2/114: rename dd/de to dd/de/d24 22 2024-07-16T09:55:29.176 INFO:tasks.workunit.client.1.smithi176.stdout:2/115: write dd/d16/f1c [648210,13076] 0 2024-07-16T09:55:29.176 INFO:tasks.workunit.client.1.smithi176.stdout:4/141: getdents da/dc/d10 0 2024-07-16T09:55:29.180 INFO:tasks.workunit.client.0.smithi156.stdout:2/136: dwrite d0/f16 [0,4194304] 0 2024-07-16T09:55:29.180 INFO:tasks.workunit.client.1.smithi176.stdout:2/116: unlink fa 0 2024-07-16T09:55:29.180 INFO:tasks.workunit.client.1.smithi176.stdout:0/162: getdents d5/d1c/d24/d26 0 2024-07-16T09:55:29.182 INFO:tasks.workunit.client.0.smithi156.stdout:2/137: write d0/d13/f26 [801472,129473] 0 2024-07-16T09:55:29.182 INFO:tasks.workunit.client.1.smithi176.stdout:4/142: creat da/d14/f29 x:0 0 0 2024-07-16T09:55:29.183 INFO:tasks.workunit.client.1.smithi176.stdout:4/143: readlink da/dc/d10/d15/d17/l21 0 2024-07-16T09:55:29.183 INFO:tasks.workunit.client.1.smithi176.stdout:2/117: symlink dd/d21/l25 0 2024-07-16T09:55:29.184 INFO:tasks.workunit.client.1.smithi176.stdout:2/118: fsync f3 0 2024-07-16T09:55:29.184 INFO:tasks.workunit.client.1.smithi176.stdout:3/21: getdents d3 0 2024-07-16T09:55:29.185 INFO:tasks.workunit.client.1.smithi176.stdout:4/144: fdatasync da/dc/d10/d15/f1b 0 2024-07-16T09:55:29.186 INFO:tasks.workunit.client.1.smithi176.stdout:3/22: fdatasync d3/f6 0 2024-07-16T09:55:29.187 INFO:tasks.workunit.client.1.smithi176.stdout:3/23: fsync d3/f6 0 2024-07-16T09:55:29.187 INFO:tasks.workunit.client.1.smithi176.stdout:4/145: readlink da/dc/d10/d15/d17/l21 0 2024-07-16T09:55:29.188 INFO:tasks.workunit.client.1.smithi176.stdout:3/24: fdatasync d3/f6 0 2024-07-16T09:55:29.188 INFO:tasks.workunit.client.1.smithi176.stdout:4/146: chown da/dc/d10/d28 1 1 2024-07-16T09:55:29.190 INFO:tasks.workunit.client.1.smithi176.stdout:4/147: chown da/d14/l1d 84 1 2024-07-16T09:55:29.190 INFO:tasks.workunit.client.1.smithi176.stdout:2/119: getdents dd 0 2024-07-16T09:55:29.192 INFO:tasks.workunit.client.1.smithi176.stdout:0/163: truncate d5/f12 2699974 0 2024-07-16T09:55:29.193 INFO:tasks.workunit.client.1.smithi176.stdout:2/120: symlink dd/d21/l26 0 2024-07-16T09:55:29.193 INFO:tasks.workunit.client.1.smithi176.stdout:3/25: dread d3/f6 [0,4194304] 0 2024-07-16T09:55:29.194 INFO:tasks.workunit.client.1.smithi176.stdout:4/148: creat da/dc/d10/d15/d17/f2a x:0 0 0 2024-07-16T09:55:29.195 INFO:tasks.workunit.client.1.smithi176.stdout:4/149: chown da/dc/d10 7155374 1 2024-07-16T09:55:29.196 INFO:tasks.workunit.client.1.smithi176.stdout:2/121: creat dd/d16/f27 x:0 0 0 2024-07-16T09:55:29.196 INFO:tasks.workunit.client.1.smithi176.stdout:4/150: write da/f19 [6548852,70843] 0 2024-07-16T09:55:29.198 INFO:tasks.workunit.client.1.smithi176.stdout:2/122: read - dd/d16/f27 zero size 2024-07-16T09:55:29.198 INFO:tasks.workunit.client.1.smithi176.stdout:0/164: dread d5/f7 [0,4194304] 0 2024-07-16T09:55:29.200 INFO:tasks.workunit.client.1.smithi176.stdout:2/123: symlink dd/de/l28 0 2024-07-16T09:55:29.201 INFO:tasks.workunit.client.1.smithi176.stdout:3/26: dwrite d3/f6 [4194304,4194304] 0 2024-07-16T09:55:29.203 INFO:tasks.workunit.client.1.smithi176.stdout:2/124: symlink dd/d16/l29 0 2024-07-16T09:55:29.205 INFO:tasks.workunit.client.1.smithi176.stdout:3/27: creat d3/f8 x:0 0 0 2024-07-16T09:55:29.211 INFO:tasks.workunit.client.1.smithi176.stdout:2/125: dread f3 [0,4194304] 0 2024-07-16T09:55:29.214 INFO:tasks.workunit.client.1.smithi176.stdout:3/28: dwrite d3/f6 [8388608,4194304] 0 2024-07-16T09:55:29.217 INFO:tasks.workunit.client.1.smithi176.stdout:3/29: write d3/f8 [873157,121576] 0 2024-07-16T09:55:29.218 INFO:tasks.workunit.client.1.smithi176.stdout:3/30: symlink d3/l9 0 2024-07-16T09:55:29.220 INFO:tasks.workunit.client.1.smithi176.stdout:2/126: dwrite f9 [0,4194304] 0 2024-07-16T09:55:29.222 INFO:tasks.workunit.client.1.smithi176.stdout:3/31: unlink d3/f6 0 2024-07-16T09:55:29.229 INFO:tasks.workunit.client.0.smithi156.stdout:2/138: truncate d0/f16 1227026 0 2024-07-16T09:55:29.231 INFO:tasks.workunit.client.0.smithi156.stdout:2/139: creat d0/d13/d17/f30 x:0 0 0 2024-07-16T09:55:29.235 INFO:tasks.workunit.client.0.smithi156.stdout:2/140: dwrite d0/d13/d17/f21 [0,4194304] 0 2024-07-16T09:55:29.239 INFO:tasks.workunit.client.0.smithi156.stdout:2/141: dread d0/f7 [0,4194304] 0 2024-07-16T09:55:29.240 INFO:tasks.workunit.client.0.smithi156.stdout:2/142: dread - d0/d5/f2f zero size 2024-07-16T09:55:29.240 INFO:tasks.workunit.client.0.smithi156.stdout:2/143: fsync d0/d5/f2f 0 2024-07-16T09:55:29.309 INFO:tasks.workunit.client.1.smithi176.stdout:1/31: sync 2024-07-16T09:55:29.312 INFO:tasks.workunit.client.1.smithi176.stdout:1/32: write f1 [1369523,118300] 0 2024-07-16T09:55:29.317 INFO:tasks.workunit.client.1.smithi176.stdout:1/33: dwrite d4/d8/fa [0,4194304] 0 2024-07-16T09:55:29.318 INFO:tasks.workunit.client.1.smithi176.stdout:1/34: fsync f1 0 2024-07-16T09:55:29.320 INFO:tasks.workunit.client.1.smithi176.stdout:1/35: symlink d4/d8/lc 0 2024-07-16T09:55:29.327 INFO:tasks.workunit.client.1.smithi176.stdout:1/36: dwrite f1 [0,4194304] 0 2024-07-16T09:55:29.334 INFO:tasks.workunit.client.1.smithi176.stdout:1/37: dwrite f1 [0,4194304] 0 2024-07-16T09:55:29.336 INFO:tasks.workunit.client.1.smithi176.stdout:1/38: symlink d4/d8/ld 0 2024-07-16T09:55:29.341 INFO:tasks.workunit.client.1.smithi176.stdout:1/39: link d4/c7 d4/db/ce 0 2024-07-16T09:55:29.343 INFO:tasks.workunit.client.1.smithi176.stdout:1/40: creat d4/ff x:0 0 0 2024-07-16T09:55:29.345 INFO:tasks.workunit.client.1.smithi176.stdout:1/41: creat d4/f10 x:0 0 0 2024-07-16T09:55:29.351 INFO:tasks.workunit.client.1.smithi176.stdout:1/42: getdents d4 0 2024-07-16T09:55:29.352 INFO:tasks.workunit.client.1.smithi176.stdout:1/43: chown c2 1 1 2024-07-16T09:55:29.352 INFO:tasks.workunit.client.1.smithi176.stdout:1/44: stat d4/f10 0 2024-07-16T09:55:29.353 INFO:tasks.workunit.client.1.smithi176.stdout:1/45: mknod d4/db/c11 0 2024-07-16T09:55:29.355 INFO:tasks.workunit.client.1.smithi176.stdout:1/46: getdents d4/d8 0 2024-07-16T09:55:29.358 INFO:tasks.workunit.client.1.smithi176.stdout:1/47: dread f1 [0,4194304] 0 2024-07-16T09:55:29.359 INFO:tasks.workunit.client.1.smithi176.stdout:1/48: write d4/d8/fa [3154036,7037] 0 2024-07-16T09:55:29.360 INFO:tasks.workunit.client.1.smithi176.stdout:1/49: write d4/f10 [738541,46949] 0 2024-07-16T09:55:29.361 INFO:tasks.workunit.client.1.smithi176.stdout:1/50: mkdir d4/d8/d12 0 2024-07-16T09:55:29.362 INFO:tasks.workunit.client.1.smithi176.stdout:1/51: chown d4/ff 0 1 2024-07-16T09:55:29.363 INFO:tasks.workunit.client.1.smithi176.stdout:1/52: symlink d4/l13 0 2024-07-16T09:55:29.367 INFO:tasks.workunit.client.1.smithi176.stdout:1/53: dwrite d4/f10 [0,4194304] 0 2024-07-16T09:55:29.454 INFO:tasks.workunit.client.1.smithi176.stdout:1/54: symlink d4/l14 0 2024-07-16T09:55:29.459 INFO:tasks.workunit.client.1.smithi176.stdout:1/55: truncate d4/d8/fa 3820991 0 2024-07-16T09:55:29.459 INFO:tasks.workunit.client.1.smithi176.stdout:1/56: readlink d4/l13 0 2024-07-16T09:55:29.461 INFO:tasks.workunit.client.1.smithi176.stdout:1/57: link d4/l14 d4/db/l15 0 2024-07-16T09:55:29.527 INFO:tasks.workunit.client.1.smithi176.stdout:1/58: creat d4/db/f16 x:0 0 0 2024-07-16T09:55:29.529 INFO:tasks.workunit.client.1.smithi176.stdout:1/59: creat d4/f17 x:0 0 0 2024-07-16T09:55:29.531 INFO:tasks.workunit.client.1.smithi176.stdout:1/60: unlink d4/d8/c9 0 2024-07-16T09:55:29.537 INFO:tasks.workunit.client.1.smithi176.stdout:1/61: dread d4/d8/fa [0,4194304] 0 2024-07-16T09:55:29.537 INFO:tasks.workunit.client.1.smithi176.stdout:1/62: write d4/db/f16 [93244,82767] 0 2024-07-16T09:55:29.540 INFO:tasks.workunit.client.1.smithi176.stdout:1/63: mknod d4/d8/d12/c18 0 2024-07-16T09:55:29.542 INFO:tasks.workunit.client.1.smithi176.stdout:1/64: rename c0 to d4/d8/d12/c19 0 2024-07-16T09:55:29.542 INFO:tasks.workunit.client.1.smithi176.stdout:1/65: read f1 [248944,83282] 0 2024-07-16T09:55:29.543 INFO:tasks.workunit.client.1.smithi176.stdout:1/66: chown d4/d8 1921 1 2024-07-16T09:55:29.543 INFO:tasks.workunit.client.1.smithi176.stdout:1/67: write d4/f17 [827623,5564] 0 2024-07-16T09:55:29.546 INFO:tasks.workunit.client.1.smithi176.stdout:1/68: dread f1 [0,4194304] 0 2024-07-16T09:55:29.547 INFO:tasks.workunit.client.1.smithi176.stdout:1/69: chown d4/d8/lc 139459746 1 2024-07-16T09:55:29.548 INFO:tasks.workunit.client.1.smithi176.stdout:1/70: chown d4/ff 7562 1 2024-07-16T09:55:29.561 INFO:tasks.workunit.client.0.smithi156.stdout:7/3: sync 2024-07-16T09:55:29.564 INFO:tasks.workunit.client.0.smithi156.stdout:7/4: unlink f0 0 2024-07-16T09:55:29.566 INFO:tasks.workunit.client.0.smithi156.stdout:7/5: creat f1 x:0 0 0 2024-07-16T09:55:29.566 INFO:tasks.workunit.client.1.smithi176.stdout:6/107: sync 2024-07-16T09:55:29.568 INFO:tasks.workunit.client.0.smithi156.stdout:7/6: rename f1 to f2 0 2024-07-16T09:55:29.569 INFO:tasks.workunit.client.1.smithi176.stdout:6/108: fdatasync d1/d5/f6 0 2024-07-16T09:55:29.570 INFO:tasks.workunit.client.0.smithi156.stdout:7/7: creat f3 x:0 0 0 2024-07-16T09:55:29.571 INFO:tasks.workunit.client.1.smithi176.stdout:6/109: mknod d1/d5/d7/c23 0 2024-07-16T09:55:29.573 INFO:tasks.workunit.client.1.smithi176.stdout:6/110: symlink d1/dc/l24 0 2024-07-16T09:55:29.574 INFO:tasks.workunit.client.1.smithi176.stdout:6/111: chown d1/f15 5999466 1 2024-07-16T09:55:29.576 INFO:tasks.workunit.client.0.smithi156.stdout:7/8: unlink f2 0 2024-07-16T09:55:29.577 INFO:tasks.workunit.client.1.smithi176.stdout:6/112: dread d1/d5/d7/fe [0,4194304] 0 2024-07-16T09:55:29.578 INFO:tasks.workunit.client.1.smithi176.stdout:6/113: mknod d1/d5/c25 0 2024-07-16T09:55:29.580 INFO:tasks.workunit.client.1.smithi176.stdout:6/114: mkdir d1/dc/d26 0 2024-07-16T09:55:29.581 INFO:tasks.workunit.client.0.smithi156.stdout:7/9: dwrite f3 [0,4194304] 0 2024-07-16T09:55:29.582 INFO:tasks.workunit.client.0.smithi156.stdout:7/10: stat f3 0 2024-07-16T09:55:29.582 INFO:tasks.workunit.client.0.smithi156.stdout:7/11: write f3 [3665192,24776] 0 2024-07-16T09:55:29.583 INFO:tasks.workunit.client.0.smithi156.stdout:7/12: chown f3 1661006 1 2024-07-16T09:55:29.584 INFO:tasks.workunit.client.1.smithi176.stdout:6/115: rename d1/dc/f12 to d1/d5/f27 0 2024-07-16T09:55:29.584 INFO:tasks.workunit.client.0.smithi156.stdout:7/13: chown f3 106583412 1 2024-07-16T09:55:29.586 INFO:tasks.workunit.client.1.smithi176.stdout:6/116: symlink d1/d5/l28 0 2024-07-16T09:55:29.588 INFO:tasks.workunit.client.0.smithi156.stdout:7/14: dread f3 [0,4194304] 0 2024-07-16T09:55:29.588 INFO:tasks.workunit.client.1.smithi176.stdout:6/117: mkdir d1/d5/d7/d1e/d29 0 2024-07-16T09:55:29.592 INFO:tasks.workunit.client.0.smithi156.stdout:7/15: truncate f3 2253774 0 2024-07-16T09:55:29.592 INFO:tasks.workunit.client.0.smithi156.stdout:7/16: stat f3 0 2024-07-16T09:55:29.592 INFO:tasks.workunit.client.0.smithi156.stdout:7/17: chown f3 44 1 2024-07-16T09:55:29.592 INFO:tasks.workunit.client.1.smithi176.stdout:6/118: dread d1/d5/d7/f19 [0,4194304] 0 2024-07-16T09:55:29.593 INFO:tasks.workunit.client.1.smithi176.stdout:6/119: stat d1/l4 0 2024-07-16T09:55:29.595 INFO:tasks.workunit.client.0.smithi156.stdout:7/18: write f3 [2345958,29180] 0 2024-07-16T09:55:29.596 INFO:tasks.workunit.client.0.smithi156.stdout:7/19: chown f3 44356290 1 2024-07-16T09:55:29.597 INFO:tasks.workunit.client.1.smithi176.stdout:6/120: link d1/d5/d7/f19 d1/d5/f2a 0 2024-07-16T09:55:29.598 INFO:tasks.workunit.client.0.smithi156.stdout:7/20: rename f3 to f4 0 2024-07-16T09:55:29.601 INFO:tasks.workunit.client.1.smithi176.stdout:6/121: dread d1/d5/d7/f9 [0,4194304] 0 2024-07-16T09:55:29.602 INFO:tasks.workunit.client.0.smithi156.stdout:7/21: fdatasync f4 0 2024-07-16T09:55:29.605 INFO:tasks.workunit.client.0.smithi156.stdout:7/22: unlink f4 0 2024-07-16T09:55:29.605 INFO:tasks.workunit.client.0.smithi156.stdout:7/23: fdatasync - no filename 2024-07-16T09:55:29.608 INFO:tasks.workunit.client.0.smithi156.stdout:7/24: getdents . 0 2024-07-16T09:55:29.608 INFO:tasks.workunit.client.0.smithi156.stdout:7/25: fdatasync - no filename 2024-07-16T09:55:29.609 INFO:tasks.workunit.client.0.smithi156.stdout:7/26: truncate - no filename 2024-07-16T09:55:29.609 INFO:tasks.workunit.client.1.smithi176.stdout:6/122: truncate d1/d5/d7/fe 2321112 0 2024-07-16T09:55:29.609 INFO:tasks.workunit.client.0.smithi156.stdout:7/27: symlink l5 0 2024-07-16T09:55:29.610 INFO:tasks.workunit.client.1.smithi176.stdout:6/123: symlink d1/dc/l2b 0 2024-07-16T09:55:29.611 INFO:tasks.workunit.client.0.smithi156.stdout:7/28: mknod c6 0 2024-07-16T09:55:29.612 INFO:tasks.workunit.client.0.smithi156.stdout:7/29: symlink l7 0 2024-07-16T09:55:29.612 INFO:tasks.workunit.client.1.smithi176.stdout:6/124: creat d1/d5/f2c x:0 0 0 2024-07-16T09:55:29.613 INFO:tasks.workunit.client.0.smithi156.stdout:7/30: symlink l8 0 2024-07-16T09:55:29.613 INFO:tasks.workunit.client.1.smithi176.stdout:6/125: rename d1/d5/d7 to d1/d5/d7/d1e/d2d 22 2024-07-16T09:55:29.614 INFO:tasks.workunit.client.0.smithi156.stdout:7/31: rename l7 to l9 0 2024-07-16T09:55:29.614 INFO:tasks.workunit.client.0.smithi156.stdout:7/32: chown c6 308 1 2024-07-16T09:55:29.615 INFO:tasks.workunit.client.0.smithi156.stdout:7/33: mknod ca 0 2024-07-16T09:55:29.616 INFO:tasks.workunit.client.0.smithi156.stdout:7/34: creat fb x:0 0 0 2024-07-16T09:55:29.618 INFO:tasks.workunit.client.1.smithi176.stdout:6/126: unlink d1/dc/d1c/f1f 0 2024-07-16T09:55:29.621 INFO:tasks.workunit.client.1.smithi176.stdout:6/127: dread d1/dc/fd [0,4194304] 0 2024-07-16T09:55:29.626 INFO:tasks.workunit.client.0.smithi156.stdout:7/35: dwrite fb [0,4194304] 0 2024-07-16T09:55:29.626 INFO:tasks.workunit.client.0.smithi156.stdout:7/36: mkdir dc 0 2024-07-16T09:55:29.627 INFO:tasks.workunit.client.1.smithi176.stdout:6/128: dread d1/d5/d7/fe [0,4194304] 0 2024-07-16T09:55:29.627 INFO:tasks.workunit.client.0.smithi156.stdout:7/37: creat dc/fd x:0 0 0 2024-07-16T09:55:29.627 INFO:tasks.workunit.client.0.smithi156.stdout:7/38: chown ca 21452176 1 2024-07-16T09:55:29.631 INFO:tasks.workunit.client.1.smithi176.stdout:6/129: write d1/d5/f2a [2735925,53774] 0 2024-07-16T09:55:29.631 INFO:tasks.workunit.client.0.smithi156.stdout:8/16: sync 2024-07-16T09:55:29.631 INFO:tasks.workunit.client.0.smithi156.stdout:0/23: sync 2024-07-16T09:55:29.634 INFO:tasks.workunit.client.0.smithi156.stdout:8/17: creat d0/f6 x:0 0 0 2024-07-16T09:55:29.634 INFO:tasks.workunit.client.1.smithi176.stdout:6/130: dread d1/dc/f17 [0,4194304] 0 2024-07-16T09:55:29.635 INFO:tasks.workunit.client.0.smithi156.stdout:0/24: getdents d0 0 2024-07-16T09:55:29.635 INFO:tasks.workunit.client.0.smithi156.stdout:8/18: chown d0/f2 509564990 1 2024-07-16T09:55:29.635 INFO:tasks.workunit.client.0.smithi156.stdout:7/39: truncate fb 3185380 0 2024-07-16T09:55:29.635 INFO:tasks.workunit.client.0.smithi156.stdout:8/19: chown d0/c3 13701 1 2024-07-16T09:55:29.636 INFO:tasks.workunit.client.0.smithi156.stdout:0/25: rmdir d0 39 2024-07-16T09:55:29.636 INFO:tasks.workunit.client.0.smithi156.stdout:8/20: chown d0 0 1 2024-07-16T09:55:29.636 INFO:tasks.workunit.client.1.smithi176.stdout:8/22: sync 2024-07-16T09:55:29.637 INFO:tasks.workunit.client.1.smithi176.stdout:5/198: sync 2024-07-16T09:55:29.637 INFO:tasks.workunit.client.0.smithi156.stdout:7/40: mknod dc/ce 0 2024-07-16T09:55:29.638 INFO:tasks.workunit.client.1.smithi176.stdout:8/23: symlink l7 0 2024-07-16T09:55:29.639 INFO:tasks.workunit.client.0.smithi156.stdout:7/41: mkdir dc/df 0 2024-07-16T09:55:29.639 INFO:tasks.workunit.client.0.smithi156.stdout:0/26: creat d0/f4 x:0 0 0 2024-07-16T09:55:29.640 INFO:tasks.workunit.client.1.smithi176.stdout:8/24: read f5 [2823069,77251] 0 2024-07-16T09:55:29.640 INFO:tasks.workunit.client.0.smithi156.stdout:0/27: truncate d0/f4 273897 0 2024-07-16T09:55:29.641 INFO:tasks.workunit.client.0.smithi156.stdout:7/42: write fb [2686887,11596] 0 2024-07-16T09:55:29.641 INFO:tasks.workunit.client.0.smithi156.stdout:8/21: write d0/f2 [839618,71569] 0 2024-07-16T09:55:29.642 INFO:tasks.workunit.client.0.smithi156.stdout:0/28: write d0/f4 [242728,11647] 0 2024-07-16T09:55:29.643 INFO:tasks.workunit.client.0.smithi156.stdout:8/22: write d0/f1 [1919733,37909] 0 2024-07-16T09:55:29.643 INFO:tasks.workunit.client.0.smithi156.stdout:8/23: stat d0/f6 0 2024-07-16T09:55:29.644 INFO:tasks.workunit.client.0.smithi156.stdout:8/24: chown d0/f1 7455358 1 2024-07-16T09:55:29.646 INFO:tasks.workunit.client.1.smithi176.stdout:6/131: dwrite d1/d5/d7/f19 [0,4194304] 0 2024-07-16T09:55:29.646 INFO:tasks.workunit.client.1.smithi176.stdout:5/199: dwrite d4/d10/f33 [0,4194304] 0 2024-07-16T09:55:29.647 INFO:tasks.workunit.client.0.smithi156.stdout:7/43: unlink dc/fd 0 2024-07-16T09:55:29.648 INFO:tasks.workunit.client.1.smithi176.stdout:6/132: read d1/d5/d7/f19 [1970756,44116] 0 2024-07-16T09:55:29.648 INFO:tasks.workunit.client.0.smithi156.stdout:7/44: write fb [3226872,95443] 0 2024-07-16T09:55:29.651 INFO:tasks.workunit.client.1.smithi176.stdout:5/200: symlink d4/d14/d19/d1b/d36/l47 0 2024-07-16T09:55:29.651 INFO:tasks.workunit.client.1.smithi176.stdout:8/25: truncate f5 4831572 0 2024-07-16T09:55:29.652 INFO:tasks.workunit.client.1.smithi176.stdout:8/26: rmdir - no directory 2024-07-16T09:55:29.652 INFO:tasks.workunit.client.0.smithi156.stdout:7/45: mknod dc/c10 0 2024-07-16T09:55:29.652 INFO:tasks.workunit.client.0.smithi156.stdout:8/25: dwrite d0/f2 [0,4194304] 0 2024-07-16T09:55:29.652 INFO:tasks.workunit.client.0.smithi156.stdout:8/26: readlink - no filename 2024-07-16T09:55:29.656 INFO:tasks.workunit.client.1.smithi176.stdout:6/133: stat d1/d5/l1a 0 2024-07-16T09:55:29.657 INFO:tasks.workunit.client.1.smithi176.stdout:8/27: mkdir d8 0 2024-07-16T09:55:29.659 INFO:tasks.workunit.client.1.smithi176.stdout:5/201: dwrite d4/d14/f2c [0,4194304] 0 2024-07-16T09:55:29.659 INFO:tasks.workunit.client.0.smithi156.stdout:8/27: mknod d0/c7 0 2024-07-16T09:55:29.661 INFO:tasks.workunit.client.0.smithi156.stdout:8/28: write d0/f6 [976051,89517] 0 2024-07-16T09:55:29.664 INFO:tasks.workunit.client.1.smithi176.stdout:5/202: write d4/d10/f22 [2066066,79666] 0 2024-07-16T09:55:29.666 INFO:tasks.workunit.client.0.smithi156.stdout:7/46: link dc/ce dc/df/c11 0 2024-07-16T09:55:29.670 INFO:tasks.workunit.client.0.smithi156.stdout:7/47: dread fb [0,4194304] 0 2024-07-16T09:55:29.672 INFO:tasks.workunit.client.0.smithi156.stdout:8/29: fdatasync d0/f6 0 2024-07-16T09:55:29.679 INFO:tasks.workunit.client.1.smithi176.stdout:8/28: dwrite f5 [4194304,4194304] 0 2024-07-16T09:55:29.681 INFO:tasks.workunit.client.1.smithi176.stdout:6/134: dread d1/f18 [0,4194304] 0 2024-07-16T09:55:29.681 INFO:tasks.workunit.client.1.smithi176.stdout:8/29: symlink d8/l9 0 2024-07-16T09:55:29.691 INFO:tasks.workunit.client.1.smithi176.stdout:6/135: getdents d1/dc/d1c 0 2024-07-16T09:55:29.692 INFO:tasks.workunit.client.1.smithi176.stdout:8/30: truncate f5 3963964 0 2024-07-16T09:55:29.692 INFO:tasks.workunit.client.1.smithi176.stdout:6/136: creat d1/dc/f2e x:0 0 0 2024-07-16T09:55:29.694 INFO:tasks.workunit.client.1.smithi176.stdout:6/137: chown d1/dc/c20 89745 1 2024-07-16T09:55:29.695 INFO:tasks.workunit.client.1.smithi176.stdout:6/138: mkdir d1/d5/d7/d2f 0 2024-07-16T09:55:29.695 INFO:tasks.workunit.client.1.smithi176.stdout:8/31: dread f6 [0,4194304] 0 2024-07-16T09:55:29.698 INFO:tasks.workunit.client.1.smithi176.stdout:6/139: creat d1/d5/f30 x:0 0 0 2024-07-16T09:55:29.699 INFO:tasks.workunit.client.1.smithi176.stdout:6/140: truncate d1/d5/f30 547309 0 2024-07-16T09:55:29.700 INFO:tasks.workunit.client.1.smithi176.stdout:6/141: chown d1/c14 0 1 2024-07-16T09:55:29.701 INFO:tasks.workunit.client.1.smithi176.stdout:8/32: dwrite f6 [0,4194304] 0 2024-07-16T09:55:29.703 INFO:tasks.workunit.client.1.smithi176.stdout:6/142: creat d1/f31 x:0 0 0 2024-07-16T09:55:29.704 INFO:tasks.workunit.client.1.smithi176.stdout:8/33: creat d8/fa x:0 0 0 2024-07-16T09:55:29.705 INFO:tasks.workunit.client.1.smithi176.stdout:8/34: truncate d8/fa 715594 0 2024-07-16T09:55:29.711 INFO:tasks.workunit.client.1.smithi176.stdout:6/143: dwrite d1/dc/f17 [0,4194304] 0 2024-07-16T09:55:29.713 INFO:tasks.workunit.client.1.smithi176.stdout:8/35: dwrite f6 [0,4194304] 0 2024-07-16T09:55:29.714 INFO:tasks.workunit.client.1.smithi176.stdout:8/36: chown d8/l9 19000 1 2024-07-16T09:55:29.718 INFO:tasks.workunit.client.1.smithi176.stdout:8/37: dread f6 [0,4194304] 0 2024-07-16T09:55:29.732 INFO:tasks.workunit.client.1.smithi176.stdout:8/38: truncate f6 665628 0 2024-07-16T09:55:29.746 INFO:tasks.workunit.client.1.smithi176.stdout:8/39: truncate f5 1030732 0 2024-07-16T09:55:29.746 INFO:tasks.workunit.client.1.smithi176.stdout:8/40: rename d8 to d8/db 22 2024-07-16T09:55:29.747 INFO:tasks.workunit.client.1.smithi176.stdout:8/41: mknod d8/cc 0 2024-07-16T09:55:29.751 INFO:tasks.workunit.client.1.smithi176.stdout:8/42: dread f5 [0,4194304] 0 2024-07-16T09:55:29.753 INFO:tasks.workunit.client.1.smithi176.stdout:8/43: dread d8/fa [0,4194304] 0 2024-07-16T09:55:29.754 INFO:tasks.workunit.client.1.smithi176.stdout:8/44: write d8/fa [923505,30202] 0 2024-07-16T09:55:29.754 INFO:tasks.workunit.client.1.smithi176.stdout:8/45: readlink d8/l9 0 2024-07-16T09:55:29.756 INFO:tasks.workunit.client.1.smithi176.stdout:8/46: creat d8/fd x:0 0 0 2024-07-16T09:55:29.757 INFO:tasks.workunit.client.1.smithi176.stdout:8/47: mknod d8/ce 0 2024-07-16T09:55:29.762 INFO:tasks.workunit.client.1.smithi176.stdout:8/48: dwrite d8/fa [0,4194304] 0 2024-07-16T09:55:29.764 INFO:tasks.workunit.client.1.smithi176.stdout:8/49: creat d8/ff x:0 0 0 2024-07-16T09:55:29.766 INFO:tasks.workunit.client.1.smithi176.stdout:8/50: rename d8/ce to d8/c10 0 2024-07-16T09:55:29.767 INFO:tasks.workunit.client.1.smithi176.stdout:8/51: unlink l7 0 2024-07-16T09:55:29.768 INFO:tasks.workunit.client.1.smithi176.stdout:9/81: sync 2024-07-16T09:55:29.768 INFO:tasks.workunit.client.1.smithi176.stdout:7/128: sync 2024-07-16T09:55:29.768 INFO:tasks.workunit.client.1.smithi176.stdout:1/71: sync 2024-07-16T09:55:29.768 INFO:tasks.workunit.client.1.smithi176.stdout:0/165: sync 2024-07-16T09:55:29.769 INFO:tasks.workunit.client.1.smithi176.stdout:4/151: sync 2024-07-16T09:55:29.769 INFO:tasks.workunit.client.1.smithi176.stdout:3/32: sync 2024-07-16T09:55:29.769 INFO:tasks.workunit.client.1.smithi176.stdout:2/127: sync 2024-07-16T09:55:29.770 INFO:tasks.workunit.client.1.smithi176.stdout:4/152: readlink l4 0 2024-07-16T09:55:29.770 INFO:tasks.workunit.client.1.smithi176.stdout:9/82: chown d5/l14 1 1 2024-07-16T09:55:29.770 INFO:tasks.workunit.client.1.smithi176.stdout:4/153: dread - da/dc/d10/f27 zero size 2024-07-16T09:55:29.771 INFO:tasks.workunit.client.0.smithi156.stdout:3/21: sync 2024-07-16T09:55:29.771 INFO:tasks.workunit.client.1.smithi176.stdout:2/128: creat dd/f2a x:0 0 0 2024-07-16T09:55:29.772 INFO:tasks.workunit.client.1.smithi176.stdout:7/129: dread d4/dd/f23 [0,4194304] 0 2024-07-16T09:55:29.774 INFO:tasks.workunit.client.0.smithi156.stdout:3/22: getdents . 0 2024-07-16T09:55:29.774 INFO:tasks.workunit.client.0.smithi156.stdout:3/23: truncate f0 4358107 0 2024-07-16T09:55:29.775 INFO:tasks.workunit.client.0.smithi156.stdout:3/24: truncate f0 4515912 0 2024-07-16T09:55:29.775 INFO:tasks.workunit.client.0.smithi156.stdout:3/25: readlink l6 0 2024-07-16T09:55:29.775 INFO:tasks.workunit.client.0.smithi156.stdout:3/26: dread - f5 zero size 2024-07-16T09:55:29.778 INFO:tasks.workunit.client.1.smithi176.stdout:7/130: unlink d4/d1f/d22/l2d 0 2024-07-16T09:55:29.783 INFO:tasks.workunit.client.1.smithi176.stdout:7/131: write d4/dd/f23 [779861,107683] 0 2024-07-16T09:55:29.784 INFO:tasks.workunit.client.0.smithi156.stdout:3/27: dwrite f0 [0,4194304] 0 2024-07-16T09:55:29.784 INFO:tasks.workunit.client.0.smithi156.stdout:3/28: write f5 [567769,29066] 0 2024-07-16T09:55:29.785 INFO:tasks.workunit.client.1.smithi176.stdout:1/72: dwrite d4/f17 [0,4194304] 0 2024-07-16T09:55:29.787 INFO:tasks.workunit.client.1.smithi176.stdout:8/52: dwrite f6 [0,4194304] 0 2024-07-16T09:55:29.788 INFO:tasks.workunit.client.1.smithi176.stdout:3/33: dwrite d3/f8 [0,4194304] 0 2024-07-16T09:55:29.789 INFO:tasks.workunit.client.1.smithi176.stdout:2/129: dread f9 [0,4194304] 0 2024-07-16T09:55:29.789 INFO:tasks.workunit.client.1.smithi176.stdout:8/53: readlink d8/l9 0 2024-07-16T09:55:29.790 INFO:tasks.workunit.client.1.smithi176.stdout:9/83: truncate d5/fd 851267 0 2024-07-16T09:55:29.791 INFO:tasks.workunit.client.1.smithi176.stdout:4/154: write da/dc/d10/d15/d17/f18 [492303,70459] 0 2024-07-16T09:55:29.793 INFO:tasks.workunit.client.1.smithi176.stdout:9/84: dread - d5/de/f15 zero size 2024-07-16T09:55:29.793 INFO:tasks.workunit.client.1.smithi176.stdout:8/54: dread - d8/fd zero size 2024-07-16T09:55:29.795 INFO:tasks.workunit.client.1.smithi176.stdout:7/132: mkdir d4/d5/d21/d33 0 2024-07-16T09:55:29.796 INFO:tasks.workunit.client.0.smithi156.stdout:3/29: dwrite f0 [0,4194304] 0 2024-07-16T09:55:29.796 INFO:tasks.workunit.client.1.smithi176.stdout:1/73: write d4/d8/fa [2697984,3071] 0 2024-07-16T09:55:29.799 INFO:tasks.workunit.client.0.smithi156.stdout:3/30: mkdir d8 0 2024-07-16T09:55:29.799 INFO:tasks.workunit.client.1.smithi176.stdout:0/166: dwrite d5/f16 [0,4194304] 0 2024-07-16T09:55:29.800 INFO:tasks.workunit.client.1.smithi176.stdout:9/85: unlink c4 0 2024-07-16T09:55:29.802 INFO:tasks.workunit.client.1.smithi176.stdout:4/155: symlink da/dc/d10/d15/d17/l2b 0 2024-07-16T09:55:29.804 INFO:tasks.workunit.client.0.smithi156.stdout:3/31: dread f0 [0,4194304] 0 2024-07-16T09:55:29.805 INFO:tasks.workunit.client.0.smithi156.stdout:3/32: stat l7 0 2024-07-16T09:55:29.807 INFO:tasks.workunit.client.0.smithi156.stdout:3/33: getdents d8 0 2024-07-16T09:55:29.811 INFO:tasks.workunit.client.1.smithi176.stdout:4/156: dread - da/dc/d10/d15/d17/f2a zero size 2024-07-16T09:55:29.817 INFO:tasks.workunit.client.1.smithi176.stdout:3/34: dwrite d3/f8 [0,4194304] 0 2024-07-16T09:55:29.819 INFO:tasks.workunit.client.1.smithi176.stdout:7/133: link d4/d5/f31 d4/d5/d21/d33/f34 0 2024-07-16T09:55:29.823 INFO:tasks.workunit.client.1.smithi176.stdout:0/167: creat d5/d1c/d24/f30 x:0 0 0 2024-07-16T09:55:29.823 INFO:tasks.workunit.client.1.smithi176.stdout:4/157: rmdir da/dc/d10 39 2024-07-16T09:55:29.825 INFO:tasks.workunit.client.1.smithi176.stdout:2/130: dwrite f9 [0,4194304] 0 2024-07-16T09:55:29.828 INFO:tasks.workunit.client.1.smithi176.stdout:8/55: dwrite d8/ff [0,4194304] 0 2024-07-16T09:55:29.829 INFO:tasks.workunit.client.1.smithi176.stdout:1/74: dwrite d4/d8/fa [0,4194304] 0 2024-07-16T09:55:29.835 INFO:tasks.workunit.client.1.smithi176.stdout:9/86: dwrite d5/f10 [0,4194304] 0 2024-07-16T09:55:29.835 INFO:tasks.workunit.client.1.smithi176.stdout:0/168: write d5/f19 [907440,77238] 0 2024-07-16T09:55:29.835 INFO:tasks.workunit.client.1.smithi176.stdout:2/131: write f3 [1606575,74275] 0 2024-07-16T09:55:29.838 INFO:tasks.workunit.client.1.smithi176.stdout:8/56: write f6 [4784583,99011] 0 2024-07-16T09:55:29.840 INFO:tasks.workunit.client.1.smithi176.stdout:1/75: mknod d4/d8/c1a 0 2024-07-16T09:55:29.841 INFO:tasks.workunit.client.1.smithi176.stdout:8/57: chown c4 324158 1 2024-07-16T09:55:29.842 INFO:tasks.workunit.client.1.smithi176.stdout:7/134: dread d4/dd/f16 [0,4194304] 0 2024-07-16T09:55:29.842 INFO:tasks.workunit.client.1.smithi176.stdout:1/76: write d4/d8/fa [2254437,2527] 0 2024-07-16T09:55:29.843 INFO:tasks.workunit.client.1.smithi176.stdout:0/169: rename d5/l2b to d5/l31 0 2024-07-16T09:55:29.844 INFO:tasks.workunit.client.1.smithi176.stdout:4/158: symlink da/dc/d10/d28/l2c 0 2024-07-16T09:55:29.846 INFO:tasks.workunit.client.1.smithi176.stdout:8/58: rename d8/cc to d8/c11 0 2024-07-16T09:55:29.847 INFO:tasks.workunit.client.1.smithi176.stdout:1/77: readlink d4/l14 0 2024-07-16T09:55:29.847 INFO:tasks.workunit.client.1.smithi176.stdout:8/59: chown d8/l9 2959651 1 2024-07-16T09:55:29.848 INFO:tasks.workunit.client.1.smithi176.stdout:0/170: creat d5/d1c/f32 x:0 0 0 2024-07-16T09:55:29.848 INFO:tasks.workunit.client.1.smithi176.stdout:9/87: dread d5/f16 [0,4194304] 0 2024-07-16T09:55:29.849 INFO:tasks.workunit.client.1.smithi176.stdout:4/159: mknod da/d14/c2d 0 2024-07-16T09:55:29.849 INFO:tasks.workunit.client.1.smithi176.stdout:2/132: dread dd/de/f10 [0,4194304] 0 2024-07-16T09:55:29.851 INFO:tasks.workunit.client.1.smithi176.stdout:2/133: write f3 [1943562,109047] 0 2024-07-16T09:55:29.851 INFO:tasks.workunit.client.1.smithi176.stdout:9/88: dread - d5/de/f15 zero size 2024-07-16T09:55:29.851 INFO:tasks.workunit.client.1.smithi176.stdout:8/60: mknod d8/c12 0 2024-07-16T09:55:29.851 INFO:tasks.workunit.client.1.smithi176.stdout:8/61: stat c4 0 2024-07-16T09:55:29.852 INFO:tasks.workunit.client.1.smithi176.stdout:4/160: unlink l4 0 2024-07-16T09:55:29.853 INFO:tasks.workunit.client.1.smithi176.stdout:1/78: unlink d4/db/ce 0 2024-07-16T09:55:29.853 INFO:tasks.workunit.client.1.smithi176.stdout:4/161: chown da/dc/d10/d28/l2c 1710850571 1 2024-07-16T09:55:29.853 INFO:tasks.workunit.client.1.smithi176.stdout:1/79: read - d4/ff zero size 2024-07-16T09:55:29.854 INFO:tasks.workunit.client.1.smithi176.stdout:2/134: mknod dd/de/c2b 0 2024-07-16T09:55:29.854 INFO:tasks.workunit.client.1.smithi176.stdout:8/62: mkdir d8/d13 0 2024-07-16T09:55:29.854 INFO:tasks.workunit.client.1.smithi176.stdout:4/162: write da/dc/d10/d15/f1b [3602728,79765] 0 2024-07-16T09:55:29.855 INFO:tasks.workunit.client.1.smithi176.stdout:1/80: rmdir d4/d8 39 2024-07-16T09:55:29.856 INFO:tasks.workunit.client.1.smithi176.stdout:2/135: symlink dd/de/l2c 0 2024-07-16T09:55:29.856 INFO:tasks.workunit.client.1.smithi176.stdout:4/163: fdatasync da/dc/fe 0 2024-07-16T09:55:29.857 INFO:tasks.workunit.client.1.smithi176.stdout:8/63: mkdir d8/d14 0 2024-07-16T09:55:29.859 INFO:tasks.workunit.client.1.smithi176.stdout:2/136: write dd/f2a [1046679,55] 0 2024-07-16T09:55:29.859 INFO:tasks.workunit.client.1.smithi176.stdout:3/35: truncate d3/f8 3582704 0 2024-07-16T09:55:29.859 INFO:tasks.workunit.client.1.smithi176.stdout:6/144: sync 2024-07-16T09:55:29.860 INFO:tasks.workunit.client.1.smithi176.stdout:5/203: sync 2024-07-16T09:55:29.860 INFO:tasks.workunit.client.1.smithi176.stdout:9/89: truncate d5/fb 857331 0 2024-07-16T09:55:29.860 INFO:tasks.workunit.client.1.smithi176.stdout:1/81: creat d4/db/f1b x:0 0 0 2024-07-16T09:55:29.862 INFO:tasks.workunit.client.1.smithi176.stdout:0/171: readlink d5/l31 0 2024-07-16T09:55:29.862 INFO:tasks.workunit.client.1.smithi176.stdout:1/82: read - d4/ff zero size 2024-07-16T09:55:29.863 INFO:tasks.workunit.client.1.smithi176.stdout:2/137: write dd/de/f10 [1331390,76848] 0 2024-07-16T09:55:29.864 INFO:tasks.workunit.client.1.smithi176.stdout:8/64: creat d8/d13/f15 x:0 0 0 2024-07-16T09:55:29.865 INFO:tasks.workunit.client.1.smithi176.stdout:6/145: write d1/d5/f2a [4591478,118918] 0 2024-07-16T09:55:29.866 INFO:tasks.workunit.client.1.smithi176.stdout:1/83: write d4/db/f16 [864077,7020] 0 2024-07-16T09:55:29.866 INFO:tasks.workunit.client.1.smithi176.stdout:9/90: mknod d5/de/c19 0 2024-07-16T09:55:29.866 INFO:tasks.workunit.client.1.smithi176.stdout:7/135: truncate f0 4682461 0 2024-07-16T09:55:29.867 INFO:tasks.workunit.client.1.smithi176.stdout:1/84: stat d4/db 0 2024-07-16T09:55:29.868 INFO:tasks.workunit.client.1.smithi176.stdout:9/91: chown d5/la 125979 1 2024-07-16T09:55:29.870 INFO:tasks.workunit.client.1.smithi176.stdout:2/138: stat dd/d16/f27 0 2024-07-16T09:55:29.870 INFO:tasks.workunit.client.1.smithi176.stdout:4/164: dwrite da/d14/f26 [0,4194304] 0 2024-07-16T09:55:29.870 INFO:tasks.workunit.client.1.smithi176.stdout:2/139: chown f3 39672862 1 2024-07-16T09:55:29.871 INFO:tasks.workunit.client.1.smithi176.stdout:0/172: chown d5/d1c/f21 2150 1 2024-07-16T09:55:29.872 INFO:tasks.workunit.client.1.smithi176.stdout:7/136: symlink d4/dd/l35 0 2024-07-16T09:55:29.873 INFO:tasks.workunit.client.1.smithi176.stdout:9/92: rmdir d5 39 2024-07-16T09:55:29.878 INFO:tasks.workunit.client.1.smithi176.stdout:7/137: dread - d4/d1f/f2a zero size 2024-07-16T09:55:29.879 INFO:tasks.workunit.client.1.smithi176.stdout:0/173: mknod d5/d1c/d24/d26/c33 0 2024-07-16T09:55:29.881 INFO:tasks.workunit.client.1.smithi176.stdout:8/65: link d8/c12 d8/d13/c16 0 2024-07-16T09:55:29.882 INFO:tasks.workunit.client.1.smithi176.stdout:9/93: fdatasync d5/de/f12 0 2024-07-16T09:55:29.885 INFO:tasks.workunit.client.1.smithi176.stdout:7/138: write d4/fb [720183,25316] 0 2024-07-16T09:55:29.887 INFO:tasks.workunit.client.1.smithi176.stdout:8/66: write d8/fa [1742683,97239] 0 2024-07-16T09:55:29.887 INFO:tasks.workunit.client.1.smithi176.stdout:2/140: dwrite dd/de/f10 [0,4194304] 0 2024-07-16T09:55:29.888 INFO:tasks.workunit.client.1.smithi176.stdout:5/204: dwrite d4/d14/d19/d1b/f26 [0,4194304] 0 2024-07-16T09:55:29.889 INFO:tasks.workunit.client.1.smithi176.stdout:1/85: rename d4/c7 to d4/c1c 0 2024-07-16T09:55:29.895 INFO:tasks.workunit.client.1.smithi176.stdout:4/165: dwrite da/f19 [0,4194304] 0 2024-07-16T09:55:29.896 INFO:tasks.workunit.client.1.smithi176.stdout:4/166: truncate f0 1350876 0 2024-07-16T09:55:29.897 INFO:tasks.workunit.client.1.smithi176.stdout:8/67: creat d8/f17 x:0 0 0 2024-07-16T09:55:29.899 INFO:tasks.workunit.client.1.smithi176.stdout:7/139: creat d4/dd/f36 x:0 0 0 2024-07-16T09:55:29.899 INFO:tasks.workunit.client.1.smithi176.stdout:2/141: chown dd/de/l14 168879 1 2024-07-16T09:55:29.899 INFO:tasks.workunit.client.1.smithi176.stdout:5/205: symlink d4/d14/d19/l48 0 2024-07-16T09:55:29.900 INFO:tasks.workunit.client.1.smithi176.stdout:9/94: dread d5/fb [0,4194304] 0 2024-07-16T09:55:29.900 INFO:tasks.workunit.client.1.smithi176.stdout:0/174: dread f4 [0,4194304] 0 2024-07-16T09:55:29.902 INFO:tasks.workunit.client.1.smithi176.stdout:3/36: truncate d3/f8 2573293 0 2024-07-16T09:55:29.905 INFO:tasks.workunit.client.1.smithi176.stdout:9/95: creat d5/de/f1a x:0 0 0 2024-07-16T09:55:29.906 INFO:tasks.workunit.client.1.smithi176.stdout:5/206: rename d4/d14/d19 to d4/d14/d19/d1b/d49 22 2024-07-16T09:55:29.908 INFO:tasks.workunit.client.1.smithi176.stdout:2/142: truncate dd/d16/f27 260986 0 2024-07-16T09:55:29.909 INFO:tasks.workunit.client.1.smithi176.stdout:0/175: dread d5/f13 [0,4194304] 0 2024-07-16T09:55:29.911 INFO:tasks.workunit.client.1.smithi176.stdout:4/167: link c5 da/c2e 0 2024-07-16T09:55:29.911 INFO:tasks.workunit.client.1.smithi176.stdout:9/96: creat d5/de/f1b x:0 0 0 2024-07-16T09:55:29.912 INFO:tasks.workunit.client.1.smithi176.stdout:5/207: write d4/d10/d17/f32 [396652,68758] 0 2024-07-16T09:55:29.913 INFO:tasks.workunit.client.1.smithi176.stdout:3/37: getdents d3 0 2024-07-16T09:55:29.913 INFO:tasks.workunit.client.1.smithi176.stdout:1/86: dwrite f1 [0,4194304] 0 2024-07-16T09:55:29.916 INFO:tasks.workunit.client.1.smithi176.stdout:9/97: read d5/f10 [3544156,97788] 0 2024-07-16T09:55:29.919 INFO:tasks.workunit.client.1.smithi176.stdout:8/68: dwrite d8/fa [0,4194304] 0 2024-07-16T09:55:29.922 INFO:tasks.workunit.client.1.smithi176.stdout:5/208: unlink d4/d10/d17/c23 0 2024-07-16T09:55:29.923 INFO:tasks.workunit.client.1.smithi176.stdout:8/69: rename d8 to d8/d13/d18 22 2024-07-16T09:55:29.924 INFO:tasks.workunit.client.1.smithi176.stdout:1/87: read d4/db/f16 [713010,6313] 0 2024-07-16T09:55:29.924 INFO:tasks.workunit.client.1.smithi176.stdout:7/140: dwrite d4/dd/f23 [0,4194304] 0 2024-07-16T09:55:29.926 INFO:tasks.workunit.client.0.smithi156.stdout:9/54: sync 2024-07-16T09:55:29.927 INFO:tasks.workunit.client.0.smithi156.stdout:9/55: write d2/d5/f8 [970554,127832] 0 2024-07-16T09:55:29.928 INFO:tasks.workunit.client.1.smithi176.stdout:8/70: dread - d8/f17 zero size 2024-07-16T09:55:29.928 INFO:tasks.workunit.client.1.smithi176.stdout:9/98: truncate d5/fd 1513121 0 2024-07-16T09:55:29.931 INFO:tasks.workunit.client.1.smithi176.stdout:1/88: write d4/f17 [1711321,63685] 0 2024-07-16T09:55:29.934 INFO:tasks.workunit.client.1.smithi176.stdout:2/143: dwrite f9 [0,4194304] 0 2024-07-16T09:55:29.935 INFO:tasks.workunit.client.1.smithi176.stdout:7/141: read d4/dd/f1a [244902,84944] 0 2024-07-16T09:55:29.936 INFO:tasks.workunit.client.0.smithi156.stdout:9/56: dwrite d2/d5/f8 [0,4194304] 0 2024-07-16T09:55:29.936 INFO:tasks.workunit.client.1.smithi176.stdout:3/38: dread d3/f8 [0,4194304] 0 2024-07-16T09:55:29.936 INFO:tasks.workunit.client.1.smithi176.stdout:4/168: dwrite da/f1a [0,4194304] 0 2024-07-16T09:55:29.937 INFO:tasks.workunit.client.1.smithi176.stdout:5/209: unlink d4/d14/d19/f35 0 2024-07-16T09:55:29.937 INFO:tasks.workunit.client.1.smithi176.stdout:0/176: link d5/f2a d5/d1c/d24/f34 0 2024-07-16T09:55:29.937 INFO:tasks.workunit.client.0.smithi156.stdout:9/57: write d2/f13 [492000,27877] 0 2024-07-16T09:55:29.939 INFO:tasks.workunit.client.1.smithi176.stdout:7/142: readlink d4/d1f/d22/l2c 0 2024-07-16T09:55:29.939 INFO:tasks.workunit.client.0.smithi156.stdout:9/58: write d2/d5/f8 [2995083,6558] 0 2024-07-16T09:55:29.941 INFO:tasks.workunit.client.0.smithi156.stdout:9/59: write d2/d5/ff [215746,24115] 0 2024-07-16T09:55:29.942 INFO:tasks.workunit.client.0.smithi156.stdout:5/66: sync 2024-07-16T09:55:29.942 INFO:tasks.workunit.client.0.smithi156.stdout:3/34: sync 2024-07-16T09:55:29.942 INFO:tasks.workunit.client.0.smithi156.stdout:1/70: sync 2024-07-16T09:55:29.942 INFO:tasks.workunit.client.0.smithi156.stdout:7/48: sync 2024-07-16T09:55:29.942 INFO:tasks.workunit.client.0.smithi156.stdout:2/144: sync 2024-07-16T09:55:29.943 INFO:tasks.workunit.client.0.smithi156.stdout:0/29: sync 2024-07-16T09:55:29.943 INFO:tasks.workunit.client.0.smithi156.stdout:6/102: sync 2024-07-16T09:55:29.943 INFO:tasks.workunit.client.0.smithi156.stdout:4/88: sync 2024-07-16T09:55:29.943 INFO:tasks.workunit.client.0.smithi156.stdout:8/30: sync 2024-07-16T09:55:29.943 INFO:tasks.workunit.client.1.smithi176.stdout:3/39: mkdir d3/da 0 2024-07-16T09:55:29.945 INFO:tasks.workunit.client.1.smithi176.stdout:3/40: stat d3/da 0 2024-07-16T09:55:29.947 INFO:tasks.workunit.client.0.smithi156.stdout:2/145: stat d0/d13/d17/d1c/l1d 0 2024-07-16T09:55:29.947 INFO:tasks.workunit.client.0.smithi156.stdout:9/60: fsync d2/f3 0 2024-07-16T09:55:29.948 INFO:tasks.workunit.client.1.smithi176.stdout:5/210: symlink d4/d14/d19/l4a 0 2024-07-16T09:55:29.950 INFO:tasks.workunit.client.1.smithi176.stdout:8/71: symlink d8/d14/l19 0 2024-07-16T09:55:29.953 INFO:tasks.workunit.client.1.smithi176.stdout:7/143: creat d4/dd/f37 x:0 0 0 2024-07-16T09:55:29.954 INFO:tasks.workunit.client.0.smithi156.stdout:1/71: chown dc/l12 19633643 1 2024-07-16T09:55:29.954 INFO:tasks.workunit.client.0.smithi156.stdout:4/89: rename d4/d1b to d4/d23 0 2024-07-16T09:55:29.954 INFO:tasks.workunit.client.0.smithi156.stdout:8/31: creat d0/f8 x:0 0 0 2024-07-16T09:55:29.954 INFO:tasks.workunit.client.1.smithi176.stdout:3/41: fdatasync d3/f8 0 2024-07-16T09:55:29.955 INFO:tasks.workunit.client.1.smithi176.stdout:2/144: dwrite f9 [0,4194304] 0 2024-07-16T09:55:29.955 INFO:tasks.workunit.client.1.smithi176.stdout:0/177: getdents d5/d25 0 2024-07-16T09:55:29.956 INFO:tasks.workunit.client.0.smithi156.stdout:4/90: fsync f1 0 2024-07-16T09:55:29.956 INFO:tasks.workunit.client.0.smithi156.stdout:6/103: creat d3/d4/f22 x:0 0 0 2024-07-16T09:55:29.957 INFO:tasks.workunit.client.1.smithi176.stdout:5/211: creat d4/d14/d3d/f4b x:0 0 0 2024-07-16T09:55:29.957 INFO:tasks.workunit.client.1.smithi176.stdout:9/99: dread f2 [0,4194304] 0 2024-07-16T09:55:29.958 INFO:tasks.workunit.client.1.smithi176.stdout:0/178: chown d5/d1c/f27 4 1 2024-07-16T09:55:29.959 INFO:tasks.workunit.client.1.smithi176.stdout:4/169: dread da/d14/f26 [0,4194304] 0 2024-07-16T09:55:29.961 INFO:tasks.workunit.client.1.smithi176.stdout:5/212: symlink d4/d14/d19/d1b/d36/d38/l4c 0 2024-07-16T09:55:29.963 INFO:tasks.workunit.client.0.smithi156.stdout:7/49: link fb dc/df/f12 0 2024-07-16T09:55:29.963 INFO:tasks.workunit.client.1.smithi176.stdout:1/89: dread d4/d8/fa [0,4194304] 0 2024-07-16T09:55:29.964 INFO:tasks.workunit.client.1.smithi176.stdout:3/42: mknod d3/da/cb 0 2024-07-16T09:55:29.964 INFO:tasks.workunit.client.1.smithi176.stdout:9/100: write d5/f10 [435818,92777] 0 2024-07-16T09:55:29.965 INFO:tasks.workunit.client.0.smithi156.stdout:1/72: creat dc/d14/f1d x:0 0 0 2024-07-16T09:55:29.967 INFO:tasks.workunit.client.0.smithi156.stdout:7/50: chown c6 42058 1 2024-07-16T09:55:29.967 INFO:tasks.workunit.client.0.smithi156.stdout:3/35: link l1 d8/l9 0 2024-07-16T09:55:29.968 INFO:tasks.workunit.client.0.smithi156.stdout:5/67: dwrite ff [0,4194304] 0 2024-07-16T09:55:29.970 INFO:tasks.workunit.client.1.smithi176.stdout:8/72: dread d8/fa [0,4194304] 0 2024-07-16T09:55:29.970 INFO:tasks.workunit.client.1.smithi176.stdout:7/144: rename d4/d5/d10/l1b to d4/d5/l38 0 2024-07-16T09:55:29.971 INFO:tasks.workunit.client.0.smithi156.stdout:8/32: unlink d0/c3 0 2024-07-16T09:55:29.972 INFO:tasks.workunit.client.1.smithi176.stdout:5/213: rmdir d4/d14 39 2024-07-16T09:55:29.973 INFO:tasks.workunit.client.0.smithi156.stdout:1/73: dread - dc/d14/f1d zero size 2024-07-16T09:55:29.974 INFO:tasks.workunit.client.1.smithi176.stdout:0/179: mkdir d5/d1c/d24/d35 0 2024-07-16T09:55:29.975 INFO:tasks.workunit.client.0.smithi156.stdout:2/146: creat d0/d5/d29/d2d/f31 x:0 0 0 2024-07-16T09:55:29.978 INFO:tasks.workunit.client.0.smithi156.stdout:5/68: creat f14 x:0 0 0 2024-07-16T09:55:29.978 INFO:tasks.workunit.client.0.smithi156.stdout:4/91: dwrite f1 [4194304,4194304] 0 2024-07-16T09:55:29.981 INFO:tasks.workunit.client.0.smithi156.stdout:1/74: unlink f7 0 2024-07-16T09:55:29.982 INFO:tasks.workunit.client.1.smithi176.stdout:2/145: dwrite dd/d16/f27 [0,4194304] 0 2024-07-16T09:55:29.983 INFO:tasks.workunit.client.0.smithi156.stdout:2/147: creat d0/d5/d29/f32 x:0 0 0 2024-07-16T09:55:29.984 INFO:tasks.workunit.client.1.smithi176.stdout:4/170: dread da/d14/f26 [0,4194304] 0 2024-07-16T09:55:29.984 INFO:tasks.workunit.client.0.smithi156.stdout:4/92: read d4/f9 [2648038,3541] 0 2024-07-16T09:55:29.987 INFO:tasks.workunit.client.1.smithi176.stdout:1/90: dwrite d4/f17 [0,4194304] 0 2024-07-16T09:55:29.987 INFO:tasks.workunit.client.1.smithi176.stdout:2/146: chown dd/de/f10 1495 1 2024-07-16T09:55:29.987 INFO:tasks.workunit.client.1.smithi176.stdout:8/73: rename c2 to d8/c1a 0 2024-07-16T09:55:29.988 INFO:tasks.workunit.client.1.smithi176.stdout:4/171: write da/f1a [2712277,114852] 0 2024-07-16T09:55:29.989 INFO:tasks.workunit.client.1.smithi176.stdout:9/101: dwrite d5/de/f1b [0,4194304] 0 2024-07-16T09:55:29.991 INFO:tasks.workunit.client.0.smithi156.stdout:9/61: dwrite d2/d5/fb [0,4194304] 0 2024-07-16T09:55:29.992 INFO:tasks.workunit.client.0.smithi156.stdout:7/51: rename dc/df to dc/d13 0 2024-07-16T09:55:29.992 INFO:tasks.workunit.client.1.smithi176.stdout:2/147: chown dd/de/c22 149 1 2024-07-16T09:55:29.993 INFO:tasks.workunit.client.1.smithi176.stdout:0/180: mknod d5/d1c/d24/c36 0 2024-07-16T09:55:29.996 INFO:tasks.workunit.client.1.smithi176.stdout:0/181: chown d5/f18 4116 1 2024-07-16T09:55:29.996 INFO:tasks.workunit.client.1.smithi176.stdout:4/172: chown da/dc/d10/d15/f1b 4128234 1 2024-07-16T09:55:29.997 INFO:tasks.workunit.client.1.smithi176.stdout:3/43: dwrite d3/f8 [0,4194304] 0 2024-07-16T09:55:29.998 INFO:tasks.workunit.client.0.smithi156.stdout:1/75: mkdir dc/d14/d1e 0 2024-07-16T09:55:29.999 INFO:tasks.workunit.client.0.smithi156.stdout:5/69: write f13 [2203879,58764] 0 2024-07-16T09:55:29.999 INFO:tasks.workunit.client.0.smithi156.stdout:0/30: dwrite d0/f3 [0,4194304] 0 2024-07-16T09:55:30.000 INFO:tasks.workunit.client.1.smithi176.stdout:5/214: unlink d4/d14/l1c 0 2024-07-16T09:55:30.003 INFO:tasks.workunit.client.1.smithi176.stdout:1/91: dread d4/f10 [0,4194304] 0 2024-07-16T09:55:30.003 INFO:tasks.workunit.client.1.smithi176.stdout:7/145: link d4/d1f/d22/l29 d4/d5/d10/d17/d1d/l39 0 2024-07-16T09:55:30.004 INFO:tasks.workunit.client.1.smithi176.stdout:8/74: creat d8/d13/f1b x:0 0 0 2024-07-16T09:55:30.006 INFO:tasks.workunit.client.1.smithi176.stdout:7/146: read d4/dd/f23 [2143838,97743] 0 2024-07-16T09:55:30.008 INFO:tasks.workunit.client.1.smithi176.stdout:3/44: mknod d3/da/cc 0 2024-07-16T09:55:30.008 INFO:tasks.workunit.client.1.smithi176.stdout:5/215: mkdir d4/d14/d3d/d4d 0 2024-07-16T09:55:30.011 INFO:tasks.workunit.client.1.smithi176.stdout:1/92: mknod d4/d8/c1d 0 2024-07-16T09:55:30.012 INFO:tasks.workunit.client.1.smithi176.stdout:8/75: chown d8/c10 3620 1 2024-07-16T09:55:30.013 INFO:tasks.workunit.client.1.smithi176.stdout:7/147: rmdir d4/d1f 39 2024-07-16T09:55:30.013 INFO:tasks.workunit.client.0.smithi156.stdout:8/33: link d0/f6 d0/f9 0 2024-07-16T09:55:30.014 INFO:tasks.workunit.client.1.smithi176.stdout:8/76: dread - d8/fd zero size 2024-07-16T09:55:30.015 INFO:tasks.workunit.client.1.smithi176.stdout:3/45: dread d3/f8 [0,4194304] 0 2024-07-16T09:55:30.015 INFO:tasks.workunit.client.0.smithi156.stdout:4/93: rename f1 to d4/d23/f24 0 2024-07-16T09:55:30.015 INFO:tasks.workunit.client.0.smithi156.stdout:7/52: creat dc/f14 x:0 0 0 2024-07-16T09:55:30.016 INFO:tasks.workunit.client.1.smithi176.stdout:0/182: symlink d5/d25/l37 0 2024-07-16T09:55:30.016 INFO:tasks.workunit.client.0.smithi156.stdout:6/104: truncate d3/d4/f14 2824670 0 2024-07-16T09:55:30.017 INFO:tasks.workunit.client.1.smithi176.stdout:5/216: dread d4/f13 [0,4194304] 0 2024-07-16T09:55:30.017 INFO:tasks.workunit.client.0.smithi156.stdout:2/148: symlink d0/d5/d29/l33 0 2024-07-16T09:55:30.017 INFO:tasks.workunit.client.0.smithi156.stdout:8/34: write d0/f2 [2562651,103844] 0 2024-07-16T09:55:30.018 INFO:tasks.workunit.client.0.smithi156.stdout:0/31: write d0/f3 [4808283,40344] 0 2024-07-16T09:55:30.019 INFO:tasks.workunit.client.1.smithi176.stdout:9/102: creat d5/f1c x:0 0 0 2024-07-16T09:55:30.019 INFO:tasks.workunit.client.0.smithi156.stdout:8/35: chown d0/f6 165752550 1 2024-07-16T09:55:30.019 INFO:tasks.workunit.client.0.smithi156.stdout:2/149: read d0/d13/d17/f21 [2374546,60746] 0 2024-07-16T09:55:30.020 INFO:tasks.workunit.client.1.smithi176.stdout:9/103: read - d5/de/f15 zero size 2024-07-16T09:55:30.022 INFO:tasks.workunit.client.1.smithi176.stdout:3/46: dread d3/f8 [0,4194304] 0 2024-07-16T09:55:30.022 INFO:tasks.workunit.client.1.smithi176.stdout:1/93: creat d4/d8/f1e x:0 0 0 2024-07-16T09:55:30.022 INFO:tasks.workunit.client.0.smithi156.stdout:9/62: dread d2/d5/f8 [0,4194304] 0 2024-07-16T09:55:30.023 INFO:tasks.workunit.client.1.smithi176.stdout:8/77: symlink d8/l1c 0 2024-07-16T09:55:30.024 INFO:tasks.workunit.client.1.smithi176.stdout:8/78: dread - d8/fd zero size 2024-07-16T09:55:30.024 INFO:tasks.workunit.client.1.smithi176.stdout:5/217: creat d4/d14/d19/d1b/d36/f4e x:0 0 0 2024-07-16T09:55:30.025 INFO:tasks.workunit.client.1.smithi176.stdout:9/104: fsync f2 0 2024-07-16T09:55:30.025 INFO:tasks.workunit.client.1.smithi176.stdout:2/148: link c1 dd/de/c2d 0 2024-07-16T09:55:30.026 INFO:tasks.workunit.client.1.smithi176.stdout:8/79: write d8/ff [3198994,116525] 0 2024-07-16T09:55:30.026 INFO:tasks.workunit.client.0.smithi156.stdout:9/63: dread d2/d5/fb [0,4194304] 0 2024-07-16T09:55:30.027 INFO:tasks.workunit.client.1.smithi176.stdout:3/47: write d3/f8 [446925,66952] 0 2024-07-16T09:55:30.027 INFO:tasks.workunit.client.1.smithi176.stdout:5/218: chown d4/d14/d19/d1b 3885 1 2024-07-16T09:55:30.027 INFO:tasks.workunit.client.1.smithi176.stdout:0/183: mknod d5/d1c/d24/d35/c38 0 2024-07-16T09:55:30.027 INFO:tasks.workunit.client.0.smithi156.stdout:9/64: chown d2/de/c11 3011587 1 2024-07-16T09:55:30.028 INFO:tasks.workunit.client.1.smithi176.stdout:9/105: chown d5/de 3019899 1 2024-07-16T09:55:30.028 INFO:tasks.workunit.client.1.smithi176.stdout:7/148: rename d4/dd/f16 to d4/d1f/d22/f3a 0 2024-07-16T09:55:30.029 INFO:tasks.workunit.client.1.smithi176.stdout:1/94: rmdir d4/d8/d12 39 2024-07-16T09:55:30.029 INFO:tasks.workunit.client.1.smithi176.stdout:9/106: chown d5 8 1 2024-07-16T09:55:30.031 INFO:tasks.workunit.client.1.smithi176.stdout:1/95: readlink d4/db/l15 0 2024-07-16T09:55:30.032 INFO:tasks.workunit.client.1.smithi176.stdout:9/107: truncate d5/de/f1a 937151 0 2024-07-16T09:55:30.034 INFO:tasks.workunit.client.1.smithi176.stdout:1/96: write d4/ff [290731,67003] 0 2024-07-16T09:55:30.035 INFO:tasks.workunit.client.1.smithi176.stdout:2/149: dread dd/de/f10 [0,4194304] 0 2024-07-16T09:55:30.035 INFO:tasks.workunit.client.1.smithi176.stdout:0/184: read d5/f22 [651895,5387] 0 2024-07-16T09:55:30.037 INFO:tasks.workunit.client.1.smithi176.stdout:7/149: creat d4/d5/d21/f3b x:0 0 0 2024-07-16T09:55:30.038 INFO:tasks.workunit.client.1.smithi176.stdout:3/48: rename d3/l9 to d3/da/ld 0 2024-07-16T09:55:30.041 INFO:tasks.workunit.client.1.smithi176.stdout:2/150: dread f9 [0,4194304] 0 2024-07-16T09:55:30.043 INFO:tasks.workunit.client.1.smithi176.stdout:2/151: chown f9 828 1 2024-07-16T09:55:30.047 INFO:tasks.workunit.client.1.smithi176.stdout:5/219: rename d4/d10/d17/f32 to d4/d14/d19/d1b/d36/f4f 0 2024-07-16T09:55:30.048 INFO:tasks.workunit.client.1.smithi176.stdout:7/150: dwrite d4/dd/f37 [0,4194304] 0 2024-07-16T09:55:30.048 INFO:tasks.workunit.client.1.smithi176.stdout:1/97: mknod d4/c1f 0 2024-07-16T09:55:30.051 INFO:tasks.workunit.client.1.smithi176.stdout:5/220: read d4/d10/d17/f28 [2596266,35835] 0 2024-07-16T09:55:30.052 INFO:tasks.workunit.client.1.smithi176.stdout:2/152: chown c1 111563 1 2024-07-16T09:55:30.053 INFO:tasks.workunit.client.1.smithi176.stdout:3/49: dwrite d3/f8 [0,4194304] 0 2024-07-16T09:55:30.054 INFO:tasks.workunit.client.1.smithi176.stdout:0/185: write d5/f6 [1714383,94109] 0 2024-07-16T09:55:30.056 INFO:tasks.workunit.client.1.smithi176.stdout:2/153: fdatasync dd/d16/f1c 0 2024-07-16T09:55:30.060 INFO:tasks.workunit.client.1.smithi176.stdout:7/151: rename d4/d5/d10/c1e to d4/d1f/c3c 0 2024-07-16T09:55:30.062 INFO:tasks.workunit.client.1.smithi176.stdout:9/108: dwrite d5/f16 [0,4194304] 0 2024-07-16T09:55:30.063 INFO:tasks.workunit.client.1.smithi176.stdout:0/186: write d5/f22 [691552,3734] 0 2024-07-16T09:55:30.068 INFO:tasks.workunit.client.1.smithi176.stdout:2/154: mkdir dd/d2e 0 2024-07-16T09:55:30.069 INFO:tasks.workunit.client.1.smithi176.stdout:0/187: mkdir d5/d39 0 2024-07-16T09:55:30.070 INFO:tasks.workunit.client.1.smithi176.stdout:8/80: dwrite f5 [0,4194304] 0 2024-07-16T09:55:30.070 INFO:tasks.workunit.client.1.smithi176.stdout:5/221: read - d4/d14/d3d/f4b zero size 2024-07-16T09:55:30.071 INFO:tasks.workunit.client.1.smithi176.stdout:7/152: symlink d4/dd/d26/l3d 0 2024-07-16T09:55:30.076 INFO:tasks.workunit.client.1.smithi176.stdout:8/81: symlink d8/l1d 0 2024-07-16T09:55:30.078 INFO:tasks.workunit.client.1.smithi176.stdout:1/98: dwrite d4/db/f16 [0,4194304] 0 2024-07-16T09:55:30.082 INFO:tasks.workunit.client.1.smithi176.stdout:1/99: fdatasync d4/ff 0 2024-07-16T09:55:30.082 INFO:tasks.workunit.client.1.smithi176.stdout:5/222: creat d4/d10/d17/f50 x:0 0 0 2024-07-16T09:55:30.082 INFO:tasks.workunit.client.1.smithi176.stdout:2/155: write dd/d16/f1d [4114745,67469] 0 2024-07-16T09:55:30.082 INFO:tasks.workunit.client.1.smithi176.stdout:9/109: fsync d5/f16 0 2024-07-16T09:55:30.083 INFO:tasks.workunit.client.1.smithi176.stdout:8/82: unlink d8/ff 0 2024-07-16T09:55:30.083 INFO:tasks.workunit.client.1.smithi176.stdout:3/50: dwrite d3/f8 [0,4194304] 0 2024-07-16T09:55:30.084 INFO:tasks.workunit.client.1.smithi176.stdout:7/153: truncate d4/d5/d21/f3b 190831 0 2024-07-16T09:55:30.086 INFO:tasks.workunit.client.1.smithi176.stdout:3/51: stat d3/l4 0 2024-07-16T09:55:30.086 INFO:tasks.workunit.client.1.smithi176.stdout:3/52: readlink d3/l7 0 2024-07-16T09:55:30.087 INFO:tasks.workunit.client.1.smithi176.stdout:0/188: dread f3 [0,4194304] 0 2024-07-16T09:55:30.091 INFO:tasks.workunit.client.1.smithi176.stdout:7/154: symlink d4/d5/d10/l3e 0 2024-07-16T09:55:30.091 INFO:tasks.workunit.client.1.smithi176.stdout:1/100: mknod d4/c20 0 2024-07-16T09:55:30.093 INFO:tasks.workunit.client.1.smithi176.stdout:9/110: mkdir d5/d1d 0 2024-07-16T09:55:30.093 INFO:tasks.workunit.client.1.smithi176.stdout:0/189: dread d5/f6 [0,4194304] 0 2024-07-16T09:55:30.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:55:29 smithi176 ceph-mon[36626]: pgmap v246: 65 pgs: 65 active+clean; 357 MiB data, 2.8 GiB used, 534 GiB / 536 GiB avail; 2.2 MiB/s rd, 25 MiB/s wr, 466 op/s 2024-07-16T09:55:30.094 INFO:tasks.workunit.client.1.smithi176.stdout:7/155: creat d4/d5/d10/f3f x:0 0 0 2024-07-16T09:55:30.095 INFO:tasks.workunit.client.1.smithi176.stdout:0/190: stat d5/d1c/f32 0 2024-07-16T09:55:30.097 INFO:tasks.workunit.client.1.smithi176.stdout:1/101: mkdir d4/d8/d12/d21 0 2024-07-16T09:55:30.097 INFO:tasks.workunit.client.1.smithi176.stdout:8/83: getdents d8/d14 0 2024-07-16T09:55:30.098 INFO:tasks.workunit.client.1.smithi176.stdout:7/156: symlink d4/dd/d26/l40 0 2024-07-16T09:55:30.101 INFO:tasks.workunit.client.1.smithi176.stdout:8/84: mkdir d8/d14/d1e 0 2024-07-16T09:55:30.101 INFO:tasks.workunit.client.1.smithi176.stdout:9/111: creat d5/f1e x:0 0 0 2024-07-16T09:55:30.101 INFO:tasks.workunit.client.1.smithi176.stdout:7/157: dread - d4/dd/f36 zero size 2024-07-16T09:55:30.102 INFO:tasks.workunit.client.1.smithi176.stdout:9/112: chown d5/l8 0 1 2024-07-16T09:55:30.102 INFO:tasks.workunit.client.1.smithi176.stdout:9/113: chown d5/d1d 4075 1 2024-07-16T09:55:30.103 INFO:tasks.workunit.client.1.smithi176.stdout:8/85: creat d8/d14/d1e/f1f x:0 0 0 2024-07-16T09:55:30.103 INFO:tasks.workunit.client.1.smithi176.stdout:7/158: mknod d4/d1f/d22/c41 0 2024-07-16T09:55:30.104 INFO:tasks.workunit.client.1.smithi176.stdout:9/114: chown d5/l8 93160 1 2024-07-16T09:55:30.104 INFO:tasks.workunit.client.1.smithi176.stdout:1/102: dread d4/db/f16 [0,4194304] 0 2024-07-16T09:55:30.104 INFO:tasks.workunit.client.1.smithi176.stdout:8/86: chown d8/f17 3788163 1 2024-07-16T09:55:30.105 INFO:tasks.workunit.client.1.smithi176.stdout:8/87: write d8/fd [1045201,51300] 0 2024-07-16T09:55:30.106 INFO:tasks.workunit.client.1.smithi176.stdout:7/159: symlink d4/d1f/d22/l42 0 2024-07-16T09:55:30.107 INFO:tasks.workunit.client.1.smithi176.stdout:3/53: fsync d3/f8 0 2024-07-16T09:55:30.108 INFO:tasks.workunit.client.1.smithi176.stdout:8/88: mkdir d8/d20 0 2024-07-16T09:55:30.108 INFO:tasks.workunit.client.1.smithi176.stdout:8/89: readlink d8/l9 0 2024-07-16T09:55:30.108 INFO:tasks.workunit.client.1.smithi176.stdout:1/103: symlink d4/d8/d12/d21/l22 0 2024-07-16T09:55:30.109 INFO:tasks.workunit.client.1.smithi176.stdout:7/160: symlink d4/dd/d26/l43 0 2024-07-16T09:55:30.110 INFO:tasks.workunit.client.1.smithi176.stdout:7/161: chown d4/d5/f2f 40428721 1 2024-07-16T09:55:30.111 INFO:tasks.workunit.client.1.smithi176.stdout:8/90: creat d8/d13/f21 x:0 0 0 2024-07-16T09:55:30.112 INFO:tasks.workunit.client.1.smithi176.stdout:8/91: stat d8/l9 0 2024-07-16T09:55:30.112 INFO:tasks.workunit.client.1.smithi176.stdout:7/162: dread - d4/d5/d21/d33/f34 zero size 2024-07-16T09:55:30.113 INFO:tasks.workunit.client.1.smithi176.stdout:8/92: chown d8/fa 906247 1 2024-07-16T09:55:30.114 INFO:tasks.workunit.client.1.smithi176.stdout:3/54: dwrite d3/f8 [0,4194304] 0 2024-07-16T09:55:30.116 INFO:tasks.workunit.client.1.smithi176.stdout:0/191: write d5/d1c/d24/f34 [198620,41957] 0 2024-07-16T09:55:30.117 INFO:tasks.workunit.client.1.smithi176.stdout:0/192: readlink d5/l9 0 2024-07-16T09:55:30.117 INFO:tasks.workunit.client.1.smithi176.stdout:8/93: chown d8/c1a 378279186 1 2024-07-16T09:55:30.120 INFO:tasks.workunit.client.1.smithi176.stdout:8/94: dread - d8/d14/d1e/f1f zero size 2024-07-16T09:55:30.120 INFO:tasks.workunit.client.1.smithi176.stdout:5/223: truncate d4/d14/d19/d1b/f26 3156075 0 2024-07-16T09:55:30.120 INFO:tasks.workunit.client.1.smithi176.stdout:7/163: creat d4/d5/d21/d33/f44 x:0 0 0 2024-07-16T09:55:30.121 INFO:tasks.workunit.client.1.smithi176.stdout:9/115: write d5/fd [2104774,86461] 0 2024-07-16T09:55:30.124 INFO:tasks.workunit.client.1.smithi176.stdout:1/104: truncate d4/f10 1328367 0 2024-07-16T09:55:30.125 INFO:tasks.workunit.client.1.smithi176.stdout:5/224: rename d4/c1a to d4/d14/d19/d1b/c51 0 2024-07-16T09:55:30.127 INFO:tasks.workunit.client.1.smithi176.stdout:9/116: mknod d5/c1f 0 2024-07-16T09:55:30.128 INFO:tasks.workunit.client.1.smithi176.stdout:3/55: fsync d3/f8 0 2024-07-16T09:55:30.128 INFO:tasks.workunit.client.1.smithi176.stdout:8/95: getdents d8/d14 0 2024-07-16T09:55:30.130 INFO:tasks.workunit.client.1.smithi176.stdout:1/105: getdents d4/d8/d12 0 2024-07-16T09:55:30.132 INFO:tasks.workunit.client.1.smithi176.stdout:1/106: dread - d4/db/f1b zero size 2024-07-16T09:55:30.132 INFO:tasks.workunit.client.1.smithi176.stdout:5/225: rename d4/d10/d17/l42 to d4/d14/l52 0 2024-07-16T09:55:30.133 INFO:tasks.workunit.client.1.smithi176.stdout:0/193: truncate d5/f10 1335012 0 2024-07-16T09:55:30.133 INFO:tasks.workunit.client.1.smithi176.stdout:3/56: dread d3/f8 [0,4194304] 0 2024-07-16T09:55:30.134 INFO:tasks.workunit.client.1.smithi176.stdout:8/96: symlink d8/d14/l22 0 2024-07-16T09:55:30.135 INFO:tasks.workunit.client.1.smithi176.stdout:9/117: mkdir d5/d1d/d20 0 2024-07-16T09:55:30.135 INFO:tasks.workunit.client.1.smithi176.stdout:0/194: read - d5/d1c/f27 zero size 2024-07-16T09:55:30.135 INFO:tasks.workunit.client.1.smithi176.stdout:3/57: fsync d3/f8 0 2024-07-16T09:55:30.136 INFO:tasks.workunit.client.1.smithi176.stdout:9/118: chown d5/d1d/d20 1346179789 1 2024-07-16T09:55:30.138 INFO:tasks.workunit.client.1.smithi176.stdout:9/119: write d5/fb [305044,25099] 0 2024-07-16T09:55:30.138 INFO:tasks.workunit.client.1.smithi176.stdout:7/164: dwrite d4/dd/f23 [0,4194304] 0 2024-07-16T09:55:30.139 INFO:tasks.workunit.client.1.smithi176.stdout:8/97: rename d8/d13 to d8/d23 0 2024-07-16T09:55:30.139 INFO:tasks.workunit.client.0.smithi156.stdout:5/70: symlink l15 0 2024-07-16T09:55:30.142 INFO:tasks.workunit.client.1.smithi176.stdout:0/195: write f4 [4715787,90502] 0 2024-07-16T09:55:30.142 INFO:tasks.workunit.client.1.smithi176.stdout:5/226: creat d4/d14/d19/d1b/d36/f53 x:0 0 0 2024-07-16T09:55:30.144 INFO:tasks.workunit.client.0.smithi156.stdout:5/71: dread f12 [0,4194304] 0 2024-07-16T09:55:30.146 INFO:tasks.workunit.client.1.smithi176.stdout:3/58: creat d3/da/fe x:0 0 0 2024-07-16T09:55:30.148 INFO:tasks.workunit.client.0.smithi156.stdout:2/150: creat d0/d13/d17/f34 x:0 0 0 2024-07-16T09:55:30.148 INFO:tasks.workunit.client.0.smithi156.stdout:2/151: readlink d0/l2 0 2024-07-16T09:55:30.149 INFO:tasks.workunit.client.0.smithi156.stdout:8/36: symlink d0/la 0 2024-07-16T09:55:30.150 INFO:tasks.workunit.client.1.smithi176.stdout:8/98: dwrite f5 [0,4194304] 0 2024-07-16T09:55:30.150 INFO:tasks.workunit.client.1.smithi176.stdout:9/120: mkdir d5/de/d21 0 2024-07-16T09:55:30.151 INFO:tasks.workunit.client.0.smithi156.stdout:0/32: rename d0/f4 to d0/f5 0 2024-07-16T09:55:30.152 INFO:tasks.workunit.client.0.smithi156.stdout:1/76: rmdir dc 39 2024-07-16T09:55:30.152 INFO:tasks.workunit.client.1.smithi176.stdout:0/196: dread f3 [0,4194304] 0 2024-07-16T09:55:30.153 INFO:tasks.workunit.client.0.smithi156.stdout:0/33: chown d0/c2 0 1 2024-07-16T09:55:30.153 INFO:tasks.workunit.client.1.smithi176.stdout:8/99: truncate d8/f17 373331 0 2024-07-16T09:55:30.154 INFO:tasks.workunit.client.0.smithi156.stdout:0/34: rename d0 to d0/d6 22 2024-07-16T09:55:30.154 INFO:tasks.workunit.client.0.smithi156.stdout:5/72: dwrite f14 [0,4194304] 0 2024-07-16T09:55:30.154 INFO:tasks.workunit.client.1.smithi176.stdout:5/227: creat d4/d10/f54 x:0 0 0 2024-07-16T09:55:30.156 INFO:tasks.workunit.client.0.smithi156.stdout:4/94: creat d4/db/dc/d22/f25 x:0 0 0 2024-07-16T09:55:30.160 INFO:tasks.workunit.client.1.smithi176.stdout:3/59: creat d3/da/ff x:0 0 0 2024-07-16T09:55:30.161 INFO:tasks.workunit.client.1.smithi176.stdout:7/165: dwrite d4/d1f/d22/f3a [0,4194304] 0 2024-07-16T09:55:30.161 INFO:tasks.workunit.client.1.smithi176.stdout:9/121: dread d5/f16 [0,4194304] 0 2024-07-16T09:55:30.162 INFO:tasks.workunit.client.0.smithi156.stdout:2/152: symlink d0/d5/d29/d2d/l35 0 2024-07-16T09:55:30.163 INFO:tasks.workunit.client.0.smithi156.stdout:2/153: chown d0/d13/f26 2821344 1 2024-07-16T09:55:30.163 INFO:tasks.workunit.client.0.smithi156.stdout:2/154: chown d0/l6 40781 1 2024-07-16T09:55:30.164 INFO:tasks.workunit.client.1.smithi176.stdout:0/197: rename d5/d1c/f27 to d5/d39/f3a 0 2024-07-16T09:55:30.165 INFO:tasks.workunit.client.0.smithi156.stdout:8/37: creat d0/fb x:0 0 0 2024-07-16T09:55:30.165 INFO:tasks.workunit.client.0.smithi156.stdout:8/38: fdatasync d0/fb 0 2024-07-16T09:55:30.167 INFO:tasks.workunit.client.1.smithi176.stdout:5/228: creat d4/d14/d3d/f55 x:0 0 0 2024-07-16T09:55:30.167 INFO:tasks.workunit.client.1.smithi176.stdout:3/60: rename d3/c5 to d3/c10 0 2024-07-16T09:55:30.168 INFO:tasks.workunit.client.0.smithi156.stdout:0/35: mkdir d0/d7 0 2024-07-16T09:55:30.170 INFO:tasks.workunit.client.0.smithi156.stdout:4/95: symlink d4/d23/l26 0 2024-07-16T09:55:30.171 INFO:tasks.workunit.client.1.smithi176.stdout:0/198: creat d5/d1c/f3b x:0 0 0 2024-07-16T09:55:30.171 INFO:tasks.workunit.client.0.smithi156.stdout:1/77: dread dc/f18 [0,4194304] 0 2024-07-16T09:55:30.172 INFO:tasks.workunit.client.0.smithi156.stdout:1/78: write f4 [2983053,100536] 0 2024-07-16T09:55:30.173 INFO:tasks.workunit.client.0.smithi156.stdout:2/155: symlink d0/d13/d17/d1c/l36 0 2024-07-16T09:55:30.174 INFO:tasks.workunit.client.1.smithi176.stdout:9/122: link d5/lc d5/de/d21/l22 0 2024-07-16T09:55:30.174 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:55:29 smithi156 ceph-mon[29302]: pgmap v246: 65 pgs: 65 active+clean; 357 MiB data, 2.8 GiB used, 534 GiB / 536 GiB avail; 2.2 MiB/s rd, 25 MiB/s wr, 466 op/s 2024-07-16T09:55:30.175 INFO:tasks.workunit.client.1.smithi176.stdout:9/123: fsync d5/de/f12 0 2024-07-16T09:55:30.175 INFO:tasks.workunit.client.0.smithi156.stdout:2/156: write d0/d5/d29/d2d/f31 [654142,8485] 0 2024-07-16T09:55:30.176 INFO:tasks.workunit.client.1.smithi176.stdout:0/199: creat d5/d1c/d24/d35/f3c x:0 0 0 2024-07-16T09:55:30.177 INFO:tasks.workunit.client.1.smithi176.stdout:7/166: dwrite d4/f11 [4194304,4194304] 0 2024-07-16T09:55:30.179 INFO:tasks.workunit.client.1.smithi176.stdout:7/167: write d4/f11 [641019,42398] 0 2024-07-16T09:55:30.180 INFO:tasks.workunit.client.1.smithi176.stdout:9/124: truncate d5/de/f17 1465673 0 2024-07-16T09:55:30.181 INFO:tasks.workunit.client.1.smithi176.stdout:5/229: creat d4/d14/d19/d1b/d36/d38/d39/f56 x:0 0 0 2024-07-16T09:55:30.182 INFO:tasks.workunit.client.0.smithi156.stdout:1/79: dwrite f6 [0,4194304] 0 2024-07-16T09:55:30.182 INFO:tasks.workunit.client.0.smithi156.stdout:9/65: getdents d2 0 2024-07-16T09:55:30.183 INFO:tasks.workunit.client.1.smithi176.stdout:3/61: truncate d3/f8 394757 0 2024-07-16T09:55:30.186 INFO:tasks.workunit.client.1.smithi176.stdout:7/168: dread d4/d5/d21/f3b [0,4194304] 0 2024-07-16T09:55:30.188 INFO:tasks.workunit.client.0.smithi156.stdout:9/66: rmdir d2/de 39 2024-07-16T09:55:30.190 INFO:tasks.workunit.client.0.smithi156.stdout:9/67: readlink d2/d5/ld 0 2024-07-16T09:55:30.191 INFO:tasks.workunit.client.1.smithi176.stdout:5/230: symlink d4/d14/d19/l57 0 2024-07-16T09:55:30.192 INFO:tasks.workunit.client.1.smithi176.stdout:3/62: mknod d3/da/c11 0 2024-07-16T09:55:30.192 INFO:tasks.workunit.client.1.smithi176.stdout:5/231: dread - d4/d37/f3f zero size 2024-07-16T09:55:30.193 INFO:tasks.workunit.client.0.smithi156.stdout:2/157: dwrite d0/d5/d29/d2d/f31 [0,4194304] 0 2024-07-16T09:55:30.193 INFO:tasks.workunit.client.1.smithi176.stdout:3/63: chown d3/l4 0 1 2024-07-16T09:55:30.194 INFO:tasks.workunit.client.1.smithi176.stdout:7/169: dread d4/f19 [0,4194304] 0 2024-07-16T09:55:30.195 INFO:tasks.workunit.client.1.smithi176.stdout:5/232: truncate d4/d10/d17/f2f 167603 0 2024-07-16T09:55:30.195 INFO:tasks.workunit.client.1.smithi176.stdout:0/200: link d5/d1c/d24/f30 d5/d1c/d24/d26/f3d 0 2024-07-16T09:55:30.196 INFO:tasks.workunit.client.1.smithi176.stdout:9/125: mknod d5/d1d/d20/c23 0 2024-07-16T09:55:30.197 INFO:tasks.workunit.client.1.smithi176.stdout:0/201: truncate d5/d1c/f21 988556 0 2024-07-16T09:55:30.197 INFO:tasks.workunit.client.1.smithi176.stdout:7/170: rmdir d4/dd/d26 39 2024-07-16T09:55:30.197 INFO:tasks.workunit.client.0.smithi156.stdout:1/80: getdents dc 0 2024-07-16T09:55:30.198 INFO:tasks.workunit.client.0.smithi156.stdout:2/158: truncate d0/d13/f1a 248756 0 2024-07-16T09:55:30.199 INFO:tasks.workunit.client.1.smithi176.stdout:5/233: readlink d4/d14/l24 0 2024-07-16T09:55:30.199 INFO:tasks.workunit.client.1.smithi176.stdout:7/171: write d4/f27 [89343,81595] 0 2024-07-16T09:55:30.200 INFO:tasks.workunit.client.0.smithi156.stdout:9/68: dread d2/f13 [0,4194304] 0 2024-07-16T09:55:30.201 INFO:tasks.workunit.client.1.smithi176.stdout:0/202: mknod d5/d1c/d24/d26/c3e 0 2024-07-16T09:55:30.202 INFO:tasks.workunit.client.1.smithi176.stdout:9/126: mknod d5/c24 0 2024-07-16T09:55:30.203 INFO:tasks.workunit.client.0.smithi156.stdout:1/81: symlink dc/d14/d1e/l1f 0 2024-07-16T09:55:30.203 INFO:tasks.workunit.client.1.smithi176.stdout:0/203: write d5/d2c/f2d [570688,130092] 0 2024-07-16T09:55:30.204 INFO:tasks.workunit.client.1.smithi176.stdout:9/127: write d5/f10 [3551913,29098] 0 2024-07-16T09:55:30.204 INFO:tasks.workunit.client.1.smithi176.stdout:7/172: rename l2 to d4/d5/d21/d33/l45 0 2024-07-16T09:55:30.206 INFO:tasks.workunit.client.1.smithi176.stdout:9/128: write d5/de/f1a [1515148,119616] 0 2024-07-16T09:55:30.207 INFO:tasks.workunit.client.1.smithi176.stdout:5/234: mknod d4/c58 0 2024-07-16T09:55:30.207 INFO:tasks.workunit.client.1.smithi176.stdout:7/173: rmdir d4 39 2024-07-16T09:55:30.208 INFO:tasks.workunit.client.1.smithi176.stdout:8/100: truncate d8/f17 45241 0 2024-07-16T09:55:30.208 INFO:tasks.workunit.client.1.smithi176.stdout:0/204: truncate f3 1377939 0 2024-07-16T09:55:30.209 INFO:tasks.workunit.client.0.smithi156.stdout:1/82: mkdir dc/d20 0 2024-07-16T09:55:30.209 INFO:tasks.workunit.client.0.smithi156.stdout:9/69: mknod d2/de/c14 0 2024-07-16T09:55:30.209 INFO:tasks.workunit.client.0.smithi156.stdout:2/159: dread d0/d13/d17/d1c/f27 [0,4194304] 0 2024-07-16T09:55:30.211 INFO:tasks.workunit.client.1.smithi176.stdout:9/129: creat d5/d1d/d20/f25 x:0 0 0 2024-07-16T09:55:30.211 INFO:tasks.workunit.client.0.smithi156.stdout:2/160: chown d0/l10 353059117 1 2024-07-16T09:55:30.212 INFO:tasks.workunit.client.1.smithi176.stdout:7/174: dread - d4/dd/f36 zero size 2024-07-16T09:55:30.212 INFO:tasks.workunit.client.1.smithi176.stdout:6/146: sync 2024-07-16T09:55:30.213 INFO:tasks.workunit.client.1.smithi176.stdout:8/101: stat d8/d23/c16 0 2024-07-16T09:55:30.214 INFO:tasks.workunit.client.1.smithi176.stdout:3/64: read d3/f8 [33099,70862] 0 2024-07-16T09:55:30.214 INFO:tasks.workunit.client.1.smithi176.stdout:8/102: write f6 [274715,2826] 0 2024-07-16T09:55:30.215 INFO:tasks.workunit.client.0.smithi156.stdout:2/161: creat d0/d13/d17/d1c/f37 x:0 0 0 2024-07-16T09:55:30.217 INFO:tasks.workunit.client.0.smithi156.stdout:1/83: link dc/d14/f17 dc/d14/d1e/f21 0 2024-07-16T09:55:30.218 INFO:tasks.workunit.client.1.smithi176.stdout:9/130: symlink d5/d1d/d20/l26 0 2024-07-16T09:55:30.219 INFO:tasks.workunit.client.0.smithi156.stdout:1/84: chown dc/le 36 1 2024-07-16T09:55:30.220 INFO:tasks.workunit.client.1.smithi176.stdout:6/147: unlink d1/l4 0 2024-07-16T09:55:30.221 INFO:tasks.workunit.client.0.smithi156.stdout:1/85: chown dc/c11 0 1 2024-07-16T09:55:30.222 INFO:tasks.workunit.client.1.smithi176.stdout:7/175: rename d4/f14 to d4/d5/d21/d33/f46 0 2024-07-16T09:55:30.223 INFO:tasks.workunit.client.0.smithi156.stdout:1/86: readlink l1 0 2024-07-16T09:55:30.224 INFO:tasks.workunit.client.1.smithi176.stdout:1/107: dwrite d4/f10 [0,4194304] 0 2024-07-16T09:55:30.224 INFO:tasks.workunit.client.1.smithi176.stdout:1/108: read d4/f17 [1382100,37152] 0 2024-07-16T09:55:30.225 INFO:tasks.workunit.client.0.smithi156.stdout:2/162: dwrite d0/fe [4194304,4194304] 0 2024-07-16T09:55:30.226 INFO:tasks.workunit.client.1.smithi176.stdout:3/65: symlink d3/da/l12 0 2024-07-16T09:55:30.226 INFO:tasks.workunit.client.1.smithi176.stdout:3/66: write d3/da/ff [520352,103375] 0 2024-07-16T09:55:30.229 INFO:tasks.workunit.client.0.smithi156.stdout:2/163: fdatasync d0/f16 0 2024-07-16T09:55:30.229 INFO:tasks.workunit.client.1.smithi176.stdout:9/131: rmdir d5/de 39 2024-07-16T09:55:30.231 INFO:tasks.workunit.client.1.smithi176.stdout:5/235: dread d4/fe [4194304,4194304] 0 2024-07-16T09:55:30.233 INFO:tasks.workunit.client.0.smithi156.stdout:2/164: creat d0/d5/d29/f38 x:0 0 0 2024-07-16T09:55:30.234 INFO:tasks.workunit.client.1.smithi176.stdout:3/67: dwrite d3/da/fe [0,4194304] 0 2024-07-16T09:55:30.236 INFO:tasks.workunit.client.1.smithi176.stdout:6/148: truncate d1/d5/f6 1524572 0 2024-07-16T09:55:30.237 INFO:tasks.workunit.client.1.smithi176.stdout:3/68: chown d3/da/ff 181134 1 2024-07-16T09:55:30.237 INFO:tasks.workunit.client.1.smithi176.stdout:5/236: fsync d4/d14/d19/d1b/d36/d38/d39/f56 0 2024-07-16T09:55:30.242 INFO:tasks.workunit.client.1.smithi176.stdout:3/69: write d3/da/ff [956739,41815] 0 2024-07-16T09:55:30.242 INFO:tasks.workunit.client.0.smithi156.stdout:2/165: link d0/d5/d29/l33 d0/d13/d17/d1c/l39 0 2024-07-16T09:55:30.243 INFO:tasks.workunit.client.1.smithi176.stdout:3/70: chown d3/da/fe 23715755 1 2024-07-16T09:55:30.245 INFO:tasks.workunit.client.1.smithi176.stdout:8/103: creat d8/d20/f24 x:0 0 0 2024-07-16T09:55:30.246 INFO:tasks.workunit.client.0.smithi156.stdout:2/166: fdatasync d0/d13/d17/d1c/f27 0 2024-07-16T09:55:30.247 INFO:tasks.workunit.client.1.smithi176.stdout:1/109: rmdir d4 39 2024-07-16T09:55:30.249 INFO:tasks.workunit.client.1.smithi176.stdout:5/237: creat d4/d14/d19/d1b/d36/f59 x:0 0 0 2024-07-16T09:55:30.249 INFO:tasks.workunit.client.0.smithi156.stdout:2/167: symlink d0/l3a 0 2024-07-16T09:55:30.251 INFO:tasks.workunit.client.0.smithi156.stdout:2/168: mknod d0/d13/d17/c3b 0 2024-07-16T09:55:30.252 INFO:tasks.workunit.client.1.smithi176.stdout:3/71: dwrite d3/da/fe [0,4194304] 0 2024-07-16T09:55:30.253 INFO:tasks.workunit.client.1.smithi176.stdout:8/104: creat d8/d23/f25 x:0 0 0 2024-07-16T09:55:30.254 INFO:tasks.workunit.client.1.smithi176.stdout:1/110: write d4/f10 [734724,101041] 0 2024-07-16T09:55:30.254 INFO:tasks.workunit.client.1.smithi176.stdout:0/205: link d5/f10 d5/d2c/f3f 0 2024-07-16T09:55:30.258 INFO:tasks.workunit.client.1.smithi176.stdout:6/149: creat d1/d5/d7/d2f/f32 x:0 0 0 2024-07-16T09:55:30.259 INFO:tasks.workunit.client.1.smithi176.stdout:4/173: sync 2024-07-16T09:55:30.259 INFO:tasks.workunit.client.1.smithi176.stdout:2/156: sync 2024-07-16T09:55:30.260 INFO:tasks.workunit.client.1.smithi176.stdout:8/105: dread - d8/d23/f1b zero size 2024-07-16T09:55:30.263 INFO:tasks.workunit.client.1.smithi176.stdout:7/176: write d4/d5/f31 [703077,7672] 0 2024-07-16T09:55:30.264 INFO:tasks.workunit.client.1.smithi176.stdout:9/132: dwrite d5/de/f1a [0,4194304] 0 2024-07-16T09:55:30.265 INFO:tasks.workunit.client.1.smithi176.stdout:1/111: creat d4/db/f23 x:0 0 0 2024-07-16T09:55:30.266 INFO:tasks.workunit.client.1.smithi176.stdout:5/238: write d4/d10/f12 [4400326,103592] 0 2024-07-16T09:55:30.267 INFO:tasks.workunit.client.1.smithi176.stdout:7/177: chown d4/d5/d21 228998 1 2024-07-16T09:55:30.269 INFO:tasks.workunit.client.1.smithi176.stdout:4/174: readlink l9 0 2024-07-16T09:55:30.270 INFO:tasks.workunit.client.0.smithi156.stdout:5/73: getdents . 0 2024-07-16T09:55:30.271 INFO:tasks.workunit.client.1.smithi176.stdout:7/178: chown c3 287 1 2024-07-16T09:55:30.271 INFO:tasks.workunit.client.1.smithi176.stdout:9/133: fdatasync d5/de/f12 0 2024-07-16T09:55:30.273 INFO:tasks.workunit.client.0.smithi156.stdout:5/74: fdatasync fc 0 2024-07-16T09:55:30.273 INFO:tasks.workunit.client.0.smithi156.stdout:6/105: sync 2024-07-16T09:55:30.273 INFO:tasks.workunit.client.0.smithi156.stdout:3/36: sync 2024-07-16T09:55:30.277 INFO:tasks.workunit.client.1.smithi176.stdout:5/239: dread d4/d10/f12 [4194304,4194304] 0 2024-07-16T09:55:30.278 INFO:tasks.workunit.client.0.smithi156.stdout:4/96: getdents d4/db/dc/d22 0 2024-07-16T09:55:30.279 INFO:tasks.workunit.client.1.smithi176.stdout:8/106: dwrite f6 [0,4194304] 0 2024-07-16T09:55:30.279 INFO:tasks.workunit.client.1.smithi176.stdout:0/206: dread d5/f6 [0,4194304] 0 2024-07-16T09:55:30.280 INFO:tasks.workunit.client.0.smithi156.stdout:6/106: write d3/d4/d1f/f10 [523459,54761] 0 2024-07-16T09:55:30.280 INFO:tasks.workunit.client.1.smithi176.stdout:9/134: creat d5/d1d/f27 x:0 0 0 2024-07-16T09:55:30.281 INFO:tasks.workunit.client.0.smithi156.stdout:4/97: write d4/f8 [3734982,13402] 0 2024-07-16T09:55:30.281 INFO:tasks.workunit.client.0.smithi156.stdout:5/75: getdents . 0 2024-07-16T09:55:30.282 INFO:tasks.workunit.client.0.smithi156.stdout:3/37: mkdir d8/da 0 2024-07-16T09:55:30.283 INFO:tasks.workunit.client.0.smithi156.stdout:9/70: truncate d2/d5/fb 1255791 0 2024-07-16T09:55:30.283 INFO:tasks.workunit.client.1.smithi176.stdout:2/157: dread dd/d16/f27 [0,4194304] 0 2024-07-16T09:55:30.284 INFO:tasks.workunit.client.1.smithi176.stdout:3/72: truncate d3/da/fe 3085521 0 2024-07-16T09:55:30.284 INFO:tasks.workunit.client.1.smithi176.stdout:9/135: stat d5/de/c11 0 2024-07-16T09:55:30.284 INFO:tasks.workunit.client.0.smithi156.stdout:4/98: chown d4/db/f16 30 1 2024-07-16T09:55:30.285 INFO:tasks.workunit.client.0.smithi156.stdout:6/107: write d3/d4/d1f/fa [4361140,48319] 0 2024-07-16T09:55:30.285 INFO:tasks.workunit.client.0.smithi156.stdout:8/39: fsync d0/fb 0 2024-07-16T09:55:30.286 INFO:tasks.workunit.client.1.smithi176.stdout:1/112: dwrite d4/f10 [0,4194304] 0 2024-07-16T09:55:30.287 INFO:tasks.workunit.client.1.smithi176.stdout:2/158: write dd/f2a [1896917,68518] 0 2024-07-16T09:55:30.287 INFO:tasks.workunit.client.1.smithi176.stdout:4/175: dread da/f1a [0,4194304] 0 2024-07-16T09:55:30.288 INFO:tasks.workunit.client.0.smithi156.stdout:5/76: symlink l16 0 2024-07-16T09:55:30.289 INFO:tasks.workunit.client.1.smithi176.stdout:8/107: mkdir d8/d20/d26 0 2024-07-16T09:55:30.290 INFO:tasks.workunit.client.0.smithi156.stdout:6/108: chown d3/d4/d1e 919 1 2024-07-16T09:55:30.290 INFO:tasks.workunit.client.0.smithi156.stdout:9/71: rmdir d2/de 39 2024-07-16T09:55:30.290 INFO:tasks.workunit.client.0.smithi156.stdout:7/53: dwrite dc/d13/f12 [0,4194304] 0 2024-07-16T09:55:30.291 INFO:tasks.workunit.client.0.smithi156.stdout:4/99: symlink d4/d23/l27 0 2024-07-16T09:55:30.291 INFO:tasks.workunit.client.0.smithi156.stdout:5/77: truncate f14 4504231 0 2024-07-16T09:55:30.291 INFO:tasks.workunit.client.0.smithi156.stdout:3/38: rename f5 to d8/fb 0 2024-07-16T09:55:30.292 INFO:tasks.workunit.client.0.smithi156.stdout:6/109: read d3/f1c [715840,64481] 0 2024-07-16T09:55:30.293 INFO:tasks.workunit.client.1.smithi176.stdout:4/176: write da/d14/f29 [310378,56172] 0 2024-07-16T09:55:30.293 INFO:tasks.workunit.client.1.smithi176.stdout:7/179: write d4/d5/d21/f3b [52542,64424] 0 2024-07-16T09:55:30.293 INFO:tasks.workunit.client.1.smithi176.stdout:9/136: fsync f2 0 2024-07-16T09:55:30.293 INFO:tasks.workunit.client.1.smithi176.stdout:8/108: truncate d8/d20/f24 854997 0 2024-07-16T09:55:30.294 INFO:tasks.workunit.client.1.smithi176.stdout:8/109: fdatasync d8/d23/f15 0 2024-07-16T09:55:30.295 INFO:tasks.workunit.client.0.smithi156.stdout:8/40: truncate d0/f9 1325757 0 2024-07-16T09:55:30.295 INFO:tasks.workunit.client.0.smithi156.stdout:9/72: dread d2/f13 [0,4194304] 0 2024-07-16T09:55:30.295 INFO:tasks.workunit.client.0.smithi156.stdout:2/169: truncate d0/f14 1225115 0 2024-07-16T09:55:30.295 INFO:tasks.workunit.client.0.smithi156.stdout:6/110: truncate d3/d4/f22 511521 0 2024-07-16T09:55:30.296 INFO:tasks.workunit.client.1.smithi176.stdout:0/207: symlink d5/d1c/d24/l40 0 2024-07-16T09:55:30.297 INFO:tasks.workunit.client.0.smithi156.stdout:1/87: getdents dc 0 2024-07-16T09:55:30.297 INFO:tasks.workunit.client.1.smithi176.stdout:0/208: readlink l2 0 2024-07-16T09:55:30.298 INFO:tasks.workunit.client.0.smithi156.stdout:7/54: mkdir dc/d13/d15 0 2024-07-16T09:55:30.299 INFO:tasks.workunit.client.0.smithi156.stdout:5/78: symlink l17 0 2024-07-16T09:55:30.300 INFO:tasks.workunit.client.0.smithi156.stdout:3/39: dread d8/fb [0,4194304] 0 2024-07-16T09:55:30.303 INFO:tasks.workunit.client.0.smithi156.stdout:5/79: chown l16 106 1 2024-07-16T09:55:30.303 INFO:tasks.workunit.client.1.smithi176.stdout:9/137: mknod d5/de/c28 0 2024-07-16T09:55:30.304 INFO:tasks.workunit.client.1.smithi176.stdout:9/138: truncate d5/f1c 202635 0 2024-07-16T09:55:30.305 INFO:tasks.workunit.client.1.smithi176.stdout:4/177: symlink da/d14/l2f 0 2024-07-16T09:55:30.306 INFO:tasks.workunit.client.1.smithi176.stdout:8/110: unlink d8/c12 0 2024-07-16T09:55:30.307 INFO:tasks.workunit.client.0.smithi156.stdout:8/41: creat d0/fc x:0 0 0 2024-07-16T09:55:30.308 INFO:tasks.workunit.client.0.smithi156.stdout:5/80: write f13 [2406266,92163] 0 2024-07-16T09:55:30.308 INFO:tasks.workunit.client.1.smithi176.stdout:3/73: dread d3/da/fe [0,4194304] 0 2024-07-16T09:55:30.309 INFO:tasks.workunit.client.1.smithi176.stdout:4/178: write f0 [1381789,14018] 0 2024-07-16T09:55:30.310 INFO:tasks.workunit.client.0.smithi156.stdout:6/111: symlink d3/d13/l23 0 2024-07-16T09:55:30.310 INFO:tasks.workunit.client.1.smithi176.stdout:5/240: link d4/d14/f45 d4/d37/f5a 0 2024-07-16T09:55:30.310 INFO:tasks.workunit.client.1.smithi176.stdout:1/113: write d4/f17 [3776571,76253] 0 2024-07-16T09:55:30.311 INFO:tasks.workunit.client.1.smithi176.stdout:2/159: creat dd/d21/f2f x:0 0 0 2024-07-16T09:55:30.311 INFO:tasks.workunit.client.0.smithi156.stdout:8/42: chown d0/f6 38 1 2024-07-16T09:55:30.312 INFO:tasks.workunit.client.1.smithi176.stdout:0/209: mkdir d5/d1c/d24/d35/d41 0 2024-07-16T09:55:30.313 INFO:tasks.workunit.client.0.smithi156.stdout:2/170: dwrite d0/d5/f2f [0,4194304] 0 2024-07-16T09:55:30.314 INFO:tasks.workunit.client.1.smithi176.stdout:3/74: dread d3/f8 [0,4194304] 0 2024-07-16T09:55:30.315 INFO:tasks.workunit.client.0.smithi156.stdout:8/43: readlink d0/la 0 2024-07-16T09:55:30.316 INFO:tasks.workunit.client.0.smithi156.stdout:6/112: write d3/d4/d1f/f10 [2946035,74048] 0 2024-07-16T09:55:30.316 INFO:tasks.workunit.client.1.smithi176.stdout:4/179: read - da/dc/d10/d15/d17/f2a zero size 2024-07-16T09:55:30.316 INFO:tasks.workunit.client.1.smithi176.stdout:8/111: creat d8/d14/f27 x:0 0 0 2024-07-16T09:55:30.318 INFO:tasks.workunit.client.1.smithi176.stdout:2/160: mknod dd/d21/c30 0 2024-07-16T09:55:30.318 INFO:tasks.workunit.client.1.smithi176.stdout:6/150: dwrite d1/f18 [0,4194304] 0 2024-07-16T09:55:30.318 INFO:tasks.workunit.client.1.smithi176.stdout:1/114: dread d4/ff [0,4194304] 0 2024-07-16T09:55:30.318 INFO:tasks.workunit.client.0.smithi156.stdout:1/88: dread f5 [0,4194304] 0 2024-07-16T09:55:30.319 INFO:tasks.workunit.client.0.smithi156.stdout:8/44: readlink d0/la 0 2024-07-16T09:55:30.319 INFO:tasks.workunit.client.0.smithi156.stdout:4/100: dwrite d4/d23/f24 [0,4194304] 0 2024-07-16T09:55:30.320 INFO:tasks.workunit.client.0.smithi156.stdout:1/89: fdatasync dc/d14/f1d 0 2024-07-16T09:55:30.320 INFO:tasks.workunit.client.1.smithi176.stdout:5/241: creat d4/d14/d19/d1b/d36/f5b x:0 0 0 2024-07-16T09:55:30.321 INFO:tasks.workunit.client.1.smithi176.stdout:8/112: read - d8/d14/f27 zero size 2024-07-16T09:55:30.322 INFO:tasks.workunit.client.0.smithi156.stdout:5/81: mknod c18 0 2024-07-16T09:55:30.323 INFO:tasks.workunit.client.0.smithi156.stdout:3/40: getdents d8/da 0 2024-07-16T09:55:30.324 INFO:tasks.workunit.client.0.smithi156.stdout:2/171: write d0/f4 [2513192,18014] 0 2024-07-16T09:55:30.324 INFO:tasks.workunit.client.1.smithi176.stdout:8/113: write d8/d14/f27 [950893,116095] 0 2024-07-16T09:55:30.325 INFO:tasks.workunit.client.0.smithi156.stdout:6/113: symlink d3/l24 0 2024-07-16T09:55:30.326 INFO:tasks.workunit.client.1.smithi176.stdout:8/114: write d8/d20/f24 [1607110,64424] 0 2024-07-16T09:55:30.327 INFO:tasks.workunit.client.1.smithi176.stdout:8/115: fsync f5 0 2024-07-16T09:55:30.329 INFO:tasks.workunit.client.0.smithi156.stdout:3/41: read f3 [2058650,95887] 0 2024-07-16T09:55:30.331 INFO:tasks.workunit.client.1.smithi176.stdout:3/75: rename d3/f8 to d3/da/f13 0 2024-07-16T09:55:30.331 INFO:tasks.workunit.client.0.smithi156.stdout:6/114: dread d3/d4/d1f/fe [0,4194304] 0 2024-07-16T09:55:30.331 INFO:tasks.workunit.client.0.smithi156.stdout:3/42: chown d8/fb 1353 1 2024-07-16T09:55:30.331 INFO:tasks.workunit.client.1.smithi176.stdout:4/180: truncate da/fb 4065958 0 2024-07-16T09:55:30.332 INFO:tasks.workunit.client.0.smithi156.stdout:4/101: symlink d4/d23/l28 0 2024-07-16T09:55:30.332 INFO:tasks.workunit.client.1.smithi176.stdout:1/115: unlink d4/d8/c1a 0 2024-07-16T09:55:30.332 INFO:tasks.workunit.client.0.smithi156.stdout:4/102: stat d4/d23/l26 0 2024-07-16T09:55:30.333 INFO:tasks.workunit.client.0.smithi156.stdout:8/45: dwrite d0/fb [0,4194304] 0 2024-07-16T09:55:30.334 INFO:tasks.workunit.client.1.smithi176.stdout:6/151: creat d1/dc/f33 x:0 0 0 2024-07-16T09:55:30.334 INFO:tasks.workunit.client.1.smithi176.stdout:4/181: readlink da/d14/l2f 0 2024-07-16T09:55:30.335 INFO:tasks.workunit.client.1.smithi176.stdout:0/210: symlink d5/d1c/d24/d35/d41/l42 0 2024-07-16T09:55:30.336 INFO:tasks.workunit.client.0.smithi156.stdout:4/103: write d4/d23/f17 [478530,5827] 0 2024-07-16T09:55:30.336 INFO:tasks.workunit.client.0.smithi156.stdout:5/82: creat f19 x:0 0 0 2024-07-16T09:55:30.336 INFO:tasks.workunit.client.1.smithi176.stdout:2/161: symlink dd/l31 0 2024-07-16T09:55:30.337 INFO:tasks.workunit.client.0.smithi156.stdout:5/83: write f19 [891546,105898] 0 2024-07-16T09:55:30.338 INFO:tasks.workunit.client.0.smithi156.stdout:2/172: dread d0/d5/f2a [0,4194304] 0 2024-07-16T09:55:30.339 INFO:tasks.workunit.client.1.smithi176.stdout:9/139: getdents d5 0 2024-07-16T09:55:30.339 INFO:tasks.workunit.client.0.smithi156.stdout:8/46: truncate d0/f6 2266442 0 2024-07-16T09:55:30.341 INFO:tasks.workunit.client.0.smithi156.stdout:6/115: creat d3/f25 x:0 0 0 2024-07-16T09:55:30.341 INFO:tasks.workunit.client.0.smithi156.stdout:2/173: chown d0/f2c 1308 1 2024-07-16T09:55:30.344 INFO:tasks.workunit.client.0.smithi156.stdout:0/36: sync 2024-07-16T09:55:30.344 INFO:tasks.workunit.client.0.smithi156.stdout:1/90: creat dc/d20/f22 x:0 0 0 2024-07-16T09:55:30.345 INFO:tasks.workunit.client.1.smithi176.stdout:4/182: chown da/dc/d10/d15/d17/l21 150903 1 2024-07-16T09:55:30.345 INFO:tasks.workunit.client.1.smithi176.stdout:3/76: rename d3/da/c11 to d3/da/c14 0 2024-07-16T09:55:30.346 INFO:tasks.workunit.client.1.smithi176.stdout:9/140: write d5/d1d/d20/f25 [35351,120115] 0 2024-07-16T09:55:30.347 INFO:tasks.workunit.client.1.smithi176.stdout:8/116: dwrite f6 [0,4194304] 0 2024-07-16T09:55:30.347 INFO:tasks.workunit.client.0.smithi156.stdout:7/55: truncate dc/d13/f12 2477381 0 2024-07-16T09:55:30.349 INFO:tasks.workunit.client.1.smithi176.stdout:6/152: symlink d1/d5/d7/d1e/l34 0 2024-07-16T09:55:30.351 INFO:tasks.workunit.client.1.smithi176.stdout:9/141: write d5/f1c [303779,67938] 0 2024-07-16T09:55:30.352 INFO:tasks.workunit.client.0.smithi156.stdout:3/43: creat d8/da/fc x:0 0 0 2024-07-16T09:55:30.353 INFO:tasks.workunit.client.0.smithi156.stdout:0/37: dwrite d0/f3 [0,4194304] 0 2024-07-16T09:55:30.353 INFO:tasks.workunit.client.1.smithi176.stdout:5/242: link d4/d14/d19/d1b/l43 d4/d10/l5c 0 2024-07-16T09:55:30.354 INFO:tasks.workunit.client.1.smithi176.stdout:1/116: dwrite f1 [4194304,4194304] 0 2024-07-16T09:55:30.356 INFO:tasks.workunit.client.0.smithi156.stdout:8/47: mkdir d0/dd 0 2024-07-16T09:55:30.356 INFO:tasks.workunit.client.1.smithi176.stdout:4/183: mknod da/dc/d10/c30 0 2024-07-16T09:55:30.359 INFO:tasks.workunit.client.0.smithi156.stdout:2/174: creat d0/d5/d29/f3c x:0 0 0 2024-07-16T09:55:30.359 INFO:tasks.workunit.client.0.smithi156.stdout:6/116: rename d3/d4 to d3/d26 0 2024-07-16T09:55:30.360 INFO:tasks.workunit.client.0.smithi156.stdout:7/56: symlink dc/l16 0 2024-07-16T09:55:30.360 INFO:tasks.workunit.client.0.smithi156.stdout:4/104: link d4/f9 d4/db/dc/f29 0 2024-07-16T09:55:30.360 INFO:tasks.workunit.client.0.smithi156.stdout:3/44: fsync d8/fb 0 2024-07-16T09:55:30.361 INFO:tasks.workunit.client.0.smithi156.stdout:3/45: write d8/fb [367930,125162] 0 2024-07-16T09:55:30.362 INFO:tasks.workunit.client.1.smithi176.stdout:3/77: dread d3/da/f13 [0,4194304] 0 2024-07-16T09:55:30.369 INFO:tasks.workunit.client.0.smithi156.stdout:0/38: creat d0/d7/f8 x:0 0 0 2024-07-16T09:55:30.369 INFO:tasks.workunit.client.1.smithi176.stdout:8/117: symlink d8/l28 0 2024-07-16T09:55:30.370 INFO:tasks.workunit.client.0.smithi156.stdout:4/105: creat d4/db/dc/f2a x:0 0 0 2024-07-16T09:55:30.371 INFO:tasks.workunit.client.1.smithi176.stdout:2/162: dwrite dd/d21/f2f [0,4194304] 0 2024-07-16T09:55:30.371 INFO:tasks.workunit.client.1.smithi176.stdout:6/153: dread d1/d5/f6 [0,4194304] 0 2024-07-16T09:55:30.371 INFO:tasks.workunit.client.1.smithi176.stdout:9/142: rmdir d5 39 2024-07-16T09:55:30.373 INFO:tasks.workunit.client.1.smithi176.stdout:5/243: creat d4/d14/d19/d1b/d36/d38/d39/f5d x:0 0 0 2024-07-16T09:55:30.374 INFO:tasks.workunit.client.1.smithi176.stdout:1/117: dwrite d4/d8/f1e [0,4194304] 0 2024-07-16T09:55:30.374 INFO:tasks.workunit.client.1.smithi176.stdout:3/78: rmdir d3 39 2024-07-16T09:55:30.375 INFO:tasks.workunit.client.0.smithi156.stdout:3/46: fdatasync d8/fb 0 2024-07-16T09:55:30.375 INFO:tasks.workunit.client.0.smithi156.stdout:9/73: truncate d2/d5/fb 516114 0 2024-07-16T09:55:30.378 INFO:tasks.workunit.client.1.smithi176.stdout:0/211: truncate d5/ff 1217283 0 2024-07-16T09:55:30.379 INFO:tasks.workunit.client.0.smithi156.stdout:8/48: link d0/c7 d0/ce 0 2024-07-16T09:55:30.381 INFO:tasks.workunit.client.1.smithi176.stdout:0/212: dread - d5/d1c/d24/f30 zero size 2024-07-16T09:55:30.385 INFO:tasks.workunit.client.1.smithi176.stdout:6/154: creat d1/dc/d1c/f35 x:0 0 0 2024-07-16T09:55:30.385 INFO:tasks.workunit.client.0.smithi156.stdout:3/47: mknod d8/da/cd 0 2024-07-16T09:55:30.386 INFO:tasks.workunit.client.1.smithi176.stdout:2/163: creat dd/d16/f32 x:0 0 0 2024-07-16T09:55:30.386 INFO:tasks.workunit.client.1.smithi176.stdout:5/244: symlink d4/d14/d19/l5e 0 2024-07-16T09:55:30.386 INFO:tasks.workunit.client.1.smithi176.stdout:4/184: dwrite da/dc/d10/f27 [0,4194304] 0 2024-07-16T09:55:30.387 INFO:tasks.workunit.client.1.smithi176.stdout:8/118: fdatasync d8/f17 0 2024-07-16T09:55:30.387 INFO:tasks.workunit.client.1.smithi176.stdout:3/79: mknod d3/c15 0 2024-07-16T09:55:30.388 INFO:tasks.workunit.client.1.smithi176.stdout:9/143: dread d5/fd [0,4194304] 0 2024-07-16T09:55:30.388 INFO:tasks.workunit.client.0.smithi156.stdout:0/39: dwrite d0/d7/f8 [0,4194304] 0 2024-07-16T09:55:30.389 INFO:tasks.workunit.client.1.smithi176.stdout:8/119: fdatasync d8/d14/d1e/f1f 0 2024-07-16T09:55:30.389 INFO:tasks.workunit.client.1.smithi176.stdout:4/185: chown da/d14/c2d 17 1 2024-07-16T09:55:30.389 INFO:tasks.workunit.client.1.smithi176.stdout:2/164: write dd/d16/f1c [124949,83828] 0 2024-07-16T09:55:30.390 INFO:tasks.workunit.client.1.smithi176.stdout:6/155: rmdir d1/d5/d7/d2f 39 2024-07-16T09:55:30.391 INFO:tasks.workunit.client.0.smithi156.stdout:4/106: dwrite d4/db/dc/f20 [0,4194304] 0 2024-07-16T09:55:30.392 INFO:tasks.workunit.client.1.smithi176.stdout:1/118: creat d4/f24 x:0 0 0 2024-07-16T09:55:30.392 INFO:tasks.workunit.client.1.smithi176.stdout:6/156: read d1/d5/d7/fe [728715,80606] 0 2024-07-16T09:55:30.392 INFO:tasks.workunit.client.1.smithi176.stdout:5/245: truncate d4/d14/d3d/f55 457296 0 2024-07-16T09:55:30.393 INFO:tasks.workunit.client.1.smithi176.stdout:3/80: mknod d3/da/c16 0 2024-07-16T09:55:30.396 INFO:tasks.workunit.client.0.smithi156.stdout:9/74: creat d2/f15 x:0 0 0 2024-07-16T09:55:30.396 INFO:tasks.workunit.client.0.smithi156.stdout:3/48: fsync d8/da/fc 0 2024-07-16T09:55:30.401 INFO:tasks.workunit.client.0.smithi156.stdout:4/107: symlink d4/d23/l2b 0 2024-07-16T09:55:30.402 INFO:tasks.workunit.client.0.smithi156.stdout:5/84: dwrite f12 [0,4194304] 0 2024-07-16T09:55:30.402 INFO:tasks.workunit.client.0.smithi156.stdout:3/49: rename l4 to d8/da/le 0 2024-07-16T09:55:30.406 INFO:tasks.workunit.client.0.smithi156.stdout:8/49: dread d0/f1 [0,4194304] 0 2024-07-16T09:55:30.406 INFO:tasks.workunit.client.0.smithi156.stdout:0/40: mkdir d0/d7/d9 0 2024-07-16T09:55:30.409 INFO:tasks.workunit.client.0.smithi156.stdout:8/50: truncate d0/f9 3020339 0 2024-07-16T09:55:30.409 INFO:tasks.workunit.client.0.smithi156.stdout:4/108: dread f2 [0,4194304] 0 2024-07-16T09:55:30.410 INFO:tasks.workunit.client.0.smithi156.stdout:5/85: creat f1a x:0 0 0 2024-07-16T09:55:30.411 INFO:tasks.workunit.client.0.smithi156.stdout:5/86: write fc [4947521,109909] 0 2024-07-16T09:55:30.413 INFO:tasks.workunit.client.0.smithi156.stdout:5/87: stat c8 0 2024-07-16T09:55:30.413 INFO:tasks.workunit.client.0.smithi156.stdout:5/88: rmdir - no directory 2024-07-16T09:55:30.414 INFO:tasks.workunit.client.0.smithi156.stdout:3/50: write f0 [5542983,105614] 0 2024-07-16T09:55:30.416 INFO:tasks.workunit.client.0.smithi156.stdout:4/109: unlink d4/d23/l2b 0 2024-07-16T09:55:30.418 INFO:tasks.workunit.client.0.smithi156.stdout:9/75: dread d2/d5/fb [0,4194304] 0 2024-07-16T09:55:30.419 INFO:tasks.workunit.client.0.smithi156.stdout:8/51: link d0/f9 d0/ff 0 2024-07-16T09:55:30.420 INFO:tasks.workunit.client.0.smithi156.stdout:5/89: mkdir d1b 0 2024-07-16T09:55:30.420 INFO:tasks.workunit.client.0.smithi156.stdout:4/110: write d4/db/f1d [700984,14503] 0 2024-07-16T09:55:30.421 INFO:tasks.workunit.client.0.smithi156.stdout:8/52: readlink d0/la 0 2024-07-16T09:55:30.421 INFO:tasks.workunit.client.0.smithi156.stdout:3/51: mknod d8/cf 0 2024-07-16T09:55:30.424 INFO:tasks.workunit.client.0.smithi156.stdout:0/41: getdents d0/d7/d9 0 2024-07-16T09:55:30.425 INFO:tasks.workunit.client.0.smithi156.stdout:8/53: read d0/f6 [329484,6989] 0 2024-07-16T09:55:30.427 INFO:tasks.workunit.client.0.smithi156.stdout:9/76: mkdir d2/d5/d16 0 2024-07-16T09:55:30.429 INFO:tasks.workunit.client.0.smithi156.stdout:3/52: readlink d8/da/le 0 2024-07-16T09:55:30.430 INFO:tasks.workunit.client.0.smithi156.stdout:0/42: chown d0/c2 3578 1 2024-07-16T09:55:30.431 INFO:tasks.workunit.client.0.smithi156.stdout:6/117: dwrite d3/f1c [0,4194304] 0 2024-07-16T09:55:30.434 INFO:tasks.workunit.client.0.smithi156.stdout:7/57: dwrite fb [0,4194304] 0 2024-07-16T09:55:30.435 INFO:tasks.workunit.client.0.smithi156.stdout:2/175: dwrite d0/d13/d17/d1c/f27 [0,4194304] 0 2024-07-16T09:55:30.442 INFO:tasks.workunit.client.0.smithi156.stdout:0/43: unlink d0/f5 0 2024-07-16T09:55:30.445 INFO:tasks.workunit.client.0.smithi156.stdout:9/77: write d2/f3 [4707264,47506] 0 2024-07-16T09:55:30.446 INFO:tasks.workunit.client.0.smithi156.stdout:7/58: symlink dc/d13/l17 0 2024-07-16T09:55:30.448 INFO:tasks.workunit.client.0.smithi156.stdout:8/54: link d0/f1 d0/dd/f10 0 2024-07-16T09:55:30.449 INFO:tasks.workunit.client.0.smithi156.stdout:5/90: dwrite ff [0,4194304] 0 2024-07-16T09:55:30.450 INFO:tasks.workunit.client.0.smithi156.stdout:0/44: creat d0/fa x:0 0 0 2024-07-16T09:55:30.450 INFO:tasks.workunit.client.0.smithi156.stdout:2/176: symlink d0/l3d 0 2024-07-16T09:55:30.453 INFO:tasks.workunit.client.0.smithi156.stdout:7/59: mkdir dc/d18 0 2024-07-16T09:55:30.455 INFO:tasks.workunit.client.0.smithi156.stdout:2/177: readlink d0/l15 0 2024-07-16T09:55:30.456 INFO:tasks.workunit.client.0.smithi156.stdout:9/78: mknod d2/d5/d16/c17 0 2024-07-16T09:55:30.458 INFO:tasks.workunit.client.0.smithi156.stdout:0/45: mknod d0/cb 0 2024-07-16T09:55:30.460 INFO:tasks.workunit.client.0.smithi156.stdout:4/111: dwrite d4/f9 [0,4194304] 0 2024-07-16T09:55:30.465 INFO:tasks.workunit.client.0.smithi156.stdout:3/53: dwrite f0 [0,4194304] 0 2024-07-16T09:55:30.468 INFO:tasks.workunit.client.0.smithi156.stdout:6/118: dread d3/d26/d1f/f10 [0,4194304] 0 2024-07-16T09:55:30.468 INFO:tasks.workunit.client.0.smithi156.stdout:2/178: chown d0/d5/d29/d2d/f31 6575318 1 2024-07-16T09:55:30.472 INFO:tasks.workunit.client.0.smithi156.stdout:9/79: truncate d2/f4 1240083 0 2024-07-16T09:55:30.472 INFO:tasks.workunit.client.0.smithi156.stdout:4/112: rename d4/db/dc/f20 to d4/db/f2c 0 2024-07-16T09:55:30.474 INFO:tasks.workunit.client.0.smithi156.stdout:3/54: chown d8/da/cd 345178 1 2024-07-16T09:55:30.475 INFO:tasks.workunit.client.1.smithi176.stdout:2/165: creat dd/d16/f33 x:0 0 0 2024-07-16T09:55:30.475 INFO:tasks.workunit.client.1.smithi176.stdout:5/246: creat d4/d10/f5f x:0 0 0 2024-07-16T09:55:30.476 INFO:tasks.workunit.client.1.smithi176.stdout:1/119: mkdir d4/d8/d12/d21/d25 0 2024-07-16T09:55:30.476 INFO:tasks.workunit.client.0.smithi156.stdout:5/91: dread f14 [0,4194304] 0 2024-07-16T09:55:30.476 INFO:tasks.workunit.client.0.smithi156.stdout:8/55: dwrite d0/f8 [0,4194304] 0 2024-07-16T09:55:30.477 INFO:tasks.workunit.client.1.smithi176.stdout:6/157: creat d1/d5/d7/d1e/f36 x:0 0 0 2024-07-16T09:55:30.478 INFO:tasks.workunit.client.0.smithi156.stdout:9/80: chown d2/la 24 1 2024-07-16T09:55:30.479 INFO:tasks.workunit.client.1.smithi176.stdout:0/213: rename d5/ff to d5/d39/f43 0 2024-07-16T09:55:30.479 INFO:tasks.workunit.client.1.smithi176.stdout:9/144: dwrite d5/de/f1b [4194304,4194304] 0 2024-07-16T09:55:30.479 INFO:tasks.workunit.client.1.smithi176.stdout:6/158: chown d1/d5/l11 74270 1 2024-07-16T09:55:30.482 INFO:tasks.workunit.client.0.smithi156.stdout:4/113: write d4/db/f16 [1472358,44965] 0 2024-07-16T09:55:30.483 INFO:tasks.workunit.client.0.smithi156.stdout:6/119: rename d3/d13/l1a to d3/d26/d1e/l27 0 2024-07-16T09:55:30.483 INFO:tasks.workunit.client.0.smithi156.stdout:7/60: dwrite dc/d13/f12 [0,4194304] 0 2024-07-16T09:55:30.483 INFO:tasks.workunit.client.1.smithi176.stdout:5/247: fsync d4/d37/f5a 0 2024-07-16T09:55:30.484 INFO:tasks.workunit.client.0.smithi156.stdout:8/56: fdatasync d0/dd/f10 0 2024-07-16T09:55:30.484 INFO:tasks.workunit.client.1.smithi176.stdout:0/214: mkdir d5/d44 0 2024-07-16T09:55:30.487 INFO:tasks.workunit.client.0.smithi156.stdout:4/114: dread - d4/db/dc/d22/f25 zero size 2024-07-16T09:55:30.488 INFO:tasks.workunit.client.1.smithi176.stdout:4/186: rename f0 to da/dc/d10/d15/d17/f31 0 2024-07-16T09:55:30.489 INFO:tasks.workunit.client.0.smithi156.stdout:3/55: unlink d8/l9 0 2024-07-16T09:55:30.489 INFO:tasks.workunit.client.0.smithi156.stdout:6/120: mknod d3/d13/d17/c28 0 2024-07-16T09:55:30.490 INFO:tasks.workunit.client.0.smithi156.stdout:9/81: unlink d2/de/c14 0 2024-07-16T09:55:30.491 INFO:tasks.workunit.client.1.smithi176.stdout:3/81: dwrite d3/da/fe [0,4194304] 0 2024-07-16T09:55:30.491 INFO:tasks.workunit.client.0.smithi156.stdout:6/121: stat d3/d13/d17/l1b 0 2024-07-16T09:55:30.492 INFO:tasks.workunit.client.0.smithi156.stdout:8/57: mkdir d0/d11 0 2024-07-16T09:55:30.493 INFO:tasks.workunit.client.1.smithi176.stdout:2/166: mknod dd/de/c34 0 2024-07-16T09:55:30.495 INFO:tasks.workunit.client.1.smithi176.stdout:0/215: symlink d5/l45 0 2024-07-16T09:55:30.496 INFO:tasks.workunit.client.0.smithi156.stdout:4/115: chown d4/db/dc/l13 85187 1 2024-07-16T09:55:30.497 INFO:tasks.workunit.client.1.smithi176.stdout:0/216: chown d5/f7 154 1 2024-07-16T09:55:30.497 INFO:tasks.workunit.client.0.smithi156.stdout:5/92: dread f14 [0,4194304] 0 2024-07-16T09:55:30.497 INFO:tasks.workunit.client.0.smithi156.stdout:2/179: dwrite d0/d13/f26 [0,4194304] 0 2024-07-16T09:55:30.497 INFO:tasks.workunit.client.0.smithi156.stdout:3/56: creat d8/f10 x:0 0 0 2024-07-16T09:55:30.498 INFO:tasks.workunit.client.1.smithi176.stdout:6/159: symlink d1/d5/d7/d2f/l37 0 2024-07-16T09:55:30.501 INFO:tasks.workunit.client.0.smithi156.stdout:8/58: creat d0/f12 x:0 0 0 2024-07-16T09:55:30.502 INFO:tasks.workunit.client.1.smithi176.stdout:9/145: mknod d5/de/d21/c29 0 2024-07-16T09:55:30.502 INFO:tasks.workunit.client.1.smithi176.stdout:5/248: rename d4/l2d to d4/d14/d19/d1b/d36/d38/l60 0 2024-07-16T09:55:30.502 INFO:tasks.workunit.client.1.smithi176.stdout:4/187: fdatasync da/dc/d10/d15/d17/f31 0 2024-07-16T09:55:30.504 INFO:tasks.workunit.client.1.smithi176.stdout:3/82: dread d3/da/ff [0,4194304] 0 2024-07-16T09:55:30.504 INFO:tasks.workunit.client.0.smithi156.stdout:8/59: read d0/f8 [3795666,6405] 0 2024-07-16T09:55:30.505 INFO:tasks.workunit.client.1.smithi176.stdout:0/217: unlink d5/d1c/f32 0 2024-07-16T09:55:30.505 INFO:tasks.workunit.client.1.smithi176.stdout:1/120: creat d4/d8/f26 x:0 0 0 2024-07-16T09:55:30.506 INFO:tasks.workunit.client.0.smithi156.stdout:8/60: write d0/f12 [851848,47652] 0 2024-07-16T09:55:30.506 INFO:tasks.workunit.client.0.smithi156.stdout:9/82: symlink d2/de/l18 0 2024-07-16T09:55:30.506 INFO:tasks.workunit.client.1.smithi176.stdout:2/167: dwrite dd/d16/f1d [4194304,4194304] 0 2024-07-16T09:55:30.506 INFO:tasks.workunit.client.1.smithi176.stdout:9/146: fdatasync d5/d1d/f27 0 2024-07-16T09:55:30.507 INFO:tasks.workunit.client.0.smithi156.stdout:9/83: chown d2/d5/d16/c17 82 1 2024-07-16T09:55:30.507 INFO:tasks.workunit.client.0.smithi156.stdout:4/116: unlink d4/db/c14 0 2024-07-16T09:55:30.508 INFO:tasks.workunit.client.1.smithi176.stdout:6/160: mknod d1/d5/d7/d1e/d29/c38 0 2024-07-16T09:55:30.509 INFO:tasks.workunit.client.0.smithi156.stdout:4/117: fsync d4/f8 0 2024-07-16T09:55:30.510 INFO:tasks.workunit.client.1.smithi176.stdout:4/188: dread - da/dc/d10/d15/d17/f2a zero size 2024-07-16T09:55:30.514 INFO:tasks.workunit.client.0.smithi156.stdout:7/61: link l8 dc/d13/d15/l19 0 2024-07-16T09:55:30.515 INFO:tasks.workunit.client.0.smithi156.stdout:5/93: write f14 [5397898,12942] 0 2024-07-16T09:55:30.516 INFO:tasks.workunit.client.1.smithi176.stdout:0/218: symlink d5/d25/l46 0 2024-07-16T09:55:30.516 INFO:tasks.workunit.client.0.smithi156.stdout:6/122: mkdir d3/d26/d1f/d19/d29 0 2024-07-16T09:55:30.517 INFO:tasks.workunit.client.1.smithi176.stdout:9/147: stat d5/f16 0 2024-07-16T09:55:30.517 INFO:tasks.workunit.client.1.smithi176.stdout:1/121: mknod d4/db/c27 0 2024-07-16T09:55:30.517 INFO:tasks.workunit.client.1.smithi176.stdout:9/148: stat d5/de/c28 0 2024-07-16T09:55:30.517 INFO:tasks.workunit.client.1.smithi176.stdout:9/149: readlink d5/l14 0 2024-07-16T09:55:30.517 INFO:tasks.workunit.client.1.smithi176.stdout:4/189: mknod da/dc/d10/d28/c32 0 2024-07-16T09:55:30.518 INFO:tasks.workunit.client.1.smithi176.stdout:6/161: symlink d1/dc/d1c/l39 0 2024-07-16T09:55:30.518 INFO:tasks.workunit.client.0.smithi156.stdout:8/61: fdatasync d0/f6 0 2024-07-16T09:55:30.518 INFO:tasks.workunit.client.1.smithi176.stdout:5/249: mkdir d4/d14/d19/d1b/d61 0 2024-07-16T09:55:30.519 INFO:tasks.workunit.client.1.smithi176.stdout:3/83: fsync d3/da/fe 0 2024-07-16T09:55:30.519 INFO:tasks.workunit.client.1.smithi176.stdout:2/168: fsync dd/d16/f27 0 2024-07-16T09:55:30.520 INFO:tasks.workunit.client.0.smithi156.stdout:2/180: symlink d0/l3e 0 2024-07-16T09:55:30.521 INFO:tasks.workunit.client.1.smithi176.stdout:9/150: dread - d5/d1d/f27 zero size 2024-07-16T09:55:30.523 INFO:tasks.workunit.client.0.smithi156.stdout:2/181: stat d0/d13/d17/d1c/l1d 0 2024-07-16T09:55:30.524 INFO:tasks.workunit.client.0.smithi156.stdout:9/84: dwrite d2/f15 [0,4194304] 0 2024-07-16T09:55:30.524 INFO:tasks.workunit.client.1.smithi176.stdout:3/84: chown d3/da/l12 1055 1 2024-07-16T09:55:30.524 INFO:tasks.workunit.client.1.smithi176.stdout:5/250: dread - d4/d14/d19/d1b/d36/f59 zero size 2024-07-16T09:55:30.525 INFO:tasks.workunit.client.1.smithi176.stdout:4/190: mkdir da/dc/d10/d15/d33 0 2024-07-16T09:55:30.526 INFO:tasks.workunit.client.1.smithi176.stdout:9/151: mknod d5/d1d/d20/c2a 0 2024-07-16T09:55:30.527 INFO:tasks.workunit.client.0.smithi156.stdout:7/62: mknod dc/c1a 0 2024-07-16T09:55:30.528 INFO:tasks.workunit.client.1.smithi176.stdout:2/169: mkdir dd/d2e/d35 0 2024-07-16T09:55:30.528 INFO:tasks.workunit.client.1.smithi176.stdout:5/251: chown d4/d14/d19/f21 48298 1 2024-07-16T09:55:30.529 INFO:tasks.workunit.client.1.smithi176.stdout:1/122: dwrite d4/f24 [0,4194304] 0 2024-07-16T09:55:30.529 INFO:tasks.workunit.client.1.smithi176.stdout:3/85: rmdir d3/da 39 2024-07-16T09:55:30.531 INFO:tasks.workunit.client.0.smithi156.stdout:9/85: write d2/f15 [2487025,24918] 0 2024-07-16T09:55:30.531 INFO:tasks.workunit.client.0.smithi156.stdout:8/62: mkdir d0/d11/d13 0 2024-07-16T09:55:30.531 INFO:tasks.workunit.client.1.smithi176.stdout:2/170: read dd/d21/f2f [2054798,56388] 0 2024-07-16T09:55:30.531 INFO:tasks.workunit.client.1.smithi176.stdout:6/162: dread d1/d5/f6 [0,4194304] 0 2024-07-16T09:55:30.532 INFO:tasks.workunit.client.1.smithi176.stdout:5/252: fdatasync d4/d10/f22 0 2024-07-16T09:55:30.538 INFO:tasks.workunit.client.1.smithi176.stdout:6/163: dread d1/dc/fd [0,4194304] 0 2024-07-16T09:55:30.539 INFO:tasks.workunit.client.0.smithi156.stdout:5/94: dwrite f19 [0,4194304] 0 2024-07-16T09:55:30.544 INFO:tasks.workunit.client.0.smithi156.stdout:9/86: creat d2/de/f19 x:0 0 0 2024-07-16T09:55:30.544 INFO:tasks.workunit.client.0.smithi156.stdout:5/95: write f1a [151207,2567] 0 2024-07-16T09:55:30.545 INFO:tasks.workunit.client.0.smithi156.stdout:5/96: stat l17 0 2024-07-16T09:55:30.545 INFO:tasks.workunit.client.0.smithi156.stdout:8/63: rename d0/f9 to d0/d11/d13/f14 0 2024-07-16T09:55:30.546 INFO:tasks.workunit.client.0.smithi156.stdout:3/57: write f3 [5111777,94698] 0 2024-07-16T09:55:30.546 INFO:tasks.workunit.client.0.smithi156.stdout:4/118: dwrite d4/db/f2c [0,4194304] 0 2024-07-16T09:55:30.546 INFO:tasks.workunit.client.0.smithi156.stdout:6/123: dread d3/d26/d1f/f10 [0,4194304] 0 2024-07-16T09:55:30.547 INFO:tasks.workunit.client.1.smithi176.stdout:9/152: link d5/la d5/d1d/d20/l2b 0 2024-07-16T09:55:30.548 INFO:tasks.workunit.client.0.smithi156.stdout:5/97: write f14 [6045099,44951] 0 2024-07-16T09:55:30.549 INFO:tasks.workunit.client.1.smithi176.stdout:4/191: dread da/fb [0,4194304] 0 2024-07-16T09:55:30.551 INFO:tasks.workunit.client.0.smithi156.stdout:7/63: dwrite fb [0,4194304] 0 2024-07-16T09:55:30.552 INFO:tasks.workunit.client.0.smithi156.stdout:9/87: mkdir d2/d5/d1a 0 2024-07-16T09:55:30.554 INFO:tasks.workunit.client.0.smithi156.stdout:8/64: creat d0/d11/f15 x:0 0 0 2024-07-16T09:55:30.555 INFO:tasks.workunit.client.1.smithi176.stdout:1/123: dwrite d4/db/f1b [0,4194304] 0 2024-07-16T09:55:30.556 INFO:tasks.workunit.client.0.smithi156.stdout:3/58: symlink d8/da/l11 0 2024-07-16T09:55:30.557 INFO:tasks.workunit.client.0.smithi156.stdout:3/59: write d8/da/fc [226864,103376] 0 2024-07-16T09:55:30.557 INFO:tasks.workunit.client.0.smithi156.stdout:5/98: mknod d1b/c1c 0 2024-07-16T09:55:30.558 INFO:tasks.workunit.client.1.smithi176.stdout:5/253: dwrite d4/d14/d19/d1b/d36/f5b [0,4194304] 0 2024-07-16T09:55:30.558 INFO:tasks.workunit.client.0.smithi156.stdout:3/60: fsync f0 0 2024-07-16T09:55:30.560 INFO:tasks.workunit.client.1.smithi176.stdout:0/219: truncate d5/f14 2889249 0 2024-07-16T09:55:30.560 INFO:tasks.workunit.client.0.smithi156.stdout:8/65: creat d0/d11/f16 x:0 0 0 2024-07-16T09:55:30.562 INFO:tasks.workunit.client.1.smithi176.stdout:5/254: chown d4/d37 101550 1 2024-07-16T09:55:30.563 INFO:tasks.workunit.client.1.smithi176.stdout:3/86: dwrite d3/da/fe [0,4194304] 0 2024-07-16T09:55:30.565 INFO:tasks.workunit.client.0.smithi156.stdout:9/88: mkdir d2/d5/d1a/d1b 0 2024-07-16T09:55:30.565 INFO:tasks.workunit.client.1.smithi176.stdout:4/192: symlink da/dc/d10/d15/d17/l34 0 2024-07-16T09:55:30.567 INFO:tasks.workunit.client.0.smithi156.stdout:9/89: stat d2/de/c11 0 2024-07-16T09:55:30.568 INFO:tasks.workunit.client.1.smithi176.stdout:0/220: creat d5/d2c/f47 x:0 0 0 2024-07-16T09:55:30.569 INFO:tasks.workunit.client.0.smithi156.stdout:5/99: dwrite f14 [0,4194304] 0 2024-07-16T09:55:30.569 INFO:tasks.workunit.client.0.smithi156.stdout:3/61: dwrite d8/fb [0,4194304] 0 2024-07-16T09:55:30.571 INFO:tasks.workunit.client.1.smithi176.stdout:5/255: chown d4/d10/c11 1271627 1 2024-07-16T09:55:30.571 INFO:tasks.workunit.client.1.smithi176.stdout:4/193: creat da/dc/d10/d15/d17/f35 x:0 0 0 2024-07-16T09:55:30.572 INFO:tasks.workunit.client.1.smithi176.stdout:3/87: mkdir d3/d17 0 2024-07-16T09:55:30.574 INFO:tasks.workunit.client.1.smithi176.stdout:4/194: write da/dc/d10/d15/d17/f35 [667728,64297] 0 2024-07-16T09:55:30.574 INFO:tasks.workunit.client.1.smithi176.stdout:1/124: creat d4/d8/f28 x:0 0 0 2024-07-16T09:55:30.575 INFO:tasks.workunit.client.0.smithi156.stdout:3/62: truncate d8/f10 669003 0 2024-07-16T09:55:30.575 INFO:tasks.workunit.client.0.smithi156.stdout:7/64: rmdir dc/d18 0 2024-07-16T09:55:30.576 INFO:tasks.workunit.client.1.smithi176.stdout:1/125: write d4/f17 [2005455,95661] 0 2024-07-16T09:55:30.579 INFO:tasks.workunit.client.1.smithi176.stdout:0/221: fsync d5/d39/f43 0 2024-07-16T09:55:30.580 INFO:tasks.workunit.client.1.smithi176.stdout:0/222: dread - d5/d2c/f47 zero size 2024-07-16T09:55:30.582 INFO:tasks.workunit.client.0.smithi156.stdout:8/66: creat d0/dd/f17 x:0 0 0 2024-07-16T09:55:30.582 INFO:tasks.workunit.client.1.smithi176.stdout:5/256: mknod d4/d14/c62 0 2024-07-16T09:55:30.585 INFO:tasks.workunit.client.1.smithi176.stdout:2/171: dread dd/d16/f1d [0,4194304] 0 2024-07-16T09:55:30.585 INFO:tasks.workunit.client.0.smithi156.stdout:9/90: mknod d2/c1c 0 2024-07-16T09:55:30.585 INFO:tasks.workunit.client.0.smithi156.stdout:3/63: dread f3 [0,4194304] 0 2024-07-16T09:55:30.587 INFO:tasks.workunit.client.1.smithi176.stdout:5/257: chown d4/d10/f12 89349 1 2024-07-16T09:55:30.588 INFO:tasks.workunit.client.1.smithi176.stdout:3/88: rename d3/l7 to d3/l18 0 2024-07-16T09:55:30.589 INFO:tasks.workunit.client.0.smithi156.stdout:8/67: mknod d0/d11/c18 0 2024-07-16T09:55:30.592 INFO:tasks.workunit.client.1.smithi176.stdout:1/126: dwrite d4/d8/f28 [0,4194304] 0 2024-07-16T09:55:30.592 INFO:tasks.workunit.client.0.smithi156.stdout:5/100: link c8 d1b/c1d 0 2024-07-16T09:55:30.593 INFO:tasks.workunit.client.1.smithi176.stdout:3/89: creat d3/da/f19 x:0 0 0 2024-07-16T09:55:30.595 INFO:tasks.workunit.client.1.smithi176.stdout:5/258: creat d4/d10/d17/f63 x:0 0 0 2024-07-16T09:55:30.595 INFO:tasks.workunit.client.1.smithi176.stdout:0/223: link d5/d25/l37 d5/l48 0 2024-07-16T09:55:30.596 INFO:tasks.workunit.client.1.smithi176.stdout:5/259: write d4/d37/f3f [510150,24136] 0 2024-07-16T09:55:30.596 INFO:tasks.workunit.client.1.smithi176.stdout:0/224: truncate d5/d1c/f3b 673878 0 2024-07-16T09:55:30.597 INFO:tasks.workunit.client.0.smithi156.stdout:2/182: truncate d0/d5/f2a 43748 0 2024-07-16T09:55:30.597 INFO:tasks.workunit.client.1.smithi176.stdout:1/127: unlink d4/db/l15 0 2024-07-16T09:55:30.598 INFO:tasks.workunit.client.0.smithi156.stdout:5/101: mkdir d1b/d1e 0 2024-07-16T09:55:30.600 INFO:tasks.workunit.client.1.smithi176.stdout:2/172: rename dd/d21 to dd/d36 0 2024-07-16T09:55:30.600 INFO:tasks.workunit.client.1.smithi176.stdout:5/260: dread d4/d14/d19/d1b/d36/f4f [0,4194304] 0 2024-07-16T09:55:30.601 INFO:tasks.workunit.client.0.smithi156.stdout:3/64: dwrite d8/f10 [0,4194304] 0 2024-07-16T09:55:30.602 INFO:tasks.workunit.client.0.smithi156.stdout:9/91: getdents d2/d5/d16 0 2024-07-16T09:55:30.603 INFO:tasks.workunit.client.1.smithi176.stdout:3/90: symlink d3/d17/l1a 0 2024-07-16T09:55:30.603 INFO:tasks.workunit.client.0.smithi156.stdout:7/65: dwrite dc/d13/f12 [0,4194304] 0 2024-07-16T09:55:30.603 INFO:tasks.workunit.client.0.smithi156.stdout:5/102: mkdir d1b/d1f 0 2024-07-16T09:55:30.604 INFO:tasks.workunit.client.0.smithi156.stdout:9/92: stat d2/d5/ld 0 2024-07-16T09:55:30.605 INFO:tasks.workunit.client.0.smithi156.stdout:9/93: readlink d2/l10 0 2024-07-16T09:55:30.605 INFO:tasks.workunit.client.1.smithi176.stdout:0/225: dread d5/d1c/f20 [0,4194304] 0 2024-07-16T09:55:30.605 INFO:tasks.workunit.client.0.smithi156.stdout:7/66: write dc/f14 [592405,80777] 0 2024-07-16T09:55:30.607 INFO:tasks.workunit.client.1.smithi176.stdout:3/91: write d3/da/fe [4581193,4160] 0 2024-07-16T09:55:30.608 INFO:tasks.workunit.client.1.smithi176.stdout:1/128: dwrite d4/ff [0,4194304] 0 2024-07-16T09:55:30.614 INFO:tasks.workunit.client.0.smithi156.stdout:9/94: mkdir d2/d5/d1d 0 2024-07-16T09:55:30.614 INFO:tasks.workunit.client.1.smithi176.stdout:6/164: write d1/d5/d7/fe [278403,80189] 0 2024-07-16T09:55:30.614 INFO:tasks.workunit.client.1.smithi176.stdout:1/129: truncate d4/d8/f1e 4698663 0 2024-07-16T09:55:30.615 INFO:tasks.workunit.client.0.smithi156.stdout:3/65: dwrite f0 [0,4194304] 0 2024-07-16T09:55:30.616 INFO:tasks.workunit.client.0.smithi156.stdout:9/95: dread - d2/de/f19 zero size 2024-07-16T09:55:30.617 INFO:tasks.workunit.client.1.smithi176.stdout:3/92: fsync d3/da/fe 0 2024-07-16T09:55:30.618 INFO:tasks.workunit.client.1.smithi176.stdout:1/130: chown d4/d8/d12 167032 1 2024-07-16T09:55:30.618 INFO:tasks.workunit.client.1.smithi176.stdout:2/173: dwrite dd/d16/f33 [0,4194304] 0 2024-07-16T09:55:30.619 INFO:tasks.workunit.client.0.smithi156.stdout:6/124: truncate d3/f1d 1298451 0 2024-07-16T09:55:30.620 INFO:tasks.workunit.client.0.smithi156.stdout:2/183: getdents d0/d13/d17/d1c 0 2024-07-16T09:55:30.621 INFO:tasks.workunit.client.0.smithi156.stdout:6/125: readlink d3/d26/lb 0 2024-07-16T09:55:30.622 INFO:tasks.workunit.client.1.smithi176.stdout:3/93: read d3/da/f13 [54377,78698] 0 2024-07-16T09:55:30.623 INFO:tasks.workunit.client.0.smithi156.stdout:5/103: read f1a [45266,99290] 0 2024-07-16T09:55:30.623 INFO:tasks.workunit.client.0.smithi156.stdout:7/67: dread dc/f14 [0,4194304] 0 2024-07-16T09:55:30.625 INFO:tasks.workunit.client.1.smithi176.stdout:1/131: fsync d4/d8/f1e 0 2024-07-16T09:55:30.626 INFO:tasks.workunit.client.0.smithi156.stdout:2/184: write d0/d5/d29/f3c [473902,26861] 0 2024-07-16T09:55:30.626 INFO:tasks.workunit.client.0.smithi156.stdout:5/104: chown d1b/d1f 12484 1 2024-07-16T09:55:30.626 INFO:tasks.workunit.client.0.smithi156.stdout:3/66: symlink d8/l12 0 2024-07-16T09:55:30.626 INFO:tasks.workunit.client.0.smithi156.stdout:1/91: sync 2024-07-16T09:55:30.628 INFO:tasks.workunit.client.1.smithi176.stdout:0/226: dwrite d5/d1c/d24/d35/f3c [0,4194304] 0 2024-07-16T09:55:30.629 INFO:tasks.workunit.client.1.smithi176.stdout:1/132: write d4/f10 [4200520,37816] 0 2024-07-16T09:55:30.630 INFO:tasks.workunit.client.1.smithi176.stdout:2/174: creat dd/d2e/f37 x:0 0 0 2024-07-16T09:55:30.630 INFO:tasks.workunit.client.1.smithi176.stdout:0/227: readlink d5/d1c/d24/l40 0 2024-07-16T09:55:30.631 INFO:tasks.workunit.client.0.smithi156.stdout:1/92: read - dc/d14/f1d zero size 2024-07-16T09:55:30.631 INFO:tasks.workunit.client.1.smithi176.stdout:5/261: write d4/f6 [3646659,120598] 0 2024-07-16T09:55:30.633 INFO:tasks.workunit.client.0.smithi156.stdout:6/126: chown d3/d26/d1f/fe 108 1 2024-07-16T09:55:30.638 INFO:tasks.workunit.client.0.smithi156.stdout:5/105: dwrite fc [0,4194304] 0 2024-07-16T09:55:30.653 INFO:tasks.workunit.client.0.smithi156.stdout:1/93: dwrite dc/f13 [0,4194304] 0 2024-07-16T09:55:30.655 INFO:tasks.workunit.client.0.smithi156.stdout:5/106: dread f12 [0,4194304] 0 2024-07-16T09:55:30.655 INFO:tasks.workunit.client.0.smithi156.stdout:3/67: dwrite d8/da/fc [0,4194304] 0 2024-07-16T09:55:30.660 INFO:tasks.workunit.client.1.smithi176.stdout:5/262: fdatasync d4/d14/d19/d1b/d36/f5b 0 2024-07-16T09:55:30.662 INFO:tasks.workunit.client.0.smithi156.stdout:2/185: creat d0/d5/d29/f3f x:0 0 0 2024-07-16T09:55:30.664 INFO:tasks.workunit.client.0.smithi156.stdout:6/127: creat d3/f2a x:0 0 0 2024-07-16T09:55:30.664 INFO:tasks.workunit.client.0.smithi156.stdout:6/128: dread - d3/f2a zero size 2024-07-16T09:55:30.665 INFO:tasks.workunit.client.0.smithi156.stdout:6/129: truncate d3/f25 705408 0 2024-07-16T09:55:30.665 INFO:tasks.workunit.client.1.smithi176.stdout:2/175: unlink dd/d16/c20 0 2024-07-16T09:55:30.666 INFO:tasks.workunit.client.1.smithi176.stdout:6/165: link d1/d5/d7/lb d1/l3a 0 2024-07-16T09:55:30.666 INFO:tasks.workunit.client.1.smithi176.stdout:1/133: write d4/d8/fa [517342,108325] 0 2024-07-16T09:55:30.668 INFO:tasks.workunit.client.1.smithi176.stdout:0/228: dread d5/d1c/f3b [0,4194304] 0 2024-07-16T09:55:30.670 INFO:tasks.workunit.client.1.smithi176.stdout:6/166: fdatasync d1/d5/d7/fe 0 2024-07-16T09:55:30.671 INFO:tasks.workunit.client.1.smithi176.stdout:3/94: dwrite d3/da/fe [4194304,4194304] 0 2024-07-16T09:55:30.673 INFO:tasks.workunit.client.0.smithi156.stdout:1/94: creat dc/d20/f23 x:0 0 0 2024-07-16T09:55:30.674 INFO:tasks.workunit.client.0.smithi156.stdout:6/130: dwrite d3/f25 [0,4194304] 0 2024-07-16T09:55:30.675 INFO:tasks.workunit.client.1.smithi176.stdout:3/95: fdatasync d3/da/f19 0 2024-07-16T09:55:30.675 INFO:tasks.workunit.client.0.smithi156.stdout:3/68: mkdir d8/da/d13 0 2024-07-16T09:55:30.675 INFO:tasks.workunit.client.0.smithi156.stdout:9/96: link d2/de/c11 d2/d5/d1a/d1b/c1e 0 2024-07-16T09:55:30.677 INFO:tasks.workunit.client.0.smithi156.stdout:6/131: write d3/f1c [4095086,6776] 0 2024-07-16T09:55:30.678 INFO:tasks.workunit.client.1.smithi176.stdout:3/96: mknod d3/d17/c1b 0 2024-07-16T09:55:30.678 INFO:tasks.workunit.client.1.smithi176.stdout:6/167: fsync d1/d5/f27 0 2024-07-16T09:55:30.682 INFO:tasks.workunit.client.0.smithi156.stdout:7/68: mknod dc/d13/d15/c1b 0 2024-07-16T09:55:30.684 INFO:tasks.workunit.client.0.smithi156.stdout:2/186: creat d0/d5/d29/d2d/f40 x:0 0 0 2024-07-16T09:55:30.685 INFO:tasks.workunit.client.0.smithi156.stdout:1/95: dwrite dc/f10 [0,4194304] 0 2024-07-16T09:55:30.686 INFO:tasks.workunit.client.1.smithi176.stdout:2/176: dwrite f3 [4194304,4194304] 0 2024-07-16T09:55:30.686 INFO:tasks.workunit.client.1.smithi176.stdout:0/229: getdents d5/d1c/d24/d35 0 2024-07-16T09:55:30.687 INFO:tasks.workunit.client.1.smithi176.stdout:2/177: fdatasync dd/d16/f32 0 2024-07-16T09:55:30.688 INFO:tasks.workunit.client.0.smithi156.stdout:3/69: mkdir d8/d14 0 2024-07-16T09:55:30.690 INFO:tasks.workunit.client.0.smithi156.stdout:6/132: fsync d3/d26/d1f/f10 0 2024-07-16T09:55:30.691 INFO:tasks.workunit.client.1.smithi176.stdout:3/97: dread d3/da/fe [4194304,4194304] 0 2024-07-16T09:55:30.693 INFO:tasks.workunit.client.1.smithi176.stdout:6/168: getdents d1/d5/d7/d1e/d29 0 2024-07-16T09:55:30.693 INFO:tasks.workunit.client.0.smithi156.stdout:7/69: unlink dc/c10 0 2024-07-16T09:55:30.694 INFO:tasks.workunit.client.1.smithi176.stdout:6/169: readlink d1/d5/la 0 2024-07-16T09:55:30.694 INFO:tasks.workunit.client.0.smithi156.stdout:7/70: truncate dc/f14 1049237 0 2024-07-16T09:55:30.695 INFO:tasks.workunit.client.1.smithi176.stdout:0/230: rename d5/d1c/d24/c36 to d5/d25/c49 0 2024-07-16T09:55:30.695 INFO:tasks.workunit.client.1.smithi176.stdout:3/98: mknod d3/d17/c1c 0 2024-07-16T09:55:30.697 INFO:tasks.workunit.client.1.smithi176.stdout:2/178: dread dd/d36/f2f [0,4194304] 0 2024-07-16T09:55:30.698 INFO:tasks.workunit.client.0.smithi156.stdout:6/133: dread d3/f25 [0,4194304] 0 2024-07-16T09:55:30.698 INFO:tasks.workunit.client.0.smithi156.stdout:2/187: symlink d0/d5/d29/d2d/l41 0 2024-07-16T09:55:30.698 INFO:tasks.workunit.client.1.smithi176.stdout:2/179: stat dd/de/c13 0 2024-07-16T09:55:30.699 INFO:tasks.workunit.client.0.smithi156.stdout:1/96: mkdir dc/d24 0 2024-07-16T09:55:30.700 INFO:tasks.workunit.client.1.smithi176.stdout:3/99: write d3/da/ff [1925606,44138] 0 2024-07-16T09:55:30.700 INFO:tasks.workunit.client.1.smithi176.stdout:3/100: fdatasync d3/da/f19 0 2024-07-16T09:55:30.701 INFO:tasks.workunit.client.0.smithi156.stdout:2/188: write d0/f8 [2214549,91632] 0 2024-07-16T09:55:30.701 INFO:tasks.workunit.client.0.smithi156.stdout:1/97: truncate dc/f13 4389224 0 2024-07-16T09:55:30.702 INFO:tasks.workunit.client.0.smithi156.stdout:3/70: creat d8/d14/f15 x:0 0 0 2024-07-16T09:55:30.702 INFO:tasks.workunit.client.0.smithi156.stdout:1/98: chown dc/cf 178 1 2024-07-16T09:55:30.704 INFO:tasks.workunit.client.0.smithi156.stdout:5/107: truncate f12 783212 0 2024-07-16T09:55:30.705 INFO:tasks.workunit.client.0.smithi156.stdout:6/134: creat d3/d13/d17/f2b x:0 0 0 2024-07-16T09:55:30.705 INFO:tasks.workunit.client.1.smithi176.stdout:6/170: link d1/f15 d1/f3b 0 2024-07-16T09:55:30.706 INFO:tasks.workunit.client.1.smithi176.stdout:5/263: write d4/d10/d17/f18 [966958,93144] 0 2024-07-16T09:55:30.706 INFO:tasks.workunit.client.1.smithi176.stdout:2/180: dread f9 [0,4194304] 0 2024-07-16T09:55:30.706 INFO:tasks.workunit.client.1.smithi176.stdout:3/101: mkdir d3/d17/d1d 0 2024-07-16T09:55:30.707 INFO:tasks.workunit.client.0.smithi156.stdout:7/71: link dc/d13/f12 dc/d13/f1c 0 2024-07-16T09:55:30.708 INFO:tasks.workunit.client.1.smithi176.stdout:5/264: dread - d4/d10/f54 zero size 2024-07-16T09:55:30.708 INFO:tasks.workunit.client.0.smithi156.stdout:2/189: truncate d0/d5/d29/d2d/f40 714503 0 2024-07-16T09:55:30.709 INFO:tasks.workunit.client.1.smithi176.stdout:5/265: chown d4/d10/d17/l29 23 1 2024-07-16T09:55:30.709 INFO:tasks.workunit.client.1.smithi176.stdout:6/171: unlink d1/dc/d1c/f35 0 2024-07-16T09:55:30.711 INFO:tasks.workunit.client.0.smithi156.stdout:3/71: dwrite d8/f10 [0,4194304] 0 2024-07-16T09:55:30.711 INFO:tasks.workunit.client.0.smithi156.stdout:6/135: rename d3/d26/d1f/f15 to d3/d26/d1f/f2c 0 2024-07-16T09:55:30.712 INFO:tasks.workunit.client.0.smithi156.stdout:7/72: creat dc/d13/f1d x:0 0 0 2024-07-16T09:55:30.713 INFO:tasks.workunit.client.1.smithi176.stdout:5/266: rmdir d4/d14/d19/d1b/d36/d38/d39 39 2024-07-16T09:55:30.713 INFO:tasks.workunit.client.0.smithi156.stdout:6/136: stat d3/d26/d1f/f2c 0 2024-07-16T09:55:30.713 INFO:tasks.workunit.client.1.smithi176.stdout:6/172: symlink d1/d5/d7/d2f/l3c 0 2024-07-16T09:55:30.714 INFO:tasks.workunit.client.1.smithi176.stdout:3/102: mkdir d3/d17/d1d/d1e 0 2024-07-16T09:55:30.715 INFO:tasks.workunit.client.1.smithi176.stdout:3/103: chown d3/l4 524114 1 2024-07-16T09:55:30.716 INFO:tasks.workunit.client.1.smithi176.stdout:3/104: dread - d3/da/f19 zero size 2024-07-16T09:55:30.717 INFO:tasks.workunit.client.1.smithi176.stdout:6/173: dread d1/d5/d7/fe [0,4194304] 0 2024-07-16T09:55:30.718 INFO:tasks.workunit.client.1.smithi176.stdout:5/267: dread f2 [0,4194304] 0 2024-07-16T09:55:30.718 INFO:tasks.workunit.client.0.smithi156.stdout:7/73: rename l5 to dc/d13/d15/l1e 0 2024-07-16T09:55:30.719 INFO:tasks.workunit.client.0.smithi156.stdout:7/74: stat dc/f14 0 2024-07-16T09:55:30.720 INFO:tasks.workunit.client.0.smithi156.stdout:2/190: dwrite d0/d13/d17/f34 [0,4194304] 0 2024-07-16T09:55:30.720 INFO:tasks.workunit.client.0.smithi156.stdout:3/72: dread d8/fb [0,4194304] 0 2024-07-16T09:55:30.721 INFO:tasks.workunit.client.1.smithi176.stdout:6/174: dread d1/dc/fd [0,4194304] 0 2024-07-16T09:55:30.721 INFO:tasks.workunit.client.0.smithi156.stdout:6/137: symlink d3/d26/d1f/d19/d29/l2d 0 2024-07-16T09:55:30.722 INFO:tasks.workunit.client.1.smithi176.stdout:6/175: read - d1/d5/d7/d1e/f36 zero size 2024-07-16T09:55:30.723 INFO:tasks.workunit.client.1.smithi176.stdout:6/176: truncate d1/f15 436682 0 2024-07-16T09:55:30.724 INFO:tasks.workunit.client.0.smithi156.stdout:7/75: symlink dc/d13/l1f 0 2024-07-16T09:55:30.726 INFO:tasks.workunit.client.0.smithi156.stdout:6/138: read d3/d26/d1f/f2c [3860698,22855] 0 2024-07-16T09:55:30.727 INFO:tasks.workunit.client.1.smithi176.stdout:3/105: link d3/da/f19 d3/d17/d1d/d1e/f1f 0 2024-07-16T09:55:30.727 INFO:tasks.workunit.client.0.smithi156.stdout:3/73: symlink d8/l16 0 2024-07-16T09:55:30.727 INFO:tasks.workunit.client.1.smithi176.stdout:0/231: write d5/d1c/d24/f30 [917517,97472] 0 2024-07-16T09:55:30.728 INFO:tasks.workunit.client.0.smithi156.stdout:6/139: dread - d3/d13/d17/f2b zero size 2024-07-16T09:55:30.728 INFO:tasks.workunit.client.0.smithi156.stdout:9/97: write d2/d5/fb [977497,82958] 0 2024-07-16T09:55:30.728 INFO:tasks.workunit.client.1.smithi176.stdout:5/268: link d4/f27 d4/d14/d3d/d4d/f64 0 2024-07-16T09:55:30.730 INFO:tasks.workunit.client.0.smithi156.stdout:9/98: fdatasync d2/d5/ff 0 2024-07-16T09:55:30.732 INFO:tasks.workunit.client.1.smithi176.stdout:0/232: chown d5/d25/c49 91553 1 2024-07-16T09:55:30.732 INFO:tasks.workunit.client.0.smithi156.stdout:6/140: rmdir d3/d26 39 2024-07-16T09:55:30.734 INFO:tasks.workunit.client.1.smithi176.stdout:5/269: symlink d4/d10/l65 0 2024-07-16T09:55:30.734 INFO:tasks.workunit.client.0.smithi156.stdout:6/141: dread - d3/d13/d17/f2b zero size 2024-07-16T09:55:30.735 INFO:tasks.workunit.client.0.smithi156.stdout:3/74: dread d8/da/fc [0,4194304] 0 2024-07-16T09:55:30.736 INFO:tasks.workunit.client.0.smithi156.stdout:6/142: write d3/f1c [2802596,129379] 0 2024-07-16T09:55:30.737 INFO:tasks.workunit.client.0.smithi156.stdout:9/99: write d2/d5/f8 [3834021,97584] 0 2024-07-16T09:55:30.739 INFO:tasks.workunit.client.1.smithi176.stdout:6/177: getdents d1/d5 0 2024-07-16T09:55:30.739 INFO:tasks.workunit.client.1.smithi176.stdout:5/270: rename d4/d14/d3d/f4b to d4/d10/d17/f66 0 2024-07-16T09:55:30.740 INFO:tasks.workunit.client.1.smithi176.stdout:2/181: dwrite f9 [0,4194304] 0 2024-07-16T09:55:30.741 INFO:tasks.workunit.client.0.smithi156.stdout:9/100: dread d2/d5/ff [0,4194304] 0 2024-07-16T09:55:30.744 INFO:tasks.workunit.client.1.smithi176.stdout:6/178: write d1/d5/d7/d1e/f36 [892365,100451] 0 2024-07-16T09:55:30.744 INFO:tasks.workunit.client.1.smithi176.stdout:0/233: dread d5/d2c/f2d [0,4194304] 0 2024-07-16T09:55:30.745 INFO:tasks.workunit.client.0.smithi156.stdout:7/76: fsync dc/d13/f12 0 2024-07-16T09:55:30.745 INFO:tasks.workunit.client.0.smithi156.stdout:6/143: dread d3/f1c [0,4194304] 0 2024-07-16T09:55:30.748 INFO:tasks.workunit.client.1.smithi176.stdout:6/179: unlink d1/dc/c20 0 2024-07-16T09:55:30.749 INFO:tasks.workunit.client.1.smithi176.stdout:0/234: readlink d5/d25/l37 0 2024-07-16T09:55:30.750 INFO:tasks.workunit.client.0.smithi156.stdout:3/75: dwrite f3 [0,4194304] 0 2024-07-16T09:55:30.750 INFO:tasks.workunit.client.0.smithi156.stdout:9/101: dread d2/f3 [4194304,4194304] 0 2024-07-16T09:55:30.751 INFO:tasks.workunit.client.1.smithi176.stdout:3/106: truncate d3/da/fe 8373539 0 2024-07-16T09:55:30.752 INFO:tasks.workunit.client.1.smithi176.stdout:2/182: dread dd/f2a [0,4194304] 0 2024-07-16T09:55:30.752 INFO:tasks.workunit.client.1.smithi176.stdout:6/180: symlink d1/dc/l3d 0 2024-07-16T09:55:30.754 INFO:tasks.workunit.client.1.smithi176.stdout:6/181: dread d1/dc/fd [0,4194304] 0 2024-07-16T09:55:30.754 INFO:tasks.workunit.client.1.smithi176.stdout:0/235: unlink d5/d1c/d24/d35/c38 0 2024-07-16T09:55:30.754 INFO:tasks.workunit.client.0.smithi156.stdout:3/76: fdatasync d8/da/fc 0 2024-07-16T09:55:30.756 INFO:tasks.workunit.client.0.smithi156.stdout:3/77: truncate d8/fb 5123032 0 2024-07-16T09:55:30.757 INFO:tasks.workunit.client.0.smithi156.stdout:2/191: truncate d0/d13/d17/d1c/f27 913900 0 2024-07-16T09:55:30.758 INFO:tasks.workunit.client.1.smithi176.stdout:0/236: symlink d5/d39/l4a 0 2024-07-16T09:55:30.759 INFO:tasks.workunit.client.1.smithi176.stdout:6/182: creat d1/d5/f3e x:0 0 0 2024-07-16T09:55:30.760 INFO:tasks.workunit.client.0.smithi156.stdout:3/78: read d8/da/fc [2463165,15709] 0 2024-07-16T09:55:30.763 INFO:tasks.workunit.client.0.smithi156.stdout:2/192: creat d0/d13/d17/f42 x:0 0 0 2024-07-16T09:55:30.763 INFO:tasks.workunit.client.0.smithi156.stdout:6/144: dwrite d3/f25 [0,4194304] 0 2024-07-16T09:55:30.767 INFO:tasks.workunit.client.0.smithi156.stdout:7/77: dwrite dc/d13/f12 [0,4194304] 0 2024-07-16T09:55:30.768 INFO:tasks.workunit.client.1.smithi176.stdout:8/120: sync 2024-07-16T09:55:30.768 INFO:tasks.workunit.client.1.smithi176.stdout:7/180: sync 2024-07-16T09:55:30.769 INFO:tasks.workunit.client.1.smithi176.stdout:2/183: write dd/d36/f2f [2502143,62156] 0 2024-07-16T09:55:30.771 INFO:tasks.workunit.client.0.smithi156.stdout:3/79: dread f3 [0,4194304] 0 2024-07-16T09:55:30.772 INFO:tasks.workunit.client.0.smithi156.stdout:9/102: truncate d2/f3 1653049 0 2024-07-16T09:55:30.773 INFO:tasks.workunit.client.1.smithi176.stdout:2/184: symlink dd/de/l38 0 2024-07-16T09:55:30.774 INFO:tasks.workunit.client.0.smithi156.stdout:9/103: stat d2/l10 0 2024-07-16T09:55:30.774 INFO:tasks.workunit.client.0.smithi156.stdout:7/78: mkdir dc/d13/d20 0 2024-07-16T09:55:30.776 INFO:tasks.workunit.client.0.smithi156.stdout:2/193: link d0/f2c d0/d13/f43 0 2024-07-16T09:55:30.778 INFO:tasks.workunit.client.0.smithi156.stdout:3/80: rename d8/d14/f15 to d8/da/d13/f17 0 2024-07-16T09:55:30.779 INFO:tasks.workunit.client.1.smithi176.stdout:7/181: rename d4/d5/d21/d33 to d4/d47 0 2024-07-16T09:55:30.779 INFO:tasks.workunit.client.0.smithi156.stdout:6/145: mkdir d3/d26/d1f/d19/d2e 0 2024-07-16T09:55:30.780 INFO:tasks.workunit.client.0.smithi156.stdout:6/146: write d3/d13/d17/f2b [447822,67080] 0 2024-07-16T09:55:30.781 INFO:tasks.workunit.client.1.smithi176.stdout:8/121: getdents d8/d20 0 2024-07-16T09:55:30.781 INFO:tasks.workunit.client.1.smithi176.stdout:8/122: truncate d8/fd 2015849 0 2024-07-16T09:55:30.783 INFO:tasks.workunit.client.0.smithi156.stdout:2/194: creat d0/d5/f44 x:0 0 0 2024-07-16T09:55:30.784 INFO:tasks.workunit.client.0.smithi156.stdout:3/81: symlink d8/l18 0 2024-07-16T09:55:30.784 INFO:tasks.workunit.client.1.smithi176.stdout:8/123: symlink d8/l29 0 2024-07-16T09:55:30.785 INFO:tasks.workunit.client.0.smithi156.stdout:7/79: mknod dc/d13/d20/c21 0 2024-07-16T09:55:30.786 INFO:tasks.workunit.client.1.smithi176.stdout:7/182: rename d4/d5/d21/f3b to d4/f48 0 2024-07-16T09:55:30.786 INFO:tasks.workunit.client.0.smithi156.stdout:2/195: mknod d0/d13/d17/d1c/c45 0 2024-07-16T09:55:30.787 INFO:tasks.workunit.client.0.smithi156.stdout:3/82: write f3 [5642425,73581] 0 2024-07-16T09:55:30.787 INFO:tasks.workunit.client.1.smithi176.stdout:7/183: write d4/dd/f15 [994425,79726] 0 2024-07-16T09:55:30.787 INFO:tasks.workunit.client.1.smithi176.stdout:8/124: mkdir d8/d20/d2a 0 2024-07-16T09:55:30.788 INFO:tasks.workunit.client.0.smithi156.stdout:3/83: read - d8/da/d13/f17 zero size 2024-07-16T09:55:30.788 INFO:tasks.workunit.client.1.smithi176.stdout:7/184: chown d4/f11 0 1 2024-07-16T09:55:30.789 INFO:tasks.workunit.client.1.smithi176.stdout:8/125: write d8/fd [2276081,19856] 0 2024-07-16T09:55:30.790 INFO:tasks.workunit.client.1.smithi176.stdout:8/126: chown c3 15253 1 2024-07-16T09:55:30.790 INFO:tasks.workunit.client.0.smithi156.stdout:3/84: mknod d8/da/c19 0 2024-07-16T09:55:30.791 INFO:tasks.workunit.client.1.smithi176.stdout:7/185: mkdir d4/dd/d49 0 2024-07-16T09:55:30.792 INFO:tasks.workunit.client.1.smithi176.stdout:7/186: stat d4/dd/le 0 2024-07-16T09:55:30.792 INFO:tasks.workunit.client.0.smithi156.stdout:3/85: creat d8/f1a x:0 0 0 2024-07-16T09:55:30.793 INFO:tasks.workunit.client.1.smithi176.stdout:7/187: chown d4/d47 52 1 2024-07-16T09:55:30.793 INFO:tasks.workunit.client.1.smithi176.stdout:8/127: creat d8/d20/f2b x:0 0 0 2024-07-16T09:55:30.793 INFO:tasks.workunit.client.1.smithi176.stdout:8/128: chown d8/l1c 1981758924 1 2024-07-16T09:55:30.794 INFO:tasks.workunit.client.1.smithi176.stdout:8/129: chown d8/d14 1035014 1 2024-07-16T09:55:30.799 INFO:tasks.workunit.client.1.smithi176.stdout:7/188: dwrite d4/d5/f31 [0,4194304] 0 2024-07-16T09:55:30.799 INFO:tasks.workunit.client.1.smithi176.stdout:8/130: mkdir d8/d23/d2c 0 2024-07-16T09:55:30.800 INFO:tasks.workunit.client.1.smithi176.stdout:8/131: write f5 [3418901,5648] 0 2024-07-16T09:55:30.801 INFO:tasks.workunit.client.0.smithi156.stdout:7/80: fsync dc/d13/f1c 0 2024-07-16T09:55:30.801 INFO:tasks.workunit.client.0.smithi156.stdout:0/46: sync 2024-07-16T09:55:30.803 INFO:tasks.workunit.client.0.smithi156.stdout:7/81: write dc/d13/f1c [3323666,52554] 0 2024-07-16T09:55:30.806 INFO:tasks.workunit.client.1.smithi176.stdout:0/237: write f3 [2049804,127410] 0 2024-07-16T09:55:30.806 INFO:tasks.workunit.client.1.smithi176.stdout:3/107: write d3/da/fe [7382582,82624] 0 2024-07-16T09:55:30.808 INFO:tasks.workunit.client.1.smithi176.stdout:3/108: readlink d3/da/ld 0 2024-07-16T09:55:30.810 INFO:tasks.workunit.client.0.smithi156.stdout:2/196: truncate d0/fe 673046 0 2024-07-16T09:55:30.811 INFO:tasks.workunit.client.0.smithi156.stdout:7/82: fdatasync fb 0 2024-07-16T09:55:30.812 INFO:tasks.workunit.client.0.smithi156.stdout:2/197: chown d0/d13/d17 135 1 2024-07-16T09:55:30.814 INFO:tasks.workunit.client.0.smithi156.stdout:2/198: rmdir d0/d5/d29 39 2024-07-16T09:55:30.816 INFO:tasks.workunit.client.0.smithi156.stdout:2/199: write d0/f4 [4658752,54984] 0 2024-07-16T09:55:30.818 INFO:tasks.workunit.client.1.smithi176.stdout:8/132: chown d8/d23/d2c 0 1 2024-07-16T09:55:30.820 INFO:tasks.workunit.client.1.smithi176.stdout:7/189: write d4/dd/f1a [2434001,113375] 0 2024-07-16T09:55:30.821 INFO:tasks.workunit.client.1.smithi176.stdout:3/109: write d3/da/f19 [481899,111360] 0 2024-07-16T09:55:30.823 INFO:tasks.workunit.client.0.smithi156.stdout:7/83: dwrite dc/d13/f12 [0,4194304] 0 2024-07-16T09:55:30.823 INFO:tasks.workunit.client.0.smithi156.stdout:9/104: dwrite d2/f3 [0,4194304] 0 2024-07-16T09:55:30.824 INFO:tasks.workunit.client.1.smithi176.stdout:7/190: fsync d4/d47/f44 0 2024-07-16T09:55:30.825 INFO:tasks.workunit.client.1.smithi176.stdout:3/110: readlink d3/l18 0 2024-07-16T09:55:30.826 INFO:tasks.workunit.client.0.smithi156.stdout:2/200: rename d0/f4 to d0/d13/d17/d1c/f46 0 2024-07-16T09:55:30.829 INFO:tasks.workunit.client.1.smithi176.stdout:0/238: dwrite d5/f13 [0,4194304] 0 2024-07-16T09:55:30.829 INFO:tasks.workunit.client.1.smithi176.stdout:3/111: dread d3/da/f13 [0,4194304] 0 2024-07-16T09:55:30.830 INFO:tasks.workunit.client.0.smithi156.stdout:2/201: dread - d0/d5/d29/f3f zero size 2024-07-16T09:55:30.830 INFO:tasks.workunit.client.0.smithi156.stdout:7/84: chown dc/ce 1846355 1 2024-07-16T09:55:30.831 INFO:tasks.workunit.client.1.smithi176.stdout:6/183: dwrite d1/d5/d7/f9 [0,4194304] 0 2024-07-16T09:55:30.831 INFO:tasks.workunit.client.0.smithi156.stdout:9/105: dread d2/f4 [0,4194304] 0 2024-07-16T09:55:30.832 INFO:tasks.workunit.client.0.smithi156.stdout:2/202: write d0/d13/d17/f42 [276894,117440] 0 2024-07-16T09:55:30.833 INFO:tasks.workunit.client.0.smithi156.stdout:7/85: mknod dc/c22 0 2024-07-16T09:55:30.834 INFO:tasks.workunit.client.1.smithi176.stdout:8/133: truncate d8/d14/f27 56677 0 2024-07-16T09:55:30.834 INFO:tasks.workunit.client.1.smithi176.stdout:6/184: fdatasync d1/d5/d7/d2f/f32 0 2024-07-16T09:55:30.834 INFO:tasks.workunit.client.0.smithi156.stdout:2/203: fdatasync d0/d13/d17/d1c/f24 0 2024-07-16T09:55:30.835 INFO:tasks.workunit.client.0.smithi156.stdout:9/106: mknod d2/d5/d1d/c1f 0 2024-07-16T09:55:30.836 INFO:tasks.workunit.client.1.smithi176.stdout:0/239: symlink d5/d1c/l4b 0 2024-07-16T09:55:30.838 INFO:tasks.workunit.client.1.smithi176.stdout:3/112: creat d3/d17/d1d/f20 x:0 0 0 2024-07-16T09:55:30.838 INFO:tasks.workunit.client.1.smithi176.stdout:6/185: read - d1/d5/f2c zero size 2024-07-16T09:55:30.838 INFO:tasks.workunit.client.0.smithi156.stdout:7/86: rename dc/c22 to dc/d13/d20/c23 0 2024-07-16T09:55:30.839 INFO:tasks.workunit.client.0.smithi156.stdout:7/87: chown c6 62322 1 2024-07-16T09:55:30.840 INFO:tasks.workunit.client.1.smithi176.stdout:8/134: unlink d8/d20/f24 0 2024-07-16T09:55:30.840 INFO:tasks.workunit.client.1.smithi176.stdout:7/191: dread d4/dd/f15 [0,4194304] 0 2024-07-16T09:55:30.841 INFO:tasks.workunit.client.0.smithi156.stdout:7/88: write dc/d13/f1d [947898,87633] 0 2024-07-16T09:55:30.841 INFO:tasks.workunit.client.0.smithi156.stdout:9/107: creat d2/d5/d16/f20 x:0 0 0 2024-07-16T09:55:30.841 INFO:tasks.workunit.client.1.smithi176.stdout:0/240: fdatasync d5/d1c/f3b 0 2024-07-16T09:55:30.843 INFO:tasks.workunit.client.0.smithi156.stdout:7/89: write dc/d13/f1c [2634440,32525] 0 2024-07-16T09:55:30.843 INFO:tasks.workunit.client.0.smithi156.stdout:2/204: truncate d0/d5/f2a 1075388 0 2024-07-16T09:55:30.843 INFO:tasks.workunit.client.1.smithi176.stdout:3/113: rename d3/da/ff to d3/d17/d1d/d1e/f21 0 2024-07-16T09:55:30.847 INFO:tasks.workunit.client.1.smithi176.stdout:7/192: mknod d4/d5/d10/d17/c4a 0 2024-07-16T09:55:30.848 INFO:tasks.workunit.client.0.smithi156.stdout:9/108: dread d2/f15 [0,4194304] 0 2024-07-16T09:55:30.848 INFO:tasks.workunit.client.1.smithi176.stdout:7/193: fsync d4/dd/f37 0 2024-07-16T09:55:30.849 INFO:tasks.workunit.client.1.smithi176.stdout:8/135: dwrite d8/fd [0,4194304] 0 2024-07-16T09:55:30.849 INFO:tasks.workunit.client.0.smithi156.stdout:2/205: creat d0/d13/d17/d1c/f47 x:0 0 0 2024-07-16T09:55:30.849 INFO:tasks.workunit.client.1.smithi176.stdout:0/241: creat d5/d1c/d24/d35/f4c x:0 0 0 2024-07-16T09:55:30.851 INFO:tasks.workunit.client.1.smithi176.stdout:3/114: creat d3/d17/d1d/d1e/f22 x:0 0 0 2024-07-16T09:55:30.856 INFO:tasks.workunit.client.1.smithi176.stdout:8/136: creat d8/d14/f2d x:0 0 0 2024-07-16T09:55:30.857 INFO:tasks.workunit.client.1.smithi176.stdout:0/242: creat d5/d2c/f4d x:0 0 0 2024-07-16T09:55:30.857 INFO:tasks.workunit.client.1.smithi176.stdout:7/194: mkdir d4/d5/d4b 0 2024-07-16T09:55:30.858 INFO:tasks.workunit.client.1.smithi176.stdout:6/186: getdents d1/d5/d7 0 2024-07-16T09:55:30.861 INFO:tasks.workunit.client.0.smithi156.stdout:2/206: dread d0/f7 [0,4194304] 0 2024-07-16T09:55:30.861 INFO:tasks.workunit.client.0.smithi156.stdout:9/109: dwrite d2/f3 [4194304,4194304] 0 2024-07-16T09:55:30.862 INFO:tasks.workunit.client.1.smithi176.stdout:8/137: creat d8/f2e x:0 0 0 2024-07-16T09:55:30.865 INFO:tasks.workunit.client.1.smithi176.stdout:0/243: write d5/d2c/f4d [301483,6984] 0 2024-07-16T09:55:30.865 INFO:tasks.workunit.client.1.smithi176.stdout:6/187: write d1/d5/d7/f9 [1784581,65272] 0 2024-07-16T09:55:30.866 INFO:tasks.workunit.client.1.smithi176.stdout:7/195: chown d4/d5/lc 0 1 2024-07-16T09:55:30.869 INFO:tasks.workunit.client.0.smithi156.stdout:9/110: dwrite d2/f13 [0,4194304] 0 2024-07-16T09:55:30.869 INFO:tasks.workunit.client.1.smithi176.stdout:3/115: dwrite d3/da/f19 [0,4194304] 0 2024-07-16T09:55:30.869 INFO:tasks.workunit.client.1.smithi176.stdout:7/196: chown d4/f2b 15901972 1 2024-07-16T09:55:30.870 INFO:tasks.workunit.client.1.smithi176.stdout:0/244: readlink d5/d1c/l1d 0 2024-07-16T09:55:30.870 INFO:tasks.workunit.client.1.smithi176.stdout:8/138: mkdir d8/d20/d26/d2f 0 2024-07-16T09:55:30.871 INFO:tasks.workunit.client.1.smithi176.stdout:0/245: write d5/d1c/d24/d35/f4c [978196,99521] 0 2024-07-16T09:55:30.871 INFO:tasks.workunit.client.1.smithi176.stdout:3/116: fdatasync d3/d17/d1d/d1e/f22 0 2024-07-16T09:55:30.872 INFO:tasks.workunit.client.1.smithi176.stdout:7/197: readlink d4/lf 0 2024-07-16T09:55:30.872 INFO:tasks.workunit.client.1.smithi176.stdout:3/117: fsync d3/d17/d1d/f20 0 2024-07-16T09:55:30.874 INFO:tasks.workunit.client.1.smithi176.stdout:6/188: mknod d1/d5/c3f 0 2024-07-16T09:55:30.875 INFO:tasks.workunit.client.0.smithi156.stdout:9/111: getdents d2/d5/d1a/d1b 0 2024-07-16T09:55:30.875 INFO:tasks.workunit.client.1.smithi176.stdout:8/139: symlink d8/l30 0 2024-07-16T09:55:30.876 INFO:tasks.workunit.client.0.smithi156.stdout:9/112: readlink d2/l10 0 2024-07-16T09:55:30.877 INFO:tasks.workunit.client.1.smithi176.stdout:7/198: readlink d4/dd/d26/l3d 0 2024-07-16T09:55:30.877 INFO:tasks.workunit.client.1.smithi176.stdout:8/140: chown d8/c1a 2355 1 2024-07-16T09:55:30.878 INFO:tasks.workunit.client.1.smithi176.stdout:7/199: read - d4/d1f/f2a zero size 2024-07-16T09:55:30.879 INFO:tasks.workunit.client.1.smithi176.stdout:6/189: mkdir d1/dc/d40 0 2024-07-16T09:55:30.880 INFO:tasks.workunit.client.0.smithi156.stdout:9/113: write d2/f15 [660846,19644] 0 2024-07-16T09:55:30.881 INFO:tasks.workunit.client.1.smithi176.stdout:0/246: link d5/d39/f3a d5/d25/f4e 0 2024-07-16T09:55:30.881 INFO:tasks.workunit.client.0.smithi156.stdout:9/114: write d2/f3 [9402915,88363] 0 2024-07-16T09:55:30.881 INFO:tasks.workunit.client.1.smithi176.stdout:3/118: dwrite d3/da/fe [4194304,4194304] 0 2024-07-16T09:55:30.882 INFO:tasks.workunit.client.0.smithi156.stdout:9/115: read d2/f13 [570829,88811] 0 2024-07-16T09:55:30.883 INFO:tasks.workunit.client.0.smithi156.stdout:2/207: write d0/d13/d17/d1c/f27 [272532,73338] 0 2024-07-16T09:55:30.887 INFO:tasks.workunit.client.1.smithi176.stdout:7/200: creat d4/dd/d49/f4c x:0 0 0 2024-07-16T09:55:30.888 INFO:tasks.workunit.client.0.smithi156.stdout:9/116: symlink d2/l21 0 2024-07-16T09:55:30.889 INFO:tasks.workunit.client.1.smithi176.stdout:6/190: mknod d1/dc/d26/c41 0 2024-07-16T09:55:30.890 INFO:tasks.workunit.client.0.smithi156.stdout:9/117: chown d2/d5/fb 43336 1 2024-07-16T09:55:30.891 INFO:tasks.workunit.client.1.smithi176.stdout:7/201: write d4/d47/f44 [259442,63727] 0 2024-07-16T09:55:30.891 INFO:tasks.workunit.client.0.smithi156.stdout:2/208: dwrite d0/f8 [0,4194304] 0 2024-07-16T09:55:30.892 INFO:tasks.workunit.client.1.smithi176.stdout:8/141: rename d8/d20/d26 to d8/d20/d2a/d31 0 2024-07-16T09:55:30.893 INFO:tasks.workunit.client.0.smithi156.stdout:9/118: write d2/d5/ff [988674,124387] 0 2024-07-16T09:55:30.893 INFO:tasks.workunit.client.1.smithi176.stdout:6/191: readlink d1/d5/l8 0 2024-07-16T09:55:30.895 INFO:tasks.workunit.client.1.smithi176.stdout:6/192: dread - d1/dc/f2e zero size 2024-07-16T09:55:30.895 INFO:tasks.workunit.client.1.smithi176.stdout:3/119: getdents d3/d17 0 2024-07-16T09:55:30.896 INFO:tasks.workunit.client.0.smithi156.stdout:2/209: creat d0/d13/d17/d1c/f48 x:0 0 0 2024-07-16T09:55:30.897 INFO:tasks.workunit.client.0.smithi156.stdout:2/210: read d0/d5/f2f [237769,16219] 0 2024-07-16T09:55:30.898 INFO:tasks.workunit.client.0.smithi156.stdout:9/119: rename d2/d5/d1a/d1b to d2/d5/d16/d22 0 2024-07-16T09:55:30.899 INFO:tasks.workunit.client.1.smithi176.stdout:3/120: read d3/d17/d1d/d1e/f1f [3267648,40491] 0 2024-07-16T09:55:30.900 INFO:tasks.workunit.client.0.smithi156.stdout:2/211: mknod d0/d5/d29/d2d/c49 0 2024-07-16T09:55:30.901 INFO:tasks.workunit.client.1.smithi176.stdout:6/193: mkdir d1/d5/d42 0 2024-07-16T09:55:30.901 INFO:tasks.workunit.client.1.smithi176.stdout:7/202: dwrite d4/dd/d49/f4c [0,4194304] 0 2024-07-16T09:55:30.902 INFO:tasks.workunit.client.0.smithi156.stdout:9/120: mkdir d2/d5/d1d/d23 0 2024-07-16T09:55:30.903 INFO:tasks.workunit.client.0.smithi156.stdout:2/212: chown d0/fe 18263272 1 2024-07-16T09:55:30.903 INFO:tasks.workunit.client.1.smithi176.stdout:7/203: write d4/dd/f23 [4634561,125640] 0 2024-07-16T09:55:30.905 INFO:tasks.workunit.client.0.smithi156.stdout:9/121: mknod d2/c24 0 2024-07-16T09:55:30.907 INFO:tasks.workunit.client.1.smithi176.stdout:8/142: link d8/l9 d8/d23/l32 0 2024-07-16T09:55:30.907 INFO:tasks.workunit.client.1.smithi176.stdout:6/194: symlink d1/d5/d7/d2f/l43 0 2024-07-16T09:55:30.908 INFO:tasks.workunit.client.1.smithi176.stdout:7/204: creat d4/d5/d10/d17/f4d x:0 0 0 2024-07-16T09:55:30.909 INFO:tasks.workunit.client.0.smithi156.stdout:9/122: symlink d2/d5/l25 0 2024-07-16T09:55:30.909 INFO:tasks.workunit.client.1.smithi176.stdout:8/143: mkdir d8/d14/d1e/d33 0 2024-07-16T09:55:30.909 INFO:tasks.workunit.client.1.smithi176.stdout:6/195: creat d1/d5/d7/d2f/f44 x:0 0 0 2024-07-16T09:55:30.910 INFO:tasks.workunit.client.1.smithi176.stdout:7/205: mknod d4/d47/c4e 0 2024-07-16T09:55:30.911 INFO:tasks.workunit.client.0.smithi156.stdout:9/123: write d2/f13 [2246574,91999] 0 2024-07-16T09:55:30.912 INFO:tasks.workunit.client.1.smithi176.stdout:8/144: write d8/fa [565654,37442] 0 2024-07-16T09:55:30.912 INFO:tasks.workunit.client.1.smithi176.stdout:7/206: dread d4/f19 [0,4194304] 0 2024-07-16T09:55:30.914 INFO:tasks.workunit.client.1.smithi176.stdout:8/145: mkdir d8/d23/d2c/d34 0 2024-07-16T09:55:30.915 INFO:tasks.workunit.client.1.smithi176.stdout:8/146: dread - d8/d14/d1e/f1f zero size 2024-07-16T09:55:30.916 INFO:tasks.workunit.client.0.smithi156.stdout:2/213: truncate d0/d13/d17/f23 223667 0 2024-07-16T09:55:30.919 INFO:tasks.workunit.client.1.smithi176.stdout:7/207: link d4/dd/d26/l3d d4/d1f/d22/l4f 0 2024-07-16T09:55:30.920 INFO:tasks.workunit.client.0.smithi156.stdout:2/214: mkdir d0/d5/d29/d2d/d4a 0 2024-07-16T09:55:30.920 INFO:tasks.workunit.client.0.smithi156.stdout:9/124: dwrite d2/de/f19 [0,4194304] 0 2024-07-16T09:55:30.921 INFO:tasks.workunit.client.0.smithi156.stdout:9/125: readlink d2/d5/l9 0 2024-07-16T09:55:30.921 INFO:tasks.workunit.client.1.smithi176.stdout:8/147: dread d8/d14/f27 [0,4194304] 0 2024-07-16T09:55:30.922 INFO:tasks.workunit.client.1.smithi176.stdout:7/208: creat d4/d5/d21/f50 x:0 0 0 2024-07-16T09:55:30.926 INFO:tasks.workunit.client.0.smithi156.stdout:2/215: rmdir d0/d5/d29/d2d/d4a 0 2024-07-16T09:55:30.927 INFO:tasks.workunit.client.0.smithi156.stdout:9/126: dread d2/f13 [0,4194304] 0 2024-07-16T09:55:30.928 INFO:tasks.workunit.client.1.smithi176.stdout:8/148: creat d8/d14/d1e/d33/f35 x:0 0 0 2024-07-16T09:55:30.929 INFO:tasks.workunit.client.0.smithi156.stdout:2/216: mkdir d0/d5/d4b 0 2024-07-16T09:55:30.929 INFO:tasks.workunit.client.1.smithi176.stdout:8/149: dread - d8/d14/f2d zero size 2024-07-16T09:55:30.929 INFO:tasks.workunit.client.1.smithi176.stdout:7/209: rmdir d4/d5/d10 39 2024-07-16T09:55:30.930 INFO:tasks.workunit.client.0.smithi156.stdout:9/127: rename d2/d5/d1a to d2/d5/d16/d22/d26 0 2024-07-16T09:55:30.931 INFO:tasks.workunit.client.1.smithi176.stdout:8/150: creat d8/d23/d2c/f36 x:0 0 0 2024-07-16T09:55:30.932 INFO:tasks.workunit.client.1.smithi176.stdout:7/210: mkdir d4/d1f/d22/d51 0 2024-07-16T09:55:30.933 INFO:tasks.workunit.client.0.smithi156.stdout:2/217: rmdir d0/d5/d4b 0 2024-07-16T09:55:30.933 INFO:tasks.workunit.client.0.smithi156.stdout:9/128: dread d2/d5/fb [0,4194304] 0 2024-07-16T09:55:30.934 INFO:tasks.workunit.client.1.smithi176.stdout:7/211: creat d4/d1f/d22/f52 x:0 0 0 2024-07-16T09:55:30.934 INFO:tasks.workunit.client.1.smithi176.stdout:7/212: chown d4 175093232 1 2024-07-16T09:55:30.935 INFO:tasks.workunit.client.0.smithi156.stdout:2/218: rmdir d0/d13 39 2024-07-16T09:55:30.939 INFO:tasks.workunit.client.0.smithi156.stdout:2/219: dread d0/d13/d17/f42 [0,4194304] 0 2024-07-16T09:55:30.942 INFO:tasks.workunit.client.1.smithi176.stdout:8/151: dwrite f5 [0,4194304] 0 2024-07-16T09:55:30.942 INFO:tasks.workunit.client.0.smithi156.stdout:9/129: write d2/f4 [573902,114319] 0 2024-07-16T09:55:30.943 INFO:tasks.workunit.client.1.smithi176.stdout:8/152: readlink d8/d14/l19 0 2024-07-16T09:55:30.944 INFO:tasks.workunit.client.1.smithi176.stdout:7/213: dread d4/fb [0,4194304] 0 2024-07-16T09:55:30.945 INFO:tasks.workunit.client.1.smithi176.stdout:8/153: write d8/d23/f21 [270330,33655] 0 2024-07-16T09:55:30.946 INFO:tasks.workunit.client.1.smithi176.stdout:7/214: mknod d4/d5/d21/c53 0 2024-07-16T09:55:30.946 INFO:tasks.workunit.client.1.smithi176.stdout:8/154: read d8/fa [2917668,108104] 0 2024-07-16T09:55:30.947 INFO:tasks.workunit.client.0.smithi156.stdout:2/220: dwrite d0/d5/d29/f38 [0,4194304] 0 2024-07-16T09:55:30.947 INFO:tasks.workunit.client.1.smithi176.stdout:7/215: write d4/f2b [717950,100407] 0 2024-07-16T09:55:30.951 INFO:tasks.workunit.client.0.smithi156.stdout:2/221: write d0/f7 [2407778,52078] 0 2024-07-16T09:55:30.951 INFO:tasks.workunit.client.0.smithi156.stdout:9/130: getdents d2/d5/d16/d22 0 2024-07-16T09:55:30.952 INFO:tasks.workunit.client.0.smithi156.stdout:2/222: write d0/d5/d29/d2d/f40 [371659,73718] 0 2024-07-16T09:55:30.958 INFO:tasks.workunit.client.0.smithi156.stdout:9/131: dwrite d2/f3 [4194304,4194304] 0 2024-07-16T09:55:30.961 INFO:tasks.workunit.client.0.smithi156.stdout:9/132: rename d2/c1c to d2/d5/c27 0 2024-07-16T09:55:30.989 INFO:tasks.workunit.client.0.smithi156.stdout:2/223: truncate d0/d13/d17/f42 245297 0 2024-07-16T09:55:30.989 INFO:tasks.workunit.client.0.smithi156.stdout:2/224: fdatasync d0/f7 0 2024-07-16T09:55:30.990 INFO:tasks.workunit.client.0.smithi156.stdout:9/133: unlink d2/d5/c27 0 2024-07-16T09:55:30.991 INFO:tasks.workunit.client.0.smithi156.stdout:2/225: mknod d0/d5/d29/d2d/c4c 0 2024-07-16T09:55:30.992 INFO:tasks.workunit.client.0.smithi156.stdout:9/134: symlink d2/d5/d16/d22/d26/l28 0 2024-07-16T09:55:30.993 INFO:tasks.workunit.client.0.smithi156.stdout:9/135: symlink d2/d5/l29 0 2024-07-16T09:55:30.994 INFO:tasks.workunit.client.0.smithi156.stdout:4/119: sync 2024-07-16T09:55:30.995 INFO:tasks.workunit.client.0.smithi156.stdout:9/136: creat d2/d5/d16/f2a x:0 0 0 2024-07-16T09:55:30.996 INFO:tasks.workunit.client.0.smithi156.stdout:2/226: rmdir d0/d5 39 2024-07-16T09:55:30.998 INFO:tasks.workunit.client.0.smithi156.stdout:4/120: getdents d4/db/dc 0 2024-07-16T09:55:30.998 INFO:tasks.workunit.client.0.smithi156.stdout:2/227: mknod d0/d13/c4d 0 2024-07-16T09:55:31.001 INFO:tasks.workunit.client.0.smithi156.stdout:4/121: dread d4/db/f2c [0,4194304] 0 2024-07-16T09:55:31.001 INFO:tasks.workunit.client.0.smithi156.stdout:2/228: symlink d0/d5/l4e 0 2024-07-16T09:55:31.027 INFO:tasks.workunit.client.1.smithi176.stdout:8/155: readlink d8/d23/l32 0 2024-07-16T09:55:31.028 INFO:tasks.workunit.client.1.smithi176.stdout:7/216: stat d4/d5/d10/d17 0 2024-07-16T09:55:31.029 INFO:tasks.workunit.client.1.smithi176.stdout:7/217: dread - d4/d1f/d22/f52 zero size 2024-07-16T09:55:31.029 INFO:tasks.workunit.client.1.smithi176.stdout:8/156: rmdir d8/d20/d2a 39 2024-07-16T09:55:31.030 INFO:tasks.workunit.client.1.smithi176.stdout:7/218: write d4/dd/f15 [1399944,124976] 0 2024-07-16T09:55:31.031 INFO:tasks.workunit.client.1.smithi176.stdout:8/157: mkdir d8/d23/d2c/d37 0 2024-07-16T09:55:31.033 INFO:tasks.workunit.client.1.smithi176.stdout:8/158: mknod d8/d14/d1e/d33/c38 0 2024-07-16T09:55:31.033 INFO:tasks.workunit.client.1.smithi176.stdout:7/219: dread d4/dd/f15 [0,4194304] 0 2024-07-16T09:55:31.035 INFO:tasks.workunit.client.1.smithi176.stdout:7/220: dread d4/f19 [0,4194304] 0 2024-07-16T09:55:31.039 INFO:tasks.workunit.client.1.smithi176.stdout:8/159: dwrite d8/d14/f2d [0,4194304] 0 2024-07-16T09:55:31.041 INFO:tasks.workunit.client.1.smithi176.stdout:7/221: rename d4/d5/d10/d17/c4a to d4/dd/c54 0 2024-07-16T09:55:31.041 INFO:tasks.workunit.client.1.smithi176.stdout:7/222: chown d4/c9 3019471 1 2024-07-16T09:55:31.042 INFO:tasks.workunit.client.1.smithi176.stdout:8/160: symlink d8/d23/l39 0 2024-07-16T09:55:31.045 INFO:tasks.workunit.client.1.smithi176.stdout:7/223: link d4/dd/f37 d4/dd/d49/f55 0 2024-07-16T09:55:31.046 INFO:tasks.workunit.client.1.smithi176.stdout:8/161: symlink d8/d20/d2a/l3a 0 2024-07-16T09:55:31.047 INFO:tasks.workunit.client.1.smithi176.stdout:7/224: symlink d4/l56 0 2024-07-16T09:55:31.048 INFO:tasks.workunit.client.1.smithi176.stdout:8/162: creat d8/d23/d2c/f3b x:0 0 0 2024-07-16T09:55:31.049 INFO:tasks.workunit.client.1.smithi176.stdout:8/163: dread d8/f17 [0,4194304] 0 2024-07-16T09:55:31.051 INFO:tasks.workunit.client.1.smithi176.stdout:7/225: dread d4/d5/f31 [0,4194304] 0 2024-07-16T09:55:31.053 INFO:tasks.workunit.client.1.smithi176.stdout:7/226: rmdir d4/d5/d10/d17/d1d 39 2024-07-16T09:55:31.054 INFO:tasks.workunit.client.1.smithi176.stdout:7/227: fsync d4/dd/d49/f4c 0 2024-07-16T09:55:31.055 INFO:tasks.workunit.client.1.smithi176.stdout:7/228: rename d4 to d4/d1f/d57 22 2024-07-16T09:55:31.056 INFO:tasks.workunit.client.1.smithi176.stdout:7/229: fdatasync d4/d5/d10/d17/f4d 0 2024-07-16T09:55:31.058 INFO:tasks.workunit.client.1.smithi176.stdout:8/164: write d8/f17 [209886,36978] 0 2024-07-16T09:55:31.061 INFO:tasks.workunit.client.1.smithi176.stdout:8/165: creat d8/d23/d2c/d37/f3c x:0 0 0 2024-07-16T09:55:31.063 INFO:tasks.workunit.client.1.smithi176.stdout:8/166: read - d8/d14/d1e/d33/f35 zero size 2024-07-16T09:55:31.063 INFO:tasks.workunit.client.1.smithi176.stdout:7/230: dwrite d4/d1f/d22/f52 [0,4194304] 0 2024-07-16T09:55:31.066 INFO:tasks.workunit.client.1.smithi176.stdout:7/231: fdatasync d4/f48 0 2024-07-16T09:55:31.070 INFO:tasks.workunit.client.1.smithi176.stdout:7/232: chown d4/d1f/d22 40042808 1 2024-07-16T09:55:31.070 INFO:tasks.workunit.client.1.smithi176.stdout:7/233: fdatasync d4/f2b 0 2024-07-16T09:55:31.070 INFO:tasks.workunit.client.1.smithi176.stdout:7/234: rmdir d4/d1f 39 2024-07-16T09:55:31.073 INFO:tasks.workunit.client.1.smithi176.stdout:8/167: dwrite d8/d23/d2c/f3b [0,4194304] 0 2024-07-16T09:55:31.073 INFO:tasks.workunit.client.1.smithi176.stdout:7/235: unlink d4/d5/d10/f3f 0 2024-07-16T09:55:31.077 INFO:tasks.workunit.client.1.smithi176.stdout:8/168: truncate d8/fa 329818 0 2024-07-16T09:55:31.083 INFO:tasks.workunit.client.1.smithi176.stdout:7/236: dwrite d4/d47/f44 [0,4194304] 0 2024-07-16T09:55:31.086 INFO:tasks.workunit.client.1.smithi176.stdout:7/237: chown d4/d1f/d22 114583259 1 2024-07-16T09:55:31.088 INFO:tasks.workunit.client.1.smithi176.stdout:8/169: dwrite d8/d14/d1e/d33/f35 [0,4194304] 0 2024-07-16T09:55:31.093 INFO:tasks.workunit.client.1.smithi176.stdout:7/238: creat d4/d1f/d22/d51/f58 x:0 0 0 2024-07-16T09:55:31.093 INFO:tasks.workunit.client.1.smithi176.stdout:8/170: creat d8/f3d x:0 0 0 2024-07-16T09:55:31.094 INFO:tasks.workunit.client.1.smithi176.stdout:8/171: chown d8/d20/d2a/d31/d2f 11 1 2024-07-16T09:55:31.095 INFO:tasks.workunit.client.1.smithi176.stdout:7/239: write d4/d47/f46 [1624478,44682] 0 2024-07-16T09:55:31.096 INFO:tasks.workunit.client.1.smithi176.stdout:7/240: chown d4/d47/f44 443740 1 2024-07-16T09:55:31.099 INFO:tasks.workunit.client.1.smithi176.stdout:7/241: creat d4/d1f/f59 x:0 0 0 2024-07-16T09:55:31.100 INFO:tasks.workunit.client.1.smithi176.stdout:8/172: unlink d8/d23/f15 0 2024-07-16T09:55:31.102 INFO:tasks.workunit.client.1.smithi176.stdout:8/173: chown d8/d14/f27 1 1 2024-07-16T09:55:31.107 INFO:tasks.workunit.client.1.smithi176.stdout:7/242: dwrite d4/f27 [0,4194304] 0 2024-07-16T09:55:31.110 INFO:tasks.workunit.client.1.smithi176.stdout:8/174: dwrite d8/f3d [0,4194304] 0 2024-07-16T09:55:31.113 INFO:tasks.workunit.client.1.smithi176.stdout:8/175: write d8/d23/d2c/f36 [586623,125401] 0 2024-07-16T09:55:31.116 INFO:tasks.workunit.client.1.smithi176.stdout:7/243: unlink d4/d47/l45 0 2024-07-16T09:55:31.117 INFO:tasks.workunit.client.1.smithi176.stdout:8/176: creat d8/d20/d2a/d31/f3e x:0 0 0 2024-07-16T09:55:31.118 INFO:tasks.workunit.client.1.smithi176.stdout:7/244: creat d4/d5/d21/f5a x:0 0 0 2024-07-16T09:55:31.118 INFO:tasks.workunit.client.1.smithi176.stdout:8/177: write d8/d14/d1e/f1f [929217,94548] 0 2024-07-16T09:55:31.120 INFO:tasks.workunit.client.1.smithi176.stdout:8/178: chown d8/d23/d2c/d37 2988091 1 2024-07-16T09:55:31.120 INFO:tasks.workunit.client.1.smithi176.stdout:8/179: chown d8/f2e 22161 1 2024-07-16T09:55:31.121 INFO:tasks.workunit.client.1.smithi176.stdout:7/245: unlink d4/f48 0 2024-07-16T09:55:31.124 INFO:tasks.workunit.client.1.smithi176.stdout:7/246: creat d4/d5/d10/d17/d1d/f5b x:0 0 0 2024-07-16T09:55:31.124 INFO:tasks.workunit.client.1.smithi176.stdout:8/180: getdents d8/d20 0 2024-07-16T09:55:31.124 INFO:tasks.workunit.client.1.smithi176.stdout:7/247: chown d4/dd/l35 15 1 2024-07-16T09:55:31.126 INFO:tasks.workunit.client.1.smithi176.stdout:8/181: truncate d8/d23/f25 698295 0 2024-07-16T09:55:31.127 INFO:tasks.workunit.client.1.smithi176.stdout:8/182: read - d8/d20/d2a/d31/f3e zero size 2024-07-16T09:55:31.127 INFO:tasks.workunit.client.1.smithi176.stdout:8/183: readlink d8/l1d 0 2024-07-16T09:55:31.147 INFO:tasks.workunit.client.1.smithi176.stdout:9/153: sync 2024-07-16T09:55:31.147 INFO:tasks.workunit.client.1.smithi176.stdout:4/195: sync 2024-07-16T09:55:31.148 INFO:tasks.workunit.client.1.smithi176.stdout:8/184: creat d8/d20/d2a/f3f x:0 0 0 2024-07-16T09:55:31.149 INFO:tasks.workunit.client.1.smithi176.stdout:7/248: dwrite d4/d5/d21/f50 [0,4194304] 0 2024-07-16T09:55:31.151 INFO:tasks.workunit.client.1.smithi176.stdout:7/249: readlink d4/dd/d26/l43 0 2024-07-16T09:55:31.153 INFO:tasks.workunit.client.1.smithi176.stdout:8/185: rmdir d8/d23/d2c/d37 39 2024-07-16T09:55:31.155 INFO:tasks.workunit.client.1.smithi176.stdout:8/186: write d8/f17 [799345,51336] 0 2024-07-16T09:55:31.156 INFO:tasks.workunit.client.1.smithi176.stdout:4/196: rename da/dc/d10/d15/d17/l1f to da/dc/l36 0 2024-07-16T09:55:31.160 INFO:tasks.workunit.client.1.smithi176.stdout:9/154: dwrite d5/de/f12 [4194304,4194304] 0 2024-07-16T09:55:31.162 INFO:tasks.workunit.client.1.smithi176.stdout:9/155: readlink d5/de/l18 0 2024-07-16T09:55:31.162 INFO:tasks.workunit.client.1.smithi176.stdout:7/250: dwrite d4/d1f/f59 [0,4194304] 0 2024-07-16T09:55:31.166 INFO:tasks.workunit.client.1.smithi176.stdout:9/156: mknod d5/d1d/d20/c2c 0 2024-07-16T09:55:31.168 INFO:tasks.workunit.client.0.smithi156.stdout:8/68: sync 2024-07-16T09:55:31.168 INFO:tasks.workunit.client.0.smithi156.stdout:1/99: sync 2024-07-16T09:55:31.171 INFO:tasks.workunit.client.1.smithi176.stdout:9/157: creat d5/f2d x:0 0 0 2024-07-16T09:55:31.173 INFO:tasks.workunit.client.0.smithi156.stdout:8/69: dread d0/dd/f10 [0,4194304] 0 2024-07-16T09:55:31.173 INFO:tasks.workunit.client.1.smithi176.stdout:4/197: dwrite da/dc/d10/d15/d17/f35 [0,4194304] 0 2024-07-16T09:55:31.173 INFO:tasks.workunit.client.1.smithi176.stdout:8/187: dwrite d8/d20/d2a/d31/f3e [0,4194304] 0 2024-07-16T09:55:31.175 INFO:tasks.workunit.client.1.smithi176.stdout:7/251: dread d4/dd/d49/f4c [0,4194304] 0 2024-07-16T09:55:31.176 INFO:tasks.workunit.client.1.smithi176.stdout:4/198: read da/dc/d10/d15/f1b [504809,25984] 0 2024-07-16T09:55:31.179 INFO:tasks.workunit.client.1.smithi176.stdout:4/199: chown da/cd 603612 1 2024-07-16T09:55:31.181 INFO:tasks.workunit.client.1.smithi176.stdout:4/200: truncate da/d14/f29 430207 0 2024-07-16T09:55:31.183 INFO:tasks.workunit.client.1.smithi176.stdout:9/158: read d5/de/f17 [953169,103504] 0 2024-07-16T09:55:31.184 INFO:tasks.workunit.client.1.smithi176.stdout:8/188: rename d8/c11 to d8/d14/c40 0 2024-07-16T09:55:31.184 INFO:tasks.workunit.client.1.smithi176.stdout:8/189: stat d8/d23/f25 0 2024-07-16T09:55:31.187 INFO:tasks.workunit.client.1.smithi176.stdout:9/159: write d5/fd [624193,6863] 0 2024-07-16T09:55:31.187 INFO:tasks.workunit.client.1.smithi176.stdout:8/190: mknod d8/d23/d2c/d34/c41 0 2024-07-16T09:55:31.190 INFO:tasks.workunit.client.1.smithi176.stdout:8/191: mknod d8/d20/d2a/d31/c42 0 2024-07-16T09:55:31.191 INFO:tasks.workunit.client.1.smithi176.stdout:4/201: dwrite da/d14/f26 [0,4194304] 0 2024-07-16T09:55:31.198 INFO:tasks.workunit.client.1.smithi176.stdout:8/192: creat d8/d20/d2a/d31/d2f/f43 x:0 0 0 2024-07-16T09:55:31.200 INFO:tasks.workunit.client.1.smithi176.stdout:4/202: write da/fb [4819834,54602] 0 2024-07-16T09:55:31.200 INFO:tasks.workunit.client.1.smithi176.stdout:9/160: dwrite d5/d1d/d20/f25 [0,4194304] 0 2024-07-16T09:55:31.203 INFO:tasks.workunit.client.1.smithi176.stdout:4/203: write da/d14/f26 [3146250,47087] 0 2024-07-16T09:55:31.203 INFO:tasks.workunit.client.1.smithi176.stdout:8/193: mknod d8/d23/c44 0 2024-07-16T09:55:31.206 INFO:tasks.workunit.client.1.smithi176.stdout:4/204: fdatasync da/dc/d10/d15/d17/f18 0 2024-07-16T09:55:31.209 INFO:tasks.workunit.client.1.smithi176.stdout:8/194: getdents d8/d23/d2c 0 2024-07-16T09:55:31.210 INFO:tasks.workunit.client.1.smithi176.stdout:8/195: chown d8/l1c 11 1 2024-07-16T09:55:31.212 INFO:tasks.workunit.client.1.smithi176.stdout:9/161: dwrite d5/d1d/d20/f25 [0,4194304] 0 2024-07-16T09:55:31.215 INFO:tasks.workunit.client.1.smithi176.stdout:8/196: creat d8/d23/d2c/d37/f45 x:0 0 0 2024-07-16T09:55:31.216 INFO:tasks.workunit.client.1.smithi176.stdout:8/197: chown d8 839 1 2024-07-16T09:55:31.217 INFO:tasks.workunit.client.1.smithi176.stdout:8/198: read f5 [3189789,85817] 0 2024-07-16T09:55:31.218 INFO:tasks.workunit.client.1.smithi176.stdout:9/162: symlink d5/d1d/l2e 0 2024-07-16T09:55:31.220 INFO:tasks.workunit.client.1.smithi176.stdout:4/205: dread da/dc/d10/d15/d17/f18 [0,4194304] 0 2024-07-16T09:55:31.221 INFO:tasks.workunit.client.1.smithi176.stdout:9/163: creat d5/de/f2f x:0 0 0 2024-07-16T09:55:31.223 INFO:tasks.workunit.client.1.smithi176.stdout:8/199: dread d8/d14/d1e/f1f [0,4194304] 0 2024-07-16T09:55:31.226 INFO:tasks.workunit.client.1.smithi176.stdout:8/200: mkdir d8/d20/d2a/d31/d46 0 2024-07-16T09:55:31.229 INFO:tasks.workunit.client.1.smithi176.stdout:4/206: dwrite da/dc/d10/d15/d17/f31 [0,4194304] 0 2024-07-16T09:55:31.230 INFO:tasks.workunit.client.1.smithi176.stdout:8/201: rename d8/fd to d8/d14/d1e/d33/f47 0 2024-07-16T09:55:31.231 INFO:tasks.workunit.client.1.smithi176.stdout:4/207: write da/dc/d10/f27 [2995310,64683] 0 2024-07-16T09:55:31.231 INFO:tasks.workunit.client.1.smithi176.stdout:4/208: readlink l9 0 2024-07-16T09:55:31.234 INFO:tasks.workunit.client.1.smithi176.stdout:4/209: unlink da/d14/l1d 0 2024-07-16T09:55:31.235 INFO:tasks.workunit.client.1.smithi176.stdout:8/202: symlink d8/d23/d2c/l48 0 2024-07-16T09:55:31.236 INFO:tasks.workunit.client.1.smithi176.stdout:8/203: dread d8/d14/f27 [0,4194304] 0 2024-07-16T09:55:31.241 INFO:tasks.workunit.client.1.smithi176.stdout:4/210: rename da/c13 to da/dc/d10/d15/c37 0 2024-07-16T09:55:31.242 INFO:tasks.workunit.client.1.smithi176.stdout:4/211: creat da/dc/d10/d15/d17/f38 x:0 0 0 2024-07-16T09:55:31.243 INFO:tasks.workunit.client.1.smithi176.stdout:9/164: dread d5/f16 [0,4194304] 0 2024-07-16T09:55:31.244 INFO:tasks.workunit.client.1.smithi176.stdout:9/165: rmdir d5 39 2024-07-16T09:55:31.245 INFO:tasks.workunit.client.1.smithi176.stdout:4/212: read da/f19 [1482205,19688] 0 2024-07-16T09:55:31.247 INFO:tasks.workunit.client.0.smithi156.stdout:2/229: rmdir d0/d5/d29 39 2024-07-16T09:55:31.248 INFO:tasks.workunit.client.1.smithi176.stdout:9/166: mkdir d5/de/d30 0 2024-07-16T09:55:31.249 INFO:tasks.workunit.client.1.smithi176.stdout:8/204: unlink d8/d23/d2c/d37/f45 0 2024-07-16T09:55:31.249 INFO:tasks.workunit.client.0.smithi156.stdout:2/230: write d0/d5/f1e [126514,24318] 0 2024-07-16T09:55:31.249 INFO:tasks.workunit.client.0.smithi156.stdout:8/70: creat d0/dd/f19 x:0 0 0 2024-07-16T09:55:31.250 INFO:tasks.workunit.client.1.smithi176.stdout:4/213: dread da/dc/d10/d15/d17/f31 [0,4194304] 0 2024-07-16T09:55:31.252 INFO:tasks.workunit.client.1.smithi176.stdout:9/167: creat d5/d1d/d20/f31 x:0 0 0 2024-07-16T09:55:31.252 INFO:tasks.workunit.client.0.smithi156.stdout:4/122: getdents d4/d23 0 2024-07-16T09:55:31.253 INFO:tasks.workunit.client.1.smithi176.stdout:9/168: readlink d5/de/l18 0 2024-07-16T09:55:31.254 INFO:tasks.workunit.client.1.smithi176.stdout:8/205: link d8/f2e d8/d20/d2a/f49 0 2024-07-16T09:55:31.254 INFO:tasks.workunit.client.1.smithi176.stdout:4/214: mkdir da/dc/d39 0 2024-07-16T09:55:31.256 INFO:tasks.workunit.client.0.smithi156.stdout:2/231: mknod d0/d5/d29/d2d/c4f 0 2024-07-16T09:55:31.256 INFO:tasks.workunit.client.0.smithi156.stdout:4/123: dread d4/db/f2c [0,4194304] 0 2024-07-16T09:55:31.257 INFO:tasks.workunit.client.1.smithi176.stdout:8/206: rename f6 to d8/d23/f4a 0 2024-07-16T09:55:31.258 INFO:tasks.workunit.client.0.smithi156.stdout:2/232: creat d0/d13/f50 x:0 0 0 2024-07-16T09:55:31.260 INFO:tasks.workunit.client.1.smithi176.stdout:9/169: dwrite d5/f16 [0,4194304] 0 2024-07-16T09:55:31.260 INFO:tasks.workunit.client.0.smithi156.stdout:4/124: creat d4/db/dc/d22/f2d x:0 0 0 2024-07-16T09:55:31.260 INFO:tasks.workunit.client.1.smithi176.stdout:4/215: link da/dc/d10/f27 da/dc/d10/d15/d33/f3a 0 2024-07-16T09:55:31.261 INFO:tasks.workunit.client.0.smithi156.stdout:2/233: unlink d0/l2 0 2024-07-16T09:55:31.262 INFO:tasks.workunit.client.0.smithi156.stdout:2/234: fsync d0/f16 0 2024-07-16T09:55:31.263 INFO:tasks.workunit.client.0.smithi156.stdout:4/125: creat d4/db/dc/f2e x:0 0 0 2024-07-16T09:55:31.263 INFO:tasks.workunit.client.1.smithi176.stdout:9/170: write d5/de/f1a [2585323,125985] 0 2024-07-16T09:55:31.264 INFO:tasks.workunit.client.1.smithi176.stdout:4/216: creat da/dc/d10/d15/f3b x:0 0 0 2024-07-16T09:55:31.264 INFO:tasks.workunit.client.0.smithi156.stdout:2/235: chown d0/d5/f2a 7918 1 2024-07-16T09:55:31.265 INFO:tasks.workunit.client.0.smithi156.stdout:4/126: mknod d4/d23/c2f 0 2024-07-16T09:55:31.266 INFO:tasks.workunit.client.1.smithi176.stdout:4/217: write da/d14/f29 [135540,22660] 0 2024-07-16T09:55:31.267 INFO:tasks.workunit.client.0.smithi156.stdout:2/236: mkdir d0/d13/d17/d1c/d51 0 2024-07-16T09:55:31.269 INFO:tasks.workunit.client.1.smithi176.stdout:8/207: dwrite d8/d20/d2a/d31/d2f/f43 [0,4194304] 0 2024-07-16T09:55:31.274 INFO:tasks.workunit.client.1.smithi176.stdout:9/171: link d5/l14 d5/de/d21/l32 0 2024-07-16T09:55:31.275 INFO:tasks.workunit.client.1.smithi176.stdout:8/208: truncate d8/d23/d2c/f36 735456 0 2024-07-16T09:55:31.276 INFO:tasks.workunit.client.0.smithi156.stdout:2/237: dwrite d0/d13/d17/d1c/f48 [0,4194304] 0 2024-07-16T09:55:31.277 INFO:tasks.workunit.client.1.smithi176.stdout:9/172: write d5/de/f1a [228018,41100] 0 2024-07-16T09:55:31.277 INFO:tasks.workunit.client.1.smithi176.stdout:4/218: dread da/f1a [0,4194304] 0 2024-07-16T09:55:31.277 INFO:tasks.workunit.client.1.smithi176.stdout:8/209: rmdir d8/d14 39 2024-07-16T09:55:31.277 INFO:tasks.workunit.client.1.smithi176.stdout:9/173: write d5/f16 [2955448,130534] 0 2024-07-16T09:55:31.278 INFO:tasks.workunit.client.0.smithi156.stdout:2/238: dread - d0/d13/d17/f30 zero size 2024-07-16T09:55:31.279 INFO:tasks.workunit.client.1.smithi176.stdout:9/174: chown d5/d1d/f27 5 1 2024-07-16T09:55:31.279 INFO:tasks.workunit.client.0.smithi156.stdout:2/239: stat d0/f16 0 2024-07-16T09:55:31.279 INFO:tasks.workunit.client.1.smithi176.stdout:4/219: truncate da/dc/d10/d15/d17/f38 507436 0 2024-07-16T09:55:31.280 INFO:tasks.workunit.client.0.smithi156.stdout:2/240: dread - d0/d13/d17/f30 zero size 2024-07-16T09:55:31.283 INFO:tasks.workunit.client.0.smithi156.stdout:2/241: unlink d0/f2c 0 2024-07-16T09:55:31.288 INFO:tasks.workunit.client.1.smithi176.stdout:8/210: creat d8/f4b x:0 0 0 2024-07-16T09:55:31.289 INFO:tasks.workunit.client.1.smithi176.stdout:8/211: fsync d8/d23/d2c/d37/f3c 0 2024-07-16T09:55:31.292 INFO:tasks.workunit.client.1.smithi176.stdout:8/212: creat d8/f4c x:0 0 0 2024-07-16T09:55:31.294 INFO:tasks.workunit.client.1.smithi176.stdout:4/220: dwrite da/dc/d10/f27 [0,4194304] 0 2024-07-16T09:55:31.297 INFO:tasks.workunit.client.1.smithi176.stdout:4/221: creat da/dc/d39/f3c x:0 0 0 2024-07-16T09:55:31.298 INFO:tasks.workunit.client.1.smithi176.stdout:4/222: rmdir da/d14 39 2024-07-16T09:55:31.299 INFO:tasks.workunit.client.1.smithi176.stdout:4/223: fsync da/fb 0 2024-07-16T09:55:31.302 INFO:tasks.workunit.client.1.smithi176.stdout:8/213: write d8/f2e [343282,23137] 0 2024-07-16T09:55:31.303 INFO:tasks.workunit.client.1.smithi176.stdout:8/214: fdatasync d8/d20/d2a/d31/d2f/f43 0 2024-07-16T09:55:31.304 INFO:tasks.workunit.client.1.smithi176.stdout:4/224: rename da/dc/d10/d15 to da/d3d 0 2024-07-16T09:55:31.306 INFO:tasks.workunit.client.1.smithi176.stdout:4/225: readlink da/d14/l2f 0 2024-07-16T09:55:31.307 INFO:tasks.workunit.client.1.smithi176.stdout:4/226: stat da/dc/fe 0 2024-07-16T09:55:31.308 INFO:tasks.workunit.client.1.smithi176.stdout:8/215: link d8/f2e d8/d20/d2a/d31/d2f/f4d 0 2024-07-16T09:55:31.309 INFO:tasks.workunit.client.1.smithi176.stdout:4/227: write da/dc/fe [1849906,109718] 0 2024-07-16T09:55:31.312 INFO:tasks.workunit.client.1.smithi176.stdout:4/228: dread da/d3d/d17/f38 [0,4194304] 0 2024-07-16T09:55:31.315 INFO:tasks.workunit.client.1.smithi176.stdout:4/229: rename da/d3d/f3b to da/d3d/f3e 0 2024-07-16T09:55:31.316 INFO:tasks.workunit.client.1.smithi176.stdout:8/216: dwrite d8/d23/d2c/d37/f3c [0,4194304] 0 2024-07-16T09:55:31.317 INFO:tasks.workunit.client.1.smithi176.stdout:4/230: rmdir da/dc/d39 39 2024-07-16T09:55:31.318 INFO:tasks.workunit.client.0.smithi156.stdout:5/108: sync 2024-07-16T09:55:31.319 INFO:tasks.workunit.client.1.smithi176.stdout:8/217: mknod d8/c4e 0 2024-07-16T09:55:31.322 INFO:tasks.workunit.client.1.smithi176.stdout:4/231: creat da/f3f x:0 0 0 2024-07-16T09:55:31.323 INFO:tasks.workunit.client.0.smithi156.stdout:5/109: creat d1b/d1f/f20 x:0 0 0 2024-07-16T09:55:31.326 INFO:tasks.workunit.client.1.smithi176.stdout:4/232: dread da/d3d/d17/f35 [0,4194304] 0 2024-07-16T09:55:31.327 INFO:tasks.workunit.client.0.smithi156.stdout:5/110: creat d1b/d1e/f21 x:0 0 0 2024-07-16T09:55:31.327 INFO:tasks.workunit.client.0.smithi156.stdout:5/111: stat c18 0 2024-07-16T09:55:31.328 INFO:tasks.workunit.client.0.smithi156.stdout:5/112: rename l7 to d1b/d1e/l22 0 2024-07-16T09:55:31.328 INFO:tasks.workunit.client.0.smithi156.stdout:5/113: write f14 [2520969,4686] 0 2024-07-16T09:55:31.330 INFO:tasks.workunit.client.0.smithi156.stdout:5/114: creat d1b/f23 x:0 0 0 2024-07-16T09:55:31.331 INFO:tasks.workunit.client.1.smithi176.stdout:8/218: write d8/d20/d2a/f49 [161907,84951] 0 2024-07-16T09:55:31.331 INFO:tasks.workunit.client.0.smithi156.stdout:5/115: truncate f19 5013575 0 2024-07-16T09:55:31.332 INFO:tasks.workunit.client.0.smithi156.stdout:5/116: mknod d1b/d1f/c24 0 2024-07-16T09:55:31.333 INFO:tasks.workunit.client.1.smithi176.stdout:8/219: creat d8/d23/d2c/d37/f4f x:0 0 0 2024-07-16T09:55:31.334 INFO:tasks.workunit.client.0.smithi156.stdout:8/71: write d0/f2 [5179422,93215] 0 2024-07-16T09:55:31.334 INFO:tasks.workunit.client.0.smithi156.stdout:5/117: write fc [1008551,108912] 0 2024-07-16T09:55:31.335 INFO:tasks.workunit.client.0.smithi156.stdout:4/127: rmdir d4/db/dc/d22 39 2024-07-16T09:55:31.336 INFO:tasks.workunit.client.1.smithi176.stdout:8/220: rename d8/c1a to d8/d23/d2c/d37/c50 0 2024-07-16T09:55:31.337 INFO:tasks.workunit.client.0.smithi156.stdout:4/128: chown d4/d23/l28 1 1 2024-07-16T09:55:31.337 INFO:tasks.workunit.client.0.smithi156.stdout:8/72: creat d0/dd/f1a x:0 0 0 2024-07-16T09:55:31.338 INFO:tasks.workunit.client.1.smithi176.stdout:8/221: truncate d8/d20/d2a/d31/d2f/f4d 821227 0 2024-07-16T09:55:31.340 INFO:tasks.workunit.client.1.smithi176.stdout:8/222: dread d8/d23/f21 [0,4194304] 0 2024-07-16T09:55:31.343 INFO:tasks.workunit.client.0.smithi156.stdout:8/73: truncate d0/fc 983786 0 2024-07-16T09:55:31.343 INFO:tasks.workunit.client.0.smithi156.stdout:1/100: dwrite dc/d14/f17 [0,4194304] 0 2024-07-16T09:55:31.344 INFO:tasks.workunit.client.0.smithi156.stdout:3/86: sync 2024-07-16T09:55:31.344 INFO:tasks.workunit.client.0.smithi156.stdout:0/47: sync 2024-07-16T09:55:31.344 INFO:tasks.workunit.client.0.smithi156.stdout:6/147: sync 2024-07-16T09:55:31.345 INFO:tasks.workunit.client.0.smithi156.stdout:9/137: sync 2024-07-16T09:55:31.345 INFO:tasks.workunit.client.0.smithi156.stdout:7/90: sync 2024-07-16T09:55:31.345 INFO:tasks.workunit.client.1.smithi176.stdout:8/223: write f5 [467232,98066] 0 2024-07-16T09:55:31.348 INFO:tasks.workunit.client.0.smithi156.stdout:8/74: mknod d0/c1b 0 2024-07-16T09:55:31.349 INFO:tasks.workunit.client.1.smithi176.stdout:8/224: symlink d8/d23/d2c/d34/l51 0 2024-07-16T09:55:31.350 INFO:tasks.workunit.client.0.smithi156.stdout:0/48: chown d0/d7/f8 18387667 1 2024-07-16T09:55:31.350 INFO:tasks.workunit.client.0.smithi156.stdout:4/129: rename d4/db/dc/d22 to d4/d23/d30 0 2024-07-16T09:55:31.351 INFO:tasks.workunit.client.1.smithi176.stdout:8/225: truncate d8/d20/f2b 558327 0 2024-07-16T09:55:31.352 INFO:tasks.workunit.client.0.smithi156.stdout:1/101: symlink dc/d20/l25 0 2024-07-16T09:55:31.352 INFO:tasks.workunit.client.0.smithi156.stdout:2/242: dwrite d0/d13/d17/f42 [0,4194304] 0 2024-07-16T09:55:31.352 INFO:tasks.workunit.client.0.smithi156.stdout:5/118: dwrite d1b/d1f/f20 [0,4194304] 0 2024-07-16T09:55:31.353 INFO:tasks.workunit.client.0.smithi156.stdout:3/87: symlink d8/d14/l1b 0 2024-07-16T09:55:31.354 INFO:tasks.workunit.client.1.smithi176.stdout:8/226: mknod d8/d14/d1e/c52 0 2024-07-16T09:55:31.355 INFO:tasks.workunit.client.0.smithi156.stdout:7/91: mkdir dc/d13/d24 0 2024-07-16T09:55:31.355 INFO:tasks.workunit.client.0.smithi156.stdout:5/119: fdatasync f13 0 2024-07-16T09:55:31.356 INFO:tasks.workunit.client.0.smithi156.stdout:6/148: truncate d3/d26/d1f/fe 439693 0 2024-07-16T09:55:31.357 INFO:tasks.workunit.client.1.smithi176.stdout:8/227: mkdir d8/d20/d2a/d53 0 2024-07-16T09:55:31.361 INFO:tasks.workunit.client.0.smithi156.stdout:5/120: dread - d1b/f23 zero size 2024-07-16T09:55:31.362 INFO:tasks.workunit.client.0.smithi156.stdout:8/75: mkdir d0/d11/d13/d1c 0 2024-07-16T09:55:31.365 INFO:tasks.workunit.client.0.smithi156.stdout:3/88: creat d8/d14/f1c x:0 0 0 2024-07-16T09:55:31.366 INFO:tasks.workunit.client.0.smithi156.stdout:6/149: symlink d3/d26/d1f/d19/l2f 0 2024-07-16T09:55:31.366 INFO:tasks.workunit.client.0.smithi156.stdout:4/130: dread d4/db/f16 [0,4194304] 0 2024-07-16T09:55:31.366 INFO:tasks.workunit.client.0.smithi156.stdout:9/138: dread d2/de/f19 [0,4194304] 0 2024-07-16T09:55:31.369 INFO:tasks.workunit.client.0.smithi156.stdout:9/139: chown d2/f3 968 1 2024-07-16T09:55:31.369 INFO:tasks.workunit.client.0.smithi156.stdout:5/121: rename d1b/f23 to d1b/d1e/f25 0 2024-07-16T09:55:31.369 INFO:tasks.workunit.client.0.smithi156.stdout:8/76: dread d0/fc [0,4194304] 0 2024-07-16T09:55:31.370 INFO:tasks.workunit.client.0.smithi156.stdout:2/243: chown d0/d13/f26 97856 1 2024-07-16T09:55:31.370 INFO:tasks.workunit.client.0.smithi156.stdout:8/77: readlink d0/la 0 2024-07-16T09:55:31.371 INFO:tasks.workunit.client.0.smithi156.stdout:4/131: mknod d4/c31 0 2024-07-16T09:55:31.371 INFO:tasks.workunit.client.0.smithi156.stdout:6/150: rmdir d3/d26/d1f 39 2024-07-16T09:55:31.372 INFO:tasks.workunit.client.0.smithi156.stdout:3/89: write d8/fb [4505445,78574] 0 2024-07-16T09:55:31.372 INFO:tasks.workunit.client.0.smithi156.stdout:7/92: link dc/d13/d15/l19 dc/d13/d24/l25 0 2024-07-16T09:55:31.373 INFO:tasks.workunit.client.0.smithi156.stdout:5/122: rmdir d1b/d1f 39 2024-07-16T09:55:31.373 INFO:tasks.workunit.client.0.smithi156.stdout:1/102: dwrite dc/f10 [0,4194304] 0 2024-07-16T09:55:31.374 INFO:tasks.workunit.client.1.smithi176.stdout:8/228: dwrite d8/d14/f27 [0,4194304] 0 2024-07-16T09:55:31.374 INFO:tasks.workunit.client.0.smithi156.stdout:5/123: readlink l15 0 2024-07-16T09:55:31.374 INFO:tasks.workunit.client.0.smithi156.stdout:3/90: write f0 [1349097,62035] 0 2024-07-16T09:55:31.375 INFO:tasks.workunit.client.0.smithi156.stdout:9/140: mkdir d2/d5/d16/d22/d2b 0 2024-07-16T09:55:31.376 INFO:tasks.workunit.client.1.smithi176.stdout:8/229: symlink d8/d23/d2c/d34/l54 0 2024-07-16T09:55:31.377 INFO:tasks.workunit.client.1.smithi176.stdout:8/230: readlink d8/l28 0 2024-07-16T09:55:31.377 INFO:tasks.workunit.client.0.smithi156.stdout:5/124: write d1b/d1e/f21 [519077,1931] 0 2024-07-16T09:55:31.378 INFO:tasks.workunit.client.0.smithi156.stdout:3/91: dread - d8/d14/f1c zero size 2024-07-16T09:55:31.378 INFO:tasks.workunit.client.0.smithi156.stdout:8/78: mknod d0/c1d 0 2024-07-16T09:55:31.380 INFO:tasks.workunit.client.0.smithi156.stdout:4/132: creat d4/d23/f32 x:0 0 0 2024-07-16T09:55:31.381 INFO:tasks.workunit.client.1.smithi176.stdout:8/231: mkdir d8/d14/d1e/d55 0 2024-07-16T09:55:31.381 INFO:tasks.workunit.client.0.smithi156.stdout:1/103: mknod dc/d14/d1e/c26 0 2024-07-16T09:55:31.381 INFO:tasks.workunit.client.0.smithi156.stdout:2/244: symlink d0/d13/d17/l52 0 2024-07-16T09:55:31.382 INFO:tasks.workunit.client.0.smithi156.stdout:5/125: dread - d1b/d1e/f25 zero size 2024-07-16T09:55:31.384 INFO:tasks.workunit.client.0.smithi156.stdout:1/104: truncate dc/f18 676392 0 2024-07-16T09:55:31.384 INFO:tasks.workunit.client.0.smithi156.stdout:7/93: dread dc/d13/f12 [0,4194304] 0 2024-07-16T09:55:31.385 INFO:tasks.workunit.client.0.smithi156.stdout:4/133: chown d4/db/dc/f2e 0 1 2024-07-16T09:55:31.385 INFO:tasks.workunit.client.0.smithi156.stdout:3/92: mkdir d8/da/d1d 0 2024-07-16T09:55:31.385 INFO:tasks.workunit.client.1.smithi176.stdout:8/232: rename d8/d14/l19 to d8/d23/l56 0 2024-07-16T09:55:31.386 INFO:tasks.workunit.client.0.smithi156.stdout:4/134: readlink d4/d23/l28 0 2024-07-16T09:55:31.386 INFO:tasks.workunit.client.0.smithi156.stdout:6/151: mkdir d3/d26/d30 0 2024-07-16T09:55:31.388 INFO:tasks.workunit.client.0.smithi156.stdout:5/126: dread d1b/d1e/f21 [0,4194304] 0 2024-07-16T09:55:31.389 INFO:tasks.workunit.client.0.smithi156.stdout:8/79: creat d0/dd/f1e x:0 0 0 2024-07-16T09:55:31.391 INFO:tasks.workunit.client.0.smithi156.stdout:7/94: mknod dc/c26 0 2024-07-16T09:55:31.391 INFO:tasks.workunit.client.0.smithi156.stdout:2/245: mknod d0/d5/d29/d2d/c53 0 2024-07-16T09:55:31.392 INFO:tasks.workunit.client.1.smithi176.stdout:8/233: dwrite d8/f2e [0,4194304] 0 2024-07-16T09:55:31.393 INFO:tasks.workunit.client.0.smithi156.stdout:4/135: mkdir d4/d23/d30/d33 0 2024-07-16T09:55:31.394 INFO:tasks.workunit.client.1.smithi176.stdout:8/234: chown d8/fa 924008 1 2024-07-16T09:55:31.397 INFO:tasks.workunit.client.0.smithi156.stdout:1/105: dwrite dc/f19 [0,4194304] 0 2024-07-16T09:55:31.398 INFO:tasks.workunit.client.1.smithi176.stdout:8/235: unlink d8/d23/d2c/l48 0 2024-07-16T09:55:31.398 INFO:tasks.workunit.client.1.smithi176.stdout:8/236: truncate d8/f17 1175383 0 2024-07-16T09:55:31.399 INFO:tasks.workunit.client.0.smithi156.stdout:3/93: dwrite f3 [0,4194304] 0 2024-07-16T09:55:31.399 INFO:tasks.workunit.client.0.smithi156.stdout:9/141: getdents d2/d5/d16 0 2024-07-16T09:55:31.400 INFO:tasks.workunit.client.1.smithi176.stdout:8/237: write d8/d20/d2a/d31/f3e [468375,53020] 0 2024-07-16T09:55:31.401 INFO:tasks.workunit.client.1.smithi176.stdout:8/238: write d8/d20/f2b [1492717,54362] 0 2024-07-16T09:55:31.401 INFO:tasks.workunit.client.0.smithi156.stdout:6/152: mkdir d3/d26/d1f/d19/d31 0 2024-07-16T09:55:31.402 INFO:tasks.workunit.client.1.smithi176.stdout:8/239: chown d8/d14/d1e 857668 1 2024-07-16T09:55:31.402 INFO:tasks.workunit.client.0.smithi156.stdout:3/94: fsync d8/f10 0 2024-07-16T09:55:31.402 INFO:tasks.workunit.client.1.smithi176.stdout:8/240: fdatasync d8/d23/f25 0 2024-07-16T09:55:31.406 INFO:tasks.workunit.client.0.smithi156.stdout:5/127: symlink d1b/l26 0 2024-07-16T09:55:31.408 INFO:tasks.workunit.client.1.smithi176.stdout:8/241: creat d8/d23/d2c/d37/f57 x:0 0 0 2024-07-16T09:55:31.409 INFO:tasks.workunit.client.0.smithi156.stdout:4/136: creat d4/db/f34 x:0 0 0 2024-07-16T09:55:31.409 INFO:tasks.workunit.client.0.smithi156.stdout:2/246: creat d0/d5/d29/f54 x:0 0 0 2024-07-16T09:55:31.411 INFO:tasks.workunit.client.0.smithi156.stdout:7/95: dwrite dc/d13/f1d [0,4194304] 0 2024-07-16T09:55:31.412 INFO:tasks.workunit.client.1.smithi176.stdout:8/242: unlink c3 0 2024-07-16T09:55:31.412 INFO:tasks.workunit.client.0.smithi156.stdout:6/153: mknod d3/d26/d1f/c32 0 2024-07-16T09:55:31.413 INFO:tasks.workunit.client.1.smithi176.stdout:8/243: read d8/d23/f21 [98847,52860] 0 2024-07-16T09:55:31.414 INFO:tasks.workunit.client.0.smithi156.stdout:9/142: creat d2/d5/d1d/f2c x:0 0 0 2024-07-16T09:55:31.414 INFO:tasks.workunit.client.0.smithi156.stdout:8/80: dwrite d0/fc [0,4194304] 0 2024-07-16T09:55:31.415 INFO:tasks.workunit.client.1.smithi176.stdout:8/244: dread d8/d14/d1e/f1f [0,4194304] 0 2024-07-16T09:55:31.416 INFO:tasks.workunit.client.1.smithi176.stdout:8/245: fsync d8/f17 0 2024-07-16T09:55:31.418 INFO:tasks.workunit.client.1.smithi176.stdout:8/246: truncate d8/d23/d2c/d37/f4f 742113 0 2024-07-16T09:55:31.421 INFO:tasks.workunit.client.1.smithi176.stdout:8/247: symlink d8/d20/l58 0 2024-07-16T09:55:31.424 INFO:tasks.workunit.client.0.smithi156.stdout:7/96: readlink dc/d13/d15/l19 0 2024-07-16T09:55:31.424 INFO:tasks.workunit.client.0.smithi156.stdout:3/95: mkdir d8/da/d1d/d1e 0 2024-07-16T09:55:31.425 INFO:tasks.workunit.client.1.smithi176.stdout:8/248: write d8/d23/f4a [3688839,66497] 0 2024-07-16T09:55:31.425 INFO:tasks.workunit.client.0.smithi156.stdout:8/81: mkdir d0/dd/d1f 0 2024-07-16T09:55:31.426 INFO:tasks.workunit.client.0.smithi156.stdout:6/154: symlink d3/d26/d1f/l33 0 2024-07-16T09:55:31.427 INFO:tasks.workunit.client.0.smithi156.stdout:5/128: symlink d1b/l27 0 2024-07-16T09:55:31.427 INFO:tasks.workunit.client.0.smithi156.stdout:4/137: dwrite d4/db/f34 [0,4194304] 0 2024-07-16T09:55:31.428 INFO:tasks.workunit.client.0.smithi156.stdout:9/143: dread d2/f4 [0,4194304] 0 2024-07-16T09:55:31.429 INFO:tasks.workunit.client.0.smithi156.stdout:3/96: unlink l7 0 2024-07-16T09:55:31.430 INFO:tasks.workunit.client.0.smithi156.stdout:4/138: chown d4/f5 7668723 1 2024-07-16T09:55:31.431 INFO:tasks.workunit.client.0.smithi156.stdout:8/82: read - d0/dd/f1a zero size 2024-07-16T09:55:31.431 INFO:tasks.workunit.client.0.smithi156.stdout:1/106: getdents dc 0 2024-07-16T09:55:31.431 INFO:tasks.workunit.client.0.smithi156.stdout:7/97: chown dc/d13/d24/l25 5 1 2024-07-16T09:55:31.432 INFO:tasks.workunit.client.0.smithi156.stdout:9/144: stat d2/de 0 2024-07-16T09:55:31.433 INFO:tasks.workunit.client.0.smithi156.stdout:3/97: write f0 [4206853,96935] 0 2024-07-16T09:55:31.436 INFO:tasks.workunit.client.0.smithi156.stdout:8/83: mkdir d0/d11/d20 0 2024-07-16T09:55:31.437 INFO:tasks.workunit.client.0.smithi156.stdout:6/155: creat d3/d26/f34 x:0 0 0 2024-07-16T09:55:31.437 INFO:tasks.workunit.client.0.smithi156.stdout:9/145: write d2/d5/f8 [2287990,79015] 0 2024-07-16T09:55:31.438 INFO:tasks.workunit.client.0.smithi156.stdout:3/98: mkdir d8/d14/d1f 0 2024-07-16T09:55:31.438 INFO:tasks.workunit.client.0.smithi156.stdout:7/98: write dc/d13/f12 [1837219,56569] 0 2024-07-16T09:55:31.438 INFO:tasks.workunit.client.0.smithi156.stdout:1/107: mknod dc/d20/c27 0 2024-07-16T09:55:31.441 INFO:tasks.workunit.client.0.smithi156.stdout:1/108: readlink lb 0 2024-07-16T09:55:31.443 INFO:tasks.workunit.client.0.smithi156.stdout:7/99: rename c6 to dc/d13/d20/c27 0 2024-07-16T09:55:31.444 INFO:tasks.workunit.client.0.smithi156.stdout:1/109: write dc/f19 [4607371,124372] 0 2024-07-16T09:55:31.445 INFO:tasks.workunit.client.0.smithi156.stdout:8/84: getdents d0/d11/d20 0 2024-07-16T09:55:31.446 INFO:tasks.workunit.client.0.smithi156.stdout:5/129: dwrite f19 [0,4194304] 0 2024-07-16T09:55:31.449 INFO:tasks.workunit.client.0.smithi156.stdout:5/130: write f19 [1246808,10138] 0 2024-07-16T09:55:31.449 INFO:tasks.workunit.client.1.smithi176.stdout:8/249: dread d8/d20/f2b [0,4194304] 0 2024-07-16T09:55:31.450 INFO:tasks.workunit.client.0.smithi156.stdout:9/146: rename d2/d5/l29 to d2/d5/d16/d22/l2d 0 2024-07-16T09:55:31.452 INFO:tasks.workunit.client.0.smithi156.stdout:6/156: getdents d3/d26/d1e 0 2024-07-16T09:55:31.452 INFO:tasks.workunit.client.0.smithi156.stdout:5/131: chown d1b/d1e/f21 1266 1 2024-07-16T09:55:31.453 INFO:tasks.workunit.client.0.smithi156.stdout:9/147: chown d2/l21 28 1 2024-07-16T09:55:31.455 INFO:tasks.workunit.client.0.smithi156.stdout:4/139: write d4/db/f34 [4784235,24481] 0 2024-07-16T09:55:31.455 INFO:tasks.workunit.client.0.smithi156.stdout:7/100: dread dc/d13/f12 [0,4194304] 0 2024-07-16T09:55:31.456 INFO:tasks.workunit.client.0.smithi156.stdout:3/99: rmdir d8/da/d1d/d1e 0 2024-07-16T09:55:31.456 INFO:tasks.workunit.client.0.smithi156.stdout:2/247: dread d0/d13/d17/d1c/f46 [0,4194304] 0 2024-07-16T09:55:31.457 INFO:tasks.workunit.client.0.smithi156.stdout:1/110: dread dc/d14/d1e/f21 [0,4194304] 0 2024-07-16T09:55:31.458 INFO:tasks.workunit.client.1.smithi176.stdout:8/250: dwrite d8/d23/f21 [0,4194304] 0 2024-07-16T09:55:31.458 INFO:tasks.workunit.client.1.smithi176.stdout:8/251: fsync d8/f4b 0 2024-07-16T09:55:31.460 INFO:tasks.workunit.client.0.smithi156.stdout:4/140: dread - d4/db/dc/f2a zero size 2024-07-16T09:55:31.461 INFO:tasks.workunit.client.0.smithi156.stdout:8/85: dwrite d0/f2 [0,4194304] 0 2024-07-16T09:55:31.462 INFO:tasks.workunit.client.0.smithi156.stdout:5/132: mkdir d1b/d28 0 2024-07-16T09:55:31.463 INFO:tasks.workunit.client.0.smithi156.stdout:4/141: write d4/d23/d30/f2d [176152,76509] 0 2024-07-16T09:55:31.464 INFO:tasks.workunit.client.1.smithi176.stdout:8/252: symlink d8/d20/d2a/d31/l59 0 2024-07-16T09:55:31.464 INFO:tasks.workunit.client.0.smithi156.stdout:3/100: creat d8/da/d13/f20 x:0 0 0 2024-07-16T09:55:31.464 INFO:tasks.workunit.client.0.smithi156.stdout:6/157: mkdir d3/d26/d1f/d19/d31/d35 0 2024-07-16T09:55:31.465 INFO:tasks.workunit.client.1.smithi176.stdout:8/253: truncate d8/f17 1523946 0 2024-07-16T09:55:31.468 INFO:tasks.workunit.client.0.smithi156.stdout:4/142: fsync d4/d23/f24 0 2024-07-16T09:55:31.468 INFO:tasks.workunit.client.0.smithi156.stdout:1/111: creat dc/f28 x:0 0 0 2024-07-16T09:55:31.470 INFO:tasks.workunit.client.0.smithi156.stdout:1/112: write dc/f10 [4779134,97910] 0 2024-07-16T09:55:31.470 INFO:tasks.workunit.client.0.smithi156.stdout:9/148: creat d2/d5/d1d/d23/f2e x:0 0 0 2024-07-16T09:55:31.473 INFO:tasks.workunit.client.1.smithi176.stdout:8/254: rename d8/d14/d1e/d33/f35 to d8/d20/f5a 0 2024-07-16T09:55:31.479 INFO:tasks.workunit.client.1.smithi176.stdout:8/255: creat d8/d14/d1e/d33/f5b x:0 0 0 2024-07-16T09:55:31.482 INFO:tasks.workunit.client.1.smithi176.stdout:8/256: rename d8/f4b to d8/d20/d2a/d31/f5c 0 2024-07-16T09:55:31.484 INFO:tasks.workunit.client.1.smithi176.stdout:8/257: mkdir d8/d5d 0 2024-07-16T09:55:31.484 INFO:tasks.workunit.client.1.smithi176.stdout:8/258: write d8/d20/d2a/d31/d2f/f4d [1356137,10623] 0 2024-07-16T09:55:31.487 INFO:tasks.workunit.client.1.smithi176.stdout:8/259: symlink d8/d20/d2a/l5e 0 2024-07-16T09:55:31.490 INFO:tasks.workunit.client.1.smithi176.stdout:8/260: dread d8/f3d [0,4194304] 0 2024-07-16T09:55:31.490 INFO:tasks.workunit.client.1.smithi176.stdout:1/134: sync 2024-07-16T09:55:31.494 INFO:tasks.workunit.client.1.smithi176.stdout:1/135: dread d4/d8/f28 [0,4194304] 0 2024-07-16T09:55:31.495 INFO:tasks.workunit.client.1.smithi176.stdout:8/261: getdents d8/d20/d2a 0 2024-07-16T09:55:31.496 INFO:tasks.workunit.client.1.smithi176.stdout:8/262: mknod d8/d23/d2c/c5f 0 2024-07-16T09:55:31.497 INFO:tasks.workunit.client.1.smithi176.stdout:8/263: chown d8/d20 666 1 2024-07-16T09:55:31.497 INFO:tasks.workunit.client.1.smithi176.stdout:1/136: creat d4/d8/f29 x:0 0 0 2024-07-16T09:55:31.498 INFO:tasks.workunit.client.1.smithi176.stdout:1/137: rename d4/d8 to d4/d8/d2a 22 2024-07-16T09:55:31.500 INFO:tasks.workunit.client.1.smithi176.stdout:1/138: symlink d4/d8/d12/l2b 0 2024-07-16T09:55:31.503 INFO:tasks.workunit.client.1.smithi176.stdout:8/264: dwrite d8/d20/d2a/f3f [0,4194304] 0 2024-07-16T09:55:31.507 INFO:tasks.workunit.client.1.smithi176.stdout:8/265: dread d8/f2e [0,4194304] 0 2024-07-16T09:55:31.509 INFO:tasks.workunit.client.1.smithi176.stdout:8/266: creat d8/d14/d1e/f60 x:0 0 0 2024-07-16T09:55:31.511 INFO:tasks.workunit.client.1.smithi176.stdout:8/267: write d8/d23/d2c/f36 [1161968,21795] 0 2024-07-16T09:55:31.513 INFO:tasks.workunit.client.1.smithi176.stdout:8/268: fdatasync d8/d23/f1b 0 2024-07-16T09:55:31.514 INFO:tasks.workunit.client.1.smithi176.stdout:8/269: mkdir d8/d20/d2a/d61 0 2024-07-16T09:55:31.514 INFO:tasks.workunit.client.1.smithi176.stdout:8/270: chown d8/l1c 702 1 2024-07-16T09:55:31.515 INFO:tasks.workunit.client.1.smithi176.stdout:8/271: read - d8/d14/d1e/d33/f5b zero size 2024-07-16T09:55:31.516 INFO:tasks.workunit.client.1.smithi176.stdout:8/272: write d8/d23/f25 [768759,49283] 0 2024-07-16T09:55:31.518 INFO:tasks.workunit.client.1.smithi176.stdout:8/273: write d8/d23/d2c/d37/f3c [1765492,22143] 0 2024-07-16T09:55:31.524 INFO:tasks.workunit.client.1.smithi176.stdout:8/274: dwrite d8/f3d [0,4194304] 0 2024-07-16T09:55:31.527 INFO:tasks.workunit.client.1.smithi176.stdout:8/275: readlink d8/d23/d2c/d34/l51 0 2024-07-16T09:55:31.528 INFO:tasks.workunit.client.1.smithi176.stdout:8/276: chown d8/d20/d2a/d31/d2f/f4d 0 1 2024-07-16T09:55:31.530 INFO:tasks.workunit.client.0.smithi156.stdout:9/149: creat d2/de/f2f x:0 0 0 2024-07-16T09:55:31.531 INFO:tasks.workunit.client.0.smithi156.stdout:6/158: write d3/f1c [735878,26550] 0 2024-07-16T09:55:31.531 INFO:tasks.workunit.client.1.smithi176.stdout:8/277: getdents d8/d20 0 2024-07-16T09:55:31.532 INFO:tasks.workunit.client.1.smithi176.stdout:8/278: stat d8/l28 0 2024-07-16T09:55:31.532 INFO:tasks.workunit.client.0.smithi156.stdout:8/86: dread d0/fc [0,4194304] 0 2024-07-16T09:55:31.534 INFO:tasks.workunit.client.0.smithi156.stdout:6/159: dread - d3/f2a zero size 2024-07-16T09:55:31.535 INFO:tasks.workunit.client.0.smithi156.stdout:8/87: read - d0/d11/f16 zero size 2024-07-16T09:55:31.535 INFO:tasks.workunit.client.1.smithi176.stdout:8/279: rename d8/d5d to d8/d23/d2c/d34/d62 0 2024-07-16T09:55:31.535 INFO:tasks.workunit.client.0.smithi156.stdout:3/101: dread f0 [0,4194304] 0 2024-07-16T09:55:31.536 INFO:tasks.workunit.client.1.smithi176.stdout:8/280: read d8/fa [140329,23604] 0 2024-07-16T09:55:31.536 INFO:tasks.workunit.client.0.smithi156.stdout:7/101: truncate dc/d13/f12 3171362 0 2024-07-16T09:55:31.537 INFO:tasks.workunit.client.0.smithi156.stdout:4/143: mkdir d4/d23/d30/d33/d35 0 2024-07-16T09:55:31.537 INFO:tasks.workunit.client.0.smithi156.stdout:2/248: getdents d0/d5 0 2024-07-16T09:55:31.538 INFO:tasks.workunit.client.0.smithi156.stdout:1/113: dwrite dc/f19 [0,4194304] 0 2024-07-16T09:55:31.540 INFO:tasks.workunit.client.0.smithi156.stdout:9/150: link d2/l21 d2/d5/d16/d22/l30 0 2024-07-16T09:55:31.541 INFO:tasks.workunit.client.1.smithi176.stdout:8/281: truncate d8/d14/f2d 2037341 0 2024-07-16T09:55:31.541 INFO:tasks.workunit.client.0.smithi156.stdout:6/160: symlink d3/d26/d1f/d19/d31/d35/l36 0 2024-07-16T09:55:31.543 INFO:tasks.workunit.client.1.smithi176.stdout:8/282: write d8/d14/d1e/d33/f5b [469832,61483] 0 2024-07-16T09:55:31.543 INFO:tasks.workunit.client.0.smithi156.stdout:9/151: write d2/f4 [879112,113170] 0 2024-07-16T09:55:31.544 INFO:tasks.workunit.client.0.smithi156.stdout:1/114: creat dc/d14/d1e/f29 x:0 0 0 2024-07-16T09:55:31.546 INFO:tasks.workunit.client.1.smithi176.stdout:8/283: symlink d8/d20/l63 0 2024-07-16T09:55:31.547 INFO:tasks.workunit.client.0.smithi156.stdout:3/102: rename d8/d14/d1f to d8/da/d21 0 2024-07-16T09:55:31.548 INFO:tasks.workunit.client.0.smithi156.stdout:1/115: chown dc/fd 2657 1 2024-07-16T09:55:31.548 INFO:tasks.workunit.client.0.smithi156.stdout:8/88: link d0/dd/f17 d0/d11/d20/f21 0 2024-07-16T09:55:31.549 INFO:tasks.workunit.client.0.smithi156.stdout:9/152: fsync d2/d5/fb 0 2024-07-16T09:55:31.550 INFO:tasks.workunit.client.0.smithi156.stdout:7/102: getdents dc/d13/d24 0 2024-07-16T09:55:31.551 INFO:tasks.workunit.client.0.smithi156.stdout:3/103: mknod d8/d14/c22 0 2024-07-16T09:55:31.551 INFO:tasks.workunit.client.0.smithi156.stdout:2/249: dread d0/f16 [0,4194304] 0 2024-07-16T09:55:31.553 INFO:tasks.workunit.client.1.smithi176.stdout:8/284: dwrite d8/d23/d2c/d37/f57 [0,4194304] 0 2024-07-16T09:55:31.554 INFO:tasks.workunit.client.0.smithi156.stdout:1/116: write dc/d20/f23 [568182,43024] 0 2024-07-16T09:55:31.555 INFO:tasks.workunit.client.0.smithi156.stdout:4/144: dwrite d4/db/f2c [0,4194304] 0 2024-07-16T09:55:31.557 INFO:tasks.workunit.client.0.smithi156.stdout:6/161: dwrite d3/f1c [0,4194304] 0 2024-07-16T09:55:31.563 INFO:tasks.workunit.client.0.smithi156.stdout:2/250: mknod d0/d13/d17/c55 0 2024-07-16T09:55:31.565 INFO:tasks.workunit.client.0.smithi156.stdout:1/117: fdatasync f5 0 2024-07-16T09:55:31.565 INFO:tasks.workunit.client.0.smithi156.stdout:9/153: rmdir d2/de 39 2024-07-16T09:55:31.565 INFO:tasks.workunit.client.1.smithi176.stdout:8/285: write d8/d20/d2a/d31/f3e [5147240,120848] 0 2024-07-16T09:55:31.568 INFO:tasks.workunit.client.1.smithi176.stdout:8/286: mknod d8/d20/d2a/d31/d2f/c64 0 2024-07-16T09:55:31.569 INFO:tasks.workunit.client.0.smithi156.stdout:4/145: write d4/db/dc/f2e [849722,34801] 0 2024-07-16T09:55:31.569 INFO:tasks.workunit.client.1.smithi176.stdout:8/287: stat d8/d23/d2c/d37/f4f 0 2024-07-16T09:55:31.570 INFO:tasks.workunit.client.0.smithi156.stdout:8/89: dread d0/f6 [0,4194304] 0 2024-07-16T09:55:31.571 INFO:tasks.workunit.client.0.smithi156.stdout:3/104: dwrite d8/f10 [0,4194304] 0 2024-07-16T09:55:31.574 INFO:tasks.workunit.client.1.smithi176.stdout:8/288: mknod d8/d20/d2a/d61/c65 0 2024-07-16T09:55:31.585 INFO:tasks.workunit.client.1.smithi176.stdout:8/289: dwrite d8/d14/d1e/d33/f47 [4194304,4194304] 0 2024-07-16T09:55:31.588 INFO:tasks.workunit.client.1.smithi176.stdout:8/290: symlink d8/d23/d2c/d34/l66 0 2024-07-16T09:55:31.588 INFO:tasks.workunit.client.0.smithi156.stdout:8/90: read d0/f1 [4772468,31477] 0 2024-07-16T09:55:31.589 INFO:tasks.workunit.client.1.smithi176.stdout:8/291: fdatasync d8/d14/d1e/d33/f5b 0 2024-07-16T09:55:31.590 INFO:tasks.workunit.client.0.smithi156.stdout:6/162: dread d3/d13/d17/f2b [0,4194304] 0 2024-07-16T09:55:31.591 INFO:tasks.workunit.client.0.smithi156.stdout:6/163: dread - d3/f2a zero size 2024-07-16T09:55:31.592 INFO:tasks.workunit.client.0.smithi156.stdout:2/251: dread d0/fe [0,4194304] 0 2024-07-16T09:55:31.592 INFO:tasks.workunit.client.0.smithi156.stdout:6/164: stat d3/d26/d1f/d19/d31/d35/l36 0 2024-07-16T09:55:31.594 INFO:tasks.workunit.client.0.smithi156.stdout:6/165: write d3/f25 [4929725,94140] 0 2024-07-16T09:55:31.605 INFO:tasks.workunit.client.0.smithi156.stdout:4/146: rmdir d4 39 2024-07-16T09:55:31.605 INFO:tasks.workunit.client.0.smithi156.stdout:7/103: write fb [3195240,130444] 0 2024-07-16T09:55:31.610 INFO:tasks.workunit.client.0.smithi156.stdout:3/105: mknod d8/da/d1d/c23 0 2024-07-16T09:55:31.611 INFO:tasks.workunit.client.0.smithi156.stdout:0/49: sync 2024-07-16T09:55:31.613 INFO:tasks.workunit.client.1.smithi176.stdout:8/292: dwrite d8/fa [0,4194304] 0 2024-07-16T09:55:31.614 INFO:tasks.workunit.client.0.smithi156.stdout:2/252: creat d0/d5/d29/f56 x:0 0 0 2024-07-16T09:55:31.618 INFO:tasks.workunit.client.0.smithi156.stdout:0/50: dwrite d0/d7/f8 [0,4194304] 0 2024-07-16T09:55:31.619 INFO:tasks.workunit.client.0.smithi156.stdout:1/118: creat dc/d24/f2a x:0 0 0 2024-07-16T09:55:31.620 INFO:tasks.workunit.client.1.smithi176.stdout:8/293: link d8/d23/f1b d8/d20/d2a/f67 0 2024-07-16T09:55:31.621 INFO:tasks.workunit.client.0.smithi156.stdout:0/51: write d0/fa [749167,42146] 0 2024-07-16T09:55:31.624 INFO:tasks.workunit.client.0.smithi156.stdout:9/154: truncate d2/d5/fb 478449 0 2024-07-16T09:55:31.625 INFO:tasks.workunit.client.0.smithi156.stdout:4/147: chown d4/db/dc 2 1 2024-07-16T09:55:31.625 INFO:tasks.workunit.client.0.smithi156.stdout:7/104: rmdir dc/d13 39 2024-07-16T09:55:31.627 INFO:tasks.workunit.client.0.smithi156.stdout:8/91: mknod d0/dd/d1f/c22 0 2024-07-16T09:55:31.630 INFO:tasks.workunit.client.0.smithi156.stdout:3/106: creat d8/da/d13/f24 x:0 0 0 2024-07-16T09:55:31.630 INFO:tasks.workunit.client.0.smithi156.stdout:7/105: read dc/f14 [817797,17049] 0 2024-07-16T09:55:31.630 INFO:tasks.workunit.client.0.smithi156.stdout:9/155: readlink d2/d5/l25 0 2024-07-16T09:55:31.631 INFO:tasks.workunit.client.0.smithi156.stdout:2/253: mknod d0/d13/d17/c57 0 2024-07-16T09:55:31.633 INFO:tasks.workunit.client.0.smithi156.stdout:3/107: fsync d8/f1a 0 2024-07-16T09:55:31.633 INFO:tasks.workunit.client.0.smithi156.stdout:6/166: dwrite d3/d26/d1f/f2c [0,4194304] 0 2024-07-16T09:55:31.635 INFO:tasks.workunit.client.0.smithi156.stdout:1/119: dwrite dc/fd [0,4194304] 0 2024-07-16T09:55:31.636 INFO:tasks.workunit.client.0.smithi156.stdout:1/120: write dc/d24/f2a [409755,115274] 0 2024-07-16T09:55:31.637 INFO:tasks.workunit.client.0.smithi156.stdout:8/92: write d0/f2 [6137276,115496] 0 2024-07-16T09:55:31.637 INFO:tasks.workunit.client.0.smithi156.stdout:1/121: stat f6 0 2024-07-16T09:55:31.640 INFO:tasks.workunit.client.0.smithi156.stdout:8/93: chown d0/dd/f10 1659638 1 2024-07-16T09:55:31.641 INFO:tasks.workunit.client.0.smithi156.stdout:7/106: write dc/f14 [59259,61594] 0 2024-07-16T09:55:31.642 INFO:tasks.workunit.client.0.smithi156.stdout:3/108: mkdir d8/da/d1d/d25 0 2024-07-16T09:55:31.644 INFO:tasks.workunit.client.0.smithi156.stdout:6/167: creat d3/d26/d1f/f37 x:0 0 0 2024-07-16T09:55:31.646 INFO:tasks.workunit.client.1.smithi176.stdout:8/294: dread d8/d23/f4a [0,4194304] 0 2024-07-16T09:55:31.647 INFO:tasks.workunit.client.1.smithi176.stdout:8/295: chown d8/c4e 44 1 2024-07-16T09:55:31.647 INFO:tasks.workunit.client.0.smithi156.stdout:6/168: truncate d3/d26/f22 1328836 0 2024-07-16T09:55:31.647 INFO:tasks.workunit.client.0.smithi156.stdout:8/94: dwrite d0/fb [4194304,4194304] 0 2024-07-16T09:55:31.648 INFO:tasks.workunit.client.1.smithi176.stdout:8/296: write d8/d14/d1e/f60 [867284,117551] 0 2024-07-16T09:55:31.651 INFO:tasks.workunit.client.0.smithi156.stdout:0/52: link d0/c2 d0/d7/d9/cc 0 2024-07-16T09:55:31.652 INFO:tasks.workunit.client.1.smithi176.stdout:8/297: fsync d8/d14/f2d 0 2024-07-16T09:55:31.653 INFO:tasks.workunit.client.0.smithi156.stdout:3/109: mknod d8/da/c26 0 2024-07-16T09:55:31.654 INFO:tasks.workunit.client.0.smithi156.stdout:3/110: readlink d8/l18 0 2024-07-16T09:55:31.655 INFO:tasks.workunit.client.0.smithi156.stdout:4/148: rename d4/d23/f24 to d4/db/dc/f36 0 2024-07-16T09:55:31.655 INFO:tasks.workunit.client.0.smithi156.stdout:3/111: fsync d8/d14/f1c 0 2024-07-16T09:55:31.656 INFO:tasks.workunit.client.0.smithi156.stdout:1/122: dwrite dc/d14/f17 [0,4194304] 0 2024-07-16T09:55:31.657 INFO:tasks.workunit.client.0.smithi156.stdout:1/123: stat dc/d20/l25 0 2024-07-16T09:55:31.657 INFO:tasks.workunit.client.1.smithi176.stdout:8/298: getdents d8/d23/d2c 0 2024-07-16T09:55:31.658 INFO:tasks.workunit.client.0.smithi156.stdout:4/149: write d4/d23/f32 [22251,92710] 0 2024-07-16T09:55:31.658 INFO:tasks.workunit.client.0.smithi156.stdout:8/95: symlink d0/d11/l23 0 2024-07-16T09:55:31.660 INFO:tasks.workunit.client.0.smithi156.stdout:8/96: chown d0/d11/d20/f21 150357 1 2024-07-16T09:55:31.660 INFO:tasks.workunit.client.0.smithi156.stdout:8/97: stat d0/dd/f17 0 2024-07-16T09:55:31.662 INFO:tasks.workunit.client.0.smithi156.stdout:6/169: rename d3/d26/lb to d3/d26/d1f/l38 0 2024-07-16T09:55:31.663 INFO:tasks.workunit.client.0.smithi156.stdout:6/170: stat d3/d26/d1f/fe 0 2024-07-16T09:55:31.664 INFO:tasks.workunit.client.1.smithi176.stdout:8/299: dread d8/d20/f5a [0,4194304] 0 2024-07-16T09:55:31.665 INFO:tasks.workunit.client.0.smithi156.stdout:1/124: mknod dc/c2b 0 2024-07-16T09:55:31.667 INFO:tasks.workunit.client.0.smithi156.stdout:0/53: mknod d0/d7/d9/cd 0 2024-07-16T09:55:31.667 INFO:tasks.workunit.client.1.smithi176.stdout:8/300: mknod d8/d20/c68 0 2024-07-16T09:55:31.672 INFO:tasks.workunit.client.0.smithi156.stdout:7/107: rename dc/ce to dc/d13/d20/c28 0 2024-07-16T09:55:31.672 INFO:tasks.workunit.client.0.smithi156.stdout:1/125: dwrite dc/d24/f2a [0,4194304] 0 2024-07-16T09:55:31.673 INFO:tasks.workunit.client.0.smithi156.stdout:1/126: stat dc/l12 0 2024-07-16T09:55:31.673 INFO:tasks.workunit.client.0.smithi156.stdout:4/150: symlink d4/db/dc/l37 0 2024-07-16T09:55:31.674 INFO:tasks.workunit.client.1.smithi176.stdout:8/301: dwrite d8/d23/d2c/f36 [0,4194304] 0 2024-07-16T09:55:31.675 INFO:tasks.workunit.client.0.smithi156.stdout:4/151: write d4/d23/d30/f2d [175929,129373] 0 2024-07-16T09:55:31.677 INFO:tasks.workunit.client.0.smithi156.stdout:4/152: write d4/d23/f17 [589543,124258] 0 2024-07-16T09:55:31.677 INFO:tasks.workunit.client.0.smithi156.stdout:1/127: symlink dc/l2c 0 2024-07-16T09:55:31.678 INFO:tasks.workunit.client.0.smithi156.stdout:1/128: chown dc/d14/f1d 3929 1 2024-07-16T09:55:31.678 INFO:tasks.workunit.client.0.smithi156.stdout:6/171: creat d3/f39 x:0 0 0 2024-07-16T09:55:31.678 INFO:tasks.workunit.client.0.smithi156.stdout:1/129: stat dc/f13 0 2024-07-16T09:55:31.679 INFO:tasks.workunit.client.0.smithi156.stdout:0/54: unlink d0/d7/d9/cc 0 2024-07-16T09:55:31.679 INFO:tasks.workunit.client.1.smithi176.stdout:8/302: creat d8/d20/d2a/d31/f69 x:0 0 0 2024-07-16T09:55:31.681 INFO:tasks.workunit.client.1.smithi176.stdout:8/303: write d8/d14/d1e/d33/f47 [4406115,65727] 0 2024-07-16T09:55:31.683 INFO:tasks.workunit.client.0.smithi156.stdout:7/108: mknod dc/c29 0 2024-07-16T09:55:31.685 INFO:tasks.workunit.client.1.smithi176.stdout:8/304: mkdir d8/d20/d2a/d31/d46/d6a 0 2024-07-16T09:55:31.685 INFO:tasks.workunit.client.1.smithi176.stdout:8/305: chown d8/f3d 441 1 2024-07-16T09:55:31.686 INFO:tasks.workunit.client.0.smithi156.stdout:0/55: dread d0/d7/f8 [0,4194304] 0 2024-07-16T09:55:31.689 INFO:tasks.workunit.client.0.smithi156.stdout:1/130: dread dc/d20/f23 [0,4194304] 0 2024-07-16T09:55:31.689 INFO:tasks.workunit.client.0.smithi156.stdout:7/109: unlink dc/d13/f1c 0 2024-07-16T09:55:31.693 INFO:tasks.workunit.client.1.smithi176.stdout:8/306: dwrite d8/d23/d2c/d37/f4f [0,4194304] 0 2024-07-16T09:55:31.694 INFO:tasks.workunit.client.1.smithi176.stdout:8/307: dread - d8/d20/d2a/d31/f69 zero size 2024-07-16T09:55:31.694 INFO:tasks.workunit.client.0.smithi156.stdout:7/110: symlink dc/d13/d15/l2a 0 2024-07-16T09:55:31.698 INFO:tasks.workunit.client.1.smithi176.stdout:8/308: dread d8/d20/d2a/f3f [0,4194304] 0 2024-07-16T09:55:31.699 INFO:tasks.workunit.client.0.smithi156.stdout:2/254: write d0/d13/f43 [940197,69415] 0 2024-07-16T09:55:31.703 INFO:tasks.workunit.client.0.smithi156.stdout:2/255: symlink d0/d13/l58 0 2024-07-16T09:55:31.703 INFO:tasks.workunit.client.0.smithi156.stdout:8/98: write d0/d11/d20/f21 [517117,112896] 0 2024-07-16T09:55:31.704 INFO:tasks.workunit.client.0.smithi156.stdout:3/112: truncate d8/fb 2630104 0 2024-07-16T09:55:31.704 INFO:tasks.workunit.client.0.smithi156.stdout:2/256: chown d0/d5/f2f 31999989 1 2024-07-16T09:55:31.707 INFO:tasks.workunit.client.1.smithi176.stdout:8/309: dwrite d8/d20/f5a [4194304,4194304] 0 2024-07-16T09:55:31.708 INFO:tasks.workunit.client.1.smithi176.stdout:8/310: write f5 [545532,13568] 0 2024-07-16T09:55:31.710 INFO:tasks.workunit.client.1.smithi176.stdout:8/311: read d8/d20/f5a [4059833,55151] 0 2024-07-16T09:55:31.711 INFO:tasks.workunit.client.0.smithi156.stdout:3/113: dwrite d8/d14/f1c [0,4194304] 0 2024-07-16T09:55:31.711 INFO:tasks.workunit.client.0.smithi156.stdout:3/114: dread - d8/f1a zero size 2024-07-16T09:55:31.714 INFO:tasks.workunit.client.1.smithi176.stdout:8/312: creat d8/d14/d1e/d55/f6b x:0 0 0 2024-07-16T09:55:31.717 INFO:tasks.workunit.client.1.smithi176.stdout:8/313: mknod d8/d20/d2a/d31/d2f/c6c 0 2024-07-16T09:55:31.719 INFO:tasks.workunit.client.1.smithi176.stdout:8/314: symlink d8/d20/d2a/d31/d2f/l6d 0 2024-07-16T09:55:31.720 INFO:tasks.workunit.client.1.smithi176.stdout:8/315: chown d8/d20/d2a/d31/d2f/f43 4 1 2024-07-16T09:55:31.720 INFO:tasks.workunit.client.1.smithi176.stdout:8/316: stat d8/d23 0 2024-07-16T09:55:31.723 INFO:tasks.workunit.client.1.smithi176.stdout:8/317: rename d8/d20/d2a/d31/d2f/f43 to d8/d20/d2a/d61/f6e 0 2024-07-16T09:55:31.732 INFO:tasks.workunit.client.1.smithi176.stdout:8/318: rmdir d8 39 2024-07-16T09:55:31.734 INFO:tasks.workunit.client.1.smithi176.stdout:8/319: write d8/d23/f25 [1731344,36454] 0 2024-07-16T09:55:31.737 INFO:tasks.workunit.client.1.smithi176.stdout:8/320: link d8/d23/d2c/d34/c41 d8/d14/d1e/d55/c6f 0 2024-07-16T09:55:31.738 INFO:tasks.workunit.client.1.smithi176.stdout:8/321: symlink d8/d20/d2a/d31/d46/d6a/l70 0 2024-07-16T09:55:31.744 INFO:tasks.workunit.client.1.smithi176.stdout:8/322: dwrite d8/f2e [4194304,4194304] 0 2024-07-16T09:55:31.746 INFO:tasks.workunit.client.1.smithi176.stdout:8/323: symlink d8/d14/l71 0 2024-07-16T09:55:31.749 INFO:tasks.workunit.client.1.smithi176.stdout:8/324: dread d8/d14/d1e/f60 [0,4194304] 0 2024-07-16T09:55:31.751 INFO:tasks.workunit.client.1.smithi176.stdout:8/325: getdents d8/d20/d2a/d53 0 2024-07-16T09:55:31.753 INFO:tasks.workunit.client.1.smithi176.stdout:8/326: symlink d8/d23/l72 0 2024-07-16T09:55:31.754 INFO:tasks.workunit.client.0.smithi156.stdout:5/133: sync 2024-07-16T09:55:31.754 INFO:tasks.workunit.client.0.smithi156.stdout:5/134: write f13 [2043771,52840] 0 2024-07-16T09:55:31.756 INFO:tasks.workunit.client.1.smithi176.stdout:8/327: link d8/d23/d2c/d34/l66 d8/d20/d2a/d61/l73 0 2024-07-16T09:55:31.762 INFO:tasks.workunit.client.1.smithi176.stdout:8/328: write d8/d23/f1b [674659,51779] 0 2024-07-16T09:55:31.763 INFO:tasks.workunit.client.1.smithi176.stdout:8/329: mknod d8/d20/c74 0 2024-07-16T09:55:31.765 INFO:tasks.workunit.client.1.smithi176.stdout:8/330: read d8/d20/f2b [450452,119224] 0 2024-07-16T09:55:31.766 INFO:tasks.workunit.client.1.smithi176.stdout:8/331: creat d8/d20/d2a/f75 x:0 0 0 2024-07-16T09:55:31.768 INFO:tasks.workunit.client.1.smithi176.stdout:8/332: mkdir d8/d23/d76 0 2024-07-16T09:55:31.769 INFO:tasks.workunit.client.1.smithi176.stdout:8/333: rmdir d8/d20/d2a/d61 39 2024-07-16T09:55:31.771 INFO:tasks.workunit.client.1.smithi176.stdout:8/334: mkdir d8/d20/d2a/d31/d46/d77 0 2024-07-16T09:55:31.773 INFO:tasks.workunit.client.1.smithi176.stdout:8/335: link d8/d23/d2c/d37/f4f d8/d14/d1e/d33/f78 0 2024-07-16T09:55:31.775 INFO:tasks.workunit.client.1.smithi176.stdout:8/336: write d8/d14/d1e/f60 [7308,44226] 0 2024-07-16T09:55:31.776 INFO:tasks.workunit.client.1.smithi176.stdout:8/337: getdents d8/d23/d76 0 2024-07-16T09:55:31.778 INFO:tasks.workunit.client.1.smithi176.stdout:8/338: symlink d8/d20/d2a/l79 0 2024-07-16T09:55:31.779 INFO:tasks.workunit.client.1.smithi176.stdout:8/339: creat d8/d20/d2a/d31/d46/d6a/f7a x:0 0 0 2024-07-16T09:55:31.781 INFO:tasks.workunit.client.1.smithi176.stdout:8/340: symlink d8/d20/d2a/d31/d2f/l7b 0 2024-07-16T09:55:31.782 INFO:tasks.workunit.client.1.smithi176.stdout:8/341: chown d8/d20/d2a/d31/d46/d77 32703977 1 2024-07-16T09:55:31.783 INFO:tasks.workunit.client.1.smithi176.stdout:8/342: truncate d8/f4c 259302 0 2024-07-16T09:55:31.783 INFO:tasks.workunit.client.1.smithi176.stdout:8/343: chown f5 242143766 1 2024-07-16T09:55:31.783 INFO:tasks.workunit.client.0.smithi156.stdout:4/153: getdents d4/db/dc 0 2024-07-16T09:55:31.784 INFO:tasks.workunit.client.0.smithi156.stdout:1/131: getdents dc 0 2024-07-16T09:55:31.785 INFO:tasks.workunit.client.1.smithi176.stdout:8/344: mkdir d8/d14/d1e/d7c 0 2024-07-16T09:55:31.786 INFO:tasks.workunit.client.1.smithi176.stdout:8/345: write d8/d23/d2c/d37/f3c [2037004,45055] 0 2024-07-16T09:55:31.786 INFO:tasks.workunit.client.0.smithi156.stdout:6/172: link d3/d26/d1f/l38 d3/d26/d1f/l3a 0 2024-07-16T09:55:31.787 INFO:tasks.workunit.client.0.smithi156.stdout:6/173: fsync d3/f2a 0 2024-07-16T09:55:31.787 INFO:tasks.workunit.client.0.smithi156.stdout:0/56: rmdir d0 39 2024-07-16T09:55:31.788 INFO:tasks.workunit.client.0.smithi156.stdout:8/99: mkdir d0/d11/d13/d1c/d24 0 2024-07-16T09:55:31.789 INFO:tasks.workunit.client.1.smithi176.stdout:8/346: dread d8/d23/f1b [0,4194304] 0 2024-07-16T09:55:31.790 INFO:tasks.workunit.client.1.smithi176.stdout:8/347: write d8/f2e [8595943,128505] 0 2024-07-16T09:55:31.794 INFO:tasks.workunit.client.0.smithi156.stdout:5/135: symlink d1b/d1f/l29 0 2024-07-16T09:55:31.794 INFO:tasks.workunit.client.0.smithi156.stdout:8/100: dwrite d0/dd/f10 [0,4194304] 0 2024-07-16T09:55:31.795 INFO:tasks.workunit.client.1.smithi176.stdout:8/348: dread d8/d23/d2c/d37/f4f [0,4194304] 0 2024-07-16T09:55:31.795 INFO:tasks.workunit.client.0.smithi156.stdout:0/57: chown d0/d7/d9 21502936 1 2024-07-16T09:55:31.796 INFO:tasks.workunit.client.0.smithi156.stdout:6/174: mkdir d3/d26/d1f/d19/d29/d3b 0 2024-07-16T09:55:31.796 INFO:tasks.workunit.client.0.smithi156.stdout:0/58: stat d0/d7/d9/cd 0 2024-07-16T09:55:31.796 INFO:tasks.workunit.client.0.smithi156.stdout:9/156: sync 2024-07-16T09:55:31.799 INFO:tasks.workunit.client.0.smithi156.stdout:2/257: link d0/d13/d17/d1c/f1f d0/d13/f59 0 2024-07-16T09:55:31.800 INFO:tasks.workunit.client.0.smithi156.stdout:2/258: stat d0/l3d 0 2024-07-16T09:55:31.802 INFO:tasks.workunit.client.1.smithi176.stdout:8/349: getdents d8/d20 0 2024-07-16T09:55:31.802 INFO:tasks.workunit.client.0.smithi156.stdout:2/259: truncate d0/d13/d17/d1c/f37 280145 0 2024-07-16T09:55:31.803 INFO:tasks.workunit.client.0.smithi156.stdout:2/260: write d0/d13/f26 [2650910,61951] 0 2024-07-16T09:55:31.804 INFO:tasks.workunit.client.0.smithi156.stdout:3/115: truncate d8/f10 2605263 0 2024-07-16T09:55:31.805 INFO:tasks.workunit.client.0.smithi156.stdout:8/101: rmdir d0/d11/d20 39 2024-07-16T09:55:31.807 INFO:tasks.workunit.client.0.smithi156.stdout:6/175: unlink d3/d13/d17/l21 0 2024-07-16T09:55:31.808 INFO:tasks.workunit.client.0.smithi156.stdout:4/154: link d4/db/dc/l15 d4/d23/d30/l38 0 2024-07-16T09:55:31.810 INFO:tasks.workunit.client.0.smithi156.stdout:5/136: mknod d1b/d28/c2a 0 2024-07-16T09:55:31.811 INFO:tasks.workunit.client.0.smithi156.stdout:5/137: fsync d1b/d1f/f20 0 2024-07-16T09:55:31.811 INFO:tasks.workunit.client.0.smithi156.stdout:4/155: dread d4/db/dc/f29 [0,4194304] 0 2024-07-16T09:55:31.812 INFO:tasks.workunit.client.0.smithi156.stdout:5/138: chown d1b/c1c 1510 1 2024-07-16T09:55:31.815 INFO:tasks.workunit.client.0.smithi156.stdout:8/102: write d0/f6 [933435,123189] 0 2024-07-16T09:55:31.817 INFO:tasks.workunit.client.0.smithi156.stdout:1/132: getdents dc/d14/d1e 0 2024-07-16T09:55:31.818 INFO:tasks.workunit.client.0.smithi156.stdout:7/111: dwrite fb [0,4194304] 0 2024-07-16T09:55:31.821 INFO:tasks.workunit.client.0.smithi156.stdout:3/116: mknod d8/da/d1d/d25/c27 0 2024-07-16T09:55:31.823 INFO:tasks.workunit.client.0.smithi156.stdout:3/117: write d8/f1a [173260,28918] 0 2024-07-16T09:55:31.826 INFO:tasks.workunit.client.0.smithi156.stdout:8/103: creat d0/d11/d20/f25 x:0 0 0 2024-07-16T09:55:31.827 INFO:tasks.workunit.client.0.smithi156.stdout:4/156: dwrite d4/db/f16 [0,4194304] 0 2024-07-16T09:55:31.828 INFO:tasks.workunit.client.0.smithi156.stdout:1/133: creat dc/d14/f2d x:0 0 0 2024-07-16T09:55:31.828 INFO:tasks.workunit.client.0.smithi156.stdout:8/104: dread - d0/d11/d20/f25 zero size 2024-07-16T09:55:31.829 INFO:tasks.workunit.client.0.smithi156.stdout:2/261: getdents d0/d5 0 2024-07-16T09:55:31.830 INFO:tasks.workunit.client.1.smithi176.stdout:5/271: sync 2024-07-16T09:55:31.831 INFO:tasks.workunit.client.0.smithi156.stdout:0/59: getdents d0/d7/d9 0 2024-07-16T09:55:31.831 INFO:tasks.workunit.client.1.smithi176.stdout:5/272: truncate d4/d10/d17/f28 4817648 0 2024-07-16T09:55:31.832 INFO:tasks.workunit.client.0.smithi156.stdout:2/262: fdatasync d0/d5/d29/d2d/f40 0 2024-07-16T09:55:31.833 INFO:tasks.workunit.client.0.smithi156.stdout:3/118: write f0 [1543086,65876] 0 2024-07-16T09:55:31.834 INFO:tasks.workunit.client.0.smithi156.stdout:0/60: chown d0/f3 1205132 1 2024-07-16T09:55:31.835 INFO:tasks.workunit.client.0.smithi156.stdout:8/105: creat d0/dd/f26 x:0 0 0 2024-07-16T09:55:31.836 INFO:tasks.workunit.client.0.smithi156.stdout:1/134: mknod dc/d14/d1e/c2e 0 2024-07-16T09:55:31.836 INFO:tasks.workunit.client.0.smithi156.stdout:0/61: write d0/fa [1159688,90566] 0 2024-07-16T09:55:31.836 INFO:tasks.workunit.client.0.smithi156.stdout:6/176: getdents d3/d13 0 2024-07-16T09:55:31.839 INFO:tasks.workunit.client.0.smithi156.stdout:3/119: mkdir d8/da/d28 0 2024-07-16T09:55:31.840 INFO:tasks.workunit.client.1.smithi176.stdout:5/273: write d4/fd [332707,5040] 0 2024-07-16T09:55:31.840 INFO:tasks.workunit.client.0.smithi156.stdout:1/135: chown dc/c2b 342 1 2024-07-16T09:55:31.840 INFO:tasks.workunit.client.1.smithi176.stdout:5/274: dread - d4/d10/f5f zero size 2024-07-16T09:55:31.841 INFO:tasks.workunit.client.0.smithi156.stdout:4/157: dread d4/db/f16 [0,4194304] 0 2024-07-16T09:55:31.841 INFO:tasks.workunit.client.1.smithi176.stdout:5/275: chown d4/d14/d19/l57 78671 1 2024-07-16T09:55:31.844 INFO:tasks.workunit.client.1.smithi176.stdout:5/276: creat d4/d10/d17/f67 x:0 0 0 2024-07-16T09:55:31.846 INFO:tasks.workunit.client.0.smithi156.stdout:7/112: truncate fb 3075179 0 2024-07-16T09:55:31.846 INFO:tasks.workunit.client.0.smithi156.stdout:1/136: rmdir dc/d24 39 2024-07-16T09:55:31.847 INFO:tasks.workunit.client.1.smithi176.stdout:5/277: creat d4/f68 x:0 0 0 2024-07-16T09:55:31.847 INFO:tasks.workunit.client.0.smithi156.stdout:2/263: dwrite d0/f14 [0,4194304] 0 2024-07-16T09:55:31.849 INFO:tasks.workunit.client.0.smithi156.stdout:0/62: getdents d0 0 2024-07-16T09:55:31.849 INFO:tasks.workunit.client.0.smithi156.stdout:5/139: dread f13 [0,4194304] 0 2024-07-16T09:55:31.850 INFO:tasks.workunit.client.0.smithi156.stdout:8/106: dwrite d0/ff [0,4194304] 0 2024-07-16T09:55:31.850 INFO:tasks.workunit.client.1.smithi176.stdout:5/278: mknod d4/d14/d19/d1b/d61/c69 0 2024-07-16T09:55:31.856 INFO:tasks.workunit.client.0.smithi156.stdout:4/158: dread d4/d23/f17 [0,4194304] 0 2024-07-16T09:55:31.856 INFO:tasks.workunit.client.0.smithi156.stdout:8/107: fdatasync d0/dd/f10 0 2024-07-16T09:55:31.856 INFO:tasks.workunit.client.0.smithi156.stdout:6/177: creat d3/d26/d1f/d19/d29/d3b/f3c x:0 0 0 2024-07-16T09:55:31.859 INFO:tasks.workunit.client.1.smithi176.stdout:5/279: symlink d4/d14/d19/d1b/d36/d38/d39/l6a 0 2024-07-16T09:55:31.859 INFO:tasks.workunit.client.0.smithi156.stdout:3/120: link l6 d8/da/l29 0 2024-07-16T09:55:31.860 INFO:tasks.workunit.client.0.smithi156.stdout:1/137: write dc/d24/f2a [2420575,77154] 0 2024-07-16T09:55:31.861 INFO:tasks.workunit.client.0.smithi156.stdout:2/264: creat d0/d5/d29/d2d/f5a x:0 0 0 2024-07-16T09:55:31.863 INFO:tasks.workunit.client.1.smithi176.stdout:5/280: mkdir d4/d14/d3d/d4d/d6b 0 2024-07-16T09:55:31.863 INFO:tasks.workunit.client.0.smithi156.stdout:6/178: creat d3/d26/d1e/f3d x:0 0 0 2024-07-16T09:55:31.864 INFO:tasks.workunit.client.1.smithi176.stdout:5/281: fsync d4/f6 0 2024-07-16T09:55:31.864 INFO:tasks.workunit.client.0.smithi156.stdout:0/63: creat d0/d7/d9/fe x:0 0 0 2024-07-16T09:55:31.864 INFO:tasks.workunit.client.0.smithi156.stdout:1/138: write f4 [3320897,22850] 0 2024-07-16T09:55:31.866 INFO:tasks.workunit.client.0.smithi156.stdout:4/159: creat d4/d23/f39 x:0 0 0 2024-07-16T09:55:31.867 INFO:tasks.workunit.client.0.smithi156.stdout:5/140: dread f13 [0,4194304] 0 2024-07-16T09:55:31.867 INFO:tasks.workunit.client.0.smithi156.stdout:2/265: mkdir d0/d5/d29/d2d/d5b 0 2024-07-16T09:55:31.869 INFO:tasks.workunit.client.0.smithi156.stdout:3/121: creat d8/da/d28/f2a x:0 0 0 2024-07-16T09:55:31.871 INFO:tasks.workunit.client.0.smithi156.stdout:0/64: creat d0/ff x:0 0 0 2024-07-16T09:55:31.872 INFO:tasks.workunit.client.0.smithi156.stdout:7/113: dread dc/d13/f12 [0,4194304] 0 2024-07-16T09:55:31.872 INFO:tasks.workunit.client.0.smithi156.stdout:4/160: read d4/db/f2c [1571978,4159] 0 2024-07-16T09:55:31.873 INFO:tasks.workunit.client.1.smithi176.stdout:5/282: rmdir d4/d14/d19 39 2024-07-16T09:55:31.875 INFO:tasks.workunit.client.0.smithi156.stdout:8/108: dwrite d0/dd/f26 [0,4194304] 0 2024-07-16T09:55:31.875 INFO:tasks.workunit.client.1.smithi176.stdout:5/283: mkdir d4/d6c 0 2024-07-16T09:55:31.876 INFO:tasks.workunit.client.0.smithi156.stdout:0/65: fdatasync d0/d7/f8 0 2024-07-16T09:55:31.877 INFO:tasks.workunit.client.0.smithi156.stdout:5/141: symlink d1b/d1e/l2b 0 2024-07-16T09:55:31.878 INFO:tasks.workunit.client.0.smithi156.stdout:3/122: symlink d8/da/d13/l2b 0 2024-07-16T09:55:31.878 INFO:tasks.workunit.client.0.smithi156.stdout:8/109: dread - d0/d11/d20/f25 zero size 2024-07-16T09:55:31.878 INFO:tasks.workunit.client.0.smithi156.stdout:7/114: creat dc/d13/d20/f2b x:0 0 0 2024-07-16T09:55:31.878 INFO:tasks.workunit.client.1.smithi176.stdout:5/284: write d4/d14/d19/d1b/d36/d38/d39/f5d [1043899,60557] 0 2024-07-16T09:55:31.879 INFO:tasks.workunit.client.1.smithi176.stdout:5/285: chown d4/fd 60 1 2024-07-16T09:55:31.880 INFO:tasks.workunit.client.0.smithi156.stdout:0/66: write d0/fa [582618,17146] 0 2024-07-16T09:55:31.880 INFO:tasks.workunit.client.0.smithi156.stdout:7/115: fsync dc/f14 0 2024-07-16T09:55:31.880 INFO:tasks.workunit.client.0.smithi156.stdout:1/139: rename dc/d20/f22 to dc/f2f 0 2024-07-16T09:55:31.882 INFO:tasks.workunit.client.1.smithi176.stdout:5/286: symlink d4/d14/d19/d1b/d36/l6d 0 2024-07-16T09:55:31.883 INFO:tasks.workunit.client.0.smithi156.stdout:4/161: creat d4/d23/d30/d33/f3a x:0 0 0 2024-07-16T09:55:31.883 INFO:tasks.workunit.client.0.smithi156.stdout:6/179: dread d3/d26/d1f/f10 [0,4194304] 0 2024-07-16T09:55:31.884 INFO:tasks.workunit.client.1.smithi176.stdout:5/287: read d4/d37/f3f [19016,36204] 0 2024-07-16T09:55:31.885 INFO:tasks.workunit.client.1.smithi176.stdout:5/288: chown d4/d14/d3d/d4d 7011441 1 2024-07-16T09:55:31.886 INFO:tasks.workunit.client.0.smithi156.stdout:0/67: creat d0/d7/f10 x:0 0 0 2024-07-16T09:55:31.886 INFO:tasks.workunit.client.0.smithi156.stdout:4/162: write d4/db/f34 [2187215,40105] 0 2024-07-16T09:55:31.887 INFO:tasks.workunit.client.1.smithi176.stdout:5/289: truncate d4/d10/d17/f18 1938029 0 2024-07-16T09:55:31.888 INFO:tasks.workunit.client.1.smithi176.stdout:5/290: chown d4/d14/d19/l57 0 1 2024-07-16T09:55:31.890 INFO:tasks.workunit.client.0.smithi156.stdout:3/123: rename d8/da/d1d/d25 to d8/da/d1d/d2c 0 2024-07-16T09:55:31.890 INFO:tasks.workunit.client.0.smithi156.stdout:5/142: dwrite f14 [0,4194304] 0 2024-07-16T09:55:31.891 INFO:tasks.workunit.client.1.smithi176.stdout:5/291: mkdir d4/d14/d19/d1b/d36/d38/d6e 0 2024-07-16T09:55:31.891 INFO:tasks.workunit.client.0.smithi156.stdout:8/110: truncate d0/f2 1790618 0 2024-07-16T09:55:31.891 INFO:tasks.workunit.client.1.smithi176.stdout:5/292: read - d4/f68 zero size 2024-07-16T09:55:31.892 INFO:tasks.workunit.client.1.smithi176.stdout:5/293: chown d4/d14/c62 28 1 2024-07-16T09:55:31.892 INFO:tasks.workunit.client.0.smithi156.stdout:7/116: link dc/d13/d15/l2a dc/d13/d15/l2c 0 2024-07-16T09:55:31.892 INFO:tasks.workunit.client.0.smithi156.stdout:3/124: write f0 [3213040,3398] 0 2024-07-16T09:55:31.892 INFO:tasks.workunit.client.0.smithi156.stdout:4/163: mknod d4/d23/c3b 0 2024-07-16T09:55:31.895 INFO:tasks.workunit.client.0.smithi156.stdout:4/164: fdatasync d4/d23/f39 0 2024-07-16T09:55:31.895 INFO:tasks.workunit.client.0.smithi156.stdout:0/68: creat d0/d7/f11 x:0 0 0 2024-07-16T09:55:31.896 INFO:tasks.workunit.client.0.smithi156.stdout:3/125: fdatasync d8/da/d13/f20 0 2024-07-16T09:55:31.897 INFO:tasks.workunit.client.0.smithi156.stdout:5/143: creat d1b/d1e/f2c x:0 0 0 2024-07-16T09:55:31.899 INFO:tasks.workunit.client.0.smithi156.stdout:7/117: truncate dc/d13/d20/f2b 758162 0 2024-07-16T09:55:31.900 INFO:tasks.workunit.client.1.smithi176.stdout:5/294: link d4/d14/d3d/d4d/f64 d4/d6c/f6f 0 2024-07-16T09:55:31.900 INFO:tasks.workunit.client.0.smithi156.stdout:4/165: creat d4/d23/d30/d33/f3c x:0 0 0 2024-07-16T09:55:31.900 INFO:tasks.workunit.client.0.smithi156.stdout:7/118: readlink dc/d13/l17 0 2024-07-16T09:55:31.902 INFO:tasks.workunit.client.1.smithi176.stdout:5/295: write d4/d14/d19/d1b/d36/d38/d39/f56 [400641,91791] 0 2024-07-16T09:55:31.903 INFO:tasks.workunit.client.0.smithi156.stdout:0/69: dread d0/d7/f8 [0,4194304] 0 2024-07-16T09:55:31.903 INFO:tasks.workunit.client.1.smithi176.stdout:5/296: fdatasync d4/d10/d17/f2f 0 2024-07-16T09:55:31.904 INFO:tasks.workunit.client.0.smithi156.stdout:0/70: fsync d0/ff 0 2024-07-16T09:55:31.904 INFO:tasks.workunit.client.0.smithi156.stdout:1/140: getdents dc/d20 0 2024-07-16T09:55:31.907 INFO:tasks.workunit.client.0.smithi156.stdout:7/119: dread fb [0,4194304] 0 2024-07-16T09:55:31.907 INFO:tasks.workunit.client.0.smithi156.stdout:4/166: dread d4/db/f1d [0,4194304] 0 2024-07-16T09:55:31.908 INFO:tasks.workunit.client.0.smithi156.stdout:1/141: write dc/f2f [589411,76554] 0 2024-07-16T09:55:31.909 INFO:tasks.workunit.client.0.smithi156.stdout:0/71: creat d0/d7/d9/f12 x:0 0 0 2024-07-16T09:55:31.909 INFO:tasks.workunit.client.1.smithi176.stdout:5/297: truncate d4/f27 2551915 0 2024-07-16T09:55:31.910 INFO:tasks.workunit.client.0.smithi156.stdout:0/72: fsync d0/d7/f8 0 2024-07-16T09:55:31.910 INFO:tasks.workunit.client.0.smithi156.stdout:6/180: dread d3/d26/f14 [0,4194304] 0 2024-07-16T09:55:31.911 INFO:tasks.workunit.client.0.smithi156.stdout:4/167: creat d4/f3d x:0 0 0 2024-07-16T09:55:31.913 INFO:tasks.workunit.client.0.smithi156.stdout:4/168: write d4/d23/d30/d33/f3a [587763,5868] 0 2024-07-16T09:55:31.914 INFO:tasks.workunit.client.0.smithi156.stdout:1/142: creat dc/d20/f30 x:0 0 0 2024-07-16T09:55:31.914 INFO:tasks.workunit.client.0.smithi156.stdout:7/120: mknod dc/c2d 0 2024-07-16T09:55:31.915 INFO:tasks.workunit.client.0.smithi156.stdout:0/73: rename d0/cb to d0/d7/c13 0 2024-07-16T09:55:31.916 INFO:tasks.workunit.client.0.smithi156.stdout:8/111: truncate d0/ff 591262 0 2024-07-16T09:55:31.917 INFO:tasks.workunit.client.0.smithi156.stdout:8/112: dread - d0/d11/f16 zero size 2024-07-16T09:55:31.917 INFO:tasks.workunit.client.0.smithi156.stdout:1/143: mknod dc/d20/c31 0 2024-07-16T09:55:31.918 INFO:tasks.workunit.client.1.smithi176.stdout:5/298: write d4/d14/d19/d1b/f26 [2032566,10285] 0 2024-07-16T09:55:31.920 INFO:tasks.workunit.client.0.smithi156.stdout:0/74: creat d0/f14 x:0 0 0 2024-07-16T09:55:31.922 INFO:tasks.workunit.client.0.smithi156.stdout:4/169: dread d4/d23/d30/d33/f3a [0,4194304] 0 2024-07-16T09:55:31.922 INFO:tasks.workunit.client.0.smithi156.stdout:1/144: creat dc/d14/f32 x:0 0 0 2024-07-16T09:55:31.923 INFO:tasks.workunit.client.1.smithi176.stdout:5/299: creat d4/d14/d19/d1b/d36/d38/f70 x:0 0 0 2024-07-16T09:55:31.924 INFO:tasks.workunit.client.0.smithi156.stdout:4/170: fsync d4/f5 0 2024-07-16T09:55:31.925 INFO:tasks.workunit.client.0.smithi156.stdout:0/75: creat d0/d7/d9/f15 x:0 0 0 2024-07-16T09:55:31.927 INFO:tasks.workunit.client.1.smithi176.stdout:5/300: rmdir d4/d14/d3d/d4d/d6b 0 2024-07-16T09:55:31.927 INFO:tasks.workunit.client.0.smithi156.stdout:0/76: stat d0/f14 0 2024-07-16T09:55:31.928 INFO:tasks.workunit.client.0.smithi156.stdout:1/145: write dc/d14/d1e/f29 [322406,78657] 0 2024-07-16T09:55:31.930 INFO:tasks.workunit.client.1.smithi176.stdout:5/301: creat d4/f71 x:0 0 0 2024-07-16T09:55:31.931 INFO:tasks.workunit.client.0.smithi156.stdout:3/126: dwrite d8/fb [0,4194304] 0 2024-07-16T09:55:31.931 INFO:tasks.workunit.client.0.smithi156.stdout:0/77: dread - d0/d7/d9/f12 zero size 2024-07-16T09:55:31.932 INFO:tasks.workunit.client.1.smithi176.stdout:5/302: fdatasync d4/d10/d17/f28 0 2024-07-16T09:55:31.932 INFO:tasks.workunit.client.0.smithi156.stdout:1/146: dread - dc/d14/f1d zero size 2024-07-16T09:55:31.932 INFO:tasks.workunit.client.0.smithi156.stdout:7/121: dwrite dc/d13/f12 [0,4194304] 0 2024-07-16T09:55:31.932 INFO:tasks.workunit.client.1.smithi176.stdout:5/303: chown d4/d10/d17/c2b 0 1 2024-07-16T09:55:31.933 INFO:tasks.workunit.client.0.smithi156.stdout:4/171: rmdir d4/d23/d30/d33/d35 0 2024-07-16T09:55:31.939 INFO:tasks.workunit.client.0.smithi156.stdout:0/78: chown d0/d7/d9/fe 16328315 1 2024-07-16T09:55:31.940 INFO:tasks.workunit.client.0.smithi156.stdout:7/122: write dc/d13/d20/f2b [230267,65737] 0 2024-07-16T09:55:31.941 INFO:tasks.workunit.client.1.smithi176.stdout:5/304: rename d4/d14/d19/d1b/d36 to d4/d37/d72 0 2024-07-16T09:55:31.942 INFO:tasks.workunit.client.0.smithi156.stdout:8/113: dwrite d0/f1 [0,4194304] 0 2024-07-16T09:55:31.943 INFO:tasks.workunit.client.0.smithi156.stdout:4/172: truncate d4/db/dc/f2a 714766 0 2024-07-16T09:55:31.946 INFO:tasks.workunit.client.0.smithi156.stdout:3/127: dread d8/fb [0,4194304] 0 2024-07-16T09:55:31.946 INFO:tasks.workunit.client.1.smithi176.stdout:5/305: rename d4/d37/d72/l44 to d4/d37/d72/l73 0 2024-07-16T09:55:31.947 INFO:tasks.workunit.client.1.smithi176.stdout:5/306: write d4/d14/d3d/f41 [2544990,69595] 0 2024-07-16T09:55:31.953 INFO:tasks.workunit.client.0.smithi156.stdout:0/79: dread d0/fa [0,4194304] 0 2024-07-16T09:55:31.956 INFO:tasks.workunit.client.0.smithi156.stdout:0/80: chown d0/d7/f10 1 1 2024-07-16T09:55:31.957 INFO:tasks.workunit.client.0.smithi156.stdout:0/81: dread - d0/f14 zero size 2024-07-16T09:55:31.958 INFO:tasks.workunit.client.0.smithi156.stdout:7/123: dwrite dc/d13/d20/f2b [0,4194304] 0 2024-07-16T09:55:31.959 INFO:tasks.workunit.client.1.smithi176.stdout:5/307: write d4/d6c/f6f [764965,19827] 0 2024-07-16T09:55:31.959 INFO:tasks.workunit.client.0.smithi156.stdout:3/128: dwrite d8/f1a [0,4194304] 0 2024-07-16T09:55:31.961 INFO:tasks.workunit.client.1.smithi176.stdout:5/308: creat d4/d37/d72/d38/d6e/f74 x:0 0 0 2024-07-16T09:55:31.961 INFO:tasks.workunit.client.0.smithi156.stdout:3/129: write f3 [4206056,83268] 0 2024-07-16T09:55:31.964 INFO:tasks.workunit.client.1.smithi176.stdout:5/309: rename d4/la to d4/d14/d19/d1b/l75 0 2024-07-16T09:55:31.967 INFO:tasks.workunit.client.1.smithi176.stdout:5/310: creat d4/d14/d19/d1b/d61/f76 x:0 0 0 2024-07-16T09:55:31.973 INFO:tasks.workunit.client.1.smithi176.stdout:5/311: dwrite d4/d37/d72/d38/d6e/f74 [0,4194304] 0 2024-07-16T09:55:31.981 INFO:tasks.workunit.client.1.smithi176.stdout:5/312: dwrite d4/d10/d17/f50 [0,4194304] 0 2024-07-16T09:55:31.982 INFO:tasks.workunit.client.1.smithi176.stdout:5/313: readlink d4/d10/d17/l46 0 2024-07-16T09:55:31.994 INFO:tasks.workunit.client.0.smithi156.stdout:9/157: sync 2024-07-16T09:55:31.995 INFO:tasks.workunit.client.0.smithi156.stdout:9/158: chown c0 1 1 2024-07-16T09:55:32.001 INFO:tasks.workunit.client.1.smithi176.stdout:5/314: dread d4/d10/f15 [0,4194304] 0 2024-07-16T09:55:32.005 INFO:tasks.workunit.client.1.smithi176.stdout:5/315: dwrite d4/d10/f22 [0,4194304] 0 2024-07-16T09:55:32.006 INFO:tasks.workunit.client.1.smithi176.stdout:5/316: mknod d4/d10/d17/c77 0 2024-07-16T09:55:32.012 INFO:tasks.workunit.client.1.smithi176.stdout:5/317: write d4/f5 [721622,47078] 0 2024-07-16T09:55:32.013 INFO:tasks.workunit.client.1.smithi176.stdout:5/318: truncate d4/d37/f3f 987241 0 2024-07-16T09:55:32.015 INFO:tasks.workunit.client.1.smithi176.stdout:5/319: mkdir d4/d37/d72/d78 0 2024-07-16T09:55:32.021 INFO:tasks.workunit.client.1.smithi176.stdout:5/320: dwrite d4/d37/d72/f4e [0,4194304] 0 2024-07-16T09:55:32.023 INFO:tasks.workunit.client.1.smithi176.stdout:5/321: creat d4/d14/d3d/f79 x:0 0 0 2024-07-16T09:55:32.025 INFO:tasks.workunit.client.1.smithi176.stdout:5/322: symlink d4/d37/d72/d38/d39/l7a 0 2024-07-16T09:55:32.027 INFO:tasks.workunit.client.1.smithi176.stdout:5/323: creat d4/d37/d72/d38/d6e/f7b x:0 0 0 2024-07-16T09:55:32.033 INFO:tasks.workunit.client.0.smithi156.stdout:8/114: symlink d0/l27 0 2024-07-16T09:55:32.036 INFO:tasks.workunit.client.0.smithi156.stdout:0/82: unlink d0/d7/f8 0 2024-07-16T09:55:32.038 INFO:tasks.workunit.client.0.smithi156.stdout:7/124: symlink dc/d13/d15/l2e 0 2024-07-16T09:55:32.038 INFO:tasks.workunit.client.0.smithi156.stdout:3/130: symlink d8/da/l2d 0 2024-07-16T09:55:32.040 INFO:tasks.workunit.client.0.smithi156.stdout:7/125: readlink dc/d13/l17 0 2024-07-16T09:55:32.040 INFO:tasks.workunit.client.0.smithi156.stdout:1/147: dwrite f5 [0,4194304] 0 2024-07-16T09:55:32.041 INFO:tasks.workunit.client.1.smithi176.stdout:5/324: dread d4/d14/d19/f21 [0,4194304] 0 2024-07-16T09:55:32.042 INFO:tasks.workunit.client.0.smithi156.stdout:7/126: fsync fb 0 2024-07-16T09:55:32.044 INFO:tasks.workunit.client.0.smithi156.stdout:3/131: creat d8/da/f2e x:0 0 0 2024-07-16T09:55:32.045 INFO:tasks.workunit.client.0.smithi156.stdout:4/173: link d4/db/dc/l13 d4/l3e 0 2024-07-16T09:55:32.045 INFO:tasks.workunit.client.0.smithi156.stdout:9/159: truncate d2/d5/fb 1270109 0 2024-07-16T09:55:32.046 INFO:tasks.workunit.client.1.smithi176.stdout:5/325: creat d4/d6c/f7c x:0 0 0 2024-07-16T09:55:32.047 INFO:tasks.workunit.client.0.smithi156.stdout:8/115: dwrite d0/d11/d20/f25 [0,4194304] 0 2024-07-16T09:55:32.047 INFO:tasks.workunit.client.0.smithi156.stdout:1/148: write dc/d20/f30 [696328,53269] 0 2024-07-16T09:55:32.049 INFO:tasks.workunit.client.0.smithi156.stdout:7/127: creat dc/d13/d20/f2f x:0 0 0 2024-07-16T09:55:32.050 INFO:tasks.workunit.client.0.smithi156.stdout:0/83: dwrite d0/d7/d9/fe [0,4194304] 0 2024-07-16T09:55:32.051 INFO:tasks.workunit.client.1.smithi176.stdout:5/326: dread d4/d37/f5a [0,4194304] 0 2024-07-16T09:55:32.052 INFO:tasks.workunit.client.0.smithi156.stdout:7/128: write fb [622052,53190] 0 2024-07-16T09:55:32.053 INFO:tasks.workunit.client.1.smithi176.stdout:5/327: fdatasync d4/d10/d17/f66 0 2024-07-16T09:55:32.053 INFO:tasks.workunit.client.0.smithi156.stdout:9/160: rmdir d2/d5/d1d 39 2024-07-16T09:55:32.054 INFO:tasks.workunit.client.0.smithi156.stdout:4/174: write d4/d23/f17 [1015350,124697] 0 2024-07-16T09:55:32.057 INFO:tasks.workunit.client.0.smithi156.stdout:8/116: symlink d0/dd/d1f/l28 0 2024-07-16T09:55:32.057 INFO:tasks.workunit.client.0.smithi156.stdout:0/84: mknod d0/d7/c16 0 2024-07-16T09:55:32.059 INFO:tasks.workunit.client.0.smithi156.stdout:4/175: dread d4/d23/f32 [0,4194304] 0 2024-07-16T09:55:32.059 INFO:tasks.workunit.client.0.smithi156.stdout:7/129: mknod dc/d13/c30 0 2024-07-16T09:55:32.060 INFO:tasks.workunit.client.0.smithi156.stdout:0/85: write d0/d7/d9/f12 [771591,58728] 0 2024-07-16T09:55:32.061 INFO:tasks.workunit.client.1.smithi176.stdout:5/328: dwrite d4/f13 [0,4194304] 0 2024-07-16T09:55:32.064 INFO:tasks.workunit.client.1.smithi176.stdout:5/329: read d4/d6c/f6f [2122613,64140] 0 2024-07-16T09:55:32.065 INFO:tasks.workunit.client.1.smithi176.stdout:5/330: chown d4/d10/f5f 2141 1 2024-07-16T09:55:32.073 INFO:tasks.workunit.client.1.smithi176.stdout:5/331: dwrite d4/d14/d3d/f55 [0,4194304] 0 2024-07-16T09:55:32.076 INFO:tasks.workunit.client.1.smithi176.stdout:5/332: mknod d4/d37/d72/c7d 0 2024-07-16T09:55:32.079 INFO:tasks.workunit.client.0.smithi156.stdout:9/161: unlink d2/f3 0 2024-07-16T09:55:32.081 INFO:tasks.workunit.client.0.smithi156.stdout:7/130: read dc/d13/d20/f2b [2961329,93487] 0 2024-07-16T09:55:32.082 INFO:tasks.workunit.client.0.smithi156.stdout:3/132: dwrite d8/da/d13/f17 [0,4194304] 0 2024-07-16T09:55:32.083 INFO:tasks.workunit.client.0.smithi156.stdout:9/162: write d2/f4 [240107,70539] 0 2024-07-16T09:55:32.087 INFO:tasks.workunit.client.0.smithi156.stdout:2/266: sync 2024-07-16T09:55:32.088 INFO:tasks.workunit.client.0.smithi156.stdout:2/267: write d0/d5/d29/d2d/f5a [242069,96935] 0 2024-07-16T09:55:32.105 INFO:tasks.workunit.client.0.smithi156.stdout:3/133: dread d8/da/fc [0,4194304] 0 2024-07-16T09:55:32.158 INFO:tasks.workunit.client.1.smithi176.stdout:2/185: sync 2024-07-16T09:55:32.160 INFO:tasks.workunit.client.1.smithi176.stdout:2/186: mkdir dd/d16/d39 0 2024-07-16T09:55:32.162 INFO:tasks.workunit.client.1.smithi176.stdout:2/187: creat dd/d16/f3a x:0 0 0 2024-07-16T09:55:32.164 INFO:tasks.workunit.client.1.smithi176.stdout:2/188: creat dd/d2e/d35/f3b x:0 0 0 2024-07-16T09:55:32.165 INFO:tasks.workunit.client.1.smithi176.stdout:2/189: write f3 [7227362,11405] 0 2024-07-16T09:55:32.168 INFO:tasks.workunit.client.1.smithi176.stdout:2/190: link f9 dd/f3c 0 2024-07-16T09:55:32.169 INFO:tasks.workunit.client.1.smithi176.stdout:2/191: read f3 [4806305,56589] 0 2024-07-16T09:55:32.170 INFO:tasks.workunit.client.1.smithi176.stdout:2/192: write dd/d2e/f37 [856070,31988] 0 2024-07-16T09:55:32.174 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:55:31 smithi156 ceph-mon[29302]: pgmap v247: 65 pgs: 65 active+clean; 523 MiB data, 3.3 GiB used, 533 GiB / 536 GiB avail; 3.3 MiB/s rd, 46 MiB/s wr, 411 op/s 2024-07-16T09:55:32.175 INFO:tasks.workunit.client.1.smithi176.stdout:2/193: dread dd/f2a [0,4194304] 0 2024-07-16T09:55:32.176 INFO:tasks.workunit.client.1.smithi176.stdout:2/194: chown dd/de/l14 229901666 1 2024-07-16T09:55:32.178 INFO:tasks.workunit.client.1.smithi176.stdout:2/195: mkdir dd/d16/d39/d3d 0 2024-07-16T09:55:32.178 INFO:tasks.workunit.client.1.smithi176.stdout:2/196: readlink dd/l31 0 2024-07-16T09:55:32.179 INFO:tasks.workunit.client.1.smithi176.stdout:2/197: creat dd/d2e/f3e x:0 0 0 2024-07-16T09:55:32.181 INFO:tasks.workunit.client.1.smithi176.stdout:2/198: truncate dd/de/f10 4740284 0 2024-07-16T09:55:32.181 INFO:tasks.workunit.client.1.smithi176.stdout:2/199: chown dd/d16 10762 1 2024-07-16T09:55:32.182 INFO:tasks.workunit.client.1.smithi176.stdout:2/200: creat dd/d2e/d35/f3f x:0 0 0 2024-07-16T09:55:32.183 INFO:tasks.workunit.client.1.smithi176.stdout:2/201: chown dd/cf 389483 1 2024-07-16T09:55:32.184 INFO:tasks.workunit.client.1.smithi176.stdout:2/202: truncate dd/d2e/f3e 750156 0 2024-07-16T09:55:32.185 INFO:tasks.workunit.client.1.smithi176.stdout:2/203: write dd/d16/f32 [719518,81891] 0 2024-07-16T09:55:32.187 INFO:tasks.workunit.client.1.smithi176.stdout:2/204: mknod dd/d16/c40 0 2024-07-16T09:55:32.188 INFO:tasks.workunit.client.1.smithi176.stdout:2/205: symlink dd/l41 0 2024-07-16T09:55:32.191 INFO:tasks.workunit.client.1.smithi176.stdout:2/206: link dd/de/l38 dd/de/l42 0 2024-07-16T09:55:32.191 INFO:tasks.workunit.client.1.smithi176.stdout:2/207: chown dd/d36/l23 3789178 1 2024-07-16T09:55:32.198 INFO:tasks.workunit.client.1.smithi176.stdout:2/208: getdents dd/d16 0 2024-07-16T09:55:32.200 INFO:tasks.workunit.client.1.smithi176.stdout:2/209: link dd/l31 dd/de/l43 0 2024-07-16T09:55:32.201 INFO:tasks.workunit.client.1.smithi176.stdout:2/210: dread - dd/d2e/d35/f3b zero size 2024-07-16T09:55:32.201 INFO:tasks.workunit.client.1.smithi176.stdout:2/211: stat dd/cf 0 2024-07-16T09:55:32.203 INFO:tasks.workunit.client.0.smithi156.stdout:1/149: creat dc/f33 x:0 0 0 2024-07-16T09:55:32.203 INFO:tasks.workunit.client.0.smithi156.stdout:0/86: mknod d0/d7/c17 0 2024-07-16T09:55:32.204 INFO:tasks.workunit.client.0.smithi156.stdout:0/87: dread - d0/d7/d9/f15 zero size 2024-07-16T09:55:32.206 INFO:tasks.workunit.client.0.smithi156.stdout:0/88: read d0/f3 [4034024,126942] 0 2024-07-16T09:55:32.206 INFO:tasks.workunit.client.0.smithi156.stdout:4/176: rename d4/d23/d30/d33/f3c to d4/db/f3f 0 2024-07-16T09:55:32.208 INFO:tasks.workunit.client.0.smithi156.stdout:7/131: symlink dc/d13/l31 0 2024-07-16T09:55:32.208 INFO:tasks.workunit.client.1.smithi176.stdout:2/212: dwrite dd/d2e/d35/f3f [0,4194304] 0 2024-07-16T09:55:32.208 INFO:tasks.workunit.client.1.smithi176.stdout:2/213: fsync dd/d16/f1c 0 2024-07-16T09:55:32.209 INFO:tasks.workunit.client.0.smithi156.stdout:8/117: mkdir d0/d11/d29 0 2024-07-16T09:55:32.210 INFO:tasks.workunit.client.1.smithi176.stdout:2/214: mknod dd/d16/d39/c44 0 2024-07-16T09:55:32.211 INFO:tasks.workunit.client.0.smithi156.stdout:8/118: chown d0/d11/l23 342 1 2024-07-16T09:55:32.211 INFO:tasks.workunit.client.0.smithi156.stdout:9/163: fsync d2/de/f2f 0 2024-07-16T09:55:32.212 INFO:tasks.workunit.client.0.smithi156.stdout:0/89: dread d0/d7/d9/fe [0,4194304] 0 2024-07-16T09:55:32.213 INFO:tasks.workunit.client.1.smithi176.stdout:2/215: mkdir dd/d36/d45 0 2024-07-16T09:55:32.213 INFO:tasks.workunit.client.0.smithi156.stdout:1/150: creat dc/d14/f34 x:0 0 0 2024-07-16T09:55:32.214 INFO:tasks.workunit.client.1.smithi176.stdout:2/216: read dd/d16/f1c [1056095,21558] 0 2024-07-16T09:55:32.215 INFO:tasks.workunit.client.1.smithi176.stdout:2/217: stat dd/d2e 0 2024-07-16T09:55:32.216 INFO:tasks.workunit.client.0.smithi156.stdout:4/177: dwrite d4/d23/d30/f25 [0,4194304] 0 2024-07-16T09:55:32.217 INFO:tasks.workunit.client.1.smithi176.stdout:2/218: dread dd/d2e/f3e [0,4194304] 0 2024-07-16T09:55:32.217 INFO:tasks.workunit.client.1.smithi176.stdout:2/219: fsync dd/d16/f32 0 2024-07-16T09:55:32.219 INFO:tasks.workunit.client.1.smithi176.stdout:2/220: write dd/de/f10 [1508736,104409] 0 2024-07-16T09:55:32.220 INFO:tasks.workunit.client.0.smithi156.stdout:6/181: sync 2024-07-16T09:55:32.220 INFO:tasks.workunit.client.0.smithi156.stdout:5/144: sync 2024-07-16T09:55:32.222 INFO:tasks.workunit.client.0.smithi156.stdout:9/164: creat d2/d5/d16/d22/d26/f31 x:0 0 0 2024-07-16T09:55:32.223 INFO:tasks.workunit.client.1.smithi176.stdout:2/221: rmdir dd/d2e 39 2024-07-16T09:55:32.223 INFO:tasks.workunit.client.0.smithi156.stdout:7/132: mknod dc/c32 0 2024-07-16T09:55:32.223 INFO:tasks.workunit.client.0.smithi156.stdout:9/165: stat d2/de/f2f 0 2024-07-16T09:55:32.224 INFO:tasks.workunit.client.1.smithi176.stdout:2/222: truncate f3 8746148 0 2024-07-16T09:55:32.224 INFO:tasks.workunit.client.0.smithi156.stdout:1/151: unlink dc/d20/l25 0 2024-07-16T09:55:32.225 INFO:tasks.workunit.client.0.smithi156.stdout:3/134: link d8/l16 d8/da/d1d/l2f 0 2024-07-16T09:55:32.225 INFO:tasks.workunit.client.0.smithi156.stdout:1/152: write dc/f28 [790768,115965] 0 2024-07-16T09:55:32.227 INFO:tasks.workunit.client.0.smithi156.stdout:4/178: mknod d4/d23/d30/d33/c40 0 2024-07-16T09:55:32.227 INFO:tasks.workunit.client.1.smithi176.stdout:2/223: mknod dd/d2e/c46 0 2024-07-16T09:55:32.228 INFO:tasks.workunit.client.0.smithi156.stdout:8/119: mknod d0/d11/c2a 0 2024-07-16T09:55:32.229 INFO:tasks.workunit.client.1.smithi176.stdout:2/224: creat dd/d2e/d35/f47 x:0 0 0 2024-07-16T09:55:32.230 INFO:tasks.workunit.client.0.smithi156.stdout:5/145: mkdir d1b/d28/d2d 0 2024-07-16T09:55:32.231 INFO:tasks.workunit.client.0.smithi156.stdout:7/133: unlink dc/c2d 0 2024-07-16T09:55:32.231 INFO:tasks.workunit.client.1.smithi176.stdout:2/225: write f3 [7206463,63079] 0 2024-07-16T09:55:32.232 INFO:tasks.workunit.client.0.smithi156.stdout:9/166: symlink d2/d5/d16/d22/d26/l32 0 2024-07-16T09:55:32.233 INFO:tasks.workunit.client.0.smithi156.stdout:3/135: rmdir d8/da/d1d/d2c 39 2024-07-16T09:55:32.233 INFO:tasks.workunit.client.0.smithi156.stdout:1/153: fdatasync dc/d20/f23 0 2024-07-16T09:55:32.234 INFO:tasks.workunit.client.1.smithi176.stdout:2/226: creat dd/f48 x:0 0 0 2024-07-16T09:55:32.234 INFO:tasks.workunit.client.0.smithi156.stdout:9/167: write d2/f15 [2437554,125651] 0 2024-07-16T09:55:32.236 INFO:tasks.workunit.client.0.smithi156.stdout:8/120: dwrite d0/dd/f26 [0,4194304] 0 2024-07-16T09:55:32.236 INFO:tasks.workunit.client.1.smithi176.stdout:2/227: read f9 [2783094,96669] 0 2024-07-16T09:55:32.239 INFO:tasks.workunit.client.0.smithi156.stdout:6/182: creat d3/d26/d1f/d19/d2e/f3e x:0 0 0 2024-07-16T09:55:32.239 INFO:tasks.workunit.client.1.smithi176.stdout:2/228: mkdir dd/d49 0 2024-07-16T09:55:32.240 INFO:tasks.workunit.client.0.smithi156.stdout:0/90: dread d0/f3 [0,4194304] 0 2024-07-16T09:55:32.240 INFO:tasks.workunit.client.0.smithi156.stdout:6/183: dread - d3/f2a zero size 2024-07-16T09:55:32.240 INFO:tasks.workunit.client.1.smithi176.stdout:2/229: fsync dd/d36/f2f 0 2024-07-16T09:55:32.241 INFO:tasks.workunit.client.0.smithi156.stdout:7/134: creat dc/d13/f33 x:0 0 0 2024-07-16T09:55:32.242 INFO:tasks.workunit.client.0.smithi156.stdout:0/91: dread - d0/d7/f10 zero size 2024-07-16T09:55:32.243 INFO:tasks.workunit.client.1.smithi176.stdout:2/230: stat dd/l31 0 2024-07-16T09:55:32.244 INFO:tasks.workunit.client.0.smithi156.stdout:1/154: dwrite dc/f28 [0,4194304] 0 2024-07-16T09:55:32.245 INFO:tasks.workunit.client.1.smithi176.stdout:2/231: mkdir dd/d2e/d35/d4a 0 2024-07-16T09:55:32.246 INFO:tasks.workunit.client.0.smithi156.stdout:8/121: symlink d0/dd/l2b 0 2024-07-16T09:55:32.247 INFO:tasks.workunit.client.0.smithi156.stdout:2/268: sync 2024-07-16T09:55:32.248 INFO:tasks.workunit.client.0.smithi156.stdout:5/146: creat d1b/d28/d2d/f2e x:0 0 0 2024-07-16T09:55:32.249 INFO:tasks.workunit.client.0.smithi156.stdout:7/135: dread fb [0,4194304] 0 2024-07-16T09:55:32.250 INFO:tasks.workunit.client.0.smithi156.stdout:6/184: rmdir d3/d26/d1e 39 2024-07-16T09:55:32.250 INFO:tasks.workunit.client.1.smithi176.stdout:2/232: truncate f9 4332643 0 2024-07-16T09:55:32.250 INFO:tasks.workunit.client.0.smithi156.stdout:3/136: rmdir d8/da/d1d/d2c 39 2024-07-16T09:55:32.251 INFO:tasks.workunit.client.1.smithi176.stdout:2/233: chown dd/d2e/d35/f47 495879 1 2024-07-16T09:55:32.252 INFO:tasks.workunit.client.0.smithi156.stdout:0/92: dread d0/f3 [0,4194304] 0 2024-07-16T09:55:32.252 INFO:tasks.workunit.client.1.smithi176.stdout:0/247: sync 2024-07-16T09:55:32.252 INFO:tasks.workunit.client.1.smithi176.stdout:3/121: sync 2024-07-16T09:55:32.253 INFO:tasks.workunit.client.1.smithi176.stdout:6/196: sync 2024-07-16T09:55:32.253 INFO:tasks.workunit.client.0.smithi156.stdout:9/168: rename d2/d5/d1d to d2/d5/d16/d22/d26/d33 0 2024-07-16T09:55:32.253 INFO:tasks.workunit.client.1.smithi176.stdout:2/234: fdatasync dd/d2e/f3e 0 2024-07-16T09:55:32.255 INFO:tasks.workunit.client.0.smithi156.stdout:1/155: dwrite dc/d24/f2a [0,4194304] 0 2024-07-16T09:55:32.255 INFO:tasks.workunit.client.0.smithi156.stdout:9/169: chown d2/d5/d16/f20 388865 1 2024-07-16T09:55:32.256 INFO:tasks.workunit.client.1.smithi176.stdout:6/197: truncate d1/d5/f2c 358349 0 2024-07-16T09:55:32.256 INFO:tasks.workunit.client.0.smithi156.stdout:8/122: unlink d0/d11/c2a 0 2024-07-16T09:55:32.257 INFO:tasks.workunit.client.0.smithi156.stdout:1/156: write dc/d14/d1e/f29 [417730,124641] 0 2024-07-16T09:55:32.258 INFO:tasks.workunit.client.1.smithi176.stdout:0/248: rename d5/f16 to d5/d1c/d24/d26/f4f 0 2024-07-16T09:55:32.259 INFO:tasks.workunit.client.1.smithi176.stdout:2/235: dread dd/d16/f32 [0,4194304] 0 2024-07-16T09:55:32.259 INFO:tasks.workunit.client.1.smithi176.stdout:3/122: truncate d3/da/f13 596568 0 2024-07-16T09:55:32.261 INFO:tasks.workunit.client.1.smithi176.stdout:2/236: write dd/d2e/d35/f3f [5042001,68648] 0 2024-07-16T09:55:32.262 INFO:tasks.workunit.client.1.smithi176.stdout:2/237: chown c8 0 1 2024-07-16T09:55:32.263 INFO:tasks.workunit.client.1.smithi176.stdout:6/198: creat d1/f45 x:0 0 0 2024-07-16T09:55:32.267 INFO:tasks.workunit.client.1.smithi176.stdout:6/199: mknod d1/d5/d7/d1e/d29/c46 0 2024-07-16T09:55:32.268 INFO:tasks.workunit.client.1.smithi176.stdout:6/200: write d1/dc/f33 [116365,26867] 0 2024-07-16T09:55:32.269 INFO:tasks.workunit.client.1.smithi176.stdout:0/249: write d5/d39/f3a [1031054,114881] 0 2024-07-16T09:55:32.271 INFO:tasks.workunit.client.1.smithi176.stdout:0/250: mkdir d5/d1c/d24/d35/d50 0 2024-07-16T09:55:32.272 INFO:tasks.workunit.client.1.smithi176.stdout:0/251: creat d5/d39/f51 x:0 0 0 2024-07-16T09:55:32.273 INFO:tasks.workunit.client.1.smithi176.stdout:0/252: chown d5/ca 420571063 1 2024-07-16T09:55:32.273 INFO:tasks.workunit.client.1.smithi176.stdout:0/253: chown d5 42820 1 2024-07-16T09:55:32.277 INFO:tasks.workunit.client.1.smithi176.stdout:3/123: truncate d3/da/f19 1411504 0 2024-07-16T09:55:32.279 INFO:tasks.workunit.client.1.smithi176.stdout:0/254: write d5/d1c/f20 [1147797,59429] 0 2024-07-16T09:55:32.280 INFO:tasks.workunit.client.1.smithi176.stdout:3/124: getdents d3 0 2024-07-16T09:55:32.282 INFO:tasks.workunit.client.1.smithi176.stdout:0/255: write d5/d39/f51 [673071,10985] 0 2024-07-16T09:55:32.284 INFO:tasks.workunit.client.0.smithi156.stdout:0/93: rename d0/d7/d9/cd to d0/d7/c18 0 2024-07-16T09:55:32.284 INFO:tasks.workunit.client.0.smithi156.stdout:9/170: symlink d2/d5/d16/l34 0 2024-07-16T09:55:32.284 INFO:tasks.workunit.client.1.smithi176.stdout:6/201: dwrite d1/f15 [0,4194304] 0 2024-07-16T09:55:32.285 INFO:tasks.workunit.client.0.smithi156.stdout:8/123: creat d0/f2c x:0 0 0 2024-07-16T09:55:32.285 INFO:tasks.workunit.client.1.smithi176.stdout:6/202: dread - d1/f31 zero size 2024-07-16T09:55:32.286 INFO:tasks.workunit.client.0.smithi156.stdout:0/94: chown d0/f3 15697200 1 2024-07-16T09:55:32.286 INFO:tasks.workunit.client.0.smithi156.stdout:1/157: creat dc/d14/d1e/f35 x:0 0 0 2024-07-16T09:55:32.286 INFO:tasks.workunit.client.1.smithi176.stdout:6/203: chown d1/dc/d40 53339 1 2024-07-16T09:55:32.291 INFO:tasks.workunit.client.1.smithi176.stdout:3/125: dwrite d3/da/fe [0,4194304] 0 2024-07-16T09:55:32.291 INFO:tasks.workunit.client.0.smithi156.stdout:8/124: creat d0/dd/f2d x:0 0 0 2024-07-16T09:55:32.291 INFO:tasks.workunit.client.0.smithi156.stdout:5/147: creat d1b/f2f x:0 0 0 2024-07-16T09:55:32.292 INFO:tasks.workunit.client.1.smithi176.stdout:3/126: readlink l0 0 2024-07-16T09:55:32.293 INFO:tasks.workunit.client.0.smithi156.stdout:9/171: dwrite d2/f15 [0,4194304] 0 2024-07-16T09:55:32.293 INFO:tasks.workunit.client.1.smithi176.stdout:6/204: mkdir d1/d5/d47 0 2024-07-16T09:55:32.294 INFO:tasks.workunit.client.0.smithi156.stdout:1/158: dread dc/f10 [0,4194304] 0 2024-07-16T09:55:32.294 INFO:tasks.workunit.client.0.smithi156.stdout:8/125: chown d0/dd/d1f/l28 9 1 2024-07-16T09:55:32.295 INFO:tasks.workunit.client.0.smithi156.stdout:1/159: chown l1 13207688 1 2024-07-16T09:55:32.295 INFO:tasks.workunit.client.1.smithi176.stdout:3/127: write d3/d17/d1d/f20 [115941,37070] 0 2024-07-16T09:55:32.296 INFO:tasks.workunit.client.1.smithi176.stdout:0/256: dwrite d5/d1c/f1f [0,4194304] 0 2024-07-16T09:55:32.297 INFO:tasks.workunit.client.1.smithi176.stdout:6/205: write d1/d5/d7/d2f/f44 [489947,29123] 0 2024-07-16T09:55:32.298 INFO:tasks.workunit.client.0.smithi156.stdout:2/269: getdents d0/d5/d29/d2d 0 2024-07-16T09:55:32.303 INFO:tasks.workunit.client.0.smithi156.stdout:5/148: unlink d1b/d1e/f2c 0 2024-07-16T09:55:32.305 INFO:tasks.workunit.client.0.smithi156.stdout:9/172: symlink d2/de/l35 0 2024-07-16T09:55:32.306 INFO:tasks.workunit.client.0.smithi156.stdout:2/270: dwrite d0/d5/d29/d2d/f40 [0,4194304] 0 2024-07-16T09:55:32.310 INFO:tasks.workunit.client.0.smithi156.stdout:1/160: creat dc/d20/f36 x:0 0 0 2024-07-16T09:55:32.311 INFO:tasks.workunit.client.1.smithi176.stdout:6/206: dwrite d1/f45 [0,4194304] 0 2024-07-16T09:55:32.311 INFO:tasks.workunit.client.1.smithi176.stdout:0/257: dwrite d5/d39/f3a [0,4194304] 0 2024-07-16T09:55:32.312 INFO:tasks.workunit.client.0.smithi156.stdout:5/149: symlink d1b/d28/d2d/l30 0 2024-07-16T09:55:32.315 INFO:tasks.workunit.client.1.smithi176.stdout:6/207: fsync d1/f15 0 2024-07-16T09:55:32.315 INFO:tasks.workunit.client.1.smithi176.stdout:0/258: write d5/d1c/d24/d26/f4f [407427,88159] 0 2024-07-16T09:55:32.315 INFO:tasks.workunit.client.1.smithi176.stdout:6/208: chown d1/f15 72 1 2024-07-16T09:55:32.315 INFO:tasks.workunit.client.0.smithi156.stdout:8/126: mkdir d0/d11/d2e 0 2024-07-16T09:55:32.316 INFO:tasks.workunit.client.1.smithi176.stdout:0/259: fsync d5/d1c/d24/d35/f3c 0 2024-07-16T09:55:32.320 INFO:tasks.workunit.client.0.smithi156.stdout:2/271: creat d0/d13/d17/d1c/d51/f5c x:0 0 0 2024-07-16T09:55:32.327 INFO:tasks.workunit.client.0.smithi156.stdout:1/161: rename dc/d14/f32 to dc/f37 0 2024-07-16T09:55:32.327 INFO:tasks.workunit.client.0.smithi156.stdout:1/162: write dc/d20/f36 [484964,15903] 0 2024-07-16T09:55:32.328 INFO:tasks.workunit.client.0.smithi156.stdout:5/150: creat d1b/f31 x:0 0 0 2024-07-16T09:55:32.329 INFO:tasks.workunit.client.0.smithi156.stdout:5/151: write fc [1469551,94531] 0 2024-07-16T09:55:32.329 INFO:tasks.workunit.client.0.smithi156.stdout:8/127: symlink d0/d11/d13/d1c/d24/l2f 0 2024-07-16T09:55:32.330 INFO:tasks.workunit.client.0.smithi156.stdout:8/128: write d0/d11/d20/f25 [4824981,74757] 0 2024-07-16T09:55:32.331 INFO:tasks.workunit.client.0.smithi156.stdout:2/272: creat d0/d13/f5d x:0 0 0 2024-07-16T09:55:32.332 INFO:tasks.workunit.client.0.smithi156.stdout:2/273: write d0/d5/f1e [3500901,128037] 0 2024-07-16T09:55:32.332 INFO:tasks.workunit.client.0.smithi156.stdout:5/152: symlink d1b/d1e/l32 0 2024-07-16T09:55:32.333 INFO:tasks.workunit.client.0.smithi156.stdout:2/274: chown d0/l3e 1755166 1 2024-07-16T09:55:32.334 INFO:tasks.workunit.client.0.smithi156.stdout:8/129: creat d0/dd/f30 x:0 0 0 2024-07-16T09:55:32.336 INFO:tasks.workunit.client.0.smithi156.stdout:5/153: symlink d1b/d1e/l33 0 2024-07-16T09:55:32.336 INFO:tasks.workunit.client.0.smithi156.stdout:8/130: write d0/dd/f1a [50692,91561] 0 2024-07-16T09:55:32.339 INFO:tasks.workunit.client.0.smithi156.stdout:8/131: rmdir d0/d11/d13/d1c/d24 39 2024-07-16T09:55:32.339 INFO:tasks.workunit.client.0.smithi156.stdout:2/275: dwrite d0/d13/d17/d1c/f48 [0,4194304] 0 2024-07-16T09:55:32.340 INFO:tasks.workunit.client.0.smithi156.stdout:2/276: readlink d0/l10 0 2024-07-16T09:55:32.340 INFO:tasks.workunit.client.0.smithi156.stdout:2/277: chown d0/d5 9298954 1 2024-07-16T09:55:32.341 INFO:tasks.workunit.client.0.smithi156.stdout:8/132: symlink d0/d11/d20/l31 0 2024-07-16T09:55:32.343 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:55:31 smithi176 ceph-mon[36626]: pgmap v247: 65 pgs: 65 active+clean; 523 MiB data, 3.3 GiB used, 533 GiB / 536 GiB avail; 3.3 MiB/s rd, 46 MiB/s wr, 411 op/s 2024-07-16T09:55:32.345 INFO:tasks.workunit.client.0.smithi156.stdout:8/133: creat d0/d11/d13/d1c/f32 x:0 0 0 2024-07-16T09:55:32.351 INFO:tasks.workunit.client.0.smithi156.stdout:8/134: dwrite d0/d11/d20/f21 [0,4194304] 0 2024-07-16T09:55:32.353 INFO:tasks.workunit.client.0.smithi156.stdout:8/135: stat d0/f6 0 2024-07-16T09:55:32.355 INFO:tasks.workunit.client.0.smithi156.stdout:8/136: rename d0/dd/f2d to d0/dd/f33 0 2024-07-16T09:55:32.356 INFO:tasks.workunit.client.0.smithi156.stdout:8/137: read - d0/d11/f15 zero size 2024-07-16T09:55:32.358 INFO:tasks.workunit.client.0.smithi156.stdout:8/138: mknod d0/d11/d13/c34 0 2024-07-16T09:55:32.360 INFO:tasks.workunit.client.0.smithi156.stdout:8/139: mkdir d0/d11/d2e/d35 0 2024-07-16T09:55:32.364 INFO:tasks.workunit.client.0.smithi156.stdout:8/140: creat d0/d11/d13/d1c/d24/f36 x:0 0 0 2024-07-16T09:55:32.366 INFO:tasks.workunit.client.0.smithi156.stdout:8/141: symlink d0/d11/d13/d1c/l37 0 2024-07-16T09:55:32.366 INFO:tasks.workunit.client.0.smithi156.stdout:8/142: readlink d0/l27 0 2024-07-16T09:55:32.391 INFO:tasks.workunit.client.0.smithi156.stdout:4/179: rmdir d4 39 2024-07-16T09:55:32.392 INFO:tasks.workunit.client.0.smithi156.stdout:4/180: mkdir d4/d41 0 2024-07-16T09:55:32.393 INFO:tasks.workunit.client.0.smithi156.stdout:6/185: rmdir d3/d26 39 2024-07-16T09:55:32.396 INFO:tasks.workunit.client.0.smithi156.stdout:4/181: getdents d4/d23/d30 0 2024-07-16T09:55:32.398 INFO:tasks.workunit.client.0.smithi156.stdout:4/182: write d4/db/dc/f2e [1330166,11355] 0 2024-07-16T09:55:32.399 INFO:tasks.workunit.client.0.smithi156.stdout:9/173: rename d2/d5/d16/d22/d26 to d2/d5/d16/d22/d36 0 2024-07-16T09:55:32.403 INFO:tasks.workunit.client.0.smithi156.stdout:9/174: creat d2/d5/d16/f37 x:0 0 0 2024-07-16T09:55:32.403 INFO:tasks.workunit.client.0.smithi156.stdout:6/186: getdents d3/d26/d1f/d19/d2e 0 2024-07-16T09:55:32.403 INFO:tasks.workunit.client.0.smithi156.stdout:9/175: readlink d2/d5/ld 0 2024-07-16T09:55:32.403 INFO:tasks.workunit.client.1.smithi176.stdout:3/128: write d3/da/f19 [1131037,84223] 0 2024-07-16T09:55:32.404 INFO:tasks.workunit.client.0.smithi156.stdout:7/136: truncate fb 4166184 0 2024-07-16T09:55:32.404 INFO:tasks.workunit.client.1.smithi176.stdout:6/209: rmdir d1/d5/d42 0 2024-07-16T09:55:32.404 INFO:tasks.workunit.client.1.smithi176.stdout:3/129: chown d3/c10 0 1 2024-07-16T09:55:32.405 INFO:tasks.workunit.client.0.smithi156.stdout:9/176: truncate d2/d5/d16/f20 993083 0 2024-07-16T09:55:32.405 INFO:tasks.workunit.client.0.smithi156.stdout:6/187: rmdir d3/d26/d1f/d19/d29/d3b 39 2024-07-16T09:55:32.406 INFO:tasks.workunit.client.0.smithi156.stdout:9/177: write d2/de/f2f [320952,86194] 0 2024-07-16T09:55:32.407 INFO:tasks.workunit.client.1.smithi176.stdout:6/210: mknod d1/d5/c48 0 2024-07-16T09:55:32.409 INFO:tasks.workunit.client.1.smithi176.stdout:6/211: chown d1/d5/d7/d2f/l3c 183 1 2024-07-16T09:55:32.410 INFO:tasks.workunit.client.0.smithi156.stdout:6/188: creat d3/d26/d1f/d19/d29/f3f x:0 0 0 2024-07-16T09:55:32.410 INFO:tasks.workunit.client.0.smithi156.stdout:9/178: truncate d2/d5/d16/d22/d36/f31 1010268 0 2024-07-16T09:55:32.411 INFO:tasks.workunit.client.1.smithi176.stdout:0/260: dread d5/f10 [0,4194304] 0 2024-07-16T09:55:32.411 INFO:tasks.workunit.client.0.smithi156.stdout:2/278: fsync d0/d13/d17/d1c/d51/f5c 0 2024-07-16T09:55:32.411 INFO:tasks.workunit.client.1.smithi176.stdout:6/212: mkdir d1/d5/d7/d49 0 2024-07-16T09:55:32.412 INFO:tasks.workunit.client.0.smithi156.stdout:6/189: chown d3/d26/f22 23215539 1 2024-07-16T09:55:32.412 INFO:tasks.workunit.client.0.smithi156.stdout:7/137: dwrite dc/f14 [0,4194304] 0 2024-07-16T09:55:32.413 INFO:tasks.workunit.client.0.smithi156.stdout:6/190: chown d3 4195813 1 2024-07-16T09:55:32.413 INFO:tasks.workunit.client.0.smithi156.stdout:9/179: truncate d2/f4 1486683 0 2024-07-16T09:55:32.417 INFO:tasks.workunit.client.0.smithi156.stdout:5/154: rmdir d1b/d1e 39 2024-07-16T09:55:32.418 INFO:tasks.workunit.client.0.smithi156.stdout:7/138: symlink dc/d13/d24/l34 0 2024-07-16T09:55:32.419 INFO:tasks.workunit.client.1.smithi176.stdout:6/213: symlink d1/d5/d7/d1e/l4a 0 2024-07-16T09:55:32.420 INFO:tasks.workunit.client.0.smithi156.stdout:5/155: write d1b/d1f/f20 [2717659,79244] 0 2024-07-16T09:55:32.420 INFO:tasks.workunit.client.0.smithi156.stdout:9/180: rename d2/d5/d16/l34 to d2/d5/d16/d22/d36/l38 0 2024-07-16T09:55:32.422 INFO:tasks.workunit.client.0.smithi156.stdout:2/279: dwrite d0/d13/d17/f30 [0,4194304] 0 2024-07-16T09:55:32.424 INFO:tasks.workunit.client.1.smithi176.stdout:3/130: dread d3/da/fe [4194304,4194304] 0 2024-07-16T09:55:32.425 INFO:tasks.workunit.client.0.smithi156.stdout:7/139: creat dc/d13/f35 x:0 0 0 2024-07-16T09:55:32.426 INFO:tasks.workunit.client.1.smithi176.stdout:4/233: sync 2024-07-16T09:55:32.426 INFO:tasks.workunit.client.1.smithi176.stdout:1/139: sync 2024-07-16T09:55:32.426 INFO:tasks.workunit.client.1.smithi176.stdout:5/333: sync 2024-07-16T09:55:32.426 INFO:tasks.workunit.client.1.smithi176.stdout:9/175: sync 2024-07-16T09:55:32.426 INFO:tasks.workunit.client.1.smithi176.stdout:7/252: sync 2024-07-16T09:55:32.426 INFO:tasks.workunit.client.1.smithi176.stdout:8/350: sync 2024-07-16T09:55:32.427 INFO:tasks.workunit.client.1.smithi176.stdout:3/131: creat d3/d17/f23 x:0 0 0 2024-07-16T09:55:32.429 INFO:tasks.workunit.client.0.smithi156.stdout:2/280: creat d0/d13/d17/d1c/f5e x:0 0 0 2024-07-16T09:55:32.430 INFO:tasks.workunit.client.1.smithi176.stdout:4/234: mkdir da/d14/d40 0 2024-07-16T09:55:32.430 INFO:tasks.workunit.client.1.smithi176.stdout:3/132: fsync d3/d17/d1d/f20 0 2024-07-16T09:55:32.431 INFO:tasks.workunit.client.0.smithi156.stdout:7/140: mkdir dc/d13/d36 0 2024-07-16T09:55:32.432 INFO:tasks.workunit.client.0.smithi156.stdout:9/181: mkdir d2/d39 0 2024-07-16T09:55:32.432 INFO:tasks.workunit.client.0.smithi156.stdout:6/191: getdents d3/d26/d1f/d19/d31/d35 0 2024-07-16T09:55:32.434 INFO:tasks.workunit.client.1.smithi176.stdout:1/140: write d4/d8/f28 [4613344,74589] 0 2024-07-16T09:55:32.434 INFO:tasks.workunit.client.0.smithi156.stdout:9/182: chown d2/d5/l25 105 1 2024-07-16T09:55:32.434 INFO:tasks.workunit.client.0.smithi156.stdout:5/156: creat d1b/f34 x:0 0 0 2024-07-16T09:55:32.437 INFO:tasks.workunit.client.1.smithi176.stdout:8/351: creat d8/d23/d2c/d34/d62/f7d x:0 0 0 2024-07-16T09:55:32.437 INFO:tasks.workunit.client.0.smithi156.stdout:7/141: mknod dc/d13/c37 0 2024-07-16T09:55:32.438 INFO:tasks.workunit.client.0.smithi156.stdout:2/281: mkdir d0/d5/d29/d2d/d5b/d5f 0 2024-07-16T09:55:32.439 INFO:tasks.workunit.client.1.smithi176.stdout:5/334: write d4/d14/d19/f21 [5071394,54526] 0 2024-07-16T09:55:32.439 INFO:tasks.workunit.client.1.smithi176.stdout:4/235: mkdir da/d41 0 2024-07-16T09:55:32.440 INFO:tasks.workunit.client.1.smithi176.stdout:3/133: fdatasync d3/d17/d1d/d1e/f21 0 2024-07-16T09:55:32.440 INFO:tasks.workunit.client.0.smithi156.stdout:9/183: readlink d2/d5/d16/d22/d36/l38 0 2024-07-16T09:55:32.441 INFO:tasks.workunit.client.0.smithi156.stdout:9/184: truncate d2/de/f2f 1222917 0 2024-07-16T09:55:32.441 INFO:tasks.workunit.client.0.smithi156.stdout:5/157: creat d1b/d28/f35 x:0 0 0 2024-07-16T09:55:32.442 INFO:tasks.workunit.client.1.smithi176.stdout:8/352: write d8/d20/d2a/d31/f3e [5210770,108434] 0 2024-07-16T09:55:32.444 INFO:tasks.workunit.client.1.smithi176.stdout:1/141: read d4/d8/f1e [3016472,123193] 0 2024-07-16T09:55:32.444 INFO:tasks.workunit.client.0.smithi156.stdout:6/192: rename d3/d26/f22 to d3/d26/d1f/d19/d29/d3b/f40 0 2024-07-16T09:55:32.447 INFO:tasks.workunit.client.1.smithi176.stdout:7/253: dread d4/d47/f46 [0,4194304] 0 2024-07-16T09:55:32.447 INFO:tasks.workunit.client.1.smithi176.stdout:9/176: dwrite d5/f1c [0,4194304] 0 2024-07-16T09:55:32.447 INFO:tasks.workunit.client.0.smithi156.stdout:9/185: mkdir d2/d5/d16/d22/d36/d33/d3a 0 2024-07-16T09:55:32.447 INFO:tasks.workunit.client.0.smithi156.stdout:7/142: mknod dc/d13/d36/c38 0 2024-07-16T09:55:32.448 INFO:tasks.workunit.client.0.smithi156.stdout:9/186: truncate d2/d5/d16/d22/d36/f31 1085173 0 2024-07-16T09:55:32.448 INFO:tasks.workunit.client.1.smithi176.stdout:6/214: dread d1/d5/f2a [0,4194304] 0 2024-07-16T09:55:32.449 INFO:tasks.workunit.client.0.smithi156.stdout:2/282: symlink d0/l60 0 2024-07-16T09:55:32.449 INFO:tasks.workunit.client.1.smithi176.stdout:8/353: symlink d8/d23/d2c/l7e 0 2024-07-16T09:55:32.450 INFO:tasks.workunit.client.1.smithi176.stdout:3/134: truncate d3/d17/d1d/d1e/f1f 1609616 0 2024-07-16T09:55:32.451 INFO:tasks.workunit.client.1.smithi176.stdout:4/236: unlink da/d3d/c37 0 2024-07-16T09:55:32.451 INFO:tasks.workunit.client.1.smithi176.stdout:9/177: write d5/fd [2990656,44648] 0 2024-07-16T09:55:32.452 INFO:tasks.workunit.client.0.smithi156.stdout:7/143: mknod dc/d13/d20/c39 0 2024-07-16T09:55:32.453 INFO:tasks.workunit.client.1.smithi176.stdout:1/142: mkdir d4/d8/d2c 0 2024-07-16T09:55:32.454 INFO:tasks.workunit.client.1.smithi176.stdout:7/254: unlink d4/dd/d49/f4c 0 2024-07-16T09:55:32.455 INFO:tasks.workunit.client.0.smithi156.stdout:2/283: mknod d0/c61 0 2024-07-16T09:55:32.456 INFO:tasks.workunit.client.0.smithi156.stdout:2/284: readlink d0/d13/d17/d1c/l1d 0 2024-07-16T09:55:32.457 INFO:tasks.workunit.client.1.smithi176.stdout:3/135: dread d3/da/fe [4194304,4194304] 0 2024-07-16T09:55:32.457 INFO:tasks.workunit.client.0.smithi156.stdout:5/158: rename d1b/d28/c2a to d1b/c36 0 2024-07-16T09:55:32.458 INFO:tasks.workunit.client.1.smithi176.stdout:9/178: creat d5/d1d/d20/f33 x:0 0 0 2024-07-16T09:55:32.459 INFO:tasks.workunit.client.1.smithi176.stdout:6/215: symlink d1/d5/d7/l4b 0 2024-07-16T09:55:32.459 INFO:tasks.workunit.client.0.smithi156.stdout:9/187: dwrite d2/d5/d16/d22/d36/d33/d23/f2e [0,4194304] 0 2024-07-16T09:55:32.460 INFO:tasks.workunit.client.1.smithi176.stdout:3/136: write d3/d17/f23 [593211,10390] 0 2024-07-16T09:55:32.462 INFO:tasks.workunit.client.0.smithi156.stdout:9/188: truncate d2/d5/d16/d22/d36/d33/f2c 68620 0 2024-07-16T09:55:32.462 INFO:tasks.workunit.client.0.smithi156.stdout:3/137: sync 2024-07-16T09:55:32.463 INFO:tasks.workunit.client.1.smithi176.stdout:7/255: rename d4/d5/d10/l3e to d4/d5/d10/d17/d1d/l5c 0 2024-07-16T09:55:32.463 INFO:tasks.workunit.client.1.smithi176.stdout:1/143: mkdir d4/d8/d2c/d2d 0 2024-07-16T09:55:32.464 INFO:tasks.workunit.client.1.smithi176.stdout:9/179: creat d5/de/f34 x:0 0 0 2024-07-16T09:55:32.465 INFO:tasks.workunit.client.0.smithi156.stdout:2/285: mkdir d0/d5/d29/d2d/d5b/d62 0 2024-07-16T09:55:32.465 INFO:tasks.workunit.client.0.smithi156.stdout:6/193: link d3/d26/d1f/d19/d31/d35/l36 d3/d26/l41 0 2024-07-16T09:55:32.466 INFO:tasks.workunit.client.1.smithi176.stdout:4/237: creat da/f42 x:0 0 0 2024-07-16T09:55:32.467 INFO:tasks.workunit.client.1.smithi176.stdout:5/335: dread d4/d10/f33 [0,4194304] 0 2024-07-16T09:55:32.469 INFO:tasks.workunit.client.1.smithi176.stdout:3/137: rename d3/da/cb to d3/c24 0 2024-07-16T09:55:32.469 INFO:tasks.workunit.client.1.smithi176.stdout:9/180: write d5/d1d/d20/f25 [814384,130187] 0 2024-07-16T09:55:32.470 INFO:tasks.workunit.client.1.smithi176.stdout:1/144: creat d4/d8/d12/f2e x:0 0 0 2024-07-16T09:55:32.470 INFO:tasks.workunit.client.0.smithi156.stdout:3/138: dread d8/da/fc [0,4194304] 0 2024-07-16T09:55:32.470 INFO:tasks.workunit.client.0.smithi156.stdout:7/144: rename dc/d13/c11 to dc/d13/d15/c3a 0 2024-07-16T09:55:32.470 INFO:tasks.workunit.client.0.smithi156.stdout:6/194: dread d3/d26/f14 [0,4194304] 0 2024-07-16T09:55:32.471 INFO:tasks.workunit.client.0.smithi156.stdout:9/189: creat d2/d5/d16/d22/d36/d33/d3a/f3b x:0 0 0 2024-07-16T09:55:32.471 INFO:tasks.workunit.client.1.smithi176.stdout:8/354: dwrite d8/d23/d2c/d37/f57 [0,4194304] 0 2024-07-16T09:55:32.472 INFO:tasks.workunit.client.0.smithi156.stdout:7/145: chown dc/d13/d20/c39 9966 1 2024-07-16T09:55:32.472 INFO:tasks.workunit.client.0.smithi156.stdout:7/146: readlink dc/d13/l17 0 2024-07-16T09:55:32.473 INFO:tasks.workunit.client.1.smithi176.stdout:9/181: fdatasync d5/de/f34 0 2024-07-16T09:55:32.474 INFO:tasks.workunit.client.0.smithi156.stdout:2/286: rename d0/c11 to d0/d5/d29/d2d/d5b/d62/c63 0 2024-07-16T09:55:32.474 INFO:tasks.workunit.client.1.smithi176.stdout:9/182: chown d5/d1d/d20 6255129 1 2024-07-16T09:55:32.474 INFO:tasks.workunit.client.1.smithi176.stdout:8/355: chown d8/c10 290414999 1 2024-07-16T09:55:32.475 INFO:tasks.workunit.client.1.smithi176.stdout:6/216: creat d1/d5/d7/f4c x:0 0 0 2024-07-16T09:55:32.476 INFO:tasks.workunit.client.0.smithi156.stdout:3/139: creat d8/d14/f30 x:0 0 0 2024-07-16T09:55:32.480 INFO:tasks.workunit.client.1.smithi176.stdout:5/336: chown d4/d14/f25 777 1 2024-07-16T09:55:32.481 INFO:tasks.workunit.client.0.smithi156.stdout:3/140: write d8/da/d28/f2a [746727,113070] 0 2024-07-16T09:55:32.481 INFO:tasks.workunit.client.0.smithi156.stdout:2/287: mknod d0/d13/d17/c64 0 2024-07-16T09:55:32.483 INFO:tasks.workunit.client.1.smithi176.stdout:8/356: symlink d8/d14/d1e/d33/l7f 0 2024-07-16T09:55:32.483 INFO:tasks.workunit.client.1.smithi176.stdout:4/238: dread da/d3d/d17/f31 [0,4194304] 0 2024-07-16T09:55:32.483 INFO:tasks.workunit.client.0.smithi156.stdout:7/147: dread dc/f14 [0,4194304] 0 2024-07-16T09:55:32.484 INFO:tasks.workunit.client.0.smithi156.stdout:9/190: dwrite d2/d5/ff [0,4194304] 0 2024-07-16T09:55:32.485 INFO:tasks.workunit.client.0.smithi156.stdout:6/195: link d3/d26/d1f/l33 d3/d26/d1f/d19/d2e/l42 0 2024-07-16T09:55:32.486 INFO:tasks.workunit.client.1.smithi176.stdout:6/217: creat d1/dc/d40/f4d x:0 0 0 2024-07-16T09:55:32.486 INFO:tasks.workunit.client.1.smithi176.stdout:1/145: dwrite d4/d8/f29 [0,4194304] 0 2024-07-16T09:55:32.487 INFO:tasks.workunit.client.1.smithi176.stdout:4/239: chown da/f19 13874 1 2024-07-16T09:55:32.487 INFO:tasks.workunit.client.0.smithi156.stdout:3/141: dread - d8/da/d13/f20 zero size 2024-07-16T09:55:32.487 INFO:tasks.workunit.client.0.smithi156.stdout:9/191: write d2/d5/f8 [3176840,29701] 0 2024-07-16T09:55:32.487 INFO:tasks.workunit.client.0.smithi156.stdout:7/148: creat dc/d13/d20/f3b x:0 0 0 2024-07-16T09:55:32.488 INFO:tasks.workunit.client.0.smithi156.stdout:3/142: chown d8/d14/l1b 220477 1 2024-07-16T09:55:32.488 INFO:tasks.workunit.client.0.smithi156.stdout:7/149: chown dc/d13/c37 152 1 2024-07-16T09:55:32.489 INFO:tasks.workunit.client.1.smithi176.stdout:1/146: chown d4/d8/d12/f2e 2864080 1 2024-07-16T09:55:32.489 INFO:tasks.workunit.client.1.smithi176.stdout:8/357: rmdir d8/d23 39 2024-07-16T09:55:32.489 INFO:tasks.workunit.client.1.smithi176.stdout:1/147: stat d4/d8 0 2024-07-16T09:55:32.491 INFO:tasks.workunit.client.0.smithi156.stdout:6/196: chown d3/d26/l41 827602 1 2024-07-16T09:55:32.492 INFO:tasks.workunit.client.0.smithi156.stdout:9/192: dread d2/de/f19 [0,4194304] 0 2024-07-16T09:55:32.492 INFO:tasks.workunit.client.1.smithi176.stdout:1/148: dread - d4/d8/f26 zero size 2024-07-16T09:55:32.493 INFO:tasks.workunit.client.0.smithi156.stdout:7/150: fsync dc/d13/d20/f2b 0 2024-07-16T09:55:32.494 INFO:tasks.workunit.client.0.smithi156.stdout:3/143: dread f3 [0,4194304] 0 2024-07-16T09:55:32.496 INFO:tasks.workunit.client.0.smithi156.stdout:7/151: chown dc/d13/l1f 1 1 2024-07-16T09:55:32.499 INFO:tasks.workunit.client.0.smithi156.stdout:3/144: write d8/d14/f1c [2649008,118647] 0 2024-07-16T09:55:32.501 INFO:tasks.workunit.client.1.smithi176.stdout:5/337: creat d4/d37/d72/d38/f7e x:0 0 0 2024-07-16T09:55:32.502 INFO:tasks.workunit.client.0.smithi156.stdout:7/152: write dc/d13/f35 [702502,128745] 0 2024-07-16T09:55:32.502 INFO:tasks.workunit.client.0.smithi156.stdout:3/145: read d8/f1a [154100,123386] 0 2024-07-16T09:55:32.504 INFO:tasks.workunit.client.1.smithi176.stdout:7/256: truncate d4/d47/f44 3617099 0 2024-07-16T09:55:32.506 INFO:tasks.workunit.client.0.smithi156.stdout:9/193: dwrite d2/f4 [0,4194304] 0 2024-07-16T09:55:32.506 INFO:tasks.workunit.client.1.smithi176.stdout:9/183: getdents d5/de/d21 0 2024-07-16T09:55:32.509 INFO:tasks.workunit.client.0.smithi156.stdout:7/153: write dc/d13/f35 [1302788,17586] 0 2024-07-16T09:55:32.510 INFO:tasks.workunit.client.0.smithi156.stdout:6/197: dwrite d3/d26/d1f/f10 [0,4194304] 0 2024-07-16T09:55:32.510 INFO:tasks.workunit.client.1.smithi176.stdout:6/218: dread d1/d5/f27 [0,4194304] 0 2024-07-16T09:55:32.510 INFO:tasks.workunit.client.0.smithi156.stdout:9/194: mkdir d2/d39/d3c 0 2024-07-16T09:55:32.511 INFO:tasks.workunit.client.1.smithi176.stdout:7/257: write d4/d47/f46 [2417017,64882] 0 2024-07-16T09:55:32.511 INFO:tasks.workunit.client.1.smithi176.stdout:1/149: dwrite d4/ff [4194304,4194304] 0 2024-07-16T09:55:32.512 INFO:tasks.workunit.client.1.smithi176.stdout:3/138: chown d3/da/cc 63579 1 2024-07-16T09:55:32.512 INFO:tasks.workunit.client.0.smithi156.stdout:3/146: link d8/da/c19 d8/da/d1d/c31 0 2024-07-16T09:55:32.512 INFO:tasks.workunit.client.1.smithi176.stdout:8/358: dwrite d8/d20/d2a/d31/f69 [0,4194304] 0 2024-07-16T09:55:32.513 INFO:tasks.workunit.client.1.smithi176.stdout:9/184: unlink d5/de/d21/l22 0 2024-07-16T09:55:32.515 INFO:tasks.workunit.client.0.smithi156.stdout:6/198: mknod d3/c43 0 2024-07-16T09:55:32.517 INFO:tasks.workunit.client.0.smithi156.stdout:6/199: write d3/d26/d1f/f37 [422492,114473] 0 2024-07-16T09:55:32.520 INFO:tasks.workunit.client.0.smithi156.stdout:9/195: getdents d2/de 0 2024-07-16T09:55:32.520 INFO:tasks.workunit.client.1.smithi176.stdout:9/185: write d5/de/f17 [626320,124180] 0 2024-07-16T09:55:32.522 INFO:tasks.workunit.client.0.smithi156.stdout:7/154: dwrite dc/d13/d20/f2f [0,4194304] 0 2024-07-16T09:55:32.522 INFO:tasks.workunit.client.1.smithi176.stdout:7/258: readlink d4/d5/d10/l18 0 2024-07-16T09:55:32.524 INFO:tasks.workunit.client.0.smithi156.stdout:6/200: fsync d3/d26/d1f/d19/d29/d3b/f3c 0 2024-07-16T09:55:32.526 INFO:tasks.workunit.client.1.smithi176.stdout:7/259: stat d4/d5/lc 0 2024-07-16T09:55:32.526 INFO:tasks.workunit.client.1.smithi176.stdout:1/150: write d4/db/f1b [3596603,89043] 0 2024-07-16T09:55:32.527 INFO:tasks.workunit.client.0.smithi156.stdout:3/147: dwrite d8/fb [0,4194304] 0 2024-07-16T09:55:32.528 INFO:tasks.workunit.client.1.smithi176.stdout:6/219: rename d1/dc/l24 to d1/d5/d7/d49/l4e 0 2024-07-16T09:55:32.531 INFO:tasks.workunit.client.0.smithi156.stdout:7/155: creat dc/d13/d20/f3c x:0 0 0 2024-07-16T09:55:32.533 INFO:tasks.workunit.client.0.smithi156.stdout:2/288: write d0/fe [1151348,111650] 0 2024-07-16T09:55:32.535 INFO:tasks.workunit.client.1.smithi176.stdout:1/151: write d4/f10 [1149995,34954] 0 2024-07-16T09:55:32.535 INFO:tasks.workunit.client.1.smithi176.stdout:7/260: fdatasync d4/f27 0 2024-07-16T09:55:32.535 INFO:tasks.workunit.client.1.smithi176.stdout:3/139: dread d3/da/f19 [0,4194304] 0 2024-07-16T09:55:32.535 INFO:tasks.workunit.client.1.smithi176.stdout:8/359: symlink d8/d14/d1e/d7c/l80 0 2024-07-16T09:55:32.535 INFO:tasks.workunit.client.1.smithi176.stdout:7/261: readlink d4/dd/le 0 2024-07-16T09:55:32.537 INFO:tasks.workunit.client.0.smithi156.stdout:7/156: symlink dc/d13/d20/l3d 0 2024-07-16T09:55:32.537 INFO:tasks.workunit.client.0.smithi156.stdout:2/289: rmdir d0/d5/d29/d2d/d5b/d62 39 2024-07-16T09:55:32.538 INFO:tasks.workunit.client.0.smithi156.stdout:3/148: creat d8/da/f32 x:0 0 0 2024-07-16T09:55:32.540 INFO:tasks.workunit.client.1.smithi176.stdout:9/186: dwrite d5/de/f15 [0,4194304] 0 2024-07-16T09:55:32.542 INFO:tasks.workunit.client.0.smithi156.stdout:7/157: mkdir dc/d3e 0 2024-07-16T09:55:32.544 INFO:tasks.workunit.client.1.smithi176.stdout:9/187: readlink d5/d1d/l2e 0 2024-07-16T09:55:32.544 INFO:tasks.workunit.client.0.smithi156.stdout:3/149: mkdir d8/da/d1d/d2c/d33 0 2024-07-16T09:55:32.546 INFO:tasks.workunit.client.0.smithi156.stdout:2/290: getdents d0/d13/d17 0 2024-07-16T09:55:32.548 INFO:tasks.workunit.client.1.smithi176.stdout:3/140: write d3/d17/d1d/d1e/f22 [1045036,101612] 0 2024-07-16T09:55:32.548 INFO:tasks.workunit.client.0.smithi156.stdout:2/291: chown d0/d5/d29/d2d/c53 452053033 1 2024-07-16T09:55:32.550 INFO:tasks.workunit.client.1.smithi176.stdout:7/262: getdents d4/d5/d21 0 2024-07-16T09:55:32.550 INFO:tasks.workunit.client.0.smithi156.stdout:2/292: write d0/d13/d17/d1c/f48 [4343141,92939] 0 2024-07-16T09:55:32.551 INFO:tasks.workunit.client.0.smithi156.stdout:7/158: getdents dc/d13/d36 0 2024-07-16T09:55:32.553 INFO:tasks.workunit.client.0.smithi156.stdout:7/159: chown dc/d13/d20/f3b 7265 1 2024-07-16T09:55:32.555 INFO:tasks.workunit.client.0.smithi156.stdout:3/150: dwrite d8/f1a [0,4194304] 0 2024-07-16T09:55:32.556 INFO:tasks.workunit.client.0.smithi156.stdout:7/160: mkdir dc/d3e/d3f 0 2024-07-16T09:55:32.556 INFO:tasks.workunit.client.1.smithi176.stdout:5/338: dwrite f2 [0,4194304] 0 2024-07-16T09:55:32.557 INFO:tasks.workunit.client.1.smithi176.stdout:1/152: dwrite f1 [0,4194304] 0 2024-07-16T09:55:32.557 INFO:tasks.workunit.client.0.smithi156.stdout:7/161: chown dc/d13/c30 435712 1 2024-07-16T09:55:32.558 INFO:tasks.workunit.client.0.smithi156.stdout:3/151: mkdir d8/d34 0 2024-07-16T09:55:32.559 INFO:tasks.workunit.client.0.smithi156.stdout:3/152: chown d8/da 15715 1 2024-07-16T09:55:32.560 INFO:tasks.workunit.client.1.smithi176.stdout:3/141: link d3/da/fe d3/f25 0 2024-07-16T09:55:32.560 INFO:tasks.workunit.client.1.smithi176.stdout:6/220: dread d1/d5/d7/fe [0,4194304] 0 2024-07-16T09:55:32.562 INFO:tasks.workunit.client.1.smithi176.stdout:8/360: dwrite d8/d23/f21 [0,4194304] 0 2024-07-16T09:55:32.563 INFO:tasks.workunit.client.0.smithi156.stdout:7/162: creat dc/f40 x:0 0 0 2024-07-16T09:55:32.564 INFO:tasks.workunit.client.0.smithi156.stdout:7/163: chown dc/d13/d20/c39 173 1 2024-07-16T09:55:32.565 INFO:tasks.workunit.client.1.smithi176.stdout:5/339: symlink d4/d37/d72/d38/l7f 0 2024-07-16T09:55:32.567 INFO:tasks.workunit.client.1.smithi176.stdout:5/340: readlink d4/d37/d72/l47 0 2024-07-16T09:55:32.570 INFO:tasks.workunit.client.1.smithi176.stdout:1/153: link d4/d8/d12/f2e d4/d8/d2c/f2f 0 2024-07-16T09:55:32.573 INFO:tasks.workunit.client.1.smithi176.stdout:3/142: unlink d3/d17/d1d/d1e/f22 0 2024-07-16T09:55:32.574 INFO:tasks.workunit.client.0.smithi156.stdout:3/153: dwrite d8/da/f2e [0,4194304] 0 2024-07-16T09:55:32.575 INFO:tasks.workunit.client.1.smithi176.stdout:9/188: rmdir d5/de 39 2024-07-16T09:55:32.578 INFO:tasks.workunit.client.0.smithi156.stdout:7/164: dwrite dc/d13/f35 [0,4194304] 0 2024-07-16T09:55:32.578 INFO:tasks.workunit.client.0.smithi156.stdout:9/196: rename d2/d39 to d2/d5/d16/d22/d2b/d3d 0 2024-07-16T09:55:32.579 INFO:tasks.workunit.client.1.smithi176.stdout:5/341: dread d4/d14/f25 [0,4194304] 0 2024-07-16T09:55:32.581 INFO:tasks.workunit.client.1.smithi176.stdout:6/221: dread d1/dc/fd [0,4194304] 0 2024-07-16T09:55:32.582 INFO:tasks.workunit.client.1.smithi176.stdout:6/222: fdatasync d1/d5/f27 0 2024-07-16T09:55:32.582 INFO:tasks.workunit.client.0.smithi156.stdout:9/197: write d2/d5/d16/d22/d36/d33/d3a/f3b [257219,81100] 0 2024-07-16T09:55:32.583 INFO:tasks.workunit.client.1.smithi176.stdout:5/342: dread f2 [0,4194304] 0 2024-07-16T09:55:32.586 INFO:tasks.workunit.client.1.smithi176.stdout:1/154: mkdir d4/d8/d12/d21/d25/d30 0 2024-07-16T09:55:32.588 INFO:tasks.workunit.client.1.smithi176.stdout:9/189: rmdir d5 39 2024-07-16T09:55:32.593 INFO:tasks.workunit.client.0.smithi156.stdout:6/201: rename d3/f1d to d3/d26/d30/f44 0 2024-07-16T09:55:32.594 INFO:tasks.workunit.client.0.smithi156.stdout:2/293: rename d0/d5/d29 to d0/d5/d29/d2d/d5b/d62/d65 22 2024-07-16T09:55:32.594 INFO:tasks.workunit.client.0.smithi156.stdout:7/165: link dc/d13/l17 dc/d13/d36/l41 0 2024-07-16T09:55:32.596 INFO:tasks.workunit.client.0.smithi156.stdout:9/198: dwrite d2/f13 [4194304,4194304] 0 2024-07-16T09:55:32.596 INFO:tasks.workunit.client.0.smithi156.stdout:6/202: dread - d3/d26/d1f/d19/d29/f3f zero size 2024-07-16T09:55:32.597 INFO:tasks.workunit.client.0.smithi156.stdout:7/166: write dc/d13/f35 [182449,96893] 0 2024-07-16T09:55:32.600 INFO:tasks.workunit.client.0.smithi156.stdout:7/167: chown dc/d13/d24/l34 12475 1 2024-07-16T09:55:32.601 INFO:tasks.workunit.client.0.smithi156.stdout:6/203: mkdir d3/d26/d30/d45 0 2024-07-16T09:55:32.604 INFO:tasks.workunit.client.0.smithi156.stdout:2/294: dwrite d0/d13/d17/d1c/d51/f5c [0,4194304] 0 2024-07-16T09:55:32.604 INFO:tasks.workunit.client.0.smithi156.stdout:7/168: creat dc/f42 x:0 0 0 2024-07-16T09:55:32.607 INFO:tasks.workunit.client.0.smithi156.stdout:7/169: creat dc/d13/f43 x:0 0 0 2024-07-16T09:55:32.608 INFO:tasks.workunit.client.0.smithi156.stdout:2/295: unlink d0/d5/d29/d2d/c53 0 2024-07-16T09:55:32.609 INFO:tasks.workunit.client.0.smithi156.stdout:7/170: readlink dc/d13/d36/l41 0 2024-07-16T09:55:32.611 INFO:tasks.workunit.client.0.smithi156.stdout:2/296: truncate d0/d13/d17/d1c/f37 1072049 0 2024-07-16T09:55:32.615 INFO:tasks.workunit.client.0.smithi156.stdout:2/297: getdents d0/d13 0 2024-07-16T09:55:32.615 INFO:tasks.workunit.client.0.smithi156.stdout:2/298: stat d0/f8 0 2024-07-16T09:55:32.616 INFO:tasks.workunit.client.0.smithi156.stdout:9/199: rmdir d2/d5/d16 39 2024-07-16T09:55:32.617 INFO:tasks.workunit.client.0.smithi156.stdout:3/154: write d8/f10 [646685,14697] 0 2024-07-16T09:55:32.620 INFO:tasks.workunit.client.0.smithi156.stdout:3/155: readlink d8/da/d1d/l2f 0 2024-07-16T09:55:32.626 INFO:tasks.workunit.client.0.smithi156.stdout:3/156: truncate d8/da/fc 2614142 0 2024-07-16T09:55:32.627 INFO:tasks.workunit.client.0.smithi156.stdout:2/299: fsync d0/d13/d17/d1c/d51/f5c 0 2024-07-16T09:55:32.628 INFO:tasks.workunit.client.0.smithi156.stdout:3/157: write f3 [6627592,75712] 0 2024-07-16T09:55:32.629 INFO:tasks.workunit.client.0.smithi156.stdout:3/158: fsync d8/d14/f1c 0 2024-07-16T09:55:32.629 INFO:tasks.workunit.client.0.smithi156.stdout:2/300: fsync d0/d13/d17/f21 0 2024-07-16T09:55:32.630 INFO:tasks.workunit.client.0.smithi156.stdout:7/171: write fb [326170,29196] 0 2024-07-16T09:55:32.630 INFO:tasks.workunit.client.0.smithi156.stdout:3/159: fsync d8/d14/f1c 0 2024-07-16T09:55:32.635 INFO:tasks.workunit.client.0.smithi156.stdout:3/160: rename l6 to d8/d14/l35 0 2024-07-16T09:55:32.636 INFO:tasks.workunit.client.0.smithi156.stdout:2/301: creat d0/d5/d29/d2d/d5b/d5f/f66 x:0 0 0 2024-07-16T09:55:32.637 INFO:tasks.workunit.client.0.smithi156.stdout:3/161: chown d8/d14/f1c 208 1 2024-07-16T09:55:32.638 INFO:tasks.workunit.client.0.smithi156.stdout:7/172: dwrite dc/d13/f35 [0,4194304] 0 2024-07-16T09:55:32.638 INFO:tasks.workunit.client.0.smithi156.stdout:6/204: dwrite d3/d26/d30/f44 [0,4194304] 0 2024-07-16T09:55:32.639 INFO:tasks.workunit.client.0.smithi156.stdout:3/162: write d8/d14/f1c [14615,87582] 0 2024-07-16T09:55:32.641 INFO:tasks.workunit.client.0.smithi156.stdout:6/205: write d3/d26/d1f/f2c [1634484,26277] 0 2024-07-16T09:55:32.641 INFO:tasks.workunit.client.0.smithi156.stdout:3/163: truncate d8/d14/f30 719607 0 2024-07-16T09:55:32.647 INFO:tasks.workunit.client.0.smithi156.stdout:6/206: write d3/d26/d30/f44 [4145843,130375] 0 2024-07-16T09:55:32.648 INFO:tasks.workunit.client.0.smithi156.stdout:3/164: write d8/da/d13/f20 [758522,63101] 0 2024-07-16T09:55:32.649 INFO:tasks.workunit.client.1.smithi176.stdout:2/238: sync 2024-07-16T09:55:32.649 INFO:tasks.workunit.client.0.smithi156.stdout:3/165: stat d8/fb 0 2024-07-16T09:55:32.650 INFO:tasks.workunit.client.1.smithi176.stdout:8/361: getdents d8/d14/d1e/d7c 0 2024-07-16T09:55:32.652 INFO:tasks.workunit.client.1.smithi176.stdout:5/343: creat d4/d14/d19/d1b/d61/f80 x:0 0 0 2024-07-16T09:55:32.652 INFO:tasks.workunit.client.1.smithi176.stdout:6/223: fsync d1/d5/d7/f9 0 2024-07-16T09:55:32.652 INFO:tasks.workunit.client.0.smithi156.stdout:7/173: dwrite dc/d13/f35 [0,4194304] 0 2024-07-16T09:55:32.654 INFO:tasks.workunit.client.1.smithi176.stdout:2/239: write dd/d2e/d35/f3f [4113956,48120] 0 2024-07-16T09:55:32.655 INFO:tasks.workunit.client.0.smithi156.stdout:9/200: dread d2/d5/f8 [0,4194304] 0 2024-07-16T09:55:32.655 INFO:tasks.workunit.client.0.smithi156.stdout:6/207: mknod d3/d26/d1f/c46 0 2024-07-16T09:55:32.655 INFO:tasks.workunit.client.1.smithi176.stdout:6/224: write d1/f3b [617459,128359] 0 2024-07-16T09:55:32.655 INFO:tasks.workunit.client.0.smithi156.stdout:3/166: symlink d8/d14/l36 0 2024-07-16T09:55:32.656 INFO:tasks.workunit.client.0.smithi156.stdout:2/302: dwrite d0/d13/d17/d1c/f1f [0,4194304] 0 2024-07-16T09:55:32.656 INFO:tasks.workunit.client.0.smithi156.stdout:3/167: readlink d8/l16 0 2024-07-16T09:55:32.657 INFO:tasks.workunit.client.1.smithi176.stdout:8/362: unlink d8/d23/l32 0 2024-07-16T09:55:32.657 INFO:tasks.workunit.client.1.smithi176.stdout:2/240: chown dd/de/l43 2 1 2024-07-16T09:55:32.658 INFO:tasks.workunit.client.0.smithi156.stdout:7/174: mkdir dc/d44 0 2024-07-16T09:55:32.660 INFO:tasks.workunit.client.0.smithi156.stdout:2/303: write d0/d5/d29/d2d/f40 [1721250,48377] 0 2024-07-16T09:55:32.660 INFO:tasks.workunit.client.0.smithi156.stdout:6/208: creat d3/d26/d1f/d19/d29/d3b/f47 x:0 0 0 2024-07-16T09:55:32.660 INFO:tasks.workunit.client.1.smithi176.stdout:6/225: write d1/d5/d7/f4c [1043271,95867] 0 2024-07-16T09:55:32.660 INFO:tasks.workunit.client.1.smithi176.stdout:5/344: mkdir d4/d14/d3d/d81 0 2024-07-16T09:55:32.661 INFO:tasks.workunit.client.1.smithi176.stdout:1/155: creat d4/f31 x:0 0 0 2024-07-16T09:55:32.665 INFO:tasks.workunit.client.1.smithi176.stdout:2/241: creat dd/d16/d39/f4b x:0 0 0 2024-07-16T09:55:32.665 INFO:tasks.workunit.client.0.smithi156.stdout:2/304: read - d0/d13/d17/d1c/f5e zero size 2024-07-16T09:55:32.667 INFO:tasks.workunit.client.0.smithi156.stdout:6/209: mknod d3/d13/c48 0 2024-07-16T09:55:32.667 INFO:tasks.workunit.client.1.smithi176.stdout:2/242: rmdir dd/d36 39 2024-07-16T09:55:32.668 INFO:tasks.workunit.client.0.smithi156.stdout:3/168: dwrite d8/da/d28/f2a [0,4194304] 0 2024-07-16T09:55:32.670 INFO:tasks.workunit.client.0.smithi156.stdout:0/95: sync 2024-07-16T09:55:32.670 INFO:tasks.workunit.client.1.smithi176.stdout:6/226: fdatasync d1/d5/d7/f19 0 2024-07-16T09:55:32.671 INFO:tasks.workunit.client.0.smithi156.stdout:0/96: stat d0/f3 0 2024-07-16T09:55:32.672 INFO:tasks.workunit.client.1.smithi176.stdout:8/363: dwrite d8/f2e [0,4194304] 0 2024-07-16T09:55:32.672 INFO:tasks.workunit.client.1.smithi176.stdout:5/345: rename d4/d14/d3d/d4d to d4/d37/d72/d82 0 2024-07-16T09:55:32.673 INFO:tasks.workunit.client.1.smithi176.stdout:7/263: dwrite d4/d47/f44 [0,4194304] 0 2024-07-16T09:55:32.674 INFO:tasks.workunit.client.0.smithi156.stdout:9/201: link d2/de/l35 d2/d5/d16/d22/l3e 0 2024-07-16T09:55:32.675 INFO:tasks.workunit.client.0.smithi156.stdout:0/97: write d0/f14 [485096,14514] 0 2024-07-16T09:55:32.676 INFO:tasks.workunit.client.0.smithi156.stdout:6/210: creat d3/d26/d30/d45/f49 x:0 0 0 2024-07-16T09:55:32.676 INFO:tasks.workunit.client.1.smithi176.stdout:1/156: link d4/db/f16 d4/d8/d2c/d2d/f32 0 2024-07-16T09:55:32.677 INFO:tasks.workunit.client.0.smithi156.stdout:2/305: dread d0/d13/d17/d1c/f27 [0,4194304] 0 2024-07-16T09:55:32.677 INFO:tasks.workunit.client.0.smithi156.stdout:3/169: dread d8/fb [0,4194304] 0 2024-07-16T09:55:32.678 INFO:tasks.workunit.client.1.smithi176.stdout:2/243: rename dd/d16/l29 to dd/d36/d45/l4c 0 2024-07-16T09:55:32.679 INFO:tasks.workunit.client.0.smithi156.stdout:6/211: write d3/f25 [1668135,26320] 0 2024-07-16T09:55:32.680 INFO:tasks.workunit.client.1.smithi176.stdout:1/157: chown d4/f10 689133094 1 2024-07-16T09:55:32.680 INFO:tasks.workunit.client.1.smithi176.stdout:5/346: chown d4/d14/d19/d1b/d61/f76 5623 1 2024-07-16T09:55:32.682 INFO:tasks.workunit.client.1.smithi176.stdout:8/364: symlink d8/d20/d2a/d31/d2f/l81 0 2024-07-16T09:55:32.682 INFO:tasks.workunit.client.0.smithi156.stdout:9/202: getdents d2/de 0 2024-07-16T09:55:32.684 INFO:tasks.workunit.client.1.smithi176.stdout:2/244: stat dd/d16/d39/d3d 0 2024-07-16T09:55:32.685 INFO:tasks.workunit.client.1.smithi176.stdout:5/347: dread - d4/d10/d17/f63 zero size 2024-07-16T09:55:32.686 INFO:tasks.workunit.client.1.smithi176.stdout:2/245: write dd/de/f10 [2065098,121906] 0 2024-07-16T09:55:32.687 INFO:tasks.workunit.client.0.smithi156.stdout:2/306: symlink d0/d13/d17/l67 0 2024-07-16T09:55:32.687 INFO:tasks.workunit.client.0.smithi156.stdout:7/175: getdents dc 0 2024-07-16T09:55:32.689 INFO:tasks.workunit.client.1.smithi176.stdout:8/365: mkdir d8/d14/d1e/d7c/d82 0 2024-07-16T09:55:32.689 INFO:tasks.workunit.client.1.smithi176.stdout:6/227: dwrite d1/f31 [0,4194304] 0 2024-07-16T09:55:32.692 INFO:tasks.workunit.client.0.smithi156.stdout:6/212: fsync d3/d26/d1f/d19/d29/d3b/f3c 0 2024-07-16T09:55:32.692 INFO:tasks.workunit.client.0.smithi156.stdout:9/203: rename d2/d5 to d2/d5/d16/d22/d36/d33/d3a/d3f 22 2024-07-16T09:55:32.692 INFO:tasks.workunit.client.0.smithi156.stdout:3/170: dread d8/f1a [0,4194304] 0 2024-07-16T09:55:32.693 INFO:tasks.workunit.client.0.smithi156.stdout:2/307: mknod d0/d5/d29/d2d/d5b/d5f/c68 0 2024-07-16T09:55:32.693 INFO:tasks.workunit.client.1.smithi176.stdout:8/366: fdatasync d8/d14/d1e/d33/f78 0 2024-07-16T09:55:32.694 INFO:tasks.workunit.client.0.smithi156.stdout:6/213: read d3/d26/d30/f44 [1403464,39374] 0 2024-07-16T09:55:32.694 INFO:tasks.workunit.client.1.smithi176.stdout:2/246: creat dd/d36/f4d x:0 0 0 2024-07-16T09:55:32.696 INFO:tasks.workunit.client.0.smithi156.stdout:0/98: dwrite d0/d7/d9/f15 [0,4194304] 0 2024-07-16T09:55:32.697 INFO:tasks.workunit.client.0.smithi156.stdout:3/171: fdatasync d8/da/f2e 0 2024-07-16T09:55:32.700 INFO:tasks.workunit.client.1.smithi176.stdout:2/247: symlink dd/d36/l4e 0 2024-07-16T09:55:32.700 INFO:tasks.workunit.client.1.smithi176.stdout:6/228: creat d1/d5/f4f x:0 0 0 2024-07-16T09:55:32.701 INFO:tasks.workunit.client.0.smithi156.stdout:0/99: chown d0/d7/c16 1 1 2024-07-16T09:55:32.701 INFO:tasks.workunit.client.0.smithi156.stdout:3/172: write f3 [4454692,67735] 0 2024-07-16T09:55:32.701 INFO:tasks.workunit.client.0.smithi156.stdout:9/204: dread d2/d5/d16/f20 [0,4194304] 0 2024-07-16T09:55:32.702 INFO:tasks.workunit.client.1.smithi176.stdout:1/158: rmdir d4/d8 39 2024-07-16T09:55:32.703 INFO:tasks.workunit.client.1.smithi176.stdout:2/248: write dd/d16/d39/f4b [873417,122846] 0 2024-07-16T09:55:32.704 INFO:tasks.workunit.client.0.smithi156.stdout:2/308: creat d0/d13/d17/d1c/f69 x:0 0 0 2024-07-16T09:55:32.704 INFO:tasks.workunit.client.1.smithi176.stdout:7/264: write d4/d5/f12 [368200,103443] 0 2024-07-16T09:55:32.704 INFO:tasks.workunit.client.1.smithi176.stdout:9/190: dread d5/de/f1b [4194304,4194304] 0 2024-07-16T09:55:32.705 INFO:tasks.workunit.client.1.smithi176.stdout:2/249: dread - dd/d16/f3a zero size 2024-07-16T09:55:32.707 INFO:tasks.workunit.client.1.smithi176.stdout:6/229: symlink d1/dc/d1c/l50 0 2024-07-16T09:55:32.707 INFO:tasks.workunit.client.0.smithi156.stdout:0/100: creat d0/d7/f19 x:0 0 0 2024-07-16T09:55:32.707 INFO:tasks.workunit.client.0.smithi156.stdout:0/101: readlink - no filename 2024-07-16T09:55:32.707 INFO:tasks.workunit.client.0.smithi156.stdout:0/102: readlink - no filename 2024-07-16T09:55:32.707 INFO:tasks.workunit.client.1.smithi176.stdout:7/265: fsync d4/d1f/f59 0 2024-07-16T09:55:32.709 INFO:tasks.workunit.client.1.smithi176.stdout:6/230: write d1/d5/f2a [5045992,53082] 0 2024-07-16T09:55:32.710 INFO:tasks.workunit.client.0.smithi156.stdout:7/176: dwrite dc/d13/f43 [0,4194304] 0 2024-07-16T09:55:32.710 INFO:tasks.workunit.client.0.smithi156.stdout:0/103: read - d0/ff zero size 2024-07-16T09:55:32.710 INFO:tasks.workunit.client.0.smithi156.stdout:0/104: readlink - no filename 2024-07-16T09:55:32.711 INFO:tasks.workunit.client.1.smithi176.stdout:8/367: getdents d8/d14/d1e 0 2024-07-16T09:55:32.714 INFO:tasks.workunit.client.1.smithi176.stdout:1/159: symlink d4/d8/d12/d21/d25/d30/l33 0 2024-07-16T09:55:32.717 INFO:tasks.workunit.client.1.smithi176.stdout:9/191: creat d5/f35 x:0 0 0 2024-07-16T09:55:32.719 INFO:tasks.workunit.client.1.smithi176.stdout:1/160: read d4/ff [4486915,110706] 0 2024-07-16T09:55:32.719 INFO:tasks.workunit.client.1.smithi176.stdout:6/231: rename d1/dc/f2e to d1/d5/d47/f51 0 2024-07-16T09:55:32.721 INFO:tasks.workunit.client.0.smithi156.stdout:6/214: dwrite d3/d26/d30/d45/f49 [0,4194304] 0 2024-07-16T09:55:32.723 INFO:tasks.workunit.client.0.smithi156.stdout:2/309: creat d0/d5/d29/d2d/d5b/d62/f6a x:0 0 0 2024-07-16T09:55:32.724 INFO:tasks.workunit.client.1.smithi176.stdout:6/232: chown d1/d5/d47 176 1 2024-07-16T09:55:32.725 INFO:tasks.workunit.client.1.smithi176.stdout:2/250: dwrite f3 [0,4194304] 0 2024-07-16T09:55:32.727 INFO:tasks.workunit.client.0.smithi156.stdout:3/173: dwrite d8/d14/f30 [0,4194304] 0 2024-07-16T09:55:32.727 INFO:tasks.workunit.client.1.smithi176.stdout:5/348: dwrite d4/d14/f25 [0,4194304] 0 2024-07-16T09:55:32.729 INFO:tasks.workunit.client.0.smithi156.stdout:6/215: dread - d3/d26/d1f/d19/d29/f3f zero size 2024-07-16T09:55:32.731 INFO:tasks.workunit.client.0.smithi156.stdout:2/310: chown d0/d5/d29/d2d/f5a 52538 1 2024-07-16T09:55:32.732 INFO:tasks.workunit.client.1.smithi176.stdout:1/161: chown d4/d8/f1e 1039752419 1 2024-07-16T09:55:32.732 INFO:tasks.workunit.client.1.smithi176.stdout:2/251: readlink dd/de/l2c 0 2024-07-16T09:55:32.732 INFO:tasks.workunit.client.0.smithi156.stdout:9/205: dwrite d2/d5/d16/d22/d36/d33/f2c [0,4194304] 0 2024-07-16T09:55:32.733 INFO:tasks.workunit.client.0.smithi156.stdout:3/174: mknod d8/da/d13/c37 0 2024-07-16T09:55:32.735 INFO:tasks.workunit.client.0.smithi156.stdout:0/105: dread d0/d7/d9/fe [0,4194304] 0 2024-07-16T09:55:32.737 INFO:tasks.workunit.client.0.smithi156.stdout:3/175: dread - d8/da/f32 zero size 2024-07-16T09:55:32.738 INFO:tasks.workunit.client.1.smithi176.stdout:8/368: dwrite d8/d23/d2c/d37/f3c [4194304,4194304] 0 2024-07-16T09:55:32.739 INFO:tasks.workunit.client.0.smithi156.stdout:0/106: write d0/d7/f10 [424322,84111] 0 2024-07-16T09:55:32.741 INFO:tasks.workunit.client.1.smithi176.stdout:9/192: dwrite d5/d1d/f27 [0,4194304] 0 2024-07-16T09:55:32.744 INFO:tasks.workunit.client.0.smithi156.stdout:9/206: symlink d2/d5/d16/d22/d2b/d3d/d3c/l40 0 2024-07-16T09:55:32.748 INFO:tasks.workunit.client.0.smithi156.stdout:3/176: mknod d8/da/d1d/c38 0 2024-07-16T09:55:32.749 INFO:tasks.workunit.client.1.smithi176.stdout:2/252: creat dd/d16/d39/f4f x:0 0 0 2024-07-16T09:55:32.749 INFO:tasks.workunit.client.1.smithi176.stdout:7/266: dread d4/d5/d21/f50 [0,4194304] 0 2024-07-16T09:55:32.750 INFO:tasks.workunit.client.1.smithi176.stdout:2/253: read dd/f2a [792042,91110] 0 2024-07-16T09:55:32.751 INFO:tasks.workunit.client.1.smithi176.stdout:6/233: rename d1/f18 to d1/d5/d7/f52 0 2024-07-16T09:55:32.752 INFO:tasks.workunit.client.1.smithi176.stdout:7/267: dread - d4/d5/d21/f5a zero size 2024-07-16T09:55:32.753 INFO:tasks.workunit.client.1.smithi176.stdout:2/254: dread - dd/d16/d39/f4f zero size 2024-07-16T09:55:32.753 INFO:tasks.workunit.client.1.smithi176.stdout:8/369: mkdir d8/d23/d76/d83 0 2024-07-16T09:55:32.754 INFO:tasks.workunit.client.0.smithi156.stdout:2/311: getdents d0/d13 0 2024-07-16T09:55:32.754 INFO:tasks.workunit.client.0.smithi156.stdout:6/216: link d3/d26/d1e/l27 d3/d26/d30/l4a 0 2024-07-16T09:55:32.755 INFO:tasks.workunit.client.1.smithi176.stdout:5/349: dread d4/d10/f22 [0,4194304] 0 2024-07-16T09:55:32.756 INFO:tasks.workunit.client.0.smithi156.stdout:9/207: dwrite d2/d5/d16/d22/d36/d33/f2c [0,4194304] 0 2024-07-16T09:55:32.756 INFO:tasks.workunit.client.1.smithi176.stdout:7/268: chown d4/d47/f34 0 1 2024-07-16T09:55:32.757 INFO:tasks.workunit.client.0.smithi156.stdout:3/177: symlink d8/da/d1d/d2c/l39 0 2024-07-16T09:55:32.757 INFO:tasks.workunit.client.0.smithi156.stdout:6/217: read d3/d26/d30/d45/f49 [1257990,54528] 0 2024-07-16T09:55:32.757 INFO:tasks.workunit.client.1.smithi176.stdout:9/193: mknod d5/c36 0 2024-07-16T09:55:32.758 INFO:tasks.workunit.client.1.smithi176.stdout:2/255: creat dd/d2e/f50 x:0 0 0 2024-07-16T09:55:32.759 INFO:tasks.workunit.client.0.smithi156.stdout:2/312: creat d0/d13/f6b x:0 0 0 2024-07-16T09:55:32.760 INFO:tasks.workunit.client.0.smithi156.stdout:6/218: write d3/d26/d1f/f10 [1617126,73386] 0 2024-07-16T09:55:32.762 INFO:tasks.workunit.client.0.smithi156.stdout:6/219: chown d3/d26/d1f/d19/d31 81 1 2024-07-16T09:55:32.763 INFO:tasks.workunit.client.1.smithi176.stdout:6/234: dread d1/f31 [0,4194304] 0 2024-07-16T09:55:32.764 INFO:tasks.workunit.client.0.smithi156.stdout:6/220: write d3/d26/d1f/d19/d29/f3f [861205,71269] 0 2024-07-16T09:55:32.765 INFO:tasks.workunit.client.1.smithi176.stdout:5/350: creat d4/d37/d72/d82/f83 x:0 0 0 2024-07-16T09:55:32.766 INFO:tasks.workunit.client.1.smithi176.stdout:2/256: write dd/f2a [2706227,21195] 0 2024-07-16T09:55:32.766 INFO:tasks.workunit.client.0.smithi156.stdout:3/178: write d8/fb [4063446,16167] 0 2024-07-16T09:55:32.767 INFO:tasks.workunit.client.0.smithi156.stdout:9/208: symlink d2/d5/l41 0 2024-07-16T09:55:32.768 INFO:tasks.workunit.client.0.smithi156.stdout:9/209: write d2/f13 [5198955,50079] 0 2024-07-16T09:55:32.768 INFO:tasks.workunit.client.1.smithi176.stdout:0/261: sync 2024-07-16T09:55:32.769 INFO:tasks.workunit.client.0.smithi156.stdout:3/179: readlink d8/l16 0 2024-07-16T09:55:32.770 INFO:tasks.workunit.client.0.smithi156.stdout:2/313: link d0/d13/d17/d1c/d51/f5c d0/d5/d29/d2d/d5b/f6c 0 2024-07-16T09:55:32.771 INFO:tasks.workunit.client.1.smithi176.stdout:9/194: mkdir d5/de/d37 0 2024-07-16T09:55:32.772 INFO:tasks.workunit.client.1.smithi176.stdout:8/370: creat d8/d23/d2c/d34/f84 x:0 0 0 2024-07-16T09:55:32.772 INFO:tasks.workunit.client.1.smithi176.stdout:6/235: creat d1/dc/d1c/f53 x:0 0 0 2024-07-16T09:55:32.773 INFO:tasks.workunit.client.0.smithi156.stdout:6/221: creat d3/d26/f4b x:0 0 0 2024-07-16T09:55:32.773 INFO:tasks.workunit.client.0.smithi156.stdout:0/107: read d0/d7/f10 [37360,137] 0 2024-07-16T09:55:32.774 INFO:tasks.workunit.client.0.smithi156.stdout:0/108: readlink - no filename 2024-07-16T09:55:32.774 INFO:tasks.workunit.client.0.smithi156.stdout:0/109: readlink - no filename 2024-07-16T09:55:32.775 INFO:tasks.workunit.client.0.smithi156.stdout:6/222: fdatasync d3/f25 0 2024-07-16T09:55:32.775 INFO:tasks.workunit.client.1.smithi176.stdout:0/262: dread d5/d1c/f21 [0,4194304] 0 2024-07-16T09:55:32.776 INFO:tasks.workunit.client.0.smithi156.stdout:0/110: read - d0/d7/f19 zero size 2024-07-16T09:55:32.776 INFO:tasks.workunit.client.1.smithi176.stdout:2/257: mknod dd/d16/d39/d3d/c51 0 2024-07-16T09:55:32.777 INFO:tasks.workunit.client.1.smithi176.stdout:9/195: mkdir d5/de/d38 0 2024-07-16T09:55:32.778 INFO:tasks.workunit.client.0.smithi156.stdout:2/314: creat d0/d5/d29/d2d/d5b/f6d x:0 0 0 2024-07-16T09:55:32.779 INFO:tasks.workunit.client.1.smithi176.stdout:5/351: dwrite d4/d37/d72/d38/d39/f5d [0,4194304] 0 2024-07-16T09:55:32.780 INFO:tasks.workunit.client.0.smithi156.stdout:0/111: mkdir d0/d1a 0 2024-07-16T09:55:32.780 INFO:tasks.workunit.client.1.smithi176.stdout:1/162: dread d4/ff [0,4194304] 0 2024-07-16T09:55:32.781 INFO:tasks.workunit.client.0.smithi156.stdout:2/315: mkdir d0/d13/d6e 0 2024-07-16T09:55:32.781 INFO:tasks.workunit.client.1.smithi176.stdout:8/371: unlink d8/d20/d2a/d31/d2f/c64 0 2024-07-16T09:55:32.782 INFO:tasks.workunit.client.1.smithi176.stdout:9/196: write d5/f2d [820781,130727] 0 2024-07-16T09:55:32.784 INFO:tasks.workunit.client.1.smithi176.stdout:0/263: write d5/f22 [63015,113891] 0 2024-07-16T09:55:32.784 INFO:tasks.workunit.client.1.smithi176.stdout:1/163: chown d4/d8/d2c/f2f 91838113 1 2024-07-16T09:55:32.784 INFO:tasks.workunit.client.0.smithi156.stdout:6/223: dwrite d3/d26/d1f/fa [0,4194304] 0 2024-07-16T09:55:32.784 INFO:tasks.workunit.client.0.smithi156.stdout:2/316: write d0/d13/f59 [4340814,69619] 0 2024-07-16T09:55:32.785 INFO:tasks.workunit.client.0.smithi156.stdout:0/112: dread d0/f14 [0,4194304] 0 2024-07-16T09:55:32.786 INFO:tasks.workunit.client.1.smithi176.stdout:2/258: mkdir dd/d49/d52 0 2024-07-16T09:55:32.786 INFO:tasks.workunit.client.0.smithi156.stdout:6/224: write d3/d26/d1f/f37 [1485669,103967] 0 2024-07-16T09:55:32.787 INFO:tasks.workunit.client.1.smithi176.stdout:8/372: creat d8/d14/d1e/d7c/f85 x:0 0 0 2024-07-16T09:55:32.788 INFO:tasks.workunit.client.0.smithi156.stdout:6/225: chown d3/l18 1207069 1 2024-07-16T09:55:32.790 INFO:tasks.workunit.client.0.smithi156.stdout:9/210: write d2/d5/d16/f20 [1289133,29204] 0 2024-07-16T09:55:32.792 INFO:tasks.workunit.client.0.smithi156.stdout:6/226: rename d3/l18 to d3/d26/d1e/l4c 0 2024-07-16T09:55:32.792 INFO:tasks.workunit.client.0.smithi156.stdout:3/180: truncate d8/d14/f1c 2856715 0 2024-07-16T09:55:32.792 INFO:tasks.workunit.client.1.smithi176.stdout:1/164: mkdir d4/d8/d12/d21/d34 0 2024-07-16T09:55:32.792 INFO:tasks.workunit.client.1.smithi176.stdout:9/197: rename d5/de/f1b to d5/de/d21/f39 0 2024-07-16T09:55:32.793 INFO:tasks.workunit.client.0.smithi156.stdout:0/113: mkdir d0/d7/d9/d1b 0 2024-07-16T09:55:32.794 INFO:tasks.workunit.client.0.smithi156.stdout:9/211: mkdir d2/de/d42 0 2024-07-16T09:55:32.795 INFO:tasks.workunit.client.0.smithi156.stdout:9/212: stat d2/c7 0 2024-07-16T09:55:32.796 INFO:tasks.workunit.client.1.smithi176.stdout:2/259: symlink dd/d49/l53 0 2024-07-16T09:55:32.796 INFO:tasks.workunit.client.0.smithi156.stdout:2/317: getdents d0/d5/d29 0 2024-07-16T09:55:32.797 INFO:tasks.workunit.client.0.smithi156.stdout:0/114: chown d0/d7/d9/f12 1028905 1 2024-07-16T09:55:32.797 INFO:tasks.workunit.client.0.smithi156.stdout:9/213: chown d2/la 420 1 2024-07-16T09:55:32.798 INFO:tasks.workunit.client.1.smithi176.stdout:6/236: rmdir d1 39 2024-07-16T09:55:32.798 INFO:tasks.workunit.client.0.smithi156.stdout:9/214: chown d2/d5/d16/d22/d36/d33/f2c 4 1 2024-07-16T09:55:32.799 INFO:tasks.workunit.client.1.smithi176.stdout:0/264: rename d5/f15 to d5/d39/f52 0 2024-07-16T09:55:32.800 INFO:tasks.workunit.client.1.smithi176.stdout:1/165: mknod d4/d8/d2c/c35 0 2024-07-16T09:55:32.800 INFO:tasks.workunit.client.1.smithi176.stdout:7/269: dwrite d4/d5/f2f [0,4194304] 0 2024-07-16T09:55:32.803 INFO:tasks.workunit.client.0.smithi156.stdout:0/115: mknod d0/d1a/c1c 0 2024-07-16T09:55:32.803 INFO:tasks.workunit.client.1.smithi176.stdout:0/265: chown d5/d1c/d24/d26/f3d 35387 1 2024-07-16T09:55:32.804 INFO:tasks.workunit.client.0.smithi156.stdout:9/215: symlink d2/l43 0 2024-07-16T09:55:32.805 INFO:tasks.workunit.client.0.smithi156.stdout:9/216: chown d2/d5 23031 1 2024-07-16T09:55:32.805 INFO:tasks.workunit.client.1.smithi176.stdout:2/260: mknod dd/d16/d39/c54 0 2024-07-16T09:55:32.805 INFO:tasks.workunit.client.1.smithi176.stdout:9/198: mkdir d5/d3a 0 2024-07-16T09:55:32.806 INFO:tasks.workunit.client.0.smithi156.stdout:2/318: creat d0/d5/d29/d2d/d5b/f6f x:0 0 0 2024-07-16T09:55:32.807 INFO:tasks.workunit.client.1.smithi176.stdout:8/373: rename d8/d20/d2a/d61/c65 to d8/d20/d2a/d31/d46/d6a/c86 0 2024-07-16T09:55:32.807 INFO:tasks.workunit.client.0.smithi156.stdout:0/116: creat d0/d7/d9/f1d x:0 0 0 2024-07-16T09:55:32.808 INFO:tasks.workunit.client.0.smithi156.stdout:0/117: rename d0 to d0/d7/d9/d1e 22 2024-07-16T09:55:32.809 INFO:tasks.workunit.client.1.smithi176.stdout:7/270: mkdir d4/d47/d5d 0 2024-07-16T09:55:32.811 INFO:tasks.workunit.client.1.smithi176.stdout:0/266: rename d5/d2c/f47 to d5/d2c/f53 0 2024-07-16T09:55:32.813 INFO:tasks.workunit.client.0.smithi156.stdout:2/319: symlink d0/d13/d6e/l70 0 2024-07-16T09:55:32.813 INFO:tasks.workunit.client.1.smithi176.stdout:2/261: creat dd/d49/f55 x:0 0 0 2024-07-16T09:55:32.813 INFO:tasks.workunit.client.0.smithi156.stdout:0/118: write d0/f14 [917603,121551] 0 2024-07-16T09:55:32.814 INFO:tasks.workunit.client.1.smithi176.stdout:9/199: dwrite d5/f1c [0,4194304] 0 2024-07-16T09:55:32.814 INFO:tasks.workunit.client.0.smithi156.stdout:9/217: creat d2/d5/f44 x:0 0 0 2024-07-16T09:55:32.815 INFO:tasks.workunit.client.0.smithi156.stdout:2/320: truncate d0/d13/f43 1920659 0 2024-07-16T09:55:32.816 INFO:tasks.workunit.client.1.smithi176.stdout:7/271: dread d4/fb [0,4194304] 0 2024-07-16T09:55:32.816 INFO:tasks.workunit.client.1.smithi176.stdout:6/237: mkdir d1/d5/d7/d54 0 2024-07-16T09:55:32.816 INFO:tasks.workunit.client.0.smithi156.stdout:2/321: dread - d0/d5/d29/f3f zero size 2024-07-16T09:55:32.817 INFO:tasks.workunit.client.0.smithi156.stdout:6/227: truncate d3/d26/d1f/d19/d29/d3b/f40 232209 0 2024-07-16T09:55:32.818 INFO:tasks.workunit.client.1.smithi176.stdout:8/374: symlink d8/d14/d1e/d7c/d82/l87 0 2024-07-16T09:55:32.819 INFO:tasks.workunit.client.1.smithi176.stdout:7/272: symlink d4/d1f/d22/l5e 0 2024-07-16T09:55:32.819 INFO:tasks.workunit.client.0.smithi156.stdout:0/119: symlink d0/d7/d9/d1b/l1f 0 2024-07-16T09:55:32.820 INFO:tasks.workunit.client.0.smithi156.stdout:9/218: symlink d2/de/d42/l45 0 2024-07-16T09:55:32.821 INFO:tasks.workunit.client.1.smithi176.stdout:1/166: rename d4/d8/d2c/c35 to d4/c36 0 2024-07-16T09:55:32.822 INFO:tasks.workunit.client.0.smithi156.stdout:2/322: creat d0/d13/d17/d1c/f71 x:0 0 0 2024-07-16T09:55:32.823 INFO:tasks.workunit.client.0.smithi156.stdout:0/120: creat d0/d7/f20 x:0 0 0 2024-07-16T09:55:32.824 INFO:tasks.workunit.client.1.smithi176.stdout:6/238: creat d1/d5/d7/d54/f55 x:0 0 0 2024-07-16T09:55:32.824 INFO:tasks.workunit.client.0.smithi156.stdout:9/219: mknod d2/d5/d16/d22/d36/d33/c46 0 2024-07-16T09:55:32.825 INFO:tasks.workunit.client.1.smithi176.stdout:8/375: mkdir d8/d23/d2c/d34/d62/d88 0 2024-07-16T09:55:32.825 INFO:tasks.workunit.client.1.smithi176.stdout:0/267: dwrite d5/d2c/f2d [0,4194304] 0 2024-07-16T09:55:32.826 INFO:tasks.workunit.client.1.smithi176.stdout:7/273: mknod d4/dd/c5f 0 2024-07-16T09:55:32.827 INFO:tasks.workunit.client.1.smithi176.stdout:6/239: write d1/d5/d7/fe [267778,85087] 0 2024-07-16T09:55:32.827 INFO:tasks.workunit.client.0.smithi156.stdout:0/121: creat d0/d7/f21 x:0 0 0 2024-07-16T09:55:32.829 INFO:tasks.workunit.client.0.smithi156.stdout:9/220: creat d2/de/d42/f47 x:0 0 0 2024-07-16T09:55:32.831 INFO:tasks.workunit.client.0.smithi156.stdout:0/122: mkdir d0/d22 0 2024-07-16T09:55:32.831 INFO:tasks.workunit.client.0.smithi156.stdout:0/123: stat d0/d7/f10 0 2024-07-16T09:55:32.831 INFO:tasks.workunit.client.0.smithi156.stdout:0/124: chown d0/d1a 17 1 2024-07-16T09:55:32.832 INFO:tasks.workunit.client.0.smithi156.stdout:0/125: dread - d0/d7/f21 zero size 2024-07-16T09:55:32.834 INFO:tasks.workunit.client.1.smithi176.stdout:7/274: write d4/d1f/d22/f52 [2404794,15606] 0 2024-07-16T09:55:32.835 INFO:tasks.workunit.client.0.smithi156.stdout:2/323: link d0/d13/d17/d1c/l1d d0/d5/d29/d2d/d5b/d62/l72 0 2024-07-16T09:55:32.835 INFO:tasks.workunit.client.0.smithi156.stdout:2/324: fsync d0/d5/f44 0 2024-07-16T09:55:32.836 INFO:tasks.workunit.client.0.smithi156.stdout:2/325: chown d0 6478 1 2024-07-16T09:55:32.837 INFO:tasks.workunit.client.0.smithi156.stdout:0/126: symlink d0/d7/d9/d1b/l23 0 2024-07-16T09:55:32.838 INFO:tasks.workunit.client.0.smithi156.stdout:2/326: dread - d0/d5/d29/d2d/d5b/d5f/f66 zero size 2024-07-16T09:55:32.839 INFO:tasks.workunit.client.0.smithi156.stdout:0/127: write d0/d7/f20 [823709,36271] 0 2024-07-16T09:55:32.839 INFO:tasks.workunit.client.1.smithi176.stdout:8/376: dwrite d8/d23/f21 [0,4194304] 0 2024-07-16T09:55:32.840 INFO:tasks.workunit.client.1.smithi176.stdout:7/275: write d4/d5/f31 [3343459,48450] 0 2024-07-16T09:55:32.841 INFO:tasks.workunit.client.0.smithi156.stdout:6/228: rename d3/d26/d1f/d19/d29 to d3/d26/d4d 0 2024-07-16T09:55:32.842 INFO:tasks.workunit.client.0.smithi156.stdout:9/221: getdents d2/d5/d16/d22 0 2024-07-16T09:55:32.844 INFO:tasks.workunit.client.1.smithi176.stdout:7/276: symlink d4/d1f/d22/l60 0 2024-07-16T09:55:32.845 INFO:tasks.workunit.client.0.smithi156.stdout:6/229: fdatasync d3/d26/d4d/f3f 0 2024-07-16T09:55:32.845 INFO:tasks.workunit.client.1.smithi176.stdout:9/200: dread d5/f10 [0,4194304] 0 2024-07-16T09:55:32.846 INFO:tasks.workunit.client.1.smithi176.stdout:6/240: getdents d1/d5/d7/d49 0 2024-07-16T09:55:32.846 INFO:tasks.workunit.client.0.smithi156.stdout:1/163: sync 2024-07-16T09:55:32.847 INFO:tasks.workunit.client.1.smithi176.stdout:0/268: write d5/f14 [183092,9788] 0 2024-07-16T09:55:32.847 INFO:tasks.workunit.client.0.smithi156.stdout:6/230: dread - d3/d26/d4d/d3b/f3c zero size 2024-07-16T09:55:32.848 INFO:tasks.workunit.client.1.smithi176.stdout:9/201: chown d5/de/f17 5 1 2024-07-16T09:55:32.850 INFO:tasks.workunit.client.1.smithi176.stdout:8/377: dwrite d8/d14/d1e/f60 [0,4194304] 0 2024-07-16T09:55:32.851 INFO:tasks.workunit.client.1.smithi176.stdout:7/277: creat d4/d5/d10/d17/d1d/f61 x:0 0 0 2024-07-16T09:55:32.851 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:32.848+0000 7f8904a99640 1 -- 172.21.15.156:0/898170563 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8900064910 msgr2=0x7f89000faf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:55:32.851 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:32.848+0000 7f8904a99640 1 --2- 172.21.15.156:0/898170563 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8900064910 0x7f89000faf00 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f88f40048d0 tx=0x7f88f4009da0 comp rx=0 tx=0).stop 2024-07-16T09:55:32.851 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:32.849+0000 7f8904a99640 1 -- 172.21.15.156:0/898170563 shutdown_connections 2024-07-16T09:55:32.852 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:32.849+0000 7f8904a99640 1 --2- 172.21.15.156:0/898170563 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8900064910 0x7f89000faf00 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:55:32.852 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:32.849+0000 7f8904a99640 1 --2- 172.21.15.156:0/898170563 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8900064500 0x7f89000fa9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:55:32.852 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:32.849+0000 7f8904a99640 1 -- 172.21.15.156:0/898170563 >> 172.21.15.156:0/898170563 conn(0x7f89000f61a0 msgr2=0x7f89000f85b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:55:32.852 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:32.849+0000 7f8904a99640 1 -- 172.21.15.156:0/898170563 shutdown_connections 2024-07-16T09:55:32.852 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:32.849+0000 7f8904a99640 1 -- 172.21.15.156:0/898170563 wait complete. 2024-07-16T09:55:32.852 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:32.850+0000 7f8904a99640 1 Processor -- start 2024-07-16T09:55:32.852 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:32.850+0000 7f8904a99640 1 -- start start 2024-07-16T09:55:32.852 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:32.850+0000 7f8904a99640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8900064500 0x7f8900106680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:55:32.852 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:32.850+0000 7f8904a99640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8900064910 0x7f8900106bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:55:32.852 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:32.850+0000 7f8904a99640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89001071c0 con 0x7f8900064910 2024-07-16T09:55:32.852 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:32.850+0000 7f8904a99640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8900107a80 con 0x7f8900064500 2024-07-16T09:55:32.853 INFO:tasks.workunit.client.0.smithi156.stdout:1/164: fsync dc/d20/f30 0 2024-07-16T09:55:32.853 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:32.850+0000 7f88fdd9b640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8900064910 0x7f8900106bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:55:32.853 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:32.850+0000 7f88fdd9b640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8900064910 0x7f8900106bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:40842/0 (socket says 172.21.15.156:40842) 2024-07-16T09:55:32.853 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:32.850+0000 7f88fe59c640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8900064500 0x7f8900106680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:55:32.853 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:32.850+0000 7f88fdd9b640 1 -- 172.21.15.156:0/135795873 learned_addr learned my addr 172.21.15.156:0/135795873 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T09:55:32.853 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:32.851+0000 7f88fe59c640 1 -- 172.21.15.156:0/135795873 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8900064910 msgr2=0x7f8900106bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:55:32.854 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:32.851+0000 7f88fe59c640 1 --2- 172.21.15.156:0/135795873 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8900064910 0x7f8900106bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:55:32.854 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:32.851+0000 7f88fe59c640 1 -- 172.21.15.156:0/135795873 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88f4005040 con 0x7f8900064500 2024-07-16T09:55:32.854 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:32.852+0000 7f88fe59c640 1 --2- 172.21.15.156:0/135795873 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8900064500 0x7f8900106680 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f88f000c7e0 tx=0x7f88f0007f20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:55:32.854 INFO:tasks.workunit.client.0.smithi156.stdout:2/327: dwrite d0/d5/d29/f32 [0,4194304] 0 2024-07-16T09:55:32.854 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:32.852+0000 7f88eeffd640 1 -- 172.21.15.156:0/135795873 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f88f000a820 con 0x7f8900064500 2024-07-16T09:55:32.855 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:32.852+0000 7f88eeffd640 1 -- 172.21.15.156:0/135795873 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f88f000ae50 con 0x7f8900064500 2024-07-16T09:55:32.855 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:32.852+0000 7f88eeffd640 1 -- 172.21.15.156:0/135795873 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f88f00113a0 con 0x7f8900064500 2024-07-16T09:55:32.855 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:32.852+0000 7f8904a99640 1 -- 172.21.15.156:0/135795873 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8900107d60 con 0x7f8900064500 2024-07-16T09:55:32.855 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:32.852+0000 7f8904a99640 1 -- 172.21.15.156:0/135795873 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8900108280 con 0x7f8900064500 2024-07-16T09:55:32.855 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:32.853+0000 7f8904a99640 1 -- 172.21.15.156:0/135795873 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89000f9e20 con 0x7f8900064500 2024-07-16T09:55:32.856 INFO:tasks.workunit.client.1.smithi176.stdout:7/278: stat d4/dd/le 0 2024-07-16T09:55:32.856 INFO:tasks.workunit.client.1.smithi176.stdout:9/202: unlink d5/c36 0 2024-07-16T09:55:32.856 INFO:tasks.workunit.client.1.smithi176.stdout:7/279: write d4/d5/d10/d17/d1d/f61 [686321,55520] 0 2024-07-16T09:55:32.856 INFO:tasks.workunit.client.1.smithi176.stdout:0/269: creat d5/d2c/f54 x:0 0 0 2024-07-16T09:55:32.857 INFO:tasks.workunit.client.1.smithi176.stdout:6/241: unlink d1/d5/d7/f4c 0 2024-07-16T09:55:32.857 INFO:tasks.workunit.client.1.smithi176.stdout:8/378: mknod d8/d23/d76/c89 0 2024-07-16T09:55:32.858 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:32.857+0000 7f88eeffd640 1 -- 172.21.15.156:0/135795873 <== mon.1 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 20) v1 ==== 88539+0+0 (secure 0 0 0) 0x7f88f0012400 con 0x7f8900064500 2024-07-16T09:55:32.859 INFO:tasks.workunit.client.1.smithi176.stdout:7/280: read d4/f11 [3274532,7146] 0 2024-07-16T09:55:32.859 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:32.857+0000 7f88eeffd640 1 --2- 172.21.15.156:0/135795873 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f88e806a5a0 0x7f88e806ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:55:32.859 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:32.858+0000 7f88eeffd640 1 -- 172.21.15.156:0/135795873 <== mon.1 v2:172.21.15.176:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f88f0089310 con 0x7f8900064500 2024-07-16T09:55:32.859 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:32.858+0000 7f88fdd9b640 1 --2- 172.21.15.156:0/135795873 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f88e806a5a0 0x7f88e806ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:55:32.860 INFO:tasks.workunit.client.0.smithi156.stdout:9/222: dread d2/f13 [4194304,4194304] 0 2024-07-16T09:55:32.860 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:32.859+0000 7f88fdd9b640 1 --2- 172.21.15.156:0/135795873 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f88e806a5a0 0x7f88e806ca50 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f88f4007ac0 tx=0x7f88f4035040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:55:32.862 INFO:tasks.workunit.client.0.smithi156.stdout:6/231: dwrite d3/d26/d1e/f20 [0,4194304] 0 2024-07-16T09:55:32.862 INFO:tasks.workunit.client.0.smithi156.stdout:2/328: creat d0/d5/d29/d2d/f73 x:0 0 0 2024-07-16T09:55:32.863 INFO:tasks.workunit.client.1.smithi176.stdout:0/270: creat d5/d44/f55 x:0 0 0 2024-07-16T09:55:32.863 INFO:tasks.workunit.client.0.smithi156.stdout:9/223: fsync d2/d5/d16/d22/d36/d33/f2c 0 2024-07-16T09:55:32.865 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:32.864+0000 7f88eeffd640 1 -- 172.21.15.156:0/135795873 <== mon.1 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f88f00566d0 con 0x7f8900064500 2024-07-16T09:55:32.866 INFO:tasks.workunit.client.0.smithi156.stdout:6/232: truncate d3/d26/d1f/f10 4277263 0 2024-07-16T09:55:32.869 INFO:tasks.workunit.client.0.smithi156.stdout:1/165: dwrite dc/f2f [0,4194304] 0 2024-07-16T09:55:32.869 INFO:tasks.workunit.client.1.smithi176.stdout:0/271: unlink d5/d1c/l4b 0 2024-07-16T09:55:32.871 INFO:tasks.workunit.client.0.smithi156.stdout:9/224: mknod d2/de/d42/c48 0 2024-07-16T09:55:32.871 INFO:tasks.workunit.client.1.smithi176.stdout:9/203: dwrite d5/de/d21/f39 [8388608,4194304] 0 2024-07-16T09:55:32.871 INFO:tasks.workunit.client.1.smithi176.stdout:8/379: creat d8/d14/d1e/d55/f8a x:0 0 0 2024-07-16T09:55:32.872 INFO:tasks.workunit.client.1.smithi176.stdout:6/242: dread d1/f15 [0,4194304] 0 2024-07-16T09:55:32.872 INFO:tasks.workunit.client.0.smithi156.stdout:6/233: creat d3/d26/d1f/d19/d31/f4e x:0 0 0 2024-07-16T09:55:32.873 INFO:tasks.workunit.client.1.smithi176.stdout:9/204: chown d5/de 509500296 1 2024-07-16T09:55:32.873 INFO:tasks.workunit.client.0.smithi156.stdout:6/234: chown d3/d26/d1f/f37 31345084 1 2024-07-16T09:55:32.877 INFO:tasks.workunit.client.0.smithi156.stdout:1/166: link dc/cf dc/d14/d1e/c38 0 2024-07-16T09:55:32.877 INFO:tasks.workunit.client.0.smithi156.stdout:9/225: dread d2/de/f19 [0,4194304] 0 2024-07-16T09:55:32.878 INFO:tasks.workunit.client.0.smithi156.stdout:1/167: stat dc/f10 0 2024-07-16T09:55:32.879 INFO:tasks.workunit.client.1.smithi176.stdout:7/281: truncate d4/f2b 265035 0 2024-07-16T09:55:32.882 INFO:tasks.workunit.client.0.smithi156.stdout:9/226: chown d2/l21 18 1 2024-07-16T09:55:32.882 INFO:tasks.workunit.client.0.smithi156.stdout:6/235: truncate d3/d13/d17/f2b 1446023 0 2024-07-16T09:55:32.883 INFO:tasks.workunit.client.1.smithi176.stdout:0/272: rename d5/d25/l46 to d5/d44/l56 0 2024-07-16T09:55:32.883 INFO:tasks.workunit.client.0.smithi156.stdout:1/168: symlink dc/d24/l39 0 2024-07-16T09:55:32.883 INFO:tasks.workunit.client.1.smithi176.stdout:8/380: dread d8/d20/d2a/d31/d2f/f4d [4194304,4194304] 0 2024-07-16T09:55:32.884 INFO:tasks.workunit.client.0.smithi156.stdout:6/236: chown d3/d26/l16 3 1 2024-07-16T09:55:32.885 INFO:tasks.workunit.client.0.smithi156.stdout:6/237: dread - d3/d26/f34 zero size 2024-07-16T09:55:32.887 INFO:tasks.workunit.client.1.smithi176.stdout:7/282: creat d4/d1f/f62 x:0 0 0 2024-07-16T09:55:32.887 INFO:tasks.workunit.client.0.smithi156.stdout:9/227: creat d2/d5/d16/d22/d36/d33/d3a/f49 x:0 0 0 2024-07-16T09:55:32.888 INFO:tasks.workunit.client.1.smithi176.stdout:9/205: dwrite d5/de/d21/f39 [0,4194304] 0 2024-07-16T09:55:32.889 INFO:tasks.workunit.client.0.smithi156.stdout:6/238: fdatasync d3/d26/d30/f44 0 2024-07-16T09:55:32.890 INFO:tasks.workunit.client.0.smithi156.stdout:6/239: dread - d3/d26/f34 zero size 2024-07-16T09:55:32.891 INFO:tasks.workunit.client.0.smithi156.stdout:1/169: link dc/f19 dc/d24/f3a 0 2024-07-16T09:55:32.891 INFO:tasks.workunit.client.1.smithi176.stdout:6/243: link d1/d5/d7/l21 d1/dc/d1c/l56 0 2024-07-16T09:55:32.892 INFO:tasks.workunit.client.0.smithi156.stdout:9/228: mknod d2/d5/d16/d22/d36/d33/d3a/c4a 0 2024-07-16T09:55:32.894 INFO:tasks.workunit.client.0.smithi156.stdout:6/240: fsync d3/d26/d30/f44 0 2024-07-16T09:55:32.895 INFO:tasks.workunit.client.1.smithi176.stdout:7/283: mkdir d4/dd/d63 0 2024-07-16T09:55:32.897 INFO:tasks.workunit.client.0.smithi156.stdout:9/229: creat d2/d5/d16/d22/d2b/d3d/d3c/f4b x:0 0 0 2024-07-16T09:55:32.900 INFO:tasks.workunit.client.0.smithi156.stdout:9/230: creat d2/de/d42/f4c x:0 0 0 2024-07-16T09:55:32.902 INFO:tasks.workunit.client.0.smithi156.stdout:1/170: dread f4 [0,4194304] 0 2024-07-16T09:55:32.903 INFO:tasks.workunit.client.0.smithi156.stdout:1/171: write dc/d14/f17 [2503392,60698] 0 2024-07-16T09:55:32.904 INFO:tasks.workunit.client.1.smithi176.stdout:0/273: dwrite d5/d1c/d24/d35/f4c [0,4194304] 0 2024-07-16T09:55:32.905 INFO:tasks.workunit.client.0.smithi156.stdout:1/172: read dc/f18 [585961,8960] 0 2024-07-16T09:55:32.907 INFO:tasks.workunit.client.1.smithi176.stdout:6/244: dwrite d1/dc/f33 [0,4194304] 0 2024-07-16T09:55:32.909 INFO:tasks.workunit.client.0.smithi156.stdout:2/329: rmdir d0/d5/d29/d2d 39 2024-07-16T09:55:32.913 INFO:tasks.workunit.client.1.smithi176.stdout:9/206: truncate d5/f10 1321493 0 2024-07-16T09:55:32.914 INFO:tasks.workunit.client.0.smithi156.stdout:1/173: unlink dc/d14/f17 0 2024-07-16T09:55:32.916 INFO:tasks.workunit.client.0.smithi156.stdout:9/231: link d2/f15 d2/f4d 0 2024-07-16T09:55:32.917 INFO:tasks.workunit.client.0.smithi156.stdout:6/241: rename d3/d26 to d3/d13/d17/d4f 0 2024-07-16T09:55:32.917 INFO:tasks.workunit.client.1.smithi176.stdout:8/381: dwrite d8/d23/d2c/d34/f84 [0,4194304] 0 2024-07-16T09:55:32.920 INFO:tasks.workunit.client.1.smithi176.stdout:9/207: symlink d5/d1d/l3b 0 2024-07-16T09:55:32.921 INFO:tasks.workunit.client.1.smithi176.stdout:6/245: read d1/d5/d7/f19 [1735650,57423] 0 2024-07-16T09:55:32.923 INFO:tasks.workunit.client.0.smithi156.stdout:2/330: dwrite d0/d5/d29/f3c [0,4194304] 0 2024-07-16T09:55:32.925 INFO:tasks.workunit.client.0.smithi156.stdout:9/232: write d2/d5/f8 [2046234,46395] 0 2024-07-16T09:55:32.925 INFO:tasks.workunit.client.0.smithi156.stdout:1/174: dwrite f6 [0,4194304] 0 2024-07-16T09:55:32.925 INFO:tasks.workunit.client.1.smithi176.stdout:7/284: dwrite d4/d5/d21/f5a [0,4194304] 0 2024-07-16T09:55:32.926 INFO:tasks.workunit.client.0.smithi156.stdout:6/242: symlink d3/d13/d17/d4f/d1e/l50 0 2024-07-16T09:55:32.927 INFO:tasks.workunit.client.0.smithi156.stdout:2/331: chown d0/d5/d29/f32 54 1 2024-07-16T09:55:32.927 INFO:tasks.workunit.client.0.smithi156.stdout:6/243: chown d3/d13/d17/d4f/d1e 97547 1 2024-07-16T09:55:32.929 INFO:tasks.workunit.client.1.smithi176.stdout:9/208: write d5/de/d21/f39 [9063363,21563] 0 2024-07-16T09:55:32.929 INFO:tasks.workunit.client.1.smithi176.stdout:0/274: dread d5/d1c/d24/d35/f4c [0,4194304] 0 2024-07-16T09:55:32.930 INFO:tasks.workunit.client.0.smithi156.stdout:6/244: chown d3/d13/d17/d4f/d1e/l50 3150 1 2024-07-16T09:55:32.931 INFO:tasks.workunit.client.0.smithi156.stdout:2/332: truncate d0/d13/d17/d1c/f24 404392 0 2024-07-16T09:55:32.934 INFO:tasks.workunit.client.0.smithi156.stdout:1/175: symlink dc/d14/d1e/l3b 0 2024-07-16T09:55:32.936 INFO:tasks.workunit.client.0.smithi156.stdout:1/176: dread - dc/d14/f34 zero size 2024-07-16T09:55:32.938 INFO:tasks.workunit.client.1.smithi176.stdout:0/275: mknod d5/d1c/d24/d35/c57 0 2024-07-16T09:55:32.938 INFO:tasks.workunit.client.1.smithi176.stdout:6/246: dwrite d1/d5/d7/f9 [0,4194304] 0 2024-07-16T09:55:32.940 INFO:tasks.workunit.client.1.smithi176.stdout:0/276: read d5/d1c/d24/d26/f4f [173164,86121] 0 2024-07-16T09:55:32.941 INFO:tasks.workunit.client.0.smithi156.stdout:9/233: dwrite d2/d5/d16/d22/d36/d33/f2c [0,4194304] 0 2024-07-16T09:55:32.941 INFO:tasks.workunit.client.1.smithi176.stdout:0/277: write f4 [3623996,101136] 0 2024-07-16T09:55:32.943 INFO:tasks.workunit.client.1.smithi176.stdout:0/278: fsync d5/f2f 0 2024-07-16T09:55:32.947 INFO:tasks.workunit.client.1.smithi176.stdout:8/382: link d8/d14/d1e/d7c/d82/l87 d8/d23/d2c/d34/l8b 0 2024-07-16T09:55:32.950 INFO:tasks.workunit.client.1.smithi176.stdout:9/209: rmdir d5/d3a 0 2024-07-16T09:55:32.951 INFO:tasks.workunit.client.1.smithi176.stdout:0/279: mkdir d5/d1c/d24/d35/d50/d58 0 2024-07-16T09:55:32.953 INFO:tasks.workunit.client.1.smithi176.stdout:6/247: rename d1/d5/d7/d1e/d29/c46 to d1/d5/d7/d2f/c57 0 2024-07-16T09:55:32.954 INFO:tasks.workunit.client.0.smithi156.stdout:1/177: rename dc/fd to dc/d24/f3c 0 2024-07-16T09:55:32.955 INFO:tasks.workunit.client.0.smithi156.stdout:6/245: link d3/d13/d17/d4f/d1f/fa d3/d13/d17/d4f/d30/d45/f51 0 2024-07-16T09:55:32.956 INFO:tasks.workunit.client.0.smithi156.stdout:2/333: mkdir d0/d5/d29/d2d/d5b/d62/d74 0 2024-07-16T09:55:32.956 INFO:tasks.workunit.client.0.smithi156.stdout:6/246: dread - d3/f39 zero size 2024-07-16T09:55:32.957 INFO:tasks.workunit.client.1.smithi176.stdout:9/210: creat d5/d1d/f3c x:0 0 0 2024-07-16T09:55:32.957 INFO:tasks.workunit.client.1.smithi176.stdout:0/280: mkdir d5/d1c/d24/d35/d59 0 2024-07-16T09:55:32.957 INFO:tasks.workunit.client.1.smithi176.stdout:0/281: truncate d5/d2c/f54 336335 0 2024-07-16T09:55:32.958 INFO:tasks.workunit.client.0.smithi156.stdout:9/234: symlink d2/d5/l4e 0 2024-07-16T09:55:32.958 INFO:tasks.workunit.client.0.smithi156.stdout:6/247: write d3/f1c [1530256,106883] 0 2024-07-16T09:55:32.960 INFO:tasks.workunit.client.0.smithi156.stdout:2/334: write d0/d5/d29/d2d/f5a [926662,91046] 0 2024-07-16T09:55:32.960 INFO:tasks.workunit.client.0.smithi156.stdout:9/235: chown d2/d5/d16/d22/d36/d33/c46 1214 1 2024-07-16T09:55:32.961 INFO:tasks.workunit.client.1.smithi176.stdout:9/211: dwrite d5/d1d/f27 [0,4194304] 0 2024-07-16T09:55:32.964 INFO:tasks.workunit.client.0.smithi156.stdout:2/335: unlink d0/d5/f2f 0 2024-07-16T09:55:32.966 INFO:tasks.workunit.client.0.smithi156.stdout:2/336: chown d0/l60 19 1 2024-07-16T09:55:32.969 INFO:tasks.workunit.client.1.smithi176.stdout:9/212: getdents d5/de/d21 0 2024-07-16T09:55:32.973 INFO:tasks.workunit.client.0.smithi156.stdout:6/248: dwrite d3/d13/d17/d4f/f34 [0,4194304] 0 2024-07-16T09:55:32.974 INFO:tasks.workunit.client.1.smithi176.stdout:9/213: dread d5/de/f15 [0,4194304] 0 2024-07-16T09:55:32.979 INFO:tasks.workunit.client.0.smithi156.stdout:2/337: mknod d0/d13/d17/d1c/d51/c75 0 2024-07-16T09:55:32.979 INFO:tasks.workunit.client.0.smithi156.stdout:2/338: read - d0/d13/d17/d1c/f47 zero size 2024-07-16T09:55:32.980 INFO:tasks.workunit.client.1.smithi176.stdout:9/214: write d5/fb [1148133,89987] 0 2024-07-16T09:55:32.982 INFO:tasks.workunit.client.0.smithi156.stdout:9/236: link d2/d5/l41 d2/d5/d16/d22/d36/d33/l4f 0 2024-07-16T09:55:32.984 INFO:tasks.workunit.client.0.smithi156.stdout:6/249: dwrite d3/d13/d17/d4f/f4b [0,4194304] 0 2024-07-16T09:55:32.984 INFO:tasks.workunit.client.0.smithi156.stdout:2/339: mkdir d0/d5/d76 0 2024-07-16T09:55:32.985 INFO:tasks.workunit.client.1.smithi176.stdout:8/383: dwrite d8/d14/d1e/f60 [4194304,4194304] 0 2024-07-16T09:55:32.992 INFO:tasks.workunit.client.0.smithi156.stdout:6/250: symlink d3/d13/d17/d4f/d1f/d19/d2e/l52 0 2024-07-16T09:55:32.994 INFO:tasks.workunit.client.0.smithi156.stdout:8/143: sync 2024-07-16T09:55:32.994 INFO:tasks.workunit.client.0.smithi156.stdout:4/183: sync 2024-07-16T09:55:32.994 INFO:tasks.workunit.client.0.smithi156.stdout:5/159: sync 2024-07-16T09:55:32.996 INFO:tasks.workunit.client.0.smithi156.stdout:4/184: chown d4/l1c 397 1 2024-07-16T09:55:32.996 INFO:tasks.workunit.client.0.smithi156.stdout:9/237: dwrite d2/d5/ff [0,4194304] 0 2024-07-16T09:55:32.998 INFO:tasks.workunit.client.0.smithi156.stdout:8/144: write d0/f2c [627816,111002] 0 2024-07-16T09:55:32.998 INFO:tasks.workunit.client.0.smithi156.stdout:6/251: mknod d3/c53 0 2024-07-16T09:55:33.001 INFO:tasks.workunit.client.1.smithi176.stdout:3/143: sync 2024-07-16T09:55:33.001 INFO:tasks.workunit.client.1.smithi176.stdout:4/240: sync 2024-07-16T09:55:33.001 INFO:tasks.workunit.client.0.smithi156.stdout:6/252: dread - d3/d13/d17/d4f/d1f/d19/d31/f4e zero size 2024-07-16T09:55:33.002 INFO:tasks.workunit.client.0.smithi156.stdout:6/253: write d3/f25 [185157,102719] 0 2024-07-16T09:55:33.002 INFO:tasks.workunit.client.0.smithi156.stdout:1/178: creat dc/d14/d1e/f3d x:0 0 0 2024-07-16T09:55:33.002 INFO:tasks.workunit.client.0.smithi156.stdout:4/185: rmdir d4 39 2024-07-16T09:55:33.004 INFO:tasks.workunit.client.0.smithi156.stdout:5/160: mkdir d1b/d1e/d37 0 2024-07-16T09:55:33.004 INFO:tasks.workunit.client.0.smithi156.stdout:8/145: creat d0/d11/d20/f38 x:0 0 0 2024-07-16T09:55:33.005 INFO:tasks.workunit.client.1.smithi176.stdout:4/241: mknod da/d3d/c43 0 2024-07-16T09:55:33.006 INFO:tasks.workunit.client.0.smithi156.stdout:6/254: truncate d3/d13/d17/d4f/d1f/f10 4673931 0 2024-07-16T09:55:33.006 INFO:tasks.workunit.client.1.smithi176.stdout:3/144: truncate d3/f25 1513960 0 2024-07-16T09:55:33.007 INFO:tasks.workunit.client.0.smithi156.stdout:4/186: dread - d4/d23/f39 zero size 2024-07-16T09:55:33.008 INFO:tasks.workunit.client.0.smithi156.stdout:4/187: write d4/f3d [513680,25092] 0 2024-07-16T09:55:33.008 INFO:tasks.workunit.client.0.smithi156.stdout:5/161: mknod d1b/d1f/c38 0 2024-07-16T09:55:33.011 INFO:tasks.workunit.client.1.smithi176.stdout:4/242: rename da/d3d/d17/f38 to da/d14/f44 0 2024-07-16T09:55:33.012 INFO:tasks.workunit.client.0.smithi156.stdout:5/162: creat d1b/d28/f39 x:0 0 0 2024-07-16T09:55:33.014 INFO:tasks.workunit.client.1.smithi176.stdout:9/215: dread d5/f16 [0,4194304] 0 2024-07-16T09:55:33.014 INFO:tasks.workunit.client.0.smithi156.stdout:4/188: truncate d4/d23/f32 324074 0 2024-07-16T09:55:33.014 INFO:tasks.workunit.client.0.smithi156.stdout:8/146: dwrite d0/dd/f17 [0,4194304] 0 2024-07-16T09:55:33.015 INFO:tasks.workunit.client.1.smithi176.stdout:3/145: truncate d3/da/f13 381780 0 2024-07-16T09:55:33.016 INFO:tasks.workunit.client.1.smithi176.stdout:9/216: write d5/de/f34 [775253,23962] 0 2024-07-16T09:55:33.016 INFO:tasks.workunit.client.0.smithi156.stdout:6/255: mknod d3/d13/d17/d4f/d1f/d19/d31/d35/c54 0 2024-07-16T09:55:33.017 INFO:tasks.workunit.client.0.smithi156.stdout:3/181: sync 2024-07-16T09:55:33.017 INFO:tasks.workunit.client.0.smithi156.stdout:7/177: sync 2024-07-16T09:55:33.017 INFO:tasks.workunit.client.0.smithi156.stdout:0/128: sync 2024-07-16T09:55:33.018 INFO:tasks.workunit.client.1.smithi176.stdout:3/146: chown d3/da/c14 1 1 2024-07-16T09:55:33.018 INFO:tasks.workunit.client.0.smithi156.stdout:8/147: chown d0/d11/d13/d1c/f32 1 1 2024-07-16T09:55:33.018 INFO:tasks.workunit.client.0.smithi156.stdout:2/340: truncate d0/d5/d29/f32 3548619 0 2024-07-16T09:55:33.019 INFO:tasks.workunit.client.0.smithi156.stdout:4/189: creat d4/db/dc/f42 x:0 0 0 2024-07-16T09:55:33.020 INFO:tasks.workunit.client.0.smithi156.stdout:8/148: stat d0/d11/d13/d1c/d24/l2f 0 2024-07-16T09:55:33.020 INFO:tasks.workunit.client.0.smithi156.stdout:3/182: chown d8/da/d1d/d2c/c27 290571 1 2024-07-16T09:55:33.022 INFO:tasks.workunit.client.0.smithi156.stdout:6/256: creat d3/d13/d17/f55 x:0 0 0 2024-07-16T09:55:33.023 INFO:tasks.workunit.client.0.smithi156.stdout:8/149: read d0/d11/d20/f21 [2655415,28958] 0 2024-07-16T09:55:33.024 INFO:tasks.workunit.client.0.smithi156.stdout:3/183: stat d8/da/l11 0 2024-07-16T09:55:33.024 INFO:tasks.workunit.client.0.smithi156.stdout:0/129: symlink d0/d7/l24 0 2024-07-16T09:55:33.028 INFO:tasks.workunit.client.0.smithi156.stdout:3/184: dread - d8/da/f32 zero size 2024-07-16T09:55:33.030 INFO:tasks.workunit.client.0.smithi156.stdout:7/178: mkdir dc/d3e/d3f/d45 0 2024-07-16T09:55:33.033 INFO:tasks.workunit.client.0.smithi156.stdout:9/238: dwrite d2/f4d [0,4194304] 0 2024-07-16T09:55:33.033 INFO:tasks.workunit.client.0.smithi156.stdout:7/179: read dc/d13/f35 [3635377,125347] 0 2024-07-16T09:55:33.033 INFO:tasks.workunit.client.0.smithi156.stdout:6/257: readlink d3/d13/d17/d4f/d1f/l3a 0 2024-07-16T09:55:33.033 INFO:tasks.workunit.client.0.smithi156.stdout:5/163: link c8 d1b/c3a 0 2024-07-16T09:55:33.035 INFO:tasks.workunit.client.0.smithi156.stdout:1/179: unlink dc/d14/d1e/c38 0 2024-07-16T09:55:33.035 INFO:tasks.workunit.client.0.smithi156.stdout:0/130: rename d0/d7/d9/d1b to d0/d7/d25 0 2024-07-16T09:55:33.035 INFO:tasks.workunit.client.0.smithi156.stdout:2/341: dread d0/d13/d17/d1c/f48 [0,4194304] 0 2024-07-16T09:55:33.039 INFO:tasks.workunit.client.0.smithi156.stdout:4/190: rmdir d4/db/dc 39 2024-07-16T09:55:33.039 INFO:tasks.workunit.client.0.smithi156.stdout:0/131: truncate d0/f14 1054907 0 2024-07-16T09:55:33.040 INFO:tasks.workunit.client.0.smithi156.stdout:2/342: fsync d0/d13/d17/d1c/f37 0 2024-07-16T09:55:33.042 INFO:tasks.workunit.client.0.smithi156.stdout:8/150: truncate d0/d11/d20/f21 1613747 0 2024-07-16T09:55:33.042 INFO:tasks.workunit.client.0.smithi156.stdout:3/185: dwrite d8/da/d13/f20 [0,4194304] 0 2024-07-16T09:55:33.043 INFO:tasks.workunit.client.0.smithi156.stdout:4/191: truncate d4/d23/f39 549159 0 2024-07-16T09:55:33.044 INFO:tasks.workunit.client.0.smithi156.stdout:1/180: write dc/d24/f3c [601906,53244] 0 2024-07-16T09:55:33.044 INFO:tasks.workunit.client.0.smithi156.stdout:4/192: dread - d4/db/f3f zero size 2024-07-16T09:55:33.044 INFO:tasks.workunit.client.0.smithi156.stdout:6/258: mkdir d3/d13/d17/d56 0 2024-07-16T09:55:33.045 INFO:tasks.workunit.client.0.smithi156.stdout:1/181: readlink dc/l12 0 2024-07-16T09:55:33.046 INFO:tasks.workunit.client.0.smithi156.stdout:7/180: mknod dc/d13/d20/c46 0 2024-07-16T09:55:33.046 INFO:tasks.workunit.client.0.smithi156.stdout:1/182: dread - dc/f1c zero size 2024-07-16T09:55:33.046 INFO:tasks.workunit.client.0.smithi156.stdout:4/193: write d4/db/f18 [1675077,85339] 0 2024-07-16T09:55:33.050 INFO:tasks.workunit.client.0.smithi156.stdout:0/132: creat d0/d7/d9/f26 x:0 0 0 2024-07-16T09:55:33.051 INFO:tasks.workunit.client.0.smithi156.stdout:2/343: readlink d0/d5/d29/l33 0 2024-07-16T09:55:33.054 INFO:tasks.workunit.client.0.smithi156.stdout:4/194: creat d4/f43 x:0 0 0 2024-07-16T09:55:33.056 INFO:tasks.workunit.client.0.smithi156.stdout:7/181: rename dc/d13/d36/c38 to dc/d3e/c47 0 2024-07-16T09:55:33.057 INFO:tasks.workunit.client.0.smithi156.stdout:0/133: rmdir d0/d1a 39 2024-07-16T09:55:33.058 INFO:tasks.workunit.client.0.smithi156.stdout:3/186: creat d8/d34/f3a x:0 0 0 2024-07-16T09:55:33.058 INFO:tasks.workunit.client.0.smithi156.stdout:2/344: creat d0/d5/d29/d2d/d5b/d62/f77 x:0 0 0 2024-07-16T09:55:33.060 INFO:tasks.workunit.client.0.smithi156.stdout:7/182: mkdir dc/d48 0 2024-07-16T09:55:33.061 INFO:tasks.workunit.client.0.smithi156.stdout:0/134: unlink d0/d7/f20 0 2024-07-16T09:55:33.061 INFO:tasks.workunit.client.0.smithi156.stdout:0/135: chown d0/d7/c16 162583957 1 2024-07-16T09:55:33.062 INFO:tasks.workunit.client.0.smithi156.stdout:3/187: chown d8/d14/l35 848206 1 2024-07-16T09:55:33.062 INFO:tasks.workunit.client.0.smithi156.stdout:0/136: stat d0/d7/c16 0 2024-07-16T09:55:33.063 INFO:tasks.workunit.client.0.smithi156.stdout:8/151: fsync d0/dd/f17 0 2024-07-16T09:55:33.064 INFO:tasks.workunit.client.0.smithi156.stdout:4/195: truncate d4/d23/f32 1263058 0 2024-07-16T09:55:33.067 INFO:tasks.workunit.client.0.smithi156.stdout:2/345: getdents d0/d5/d76 0 2024-07-16T09:55:33.068 INFO:tasks.workunit.client.0.smithi156.stdout:3/188: mknod d8/da/d1d/d2c/c3b 0 2024-07-16T09:55:33.069 INFO:tasks.workunit.client.0.smithi156.stdout:8/152: rmdir d0/d11/d13 39 2024-07-16T09:55:33.069 INFO:tasks.workunit.client.0.smithi156.stdout:0/137: dwrite d0/d7/d9/f15 [4194304,4194304] 0 2024-07-16T09:55:33.071 INFO:tasks.workunit.client.0.smithi156.stdout:2/346: symlink d0/d5/d29/d2d/l78 0 2024-07-16T09:55:33.072 INFO:tasks.workunit.client.0.smithi156.stdout:5/164: dread f19 [0,4194304] 0 2024-07-16T09:55:33.072 INFO:tasks.workunit.client.0.smithi156.stdout:9/239: truncate d2/f4 1211255 0 2024-07-16T09:55:33.074 INFO:tasks.workunit.client.0.smithi156.stdout:4/196: mkdir d4/db/d44 0 2024-07-16T09:55:33.074 INFO:tasks.workunit.client.0.smithi156.stdout:5/165: readlink d1b/l26 0 2024-07-16T09:55:33.075 INFO:tasks.workunit.client.0.smithi156.stdout:7/183: symlink dc/d3e/d3f/d45/l49 0 2024-07-16T09:55:33.076 INFO:tasks.workunit.client.0.smithi156.stdout:3/189: rename d8/da/d1d/c38 to d8/da/d13/c3c 0 2024-07-16T09:55:33.078 INFO:tasks.workunit.client.0.smithi156.stdout:2/347: creat d0/d5/d29/d2d/f79 x:0 0 0 2024-07-16T09:55:33.078 INFO:tasks.workunit.client.0.smithi156.stdout:1/183: truncate dc/d14/d1e/f21 3317007 0 2024-07-16T09:55:33.079 INFO:tasks.workunit.client.0.smithi156.stdout:9/240: write d2/d5/d16/f2a [146692,100718] 0 2024-07-16T09:55:33.090 INFO:tasks.workunit.client.0.smithi156.stdout:5/166: unlink f1a 0 2024-07-16T09:55:33.092 INFO:tasks.workunit.client.0.smithi156.stdout:0/138: dwrite d0/ff [0,4194304] 0 2024-07-16T09:55:33.092 INFO:tasks.workunit.client.0.smithi156.stdout:4/197: dread d4/d23/d30/f2d [0,4194304] 0 2024-07-16T09:55:33.092 INFO:tasks.workunit.client.0.smithi156.stdout:7/184: creat dc/d3e/f4a x:0 0 0 2024-07-16T09:55:33.092 INFO:tasks.workunit.client.0.smithi156.stdout:8/153: mkdir d0/d11/d13/d1c/d39 0 2024-07-16T09:55:33.095 INFO:tasks.workunit.client.0.smithi156.stdout:4/198: dread - d4/db/f3f zero size 2024-07-16T09:55:33.095 INFO:tasks.workunit.client.0.smithi156.stdout:0/139: chown d0/f3 64944619 1 2024-07-16T09:55:33.098 INFO:tasks.workunit.client.0.smithi156.stdout:2/348: dwrite d0/d13/d17/f30 [0,4194304] 0 2024-07-16T09:55:33.126 INFO:tasks.workunit.client.1.smithi176.stdout:8/384: dread d8/d20/d2a/d31/f3e [0,4194304] 0 2024-07-16T09:55:33.128 INFO:tasks.workunit.client.1.smithi176.stdout:9/217: rename d5/d1d/d20/l2b to d5/de/d38/l3d 0 2024-07-16T09:55:33.129 INFO:tasks.workunit.client.1.smithi176.stdout:9/218: write d5/f35 [64761,117204] 0 2024-07-16T09:55:33.130 INFO:tasks.workunit.client.1.smithi176.stdout:4/243: unlink da/d14/f44 0 2024-07-16T09:55:33.130 INFO:tasks.workunit.client.1.smithi176.stdout:4/244: chown da/c20 2887 1 2024-07-16T09:55:33.131 INFO:tasks.workunit.client.1.smithi176.stdout:8/385: mknod d8/d20/d2a/d31/d2f/c8c 0 2024-07-16T09:55:33.132 INFO:tasks.workunit.client.1.smithi176.stdout:9/219: rmdir d5/de/d21 39 2024-07-16T09:55:33.133 INFO:tasks.workunit.client.1.smithi176.stdout:8/386: dread - d8/d20/d2a/d31/d46/d6a/f7a zero size 2024-07-16T09:55:33.135 INFO:tasks.workunit.client.1.smithi176.stdout:4/245: rename da/f3f to da/dc/f45 0 2024-07-16T09:55:33.135 INFO:tasks.workunit.client.1.smithi176.stdout:9/220: symlink d5/l3e 0 2024-07-16T09:55:33.138 INFO:tasks.workunit.client.1.smithi176.stdout:4/246: write da/dc/d39/f3c [495829,87663] 0 2024-07-16T09:55:33.139 INFO:tasks.workunit.client.1.smithi176.stdout:9/221: link d5/de/f1a d5/d1d/f3f 0 2024-07-16T09:55:33.140 INFO:tasks.workunit.client.1.smithi176.stdout:3/147: truncate d3/da/f13 1286338 0 2024-07-16T09:55:33.141 INFO:tasks.workunit.client.1.smithi176.stdout:4/247: chown da/d14/c2d 1250851102 1 2024-07-16T09:55:33.142 INFO:tasks.workunit.client.1.smithi176.stdout:4/248: stat da/d14/c2d 0 2024-07-16T09:55:33.143 INFO:tasks.workunit.client.1.smithi176.stdout:3/148: creat d3/d17/f26 x:0 0 0 2024-07-16T09:55:33.144 INFO:tasks.workunit.client.1.smithi176.stdout:9/222: creat d5/de/f40 x:0 0 0 2024-07-16T09:55:33.145 INFO:tasks.workunit.client.1.smithi176.stdout:8/387: dwrite d8/d23/f1b [0,4194304] 0 2024-07-16T09:55:33.146 INFO:tasks.workunit.client.1.smithi176.stdout:4/249: read da/dc/d10/f27 [2532673,109272] 0 2024-07-16T09:55:33.146 INFO:tasks.workunit.client.1.smithi176.stdout:8/388: chown d8/d23/f25 131 1 2024-07-16T09:55:33.149 INFO:tasks.workunit.client.1.smithi176.stdout:3/149: creat d3/d17/d1d/d1e/f27 x:0 0 0 2024-07-16T09:55:33.152 INFO:tasks.workunit.client.1.smithi176.stdout:8/389: creat d8/d20/d2a/d31/d46/d77/f8d x:0 0 0 2024-07-16T09:55:33.152 INFO:tasks.workunit.client.1.smithi176.stdout:8/390: readlink d8/d20/d2a/l79 0 2024-07-16T09:55:33.153 INFO:tasks.workunit.client.1.smithi176.stdout:4/250: symlink da/dc/l46 0 2024-07-16T09:55:33.154 INFO:tasks.workunit.client.1.smithi176.stdout:4/251: fsync da/d14/f29 0 2024-07-16T09:55:33.155 INFO:tasks.workunit.client.1.smithi176.stdout:8/391: write d8/d23/f1b [3998224,51334] 0 2024-07-16T09:55:33.155 INFO:tasks.workunit.client.1.smithi176.stdout:3/150: symlink d3/l28 0 2024-07-16T09:55:33.157 INFO:tasks.workunit.client.1.smithi176.stdout:4/252: write da/d3d/f1b [3583025,68217] 0 2024-07-16T09:55:33.161 INFO:tasks.workunit.client.1.smithi176.stdout:8/392: dread d8/d20/d2a/d31/f69 [0,4194304] 0 2024-07-16T09:55:33.161 INFO:tasks.workunit.client.1.smithi176.stdout:9/223: dread d5/de/f12 [0,4194304] 0 2024-07-16T09:55:33.162 INFO:tasks.workunit.client.1.smithi176.stdout:8/393: write d8/d14/d1e/d33/f5b [717580,111658] 0 2024-07-16T09:55:33.163 INFO:tasks.workunit.client.1.smithi176.stdout:4/253: truncate da/d3d/d17/f18 357521 0 2024-07-16T09:55:33.165 INFO:tasks.workunit.client.0.smithi156.stdout:1/184: mkdir dc/d14/d3e 0 2024-07-16T09:55:33.166 INFO:tasks.workunit.client.1.smithi176.stdout:8/394: write d8/d20/f2b [2165789,53359] 0 2024-07-16T09:55:33.167 INFO:tasks.workunit.client.1.smithi176.stdout:8/395: chown d8/l1c 610 1 2024-07-16T09:55:33.169 INFO:tasks.workunit.client.0.smithi156.stdout:9/241: rename d2/f13 to d2/d5/d16/d22/d36/f50 0 2024-07-16T09:55:33.169 INFO:tasks.workunit.client.0.smithi156.stdout:8/154: mkdir d0/d11/d20/d3a 0 2024-07-16T09:55:33.170 INFO:tasks.workunit.client.1.smithi176.stdout:3/151: getdents d3 0 2024-07-16T09:55:33.171 INFO:tasks.workunit.client.1.smithi176.stdout:4/254: creat da/f47 x:0 0 0 2024-07-16T09:55:33.171 INFO:tasks.workunit.client.1.smithi176.stdout:3/152: fsync d3/d17/f23 0 2024-07-16T09:55:33.172 INFO:tasks.workunit.client.0.smithi156.stdout:0/140: mkdir d0/d7/d9/d27 0 2024-07-16T09:55:33.173 INFO:tasks.workunit.client.1.smithi176.stdout:9/224: rename d5/c24 to d5/c41 0 2024-07-16T09:55:33.174 INFO:tasks.workunit.client.0.smithi156.stdout:0/141: write d0/d7/f21 [483949,18807] 0 2024-07-16T09:55:33.175 INFO:tasks.workunit.client.1.smithi176.stdout:3/153: creat d3/da/f29 x:0 0 0 2024-07-16T09:55:33.175 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:33.174+0000 7f8904a99640 1 -- 172.21.15.156:0/135795873 --> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f8900105020 con 0x7f88e806a5a0 2024-07-16T09:55:33.176 INFO:tasks.workunit.client.1.smithi176.stdout:8/396: truncate d8/f3d 2323403 0 2024-07-16T09:55:33.177 INFO:tasks.workunit.client.0.smithi156.stdout:9/242: symlink d2/de/l51 0 2024-07-16T09:55:33.177 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:33.176+0000 7f88eeffd640 1 -- 172.21.15.156:0/135795873 <== mgr.14208 v2:172.21.15.156:6800/2615741717 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+349 (secure 0 0 0) 0x7f8900105020 con 0x7f88e806a5a0 2024-07-16T09:55:33.178 INFO:tasks.workunit.client.0.smithi156.stdout:8/155: creat d0/dd/f3b x:0 0 0 2024-07-16T09:55:33.179 INFO:tasks.workunit.client.1.smithi176.stdout:3/154: mknod d3/d17/d1d/d1e/c2a 0 2024-07-16T09:55:33.179 INFO:tasks.workunit.client.1.smithi176.stdout:8/397: chown d8/d20/d2a/d61/f6e 3961470 1 2024-07-16T09:55:33.179 INFO:tasks.workunit.client.1.smithi176.stdout:4/255: dwrite da/d14/f26 [4194304,4194304] 0 2024-07-16T09:55:33.180 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:33.178+0000 7f8904a99640 1 -- 172.21.15.156:0/135795873 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f88e806a5a0 msgr2=0x7f88e806ca50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:55:33.180 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:33.178+0000 7f8904a99640 1 --2- 172.21.15.156:0/135795873 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f88e806a5a0 0x7f88e806ca50 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f88f4007ac0 tx=0x7f88f4035040 comp rx=0 tx=0).stop 2024-07-16T09:55:33.180 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:33.179+0000 7f8904a99640 1 -- 172.21.15.156:0/135795873 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8900064500 msgr2=0x7f8900106680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:55:33.180 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:33.179+0000 7f8904a99640 1 --2- 172.21.15.156:0/135795873 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8900064500 0x7f8900106680 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f88f000c7e0 tx=0x7f88f0007f20 comp rx=0 tx=0).stop 2024-07-16T09:55:33.180 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:33.179+0000 7f8904a99640 1 -- 172.21.15.156:0/135795873 shutdown_connections 2024-07-16T09:55:33.180 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:33.179+0000 7f8904a99640 1 --2- 172.21.15.156:0/135795873 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f88e806a5a0 0x7f88e806ca50 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:55:33.180 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:33.179+0000 7f8904a99640 1 --2- 172.21.15.156:0/135795873 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8900064910 0x7f8900106bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:55:33.181 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:33.179+0000 7f8904a99640 1 --2- 172.21.15.156:0/135795873 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8900064500 0x7f8900106680 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:55:33.181 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:33.179+0000 7f8904a99640 1 -- 172.21.15.156:0/135795873 >> 172.21.15.156:0/135795873 conn(0x7f89000f61a0 msgr2=0x7f89000f77d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:55:33.181 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:33.179+0000 7f8904a99640 1 -- 172.21.15.156:0/135795873 shutdown_connections 2024-07-16T09:55:33.181 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:33.179+0000 7f8904a99640 1 -- 172.21.15.156:0/135795873 wait complete. 2024-07-16T09:55:33.181 INFO:tasks.workunit.client.1.smithi176.stdout:4/256: fdatasync da/d3d/f1b 0 2024-07-16T09:55:33.181 INFO:tasks.workunit.client.1.smithi176.stdout:9/225: rename d5/l14 to d5/de/l42 0 2024-07-16T09:55:33.182 INFO:tasks.workunit.client.0.smithi156.stdout:5/167: rename d1b/c1d to d1b/d1f/c3b 0 2024-07-16T09:55:33.182 INFO:tasks.workunit.client.1.smithi176.stdout:4/257: write da/d14/f26 [6623245,29942] 0 2024-07-16T09:55:33.183 INFO:tasks.workunit.client.0.smithi156.stdout:5/168: stat d1b/c1c 0 2024-07-16T09:55:33.183 INFO:tasks.workunit.client.0.smithi156.stdout:9/243: symlink d2/de/l52 0 2024-07-16T09:55:33.183 INFO:tasks.workunit.client.0.smithi156.stdout:8/156: dread d0/f12 [0,4194304] 0 2024-07-16T09:55:33.184 INFO:tasks.workunit.client.0.smithi156.stdout:1/185: creat dc/f3f x:0 0 0 2024-07-16T09:55:33.184 INFO:tasks.workunit.client.0.smithi156.stdout:8/157: dread - d0/dd/f30 zero size 2024-07-16T09:55:33.186 INFO:tasks.workunit.client.1.smithi176.stdout:4/258: truncate da/dc/d10/f27 2295029 0 2024-07-16T09:55:33.187 INFO:tasks.workunit.client.0.smithi156.stdout:0/142: truncate d0/f3 94970 0 2024-07-16T09:55:33.187 INFO:tasks.workunit.client.1.smithi176.stdout:3/155: dread d3/da/f13 [0,4194304] 0 2024-07-16T09:55:33.187 INFO:tasks.workunit.client.1.smithi176.stdout:9/226: mkdir d5/d43 0 2024-07-16T09:55:33.189 INFO:tasks.workunit.client.1.smithi176.stdout:9/227: chown d5/d1d/f27 0 1 2024-07-16T09:55:33.190 INFO:tasks.workunit.client.1.smithi176.stdout:8/398: dread d8/f17 [0,4194304] 0 2024-07-16T09:55:33.190 INFO:tasks.workunit.client.0.smithi156.stdout:9/244: dread d2/d5/d16/d22/d36/f31 [0,4194304] 0 2024-07-16T09:55:33.190 INFO:tasks.workunit.client.0.smithi156.stdout:0/143: fdatasync d0/d7/d9/f26 0 2024-07-16T09:55:33.192 INFO:tasks.workunit.client.0.smithi156.stdout:1/186: symlink dc/d24/l40 0 2024-07-16T09:55:33.192 INFO:tasks.workunit.client.1.smithi176.stdout:3/156: truncate d3/d17/d1d/d1e/f27 647877 0 2024-07-16T09:55:33.193 INFO:tasks.workunit.client.0.smithi156.stdout:2/349: link d0/d13/d17/d1c/l1d d0/d5/d29/d2d/l7a 0 2024-07-16T09:55:33.193 INFO:tasks.workunit.client.0.smithi156.stdout:1/187: write dc/f13 [4515442,108805] 0 2024-07-16T09:55:33.194 INFO:tasks.workunit.client.1.smithi176.stdout:9/228: mknod d5/c44 0 2024-07-16T09:55:33.195 INFO:tasks.workunit.client.1.smithi176.stdout:8/399: symlink d8/d20/d2a/l8e 0 2024-07-16T09:55:33.196 INFO:teuthology.orchestra.run.smithi156.stdout:true 2024-07-16T09:55:33.197 INFO:tasks.workunit.client.0.smithi156.stdout:8/158: creat d0/dd/f3c x:0 0 0 2024-07-16T09:55:33.198 INFO:tasks.workunit.client.0.smithi156.stdout:4/199: rename d4/d23/f32 to d4/d23/d30/d33/f45 0 2024-07-16T09:55:33.198 INFO:tasks.workunit.client.1.smithi176.stdout:3/157: rename d3/da/f19 to d3/f2b 0 2024-07-16T09:55:33.199 INFO:tasks.workunit.client.0.smithi156.stdout:5/169: link d1b/d1f/f20 d1b/d28/f3c 0 2024-07-16T09:55:33.199 INFO:tasks.workunit.client.1.smithi176.stdout:8/400: fsync d8/d20/f5a 0 2024-07-16T09:55:33.200 INFO:tasks.workunit.client.1.smithi176.stdout:4/259: dwrite da/dc/fe [0,4194304] 0 2024-07-16T09:55:33.201 INFO:tasks.workunit.client.0.smithi156.stdout:0/144: unlink d0/d7/d25/l23 0 2024-07-16T09:55:33.202 INFO:tasks.workunit.client.0.smithi156.stdout:0/145: chown d0/d7/d25/l1f 852469 1 2024-07-16T09:55:33.203 INFO:tasks.workunit.client.0.smithi156.stdout:9/245: creat d2/d5/d16/d22/d2b/d3d/d3c/f53 x:0 0 0 2024-07-16T09:55:33.203 INFO:tasks.workunit.client.0.smithi156.stdout:0/146: dread - d0/d7/d9/f26 zero size 2024-07-16T09:55:33.204 INFO:tasks.workunit.client.0.smithi156.stdout:9/246: fsync d2/f15 0 2024-07-16T09:55:33.204 INFO:tasks.workunit.client.1.smithi176.stdout:9/229: getdents d5/de/d21 0 2024-07-16T09:55:33.205 INFO:tasks.workunit.client.1.smithi176.stdout:8/401: creat d8/d14/d1e/d7c/f8f x:0 0 0 2024-07-16T09:55:33.205 INFO:tasks.workunit.client.1.smithi176.stdout:8/402: stat d8/d20/d2a/d31/c42 0 2024-07-16T09:55:33.207 INFO:tasks.workunit.client.0.smithi156.stdout:1/188: unlink f5 0 2024-07-16T09:55:33.207 INFO:tasks.workunit.client.1.smithi176.stdout:3/158: mknod d3/d17/d1d/c2c 0 2024-07-16T09:55:33.208 INFO:tasks.workunit.client.1.smithi176.stdout:4/260: mkdir da/d3d/d17/d48 0 2024-07-16T09:55:33.210 INFO:tasks.workunit.client.0.smithi156.stdout:4/200: creat d4/d23/d30/f46 x:0 0 0 2024-07-16T09:55:33.212 INFO:tasks.workunit.client.0.smithi156.stdout:5/170: creat d1b/d1e/f3d x:0 0 0 2024-07-16T09:55:33.214 INFO:tasks.workunit.client.0.smithi156.stdout:8/159: dread d0/dd/f10 [0,4194304] 0 2024-07-16T09:55:33.215 INFO:tasks.workunit.client.1.smithi176.stdout:9/230: dwrite d5/de/f40 [0,4194304] 0 2024-07-16T09:55:33.215 INFO:tasks.workunit.client.1.smithi176.stdout:9/231: read - d5/de/f2f zero size 2024-07-16T09:55:33.216 INFO:tasks.workunit.client.0.smithi156.stdout:2/350: creat d0/d5/d76/f7b x:0 0 0 2024-07-16T09:55:33.217 INFO:tasks.workunit.client.1.smithi176.stdout:4/261: rename da/dc/d10/d28/l2c to da/d3d/d17/d48/l49 0 2024-07-16T09:55:33.217 INFO:tasks.workunit.client.1.smithi176.stdout:8/403: dread d8/d14/d1e/f60 [0,4194304] 0 2024-07-16T09:55:33.218 INFO:tasks.workunit.client.0.smithi156.stdout:1/189: readlink dc/d14/d1e/l3b 0 2024-07-16T09:55:33.219 INFO:tasks.workunit.client.0.smithi156.stdout:2/351: chown d0/d5/d29/d2d/f73 1 1 2024-07-16T09:55:33.220 INFO:tasks.workunit.client.1.smithi176.stdout:9/232: creat d5/de/d38/f45 x:0 0 0 2024-07-16T09:55:33.220 INFO:tasks.workunit.client.0.smithi156.stdout:8/160: symlink d0/dd/d1f/l3d 0 2024-07-16T09:55:33.223 INFO:tasks.workunit.client.1.smithi176.stdout:3/159: truncate d3/d17/d1d/d1e/f1f 1451534 0 2024-07-16T09:55:33.224 INFO:tasks.workunit.client.0.smithi156.stdout:2/352: chown d0/d13/f1a 95 1 2024-07-16T09:55:33.224 INFO:tasks.workunit.client.0.smithi156.stdout:0/147: dwrite d0/d7/f10 [0,4194304] 0 2024-07-16T09:55:33.227 INFO:tasks.workunit.client.0.smithi156.stdout:7/185: sync 2024-07-16T09:55:33.227 INFO:tasks.workunit.client.0.smithi156.stdout:6/259: sync 2024-07-16T09:55:33.227 INFO:tasks.workunit.client.1.smithi176.stdout:9/233: dread d5/de/f34 [0,4194304] 0 2024-07-16T09:55:33.227 INFO:tasks.workunit.client.1.smithi176.stdout:3/160: creat d3/d17/d1d/f2d x:0 0 0 2024-07-16T09:55:33.228 INFO:tasks.workunit.client.1.smithi176.stdout:9/234: chown d5/l3e 232896037 1 2024-07-16T09:55:33.231 INFO:tasks.workunit.client.0.smithi156.stdout:8/161: mknod d0/d11/d2e/d35/c3e 0 2024-07-16T09:55:33.235 INFO:tasks.workunit.client.0.smithi156.stdout:6/260: readlink d3/d13/d17/d4f/d1e/l50 0 2024-07-16T09:55:33.235 INFO:tasks.workunit.client.0.smithi156.stdout:2/353: symlink d0/l7c 0 2024-07-16T09:55:33.236 INFO:tasks.workunit.client.1.smithi176.stdout:9/235: readlink d5/l7 0 2024-07-16T09:55:33.236 INFO:tasks.workunit.client.1.smithi176.stdout:8/404: getdents d8/d14/d1e/d7c/d82 0 2024-07-16T09:55:33.236 INFO:tasks.workunit.client.1.smithi176.stdout:8/405: fsync d8/d23/f25 0 2024-07-16T09:55:33.237 INFO:tasks.workunit.client.1.smithi176.stdout:4/262: truncate da/dc/d10/f27 1535278 0 2024-07-16T09:55:33.238 INFO:tasks.workunit.client.0.smithi156.stdout:4/201: dwrite d4/db/f1d [0,4194304] 0 2024-07-16T09:55:33.238 INFO:tasks.workunit.client.1.smithi176.stdout:9/236: creat d5/d1d/f46 x:0 0 0 2024-07-16T09:55:33.240 INFO:tasks.workunit.client.0.smithi156.stdout:7/186: write dc/d13/d20/f3c [592848,18039] 0 2024-07-16T09:55:33.241 INFO:tasks.workunit.client.1.smithi176.stdout:8/406: fsync d8/d23/d2c/d37/f4f 0 2024-07-16T09:55:33.242 INFO:tasks.workunit.client.0.smithi156.stdout:0/148: rename d0/d7/c18 to d0/d7/d9/d27/c28 0 2024-07-16T09:55:33.242 INFO:tasks.workunit.client.1.smithi176.stdout:4/263: read da/d3d/d17/f18 [173071,27566] 0 2024-07-16T09:55:33.243 INFO:tasks.workunit.client.0.smithi156.stdout:1/190: dwrite dc/d14/d1e/f35 [0,4194304] 0 2024-07-16T09:55:33.244 INFO:tasks.workunit.client.0.smithi156.stdout:6/261: write d3/d13/d17/d4f/d1f/f37 [1815030,18086] 0 2024-07-16T09:55:33.244 INFO:tasks.workunit.client.1.smithi176.stdout:8/407: dread - d8/d23/d2c/d34/d62/f7d zero size 2024-07-16T09:55:33.244 INFO:tasks.workunit.client.0.smithi156.stdout:8/162: mkdir d0/d11/d20/d3f 0 2024-07-16T09:55:33.245 INFO:tasks.workunit.client.1.smithi176.stdout:8/408: write d8/d20/f2b [339944,100218] 0 2024-07-16T09:55:33.249 INFO:tasks.workunit.client.0.smithi156.stdout:7/187: write dc/d13/d20/f3c [533247,73031] 0 2024-07-16T09:55:33.250 INFO:tasks.workunit.client.0.smithi156.stdout:5/171: dread d1b/d1f/f20 [0,4194304] 0 2024-07-16T09:55:33.251 INFO:tasks.workunit.client.1.smithi176.stdout:8/409: mkdir d8/d20/d2a/d61/d90 0 2024-07-16T09:55:33.256 INFO:tasks.workunit.client.0.smithi156.stdout:2/354: rename d0/d5/d29/d2d/d5b/f6c to d0/d5/d29/d2d/d5b/d5f/f7d 0 2024-07-16T09:55:33.256 INFO:tasks.workunit.client.0.smithi156.stdout:5/172: chown d1b/l27 305956 1 2024-07-16T09:55:33.257 INFO:tasks.workunit.client.1.smithi176.stdout:8/410: mknod d8/d23/d2c/d37/c91 0 2024-07-16T09:55:33.261 INFO:tasks.workunit.client.1.smithi176.stdout:8/411: symlink d8/d20/d2a/d31/d46/d77/l92 0 2024-07-16T09:55:33.263 INFO:tasks.workunit.client.0.smithi156.stdout:7/188: fsync dc/d13/f35 0 2024-07-16T09:55:33.264 INFO:tasks.workunit.client.1.smithi176.stdout:8/412: write d8/d23/d2c/f36 [438089,2296] 0 2024-07-16T09:55:33.267 INFO:tasks.workunit.client.1.smithi176.stdout:8/413: dread - d8/d20/d2a/d31/d46/d77/f8d zero size 2024-07-16T09:55:33.267 INFO:tasks.workunit.client.0.smithi156.stdout:5/173: write d1b/d28/f39 [619353,16950] 0 2024-07-16T09:55:33.268 INFO:tasks.workunit.client.1.smithi176.stdout:8/414: write f5 [172331,46317] 0 2024-07-16T09:55:33.270 INFO:tasks.workunit.client.0.smithi156.stdout:0/149: dwrite d0/d7/f10 [0,4194304] 0 2024-07-16T09:55:33.272 INFO:tasks.workunit.client.0.smithi156.stdout:9/247: stat d2/f4 0 2024-07-16T09:55:33.273 INFO:tasks.workunit.client.0.smithi156.stdout:1/191: dwrite dc/d14/d1e/f35 [0,4194304] 0 2024-07-16T09:55:33.275 INFO:tasks.workunit.client.0.smithi156.stdout:1/192: chown f4 14 1 2024-07-16T09:55:33.276 INFO:tasks.workunit.client.0.smithi156.stdout:2/355: creat d0/d5/d29/d2d/d5b/d5f/f7e x:0 0 0 2024-07-16T09:55:33.277 INFO:tasks.workunit.client.0.smithi156.stdout:0/150: dread - d0/d7/f19 zero size 2024-07-16T09:55:33.280 INFO:tasks.workunit.client.0.smithi156.stdout:2/356: stat d0/d5/f1e 0 2024-07-16T09:55:33.281 INFO:tasks.workunit.client.0.smithi156.stdout:8/163: dwrite d0/f2 [4194304,4194304] 0 2024-07-16T09:55:33.282 INFO:tasks.workunit.client.0.smithi156.stdout:0/151: read - d0/d7/f19 zero size 2024-07-16T09:55:33.284 INFO:tasks.workunit.client.0.smithi156.stdout:9/248: write d2/d5/d16/d22/d36/d33/f2c [4133209,53564] 0 2024-07-16T09:55:33.285 INFO:tasks.workunit.client.0.smithi156.stdout:3/190: dwrite d8/da/fc [0,4194304] 0 2024-07-16T09:55:33.286 INFO:tasks.workunit.client.0.smithi156.stdout:8/164: read - d0/dd/f33 zero size 2024-07-16T09:55:33.287 INFO:tasks.workunit.client.0.smithi156.stdout:2/357: read d0/d5/f1e [762630,46594] 0 2024-07-16T09:55:33.289 INFO:tasks.workunit.client.0.smithi156.stdout:5/174: rename c11 to d1b/d28/d2d/c3e 0 2024-07-16T09:55:33.291 INFO:tasks.workunit.client.0.smithi156.stdout:1/193: unlink dc/d20/c27 0 2024-07-16T09:55:33.292 INFO:tasks.workunit.client.0.smithi156.stdout:5/175: write d1b/f31 [922263,56127] 0 2024-07-16T09:55:33.295 INFO:tasks.workunit.client.0.smithi156.stdout:4/202: truncate d4/d23/d30/d33/f45 1273341 0 2024-07-16T09:55:33.295 INFO:tasks.workunit.client.0.smithi156.stdout:0/152: stat d0/d7/c13 0 2024-07-16T09:55:33.298 INFO:tasks.workunit.client.0.smithi156.stdout:7/189: creat dc/d48/f4b x:0 0 0 2024-07-16T09:55:33.298 INFO:tasks.workunit.client.0.smithi156.stdout:6/262: truncate d3/d13/d17/d4f/d1f/f2c 2363527 0 2024-07-16T09:55:33.299 INFO:tasks.workunit.client.0.smithi156.stdout:3/191: dwrite d8/da/f2e [0,4194304] 0 2024-07-16T09:55:33.299 INFO:tasks.workunit.client.0.smithi156.stdout:4/203: chown d4/f3d 4161 1 2024-07-16T09:55:33.301 INFO:tasks.workunit.client.0.smithi156.stdout:2/358: dread d0/d13/d17/d1c/f48 [0,4194304] 0 2024-07-16T09:55:33.304 INFO:tasks.workunit.client.0.smithi156.stdout:3/192: write d8/da/d13/f24 [921519,99880] 0 2024-07-16T09:55:33.306 INFO:tasks.workunit.client.0.smithi156.stdout:7/190: creat dc/d13/d36/f4c x:0 0 0 2024-07-16T09:55:33.306 INFO:tasks.workunit.client.0.smithi156.stdout:6/263: truncate d3/d13/d17/d4f/d1f/fe 1397793 0 2024-07-16T09:55:33.307 INFO:tasks.workunit.client.0.smithi156.stdout:5/176: symlink d1b/d1e/d37/l3f 0 2024-07-16T09:55:33.307 INFO:tasks.workunit.client.0.smithi156.stdout:6/264: stat d3/d13/d17/d4f/d1f/d19 0 2024-07-16T09:55:33.309 INFO:tasks.workunit.client.0.smithi156.stdout:1/194: creat dc/d14/d1e/f41 x:0 0 0 2024-07-16T09:55:33.310 INFO:tasks.workunit.client.0.smithi156.stdout:2/359: mknod d0/d13/c7f 0 2024-07-16T09:55:33.310 INFO:tasks.workunit.client.0.smithi156.stdout:8/165: link d0/f8 d0/d11/f40 0 2024-07-16T09:55:33.311 INFO:tasks.workunit.client.0.smithi156.stdout:3/193: creat d8/da/f3d x:0 0 0 2024-07-16T09:55:33.312 INFO:tasks.workunit.client.0.smithi156.stdout:2/360: dread - d0/d13/d17/d1c/f47 zero size 2024-07-16T09:55:33.312 INFO:tasks.workunit.client.0.smithi156.stdout:7/191: mkdir dc/d13/d20/d4d 0 2024-07-16T09:55:33.313 INFO:tasks.workunit.client.0.smithi156.stdout:9/249: dwrite d2/d5/d16/d22/d36/f31 [0,4194304] 0 2024-07-16T09:55:33.315 INFO:tasks.workunit.client.0.smithi156.stdout:6/265: readlink d3/d13/d17/d4f/d1e/l4c 0 2024-07-16T09:55:33.315 INFO:tasks.workunit.client.0.smithi156.stdout:1/195: mknod dc/d14/c42 0 2024-07-16T09:55:33.316 INFO:tasks.workunit.client.0.smithi156.stdout:2/361: truncate d0/d5/d29/d2d/d5b/d62/f77 618638 0 2024-07-16T09:55:33.317 INFO:tasks.workunit.client.0.smithi156.stdout:8/166: rmdir d0/d11/d20 39 2024-07-16T09:55:33.318 INFO:tasks.workunit.client.0.smithi156.stdout:2/362: read d0/d13/d17/d1c/f27 [1774349,55869] 0 2024-07-16T09:55:33.318 INFO:tasks.workunit.client.0.smithi156.stdout:7/192: fdatasync dc/d13/d20/f2b 0 2024-07-16T09:55:33.321 INFO:tasks.workunit.client.0.smithi156.stdout:9/250: symlink d2/d5/d16/d22/d2b/d3d/d3c/l54 0 2024-07-16T09:55:33.322 INFO:tasks.workunit.client.0.smithi156.stdout:6/266: rmdir d3/d13/d17/d4f/d1f/d19/d31/d35 39 2024-07-16T09:55:33.322 INFO:tasks.workunit.client.0.smithi156.stdout:9/251: chown d2/d5/d16/d22/d2b/d3d/d3c/f53 112416520 1 2024-07-16T09:55:33.322 INFO:tasks.workunit.client.0.smithi156.stdout:5/177: creat d1b/f40 x:0 0 0 2024-07-16T09:55:33.323 INFO:tasks.workunit.client.0.smithi156.stdout:6/267: write d3/d13/d17/d4f/d1f/f10 [3902307,117295] 0 2024-07-16T09:55:33.324 INFO:tasks.workunit.client.0.smithi156.stdout:9/252: fsync d2/d5/ff 0 2024-07-16T09:55:33.325 INFO:tasks.workunit.client.0.smithi156.stdout:5/178: dread - d1b/d1e/f3d zero size 2024-07-16T09:55:33.327 INFO:tasks.workunit.client.0.smithi156.stdout:6/268: read d3/d13/d17/d4f/f4b [3081346,14267] 0 2024-07-16T09:55:33.327 INFO:tasks.workunit.client.0.smithi156.stdout:1/196: creat dc/d14/d3e/f43 x:0 0 0 2024-07-16T09:55:33.328 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:33.327+0000 7ff337eac640 1 -- 172.21.15.156:0/1401843669 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff3300fe2d0 msgr2=0x7ff3300fa430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:55:33.328 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:33.327+0000 7ff337eac640 1 --2- 172.21.15.156:0/1401843669 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff3300fe2d0 0x7ff3300fa430 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7ff320004810 tx=0x7ff320009da0 comp rx=0 tx=0).stop 2024-07-16T09:55:33.328 INFO:tasks.workunit.client.0.smithi156.stdout:8/167: symlink d0/d11/l41 0 2024-07-16T09:55:33.329 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:33.327+0000 7ff337eac640 1 -- 172.21.15.156:0/1401843669 shutdown_connections 2024-07-16T09:55:33.329 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:33.327+0000 7ff337eac640 1 --2- 172.21.15.156:0/1401843669 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff3300fe2d0 0x7ff3300fa430 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:55:33.329 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:33.327+0000 7ff337eac640 1 --2- 172.21.15.156:0/1401843669 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff3300fd980 0x7ff3300fdd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:55:33.329 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:33.327+0000 7ff337eac640 1 -- 172.21.15.156:0/1401843669 >> 172.21.15.156:0/1401843669 conn(0x7ff3300f61a0 msgr2=0x7ff3300f85b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:55:33.329 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:33.327+0000 7ff337eac640 1 -- 172.21.15.156:0/1401843669 shutdown_connections 2024-07-16T09:55:33.330 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:33.328+0000 7ff337eac640 1 -- 172.21.15.156:0/1401843669 wait complete. 2024-07-16T09:55:33.330 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:33.328+0000 7ff337eac640 1 Processor -- start 2024-07-16T09:55:33.330 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:33.329+0000 7ff337eac640 1 -- start start 2024-07-16T09:55:33.330 INFO:tasks.workunit.client.0.smithi156.stdout:7/193: dwrite dc/d13/d20/f2f [4194304,4194304] 0 2024-07-16T09:55:33.331 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:33.329+0000 7ff337eac640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff3300fd980 0x7ff33010cf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:55:33.331 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:33.329+0000 7ff337eac640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff3300fe2d0 0x7ff33010d490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:55:33.331 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:33.329+0000 7ff337eac640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff33010dae0 con 0x7ff3300fe2d0 2024-07-16T09:55:33.332 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:33.329+0000 7ff337eac640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff33010e300 con 0x7ff3300fd980 2024-07-16T09:55:33.332 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:33.329+0000 7ff335c48640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff3300fd980 0x7ff33010cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:55:33.332 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:33.329+0000 7ff335c48640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff3300fd980 0x7ff33010cf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.156:39416/0 (socket says 172.21.15.156:39416) 2024-07-16T09:55:33.332 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:33.329+0000 7ff335c48640 1 -- 172.21.15.156:0/3443142147 learned_addr learned my addr 172.21.15.156:0/3443142147 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T09:55:33.332 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:33.329+0000 7ff335447640 1 --2- 172.21.15.156:0/3443142147 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff3300fe2d0 0x7ff33010d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:55:33.332 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:33.329+0000 7ff335c48640 1 -- 172.21.15.156:0/3443142147 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff3300fe2d0 msgr2=0x7ff33010d490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:55:33.332 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:33.329+0000 7ff335c48640 1 --2- 172.21.15.156:0/3443142147 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff3300fe2d0 0x7ff33010d490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:55:33.332 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:33.329+0000 7ff335c48640 1 -- 172.21.15.156:0/3443142147 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff320005040 con 0x7ff3300fd980 2024-07-16T09:55:33.332 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:33.330+0000 7ff335c48640 1 --2- 172.21.15.156:0/3443142147 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff3300fd980 0x7ff33010cf50 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7ff32c00eb40 tx=0x7ff32c00b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:55:33.332 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:33.330+0000 7ff3267fc640 1 -- 172.21.15.156:0/3443142147 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff32c007c30 con 0x7ff3300fd980 2024-07-16T09:55:33.333 INFO:tasks.workunit.client.0.smithi156.stdout:1/197: truncate dc/f1c 881114 0 2024-07-16T09:55:33.333 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:33.330+0000 7ff3267fc640 1 -- 172.21.15.156:0/3443142147 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff32c00f040 con 0x7ff3300fd980 2024-07-16T09:55:33.333 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:33.330+0000 7ff337eac640 1 -- 172.21.15.156:0/3443142147 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff33010e530 con 0x7ff3300fd980 2024-07-16T09:55:33.333 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:33.330+0000 7ff337eac640 1 -- 172.21.15.156:0/3443142147 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff33010eaa0 con 0x7ff3300fd980 2024-07-16T09:55:33.333 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:33.331+0000 7ff3267fc640 1 -- 172.21.15.156:0/3443142147 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff32c00cb50 con 0x7ff3300fd980 2024-07-16T09:55:33.333 INFO:tasks.workunit.client.0.smithi156.stdout:4/204: write d4/db/dc/f29 [623275,113207] 0 2024-07-16T09:55:33.334 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:33.332+0000 7ff337eac640 1 -- 172.21.15.156:0/3443142147 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3300f9e20 con 0x7ff3300fd980 2024-07-16T09:55:33.335 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:33.333+0000 7ff3267fc640 1 -- 172.21.15.156:0/3443142147 <== mon.1 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 20) v1 ==== 88539+0+0 (secure 0 0 0) 0x7ff32c00cd90 con 0x7ff3300fd980 2024-07-16T09:55:33.335 INFO:tasks.workunit.client.0.smithi156.stdout:3/194: getdents d8/da/d13 0 2024-07-16T09:55:33.335 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:33.334+0000 7ff3267fc640 1 --2- 172.21.15.156:0/3443142147 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7ff31c06a5a0 0x7ff31c06ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:55:33.335 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:33.334+0000 7ff3267fc640 1 -- 172.21.15.156:0/3443142147 <== mon.1 v2:172.21.15.176:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5464+0+0 (secure 0 0 0) 0x7ff32c0889d0 con 0x7ff3300fd980 2024-07-16T09:55:33.336 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:33.334+0000 7ff335447640 1 --2- 172.21.15.156:0/3443142147 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7ff31c06a5a0 0x7ff31c06ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:55:33.337 INFO:tasks.workunit.client.0.smithi156.stdout:3/195: fdatasync d8/da/f2e 0 2024-07-16T09:55:33.337 INFO:tasks.workunit.client.0.smithi156.stdout:2/363: creat d0/d5/d29/f80 x:0 0 0 2024-07-16T09:55:33.337 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:33.335+0000 7ff335447640 1 --2- 172.21.15.156:0/3443142147 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7ff31c06a5a0 0x7ff31c06ca50 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7ff3200047e0 tx=0x7ff320035040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:55:33.338 INFO:tasks.workunit.client.0.smithi156.stdout:5/179: mknod d1b/c41 0 2024-07-16T09:55:33.339 INFO:tasks.workunit.client.0.smithi156.stdout:5/180: write ff [2357972,59191] 0 2024-07-16T09:55:33.340 INFO:tasks.workunit.client.0.smithi156.stdout:1/198: mknod dc/d24/c44 0 2024-07-16T09:55:33.341 INFO:tasks.workunit.client.0.smithi156.stdout:5/181: write d1b/d28/f35 [990069,4641] 0 2024-07-16T09:55:33.342 INFO:tasks.workunit.client.0.smithi156.stdout:1/199: write dc/d20/f23 [1168673,26663] 0 2024-07-16T09:55:33.342 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:33.341+0000 7ff3267fc640 1 -- 172.21.15.156:0/3443142147 <== mon.1 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7ff32c055d20 con 0x7ff3300fd980 2024-07-16T09:55:33.346 INFO:tasks.workunit.client.0.smithi156.stdout:3/196: creat d8/da/d21/f3e x:0 0 0 2024-07-16T09:55:33.346 INFO:tasks.workunit.client.0.smithi156.stdout:2/364: creat d0/d13/f81 x:0 0 0 2024-07-16T09:55:33.348 INFO:tasks.workunit.client.0.smithi156.stdout:7/194: rename dc/c29 to dc/d13/d20/d4d/c4e 0 2024-07-16T09:55:33.349 INFO:tasks.workunit.client.0.smithi156.stdout:4/205: dwrite d4/db/dc/f2e [0,4194304] 0 2024-07-16T09:55:33.349 INFO:tasks.workunit.client.0.smithi156.stdout:5/182: mkdir d1b/d1f/d42 0 2024-07-16T09:55:33.350 INFO:tasks.workunit.client.0.smithi156.stdout:9/253: creat d2/d5/f55 x:0 0 0 2024-07-16T09:55:33.350 INFO:tasks.workunit.client.1.smithi176.stdout:9/237: dwrite d5/d1d/f3f [4194304,4194304] 0 2024-07-16T09:55:33.352 INFO:tasks.workunit.client.0.smithi156.stdout:9/254: write d2/f15 [1292902,14080] 0 2024-07-16T09:55:33.352 INFO:tasks.workunit.client.0.smithi156.stdout:1/200: mkdir dc/d14/d3e/d45 0 2024-07-16T09:55:33.353 INFO:tasks.workunit.client.1.smithi176.stdout:4/264: dread da/d3d/f1b [0,4194304] 0 2024-07-16T09:55:33.353 INFO:tasks.workunit.client.0.smithi156.stdout:1/201: read f6 [4175258,71118] 0 2024-07-16T09:55:33.354 INFO:tasks.workunit.client.1.smithi176.stdout:8/415: mkdir d8/d20/d2a/d53/d93 0 2024-07-16T09:55:33.355 INFO:tasks.workunit.client.0.smithi156.stdout:3/197: creat d8/d14/f3f x:0 0 0 2024-07-16T09:55:33.355 INFO:tasks.workunit.client.0.smithi156.stdout:6/269: creat d3/d13/d17/d4f/d4d/f57 x:0 0 0 2024-07-16T09:55:33.356 INFO:tasks.workunit.client.0.smithi156.stdout:8/168: write d0/f8 [2430343,83614] 0 2024-07-16T09:55:33.356 INFO:tasks.workunit.client.1.smithi176.stdout:4/265: write da/d3d/d17/f2a [748574,36701] 0 2024-07-16T09:55:33.357 INFO:tasks.workunit.client.0.smithi156.stdout:7/195: fsync dc/f40 0 2024-07-16T09:55:33.358 INFO:tasks.workunit.client.0.smithi156.stdout:5/183: mkdir d1b/d28/d43 0 2024-07-16T09:55:33.358 INFO:tasks.workunit.client.1.smithi176.stdout:4/266: chown da/f1a 32 1 2024-07-16T09:55:33.360 INFO:tasks.workunit.client.0.smithi156.stdout:5/184: truncate fc 1907936 0 2024-07-16T09:55:33.360 INFO:tasks.workunit.client.1.smithi176.stdout:3/161: dwrite d3/d17/d1d/d1e/f1f [0,4194304] 0 2024-07-16T09:55:33.362 INFO:tasks.workunit.client.0.smithi156.stdout:9/255: write d2/de/f19 [1649653,65575] 0 2024-07-16T09:55:33.362 INFO:tasks.workunit.client.1.smithi176.stdout:9/238: rename d5/d1d/d20/f31 to d5/de/f47 0 2024-07-16T09:55:33.363 INFO:tasks.workunit.client.0.smithi156.stdout:5/185: write d1b/d28/f35 [1235841,79553] 0 2024-07-16T09:55:33.364 INFO:tasks.workunit.client.0.smithi156.stdout:1/202: mknod dc/d24/c46 0 2024-07-16T09:55:33.364 INFO:tasks.workunit.client.0.smithi156.stdout:5/186: chown d1b/f2f 857897 1 2024-07-16T09:55:33.365 INFO:tasks.workunit.client.0.smithi156.stdout:3/198: mkdir d8/da/d28/d40 0 2024-07-16T09:55:33.365 INFO:tasks.workunit.client.0.smithi156.stdout:5/187: chown d1b/d1f/d42 3344 1 2024-07-16T09:55:33.366 INFO:tasks.workunit.client.0.smithi156.stdout:1/203: chown f6 2 1 2024-07-16T09:55:33.367 INFO:tasks.workunit.client.1.smithi176.stdout:3/162: link c2 d3/d17/d1d/c2e 0 2024-07-16T09:55:33.367 INFO:tasks.workunit.client.0.smithi156.stdout:4/206: mkdir d4/db/d47 0 2024-07-16T09:55:33.368 INFO:tasks.workunit.client.0.smithi156.stdout:7/196: unlink dc/d13/f33 0 2024-07-16T09:55:33.369 INFO:tasks.workunit.client.0.smithi156.stdout:8/169: creat d0/d11/d13/d1c/d24/f42 x:0 0 0 2024-07-16T09:55:33.369 INFO:tasks.workunit.client.0.smithi156.stdout:9/256: creat d2/d5/d16/d22/d36/f56 x:0 0 0 2024-07-16T09:55:33.370 INFO:tasks.workunit.client.1.smithi176.stdout:4/267: dwrite da/d3d/f1b [0,4194304] 0 2024-07-16T09:55:33.371 INFO:tasks.workunit.client.0.smithi156.stdout:5/188: fdatasync f19 0 2024-07-16T09:55:33.372 INFO:tasks.workunit.client.0.smithi156.stdout:9/257: stat d2/d5/d16/c17 0 2024-07-16T09:55:33.373 INFO:tasks.workunit.client.1.smithi176.stdout:3/163: unlink d3/d17/d1d/f20 0 2024-07-16T09:55:33.374 INFO:tasks.workunit.client.1.smithi176.stdout:9/239: dwrite d5/de/f34 [0,4194304] 0 2024-07-16T09:55:33.375 INFO:tasks.workunit.client.0.smithi156.stdout:3/199: creat d8/d14/f41 x:0 0 0 2024-07-16T09:55:33.375 INFO:tasks.workunit.client.1.smithi176.stdout:4/268: mkdir da/d14/d4a 0 2024-07-16T09:55:33.376 INFO:tasks.workunit.client.0.smithi156.stdout:8/170: truncate d0/dd/f1e 103444 0 2024-07-16T09:55:33.376 INFO:tasks.workunit.client.1.smithi176.stdout:3/164: write d3/d17/d1d/f2d [797691,15196] 0 2024-07-16T09:55:33.377 INFO:tasks.workunit.client.1.smithi176.stdout:9/240: write d5/de/f2f [297714,10230] 0 2024-07-16T09:55:33.378 INFO:tasks.workunit.client.0.smithi156.stdout:6/270: symlink d3/d13/d17/d4f/d1f/d19/d31/l58 0 2024-07-16T09:55:33.378 INFO:tasks.workunit.client.0.smithi156.stdout:8/171: chown d0/d11/d29 1731 1 2024-07-16T09:55:33.381 INFO:tasks.workunit.client.0.smithi156.stdout:1/204: dwrite dc/d20/f36 [0,4194304] 0 2024-07-16T09:55:33.381 INFO:tasks.workunit.client.0.smithi156.stdout:9/258: mkdir d2/d5/d16/d57 0 2024-07-16T09:55:33.381 INFO:tasks.workunit.client.0.smithi156.stdout:5/189: creat d1b/d1f/f44 x:0 0 0 2024-07-16T09:55:33.381 INFO:tasks.workunit.client.1.smithi176.stdout:3/165: creat d3/d17/f2f x:0 0 0 2024-07-16T09:55:33.382 INFO:tasks.workunit.client.1.smithi176.stdout:9/241: mkdir d5/d43/d48 0 2024-07-16T09:55:33.382 INFO:tasks.workunit.client.0.smithi156.stdout:4/207: mknod d4/db/c48 0 2024-07-16T09:55:33.383 INFO:tasks.workunit.client.1.smithi176.stdout:4/269: mkdir da/dc/d4b 0 2024-07-16T09:55:33.384 INFO:tasks.workunit.client.1.smithi176.stdout:3/166: dread - d3/da/f29 zero size 2024-07-16T09:55:33.385 INFO:tasks.workunit.client.0.smithi156.stdout:4/208: readlink d4/d23/l27 0 2024-07-16T09:55:33.386 INFO:tasks.workunit.client.1.smithi176.stdout:9/242: unlink d5/f16 0 2024-07-16T09:55:33.387 INFO:tasks.workunit.client.1.smithi176.stdout:9/243: chown d5/d43/d48 32384 1 2024-07-16T09:55:33.387 INFO:tasks.workunit.client.0.smithi156.stdout:6/271: unlink d3/d13/d17/d4f/d1f/l38 0 2024-07-16T09:55:33.389 INFO:tasks.workunit.client.1.smithi176.stdout:4/270: getdents da/d41 0 2024-07-16T09:55:33.389 INFO:tasks.workunit.client.0.smithi156.stdout:1/205: unlink l1 0 2024-07-16T09:55:33.390 INFO:tasks.workunit.client.1.smithi176.stdout:4/271: write da/dc/fe [55835,58028] 0 2024-07-16T09:55:33.390 INFO:tasks.workunit.client.1.smithi176.stdout:9/244: readlink d5/de/d38/l3d 0 2024-07-16T09:55:33.391 INFO:tasks.workunit.client.0.smithi156.stdout:9/259: dread - d2/d5/d16/d22/d36/d33/d3a/f49 zero size 2024-07-16T09:55:33.392 INFO:tasks.workunit.client.1.smithi176.stdout:9/245: write d5/de/f40 [4960807,34395] 0 2024-07-16T09:55:33.392 INFO:tasks.workunit.client.0.smithi156.stdout:8/172: creat d0/dd/f43 x:0 0 0 2024-07-16T09:55:33.393 INFO:tasks.workunit.client.1.smithi176.stdout:9/246: write d5/fb [1473425,65754] 0 2024-07-16T09:55:33.394 INFO:tasks.workunit.client.0.smithi156.stdout:5/190: dread d1b/d28/f3c [0,4194304] 0 2024-07-16T09:55:33.396 INFO:tasks.workunit.client.0.smithi156.stdout:4/209: rename d4/db/cd to d4/d41/c49 0 2024-07-16T09:55:33.397 INFO:tasks.workunit.client.1.smithi176.stdout:8/416: dread d8/d23/d2c/d37/f4f [0,4194304] 0 2024-07-16T09:55:33.398 INFO:tasks.workunit.client.0.smithi156.stdout:3/200: link d8/d14/l36 d8/da/d1d/d2c/l42 0 2024-07-16T09:55:33.399 INFO:tasks.workunit.client.1.smithi176.stdout:8/417: chown d8/d23/d2c/d37/c91 49781 1 2024-07-16T09:55:33.400 INFO:tasks.workunit.client.0.smithi156.stdout:5/191: dread d1b/f31 [0,4194304] 0 2024-07-16T09:55:33.400 INFO:tasks.workunit.client.1.smithi176.stdout:3/167: link d3/d17/d1d/c2e d3/d17/d1d/d1e/c30 0 2024-07-16T09:55:33.401 INFO:tasks.workunit.client.0.smithi156.stdout:3/201: read d8/da/d13/f24 [938871,55246] 0 2024-07-16T09:55:33.401 INFO:tasks.workunit.client.1.smithi176.stdout:8/418: fdatasync d8/d20/d2a/f49 0 2024-07-16T09:55:33.402 INFO:tasks.workunit.client.1.smithi176.stdout:3/168: chown d3/f2b 102128576 1 2024-07-16T09:55:33.402 INFO:tasks.workunit.client.1.smithi176.stdout:4/272: dread da/d3d/d17/f2a [0,4194304] 0 2024-07-16T09:55:33.403 INFO:tasks.workunit.client.0.smithi156.stdout:9/260: dwrite d2/d5/d16/d22/d36/d33/f2c [0,4194304] 0 2024-07-16T09:55:33.403 INFO:tasks.workunit.client.1.smithi176.stdout:4/273: readlink da/d14/l2f 0 2024-07-16T09:55:33.404 INFO:tasks.workunit.client.1.smithi176.stdout:9/247: creat d5/f49 x:0 0 0 2024-07-16T09:55:33.404 INFO:tasks.workunit.client.1.smithi176.stdout:4/274: fsync da/d3d/f1b 0 2024-07-16T09:55:33.405 INFO:tasks.workunit.client.1.smithi176.stdout:9/248: fdatasync d5/d1d/f46 0 2024-07-16T09:55:33.408 INFO:tasks.workunit.client.0.smithi156.stdout:3/202: write d8/da/d21/f3e [343770,77728] 0 2024-07-16T09:55:33.408 INFO:tasks.workunit.client.0.smithi156.stdout:9/261: write d2/d5/d16/d22/d2b/d3d/d3c/f53 [59717,9615] 0 2024-07-16T09:55:33.409 INFO:tasks.workunit.client.0.smithi156.stdout:6/272: truncate d3/d13/d17/d4f/f4b 2486956 0 2024-07-16T09:55:33.412 INFO:tasks.workunit.client.0.smithi156.stdout:6/273: fdatasync d3/d13/d17/d4f/d1e/f3d 0 2024-07-16T09:55:33.412 INFO:tasks.workunit.client.1.smithi176.stdout:8/419: dwrite d8/d20/d2a/d31/f3e [0,4194304] 0 2024-07-16T09:55:33.413 INFO:tasks.workunit.client.1.smithi176.stdout:4/275: mknod da/dc/d4b/c4c 0 2024-07-16T09:55:33.413 INFO:tasks.workunit.client.0.smithi156.stdout:8/173: dwrite d0/dd/f33 [0,4194304] 0 2024-07-16T09:55:33.413 INFO:tasks.workunit.client.0.smithi156.stdout:9/262: write d2/d5/d16/d22/d36/d33/d3a/f49 [663682,8530] 0 2024-07-16T09:55:33.414 INFO:tasks.workunit.client.1.smithi176.stdout:8/420: fdatasync d8/d14/d1e/d33/f47 0 2024-07-16T09:55:33.415 INFO:tasks.workunit.client.1.smithi176.stdout:9/249: creat d5/de/d30/f4a x:0 0 0 2024-07-16T09:55:33.416 INFO:tasks.workunit.client.0.smithi156.stdout:5/192: mkdir d1b/d1f/d45 0 2024-07-16T09:55:33.417 INFO:tasks.workunit.client.1.smithi176.stdout:4/276: fsync da/d3d/d17/f18 0 2024-07-16T09:55:33.417 INFO:tasks.workunit.client.1.smithi176.stdout:8/421: dread - d8/d20/d2a/f75 zero size 2024-07-16T09:55:33.417 INFO:tasks.workunit.client.0.smithi156.stdout:1/206: link dc/c11 dc/d14/c47 0 2024-07-16T09:55:33.417 INFO:tasks.workunit.client.0.smithi156.stdout:5/193: dread - d1b/d28/d2d/f2e zero size 2024-07-16T09:55:33.418 INFO:tasks.workunit.client.1.smithi176.stdout:4/277: readlink da/d3d/d17/l21 0 2024-07-16T09:55:33.420 INFO:tasks.workunit.client.0.smithi156.stdout:5/194: read f19 [3018767,12977] 0 2024-07-16T09:55:33.420 INFO:tasks.workunit.client.0.smithi156.stdout:3/203: mkdir d8/da/d28/d43 0 2024-07-16T09:55:33.420 INFO:tasks.workunit.client.1.smithi176.stdout:8/422: write d8/d14/d1e/d55/f8a [349616,7053] 0 2024-07-16T09:55:33.421 INFO:tasks.workunit.client.1.smithi176.stdout:9/250: unlink d5/d1d/f27 0 2024-07-16T09:55:33.422 INFO:tasks.workunit.client.0.smithi156.stdout:5/195: write fc [2815494,124208] 0 2024-07-16T09:55:33.424 INFO:tasks.workunit.client.0.smithi156.stdout:5/196: read - d1b/d1e/f3d zero size 2024-07-16T09:55:33.424 INFO:tasks.workunit.client.0.smithi156.stdout:6/274: rmdir d3/d13/d17/d4f 39 2024-07-16T09:55:33.424 INFO:tasks.workunit.client.1.smithi176.stdout:9/251: symlink d5/de/d38/l4b 0 2024-07-16T09:55:33.425 INFO:tasks.workunit.client.0.smithi156.stdout:8/174: fdatasync d0/f6 0 2024-07-16T09:55:33.426 INFO:tasks.workunit.client.1.smithi176.stdout:9/252: dread - d5/d1d/f3c zero size 2024-07-16T09:55:33.428 INFO:tasks.workunit.client.0.smithi156.stdout:3/204: dread d8/da/d28/f2a [0,4194304] 0 2024-07-16T09:55:33.429 INFO:tasks.workunit.client.0.smithi156.stdout:8/175: chown d0/f6 7397451 1 2024-07-16T09:55:33.430 INFO:tasks.workunit.client.0.smithi156.stdout:1/207: rmdir dc/d14/d1e 39 2024-07-16T09:55:33.432 INFO:tasks.workunit.client.1.smithi176.stdout:9/253: symlink d5/de/d30/l4c 0 2024-07-16T09:55:33.432 INFO:tasks.workunit.client.1.smithi176.stdout:3/169: write d3/da/fe [1815282,31138] 0 2024-07-16T09:55:33.433 INFO:tasks.workunit.client.0.smithi156.stdout:4/210: dwrite d4/d23/d30/d33/f3a [0,4194304] 0 2024-07-16T09:55:33.434 INFO:tasks.workunit.client.0.smithi156.stdout:5/197: dread d1b/d28/f3c [0,4194304] 0 2024-07-16T09:55:33.434 INFO:tasks.workunit.client.1.smithi176.stdout:3/170: fsync d3/d17/f2f 0 2024-07-16T09:55:33.435 INFO:tasks.workunit.client.0.smithi156.stdout:4/211: write d4/f9 [3115838,9774] 0 2024-07-16T09:55:33.435 INFO:tasks.workunit.client.1.smithi176.stdout:9/254: rename d5/de/c11 to d5/d43/d48/c4d 0 2024-07-16T09:55:33.436 INFO:tasks.workunit.client.1.smithi176.stdout:9/255: dread - d5/de/f47 zero size 2024-07-16T09:55:33.436 INFO:tasks.workunit.client.1.smithi176.stdout:9/256: stat d5/de/d38 0 2024-07-16T09:55:33.437 INFO:tasks.workunit.client.1.smithi176.stdout:3/171: mkdir d3/d17/d31 0 2024-07-16T09:55:33.437 INFO:tasks.workunit.client.0.smithi156.stdout:1/208: fsync f4 0 2024-07-16T09:55:33.438 INFO:tasks.workunit.client.1.smithi176.stdout:3/172: write d3/f2b [1838117,7368] 0 2024-07-16T09:55:33.440 INFO:tasks.workunit.client.0.smithi156.stdout:3/205: mkdir d8/da/d44 0 2024-07-16T09:55:33.440 INFO:tasks.workunit.client.0.smithi156.stdout:8/176: dread d0/f2c [0,4194304] 0 2024-07-16T09:55:33.441 INFO:tasks.workunit.client.1.smithi176.stdout:9/257: creat d5/de/d38/f4e x:0 0 0 2024-07-16T09:55:33.442 INFO:tasks.workunit.client.1.smithi176.stdout:9/258: chown d5/de/f12 6387513 1 2024-07-16T09:55:33.443 INFO:tasks.workunit.client.1.smithi176.stdout:3/173: creat d3/d17/f32 x:0 0 0 2024-07-16T09:55:33.444 INFO:tasks.workunit.client.1.smithi176.stdout:3/174: stat d3/d17/f26 0 2024-07-16T09:55:33.444 INFO:tasks.workunit.client.1.smithi176.stdout:3/175: fdatasync d3/d17/f32 0 2024-07-16T09:55:33.445 INFO:tasks.workunit.client.1.smithi176.stdout:9/259: creat d5/d43/f4f x:0 0 0 2024-07-16T09:55:33.447 INFO:tasks.workunit.client.1.smithi176.stdout:4/278: dread da/fb [0,4194304] 0 2024-07-16T09:55:33.448 INFO:tasks.workunit.client.0.smithi156.stdout:6/275: creat d3/d13/d17/d4f/d4d/d3b/f59 x:0 0 0 2024-07-16T09:55:33.448 INFO:tasks.workunit.client.1.smithi176.stdout:3/176: mkdir d3/da/d33 0 2024-07-16T09:55:33.449 INFO:tasks.workunit.client.0.smithi156.stdout:4/212: mknod d4/d23/d30/d33/c4a 0 2024-07-16T09:55:33.450 INFO:tasks.workunit.client.0.smithi156.stdout:9/263: link d2/l43 d2/d5/d16/d22/l58 0 2024-07-16T09:55:33.450 INFO:tasks.workunit.client.0.smithi156.stdout:1/209: dwrite dc/d20/f36 [0,4194304] 0 2024-07-16T09:55:33.452 INFO:tasks.workunit.client.0.smithi156.stdout:9/264: fsync d2/d5/d16/f20 0 2024-07-16T09:55:33.453 INFO:tasks.workunit.client.0.smithi156.stdout:8/177: rmdir d0/dd 39 2024-07-16T09:55:33.454 INFO:tasks.workunit.client.1.smithi176.stdout:3/177: dread d3/d17/d1d/d1e/f27 [0,4194304] 0 2024-07-16T09:55:33.457 INFO:tasks.workunit.client.0.smithi156.stdout:5/198: dread f13 [0,4194304] 0 2024-07-16T09:55:33.458 INFO:tasks.workunit.client.1.smithi176.stdout:9/260: dwrite d5/de/f1a [8388608,4194304] 0 2024-07-16T09:55:33.459 INFO:tasks.workunit.client.0.smithi156.stdout:5/199: readlink d1b/d1e/d37/l3f 0 2024-07-16T09:55:33.459 INFO:tasks.workunit.client.0.smithi156.stdout:3/206: truncate d8/da/d13/f24 2038581 0 2024-07-16T09:55:33.460 INFO:tasks.workunit.client.1.smithi176.stdout:9/261: chown d5/d1d/d20/c2c 952618 1 2024-07-16T09:55:33.461 INFO:tasks.workunit.client.0.smithi156.stdout:9/265: dwrite d2/d5/f55 [0,4194304] 0 2024-07-16T09:55:33.462 INFO:tasks.workunit.client.0.smithi156.stdout:3/207: truncate d8/f10 2747889 0 2024-07-16T09:55:33.464 INFO:tasks.workunit.client.1.smithi176.stdout:9/262: mknod d5/de/c50 0 2024-07-16T09:55:33.465 INFO:tasks.workunit.client.1.smithi176.stdout:4/279: write da/d3d/d33/f3a [753198,60743] 0 2024-07-16T09:55:33.468 INFO:tasks.workunit.client.1.smithi176.stdout:4/280: stat da/d3d/d17/l21 0 2024-07-16T09:55:33.473 INFO:tasks.workunit.client.1.smithi176.stdout:4/281: mknod da/d3d/d33/c4d 0 2024-07-16T09:55:33.474 INFO:tasks.workunit.client.0.smithi156.stdout:6/276: mkdir d3/d13/d17/d4f/d4d/d3b/d5a 0 2024-07-16T09:55:33.474 INFO:tasks.workunit.client.1.smithi176.stdout:3/178: dread d3/d17/d1d/d1e/f21 [0,4194304] 0 2024-07-16T09:55:33.474 INFO:tasks.workunit.client.0.smithi156.stdout:6/277: fdatasync d3/d13/d17/d4f/d1e/f3d 0 2024-07-16T09:55:33.475 INFO:tasks.workunit.client.0.smithi156.stdout:4/213: rmdir d4/d23/d30 39 2024-07-16T09:55:33.475 INFO:tasks.workunit.client.1.smithi176.stdout:9/263: getdents d5/de/d38 0 2024-07-16T09:55:33.477 INFO:tasks.workunit.client.0.smithi156.stdout:8/178: dread - d0/dd/f30 zero size 2024-07-16T09:55:33.477 INFO:tasks.workunit.client.1.smithi176.stdout:3/179: chown d3/da/fe 24984595 1 2024-07-16T09:55:33.478 INFO:tasks.workunit.client.0.smithi156.stdout:8/179: readlink d0/d11/d13/d1c/l37 0 2024-07-16T09:55:33.478 INFO:tasks.workunit.client.1.smithi176.stdout:3/180: chown d3/d17 182212264 1 2024-07-16T09:55:33.478 INFO:tasks.workunit.client.1.smithi176.stdout:9/264: mknod d5/d1d/c51 0 2024-07-16T09:55:33.479 INFO:tasks.workunit.client.0.smithi156.stdout:5/200: mknod d1b/d28/c46 0 2024-07-16T09:55:33.481 INFO:tasks.workunit.client.0.smithi156.stdout:9/266: rmdir d2/d5/d16/d22/d36 39 2024-07-16T09:55:33.482 INFO:tasks.workunit.client.0.smithi156.stdout:4/214: read d4/db/f18 [246581,128347] 0 2024-07-16T09:55:33.482 INFO:tasks.workunit.client.0.smithi156.stdout:3/208: rmdir d8/da/d21 39 2024-07-16T09:55:33.483 INFO:tasks.workunit.client.1.smithi176.stdout:3/181: write d3/da/fe [1626732,33914] 0 2024-07-16T09:55:33.485 INFO:tasks.workunit.client.1.smithi176.stdout:4/282: dwrite da/f1a [0,4194304] 0 2024-07-16T09:55:33.486 INFO:tasks.workunit.client.1.smithi176.stdout:3/182: write d3/d17/d1d/d1e/f1f [3294478,115040] 0 2024-07-16T09:55:33.488 INFO:tasks.workunit.client.1.smithi176.stdout:4/283: readlink da/d3d/d17/l2b 0 2024-07-16T09:55:33.491 INFO:tasks.workunit.client.1.smithi176.stdout:3/183: fsync d3/f25 0 2024-07-16T09:55:33.491 INFO:tasks.workunit.client.0.smithi156.stdout:0/153: sync 2024-07-16T09:55:33.492 INFO:tasks.workunit.client.1.smithi176.stdout:9/265: dwrite d5/d1d/f46 [0,4194304] 0 2024-07-16T09:55:33.493 INFO:tasks.workunit.client.0.smithi156.stdout:0/154: fdatasync d0/d7/f10 0 2024-07-16T09:55:33.493 INFO:tasks.workunit.client.0.smithi156.stdout:8/180: symlink d0/d11/d20/l44 0 2024-07-16T09:55:33.493 INFO:tasks.workunit.client.1.smithi176.stdout:5/352: sync 2024-07-16T09:55:33.494 INFO:tasks.workunit.client.1.smithi176.stdout:5/353: chown d4/fd 220663336 1 2024-07-16T09:55:33.495 INFO:tasks.workunit.client.1.smithi176.stdout:5/354: readlink d4/d37/d72/l47 0 2024-07-16T09:55:33.495 INFO:tasks.workunit.client.0.smithi156.stdout:0/155: readlink d0/d7/d25/l1f 0 2024-07-16T09:55:33.497 INFO:tasks.workunit.client.0.smithi156.stdout:0/156: dread - d0/d7/f19 zero size 2024-07-16T09:55:33.497 INFO:tasks.workunit.client.0.smithi156.stdout:0/157: stat d0/d7/c16 0 2024-07-16T09:55:33.498 INFO:tasks.workunit.client.0.smithi156.stdout:9/267: creat d2/d5/d16/d22/d36/f59 x:0 0 0 2024-07-16T09:55:33.498 INFO:tasks.workunit.client.1.smithi176.stdout:4/284: creat da/d14/d4a/f4e x:0 0 0 2024-07-16T09:55:33.499 INFO:tasks.workunit.client.0.smithi156.stdout:5/201: write d1b/f31 [851686,113734] 0 2024-07-16T09:55:33.500 INFO:tasks.workunit.client.0.smithi156.stdout:9/268: read - d2/d5/d16/d22/d36/f59 zero size 2024-07-16T09:55:33.501 INFO:tasks.workunit.client.0.smithi156.stdout:3/209: symlink d8/da/d1d/d2c/d33/l45 0 2024-07-16T09:55:33.502 INFO:tasks.workunit.client.1.smithi176.stdout:5/355: link d4/d37/d72/d38/d39/f3a d4/d37/f84 0 2024-07-16T09:55:33.502 INFO:tasks.workunit.client.1.smithi176.stdout:3/184: dread d3/da/fe [0,4194304] 0 2024-07-16T09:55:33.502 INFO:tasks.workunit.client.0.smithi156.stdout:6/278: creat d3/d13/d17/d4f/d1f/d19/f5b x:0 0 0 2024-07-16T09:55:33.503 INFO:tasks.workunit.client.0.smithi156.stdout:0/158: symlink d0/d7/d25/l29 0 2024-07-16T09:55:33.505 INFO:tasks.workunit.client.1.smithi176.stdout:9/266: rmdir d5 39 2024-07-16T09:55:33.505 INFO:tasks.workunit.client.1.smithi176.stdout:5/356: truncate d4/d10/f5f 59176 0 2024-07-16T09:55:33.509 INFO:tasks.workunit.client.0.smithi156.stdout:0/159: write d0/d7/f10 [3263544,93573] 0 2024-07-16T09:55:33.510 INFO:tasks.workunit.client.0.smithi156.stdout:8/181: mkdir d0/d11/d13/d1c/d45 0 2024-07-16T09:55:33.510 INFO:tasks.workunit.client.1.smithi176.stdout:3/185: creat d3/d17/d1d/d1e/f34 x:0 0 0 2024-07-16T09:55:33.510 INFO:tasks.workunit.client.1.smithi176.stdout:4/285: dwrite da/d3d/d17/f2a [0,4194304] 0 2024-07-16T09:55:33.512 INFO:tasks.workunit.client.0.smithi156.stdout:3/210: dwrite d8/da/f32 [0,4194304] 0 2024-07-16T09:55:33.512 INFO:tasks.workunit.client.0.smithi156.stdout:4/215: getdents d4/d23 0 2024-07-16T09:55:33.513 INFO:tasks.workunit.client.1.smithi176.stdout:3/186: symlink d3/d17/d1d/d1e/l35 0 2024-07-16T09:55:33.515 INFO:tasks.workunit.client.0.smithi156.stdout:0/160: truncate d0/d7/f11 908677 0 2024-07-16T09:55:33.517 INFO:tasks.workunit.client.1.smithi176.stdout:3/187: dread d3/da/fe [0,4194304] 0 2024-07-16T09:55:33.518 INFO:tasks.workunit.client.1.smithi176.stdout:4/286: creat da/dc/d39/f4f x:0 0 0 2024-07-16T09:55:33.518 INFO:tasks.workunit.client.0.smithi156.stdout:1/210: creat dc/d14/f48 x:0 0 0 2024-07-16T09:55:33.519 INFO:tasks.workunit.client.1.smithi176.stdout:9/267: rename d5/d43/d48 to d5/d52 0 2024-07-16T09:55:33.522 INFO:tasks.workunit.client.0.smithi156.stdout:8/182: dwrite d0/f2 [0,4194304] 0 2024-07-16T09:55:33.523 INFO:tasks.workunit.client.1.smithi176.stdout:3/188: symlink d3/d17/d31/l36 0 2024-07-16T09:55:33.523 INFO:tasks.workunit.client.1.smithi176.stdout:5/357: write d4/d37/d72/d82/f64 [2301770,106144] 0 2024-07-16T09:55:33.523 INFO:tasks.workunit.client.1.smithi176.stdout:3/189: fsync d3/da/fe 0 2024-07-16T09:55:33.525 INFO:tasks.workunit.client.1.smithi176.stdout:5/358: write d4/d37/d72/d38/d39/f5d [2221055,53501] 0 2024-07-16T09:55:33.526 INFO:tasks.workunit.client.0.smithi156.stdout:2/365: sync 2024-07-16T09:55:33.526 INFO:tasks.workunit.client.0.smithi156.stdout:7/197: sync 2024-07-16T09:55:33.527 INFO:tasks.workunit.client.1.smithi176.stdout:4/287: creat da/dc/f50 x:0 0 0 2024-07-16T09:55:33.528 INFO:tasks.workunit.client.0.smithi156.stdout:9/269: getdents d2/d5 0 2024-07-16T09:55:33.528 INFO:tasks.workunit.client.1.smithi176.stdout:3/190: mknod d3/d17/c37 0 2024-07-16T09:55:33.530 INFO:tasks.workunit.client.0.smithi156.stdout:0/161: creat d0/d7/d25/f2a x:0 0 0 2024-07-16T09:55:33.530 INFO:tasks.workunit.client.1.smithi176.stdout:9/268: rename d5/c44 to d5/de/c53 0 2024-07-16T09:55:33.531 INFO:tasks.workunit.client.1.smithi176.stdout:4/288: rename da to da/d14/d40/d51 22 2024-07-16T09:55:33.532 INFO:tasks.workunit.client.0.smithi156.stdout:5/202: dwrite d1b/d1f/f20 [0,4194304] 0 2024-07-16T09:55:33.533 INFO:tasks.workunit.client.0.smithi156.stdout:8/183: creat d0/d11/d2e/d35/f46 x:0 0 0 2024-07-16T09:55:33.533 INFO:tasks.workunit.client.0.smithi156.stdout:9/270: dread - d2/de/d42/f4c zero size 2024-07-16T09:55:33.535 INFO:tasks.workunit.client.1.smithi176.stdout:5/359: dread d4/d37/d72/f4f [0,4194304] 0 2024-07-16T09:55:33.536 INFO:tasks.workunit.client.0.smithi156.stdout:9/271: stat d2/d5 0 2024-07-16T09:55:33.536 INFO:tasks.workunit.client.0.smithi156.stdout:5/203: write f14 [3308941,65837] 0 2024-07-16T09:55:33.536 INFO:tasks.workunit.client.0.smithi156.stdout:4/216: mkdir d4/db/d47/d4b 0 2024-07-16T09:55:33.537 INFO:tasks.workunit.client.1.smithi176.stdout:9/269: rename d5/d1d/d20/c2a to d5/d52/c54 0 2024-07-16T09:55:33.537 INFO:tasks.workunit.client.0.smithi156.stdout:7/198: creat dc/d3e/d3f/d45/f4f x:0 0 0 2024-07-16T09:55:33.538 INFO:tasks.workunit.client.0.smithi156.stdout:5/204: truncate d1b/d28/f39 804178 0 2024-07-16T09:55:33.538 INFO:tasks.workunit.client.0.smithi156.stdout:4/217: chown d4/d23/l27 1856673919 1 2024-07-16T09:55:33.538 INFO:tasks.workunit.client.1.smithi176.stdout:4/289: creat da/dc/d4b/f52 x:0 0 0 2024-07-16T09:55:33.539 INFO:tasks.workunit.client.0.smithi156.stdout:2/366: mknod d0/d5/d29/d2d/d5b/d62/c82 0 2024-07-16T09:55:33.540 INFO:tasks.workunit.client.0.smithi156.stdout:9/272: dread - d2/de/d42/f47 zero size 2024-07-16T09:55:33.541 INFO:tasks.workunit.client.0.smithi156.stdout:5/205: read d1b/d1f/f20 [1189844,7898] 0 2024-07-16T09:55:33.542 INFO:tasks.workunit.client.1.smithi176.stdout:3/191: dwrite d3/d17/f26 [0,4194304] 0 2024-07-16T09:55:33.543 INFO:tasks.workunit.client.0.smithi156.stdout:8/184: creat d0/dd/d1f/f47 x:0 0 0 2024-07-16T09:55:33.543 INFO:tasks.workunit.client.1.smithi176.stdout:9/270: chown d5/de/cf 148 1 2024-07-16T09:55:33.543 INFO:tasks.workunit.client.1.smithi176.stdout:3/192: chown d3/da/ld 601 1 2024-07-16T09:55:33.544 INFO:tasks.workunit.client.1.smithi176.stdout:4/290: mknod da/dc/d10/d28/c53 0 2024-07-16T09:55:33.545 INFO:tasks.workunit.client.0.smithi156.stdout:4/218: mknod d4/d41/c4c 0 2024-07-16T09:55:33.549 INFO:tasks.workunit.client.1.smithi176.stdout:3/193: mknod d3/d17/d1d/d1e/c38 0 2024-07-16T09:55:33.549 INFO:tasks.workunit.client.0.smithi156.stdout:4/219: truncate d4/f9 5068473 0 2024-07-16T09:55:33.550 INFO:tasks.workunit.client.0.smithi156.stdout:3/211: link d8/da/d13/l2b d8/da/d28/d40/l46 0 2024-07-16T09:55:33.550 INFO:tasks.workunit.client.1.smithi176.stdout:4/291: write da/d3d/f1b [1521533,32480] 0 2024-07-16T09:55:33.551 INFO:tasks.workunit.client.0.smithi156.stdout:1/211: unlink dc/f18 0 2024-07-16T09:55:33.551 INFO:tasks.workunit.client.0.smithi156.stdout:7/199: dwrite dc/d3e/f4a [0,4194304] 0 2024-07-16T09:55:33.553 INFO:tasks.workunit.client.0.smithi156.stdout:9/273: fdatasync d2/d5/d16/d22/d36/f50 0 2024-07-16T09:55:33.556 INFO:tasks.workunit.client.1.smithi176.stdout:4/292: mkdir da/d3d/d17/d54 0 2024-07-16T09:55:33.556 INFO:tasks.workunit.client.0.smithi156.stdout:3/212: truncate d8/d34/f3a 373443 0 2024-07-16T09:55:33.557 INFO:tasks.workunit.client.1.smithi176.stdout:5/360: dwrite d4/d14/f2c [4194304,4194304] 0 2024-07-16T09:55:33.557 INFO:tasks.workunit.client.1.smithi176.stdout:9/271: dread d5/de/f1a [8388608,4194304] 0 2024-07-16T09:55:33.557 INFO:tasks.workunit.client.0.smithi156.stdout:2/367: chown d0/d5/d29/d2d/d5b/d62/c63 31661917 1 2024-07-16T09:55:33.557 INFO:tasks.workunit.client.0.smithi156.stdout:7/200: fdatasync dc/d13/f1d 0 2024-07-16T09:55:33.557 INFO:tasks.workunit.client.1.smithi176.stdout:4/293: stat da/f1a 0 2024-07-16T09:55:33.558 INFO:tasks.workunit.client.0.smithi156.stdout:0/162: creat d0/f2b x:0 0 0 2024-07-16T09:55:33.561 INFO:tasks.workunit.client.0.smithi156.stdout:5/206: fdatasync d1b/d28/f3c 0 2024-07-16T09:55:33.563 INFO:tasks.workunit.client.1.smithi176.stdout:9/272: fsync d5/f1e 0 2024-07-16T09:55:33.564 INFO:tasks.workunit.client.1.smithi176.stdout:4/294: truncate da/dc/d39/f4f 56046 0 2024-07-16T09:55:33.564 INFO:tasks.workunit.client.0.smithi156.stdout:3/213: read d8/fb [1361379,42765] 0 2024-07-16T09:55:33.565 INFO:tasks.workunit.client.1.smithi176.stdout:5/361: symlink d4/d37/d72/d38/l85 0 2024-07-16T09:55:33.565 INFO:tasks.workunit.client.1.smithi176.stdout:3/194: dwrite d3/d17/f32 [0,4194304] 0 2024-07-16T09:55:33.567 INFO:tasks.workunit.client.0.smithi156.stdout:4/220: dwrite d4/d23/d30/f25 [4194304,4194304] 0 2024-07-16T09:55:33.567 INFO:tasks.workunit.client.0.smithi156.stdout:6/279: dread d3/d13/d17/d4f/d1f/f10 [0,4194304] 0 2024-07-16T09:55:33.567 INFO:tasks.workunit.client.0.smithi156.stdout:7/201: creat dc/d13/d15/f50 x:0 0 0 2024-07-16T09:55:33.567 INFO:tasks.workunit.client.1.smithi176.stdout:3/195: stat l0 0 2024-07-16T09:55:33.569 INFO:tasks.workunit.client.1.smithi176.stdout:9/273: creat d5/d52/f55 x:0 0 0 2024-07-16T09:55:33.570 INFO:tasks.workunit.client.1.smithi176.stdout:3/196: chown d3/d17/c1c 63 1 2024-07-16T09:55:33.570 INFO:tasks.workunit.client.0.smithi156.stdout:0/163: mkdir d0/d7/d9/d27/d2c 0 2024-07-16T09:55:33.574 INFO:tasks.workunit.client.1.smithi176.stdout:9/274: write d5/de/d38/f45 [603591,87258] 0 2024-07-16T09:55:33.575 INFO:tasks.workunit.client.0.smithi156.stdout:8/185: creat d0/d11/f48 x:0 0 0 2024-07-16T09:55:33.577 INFO:tasks.workunit.client.1.smithi176.stdout:9/275: chown d5/d1d/d20 1415 1 2024-07-16T09:55:33.581 INFO:tasks.workunit.client.0.smithi156.stdout:3/214: creat d8/d14/f47 x:0 0 0 2024-07-16T09:55:33.583 INFO:tasks.workunit.client.0.smithi156.stdout:6/280: dwrite d3/d13/d17/d4f/d1f/d19/d31/f4e [0,4194304] 0 2024-07-16T09:55:33.583 INFO:tasks.workunit.client.1.smithi176.stdout:4/295: dwrite da/f42 [0,4194304] 0 2024-07-16T09:55:33.583 INFO:tasks.workunit.client.0.smithi156.stdout:0/164: dwrite d0/d7/d9/f1d [0,4194304] 0 2024-07-16T09:55:33.584 INFO:tasks.workunit.client.0.smithi156.stdout:7/202: dread dc/d13/f1d [0,4194304] 0 2024-07-16T09:55:33.585 INFO:tasks.workunit.client.0.smithi156.stdout:2/368: symlink d0/d13/d17/d1c/l83 0 2024-07-16T09:55:33.586 INFO:tasks.workunit.client.1.smithi176.stdout:5/362: dwrite d4/d10/d17/f67 [0,4194304] 0 2024-07-16T09:55:33.586 INFO:tasks.workunit.client.0.smithi156.stdout:4/221: fdatasync d4/db/f18 0 2024-07-16T09:55:33.586 INFO:tasks.workunit.client.0.smithi156.stdout:4/222: fdatasync d4/d23/f39 0 2024-07-16T09:55:33.588 INFO:tasks.workunit.client.0.smithi156.stdout:4/223: write d4/db/dc/f42 [701543,58481] 0 2024-07-16T09:55:33.589 INFO:tasks.workunit.client.1.smithi176.stdout:4/296: read da/d3d/f1b [3939842,38984] 0 2024-07-16T09:55:33.589 INFO:tasks.workunit.client.0.smithi156.stdout:1/212: rename dc/f1c to dc/d20/f49 0 2024-07-16T09:55:33.591 INFO:tasks.workunit.client.0.smithi156.stdout:5/207: creat d1b/d1f/d42/f47 x:0 0 0 2024-07-16T09:55:33.591 INFO:tasks.workunit.client.0.smithi156.stdout:5/208: fdatasync f12 0 2024-07-16T09:55:33.594 INFO:tasks.workunit.client.1.smithi176.stdout:9/276: dread d5/f2d [0,4194304] 0 2024-07-16T09:55:33.597 INFO:tasks.workunit.client.1.smithi176.stdout:3/197: dwrite d3/da/fe [0,4194304] 0 2024-07-16T09:55:33.597 INFO:tasks.workunit.client.0.smithi156.stdout:0/165: mkdir d0/d7/d9/d2d 0 2024-07-16T09:55:33.599 INFO:tasks.workunit.client.0.smithi156.stdout:7/203: mkdir dc/d3e/d3f/d51 0 2024-07-16T09:55:33.601 INFO:tasks.workunit.client.1.smithi176.stdout:9/277: creat d5/d1d/f56 x:0 0 0 2024-07-16T09:55:33.602 INFO:tasks.workunit.client.1.smithi176.stdout:9/278: chown d5/de/d37 1224611 1 2024-07-16T09:55:33.602 INFO:tasks.workunit.client.1.smithi176.stdout:4/297: rename da/f1a to da/dc/f55 0 2024-07-16T09:55:33.606 INFO:tasks.workunit.client.0.smithi156.stdout:0/166: dwrite d0/d7/f21 [0,4194304] 0 2024-07-16T09:55:33.606 INFO:tasks.workunit.client.1.smithi176.stdout:3/198: creat d3/da/d33/f39 x:0 0 0 2024-07-16T09:55:33.609 INFO:tasks.workunit.client.0.smithi156.stdout:0/167: write d0/f2b [413119,5624] 0 2024-07-16T09:55:33.610 INFO:tasks.workunit.client.1.smithi176.stdout:9/279: unlink d5/d1d/c51 0 2024-07-16T09:55:33.610 INFO:tasks.workunit.client.1.smithi176.stdout:4/298: mknod da/dc/d4b/c56 0 2024-07-16T09:55:33.612 INFO:tasks.workunit.client.0.smithi156.stdout:5/209: creat d1b/d1e/f48 x:0 0 0 2024-07-16T09:55:33.612 INFO:tasks.workunit.client.1.smithi176.stdout:5/363: dwrite d4/d14/d19/d1b/d61/f76 [0,4194304] 0 2024-07-16T09:55:33.614 INFO:tasks.workunit.client.1.smithi176.stdout:3/199: stat d3/d17/d1d/d1e/c30 0 2024-07-16T09:55:33.614 INFO:tasks.workunit.client.1.smithi176.stdout:4/299: write da/dc/f50 [192555,93527] 0 2024-07-16T09:55:33.617 INFO:tasks.workunit.client.0.smithi156.stdout:3/215: symlink d8/da/d21/l48 0 2024-07-16T09:55:33.617 INFO:tasks.workunit.client.0.smithi156.stdout:6/281: mkdir d3/d13/d17/d4f/d5c 0 2024-07-16T09:55:33.618 INFO:tasks.workunit.client.1.smithi176.stdout:3/200: dread - d3/d17/f2f zero size 2024-07-16T09:55:33.618 INFO:tasks.workunit.client.0.smithi156.stdout:2/369: symlink d0/d5/d29/l84 0 2024-07-16T09:55:33.618 INFO:tasks.workunit.client.0.smithi156.stdout:9/274: link d2/de/c11 d2/de/c5a 0 2024-07-16T09:55:33.618 INFO:tasks.workunit.client.0.smithi156.stdout:7/204: creat dc/d48/f52 x:0 0 0 2024-07-16T09:55:33.619 INFO:tasks.workunit.client.1.smithi176.stdout:4/300: symlink da/dc/d10/l57 0 2024-07-16T09:55:33.622 INFO:tasks.workunit.client.0.smithi156.stdout:5/210: mkdir d1b/d28/d49 0 2024-07-16T09:55:33.623 INFO:tasks.workunit.client.1.smithi176.stdout:9/280: rename d5/de/d38/f45 to d5/de/d30/f57 0 2024-07-16T09:55:33.623 INFO:tasks.workunit.client.0.smithi156.stdout:8/186: link d0/c1d d0/d11/d13/c49 0 2024-07-16T09:55:33.624 INFO:tasks.workunit.client.1.smithi176.stdout:4/301: write da/dc/d10/f27 [2140453,104208] 0 2024-07-16T09:55:33.624 INFO:tasks.workunit.client.0.smithi156.stdout:2/370: chown d0/d13/d17/d1c/l39 19815141 1 2024-07-16T09:55:33.627 INFO:tasks.workunit.client.0.smithi156.stdout:4/224: link d4/db/f18 d4/db/dc/f4d 0 2024-07-16T09:55:33.627 INFO:tasks.workunit.client.1.smithi176.stdout:9/281: mkdir d5/de/d37/d58 0 2024-07-16T09:55:33.628 INFO:tasks.workunit.client.0.smithi156.stdout:0/168: dread d0/d7/d9/f12 [0,4194304] 0 2024-07-16T09:55:33.628 INFO:tasks.workunit.client.1.smithi176.stdout:9/282: chown d5/de/d37/d58 2427403 1 2024-07-16T09:55:33.628 INFO:tasks.workunit.client.0.smithi156.stdout:8/187: chown d0/dd/l2b 7577579 1 2024-07-16T09:55:33.628 INFO:tasks.workunit.client.0.smithi156.stdout:2/371: stat d0/d13/d17/f42 0 2024-07-16T09:55:33.629 INFO:tasks.workunit.client.0.smithi156.stdout:3/216: dwrite d8/da/f32 [0,4194304] 0 2024-07-16T09:55:33.631 INFO:tasks.workunit.client.1.smithi176.stdout:9/283: unlink d5/d1d/f56 0 2024-07-16T09:55:33.633 INFO:tasks.workunit.client.0.smithi156.stdout:5/211: rename d1b/d28/c46 to d1b/d28/c4a 0 2024-07-16T09:55:33.633 INFO:tasks.workunit.client.0.smithi156.stdout:4/225: write d4/d23/d30/f46 [380041,59851] 0 2024-07-16T09:55:33.635 INFO:tasks.workunit.client.0.smithi156.stdout:8/188: write d0/d11/f15 [71572,65794] 0 2024-07-16T09:55:33.636 INFO:tasks.workunit.client.0.smithi156.stdout:5/212: write ff [2148072,62044] 0 2024-07-16T09:55:33.636 INFO:tasks.workunit.client.1.smithi176.stdout:9/284: rename d5/d1d/l3b to d5/d1d/d20/l59 0 2024-07-16T09:55:33.636 INFO:tasks.workunit.client.1.smithi176.stdout:9/285: stat d5/d1d/l2e 0 2024-07-16T09:55:33.637 INFO:tasks.workunit.client.0.smithi156.stdout:2/372: creat d0/d13/d17/d1c/f85 x:0 0 0 2024-07-16T09:55:33.639 INFO:tasks.workunit.client.1.smithi176.stdout:9/286: creat d5/de/d38/f5a x:0 0 0 2024-07-16T09:55:33.639 INFO:tasks.workunit.client.1.smithi176.stdout:9/287: chown d5/de/cf 1438 1 2024-07-16T09:55:33.640 INFO:tasks.workunit.client.0.smithi156.stdout:8/189: fsync d0/fb 0 2024-07-16T09:55:33.641 INFO:tasks.workunit.client.0.smithi156.stdout:9/275: dwrite d2/d5/d16/d22/d2b/d3d/d3c/f53 [0,4194304] 0 2024-07-16T09:55:33.641 INFO:tasks.workunit.client.0.smithi156.stdout:2/373: dread - d0/d5/d29/f54 zero size 2024-07-16T09:55:33.643 INFO:tasks.workunit.client.0.smithi156.stdout:3/217: creat d8/da/d1d/d2c/d33/f49 x:0 0 0 2024-07-16T09:55:33.645 INFO:tasks.workunit.client.0.smithi156.stdout:8/190: read d0/f6 [489403,111544] 0 2024-07-16T09:55:33.645 INFO:tasks.workunit.client.1.smithi176.stdout:9/288: creat d5/de/f5b x:0 0 0 2024-07-16T09:55:33.647 INFO:tasks.workunit.client.1.smithi176.stdout:9/289: readlink d5/de/l42 0 2024-07-16T09:55:33.648 INFO:tasks.workunit.client.0.smithi156.stdout:2/374: mkdir d0/d13/d17/d1c/d86 0 2024-07-16T09:55:33.648 INFO:tasks.workunit.client.0.smithi156.stdout:5/213: symlink d1b/l4b 0 2024-07-16T09:55:33.649 INFO:tasks.workunit.client.0.smithi156.stdout:7/205: dwrite dc/f14 [0,4194304] 0 2024-07-16T09:55:33.649 INFO:tasks.workunit.client.0.smithi156.stdout:3/218: readlink d8/da/d28/d40/l46 0 2024-07-16T09:55:33.649 INFO:tasks.workunit.client.0.smithi156.stdout:5/214: chown d1b/d28 1081 1 2024-07-16T09:55:33.651 INFO:tasks.workunit.client.0.smithi156.stdout:3/219: readlink d8/da/d13/l2b 0 2024-07-16T09:55:33.651 INFO:tasks.workunit.client.1.smithi176.stdout:9/290: rename d5/f49 to d5/de/d37/d58/f5c 0 2024-07-16T09:55:33.652 INFO:tasks.workunit.client.0.smithi156.stdout:2/375: write d0/d5/d29/f3c [277848,82915] 0 2024-07-16T09:55:33.652 INFO:tasks.workunit.client.1.smithi176.stdout:9/291: dread - d5/f1e zero size 2024-07-16T09:55:33.652 INFO:tasks.workunit.client.0.smithi156.stdout:9/276: mknod d2/d5/d16/d22/d2b/d3d/d3c/c5b 0 2024-07-16T09:55:33.654 INFO:tasks.workunit.client.1.smithi176.stdout:9/292: write d5/de/d38/f4e [408662,83862] 0 2024-07-16T09:55:33.654 INFO:tasks.workunit.client.0.smithi156.stdout:8/191: mkdir d0/d11/d4a 0 2024-07-16T09:55:33.655 INFO:tasks.workunit.client.1.smithi176.stdout:9/293: write d5/d1d/f3c [822405,60171] 0 2024-07-16T09:55:33.656 INFO:tasks.workunit.client.0.smithi156.stdout:5/215: write f13 [4801113,109766] 0 2024-07-16T09:55:33.657 INFO:tasks.workunit.client.0.smithi156.stdout:2/376: mknod d0/d13/d6e/c87 0 2024-07-16T09:55:33.657 INFO:tasks.workunit.client.0.smithi156.stdout:5/216: readlink d1b/l26 0 2024-07-16T09:55:33.657 INFO:tasks.workunit.client.0.smithi156.stdout:2/377: stat d0/d13/f1a 0 2024-07-16T09:55:33.659 INFO:tasks.workunit.client.0.smithi156.stdout:2/378: write d0/d13/d17/d1c/f71 [159332,19388] 0 2024-07-16T09:55:33.659 INFO:tasks.workunit.client.0.smithi156.stdout:7/206: link dc/d13/d20/f3b dc/d3e/d3f/d51/f53 0 2024-07-16T09:55:33.659 INFO:tasks.workunit.client.0.smithi156.stdout:2/379: readlink d0/l6 0 2024-07-16T09:55:33.660 INFO:tasks.workunit.client.0.smithi156.stdout:2/380: chown d0/d5/d29/d2d/l41 120737883 1 2024-07-16T09:55:33.662 INFO:tasks.workunit.client.0.smithi156.stdout:7/207: mknod dc/d48/c54 0 2024-07-16T09:55:33.663 INFO:tasks.workunit.client.0.smithi156.stdout:8/192: mknod d0/d11/d4a/c4b 0 2024-07-16T09:55:33.663 INFO:tasks.workunit.client.0.smithi156.stdout:5/217: mknod d1b/d1e/d37/c4c 0 2024-07-16T09:55:33.664 INFO:tasks.workunit.client.0.smithi156.stdout:8/193: dread - d0/dd/d1f/f47 zero size 2024-07-16T09:55:33.665 INFO:tasks.workunit.client.0.smithi156.stdout:3/220: rename d8/da/d21/l48 to d8/da/d1d/l4a 0 2024-07-16T09:55:33.667 INFO:tasks.workunit.client.0.smithi156.stdout:0/169: write d0/d7/d9/f1d [4986992,101932] 0 2024-07-16T09:55:33.667 INFO:tasks.workunit.client.0.smithi156.stdout:9/277: rename d2/d5 to d2/d5/d16/d22/d36/d33/d23/d5c 22 2024-07-16T09:55:33.667 INFO:tasks.workunit.client.0.smithi156.stdout:7/208: chown dc/d13/d15/l2a 11434 1 2024-07-16T09:55:33.667 INFO:tasks.workunit.client.0.smithi156.stdout:8/194: write d0/dd/f26 [889068,74276] 0 2024-07-16T09:55:33.669 INFO:tasks.workunit.client.0.smithi156.stdout:5/218: mkdir d1b/d28/d49/d4d 0 2024-07-16T09:55:33.671 INFO:tasks.workunit.client.0.smithi156.stdout:9/278: symlink d2/de/l5d 0 2024-07-16T09:55:33.671 INFO:tasks.workunit.client.0.smithi156.stdout:3/221: truncate d8/d14/f47 758720 0 2024-07-16T09:55:33.673 INFO:tasks.workunit.client.0.smithi156.stdout:5/219: write fc [750744,89980] 0 2024-07-16T09:55:33.673 INFO:tasks.workunit.client.0.smithi156.stdout:0/170: dread d0/f14 [0,4194304] 0 2024-07-16T09:55:33.675 INFO:tasks.workunit.client.0.smithi156.stdout:2/381: dwrite d0/d13/d17/f34 [0,4194304] 0 2024-07-16T09:55:33.678 INFO:tasks.workunit.client.0.smithi156.stdout:9/279: creat d2/de/f5e x:0 0 0 2024-07-16T09:55:33.679 INFO:tasks.workunit.client.1.smithi176.stdout:5/364: truncate d4/d10/d17/f67 326997 0 2024-07-16T09:55:33.681 INFO:tasks.workunit.client.1.smithi176.stdout:5/365: read d4/d10/f5f [26105,61981] 0 2024-07-16T09:55:33.681 INFO:tasks.workunit.client.0.smithi156.stdout:5/220: creat d1b/d1e/f4e x:0 0 0 2024-07-16T09:55:33.684 INFO:tasks.workunit.client.0.smithi156.stdout:0/171: creat d0/d7/f2e x:0 0 0 2024-07-16T09:55:33.685 INFO:tasks.workunit.client.0.smithi156.stdout:2/382: write d0/d13/d17/f34 [534094,53622] 0 2024-07-16T09:55:33.686 INFO:tasks.workunit.client.0.smithi156.stdout:1/213: truncate dc/d24/f2a 1096956 0 2024-07-16T09:55:33.686 INFO:tasks.workunit.client.0.smithi156.stdout:7/209: dwrite dc/d13/f12 [0,4194304] 0 2024-07-16T09:55:33.687 INFO:tasks.workunit.client.1.smithi176.stdout:5/366: creat d4/f86 x:0 0 0 2024-07-16T09:55:33.688 INFO:tasks.workunit.client.1.smithi176.stdout:9/294: truncate d5/d1d/f46 1497784 0 2024-07-16T09:55:33.689 INFO:tasks.workunit.client.0.smithi156.stdout:4/226: dread d4/db/f18 [0,4194304] 0 2024-07-16T09:55:33.690 INFO:tasks.workunit.client.0.smithi156.stdout:5/221: symlink d1b/d1e/l4f 0 2024-07-16T09:55:33.691 INFO:tasks.workunit.client.0.smithi156.stdout:4/227: fdatasync d4/f43 0 2024-07-16T09:55:33.691 INFO:tasks.workunit.client.1.smithi176.stdout:5/367: rmdir d4/d37/d72/d82 39 2024-07-16T09:55:33.692 INFO:tasks.workunit.client.0.smithi156.stdout:7/210: truncate dc/d13/d15/f50 62747 0 2024-07-16T09:55:33.692 INFO:tasks.workunit.client.1.smithi176.stdout:9/295: symlink d5/de/d37/d58/l5d 0 2024-07-16T09:55:33.692 INFO:tasks.workunit.client.0.smithi156.stdout:2/383: write d0/d13/d17/f42 [844621,66380] 0 2024-07-16T09:55:33.693 INFO:tasks.workunit.client.1.smithi176.stdout:9/296: chown d5/de/f40 20100207 1 2024-07-16T09:55:33.693 INFO:tasks.workunit.client.0.smithi156.stdout:9/280: dread d2/f4 [0,4194304] 0 2024-07-16T09:55:33.696 INFO:tasks.workunit.client.0.smithi156.stdout:9/281: dread - d2/d5/f44 zero size 2024-07-16T09:55:33.696 INFO:tasks.workunit.client.0.smithi156.stdout:3/222: link d8/da/d1d/d2c/d33/f49 d8/da/d28/d43/f4b 0 2024-07-16T09:55:33.697 INFO:tasks.workunit.client.0.smithi156.stdout:3/223: chown d8/da/f32 79491 1 2024-07-16T09:55:33.697 INFO:tasks.workunit.client.1.smithi176.stdout:5/368: link d4/f71 d4/d37/d72/d38/f87 0 2024-07-16T09:55:33.700 INFO:tasks.workunit.client.1.smithi176.stdout:9/297: rename f2 to d5/de/d30/f5e 0 2024-07-16T09:55:33.700 INFO:tasks.workunit.client.1.smithi176.stdout:5/369: mkdir d4/d37/d88 0 2024-07-16T09:55:33.701 INFO:tasks.workunit.client.1.smithi176.stdout:9/298: write d5/f1e [400642,33053] 0 2024-07-16T09:55:33.702 INFO:tasks.workunit.client.0.smithi156.stdout:6/282: truncate d3/d13/d17/d4f/d1f/d19/d31/f4e 3854084 0 2024-07-16T09:55:33.702 INFO:tasks.workunit.client.0.smithi156.stdout:0/172: symlink d0/d7/d9/d27/l2f 0 2024-07-16T09:55:33.702 INFO:tasks.workunit.client.0.smithi156.stdout:7/211: truncate dc/d13/f1d 4931135 0 2024-07-16T09:55:33.704 INFO:tasks.workunit.client.0.smithi156.stdout:6/283: readlink d3/d13/d17/d4f/d1f/d19/l2f 0 2024-07-16T09:55:33.705 INFO:tasks.workunit.client.1.smithi176.stdout:9/299: rmdir d5/de/d37/d58 39 2024-07-16T09:55:33.707 INFO:tasks.workunit.client.0.smithi156.stdout:2/384: symlink d0/d5/d76/l88 0 2024-07-16T09:55:33.707 INFO:tasks.workunit.client.1.smithi176.stdout:9/300: chown d5/de/d38/l4b 521636 1 2024-07-16T09:55:33.708 INFO:tasks.workunit.client.0.smithi156.stdout:6/284: chown d3/d13/d17/d4f/d1e/l4c 202511673 1 2024-07-16T09:55:33.710 INFO:tasks.workunit.client.0.smithi156.stdout:1/214: truncate f4 956234 0 2024-07-16T09:55:33.711 INFO:tasks.workunit.client.1.smithi176.stdout:5/370: dwrite d4/d37/f3f [0,4194304] 0 2024-07-16T09:55:33.711 INFO:tasks.workunit.client.0.smithi156.stdout:6/285: readlink d3/d13/d17/l1b 0 2024-07-16T09:55:33.711 INFO:tasks.workunit.client.0.smithi156.stdout:1/215: chown dc/d14/d3e/f43 61684 1 2024-07-16T09:55:33.714 INFO:tasks.workunit.client.0.smithi156.stdout:5/222: dread d1b/d28/f39 [0,4194304] 0 2024-07-16T09:55:33.714 INFO:tasks.workunit.client.0.smithi156.stdout:7/212: rename dc/d13/d20/c39 to dc/d3e/d3f/c55 0 2024-07-16T09:55:33.714 INFO:tasks.workunit.client.1.smithi176.stdout:9/301: write d5/de/d37/d58/f5c [503703,8570] 0 2024-07-16T09:55:33.714 INFO:tasks.workunit.client.0.smithi156.stdout:8/195: dread d0/fb [0,4194304] 0 2024-07-16T09:55:33.715 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:33.713+0000 7ff337eac640 1 -- 172.21.15.156:0/3443142147 --> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff33010ee80 con 0x7ff31c06a5a0 2024-07-16T09:55:33.716 INFO:tasks.workunit.client.0.smithi156.stdout:5/223: write d1b/d28/f35 [1676875,111278] 0 2024-07-16T09:55:33.716 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:33.715+0000 7ff3267fc640 1 -- 172.21.15.156:0/3443142147 <== mgr.14208 v2:172.21.15.156:6800/2615741717 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+349 (secure 0 0 0) 0x7ff33010ee80 con 0x7ff31c06a5a0 2024-07-16T09:55:33.718 INFO:tasks.workunit.client.0.smithi156.stdout:0/173: symlink d0/d7/d25/l30 0 2024-07-16T09:55:33.719 INFO:tasks.workunit.client.0.smithi156.stdout:9/282: dread d2/d5/d16/d22/d36/d33/d3a/f3b [0,4194304] 0 2024-07-16T09:55:33.721 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:33.718+0000 7ff337eac640 1 -- 172.21.15.156:0/3443142147 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7ff31c06a5a0 msgr2=0x7ff31c06ca50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:55:33.722 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:33.718+0000 7ff337eac640 1 --2- 172.21.15.156:0/3443142147 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7ff31c06a5a0 0x7ff31c06ca50 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7ff3200047e0 tx=0x7ff320035040 comp rx=0 tx=0).stop 2024-07-16T09:55:33.722 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:33.719+0000 7ff337eac640 1 -- 172.21.15.156:0/3443142147 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff3300fd980 msgr2=0x7ff33010cf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:55:33.722 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:33.719+0000 7ff337eac640 1 --2- 172.21.15.156:0/3443142147 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff3300fd980 0x7ff33010cf50 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7ff32c00eb40 tx=0x7ff32c00b040 comp rx=0 tx=0).stop 2024-07-16T09:55:33.722 INFO:tasks.workunit.client.0.smithi156.stdout:8/196: chown d0/d11/d20/f25 1879776 1 2024-07-16T09:55:33.722 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:33.720+0000 7ff337eac640 1 -- 172.21.15.156:0/3443142147 shutdown_connections 2024-07-16T09:55:33.723 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:33.720+0000 7ff337eac640 1 --2- 172.21.15.156:0/3443142147 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7ff31c06a5a0 0x7ff31c06ca50 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:55:33.723 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:33.720+0000 7ff337eac640 1 --2- 172.21.15.156:0/3443142147 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff3300fe2d0 0x7ff33010d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:55:33.723 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:33.720+0000 7ff337eac640 1 --2- 172.21.15.156:0/3443142147 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff3300fd980 0x7ff33010cf50 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:55:33.723 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:33.720+0000 7ff337eac640 1 -- 172.21.15.156:0/3443142147 >> 172.21.15.156:0/3443142147 conn(0x7ff3300f61a0 msgr2=0x7ff3300f7850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:55:33.723 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:33.720+0000 7ff337eac640 1 -- 172.21.15.156:0/3443142147 shutdown_connections 2024-07-16T09:55:33.723 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:33.720+0000 7ff337eac640 1 -- 172.21.15.156:0/3443142147 wait complete. 2024-07-16T09:55:33.723 INFO:tasks.workunit.client.0.smithi156.stdout:2/385: mkdir d0/d5/d29/d2d/d5b/d5f/d89 0 2024-07-16T09:55:33.724 INFO:tasks.workunit.client.0.smithi156.stdout:0/174: write d0/d7/f21 [2696546,124865] 0 2024-07-16T09:55:33.724 INFO:tasks.workunit.client.1.smithi176.stdout:9/302: dread d5/de/d30/f57 [0,4194304] 0 2024-07-16T09:55:33.725 INFO:tasks.workunit.client.0.smithi156.stdout:6/286: creat d3/d13/d17/d4f/d4d/d3b/f5d x:0 0 0 2024-07-16T09:55:33.728 INFO:tasks.workunit.client.0.smithi156.stdout:7/213: dwrite dc/f14 [0,4194304] 0 2024-07-16T09:55:33.730 INFO:tasks.workunit.client.0.smithi156.stdout:5/224: unlink d1b/d1e/l2b 0 2024-07-16T09:55:33.733 INFO:tasks.workunit.client.0.smithi156.stdout:8/197: creat d0/dd/f4c x:0 0 0 2024-07-16T09:55:33.735 INFO:tasks.workunit.client.1.smithi176.stdout:9/303: fdatasync d5/d1d/f46 0 2024-07-16T09:55:33.736 INFO:tasks.workunit.client.1.smithi176.stdout:5/371: dread d4/d10/d17/f50 [0,4194304] 0 2024-07-16T09:55:33.737 INFO:tasks.workunit.client.0.smithi156.stdout:9/283: unlink d2/d5/d16/d22/d2b/d3d/d3c/f4b 0 2024-07-16T09:55:33.737 INFO:tasks.workunit.client.1.smithi176.stdout:9/304: creat d5/de/d21/f5f x:0 0 0 2024-07-16T09:55:33.738 INFO:tasks.workunit.client.0.smithi156.stdout:9/284: write d2/de/f19 [810988,22963] 0 2024-07-16T09:55:33.739 INFO:tasks.workunit.client.1.smithi176.stdout:5/372: mknod d4/d37/d72/c89 0 2024-07-16T09:55:33.740 INFO:tasks.workunit.client.0.smithi156.stdout:9/285: stat d2/d5/d16/d22/d36/d33/d3a/f49 0 2024-07-16T09:55:33.741 INFO:tasks.workunit.client.1.smithi176.stdout:5/373: rmdir d4/d14 39 2024-07-16T09:55:33.742 INFO:tasks.workunit.client.0.smithi156.stdout:9/286: readlink d2/d5/l9 0 2024-07-16T09:55:33.742 INFO:tasks.workunit.client.0.smithi156.stdout:2/386: unlink d0/d5/d29/d2d/d5b/d62/f77 0 2024-07-16T09:55:33.742 INFO:tasks.workunit.client.1.smithi176.stdout:5/374: write d4/f27 [1689361,50973] 0 2024-07-16T09:55:33.745 INFO:tasks.workunit.client.1.smithi176.stdout:9/305: getdents d5 0 2024-07-16T09:55:33.746 INFO:tasks.workunit.client.0.smithi156.stdout:7/214: mkdir dc/d13/d36/d56 0 2024-07-16T09:55:33.746 INFO:tasks.workunit.client.1.smithi176.stdout:9/306: fdatasync d5/de/d38/f5a 0 2024-07-16T09:55:33.747 INFO:tasks.workunit.client.0.smithi156.stdout:7/215: write dc/d13/d36/f4c [742521,82215] 0 2024-07-16T09:55:33.748 INFO:tasks.workunit.client.0.smithi156.stdout:5/225: unlink f13 0 2024-07-16T09:55:33.749 INFO:tasks.workunit.client.0.smithi156.stdout:5/226: write d1b/d1e/f3d [997951,23599] 0 2024-07-16T09:55:33.749 INFO:tasks.workunit.client.0.smithi156.stdout:1/216: creat dc/f4a x:0 0 0 2024-07-16T09:55:33.750 INFO:tasks.workunit.client.1.smithi176.stdout:5/375: dwrite d4/d37/f3f [0,4194304] 0 2024-07-16T09:55:33.751 INFO:tasks.workunit.client.0.smithi156.stdout:1/217: chown dc/d20/c31 96 1 2024-07-16T09:55:33.751 INFO:tasks.workunit.client.0.smithi156.stdout:4/228: truncate d4/d23/d30/f25 914102 0 2024-07-16T09:55:33.753 INFO:tasks.workunit.client.1.smithi176.stdout:5/376: fsync d4/d6c/f6f 0 2024-07-16T09:55:33.754 INFO:tasks.workunit.client.0.smithi156.stdout:5/227: dread d1b/d28/f35 [0,4194304] 0 2024-07-16T09:55:33.755 INFO:tasks.workunit.client.1.smithi176.stdout:9/307: dwrite d5/d1d/d20/f25 [0,4194304] 0 2024-07-16T09:55:33.757 INFO:tasks.workunit.client.1.smithi176.stdout:5/377: creat d4/d10/d17/f8a x:0 0 0 2024-07-16T09:55:33.758 INFO:tasks.workunit.client.0.smithi156.stdout:4/229: dread d4/db/f18 [0,4194304] 0 2024-07-16T09:55:33.759 INFO:tasks.workunit.client.1.smithi176.stdout:9/308: creat d5/de/d37/d58/f60 x:0 0 0 2024-07-16T09:55:33.760 INFO:tasks.workunit.client.1.smithi176.stdout:9/309: chown d5/de/c28 59980 1 2024-07-16T09:55:33.760 INFO:tasks.workunit.client.1.smithi176.stdout:5/378: symlink d4/d37/d72/d38/d6e/l8b 0 2024-07-16T09:55:33.762 INFO:tasks.workunit.client.0.smithi156.stdout:6/287: rename d3/d13/d17/d4f/d30/l4a to d3/d13/d17/d4f/d1e/l5e 0 2024-07-16T09:55:33.764 INFO:tasks.workunit.client.0.smithi156.stdout:7/216: fsync dc/d13/f35 0 2024-07-16T09:55:33.764 INFO:tasks.workunit.client.1.smithi176.stdout:9/310: creat d5/de/f61 x:0 0 0 2024-07-16T09:55:33.765 INFO:tasks.workunit.client.1.smithi176.stdout:9/311: stat d5/d1d/d20/f25 0 2024-07-16T09:55:33.768 INFO:tasks.workunit.client.1.smithi176.stdout:9/312: symlink d5/d52/l62 0 2024-07-16T09:55:33.769 INFO:tasks.workunit.client.0.smithi156.stdout:4/230: dwrite d4/f9 [0,4194304] 0 2024-07-16T09:55:33.769 INFO:tasks.workunit.client.0.smithi156.stdout:1/218: write dc/f13 [457558,76364] 0 2024-07-16T09:55:33.769 INFO:tasks.workunit.client.0.smithi156.stdout:8/198: rename d0/d11/l41 to d0/d11/d13/d1c/d45/l4d 0 2024-07-16T09:55:33.770 INFO:tasks.workunit.client.1.smithi176.stdout:9/313: symlink d5/de/d30/l63 0 2024-07-16T09:55:33.773 INFO:tasks.workunit.client.0.smithi156.stdout:8/199: write d0/dd/f43 [599197,47711] 0 2024-07-16T09:55:33.774 INFO:tasks.workunit.client.0.smithi156.stdout:0/175: dwrite d0/fa [0,4194304] 0 2024-07-16T09:55:33.774 INFO:tasks.workunit.client.0.smithi156.stdout:5/228: link d1b/d1e/f3d d1b/d1f/d45/f50 0 2024-07-16T09:55:33.774 INFO:tasks.workunit.client.1.smithi176.stdout:9/314: dread d5/de/f1a [8388608,4194304] 0 2024-07-16T09:55:33.775 INFO:tasks.workunit.client.0.smithi156.stdout:5/229: chown d1b/d1f 4 1 2024-07-16T09:55:33.776 INFO:tasks.workunit.client.1.smithi176.stdout:9/315: write d5/de/d21/f5f [403831,59821] 0 2024-07-16T09:55:33.776 INFO:tasks.workunit.client.0.smithi156.stdout:6/288: mkdir d3/d13/d17/d4f/d30/d45/d5f 0 2024-07-16T09:55:33.777 INFO:tasks.workunit.client.0.smithi156.stdout:8/200: chown d0/d11/d13/d1c/d24/f36 91856 1 2024-07-16T09:55:33.779 INFO:tasks.workunit.client.0.smithi156.stdout:8/201: write d0/dd/f10 [6570482,17068] 0 2024-07-16T09:55:33.781 INFO:tasks.workunit.client.0.smithi156.stdout:9/287: rename d2 to d2/d5/d16/d22/d5f 22 2024-07-16T09:55:33.781 INFO:tasks.workunit.client.0.smithi156.stdout:8/202: truncate d0/d11/d13/d1c/d24/f42 199639 0 2024-07-16T09:55:33.784 INFO:tasks.workunit.client.0.smithi156.stdout:9/288: dread d2/f4 [0,4194304] 0 2024-07-16T09:55:33.791 INFO:tasks.workunit.client.0.smithi156.stdout:5/230: creat d1b/d1f/f51 x:0 0 0 2024-07-16T09:55:33.793 INFO:tasks.workunit.client.1.smithi176.stdout:9/316: dwrite d5/f10 [0,4194304] 0 2024-07-16T09:55:33.796 INFO:tasks.workunit.client.1.smithi176.stdout:9/317: mknod d5/d43/c64 0 2024-07-16T09:55:33.797 INFO:tasks.workunit.client.0.smithi156.stdout:9/289: rmdir d2/d5/d16/d22/d36/d33 39 2024-07-16T09:55:33.798 INFO:tasks.workunit.client.1.smithi176.stdout:9/318: mknod d5/de/d38/c65 0 2024-07-16T09:55:33.802 INFO:tasks.workunit.client.0.smithi156.stdout:7/217: rename dc/l16 to dc/d13/d15/l57 0 2024-07-16T09:55:33.803 INFO:tasks.workunit.client.0.smithi156.stdout:8/203: creat d0/d11/d20/d3a/f4e x:0 0 0 2024-07-16T09:55:33.803 INFO:tasks.workunit.client.0.smithi156.stdout:2/387: dwrite d0/d5/d29/f32 [0,4194304] 0 2024-07-16T09:55:33.804 INFO:tasks.workunit.client.0.smithi156.stdout:8/204: readlink d0/d11/d13/d1c/l37 0 2024-07-16T09:55:33.804 INFO:tasks.workunit.client.0.smithi156.stdout:2/388: read d0/f14 [1333506,37881] 0 2024-07-16T09:55:33.805 INFO:tasks.workunit.client.0.smithi156.stdout:4/231: creat d4/db/dc/f4e x:0 0 0 2024-07-16T09:55:33.806 INFO:tasks.workunit.client.0.smithi156.stdout:2/389: write d0/d13/f50 [776655,78656] 0 2024-07-16T09:55:33.806 INFO:tasks.workunit.client.0.smithi156.stdout:8/205: write d0/d11/d20/f25 [3332213,1507] 0 2024-07-16T09:55:33.808 INFO:tasks.workunit.client.0.smithi156.stdout:0/176: write d0/f14 [1428793,70253] 0 2024-07-16T09:55:33.809 INFO:tasks.workunit.client.0.smithi156.stdout:0/177: fdatasync d0/d7/f11 0 2024-07-16T09:55:33.810 INFO:tasks.workunit.client.0.smithi156.stdout:6/289: getdents d3/d13/d17/d4f/d1e 0 2024-07-16T09:55:33.812 INFO:tasks.workunit.client.0.smithi156.stdout:0/178: write d0/f14 [1281124,113305] 0 2024-07-16T09:55:33.816 INFO:tasks.workunit.client.0.smithi156.stdout:0/179: dread - d0/d7/d9/f26 zero size 2024-07-16T09:55:33.816 INFO:tasks.workunit.client.0.smithi156.stdout:4/232: symlink d4/db/dc/l4f 0 2024-07-16T09:55:33.817 INFO:tasks.workunit.client.0.smithi156.stdout:1/219: rename dc/d20/f30 to dc/d14/f4b 0 2024-07-16T09:55:33.820 INFO:tasks.workunit.client.0.smithi156.stdout:4/233: fdatasync d4/db/f1d 0 2024-07-16T09:55:33.821 INFO:tasks.workunit.client.0.smithi156.stdout:6/290: creat d3/d13/d17/d56/f60 x:0 0 0 2024-07-16T09:55:33.821 INFO:tasks.workunit.client.1.smithi176.stdout:9/319: dread d5/de/d30/f5e [0,4194304] 0 2024-07-16T09:55:33.822 INFO:tasks.workunit.client.0.smithi156.stdout:7/218: dwrite dc/f42 [0,4194304] 0 2024-07-16T09:55:33.823 INFO:tasks.workunit.client.1.smithi176.stdout:9/320: write d5/f2d [20191,89643] 0 2024-07-16T09:55:33.824 INFO:tasks.workunit.client.0.smithi156.stdout:0/180: symlink d0/d22/l31 0 2024-07-16T09:55:33.824 INFO:tasks.workunit.client.1.smithi176.stdout:9/321: chown d5/de/c28 412009 1 2024-07-16T09:55:33.825 INFO:tasks.workunit.client.0.smithi156.stdout:9/290: dwrite d2/d5/d16/d22/d36/d33/d3a/f3b [0,4194304] 0 2024-07-16T09:55:33.826 INFO:tasks.workunit.client.1.smithi176.stdout:9/322: symlink d5/de/d21/l66 0 2024-07-16T09:55:33.827 INFO:tasks.workunit.client.0.smithi156.stdout:5/231: rename d1b/d28 to d1b/d1f/d45/d52 0 2024-07-16T09:55:33.828 INFO:tasks.workunit.client.0.smithi156.stdout:9/291: chown d2/d5/d16/d22/d36/d33/d23/f2e 482 1 2024-07-16T09:55:33.833 INFO:tasks.workunit.client.1.smithi176.stdout:2/262: sync 2024-07-16T09:55:33.835 INFO:tasks.workunit.client.1.smithi176.stdout:9/323: dwrite d5/de/d37/d58/f5c [0,4194304] 0 2024-07-16T09:55:33.836 INFO:tasks.workunit.client.1.smithi176.stdout:2/263: stat dd/de/c13 0 2024-07-16T09:55:33.837 INFO:tasks.workunit.client.1.smithi176.stdout:9/324: fsync d5/de/f34 0 2024-07-16T09:55:33.837 INFO:tasks.workunit.client.1.smithi176.stdout:2/264: fdatasync dd/d16/d39/f4f 0 2024-07-16T09:55:33.839 INFO:tasks.workunit.client.0.smithi156.stdout:1/220: mkdir dc/d14/d4c 0 2024-07-16T09:55:33.840 INFO:tasks.workunit.client.0.smithi156.stdout:1/221: chown dc/d14 1404 1 2024-07-16T09:55:33.843 INFO:tasks.workunit.client.0.smithi156.stdout:8/206: write d0/f2c [1569341,112375] 0 2024-07-16T09:55:33.846 INFO:tasks.workunit.client.1.smithi176.stdout:2/265: dwrite dd/d2e/d35/f47 [0,4194304] 0 2024-07-16T09:55:33.846 INFO:tasks.workunit.client.0.smithi156.stdout:2/390: rename d0/d13/d17/d1c/f69 to d0/d5/d76/f8a 0 2024-07-16T09:55:33.847 INFO:tasks.workunit.client.1.smithi176.stdout:2/266: readlink dd/de/l28 0 2024-07-16T09:55:33.849 INFO:tasks.workunit.client.0.smithi156.stdout:8/207: dwrite d0/fb [0,4194304] 0 2024-07-16T09:55:33.849 INFO:tasks.workunit.client.0.smithi156.stdout:2/391: write d0/d13/d17/f34 [303246,5898] 0 2024-07-16T09:55:33.849 INFO:tasks.workunit.client.0.smithi156.stdout:9/292: creat d2/d5/d16/d22/d36/d33/d23/f60 x:0 0 0 2024-07-16T09:55:33.850 INFO:tasks.workunit.client.1.smithi176.stdout:2/267: creat dd/d16/f56 x:0 0 0 2024-07-16T09:55:33.850 INFO:tasks.workunit.client.0.smithi156.stdout:2/392: fsync d0/d5/f44 0 2024-07-16T09:55:33.851 INFO:tasks.workunit.client.0.smithi156.stdout:8/208: write d0/d11/f40 [4751409,27282] 0 2024-07-16T09:55:33.852 INFO:tasks.workunit.client.1.smithi176.stdout:2/268: mknod dd/d16/d39/d3d/c57 0 2024-07-16T09:55:33.854 INFO:tasks.workunit.client.1.smithi176.stdout:2/269: truncate dd/d2e/d35/f3f 5560497 0 2024-07-16T09:55:33.856 INFO:tasks.workunit.client.0.smithi156.stdout:8/209: chown d0/dd/f3c 3389 1 2024-07-16T09:55:33.856 INFO:tasks.workunit.client.0.smithi156.stdout:0/181: creat d0/d7/d9/d27/d2c/f32 x:0 0 0 2024-07-16T09:55:33.857 INFO:tasks.workunit.client.1.smithi176.stdout:2/270: creat dd/de/f58 x:0 0 0 2024-07-16T09:55:33.858 INFO:tasks.workunit.client.0.smithi156.stdout:5/232: rename d1b/d1f/d45/d52/d2d/f2e to d1b/d1f/d45/d52/d49/f53 0 2024-07-16T09:55:33.861 INFO:tasks.workunit.client.0.smithi156.stdout:2/393: dwrite d0/d13/d17/f42 [0,4194304] 0 2024-07-16T09:55:33.862 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:33.861+0000 7f77100ca640 1 -- 172.21.15.156:0/2904443000 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7708071380 msgr2=0x7f770806f8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:55:33.862 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:33.861+0000 7f77100ca640 1 --2- 172.21.15.156:0/2904443000 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7708071380 0x7f770806f8b0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f77040048d0 tx=0x7f7704009da0 comp rx=0 tx=0).stop 2024-07-16T09:55:33.863 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:33.861+0000 7f77100ca640 1 -- 172.21.15.156:0/2904443000 shutdown_connections 2024-07-16T09:55:33.863 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:33.861+0000 7f77100ca640 1 --2- 172.21.15.156:0/2904443000 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f770806fdf0 0x7f7708070260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:55:33.863 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:33.861+0000 7f77100ca640 1 --2- 172.21.15.156:0/2904443000 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7708071380 0x7f770806f8b0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:55:33.863 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:33.861+0000 7f77100ca640 1 -- 172.21.15.156:0/2904443000 >> 172.21.15.156:0/2904443000 conn(0x7f77080f7b40 msgr2=0x7f77080f9f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:55:33.863 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:33.862+0000 7f77100ca640 1 -- 172.21.15.156:0/2904443000 shutdown_connections 2024-07-16T09:55:33.863 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:33.862+0000 7f77100ca640 1 -- 172.21.15.156:0/2904443000 wait complete. 2024-07-16T09:55:33.864 INFO:tasks.workunit.client.0.smithi156.stdout:9/293: mkdir d2/d5/d61 0 2024-07-16T09:55:33.864 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:33.862+0000 7f77100ca640 1 Processor -- start 2024-07-16T09:55:33.864 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:33.862+0000 7f77100ca640 1 -- start start 2024-07-16T09:55:33.865 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:33.863+0000 7f77100ca640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f770806fdf0 0x7f7708118150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:55:33.865 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:33.863+0000 7f77100ca640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7708071380 0x7f7708118690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:55:33.865 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:33.863+0000 7f77100ca640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7708118c60 con 0x7f770806fdf0 2024-07-16T09:55:33.865 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:33.863+0000 7f77100ca640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7708118da0 con 0x7f7708071380 2024-07-16T09:55:33.865 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:33.863+0000 7f770de66640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f770806fdf0 0x7f7708118150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:55:33.865 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:33.863+0000 7f770de66640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f770806fdf0 0x7f7708118150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:40876/0 (socket says 172.21.15.156:40876) 2024-07-16T09:55:33.865 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:33.863+0000 7f770de66640 1 -- 172.21.15.156:0/3747026169 learned_addr learned my addr 172.21.15.156:0/3747026169 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T09:55:33.865 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:33.863+0000 7f770d665640 1 --2- 172.21.15.156:0/3747026169 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7708071380 0x7f7708118690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:55:33.866 INFO:tasks.workunit.client.1.smithi176.stdout:2/271: dwrite f3 [4194304,4194304] 0 2024-07-16T09:55:33.866 INFO:tasks.workunit.client.0.smithi156.stdout:9/294: write d2/de/f5e [923036,2584] 0 2024-07-16T09:55:33.866 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:33.864+0000 7f770d665640 1 -- 172.21.15.156:0/3747026169 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f770806fdf0 msgr2=0x7f7708118150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:55:33.866 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:33.864+0000 7f770d665640 1 --2- 172.21.15.156:0/3747026169 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f770806fdf0 0x7f7708118150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:55:33.866 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:33.864+0000 7f770d665640 1 -- 172.21.15.156:0/3747026169 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7704005040 con 0x7f7708071380 2024-07-16T09:55:33.866 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:33.864+0000 7f770d665640 1 --2- 172.21.15.156:0/3747026169 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7708071380 0x7f7708118690 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f76f800b4e0 tx=0x7f76f8009120 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:55:33.867 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:33.865+0000 7f76fe7fc640 1 -- 172.21.15.156:0/3747026169 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f76f800f6f0 con 0x7f7708071380 2024-07-16T09:55:33.867 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:33.865+0000 7f76fe7fc640 1 -- 172.21.15.156:0/3747026169 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f76f800fd20 con 0x7f7708071380 2024-07-16T09:55:33.867 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:33.865+0000 7f76fe7fc640 1 -- 172.21.15.156:0/3747026169 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f76f80103a0 con 0x7f7708071380 2024-07-16T09:55:33.867 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:33.865+0000 7f77100ca640 1 -- 172.21.15.156:0/3747026169 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7708119080 con 0x7f7708071380 2024-07-16T09:55:33.868 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:33.866+0000 7f77100ca640 1 -- 172.21.15.156:0/3747026169 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7708119540 con 0x7f7708071380 2024-07-16T09:55:33.868 INFO:tasks.workunit.client.0.smithi156.stdout:1/222: link dc/f4a dc/d20/f4d 0 2024-07-16T09:55:33.869 INFO:tasks.workunit.client.0.smithi156.stdout:6/291: write d3/d13/d17/d4f/d1f/fa [3762742,71668] 0 2024-07-16T09:55:33.869 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:33.867+0000 7f77100ca640 1 -- 172.21.15.156:0/3747026169 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77080fb7c0 con 0x7f7708071380 2024-07-16T09:55:33.870 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:33.869+0000 7f76fe7fc640 1 -- 172.21.15.156:0/3747026169 <== mon.1 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 20) v1 ==== 88539+0+0 (secure 0 0 0) 0x7f76f800f850 con 0x7f7708071380 2024-07-16T09:55:33.871 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:33.869+0000 7f76fe7fc640 1 --2- 172.21.15.156:0/3747026169 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f76f406a730 0x7f76f406cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:55:33.871 INFO:tasks.workunit.client.0.smithi156.stdout:4/234: getdents d4/d41 0 2024-07-16T09:55:33.871 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:33.870+0000 7f76fe7fc640 1 -- 172.21.15.156:0/3747026169 <== mon.1 v2:172.21.15.176:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f76f8089110 con 0x7f7708071380 2024-07-16T09:55:33.872 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:33.870+0000 7f770de66640 1 --2- 172.21.15.156:0/3747026169 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f76f406a730 0x7f76f406cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:55:33.872 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:33.871+0000 7f770de66640 1 --2- 172.21.15.156:0/3747026169 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f76f406a730 0x7f76f406cbe0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f7704007ac0 tx=0x7f7704035040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:55:33.873 INFO:tasks.workunit.client.0.smithi156.stdout:2/394: mkdir d0/d5/d8b 0 2024-07-16T09:55:33.873 INFO:tasks.workunit.client.0.smithi156.stdout:0/182: dread d0/f3 [0,4194304] 0 2024-07-16T09:55:33.876 INFO:tasks.workunit.client.0.smithi156.stdout:7/219: getdents dc/d3e/d3f/d45 0 2024-07-16T09:55:33.876 INFO:tasks.workunit.client.0.smithi156.stdout:2/395: fdatasync d0/d13/f43 0 2024-07-16T09:55:33.878 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:33.877+0000 7f76fe7fc640 1 -- 172.21.15.156:0/3747026169 <== mon.1 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f76f8056410 con 0x7f7708071380 2024-07-16T09:55:33.881 INFO:tasks.workunit.client.0.smithi156.stdout:7/220: chown dc/d48/c54 1 1 2024-07-16T09:55:33.881 INFO:tasks.workunit.client.0.smithi156.stdout:0/183: creat d0/d7/d9/d27/f33 x:0 0 0 2024-07-16T09:55:33.882 INFO:tasks.workunit.client.0.smithi156.stdout:2/396: symlink d0/d5/d29/d2d/d5b/d62/l8c 0 2024-07-16T09:55:33.884 INFO:tasks.workunit.client.0.smithi156.stdout:4/235: link d4/f5 d4/d23/d30/d33/f50 0 2024-07-16T09:55:33.884 INFO:tasks.workunit.client.0.smithi156.stdout:6/292: dwrite d3/f1c [0,4194304] 0 2024-07-16T09:55:33.887 INFO:tasks.workunit.client.1.smithi176.stdout:2/272: fsync f3 0 2024-07-16T09:55:33.888 INFO:tasks.workunit.client.0.smithi156.stdout:8/210: truncate d0/fc 1923647 0 2024-07-16T09:55:33.888 INFO:tasks.workunit.client.0.smithi156.stdout:0/184: write d0/d7/d25/f2a [237172,47733] 0 2024-07-16T09:55:33.890 INFO:tasks.workunit.client.0.smithi156.stdout:1/223: dwrite dc/f28 [0,4194304] 0 2024-07-16T09:55:33.892 INFO:tasks.workunit.client.0.smithi156.stdout:4/236: chown d4/d23/d30/l38 1182 1 2024-07-16T09:55:33.893 INFO:tasks.workunit.client.0.smithi156.stdout:2/397: truncate d0/d5/d29/d2d/d5b/f6d 526556 0 2024-07-16T09:55:33.894 INFO:tasks.workunit.client.0.smithi156.stdout:2/398: chown d0/d5/d29/d2d/l41 4000987 1 2024-07-16T09:55:33.895 INFO:tasks.workunit.client.0.smithi156.stdout:8/211: unlink d0/d11/l23 0 2024-07-16T09:55:33.897 INFO:tasks.workunit.client.0.smithi156.stdout:9/295: dread d2/d5/d16/d22/d36/d33/f2c [0,4194304] 0 2024-07-16T09:55:33.897 INFO:tasks.workunit.client.0.smithi156.stdout:7/221: rename dc/d13/d36/l41 to dc/d13/d15/l58 0 2024-07-16T09:55:33.898 INFO:tasks.workunit.client.1.smithi176.stdout:2/273: dread dd/d16/f1d [0,4194304] 0 2024-07-16T09:55:33.899 INFO:tasks.workunit.client.0.smithi156.stdout:4/237: write d4/db/f3f [757781,2152] 0 2024-07-16T09:55:33.900 INFO:tasks.workunit.client.0.smithi156.stdout:2/399: creat d0/d13/f8d x:0 0 0 2024-07-16T09:55:33.904 INFO:tasks.workunit.client.0.smithi156.stdout:8/212: creat d0/d11/d4a/f4f x:0 0 0 2024-07-16T09:55:33.904 INFO:tasks.workunit.client.0.smithi156.stdout:9/296: rmdir d2/d5/d16/d22 39 2024-07-16T09:55:33.906 INFO:tasks.workunit.client.1.smithi176.stdout:2/274: dwrite dd/de/f58 [0,4194304] 0 2024-07-16T09:55:33.907 INFO:tasks.workunit.client.0.smithi156.stdout:1/224: symlink dc/d14/d3e/d45/l4e 0 2024-07-16T09:55:33.907 INFO:tasks.workunit.client.0.smithi156.stdout:8/213: write d0/dd/f33 [172653,48151] 0 2024-07-16T09:55:33.909 INFO:tasks.workunit.client.1.smithi176.stdout:2/275: readlink dd/de/l42 0 2024-07-16T09:55:33.909 INFO:tasks.workunit.client.0.smithi156.stdout:0/185: link d0/d7/d25/f2a d0/f34 0 2024-07-16T09:55:33.910 INFO:tasks.workunit.client.0.smithi156.stdout:0/186: chown d0/f14 510556 1 2024-07-16T09:55:33.913 INFO:tasks.workunit.client.0.smithi156.stdout:7/222: rename dc/d13/d20/c27 to dc/d13/d20/c59 0 2024-07-16T09:55:33.913 INFO:tasks.workunit.client.1.smithi176.stdout:2/276: truncate dd/f3c 4646222 0 2024-07-16T09:55:33.915 INFO:tasks.workunit.client.0.smithi156.stdout:4/238: mknod d4/db/d44/c51 0 2024-07-16T09:55:33.915 INFO:tasks.workunit.client.0.smithi156.stdout:7/223: write dc/d48/f4b [197384,83462] 0 2024-07-16T09:55:33.916 INFO:tasks.workunit.client.0.smithi156.stdout:1/225: rename dc/d20/f4d to dc/d24/f4f 0 2024-07-16T09:55:33.916 INFO:tasks.workunit.client.1.smithi176.stdout:2/277: mkdir dd/d2e/d35/d59 0 2024-07-16T09:55:33.917 INFO:tasks.workunit.client.0.smithi156.stdout:8/214: dwrite d0/dd/f26 [0,4194304] 0 2024-07-16T09:55:33.918 INFO:tasks.workunit.client.1.smithi176.stdout:2/278: write dd/d2e/f37 [478387,1414] 0 2024-07-16T09:55:33.918 INFO:tasks.workunit.client.1.smithi176.stdout:2/279: chown dd/d2e 1961288 1 2024-07-16T09:55:33.919 INFO:tasks.workunit.client.0.smithi156.stdout:1/226: dread - dc/d14/f34 zero size 2024-07-16T09:55:33.919 INFO:tasks.workunit.client.0.smithi156.stdout:7/224: write dc/d13/d20/f3c [1138886,61583] 0 2024-07-16T09:55:33.923 INFO:tasks.workunit.client.0.smithi156.stdout:0/187: dread d0/d7/d9/f15 [4194304,4194304] 0 2024-07-16T09:55:33.923 INFO:tasks.workunit.client.0.smithi156.stdout:4/239: mkdir d4/d23/d52 0 2024-07-16T09:55:33.925 INFO:tasks.workunit.client.0.smithi156.stdout:2/400: dread d0/d5/d29/d2d/f5a [0,4194304] 0 2024-07-16T09:55:33.926 INFO:tasks.workunit.client.1.smithi176.stdout:2/280: dwrite dd/d2e/f50 [0,4194304] 0 2024-07-16T09:55:33.928 INFO:tasks.workunit.client.1.smithi176.stdout:2/281: write dd/d16/f3a [1006632,22024] 0 2024-07-16T09:55:33.930 INFO:tasks.workunit.client.0.smithi156.stdout:7/225: fsync dc/d13/f35 0 2024-07-16T09:55:33.931 INFO:tasks.workunit.client.1.smithi176.stdout:2/282: unlink dd/d16/f1c 0 2024-07-16T09:55:33.933 INFO:tasks.workunit.client.0.smithi156.stdout:8/215: dwrite d0/dd/f1a [0,4194304] 0 2024-07-16T09:55:33.936 INFO:tasks.workunit.client.0.smithi156.stdout:4/240: fdatasync f2 0 2024-07-16T09:55:33.936 INFO:tasks.workunit.client.0.smithi156.stdout:9/297: getdents d2 0 2024-07-16T09:55:33.939 INFO:tasks.workunit.client.0.smithi156.stdout:1/227: dwrite dc/d14/f2d [0,4194304] 0 2024-07-16T09:55:33.939 INFO:tasks.workunit.client.0.smithi156.stdout:2/401: unlink d0/d13/f5d 0 2024-07-16T09:55:33.940 INFO:tasks.workunit.client.1.smithi176.stdout:2/283: dwrite dd/d16/f56 [0,4194304] 0 2024-07-16T09:55:33.942 INFO:tasks.workunit.client.1.smithi176.stdout:2/284: chown dd/d2e/d35/f3b 575977 1 2024-07-16T09:55:33.947 INFO:tasks.workunit.client.1.smithi176.stdout:2/285: mknod dd/d49/d52/c5a 0 2024-07-16T09:55:33.949 INFO:tasks.workunit.client.0.smithi156.stdout:0/188: dwrite d0/d7/d9/d27/d2c/f32 [0,4194304] 0 2024-07-16T09:55:33.952 INFO:tasks.workunit.client.0.smithi156.stdout:6/293: dwrite d3/d13/d17/d4f/d1f/f2c [0,4194304] 0 2024-07-16T09:55:33.952 INFO:tasks.workunit.client.1.smithi176.stdout:2/286: link dd/d36/l4e dd/d16/l5b 0 2024-07-16T09:55:33.956 INFO:tasks.workunit.client.1.smithi176.stdout:2/287: mknod dd/d36/d45/c5c 0 2024-07-16T09:55:33.958 INFO:tasks.workunit.client.0.smithi156.stdout:7/226: creat dc/d13/d36/d56/f5a x:0 0 0 2024-07-16T09:55:33.958 INFO:tasks.workunit.client.0.smithi156.stdout:8/216: dwrite d0/f2c [0,4194304] 0 2024-07-16T09:55:33.959 INFO:tasks.workunit.client.1.smithi176.stdout:2/288: unlink dd/de/l43 0 2024-07-16T09:55:33.959 INFO:tasks.workunit.client.0.smithi156.stdout:4/241: write d4/db/dc/f2a [1402553,37292] 0 2024-07-16T09:55:33.959 INFO:tasks.workunit.client.0.smithi156.stdout:1/228: rmdir dc/d14/d3e 39 2024-07-16T09:55:33.961 INFO:tasks.workunit.client.0.smithi156.stdout:9/298: truncate d2/d5/d16/f20 882464 0 2024-07-16T09:55:33.962 INFO:tasks.workunit.client.0.smithi156.stdout:2/402: creat d0/d5/d8b/f8e x:0 0 0 2024-07-16T09:55:33.970 INFO:tasks.workunit.client.0.smithi156.stdout:2/403: write d0/d13/d17/d1c/f37 [1663689,15014] 0 2024-07-16T09:55:33.971 INFO:tasks.workunit.client.0.smithi156.stdout:9/299: readlink d2/d5/ld 0 2024-07-16T09:55:33.971 INFO:tasks.workunit.client.0.smithi156.stdout:2/404: readlink d0/l60 0 2024-07-16T09:55:33.973 INFO:tasks.workunit.client.0.smithi156.stdout:0/189: getdents d0/d7/d9/d27/d2c 0 2024-07-16T09:55:33.975 INFO:tasks.workunit.client.0.smithi156.stdout:6/294: rename d3/d13/d17/d4f/d1f/f37 to d3/d13/d17/d4f/d4d/f61 0 2024-07-16T09:55:33.978 INFO:tasks.workunit.client.0.smithi156.stdout:0/190: chown d0/d7/d9/d27/c28 9540676 1 2024-07-16T09:55:33.980 INFO:tasks.workunit.client.1.smithi176.stdout:2/289: getdents dd/d49/d52 0 2024-07-16T09:55:33.980 INFO:tasks.workunit.client.0.smithi156.stdout:8/217: getdents d0 0 2024-07-16T09:55:33.980 INFO:tasks.workunit.client.0.smithi156.stdout:2/405: creat d0/d5/d29/d2d/d5b/d62/d74/f8f x:0 0 0 2024-07-16T09:55:33.982 INFO:tasks.workunit.client.1.smithi176.stdout:2/290: creat dd/d36/f5d x:0 0 0 2024-07-16T09:55:33.983 INFO:tasks.workunit.client.0.smithi156.stdout:2/406: chown d0/d13/d17/d1c 313207 1 2024-07-16T09:55:33.983 INFO:tasks.workunit.client.0.smithi156.stdout:6/295: dwrite d3/d13/d17/d4f/d4d/f3f [0,4194304] 0 2024-07-16T09:55:33.983 INFO:tasks.workunit.client.0.smithi156.stdout:8/218: write d0/d11/f40 [196239,114816] 0 2024-07-16T09:55:33.983 INFO:tasks.workunit.client.0.smithi156.stdout:0/191: unlink d0/fa 0 2024-07-16T09:55:33.988 INFO:tasks.workunit.client.0.smithi156.stdout:9/300: link d2/f4 d2/d5/d16/d22/d2b/d3d/d3c/f62 0 2024-07-16T09:55:33.990 INFO:tasks.workunit.client.1.smithi176.stdout:2/291: dwrite dd/d49/f55 [0,4194304] 0 2024-07-16T09:55:33.993 INFO:tasks.workunit.client.0.smithi156.stdout:8/219: rename d0/d11/d13/d1c/d45 to d0/dd/d1f/d50 0 2024-07-16T09:55:33.994 INFO:tasks.workunit.client.1.smithi176.stdout:2/292: mknod dd/d16/c5e 0 2024-07-16T09:55:33.994 INFO:tasks.workunit.client.0.smithi156.stdout:0/192: dwrite d0/d7/f11 [0,4194304] 0 2024-07-16T09:55:33.995 INFO:tasks.workunit.client.0.smithi156.stdout:7/227: dwrite dc/d13/d20/f2b [4194304,4194304] 0 2024-07-16T09:55:33.996 INFO:tasks.workunit.client.0.smithi156.stdout:6/296: creat d3/d13/d17/d4f/d1f/d19/d31/d35/f62 x:0 0 0 2024-07-16T09:55:33.997 INFO:tasks.workunit.client.1.smithi176.stdout:2/293: mknod dd/d16/c5f 0 2024-07-16T09:55:33.998 INFO:tasks.workunit.client.0.smithi156.stdout:2/407: rename d0/d13/d17/d1c/d51/f5c to d0/d13/d6e/f90 0 2024-07-16T09:55:33.998 INFO:tasks.workunit.client.0.smithi156.stdout:7/228: write dc/f40 [427686,9469] 0 2024-07-16T09:55:34.000 INFO:tasks.workunit.client.1.smithi176.stdout:2/294: creat dd/d49/f60 x:0 0 0 2024-07-16T09:55:34.000 INFO:tasks.workunit.client.0.smithi156.stdout:9/301: mkdir d2/d5/d16/d57/d63 0 2024-07-16T09:55:34.000 INFO:tasks.workunit.client.0.smithi156.stdout:2/408: fsync d0/d5/d29/f3f 0 2024-07-16T09:55:34.003 INFO:tasks.workunit.client.0.smithi156.stdout:8/220: link d0/d11/c18 d0/d11/d13/c51 0 2024-07-16T09:55:34.006 INFO:tasks.workunit.client.0.smithi156.stdout:7/229: rename dc/c1a to dc/d3e/d3f/c5b 0 2024-07-16T09:55:34.033 INFO:tasks.workunit.client.0.smithi156.stdout:3/224: sync 2024-07-16T09:55:34.043 INFO:tasks.workunit.client.0.smithi156.stdout:0/193: dwrite d0/d7/f21 [0,4194304] 0 2024-07-16T09:55:34.045 INFO:tasks.workunit.client.0.smithi156.stdout:7/230: stat dc/d48/f52 0 2024-07-16T09:55:34.049 INFO:tasks.workunit.client.0.smithi156.stdout:6/297: dwrite d3/d13/d17/d4f/d4d/d3b/f5d [0,4194304] 0 2024-07-16T09:55:34.049 INFO:tasks.workunit.client.0.smithi156.stdout:3/225: rmdir d8/da/d28/d43 39 2024-07-16T09:55:34.053 INFO:tasks.workunit.client.1.smithi176.stdout:2/295: dread dd/f2a [0,4194304] 0 2024-07-16T09:55:34.055 INFO:tasks.workunit.client.0.smithi156.stdout:0/194: mknod d0/d7/d9/d27/c35 0 2024-07-16T09:55:34.056 INFO:tasks.workunit.client.1.smithi176.stdout:2/296: creat dd/d36/f61 x:0 0 0 2024-07-16T09:55:34.057 INFO:tasks.workunit.client.0.smithi156.stdout:2/409: dwrite d0/d13/f81 [0,4194304] 0 2024-07-16T09:55:34.058 INFO:tasks.workunit.client.1.smithi176.stdout:2/297: symlink dd/d2e/l62 0 2024-07-16T09:55:34.060 INFO:tasks.workunit.client.0.smithi156.stdout:3/226: write d8/da/d13/f17 [5096516,89447] 0 2024-07-16T09:55:34.060 INFO:tasks.workunit.client.0.smithi156.stdout:7/231: creat dc/d44/f5c x:0 0 0 2024-07-16T09:55:34.064 INFO:tasks.workunit.client.0.smithi156.stdout:9/302: dwrite d2/d5/d16/d22/d36/f56 [0,4194304] 0 2024-07-16T09:55:34.064 INFO:tasks.workunit.client.0.smithi156.stdout:6/298: getdents d3/d13 0 2024-07-16T09:55:34.064 INFO:tasks.workunit.client.1.smithi176.stdout:2/298: read dd/d36/f2f [1168390,91445] 0 2024-07-16T09:55:34.066 INFO:tasks.workunit.client.0.smithi156.stdout:3/227: fsync d8/da/d21/f3e 0 2024-07-16T09:55:34.067 INFO:tasks.workunit.client.0.smithi156.stdout:8/221: dwrite d0/d11/d20/f21 [0,4194304] 0 2024-07-16T09:55:34.067 INFO:tasks.workunit.client.0.smithi156.stdout:3/228: fsync d8/f10 0 2024-07-16T09:55:34.068 INFO:tasks.workunit.client.1.smithi176.stdout:2/299: creat dd/d49/d52/f63 x:0 0 0 2024-07-16T09:55:34.069 INFO:tasks.workunit.client.1.smithi176.stdout:2/300: stat dd/d2e/d35/d59 0 2024-07-16T09:55:34.070 INFO:tasks.workunit.client.0.smithi156.stdout:6/299: chown d3/d13/d17/d4f/d4d/f61 203960339 1 2024-07-16T09:55:34.070 INFO:tasks.workunit.client.0.smithi156.stdout:0/195: creat d0/f36 x:0 0 0 2024-07-16T09:55:34.074 INFO:tasks.workunit.client.0.smithi156.stdout:7/232: write dc/d13/d20/f2b [8572595,42223] 0 2024-07-16T09:55:34.075 INFO:tasks.workunit.client.0.smithi156.stdout:6/300: readlink d3/d13/d17/d4f/d1f/l33 0 2024-07-16T09:55:34.076 INFO:tasks.workunit.client.1.smithi176.stdout:2/301: link dd/d2e/f3e dd/d2e/d35/f64 0 2024-07-16T09:55:34.078 INFO:tasks.workunit.client.0.smithi156.stdout:9/303: mkdir d2/d5/d61/d64 0 2024-07-16T09:55:34.079 INFO:tasks.workunit.client.0.smithi156.stdout:3/229: rmdir d8/da 39 2024-07-16T09:55:34.084 INFO:tasks.workunit.client.0.smithi156.stdout:2/410: rename d0/d13/d17/d1c/f37 to d0/f91 0 2024-07-16T09:55:34.084 INFO:tasks.workunit.client.0.smithi156.stdout:7/233: mkdir dc/d3e/d5d 0 2024-07-16T09:55:34.085 INFO:tasks.workunit.client.0.smithi156.stdout:2/411: fdatasync d0/d13/d17/d1c/f71 0 2024-07-16T09:55:34.086 INFO:tasks.workunit.client.1.smithi176.stdout:2/302: dread dd/d2e/f37 [0,4194304] 0 2024-07-16T09:55:34.088 INFO:tasks.workunit.client.0.smithi156.stdout:9/304: symlink d2/d5/d16/d22/d36/d33/l65 0 2024-07-16T09:55:34.088 INFO:tasks.workunit.client.0.smithi156.stdout:6/301: symlink d3/d13/d17/d4f/d1f/d19/d31/d35/l63 0 2024-07-16T09:55:34.093 INFO:tasks.workunit.client.1.smithi176.stdout:2/303: dwrite dd/d16/f27 [4194304,4194304] 0 2024-07-16T09:55:34.095 INFO:tasks.workunit.client.1.smithi176.stdout:2/304: write dd/d49/f55 [4055914,127525] 0 2024-07-16T09:55:34.095 INFO:tasks.workunit.client.0.smithi156.stdout:0/196: write d0/d7/d9/fe [1912358,96322] 0 2024-07-16T09:55:34.097 INFO:tasks.workunit.client.0.smithi156.stdout:8/222: write d0/d11/d13/f14 [193126,80224] 0 2024-07-16T09:55:34.097 INFO:tasks.workunit.client.0.smithi156.stdout:9/305: write d2/f15 [440637,123868] 0 2024-07-16T09:55:34.099 INFO:tasks.workunit.client.0.smithi156.stdout:0/197: write d0/f14 [56032,117357] 0 2024-07-16T09:55:34.102 INFO:tasks.workunit.client.0.smithi156.stdout:7/234: creat dc/d3e/d3f/d51/f5e x:0 0 0 2024-07-16T09:55:34.103 INFO:tasks.workunit.client.0.smithi156.stdout:2/412: write d0/d5/f2a [658611,30112] 0 2024-07-16T09:55:34.103 INFO:tasks.workunit.client.0.smithi156.stdout:6/302: creat d3/d13/d17/d56/f64 x:0 0 0 2024-07-16T09:55:34.109 INFO:tasks.workunit.client.0.smithi156.stdout:0/198: fdatasync d0/d7/d9/fe 0 2024-07-16T09:55:34.110 INFO:tasks.workunit.client.1.smithi176.stdout:2/305: truncate dd/d49/f55 2837763 0 2024-07-16T09:55:34.111 INFO:tasks.workunit.client.1.smithi176.stdout:2/306: stat dd/d36/f2f 0 2024-07-16T09:55:34.113 INFO:tasks.workunit.client.1.smithi176.stdout:2/307: creat dd/d49/f65 x:0 0 0 2024-07-16T09:55:34.116 INFO:tasks.workunit.client.0.smithi156.stdout:0/199: unlink d0/d7/f10 0 2024-07-16T09:55:34.120 INFO:tasks.workunit.client.1.smithi176.stdout:2/308: dwrite dd/d49/d52/f63 [0,4194304] 0 2024-07-16T09:55:34.122 INFO:tasks.workunit.client.1.smithi176.stdout:2/309: rmdir dd/d36/d45 39 2024-07-16T09:55:34.123 INFO:tasks.workunit.client.0.smithi156.stdout:6/303: dwrite d3/d13/d17/d4f/d4d/f57 [0,4194304] 0 2024-07-16T09:55:34.123 INFO:tasks.workunit.client.1.smithi176.stdout:2/310: write dd/d16/f56 [3444333,306] 0 2024-07-16T09:55:34.124 INFO:tasks.workunit.client.0.smithi156.stdout:7/235: dwrite dc/d3e/d3f/d51/f5e [0,4194304] 0 2024-07-16T09:55:34.124 INFO:tasks.workunit.client.1.smithi176.stdout:2/311: chown dd/d16/f32 6158932 1 2024-07-16T09:55:34.126 INFO:tasks.workunit.client.0.smithi156.stdout:2/413: dwrite d0/d5/d29/d2d/f5a [0,4194304] 0 2024-07-16T09:55:34.128 INFO:tasks.workunit.client.0.smithi156.stdout:2/414: write d0/fe [1520232,86629] 0 2024-07-16T09:55:34.131 INFO:tasks.workunit.client.1.smithi176.stdout:2/312: unlink dd/f2a 0 2024-07-16T09:55:34.133 INFO:tasks.workunit.client.0.smithi156.stdout:0/200: write d0/f2b [539808,109347] 0 2024-07-16T09:55:34.134 INFO:tasks.workunit.client.0.smithi156.stdout:9/306: dread d2/d5/f8 [0,4194304] 0 2024-07-16T09:55:34.136 INFO:tasks.workunit.client.0.smithi156.stdout:3/230: dwrite d8/da/d13/f24 [0,4194304] 0 2024-07-16T09:55:34.140 INFO:tasks.workunit.client.1.smithi176.stdout:2/313: dwrite dd/d36/f4d [0,4194304] 0 2024-07-16T09:55:34.140 INFO:tasks.workunit.client.0.smithi156.stdout:7/236: symlink dc/d13/d36/d56/l5f 0 2024-07-16T09:55:34.143 INFO:tasks.workunit.client.1.smithi176.stdout:2/314: write dd/de/f58 [3540309,48471] 0 2024-07-16T09:55:34.143 INFO:tasks.workunit.client.0.smithi156.stdout:9/307: truncate d2/d5/d16/f37 438024 0 2024-07-16T09:55:34.144 INFO:tasks.workunit.client.0.smithi156.stdout:7/237: chown ca 51637050 1 2024-07-16T09:55:34.147 INFO:tasks.workunit.client.1.smithi176.stdout:1/167: sync 2024-07-16T09:55:34.148 INFO:tasks.workunit.client.0.smithi156.stdout:6/304: rename d3/d13/d17/d4f/d4d/d3b/f3c to d3/d13/d17/d4f/d5c/f65 0 2024-07-16T09:55:34.148 INFO:tasks.workunit.client.0.smithi156.stdout:8/223: dread d0/dd/f10 [4194304,4194304] 0 2024-07-16T09:55:34.148 INFO:tasks.workunit.client.1.smithi176.stdout:1/168: dread - d4/f31 zero size 2024-07-16T09:55:34.153 INFO:tasks.workunit.client.1.smithi176.stdout:1/169: rename d4/c1f to d4/d8/d2c/d2d/c37 0 2024-07-16T09:55:34.160 INFO:tasks.workunit.client.1.smithi176.stdout:1/170: rename f1 to d4/d8/d12/d21/d25/d30/f38 0 2024-07-16T09:55:34.160 INFO:tasks.workunit.client.0.smithi156.stdout:0/201: dwrite d0/d7/d9/f26 [0,4194304] 0 2024-07-16T09:55:34.162 INFO:tasks.workunit.client.0.smithi156.stdout:8/224: mknod d0/d11/d29/c52 0 2024-07-16T09:55:34.162 INFO:tasks.workunit.client.0.smithi156.stdout:2/415: dwrite d0/d5/d29/d2d/d5b/d62/f6a [0,4194304] 0 2024-07-16T09:55:34.167 INFO:tasks.workunit.client.1.smithi176.stdout:2/315: dread dd/d16/f27 [0,4194304] 0 2024-07-16T09:55:34.169 INFO:tasks.workunit.client.0.smithi156.stdout:2/416: fdatasync d0/d13/f1a 0 2024-07-16T09:55:34.174 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:55:33 smithi156 ceph-mon[29302]: pgmap v248: 65 pgs: 65 active+clean; 737 MiB data, 3.9 GiB used, 533 GiB / 536 GiB avail; 5.6 MiB/s rd, 83 MiB/s wr, 401 op/s 2024-07-16T09:55:34.174 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:55:33 smithi156 ceph-mon[29302]: from='client.24399 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-16T09:55:34.175 INFO:tasks.workunit.client.0.smithi156.stdout:8/225: symlink d0/l53 0 2024-07-16T09:55:34.176 INFO:tasks.workunit.client.0.smithi156.stdout:1/229: sync 2024-07-16T09:55:34.179 INFO:tasks.workunit.client.0.smithi156.stdout:6/305: dwrite d3/f1c [0,4194304] 0 2024-07-16T09:55:34.179 INFO:tasks.workunit.client.0.smithi156.stdout:5/233: sync 2024-07-16T09:55:34.181 INFO:tasks.workunit.client.0.smithi156.stdout:7/238: dwrite dc/d3e/d3f/d51/f5e [4194304,4194304] 0 2024-07-16T09:55:34.184 INFO:tasks.workunit.client.0.smithi156.stdout:7/239: chown dc/f40 692 1 2024-07-16T09:55:34.186 INFO:tasks.workunit.client.0.smithi156.stdout:5/234: fsync d1b/d1e/f48 0 2024-07-16T09:55:34.186 INFO:tasks.workunit.client.0.smithi156.stdout:9/308: dwrite d2/de/d42/f47 [0,4194304] 0 2024-07-16T09:55:34.188 INFO:tasks.workunit.client.0.smithi156.stdout:2/417: creat d0/d5/d8b/f92 x:0 0 0 2024-07-16T09:55:34.188 INFO:tasks.workunit.client.0.smithi156.stdout:0/202: getdents d0/d22 0 2024-07-16T09:55:34.190 INFO:tasks.workunit.client.1.smithi176.stdout:1/171: truncate d4/f10 3363603 0 2024-07-16T09:55:34.190 INFO:tasks.workunit.client.0.smithi156.stdout:6/306: fsync d3/d13/d17/d4f/d30/d45/f51 0 2024-07-16T09:55:34.190 INFO:tasks.workunit.client.1.smithi176.stdout:2/316: write dd/d2e/f3e [1424595,126280] 0 2024-07-16T09:55:34.192 INFO:tasks.workunit.client.0.smithi156.stdout:7/240: readlink dc/d13/d15/l58 0 2024-07-16T09:55:34.196 INFO:tasks.workunit.client.0.smithi156.stdout:0/203: dread - d0/d7/d9/d27/f33 zero size 2024-07-16T09:55:34.197 INFO:tasks.workunit.client.0.smithi156.stdout:2/418: creat d0/d5/f93 x:0 0 0 2024-07-16T09:55:34.198 INFO:tasks.workunit.client.0.smithi156.stdout:3/231: dwrite d8/d14/f1c [0,4194304] 0 2024-07-16T09:55:34.199 INFO:tasks.workunit.client.0.smithi156.stdout:9/309: truncate d2/d5/d16/d22/d36/f59 626074 0 2024-07-16T09:55:34.199 INFO:tasks.workunit.client.0.smithi156.stdout:5/235: rmdir d1b/d1f/d45 39 2024-07-16T09:55:34.200 INFO:tasks.workunit.client.0.smithi156.stdout:8/226: rename d0/d11/d13 to d0/d11/d13/d1c/d54 22 2024-07-16T09:55:34.201 INFO:tasks.workunit.client.1.smithi176.stdout:4/302: sync 2024-07-16T09:55:34.201 INFO:tasks.workunit.client.1.smithi176.stdout:8/423: sync 2024-07-16T09:55:34.201 INFO:tasks.workunit.client.1.smithi176.stdout:6/248: sync 2024-07-16T09:55:34.201 INFO:tasks.workunit.client.1.smithi176.stdout:7/285: sync 2024-07-16T09:55:34.201 INFO:tasks.workunit.client.1.smithi176.stdout:3/201: sync 2024-07-16T09:55:34.201 INFO:tasks.workunit.client.1.smithi176.stdout:5/379: sync 2024-07-16T09:55:34.201 INFO:tasks.workunit.client.1.smithi176.stdout:0/282: sync 2024-07-16T09:55:34.201 INFO:tasks.workunit.client.1.smithi176.stdout:9/325: sync 2024-07-16T09:55:34.202 INFO:tasks.workunit.client.0.smithi156.stdout:7/241: dread dc/f42 [0,4194304] 0 2024-07-16T09:55:34.203 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:34.202+0000 7f77100ca640 1 -- 172.21.15.156:0/3747026169 --> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f7708119a60 con 0x7f76f406a730 2024-07-16T09:55:34.204 INFO:tasks.workunit.client.1.smithi176.stdout:1/172: dwrite d4/db/f1b [4194304,4194304] 0 2024-07-16T09:55:34.205 INFO:tasks.workunit.client.0.smithi156.stdout:1/230: write dc/f4a [563440,105892] 0 2024-07-16T09:55:34.205 INFO:tasks.workunit.client.0.smithi156.stdout:5/236: fsync d1b/d1e/f3d 0 2024-07-16T09:55:34.206 INFO:tasks.workunit.client.0.smithi156.stdout:5/237: fsync d1b/d1e/f48 0 2024-07-16T09:55:34.206 INFO:tasks.workunit.client.1.smithi176.stdout:3/202: chown d3/c24 0 1 2024-07-16T09:55:34.207 INFO:tasks.workunit.client.0.smithi156.stdout:5/238: write ff [2083197,57003] 0 2024-07-16T09:55:34.207 INFO:tasks.workunit.client.1.smithi176.stdout:2/317: dwrite dd/d2e/f37 [0,4194304] 0 2024-07-16T09:55:34.209 INFO:tasks.workunit.client.0.smithi156.stdout:8/227: chown d0/fc 26465512 1 2024-07-16T09:55:34.209 INFO:tasks.workunit.client.1.smithi176.stdout:4/303: write da/dc/d39/f3c [757604,24645] 0 2024-07-16T09:55:34.209 INFO:tasks.workunit.client.0.smithi156.stdout:9/310: mkdir d2/d5/d16/d22/d36/d33/d66 0 2024-07-16T09:55:34.211 INFO:tasks.workunit.client.0.smithi156.stdout:2/419: creat d0/d13/d17/d1c/d86/f94 x:0 0 0 2024-07-16T09:55:34.212 INFO:tasks.workunit.client.0.smithi156.stdout:3/232: rmdir d8/da/d28/d43 39 2024-07-16T09:55:34.214 INFO:tasks.workunit.client.1.smithi176.stdout:3/203: creat d3/d17/d1d/d1e/f3a x:0 0 0 2024-07-16T09:55:34.215 INFO:tasks.workunit.client.1.smithi176.stdout:0/283: write f4 [4826286,4907] 0 2024-07-16T09:55:34.216 INFO:tasks.workunit.client.0.smithi156.stdout:0/204: rename d0/f3 to d0/d7/d9/d2d/f37 0 2024-07-16T09:55:34.217 INFO:tasks.workunit.client.1.smithi176.stdout:7/286: rename d4/d5/f31 to d4/dd/d63/f64 0 2024-07-16T09:55:34.218 INFO:tasks.workunit.client.0.smithi156.stdout:7/242: readlink dc/d13/d15/l57 0 2024-07-16T09:55:34.219 INFO:tasks.workunit.client.1.smithi176.stdout:5/380: symlink d4/l8c 0 2024-07-16T09:55:34.219 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:34.217+0000 7f76fe7fc640 1 -- 172.21.15.156:0/3747026169 <== mgr.14208 v2:172.21.15.156:6800/2615741717 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3388 (secure 0 0 0) 0x7f7708119a60 con 0x7f76f406a730 2024-07-16T09:55:34.219 INFO:teuthology.orchestra.run.smithi156.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-07-16T09:55:34.219 INFO:teuthology.orchestra.run.smithi156.stdout:alertmanager.smithi156 smithi156 *:9093,9094 running (6m) 2m ago 7m 24.6M - 0.25.0 c8568f914cd2 83419a2a37f4 2024-07-16T09:55:34.219 INFO:teuthology.orchestra.run.smithi156.stdout:crash.smithi156 smithi156 running (7m) 2m ago 7m 6899k - 17.2.7-1104-ge6231b04 d5de5f953b95 bf6d49ea06b4 2024-07-16T09:55:34.220 INFO:teuthology.orchestra.run.smithi156.stdout:crash.smithi176 smithi176 running (7m) 10s ago 6m 6903k - 17.2.7-1104-ge6231b04 d5de5f953b95 907c9fa1e890 2024-07-16T09:55:34.220 INFO:teuthology.orchestra.run.smithi156.stdout:grafana.smithi156 smithi156 *:3000 running (6m) 2m ago 7m 85.0M - 9.4.7 954c08fa6188 184256cf055d 2024-07-16T09:55:34.220 INFO:teuthology.orchestra.run.smithi156.stdout:mds.cephfs.smithi156.evwwaz smithi156 running (2m) 2m ago 2m 14.9M - 17.2.7-1104-ge6231b04 d5de5f953b95 7c0ff9ae43f8 2024-07-16T09:55:34.220 INFO:teuthology.orchestra.run.smithi156.stdout:mds.cephfs.smithi156.rdibys smithi156 running (2m) 2m ago 2m 15.8M - 17.2.7-1104-ge6231b04 d5de5f953b95 b985cd1e6924 2024-07-16T09:55:34.220 INFO:teuthology.orchestra.run.smithi156.stdout:mds.cephfs.smithi176.dvvbud smithi176 running (2m) 10s ago 2m 68.4M - 17.2.7-1104-ge6231b04 d5de5f953b95 1b00147950f9 2024-07-16T09:55:34.220 INFO:teuthology.orchestra.run.smithi156.stdout:mds.cephfs.smithi176.wahbsi smithi176 running (2m) 10s ago 2m 44.2M - 17.2.7-1104-ge6231b04 d5de5f953b95 866664af2311 2024-07-16T09:55:34.220 INFO:teuthology.orchestra.run.smithi156.stdout:mgr.smithi156.zkvylv smithi156 *:9283 running (9m) 2m ago 9m 450M - 17.2.7-1104-ge6231b04 d5de5f953b95 a6937c84750a 2024-07-16T09:55:34.220 INFO:teuthology.orchestra.run.smithi156.stdout:mgr.smithi176.dqbrvx smithi176 *:8443,9283 running (36s) 10s ago 6m 113M - 19.0.0-3495-ga6ca461b 16f7feb04342 a7c332f2d676 2024-07-16T09:55:34.220 INFO:teuthology.orchestra.run.smithi156.stdout:mon.smithi156 smithi156 running (9m) 2m ago 9m 47.6M 2048M 17.2.7-1104-ge6231b04 d5de5f953b95 6471c8acff40 2024-07-16T09:55:34.220 INFO:teuthology.orchestra.run.smithi156.stdout:mon.smithi176 smithi176 running (6m) 10s ago 6m 37.5M 2048M 17.2.7-1104-ge6231b04 d5de5f953b95 3a293ac6ef01 2024-07-16T09:55:34.220 INFO:teuthology.orchestra.run.smithi156.stdout:node-exporter.smithi156 smithi156 *:9100 running (7m) 2m ago 7m 15.9M - 1.5.0 0da6a335fe13 2c136840a7a9 2024-07-16T09:55:34.220 INFO:teuthology.orchestra.run.smithi156.stdout:node-exporter.smithi176 smithi176 *:9100 running (6m) 10s ago 6m 16.3M - 1.5.0 0da6a335fe13 536716db6afe 2024-07-16T09:55:34.220 INFO:teuthology.orchestra.run.smithi156.stdout:osd.0 smithi156 running (6m) 2m ago 6m 48.5M 4288M 17.2.7-1104-ge6231b04 d5de5f953b95 0471758ec349 2024-07-16T09:55:34.221 INFO:teuthology.orchestra.run.smithi156.stdout:osd.1 smithi156 running (5m) 2m ago 5m 49.0M 4288M 17.2.7-1104-ge6231b04 d5de5f953b95 149a0bda2f61 2024-07-16T09:55:34.221 INFO:teuthology.orchestra.run.smithi156.stdout:osd.2 smithi156 running (5m) 2m ago 5m 44.5M 4288M 17.2.7-1104-ge6231b04 d5de5f953b95 f28acec5cb79 2024-07-16T09:55:34.221 INFO:teuthology.orchestra.run.smithi156.stdout:osd.3 smithi176 running (4m) 10s ago 4m 95.8M 5312M 17.2.7-1104-ge6231b04 d5de5f953b95 733861ba84dc 2024-07-16T09:55:34.221 INFO:teuthology.orchestra.run.smithi156.stdout:osd.4 smithi176 running (4m) 10s ago 4m 89.1M 5312M 17.2.7-1104-ge6231b04 d5de5f953b95 ef0c67cec086 2024-07-16T09:55:34.221 INFO:teuthology.orchestra.run.smithi156.stdout:osd.5 smithi176 running (4m) 10s ago 4m 88.0M 5312M 17.2.7-1104-ge6231b04 d5de5f953b95 996e15448a52 2024-07-16T09:55:34.221 INFO:teuthology.orchestra.run.smithi156.stdout:prometheus.smithi156 smithi156 *:9095 running (6m) 2m ago 7m 55.5M - 2.43.0 a07b618ecd1d 1655b4227cd0 2024-07-16T09:55:34.222 INFO:tasks.workunit.client.0.smithi156.stdout:9/311: unlink d2/d5/f8 0 2024-07-16T09:55:34.223 INFO:tasks.workunit.client.0.smithi156.stdout:2/420: mknod d0/d5/c95 0 2024-07-16T09:55:34.223 INFO:tasks.workunit.client.1.smithi176.stdout:0/284: mkdir d5/d1c/d5a 0 2024-07-16T09:55:34.224 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:34.222+0000 7f77100ca640 1 -- 172.21.15.156:0/3747026169 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f76f406a730 msgr2=0x7f76f406cbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:55:34.224 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:34.222+0000 7f77100ca640 1 --2- 172.21.15.156:0/3747026169 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f76f406a730 0x7f76f406cbe0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f7704007ac0 tx=0x7f7704035040 comp rx=0 tx=0).stop 2024-07-16T09:55:34.224 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:34.222+0000 7f77100ca640 1 -- 172.21.15.156:0/3747026169 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7708071380 msgr2=0x7f7708118690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:55:34.224 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:34.222+0000 7f77100ca640 1 --2- 172.21.15.156:0/3747026169 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7708071380 0x7f7708118690 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f76f800b4e0 tx=0x7f76f8009120 comp rx=0 tx=0).stop 2024-07-16T09:55:34.224 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:34.222+0000 7f77100ca640 1 -- 172.21.15.156:0/3747026169 shutdown_connections 2024-07-16T09:55:34.224 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:34.222+0000 7f77100ca640 1 --2- 172.21.15.156:0/3747026169 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f76f406a730 0x7f76f406cbe0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:55:34.224 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:34.222+0000 7f77100ca640 1 --2- 172.21.15.156:0/3747026169 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7708071380 0x7f7708118690 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:55:34.224 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:34.222+0000 7f77100ca640 1 --2- 172.21.15.156:0/3747026169 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f770806fdf0 0x7f7708118150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:55:34.224 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:34.222+0000 7f77100ca640 1 -- 172.21.15.156:0/3747026169 >> 172.21.15.156:0/3747026169 conn(0x7f77080f7b40 msgr2=0x7f77080f92a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:55:34.224 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:34.223+0000 7f77100ca640 1 -- 172.21.15.156:0/3747026169 shutdown_connections 2024-07-16T09:55:34.225 INFO:tasks.workunit.client.0.smithi156.stdout:2/421: dread - d0/d5/f93 zero size 2024-07-16T09:55:34.225 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:34.223+0000 7f77100ca640 1 -- 172.21.15.156:0/3747026169 wait complete. 2024-07-16T09:55:34.226 INFO:tasks.workunit.client.1.smithi176.stdout:9/326: dwrite d5/de/f15 [0,4194304] 0 2024-07-16T09:55:34.226 INFO:tasks.workunit.client.0.smithi156.stdout:7/243: write dc/f42 [234123,104666] 0 2024-07-16T09:55:34.227 INFO:tasks.workunit.client.0.smithi156.stdout:7/244: chown dc/f14 5854 1 2024-07-16T09:55:34.228 INFO:tasks.workunit.client.0.smithi156.stdout:5/239: write d1b/d1f/d45/f50 [57659,81995] 0 2024-07-16T09:55:34.230 INFO:tasks.workunit.client.1.smithi176.stdout:0/285: chown d5/d25/c49 0 1 2024-07-16T09:55:34.230 INFO:tasks.workunit.client.1.smithi176.stdout:3/204: dread d3/d17/f23 [0,4194304] 0 2024-07-16T09:55:34.231 INFO:tasks.workunit.client.1.smithi176.stdout:7/287: rename d4/dd/l13 to d4/d47/d5d/l65 0 2024-07-16T09:55:34.231 INFO:tasks.workunit.client.0.smithi156.stdout:3/233: symlink d8/da/l4c 0 2024-07-16T09:55:34.233 INFO:tasks.workunit.client.0.smithi156.stdout:3/234: fsync d8/da/f3d 0 2024-07-16T09:55:34.233 INFO:tasks.workunit.client.0.smithi156.stdout:1/231: rename dc/le to dc/d24/l50 0 2024-07-16T09:55:34.234 INFO:tasks.workunit.client.1.smithi176.stdout:0/286: mkdir d5/d1c/d24/d35/d50/d5b 0 2024-07-16T09:55:34.234 INFO:tasks.workunit.client.0.smithi156.stdout:7/245: rmdir dc/d48 39 2024-07-16T09:55:34.235 INFO:tasks.workunit.client.1.smithi176.stdout:5/381: rename d4/d37/d72/l47 to d4/d37/d72/d38/d6e/l8d 0 2024-07-16T09:55:34.238 INFO:tasks.workunit.client.0.smithi156.stdout:3/235: rmdir d8/da/d21 39 2024-07-16T09:55:34.238 INFO:tasks.workunit.client.1.smithi176.stdout:7/288: dread d4/fb [0,4194304] 0 2024-07-16T09:55:34.238 INFO:tasks.workunit.client.1.smithi176.stdout:3/205: unlink d3/d17/f32 0 2024-07-16T09:55:34.239 INFO:tasks.workunit.client.1.smithi176.stdout:4/304: dwrite da/f42 [0,4194304] 0 2024-07-16T09:55:34.240 INFO:tasks.workunit.client.0.smithi156.stdout:6/307: dwrite d3/d13/d17/d4f/d4d/d3b/f40 [0,4194304] 0 2024-07-16T09:55:34.240 INFO:tasks.workunit.client.0.smithi156.stdout:7/246: fsync dc/d48/f52 0 2024-07-16T09:55:34.240 INFO:tasks.workunit.client.1.smithi176.stdout:6/249: truncate d1/f3b 310997 0 2024-07-16T09:55:34.241 INFO:tasks.workunit.client.0.smithi156.stdout:3/236: truncate d8/f1a 4700619 0 2024-07-16T09:55:34.242 INFO:tasks.workunit.client.0.smithi156.stdout:0/205: rename d0/f2b to d0/d7/d25/f38 0 2024-07-16T09:55:34.246 INFO:tasks.workunit.client.0.smithi156.stdout:5/240: mkdir d1b/d1f/d54 0 2024-07-16T09:55:34.246 INFO:tasks.workunit.client.1.smithi176.stdout:1/173: dread d4/f17 [0,4194304] 0 2024-07-16T09:55:34.247 INFO:tasks.workunit.client.0.smithi156.stdout:0/206: dread - d0/d7/d9/d27/f33 zero size 2024-07-16T09:55:34.247 INFO:tasks.workunit.client.0.smithi156.stdout:8/228: dwrite d0/f1 [4194304,4194304] 0 2024-07-16T09:55:34.247 INFO:tasks.workunit.client.1.smithi176.stdout:6/250: creat d1/dc/d26/f58 x:0 0 0 2024-07-16T09:55:34.248 INFO:tasks.workunit.client.1.smithi176.stdout:7/289: dwrite d4/dd/f23 [0,4194304] 0 2024-07-16T09:55:34.249 INFO:tasks.workunit.client.0.smithi156.stdout:0/207: readlink d0/d7/d25/l30 0 2024-07-16T09:55:34.249 INFO:tasks.workunit.client.0.smithi156.stdout:3/237: rmdir d8/da/d28 39 2024-07-16T09:55:34.250 INFO:tasks.workunit.client.1.smithi176.stdout:7/290: fsync d4/d5/d10/d17/d1d/f5b 0 2024-07-16T09:55:34.250 INFO:tasks.workunit.client.0.smithi156.stdout:7/247: dread dc/d13/f12 [0,4194304] 0 2024-07-16T09:55:34.251 INFO:tasks.workunit.client.1.smithi176.stdout:5/382: rename d4/d37/d72/d38/l85 to d4/d37/d72/d82/l8e 0 2024-07-16T09:55:34.251 INFO:tasks.workunit.client.0.smithi156.stdout:6/308: creat d3/d13/d17/d4f/d1f/d19/f66 x:0 0 0 2024-07-16T09:55:34.251 INFO:tasks.workunit.client.1.smithi176.stdout:7/291: chown d4/dd/f15 1202 1 2024-07-16T09:55:34.252 INFO:tasks.workunit.client.0.smithi156.stdout:5/241: rmdir d1b/d1f/d45/d52 39 2024-07-16T09:55:34.253 INFO:tasks.workunit.client.0.smithi156.stdout:6/309: chown d3/d13/d17/d56 13 1 2024-07-16T09:55:34.253 INFO:tasks.workunit.client.1.smithi176.stdout:7/292: truncate d4/dd/f36 344384 0 2024-07-16T09:55:34.254 INFO:tasks.workunit.client.1.smithi176.stdout:1/174: symlink d4/d8/d12/l39 0 2024-07-16T09:55:34.256 INFO:tasks.workunit.client.0.smithi156.stdout:9/312: getdents d2/d5/d16/d22/d2b/d3d/d3c 0 2024-07-16T09:55:34.256 INFO:tasks.workunit.client.0.smithi156.stdout:0/208: symlink d0/d7/d9/d27/d2c/l39 0 2024-07-16T09:55:34.257 INFO:tasks.workunit.client.1.smithi176.stdout:6/251: mknod d1/d5/d7/d1e/c59 0 2024-07-16T09:55:34.258 INFO:tasks.workunit.client.0.smithi156.stdout:1/232: write dc/d14/d1e/f21 [1499865,115248] 0 2024-07-16T09:55:34.258 INFO:tasks.workunit.client.1.smithi176.stdout:6/252: readlink d1/dc/l2b 0 2024-07-16T09:55:34.259 INFO:tasks.workunit.client.0.smithi156.stdout:3/238: mknod d8/da/c4d 0 2024-07-16T09:55:34.259 INFO:tasks.workunit.client.0.smithi156.stdout:6/310: creat d3/d13/d17/d4f/d30/f67 x:0 0 0 2024-07-16T09:55:34.259 INFO:tasks.workunit.client.1.smithi176.stdout:6/253: write d1/dc/d1c/f53 [96204,99187] 0 2024-07-16T09:55:34.262 INFO:tasks.workunit.client.1.smithi176.stdout:6/254: truncate d1/dc/d26/f58 597460 0 2024-07-16T09:55:34.263 INFO:tasks.workunit.client.0.smithi156.stdout:3/239: write d8/da/d13/f17 [2116024,98137] 0 2024-07-16T09:55:34.265 INFO:tasks.workunit.client.0.smithi156.stdout:6/311: fsync d3/d13/d17/d4f/d4d/d3b/f59 0 2024-07-16T09:55:34.268 INFO:tasks.workunit.client.0.smithi156.stdout:6/312: dread - d3/d13/d17/d56/f60 zero size 2024-07-16T09:55:34.269 INFO:tasks.workunit.client.1.smithi176.stdout:6/255: dwrite d1/dc/f33 [0,4194304] 0 2024-07-16T09:55:34.269 INFO:tasks.workunit.client.0.smithi156.stdout:7/248: dwrite dc/d3e/f4a [0,4194304] 0 2024-07-16T09:55:34.271 INFO:tasks.workunit.client.0.smithi156.stdout:7/249: stat dc/d44 0 2024-07-16T09:55:34.272 INFO:tasks.workunit.client.0.smithi156.stdout:5/242: dwrite d1b/d1e/f4e [0,4194304] 0 2024-07-16T09:55:34.274 INFO:tasks.workunit.client.0.smithi156.stdout:5/243: truncate f19 5609078 0 2024-07-16T09:55:34.275 INFO:tasks.workunit.client.0.smithi156.stdout:5/244: readlink d1b/l26 0 2024-07-16T09:55:34.276 INFO:tasks.workunit.client.0.smithi156.stdout:0/209: creat d0/d7/d25/f3a x:0 0 0 2024-07-16T09:55:34.288 INFO:tasks.workunit.client.0.smithi156.stdout:0/210: rename d0/d7 to d0/d7/d9/d2d/d3b 22 2024-07-16T09:55:34.288 INFO:tasks.workunit.client.0.smithi156.stdout:7/250: mknod dc/d44/c60 0 2024-07-16T09:55:34.289 INFO:tasks.workunit.client.1.smithi176.stdout:3/206: link d3/d17/d1d/c2e d3/da/c3b 0 2024-07-16T09:55:34.293 INFO:tasks.workunit.client.0.smithi156.stdout:0/211: creat d0/d22/f3c x:0 0 0 2024-07-16T09:55:34.294 INFO:tasks.workunit.client.1.smithi176.stdout:4/305: dwrite da/d3d/f1b [0,4194304] 0 2024-07-16T09:55:34.295 INFO:tasks.workunit.client.0.smithi156.stdout:5/245: creat d1b/f55 x:0 0 0 2024-07-16T09:55:34.295 INFO:tasks.workunit.client.1.smithi176.stdout:4/306: dread - da/d14/d4a/f4e zero size 2024-07-16T09:55:34.296 INFO:tasks.workunit.client.0.smithi156.stdout:6/313: dread d3/d13/d17/d4f/d1f/fe [0,4194304] 0 2024-07-16T09:55:34.300 INFO:tasks.workunit.client.1.smithi176.stdout:6/256: creat d1/dc/d26/f5a x:0 0 0 2024-07-16T09:55:34.303 INFO:tasks.workunit.client.1.smithi176.stdout:1/175: mkdir d4/d3a 0 2024-07-16T09:55:34.303 INFO:tasks.workunit.client.0.smithi156.stdout:8/229: truncate d0/fb 665220 0 2024-07-16T09:55:34.305 INFO:tasks.workunit.client.0.smithi156.stdout:8/230: write d0/d11/f15 [783291,54627] 0 2024-07-16T09:55:34.305 INFO:tasks.workunit.client.0.smithi156.stdout:9/313: write d2/d5/fb [1541381,19733] 0 2024-07-16T09:55:34.307 INFO:tasks.workunit.client.0.smithi156.stdout:8/231: fdatasync d0/d11/f48 0 2024-07-16T09:55:34.307 INFO:tasks.workunit.client.1.smithi176.stdout:0/287: dread d5/f10 [0,4194304] 0 2024-07-16T09:55:34.309 INFO:tasks.workunit.client.0.smithi156.stdout:9/314: fsync d2/de/f19 0 2024-07-16T09:55:34.311 INFO:tasks.workunit.client.1.smithi176.stdout:1/176: fsync d4/d8/d2c/f2f 0 2024-07-16T09:55:34.314 INFO:tasks.workunit.client.1.smithi176.stdout:0/288: symlink d5/d1c/d24/d26/l5c 0 2024-07-16T09:55:34.315 INFO:tasks.workunit.client.0.smithi156.stdout:8/232: mkdir d0/d11/d4a/d55 0 2024-07-16T09:55:34.315 INFO:tasks.workunit.client.1.smithi176.stdout:0/289: write f4 [2192901,107418] 0 2024-07-16T09:55:34.316 INFO:tasks.workunit.client.0.smithi156.stdout:7/251: getdents dc/d13/d24 0 2024-07-16T09:55:34.316 INFO:tasks.workunit.client.0.smithi156.stdout:7/252: chown dc/d44/f5c 1486764 1 2024-07-16T09:55:34.316 INFO:tasks.workunit.client.0.smithi156.stdout:5/246: mknod d1b/d1f/d45/d52/c56 0 2024-07-16T09:55:34.318 INFO:tasks.workunit.client.0.smithi156.stdout:5/247: fsync d1b/d1f/f44 0 2024-07-16T09:55:34.321 INFO:tasks.workunit.client.0.smithi156.stdout:0/212: getdents d0/d7 0 2024-07-16T09:55:34.322 INFO:tasks.workunit.client.0.smithi156.stdout:8/233: dread - d0/dd/f4c zero size 2024-07-16T09:55:34.323 INFO:tasks.workunit.client.1.smithi176.stdout:1/177: mknod d4/c3b 0 2024-07-16T09:55:34.324 INFO:tasks.workunit.client.0.smithi156.stdout:5/248: write d1b/d1e/f4e [1936064,100742] 0 2024-07-16T09:55:34.325 INFO:tasks.workunit.client.0.smithi156.stdout:0/213: rmdir d0/d7/d25 39 2024-07-16T09:55:34.327 INFO:tasks.workunit.client.1.smithi176.stdout:1/178: unlink d4/c36 0 2024-07-16T09:55:34.328 INFO:tasks.workunit.client.1.smithi176.stdout:0/290: getdents d5/d44 0 2024-07-16T09:55:34.329 INFO:tasks.workunit.client.0.smithi156.stdout:9/315: dwrite d2/d5/f55 [4194304,4194304] 0 2024-07-16T09:55:34.330 INFO:tasks.workunit.client.0.smithi156.stdout:8/234: write d0/d11/d13/d1c/f32 [671870,10193] 0 2024-07-16T09:55:34.331 INFO:tasks.workunit.client.1.smithi176.stdout:0/291: symlink d5/d25/l5d 0 2024-07-16T09:55:34.332 INFO:tasks.workunit.client.1.smithi176.stdout:1/179: symlink d4/d8/d12/d21/d34/l3c 0 2024-07-16T09:55:34.333 INFO:tasks.workunit.client.0.smithi156.stdout:9/316: symlink d2/d5/d16/d57/l67 0 2024-07-16T09:55:34.334 INFO:tasks.workunit.client.0.smithi156.stdout:0/214: truncate d0/d7/d9/f26 4298900 0 2024-07-16T09:55:34.334 INFO:tasks.workunit.client.1.smithi176.stdout:1/180: creat d4/d8/d12/d21/d34/f3d x:0 0 0 2024-07-16T09:55:34.335 INFO:tasks.workunit.client.0.smithi156.stdout:5/249: mkdir d1b/d1f/d57 0 2024-07-16T09:55:34.336 INFO:tasks.workunit.client.0.smithi156.stdout:9/317: write d2/d5/fb [728739,49960] 0 2024-07-16T09:55:34.342 INFO:tasks.workunit.client.0.smithi156.stdout:8/235: dwrite d0/dd/f26 [0,4194304] 0 2024-07-16T09:55:34.342 INFO:tasks.workunit.client.0.smithi156.stdout:0/215: rename d0/d7/d9/d2d to d0/d7/d9/d27/d3d 0 2024-07-16T09:55:34.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:55:33 smithi176 ceph-mon[36626]: pgmap v248: 65 pgs: 65 active+clean; 737 MiB data, 3.9 GiB used, 533 GiB / 536 GiB avail; 5.6 MiB/s rd, 83 MiB/s wr, 401 op/s 2024-07-16T09:55:34.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:55:33 smithi176 ceph-mon[36626]: from='client.24399 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-16T09:55:34.344 INFO:tasks.workunit.client.0.smithi156.stdout:0/216: chown d0/d7/d9/d27 256870318 1 2024-07-16T09:55:34.344 INFO:tasks.workunit.client.0.smithi156.stdout:3/240: dread d8/f1a [0,4194304] 0 2024-07-16T09:55:34.346 INFO:tasks.workunit.client.0.smithi156.stdout:8/236: chown d0/dd/d1f/f47 982 1 2024-07-16T09:55:34.356 INFO:tasks.workunit.client.0.smithi156.stdout:5/250: dread f12 [0,4194304] 0 2024-07-16T09:55:34.357 INFO:tasks.workunit.client.0.smithi156.stdout:8/237: mknod d0/d11/d20/d3f/c56 0 2024-07-16T09:55:34.359 INFO:tasks.workunit.client.0.smithi156.stdout:5/251: write d1b/f55 [219996,110718] 0 2024-07-16T09:55:34.359 INFO:tasks.workunit.client.0.smithi156.stdout:0/217: mkdir d0/d3e 0 2024-07-16T09:55:34.359 INFO:tasks.workunit.client.0.smithi156.stdout:3/241: link d8/l12 d8/da/d13/l4e 0 2024-07-16T09:55:34.360 INFO:tasks.workunit.client.0.smithi156.stdout:0/218: readlink d0/d22/l31 0 2024-07-16T09:55:34.363 INFO:tasks.workunit.client.1.smithi176.stdout:5/383: truncate d4/d14/d19/d1b/d61/f76 1153718 0 2024-07-16T09:55:34.365 INFO:tasks.workunit.client.1.smithi176.stdout:5/384: readlink d4/d14/d19/l5e 0 2024-07-16T09:55:34.365 INFO:tasks.workunit.client.0.smithi156.stdout:5/252: creat d1b/d1f/d45/d52/d43/f58 x:0 0 0 2024-07-16T09:55:34.366 INFO:tasks.workunit.client.0.smithi156.stdout:6/314: dwrite d3/d13/d17/d4f/f4b [0,4194304] 0 2024-07-16T09:55:34.366 INFO:tasks.workunit.client.1.smithi176.stdout:5/385: stat d4/d6c 0 2024-07-16T09:55:34.367 INFO:tasks.workunit.client.1.smithi176.stdout:1/181: truncate d4/d8/f29 3376139 0 2024-07-16T09:55:34.368 INFO:tasks.workunit.client.1.smithi176.stdout:3/207: dwrite d3/da/f13 [0,4194304] 0 2024-07-16T09:55:34.368 INFO:tasks.workunit.client.0.smithi156.stdout:5/253: write d1b/d1f/d45/d52/d43/f58 [417088,86474] 0 2024-07-16T09:55:34.369 INFO:tasks.workunit.client.1.smithi176.stdout:1/182: dread - d4/d8/d12/f2e zero size 2024-07-16T09:55:34.369 INFO:tasks.workunit.client.0.smithi156.stdout:6/315: write d3/d13/d17/d4f/d4d/d3b/f5d [3289929,108755] 0 2024-07-16T09:55:34.373 INFO:tasks.workunit.client.1.smithi176.stdout:1/183: mknod d4/d8/d2c/c3e 0 2024-07-16T09:55:34.373 INFO:tasks.workunit.client.1.smithi176.stdout:3/208: chown d3/c10 322555289 1 2024-07-16T09:55:34.376 INFO:tasks.workunit.client.1.smithi176.stdout:3/209: dread - d3/d17/d1d/d1e/f3a zero size 2024-07-16T09:55:34.377 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:34.376+0000 7f8ec5e52640 1 -- 172.21.15.156:0/3909826185 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8ec00fd640 msgr2=0x7f8ec00fdab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:55:34.377 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:34.376+0000 7f8ec5e52640 1 --2- 172.21.15.156:0/3909826185 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8ec00fd640 0x7f8ec00fdab0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f8eb00048d0 tx=0x7f8eb0009da0 comp rx=0 tx=0).stop 2024-07-16T09:55:34.377 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:34.376+0000 7f8ec5e52640 1 -- 172.21.15.156:0/3909826185 shutdown_connections 2024-07-16T09:55:34.377 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:34.376+0000 7f8ec5e52640 1 --2- 172.21.15.156:0/3909826185 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8ec00fd640 0x7f8ec00fdab0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:55:34.378 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:34.376+0000 7f8ec5e52640 1 --2- 172.21.15.156:0/3909826185 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8ec00fc370 0x7f8ec00fc780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:55:34.378 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:34.376+0000 7f8ec5e52640 1 -- 172.21.15.156:0/3909826185 >> 172.21.15.156:0/3909826185 conn(0x7f8ec00f7b30 msgr2=0x7f8ec00f9f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:55:34.378 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:34.376+0000 7f8ec5e52640 1 -- 172.21.15.156:0/3909826185 shutdown_connections 2024-07-16T09:55:34.378 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:34.376+0000 7f8ec5e52640 1 -- 172.21.15.156:0/3909826185 wait complete. 2024-07-16T09:55:34.378 INFO:tasks.workunit.client.1.smithi176.stdout:4/307: creat da/dc/f58 x:0 0 0 2024-07-16T09:55:34.378 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:34.377+0000 7f8ec5e52640 1 Processor -- start 2024-07-16T09:55:34.379 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:34.378+0000 7f8ec5e52640 1 -- start start 2024-07-16T09:55:34.379 INFO:tasks.workunit.client.0.smithi156.stdout:5/254: rename d1b/d1e/f4e to d1b/d1f/d57/f59 0 2024-07-16T09:55:34.379 INFO:tasks.workunit.client.0.smithi156.stdout:6/316: dwrite d3/d13/d17/d4f/d1f/fa [0,4194304] 0 2024-07-16T09:55:34.380 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:34.378+0000 7f8ec5e52640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8ec00fc370 0x7f8ec010cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:55:34.380 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:34.378+0000 7f8ec5e52640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8ec00fd640 0x7f8ec010d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:55:34.380 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:34.378+0000 7f8ec5e52640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ec010dab0 con 0x7f8ec00fd640 2024-07-16T09:55:34.380 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:34.378+0000 7f8ec5e52640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ec010e2d0 con 0x7f8ec00fc370 2024-07-16T09:55:34.381 INFO:tasks.workunit.client.1.smithi176.stdout:6/257: rename d1/d5/d7/d49 to d1/dc/d5b 0 2024-07-16T09:55:34.381 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:34.379+0000 7f8ebeffd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8ec00fd640 0x7f8ec010d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:55:34.381 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:34.379+0000 7f8ebeffd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8ec00fd640 0x7f8ec010d460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:40906/0 (socket says 172.21.15.156:40906) 2024-07-16T09:55:34.381 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:34.379+0000 7f8ebeffd640 1 -- 172.21.15.156:0/2049269205 learned_addr learned my addr 172.21.15.156:0/2049269205 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T09:55:34.381 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:34.379+0000 7f8ebf7fe640 1 --2- 172.21.15.156:0/2049269205 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8ec00fc370 0x7f8ec010cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:55:34.381 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:34.379+0000 7f8ebeffd640 1 -- 172.21.15.156:0/2049269205 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8ec00fc370 msgr2=0x7f8ec010cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:55:34.381 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:34.379+0000 7f8ebeffd640 1 --2- 172.21.15.156:0/2049269205 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8ec00fc370 0x7f8ec010cf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:55:34.381 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:34.379+0000 7f8ebeffd640 1 -- 172.21.15.156:0/2049269205 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8eb0005040 con 0x7f8ec00fd640 2024-07-16T09:55:34.382 INFO:tasks.workunit.client.0.smithi156.stdout:5/255: write d1b/f34 [654532,109994] 0 2024-07-16T09:55:34.382 INFO:tasks.workunit.client.0.smithi156.stdout:6/317: write d3/f25 [125084,47304] 0 2024-07-16T09:55:34.382 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:34.380+0000 7f8ebeffd640 1 --2- 172.21.15.156:0/2049269205 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8ec00fd640 0x7f8ec010d460 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f8eb00048a0 tx=0x7f8eb0009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:55:34.383 INFO:tasks.workunit.client.1.smithi176.stdout:7/293: rmdir d4/d5/d4b 0 2024-07-16T09:55:34.383 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:34.380+0000 7f8ea7fff640 1 -- 172.21.15.156:0/2049269205 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8eb0038070 con 0x7f8ec00fd640 2024-07-16T09:55:34.383 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:34.380+0000 7f8ea7fff640 1 -- 172.21.15.156:0/2049269205 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8eb002abc0 con 0x7f8ec00fd640 2024-07-16T09:55:34.383 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:34.380+0000 7f8ec5e52640 1 -- 172.21.15.156:0/2049269205 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ec010e4a0 con 0x7f8ec00fd640 2024-07-16T09:55:34.384 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:34.381+0000 7f8ea7fff640 1 -- 172.21.15.156:0/2049269205 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8eb003dc00 con 0x7f8ec00fd640 2024-07-16T09:55:34.384 INFO:tasks.workunit.client.1.smithi176.stdout:7/294: write d4/d5/f2f [3532450,45719] 0 2024-07-16T09:55:34.384 INFO:tasks.workunit.client.0.smithi156.stdout:4/242: sync 2024-07-16T09:55:34.384 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:34.382+0000 7f8ec5e52640 1 -- 172.21.15.156:0/2049269205 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ec010e9c0 con 0x7f8ec00fd640 2024-07-16T09:55:34.385 INFO:tasks.workunit.client.1.smithi176.stdout:7/295: chown d4 477674862 1 2024-07-16T09:55:34.386 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:34.384+0000 7f8ec5e52640 1 -- 172.21.15.156:0/2049269205 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ec00fb7d0 con 0x7f8ec00fd640 2024-07-16T09:55:34.386 INFO:tasks.workunit.client.1.smithi176.stdout:4/308: creat da/dc/d10/f59 x:0 0 0 2024-07-16T09:55:34.386 INFO:tasks.workunit.client.1.smithi176.stdout:3/210: dread d3/da/fe [0,4194304] 0 2024-07-16T09:55:34.387 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:34.385+0000 7f8ea7fff640 1 -- 172.21.15.156:0/2049269205 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 20) v1 ==== 88539+0+0 (secure 0 0 0) 0x7f8eb0030020 con 0x7f8ec00fd640 2024-07-16T09:55:34.387 INFO:tasks.workunit.client.1.smithi176.stdout:4/309: write da/f42 [445934,58457] 0 2024-07-16T09:55:34.387 INFO:tasks.workunit.client.0.smithi156.stdout:5/256: truncate d1b/d1e/f21 1339508 0 2024-07-16T09:55:34.388 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:34.386+0000 7f8ea7fff640 1 --2- 172.21.15.156:0/2049269205 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f8ea806a730 0x7f8ea806cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:55:34.388 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:34.386+0000 7f8ea7fff640 1 -- 172.21.15.156:0/2049269205 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f8eb00a94d0 con 0x7f8ec00fd640 2024-07-16T09:55:34.388 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:34.386+0000 7f8ebf7fe640 1 --2- 172.21.15.156:0/2049269205 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f8ea806a730 0x7f8ea806cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:55:34.388 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:34.387+0000 7f8ebf7fe640 1 --2- 172.21.15.156:0/2049269205 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f8ea806a730 0x7f8ea806cbe0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f8eb4009fd0 tx=0x7f8eb4009e50 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:55:34.390 INFO:tasks.workunit.client.0.smithi156.stdout:4/243: write d4/db/dc/f4d [1188022,72777] 0 2024-07-16T09:55:34.394 INFO:tasks.workunit.client.0.smithi156.stdout:4/244: getdents d4/d41 0 2024-07-16T09:55:34.395 INFO:tasks.workunit.client.1.smithi176.stdout:0/292: rename d5/d2c/f53 to d5/d1c/d24/d35/d41/f5e 0 2024-07-16T09:55:34.395 INFO:tasks.workunit.client.1.smithi176.stdout:5/386: dread d4/d14/d19/d1b/f26 [0,4194304] 0 2024-07-16T09:55:34.396 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:34.395+0000 7f8ea7fff640 1 -- 172.21.15.156:0/2049269205 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f8eb00767d0 con 0x7f8ec00fd640 2024-07-16T09:55:34.397 INFO:tasks.workunit.client.1.smithi176.stdout:5/387: chown d4/d10/d17/f63 15150 1 2024-07-16T09:55:34.397 INFO:tasks.workunit.client.0.smithi156.stdout:5/257: dread d1b/f31 [0,4194304] 0 2024-07-16T09:55:34.398 INFO:tasks.workunit.client.0.smithi156.stdout:3/242: dread d8/f10 [0,4194304] 0 2024-07-16T09:55:34.399 INFO:tasks.workunit.client.1.smithi176.stdout:6/258: creat d1/dc/d26/f5c x:0 0 0 2024-07-16T09:55:34.400 INFO:tasks.workunit.client.0.smithi156.stdout:5/258: write d1b/d1f/f44 [9471,2771] 0 2024-07-16T09:55:34.401 INFO:tasks.workunit.client.1.smithi176.stdout:6/259: write d1/dc/d40/f4d [144657,50591] 0 2024-07-16T09:55:34.402 INFO:tasks.workunit.client.0.smithi156.stdout:5/259: fdatasync ff 0 2024-07-16T09:55:34.402 INFO:tasks.workunit.client.0.smithi156.stdout:9/318: dwrite d2/f4 [0,4194304] 0 2024-07-16T09:55:34.403 INFO:tasks.workunit.client.1.smithi176.stdout:6/260: chown d1/d5/d7/d2f 1044 1 2024-07-16T09:55:34.403 INFO:tasks.workunit.client.1.smithi176.stdout:4/310: mknod da/dc/d10/c5a 0 2024-07-16T09:55:34.403 INFO:tasks.workunit.client.1.smithi176.stdout:6/261: stat d1/d5/d7 0 2024-07-16T09:55:34.404 INFO:tasks.workunit.client.0.smithi156.stdout:9/319: readlink d2/de/l18 0 2024-07-16T09:55:34.404 INFO:tasks.workunit.client.1.smithi176.stdout:4/311: fsync da/d3d/f1b 0 2024-07-16T09:55:34.405 INFO:tasks.workunit.client.1.smithi176.stdout:0/293: dread d5/d1c/f20 [0,4194304] 0 2024-07-16T09:55:34.406 INFO:tasks.workunit.client.1.smithi176.stdout:0/294: write d5/f23 [589094,4801] 0 2024-07-16T09:55:34.413 INFO:tasks.workunit.client.1.smithi176.stdout:6/262: rmdir d1/dc 39 2024-07-16T09:55:34.414 INFO:tasks.workunit.client.0.smithi156.stdout:4/245: rename d4/db/dc/f36 to d4/d23/f53 0 2024-07-16T09:55:34.414 INFO:tasks.workunit.client.0.smithi156.stdout:8/238: dwrite d0/f12 [0,4194304] 0 2024-07-16T09:55:34.415 INFO:tasks.workunit.client.1.smithi176.stdout:5/388: creat d4/d37/d88/f8f x:0 0 0 2024-07-16T09:55:34.416 INFO:tasks.workunit.client.0.smithi156.stdout:4/246: chown d4/db/d44 942783 1 2024-07-16T09:55:34.417 INFO:tasks.workunit.client.0.smithi156.stdout:3/243: rename d8/da/l11 to d8/da/d28/d40/l4f 0 2024-07-16T09:55:34.417 INFO:tasks.workunit.client.1.smithi176.stdout:3/211: rename d3/f2b to d3/f3c 0 2024-07-16T09:55:34.419 INFO:tasks.workunit.client.0.smithi156.stdout:8/239: unlink d0/fc 0 2024-07-16T09:55:34.420 INFO:tasks.workunit.client.1.smithi176.stdout:5/389: rmdir d4/d37/d72 39 2024-07-16T09:55:34.421 INFO:tasks.workunit.client.0.smithi156.stdout:9/320: dread d2/d5/d16/d22/d36/d33/d3a/f3b [0,4194304] 0 2024-07-16T09:55:34.424 INFO:tasks.workunit.client.1.smithi176.stdout:0/295: getdents d5/d2c 0 2024-07-16T09:55:34.425 INFO:tasks.workunit.client.0.smithi156.stdout:3/244: creat d8/da/d1d/d2c/d33/f50 x:0 0 0 2024-07-16T09:55:34.425 INFO:tasks.workunit.client.0.smithi156.stdout:8/240: mkdir d0/dd/d1f/d57 0 2024-07-16T09:55:34.426 INFO:tasks.workunit.client.0.smithi156.stdout:9/321: creat d2/d5/d16/d22/d2b/d3d/f68 x:0 0 0 2024-07-16T09:55:34.427 INFO:tasks.workunit.client.1.smithi176.stdout:6/263: dread d1/d5/d7/fe [0,4194304] 0 2024-07-16T09:55:34.427 INFO:tasks.workunit.client.1.smithi176.stdout:3/212: link d3/d17/d1d/d1e/c38 d3/da/c3d 0 2024-07-16T09:55:34.427 INFO:tasks.workunit.client.1.smithi176.stdout:5/390: creat d4/d37/d72/d82/f90 x:0 0 0 2024-07-16T09:55:34.427 INFO:tasks.workunit.client.0.smithi156.stdout:5/260: dread f14 [0,4194304] 0 2024-07-16T09:55:34.427 INFO:tasks.workunit.client.1.smithi176.stdout:7/296: dread d4/d5/f2f [0,4194304] 0 2024-07-16T09:55:34.428 INFO:tasks.workunit.client.0.smithi156.stdout:8/241: fdatasync d0/f8 0 2024-07-16T09:55:34.430 INFO:tasks.workunit.client.1.smithi176.stdout:5/391: truncate d4/d37/d72/d82/f64 3476098 0 2024-07-16T09:55:34.431 INFO:tasks.workunit.client.0.smithi156.stdout:5/261: creat d1b/d1e/d37/f5a x:0 0 0 2024-07-16T09:55:34.432 INFO:tasks.workunit.client.0.smithi156.stdout:9/322: fsync d2/d5/d16/d22/d36/d33/f2c 0 2024-07-16T09:55:34.432 INFO:tasks.workunit.client.0.smithi156.stdout:5/262: dread - d1b/d1e/f48 zero size 2024-07-16T09:55:34.433 INFO:tasks.workunit.client.0.smithi156.stdout:9/323: chown d2/de/l52 417 1 2024-07-16T09:55:34.433 INFO:tasks.workunit.client.1.smithi176.stdout:7/297: unlink d4/d47/f44 0 2024-07-16T09:55:34.434 INFO:tasks.workunit.client.0.smithi156.stdout:8/242: unlink d0/d11/d4a/c4b 0 2024-07-16T09:55:34.435 INFO:tasks.workunit.client.0.smithi156.stdout:5/263: creat d1b/d1f/d45/d52/d43/f5b x:0 0 0 2024-07-16T09:55:34.437 INFO:tasks.workunit.client.1.smithi176.stdout:0/296: link d5/d1c/d24/d26/c3e d5/d1c/d24/d35/d41/c5f 0 2024-07-16T09:55:34.438 INFO:tasks.workunit.client.0.smithi156.stdout:9/324: creat d2/d5/d16/d22/d2b/d3d/f69 x:0 0 0 2024-07-16T09:55:34.439 INFO:tasks.workunit.client.0.smithi156.stdout:5/264: rmdir d1b/d1f/d45 39 2024-07-16T09:55:34.439 INFO:tasks.workunit.client.1.smithi176.stdout:3/213: dwrite d3/d17/f26 [0,4194304] 0 2024-07-16T09:55:34.439 INFO:tasks.workunit.client.0.smithi156.stdout:3/245: link d8/da/d1d/d2c/l39 d8/da/d28/l51 0 2024-07-16T09:55:34.440 INFO:tasks.workunit.client.1.smithi176.stdout:7/298: dread d4/dd/f36 [0,4194304] 0 2024-07-16T09:55:34.442 INFO:tasks.workunit.client.0.smithi156.stdout:3/246: readlink d8/da/le 0 2024-07-16T09:55:34.443 INFO:tasks.workunit.client.1.smithi176.stdout:3/214: readlink d3/da/ld 0 2024-07-16T09:55:34.446 INFO:tasks.workunit.client.0.smithi156.stdout:5/265: creat d1b/d1f/d42/f5c x:0 0 0 2024-07-16T09:55:34.446 INFO:tasks.workunit.client.0.smithi156.stdout:8/243: dwrite d0/dd/f1e [0,4194304] 0 2024-07-16T09:55:34.446 INFO:tasks.workunit.client.1.smithi176.stdout:6/264: dwrite d1/d5/d47/f51 [0,4194304] 0 2024-07-16T09:55:34.447 INFO:tasks.workunit.client.1.smithi176.stdout:0/297: write d5/d1c/f20 [277504,92911] 0 2024-07-16T09:55:34.447 INFO:tasks.workunit.client.1.smithi176.stdout:5/392: dwrite d4/d37/d72/f4e [0,4194304] 0 2024-07-16T09:55:34.452 INFO:tasks.workunit.client.1.smithi176.stdout:3/215: unlink d3/f3c 0 2024-07-16T09:55:34.453 INFO:tasks.workunit.client.0.smithi156.stdout:3/247: dwrite d8/d14/f30 [0,4194304] 0 2024-07-16T09:55:34.454 INFO:tasks.workunit.client.0.smithi156.stdout:5/266: fsync d1b/d1f/d45/d52/d43/f58 0 2024-07-16T09:55:34.456 INFO:tasks.workunit.client.0.smithi156.stdout:3/248: chown d8/da/d13/f17 64401398 1 2024-07-16T09:55:34.459 INFO:tasks.workunit.client.1.smithi176.stdout:7/299: rmdir d4/d5/d10/d17 39 2024-07-16T09:55:34.459 INFO:tasks.workunit.client.1.smithi176.stdout:0/298: dread d5/f23 [0,4194304] 0 2024-07-16T09:55:34.462 INFO:tasks.workunit.client.0.smithi156.stdout:9/325: dread d2/d5/d16/d22/d36/d33/d23/f2e [0,4194304] 0 2024-07-16T09:55:34.463 INFO:tasks.workunit.client.1.smithi176.stdout:5/393: symlink d4/d14/d19/l91 0 2024-07-16T09:55:34.464 INFO:tasks.workunit.client.1.smithi176.stdout:0/299: write d5/d1c/d24/f34 [449416,103166] 0 2024-07-16T09:55:34.465 INFO:tasks.workunit.client.0.smithi156.stdout:9/326: dread d2/d5/d16/d22/d36/f59 [0,4194304] 0 2024-07-16T09:55:34.467 INFO:tasks.workunit.client.0.smithi156.stdout:8/244: rename d0/f6 to d0/dd/f58 0 2024-07-16T09:55:34.469 INFO:tasks.workunit.client.0.smithi156.stdout:5/267: mknod d1b/d1f/d45/d52/d49/c5d 0 2024-07-16T09:55:34.469 INFO:tasks.workunit.client.0.smithi156.stdout:8/245: fsync d0/d11/f48 0 2024-07-16T09:55:34.471 INFO:tasks.workunit.client.1.smithi176.stdout:7/300: dwrite d4/f27 [0,4194304] 0 2024-07-16T09:55:34.471 INFO:tasks.workunit.client.0.smithi156.stdout:9/327: dread - d2/d5/d16/d22/d36/d33/d23/f60 zero size 2024-07-16T09:55:34.472 INFO:tasks.workunit.client.0.smithi156.stdout:9/328: fdatasync d2/d5/d16/f37 0 2024-07-16T09:55:34.473 INFO:tasks.workunit.client.1.smithi176.stdout:6/265: rename d1/dc/d1c to d1/d5d 0 2024-07-16T09:55:34.473 INFO:tasks.workunit.client.0.smithi156.stdout:3/249: rename d8/d14/f47 to d8/da/d1d/d2c/f52 0 2024-07-16T09:55:34.475 INFO:tasks.workunit.client.1.smithi176.stdout:7/301: mkdir d4/d5/d21/d66 0 2024-07-16T09:55:34.476 INFO:tasks.workunit.client.0.smithi156.stdout:9/329: read d2/d5/d16/d22/d36/d33/d3a/f3b [1116978,65289] 0 2024-07-16T09:55:34.477 INFO:tasks.workunit.client.0.smithi156.stdout:8/246: dread d0/dd/f1a [0,4194304] 0 2024-07-16T09:55:34.479 INFO:tasks.workunit.client.0.smithi156.stdout:9/330: write d2/f15 [3681948,107673] 0 2024-07-16T09:55:34.480 INFO:tasks.workunit.client.0.smithi156.stdout:5/268: mkdir d1b/d1f/d45/d52/d5e 0 2024-07-16T09:55:34.481 INFO:tasks.workunit.client.0.smithi156.stdout:3/250: dwrite d8/da/d13/f20 [0,4194304] 0 2024-07-16T09:55:34.485 INFO:tasks.workunit.client.0.smithi156.stdout:5/269: creat d1b/d1f/d42/f5f x:0 0 0 2024-07-16T09:55:34.486 INFO:tasks.workunit.client.0.smithi156.stdout:8/247: mknod d0/d11/d4a/d55/c59 0 2024-07-16T09:55:34.486 INFO:tasks.workunit.client.0.smithi156.stdout:5/270: read - d1b/d1f/d42/f47 zero size 2024-07-16T09:55:34.487 INFO:tasks.workunit.client.0.smithi156.stdout:8/248: chown d0/d11/d4a/f4f 6050 1 2024-07-16T09:55:34.489 INFO:tasks.workunit.client.1.smithi176.stdout:6/266: dwrite d1/d5/f6 [4194304,4194304] 0 2024-07-16T09:55:34.489 INFO:tasks.workunit.client.0.smithi156.stdout:5/271: creat d1b/d1f/d45/d52/d49/f60 x:0 0 0 2024-07-16T09:55:34.490 INFO:tasks.workunit.client.0.smithi156.stdout:3/251: getdents d8/da/d28 0 2024-07-16T09:55:34.497 INFO:tasks.workunit.client.0.smithi156.stdout:5/272: mkdir d1b/d1f/d45/d52/d5e/d61 0 2024-07-16T09:55:34.497 INFO:tasks.workunit.client.1.smithi176.stdout:6/267: symlink d1/d5/d7/l5e 0 2024-07-16T09:55:34.498 INFO:tasks.workunit.client.0.smithi156.stdout:8/249: link d0/dd/d1f/l3d d0/d11/d13/d1c/l5a 0 2024-07-16T09:55:34.499 INFO:tasks.workunit.client.1.smithi176.stdout:6/268: write d1/d5/f27 [155745,61674] 0 2024-07-16T09:55:34.500 INFO:tasks.workunit.client.0.smithi156.stdout:3/252: getdents d8/da/d1d/d2c/d33 0 2024-07-16T09:55:34.501 INFO:tasks.workunit.client.1.smithi176.stdout:0/300: truncate d5/d1c/f1f 2095940 0 2024-07-16T09:55:34.503 INFO:tasks.workunit.client.1.smithi176.stdout:5/394: dwrite d4/d14/f25 [0,4194304] 0 2024-07-16T09:55:34.509 INFO:tasks.workunit.client.1.smithi176.stdout:0/301: rename d5/f1b to d5/d1c/d24/d35/d50/d58/f60 0 2024-07-16T09:55:34.509 INFO:tasks.workunit.client.0.smithi156.stdout:3/253: unlink d8/d34/f3a 0 2024-07-16T09:55:34.510 INFO:tasks.workunit.client.0.smithi156.stdout:5/273: dwrite d1b/d1e/f25 [0,4194304] 0 2024-07-16T09:55:34.511 INFO:tasks.workunit.client.1.smithi176.stdout:6/269: symlink d1/l5f 0 2024-07-16T09:55:34.512 INFO:tasks.workunit.client.1.smithi176.stdout:5/395: symlink d4/l92 0 2024-07-16T09:55:34.518 INFO:tasks.workunit.client.1.smithi176.stdout:5/396: link d4/d6c/f6f d4/d37/d72/d82/f93 0 2024-07-16T09:55:34.519 INFO:tasks.workunit.client.0.smithi156.stdout:8/250: link d0/d11/d13/d1c/l37 d0/d11/d29/l5b 0 2024-07-16T09:55:34.520 INFO:tasks.workunit.client.1.smithi176.stdout:6/270: link d1/dc/d5b/l4e d1/d5/d7/d2f/l60 0 2024-07-16T09:55:34.520 INFO:tasks.workunit.client.0.smithi156.stdout:8/251: truncate d0/dd/f3b 534771 0 2024-07-16T09:55:34.521 INFO:tasks.workunit.client.1.smithi176.stdout:7/302: dread d4/f11 [0,4194304] 0 2024-07-16T09:55:34.521 INFO:tasks.workunit.client.1.smithi176.stdout:0/302: dread d5/d39/f51 [0,4194304] 0 2024-07-16T09:55:34.522 INFO:tasks.workunit.client.0.smithi156.stdout:9/331: truncate d2/de/f5e 319760 0 2024-07-16T09:55:34.522 INFO:tasks.workunit.client.1.smithi176.stdout:6/271: fsync d1/dc/d26/f5c 0 2024-07-16T09:55:34.524 INFO:tasks.workunit.client.0.smithi156.stdout:5/274: creat d1b/d1f/d45/d52/d49/d4d/f62 x:0 0 0 2024-07-16T09:55:34.525 INFO:tasks.workunit.client.0.smithi156.stdout:3/254: link d8/da/f3d d8/da/d1d/f53 0 2024-07-16T09:55:34.525 INFO:tasks.workunit.client.1.smithi176.stdout:6/272: mkdir d1/d61 0 2024-07-16T09:55:34.526 INFO:tasks.workunit.client.1.smithi176.stdout:0/303: creat d5/d1c/d24/d35/f61 x:0 0 0 2024-07-16T09:55:34.528 INFO:tasks.workunit.client.1.smithi176.stdout:7/303: truncate d4/dd/f36 289915 0 2024-07-16T09:55:34.529 INFO:tasks.workunit.client.0.smithi156.stdout:8/252: rename d0/dd/f43 to d0/d11/d2e/f5c 0 2024-07-16T09:55:34.529 INFO:tasks.workunit.client.1.smithi176.stdout:5/397: dwrite d4/f27 [0,4194304] 0 2024-07-16T09:55:34.529 INFO:tasks.workunit.client.1.smithi176.stdout:6/273: mkdir d1/d5/d47/d62 0 2024-07-16T09:55:34.532 INFO:tasks.workunit.client.1.smithi176.stdout:7/304: truncate d4/d1f/d22/d51/f58 556758 0 2024-07-16T09:55:34.533 INFO:tasks.workunit.client.1.smithi176.stdout:0/304: creat d5/d1c/d24/d35/d50/f62 x:0 0 0 2024-07-16T09:55:34.533 INFO:tasks.workunit.client.1.smithi176.stdout:6/274: chown d1/d5/f27 166467850 1 2024-07-16T09:55:34.534 INFO:tasks.workunit.client.1.smithi176.stdout:5/398: mkdir d4/d6c/d94 0 2024-07-16T09:55:34.534 INFO:tasks.workunit.client.1.smithi176.stdout:0/305: write d5/f6 [4512710,5509] 0 2024-07-16T09:55:34.535 INFO:tasks.workunit.client.0.smithi156.stdout:3/255: rename f0 to d8/da/d28/d40/f54 0 2024-07-16T09:55:34.536 INFO:tasks.workunit.client.1.smithi176.stdout:5/399: write d4/d10/d17/f63 [298914,66973] 0 2024-07-16T09:55:34.536 INFO:tasks.workunit.client.1.smithi176.stdout:6/275: write d1/d5/d7/d1e/f36 [1477321,118387] 0 2024-07-16T09:55:34.538 INFO:tasks.workunit.client.1.smithi176.stdout:5/400: chown d4/f71 174 1 2024-07-16T09:55:34.539 INFO:tasks.workunit.client.1.smithi176.stdout:7/305: creat d4/d1f/d22/d51/f67 x:0 0 0 2024-07-16T09:55:34.539 INFO:tasks.workunit.client.0.smithi156.stdout:9/332: mkdir d2/d5/d16/d22/d36/d6a 0 2024-07-16T09:55:34.539 INFO:tasks.workunit.client.0.smithi156.stdout:3/256: dread - d8/da/d1d/d2c/d33/f50 zero size 2024-07-16T09:55:34.540 INFO:tasks.workunit.client.0.smithi156.stdout:5/275: dread ff [0,4194304] 0 2024-07-16T09:55:34.541 INFO:tasks.workunit.client.0.smithi156.stdout:9/333: stat d2/d5/f44 0 2024-07-16T09:55:34.541 INFO:tasks.workunit.client.0.smithi156.stdout:9/334: readlink d2/d5/d16/d22/l2d 0 2024-07-16T09:55:34.541 INFO:tasks.workunit.client.1.smithi176.stdout:7/306: chown d4/dd/f1a 6069 1 2024-07-16T09:55:34.545 INFO:tasks.workunit.client.0.smithi156.stdout:5/276: write d1b/f31 [1553583,104669] 0 2024-07-16T09:55:34.545 INFO:tasks.workunit.client.0.smithi156.stdout:3/257: rmdir d8/d34 0 2024-07-16T09:55:34.546 INFO:tasks.workunit.client.1.smithi176.stdout:0/306: mkdir d5/d1c/d24/d35/d50/d58/d63 0 2024-07-16T09:55:34.546 INFO:tasks.workunit.client.1.smithi176.stdout:5/401: dwrite d4/f68 [0,4194304] 0 2024-07-16T09:55:34.546 INFO:tasks.workunit.client.1.smithi176.stdout:0/307: chown d5/d1c/d5a 454852116 1 2024-07-16T09:55:34.548 INFO:tasks.workunit.client.1.smithi176.stdout:6/276: write d1/f31 [4814447,15335] 0 2024-07-16T09:55:34.550 INFO:tasks.workunit.client.0.smithi156.stdout:5/277: creat d1b/d1f/d45/d52/d43/f63 x:0 0 0 2024-07-16T09:55:34.550 INFO:tasks.workunit.client.1.smithi176.stdout:5/402: mkdir d4/d14/d19/d1b/d95 0 2024-07-16T09:55:34.551 INFO:tasks.workunit.client.0.smithi156.stdout:9/335: dread d2/d5/fb [0,4194304] 0 2024-07-16T09:55:34.552 INFO:tasks.workunit.client.1.smithi176.stdout:0/308: creat d5/d1c/d24/d35/d41/f64 x:0 0 0 2024-07-16T09:55:34.556 INFO:tasks.workunit.client.1.smithi176.stdout:7/307: creat d4/d5/d21/d66/f68 x:0 0 0 2024-07-16T09:55:34.558 INFO:tasks.workunit.client.1.smithi176.stdout:0/309: write d5/f23 [1408506,17431] 0 2024-07-16T09:55:34.560 INFO:tasks.workunit.client.1.smithi176.stdout:6/277: dread d1/d5/d7/d1e/f36 [0,4194304] 0 2024-07-16T09:55:34.560 INFO:tasks.workunit.client.1.smithi176.stdout:0/310: truncate d5/d44/f55 971752 0 2024-07-16T09:55:34.562 INFO:tasks.workunit.client.1.smithi176.stdout:0/311: stat d5/d1c/d24/d35/d41/f5e 0 2024-07-16T09:55:34.562 INFO:tasks.workunit.client.0.smithi156.stdout:9/336: dwrite d2/d5/d16/d22/d2b/d3d/f69 [0,4194304] 0 2024-07-16T09:55:34.564 INFO:tasks.workunit.client.0.smithi156.stdout:3/258: truncate d8/da/d13/f20 3841160 0 2024-07-16T09:55:34.565 INFO:tasks.workunit.client.1.smithi176.stdout:0/312: unlink d5/d39/f3a 0 2024-07-16T09:55:34.568 INFO:tasks.workunit.client.0.smithi156.stdout:8/253: dwrite d0/fb [0,4194304] 0 2024-07-16T09:55:34.568 INFO:tasks.workunit.client.0.smithi156.stdout:3/259: write d8/da/d13/f24 [2279688,97879] 0 2024-07-16T09:55:34.574 INFO:tasks.workunit.client.1.smithi176.stdout:7/308: write d4/f2b [50875,56096] 0 2024-07-16T09:55:34.575 INFO:tasks.workunit.client.0.smithi156.stdout:5/278: dwrite d1b/d1f/d45/d52/f35 [0,4194304] 0 2024-07-16T09:55:34.575 INFO:tasks.workunit.client.0.smithi156.stdout:5/279: chown d1b/d1f/f20 20 1 2024-07-16T09:55:34.576 INFO:tasks.workunit.client.0.smithi156.stdout:5/280: dread - d1b/d1f/d45/d52/d49/f53 zero size 2024-07-16T09:55:34.577 INFO:tasks.workunit.client.1.smithi176.stdout:7/309: fdatasync d4/f19 0 2024-07-16T09:55:34.578 INFO:tasks.workunit.client.1.smithi176.stdout:7/310: write d4/d5/d21/f5a [108629,116590] 0 2024-07-16T09:55:34.581 INFO:tasks.workunit.client.0.smithi156.stdout:5/281: creat d1b/d1f/d45/d52/d43/f64 x:0 0 0 2024-07-16T09:55:34.583 INFO:tasks.workunit.client.0.smithi156.stdout:9/337: write d2/d5/d16/d22/d36/f59 [307365,29831] 0 2024-07-16T09:55:34.586 INFO:tasks.workunit.client.0.smithi156.stdout:3/260: getdents d8/da/d21 0 2024-07-16T09:55:34.588 INFO:tasks.workunit.client.1.smithi176.stdout:6/278: dread d1/f31 [0,4194304] 0 2024-07-16T09:55:34.588 INFO:tasks.workunit.client.0.smithi156.stdout:8/254: dwrite d0/d11/d2e/f5c [0,4194304] 0 2024-07-16T09:55:34.591 INFO:tasks.workunit.client.0.smithi156.stdout:3/261: creat d8/d14/f55 x:0 0 0 2024-07-16T09:55:34.591 INFO:tasks.workunit.client.1.smithi176.stdout:6/279: write d1/d5/d7/d54/f55 [626142,10966] 0 2024-07-16T09:55:34.592 INFO:tasks.workunit.client.0.smithi156.stdout:8/255: unlink d0/d11/d20/f38 0 2024-07-16T09:55:34.595 INFO:tasks.workunit.client.0.smithi156.stdout:3/262: creat d8/da/d28/d40/f56 x:0 0 0 2024-07-16T09:55:34.595 INFO:tasks.workunit.client.0.smithi156.stdout:3/263: fdatasync d8/da/f2e 0 2024-07-16T09:55:34.596 INFO:tasks.workunit.client.0.smithi156.stdout:9/338: rename d2/d5/d61/d64 to d2/d5/d16/d22/d6b 0 2024-07-16T09:55:34.598 INFO:tasks.workunit.client.0.smithi156.stdout:8/256: creat d0/d11/d29/f5d x:0 0 0 2024-07-16T09:55:34.598 INFO:tasks.workunit.client.1.smithi176.stdout:6/280: getdents d1/dc 0 2024-07-16T09:55:34.598 INFO:tasks.workunit.client.1.smithi176.stdout:0/313: dread d5/f22 [0,4194304] 0 2024-07-16T09:55:34.599 INFO:tasks.workunit.client.1.smithi176.stdout:7/311: write f0 [1748284,54811] 0 2024-07-16T09:55:34.600 INFO:tasks.workunit.client.1.smithi176.stdout:7/312: read - d4/d1f/f62 zero size 2024-07-16T09:55:34.601 INFO:tasks.workunit.client.0.smithi156.stdout:8/257: unlink d0/d11/d29/c52 0 2024-07-16T09:55:34.601 INFO:tasks.workunit.client.1.smithi176.stdout:6/281: chown d1/d5/l11 0 1 2024-07-16T09:55:34.603 INFO:tasks.workunit.client.1.smithi176.stdout:0/314: rename d5/f14 to d5/d1c/d24/d35/d50/d5b/f65 0 2024-07-16T09:55:34.604 INFO:tasks.workunit.client.0.smithi156.stdout:8/258: dread - d0/d11/d20/d3a/f4e zero size 2024-07-16T09:55:34.604 INFO:tasks.workunit.client.0.smithi156.stdout:5/282: write d1b/d1e/f21 [1385942,87775] 0 2024-07-16T09:55:34.604 INFO:tasks.workunit.client.0.smithi156.stdout:5/283: symlink d1b/d1f/d42/l65 2 2024-07-16T09:55:34.606 INFO:tasks.workunit.client.0.smithi156.stdout:9/339: dwrite d2/d5/fb [0,4194304] 0 2024-07-16T09:55:34.608 INFO:tasks.workunit.client.1.smithi176.stdout:6/282: creat d1/d5/d7/f63 x:0 0 0 2024-07-16T09:55:34.608 INFO:tasks.workunit.client.0.smithi156.stdout:5/284: dread - d1b/d1f/d45/d52/d43/f64 zero size 2024-07-16T09:55:34.611 INFO:tasks.workunit.client.0.smithi156.stdout:9/340: dread d2/d5/d16/d22/d36/f59 [0,4194304] 0 2024-07-16T09:55:34.611 INFO:tasks.workunit.client.0.smithi156.stdout:5/285: mknod d1b/d1f/d42/c66 0 2024-07-16T09:55:34.612 INFO:tasks.workunit.client.1.smithi176.stdout:6/283: dread d1/f31 [0,4194304] 0 2024-07-16T09:55:34.613 INFO:tasks.workunit.client.1.smithi176.stdout:6/284: dread - d1/d5/f3e zero size 2024-07-16T09:55:34.613 INFO:tasks.workunit.client.1.smithi176.stdout:0/315: dread d5/f8 [0,4194304] 0 2024-07-16T09:55:34.615 INFO:tasks.workunit.client.0.smithi156.stdout:5/286: mkdir d1b/d1f/d45/d52/d67 0 2024-07-16T09:55:34.615 INFO:tasks.workunit.client.0.smithi156.stdout:5/287: chown l17 480768 1 2024-07-16T09:55:34.616 INFO:tasks.workunit.client.1.smithi176.stdout:6/285: write d1/dc/d26/f5c [314326,70879] 0 2024-07-16T09:55:34.617 INFO:tasks.workunit.client.0.smithi156.stdout:5/288: dread - d1b/d1f/d45/d52/d43/f63 zero size 2024-07-16T09:55:34.618 INFO:tasks.workunit.client.0.smithi156.stdout:5/289: write d1b/d1e/f25 [2118558,122621] 0 2024-07-16T09:55:34.619 INFO:tasks.workunit.client.1.smithi176.stdout:0/316: write d5/d1c/d24/d35/d41/f5e [583782,53112] 0 2024-07-16T09:55:34.619 INFO:tasks.workunit.client.1.smithi176.stdout:7/313: dwrite d4/dd/f2e [0,4194304] 0 2024-07-16T09:55:34.621 INFO:tasks.workunit.client.1.smithi176.stdout:0/317: fsync d5/f2a 0 2024-07-16T09:55:34.624 INFO:tasks.workunit.client.0.smithi156.stdout:5/290: getdents d1b/d1e/d37 0 2024-07-16T09:55:34.625 INFO:tasks.workunit.client.1.smithi176.stdout:6/286: link d1/d5/f3e d1/f64 0 2024-07-16T09:55:34.626 INFO:tasks.workunit.client.1.smithi176.stdout:6/287: write d1/dc/f17 [1656749,30878] 0 2024-07-16T09:55:34.628 INFO:tasks.workunit.client.0.smithi156.stdout:5/291: creat d1b/d1f/d45/d52/f68 x:0 0 0 2024-07-16T09:55:34.631 INFO:tasks.workunit.client.1.smithi176.stdout:6/288: unlink d1/d5/d7/fe 0 2024-07-16T09:55:34.631 INFO:tasks.workunit.client.0.smithi156.stdout:5/292: write d1b/d1e/f25 [2461178,93376] 0 2024-07-16T09:55:34.632 INFO:tasks.workunit.client.1.smithi176.stdout:0/318: write d5/d2c/f3f [745750,72965] 0 2024-07-16T09:55:34.632 INFO:tasks.workunit.client.0.smithi156.stdout:5/293: dread - d1b/d1e/f48 zero size 2024-07-16T09:55:34.632 INFO:tasks.workunit.client.1.smithi176.stdout:6/289: chown d1/dc/d26/f5a 1271943 1 2024-07-16T09:55:34.635 INFO:tasks.workunit.client.0.smithi156.stdout:9/341: dread d2/d5/ff [0,4194304] 0 2024-07-16T09:55:34.635 INFO:tasks.workunit.client.0.smithi156.stdout:8/259: dread d0/f8 [0,4194304] 0 2024-07-16T09:55:34.636 INFO:tasks.workunit.client.1.smithi176.stdout:6/290: mkdir d1/d61/d65 0 2024-07-16T09:55:34.636 INFO:tasks.workunit.client.0.smithi156.stdout:9/342: chown d2/la 3 1 2024-07-16T09:55:34.636 INFO:tasks.workunit.client.0.smithi156.stdout:5/294: creat d1b/d1f/d45/d52/d5e/d61/f69 x:0 0 0 2024-07-16T09:55:34.638 INFO:tasks.workunit.client.0.smithi156.stdout:5/295: write d1b/d1e/f48 [595747,118689] 0 2024-07-16T09:55:34.638 INFO:tasks.workunit.client.1.smithi176.stdout:0/319: creat d5/d1c/d24/d35/d50/d58/d63/f66 x:0 0 0 2024-07-16T09:55:34.639 INFO:tasks.workunit.client.0.smithi156.stdout:9/343: chown d2/d5/l4e 29452113 1 2024-07-16T09:55:34.639 INFO:tasks.workunit.client.0.smithi156.stdout:5/296: truncate f12 3829774 0 2024-07-16T09:55:34.639 INFO:tasks.workunit.client.1.smithi176.stdout:6/291: dread d1/dc/d26/f5c [0,4194304] 0 2024-07-16T09:55:34.639 INFO:tasks.workunit.client.0.smithi156.stdout:8/260: mkdir d0/d11/d29/d5e 0 2024-07-16T09:55:34.641 INFO:tasks.workunit.client.0.smithi156.stdout:9/344: dread d2/d5/d16/f2a [0,4194304] 0 2024-07-16T09:55:34.642 INFO:tasks.workunit.client.1.smithi176.stdout:0/320: symlink d5/d1c/d5a/l67 0 2024-07-16T09:55:34.644 INFO:tasks.workunit.client.0.smithi156.stdout:5/297: dread d1b/d1f/d45/d52/f39 [0,4194304] 0 2024-07-16T09:55:34.644 INFO:tasks.workunit.client.0.smithi156.stdout:5/298: chown d1b/d1e 1 1 2024-07-16T09:55:34.645 INFO:tasks.workunit.client.0.smithi156.stdout:9/345: unlink d2/d5/d16/d22/d36/d33/l4f 0 2024-07-16T09:55:34.646 INFO:tasks.workunit.client.1.smithi176.stdout:7/314: dread d4/d5/d21/f5a [0,4194304] 0 2024-07-16T09:55:34.647 INFO:tasks.workunit.client.1.smithi176.stdout:0/321: dread d5/d2c/f4d [0,4194304] 0 2024-07-16T09:55:34.647 INFO:tasks.workunit.client.1.smithi176.stdout:6/292: dwrite d1/d5/f2a [0,4194304] 0 2024-07-16T09:55:34.648 INFO:tasks.workunit.client.0.smithi156.stdout:5/299: creat d1b/d1f/d57/f6a x:0 0 0 2024-07-16T09:55:34.648 INFO:tasks.workunit.client.0.smithi156.stdout:5/300: chown d1b/d1f/d45/d52/d2d 693766768 1 2024-07-16T09:55:34.649 INFO:tasks.workunit.client.1.smithi176.stdout:0/322: truncate d5/f2f 959836 0 2024-07-16T09:55:34.650 INFO:tasks.workunit.client.0.smithi156.stdout:5/301: write d1b/d1f/f51 [98024,32068] 0 2024-07-16T09:55:34.650 INFO:tasks.workunit.client.0.smithi156.stdout:8/261: truncate d0/d11/d20/f25 4644472 0 2024-07-16T09:55:34.650 INFO:tasks.workunit.client.1.smithi176.stdout:7/315: symlink d4/d5/d21/l69 0 2024-07-16T09:55:34.651 INFO:tasks.workunit.client.0.smithi156.stdout:8/262: fdatasync d0/dd/f58 0 2024-07-16T09:55:34.652 INFO:tasks.workunit.client.1.smithi176.stdout:6/293: write d1/dc/fd [1168043,129015] 0 2024-07-16T09:55:34.652 INFO:tasks.workunit.client.0.smithi156.stdout:9/346: link d2/d5/l4e d2/d5/d16/d22/l6c 0 2024-07-16T09:55:34.653 INFO:tasks.workunit.client.0.smithi156.stdout:5/302: mknod d1b/d1f/d42/c6b 0 2024-07-16T09:55:34.653 INFO:tasks.workunit.client.1.smithi176.stdout:0/323: mknod d5/d1c/d5a/c68 0 2024-07-16T09:55:34.654 INFO:tasks.workunit.client.0.smithi156.stdout:5/303: readlink l17 0 2024-07-16T09:55:34.657 INFO:tasks.workunit.client.1.smithi176.stdout:0/324: truncate d5/d1c/d24/d35/d50/f62 207407 0 2024-07-16T09:55:34.657 INFO:tasks.workunit.client.1.smithi176.stdout:6/294: getdents d1 0 2024-07-16T09:55:34.658 INFO:tasks.workunit.client.1.smithi176.stdout:6/295: read d1/d5/f6 [8073258,120679] 0 2024-07-16T09:55:34.658 INFO:tasks.workunit.client.0.smithi156.stdout:9/347: read d2/d5/d16/d22/d36/d33/d23/f2e [3524804,49496] 0 2024-07-16T09:55:34.658 INFO:tasks.workunit.client.0.smithi156.stdout:8/263: mknod d0/d11/d29/c5f 0 2024-07-16T09:55:34.661 INFO:tasks.workunit.client.0.smithi156.stdout:8/264: mkdir d0/dd/d1f/d60 0 2024-07-16T09:55:34.663 INFO:tasks.workunit.client.0.smithi156.stdout:5/304: link l16 d1b/d1f/d45/d52/d2d/l6c 0 2024-07-16T09:55:34.664 INFO:tasks.workunit.client.0.smithi156.stdout:9/348: dwrite d2/d5/d16/d22/d2b/d3d/f69 [4194304,4194304] 0 2024-07-16T09:55:34.665 INFO:tasks.workunit.client.0.smithi156.stdout:8/265: mknod d0/d11/d20/d3a/c61 0 2024-07-16T09:55:34.665 INFO:tasks.workunit.client.0.smithi156.stdout:5/305: chown d1b/d1f/f44 5503815 1 2024-07-16T09:55:34.666 INFO:tasks.workunit.client.1.smithi176.stdout:6/296: dwrite d1/dc/d26/f58 [0,4194304] 0 2024-07-16T09:55:34.667 INFO:tasks.workunit.client.0.smithi156.stdout:5/306: write d1b/d1e/f25 [3789193,103147] 0 2024-07-16T09:55:34.672 INFO:tasks.workunit.client.1.smithi176.stdout:6/297: creat d1/d5/d7/d1e/f66 x:0 0 0 2024-07-16T09:55:34.673 INFO:tasks.workunit.client.0.smithi156.stdout:8/266: mkdir d0/d11/d62 0 2024-07-16T09:55:34.673 INFO:tasks.workunit.client.1.smithi176.stdout:6/298: fdatasync d1/d5/d7/d2f/f32 0 2024-07-16T09:55:34.674 INFO:tasks.workunit.client.0.smithi156.stdout:8/267: chown d0/d11/f16 4524 1 2024-07-16T09:55:34.674 INFO:tasks.workunit.client.0.smithi156.stdout:9/349: dwrite d2/de/d42/f4c [0,4194304] 0 2024-07-16T09:55:34.678 INFO:tasks.workunit.client.1.smithi176.stdout:6/299: write d1/d5/f2c [720220,100247] 0 2024-07-16T09:55:34.678 INFO:tasks.workunit.client.1.smithi176.stdout:7/316: dread f0 [0,4194304] 0 2024-07-16T09:55:34.678 INFO:tasks.workunit.client.1.smithi176.stdout:0/325: write d5/fb [6466983,98995] 0 2024-07-16T09:55:34.680 INFO:tasks.workunit.client.1.smithi176.stdout:7/317: dread - d4/d1f/d22/d51/f67 zero size 2024-07-16T09:55:34.681 INFO:tasks.workunit.client.0.smithi156.stdout:9/350: dwrite d2/f4 [0,4194304] 0 2024-07-16T09:55:34.683 INFO:tasks.workunit.client.0.smithi156.stdout:8/268: creat d0/d11/d20/d3a/f63 x:0 0 0 2024-07-16T09:55:34.684 INFO:tasks.workunit.client.1.smithi176.stdout:0/326: creat d5/d1c/d24/d26/f69 x:0 0 0 2024-07-16T09:55:34.685 INFO:tasks.workunit.client.0.smithi156.stdout:5/307: getdents d1b/d1f/d45/d52 0 2024-07-16T09:55:34.686 INFO:tasks.workunit.client.1.smithi176.stdout:0/327: chown d5/f22 1947 1 2024-07-16T09:55:34.686 INFO:tasks.workunit.client.0.smithi156.stdout:8/269: symlink d0/dd/l64 0 2024-07-16T09:55:34.687 INFO:tasks.workunit.client.1.smithi176.stdout:7/318: rmdir d4/d47/d5d 39 2024-07-16T09:55:34.687 INFO:tasks.workunit.client.1.smithi176.stdout:6/300: mkdir d1/d5/d7/d1e/d67 0 2024-07-16T09:55:34.690 INFO:tasks.workunit.client.1.smithi176.stdout:0/328: mknod d5/d1c/d24/d35/d50/c6a 0 2024-07-16T09:55:34.691 INFO:tasks.workunit.client.0.smithi156.stdout:5/308: mkdir d1b/d1f/d42/d6d 0 2024-07-16T09:55:34.692 INFO:tasks.workunit.client.0.smithi156.stdout:8/270: dread d0/f8 [0,4194304] 0 2024-07-16T09:55:34.693 INFO:tasks.workunit.client.0.smithi156.stdout:9/351: creat d2/d5/d16/d22/d2b/d3d/f6d x:0 0 0 2024-07-16T09:55:34.693 INFO:tasks.workunit.client.1.smithi176.stdout:0/329: symlink d5/d1c/d24/d26/l6b 0 2024-07-16T09:55:34.695 INFO:tasks.workunit.client.0.smithi156.stdout:5/309: creat d1b/d1f/d45/d52/d5e/f6e x:0 0 0 2024-07-16T09:55:34.695 INFO:tasks.workunit.client.0.smithi156.stdout:9/352: read d2/d5/d16/d22/d2b/d3d/d3c/f62 [1326166,19890] 0 2024-07-16T09:55:34.695 INFO:tasks.workunit.client.0.smithi156.stdout:8/271: truncate d0/d11/d13/d1c/d24/f36 36260 0 2024-07-16T09:55:34.696 INFO:tasks.workunit.client.1.smithi176.stdout:6/301: rename d1/d5d/l56 to d1/d5/d7/d1e/d29/l68 0 2024-07-16T09:55:34.696 INFO:tasks.workunit.client.0.smithi156.stdout:5/310: fdatasync d1b/d1e/f21 0 2024-07-16T09:55:34.696 INFO:tasks.workunit.client.0.smithi156.stdout:8/272: read - d0/d11/f16 zero size 2024-07-16T09:55:34.697 INFO:tasks.workunit.client.1.smithi176.stdout:0/330: stat d5/d1c/d24/d35/d59 0 2024-07-16T09:55:34.699 INFO:tasks.workunit.client.0.smithi156.stdout:9/353: rmdir d2/d5/d16/d22/d2b/d3d/d3c 39 2024-07-16T09:55:34.699 INFO:tasks.workunit.client.1.smithi176.stdout:7/319: dwrite d4/f19 [0,4194304] 0 2024-07-16T09:55:34.701 INFO:tasks.workunit.client.0.smithi156.stdout:8/273: mknod d0/dd/d1f/d50/c65 0 2024-07-16T09:55:34.701 INFO:tasks.workunit.client.0.smithi156.stdout:5/311: mknod d1b/d1f/d42/d6d/c6f 0 2024-07-16T09:55:34.704 INFO:tasks.workunit.client.1.smithi176.stdout:7/320: symlink d4/d47/d5d/l6a 0 2024-07-16T09:55:34.704 INFO:tasks.workunit.client.0.smithi156.stdout:9/354: creat d2/d5/d16/d57/d63/f6e x:0 0 0 2024-07-16T09:55:34.705 INFO:tasks.workunit.client.1.smithi176.stdout:0/331: getdents d5/d1c/d24 0 2024-07-16T09:55:34.706 INFO:tasks.workunit.client.1.smithi176.stdout:6/302: dwrite d1/d5/d7/d2f/f32 [0,4194304] 0 2024-07-16T09:55:34.708 INFO:tasks.workunit.client.0.smithi156.stdout:5/312: rename d1b/d1f/d45/d52/d49/c5d to d1b/c70 0 2024-07-16T09:55:34.708 INFO:tasks.workunit.client.1.smithi176.stdout:6/303: readlink d1/d5/d7/d1e/l34 0 2024-07-16T09:55:34.710 INFO:tasks.workunit.client.0.smithi156.stdout:9/355: mknod d2/d5/d16/c6f 0 2024-07-16T09:55:34.711 INFO:tasks.workunit.client.1.smithi176.stdout:0/332: rename d5/d1c/d24/d35/d41 to d5/d2c/d6c 0 2024-07-16T09:55:34.711 INFO:tasks.workunit.client.0.smithi156.stdout:5/313: mknod d1b/d1f/d45/d52/d49/c71 0 2024-07-16T09:55:34.713 INFO:tasks.workunit.client.1.smithi176.stdout:6/304: creat d1/d5/d7/d1e/f69 x:0 0 0 2024-07-16T09:55:34.713 INFO:tasks.workunit.client.0.smithi156.stdout:9/356: write d2/d5/d16/d22/d36/d33/d23/f2e [4464385,109273] 0 2024-07-16T09:55:34.715 INFO:tasks.workunit.client.0.smithi156.stdout:5/314: mkdir d1b/d1f/d54/d72 0 2024-07-16T09:55:34.716 INFO:tasks.workunit.client.0.smithi156.stdout:9/357: chown d2/d5/d16/d22/d2b/d3d/f68 14872 1 2024-07-16T09:55:34.717 INFO:tasks.workunit.client.0.smithi156.stdout:5/315: fsync f19 0 2024-07-16T09:55:34.718 INFO:tasks.workunit.client.0.smithi156.stdout:8/274: truncate d0/d11/d20/f25 1332712 0 2024-07-16T09:55:34.719 INFO:tasks.workunit.client.0.smithi156.stdout:8/275: dread - d0/d11/d29/f5d zero size 2024-07-16T09:55:34.720 INFO:tasks.workunit.client.1.smithi176.stdout:0/333: fsync f4 0 2024-07-16T09:55:34.721 INFO:tasks.workunit.client.1.smithi176.stdout:0/334: chown d5/ca 3529 1 2024-07-16T09:55:34.722 INFO:tasks.workunit.client.0.smithi156.stdout:9/358: rename d2/d5/d16/d22/d36/f56 to d2/d5/f70 0 2024-07-16T09:55:34.723 INFO:tasks.workunit.client.1.smithi176.stdout:7/321: link d4/d1f/c3c d4/d5/d10/d17/c6b 0 2024-07-16T09:55:34.724 INFO:tasks.workunit.client.0.smithi156.stdout:9/359: fdatasync d2/de/d42/f47 0 2024-07-16T09:55:34.726 INFO:tasks.workunit.client.1.smithi176.stdout:7/322: write d4/f2b [1220739,52552] 0 2024-07-16T09:55:34.726 INFO:tasks.workunit.client.1.smithi176.stdout:0/335: link d5/f23 d5/d1c/d5a/f6d 0 2024-07-16T09:55:34.727 INFO:tasks.workunit.client.1.smithi176.stdout:7/323: stat d4/d5/d10/d17/d1d/f61 0 2024-07-16T09:55:34.728 INFO:tasks.workunit.client.0.smithi156.stdout:8/276: rename d0/d11/d13/d1c/l37 to d0/dd/l66 0 2024-07-16T09:55:34.731 INFO:tasks.workunit.client.0.smithi156.stdout:8/277: symlink d0/dd/l67 0 2024-07-16T09:55:34.731 INFO:tasks.workunit.client.1.smithi176.stdout:7/324: rename d4/d5/d21/f5a to d4/d5/f6c 0 2024-07-16T09:55:34.732 INFO:tasks.workunit.client.1.smithi176.stdout:6/305: dread d1/d5/d7/f9 [0,4194304] 0 2024-07-16T09:55:34.732 INFO:tasks.workunit.client.1.smithi176.stdout:7/325: readlink d4/d1f/d22/l29 0 2024-07-16T09:55:34.733 INFO:tasks.workunit.client.0.smithi156.stdout:5/316: unlink d1b/c70 0 2024-07-16T09:55:34.733 INFO:tasks.workunit.client.0.smithi156.stdout:5/317: write d1b/d1f/d42/f47 [689653,8013] 0 2024-07-16T09:55:34.735 INFO:tasks.workunit.client.1.smithi176.stdout:6/306: symlink d1/dc/d5b/l6a 0 2024-07-16T09:55:34.736 INFO:tasks.workunit.client.0.smithi156.stdout:8/278: rename d0/d11/d20/d3a to d0/d11/d29/d68 0 2024-07-16T09:55:34.736 INFO:tasks.workunit.client.1.smithi176.stdout:6/307: readlink d1/dc/l1b 0 2024-07-16T09:55:34.736 INFO:tasks.workunit.client.1.smithi176.stdout:6/308: dread - d1/dc/d26/f5a zero size 2024-07-16T09:55:34.738 INFO:tasks.workunit.client.1.smithi176.stdout:7/326: dread d4/d5/d10/d17/d1d/f61 [0,4194304] 0 2024-07-16T09:55:34.740 INFO:tasks.workunit.client.0.smithi156.stdout:8/279: creat d0/d11/d20/d3f/f69 x:0 0 0 2024-07-16T09:55:34.741 INFO:tasks.workunit.client.1.smithi176.stdout:6/309: unlink d1/dc/d26/c41 0 2024-07-16T09:55:34.741 INFO:tasks.workunit.client.0.smithi156.stdout:5/318: dwrite d1b/d1f/f44 [0,4194304] 0 2024-07-16T09:55:34.741 INFO:tasks.workunit.client.1.smithi176.stdout:7/327: write d4/d5/f6c [159098,44465] 0 2024-07-16T09:55:34.742 INFO:tasks.workunit.client.0.smithi156.stdout:8/280: read d0/dd/f10 [3030789,26769] 0 2024-07-16T09:55:34.744 INFO:tasks.workunit.client.1.smithi176.stdout:6/310: mknod d1/d5/d7/d1e/d67/c6b 0 2024-07-16T09:55:34.748 INFO:tasks.workunit.client.1.smithi176.stdout:6/311: creat d1/d5/d47/f6c x:0 0 0 2024-07-16T09:55:34.749 INFO:tasks.workunit.client.0.smithi156.stdout:5/319: rename f12 to d1b/d1f/f73 0 2024-07-16T09:55:34.749 INFO:tasks.workunit.client.1.smithi176.stdout:7/328: getdents d4/d1f/d22/d51 0 2024-07-16T09:55:34.749 INFO:tasks.workunit.client.1.smithi176.stdout:6/312: chown d1/f45 29 1 2024-07-16T09:55:34.750 INFO:tasks.workunit.client.0.smithi156.stdout:5/320: stat d1b/d1f/d42/f47 0 2024-07-16T09:55:34.752 INFO:tasks.workunit.client.0.smithi156.stdout:8/281: rename d0/f8 to d0/dd/f6a 0 2024-07-16T09:55:34.752 INFO:tasks.workunit.client.1.smithi176.stdout:6/313: creat d1/dc/d5b/f6d x:0 0 0 2024-07-16T09:55:34.753 INFO:tasks.workunit.client.0.smithi156.stdout:5/321: creat d1b/d1f/d54/f74 x:0 0 0 2024-07-16T09:55:34.753 INFO:tasks.workunit.client.0.smithi156.stdout:8/282: fsync d0/ff 0 2024-07-16T09:55:34.754 INFO:tasks.workunit.client.1.smithi176.stdout:7/329: mkdir d4/d47/d6d 0 2024-07-16T09:55:34.755 INFO:tasks.workunit.client.0.smithi156.stdout:5/322: creat d1b/d1f/d57/f75 x:0 0 0 2024-07-16T09:55:34.759 INFO:tasks.workunit.client.1.smithi176.stdout:6/314: fsync d1/dc/d5b/f6d 0 2024-07-16T09:55:34.761 INFO:tasks.workunit.client.1.smithi176.stdout:6/315: write d1/d5/f2a [3533112,54632] 0 2024-07-16T09:55:34.762 INFO:tasks.workunit.client.0.smithi156.stdout:5/323: dwrite d1b/d1f/d42/f47 [0,4194304] 0 2024-07-16T09:55:34.762 INFO:tasks.workunit.client.0.smithi156.stdout:8/283: truncate d0/dd/f1e 2643749 0 2024-07-16T09:55:34.764 INFO:tasks.workunit.client.1.smithi176.stdout:7/330: dwrite d4/dd/f36 [0,4194304] 0 2024-07-16T09:55:34.767 INFO:tasks.workunit.client.1.smithi176.stdout:6/316: creat d1/d5/d7/f6e x:0 0 0 2024-07-16T09:55:34.768 INFO:tasks.workunit.client.1.smithi176.stdout:6/317: fdatasync d1/dc/d26/f5a 0 2024-07-16T09:55:34.769 INFO:tasks.workunit.client.1.smithi176.stdout:7/331: creat d4/d47/d6d/f6e x:0 0 0 2024-07-16T09:55:34.770 INFO:tasks.workunit.client.0.smithi156.stdout:5/324: dwrite d1b/d1f/d45/d52/d49/d4d/f62 [0,4194304] 0 2024-07-16T09:55:34.771 INFO:tasks.workunit.client.1.smithi176.stdout:6/318: write d1/d5/f2a [5958309,6807] 0 2024-07-16T09:55:34.771 INFO:tasks.workunit.client.1.smithi176.stdout:7/332: write d4/d47/f46 [1410111,102333] 0 2024-07-16T09:55:34.771 INFO:tasks.workunit.client.0.smithi156.stdout:5/325: chown d1b/d1f/c38 184202 1 2024-07-16T09:55:34.772 INFO:tasks.workunit.client.1.smithi176.stdout:7/333: readlink d4/d5/l7 0 2024-07-16T09:55:34.776 INFO:tasks.workunit.client.0.smithi156.stdout:5/326: creat d1b/d1f/d45/d52/d2d/f76 x:0 0 0 2024-07-16T09:55:34.778 INFO:tasks.workunit.client.0.smithi156.stdout:5/327: write d1b/d1f/d45/d52/d5e/f6e [472939,113150] 0 2024-07-16T09:55:34.778 INFO:tasks.workunit.client.1.smithi176.stdout:7/334: rename d4/d5/d10/d17/d1d/f61 to d4/d1f/f6f 0 2024-07-16T09:55:34.779 INFO:tasks.workunit.client.0.smithi156.stdout:5/328: chown d1b/d1f 291 1 2024-07-16T09:55:34.779 INFO:tasks.workunit.client.1.smithi176.stdout:7/335: read d4/f19 [922438,52165] 0 2024-07-16T09:55:34.780 INFO:tasks.workunit.client.0.smithi156.stdout:5/329: write d1b/d1e/d37/f5a [988290,87023] 0 2024-07-16T09:55:34.783 INFO:tasks.workunit.client.1.smithi176.stdout:7/336: creat d4/dd/d63/f70 x:0 0 0 2024-07-16T09:55:34.789 INFO:tasks.workunit.client.0.smithi156.stdout:5/330: dwrite d1b/d1f/d45/d52/d43/f64 [0,4194304] 0 2024-07-16T09:55:34.789 INFO:tasks.workunit.client.1.smithi176.stdout:7/337: dwrite d4/dd/f36 [0,4194304] 0 2024-07-16T09:55:34.790 INFO:tasks.workunit.client.0.smithi156.stdout:8/284: dwrite d0/f2c [4194304,4194304] 0 2024-07-16T09:55:34.793 INFO:tasks.workunit.client.0.smithi156.stdout:5/331: mknod d1b/d1f/d45/d52/d43/c77 0 2024-07-16T09:55:34.794 INFO:tasks.workunit.client.1.smithi176.stdout:7/338: dread f0 [0,4194304] 0 2024-07-16T09:55:34.796 INFO:tasks.workunit.client.0.smithi156.stdout:5/332: mknod d1b/d1f/d45/d52/d5e/c78 0 2024-07-16T09:55:34.797 INFO:tasks.workunit.client.0.smithi156.stdout:5/333: chown d1b/d1f/d45/d52/f39 5 1 2024-07-16T09:55:34.799 INFO:tasks.workunit.client.0.smithi156.stdout:5/334: write d1b/d1f/d45/d52/d43/f64 [1686798,88760] 0 2024-07-16T09:55:34.799 INFO:tasks.workunit.client.1.smithi176.stdout:7/339: symlink d4/dd/l71 0 2024-07-16T09:55:34.802 INFO:tasks.workunit.client.1.smithi176.stdout:7/340: read d4/dd/d49/f55 [4129472,43831] 0 2024-07-16T09:55:34.803 INFO:tasks.workunit.client.0.smithi156.stdout:5/335: creat d1b/d1f/d54/d72/f79 x:0 0 0 2024-07-16T09:55:34.804 INFO:tasks.workunit.client.0.smithi156.stdout:5/336: fdatasync d1b/d1f/d54/f74 0 2024-07-16T09:55:34.805 INFO:tasks.workunit.client.1.smithi176.stdout:6/319: dwrite d1/d5/d7/f52 [0,4194304] 0 2024-07-16T09:55:34.806 INFO:tasks.workunit.client.1.smithi176.stdout:7/341: symlink d4/d5/d10/l72 0 2024-07-16T09:55:34.807 INFO:tasks.workunit.client.0.smithi156.stdout:5/337: dread d1b/f55 [0,4194304] 0 2024-07-16T09:55:34.809 INFO:tasks.workunit.client.1.smithi176.stdout:7/342: creat d4/d5/d10/d17/d1d/f73 x:0 0 0 2024-07-16T09:55:34.814 INFO:tasks.workunit.client.1.smithi176.stdout:7/343: read d4/d47/f34 [3495507,117363] 0 2024-07-16T09:55:34.817 INFO:tasks.workunit.client.1.smithi176.stdout:7/344: unlink d4/dd/d63/f70 0 2024-07-16T09:55:34.820 INFO:tasks.workunit.client.1.smithi176.stdout:7/345: mknod d4/d47/d6d/c74 0 2024-07-16T09:55:34.823 INFO:tasks.workunit.client.0.smithi156.stdout:5/338: rmdir d1b 39 2024-07-16T09:55:34.823 INFO:tasks.workunit.client.1.smithi176.stdout:6/320: truncate d1/f45 929441 0 2024-07-16T09:55:34.826 INFO:tasks.workunit.client.0.smithi156.stdout:5/339: rmdir d1b/d1f/d45/d52/d5e/d61 39 2024-07-16T09:55:34.827 INFO:tasks.workunit.client.0.smithi156.stdout:5/340: write d1b/d1f/d42/f47 [3138320,4418] 0 2024-07-16T09:55:34.827 INFO:tasks.workunit.client.1.smithi176.stdout:7/346: link d4/d5/l38 d4/d47/l75 0 2024-07-16T09:55:34.829 INFO:tasks.workunit.client.0.smithi156.stdout:2/422: sync 2024-07-16T09:55:34.830 INFO:tasks.workunit.client.1.smithi176.stdout:6/321: dwrite d1/dc/d40/f4d [0,4194304] 0 2024-07-16T09:55:34.831 INFO:tasks.workunit.client.0.smithi156.stdout:5/341: rename d1b/d1f/d45/d52/d67 to d1b/d1f/d45/d7a 0 2024-07-16T09:55:34.832 INFO:tasks.workunit.client.0.smithi156.stdout:5/342: fdatasync d1b/d1f/d45/d52/d49/f60 0 2024-07-16T09:55:34.833 INFO:tasks.workunit.client.0.smithi156.stdout:5/343: dread - d1b/d1f/d57/f6a zero size 2024-07-16T09:55:34.834 INFO:tasks.workunit.client.1.smithi176.stdout:7/347: dread d4/dd/f36 [0,4194304] 0 2024-07-16T09:55:34.834 INFO:tasks.workunit.client.0.smithi156.stdout:5/344: write d1b/d1f/d45/d52/d49/f60 [469741,73994] 0 2024-07-16T09:55:34.837 INFO:tasks.workunit.client.0.smithi156.stdout:2/423: getdents d0/d13 0 2024-07-16T09:55:34.839 INFO:tasks.workunit.client.1.smithi176.stdout:6/322: dwrite d1/d5/f27 [0,4194304] 0 2024-07-16T09:55:34.839 INFO:tasks.workunit.client.1.smithi176.stdout:7/348: mknod d4/d5/d21/d66/c76 0 2024-07-16T09:55:34.842 INFO:tasks.workunit.client.0.smithi156.stdout:5/345: dwrite d1b/d1f/d45/d52/d43/f5b [0,4194304] 0 2024-07-16T09:55:34.843 INFO:tasks.workunit.client.0.smithi156.stdout:2/424: symlink d0/d5/d29/d2d/d5b/d5f/d89/l96 0 2024-07-16T09:55:34.843 INFO:tasks.workunit.client.0.smithi156.stdout:5/346: stat d1b/d1e/f21 0 2024-07-16T09:55:34.844 INFO:tasks.workunit.client.1.smithi176.stdout:6/323: write d1/dc/d26/f5c [1111771,113388] 0 2024-07-16T09:55:34.844 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:34.843+0000 7f8ec5e52640 1 -- 172.21.15.156:0/2049269205 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f8ec0104ec0 con 0x7f8ec00fd640 2024-07-16T09:55:34.845 INFO:tasks.workunit.client.1.smithi176.stdout:7/349: creat d4/d47/d5d/f77 x:0 0 0 2024-07-16T09:55:34.845 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:34.844+0000 7f8ea7fff640 1 -- 172.21.15.156:0/2049269205 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+849 (secure 0 0 0) 0x7f8eb0079df0 con 0x7f8ec00fd640 2024-07-16T09:55:34.846 INFO:teuthology.orchestra.run.smithi156.stdout:{ 2024-07-16T09:55:34.846 INFO:teuthology.orchestra.run.smithi156.stdout: "mon": { 2024-07-16T09:55:34.846 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph version 17.2.7-1104-ge6231b04 (e6231b04b52ebf3b9b31803282dd3074439e51ac) quincy (stable)": 2 2024-07-16T09:55:34.846 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-07-16T09:55:34.846 INFO:teuthology.orchestra.run.smithi156.stdout: "mgr": { 2024-07-16T09:55:34.846 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph version 17.2.7-1104-ge6231b04 (e6231b04b52ebf3b9b31803282dd3074439e51ac) quincy (stable)": 1, 2024-07-16T09:55:34.846 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph version 19.0.0-3495-ga6ca461b (a6ca461be7cc1aff3658c2c6df4cff975a24449a) squid (rc)": 1 2024-07-16T09:55:34.846 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-07-16T09:55:34.846 INFO:teuthology.orchestra.run.smithi156.stdout: "osd": { 2024-07-16T09:55:34.847 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph version 17.2.7-1104-ge6231b04 (e6231b04b52ebf3b9b31803282dd3074439e51ac) quincy (stable)": 6 2024-07-16T09:55:34.847 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-07-16T09:55:34.847 INFO:teuthology.orchestra.run.smithi156.stdout: "mds": { 2024-07-16T09:55:34.847 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph version 17.2.7-1104-ge6231b04 (e6231b04b52ebf3b9b31803282dd3074439e51ac) quincy (stable)": 4 2024-07-16T09:55:34.847 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-07-16T09:55:34.847 INFO:teuthology.orchestra.run.smithi156.stdout: "overall": { 2024-07-16T09:55:34.847 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph version 17.2.7-1104-ge6231b04 (e6231b04b52ebf3b9b31803282dd3074439e51ac) quincy (stable)": 13, 2024-07-16T09:55:34.847 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph version 19.0.0-3495-ga6ca461b (a6ca461be7cc1aff3658c2c6df4cff975a24449a) squid (rc)": 1 2024-07-16T09:55:34.847 INFO:teuthology.orchestra.run.smithi156.stdout: } 2024-07-16T09:55:34.847 INFO:teuthology.orchestra.run.smithi156.stdout:} 2024-07-16T09:55:34.848 INFO:tasks.workunit.client.1.smithi176.stdout:7/350: chown d4/c30 457820 1 2024-07-16T09:55:34.848 INFO:tasks.workunit.client.1.smithi176.stdout:6/324: mkdir d1/d5/d7/d1e/d29/d6f 0 2024-07-16T09:55:34.848 INFO:tasks.workunit.client.1.smithi176.stdout:8/424: sync 2024-07-16T09:55:34.849 INFO:tasks.workunit.client.1.smithi176.stdout:8/425: write d8/d14/f27 [1596617,122382] 0 2024-07-16T09:55:34.850 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:34.848+0000 7f8ec5e52640 1 -- 172.21.15.156:0/2049269205 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f8ea806a730 msgr2=0x7f8ea806cbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:55:34.850 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:34.848+0000 7f8ec5e52640 1 --2- 172.21.15.156:0/2049269205 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f8ea806a730 0x7f8ea806cbe0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f8eb4009fd0 tx=0x7f8eb4009e50 comp rx=0 tx=0).stop 2024-07-16T09:55:34.850 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:34.848+0000 7f8ec5e52640 1 -- 172.21.15.156:0/2049269205 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8ec00fd640 msgr2=0x7f8ec010d460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:55:34.850 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:34.848+0000 7f8ec5e52640 1 --2- 172.21.15.156:0/2049269205 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8ec00fd640 0x7f8ec010d460 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f8eb00048a0 tx=0x7f8eb0009da0 comp rx=0 tx=0).stop 2024-07-16T09:55:34.851 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:34.850+0000 7f8ec5e52640 1 -- 172.21.15.156:0/2049269205 shutdown_connections 2024-07-16T09:55:34.852 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:34.850+0000 7f8ec5e52640 1 --2- 172.21.15.156:0/2049269205 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f8ea806a730 0x7f8ea806cbe0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:55:34.852 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:34.850+0000 7f8ec5e52640 1 --2- 172.21.15.156:0/2049269205 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8ec00fd640 0x7f8ec010d460 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:55:34.852 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:34.850+0000 7f8ec5e52640 1 --2- 172.21.15.156:0/2049269205 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8ec00fc370 0x7f8ec010cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:55:34.852 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:34.850+0000 7f8ec5e52640 1 -- 172.21.15.156:0/2049269205 >> 172.21.15.156:0/2049269205 conn(0x7f8ec00f7b30 msgr2=0x7f8ec00f9170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:55:34.852 INFO:tasks.workunit.client.0.smithi156.stdout:2/425: getdents d0/d13/d17/d1c/d51 0 2024-07-16T09:55:34.853 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:34.851+0000 7f8ec5e52640 1 -- 172.21.15.156:0/2049269205 shutdown_connections 2024-07-16T09:55:34.853 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:34.851+0000 7f8ec5e52640 1 -- 172.21.15.156:0/2049269205 wait complete. 2024-07-16T09:55:34.853 INFO:tasks.workunit.client.0.smithi156.stdout:5/347: truncate d1b/d1f/d45/d52/d43/f5b 2292236 0 2024-07-16T09:55:34.855 INFO:tasks.workunit.client.1.smithi176.stdout:6/325: creat d1/d5/d7/d54/f70 x:0 0 0 2024-07-16T09:55:34.857 INFO:tasks.workunit.client.0.smithi156.stdout:5/348: creat d1b/f7b x:0 0 0 2024-07-16T09:55:34.858 INFO:tasks.workunit.client.1.smithi176.stdout:6/326: mkdir d1/dc/d40/d71 0 2024-07-16T09:55:34.860 INFO:tasks.workunit.client.0.smithi156.stdout:2/426: dwrite d0/d13/d17/d1c/f27 [0,4194304] 0 2024-07-16T09:55:34.862 INFO:tasks.workunit.client.1.smithi176.stdout:8/426: dwrite d8/d14/d1e/f60 [0,4194304] 0 2024-07-16T09:55:34.863 INFO:tasks.workunit.client.0.smithi156.stdout:2/427: dread - d0/d5/d29/d2d/f79 zero size 2024-07-16T09:55:34.932 INFO:tasks.workunit.client.0.smithi156.stdout:1/233: sync 2024-07-16T09:55:34.933 INFO:tasks.workunit.client.0.smithi156.stdout:1/234: dread - dc/d14/d1e/f3d zero size 2024-07-16T09:55:34.935 INFO:tasks.workunit.client.0.smithi156.stdout:1/235: dread dc/f4a [0,4194304] 0 2024-07-16T09:55:34.938 INFO:tasks.workunit.client.0.smithi156.stdout:1/236: unlink dc/c2b 0 2024-07-16T09:55:34.939 INFO:tasks.workunit.client.0.smithi156.stdout:1/237: rmdir dc/d14/d4c 0 2024-07-16T09:55:34.940 INFO:tasks.workunit.client.0.smithi156.stdout:1/238: read - dc/d14/f34 zero size 2024-07-16T09:55:34.941 INFO:tasks.workunit.client.0.smithi156.stdout:1/239: readlink dc/d24/l50 0 2024-07-16T09:55:34.942 INFO:tasks.workunit.client.0.smithi156.stdout:1/240: write dc/f3f [580056,45914] 0 2024-07-16T09:55:34.944 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:34.943+0000 7f910b815640 1 -- 172.21.15.156:0/2328200264 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f91040ff510 msgr2=0x7f91041018f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:55:34.944 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:34.943+0000 7f910b815640 1 --2- 172.21.15.156:0/2328200264 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f91040ff510 0x7f91041018f0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f90f40048d0 tx=0x7f90f4009da0 comp rx=0 tx=0).stop 2024-07-16T09:55:34.944 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:34.943+0000 7f910b815640 1 -- 172.21.15.156:0/2328200264 shutdown_connections 2024-07-16T09:55:34.945 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:34.943+0000 7f910b815640 1 --2- 172.21.15.156:0/2328200264 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f91040ff510 0x7f91041018f0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:55:34.945 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:34.943+0000 7f910b815640 1 --2- 172.21.15.156:0/2328200264 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f91040659d0 0x7f91040fefd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:55:34.945 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:34.943+0000 7f910b815640 1 -- 172.21.15.156:0/2328200264 >> 172.21.15.156:0/2328200264 conn(0x7f91040f6a20 msgr2=0x7f91040f8e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:55:34.945 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:34.943+0000 7f910b815640 1 -- 172.21.15.156:0/2328200264 shutdown_connections 2024-07-16T09:55:34.945 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:34.943+0000 7f910b815640 1 -- 172.21.15.156:0/2328200264 wait complete. 2024-07-16T09:55:34.946 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:34.944+0000 7f910b815640 1 Processor -- start 2024-07-16T09:55:34.946 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:34.944+0000 7f910b815640 1 -- start start 2024-07-16T09:55:34.946 INFO:tasks.workunit.client.0.smithi156.stdout:1/241: rename dc/f3f to dc/f51 0 2024-07-16T09:55:34.948 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:34.944+0000 7f910b815640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f91040659d0 0x7f91040fc9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:55:34.948 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:34.944+0000 7f910b815640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f91040ff510 0x7f91040faed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:55:34.948 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:34.944+0000 7f910b815640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91040fb460 con 0x7f91040659d0 2024-07-16T09:55:34.948 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:34.944+0000 7f910b815640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91040fb5a0 con 0x7f91040ff510 2024-07-16T09:55:34.948 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:34.944+0000 7f91095b1640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f91040659d0 0x7f91040fc9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:55:34.948 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:34.944+0000 7f91095b1640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f91040659d0 0x7f91040fc9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:40920/0 (socket says 172.21.15.156:40920) 2024-07-16T09:55:34.948 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:34.944+0000 7f91095b1640 1 -- 172.21.15.156:0/601728330 learned_addr learned my addr 172.21.15.156:0/601728330 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T09:55:34.948 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:34.944+0000 7f9108db0640 1 --2- 172.21.15.156:0/601728330 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f91040ff510 0x7f91040faed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:55:34.948 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:34.944+0000 7f91095b1640 1 -- 172.21.15.156:0/601728330 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f91040ff510 msgr2=0x7f91040faed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:55:34.949 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:34.944+0000 7f91095b1640 1 --2- 172.21.15.156:0/601728330 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f91040ff510 0x7f91040faed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:55:34.949 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:34.944+0000 7f91095b1640 1 -- 172.21.15.156:0/601728330 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90f4005040 con 0x7f91040659d0 2024-07-16T09:55:34.949 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:34.944+0000 7f9108db0640 1 --2- 172.21.15.156:0/601728330 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f91040ff510 0x7f91040faed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-16T09:55:34.949 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:34.945+0000 7f91095b1640 1 --2- 172.21.15.156:0/601728330 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f91040659d0 0x7f91040fc9e0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f910000eb70 tx=0x7f910000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:55:34.949 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:34.945+0000 7f90f9ffb640 1 -- 172.21.15.156:0/601728330 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9100007c30 con 0x7f91040659d0 2024-07-16T09:55:34.949 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:34.945+0000 7f90f9ffb640 1 -- 172.21.15.156:0/601728330 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f910000f040 con 0x7f91040659d0 2024-07-16T09:55:34.949 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:34.945+0000 7f910b815640 1 -- 172.21.15.156:0/601728330 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91040fb880 con 0x7f91040659d0 2024-07-16T09:55:34.949 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:34.945+0000 7f90f9ffb640 1 -- 172.21.15.156:0/601728330 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9100009680 con 0x7f91040659d0 2024-07-16T09:55:34.949 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:34.946+0000 7f910b815640 1 -- 172.21.15.156:0/601728330 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9104114ff0 con 0x7f91040659d0 2024-07-16T09:55:34.949 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:34.946+0000 7f90f9ffb640 1 -- 172.21.15.156:0/601728330 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 20) v1 ==== 88539+0+0 (secure 0 0 0) 0x7f910000c710 con 0x7f91040659d0 2024-07-16T09:55:34.949 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:34.947+0000 7f910b815640 1 -- 172.21.15.156:0/601728330 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91040fa6a0 con 0x7f91040659d0 2024-07-16T09:55:34.949 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:34.947+0000 7f90f9ffb640 1 --2- 172.21.15.156:0/601728330 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f90f006a5a0 0x7f90f006ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:55:34.950 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:34.947+0000 7f90f9ffb640 1 -- 172.21.15.156:0/601728330 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f910008c3d0 con 0x7f91040659d0 2024-07-16T09:55:34.953 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:34.947+0000 7f9108db0640 1 --2- 172.21.15.156:0/601728330 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f90f006a5a0 0x7f90f006ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:55:34.953 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:34.948+0000 7f9108db0640 1 --2- 172.21.15.156:0/601728330 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f90f006a5a0 0x7f90f006ca50 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f90f4000c00 tx=0x7f90f4035040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:55:34.954 INFO:tasks.workunit.client.1.smithi176.stdout:6/327: rename d1/d5/d7/d1e/d29 to d1/dc/d72 0 2024-07-16T09:55:34.954 INFO:tasks.workunit.client.1.smithi176.stdout:9/327: sync 2024-07-16T09:55:34.954 INFO:tasks.workunit.client.1.smithi176.stdout:2/318: sync 2024-07-16T09:55:34.954 INFO:tasks.workunit.client.1.smithi176.stdout:8/427: truncate d8/d20/d2a/d31/f69 358983 0 2024-07-16T09:55:34.954 INFO:tasks.workunit.client.1.smithi176.stdout:9/328: stat d5/d1d/f46 0 2024-07-16T09:55:34.954 INFO:tasks.workunit.client.1.smithi176.stdout:6/328: write d1/d5/d7/d1e/f69 [6857,52782] 0 2024-07-16T09:55:34.954 INFO:tasks.workunit.client.1.smithi176.stdout:9/329: mkdir d5/d1d/d67 0 2024-07-16T09:55:34.955 INFO:tasks.workunit.client.0.smithi156.stdout:1/242: truncate dc/f13 1403021 0 2024-07-16T09:55:34.955 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:34.954+0000 7f90f9ffb640 1 -- 172.21.15.156:0/601728330 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f9100055a40 con 0x7f91040659d0 2024-07-16T09:55:34.959 INFO:tasks.workunit.client.0.smithi156.stdout:1/243: truncate dc/d20/f49 193619 0 2024-07-16T09:55:34.965 INFO:tasks.workunit.client.0.smithi156.stdout:1/244: dwrite dc/d14/d1e/f29 [0,4194304] 0 2024-07-16T09:55:34.972 INFO:tasks.workunit.client.0.smithi156.stdout:1/245: chown dc/f13 28377 1 2024-07-16T09:55:34.974 INFO:tasks.workunit.client.0.smithi156.stdout:1/246: mknod dc/d14/c52 0 2024-07-16T09:55:34.975 INFO:tasks.workunit.client.0.smithi156.stdout:1/247: chown dc/d14/d1e/l3b 0 1 2024-07-16T09:55:34.987 INFO:tasks.workunit.client.0.smithi156.stdout:1/248: dwrite f4 [0,4194304] 0 2024-07-16T09:55:34.992 INFO:tasks.workunit.client.0.smithi156.stdout:1/249: creat dc/d14/d3e/d45/f53 x:0 0 0 2024-07-16T09:55:34.993 INFO:tasks.workunit.client.0.smithi156.stdout:1/250: readlink dc/l12 0 2024-07-16T09:55:35.000 INFO:tasks.workunit.client.0.smithi156.stdout:1/251: truncate dc/d14/f2d 934009 0 2024-07-16T09:55:35.001 INFO:tasks.workunit.client.0.smithi156.stdout:1/252: write f4 [3524539,35412] 0 2024-07-16T09:55:35.003 INFO:tasks.workunit.client.0.smithi156.stdout:1/253: symlink dc/d14/d1e/l54 0 2024-07-16T09:55:35.005 INFO:tasks.workunit.client.0.smithi156.stdout:1/254: mkdir dc/d14/d3e/d55 0 2024-07-16T09:55:35.009 INFO:tasks.workunit.client.0.smithi156.stdout:1/255: rename dc/d14/f1d to dc/f56 0 2024-07-16T09:55:35.010 INFO:tasks.workunit.client.0.smithi156.stdout:1/256: write dc/f56 [132981,74450] 0 2024-07-16T09:55:35.012 INFO:tasks.workunit.client.0.smithi156.stdout:1/257: fsync dc/f19 0 2024-07-16T09:55:35.012 INFO:tasks.workunit.client.0.smithi156.stdout:1/258: stat dc/d14/f34 0 2024-07-16T09:55:35.014 INFO:tasks.workunit.client.0.smithi156.stdout:1/259: mkdir dc/d14/d3e/d55/d57 0 2024-07-16T09:55:35.015 INFO:tasks.workunit.client.0.smithi156.stdout:1/260: write dc/d14/f48 [876695,90930] 0 2024-07-16T09:55:35.016 INFO:tasks.workunit.client.0.smithi156.stdout:1/261: readlink lb 0 2024-07-16T09:55:35.017 INFO:tasks.workunit.client.0.smithi156.stdout:1/262: mkdir dc/d14/d3e/d55/d58 0 2024-07-16T09:55:35.018 INFO:tasks.workunit.client.0.smithi156.stdout:1/263: chown dc/l12 30296 1 2024-07-16T09:55:35.019 INFO:tasks.workunit.client.0.smithi156.stdout:1/264: symlink dc/d14/d3e/d55/d58/l59 0 2024-07-16T09:55:35.020 INFO:tasks.workunit.client.0.smithi156.stdout:1/265: readlink dc/d24/l39 0 2024-07-16T09:55:35.020 INFO:tasks.workunit.client.0.smithi156.stdout:1/266: chown c8 0 1 2024-07-16T09:55:35.031 INFO:tasks.workunit.client.0.smithi156.stdout:1/267: dread dc/d14/d1e/f35 [0,4194304] 0 2024-07-16T09:55:35.032 INFO:tasks.workunit.client.0.smithi156.stdout:1/268: symlink dc/d14/d3e/d45/l5a 0 2024-07-16T09:55:35.035 INFO:tasks.workunit.client.0.smithi156.stdout:1/269: rename dc/d14/c52 to dc/d14/d3e/d55/d58/c5b 0 2024-07-16T09:55:35.036 INFO:tasks.workunit.client.0.smithi156.stdout:1/270: write dc/d14/d1e/f21 [563789,55720] 0 2024-07-16T09:55:35.037 INFO:tasks.workunit.client.1.smithi176.stdout:6/329: mkdir d1/d5/d73 0 2024-07-16T09:55:35.037 INFO:tasks.workunit.client.1.smithi176.stdout:7/351: dwrite d4/dd/f37 [0,4194304] 0 2024-07-16T09:55:35.038 INFO:tasks.workunit.client.0.smithi156.stdout:1/271: creat dc/d14/d3e/d55/f5c x:0 0 0 2024-07-16T09:55:35.041 INFO:tasks.workunit.client.1.smithi176.stdout:8/428: dread d8/f17 [0,4194304] 0 2024-07-16T09:55:35.041 INFO:tasks.workunit.client.1.smithi176.stdout:6/330: chown d1/d5/d7/d1e/f36 52569801 1 2024-07-16T09:55:35.042 INFO:tasks.workunit.client.0.smithi156.stdout:1/272: creat dc/d14/d3e/f5d x:0 0 0 2024-07-16T09:55:35.042 INFO:tasks.workunit.client.1.smithi176.stdout:2/319: dwrite dd/d36/f4d [4194304,4194304] 0 2024-07-16T09:55:35.043 INFO:tasks.workunit.client.1.smithi176.stdout:7/352: write d4/fb [279293,84827] 0 2024-07-16T09:55:35.043 INFO:tasks.workunit.client.1.smithi176.stdout:9/330: truncate d5/f1e 191158 0 2024-07-16T09:55:35.045 INFO:tasks.workunit.client.1.smithi176.stdout:8/429: mknod d8/c94 0 2024-07-16T09:55:35.046 INFO:tasks.workunit.client.1.smithi176.stdout:9/331: mknod d5/d1d/c68 0 2024-07-16T09:55:35.047 INFO:tasks.workunit.client.1.smithi176.stdout:9/332: chown d5/d52 3 1 2024-07-16T09:55:35.047 INFO:tasks.workunit.client.0.smithi156.stdout:1/273: getdents dc/d14/d1e 0 2024-07-16T09:55:35.048 INFO:tasks.workunit.client.1.smithi176.stdout:2/320: unlink dd/d2e/c46 0 2024-07-16T09:55:35.050 INFO:tasks.workunit.client.1.smithi176.stdout:9/333: creat d5/d43/f69 x:0 0 0 2024-07-16T09:55:35.051 INFO:tasks.workunit.client.1.smithi176.stdout:9/334: write d5/f10 [1323891,76421] 0 2024-07-16T09:55:35.055 INFO:tasks.workunit.client.1.smithi176.stdout:2/321: mkdir dd/d2e/d35/d59/d66 0 2024-07-16T09:55:35.057 INFO:tasks.workunit.client.1.smithi176.stdout:6/331: write d1/d5/f3e [129983,9068] 0 2024-07-16T09:55:35.057 INFO:tasks.workunit.client.1.smithi176.stdout:7/353: link d4/l56 d4/dd/l78 0 2024-07-16T09:55:35.058 INFO:tasks.workunit.client.1.smithi176.stdout:2/322: rmdir dd/d16/d39/d3d 39 2024-07-16T09:55:35.059 INFO:tasks.workunit.client.1.smithi176.stdout:2/323: stat dd/d16/c5f 0 2024-07-16T09:55:35.059 INFO:tasks.workunit.client.1.smithi176.stdout:6/332: readlink d1/d5/l1a 0 2024-07-16T09:55:35.060 INFO:tasks.workunit.client.1.smithi176.stdout:8/430: dread d8/d14/f2d [0,4194304] 0 2024-07-16T09:55:35.060 INFO:tasks.workunit.client.1.smithi176.stdout:9/335: creat d5/d1d/d67/f6a x:0 0 0 2024-07-16T09:55:35.060 INFO:tasks.workunit.client.1.smithi176.stdout:6/333: chown d1/dc/d26 1 1 2024-07-16T09:55:35.061 INFO:tasks.workunit.client.1.smithi176.stdout:7/354: symlink d4/d1f/l79 0 2024-07-16T09:55:35.061 INFO:tasks.workunit.client.1.smithi176.stdout:2/324: write dd/de/f10 [191799,49489] 0 2024-07-16T09:55:35.063 INFO:tasks.workunit.client.1.smithi176.stdout:9/336: write d5/de/d21/f39 [4874765,15318] 0 2024-07-16T09:55:35.064 INFO:tasks.workunit.client.1.smithi176.stdout:6/334: chown d1/l3a 1 1 2024-07-16T09:55:35.064 INFO:tasks.workunit.client.0.smithi156.stdout:1/274: write dc/d20/f49 [1028864,52779] 0 2024-07-16T09:55:35.065 INFO:tasks.workunit.client.1.smithi176.stdout:8/431: chown d8/d20/d2a/d31/d2f/l81 2 1 2024-07-16T09:55:35.065 INFO:tasks.workunit.client.1.smithi176.stdout:7/355: chown d4/d5/d10/d17/d1d/l39 79440 1 2024-07-16T09:55:35.065 INFO:tasks.workunit.client.1.smithi176.stdout:8/432: stat d8/l30 0 2024-07-16T09:55:35.066 INFO:tasks.workunit.client.1.smithi176.stdout:9/337: creat d5/de/d30/f6b x:0 0 0 2024-07-16T09:55:35.068 INFO:tasks.workunit.client.1.smithi176.stdout:6/335: creat d1/d5/d7/d1e/d67/f74 x:0 0 0 2024-07-16T09:55:35.069 INFO:tasks.workunit.client.0.smithi156.stdout:1/275: rename dc/f2f to dc/d24/f5e 0 2024-07-16T09:55:35.072 INFO:tasks.workunit.client.1.smithi176.stdout:8/433: rename d8/d20/l58 to d8/d20/d2a/d61/l95 0 2024-07-16T09:55:35.074 INFO:tasks.workunit.client.1.smithi176.stdout:6/336: mkdir d1/d5/d47/d75 0 2024-07-16T09:55:35.074 INFO:tasks.workunit.client.1.smithi176.stdout:7/356: dwrite d4/d1f/f59 [0,4194304] 0 2024-07-16T09:55:35.076 INFO:tasks.workunit.client.1.smithi176.stdout:7/357: dread - d4/d47/d6d/f6e zero size 2024-07-16T09:55:35.076 INFO:tasks.workunit.client.0.smithi156.stdout:1/276: link dc/l16 dc/d14/d3e/d55/d57/l5f 0 2024-07-16T09:55:35.076 INFO:tasks.workunit.client.1.smithi176.stdout:6/337: dread - d1/d5/f4f zero size 2024-07-16T09:55:35.079 INFO:tasks.workunit.client.0.smithi156.stdout:1/277: mkdir dc/d14/d3e/d55/d57/d60 0 2024-07-16T09:55:35.081 INFO:tasks.workunit.client.1.smithi176.stdout:7/358: mkdir d4/dd/d49/d7a 0 2024-07-16T09:55:35.081 INFO:tasks.workunit.client.1.smithi176.stdout:2/325: dread dd/d16/f33 [0,4194304] 0 2024-07-16T09:55:35.081 INFO:tasks.workunit.client.1.smithi176.stdout:9/338: truncate d5/de/f17 114896 0 2024-07-16T09:55:35.082 INFO:tasks.workunit.client.0.smithi156.stdout:1/278: symlink dc/d20/l61 0 2024-07-16T09:55:35.082 INFO:tasks.workunit.client.1.smithi176.stdout:2/326: write dd/d16/f3a [561122,53633] 0 2024-07-16T09:55:35.082 INFO:tasks.workunit.client.1.smithi176.stdout:9/339: dread - d5/de/f47 zero size 2024-07-16T09:55:35.083 INFO:tasks.workunit.client.1.smithi176.stdout:8/434: truncate d8/d23/d2c/f36 3176060 0 2024-07-16T09:55:35.084 INFO:tasks.workunit.client.1.smithi176.stdout:7/359: creat d4/dd/d26/f7b x:0 0 0 2024-07-16T09:55:35.086 INFO:tasks.workunit.client.1.smithi176.stdout:7/360: truncate d4/d5/d10/d17/f4d 446596 0 2024-07-16T09:55:35.086 INFO:tasks.workunit.client.1.smithi176.stdout:8/435: symlink d8/d23/d2c/d34/d62/l96 0 2024-07-16T09:55:35.088 INFO:tasks.workunit.client.1.smithi176.stdout:2/327: mknod dd/c67 0 2024-07-16T09:55:35.088 INFO:tasks.workunit.client.1.smithi176.stdout:1/184: sync 2024-07-16T09:55:35.089 INFO:tasks.workunit.client.1.smithi176.stdout:4/312: sync 2024-07-16T09:55:35.089 INFO:tasks.workunit.client.0.smithi156.stdout:1/279: truncate f4 4179347 0 2024-07-16T09:55:35.089 INFO:tasks.workunit.client.1.smithi176.stdout:7/361: fsync d4/dd/f36 0 2024-07-16T09:55:35.090 INFO:tasks.workunit.client.1.smithi176.stdout:8/436: symlink d8/d23/d2c/d34/d62/l97 0 2024-07-16T09:55:35.090 INFO:tasks.workunit.client.0.smithi156.stdout:1/280: mknod dc/d14/d3e/d55/c62 0 2024-07-16T09:55:35.091 INFO:tasks.workunit.client.1.smithi176.stdout:1/185: write d4/db/f1b [5662220,105530] 0 2024-07-16T09:55:35.096 INFO:tasks.workunit.client.1.smithi176.stdout:1/186: fdatasync d4/d8/d12/d21/d34/f3d 0 2024-07-16T09:55:35.100 INFO:tasks.workunit.client.1.smithi176.stdout:4/313: dread da/dc/d39/f3c [0,4194304] 0 2024-07-16T09:55:35.100 INFO:tasks.workunit.client.1.smithi176.stdout:2/328: dwrite dd/d16/f3a [0,4194304] 0 2024-07-16T09:55:35.100 INFO:tasks.workunit.client.1.smithi176.stdout:8/437: rmdir d8/d20/d2a/d31/d46/d77 39 2024-07-16T09:55:35.102 INFO:tasks.workunit.client.1.smithi176.stdout:2/329: write dd/d2e/d35/f47 [4024375,92136] 0 2024-07-16T09:55:35.103 INFO:tasks.workunit.client.1.smithi176.stdout:1/187: symlink d4/d8/d12/d21/d25/d30/l3f 0 2024-07-16T09:55:35.105 INFO:tasks.workunit.client.1.smithi176.stdout:9/340: dread d5/de/f1a [8388608,4194304] 0 2024-07-16T09:55:35.105 INFO:tasks.workunit.client.1.smithi176.stdout:1/188: write d4/d8/f28 [2588774,108011] 0 2024-07-16T09:55:35.106 INFO:tasks.workunit.client.0.smithi156.stdout:1/281: dwrite dc/d14/f2d [0,4194304] 0 2024-07-16T09:55:35.107 INFO:tasks.workunit.client.1.smithi176.stdout:8/438: creat d8/d23/d76/f98 x:0 0 0 2024-07-16T09:55:35.108 INFO:tasks.workunit.client.1.smithi176.stdout:8/439: chown d8/c10 1006566 1 2024-07-16T09:55:35.110 INFO:tasks.workunit.client.1.smithi176.stdout:2/330: mknod dd/d16/c68 0 2024-07-16T09:55:35.113 INFO:tasks.workunit.client.1.smithi176.stdout:9/341: symlink d5/de/d37/d58/l6c 0 2024-07-16T09:55:35.117 INFO:tasks.workunit.client.1.smithi176.stdout:1/189: rename d4/d8/d2c/d2d/c37 to d4/db/c40 0 2024-07-16T09:55:35.117 INFO:tasks.workunit.client.0.smithi156.stdout:1/282: dwrite dc/d14/f34 [0,4194304] 0 2024-07-16T09:55:35.118 INFO:tasks.workunit.client.1.smithi176.stdout:1/190: fdatasync d4/d8/f26 0 2024-07-16T09:55:35.118 INFO:tasks.workunit.client.1.smithi176.stdout:9/342: fdatasync d5/de/f1a 0 2024-07-16T09:55:35.119 INFO:tasks.workunit.client.0.smithi156.stdout:1/283: rmdir dc/d14/d3e/d55/d57 39 2024-07-16T09:55:35.120 INFO:tasks.workunit.client.0.smithi156.stdout:1/284: chown dc/f51 64 1 2024-07-16T09:55:35.128 INFO:tasks.workunit.client.1.smithi176.stdout:2/331: rename dd/de/c22 to dd/d2e/c69 0 2024-07-16T09:55:35.131 INFO:tasks.workunit.client.1.smithi176.stdout:1/191: write d4/d8/d12/d21/d25/d30/f38 [4505867,9968] 0 2024-07-16T09:55:35.131 INFO:tasks.workunit.client.0.smithi156.stdout:7/253: sync 2024-07-16T09:55:35.131 INFO:tasks.workunit.client.0.smithi156.stdout:7/254: fdatasync dc/d13/f35 0 2024-07-16T09:55:35.133 INFO:tasks.workunit.client.1.smithi176.stdout:1/192: rename d4/d8/d12/c19 to d4/d8/d12/c41 0 2024-07-16T09:55:35.134 INFO:tasks.workunit.client.1.smithi176.stdout:7/362: truncate d4/d47/f46 1728758 0 2024-07-16T09:55:35.135 INFO:tasks.workunit.client.0.smithi156.stdout:7/255: unlink dc/d48/f52 0 2024-07-16T09:55:35.135 INFO:tasks.workunit.client.0.smithi156.stdout:7/256: fsync dc/f40 0 2024-07-16T09:55:35.136 INFO:tasks.workunit.client.1.smithi176.stdout:2/332: dwrite dd/d36/f2f [0,4194304] 0 2024-07-16T09:55:35.138 INFO:tasks.workunit.client.1.smithi176.stdout:9/343: getdents d5/de/d30 0 2024-07-16T09:55:35.139 INFO:tasks.workunit.client.1.smithi176.stdout:9/344: chown d5/de/d30/f57 45 1 2024-07-16T09:55:35.141 INFO:tasks.workunit.client.1.smithi176.stdout:8/440: truncate d8/d14/f27 3489806 0 2024-07-16T09:55:35.142 INFO:tasks.workunit.client.1.smithi176.stdout:9/345: write d5/d1d/d20/f25 [369,12374] 0 2024-07-16T09:55:35.142 INFO:tasks.workunit.client.1.smithi176.stdout:1/193: fsync d4/d8/f29 0 2024-07-16T09:55:35.149 INFO:tasks.workunit.client.1.smithi176.stdout:9/346: mkdir d5/d1d/d67/d6d 0 2024-07-16T09:55:35.149 INFO:tasks.workunit.client.0.smithi156.stdout:1/285: dwrite f6 [0,4194304] 0 2024-07-16T09:55:35.153 INFO:tasks.workunit.client.0.smithi156.stdout:7/257: dread dc/d13/f43 [0,4194304] 0 2024-07-16T09:55:35.153 INFO:tasks.workunit.client.0.smithi156.stdout:1/286: mknod dc/d14/d3e/c63 0 2024-07-16T09:55:35.153 INFO:tasks.workunit.client.1.smithi176.stdout:8/441: mknod d8/d14/d1e/d33/c99 0 2024-07-16T09:55:35.153 INFO:tasks.workunit.client.1.smithi176.stdout:9/347: dread - d5/de/f61 zero size 2024-07-16T09:55:35.153 INFO:tasks.workunit.client.1.smithi176.stdout:1/194: symlink d4/d8/d12/d21/l42 0 2024-07-16T09:55:35.157 INFO:tasks.workunit.client.1.smithi176.stdout:9/348: mknod d5/de/d30/c6e 0 2024-07-16T09:55:35.158 INFO:tasks.workunit.client.0.smithi156.stdout:1/287: write dc/f56 [608350,64232] 0 2024-07-16T09:55:35.158 INFO:tasks.workunit.client.0.smithi156.stdout:1/288: chown dc 0 1 2024-07-16T09:55:35.158 INFO:tasks.workunit.client.1.smithi176.stdout:2/333: truncate dd/d2e/d35/f3f 1181570 0 2024-07-16T09:55:35.160 INFO:tasks.workunit.client.1.smithi176.stdout:7/363: dwrite d4/f11 [0,4194304] 0 2024-07-16T09:55:35.161 INFO:tasks.workunit.client.1.smithi176.stdout:9/349: truncate d5/d1d/d20/f25 4746518 0 2024-07-16T09:55:35.163 INFO:tasks.workunit.client.1.smithi176.stdout:1/195: rmdir d4 39 2024-07-16T09:55:35.164 INFO:tasks.workunit.client.1.smithi176.stdout:7/364: fdatasync d4/d47/d6d/f6e 0 2024-07-16T09:55:35.166 INFO:tasks.workunit.client.1.smithi176.stdout:8/442: dwrite d8/d14/d1e/d55/f8a [0,4194304] 0 2024-07-16T09:55:35.166 INFO:tasks.workunit.client.1.smithi176.stdout:2/334: symlink dd/d2e/l6a 0 2024-07-16T09:55:35.172 INFO:tasks.workunit.client.0.smithi156.stdout:1/289: rmdir dc 39 2024-07-16T09:55:35.172 INFO:tasks.workunit.client.1.smithi176.stdout:7/365: write d4/d5/f6c [478159,64755] 0 2024-07-16T09:55:35.173 INFO:tasks.workunit.client.1.smithi176.stdout:1/196: creat d4/db/f43 x:0 0 0 2024-07-16T09:55:35.174 INFO:tasks.workunit.client.1.smithi176.stdout:8/443: truncate d8/d23/d2c/d37/f4f 1244349 0 2024-07-16T09:55:35.174 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:55:34 smithi156 ceph-mon[29302]: from='client.24403 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-16T09:55:35.174 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:55:34 smithi156 ceph-mon[29302]: from='client.24405 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-07-16T09:55:35.175 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:55:34 smithi156 ceph-mon[29302]: from='client.? 172.21.15.156:0/2049269205' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T09:55:35.180 INFO:tasks.workunit.client.1.smithi176.stdout:8/444: write d8/d23/d2c/d34/d62/f7d [589655,109618] 0 2024-07-16T09:55:35.181 INFO:tasks.workunit.client.0.smithi156.stdout:1/290: write dc/f37 [470416,35552] 0 2024-07-16T09:55:35.183 INFO:tasks.workunit.client.0.smithi156.stdout:1/291: symlink dc/d24/l64 0 2024-07-16T09:55:35.184 INFO:tasks.workunit.client.1.smithi176.stdout:9/350: truncate d5/fb 2966727 0 2024-07-16T09:55:35.184 INFO:tasks.workunit.client.1.smithi176.stdout:8/445: readlink d8/d20/d2a/d31/d2f/l6d 0 2024-07-16T09:55:35.185 INFO:tasks.workunit.client.1.smithi176.stdout:7/366: creat d4/dd/d49/f7c x:0 0 0 2024-07-16T09:55:35.185 INFO:tasks.workunit.client.0.smithi156.stdout:1/292: fdatasync dc/d24/f5e 0 2024-07-16T09:55:35.185 INFO:tasks.workunit.client.0.smithi156.stdout:1/293: chown dc/d24 1927 1 2024-07-16T09:55:35.187 INFO:tasks.workunit.client.1.smithi176.stdout:1/197: dwrite d4/f31 [0,4194304] 0 2024-07-16T09:55:35.187 INFO:tasks.workunit.client.0.smithi156.stdout:1/294: write dc/d14/d1e/f21 [3729688,119391] 0 2024-07-16T09:55:35.188 INFO:tasks.workunit.client.1.smithi176.stdout:9/351: rmdir d5/de/d30 39 2024-07-16T09:55:35.188 INFO:tasks.workunit.client.0.smithi156.stdout:1/295: truncate dc/d14/d3e/d55/f5c 736848 0 2024-07-16T09:55:35.190 INFO:tasks.workunit.client.1.smithi176.stdout:8/446: mkdir d8/d20/d2a/d53/d9a 0 2024-07-16T09:55:35.190 INFO:tasks.workunit.client.0.smithi156.stdout:1/296: truncate dc/d14/d3e/d45/f53 622682 0 2024-07-16T09:55:35.193 INFO:tasks.workunit.client.0.smithi156.stdout:1/297: rmdir dc/d14/d3e/d55/d58 39 2024-07-16T09:55:35.193 INFO:tasks.workunit.client.1.smithi176.stdout:8/447: chown d8/l9 183 1 2024-07-16T09:55:35.193 INFO:tasks.workunit.client.0.smithi156.stdout:1/298: fdatasync dc/f56 0 2024-07-16T09:55:35.197 INFO:tasks.workunit.client.0.smithi156.stdout:1/299: read dc/d14/d1e/f21 [2129945,51915] 0 2024-07-16T09:55:35.198 INFO:tasks.workunit.client.1.smithi176.stdout:8/448: write d8/d23/f21 [1350047,14852] 0 2024-07-16T09:55:35.198 INFO:tasks.workunit.client.1.smithi176.stdout:7/367: dwrite f0 [0,4194304] 0 2024-07-16T09:55:35.198 INFO:tasks.workunit.client.0.smithi156.stdout:1/300: rename dc/d14/d1e to dc/d14/d1e/d65 22 2024-07-16T09:55:35.199 INFO:tasks.workunit.client.0.smithi156.stdout:1/301: chown dc/d24/c44 1381394 1 2024-07-16T09:55:35.202 INFO:tasks.workunit.client.0.smithi156.stdout:1/302: mknod dc/c66 0 2024-07-16T09:55:35.205 INFO:tasks.workunit.client.1.smithi176.stdout:8/449: write d8/d14/d1e/d7c/f85 [162339,67839] 0 2024-07-16T09:55:35.205 INFO:tasks.workunit.client.1.smithi176.stdout:2/335: dread dd/f3c [0,4194304] 0 2024-07-16T09:55:35.206 INFO:tasks.workunit.client.1.smithi176.stdout:9/352: dwrite d5/de/f12 [4194304,4194304] 0 2024-07-16T09:55:35.206 INFO:tasks.workunit.client.0.smithi156.stdout:1/303: creat dc/d14/d3e/d55/d57/f67 x:0 0 0 2024-07-16T09:55:35.209 INFO:tasks.workunit.client.1.smithi176.stdout:9/353: write d5/d1d/f3c [193449,86304] 0 2024-07-16T09:55:35.211 INFO:tasks.workunit.client.0.smithi156.stdout:1/304: rmdir dc/d14/d3e/d55/d57/d60 0 2024-07-16T09:55:35.213 INFO:tasks.workunit.client.1.smithi176.stdout:7/368: dwrite d4/dd/f37 [0,4194304] 0 2024-07-16T09:55:35.214 INFO:tasks.workunit.client.0.smithi156.stdout:1/305: mknod dc/d24/c68 0 2024-07-16T09:55:35.214 INFO:tasks.workunit.client.1.smithi176.stdout:9/354: dread - d5/d1d/d20/f33 zero size 2024-07-16T09:55:35.215 INFO:tasks.workunit.client.1.smithi176.stdout:8/450: read d8/d23/f21 [3973936,57142] 0 2024-07-16T09:55:35.215 INFO:tasks.workunit.client.0.smithi156.stdout:1/306: chown dc/d24/f3c 5926486 1 2024-07-16T09:55:35.216 INFO:tasks.workunit.client.1.smithi176.stdout:9/355: fdatasync d5/d1d/f3f 0 2024-07-16T09:55:35.217 INFO:tasks.workunit.client.0.smithi156.stdout:1/307: write dc/d24/f3c [2788279,88548] 0 2024-07-16T09:55:35.217 INFO:tasks.workunit.client.1.smithi176.stdout:5/403: sync 2024-07-16T09:55:35.217 INFO:tasks.workunit.client.1.smithi176.stdout:3/216: sync 2024-07-16T09:55:35.218 INFO:tasks.workunit.client.0.smithi156.stdout:1/308: write dc/d14/d1e/f41 [811528,70428] 0 2024-07-16T09:55:35.221 INFO:tasks.workunit.client.0.smithi156.stdout:1/309: creat dc/d14/d3e/d45/f69 x:0 0 0 2024-07-16T09:55:35.221 INFO:tasks.workunit.client.1.smithi176.stdout:1/198: dread d4/d8/d2c/d2d/f32 [0,4194304] 0 2024-07-16T09:55:35.221 INFO:tasks.workunit.client.1.smithi176.stdout:5/404: mkdir d4/d37/d72/d82/d96 0 2024-07-16T09:55:35.223 INFO:tasks.workunit.client.1.smithi176.stdout:5/405: write d4/d14/d19/f21 [1852115,8886] 0 2024-07-16T09:55:35.227 INFO:tasks.workunit.client.1.smithi176.stdout:3/217: stat d3/d17/d1d/c2e 0 2024-07-16T09:55:35.230 INFO:tasks.workunit.client.0.smithi156.stdout:1/310: dwrite dc/d20/f36 [4194304,4194304] 0 2024-07-16T09:55:35.230 INFO:tasks.workunit.client.1.smithi176.stdout:1/199: dwrite d4/f24 [0,4194304] 0 2024-07-16T09:55:35.231 INFO:tasks.workunit.client.0.smithi156.stdout:1/311: readlink dc/d14/d3e/d45/l5a 0 2024-07-16T09:55:35.232 INFO:tasks.workunit.client.1.smithi176.stdout:7/369: getdents d4/d1f/d22 0 2024-07-16T09:55:35.234 INFO:tasks.workunit.client.0.smithi156.stdout:1/312: mknod dc/d20/c6a 0 2024-07-16T09:55:35.237 INFO:tasks.workunit.client.0.smithi156.stdout:1/313: readlink dc/d14/d3e/d55/d57/l5f 0 2024-07-16T09:55:35.238 INFO:tasks.workunit.client.1.smithi176.stdout:7/370: dread - d4/d5/d21/d66/f68 zero size 2024-07-16T09:55:35.238 INFO:tasks.workunit.client.1.smithi176.stdout:9/356: creat d5/d1d/d67/d6d/f6f x:0 0 0 2024-07-16T09:55:35.238 INFO:tasks.workunit.client.1.smithi176.stdout:1/200: mkdir d4/d8/d12/d21/d25/d30/d44 0 2024-07-16T09:55:35.241 INFO:tasks.workunit.client.0.smithi156.stdout:1/314: mkdir dc/d6b 0 2024-07-16T09:55:35.242 INFO:tasks.workunit.client.1.smithi176.stdout:5/406: dwrite d4/d6c/f6f [4194304,4194304] 0 2024-07-16T09:55:35.242 INFO:tasks.workunit.client.0.smithi156.stdout:1/315: chown dc/d14/d1e 23952 1 2024-07-16T09:55:35.245 INFO:tasks.workunit.client.1.smithi176.stdout:9/357: rmdir d5/d1d/d20 39 2024-07-16T09:55:35.245 INFO:tasks.workunit.client.1.smithi176.stdout:3/218: unlink d3/d17/d1d/d1e/f1f 0 2024-07-16T09:55:35.245 INFO:tasks.workunit.client.1.smithi176.stdout:8/451: link d8/d20/d2a/d31/d46/d77/l92 d8/d23/d2c/l9b 0 2024-07-16T09:55:35.246 INFO:tasks.workunit.client.0.smithi156.stdout:1/316: mkdir dc/d14/d3e/d6c 0 2024-07-16T09:55:35.246 INFO:tasks.workunit.client.1.smithi176.stdout:1/201: symlink d4/d8/d12/d21/d25/d30/l45 0 2024-07-16T09:55:35.247 INFO:tasks.workunit.client.1.smithi176.stdout:3/219: readlink l0 0 2024-07-16T09:55:35.248 INFO:tasks.workunit.client.1.smithi176.stdout:2/336: dread dd/d49/d52/f63 [0,4194304] 0 2024-07-16T09:55:35.248 INFO:tasks.workunit.client.0.smithi156.stdout:1/317: read dc/d14/d3e/d45/f53 [29482,23385] 0 2024-07-16T09:55:35.250 INFO:tasks.workunit.client.1.smithi176.stdout:5/407: creat d4/d37/d72/d38/f97 x:0 0 0 2024-07-16T09:55:35.252 INFO:tasks.workunit.client.0.smithi156.stdout:1/318: mknod dc/d14/c6d 0 2024-07-16T09:55:35.253 INFO:tasks.workunit.client.1.smithi176.stdout:2/337: write dd/d16/d39/f4b [341988,109799] 0 2024-07-16T09:55:35.254 INFO:tasks.workunit.client.1.smithi176.stdout:9/358: truncate d5/de/d30/f6b 775853 0 2024-07-16T09:55:35.255 INFO:tasks.workunit.client.1.smithi176.stdout:1/202: rename d4/c20 to d4/d8/d12/d21/d25/c46 0 2024-07-16T09:55:35.256 INFO:tasks.workunit.client.1.smithi176.stdout:2/338: chown dd/d49/f65 406705317 1 2024-07-16T09:55:35.257 INFO:tasks.workunit.client.1.smithi176.stdout:5/408: symlink d4/d10/d17/l98 0 2024-07-16T09:55:35.259 INFO:tasks.workunit.client.1.smithi176.stdout:1/203: fdatasync d4/d8/d12/d21/d25/d30/f38 0 2024-07-16T09:55:35.259 INFO:tasks.workunit.client.0.smithi156.stdout:1/319: dwrite dc/d14/d3e/f43 [0,4194304] 0 2024-07-16T09:55:35.260 INFO:tasks.workunit.client.1.smithi176.stdout:3/220: dwrite d3/d17/f26 [0,4194304] 0 2024-07-16T09:55:35.260 INFO:tasks.workunit.client.1.smithi176.stdout:5/409: chown d4/d37/d72/d38/l4c 14 1 2024-07-16T09:55:35.261 INFO:tasks.workunit.client.1.smithi176.stdout:2/339: mknod dd/d2e/d35/d4a/c6b 0 2024-07-16T09:55:35.261 INFO:tasks.workunit.client.1.smithi176.stdout:3/221: chown d3/d17/d1d/c2e 2058 1 2024-07-16T09:55:35.262 INFO:tasks.workunit.client.0.smithi156.stdout:1/320: mknod dc/d20/c6e 0 2024-07-16T09:55:35.262 INFO:tasks.workunit.client.1.smithi176.stdout:1/204: mknod d4/d8/d12/d21/d25/c47 0 2024-07-16T09:55:35.266 INFO:tasks.workunit.client.1.smithi176.stdout:3/222: symlink d3/d17/l3e 0 2024-07-16T09:55:35.266 INFO:tasks.workunit.client.0.smithi156.stdout:1/321: symlink dc/d14/d3e/d55/l6f 0 2024-07-16T09:55:35.267 INFO:tasks.workunit.client.1.smithi176.stdout:9/359: dread d5/d1d/f3c [0,4194304] 0 2024-07-16T09:55:35.268 INFO:tasks.workunit.client.1.smithi176.stdout:2/340: mknod dd/d2e/d35/d4a/c6c 0 2024-07-16T09:55:35.269 INFO:tasks.workunit.client.1.smithi176.stdout:9/360: write d5/de/d38/f4e [476620,18411] 0 2024-07-16T09:55:35.270 INFO:tasks.workunit.client.1.smithi176.stdout:3/223: write d3/d17/d1d/d1e/f3a [622925,70131] 0 2024-07-16T09:55:35.271 INFO:tasks.workunit.client.0.smithi156.stdout:1/322: creat dc/d14/d3e/d55/f70 x:0 0 0 2024-07-16T09:55:35.274 INFO:tasks.workunit.client.1.smithi176.stdout:5/410: link d4/f86 d4/d37/d72/d38/f99 0 2024-07-16T09:55:35.275 INFO:tasks.workunit.client.1.smithi176.stdout:3/224: creat d3/d17/d31/f3f x:0 0 0 2024-07-16T09:55:35.277 INFO:tasks.workunit.client.0.smithi156.stdout:1/323: link dc/d14/d1e/l54 dc/d20/l71 0 2024-07-16T09:55:35.277 INFO:tasks.workunit.client.1.smithi176.stdout:9/361: rename d5/de/c53 to d5/de/d30/c70 0 2024-07-16T09:55:35.279 INFO:tasks.workunit.client.1.smithi176.stdout:3/225: creat d3/d17/f40 x:0 0 0 2024-07-16T09:55:35.280 INFO:tasks.workunit.client.1.smithi176.stdout:5/411: chown d4/d14/d19/d1b/l75 6150254 1 2024-07-16T09:55:35.280 INFO:tasks.workunit.client.1.smithi176.stdout:9/362: mkdir d5/d52/d71 0 2024-07-16T09:55:35.280 INFO:tasks.workunit.client.1.smithi176.stdout:8/452: dwrite d8/d14/d1e/d33/f78 [0,4194304] 0 2024-07-16T09:55:35.281 INFO:tasks.workunit.client.1.smithi176.stdout:9/363: chown d5/la 6351843 1 2024-07-16T09:55:35.282 INFO:tasks.workunit.client.1.smithi176.stdout:8/453: stat d8/d20/d2a/l8e 0 2024-07-16T09:55:35.283 INFO:tasks.workunit.client.1.smithi176.stdout:3/226: truncate d3/d17/d1d/f2d 1168492 0 2024-07-16T09:55:35.286 INFO:tasks.workunit.client.0.smithi156.stdout:1/324: dwrite dc/f33 [0,4194304] 0 2024-07-16T09:55:35.287 INFO:tasks.workunit.client.0.smithi156.stdout:1/325: fdatasync dc/d20/f49 0 2024-07-16T09:55:35.293 INFO:tasks.workunit.client.0.smithi156.stdout:0/219: sync 2024-07-16T09:55:35.294 INFO:tasks.workunit.client.0.smithi156.stdout:1/326: fdatasync f4 0 2024-07-16T09:55:35.294 INFO:tasks.workunit.client.1.smithi176.stdout:5/412: dwrite d4/d10/f54 [0,4194304] 0 2024-07-16T09:55:35.296 INFO:tasks.workunit.client.1.smithi176.stdout:9/364: dread d5/f2d [0,4194304] 0 2024-07-16T09:55:35.297 INFO:tasks.workunit.client.0.smithi156.stdout:0/220: symlink d0/d7/d9/d27/l3f 0 2024-07-16T09:55:35.297 INFO:tasks.workunit.client.1.smithi176.stdout:8/454: dwrite d8/d14/d1e/d55/f8a [0,4194304] 0 2024-07-16T09:55:35.298 INFO:tasks.workunit.client.1.smithi176.stdout:3/227: dread d3/da/fe [0,4194304] 0 2024-07-16T09:55:35.299 INFO:tasks.workunit.client.0.smithi156.stdout:1/327: dread dc/d14/d1e/f41 [0,4194304] 0 2024-07-16T09:55:35.300 INFO:tasks.workunit.client.1.smithi176.stdout:5/413: write d4/d10/f54 [4089598,23866] 0 2024-07-16T09:55:35.306 INFO:tasks.workunit.client.0.smithi156.stdout:0/221: rename d0/d7/d9/f1d to d0/d7/d9/d27/d3d/f40 0 2024-07-16T09:55:35.307 INFO:tasks.workunit.client.0.smithi156.stdout:0/222: chown d0/d7/d9/d27/d2c/l39 41654274 1 2024-07-16T09:55:35.307 INFO:tasks.workunit.client.0.smithi156.stdout:1/328: dread dc/d14/d3e/d45/f53 [0,4194304] 0 2024-07-16T09:55:35.308 INFO:tasks.workunit.client.1.smithi176.stdout:9/365: mknod d5/d1d/d20/c72 0 2024-07-16T09:55:35.308 INFO:tasks.workunit.client.1.smithi176.stdout:3/228: creat d3/da/f41 x:0 0 0 2024-07-16T09:55:35.311 INFO:tasks.workunit.client.0.smithi156.stdout:0/223: write d0/d7/d9/f15 [7027866,115765] 0 2024-07-16T09:55:35.312 INFO:tasks.workunit.client.1.smithi176.stdout:8/455: dwrite d8/d14/d1e/f60 [0,4194304] 0 2024-07-16T09:55:35.312 INFO:tasks.workunit.client.0.smithi156.stdout:0/224: write d0/d7/f2e [736573,101526] 0 2024-07-16T09:55:35.313 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:35.311+0000 7f910b815640 1 -- 172.21.15.156:0/601728330 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f9104104ea0 con 0x7f91040659d0 2024-07-16T09:55:35.313 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:35.311+0000 7f90f9ffb640 1 -- 172.21.15.156:0/601728330 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 17 v17) v1 ==== 76+0+1852 (secure 0 0 0) 0x7f9100059060 con 0x7f91040659d0 2024-07-16T09:55:35.313 INFO:teuthology.orchestra.run.smithi156.stdout:e17 2024-07-16T09:55:35.313 INFO:teuthology.orchestra.run.smithi156.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-07-16T09:55:35.313 INFO:teuthology.orchestra.run.smithi156.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-07-16T09:55:35.313 INFO:teuthology.orchestra.run.smithi156.stdout:legacy client fscid: 1 2024-07-16T09:55:35.313 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-07-16T09:55:35.313 INFO:teuthology.orchestra.run.smithi156.stdout:Filesystem 'cephfs' (1) 2024-07-16T09:55:35.314 INFO:teuthology.orchestra.run.smithi156.stdout:fs_name cephfs 2024-07-16T09:55:35.314 INFO:teuthology.orchestra.run.smithi156.stdout:epoch 17 2024-07-16T09:55:35.314 INFO:teuthology.orchestra.run.smithi156.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-07-16T09:55:35.314 INFO:teuthology.orchestra.run.smithi156.stdout:created 2024-07-16T09:52:55.214975+0000 2024-07-16T09:55:35.314 INFO:teuthology.orchestra.run.smithi156.stdout:modified 2024-07-16T09:54:55.108622+0000 2024-07-16T09:55:35.314 INFO:teuthology.orchestra.run.smithi156.stdout:tableserver 0 2024-07-16T09:55:35.314 INFO:teuthology.orchestra.run.smithi156.stdout:root 0 2024-07-16T09:55:35.314 INFO:teuthology.orchestra.run.smithi156.stdout:session_timeout 60 2024-07-16T09:55:35.314 INFO:teuthology.orchestra.run.smithi156.stdout:session_autoclose 300 2024-07-16T09:55:35.314 INFO:teuthology.orchestra.run.smithi156.stdout:max_file_size 1099511627776 2024-07-16T09:55:35.314 INFO:teuthology.orchestra.run.smithi156.stdout:required_client_features {} 2024-07-16T09:55:35.315 INFO:teuthology.orchestra.run.smithi156.stdout:last_failure 0 2024-07-16T09:55:35.315 INFO:teuthology.orchestra.run.smithi156.stdout:last_failure_osd_epoch 0 2024-07-16T09:55:35.315 INFO:teuthology.orchestra.run.smithi156.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-07-16T09:55:35.315 INFO:teuthology.orchestra.run.smithi156.stdout:max_mds 2 2024-07-16T09:55:35.315 INFO:teuthology.orchestra.run.smithi156.stdout:in 0,1 2024-07-16T09:55:35.315 INFO:teuthology.orchestra.run.smithi156.stdout:up {0=14458,1=24261} 2024-07-16T09:55:35.315 INFO:teuthology.orchestra.run.smithi156.stdout:failed 2024-07-16T09:55:35.315 INFO:teuthology.orchestra.run.smithi156.stdout:damaged 2024-07-16T09:55:35.315 INFO:teuthology.orchestra.run.smithi156.stdout:stopped 2024-07-16T09:55:35.315 INFO:teuthology.orchestra.run.smithi156.stdout:data_pools [3] 2024-07-16T09:55:35.315 INFO:teuthology.orchestra.run.smithi156.stdout:metadata_pool 2 2024-07-16T09:55:35.316 INFO:teuthology.orchestra.run.smithi156.stdout:inline_data disabled 2024-07-16T09:55:35.316 INFO:teuthology.orchestra.run.smithi156.stdout:balancer 2024-07-16T09:55:35.316 INFO:teuthology.orchestra.run.smithi156.stdout:standby_count_wanted 1 2024-07-16T09:55:35.316 INFO:teuthology.orchestra.run.smithi156.stdout:[mds.cephfs.smithi156.rdibys{0:14458} state up:active seq 4 export targets 1 join_fscid=1 addr [v2:172.21.15.156:6826/1080176013,v1:172.21.15.156:6827/1080176013] compat {c=[1],r=[1],i=[7ff]}] 2024-07-16T09:55:35.316 INFO:teuthology.orchestra.run.smithi156.stdout:[mds.cephfs.smithi156.evwwaz{0:14468} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.156:6828/560329502,v1:172.21.15.156:6829/560329502] compat {c=[1],r=[1],i=[7ff]}] 2024-07-16T09:55:35.316 INFO:teuthology.orchestra.run.smithi156.stdout:[mds.cephfs.smithi176.dvvbud{1:24261} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.176:6824/1577019287,v1:172.21.15.176:6825/1577019287] compat {c=[1],r=[1],i=[7ff]}] 2024-07-16T09:55:35.316 INFO:teuthology.orchestra.run.smithi156.stdout:[mds.cephfs.smithi176.wahbsi{1:24273} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.176:6826/2196775412,v1:172.21.15.176:6827/2196775412] compat {c=[1],r=[1],i=[7ff]}] 2024-07-16T09:55:35.316 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-07-16T09:55:35.316 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-07-16T09:55:35.317 INFO:teuthology.orchestra.run.smithi156.stderr:dumped fsmap epoch 17 2024-07-16T09:55:35.317 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:35.315+0000 7f910b815640 1 -- 172.21.15.156:0/601728330 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f90f006a5a0 msgr2=0x7f90f006ca50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:55:35.317 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:35.315+0000 7f910b815640 1 --2- 172.21.15.156:0/601728330 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f90f006a5a0 0x7f90f006ca50 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f90f4000c00 tx=0x7f90f4035040 comp rx=0 tx=0).stop 2024-07-16T09:55:35.317 INFO:tasks.workunit.client.0.smithi156.stdout:1/329: link dc/d14/d1e/l1f dc/d20/l72 0 2024-07-16T09:55:35.318 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:35.315+0000 7f910b815640 1 -- 172.21.15.156:0/601728330 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f91040659d0 msgr2=0x7f91040fc9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:55:35.318 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:35.315+0000 7f910b815640 1 --2- 172.21.15.156:0/601728330 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f91040659d0 0x7f91040fc9e0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f910000eb70 tx=0x7f910000b040 comp rx=0 tx=0).stop 2024-07-16T09:55:35.318 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:35.315+0000 7f910b815640 1 -- 172.21.15.156:0/601728330 shutdown_connections 2024-07-16T09:55:35.318 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:35.315+0000 7f910b815640 1 --2- 172.21.15.156:0/601728330 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f90f006a5a0 0x7f90f006ca50 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:55:35.318 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:35.315+0000 7f910b815640 1 --2- 172.21.15.156:0/601728330 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f91040ff510 0x7f91040faed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:55:35.318 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:35.315+0000 7f910b815640 1 --2- 172.21.15.156:0/601728330 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f91040659d0 0x7f91040fc9e0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:55:35.318 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:35.315+0000 7f910b815640 1 -- 172.21.15.156:0/601728330 >> 172.21.15.156:0/601728330 conn(0x7f91040f6a20 msgr2=0x7f91040f6df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:55:35.318 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:35.315+0000 7f910b815640 1 -- 172.21.15.156:0/601728330 shutdown_connections 2024-07-16T09:55:35.319 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:35.315+0000 7f910b815640 1 -- 172.21.15.156:0/601728330 wait complete. 2024-07-16T09:55:35.319 INFO:tasks.workunit.client.1.smithi176.stdout:9/366: symlink d5/d52/d71/l73 0 2024-07-16T09:55:35.319 INFO:tasks.workunit.client.1.smithi176.stdout:8/456: fsync d8/d20/d2a/d31/d46/d6a/f7a 0 2024-07-16T09:55:35.319 INFO:tasks.workunit.client.1.smithi176.stdout:3/229: creat d3/d17/d31/f42 x:0 0 0 2024-07-16T09:55:35.320 INFO:tasks.workunit.client.1.smithi176.stdout:9/367: rename d5/de/d30/f57 to d5/d1d/d20/f74 0 2024-07-16T09:55:35.321 INFO:tasks.workunit.client.1.smithi176.stdout:9/368: write d5/d1d/d67/d6d/f6f [95435,4968] 0 2024-07-16T09:55:35.323 INFO:tasks.workunit.client.0.smithi156.stdout:1/330: getdents dc/d14/d3e/d55/d58 0 2024-07-16T09:55:35.324 INFO:tasks.workunit.client.1.smithi176.stdout:9/369: creat d5/d52/d71/f75 x:0 0 0 2024-07-16T09:55:35.328 INFO:tasks.workunit.client.1.smithi176.stdout:9/370: symlink d5/d52/l76 0 2024-07-16T09:55:35.330 INFO:tasks.workunit.client.1.smithi176.stdout:3/230: dwrite d3/d17/f2f [0,4194304] 0 2024-07-16T09:55:35.330 INFO:tasks.workunit.client.1.smithi176.stdout:9/371: fsync d5/de/d30/f6b 0 2024-07-16T09:55:35.332 INFO:tasks.workunit.client.1.smithi176.stdout:3/231: fsync d3/d17/d1d/f2d 0 2024-07-16T09:55:35.332 INFO:tasks.workunit.client.1.smithi176.stdout:5/414: dwrite d4/f86 [0,4194304] 0 2024-07-16T09:55:35.338 INFO:tasks.workunit.client.1.smithi176.stdout:9/372: mkdir d5/d1d/d77 0 2024-07-16T09:55:35.343 INFO:tasks.workunit.client.1.smithi176.stdout:9/373: chown d5/de/d30/f4a 15039 1 2024-07-16T09:55:35.343 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:55:34 smithi176 ceph-mon[36626]: from='client.24403 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-16T09:55:35.343 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:55:34 smithi176 ceph-mon[36626]: from='client.24405 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-07-16T09:55:35.343 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:55:34 smithi176 ceph-mon[36626]: from='client.? 172.21.15.156:0/2049269205' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T09:55:35.345 INFO:tasks.workunit.client.1.smithi176.stdout:3/232: dread d3/da/f13 [0,4194304] 0 2024-07-16T09:55:35.346 INFO:tasks.workunit.client.1.smithi176.stdout:9/374: mkdir d5/d1d/d20/d78 0 2024-07-16T09:55:35.348 INFO:tasks.workunit.client.1.smithi176.stdout:5/415: dwrite d4/d37/d72/d82/f83 [0,4194304] 0 2024-07-16T09:55:35.349 INFO:tasks.workunit.client.1.smithi176.stdout:9/375: stat d5/d1d/d20/f33 0 2024-07-16T09:55:35.351 INFO:tasks.workunit.client.1.smithi176.stdout:5/416: write d4/d37/d72/f4e [4468496,54853] 0 2024-07-16T09:55:35.355 INFO:tasks.workunit.client.1.smithi176.stdout:3/233: dwrite d3/d17/f40 [0,4194304] 0 2024-07-16T09:55:35.357 INFO:tasks.workunit.client.1.smithi176.stdout:3/234: readlink d3/d17/l1a 0 2024-07-16T09:55:35.360 INFO:tasks.workunit.client.0.smithi156.stdout:1/331: dread dc/d24/f3a [0,4194304] 0 2024-07-16T09:55:35.360 INFO:tasks.workunit.client.1.smithi176.stdout:9/376: dwrite d5/de/f47 [0,4194304] 0 2024-07-16T09:55:35.364 INFO:tasks.workunit.client.0.smithi156.stdout:1/332: rename dc/d20/f23 to dc/d14/d3e/d6c/f73 0 2024-07-16T09:55:35.366 INFO:tasks.workunit.client.1.smithi176.stdout:9/377: symlink d5/d1d/l79 0 2024-07-16T09:55:35.367 INFO:tasks.workunit.client.1.smithi176.stdout:6/338: sync 2024-07-16T09:55:35.368 INFO:tasks.workunit.client.1.smithi176.stdout:0/336: sync 2024-07-16T09:55:35.368 INFO:tasks.workunit.client.0.smithi156.stdout:1/333: creat dc/d6b/f74 x:0 0 0 2024-07-16T09:55:35.368 INFO:tasks.workunit.client.1.smithi176.stdout:9/378: unlink d5/d1d/f46 0 2024-07-16T09:55:35.369 INFO:tasks.workunit.client.1.smithi176.stdout:6/339: write d1/f64 [243754,106579] 0 2024-07-16T09:55:35.372 INFO:tasks.workunit.client.1.smithi176.stdout:0/337: creat d5/f6e x:0 0 0 2024-07-16T09:55:35.373 INFO:tasks.workunit.client.1.smithi176.stdout:9/379: creat d5/d1d/d67/d6d/f7a x:0 0 0 2024-07-16T09:55:35.373 INFO:tasks.workunit.client.1.smithi176.stdout:3/235: dread d3/d17/d1d/d1e/f21 [0,4194304] 0 2024-07-16T09:55:35.374 INFO:tasks.workunit.client.1.smithi176.stdout:9/380: stat d5/de/d38/f4e 0 2024-07-16T09:55:35.376 INFO:tasks.workunit.client.1.smithi176.stdout:6/340: rename d1/dc/d26 to d1/d61/d65/d76 0 2024-07-16T09:55:35.376 INFO:tasks.workunit.client.1.smithi176.stdout:0/338: truncate d5/d1c/d24/f30 1475914 0 2024-07-16T09:55:35.377 INFO:tasks.workunit.client.1.smithi176.stdout:9/381: mknod d5/d52/d71/c7b 0 2024-07-16T09:55:35.380 INFO:tasks.workunit.client.1.smithi176.stdout:3/236: dread d3/d17/f2f [0,4194304] 0 2024-07-16T09:55:35.384 INFO:tasks.workunit.client.1.smithi176.stdout:0/339: getdents d5/d25 0 2024-07-16T09:55:35.385 INFO:tasks.workunit.client.1.smithi176.stdout:9/382: dread d5/d1d/d20/f74 [0,4194304] 0 2024-07-16T09:55:35.386 INFO:tasks.workunit.client.1.smithi176.stdout:6/341: dwrite d1/d5/d7/d1e/f69 [0,4194304] 0 2024-07-16T09:55:35.388 INFO:tasks.workunit.client.1.smithi176.stdout:9/383: readlink d5/la 0 2024-07-16T09:55:35.391 INFO:tasks.workunit.client.1.smithi176.stdout:9/384: fsync d5/de/f12 0 2024-07-16T09:55:35.393 INFO:tasks.workunit.client.1.smithi176.stdout:5/417: dread d4/d37/d72/d38/d6e/f74 [0,4194304] 0 2024-07-16T09:55:35.394 INFO:tasks.workunit.client.1.smithi176.stdout:3/237: dread d3/d17/f26 [0,4194304] 0 2024-07-16T09:55:35.396 INFO:tasks.workunit.client.1.smithi176.stdout:5/418: fdatasync d4/d37/d72/d82/f64 0 2024-07-16T09:55:35.398 INFO:tasks.workunit.client.1.smithi176.stdout:3/238: rmdir d3/d17/d1d/d1e 39 2024-07-16T09:55:35.400 INFO:tasks.workunit.client.1.smithi176.stdout:6/342: dwrite d1/d5/d7/d2f/f32 [0,4194304] 0 2024-07-16T09:55:35.401 INFO:tasks.workunit.client.1.smithi176.stdout:9/385: creat d5/de/d37/f7c x:0 0 0 2024-07-16T09:55:35.402 INFO:tasks.workunit.client.0.smithi156.stdout:1/334: dread dc/d14/d1e/f21 [0,4194304] 0 2024-07-16T09:55:35.402 INFO:tasks.workunit.client.1.smithi176.stdout:3/239: stat d3/d17/d1d/d1e/f3a 0 2024-07-16T09:55:35.403 INFO:tasks.workunit.client.0.smithi156.stdout:1/335: truncate dc/f37 1347479 0 2024-07-16T09:55:35.403 INFO:tasks.workunit.client.1.smithi176.stdout:0/340: truncate d5/f22 2670683 0 2024-07-16T09:55:35.405 INFO:tasks.workunit.client.0.smithi156.stdout:1/336: creat dc/d6b/f75 x:0 0 0 2024-07-16T09:55:35.405 INFO:tasks.workunit.client.1.smithi176.stdout:9/386: creat d5/d1d/d20/f7d x:0 0 0 2024-07-16T09:55:35.405 INFO:tasks.workunit.client.1.smithi176.stdout:6/343: dread d1/dc/d40/f4d [0,4194304] 0 2024-07-16T09:55:35.406 INFO:tasks.workunit.client.1.smithi176.stdout:0/341: dread d5/d1c/f3b [0,4194304] 0 2024-07-16T09:55:35.406 INFO:tasks.workunit.client.0.smithi156.stdout:1/337: dread - dc/d14/d3e/d55/d57/f67 zero size 2024-07-16T09:55:35.407 INFO:tasks.workunit.client.1.smithi176.stdout:9/387: write d5/d52/d71/f75 [777221,11210] 0 2024-07-16T09:55:35.409 INFO:tasks.workunit.client.0.smithi156.stdout:1/338: mkdir dc/d14/d3e/d55/d57/d76 0 2024-07-16T09:55:35.412 INFO:tasks.workunit.client.1.smithi176.stdout:0/342: creat d5/d1c/d24/d35/d50/d58/f6f x:0 0 0 2024-07-16T09:55:35.412 INFO:tasks.workunit.client.1.smithi176.stdout:6/344: dread d1/d5/f27 [0,4194304] 0 2024-07-16T09:55:35.413 INFO:tasks.workunit.client.1.smithi176.stdout:3/240: getdents d3/da/d33 0 2024-07-16T09:55:35.415 INFO:tasks.workunit.client.1.smithi176.stdout:6/345: unlink d1/d5/l1a 0 2024-07-16T09:55:35.420 INFO:tasks.workunit.client.1.smithi176.stdout:0/343: getdents d5/d1c/d24/d35/d50/d58 0 2024-07-16T09:55:35.420 INFO:tasks.workunit.client.1.smithi176.stdout:3/241: getdents d3/d17/d1d/d1e 0 2024-07-16T09:55:35.423 INFO:tasks.workunit.client.1.smithi176.stdout:0/344: unlink d5/d1c/d24/d26/c2e 0 2024-07-16T09:55:35.423 INFO:tasks.workunit.client.1.smithi176.stdout:6/346: dwrite d1/d5/d47/f51 [4194304,4194304] 0 2024-07-16T09:55:35.423 INFO:tasks.workunit.client.1.smithi176.stdout:0/345: readlink d5/l45 0 2024-07-16T09:55:35.425 INFO:tasks.workunit.client.1.smithi176.stdout:3/242: mknod d3/c43 0 2024-07-16T09:55:35.425 INFO:tasks.workunit.client.1.smithi176.stdout:9/388: dread d5/de/f15 [0,4194304] 0 2024-07-16T09:55:35.428 INFO:tasks.workunit.client.1.smithi176.stdout:0/346: write d5/d1c/d24/d35/d50/d5b/f65 [3172496,50266] 0 2024-07-16T09:55:35.429 INFO:tasks.workunit.client.1.smithi176.stdout:6/347: mkdir d1/d5/d47/d62/d77 0 2024-07-16T09:55:35.430 INFO:tasks.workunit.client.1.smithi176.stdout:3/243: creat d3/da/d33/f44 x:0 0 0 2024-07-16T09:55:35.434 INFO:tasks.workunit.client.1.smithi176.stdout:0/347: mknod d5/d25/c70 0 2024-07-16T09:55:35.435 INFO:tasks.workunit.client.1.smithi176.stdout:6/348: creat d1/dc/d40/f78 x:0 0 0 2024-07-16T09:55:35.437 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:35.434+0000 7f0d392ff640 1 -- 172.21.15.156:0/2575957207 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0d340fb830 msgr2=0x7f0d340fbc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:55:35.437 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:35.434+0000 7f0d392ff640 1 --2- 172.21.15.156:0/2575957207 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0d340fb830 0x7f0d340fbc40 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f0d240048d0 tx=0x7f0d24009da0 comp rx=0 tx=0).stop 2024-07-16T09:55:35.437 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:35.434+0000 7f0d392ff640 1 -- 172.21.15.156:0/2575957207 shutdown_connections 2024-07-16T09:55:35.437 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:35.434+0000 7f0d392ff640 1 --2- 172.21.15.156:0/2575957207 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0d340fc180 0x7f0d340fc5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:55:35.437 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:35.434+0000 7f0d392ff640 1 --2- 172.21.15.156:0/2575957207 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0d340fb830 0x7f0d340fbc40 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:55:35.437 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:35.434+0000 7f0d392ff640 1 -- 172.21.15.156:0/2575957207 >> 172.21.15.156:0/2575957207 conn(0x7f0d340f61a0 msgr2=0x7f0d340f85b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:55:35.437 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:35.435+0000 7f0d392ff640 1 -- 172.21.15.156:0/2575957207 shutdown_connections 2024-07-16T09:55:35.437 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:35.435+0000 7f0d392ff640 1 -- 172.21.15.156:0/2575957207 wait complete. 2024-07-16T09:55:35.438 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:35.435+0000 7f0d392ff640 1 Processor -- start 2024-07-16T09:55:35.438 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:35.435+0000 7f0d392ff640 1 -- start start 2024-07-16T09:55:35.438 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:35.435+0000 7f0d392ff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0d340fb830 0x7f0d3410cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:55:35.438 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:35.436+0000 7f0d392ff640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0d340fc180 0x7f0d3410d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:55:35.438 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:35.436+0000 7f0d392ff640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d3410db60 con 0x7f0d340fb830 2024-07-16T09:55:35.438 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:35.436+0000 7f0d392ff640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d3410e380 con 0x7f0d340fc180 2024-07-16T09:55:35.438 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:35.436+0000 7f0d32ffd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0d340fb830 0x7f0d3410cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:55:35.438 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:35.436+0000 7f0d32ffd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0d340fb830 0x7f0d3410cfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:40932/0 (socket says 172.21.15.156:40932) 2024-07-16T09:55:35.438 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:35.436+0000 7f0d32ffd640 1 -- 172.21.15.156:0/143056019 learned_addr learned my addr 172.21.15.156:0/143056019 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T09:55:35.438 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:35.436+0000 7f0d327fc640 1 --2- 172.21.15.156:0/143056019 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0d340fc180 0x7f0d3410d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:55:35.440 INFO:tasks.workunit.client.1.smithi176.stdout:3/244: mknod d3/da/d33/c45 0 2024-07-16T09:55:35.440 INFO:tasks.workunit.client.1.smithi176.stdout:3/245: dread - d3/d17/d31/f42 zero size 2024-07-16T09:55:35.440 INFO:tasks.workunit.client.1.smithi176.stdout:9/389: mknod d5/d1d/d77/c7e 0 2024-07-16T09:55:35.440 INFO:tasks.workunit.client.1.smithi176.stdout:3/246: write d3/d17/d1d/d1e/f34 [973569,49123] 0 2024-07-16T09:55:35.441 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:35.436+0000 7f0d32ffd640 1 -- 172.21.15.156:0/143056019 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0d340fc180 msgr2=0x7f0d3410d510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:55:35.441 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:35.436+0000 7f0d32ffd640 1 --2- 172.21.15.156:0/143056019 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0d340fc180 0x7f0d3410d510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:55:35.441 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:35.436+0000 7f0d32ffd640 1 -- 172.21.15.156:0/143056019 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d24005040 con 0x7f0d340fb830 2024-07-16T09:55:35.441 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:35.436+0000 7f0d32ffd640 1 --2- 172.21.15.156:0/143056019 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0d340fb830 0x7f0d3410cfd0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f0d24009f80 tx=0x7f0d24009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:55:35.441 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:35.436+0000 7f0d1b7fe640 1 -- 172.21.15.156:0/143056019 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0d24038070 con 0x7f0d340fb830 2024-07-16T09:55:35.441 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:35.437+0000 7f0d392ff640 1 -- 172.21.15.156:0/143056019 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d3410e5b0 con 0x7f0d340fb830 2024-07-16T09:55:35.441 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:35.437+0000 7f0d1b7fe640 1 -- 172.21.15.156:0/143056019 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0d24008ee0 con 0x7f0d340fb830 2024-07-16T09:55:35.442 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:35.437+0000 7f0d1b7fe640 1 -- 172.21.15.156:0/143056019 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0d240336f0 con 0x7f0d340fb830 2024-07-16T09:55:35.442 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:35.437+0000 7f0d392ff640 1 -- 172.21.15.156:0/143056019 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d3410ea30 con 0x7f0d340fb830 2024-07-16T09:55:35.442 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:35.438+0000 7f0d392ff640 1 -- 172.21.15.156:0/143056019 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d340f9e20 con 0x7f0d340fb830 2024-07-16T09:55:35.442 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:35.438+0000 7f0d1b7fe640 1 -- 172.21.15.156:0/143056019 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 20) v1 ==== 88539+0+0 (secure 0 0 0) 0x7f0d24030020 con 0x7f0d340fb830 2024-07-16T09:55:35.442 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:35.439+0000 7f0d1b7fe640 1 --2- 172.21.15.156:0/143056019 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f0d1c06a5a0 0x7f0d1c06ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:55:35.442 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:35.439+0000 7f0d1b7fe640 1 -- 172.21.15.156:0/143056019 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f0d240a8ae0 con 0x7f0d340fb830 2024-07-16T09:55:35.442 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:35.439+0000 7f0d327fc640 1 --2- 172.21.15.156:0/143056019 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f0d1c06a5a0 0x7f0d1c06ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:55:35.442 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:35.440+0000 7f0d327fc640 1 --2- 172.21.15.156:0/143056019 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f0d1c06a5a0 0x7f0d1c06ca50 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f0d3410cd80 tx=0x7f0d2800b040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:55:35.443 INFO:tasks.workunit.client.1.smithi176.stdout:6/349: write d1/dc/d40/f4d [4914084,94967] 0 2024-07-16T09:55:35.444 INFO:tasks.workunit.client.1.smithi176.stdout:0/348: write d5/d1c/d5a/f6d [343702,89455] 0 2024-07-16T09:55:35.445 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:35.444+0000 7f0d1b7fe640 1 -- 172.21.15.156:0/143056019 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f0d24075e30 con 0x7f0d340fb830 2024-07-16T09:55:35.445 INFO:tasks.workunit.client.1.smithi176.stdout:3/247: dwrite d3/d17/d31/f3f [0,4194304] 0 2024-07-16T09:55:35.446 INFO:tasks.workunit.client.1.smithi176.stdout:3/248: stat d3/da 0 2024-07-16T09:55:35.447 INFO:tasks.workunit.client.1.smithi176.stdout:9/390: truncate d5/d1d/f3f 3287785 0 2024-07-16T09:55:35.449 INFO:tasks.workunit.client.1.smithi176.stdout:3/249: truncate d3/da/f29 103659 0 2024-07-16T09:55:35.450 INFO:tasks.workunit.client.1.smithi176.stdout:6/350: creat d1/d5/d7/d1e/f79 x:0 0 0 2024-07-16T09:55:35.455 INFO:tasks.workunit.client.1.smithi176.stdout:6/351: symlink d1/d5/d7/l7a 0 2024-07-16T09:55:35.456 INFO:tasks.workunit.client.1.smithi176.stdout:6/352: chown d1/d5/d47/d62 1741 1 2024-07-16T09:55:35.456 INFO:tasks.workunit.client.1.smithi176.stdout:3/250: getdents d3/d17/d31 0 2024-07-16T09:55:35.457 INFO:tasks.workunit.client.1.smithi176.stdout:6/353: write d1/d5/f2c [472186,93782] 0 2024-07-16T09:55:35.458 INFO:tasks.workunit.client.1.smithi176.stdout:3/251: chown d3/da/d33/f39 15716852 1 2024-07-16T09:55:35.459 INFO:tasks.workunit.client.1.smithi176.stdout:3/252: chown d3/d17/d31 331710 1 2024-07-16T09:55:35.460 INFO:tasks.workunit.client.1.smithi176.stdout:6/354: creat d1/d61/d65/d76/f7b x:0 0 0 2024-07-16T09:55:35.462 INFO:tasks.workunit.client.1.smithi176.stdout:6/355: chown d1/d5/d7/f52 10473723 1 2024-07-16T09:55:35.463 INFO:tasks.workunit.client.1.smithi176.stdout:3/253: rename d3/d17/d1d/d1e/c2a to d3/d17/c46 0 2024-07-16T09:55:35.465 INFO:tasks.workunit.client.1.smithi176.stdout:6/356: mkdir d1/dc/d72/d6f/d7c 0 2024-07-16T09:55:35.465 INFO:tasks.workunit.client.1.smithi176.stdout:3/254: truncate d3/f25 5022146 0 2024-07-16T09:55:35.466 INFO:tasks.workunit.client.1.smithi176.stdout:0/349: dread d5/d25/f4e [0,4194304] 0 2024-07-16T09:55:35.467 INFO:tasks.workunit.client.1.smithi176.stdout:3/255: fdatasync d3/da/d33/f39 0 2024-07-16T09:55:35.468 INFO:tasks.workunit.client.1.smithi176.stdout:9/391: dwrite d5/fd [0,4194304] 0 2024-07-16T09:55:35.468 INFO:tasks.workunit.client.1.smithi176.stdout:6/357: mknod d1/d5/d7/d54/c7d 0 2024-07-16T09:55:35.469 INFO:tasks.workunit.client.1.smithi176.stdout:6/358: fdatasync d1/d5/d7/f52 0 2024-07-16T09:55:35.470 INFO:tasks.workunit.client.1.smithi176.stdout:3/256: write d3/d17/f26 [912059,127617] 0 2024-07-16T09:55:35.471 INFO:tasks.workunit.client.1.smithi176.stdout:9/392: read d5/de/d21/f5f [223723,33967] 0 2024-07-16T09:55:35.473 INFO:tasks.workunit.client.1.smithi176.stdout:9/393: fsync d5/de/f2f 0 2024-07-16T09:55:35.474 INFO:tasks.workunit.client.1.smithi176.stdout:6/359: mknod d1/dc/d72/c7e 0 2024-07-16T09:55:35.474 INFO:tasks.workunit.client.1.smithi176.stdout:6/360: chown d1/d5/l8 588881750 1 2024-07-16T09:55:35.478 INFO:tasks.workunit.client.1.smithi176.stdout:9/394: symlink d5/d43/l7f 0 2024-07-16T09:55:35.479 INFO:tasks.workunit.client.1.smithi176.stdout:3/257: mknod d3/d17/d1d/c47 0 2024-07-16T09:55:35.479 INFO:tasks.workunit.client.1.smithi176.stdout:0/350: write d5/d1c/f3b [542518,13424] 0 2024-07-16T09:55:35.480 INFO:tasks.workunit.client.1.smithi176.stdout:0/351: stat d5/l48 0 2024-07-16T09:55:35.480 INFO:tasks.workunit.client.1.smithi176.stdout:0/352: fsync d5/fb 0 2024-07-16T09:55:35.481 INFO:tasks.workunit.client.1.smithi176.stdout:3/258: symlink d3/d17/d31/l48 0 2024-07-16T09:55:35.481 INFO:tasks.workunit.client.1.smithi176.stdout:3/259: stat c2 0 2024-07-16T09:55:35.482 INFO:tasks.workunit.client.1.smithi176.stdout:3/260: truncate d3/d17/f2f 4819389 0 2024-07-16T09:55:35.488 INFO:tasks.workunit.client.1.smithi176.stdout:9/395: getdents d5/de 0 2024-07-16T09:55:35.499 INFO:tasks.workunit.client.1.smithi176.stdout:0/353: dwrite d5/f7 [4194304,4194304] 0 2024-07-16T09:55:35.501 INFO:tasks.workunit.client.1.smithi176.stdout:0/354: stat d5/d1c/d24/d35/d50/f62 0 2024-07-16T09:55:35.504 INFO:tasks.workunit.client.1.smithi176.stdout:3/261: write d3/d17/d1d/d1e/f27 [930217,80943] 0 2024-07-16T09:55:35.504 INFO:tasks.workunit.client.1.smithi176.stdout:9/396: truncate d5/de/f47 2444512 0 2024-07-16T09:55:35.504 INFO:tasks.workunit.client.1.smithi176.stdout:0/355: symlink d5/d1c/d24/d35/d50/d5b/l71 0 2024-07-16T09:55:35.506 INFO:tasks.workunit.client.1.smithi176.stdout:3/262: chown d3/d17/c46 10 1 2024-07-16T09:55:35.507 INFO:tasks.workunit.client.1.smithi176.stdout:9/397: rmdir d5/de/d37 39 2024-07-16T09:55:35.512 INFO:tasks.workunit.client.1.smithi176.stdout:9/398: dread d5/d1d/f3c [0,4194304] 0 2024-07-16T09:55:35.513 INFO:tasks.workunit.client.1.smithi176.stdout:0/356: dwrite d5/d1c/d24/d35/f61 [0,4194304] 0 2024-07-16T09:55:35.514 INFO:tasks.workunit.client.1.smithi176.stdout:0/357: write d5/d2c/d6c/f64 [517080,10927] 0 2024-07-16T09:55:35.518 INFO:tasks.workunit.client.1.smithi176.stdout:0/358: read - d5/f1e zero size 2024-07-16T09:55:35.518 INFO:tasks.workunit.client.1.smithi176.stdout:3/263: dread d3/d17/d1d/d1e/f3a [0,4194304] 0 2024-07-16T09:55:35.519 INFO:tasks.workunit.client.1.smithi176.stdout:0/359: symlink d5/d2c/l72 0 2024-07-16T09:55:35.520 INFO:tasks.workunit.client.1.smithi176.stdout:9/399: getdents d5/de/d30 0 2024-07-16T09:55:35.521 INFO:tasks.workunit.client.1.smithi176.stdout:0/360: write d5/f6e [897395,2848] 0 2024-07-16T09:55:35.522 INFO:tasks.workunit.client.1.smithi176.stdout:0/361: chown d5/l9 19139 1 2024-07-16T09:55:35.522 INFO:tasks.workunit.client.1.smithi176.stdout:0/362: chown d5/l48 3663518 1 2024-07-16T09:55:35.524 INFO:tasks.workunit.client.1.smithi176.stdout:9/400: mknod d5/de/d37/d58/c80 0 2024-07-16T09:55:35.524 INFO:tasks.workunit.client.1.smithi176.stdout:0/363: write d5/d2c/d6c/f64 [7558,96271] 0 2024-07-16T09:55:35.525 INFO:tasks.workunit.client.1.smithi176.stdout:9/401: dread - d5/d1d/d67/f6a zero size 2024-07-16T09:55:35.525 INFO:tasks.workunit.client.1.smithi176.stdout:3/264: link d3/l28 d3/d17/d1d/l49 0 2024-07-16T09:55:35.526 INFO:tasks.workunit.client.1.smithi176.stdout:0/364: mknod d5/d1c/d24/d35/d50/d58/c73 0 2024-07-16T09:55:35.526 INFO:tasks.workunit.client.1.smithi176.stdout:9/402: creat d5/d1d/d20/f81 x:0 0 0 2024-07-16T09:55:35.527 INFO:tasks.workunit.client.1.smithi176.stdout:0/365: readlink d5/d2c/d6c/l42 0 2024-07-16T09:55:35.529 INFO:tasks.workunit.client.1.smithi176.stdout:0/366: creat d5/d1c/d24/d35/d59/f74 x:0 0 0 2024-07-16T09:55:35.529 INFO:tasks.workunit.client.1.smithi176.stdout:3/265: dread d3/d17/d31/f3f [0,4194304] 0 2024-07-16T09:55:35.529 INFO:tasks.workunit.client.1.smithi176.stdout:9/403: creat d5/de/d38/f82 x:0 0 0 2024-07-16T09:55:35.530 INFO:tasks.workunit.client.1.smithi176.stdout:0/367: write d5/d2c/d6c/f5e [1120076,106846] 0 2024-07-16T09:55:35.551 INFO:tasks.workunit.client.1.smithi176.stdout:4/314: sync 2024-07-16T09:55:35.554 INFO:tasks.workunit.client.1.smithi176.stdout:3/266: fdatasync d3/d17/d1d/d1e/f3a 0 2024-07-16T09:55:35.555 INFO:tasks.workunit.client.1.smithi176.stdout:0/368: fdatasync d5/d1c/d24/d26/f69 0 2024-07-16T09:55:35.556 INFO:tasks.workunit.client.1.smithi176.stdout:4/315: fdatasync da/d3d/d17/f2a 0 2024-07-16T09:55:35.556 INFO:tasks.workunit.client.1.smithi176.stdout:3/267: dread - d3/da/d33/f44 zero size 2024-07-16T09:55:35.556 INFO:tasks.workunit.client.1.smithi176.stdout:0/369: fdatasync d5/f7 0 2024-07-16T09:55:35.558 INFO:tasks.workunit.client.1.smithi176.stdout:9/404: dread d5/fb [0,4194304] 0 2024-07-16T09:55:35.560 INFO:tasks.workunit.client.1.smithi176.stdout:3/268: write d3/d17/d1d/d1e/f21 [2440992,13947] 0 2024-07-16T09:55:35.560 INFO:tasks.workunit.client.1.smithi176.stdout:0/370: creat d5/d1c/d5a/f75 x:0 0 0 2024-07-16T09:55:35.561 INFO:tasks.workunit.client.0.smithi156.stdout:6/318: sync 2024-07-16T09:55:35.561 INFO:tasks.workunit.client.0.smithi156.stdout:4/247: sync 2024-07-16T09:55:35.561 INFO:tasks.workunit.client.1.smithi176.stdout:9/405: mknod d5/d52/c83 0 2024-07-16T09:55:35.561 INFO:tasks.workunit.client.1.smithi176.stdout:0/371: chown d5/f6e 28329441 1 2024-07-16T09:55:35.563 INFO:tasks.workunit.client.0.smithi156.stdout:6/319: write d3/d13/d17/d4f/d1f/fa [363642,30378] 0 2024-07-16T09:55:35.566 INFO:tasks.workunit.client.1.smithi176.stdout:9/406: dread - d5/d1d/d67/f6a zero size 2024-07-16T09:55:35.566 INFO:tasks.workunit.client.0.smithi156.stdout:4/248: creat d4/f54 x:0 0 0 2024-07-16T09:55:35.567 INFO:tasks.workunit.client.1.smithi176.stdout:0/372: write d5/d1c/d24/d35/d50/d58/d63/f66 [633676,48482] 0 2024-07-16T09:55:35.567 INFO:tasks.workunit.client.1.smithi176.stdout:9/407: readlink d5/d52/d71/l73 0 2024-07-16T09:55:35.571 INFO:tasks.workunit.client.1.smithi176.stdout:3/269: dwrite d3/d17/d1d/d1e/f27 [0,4194304] 0 2024-07-16T09:55:35.572 INFO:tasks.workunit.client.1.smithi176.stdout:4/316: dwrite da/d3d/d17/f18 [0,4194304] 0 2024-07-16T09:55:35.573 INFO:tasks.workunit.client.1.smithi176.stdout:0/373: write d5/d1c/d24/d35/d50/d5b/f65 [4248465,34807] 0 2024-07-16T09:55:35.574 INFO:tasks.workunit.client.0.smithi156.stdout:6/320: dread d3/d13/d17/d4f/d1f/f10 [0,4194304] 0 2024-07-16T09:55:35.575 INFO:tasks.workunit.client.1.smithi176.stdout:9/408: creat d5/de/d21/f84 x:0 0 0 2024-07-16T09:55:35.577 INFO:tasks.workunit.client.0.smithi156.stdout:6/321: mknod d3/d13/d17/c68 0 2024-07-16T09:55:35.578 INFO:tasks.workunit.client.1.smithi176.stdout:9/409: mknod d5/d1d/d67/d6d/c85 0 2024-07-16T09:55:35.581 INFO:tasks.workunit.client.0.smithi156.stdout:4/249: dwrite d4/f5 [0,4194304] 0 2024-07-16T09:55:35.585 INFO:tasks.workunit.client.0.smithi156.stdout:4/250: creat d4/db/dc/f55 x:0 0 0 2024-07-16T09:55:35.590 INFO:tasks.workunit.client.1.smithi176.stdout:3/270: creat d3/f4a x:0 0 0 2024-07-16T09:55:35.590 INFO:tasks.workunit.client.1.smithi176.stdout:0/374: rename d5/d1c/d24/d35/d50/d5b/l71 to d5/d25/l76 0 2024-07-16T09:55:35.591 INFO:tasks.workunit.client.0.smithi156.stdout:4/251: mknod d4/d23/c56 0 2024-07-16T09:55:35.591 INFO:tasks.workunit.client.1.smithi176.stdout:3/271: dread - d3/da/d33/f44 zero size 2024-07-16T09:55:35.593 INFO:tasks.workunit.client.1.smithi176.stdout:4/317: getdents da/d3d 0 2024-07-16T09:55:35.594 INFO:tasks.workunit.client.1.smithi176.stdout:4/318: write da/dc/f55 [4844963,72786] 0 2024-07-16T09:55:35.594 INFO:tasks.workunit.client.1.smithi176.stdout:0/375: unlink d5/d1c/d24/d35/d50/d58/c73 0 2024-07-16T09:55:35.595 INFO:tasks.workunit.client.1.smithi176.stdout:0/376: chown d5/d1c/d24/d26/c33 249218186 1 2024-07-16T09:55:35.595 INFO:tasks.workunit.client.1.smithi176.stdout:3/272: write d3/d17/f2f [3390313,30566] 0 2024-07-16T09:55:35.596 INFO:tasks.workunit.client.1.smithi176.stdout:4/319: stat da/c2e 0 2024-07-16T09:55:35.596 INFO:tasks.workunit.client.1.smithi176.stdout:9/410: rmdir d5/d1d 39 2024-07-16T09:55:35.597 INFO:tasks.workunit.client.1.smithi176.stdout:4/320: write da/d14/f26 [2822736,96111] 0 2024-07-16T09:55:35.600 INFO:tasks.workunit.client.1.smithi176.stdout:3/273: creat d3/d17/d1d/f4b x:0 0 0 2024-07-16T09:55:35.602 INFO:tasks.workunit.client.1.smithi176.stdout:4/321: mkdir da/d41/d5b 0 2024-07-16T09:55:35.605 INFO:tasks.workunit.client.1.smithi176.stdout:9/411: rename d5/de/d37/d58/l5d to d5/d1d/l86 0 2024-07-16T09:55:35.606 INFO:tasks.workunit.client.1.smithi176.stdout:4/322: write da/dc/f45 [689162,121027] 0 2024-07-16T09:55:35.606 INFO:tasks.workunit.client.1.smithi176.stdout:4/323: chown da/d3d/d33 1 1 2024-07-16T09:55:35.607 INFO:tasks.workunit.client.0.smithi156.stdout:6/322: dread d3/f25 [0,4194304] 0 2024-07-16T09:55:35.610 INFO:tasks.workunit.client.0.smithi156.stdout:4/252: getdents d4/d23 0 2024-07-16T09:55:35.611 INFO:tasks.workunit.client.1.smithi176.stdout:3/274: rename d3/d17/d1d/d1e/c38 to d3/da/d33/c4c 0 2024-07-16T09:55:35.613 INFO:tasks.workunit.client.1.smithi176.stdout:0/377: truncate d5/d1c/d24/d35/f61 3229269 0 2024-07-16T09:55:35.616 INFO:tasks.workunit.client.0.smithi156.stdout:6/323: dwrite d3/d13/d17/d4f/d1f/d19/d2e/f3e [0,4194304] 0 2024-07-16T09:55:35.617 INFO:tasks.workunit.client.1.smithi176.stdout:4/324: getdents da/dc/d10/d28 0 2024-07-16T09:55:35.618 INFO:tasks.workunit.client.0.smithi156.stdout:4/253: dwrite d4/f5 [0,4194304] 0 2024-07-16T09:55:35.619 INFO:tasks.workunit.client.1.smithi176.stdout:4/325: write da/dc/f50 [264747,91392] 0 2024-07-16T09:55:35.620 INFO:tasks.workunit.client.1.smithi176.stdout:3/275: dwrite d3/d17/f40 [0,4194304] 0 2024-07-16T09:55:35.620 INFO:tasks.workunit.client.1.smithi176.stdout:9/412: rename d5/d52/c54 to d5/d43/c87 0 2024-07-16T09:55:35.620 INFO:tasks.workunit.client.1.smithi176.stdout:0/378: mknod d5/d1c/d24/d26/c77 0 2024-07-16T09:55:35.622 INFO:tasks.workunit.client.1.smithi176.stdout:0/379: fdatasync d5/d1c/d24/d26/f3d 0 2024-07-16T09:55:35.623 INFO:tasks.workunit.client.0.smithi156.stdout:6/324: truncate d3/d13/d17/d4f/d4d/d3b/f59 630086 0 2024-07-16T09:55:35.624 INFO:tasks.workunit.client.0.smithi156.stdout:4/254: read f2 [271279,45296] 0 2024-07-16T09:55:35.630 INFO:tasks.workunit.client.1.smithi176.stdout:4/326: symlink da/d3d/d17/l5c 0 2024-07-16T09:55:35.632 INFO:tasks.workunit.client.1.smithi176.stdout:9/413: write d5/fd [2580432,57547] 0 2024-07-16T09:55:35.633 INFO:tasks.workunit.client.0.smithi156.stdout:6/325: rename d3/d13/d17/d4f/d30/d45/f51 to d3/d13/d17/d4f/d4d/d3b/d5a/f69 0 2024-07-16T09:55:35.633 INFO:tasks.workunit.client.0.smithi156.stdout:4/255: link d4/d23/c1f d4/d23/d52/c57 0 2024-07-16T09:55:35.634 INFO:tasks.workunit.client.1.smithi176.stdout:4/327: mkdir da/d14/d40/d5d 0 2024-07-16T09:55:35.636 INFO:tasks.workunit.client.0.smithi156.stdout:4/256: mkdir d4/d41/d58 0 2024-07-16T09:55:35.636 INFO:tasks.workunit.client.1.smithi176.stdout:3/276: rename d3/d17/d31 to d3/d17/d1d/d4d 0 2024-07-16T09:55:35.638 INFO:tasks.workunit.client.0.smithi156.stdout:4/257: readlink d4/l3e 0 2024-07-16T09:55:35.639 INFO:tasks.workunit.client.0.smithi156.stdout:6/326: link d3/d13/d17/c28 d3/d13/d17/d4f/d1f/d19/d2e/c6a 0 2024-07-16T09:55:35.641 INFO:tasks.workunit.client.1.smithi176.stdout:9/414: dwrite d5/de/d37/f7c [0,4194304] 0 2024-07-16T09:55:35.641 INFO:tasks.workunit.client.0.smithi156.stdout:6/327: write d3/d13/d17/d4f/d1f/f10 [1511874,88606] 0 2024-07-16T09:55:35.643 INFO:tasks.workunit.client.1.smithi176.stdout:3/277: dread d3/da/f13 [0,4194304] 0 2024-07-16T09:55:35.643 INFO:tasks.workunit.client.0.smithi156.stdout:4/258: link d4/f3d d4/d23/f59 0 2024-07-16T09:55:35.643 INFO:tasks.workunit.client.1.smithi176.stdout:3/278: stat d3/d17/d1d/c2e 0 2024-07-16T09:55:35.644 INFO:tasks.workunit.client.1.smithi176.stdout:3/279: chown d3/d17/c1b 1911 1 2024-07-16T09:55:35.645 INFO:tasks.workunit.client.1.smithi176.stdout:9/415: getdents d5/de/d30 0 2024-07-16T09:55:35.645 INFO:tasks.workunit.client.1.smithi176.stdout:3/280: write d3/d17/f40 [4415254,56755] 0 2024-07-16T09:55:35.647 INFO:tasks.workunit.client.1.smithi176.stdout:3/281: chown d3/d17/c46 61997 1 2024-07-16T09:55:35.647 INFO:tasks.workunit.client.0.smithi156.stdout:6/328: rename d3/d13/d17/d4f/d1f/c46 to d3/d13/d17/d4f/d1e/c6b 0 2024-07-16T09:55:35.650 INFO:tasks.workunit.client.1.smithi176.stdout:9/416: write d5/de/f12 [2497955,40756] 0 2024-07-16T09:55:35.651 INFO:tasks.workunit.client.1.smithi176.stdout:4/328: rmdir da 39 2024-07-16T09:55:35.651 INFO:tasks.workunit.client.0.smithi156.stdout:4/259: link d4/db/dc/l10 d4/db/l5a 0 2024-07-16T09:55:35.652 INFO:tasks.workunit.client.0.smithi156.stdout:4/260: fdatasync d4/f54 0 2024-07-16T09:55:35.653 INFO:tasks.workunit.client.0.smithi156.stdout:6/329: link d3/d13/d17/d4f/d1f/f2c d3/d13/d17/f6c 0 2024-07-16T09:55:35.655 INFO:tasks.workunit.client.0.smithi156.stdout:4/261: creat d4/db/d47/d4b/f5b x:0 0 0 2024-07-16T09:55:35.655 INFO:tasks.workunit.client.0.smithi156.stdout:6/330: fdatasync d3/d13/d17/d4f/d1f/d19/d31/f4e 0 2024-07-16T09:55:35.656 INFO:tasks.workunit.client.0.smithi156.stdout:4/262: write d4/f9 [5608557,22287] 0 2024-07-16T09:55:35.656 INFO:tasks.workunit.client.1.smithi176.stdout:3/282: dwrite d3/d17/d1d/d4d/f42 [0,4194304] 0 2024-07-16T09:55:35.658 INFO:tasks.workunit.client.1.smithi176.stdout:0/380: dwrite d5/f1e [0,4194304] 0 2024-07-16T09:55:35.658 INFO:tasks.workunit.client.0.smithi156.stdout:6/331: creat d3/d13/d17/d4f/d30/d45/f6d x:0 0 0 2024-07-16T09:55:35.659 INFO:tasks.workunit.client.0.smithi156.stdout:4/263: creat d4/db/d44/f5c x:0 0 0 2024-07-16T09:55:35.660 INFO:tasks.workunit.client.0.smithi156.stdout:6/332: fdatasync d3/d13/d17/d4f/d4d/d3b/d5a/f69 0 2024-07-16T09:55:35.664 INFO:tasks.workunit.client.0.smithi156.stdout:6/333: symlink d3/d13/d17/d4f/d1f/d19/d31/l6e 0 2024-07-16T09:55:35.664 INFO:tasks.workunit.client.0.smithi156.stdout:9/360: sync 2024-07-16T09:55:35.665 INFO:tasks.workunit.client.0.smithi156.stdout:3/264: sync 2024-07-16T09:55:35.665 INFO:tasks.workunit.client.0.smithi156.stdout:3/265: dread - d8/da/f3d zero size 2024-07-16T09:55:35.667 INFO:tasks.workunit.client.0.smithi156.stdout:6/334: fsync d3/d13/d17/d4f/d5c/f65 0 2024-07-16T09:55:35.668 INFO:tasks.workunit.client.0.smithi156.stdout:3/266: creat d8/da/d13/f57 x:0 0 0 2024-07-16T09:55:35.671 INFO:tasks.workunit.client.0.smithi156.stdout:6/335: write d3/d13/d17/d4f/d4d/d3b/d5a/f69 [3460896,53791] 0 2024-07-16T09:55:35.672 INFO:tasks.workunit.client.0.smithi156.stdout:6/336: read - d3/d13/d17/d56/f64 zero size 2024-07-16T09:55:35.674 INFO:tasks.workunit.client.1.smithi176.stdout:3/283: rename d3/d17/d1d/f2d to d3/d17/d1d/d4d/f4e 0 2024-07-16T09:55:35.674 INFO:tasks.workunit.client.0.smithi156.stdout:3/267: link d8/da/f2e d8/da/d1d/f58 0 2024-07-16T09:55:35.675 INFO:tasks.workunit.client.1.smithi176.stdout:4/329: write da/dc/d39/f3c [538360,72095] 0 2024-07-16T09:55:35.675 INFO:tasks.workunit.client.1.smithi176.stdout:3/284: chown d3/d17/d1d/d1e/f21 0 1 2024-07-16T09:55:35.675 INFO:tasks.workunit.client.0.smithi156.stdout:6/337: mknod d3/d13/d17/d4f/d1e/c6f 0 2024-07-16T09:55:35.676 INFO:tasks.workunit.client.1.smithi176.stdout:3/285: write d3/d17/d1d/d4d/f42 [1972187,96924] 0 2024-07-16T09:55:35.678 INFO:tasks.workunit.client.1.smithi176.stdout:4/330: mkdir da/d3d/d17/d54/d5e 0 2024-07-16T09:55:35.681 INFO:tasks.workunit.client.1.smithi176.stdout:3/286: mkdir d3/d17/d1d/d4f 0 2024-07-16T09:55:35.683 INFO:tasks.workunit.client.1.smithi176.stdout:4/331: creat da/f5f x:0 0 0 2024-07-16T09:55:35.686 INFO:tasks.workunit.client.1.smithi176.stdout:3/287: dread d3/da/f13 [0,4194304] 0 2024-07-16T09:55:35.686 INFO:tasks.workunit.client.1.smithi176.stdout:3/288: stat d3/da/d33/f39 0 2024-07-16T09:55:35.687 INFO:tasks.workunit.client.0.smithi156.stdout:4/264: dread d4/f8 [0,4194304] 0 2024-07-16T09:55:35.689 INFO:tasks.workunit.client.1.smithi176.stdout:3/289: readlink d3/l28 0 2024-07-16T09:55:35.689 INFO:tasks.workunit.client.1.smithi176.stdout:9/417: dread d5/d1d/f3f [0,4194304] 0 2024-07-16T09:55:35.691 INFO:tasks.workunit.client.0.smithi156.stdout:6/338: dread d3/d13/d17/d4f/f14 [0,4194304] 0 2024-07-16T09:55:35.692 INFO:tasks.workunit.client.1.smithi176.stdout:4/332: rmdir da 39 2024-07-16T09:55:35.692 INFO:tasks.workunit.client.1.smithi176.stdout:0/381: write d5/d1c/d24/d35/f61 [728003,29916] 0 2024-07-16T09:55:35.693 INFO:tasks.workunit.client.0.smithi156.stdout:6/339: write d3/d13/d17/d4f/d5c/f65 [320810,84323] 0 2024-07-16T09:55:35.694 INFO:tasks.workunit.client.1.smithi176.stdout:0/382: dread - d5/d1c/d24/d35/d50/d58/f6f zero size 2024-07-16T09:55:35.695 INFO:tasks.workunit.client.1.smithi176.stdout:0/383: dread - d5/d1c/d24/d26/f69 zero size 2024-07-16T09:55:35.695 INFO:tasks.workunit.client.1.smithi176.stdout:4/333: symlink da/d14/d4a/l60 0 2024-07-16T09:55:35.697 INFO:tasks.workunit.client.0.smithi156.stdout:6/340: creat d3/d13/d17/d4f/d4d/f70 x:0 0 0 2024-07-16T09:55:35.698 INFO:tasks.workunit.client.1.smithi176.stdout:4/334: unlink da/dc/d10/c30 0 2024-07-16T09:55:35.698 INFO:tasks.workunit.client.0.smithi156.stdout:9/361: dread d2/d5/d16/d22/d36/f50 [4194304,4194304] 0 2024-07-16T09:55:35.699 INFO:tasks.workunit.client.0.smithi156.stdout:9/362: chown d2/d5/d16/f20 325540573 1 2024-07-16T09:55:35.701 INFO:tasks.workunit.client.0.smithi156.stdout:6/341: rename d3/d13/l23 to d3/d13/d17/d4f/l71 0 2024-07-16T09:55:35.706 INFO:tasks.workunit.client.0.smithi156.stdout:9/363: truncate d2/d5/f55 7361544 0 2024-07-16T09:55:35.716 INFO:tasks.workunit.client.0.smithi156.stdout:9/364: dread d2/d5/d16/d22/d36/d33/f2c [0,4194304] 0 2024-07-16T09:55:35.718 INFO:tasks.workunit.client.0.smithi156.stdout:9/365: symlink d2/d5/d16/d22/d36/l71 0 2024-07-16T09:55:35.719 INFO:tasks.workunit.client.0.smithi156.stdout:9/366: chown d2/f15 3349899 1 2024-07-16T09:55:35.727 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:35.726+0000 7f0d392ff640 1 -- 172.21.15.156:0/143056019 --> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f0d3410ee10 con 0x7f0d1c06a5a0 2024-07-16T09:55:35.727 INFO:tasks.workunit.client.1.smithi176.stdout:2/341: sync 2024-07-16T09:55:35.727 INFO:tasks.workunit.client.1.smithi176.stdout:1/205: sync 2024-07-16T09:55:35.727 INFO:tasks.workunit.client.1.smithi176.stdout:7/371: sync 2024-07-16T09:55:35.728 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:35.727+0000 7f0d1b7fe640 1 -- 172.21.15.156:0/143056019 <== mgr.14208 v2:172.21.15.156:6800/2615741717 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+349 (secure 0 0 0) 0x7f0d3410ee10 con 0x7f0d1c06a5a0 2024-07-16T09:55:35.728 INFO:teuthology.orchestra.run.smithi156.stdout:{ 2024-07-16T09:55:35.728 INFO:teuthology.orchestra.run.smithi156.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:c15ec3aa8e6c0a9c12ee24ff79f3e7dd4d01b3d1a702dfeb7d7fff9b0811a393", 2024-07-16T09:55:35.728 INFO:teuthology.orchestra.run.smithi156.stdout: "in_progress": true, 2024-07-16T09:55:35.728 INFO:teuthology.orchestra.run.smithi156.stdout: "which": "Upgrading all daemon types on all hosts", 2024-07-16T09:55:35.729 INFO:teuthology.orchestra.run.smithi156.stdout: "services_complete": [], 2024-07-16T09:55:35.729 INFO:teuthology.orchestra.run.smithi156.stdout: "progress": "1/21 daemons upgraded", 2024-07-16T09:55:35.729 INFO:teuthology.orchestra.run.smithi156.stdout: "message": "Currently upgrading mgr daemons", 2024-07-16T09:55:35.729 INFO:teuthology.orchestra.run.smithi156.stdout: "is_paused": false 2024-07-16T09:55:35.729 INFO:teuthology.orchestra.run.smithi156.stdout:} 2024-07-16T09:55:35.731 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:35.730+0000 7f0d392ff640 1 -- 172.21.15.156:0/143056019 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f0d1c06a5a0 msgr2=0x7f0d1c06ca50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:55:35.731 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:35.730+0000 7f0d392ff640 1 --2- 172.21.15.156:0/143056019 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f0d1c06a5a0 0x7f0d1c06ca50 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f0d3410cd80 tx=0x7f0d2800b040 comp rx=0 tx=0).stop 2024-07-16T09:55:35.731 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:35.730+0000 7f0d392ff640 1 -- 172.21.15.156:0/143056019 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0d340fb830 msgr2=0x7f0d3410cfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:55:35.731 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:35.730+0000 7f0d392ff640 1 --2- 172.21.15.156:0/143056019 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0d340fb830 0x7f0d3410cfd0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f0d24009f80 tx=0x7f0d24009da0 comp rx=0 tx=0).stop 2024-07-16T09:55:35.731 INFO:tasks.workunit.client.0.smithi156.stdout:9/367: dread d2/f15 [0,4194304] 0 2024-07-16T09:55:35.732 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:35.730+0000 7f0d392ff640 1 -- 172.21.15.156:0/143056019 shutdown_connections 2024-07-16T09:55:35.732 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:35.730+0000 7f0d392ff640 1 --2- 172.21.15.156:0/143056019 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7f0d1c06a5a0 0x7f0d1c06ca50 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:55:35.732 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:35.730+0000 7f0d392ff640 1 --2- 172.21.15.156:0/143056019 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0d340fc180 0x7f0d3410d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:55:35.732 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:35.730+0000 7f0d392ff640 1 --2- 172.21.15.156:0/143056019 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0d340fb830 0x7f0d3410cfd0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:55:35.732 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:35.730+0000 7f0d392ff640 1 -- 172.21.15.156:0/143056019 >> 172.21.15.156:0/143056019 conn(0x7f0d340f61a0 msgr2=0x7f0d340f78a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:55:35.732 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:35.730+0000 7f0d392ff640 1 -- 172.21.15.156:0/143056019 shutdown_connections 2024-07-16T09:55:35.732 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:35.730+0000 7f0d392ff640 1 -- 172.21.15.156:0/143056019 wait complete. 2024-07-16T09:55:35.734 INFO:tasks.workunit.client.0.smithi156.stdout:9/368: rename d2/d5/l4e to d2/d5/d61/l72 0 2024-07-16T09:55:35.735 INFO:tasks.workunit.client.0.smithi156.stdout:9/369: rmdir d2/d5/d16/d57 39 2024-07-16T09:55:35.738 INFO:tasks.workunit.client.0.smithi156.stdout:9/370: mknod d2/d5/d16/d57/c73 0 2024-07-16T09:55:35.743 INFO:tasks.workunit.client.0.smithi156.stdout:9/371: dwrite d2/d5/d16/d22/d36/d33/f2c [0,4194304] 0 2024-07-16T09:55:35.745 INFO:tasks.workunit.client.1.smithi176.stdout:0/384: truncate d5/d1c/d24/d35/d50/d58/d63/f66 425820 0 2024-07-16T09:55:35.746 INFO:tasks.workunit.client.1.smithi176.stdout:7/372: dread - d4/d5/d10/d17/d1d/f73 zero size 2024-07-16T09:55:35.747 INFO:tasks.workunit.client.1.smithi176.stdout:4/335: dread da/d3d/d17/f18 [0,4194304] 0 2024-07-16T09:55:35.747 INFO:tasks.workunit.client.1.smithi176.stdout:2/342: rmdir dd/d36/d45 39 2024-07-16T09:55:35.748 INFO:tasks.workunit.client.1.smithi176.stdout:0/385: write d5/d39/f51 [582184,76521] 0 2024-07-16T09:55:35.748 INFO:tasks.workunit.client.1.smithi176.stdout:1/206: mknod d4/d8/d12/d21/d25/d30/c48 0 2024-07-16T09:55:35.749 INFO:tasks.workunit.client.0.smithi156.stdout:9/372: unlink d2/de/d42/f47 0 2024-07-16T09:55:35.749 INFO:tasks.workunit.client.1.smithi176.stdout:1/207: stat d4/d8/f26 0 2024-07-16T09:55:35.749 INFO:tasks.workunit.client.1.smithi176.stdout:4/336: write da/d3d/d33/f3a [488991,90638] 0 2024-07-16T09:55:35.750 INFO:tasks.workunit.client.1.smithi176.stdout:1/208: chown d4/db/c11 159 1 2024-07-16T09:55:35.751 INFO:tasks.workunit.client.1.smithi176.stdout:7/373: getdents d4/d1f/d22/d51 0 2024-07-16T09:55:35.752 INFO:tasks.workunit.client.1.smithi176.stdout:0/386: symlink d5/d2c/d6c/l78 0 2024-07-16T09:55:35.753 INFO:tasks.workunit.client.0.smithi156.stdout:9/373: mkdir d2/d5/d16/d22/d2b/d74 0 2024-07-16T09:55:35.753 INFO:tasks.workunit.client.1.smithi176.stdout:0/387: chown d5/d1c/d24/d35/c57 25065563 1 2024-07-16T09:55:35.754 INFO:tasks.workunit.client.1.smithi176.stdout:4/337: creat da/dc/d39/f61 x:0 0 0 2024-07-16T09:55:35.754 INFO:tasks.workunit.client.1.smithi176.stdout:1/209: creat d4/db/f49 x:0 0 0 2024-07-16T09:55:35.754 INFO:tasks.workunit.client.1.smithi176.stdout:7/374: mknod d4/d5/d21/c7d 0 2024-07-16T09:55:35.755 INFO:tasks.workunit.client.0.smithi156.stdout:9/374: mknod d2/de/c75 0 2024-07-16T09:55:35.755 INFO:tasks.workunit.client.1.smithi176.stdout:7/375: dread - d4/d5/d10/d17/d1d/f5b zero size 2024-07-16T09:55:35.756 INFO:tasks.workunit.client.1.smithi176.stdout:2/343: dread dd/d2e/d35/f3f [0,4194304] 0 2024-07-16T09:55:35.757 INFO:tasks.workunit.client.1.smithi176.stdout:1/210: stat c2 0 2024-07-16T09:55:35.758 INFO:tasks.workunit.client.1.smithi176.stdout:4/338: mkdir da/dc/d4b/d62 0 2024-07-16T09:55:35.758 INFO:tasks.workunit.client.0.smithi156.stdout:9/375: rename d2/c7 to d2/d5/d16/d22/d36/d33/d3a/c76 0 2024-07-16T09:55:35.759 INFO:tasks.workunit.client.1.smithi176.stdout:7/376: chown d4/l56 179847853 1 2024-07-16T09:55:35.759 INFO:tasks.workunit.client.1.smithi176.stdout:4/339: write da/dc/d39/f4f [655446,74318] 0 2024-07-16T09:55:35.760 INFO:tasks.workunit.client.0.smithi156.stdout:9/376: symlink d2/l77 0 2024-07-16T09:55:35.761 INFO:tasks.workunit.client.1.smithi176.stdout:7/377: readlink d4/dd/d26/l40 0 2024-07-16T09:55:35.761 INFO:tasks.workunit.client.1.smithi176.stdout:4/340: fsync da/dc/f45 0 2024-07-16T09:55:35.762 INFO:tasks.workunit.client.0.smithi156.stdout:9/377: dread d2/d5/d16/f37 [0,4194304] 0 2024-07-16T09:55:35.763 INFO:tasks.workunit.client.1.smithi176.stdout:7/378: chown d4/d5/d10/d17/f4d 3089 1 2024-07-16T09:55:35.764 INFO:tasks.workunit.client.1.smithi176.stdout:1/211: dwrite d4/db/f1b [4194304,4194304] 0 2024-07-16T09:55:35.770 INFO:tasks.workunit.client.0.smithi156.stdout:9/378: dread d2/d5/d16/d22/d36/d33/d3a/f49 [0,4194304] 0 2024-07-16T09:55:35.770 INFO:tasks.workunit.client.0.smithi156.stdout:9/379: readlink d2/de/d42/l45 0 2024-07-16T09:55:35.770 INFO:tasks.workunit.client.0.smithi156.stdout:9/380: chown d2/de/c11 18 1 2024-07-16T09:55:35.770 INFO:tasks.workunit.client.1.smithi176.stdout:1/212: write d4/db/f1b [9050784,103755] 0 2024-07-16T09:55:35.771 INFO:tasks.workunit.client.1.smithi176.stdout:2/344: creat dd/d16/d39/f6d x:0 0 0 2024-07-16T09:55:35.771 INFO:tasks.workunit.client.1.smithi176.stdout:7/379: mkdir d4/d5/d21/d7e 0 2024-07-16T09:55:35.771 INFO:tasks.workunit.client.1.smithi176.stdout:7/380: readlink d4/d5/d10/l72 0 2024-07-16T09:55:35.771 INFO:tasks.workunit.client.1.smithi176.stdout:2/345: write dd/d16/f56 [2236909,115485] 0 2024-07-16T09:55:35.771 INFO:tasks.workunit.client.1.smithi176.stdout:7/381: readlink d4/l56 0 2024-07-16T09:55:35.772 INFO:tasks.workunit.client.1.smithi176.stdout:0/388: dwrite d5/f8 [0,4194304] 0 2024-07-16T09:55:35.773 INFO:tasks.workunit.client.1.smithi176.stdout:2/346: write dd/d16/f32 [726569,1071] 0 2024-07-16T09:55:35.775 INFO:tasks.workunit.client.1.smithi176.stdout:1/213: unlink d4/d8/d12/d21/d25/d30/f38 0 2024-07-16T09:55:35.775 INFO:tasks.workunit.client.1.smithi176.stdout:2/347: write dd/d2e/f37 [3647451,33632] 0 2024-07-16T09:55:35.778 INFO:tasks.workunit.client.1.smithi176.stdout:1/214: write d4/db/f43 [292752,26845] 0 2024-07-16T09:55:35.778 INFO:tasks.workunit.client.1.smithi176.stdout:4/341: dread da/dc/d10/f27 [0,4194304] 0 2024-07-16T09:55:35.779 INFO:tasks.workunit.client.1.smithi176.stdout:0/389: creat d5/d1c/d24/d35/d50/d58/f79 x:0 0 0 2024-07-16T09:55:35.780 INFO:tasks.workunit.client.1.smithi176.stdout:2/348: truncate dd/d36/f61 206461 0 2024-07-16T09:55:35.782 INFO:tasks.workunit.client.1.smithi176.stdout:4/342: chown da/d14/d4a 12363 1 2024-07-16T09:55:35.782 INFO:tasks.workunit.client.1.smithi176.stdout:1/215: write d4/d8/fa [4831250,94687] 0 2024-07-16T09:55:35.784 INFO:tasks.workunit.client.1.smithi176.stdout:0/390: truncate d5/d1c/d5a/f75 106539 0 2024-07-16T09:55:35.785 INFO:tasks.workunit.client.0.smithi156.stdout:9/381: truncate d2/d5/fb 4135724 0 2024-07-16T09:55:35.787 INFO:tasks.workunit.client.1.smithi176.stdout:7/382: rename d4/dd/f36 to d4/d5/d10/d17/f7f 0 2024-07-16T09:55:35.789 INFO:tasks.workunit.client.0.smithi156.stdout:9/382: creat d2/d5/d16/d22/d36/f78 x:0 0 0 2024-07-16T09:55:35.790 INFO:tasks.workunit.client.1.smithi176.stdout:1/216: creat d4/d8/d12/f4a x:0 0 0 2024-07-16T09:55:35.791 INFO:tasks.workunit.client.1.smithi176.stdout:4/343: unlink da/dc/l36 0 2024-07-16T09:55:35.792 INFO:tasks.workunit.client.1.smithi176.stdout:4/344: write da/fb [877468,111981] 0 2024-07-16T09:55:35.792 INFO:tasks.workunit.client.1.smithi176.stdout:2/349: creat dd/d2e/d35/d59/d66/f6e x:0 0 0 2024-07-16T09:55:35.793 INFO:tasks.workunit.client.1.smithi176.stdout:1/217: creat d4/d8/d12/f4b x:0 0 0 2024-07-16T09:55:35.794 INFO:tasks.workunit.client.1.smithi176.stdout:2/350: write dd/d36/f2f [1027371,122033] 0 2024-07-16T09:55:35.795 INFO:tasks.workunit.client.1.smithi176.stdout:4/345: symlink da/dc/l63 0 2024-07-16T09:55:35.795 INFO:tasks.workunit.client.1.smithi176.stdout:2/351: stat dd/d49/d52 0 2024-07-16T09:55:35.797 INFO:tasks.workunit.client.1.smithi176.stdout:7/383: rename d4/d5/d10 to d4/d1f/d22/d80 0 2024-07-16T09:55:35.799 INFO:tasks.workunit.client.1.smithi176.stdout:7/384: fdatasync d4/d1f/f6f 0 2024-07-16T09:55:35.800 INFO:tasks.workunit.client.1.smithi176.stdout:2/352: rename dd/d49/f65 to dd/d16/d39/f6f 0 2024-07-16T09:55:35.801 INFO:tasks.workunit.client.0.smithi156.stdout:8/285: sync 2024-07-16T09:55:35.802 INFO:tasks.workunit.client.0.smithi156.stdout:8/286: write d0/dd/f17 [1895324,3781] 0 2024-07-16T09:55:35.802 INFO:tasks.workunit.client.1.smithi176.stdout:4/346: dread da/d3d/d17/f18 [0,4194304] 0 2024-07-16T09:55:35.802 INFO:tasks.workunit.client.1.smithi176.stdout:1/218: dwrite d4/d8/d12/f4b [0,4194304] 0 2024-07-16T09:55:35.802 INFO:tasks.workunit.client.1.smithi176.stdout:7/385: mkdir d4/dd/d81 0 2024-07-16T09:55:35.803 INFO:tasks.workunit.client.1.smithi176.stdout:2/353: mknod dd/c70 0 2024-07-16T09:55:35.806 INFO:tasks.workunit.client.0.smithi156.stdout:8/287: unlink d0/d11/d20/f25 0 2024-07-16T09:55:35.806 INFO:tasks.workunit.client.1.smithi176.stdout:7/386: read d4/dd/f2e [845745,76812] 0 2024-07-16T09:55:35.806 INFO:tasks.workunit.client.1.smithi176.stdout:1/219: write d4/d8/d12/d21/d34/f3d [822691,83744] 0 2024-07-16T09:55:35.807 INFO:tasks.workunit.client.1.smithi176.stdout:4/347: creat da/d3d/d17/f64 x:0 0 0 2024-07-16T09:55:35.808 INFO:tasks.workunit.client.1.smithi176.stdout:2/354: creat dd/d16/f71 x:0 0 0 2024-07-16T09:55:35.810 INFO:tasks.workunit.client.1.smithi176.stdout:7/387: unlink d4/d5/d21/c7d 0 2024-07-16T09:55:35.811 INFO:tasks.workunit.client.1.smithi176.stdout:4/348: creat da/d3d/d17/d48/f65 x:0 0 0 2024-07-16T09:55:35.812 INFO:tasks.workunit.client.1.smithi176.stdout:7/388: write d4/f2b [871179,98461] 0 2024-07-16T09:55:35.814 INFO:tasks.workunit.client.1.smithi176.stdout:7/389: readlink d4/d5/d21/l69 0 2024-07-16T09:55:35.815 INFO:tasks.workunit.client.1.smithi176.stdout:4/349: readlink da/d3d/d17/d48/l49 0 2024-07-16T09:55:35.815 INFO:tasks.workunit.client.1.smithi176.stdout:1/220: dwrite d4/d8/fa [4194304,4194304] 0 2024-07-16T09:55:35.817 INFO:tasks.workunit.client.1.smithi176.stdout:1/221: mkdir d4/d8/d12/d4c 0 2024-07-16T09:55:35.818 INFO:tasks.workunit.client.1.smithi176.stdout:7/390: rename d4/dd/c5f to d4/d5/d21/d7e/c82 0 2024-07-16T09:55:35.818 INFO:tasks.workunit.client.1.smithi176.stdout:4/350: symlink da/d3d/d17/d54/l66 0 2024-07-16T09:55:35.819 INFO:tasks.workunit.client.1.smithi176.stdout:4/351: chown da/d41 599882 1 2024-07-16T09:55:35.820 INFO:tasks.workunit.client.1.smithi176.stdout:4/352: chown da/d3d/d17/l21 5 1 2024-07-16T09:55:35.822 INFO:tasks.workunit.client.1.smithi176.stdout:4/353: write da/dc/d10/f59 [173278,92409] 0 2024-07-16T09:55:35.822 INFO:tasks.workunit.client.1.smithi176.stdout:1/222: rename d4/d8/d2c/d2d to d4/d8/d12/d21/d25/d30/d44/d4d 0 2024-07-16T09:55:35.825 INFO:tasks.workunit.client.1.smithi176.stdout:4/354: symlink da/d3d/l67 0 2024-07-16T09:55:35.826 INFO:tasks.workunit.client.1.smithi176.stdout:1/223: truncate d4/d8/d12/f4b 4982328 0 2024-07-16T09:55:35.826 INFO:tasks.workunit.client.1.smithi176.stdout:7/391: dwrite d4/dd/f23 [0,4194304] 0 2024-07-16T09:55:35.827 INFO:tasks.workunit.client.1.smithi176.stdout:4/355: write da/d3d/d17/f2a [2845115,99021] 0 2024-07-16T09:55:35.829 INFO:tasks.workunit.client.1.smithi176.stdout:1/224: truncate d4/ff 2807243 0 2024-07-16T09:55:35.830 INFO:tasks.workunit.client.1.smithi176.stdout:7/392: getdents d4/dd/d49/d7a 0 2024-07-16T09:55:35.831 INFO:tasks.workunit.client.1.smithi176.stdout:4/356: chown da/dc/d4b/d62 146511667 1 2024-07-16T09:55:35.832 INFO:tasks.workunit.client.1.smithi176.stdout:2/355: dread dd/de/f10 [0,4194304] 0 2024-07-16T09:55:35.833 INFO:tasks.workunit.client.1.smithi176.stdout:7/393: readlink d4/d1f/d22/d80/l72 0 2024-07-16T09:55:35.834 INFO:tasks.workunit.client.1.smithi176.stdout:1/225: mknod d4/d8/d12/c4e 0 2024-07-16T09:55:35.835 INFO:tasks.workunit.client.1.smithi176.stdout:7/394: mknod d4/d1f/d22/d51/c83 0 2024-07-16T09:55:35.836 INFO:tasks.workunit.client.1.smithi176.stdout:2/356: symlink dd/d16/d39/l72 0 2024-07-16T09:55:35.836 INFO:tasks.workunit.client.1.smithi176.stdout:7/395: write d4/d5/f6c [3016721,72632] 0 2024-07-16T09:55:35.838 INFO:tasks.workunit.client.1.smithi176.stdout:4/357: truncate da/dc/f45 551780 0 2024-07-16T09:55:35.839 INFO:tasks.workunit.client.1.smithi176.stdout:4/358: readlink da/d3d/d17/l21 0 2024-07-16T09:55:35.841 INFO:tasks.workunit.client.1.smithi176.stdout:1/226: rename d4/d8/d12/d21/l22 to d4/l4f 0 2024-07-16T09:55:35.842 INFO:tasks.workunit.client.1.smithi176.stdout:4/359: creat da/dc/d10/d28/f68 x:0 0 0 2024-07-16T09:55:35.844 INFO:tasks.workunit.client.1.smithi176.stdout:7/396: getdents d4/d5/d21/d66 0 2024-07-16T09:55:35.846 INFO:tasks.workunit.client.1.smithi176.stdout:4/360: mkdir da/d3d/d17/d54/d5e/d69 0 2024-07-16T09:55:35.846 INFO:tasks.workunit.client.1.smithi176.stdout:7/397: symlink d4/d1f/d22/l84 0 2024-07-16T09:55:35.847 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:35.846+0000 7fdf7d2ae640 1 -- 172.21.15.156:0/115452685 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdf780fb240 msgr2=0x7fdf780fb650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:55:35.847 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:35.846+0000 7fdf7d2ae640 1 --2- 172.21.15.156:0/115452685 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdf780fb240 0x7fdf780fb650 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fdf680048d0 tx=0x7fdf68009da0 comp rx=0 tx=0).stop 2024-07-16T09:55:35.847 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:35.846+0000 7fdf7d2ae640 1 -- 172.21.15.156:0/115452685 shutdown_connections 2024-07-16T09:55:35.848 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:35.846+0000 7fdf7d2ae640 1 --2- 172.21.15.156:0/115452685 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fdf780fc510 0x7fdf780fc980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:55:35.848 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:35.846+0000 7fdf7d2ae640 1 --2- 172.21.15.156:0/115452685 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdf780fb240 0x7fdf780fb650 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:55:35.848 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:35.846+0000 7fdf7d2ae640 1 -- 172.21.15.156:0/115452685 >> 172.21.15.156:0/115452685 conn(0x7fdf780f6a20 msgr2=0x7fdf780f8e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:55:35.848 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:35.846+0000 7fdf7d2ae640 1 -- 172.21.15.156:0/115452685 shutdown_connections 2024-07-16T09:55:35.848 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:35.846+0000 7fdf7d2ae640 1 -- 172.21.15.156:0/115452685 wait complete. 2024-07-16T09:55:35.848 INFO:tasks.workunit.client.1.smithi176.stdout:1/227: dwrite d4/d8/d12/d21/d25/d30/d44/d4d/f32 [0,4194304] 0 2024-07-16T09:55:35.849 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:35.847+0000 7fdf7d2ae640 1 Processor -- start 2024-07-16T09:55:35.849 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:35.847+0000 7fdf7d2ae640 1 -- start start 2024-07-16T09:55:35.850 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:35.847+0000 7fdf7d2ae640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fdf780fb240 0x7fdf7810bf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:55:35.850 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:35.847+0000 7fdf7d2ae640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdf780fc510 0x7fdf7810c440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:55:35.850 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:35.847+0000 7fdf7d2ae640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf7810d170 con 0x7fdf780fc510 2024-07-16T09:55:35.850 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:35.847+0000 7fdf7d2ae640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf7810d2e0 con 0x7fdf780fb240 2024-07-16T09:55:35.850 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:35.847+0000 7fdf767fc640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdf780fc510 0x7fdf7810c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:55:35.850 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:35.847+0000 7fdf767fc640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdf780fc510 0x7fdf7810c440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:40952/0 (socket says 172.21.15.156:40952) 2024-07-16T09:55:35.850 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:35.847+0000 7fdf767fc640 1 -- 172.21.15.156:0/3729096851 learned_addr learned my addr 172.21.15.156:0/3729096851 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T09:55:35.850 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:35.847+0000 7fdf76ffd640 1 --2- 172.21.15.156:0/3729096851 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fdf780fb240 0x7fdf7810bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:55:35.850 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:35.848+0000 7fdf767fc640 1 -- 172.21.15.156:0/3729096851 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fdf780fb240 msgr2=0x7fdf7810bf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:55:35.850 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:35.848+0000 7fdf767fc640 1 --2- 172.21.15.156:0/3729096851 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fdf780fb240 0x7fdf7810bf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:55:35.850 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:35.848+0000 7fdf767fc640 1 -- 172.21.15.156:0/3729096851 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf68005040 con 0x7fdf780fc510 2024-07-16T09:55:35.850 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:35.848+0000 7fdf767fc640 1 --2- 172.21.15.156:0/3729096851 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdf780fc510 0x7fdf7810c440 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fdf6c004e80 tx=0x7fdf6c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:55:35.851 INFO:tasks.workunit.client.1.smithi176.stdout:7/398: rmdir d4/d5 39 2024-07-16T09:55:35.851 INFO:tasks.workunit.client.1.smithi176.stdout:4/361: rename da/d14/l2f to da/d3d/d17/d54/d5e/l6a 0 2024-07-16T09:55:35.852 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:35.848+0000 7fdf5f7fe640 1 -- 172.21.15.156:0/3729096851 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdf6c007e10 con 0x7fdf780fc510 2024-07-16T09:55:35.852 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:35.848+0000 7fdf7d2ae640 1 -- 172.21.15.156:0/3729096851 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf7810d560 con 0x7fdf780fc510 2024-07-16T09:55:35.852 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:35.848+0000 7fdf7d2ae640 1 -- 172.21.15.156:0/3729096851 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf7810d970 con 0x7fdf780fc510 2024-07-16T09:55:35.852 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:35.848+0000 7fdf5f7fe640 1 -- 172.21.15.156:0/3729096851 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdf6c00f040 con 0x7fdf780fc510 2024-07-16T09:55:35.852 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:35.848+0000 7fdf5f7fe640 1 -- 172.21.15.156:0/3729096851 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdf6c00c940 con 0x7fdf780fc510 2024-07-16T09:55:35.852 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:35.850+0000 7fdf5f7fe640 1 -- 172.21.15.156:0/3729096851 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 20) v1 ==== 88539+0+0 (secure 0 0 0) 0x7fdf6c018030 con 0x7fdf780fc510 2024-07-16T09:55:35.853 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:35.850+0000 7fdf7d2ae640 1 -- 172.21.15.156:0/3729096851 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf780fa6a0 con 0x7fdf780fc510 2024-07-16T09:55:35.853 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:35.850+0000 7fdf5f7fe640 1 --2- 172.21.15.156:0/3729096851 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7fdf60072db0 0x7fdf60075260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:55:35.853 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:35.851+0000 7fdf5f7fe640 1 -- 172.21.15.156:0/3729096851 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5464+0+0 (secure 0 0 0) 0x7fdf6c016070 con 0x7fdf780fc510 2024-07-16T09:55:35.853 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:35.851+0000 7fdf76ffd640 1 --2- 172.21.15.156:0/3729096851 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7fdf60072db0 0x7fdf60075260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:55:35.853 INFO:tasks.workunit.client.1.smithi176.stdout:7/399: creat d4/dd/d63/f85 x:0 0 0 2024-07-16T09:55:35.854 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:35.851+0000 7fdf76ffd640 1 --2- 172.21.15.156:0/3729096851 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7fdf60072db0 0x7fdf60075260 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fdf68009f80 tx=0x7fdf68035040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:55:35.856 INFO:tasks.workunit.client.1.smithi176.stdout:4/362: dwrite da/dc/d10/f59 [0,4194304] 0 2024-07-16T09:55:35.859 INFO:tasks.workunit.client.1.smithi176.stdout:7/400: fdatasync d4/d1f/d22/f3a 0 2024-07-16T09:55:35.860 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:35.858+0000 7fdf5f7fe640 1 -- 172.21.15.156:0/3729096851 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fdf6c055c60 con 0x7fdf780fc510 2024-07-16T09:55:35.861 INFO:tasks.workunit.client.1.smithi176.stdout:4/363: rmdir da 39 2024-07-16T09:55:35.863 INFO:tasks.workunit.client.1.smithi176.stdout:4/364: dread - da/d14/d4a/f4e zero size 2024-07-16T09:55:35.863 INFO:tasks.workunit.client.1.smithi176.stdout:7/401: mknod d4/d1f/d22/d80/d17/d1d/c86 0 2024-07-16T09:55:35.866 INFO:tasks.workunit.client.1.smithi176.stdout:7/402: write d4/dd/f1a [3271686,49463] 0 2024-07-16T09:55:35.867 INFO:tasks.workunit.client.1.smithi176.stdout:4/365: symlink da/d3d/d17/d54/l6b 0 2024-07-16T09:55:35.868 INFO:tasks.workunit.client.1.smithi176.stdout:7/403: chown d4/d47/d5d/f77 21 1 2024-07-16T09:55:35.869 INFO:tasks.workunit.client.1.smithi176.stdout:2/357: dwrite dd/d2e/d35/f3f [0,4194304] 0 2024-07-16T09:55:35.869 INFO:tasks.workunit.client.1.smithi176.stdout:1/228: truncate d4/d8/d12/d21/d34/f3d 30645 0 2024-07-16T09:55:35.872 INFO:tasks.workunit.client.1.smithi176.stdout:4/366: unlink da/dc/d10/d28/f68 0 2024-07-16T09:55:35.873 INFO:tasks.workunit.client.1.smithi176.stdout:7/404: mkdir d4/d5/d21/d7e/d87 0 2024-07-16T09:55:35.874 INFO:tasks.workunit.client.1.smithi176.stdout:7/405: fdatasync d4/d1f/d22/f52 0 2024-07-16T09:55:35.875 INFO:tasks.workunit.client.1.smithi176.stdout:1/229: mknod d4/d8/d12/d4c/c50 0 2024-07-16T09:55:35.876 INFO:tasks.workunit.client.1.smithi176.stdout:1/230: write d4/d8/f26 [841118,112839] 0 2024-07-16T09:55:35.877 INFO:tasks.workunit.client.1.smithi176.stdout:1/231: readlink d4/d8/d12/d21/d34/l3c 0 2024-07-16T09:55:35.881 INFO:tasks.workunit.client.1.smithi176.stdout:1/232: creat d4/d8/d12/d21/d34/f51 x:0 0 0 2024-07-16T09:55:35.882 INFO:tasks.workunit.client.1.smithi176.stdout:4/367: link da/d3d/l67 da/d14/d40/l6c 0 2024-07-16T09:55:35.885 INFO:tasks.workunit.client.1.smithi176.stdout:1/233: symlink d4/l52 0 2024-07-16T09:55:35.888 INFO:tasks.workunit.client.1.smithi176.stdout:1/234: rmdir d4/d8/d12/d21/d25/d30/d44 39 2024-07-16T09:55:35.888 INFO:tasks.workunit.client.1.smithi176.stdout:4/368: dwrite da/dc/f50 [0,4194304] 0 2024-07-16T09:55:35.890 INFO:tasks.workunit.client.1.smithi176.stdout:7/406: dread d4/f27 [0,4194304] 0 2024-07-16T09:55:35.892 INFO:tasks.workunit.client.1.smithi176.stdout:7/407: write f0 [2265268,101261] 0 2024-07-16T09:55:35.894 INFO:tasks.workunit.client.1.smithi176.stdout:7/408: fsync d4/dd/d49/f55 0 2024-07-16T09:55:35.897 INFO:tasks.workunit.client.1.smithi176.stdout:7/409: write d4/d47/f34 [1493570,78516] 0 2024-07-16T09:55:35.898 INFO:tasks.workunit.client.1.smithi176.stdout:4/369: dwrite da/dc/d4b/f52 [0,4194304] 0 2024-07-16T09:55:35.902 INFO:tasks.workunit.client.1.smithi176.stdout:7/410: unlink d4/dd/c54 0 2024-07-16T09:55:35.904 INFO:tasks.workunit.client.1.smithi176.stdout:7/411: dread - d4/d1f/d22/d51/f67 zero size 2024-07-16T09:55:35.906 INFO:tasks.workunit.client.1.smithi176.stdout:4/370: dwrite da/d3d/f3e [0,4194304] 0 2024-07-16T09:55:35.912 INFO:tasks.workunit.client.1.smithi176.stdout:7/412: link d4/dd/d63/f64 d4/d47/f88 0 2024-07-16T09:55:35.913 INFO:tasks.workunit.client.1.smithi176.stdout:1/235: truncate d4/d8/d12/f4b 289979 0 2024-07-16T09:55:35.914 INFO:tasks.workunit.client.1.smithi176.stdout:4/371: dwrite da/dc/f55 [0,4194304] 0 2024-07-16T09:55:35.916 INFO:tasks.workunit.client.1.smithi176.stdout:1/236: mknod d4/d8/d12/c53 0 2024-07-16T09:55:35.918 INFO:tasks.workunit.client.1.smithi176.stdout:7/413: rename d4/d5/d21 to d4/d47/d6d/d89 0 2024-07-16T09:55:35.919 INFO:tasks.workunit.client.1.smithi176.stdout:4/372: write da/dc/d10/f27 [3170125,98647] 0 2024-07-16T09:55:35.921 INFO:tasks.workunit.client.1.smithi176.stdout:7/414: creat d4/d47/d6d/d89/d66/f8a x:0 0 0 2024-07-16T09:55:35.922 INFO:tasks.workunit.client.1.smithi176.stdout:1/237: mkdir d4/d3a/d54 0 2024-07-16T09:55:35.923 INFO:tasks.workunit.client.1.smithi176.stdout:1/238: dread - d4/d8/d12/d21/d34/f51 zero size 2024-07-16T09:55:35.924 INFO:tasks.workunit.client.1.smithi176.stdout:4/373: read da/dc/f45 [185154,73449] 0 2024-07-16T09:55:35.926 INFO:tasks.workunit.client.1.smithi176.stdout:1/239: creat d4/d8/d12/d21/d25/f55 x:0 0 0 2024-07-16T09:55:35.927 INFO:tasks.workunit.client.1.smithi176.stdout:4/374: creat da/d3d/d17/d54/d5e/d69/f6d x:0 0 0 2024-07-16T09:55:35.928 INFO:tasks.workunit.client.1.smithi176.stdout:4/375: chown da/d3d/d17/d54/d5e/d69/f6d 2400010 1 2024-07-16T09:55:35.930 INFO:tasks.workunit.client.1.smithi176.stdout:4/376: write da/f19 [1868181,107536] 0 2024-07-16T09:55:35.931 INFO:tasks.workunit.client.1.smithi176.stdout:1/240: dwrite d4/db/f1b [8388608,4194304] 0 2024-07-16T09:55:35.934 INFO:tasks.workunit.client.1.smithi176.stdout:4/377: mknod da/dc/d4b/c6e 0 2024-07-16T09:55:35.935 INFO:tasks.workunit.client.1.smithi176.stdout:7/415: rmdir d4/d47/d6d/d89 39 2024-07-16T09:55:35.935 INFO:tasks.workunit.client.1.smithi176.stdout:7/416: stat d4/d47/d6d/c74 0 2024-07-16T09:55:35.937 INFO:tasks.workunit.client.1.smithi176.stdout:4/378: fdatasync da/d3d/d17/f18 0 2024-07-16T09:55:35.937 INFO:tasks.workunit.client.1.smithi176.stdout:8/457: sync 2024-07-16T09:55:35.937 INFO:tasks.workunit.client.1.smithi176.stdout:7/417: write d4/dd/f1a [3710362,10425] 0 2024-07-16T09:55:35.938 INFO:tasks.workunit.client.1.smithi176.stdout:1/241: rmdir d4/d8/d12/d21/d25/d30/d44 39 2024-07-16T09:55:35.939 INFO:tasks.workunit.client.1.smithi176.stdout:4/379: write da/dc/f55 [1149349,44650] 0 2024-07-16T09:55:35.942 INFO:tasks.workunit.client.1.smithi176.stdout:8/458: mknod d8/d20/d2a/d31/d2f/c9c 0 2024-07-16T09:55:35.943 INFO:tasks.workunit.client.1.smithi176.stdout:7/418: mkdir d4/d1f/d22/d80/d17/d1d/d8b 0 2024-07-16T09:55:35.943 INFO:tasks.workunit.client.1.smithi176.stdout:7/419: chown d4/d1f/d22/l5e 0 1 2024-07-16T09:55:35.944 INFO:tasks.workunit.client.0.smithi156.stdout:5/349: sync 2024-07-16T09:55:35.944 INFO:tasks.workunit.client.0.smithi156.stdout:1/339: sync 2024-07-16T09:55:35.944 INFO:tasks.workunit.client.0.smithi156.stdout:2/428: sync 2024-07-16T09:55:35.944 INFO:tasks.workunit.client.1.smithi176.stdout:1/242: mkdir d4/d8/d2c/d56 0 2024-07-16T09:55:35.945 INFO:tasks.workunit.client.1.smithi176.stdout:4/380: creat da/d3d/d33/f6f x:0 0 0 2024-07-16T09:55:35.946 INFO:tasks.workunit.client.0.smithi156.stdout:2/429: write d0/d13/d17/f42 [3446758,14468] 0 2024-07-16T09:55:35.946 INFO:tasks.workunit.client.1.smithi176.stdout:8/459: mkdir d8/d14/d1e/d33/d9d 0 2024-07-16T09:55:35.946 INFO:tasks.workunit.client.1.smithi176.stdout:4/381: readlink l9 0 2024-07-16T09:55:35.947 INFO:tasks.workunit.client.0.smithi156.stdout:1/340: creat dc/d14/d1e/f77 x:0 0 0 2024-07-16T09:55:35.947 INFO:tasks.workunit.client.0.smithi156.stdout:5/350: dread d1b/d1f/d45/d52/f39 [0,4194304] 0 2024-07-16T09:55:35.947 INFO:tasks.workunit.client.1.smithi176.stdout:4/382: chown da/dc/l1e 72475886 1 2024-07-16T09:55:35.948 INFO:tasks.workunit.client.0.smithi156.stdout:1/341: fsync dc/d24/f5e 0 2024-07-16T09:55:35.949 INFO:tasks.workunit.client.0.smithi156.stdout:2/430: getdents d0/d13/d17/d1c/d51 0 2024-07-16T09:55:35.949 INFO:tasks.workunit.client.0.smithi156.stdout:1/342: dread - dc/d14/d3e/d55/d57/f67 zero size 2024-07-16T09:55:35.950 INFO:tasks.workunit.client.1.smithi176.stdout:1/243: creat d4/f57 x:0 0 0 2024-07-16T09:55:35.950 INFO:tasks.workunit.client.1.smithi176.stdout:4/383: truncate da/d3d/d17/f64 935237 0 2024-07-16T09:55:35.951 INFO:tasks.workunit.client.1.smithi176.stdout:7/420: dwrite d4/dd/f1a [0,4194304] 0 2024-07-16T09:55:35.951 INFO:tasks.workunit.client.1.smithi176.stdout:1/244: chown d4/db/c11 140 1 2024-07-16T09:55:35.952 INFO:tasks.workunit.client.0.smithi156.stdout:1/343: chown dc/d14/d3e/d45/f53 0 1 2024-07-16T09:55:35.953 INFO:tasks.workunit.client.1.smithi176.stdout:8/460: dread d8/d23/d2c/d34/d62/f7d [0,4194304] 0 2024-07-16T09:55:35.953 INFO:tasks.workunit.client.0.smithi156.stdout:1/344: stat dc/d24/l39 0 2024-07-16T09:55:35.954 INFO:tasks.workunit.client.1.smithi176.stdout:4/384: unlink da/d3d/f3e 0 2024-07-16T09:55:35.954 INFO:tasks.workunit.client.0.smithi156.stdout:2/431: truncate d0/d13/d17/f21 2621005 0 2024-07-16T09:55:35.955 INFO:tasks.workunit.client.0.smithi156.stdout:5/351: dwrite d1b/f2f [0,4194304] 0 2024-07-16T09:55:35.956 INFO:tasks.workunit.client.1.smithi176.stdout:7/421: mknod d4/d5/c8c 0 2024-07-16T09:55:35.958 INFO:tasks.workunit.client.0.smithi156.stdout:2/432: write d0/d13/d17/f34 [1333689,70097] 0 2024-07-16T09:55:35.961 INFO:tasks.workunit.client.0.smithi156.stdout:5/352: write fc [301958,7758] 0 2024-07-16T09:55:35.961 INFO:tasks.workunit.client.1.smithi176.stdout:8/461: symlink d8/d20/d2a/d31/d46/d6a/l9e 0 2024-07-16T09:55:35.961 INFO:tasks.workunit.client.1.smithi176.stdout:1/245: rename d4/f10 to d4/d8/d12/d21/d34/f58 0 2024-07-16T09:55:35.962 INFO:tasks.workunit.client.0.smithi156.stdout:5/353: chown d1b/l26 368 1 2024-07-16T09:55:35.963 INFO:tasks.workunit.client.0.smithi156.stdout:1/345: dread f4 [0,4194304] 0 2024-07-16T09:55:35.964 INFO:tasks.workunit.client.0.smithi156.stdout:2/433: readlink d0/d5/d29/d2d/d5b/d62/l72 0 2024-07-16T09:55:35.966 INFO:tasks.workunit.client.0.smithi156.stdout:2/434: truncate d0/d5/d29/f3c 4589246 0 2024-07-16T09:55:35.966 INFO:tasks.workunit.client.1.smithi176.stdout:4/385: symlink da/dc/l70 0 2024-07-16T09:55:35.967 INFO:tasks.workunit.client.0.smithi156.stdout:2/435: chown d0/d5 1632846447 1 2024-07-16T09:55:35.969 INFO:tasks.workunit.client.0.smithi156.stdout:1/346: mknod dc/d14/d3e/d55/d57/d76/c78 0 2024-07-16T09:55:35.970 INFO:tasks.workunit.client.0.smithi156.stdout:2/436: chown d0/d13/d17/d1c/f47 62 1 2024-07-16T09:55:35.970 INFO:tasks.workunit.client.1.smithi176.stdout:7/422: mkdir d4/d47/d6d/d89/d7e/d8d 0 2024-07-16T09:55:35.971 INFO:tasks.workunit.client.1.smithi176.stdout:7/423: fsync d4/dd/f23 0 2024-07-16T09:55:35.972 INFO:tasks.workunit.client.0.smithi156.stdout:1/347: chown dc/d20/l71 4042 1 2024-07-16T09:55:35.972 INFO:tasks.workunit.client.0.smithi156.stdout:5/354: dwrite d1b/d1f/d45/d52/d43/f5b [0,4194304] 0 2024-07-16T09:55:35.972 INFO:tasks.workunit.client.1.smithi176.stdout:1/246: rename d4/db/f23 to d4/f59 0 2024-07-16T09:55:35.973 INFO:tasks.workunit.client.1.smithi176.stdout:1/247: stat d4/c3b 0 2024-07-16T09:55:35.974 INFO:tasks.workunit.client.1.smithi176.stdout:7/424: fsync d4/d1f/f2a 0 2024-07-16T09:55:35.974 INFO:tasks.workunit.client.0.smithi156.stdout:2/437: creat d0/d5/d8b/f97 x:0 0 0 2024-07-16T09:55:35.975 INFO:tasks.workunit.client.1.smithi176.stdout:4/386: rename da/dc/d4b/c6e to da/d3d/c71 0 2024-07-16T09:55:35.976 INFO:tasks.workunit.client.0.smithi156.stdout:5/355: rmdir d1b/d1f/d42 39 2024-07-16T09:55:35.977 INFO:tasks.workunit.client.0.smithi156.stdout:2/438: symlink d0/d5/d8b/l98 0 2024-07-16T09:55:35.978 INFO:tasks.workunit.client.1.smithi176.stdout:1/248: mkdir d4/d3a/d54/d5a 0 2024-07-16T09:55:35.980 INFO:tasks.workunit.client.1.smithi176.stdout:7/425: mkdir d4/dd/d8e 0 2024-07-16T09:55:35.980 INFO:tasks.workunit.client.0.smithi156.stdout:5/356: mkdir d1b/d1f/d45/d52/d43/d7c 0 2024-07-16T09:55:35.980 INFO:tasks.workunit.client.0.smithi156.stdout:5/357: fdatasync d1b/d1e/f25 0 2024-07-16T09:55:35.981 INFO:tasks.workunit.client.0.smithi156.stdout:2/439: creat d0/d5/d29/d2d/d5b/d5f/d89/f99 x:0 0 0 2024-07-16T09:55:35.982 INFO:tasks.workunit.client.0.smithi156.stdout:1/348: getdents dc/d14/d1e 0 2024-07-16T09:55:35.983 INFO:tasks.workunit.client.0.smithi156.stdout:5/358: mknod d1b/d1f/d45/d52/d43/c7d 0 2024-07-16T09:55:35.984 INFO:tasks.workunit.client.1.smithi176.stdout:4/387: dwrite da/dc/d39/f61 [0,4194304] 0 2024-07-16T09:55:35.985 INFO:tasks.workunit.client.0.smithi156.stdout:5/359: readlink d1b/d1f/l29 0 2024-07-16T09:55:35.988 INFO:tasks.workunit.client.1.smithi176.stdout:7/426: link d4/d1f/c3c d4/d47/d6d/d89/d7e/d8d/c8f 0 2024-07-16T09:55:35.989 INFO:tasks.workunit.client.0.smithi156.stdout:1/349: link dc/d14/d3e/d45/l4e dc/d14/d3e/d55/d58/l79 0 2024-07-16T09:55:35.990 INFO:tasks.workunit.client.1.smithi176.stdout:4/388: mknod da/dc/d4b/d62/c72 0 2024-07-16T09:55:35.990 INFO:tasks.workunit.client.0.smithi156.stdout:2/440: dwrite d0/d13/d17/d1c/f5e [0,4194304] 0 2024-07-16T09:55:35.991 INFO:tasks.workunit.client.0.smithi156.stdout:2/441: chown d0/d5/d29/d2d/d5b/d62/d74/f8f 13 1 2024-07-16T09:55:35.992 INFO:tasks.workunit.client.0.smithi156.stdout:1/350: rmdir dc/d14/d3e 39 2024-07-16T09:55:35.993 INFO:tasks.workunit.client.1.smithi176.stdout:7/427: dread d4/dd/f23 [0,4194304] 0 2024-07-16T09:55:35.994 INFO:tasks.workunit.client.1.smithi176.stdout:7/428: chown d4/d1f/d22/d80/d17/d1d/l39 7 1 2024-07-16T09:55:35.995 INFO:tasks.workunit.client.1.smithi176.stdout:4/389: rename da/dc/d39 to da/d14/d40/d5d/d73 0 2024-07-16T09:55:35.995 INFO:tasks.workunit.client.1.smithi176.stdout:7/429: fdatasync d4/dd/f1a 0 2024-07-16T09:55:35.996 INFO:tasks.workunit.client.1.smithi176.stdout:4/390: fsync da/dc/f58 0 2024-07-16T09:55:35.997 INFO:tasks.workunit.client.1.smithi176.stdout:7/430: truncate d4/fb 1620023 0 2024-07-16T09:55:35.997 INFO:tasks.workunit.client.0.smithi156.stdout:2/442: rename d0/d5/d29/f80 to d0/d5/d29/d2d/d5b/d5f/d89/f9a 0 2024-07-16T09:55:36.000 INFO:tasks.workunit.client.1.smithi176.stdout:7/431: chown d4/d1f/d22/d80/d17/d1d/c86 17 1 2024-07-16T09:55:36.003 INFO:tasks.workunit.client.1.smithi176.stdout:8/462: truncate d8/d14/d1e/d33/f5b 271858 0 2024-07-16T09:55:36.004 INFO:tasks.workunit.client.1.smithi176.stdout:4/391: dwrite da/d3d/d17/f18 [0,4194304] 0 2024-07-16T09:55:36.004 INFO:tasks.workunit.client.1.smithi176.stdout:8/463: chown d8/d14/d1e/d55/f8a 9518 1 2024-07-16T09:55:36.006 INFO:tasks.workunit.client.1.smithi176.stdout:7/432: link d4/d47/d6d/d89/d66/f68 d4/d47/d5d/f90 0 2024-07-16T09:55:36.007 INFO:tasks.workunit.client.1.smithi176.stdout:7/433: read d4/dd/f23 [3085737,95880] 0 2024-07-16T09:55:36.007 INFO:tasks.workunit.client.1.smithi176.stdout:4/392: rmdir da/d14/d40 39 2024-07-16T09:55:36.012 INFO:tasks.workunit.client.1.smithi176.stdout:7/434: creat d4/d5/f91 x:0 0 0 2024-07-16T09:55:36.012 INFO:tasks.workunit.client.1.smithi176.stdout:1/249: dread d4/db/f1b [0,4194304] 0 2024-07-16T09:55:36.013 INFO:tasks.workunit.client.1.smithi176.stdout:1/250: readlink d4/l13 0 2024-07-16T09:55:36.014 INFO:tasks.workunit.client.1.smithi176.stdout:7/435: write d4/d5/f12 [618529,31888] 0 2024-07-16T09:55:36.015 INFO:tasks.workunit.client.1.smithi176.stdout:4/393: dwrite da/f5f [0,4194304] 0 2024-07-16T09:55:36.015 INFO:tasks.workunit.client.1.smithi176.stdout:1/251: truncate d4/f24 4216660 0 2024-07-16T09:55:36.017 INFO:tasks.workunit.client.1.smithi176.stdout:1/252: chown d4/d8/d12/d4c 8377 1 2024-07-16T09:55:36.018 INFO:tasks.workunit.client.1.smithi176.stdout:1/253: write d4/d8/f26 [564322,40112] 0 2024-07-16T09:55:36.024 INFO:tasks.workunit.client.1.smithi176.stdout:4/394: dread da/d14/d40/d5d/d73/f4f [0,4194304] 0 2024-07-16T09:55:36.024 INFO:tasks.workunit.client.1.smithi176.stdout:7/436: creat d4/dd/d49/d7a/f92 x:0 0 0 2024-07-16T09:55:36.027 INFO:tasks.workunit.client.0.smithi156.stdout:1/351: unlink dc/d24/l64 0 2024-07-16T09:55:36.028 INFO:tasks.workunit.client.1.smithi176.stdout:4/395: mknod da/d3d/d17/d54/c74 0 2024-07-16T09:55:36.029 INFO:tasks.workunit.client.0.smithi156.stdout:2/443: mkdir d0/d9b 0 2024-07-16T09:55:36.030 INFO:tasks.workunit.client.0.smithi156.stdout:1/352: unlink dc/d24/f4f 0 2024-07-16T09:55:36.030 INFO:tasks.workunit.client.1.smithi176.stdout:4/396: creat da/dc/d4b/f75 x:0 0 0 2024-07-16T09:55:36.031 INFO:tasks.workunit.client.0.smithi156.stdout:2/444: write d0/d5/d29/d2d/d5b/d62/f6a [502206,3888] 0 2024-07-16T09:55:36.031 INFO:tasks.workunit.client.1.smithi176.stdout:4/397: fsync da/dc/f55 0 2024-07-16T09:55:36.032 INFO:tasks.workunit.client.1.smithi176.stdout:4/398: fsync da/dc/f58 0 2024-07-16T09:55:36.032 INFO:tasks.workunit.client.0.smithi156.stdout:1/353: chown dc/d14/d3e/d45/f69 3849 1 2024-07-16T09:55:36.033 INFO:tasks.workunit.client.1.smithi176.stdout:4/399: stat da/d3d/d33/f3a 0 2024-07-16T09:55:36.034 INFO:tasks.workunit.client.0.smithi156.stdout:1/354: write dc/d14/d3e/d55/d57/f67 [180781,22925] 0 2024-07-16T09:55:36.035 INFO:tasks.workunit.client.0.smithi156.stdout:2/445: creat d0/d5/d29/d2d/d5b/f9c x:0 0 0 2024-07-16T09:55:36.036 INFO:tasks.workunit.client.1.smithi176.stdout:4/400: mknod da/d3d/d17/d54/d5e/c76 0 2024-07-16T09:55:36.036 INFO:tasks.workunit.client.0.smithi156.stdout:1/355: fdatasync dc/d14/d1e/f35 0 2024-07-16T09:55:36.036 INFO:tasks.workunit.client.0.smithi156.stdout:2/446: truncate d0/f91 1788117 0 2024-07-16T09:55:36.038 INFO:tasks.workunit.client.0.smithi156.stdout:2/447: mknod d0/d5/d8b/c9d 0 2024-07-16T09:55:36.041 INFO:tasks.workunit.client.0.smithi156.stdout:2/448: creat d0/d13/f9e x:0 0 0 2024-07-16T09:55:36.041 INFO:tasks.workunit.client.1.smithi176.stdout:1/254: dwrite d4/d8/d12/d21/d34/f58 [0,4194304] 0 2024-07-16T09:55:36.042 INFO:tasks.workunit.client.1.smithi176.stdout:7/437: dwrite d4/d1f/d22/f3a [0,4194304] 0 2024-07-16T09:55:36.043 INFO:tasks.workunit.client.0.smithi156.stdout:2/449: chown d0/d13/d17/d1c/f47 181 1 2024-07-16T09:55:36.043 INFO:tasks.workunit.client.0.smithi156.stdout:1/356: dwrite dc/d6b/f75 [0,4194304] 0 2024-07-16T09:55:36.043 INFO:tasks.workunit.client.1.smithi176.stdout:1/255: chown d4/d8/d12/d21/d34/f58 3646573 1 2024-07-16T09:55:36.044 INFO:tasks.workunit.client.0.smithi156.stdout:2/450: readlink d0/d13/d6e/l70 0 2024-07-16T09:55:36.048 INFO:tasks.workunit.client.0.smithi156.stdout:2/451: creat d0/d5/d29/d2d/d5b/d5f/d89/f9f x:0 0 0 2024-07-16T09:55:36.049 INFO:tasks.workunit.client.0.smithi156.stdout:2/452: chown d0/d5/cf 7031965 1 2024-07-16T09:55:36.049 INFO:tasks.workunit.client.0.smithi156.stdout:1/357: dread dc/d14/d1e/f35 [0,4194304] 0 2024-07-16T09:55:36.051 INFO:tasks.workunit.client.1.smithi176.stdout:7/438: unlink d4/d47/d6d/d89/d7e/d8d/c8f 0 2024-07-16T09:55:36.052 INFO:tasks.workunit.client.1.smithi176.stdout:1/256: rename d4/f59 to d4/d3a/d54/d5a/f5b 0 2024-07-16T09:55:36.057 INFO:tasks.workunit.client.1.smithi176.stdout:1/257: creat d4/db/f5c x:0 0 0 2024-07-16T09:55:36.057 INFO:tasks.workunit.client.0.smithi156.stdout:2/453: dwrite d0/d5/d29/d2d/d5b/d5f/f7e [0,4194304] 0 2024-07-16T09:55:36.060 INFO:tasks.workunit.client.0.smithi156.stdout:2/454: truncate d0/d5/d29/d2d/f73 104905 0 2024-07-16T09:55:36.065 INFO:tasks.workunit.client.0.smithi156.stdout:2/455: link d0/d13/f9e d0/d5/d29/d2d/d5b/d62/fa0 0 2024-07-16T09:55:36.068 INFO:tasks.workunit.client.0.smithi156.stdout:2/456: mkdir d0/d5/d76/da1 0 2024-07-16T09:55:36.071 INFO:tasks.workunit.client.1.smithi176.stdout:4/401: dread da/d14/f26 [4194304,4194304] 0 2024-07-16T09:55:36.072 INFO:tasks.workunit.client.0.smithi156.stdout:2/457: rename d0/d5/d29/f54 to d0/d13/d6e/fa2 0 2024-07-16T09:55:36.073 INFO:tasks.workunit.client.1.smithi176.stdout:4/402: unlink da/dc/d4b/f75 0 2024-07-16T09:55:36.078 INFO:tasks.workunit.client.0.smithi156.stdout:1/358: dread dc/d24/f2a [0,4194304] 0 2024-07-16T09:55:36.079 INFO:tasks.workunit.client.1.smithi176.stdout:4/403: dwrite da/dc/d10/f27 [0,4194304] 0 2024-07-16T09:55:36.080 INFO:tasks.workunit.client.0.smithi156.stdout:1/359: dread dc/d14/d1e/f41 [0,4194304] 0 2024-07-16T09:55:36.083 INFO:tasks.workunit.client.0.smithi156.stdout:1/360: mkdir dc/d14/d3e/d6c/d7a 0 2024-07-16T09:55:36.087 INFO:tasks.workunit.client.0.smithi156.stdout:2/458: truncate d0/d13/f59 1387211 0 2024-07-16T09:55:36.089 INFO:tasks.workunit.client.1.smithi176.stdout:7/439: dwrite d4/d47/f88 [0,4194304] 0 2024-07-16T09:55:36.090 INFO:tasks.workunit.client.0.smithi156.stdout:1/361: dwrite dc/d14/d1e/f35 [0,4194304] 0 2024-07-16T09:55:36.090 INFO:tasks.workunit.client.0.smithi156.stdout:2/459: chown d0/d5/d29/d2d/d5b/d5f/d89/f9f 45 1 2024-07-16T09:55:36.090 INFO:tasks.workunit.client.1.smithi176.stdout:1/258: dwrite d4/d8/f29 [0,4194304] 0 2024-07-16T09:55:36.090 INFO:tasks.workunit.client.1.smithi176.stdout:7/440: readlink d4/d1f/d22/l42 0 2024-07-16T09:55:36.095 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:55:35 smithi176 ceph-mon[36626]: pgmap v249: 65 pgs: 65 active+clean; 1.1 GiB data, 4.6 GiB used, 532 GiB / 536 GiB avail; 13 MiB/s rd, 141 MiB/s wr, 396 op/s 2024-07-16T09:55:36.095 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:55:35 smithi176 ceph-mon[36626]: from='client.? 172.21.15.156:0/601728330' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-07-16T09:55:36.095 INFO:tasks.workunit.client.1.smithi176.stdout:6/361: sync 2024-07-16T09:55:36.095 INFO:tasks.workunit.client.1.smithi176.stdout:5/419: sync 2024-07-16T09:55:36.098 INFO:tasks.workunit.client.0.smithi156.stdout:1/362: dread dc/d14/d1e/f29 [0,4194304] 0 2024-07-16T09:55:36.100 INFO:tasks.workunit.client.1.smithi176.stdout:6/362: symlink d1/dc/d40/l7f 0 2024-07-16T09:55:36.103 INFO:tasks.workunit.client.0.smithi156.stdout:2/460: dwrite d0/f91 [0,4194304] 0 2024-07-16T09:55:36.103 INFO:tasks.workunit.client.1.smithi176.stdout:1/259: rename d4/d8/d12/d21/d25/d30/l45 to d4/d8/l5d 0 2024-07-16T09:55:36.104 INFO:tasks.workunit.client.0.smithi156.stdout:1/363: mknod dc/d20/c7b 0 2024-07-16T09:55:36.105 INFO:tasks.workunit.client.0.smithi156.stdout:2/461: read - d0/d5/d8b/f97 zero size 2024-07-16T09:55:36.106 INFO:tasks.workunit.client.1.smithi176.stdout:6/363: mkdir d1/d5/d47/d62/d80 0 2024-07-16T09:55:36.106 INFO:tasks.workunit.client.1.smithi176.stdout:1/260: rmdir d4/d3a 39 2024-07-16T09:55:36.106 INFO:tasks.workunit.client.1.smithi176.stdout:5/420: dwrite d4/f6 [0,4194304] 0 2024-07-16T09:55:36.108 INFO:tasks.workunit.client.1.smithi176.stdout:5/421: readlink d4/d14/d19/l91 0 2024-07-16T09:55:36.109 INFO:tasks.workunit.client.1.smithi176.stdout:5/422: fdatasync d4/d14/f2c 0 2024-07-16T09:55:36.109 INFO:tasks.workunit.client.0.smithi156.stdout:2/462: symlink d0/d13/d17/d1c/d86/la3 0 2024-07-16T09:55:36.110 INFO:tasks.workunit.client.0.smithi156.stdout:1/364: mknod dc/d14/c7c 0 2024-07-16T09:55:36.111 INFO:tasks.workunit.client.1.smithi176.stdout:5/423: dread - d4/d10/d17/f8a zero size 2024-07-16T09:55:36.111 INFO:tasks.workunit.client.0.smithi156.stdout:1/365: dread - dc/d14/d3e/d45/f69 zero size 2024-07-16T09:55:36.111 INFO:tasks.workunit.client.0.smithi156.stdout:1/366: fsync dc/f28 0 2024-07-16T09:55:36.113 INFO:tasks.workunit.client.1.smithi176.stdout:1/261: rmdir d4/d8/d12/d4c 39 2024-07-16T09:55:36.113 INFO:tasks.workunit.client.1.smithi176.stdout:4/404: dread da/d14/f26 [0,4194304] 0 2024-07-16T09:55:36.114 INFO:tasks.workunit.client.0.smithi156.stdout:2/463: symlink d0/d5/d76/da1/la4 0 2024-07-16T09:55:36.116 INFO:tasks.workunit.client.1.smithi176.stdout:6/364: dread d1/d5/d47/f51 [4194304,4194304] 0 2024-07-16T09:55:36.117 INFO:tasks.workunit.client.0.smithi156.stdout:2/464: mknod d0/d5/d76/ca5 0 2024-07-16T09:55:36.119 INFO:tasks.workunit.client.1.smithi176.stdout:4/405: mknod da/d14/d40/d5d/d73/c77 0 2024-07-16T09:55:36.121 INFO:tasks.workunit.client.0.smithi156.stdout:1/367: rename dc/d14/d1e/l54 to dc/d14/l7d 0 2024-07-16T09:55:36.121 INFO:tasks.workunit.client.1.smithi176.stdout:6/365: mkdir d1/dc/d81 0 2024-07-16T09:55:36.123 INFO:tasks.workunit.client.1.smithi176.stdout:4/406: mkdir da/d3d/d17/d54/d78 0 2024-07-16T09:55:36.124 INFO:tasks.workunit.client.0.smithi156.stdout:2/465: symlink d0/d5/d29/la6 0 2024-07-16T09:55:36.125 INFO:tasks.workunit.client.1.smithi176.stdout:7/441: dread d4/d47/d6d/d89/f50 [0,4194304] 0 2024-07-16T09:55:36.125 INFO:tasks.workunit.client.0.smithi156.stdout:2/466: write d0/d5/d29/f3f [754896,12851] 0 2024-07-16T09:55:36.126 INFO:tasks.workunit.client.0.smithi156.stdout:1/368: creat dc/d24/f7e x:0 0 0 2024-07-16T09:55:36.127 INFO:tasks.workunit.client.0.smithi156.stdout:2/467: chown d0/d13/d6e/fa2 4214 1 2024-07-16T09:55:36.129 INFO:tasks.workunit.client.1.smithi176.stdout:7/442: mknod d4/d1f/d22/d80/d17/c93 0 2024-07-16T09:55:36.130 INFO:tasks.workunit.client.1.smithi176.stdout:7/443: write d4/f27 [1456876,120171] 0 2024-07-16T09:55:36.132 INFO:tasks.workunit.client.1.smithi176.stdout:7/444: symlink d4/d1f/d22/d80/d17/d1d/d8b/l94 0 2024-07-16T09:55:36.139 INFO:tasks.workunit.client.1.smithi176.stdout:5/424: truncate d4/d10/f22 886035 0 2024-07-16T09:55:36.139 INFO:tasks.workunit.client.1.smithi176.stdout:4/407: rmdir da/d3d/d17 39 2024-07-16T09:55:36.140 INFO:tasks.workunit.client.1.smithi176.stdout:4/408: stat da/fb 0 2024-07-16T09:55:36.141 INFO:tasks.workunit.client.1.smithi176.stdout:5/425: mkdir d4/d9a 0 2024-07-16T09:55:36.142 INFO:tasks.workunit.client.1.smithi176.stdout:7/445: truncate d4/d1f/d22/d51/f58 120282 0 2024-07-16T09:55:36.143 INFO:tasks.workunit.client.1.smithi176.stdout:4/409: creat da/dc/d10/d28/f79 x:0 0 0 2024-07-16T09:55:36.144 INFO:tasks.workunit.client.1.smithi176.stdout:5/426: rmdir d4/d14/d19 39 2024-07-16T09:55:36.144 INFO:tasks.workunit.client.0.smithi156.stdout:2/468: fdatasync d0/d13/d17/d1c/f1f 0 2024-07-16T09:55:36.145 INFO:tasks.workunit.client.0.smithi156.stdout:1/369: truncate dc/d6b/f75 668442 0 2024-07-16T09:55:36.146 INFO:tasks.workunit.client.1.smithi176.stdout:4/410: read da/dc/f55 [3350570,56115] 0 2024-07-16T09:55:36.146 INFO:tasks.workunit.client.1.smithi176.stdout:6/366: dwrite d1/f45 [0,4194304] 0 2024-07-16T09:55:36.146 INFO:tasks.workunit.client.0.smithi156.stdout:7/258: sync 2024-07-16T09:55:36.147 INFO:tasks.workunit.client.0.smithi156.stdout:0/225: sync 2024-07-16T09:55:36.147 INFO:tasks.workunit.client.0.smithi156.stdout:1/370: fsync dc/f33 0 2024-07-16T09:55:36.148 INFO:tasks.workunit.client.0.smithi156.stdout:7/259: rmdir dc/d13/d20 39 2024-07-16T09:55:36.151 INFO:tasks.workunit.client.0.smithi156.stdout:0/226: fsync d0/d7/d9/f12 0 2024-07-16T09:55:36.152 INFO:tasks.workunit.client.0.smithi156.stdout:1/371: symlink dc/d14/d3e/d55/d57/l7f 0 2024-07-16T09:55:36.153 INFO:tasks.workunit.client.1.smithi176.stdout:6/367: creat d1/d61/d65/f82 x:0 0 0 2024-07-16T09:55:36.153 INFO:tasks.workunit.client.0.smithi156.stdout:1/372: chown dc/d14/f34 570346 1 2024-07-16T09:55:36.154 INFO:tasks.workunit.client.0.smithi156.stdout:7/260: creat dc/d13/d36/d56/f61 x:0 0 0 2024-07-16T09:55:36.154 INFO:tasks.workunit.client.1.smithi176.stdout:5/427: dread d4/d37/d72/d38/d6e/f74 [0,4194304] 0 2024-07-16T09:55:36.154 INFO:tasks.workunit.client.1.smithi176.stdout:4/411: dread da/d3d/f1b [0,4194304] 0 2024-07-16T09:55:36.155 INFO:tasks.workunit.client.0.smithi156.stdout:0/227: mkdir d0/d22/d41 0 2024-07-16T09:55:36.156 INFO:tasks.workunit.client.0.smithi156.stdout:0/228: chown d0/ff 1105 1 2024-07-16T09:55:36.157 INFO:tasks.workunit.client.1.smithi176.stdout:6/368: creat d1/d5/d47/f83 x:0 0 0 2024-07-16T09:55:36.159 INFO:tasks.workunit.client.0.smithi156.stdout:7/261: symlink dc/d13/d36/l62 0 2024-07-16T09:55:36.160 INFO:tasks.workunit.client.0.smithi156.stdout:7/262: readlink dc/d13/l17 0 2024-07-16T09:55:36.160 INFO:tasks.workunit.client.1.smithi176.stdout:5/428: creat d4/d37/d72/d38/d39/f9b x:0 0 0 2024-07-16T09:55:36.162 INFO:tasks.workunit.client.1.smithi176.stdout:6/369: mknod d1/d5/d7/d1e/d67/c84 0 2024-07-16T09:55:36.163 INFO:tasks.workunit.client.1.smithi176.stdout:4/412: mkdir da/d3d/d17/d54/d78/d7a 0 2024-07-16T09:55:36.164 INFO:tasks.workunit.client.0.smithi156.stdout:7/263: chown dc/d13/d20/d4d/c4e 327191173 1 2024-07-16T09:55:36.164 INFO:tasks.workunit.client.1.smithi176.stdout:4/413: stat da/d3d/d17/d54/d5e 0 2024-07-16T09:55:36.165 INFO:tasks.workunit.client.0.smithi156.stdout:1/373: write dc/f19 [3540398,102668] 0 2024-07-16T09:55:36.166 INFO:tasks.workunit.client.1.smithi176.stdout:5/429: creat d4/d14/f9c x:0 0 0 2024-07-16T09:55:36.167 INFO:tasks.workunit.client.1.smithi176.stdout:4/414: creat da/d3d/d17/d54/d78/f7b x:0 0 0 2024-07-16T09:55:36.171 INFO:tasks.workunit.client.1.smithi176.stdout:4/415: symlink da/d3d/d17/d54/d5e/l7c 0 2024-07-16T09:55:36.172 INFO:tasks.workunit.client.1.smithi176.stdout:6/370: dwrite d1/d5/d7/f63 [0,4194304] 0 2024-07-16T09:55:36.174 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:55:35 smithi156 ceph-mon[29302]: pgmap v249: 65 pgs: 65 active+clean; 1.1 GiB data, 4.6 GiB used, 532 GiB / 536 GiB avail; 13 MiB/s rd, 141 MiB/s wr, 396 op/s 2024-07-16T09:55:36.174 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:55:35 smithi156 ceph-mon[29302]: from='client.? 172.21.15.156:0/601728330' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-07-16T09:55:36.175 INFO:tasks.workunit.client.1.smithi176.stdout:5/430: dwrite d4/d37/d72/d38/d39/f5d [0,4194304] 0 2024-07-16T09:55:36.178 INFO:tasks.workunit.client.0.smithi156.stdout:0/229: dwrite d0/d7/d9/d27/d3d/f37 [0,4194304] 0 2024-07-16T09:55:36.178 INFO:tasks.workunit.client.0.smithi156.stdout:1/374: dwrite dc/d14/d3e/f43 [4194304,4194304] 0 2024-07-16T09:55:36.183 INFO:tasks.workunit.client.1.smithi176.stdout:4/416: getdents da/d3d/d17/d54/d78/d7a 0 2024-07-16T09:55:36.183 INFO:tasks.workunit.client.0.smithi156.stdout:7/264: dwrite dc/d13/d20/f2f [4194304,4194304] 0 2024-07-16T09:55:36.186 INFO:tasks.workunit.client.0.smithi156.stdout:1/375: creat dc/d6b/f80 x:0 0 0 2024-07-16T09:55:36.189 INFO:tasks.workunit.client.1.smithi176.stdout:6/371: dwrite d1/f31 [0,4194304] 0 2024-07-16T09:55:36.190 INFO:tasks.workunit.client.0.smithi156.stdout:0/230: rename d0/d7/d9/d27/d2c/f32 to d0/d1a/f42 0 2024-07-16T09:55:36.191 INFO:tasks.workunit.client.1.smithi176.stdout:5/431: rename d4/d14/d19/l57 to d4/d37/l9d 0 2024-07-16T09:55:36.191 INFO:tasks.workunit.client.1.smithi176.stdout:6/372: chown d1/l5f 62 1 2024-07-16T09:55:36.191 INFO:tasks.workunit.client.0.smithi156.stdout:7/265: read dc/d3e/f4a [2220274,60015] 0 2024-07-16T09:55:36.193 INFO:tasks.workunit.client.0.smithi156.stdout:1/376: dread dc/d20/f36 [4194304,4194304] 0 2024-07-16T09:55:36.194 INFO:tasks.workunit.client.1.smithi176.stdout:6/373: write d1/d5/d7/d1e/f66 [773704,116365] 0 2024-07-16T09:55:36.197 INFO:tasks.workunit.client.1.smithi176.stdout:6/374: stat d1/dc/d5b/f6d 0 2024-07-16T09:55:36.199 INFO:tasks.workunit.client.0.smithi156.stdout:7/266: unlink dc/d13/d20/f2f 0 2024-07-16T09:55:36.202 INFO:tasks.workunit.client.0.smithi156.stdout:0/231: creat d0/f43 x:0 0 0 2024-07-16T09:55:36.203 INFO:tasks.workunit.client.1.smithi176.stdout:5/432: dwrite d4/d37/d72/d82/f90 [0,4194304] 0 2024-07-16T09:55:36.203 INFO:tasks.workunit.client.0.smithi156.stdout:0/232: write d0/d7/d9/f15 [8819529,122564] 0 2024-07-16T09:55:36.206 INFO:tasks.workunit.client.1.smithi176.stdout:6/375: creat d1/d5/d7/d2f/f85 x:0 0 0 2024-07-16T09:55:36.208 INFO:tasks.workunit.client.1.smithi176.stdout:6/376: creat d1/d5/d47/f86 x:0 0 0 2024-07-16T09:55:36.209 INFO:tasks.workunit.client.0.smithi156.stdout:0/233: getdents d0/d7/d25 0 2024-07-16T09:55:36.211 INFO:tasks.workunit.client.1.smithi176.stdout:6/377: mknod d1/d5/d47/d62/c87 0 2024-07-16T09:55:36.213 INFO:tasks.workunit.client.0.smithi156.stdout:0/234: truncate d0/d7/d25/f2a 159498 0 2024-07-16T09:55:36.214 INFO:tasks.workunit.client.1.smithi176.stdout:6/378: mknod d1/d5/d47/d75/c88 0 2024-07-16T09:55:36.215 INFO:tasks.workunit.client.1.smithi176.stdout:6/379: write d1/d5/d7/d1e/f66 [1639167,131063] 0 2024-07-16T09:55:36.216 INFO:tasks.workunit.client.0.smithi156.stdout:0/235: rmdir d0/d3e 0 2024-07-16T09:55:36.218 INFO:tasks.workunit.client.1.smithi176.stdout:6/380: symlink d1/d5/d7/d1e/d67/l89 0 2024-07-16T09:55:36.219 INFO:tasks.workunit.client.1.smithi176.stdout:6/381: symlink d1/d5d/l8a 0 2024-07-16T09:55:36.220 INFO:tasks.workunit.client.1.smithi176.stdout:6/382: write d1/d5d/f53 [548747,126196] 0 2024-07-16T09:55:36.222 INFO:tasks.workunit.client.1.smithi176.stdout:6/383: creat d1/dc/d40/f8b x:0 0 0 2024-07-16T09:55:36.223 INFO:tasks.workunit.client.0.smithi156.stdout:0/236: dwrite d0/f43 [0,4194304] 0 2024-07-16T09:55:36.224 INFO:tasks.workunit.client.1.smithi176.stdout:6/384: rmdir d1/d5/d7 39 2024-07-16T09:55:36.226 INFO:tasks.workunit.client.1.smithi176.stdout:6/385: mknod d1/dc/d40/c8c 0 2024-07-16T09:55:36.227 INFO:tasks.workunit.client.0.smithi156.stdout:0/237: unlink d0/d7/d9/d27/d3d/f40 0 2024-07-16T09:55:36.227 INFO:tasks.workunit.client.1.smithi176.stdout:6/386: chown d1/f64 7 1 2024-07-16T09:55:36.228 INFO:tasks.workunit.client.1.smithi176.stdout:4/417: truncate da/d3d/d17/f31 2223576 0 2024-07-16T09:55:36.229 INFO:tasks.workunit.client.1.smithi176.stdout:4/418: fdatasync da/dc/f50 0 2024-07-16T09:55:36.230 INFO:tasks.workunit.client.0.smithi156.stdout:0/238: unlink d0/d7/f2e 0 2024-07-16T09:55:36.232 INFO:tasks.workunit.client.1.smithi176.stdout:4/419: readlink da/d3d/d17/d54/l66 0 2024-07-16T09:55:36.233 INFO:tasks.workunit.client.1.smithi176.stdout:6/387: creat d1/d5/d7/d1e/d67/f8d x:0 0 0 2024-07-16T09:55:36.234 INFO:tasks.workunit.client.0.smithi156.stdout:0/239: link d0/d7/d9/d27/c28 d0/d22/c44 0 2024-07-16T09:55:36.235 INFO:tasks.workunit.client.0.smithi156.stdout:0/240: dread - d0/d22/f3c zero size 2024-07-16T09:55:36.236 INFO:tasks.workunit.client.1.smithi176.stdout:6/388: mkdir d1/d5/d7/d2f/d8e 0 2024-07-16T09:55:36.237 INFO:tasks.workunit.client.0.smithi156.stdout:0/241: mknod d0/d7/d9/d27/d3d/c45 0 2024-07-16T09:55:36.239 INFO:tasks.workunit.client.0.smithi156.stdout:0/242: truncate d0/d22/f3c 16611 0 2024-07-16T09:55:36.241 INFO:tasks.workunit.client.0.smithi156.stdout:0/243: truncate d0/d7/d25/f3a 530943 0 2024-07-16T09:55:36.241 INFO:tasks.workunit.client.0.smithi156.stdout:0/244: chown d0/d7/f11 640456 1 2024-07-16T09:55:36.242 INFO:tasks.workunit.client.1.smithi176.stdout:4/420: dwrite da/d14/d40/d5d/d73/f61 [0,4194304] 0 2024-07-16T09:55:36.242 INFO:tasks.workunit.client.0.smithi156.stdout:7/267: dread dc/f42 [0,4194304] 0 2024-07-16T09:55:36.244 INFO:tasks.workunit.client.0.smithi156.stdout:0/245: creat d0/d1a/f46 x:0 0 0 2024-07-16T09:55:36.245 INFO:tasks.workunit.client.0.smithi156.stdout:7/268: mkdir dc/d63 0 2024-07-16T09:55:36.246 INFO:tasks.workunit.client.0.smithi156.stdout:7/269: fsync dc/f14 0 2024-07-16T09:55:36.247 INFO:tasks.workunit.client.0.smithi156.stdout:0/246: creat d0/d22/d41/f47 x:0 0 0 2024-07-16T09:55:36.249 INFO:tasks.workunit.client.0.smithi156.stdout:7/270: rmdir dc/d48 39 2024-07-16T09:55:36.249 INFO:tasks.workunit.client.0.smithi156.stdout:7/271: chown dc/d44 374 1 2024-07-16T09:55:36.252 INFO:tasks.workunit.client.0.smithi156.stdout:1/377: write dc/d24/f2a [868448,33140] 0 2024-07-16T09:55:36.253 INFO:tasks.workunit.client.0.smithi156.stdout:7/272: symlink dc/d63/l64 0 2024-07-16T09:55:36.257 INFO:tasks.workunit.client.0.smithi156.stdout:1/378: chown dc/d14/d1e/f29 1 1 2024-07-16T09:55:36.257 INFO:tasks.workunit.client.1.smithi176.stdout:6/389: rename d1/d5/d7/d1e/d67 to d1/d61/d65/d8f 0 2024-07-16T09:55:36.264 INFO:tasks.workunit.client.0.smithi156.stdout:1/379: dwrite dc/d14/d1e/f3d [0,4194304] 0 2024-07-16T09:55:36.265 INFO:tasks.workunit.client.1.smithi176.stdout:6/390: dwrite d1/d61/d65/d8f/f74 [0,4194304] 0 2024-07-16T09:55:36.269 INFO:tasks.workunit.client.0.smithi156.stdout:0/247: dread d0/d7/d9/f26 [0,4194304] 0 2024-07-16T09:55:36.271 INFO:tasks.workunit.client.1.smithi176.stdout:6/391: creat d1/dc/d72/d6f/d7c/f90 x:0 0 0 2024-07-16T09:55:36.272 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:36.270+0000 7fdf7d2ae640 1 -- 172.21.15.156:0/3729096851 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fdf7804b530 con 0x7fdf780fc510 2024-07-16T09:55:36.272 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:36.271+0000 7fdf5f7fe640 1 -- 172.21.15.156:0/3729096851 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7fdf6c059280 con 0x7fdf780fc510 2024-07-16T09:55:36.272 INFO:teuthology.orchestra.run.smithi156.stdout:HEALTH_OK 2024-07-16T09:55:36.274 INFO:tasks.workunit.client.1.smithi176.stdout:6/392: write d1/dc/d40/f4d [5899846,15691] 0 2024-07-16T09:55:36.274 INFO:tasks.workunit.client.0.smithi156.stdout:1/380: getdents dc/d14/d1e 0 2024-07-16T09:55:36.275 INFO:tasks.workunit.client.0.smithi156.stdout:1/381: readlink dc/d24/l50 0 2024-07-16T09:55:36.276 INFO:tasks.workunit.client.1.smithi176.stdout:4/421: dwrite da/dc/fe [0,4194304] 0 2024-07-16T09:55:36.276 INFO:tasks.workunit.client.0.smithi156.stdout:0/248: dread d0/d7/d9/d27/d3d/f37 [0,4194304] 0 2024-07-16T09:55:36.277 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:36.275+0000 7fdf7d2ae640 1 -- 172.21.15.156:0/3729096851 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7fdf60072db0 msgr2=0x7fdf60075260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:55:36.277 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:36.275+0000 7fdf7d2ae640 1 --2- 172.21.15.156:0/3729096851 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7fdf60072db0 0x7fdf60075260 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fdf68009f80 tx=0x7fdf68035040 comp rx=0 tx=0).stop 2024-07-16T09:55:36.277 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:36.275+0000 7fdf7d2ae640 1 -- 172.21.15.156:0/3729096851 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdf780fc510 msgr2=0x7fdf7810c440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:55:36.277 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:36.275+0000 7fdf7d2ae640 1 --2- 172.21.15.156:0/3729096851 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdf780fc510 0x7fdf7810c440 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fdf6c004e80 tx=0x7fdf6c00b040 comp rx=0 tx=0).stop 2024-07-16T09:55:36.277 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:36.275+0000 7fdf7d2ae640 1 -- 172.21.15.156:0/3729096851 shutdown_connections 2024-07-16T09:55:36.277 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:36.275+0000 7fdf7d2ae640 1 --2- 172.21.15.156:0/3729096851 >> [v2:172.21.15.156:6800/2615741717,v1:172.21.15.156:6801/2615741717] conn(0x7fdf60072db0 0x7fdf60075260 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:55:36.277 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:36.275+0000 7fdf7d2ae640 1 --2- 172.21.15.156:0/3729096851 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdf780fc510 0x7fdf7810c440 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:55:36.277 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:36.275+0000 7fdf7d2ae640 1 --2- 172.21.15.156:0/3729096851 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fdf780fb240 0x7fdf7810bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:55:36.277 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:36.275+0000 7fdf7d2ae640 1 -- 172.21.15.156:0/3729096851 >> 172.21.15.156:0/3729096851 conn(0x7fdf780f6a20 msgr2=0x7fdf780f8110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:55:36.277 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:36.275+0000 7fdf7d2ae640 1 -- 172.21.15.156:0/3729096851 shutdown_connections 2024-07-16T09:55:36.278 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:55:36.275+0000 7fdf7d2ae640 1 -- 172.21.15.156:0/3729096851 wait complete. 2024-07-16T09:55:36.278 INFO:tasks.workunit.client.0.smithi156.stdout:0/249: chown d0/d7/d25/l30 594771 1 2024-07-16T09:55:36.278 INFO:tasks.workunit.client.1.smithi176.stdout:6/393: symlink d1/dc/d5b/l91 0 2024-07-16T09:55:36.278 INFO:tasks.workunit.client.1.smithi176.stdout:6/394: stat d1/d5/d47/f86 0 2024-07-16T09:55:36.279 INFO:tasks.workunit.client.1.smithi176.stdout:6/395: write d1/d61/d65/f82 [430304,66386] 0 2024-07-16T09:55:36.280 INFO:tasks.workunit.client.0.smithi156.stdout:1/382: symlink dc/d14/d3e/d55/d58/l81 0 2024-07-16T09:55:36.283 INFO:tasks.workunit.client.0.smithi156.stdout:7/273: dread dc/d13/d20/f2b [4194304,4194304] 0 2024-07-16T09:55:36.285 INFO:tasks.workunit.client.0.smithi156.stdout:0/250: getdents d0 0 2024-07-16T09:55:36.287 INFO:tasks.workunit.client.0.smithi156.stdout:0/251: fdatasync d0/f36 0 2024-07-16T09:55:36.287 INFO:tasks.workunit.client.1.smithi176.stdout:4/422: truncate da/d14/d40/d5d/d73/f61 2623284 0 2024-07-16T09:55:36.288 INFO:tasks.workunit.client.0.smithi156.stdout:0/252: stat d0/f14 0 2024-07-16T09:55:36.290 INFO:tasks.workunit.client.0.smithi156.stdout:7/274: getdents dc/d3e/d3f/d51 0 2024-07-16T09:55:36.291 INFO:tasks.workunit.client.0.smithi156.stdout:1/383: dwrite dc/d14/d1e/f21 [0,4194304] 0 2024-07-16T09:55:36.291 INFO:tasks.workunit.client.0.smithi156.stdout:0/253: mkdir d0/d22/d41/d48 0 2024-07-16T09:55:36.292 INFO:tasks.workunit.client.0.smithi156.stdout:0/254: write d0/d7/d9/f12 [309059,91533] 0 2024-07-16T09:55:36.295 INFO:tasks.workunit.client.0.smithi156.stdout:1/384: unlink dc/d14/d1e/l3b 0 2024-07-16T09:55:36.296 INFO:tasks.workunit.client.0.smithi156.stdout:7/275: rename l9 to dc/d3e/d5d/l65 0 2024-07-16T09:55:36.297 INFO:tasks.workunit.client.0.smithi156.stdout:0/255: mknod d0/d22/d41/d48/c49 0 2024-07-16T09:55:36.297 INFO:tasks.workunit.client.0.smithi156.stdout:6/342: sync 2024-07-16T09:55:36.297 INFO:tasks.workunit.client.0.smithi156.stdout:4/265: sync 2024-07-16T09:55:36.297 INFO:tasks.workunit.client.0.smithi156.stdout:3/268: sync 2024-07-16T09:55:36.299 INFO:tasks.workunit.client.0.smithi156.stdout:1/385: fsync dc/d20/f36 0 2024-07-16T09:55:36.300 INFO:tasks.workunit.client.0.smithi156.stdout:7/276: creat dc/d13/d36/f66 x:0 0 0 2024-07-16T09:55:36.301 INFO:tasks.workunit.client.0.smithi156.stdout:0/256: unlink d0/d7/c17 0 2024-07-16T09:55:36.303 INFO:tasks.workunit.client.0.smithi156.stdout:7/277: dread - dc/d3e/d3f/d45/f4f zero size 2024-07-16T09:55:36.303 INFO:tasks.workunit.client.0.smithi156.stdout:4/266: mknod d4/db/c5d 0 2024-07-16T09:55:36.303 INFO:tasks.workunit.client.0.smithi156.stdout:0/257: chown d0/d7/f11 61963785 1 2024-07-16T09:55:36.304 INFO:tasks.workunit.client.0.smithi156.stdout:1/386: dread dc/d14/d3e/d45/f53 [0,4194304] 0 2024-07-16T09:55:36.307 INFO:tasks.workunit.client.0.smithi156.stdout:4/267: write d4/db/dc/f55 [633222,94131] 0 2024-07-16T09:55:36.307 INFO:tasks.workunit.client.0.smithi156.stdout:3/269: dwrite d8/f10 [0,4194304] 0 2024-07-16T09:55:36.309 INFO:tasks.workunit.client.0.smithi156.stdout:6/343: dwrite d3/d13/d17/d4f/d4d/d3b/f47 [0,4194304] 0 2024-07-16T09:55:36.311 INFO:tasks.workunit.client.0.smithi156.stdout:7/278: rename dc/d13/l1f to dc/d48/l67 0 2024-07-16T09:55:36.313 INFO:tasks.workunit.client.0.smithi156.stdout:3/270: creat d8/d14/f59 x:0 0 0 2024-07-16T09:55:36.314 INFO:tasks.workunit.client.0.smithi156.stdout:4/268: creat d4/db/d47/d4b/f5e x:0 0 0 2024-07-16T09:55:36.316 INFO:tasks.workunit.client.0.smithi156.stdout:1/387: chown dc/d14/d1e/f29 60676722 1 2024-07-16T09:55:36.318 INFO:tasks.workunit.client.0.smithi156.stdout:3/271: creat d8/da/d28/d43/f5a x:0 0 0 2024-07-16T09:55:36.319 INFO:tasks.workunit.client.0.smithi156.stdout:0/258: link d0/d7/d25/f2a d0/d22/d41/f4a 0 2024-07-16T09:55:36.320 INFO:tasks.workunit.client.0.smithi156.stdout:0/259: chown d0/d7/d25/l30 3970 1 2024-07-16T09:55:36.321 INFO:tasks.workunit.client.0.smithi156.stdout:1/388: creat dc/d14/d3e/d55/d57/f82 x:0 0 0 2024-07-16T09:55:36.322 INFO:tasks.workunit.client.0.smithi156.stdout:3/272: rmdir d8/da/d1d 39 2024-07-16T09:55:36.323 INFO:tasks.workunit.client.0.smithi156.stdout:0/260: symlink d0/d7/d9/d27/l4b 0 2024-07-16T09:55:36.324 INFO:tasks.workunit.client.0.smithi156.stdout:4/269: rename d4/db/d44 to d4/db/dc/d5f 0 2024-07-16T09:55:36.325 INFO:tasks.workunit.client.0.smithi156.stdout:3/273: creat d8/da/d28/d43/f5b x:0 0 0 2024-07-16T09:55:36.327 INFO:tasks.workunit.client.0.smithi156.stdout:1/389: mkdir dc/d14/d3e/d55/d83 0 2024-07-16T09:55:36.328 INFO:tasks.workunit.client.0.smithi156.stdout:4/270: rmdir d4/db/d47 39 2024-07-16T09:55:36.333 INFO:tasks.workunit.client.0.smithi156.stdout:4/271: getdents d4/d23/d52 0 2024-07-16T09:55:36.334 INFO:tasks.workunit.client.0.smithi156.stdout:1/390: dwrite dc/d14/f2d [4194304,4194304] 0 2024-07-16T09:55:36.337 INFO:tasks.workunit.client.0.smithi156.stdout:0/261: dread d0/d7/d25/f2a [0,4194304] 0 2024-07-16T09:55:36.338 INFO:tasks.workunit.client.0.smithi156.stdout:4/272: write d4/d23/f53 [6724459,78256] 0 2024-07-16T09:55:36.339 INFO:tasks.workunit.client.0.smithi156.stdout:0/262: write d0/d7/d9/f12 [455090,3074] 0 2024-07-16T09:55:36.339 INFO:tasks.workunit.client.0.smithi156.stdout:0/263: dread - d0/d22/d41/f47 zero size 2024-07-16T09:55:36.341 INFO:tasks.workunit.client.0.smithi156.stdout:1/391: creat dc/d14/f84 x:0 0 0 2024-07-16T09:55:36.341 INFO:tasks.workunit.client.0.smithi156.stdout:1/392: fsync dc/f33 0 2024-07-16T09:55:36.342 INFO:tasks.workunit.client.0.smithi156.stdout:0/264: creat d0/d22/d41/f4c x:0 0 0 2024-07-16T09:55:36.343 INFO:tasks.workunit.client.0.smithi156.stdout:7/279: dread fb [0,4194304] 0 2024-07-16T09:55:36.343 INFO:tasks.workunit.client.0.smithi156.stdout:4/273: rmdir d4/db/d47/d4b 39 2024-07-16T09:55:36.344 INFO:tasks.workunit.client.0.smithi156.stdout:0/265: rmdir d0 39 2024-07-16T09:55:36.345 INFO:tasks.workunit.client.0.smithi156.stdout:7/280: write dc/f14 [479026,54063] 0 2024-07-16T09:55:36.347 INFO:tasks.workunit.client.0.smithi156.stdout:1/393: dread dc/f33 [0,4194304] 0 2024-07-16T09:55:36.348 INFO:tasks.workunit.client.0.smithi156.stdout:7/281: symlink dc/d3e/d3f/d45/l68 0 2024-07-16T09:55:36.350 INFO:tasks.workunit.client.0.smithi156.stdout:0/266: dread d0/d7/d25/f3a [0,4194304] 0 2024-07-16T09:55:36.350 INFO:tasks.workunit.client.0.smithi156.stdout:1/394: chown dc/d20/l72 33720 1 2024-07-16T09:55:36.352 INFO:tasks.workunit.client.0.smithi156.stdout:7/282: rename dc/d3e/f4a to dc/f69 0 2024-07-16T09:55:36.352 INFO:tasks.workunit.client.0.smithi156.stdout:4/274: dwrite d4/db/dc/f4d [0,4194304] 0 2024-07-16T09:55:36.353 INFO:tasks.workunit.client.0.smithi156.stdout:0/267: write d0/d1a/f42 [2104172,8542] 0 2024-07-16T09:55:36.353 INFO:tasks.workunit.client.0.smithi156.stdout:1/395: creat dc/d14/d3e/f85 x:0 0 0 2024-07-16T09:55:36.354 INFO:tasks.workunit.client.0.smithi156.stdout:7/283: creat dc/d13/d36/f6a x:0 0 0 2024-07-16T09:55:36.355 INFO:tasks.workunit.client.0.smithi156.stdout:4/275: write d4/db/f16 [4859316,71805] 0 2024-07-16T09:55:36.356 INFO:tasks.workunit.client.0.smithi156.stdout:0/268: write d0/d7/d25/f38 [348228,39747] 0 2024-07-16T09:55:36.356 INFO:tasks.workunit.client.0.smithi156.stdout:1/396: creat dc/d14/d3e/d45/f86 x:0 0 0 2024-07-16T09:55:36.357 INFO:tasks.workunit.client.0.smithi156.stdout:4/276: fsync d4/f9 0 2024-07-16T09:55:36.359 INFO:tasks.workunit.client.0.smithi156.stdout:1/397: fsync dc/f51 0 2024-07-16T09:55:36.361 INFO:tasks.workunit.client.0.smithi156.stdout:1/398: write dc/d14/f2d [2747453,49199] 0 2024-07-16T09:55:36.362 INFO:tasks.workunit.client.0.smithi156.stdout:4/277: read d4/d23/d30/d33/f50 [2850204,111380] 0 2024-07-16T09:55:36.364 INFO:tasks.workunit.client.0.smithi156.stdout:7/284: dwrite dc/d3e/d3f/d51/f5e [0,4194304] 0 2024-07-16T09:55:36.367 INFO:tasks.workunit.client.0.smithi156.stdout:4/278: chown d4/db/l12 2 1 2024-07-16T09:55:36.368 INFO:tasks.workunit.client.0.smithi156.stdout:7/285: creat dc/d63/f6b x:0 0 0 2024-07-16T09:55:36.371 INFO:tasks.workunit.client.0.smithi156.stdout:4/279: write d4/db/dc/d5f/f5c [718318,127141] 0 2024-07-16T09:55:36.372 INFO:tasks.workunit.client.0.smithi156.stdout:7/286: write fb [3571079,13562] 0 2024-07-16T09:55:36.374 INFO:tasks.workunit.client.0.smithi156.stdout:4/280: write f2 [2865556,53285] 0 2024-07-16T09:55:36.377 INFO:tasks.workunit.client.0.smithi156.stdout:7/287: mkdir dc/d3e/d3f/d51/d6c 0 2024-07-16T09:55:36.377 INFO:tasks.workunit.client.0.smithi156.stdout:1/399: dwrite dc/d14/f4b [0,4194304] 0 2024-07-16T09:55:36.379 INFO:tasks.workunit.client.0.smithi156.stdout:0/269: dwrite d0/d7/d25/f2a [0,4194304] 0 2024-07-16T09:55:36.380 INFO:tasks.workunit.client.0.smithi156.stdout:4/281: symlink d4/db/l60 0 2024-07-16T09:55:36.381 INFO:tasks.workunit.client.0.smithi156.stdout:1/400: dread - dc/d14/d3e/d55/d57/f82 zero size 2024-07-16T09:55:36.381 INFO:tasks.workunit.client.0.smithi156.stdout:0/270: stat d0/d22/d41/f4c 0 2024-07-16T09:55:36.382 INFO:tasks.workunit.client.0.smithi156.stdout:1/401: chown dc/d14/f2d 135 1 2024-07-16T09:55:36.383 INFO:tasks.workunit.client.0.smithi156.stdout:1/402: stat dc/d24/f2a 0 2024-07-16T09:55:36.384 INFO:tasks.workunit.client.0.smithi156.stdout:1/403: write dc/d14/d3e/d55/f5c [1223800,3323] 0 2024-07-16T09:55:36.385 INFO:tasks.workunit.client.0.smithi156.stdout:4/282: getdents d4/d23/d52 0 2024-07-16T09:55:36.391 INFO:tasks.workunit.client.0.smithi156.stdout:1/404: write dc/f10 [42775,27305] 0 2024-07-16T09:55:36.405 INFO:tasks.workunit.client.0.smithi156.stdout:7/288: getdents dc/d3e/d3f/d51 0 2024-07-16T09:55:36.407 INFO:tasks.workunit.client.0.smithi156.stdout:7/289: creat dc/d44/f6d x:0 0 0 2024-07-16T09:55:36.411 INFO:tasks.workunit.client.0.smithi156.stdout:7/290: link l8 dc/d13/d24/l6e 0 2024-07-16T09:55:36.411 INFO:tasks.workunit.client.0.smithi156.stdout:4/283: dwrite d4/db/dc/f2e [4194304,4194304] 0 2024-07-16T09:55:36.412 INFO:tasks.workunit.client.0.smithi156.stdout:4/284: truncate d4/f54 549602 0 2024-07-16T09:55:36.412 INFO:tasks.workunit.client.0.smithi156.stdout:0/271: dread d0/ff [0,4194304] 0 2024-07-16T09:55:36.416 INFO:tasks.workunit.client.0.smithi156.stdout:7/291: dread dc/d13/f43 [0,4194304] 0 2024-07-16T09:55:36.418 INFO:tasks.workunit.client.0.smithi156.stdout:0/272: write d0/d7/d25/f3a [357160,1302] 0 2024-07-16T09:55:36.418 INFO:tasks.workunit.client.0.smithi156.stdout:4/285: dread d4/f5 [0,4194304] 0 2024-07-16T09:55:36.419 INFO:tasks.workunit.client.0.smithi156.stdout:7/292: truncate dc/d13/d36/f4c 868685 0 2024-07-16T09:55:36.419 INFO:tasks.workunit.client.0.smithi156.stdout:0/273: fsync d0/d7/f11 0 2024-07-16T09:55:36.426 INFO:tasks.workunit.client.0.smithi156.stdout:4/286: dread d4/db/dc/f2e [4194304,4194304] 0 2024-07-16T09:55:36.427 INFO:tasks.workunit.client.0.smithi156.stdout:7/293: dwrite dc/d13/d15/f50 [0,4194304] 0 2024-07-16T09:55:36.427 INFO:tasks.workunit.client.0.smithi156.stdout:0/274: dwrite d0/d22/d41/f4a [0,4194304] 0 2024-07-16T09:55:36.431 INFO:tasks.workunit.client.0.smithi156.stdout:7/294: mkdir dc/d48/d6f 0 2024-07-16T09:55:36.431 INFO:tasks.workunit.client.0.smithi156.stdout:4/287: write d4/db/d47/d4b/f5b [137804,2895] 0 2024-07-16T09:55:36.431 INFO:tasks.workunit.client.0.smithi156.stdout:7/295: dread - dc/d44/f6d zero size 2024-07-16T09:55:36.433 INFO:tasks.workunit.client.0.smithi156.stdout:0/275: rename d0/d7 to d0/d1a/d4d 0 2024-07-16T09:55:36.434 INFO:tasks.workunit.client.0.smithi156.stdout:7/296: mknod dc/d3e/d3f/c70 0 2024-07-16T09:55:36.436 INFO:tasks.workunit.client.0.smithi156.stdout:7/297: stat dc/c32 0 2024-07-16T09:55:36.437 INFO:tasks.workunit.client.0.smithi156.stdout:4/288: creat d4/db/f61 x:0 0 0 2024-07-16T09:55:36.438 INFO:tasks.workunit.client.1.smithi176.stdout:3/290: sync 2024-07-16T09:55:36.439 INFO:tasks.workunit.client.0.smithi156.stdout:7/298: dread dc/d13/f12 [0,4194304] 0 2024-07-16T09:55:36.442 INFO:tasks.workunit.client.1.smithi176.stdout:3/291: mknod d3/d17/d1d/d1e/c50 0 2024-07-16T09:55:36.443 INFO:tasks.workunit.client.1.smithi176.stdout:3/292: write d3/d17/d1d/d1e/f3a [1033990,129404] 0 2024-07-16T09:55:36.463 INFO:tasks.workunit.client.1.smithi176.stdout:3/293: dread d3/d17/f26 [0,4194304] 0 2024-07-16T09:55:36.464 INFO:tasks.workunit.client.1.smithi176.stdout:3/294: creat d3/d17/f51 x:0 0 0 2024-07-16T09:55:36.467 INFO:tasks.workunit.client.1.smithi176.stdout:3/295: rename d3/d17/f26 to d3/da/d33/f52 0 2024-07-16T09:55:36.471 INFO:tasks.workunit.client.1.smithi176.stdout:3/296: getdents d3/da/d33 0 2024-07-16T09:55:36.472 INFO:tasks.workunit.client.1.smithi176.stdout:3/297: stat d3/d17/f2f 0 2024-07-16T09:55:36.473 INFO:tasks.workunit.client.0.smithi156.stdout:9/383: sync 2024-07-16T09:55:36.474 INFO:tasks.workunit.client.0.smithi156.stdout:9/384: chown d2/d5/d16/d22/l30 1663591 1 2024-07-16T09:55:36.476 INFO:tasks.workunit.client.1.smithi176.stdout:3/298: link d3/d17/f23 d3/da/d33/f53 0 2024-07-16T09:55:36.476 INFO:tasks.workunit.client.1.smithi176.stdout:3/299: dread - d3/d17/f51 zero size 2024-07-16T09:55:36.476 INFO:tasks.workunit.client.1.smithi176.stdout:3/300: stat d3/da/d33/f52 0 2024-07-16T09:55:36.478 INFO:tasks.workunit.client.1.smithi176.stdout:3/301: rmdir d3/d17/d1d/d1e 39 2024-07-16T09:55:36.479 INFO:tasks.workunit.client.1.smithi176.stdout:3/302: write d3/da/fe [2767000,48628] 0 2024-07-16T09:55:36.481 INFO:tasks.workunit.client.1.smithi176.stdout:3/303: mknod d3/da/c54 0 2024-07-16T09:55:36.485 INFO:tasks.workunit.client.1.smithi176.stdout:3/304: rmdir d3/da 39 2024-07-16T09:55:36.489 INFO:tasks.workunit.client.0.smithi156.stdout:9/385: dread d2/de/d42/f4c [0,4194304] 0 2024-07-16T09:55:36.490 INFO:tasks.workunit.client.0.smithi156.stdout:9/386: write d2/d5/d16/d22/d36/f50 [933157,99396] 0 2024-07-16T09:55:36.491 INFO:tasks.workunit.client.1.smithi176.stdout:3/305: dread d3/d17/f23 [0,4194304] 0 2024-07-16T09:55:36.494 INFO:tasks.workunit.client.1.smithi176.stdout:3/306: dread d3/d17/d1d/d4d/f3f [0,4194304] 0 2024-07-16T09:55:36.499 INFO:tasks.workunit.client.1.smithi176.stdout:3/307: creat d3/f55 x:0 0 0 2024-07-16T09:55:36.505 INFO:tasks.workunit.client.1.smithi176.stdout:3/308: dwrite d3/d17/d1d/d4d/f4e [0,4194304] 0 2024-07-16T09:55:36.509 INFO:tasks.workunit.client.1.smithi176.stdout:3/309: mkdir d3/da/d56 0 2024-07-16T09:55:36.513 INFO:tasks.workunit.client.1.smithi176.stdout:3/310: dread d3/da/f13 [0,4194304] 0 2024-07-16T09:55:36.513 INFO:tasks.workunit.client.1.smithi176.stdout:3/311: chown d3 198536 1 2024-07-16T09:55:36.518 INFO:tasks.workunit.client.1.smithi176.stdout:3/312: dwrite d3/da/d33/f52 [0,4194304] 0 2024-07-16T09:55:36.521 INFO:tasks.workunit.client.0.smithi156.stdout:0/276: fsync d0/d1a/d4d/d9/fe 0 2024-07-16T09:55:36.523 INFO:tasks.workunit.client.1.smithi176.stdout:3/313: link d3/d17/c37 d3/c57 0 2024-07-16T09:55:36.523 INFO:tasks.workunit.client.0.smithi156.stdout:4/289: getdents d4/db/dc/d5f 0 2024-07-16T09:55:36.524 INFO:tasks.workunit.client.1.smithi176.stdout:3/314: fdatasync d3/f25 0 2024-07-16T09:55:36.524 INFO:tasks.workunit.client.0.smithi156.stdout:0/277: fdatasync d0/f43 0 2024-07-16T09:55:36.526 INFO:tasks.workunit.client.1.smithi176.stdout:3/315: creat d3/d17/f58 x:0 0 0 2024-07-16T09:55:36.526 INFO:tasks.workunit.client.0.smithi156.stdout:7/299: write dc/d3e/d3f/d51/f53 [848723,16609] 0 2024-07-16T09:55:36.526 INFO:tasks.workunit.client.0.smithi156.stdout:0/278: chown d0/d1a/f46 65804 1 2024-07-16T09:55:36.527 INFO:tasks.workunit.client.0.smithi156.stdout:4/290: chown d4/d23/d52/c57 264505265 1 2024-07-16T09:55:36.529 INFO:tasks.workunit.client.0.smithi156.stdout:7/300: write dc/f42 [4712018,42444] 0 2024-07-16T09:55:36.530 INFO:tasks.workunit.client.1.smithi176.stdout:3/316: rename d3/d17/d1d/d1e to d3/d17/d1d/d4f/d59 0 2024-07-16T09:55:36.531 INFO:tasks.workunit.client.0.smithi156.stdout:9/387: dwrite d2/d5/f70 [0,4194304] 0 2024-07-16T09:55:36.531 INFO:tasks.workunit.client.0.smithi156.stdout:0/279: write d0/d22/d41/f4a [1599509,18863] 0 2024-07-16T09:55:36.531 INFO:tasks.workunit.client.0.smithi156.stdout:4/291: mkdir d4/db/dc/d62 0 2024-07-16T09:55:36.533 INFO:tasks.workunit.client.1.smithi176.stdout:3/317: stat d3/d17/c37 0 2024-07-16T09:55:36.534 INFO:tasks.workunit.client.0.smithi156.stdout:4/292: readlink d4/d23/l27 0 2024-07-16T09:55:36.536 INFO:tasks.workunit.client.0.smithi156.stdout:0/280: mknod d0/c4e 0 2024-07-16T09:55:36.542 INFO:tasks.workunit.client.0.smithi156.stdout:7/301: dwrite dc/d48/f4b [0,4194304] 0 2024-07-16T09:55:36.542 INFO:tasks.workunit.client.0.smithi156.stdout:0/281: mknod d0/d1a/d4d/d25/c4f 0 2024-07-16T09:55:36.543 INFO:tasks.workunit.client.0.smithi156.stdout:7/302: dread - dc/d44/f6d zero size 2024-07-16T09:55:36.545 INFO:tasks.workunit.client.0.smithi156.stdout:0/282: chown d0/d1a/d4d/d9/d27/l4b 12307375 1 2024-07-16T09:55:36.546 INFO:tasks.workunit.client.0.smithi156.stdout:4/293: dwrite d4/db/f18 [4194304,4194304] 0 2024-07-16T09:55:36.546 INFO:tasks.workunit.client.0.smithi156.stdout:9/388: dread d2/d5/ff [0,4194304] 0 2024-07-16T09:55:36.547 INFO:tasks.workunit.client.0.smithi156.stdout:7/303: symlink dc/d3e/d3f/d45/l71 0 2024-07-16T09:55:36.547 INFO:tasks.workunit.client.0.smithi156.stdout:7/304: stat fb 0 2024-07-16T09:55:36.550 INFO:tasks.workunit.client.0.smithi156.stdout:2/469: sync 2024-07-16T09:55:36.550 INFO:tasks.workunit.client.0.smithi156.stdout:5/360: sync 2024-07-16T09:55:36.550 INFO:tasks.workunit.client.0.smithi156.stdout:8/288: sync 2024-07-16T09:55:36.550 INFO:tasks.workunit.client.1.smithi176.stdout:3/318: rename d3/d17/c37 to d3/d17/d1d/d4f/c5a 0 2024-07-16T09:55:36.552 INFO:tasks.workunit.client.0.smithi156.stdout:9/389: mknod d2/d5/d16/d22/d36/c79 0 2024-07-16T09:55:36.553 INFO:tasks.workunit.client.1.smithi176.stdout:3/319: unlink d3/da/l12 0 2024-07-16T09:55:36.554 INFO:tasks.workunit.client.0.smithi156.stdout:8/289: creat d0/dd/d1f/f6b x:0 0 0 2024-07-16T09:55:36.555 INFO:tasks.workunit.client.0.smithi156.stdout:4/294: rename d4/d41 to d4/db/d47/d63 0 2024-07-16T09:55:36.556 INFO:tasks.workunit.client.1.smithi176.stdout:3/320: creat d3/da/d56/f5b x:0 0 0 2024-07-16T09:55:36.556 INFO:tasks.workunit.client.0.smithi156.stdout:9/390: symlink d2/l7a 0 2024-07-16T09:55:36.557 INFO:tasks.workunit.client.0.smithi156.stdout:5/361: truncate d1b/d1e/f21 583056 0 2024-07-16T09:55:36.558 INFO:tasks.workunit.client.0.smithi156.stdout:7/305: dwrite dc/d13/d36/f4c [0,4194304] 0 2024-07-16T09:55:36.560 INFO:tasks.workunit.client.0.smithi156.stdout:4/295: write d4/d23/f17 [1836288,107799] 0 2024-07-16T09:55:36.560 INFO:tasks.workunit.client.0.smithi156.stdout:0/283: rename d0/d22/d41/d48 to d0/d1a/d4d/d9/d27/d50 0 2024-07-16T09:55:36.561 INFO:tasks.workunit.client.0.smithi156.stdout:8/290: symlink d0/d11/d13/d1c/l6c 0 2024-07-16T09:55:36.563 INFO:tasks.workunit.client.0.smithi156.stdout:2/470: getdents d0 0 2024-07-16T09:55:36.564 INFO:tasks.workunit.client.0.smithi156.stdout:7/306: chown dc/d13/d15/l19 17 1 2024-07-16T09:55:36.565 INFO:tasks.workunit.client.0.smithi156.stdout:7/307: dread - dc/d13/d36/d56/f5a zero size 2024-07-16T09:55:36.566 INFO:tasks.workunit.client.0.smithi156.stdout:7/308: stat dc/d13/c37 0 2024-07-16T09:55:36.567 INFO:tasks.workunit.client.0.smithi156.stdout:7/309: write dc/f42 [1108124,73136] 0 2024-07-16T09:55:36.573 INFO:tasks.workunit.client.0.smithi156.stdout:0/284: dread d0/d1a/f42 [0,4194304] 0 2024-07-16T09:55:36.574 INFO:tasks.workunit.client.0.smithi156.stdout:7/310: dwrite dc/d13/d15/f50 [0,4194304] 0 2024-07-16T09:55:36.576 INFO:tasks.workunit.client.0.smithi156.stdout:7/311: write dc/d3e/d3f/d45/f4f [439060,94720] 0 2024-07-16T09:55:36.578 INFO:tasks.workunit.client.0.smithi156.stdout:9/391: mknod d2/d5/d16/d22/d2b/c7b 0 2024-07-16T09:55:36.582 INFO:tasks.workunit.client.0.smithi156.stdout:0/285: dwrite d0/d1a/d4d/d9/fe [0,4194304] 0 2024-07-16T09:55:36.583 INFO:tasks.workunit.client.0.smithi156.stdout:7/312: dread dc/d13/f43 [0,4194304] 0 2024-07-16T09:55:36.587 INFO:tasks.workunit.client.0.smithi156.stdout:7/313: dread dc/d13/f12 [0,4194304] 0 2024-07-16T09:55:36.593 INFO:tasks.workunit.client.0.smithi156.stdout:7/314: read dc/f14 [2178190,80240] 0 2024-07-16T09:55:36.598 INFO:tasks.workunit.client.0.smithi156.stdout:4/296: mkdir d4/db/dc/d5f/d64 0 2024-07-16T09:55:36.601 INFO:tasks.workunit.client.0.smithi156.stdout:4/297: dread d4/f8 [0,4194304] 0 2024-07-16T09:55:36.607 INFO:tasks.workunit.client.0.smithi156.stdout:0/286: symlink d0/d1a/d4d/d9/d27/d50/l51 0 2024-07-16T09:55:36.608 INFO:tasks.workunit.client.0.smithi156.stdout:0/287: chown d0/d1a/d4d/d9/d27/c35 28 1 2024-07-16T09:55:36.611 INFO:tasks.workunit.client.0.smithi156.stdout:7/315: symlink dc/d63/l72 0 2024-07-16T09:55:36.617 INFO:tasks.workunit.client.0.smithi156.stdout:0/288: mkdir d0/d22/d41/d52 0 2024-07-16T09:55:36.619 INFO:tasks.workunit.client.0.smithi156.stdout:7/316: symlink dc/d63/l73 0 2024-07-16T09:55:36.624 INFO:tasks.workunit.client.0.smithi156.stdout:0/289: dwrite d0/d1a/d4d/d25/f38 [0,4194304] 0 2024-07-16T09:55:36.625 INFO:tasks.workunit.client.0.smithi156.stdout:5/362: getdents d1b/d1f/d45 0 2024-07-16T09:55:36.626 INFO:tasks.workunit.client.0.smithi156.stdout:1/405: sync 2024-07-16T09:55:36.626 INFO:tasks.workunit.client.0.smithi156.stdout:3/274: sync 2024-07-16T09:55:36.626 INFO:tasks.workunit.client.0.smithi156.stdout:6/344: sync 2024-07-16T09:55:36.627 INFO:tasks.workunit.client.0.smithi156.stdout:2/471: link d0/c2b d0/d5/d29/d2d/d5b/ca7 0 2024-07-16T09:55:36.627 INFO:tasks.workunit.client.0.smithi156.stdout:6/345: fdatasync d3/d13/d17/d4f/d1f/d19/d2e/f3e 0 2024-07-16T09:55:36.629 INFO:tasks.workunit.client.0.smithi156.stdout:5/363: mknod d1b/d1f/d54/c7e 0 2024-07-16T09:55:36.630 INFO:tasks.workunit.client.0.smithi156.stdout:3/275: creat d8/da/d13/f5c x:0 0 0 2024-07-16T09:55:36.631 INFO:tasks.workunit.client.0.smithi156.stdout:3/276: fdatasync d8/d14/f30 0 2024-07-16T09:55:36.631 INFO:tasks.workunit.client.0.smithi156.stdout:3/277: chown d8/da/c26 190741 1 2024-07-16T09:55:36.633 INFO:tasks.workunit.client.0.smithi156.stdout:0/290: symlink d0/d1a/d4d/l53 0 2024-07-16T09:55:36.634 INFO:tasks.workunit.client.0.smithi156.stdout:2/472: symlink d0/d5/d76/da1/la8 0 2024-07-16T09:55:36.635 INFO:tasks.workunit.client.0.smithi156.stdout:2/473: write d0/d5/d29/f56 [821806,12177] 0 2024-07-16T09:55:36.635 INFO:tasks.workunit.client.0.smithi156.stdout:1/406: unlink dc/d14/d1e/l1f 0 2024-07-16T09:55:36.636 INFO:tasks.workunit.client.0.smithi156.stdout:2/474: write d0/d5/d29/d2d/d5b/f6d [345987,87103] 0 2024-07-16T09:55:36.636 INFO:tasks.workunit.client.0.smithi156.stdout:1/407: fdatasync dc/d14/d3e/d45/f69 0 2024-07-16T09:55:36.637 INFO:tasks.workunit.client.0.smithi156.stdout:1/408: dread - dc/d24/f7e zero size 2024-07-16T09:55:36.638 INFO:tasks.workunit.client.0.smithi156.stdout:2/475: dread d0/d13/f1a [0,4194304] 0 2024-07-16T09:55:36.643 INFO:tasks.workunit.client.0.smithi156.stdout:6/346: link d3/d13/d17/d4f/d4d/f3f d3/d13/d17/d4f/d5c/f72 0 2024-07-16T09:55:36.646 INFO:tasks.workunit.client.0.smithi156.stdout:2/476: symlink d0/d13/d17/d1c/d51/la9 0 2024-07-16T09:55:36.647 INFO:tasks.workunit.client.0.smithi156.stdout:2/477: write d0/d13/d17/d1c/d86/f94 [747183,118609] 0 2024-07-16T09:55:36.648 INFO:tasks.workunit.client.0.smithi156.stdout:0/291: link d0/d1a/d4d/d9/d27/d3d/f37 d0/d1a/f54 0 2024-07-16T09:55:36.648 INFO:tasks.workunit.client.0.smithi156.stdout:2/478: stat d0/d5/cf 0 2024-07-16T09:55:36.651 INFO:tasks.workunit.client.0.smithi156.stdout:6/347: symlink d3/d13/d17/d4f/d1f/d19/d31/l73 0 2024-07-16T09:55:36.652 INFO:tasks.workunit.client.0.smithi156.stdout:0/292: dread d0/d1a/d4d/d9/fe [0,4194304] 0 2024-07-16T09:55:36.652 INFO:tasks.workunit.client.0.smithi156.stdout:3/278: dread d8/da/d13/f20 [0,4194304] 0 2024-07-16T09:55:36.653 INFO:tasks.workunit.client.0.smithi156.stdout:3/279: dread - d8/d14/f59 zero size 2024-07-16T09:55:36.653 INFO:tasks.workunit.client.0.smithi156.stdout:0/293: read d0/d1a/d4d/d9/f26 [1819506,74384] 0 2024-07-16T09:55:36.654 INFO:tasks.workunit.client.0.smithi156.stdout:6/348: creat d3/d13/d17/d4f/d1f/d19/d2e/f74 x:0 0 0 2024-07-16T09:55:36.655 INFO:tasks.workunit.client.0.smithi156.stdout:3/280: creat d8/da/f5d x:0 0 0 2024-07-16T09:55:36.656 INFO:tasks.workunit.client.0.smithi156.stdout:0/294: creat d0/d1a/d4d/d25/f55 x:0 0 0 2024-07-16T09:55:36.656 INFO:tasks.workunit.client.1.smithi176.stdout:0/391: sync 2024-07-16T09:55:36.656 INFO:tasks.workunit.client.1.smithi176.stdout:9/418: sync 2024-07-16T09:55:36.657 INFO:tasks.workunit.client.1.smithi176.stdout:9/419: read d5/de/d38/f4e [172173,43061] 0 2024-07-16T09:55:36.658 INFO:tasks.workunit.client.0.smithi156.stdout:0/295: rmdir d0/d1a/d4d/d9/d27/d3d 39 2024-07-16T09:55:36.659 INFO:tasks.workunit.client.1.smithi176.stdout:0/392: mknod d5/d25/c7a 0 2024-07-16T09:55:36.660 INFO:tasks.workunit.client.1.smithi176.stdout:0/393: stat d5 0 2024-07-16T09:55:36.660 INFO:tasks.workunit.client.0.smithi156.stdout:3/281: rmdir d8/da/d44 0 2024-07-16T09:55:36.660 INFO:tasks.workunit.client.0.smithi156.stdout:0/296: fdatasync d0/d22/d41/f4a 0 2024-07-16T09:55:36.660 INFO:tasks.workunit.client.0.smithi156.stdout:6/349: dwrite d3/d13/d17/f55 [0,4194304] 0 2024-07-16T09:55:36.661 INFO:tasks.workunit.client.1.smithi176.stdout:0/394: write d5/d1c/d24/f34 [989037,67545] 0 2024-07-16T09:55:36.662 INFO:tasks.workunit.client.1.smithi176.stdout:9/420: creat d5/de/f88 x:0 0 0 2024-07-16T09:55:36.662 INFO:tasks.workunit.client.0.smithi156.stdout:3/282: creat d8/da/d28/f5e x:0 0 0 2024-07-16T09:55:36.663 INFO:tasks.workunit.client.1.smithi176.stdout:0/395: mknod d5/d2c/c7b 0 2024-07-16T09:55:36.663 INFO:tasks.workunit.client.1.smithi176.stdout:9/421: chown d5/de/d38/l3d 395457 1 2024-07-16T09:55:36.663 INFO:tasks.workunit.client.0.smithi156.stdout:6/350: write d3/d13/d17/d4f/d1e/f3d [666591,47232] 0 2024-07-16T09:55:36.664 INFO:tasks.workunit.client.1.smithi176.stdout:0/396: write d5/f10 [1693778,119544] 0 2024-07-16T09:55:36.666 INFO:tasks.workunit.client.1.smithi176.stdout:9/422: write d5/d1d/d20/f74 [550979,27674] 0 2024-07-16T09:55:36.666 INFO:tasks.workunit.client.1.smithi176.stdout:0/397: read d5/d1c/f1f [807824,67114] 0 2024-07-16T09:55:36.667 INFO:tasks.workunit.client.0.smithi156.stdout:6/351: readlink d3/d13/d17/d4f/d1e/l27 0 2024-07-16T09:55:36.669 INFO:tasks.workunit.client.1.smithi176.stdout:9/423: mkdir d5/de/d37/d58/d89 0 2024-07-16T09:55:36.670 INFO:tasks.workunit.client.1.smithi176.stdout:9/424: write d5/d1d/d67/f6a [1037708,93646] 0 2024-07-16T09:55:36.674 INFO:tasks.workunit.client.0.smithi156.stdout:3/283: dwrite d8/da/d21/f3e [0,4194304] 0 2024-07-16T09:55:36.674 INFO:tasks.workunit.client.1.smithi176.stdout:9/425: rename d5/de/d38 to d5/de/d37/d58/d89/d8a 0 2024-07-16T09:55:36.675 INFO:tasks.workunit.client.1.smithi176.stdout:9/426: write d5/d1d/d20/f74 [1429792,120919] 0 2024-07-16T09:55:36.678 INFO:tasks.workunit.client.0.smithi156.stdout:6/352: dwrite d3/d13/d17/d4f/d1f/d19/d31/d35/f62 [0,4194304] 0 2024-07-16T09:55:36.681 INFO:tasks.workunit.client.0.smithi156.stdout:6/353: fsync d3/d13/d17/d4f/d1f/fa 0 2024-07-16T09:55:36.681 INFO:tasks.workunit.client.0.smithi156.stdout:6/354: fdatasync d3/d13/d17/f55 0 2024-07-16T09:55:36.681 INFO:tasks.workunit.client.0.smithi156.stdout:6/355: fsync d3/f2a 0 2024-07-16T09:55:36.683 INFO:tasks.workunit.client.1.smithi176.stdout:9/427: link d5/de/f1a d5/de/f8b 0 2024-07-16T09:55:36.686 INFO:tasks.workunit.client.1.smithi176.stdout:0/398: dwrite f4 [0,4194304] 0 2024-07-16T09:55:36.687 INFO:tasks.workunit.client.1.smithi176.stdout:9/428: creat d5/d52/d71/f8c x:0 0 0 2024-07-16T09:55:36.689 INFO:tasks.workunit.client.1.smithi176.stdout:0/399: dread - d5/d1c/d24/d35/d50/d58/f6f zero size 2024-07-16T09:55:36.692 INFO:tasks.workunit.client.1.smithi176.stdout:0/400: rmdir d5/d25 39 2024-07-16T09:55:36.706 INFO:tasks.workunit.client.1.smithi176.stdout:9/429: rename d5/d52/d71 to d5/d1d/d20/d8d 0 2024-07-16T09:55:36.715 INFO:tasks.workunit.client.1.smithi176.stdout:9/430: write d5/de/f47 [139056,2044] 0 2024-07-16T09:55:36.721 INFO:tasks.workunit.client.0.smithi156.stdout:0/297: getdents d0/d22/d41 0 2024-07-16T09:55:36.722 INFO:tasks.workunit.client.0.smithi156.stdout:0/298: write d0/d1a/d4d/d25/f55 [211197,81010] 0 2024-07-16T09:55:36.722 INFO:tasks.workunit.client.0.smithi156.stdout:0/299: fdatasync d0/d1a/d4d/f21 0 2024-07-16T09:55:36.723 INFO:tasks.workunit.client.0.smithi156.stdout:2/479: unlink d0/fe 0 2024-07-16T09:55:36.723 INFO:tasks.workunit.client.1.smithi176.stdout:9/431: dwrite d5/de/d37/d58/f60 [0,4194304] 0 2024-07-16T09:55:36.724 INFO:tasks.workunit.client.0.smithi156.stdout:0/300: write d0/d22/d41/f4a [2126594,55967] 0 2024-07-16T09:55:36.727 INFO:tasks.workunit.client.0.smithi156.stdout:0/301: creat d0/d1a/d4d/d9/f56 x:0 0 0 2024-07-16T09:55:36.727 INFO:tasks.workunit.client.1.smithi176.stdout:0/401: dread d5/d1c/d24/d35/d50/d5b/f65 [0,4194304] 0 2024-07-16T09:55:36.728 INFO:tasks.workunit.client.1.smithi176.stdout:9/432: rename d5/d1d/f3f to d5/de/d37/d58/d89/d8a/f8e 0 2024-07-16T09:55:36.730 INFO:tasks.workunit.client.1.smithi176.stdout:0/402: mkdir d5/d1c/d24/d35/d50/d7c 0 2024-07-16T09:55:36.731 INFO:tasks.workunit.client.0.smithi156.stdout:0/302: write d0/f14 [141812,20339] 0 2024-07-16T09:55:36.733 INFO:tasks.workunit.client.0.smithi156.stdout:0/303: fsync d0/d1a/d4d/d9/f12 0 2024-07-16T09:55:36.733 INFO:tasks.workunit.client.1.smithi176.stdout:0/403: mknod d5/d1c/c7d 0 2024-07-16T09:55:36.734 INFO:tasks.workunit.client.1.smithi176.stdout:9/433: dread d5/fd [0,4194304] 0 2024-07-16T09:55:36.734 INFO:tasks.workunit.client.1.smithi176.stdout:0/404: chown d5/l48 399415962 1 2024-07-16T09:55:36.734 INFO:tasks.workunit.client.0.smithi156.stdout:4/298: truncate d4/db/dc/f4d 5800269 0 2024-07-16T09:55:36.736 INFO:tasks.workunit.client.0.smithi156.stdout:0/304: symlink d0/d1a/l57 0 2024-07-16T09:55:36.737 INFO:tasks.workunit.client.1.smithi176.stdout:9/434: creat d5/d1d/d20/d78/f8f x:0 0 0 2024-07-16T09:55:36.738 INFO:tasks.workunit.client.0.smithi156.stdout:7/317: symlink dc/l74 0 2024-07-16T09:55:36.738 INFO:tasks.workunit.client.1.smithi176.stdout:9/435: write d5/de/f12 [3753498,112240] 0 2024-07-16T09:55:36.739 INFO:tasks.workunit.client.1.smithi176.stdout:0/405: rename d5/f12 to d5/d39/f7e 0 2024-07-16T09:55:36.740 INFO:tasks.workunit.client.1.smithi176.stdout:0/406: chown d5/d1c/d24/d35/d50/d58/d63 14038173 1 2024-07-16T09:55:36.740 INFO:tasks.workunit.client.1.smithi176.stdout:9/436: chown d5/de/d37/d58/f60 2085512 1 2024-07-16T09:55:36.741 INFO:tasks.workunit.client.0.smithi156.stdout:7/318: symlink dc/d13/d36/l75 0 2024-07-16T09:55:36.742 INFO:tasks.workunit.client.1.smithi176.stdout:0/407: creat d5/d1c/d24/d35/d50/f7f x:0 0 0 2024-07-16T09:55:36.744 INFO:tasks.workunit.client.1.smithi176.stdout:0/408: creat d5/d2c/f80 x:0 0 0 2024-07-16T09:55:36.744 INFO:tasks.workunit.client.0.smithi156.stdout:7/319: getdents dc/d3e/d3f/d51/d6c 0 2024-07-16T09:55:36.746 INFO:tasks.workunit.client.1.smithi176.stdout:0/409: mknod d5/d2c/d6c/c81 0 2024-07-16T09:55:36.751 INFO:tasks.workunit.client.0.smithi156.stdout:7/320: dread dc/d13/d20/f3c [0,4194304] 0 2024-07-16T09:55:36.752 INFO:tasks.workunit.client.0.smithi156.stdout:1/409: rename dc/d14/c47 to dc/d14/d1e/c87 0 2024-07-16T09:55:36.756 INFO:tasks.workunit.client.1.smithi176.stdout:0/410: truncate d5/d2c/d6c/f5e 285187 0 2024-07-16T09:55:36.756 INFO:tasks.workunit.client.0.smithi156.stdout:8/291: creat d0/f6d x:0 0 0 2024-07-16T09:55:36.757 INFO:tasks.workunit.client.0.smithi156.stdout:7/321: rmdir dc/d48/d6f 0 2024-07-16T09:55:36.758 INFO:tasks.workunit.client.1.smithi176.stdout:0/411: truncate d5/f22 1874378 0 2024-07-16T09:55:36.758 INFO:tasks.workunit.client.0.smithi156.stdout:7/322: write dc/d13/d36/d56/f5a [441256,64248] 0 2024-07-16T09:55:36.759 INFO:tasks.workunit.client.0.smithi156.stdout:7/323: stat dc/d13/d24 0 2024-07-16T09:55:36.759 INFO:tasks.workunit.client.1.smithi176.stdout:0/412: chown d5/d1c/d24/d35/d50/d58/f60 93853174 1 2024-07-16T09:55:36.759 INFO:tasks.workunit.client.0.smithi156.stdout:8/292: mknod d0/d11/d13/d1c/d39/c6e 0 2024-07-16T09:55:36.760 INFO:tasks.workunit.client.0.smithi156.stdout:7/324: dread - dc/d13/d36/f66 zero size 2024-07-16T09:55:36.760 INFO:tasks.workunit.client.0.smithi156.stdout:8/293: chown d0/d11/d13/f14 67814 1 2024-07-16T09:55:36.760 INFO:tasks.workunit.client.0.smithi156.stdout:5/364: unlink ff 0 2024-07-16T09:55:36.762 INFO:tasks.workunit.client.1.smithi176.stdout:0/413: mkdir d5/d25/d82 0 2024-07-16T09:55:36.762 INFO:tasks.workunit.client.0.smithi156.stdout:8/294: read d0/d11/d13/d1c/d24/f36 [14475,24957] 0 2024-07-16T09:55:36.763 INFO:tasks.workunit.client.0.smithi156.stdout:8/295: read d0/ff [351133,10202] 0 2024-07-16T09:55:36.766 INFO:tasks.workunit.client.0.smithi156.stdout:8/296: creat d0/f6f x:0 0 0 2024-07-16T09:55:36.768 INFO:tasks.workunit.client.0.smithi156.stdout:8/297: write d0/dd/f3c [212262,82748] 0 2024-07-16T09:55:36.769 INFO:tasks.workunit.client.0.smithi156.stdout:8/298: write d0/f6d [908088,56197] 0 2024-07-16T09:55:36.769 INFO:tasks.workunit.client.0.smithi156.stdout:7/325: rmdir dc/d13/d24 39 2024-07-16T09:55:36.771 INFO:tasks.workunit.client.0.smithi156.stdout:9/392: creat d2/d5/d16/d22/f7c x:0 0 0 2024-07-16T09:55:36.772 INFO:tasks.workunit.client.0.smithi156.stdout:8/299: creat d0/d11/d29/d68/f70 x:0 0 0 2024-07-16T09:55:36.774 INFO:tasks.workunit.client.0.smithi156.stdout:8/300: mknod d0/d11/d29/d5e/c71 0 2024-07-16T09:55:36.775 INFO:tasks.workunit.client.1.smithi176.stdout:0/414: dwrite d5/d39/f7e [0,4194304] 0 2024-07-16T09:55:36.776 INFO:tasks.workunit.client.0.smithi156.stdout:8/301: write d0/dd/f19 [686111,68135] 0 2024-07-16T09:55:36.776 INFO:tasks.workunit.client.0.smithi156.stdout:9/393: dread d2/de/d42/f4c [0,4194304] 0 2024-07-16T09:55:36.778 INFO:tasks.workunit.client.1.smithi176.stdout:0/415: mkdir d5/d2c/d6c/d83 0 2024-07-16T09:55:36.779 INFO:tasks.workunit.client.0.smithi156.stdout:8/302: truncate d0/d11/d29/f5d 348325 0 2024-07-16T09:55:36.779 INFO:tasks.workunit.client.0.smithi156.stdout:2/480: fsync d0/d5/d29/d2d/f5a 0 2024-07-16T09:55:36.779 INFO:tasks.workunit.client.1.smithi176.stdout:0/416: rmdir d5/d1c/d5a 39 2024-07-16T09:55:36.780 INFO:tasks.workunit.client.0.smithi156.stdout:9/394: symlink d2/de/d42/l7d 0 2024-07-16T09:55:36.780 INFO:tasks.workunit.client.0.smithi156.stdout:5/365: dread d1b/d1f/d57/f59 [0,4194304] 0 2024-07-16T09:55:36.781 INFO:tasks.workunit.client.0.smithi156.stdout:9/395: read - d2/d5/d16/d22/f7c zero size 2024-07-16T09:55:36.782 INFO:tasks.workunit.client.0.smithi156.stdout:8/303: stat d0/d11/d13/c49 0 2024-07-16T09:55:36.784 INFO:tasks.workunit.client.1.smithi176.stdout:0/417: dread d5/d1c/f1f [0,4194304] 0 2024-07-16T09:55:36.784 INFO:tasks.workunit.client.0.smithi156.stdout:5/366: rmdir d1b/d1f/d45 39 2024-07-16T09:55:36.786 INFO:tasks.workunit.client.0.smithi156.stdout:4/299: write d4/f5 [4789555,115245] 0 2024-07-16T09:55:36.786 INFO:tasks.workunit.client.0.smithi156.stdout:0/305: truncate d0/d1a/d4d/d25/f38 3176659 0 2024-07-16T09:55:36.787 INFO:tasks.workunit.client.0.smithi156.stdout:2/481: chown d0/d5/d29/f3f 366382510 1 2024-07-16T09:55:36.791 INFO:tasks.workunit.client.0.smithi156.stdout:4/300: chown d4/db/dc/f29 71722 1 2024-07-16T09:55:36.792 INFO:tasks.workunit.client.0.smithi156.stdout:0/306: write d0/d22/d41/f47 [346878,75029] 0 2024-07-16T09:55:36.792 INFO:tasks.workunit.client.0.smithi156.stdout:2/482: dread - d0/d5/d8b/f92 zero size 2024-07-16T09:55:36.792 INFO:tasks.workunit.client.0.smithi156.stdout:5/367: mkdir d1b/d1f/d45/d7f 0 2024-07-16T09:55:36.795 INFO:tasks.workunit.client.0.smithi156.stdout:1/410: symlink dc/d20/l88 0 2024-07-16T09:55:36.796 INFO:tasks.workunit.client.0.smithi156.stdout:8/304: dwrite d0/dd/f3b [0,4194304] 0 2024-07-16T09:55:36.797 INFO:tasks.workunit.client.0.smithi156.stdout:9/396: dwrite d2/de/f2f [0,4194304] 0 2024-07-16T09:55:36.798 INFO:tasks.workunit.client.0.smithi156.stdout:2/483: truncate d0/d5/f93 863283 0 2024-07-16T09:55:36.801 INFO:tasks.workunit.client.0.smithi156.stdout:4/301: symlink d4/db/dc/d5f/d64/l65 0 2024-07-16T09:55:36.803 INFO:tasks.workunit.client.0.smithi156.stdout:1/411: mknod dc/d14/d3e/c89 0 2024-07-16T09:55:36.804 INFO:tasks.workunit.client.1.smithi176.stdout:2/358: sync 2024-07-16T09:55:36.804 INFO:tasks.workunit.client.1.smithi176.stdout:8/464: sync 2024-07-16T09:55:36.807 INFO:tasks.workunit.client.0.smithi156.stdout:9/397: fsync d2/de/d42/f4c 0 2024-07-16T09:55:36.808 INFO:tasks.workunit.client.0.smithi156.stdout:0/307: creat d0/d22/d41/d52/f58 x:0 0 0 2024-07-16T09:55:36.808 INFO:tasks.workunit.client.0.smithi156.stdout:9/398: write d2/d5/d16/d22/d36/f50 [7548515,46942] 0 2024-07-16T09:55:36.810 INFO:tasks.workunit.client.0.smithi156.stdout:4/302: creat d4/d23/d52/f66 x:0 0 0 2024-07-16T09:55:36.811 INFO:tasks.workunit.client.0.smithi156.stdout:1/412: mkdir dc/d6b/d8a 0 2024-07-16T09:55:36.811 INFO:tasks.workunit.client.1.smithi176.stdout:8/465: dwrite d8/d23/f25 [0,4194304] 0 2024-07-16T09:55:36.812 INFO:tasks.workunit.client.0.smithi156.stdout:2/484: creat d0/d13/d17/d1c/faa x:0 0 0 2024-07-16T09:55:36.813 INFO:tasks.workunit.client.0.smithi156.stdout:8/305: creat d0/d11/f72 x:0 0 0 2024-07-16T09:55:36.815 INFO:tasks.workunit.client.0.smithi156.stdout:5/368: creat d1b/d1f/d45/d52/d49/d4d/f80 x:0 0 0 2024-07-16T09:55:36.815 INFO:tasks.workunit.client.0.smithi156.stdout:9/399: read d2/d5/f55 [1826887,25628] 0 2024-07-16T09:55:36.818 INFO:tasks.workunit.client.0.smithi156.stdout:2/485: mknod d0/d13/d17/cab 0 2024-07-16T09:55:36.818 INFO:tasks.workunit.client.0.smithi156.stdout:5/369: read d1b/d1f/d45/d52/d5e/f6e [371232,92392] 0 2024-07-16T09:55:36.820 INFO:tasks.workunit.client.0.smithi156.stdout:2/486: rename d0/d5/d76 to d0/d5/d76/da1/dac 22 2024-07-16T09:55:36.820 INFO:tasks.workunit.client.0.smithi156.stdout:4/303: mkdir d4/db/dc/d67 0 2024-07-16T09:55:36.821 INFO:tasks.workunit.client.0.smithi156.stdout:1/413: mkdir dc/d14/d8b 0 2024-07-16T09:55:36.822 INFO:tasks.workunit.client.0.smithi156.stdout:0/308: creat d0/d1a/d4d/f59 x:0 0 0 2024-07-16T09:55:36.822 INFO:tasks.workunit.client.0.smithi156.stdout:4/304: write d4/db/dc/f29 [2453552,8788] 0 2024-07-16T09:55:36.822 INFO:tasks.workunit.client.0.smithi156.stdout:8/306: link d0/d11/f40 d0/d11/d13/d1c/d39/f73 0 2024-07-16T09:55:36.824 INFO:tasks.workunit.client.0.smithi156.stdout:5/370: rename d1b/d1f/d45/d52/d5e/f6e to d1b/d1f/d45/d52/d49/f81 0 2024-07-16T09:55:36.827 INFO:tasks.workunit.client.0.smithi156.stdout:9/400: getdents d2/d5/d16/d22/d36 0 2024-07-16T09:55:36.828 INFO:tasks.workunit.client.0.smithi156.stdout:1/414: link dc/d6b/f80 dc/d14/d3e/d6c/d7a/f8c 0 2024-07-16T09:55:36.828 INFO:tasks.workunit.client.0.smithi156.stdout:0/309: dread d0/d1a/d4d/d9/fe [0,4194304] 0 2024-07-16T09:55:36.828 INFO:tasks.workunit.client.0.smithi156.stdout:2/487: link d0/d5/c95 d0/d5/d29/d2d/cad 0 2024-07-16T09:55:36.830 INFO:tasks.workunit.client.0.smithi156.stdout:4/305: link d4/d23/d30/d33/f50 d4/db/dc/f68 0 2024-07-16T09:55:36.831 INFO:tasks.workunit.client.0.smithi156.stdout:5/371: creat d1b/d1f/d45/d52/d43/d7c/f82 x:0 0 0 2024-07-16T09:55:36.832 INFO:tasks.workunit.client.0.smithi156.stdout:9/401: read d2/d5/d16/d22/d2b/d3d/d3c/f62 [710998,12210] 0 2024-07-16T09:55:36.833 INFO:tasks.workunit.client.0.smithi156.stdout:8/307: rename d0/d11/d29/d68 to d0/d74 0 2024-07-16T09:55:36.833 INFO:tasks.workunit.client.0.smithi156.stdout:5/372: read d1b/d1f/d45/d52/d43/f64 [271040,70426] 0 2024-07-16T09:55:36.834 INFO:tasks.workunit.client.0.smithi156.stdout:2/488: creat d0/d13/d17/d1c/d86/fae x:0 0 0 2024-07-16T09:55:36.835 INFO:tasks.workunit.client.0.smithi156.stdout:1/415: mknod dc/d14/d8b/c8d 0 2024-07-16T09:55:36.835 INFO:tasks.workunit.client.0.smithi156.stdout:4/306: write d4/db/dc/f2e [4066233,114720] 0 2024-07-16T09:55:36.837 INFO:tasks.workunit.client.0.smithi156.stdout:4/307: stat d4/db/dc/d5f/c51 0 2024-07-16T09:55:36.837 INFO:tasks.workunit.client.0.smithi156.stdout:8/308: stat d0/c7 0 2024-07-16T09:55:36.838 INFO:tasks.workunit.client.0.smithi156.stdout:8/309: fdatasync d0/f12 0 2024-07-16T09:55:36.839 INFO:tasks.workunit.client.0.smithi156.stdout:4/308: truncate d4/d23/d30/f2d 753399 0 2024-07-16T09:55:36.840 INFO:tasks.workunit.client.0.smithi156.stdout:5/373: fdatasync d1b/d1f/d45/d52/f39 0 2024-07-16T09:55:36.842 INFO:tasks.workunit.client.0.smithi156.stdout:8/310: chown d0/f2 462 1 2024-07-16T09:55:36.843 INFO:tasks.workunit.client.0.smithi156.stdout:4/309: creat d4/d23/d52/f69 x:0 0 0 2024-07-16T09:55:36.844 INFO:tasks.workunit.client.0.smithi156.stdout:2/489: dread d0/d13/d17/d1c/f46 [0,4194304] 0 2024-07-16T09:55:36.845 INFO:tasks.workunit.client.0.smithi156.stdout:8/311: rmdir d0/dd/d1f/d50 39 2024-07-16T09:55:36.845 INFO:tasks.workunit.client.0.smithi156.stdout:2/490: chown d0/d5/d29/l84 905081573 1 2024-07-16T09:55:36.846 INFO:tasks.workunit.client.0.smithi156.stdout:8/312: dread - d0/d11/d2e/d35/f46 zero size 2024-07-16T09:55:36.848 INFO:tasks.workunit.client.0.smithi156.stdout:5/374: symlink d1b/d1f/d42/l83 0 2024-07-16T09:55:36.848 INFO:tasks.workunit.client.0.smithi156.stdout:5/375: chown d1b/d1e/d37 60171 1 2024-07-16T09:55:36.849 INFO:tasks.workunit.client.0.smithi156.stdout:2/491: rmdir d0/d5/d29/d2d/d5b/d62/d74 39 2024-07-16T09:55:36.852 INFO:tasks.workunit.client.0.smithi156.stdout:4/310: dwrite d4/db/dc/f68 [0,4194304] 0 2024-07-16T09:55:36.853 INFO:tasks.workunit.client.0.smithi156.stdout:2/492: symlink d0/d5/d29/d2d/laf 0 2024-07-16T09:55:36.854 INFO:tasks.workunit.client.0.smithi156.stdout:2/493: truncate d0/d13/f1a 351271 0 2024-07-16T09:55:36.856 INFO:tasks.workunit.client.0.smithi156.stdout:5/376: unlink d1b/d1f/d45/d52/d2d/c3e 0 2024-07-16T09:55:36.856 INFO:tasks.workunit.client.0.smithi156.stdout:2/494: fsync d0/d13/f26 0 2024-07-16T09:55:36.860 INFO:tasks.workunit.client.0.smithi156.stdout:4/311: dwrite d4/db/dc/f29 [4194304,4194304] 0 2024-07-16T09:55:36.860 INFO:tasks.workunit.client.0.smithi156.stdout:5/377: write d1b/d1f/d45/d52/d49/d4d/f62 [1169421,70468] 0 2024-07-16T09:55:36.860 INFO:tasks.workunit.client.0.smithi156.stdout:2/495: dread - d0/d5/d29/d2d/d5b/d62/fa0 zero size 2024-07-16T09:55:36.863 INFO:tasks.workunit.client.0.smithi156.stdout:4/312: mkdir d4/d6a 0 2024-07-16T09:55:36.866 INFO:tasks.workunit.client.0.smithi156.stdout:4/313: read - d4/d23/d52/f66 zero size 2024-07-16T09:55:36.869 INFO:tasks.workunit.client.0.smithi156.stdout:2/496: symlink d0/d5/d29/d2d/d5b/d62/lb0 0 2024-07-16T09:55:36.870 INFO:tasks.workunit.client.0.smithi156.stdout:0/310: dwrite d0/d1a/d4d/d9/d27/d3d/f37 [0,4194304] 0 2024-07-16T09:55:36.870 INFO:tasks.workunit.client.0.smithi156.stdout:4/314: unlink d4/l3e 0 2024-07-16T09:55:36.873 INFO:tasks.workunit.client.1.smithi176.stdout:0/418: write f4 [3570702,127645] 0 2024-07-16T09:55:36.874 INFO:tasks.workunit.client.0.smithi156.stdout:4/315: write d4/db/dc/f55 [96654,75982] 0 2024-07-16T09:55:36.877 INFO:tasks.workunit.client.1.smithi176.stdout:2/359: chown dd/d36/d45/l4c 940 1 2024-07-16T09:55:36.877 INFO:tasks.workunit.client.0.smithi156.stdout:8/313: dwrite d0/f1 [0,4194304] 0 2024-07-16T09:55:36.878 INFO:tasks.workunit.client.0.smithi156.stdout:2/497: write d0/d5/d29/d2d/d5b/d5f/d89/f9a [485910,67688] 0 2024-07-16T09:55:36.879 INFO:tasks.workunit.client.1.smithi176.stdout:8/466: truncate d8/d23/d2c/f3b 17543 0 2024-07-16T09:55:36.879 INFO:tasks.workunit.client.0.smithi156.stdout:5/378: dwrite d1b/d1e/d37/f5a [0,4194304] 0 2024-07-16T09:55:36.879 INFO:tasks.workunit.client.0.smithi156.stdout:4/316: readlink d4/db/l12 0 2024-07-16T09:55:36.880 INFO:tasks.workunit.client.1.smithi176.stdout:2/360: symlink dd/de/l73 0 2024-07-16T09:55:36.881 INFO:tasks.workunit.client.1.smithi176.stdout:0/419: truncate d5/f22 1558140 0 2024-07-16T09:55:36.885 INFO:tasks.workunit.client.0.smithi156.stdout:4/317: chown d4/db/d47/d4b/f5b 0 1 2024-07-16T09:55:36.885 INFO:tasks.workunit.client.0.smithi156.stdout:8/314: chown d0/d11/d13/c49 453950 1 2024-07-16T09:55:36.887 INFO:tasks.workunit.client.1.smithi176.stdout:8/467: dwrite d8/d23/d2c/d37/f4f [0,4194304] 0 2024-07-16T09:55:36.887 INFO:tasks.workunit.client.0.smithi156.stdout:2/498: write d0/d13/d17/d1c/d86/f94 [1250542,126979] 0 2024-07-16T09:55:36.889 INFO:tasks.workunit.client.0.smithi156.stdout:5/379: fsync d1b/f55 0 2024-07-16T09:55:36.890 INFO:tasks.workunit.client.1.smithi176.stdout:0/420: dwrite d5/f1e [0,4194304] 0 2024-07-16T09:55:36.891 INFO:tasks.workunit.client.0.smithi156.stdout:0/311: dwrite d0/d1a/d4d/d9/f15 [8388608,4194304] 0 2024-07-16T09:55:36.891 INFO:tasks.workunit.client.0.smithi156.stdout:4/318: mknod d4/d23/d30/d33/c6b 0 2024-07-16T09:55:36.892 INFO:tasks.workunit.client.0.smithi156.stdout:2/499: truncate d0/d13/d17/d1c/faa 695190 0 2024-07-16T09:55:36.893 INFO:tasks.workunit.client.1.smithi176.stdout:8/468: mknod d8/d14/d1e/d7c/c9f 0 2024-07-16T09:55:36.899 INFO:tasks.workunit.client.0.smithi156.stdout:2/500: dread d0/d13/d17/f23 [0,4194304] 0 2024-07-16T09:55:36.901 INFO:tasks.workunit.client.0.smithi156.stdout:5/380: creat d1b/d1f/d45/d7f/f84 x:0 0 0 2024-07-16T09:55:36.902 INFO:tasks.workunit.client.1.smithi176.stdout:0/421: dwrite d5/d2c/f3f [0,4194304] 0 2024-07-16T09:55:36.903 INFO:tasks.workunit.client.1.smithi176.stdout:0/422: write d5/f6e [985782,79386] 0 2024-07-16T09:55:36.904 INFO:tasks.workunit.client.0.smithi156.stdout:0/312: rename d0/d1a/d4d/d9/d27/f33 to d0/d1a/d4d/d9/d27/d50/f5a 0 2024-07-16T09:55:36.907 INFO:tasks.workunit.client.0.smithi156.stdout:4/319: getdents d4/db/d47/d4b 0 2024-07-16T09:55:36.908 INFO:tasks.workunit.client.0.smithi156.stdout:5/381: rename d1b/d1f/d57 to d1b/d1f/d45/d52/d43/d85 0 2024-07-16T09:55:36.909 INFO:tasks.workunit.client.1.smithi176.stdout:7/446: sync 2024-07-16T09:55:36.909 INFO:tasks.workunit.client.1.smithi176.stdout:1/262: sync 2024-07-16T09:55:36.910 INFO:tasks.workunit.client.1.smithi176.stdout:0/423: dread d5/d2c/d6c/f5e [0,4194304] 0 2024-07-16T09:55:36.910 INFO:tasks.workunit.client.1.smithi176.stdout:5/433: sync 2024-07-16T09:55:36.911 INFO:tasks.workunit.client.0.smithi156.stdout:4/320: chown d4/db/dc/l10 3 1 2024-07-16T09:55:36.913 INFO:tasks.workunit.client.1.smithi176.stdout:0/424: readlink d5/l31 0 2024-07-16T09:55:36.914 INFO:tasks.workunit.client.0.smithi156.stdout:4/321: read d4/db/dc/f2a [817030,128045] 0 2024-07-16T09:55:36.916 INFO:tasks.workunit.client.0.smithi156.stdout:0/313: link d0/d1a/f42 d0/f5b 0 2024-07-16T09:55:36.917 INFO:tasks.workunit.client.1.smithi176.stdout:1/263: rmdir d4/d8/d2c/d56 0 2024-07-16T09:55:36.919 INFO:tasks.workunit.client.0.smithi156.stdout:0/314: rmdir d0/d1a/d4d/d9/d27/d50 39 2024-07-16T09:55:36.919 INFO:tasks.workunit.client.0.smithi156.stdout:0/315: chown d0/d1a/d4d/c16 1021314288 1 2024-07-16T09:55:36.921 INFO:tasks.workunit.client.1.smithi176.stdout:8/469: dread d8/d23/d2c/f36 [0,4194304] 0 2024-07-16T09:55:36.921 INFO:tasks.workunit.client.0.smithi156.stdout:4/322: link d4/f9 d4/d23/d30/f6c 0 2024-07-16T09:55:36.924 INFO:tasks.workunit.client.1.smithi176.stdout:7/447: dwrite d4/d47/d6d/d89/d66/f8a [0,4194304] 0 2024-07-16T09:55:36.924 INFO:tasks.workunit.client.1.smithi176.stdout:1/264: chown d4/d8/d12/d21/d25/c46 29842135 1 2024-07-16T09:55:36.926 INFO:tasks.workunit.client.0.smithi156.stdout:0/316: creat d0/f5c x:0 0 0 2024-07-16T09:55:36.926 INFO:tasks.workunit.client.0.smithi156.stdout:0/317: readlink d0/d1a/d4d/d9/d27/l3f 0 2024-07-16T09:55:36.929 INFO:tasks.workunit.client.0.smithi156.stdout:0/318: creat d0/d22/d41/d52/f5d x:0 0 0 2024-07-16T09:55:36.929 INFO:tasks.workunit.client.1.smithi176.stdout:7/448: readlink d4/dd/d26/l3d 0 2024-07-16T09:55:36.932 INFO:tasks.workunit.client.1.smithi176.stdout:1/265: write d4/d8/d12/d21/d34/f58 [4736647,83020] 0 2024-07-16T09:55:36.933 INFO:tasks.workunit.client.1.smithi176.stdout:0/425: dwrite d5/d1c/d24/d35/d50/d58/f79 [0,4194304] 0 2024-07-16T09:55:36.933 INFO:tasks.workunit.client.0.smithi156.stdout:0/319: mkdir d0/d1a/d4d/d5e 0 2024-07-16T09:55:36.937 INFO:tasks.workunit.client.1.smithi176.stdout:7/449: mknod d4/d5/c95 0 2024-07-16T09:55:36.937 INFO:tasks.workunit.client.1.smithi176.stdout:0/426: fdatasync d5/d1c/d5a/f6d 0 2024-07-16T09:55:36.938 INFO:tasks.workunit.client.1.smithi176.stdout:8/470: dwrite d8/d23/d2c/d37/f3c [4194304,4194304] 0 2024-07-16T09:55:36.938 INFO:tasks.workunit.client.1.smithi176.stdout:2/361: dread dd/d16/f27 [4194304,4194304] 0 2024-07-16T09:55:36.939 INFO:tasks.workunit.client.0.smithi156.stdout:0/320: creat d0/d1a/d4d/d9/d27/d50/f5f x:0 0 0 2024-07-16T09:55:36.941 INFO:tasks.workunit.client.1.smithi176.stdout:1/266: symlink d4/d3a/l5e 0 2024-07-16T09:55:36.942 INFO:tasks.workunit.client.0.smithi156.stdout:0/321: dread d0/d1a/d4d/d25/f55 [0,4194304] 0 2024-07-16T09:55:36.943 INFO:tasks.workunit.client.0.smithi156.stdout:0/322: dread - d0/d1a/f46 zero size 2024-07-16T09:55:36.944 INFO:tasks.workunit.client.1.smithi176.stdout:7/450: creat d4/d1f/d22/d51/f96 x:0 0 0 2024-07-16T09:55:36.944 INFO:tasks.workunit.client.1.smithi176.stdout:1/267: rmdir d4/d8 39 2024-07-16T09:55:36.946 INFO:tasks.workunit.client.0.smithi156.stdout:0/323: read d0/d1a/d4d/d25/f38 [1352090,25120] 0 2024-07-16T09:55:36.950 INFO:tasks.workunit.client.0.smithi156.stdout:0/324: mkdir d0/d1a/d4d/d60 0 2024-07-16T09:55:36.951 INFO:tasks.workunit.client.0.smithi156.stdout:0/325: fsync d0/d1a/d4d/d9/f56 0 2024-07-16T09:55:36.953 INFO:tasks.workunit.client.0.smithi156.stdout:0/326: write d0/d22/f3c [876147,79657] 0 2024-07-16T09:55:36.953 INFO:tasks.workunit.client.1.smithi176.stdout:1/268: creat d4/d8/d12/d21/d25/d30/d44/f5f x:0 0 0 2024-07-16T09:55:36.954 INFO:tasks.workunit.client.0.smithi156.stdout:0/327: fsync d0/d1a/d4d/d25/f3a 0 2024-07-16T09:55:36.956 INFO:tasks.workunit.client.1.smithi176.stdout:1/269: creat d4/d8/d12/d21/d25/f60 x:0 0 0 2024-07-16T09:55:36.958 INFO:tasks.workunit.client.0.smithi156.stdout:5/382: dwrite d1b/d1f/d45/d52/d43/f64 [0,4194304] 0 2024-07-16T09:55:36.958 INFO:tasks.workunit.client.0.smithi156.stdout:0/328: mkdir d0/d1a/d4d/d61 0 2024-07-16T09:55:36.959 INFO:tasks.workunit.client.0.smithi156.stdout:0/329: readlink d0/d1a/l57 0 2024-07-16T09:55:36.959 INFO:tasks.workunit.client.0.smithi156.stdout:5/383: write d1b/d1f/d45/d52/f3c [3014613,14921] 0 2024-07-16T09:55:36.960 INFO:tasks.workunit.client.1.smithi176.stdout:7/451: truncate d4/f2b 649997 0 2024-07-16T09:55:36.960 INFO:tasks.workunit.client.1.smithi176.stdout:0/427: dread d5/d1c/d24/d35/f3c [0,4194304] 0 2024-07-16T09:55:36.961 INFO:tasks.workunit.client.0.smithi156.stdout:5/384: chown d1b/f34 720428 1 2024-07-16T09:55:36.962 INFO:tasks.workunit.client.0.smithi156.stdout:0/330: rmdir d0/d1a/d4d/d9/d27/d3d 39 2024-07-16T09:55:36.962 INFO:tasks.workunit.client.1.smithi176.stdout:1/270: creat d4/d8/d12/d21/d25/d30/f61 x:0 0 0 2024-07-16T09:55:36.962 INFO:tasks.workunit.client.1.smithi176.stdout:8/471: write d8/d23/d2c/f3b [126071,45705] 0 2024-07-16T09:55:36.963 INFO:tasks.workunit.client.0.smithi156.stdout:5/385: mkdir d1b/d1f/d45/d52/d2d/d86 0 2024-07-16T09:55:36.964 INFO:tasks.workunit.client.0.smithi156.stdout:5/386: dread - d1b/f40 zero size 2024-07-16T09:55:36.965 INFO:tasks.workunit.client.1.smithi176.stdout:7/452: mkdir d4/d5/d97 0 2024-07-16T09:55:36.965 INFO:tasks.workunit.client.1.smithi176.stdout:2/362: dwrite f9 [4194304,4194304] 0 2024-07-16T09:55:36.965 INFO:tasks.workunit.client.1.smithi176.stdout:0/428: mkdir d5/d2c/d6c/d83/d84 0 2024-07-16T09:55:36.966 INFO:tasks.workunit.client.0.smithi156.stdout:0/331: mkdir d0/d1a/d4d/d9/d27/d3d/d62 0 2024-07-16T09:55:36.967 INFO:tasks.workunit.client.1.smithi176.stdout:2/363: write dd/d36/f61 [336011,21042] 0 2024-07-16T09:55:36.968 INFO:tasks.workunit.client.1.smithi176.stdout:8/472: rmdir d8/d20/d2a/d31/d2f 39 2024-07-16T09:55:36.969 INFO:tasks.workunit.client.0.smithi156.stdout:0/332: write d0/d1a/d4d/d25/f55 [897430,92571] 0 2024-07-16T09:55:36.970 INFO:tasks.workunit.client.0.smithi156.stdout:5/387: dwrite d1b/f31 [0,4194304] 0 2024-07-16T09:55:36.971 INFO:tasks.workunit.client.0.smithi156.stdout:0/333: readlink d0/d1a/d4d/d9/d27/l2f 0 2024-07-16T09:55:36.971 INFO:tasks.workunit.client.0.smithi156.stdout:0/334: stat d0/d1a/d4d/d25 0 2024-07-16T09:55:36.972 INFO:tasks.workunit.client.0.smithi156.stdout:5/388: truncate d1b/d1e/f3d 1933558 0 2024-07-16T09:55:36.973 INFO:tasks.workunit.client.0.smithi156.stdout:0/335: write d0/d1a/d4d/f21 [1932362,61681] 0 2024-07-16T09:55:36.974 INFO:tasks.workunit.client.1.smithi176.stdout:2/364: link dd/de/c34 dd/d2e/c74 0 2024-07-16T09:55:36.977 INFO:tasks.workunit.client.1.smithi176.stdout:7/453: dwrite d4/dd/d63/f64 [0,4194304] 0 2024-07-16T09:55:36.977 INFO:tasks.workunit.client.0.smithi156.stdout:0/336: fdatasync d0/d1a/d4d/d9/fe 0 2024-07-16T09:55:36.982 INFO:tasks.workunit.client.0.smithi156.stdout:5/389: dwrite d1b/d1e/f48 [0,4194304] 0 2024-07-16T09:55:36.983 INFO:tasks.workunit.client.1.smithi176.stdout:0/429: dwrite d5/d1c/d24/d26/f69 [0,4194304] 0 2024-07-16T09:55:36.984 INFO:tasks.workunit.client.0.smithi156.stdout:0/337: unlink d0/d1a/f42 0 2024-07-16T09:55:36.985 INFO:tasks.workunit.client.0.smithi156.stdout:0/338: fdatasync d0/d22/d41/d52/f58 0 2024-07-16T09:55:36.986 INFO:tasks.workunit.client.1.smithi176.stdout:2/365: write dd/d16/d39/f6f [480932,68269] 0 2024-07-16T09:55:36.991 INFO:tasks.workunit.client.1.smithi176.stdout:7/454: dwrite d4/d1f/d22/f52 [0,4194304] 0 2024-07-16T09:55:36.994 INFO:tasks.workunit.client.1.smithi176.stdout:7/455: truncate d4/dd/d49/f7c 374477 0 2024-07-16T09:55:36.994 INFO:tasks.workunit.client.0.smithi156.stdout:5/390: dwrite d1b/d1f/d45/d52/f3c [4194304,4194304] 0 2024-07-16T09:55:36.994 INFO:tasks.workunit.client.1.smithi176.stdout:0/430: link d5/d1c/d24/d35/d50/f62 d5/d44/f85 0 2024-07-16T09:55:36.995 INFO:tasks.workunit.client.0.smithi156.stdout:5/391: chown fc 150 1 2024-07-16T09:55:36.995 INFO:tasks.workunit.client.1.smithi176.stdout:2/366: creat dd/d16/d39/d3d/f75 x:0 0 0 2024-07-16T09:55:36.995 INFO:tasks.workunit.client.1.smithi176.stdout:7/456: write d4/f11 [8948459,87553] 0 2024-07-16T09:55:36.998 INFO:tasks.workunit.client.0.smithi156.stdout:5/392: chown d1b/c3a 949 1 2024-07-16T09:55:37.002 INFO:tasks.workunit.client.1.smithi176.stdout:2/367: creat dd/d16/d39/d3d/f76 x:0 0 0 2024-07-16T09:55:37.002 INFO:tasks.workunit.client.0.smithi156.stdout:0/339: link d0/d22/d41/f47 d0/d1a/d4d/d9/d27/d3d/d62/f63 0 2024-07-16T09:55:37.002 INFO:tasks.workunit.client.0.smithi156.stdout:0/340: stat d0/d1a/f46 0 2024-07-16T09:55:37.002 INFO:tasks.workunit.client.0.smithi156.stdout:0/341: write d0/d1a/d4d/f21 [1329312,86765] 0 2024-07-16T09:55:37.003 INFO:tasks.workunit.client.1.smithi176.stdout:0/431: rename d5/d1c/d24/d26/f69 to d5/d1c/d5a/f86 0 2024-07-16T09:55:37.003 INFO:tasks.workunit.client.1.smithi176.stdout:0/432: stat d5 0 2024-07-16T09:55:37.005 INFO:tasks.workunit.client.0.smithi156.stdout:0/342: rmdir d0/d1a/d4d/d9/d27 39 2024-07-16T09:55:37.006 INFO:tasks.workunit.client.0.smithi156.stdout:5/393: dread d1b/f34 [0,4194304] 0 2024-07-16T09:55:37.007 INFO:tasks.workunit.client.0.smithi156.stdout:0/343: readlink d0/d1a/d4d/d9/d27/l4b 0 2024-07-16T09:55:37.010 INFO:tasks.workunit.client.1.smithi176.stdout:2/368: creat dd/f77 x:0 0 0 2024-07-16T09:55:37.010 INFO:tasks.workunit.client.1.smithi176.stdout:7/457: link d4/d1f/d22/d51/f58 d4/d1f/d22/d51/f98 0 2024-07-16T09:55:37.010 INFO:tasks.workunit.client.0.smithi156.stdout:0/344: symlink d0/d1a/d4d/d9/d27/d3d/d62/l64 0 2024-07-16T09:55:37.011 INFO:tasks.workunit.client.0.smithi156.stdout:5/394: read d1b/d1f/f44 [3288484,9304] 0 2024-07-16T09:55:37.013 INFO:tasks.workunit.client.0.smithi156.stdout:5/395: fdatasync d1b/d1f/d45/d52/d43/f5b 0 2024-07-16T09:55:37.013 INFO:tasks.workunit.client.1.smithi176.stdout:7/458: creat d4/dd/f99 x:0 0 0 2024-07-16T09:55:37.014 INFO:tasks.workunit.client.1.smithi176.stdout:2/369: getdents dd/d36 0 2024-07-16T09:55:37.014 INFO:tasks.workunit.client.1.smithi176.stdout:7/459: chown d4/d1f/d22/d80/d17/f4d 443464849 1 2024-07-16T09:55:37.015 INFO:tasks.workunit.client.0.smithi156.stdout:5/396: mkdir d1b/d1f/d45/d7f/d87 0 2024-07-16T09:55:37.018 INFO:tasks.workunit.client.0.smithi156.stdout:0/345: dwrite d0/d22/d41/d52/f5d [0,4194304] 0 2024-07-16T09:55:37.019 INFO:tasks.workunit.client.1.smithi176.stdout:7/460: getdents d4/d1f/d22/d80/d17/d1d 0 2024-07-16T09:55:37.020 INFO:tasks.workunit.client.0.smithi156.stdout:0/346: symlink d0/d1a/d4d/d25/l65 0 2024-07-16T09:55:37.021 INFO:tasks.workunit.client.0.smithi156.stdout:5/397: dread d1b/d1f/f51 [0,4194304] 0 2024-07-16T09:55:37.021 INFO:tasks.workunit.client.1.smithi176.stdout:7/461: mkdir d4/d47/d5d/d9a 0 2024-07-16T09:55:37.025 INFO:tasks.workunit.client.0.smithi156.stdout:5/398: rename d1b/d1f/d45/d52/d5e to d1b/d1e/d37/d88 0 2024-07-16T09:55:37.027 INFO:tasks.workunit.client.1.smithi176.stdout:7/462: link d4/d1f/f2a d4/d1f/d22/d80/d17/d1d/f9b 0 2024-07-16T09:55:37.030 INFO:tasks.workunit.client.1.smithi176.stdout:0/433: write d5/d1c/d24/d35/d50/d58/f60 [398627,3926] 0 2024-07-16T09:55:37.031 INFO:tasks.workunit.client.0.smithi156.stdout:5/399: link d1b/d1f/d45/d52/d49/f60 d1b/d1e/f89 0 2024-07-16T09:55:37.031 INFO:tasks.workunit.client.1.smithi176.stdout:7/463: write d4/d1f/d22/d80/d17/d1d/f9b [2771660,84902] 0 2024-07-16T09:55:37.031 INFO:tasks.workunit.client.1.smithi176.stdout:7/464: dread - d4/d5/f91 zero size 2024-07-16T09:55:37.033 INFO:tasks.workunit.client.0.smithi156.stdout:5/400: read f19 [2981504,95805] 0 2024-07-16T09:55:37.033 INFO:tasks.workunit.client.1.smithi176.stdout:0/434: chown d5/d1c/d24/d26/f3d 2202922 1 2024-07-16T09:55:37.034 INFO:tasks.workunit.client.1.smithi176.stdout:7/465: symlink d4/l9c 0 2024-07-16T09:55:37.035 INFO:tasks.workunit.client.0.smithi156.stdout:5/401: unlink d1b/c1c 0 2024-07-16T09:55:37.035 INFO:tasks.workunit.client.1.smithi176.stdout:7/466: fdatasync d4/dd/d49/f7c 0 2024-07-16T09:55:37.035 INFO:tasks.workunit.client.1.smithi176.stdout:2/370: truncate dd/d2e/d35/f3f 737186 0 2024-07-16T09:55:37.037 INFO:tasks.workunit.client.1.smithi176.stdout:0/435: rename d5/d1c/d24/d35/f3c to d5/d25/d82/f87 0 2024-07-16T09:55:37.038 INFO:tasks.workunit.client.0.smithi156.stdout:5/402: truncate d1b/d1e/f21 810662 0 2024-07-16T09:55:37.040 INFO:tasks.workunit.client.1.smithi176.stdout:2/371: creat dd/d49/f78 x:0 0 0 2024-07-16T09:55:37.044 INFO:tasks.workunit.client.0.smithi156.stdout:5/403: mkdir d1b/d1f/d8a 0 2024-07-16T09:55:37.045 INFO:tasks.workunit.client.0.smithi156.stdout:5/404: chown d1b/c3a 26113889 1 2024-07-16T09:55:37.047 INFO:tasks.workunit.client.1.smithi176.stdout:0/436: write d5/d1c/f21 [651290,59619] 0 2024-07-16T09:55:37.047 INFO:tasks.workunit.client.1.smithi176.stdout:2/372: truncate dd/de/f10 744141 0 2024-07-16T09:55:37.048 INFO:tasks.workunit.client.1.smithi176.stdout:0/437: write f3 [4051883,124113] 0 2024-07-16T09:55:37.049 INFO:tasks.workunit.client.1.smithi176.stdout:0/438: readlink d5/l45 0 2024-07-16T09:55:37.050 INFO:tasks.workunit.client.1.smithi176.stdout:0/439: stat d5/d1c/d24/d26/f3d 0 2024-07-16T09:55:37.050 INFO:tasks.workunit.client.1.smithi176.stdout:2/373: creat dd/d49/d52/f79 x:0 0 0 2024-07-16T09:55:37.052 INFO:tasks.workunit.client.0.smithi156.stdout:5/405: rename d1b/d1f/d45/d52/d43/d85/f59 to d1b/d1f/d45/d7a/f8b 0 2024-07-16T09:55:37.052 INFO:tasks.workunit.client.1.smithi176.stdout:0/440: read d5/d1c/f20 [16740,115880] 0 2024-07-16T09:55:37.052 INFO:tasks.workunit.client.1.smithi176.stdout:0/441: chown d5/d1c/d24/d35/d50/d58 0 1 2024-07-16T09:55:37.052 INFO:tasks.workunit.client.0.smithi156.stdout:0/347: truncate d0/d1a/d4d/d25/f55 859860 0 2024-07-16T09:55:37.053 INFO:tasks.workunit.client.0.smithi156.stdout:5/406: write d1b/d1e/f48 [3233476,126589] 0 2024-07-16T09:55:37.053 INFO:tasks.workunit.client.0.smithi156.stdout:5/407: chown d1b/c3a 3 1 2024-07-16T09:55:37.053 INFO:tasks.workunit.client.1.smithi176.stdout:0/442: fdatasync d5/d1c/f1f 0 2024-07-16T09:55:37.053 INFO:tasks.workunit.client.0.smithi156.stdout:5/408: stat d1b/d1f/d45/d52/f35 0 2024-07-16T09:55:37.057 INFO:tasks.workunit.client.0.smithi156.stdout:5/409: mknod d1b/d1f/c8c 0 2024-07-16T09:55:37.060 INFO:tasks.workunit.client.1.smithi176.stdout:2/374: getdents dd/d49/d52 0 2024-07-16T09:55:37.061 INFO:tasks.workunit.client.1.smithi176.stdout:0/443: dwrite d5/d1c/d24/d35/d50/d58/f79 [4194304,4194304] 0 2024-07-16T09:55:37.063 INFO:tasks.workunit.client.1.smithi176.stdout:2/375: symlink dd/d49/l7a 0 2024-07-16T09:55:37.066 INFO:tasks.workunit.client.1.smithi176.stdout:0/444: rename d5/d1c/f3b to d5/d1c/d24/d35/f88 0 2024-07-16T09:55:37.068 INFO:tasks.workunit.client.0.smithi156.stdout:5/410: dwrite d1b/d1f/d45/d7a/f8b [0,4194304] 0 2024-07-16T09:55:37.073 INFO:tasks.workunit.client.0.smithi156.stdout:0/348: dwrite d0/d1a/d4d/f21 [0,4194304] 0 2024-07-16T09:55:37.076 INFO:tasks.workunit.client.0.smithi156.stdout:0/349: chown d0/d1a/d4d/d9/f26 811320008 1 2024-07-16T09:55:37.076 INFO:tasks.workunit.client.0.smithi156.stdout:3/284: sync 2024-07-16T09:55:37.076 INFO:tasks.workunit.client.0.smithi156.stdout:6/356: sync 2024-07-16T09:55:37.077 INFO:tasks.workunit.client.1.smithi176.stdout:2/376: truncate dd/d2e/f50 4063782 0 2024-07-16T09:55:37.078 INFO:tasks.workunit.client.0.smithi156.stdout:3/285: stat d8/da/d13/f20 0 2024-07-16T09:55:37.078 INFO:tasks.workunit.client.1.smithi176.stdout:2/377: dread - dd/d16/d39/d3d/f75 zero size 2024-07-16T09:55:37.079 INFO:tasks.workunit.client.0.smithi156.stdout:5/411: dwrite d1b/d1f/d45/d52/d43/d85/f6a [0,4194304] 0 2024-07-16T09:55:37.080 INFO:tasks.workunit.client.0.smithi156.stdout:0/350: rmdir d0/d1a/d4d/d9/d27/d3d 39 2024-07-16T09:55:37.081 INFO:tasks.workunit.client.1.smithi176.stdout:0/445: dread d5/f6 [0,4194304] 0 2024-07-16T09:55:37.082 INFO:tasks.workunit.client.1.smithi176.stdout:2/378: rename dd/d2e/l6a to dd/d49/l7b 0 2024-07-16T09:55:37.085 INFO:tasks.workunit.client.0.smithi156.stdout:6/357: rename d3/d13/d17/d4f/d1f to d3/d13/d75 0 2024-07-16T09:55:37.086 INFO:tasks.workunit.client.0.smithi156.stdout:6/358: chown d3/d13/d17/d4f/d5c 1253970495 1 2024-07-16T09:55:37.087 INFO:tasks.workunit.client.0.smithi156.stdout:3/286: mknod d8/c5f 0 2024-07-16T09:55:37.087 INFO:tasks.workunit.client.0.smithi156.stdout:6/359: chown d3/d13/d17/d4f/d4d/d3b 657915041 1 2024-07-16T09:55:37.088 INFO:tasks.workunit.client.0.smithi156.stdout:6/360: chown d3/d13/d17/d4f/d4d/d3b/f5d 3922 1 2024-07-16T09:55:37.089 INFO:tasks.workunit.client.1.smithi176.stdout:2/379: dread dd/d16/d39/f6f [0,4194304] 0 2024-07-16T09:55:37.089 INFO:tasks.workunit.client.0.smithi156.stdout:5/412: creat d1b/d1f/d45/d52/d43/f8d x:0 0 0 2024-07-16T09:55:37.090 INFO:tasks.workunit.client.0.smithi156.stdout:6/361: fdatasync d3/d13/d17/d4f/d30/f67 0 2024-07-16T09:55:37.090 INFO:tasks.workunit.client.1.smithi176.stdout:2/380: write dd/d2e/f37 [1256449,73781] 0 2024-07-16T09:55:37.092 INFO:tasks.workunit.client.0.smithi156.stdout:3/287: rmdir d8/da 39 2024-07-16T09:55:37.092 INFO:tasks.workunit.client.1.smithi176.stdout:2/381: write dd/d16/d39/d3d/f75 [612782,22015] 0 2024-07-16T09:55:37.094 INFO:tasks.workunit.client.1.smithi176.stdout:2/382: mkdir dd/d49/d52/d7c 0 2024-07-16T09:55:37.094 INFO:tasks.workunit.client.0.smithi156.stdout:0/351: mkdir d0/d22/d41/d66 0 2024-07-16T09:55:37.095 INFO:tasks.workunit.client.1.smithi176.stdout:2/383: read - dd/d16/d39/d3d/f76 zero size 2024-07-16T09:55:37.096 INFO:tasks.workunit.client.0.smithi156.stdout:6/362: unlink d3/d13/d17/d4f/f4b 0 2024-07-16T09:55:37.098 INFO:tasks.workunit.client.0.smithi156.stdout:3/288: dread - d8/da/d13/f57 zero size 2024-07-16T09:55:37.098 INFO:tasks.workunit.client.1.smithi176.stdout:2/384: creat dd/d16/f7d x:0 0 0 2024-07-16T09:55:37.099 INFO:tasks.workunit.client.1.smithi176.stdout:0/446: write d5/d1c/d24/d35/d50/f62 [311320,112889] 0 2024-07-16T09:55:37.099 INFO:tasks.workunit.client.0.smithi156.stdout:3/289: truncate d8/d14/f55 505082 0 2024-07-16T09:55:37.099 INFO:tasks.workunit.client.1.smithi176.stdout:2/385: stat dd/d36/l23 0 2024-07-16T09:55:37.099 INFO:tasks.workunit.client.1.smithi176.stdout:0/447: fsync f4 0 2024-07-16T09:55:37.100 INFO:tasks.workunit.client.0.smithi156.stdout:5/413: link d1b/d1f/d42/c6b d1b/d1e/c8e 0 2024-07-16T09:55:37.101 INFO:tasks.workunit.client.1.smithi176.stdout:2/386: mkdir dd/d16/d39/d3d/d7e 0 2024-07-16T09:55:37.102 INFO:tasks.workunit.client.1.smithi176.stdout:2/387: dread - dd/d16/f7d zero size 2024-07-16T09:55:37.102 INFO:tasks.workunit.client.0.smithi156.stdout:0/352: creat d0/d1a/d4d/d9/d27/d3d/d62/f67 x:0 0 0 2024-07-16T09:55:37.103 INFO:tasks.workunit.client.1.smithi176.stdout:0/448: creat d5/d1c/f89 x:0 0 0 2024-07-16T09:55:37.104 INFO:tasks.workunit.client.1.smithi176.stdout:2/388: fdatasync dd/d16/d39/f6f 0 2024-07-16T09:55:37.104 INFO:tasks.workunit.client.1.smithi176.stdout:2/389: stat dd/d49 0 2024-07-16T09:55:37.104 INFO:tasks.workunit.client.1.smithi176.stdout:0/449: symlink d5/d1c/d24/d35/d59/l8a 0 2024-07-16T09:55:37.106 INFO:tasks.workunit.client.1.smithi176.stdout:0/450: symlink d5/d1c/d5a/l8b 0 2024-07-16T09:55:37.106 INFO:tasks.workunit.client.0.smithi156.stdout:5/414: creat d1b/d1f/d45/d52/d43/d85/f8f x:0 0 0 2024-07-16T09:55:37.107 INFO:tasks.workunit.client.1.smithi176.stdout:2/390: creat dd/d16/d39/d3d/d7e/f7f x:0 0 0 2024-07-16T09:55:37.107 INFO:tasks.workunit.client.1.smithi176.stdout:0/451: truncate d5/d1c/f20 1733617 0 2024-07-16T09:55:37.108 INFO:tasks.workunit.client.0.smithi156.stdout:0/353: write d0/d1a/d4d/d25/f3a [470569,86913] 0 2024-07-16T09:55:37.108 INFO:tasks.workunit.client.1.smithi176.stdout:0/452: write d5/d2c/d6c/f64 [1019509,34833] 0 2024-07-16T09:55:37.109 INFO:tasks.workunit.client.1.smithi176.stdout:2/391: creat dd/d16/d39/d3d/f80 x:0 0 0 2024-07-16T09:55:37.110 INFO:tasks.workunit.client.1.smithi176.stdout:2/392: stat dd/d2e/d35/d4a/c6c 0 2024-07-16T09:55:37.113 INFO:tasks.workunit.client.0.smithi156.stdout:0/354: dwrite d0/d1a/d4d/f11 [0,4194304] 0 2024-07-16T09:55:37.127 INFO:tasks.workunit.client.0.smithi156.stdout:6/363: creat d3/d13/d17/d4f/d4d/f76 x:0 0 0 2024-07-16T09:55:37.128 INFO:tasks.workunit.client.1.smithi176.stdout:2/393: write dd/d16/d39/f4b [1348361,47602] 0 2024-07-16T09:55:37.130 INFO:tasks.workunit.client.1.smithi176.stdout:2/394: write dd/d49/d52/f79 [248446,114804] 0 2024-07-16T09:55:37.132 INFO:tasks.workunit.client.0.smithi156.stdout:6/364: mkdir d3/d13/d17/d4f/d4d/d3b/d77 0 2024-07-16T09:55:37.133 INFO:tasks.workunit.client.1.smithi176.stdout:2/395: mkdir dd/d49/d52/d7c/d81 0 2024-07-16T09:55:37.134 INFO:tasks.workunit.client.0.smithi156.stdout:0/355: mkdir d0/d1a/d4d/d9/d68 0 2024-07-16T09:55:37.134 INFO:tasks.workunit.client.1.smithi176.stdout:0/453: dwrite d5/d2c/f4d [0,4194304] 0 2024-07-16T09:55:37.135 INFO:tasks.workunit.client.0.smithi156.stdout:5/415: link d1b/d1e/f89 d1b/d1f/d45/d7f/f90 0 2024-07-16T09:55:37.135 INFO:tasks.workunit.client.1.smithi176.stdout:0/454: fdatasync d5/d1c/f21 0 2024-07-16T09:55:37.136 INFO:tasks.workunit.client.1.smithi176.stdout:2/396: truncate dd/f48 149390 0 2024-07-16T09:55:37.138 INFO:tasks.workunit.client.0.smithi156.stdout:0/356: unlink d0/d1a/d4d/d9/d27/d50/l51 0 2024-07-16T09:55:37.138 INFO:tasks.workunit.client.0.smithi156.stdout:5/416: mknod d1b/d1f/d54/c91 0 2024-07-16T09:55:37.140 INFO:tasks.workunit.client.1.smithi176.stdout:0/455: creat d5/d1c/d24/d35/d59/f8c x:0 0 0 2024-07-16T09:55:37.140 INFO:tasks.workunit.client.1.smithi176.stdout:0/456: readlink d5/d1c/d24/d26/l6b 0 2024-07-16T09:55:37.142 INFO:tasks.workunit.client.1.smithi176.stdout:0/457: mknod d5/d39/c8d 0 2024-07-16T09:55:37.143 INFO:tasks.workunit.client.0.smithi156.stdout:5/417: symlink d1b/d1f/d42/d6d/l92 0 2024-07-16T09:55:37.145 INFO:tasks.workunit.client.0.smithi156.stdout:0/357: symlink d0/d1a/d4d/d25/l69 0 2024-07-16T09:55:37.145 INFO:tasks.workunit.client.1.smithi176.stdout:2/397: write dd/d16/f1d [2099178,120533] 0 2024-07-16T09:55:37.146 INFO:tasks.workunit.client.0.smithi156.stdout:3/290: dwrite d8/da/d1d/f53 [0,4194304] 0 2024-07-16T09:55:37.148 INFO:tasks.workunit.client.1.smithi176.stdout:2/398: truncate dd/d49/f55 3365227 0 2024-07-16T09:55:37.148 INFO:tasks.workunit.client.0.smithi156.stdout:3/291: fsync f3 0 2024-07-16T09:55:37.148 INFO:tasks.workunit.client.0.smithi156.stdout:6/365: dwrite d3/f25 [4194304,4194304] 0 2024-07-16T09:55:37.150 INFO:tasks.workunit.client.1.smithi176.stdout:2/399: write dd/d36/f61 [987777,113339] 0 2024-07-16T09:55:37.152 INFO:tasks.workunit.client.0.smithi156.stdout:5/418: rename d1b/d1f/d54/c7e to d1b/d1f/d45/d52/d43/d85/c93 0 2024-07-16T09:55:37.153 INFO:tasks.workunit.client.1.smithi176.stdout:0/458: dwrite d5/d2c/f2d [4194304,4194304] 0 2024-07-16T09:55:37.156 INFO:tasks.workunit.client.0.smithi156.stdout:0/358: fsync d0/d1a/d4d/d25/f55 0 2024-07-16T09:55:37.158 INFO:tasks.workunit.client.1.smithi176.stdout:0/459: creat d5/d1c/f8e x:0 0 0 2024-07-16T09:55:37.160 INFO:tasks.workunit.client.0.smithi156.stdout:5/419: dwrite d1b/d1f/d42/f47 [4194304,4194304] 0 2024-07-16T09:55:37.161 INFO:tasks.workunit.client.1.smithi176.stdout:0/460: creat d5/d25/d82/f8f x:0 0 0 2024-07-16T09:55:37.165 INFO:tasks.workunit.client.0.smithi156.stdout:6/366: creat d3/d13/d17/f78 x:0 0 0 2024-07-16T09:55:37.166 INFO:tasks.workunit.client.1.smithi176.stdout:0/461: link d5/c1a d5/d2c/d6c/c90 0 2024-07-16T09:55:37.167 INFO:tasks.workunit.client.0.smithi156.stdout:3/292: creat d8/da/d28/f60 x:0 0 0 2024-07-16T09:55:37.168 INFO:tasks.workunit.client.0.smithi156.stdout:3/293: write d8/da/d21/f3e [2346555,22965] 0 2024-07-16T09:55:37.168 INFO:tasks.workunit.client.0.smithi156.stdout:6/367: symlink d3/d13/d17/d4f/d5c/l79 0 2024-07-16T09:55:37.171 INFO:tasks.workunit.client.0.smithi156.stdout:0/359: creat d0/d1a/d4d/d9/d27/f6a x:0 0 0 2024-07-16T09:55:37.174 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:55:36 smithi156 ceph-mon[29302]: from='client.14638 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-16T09:55:37.174 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:55:36 smithi156 ceph-mon[29302]: from='client.? 172.21.15.156:0/3729096851' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-07-16T09:55:37.175 INFO:tasks.workunit.client.1.smithi176.stdout:0/462: dwrite d5/d1c/d24/d35/d50/d58/f79 [0,4194304] 0 2024-07-16T09:55:37.178 INFO:tasks.workunit.client.0.smithi156.stdout:6/368: dwrite d3/d13/d17/d4f/d4d/d3b/d5a/f69 [0,4194304] 0 2024-07-16T09:55:37.178 INFO:tasks.workunit.client.0.smithi156.stdout:5/420: dread d1b/d1f/d45/d52/d43/f5b [0,4194304] 0 2024-07-16T09:55:37.178 INFO:tasks.workunit.client.0.smithi156.stdout:0/360: rename d0/f5b to d0/d1a/d4d/f6b 0 2024-07-16T09:55:37.181 INFO:tasks.workunit.client.0.smithi156.stdout:0/361: mknod d0/d1a/d4d/d25/c6c 0 2024-07-16T09:55:37.183 INFO:tasks.workunit.client.0.smithi156.stdout:6/369: rename d3/c11 to d3/d13/d75/d19/d31/d35/c7a 0 2024-07-16T09:55:37.185 INFO:tasks.workunit.client.0.smithi156.stdout:6/370: dread - d3/d13/d75/d19/f66 zero size 2024-07-16T09:55:37.188 INFO:tasks.workunit.client.0.smithi156.stdout:5/421: dread d1b/d1f/d45/d7a/f8b [0,4194304] 0 2024-07-16T09:55:37.193 INFO:tasks.workunit.client.0.smithi156.stdout:5/422: unlink d1b/f7b 0 2024-07-16T09:55:37.194 INFO:tasks.workunit.client.0.smithi156.stdout:0/362: dwrite d0/d1a/d4d/d9/d27/d3d/d62/f67 [0,4194304] 0 2024-07-16T09:55:37.195 INFO:tasks.workunit.client.1.smithi176.stdout:2/400: dwrite dd/d16/f3a [4194304,4194304] 0 2024-07-16T09:55:37.197 INFO:tasks.workunit.client.0.smithi156.stdout:6/371: dwrite d3/d13/d75/d19/d2e/f3e [0,4194304] 0 2024-07-16T09:55:37.197 INFO:tasks.workunit.client.1.smithi176.stdout:0/463: dwrite d5/d1c/d24/d35/d50/d58/d63/f66 [0,4194304] 0 2024-07-16T09:55:37.198 INFO:tasks.workunit.client.1.smithi176.stdout:2/401: chown dd/d49/d52/f79 502341 1 2024-07-16T09:55:37.200 INFO:tasks.workunit.client.0.smithi156.stdout:3/294: creat d8/da/d28/d43/f61 x:0 0 0 2024-07-16T09:55:37.202 INFO:tasks.workunit.client.1.smithi176.stdout:0/464: symlink d5/d1c/d24/d26/l91 0 2024-07-16T09:55:37.202 INFO:tasks.workunit.client.0.smithi156.stdout:6/372: mknod d3/d13/d17/d4f/d4d/d3b/d77/c7b 0 2024-07-16T09:55:37.202 INFO:tasks.workunit.client.0.smithi156.stdout:0/363: dread d0/d1a/d4d/d9/d27/d3d/f37 [0,4194304] 0 2024-07-16T09:55:37.203 INFO:tasks.workunit.client.1.smithi176.stdout:2/402: dread dd/d16/d39/d3d/f75 [0,4194304] 0 2024-07-16T09:55:37.203 INFO:tasks.workunit.client.0.smithi156.stdout:3/295: stat d8/da/d28/d40 0 2024-07-16T09:55:37.203 INFO:tasks.workunit.client.1.smithi176.stdout:2/403: chown dd/d36/d45 11971 1 2024-07-16T09:55:37.205 INFO:tasks.workunit.client.1.smithi176.stdout:0/465: unlink d5/d1c/d24/d26/f3d 0 2024-07-16T09:55:37.212 INFO:tasks.workunit.client.1.smithi176.stdout:2/404: getdents dd/d16/d39 0 2024-07-16T09:55:37.213 INFO:tasks.workunit.client.1.smithi176.stdout:0/466: dread d5/d2c/f3f [0,4194304] 0 2024-07-16T09:55:37.214 INFO:tasks.workunit.client.0.smithi156.stdout:6/373: rename d3/f2a to d3/d13/d17/f7c 0 2024-07-16T09:55:37.214 INFO:tasks.workunit.client.1.smithi176.stdout:2/405: write dd/d36/f2f [461758,71418] 0 2024-07-16T09:55:37.215 INFO:tasks.workunit.client.0.smithi156.stdout:5/423: dread d1b/d1e/f3d [0,4194304] 0 2024-07-16T09:55:37.215 INFO:tasks.workunit.client.0.smithi156.stdout:5/424: chown d1b/d1e/f48 18 1 2024-07-16T09:55:37.216 INFO:tasks.workunit.client.1.smithi176.stdout:2/406: write dd/d16/f1d [323287,111212] 0 2024-07-16T09:55:37.216 INFO:tasks.workunit.client.1.smithi176.stdout:0/467: unlink d5/d2c/d6c/l78 0 2024-07-16T09:55:37.217 INFO:tasks.workunit.client.1.smithi176.stdout:0/468: stat d5/ca 0 2024-07-16T09:55:37.218 INFO:tasks.workunit.client.1.smithi176.stdout:2/407: mkdir dd/d36/d82 0 2024-07-16T09:55:37.220 INFO:tasks.workunit.client.1.smithi176.stdout:0/469: dread d5/d1c/d24/d35/d50/f62 [0,4194304] 0 2024-07-16T09:55:37.222 INFO:tasks.workunit.client.0.smithi156.stdout:6/374: dwrite d3/d13/d75/fa [0,4194304] 0 2024-07-16T09:55:37.222 INFO:tasks.workunit.client.1.smithi176.stdout:0/470: mkdir d5/d39/d92 0 2024-07-16T09:55:37.223 INFO:tasks.workunit.client.0.smithi156.stdout:6/375: chown d3/f25 555297430 1 2024-07-16T09:55:37.224 INFO:tasks.workunit.client.0.smithi156.stdout:5/425: dwrite d1b/d1f/d45/d52/d43/d85/f8f [0,4194304] 0 2024-07-16T09:55:37.228 INFO:tasks.workunit.client.1.smithi176.stdout:0/471: creat d5/d1c/d24/d35/d50/d7c/f93 x:0 0 0 2024-07-16T09:55:37.229 INFO:tasks.workunit.client.1.smithi176.stdout:0/472: chown d5/d44/f55 97 1 2024-07-16T09:55:37.229 INFO:tasks.workunit.client.0.smithi156.stdout:5/426: dread d1b/d1f/d45/f50 [0,4194304] 0 2024-07-16T09:55:37.231 INFO:tasks.workunit.client.1.smithi176.stdout:0/473: creat d5/d1c/d24/d35/f94 x:0 0 0 2024-07-16T09:55:37.232 INFO:tasks.workunit.client.0.smithi156.stdout:5/427: mkdir d1b/d1f/d45/d7a/d94 0 2024-07-16T09:55:37.232 INFO:tasks.workunit.client.0.smithi156.stdout:6/376: link d3/d13/c48 d3/d13/c7d 0 2024-07-16T09:55:37.233 INFO:tasks.workunit.client.0.smithi156.stdout:6/377: stat d3/d13/d17/d4f/d1e 0 2024-07-16T09:55:37.234 INFO:tasks.workunit.client.0.smithi156.stdout:7/326: sync 2024-07-16T09:55:37.236 INFO:tasks.workunit.client.0.smithi156.stdout:5/428: truncate d1b/d1e/f89 1016288 0 2024-07-16T09:55:37.236 INFO:tasks.workunit.client.1.smithi176.stdout:0/474: creat d5/d1c/f95 x:0 0 0 2024-07-16T09:55:37.237 INFO:tasks.workunit.client.0.smithi156.stdout:5/429: readlink d1b/d1f/l29 0 2024-07-16T09:55:37.237 INFO:tasks.workunit.client.1.smithi176.stdout:9/437: sync 2024-07-16T09:55:37.237 INFO:tasks.workunit.client.1.smithi176.stdout:3/321: sync 2024-07-16T09:55:37.237 INFO:tasks.workunit.client.1.smithi176.stdout:4/423: sync 2024-07-16T09:55:37.237 INFO:tasks.workunit.client.1.smithi176.stdout:6/396: sync 2024-07-16T09:55:37.238 INFO:tasks.workunit.client.1.smithi176.stdout:0/475: creat d5/d44/f96 x:0 0 0 2024-07-16T09:55:37.238 INFO:tasks.workunit.client.0.smithi156.stdout:7/327: mkdir dc/d3e/d76 0 2024-07-16T09:55:37.239 INFO:tasks.workunit.client.0.smithi156.stdout:7/328: dread - dc/d13/d36/f66 zero size 2024-07-16T09:55:37.239 INFO:tasks.workunit.client.0.smithi156.stdout:6/378: mknod d3/d13/d17/d4f/d30/d45/c7e 0 2024-07-16T09:55:37.240 INFO:tasks.workunit.client.1.smithi176.stdout:3/322: chown d3/da/d33/c45 3970295 1 2024-07-16T09:55:37.240 INFO:tasks.workunit.client.0.smithi156.stdout:5/430: symlink d1b/d1f/d45/d52/d43/d7c/l95 0 2024-07-16T09:55:37.240 INFO:tasks.workunit.client.1.smithi176.stdout:4/424: write da/dc/d10/f59 [3765920,47671] 0 2024-07-16T09:55:37.241 INFO:tasks.workunit.client.0.smithi156.stdout:7/329: rmdir dc/d3e/d3f 39 2024-07-16T09:55:37.241 INFO:tasks.workunit.client.1.smithi176.stdout:3/323: readlink d3/d17/l3e 0 2024-07-16T09:55:37.242 INFO:tasks.workunit.client.1.smithi176.stdout:4/425: fsync da/d3d/d33/f3a 0 2024-07-16T09:55:37.244 INFO:tasks.workunit.client.1.smithi176.stdout:6/397: mknod d1/dc/d72/d6f/d7c/c92 0 2024-07-16T09:55:37.245 INFO:tasks.workunit.client.0.smithi156.stdout:7/330: symlink dc/d13/d20/d4d/l77 0 2024-07-16T09:55:37.245 INFO:tasks.workunit.client.0.smithi156.stdout:3/296: truncate d8/da/f3d 604259 0 2024-07-16T09:55:37.246 INFO:tasks.workunit.client.1.smithi176.stdout:3/324: mknod d3/da/d56/c5c 0 2024-07-16T09:55:37.246 INFO:tasks.workunit.client.0.smithi156.stdout:7/331: write dc/f40 [891303,15752] 0 2024-07-16T09:55:37.246 INFO:tasks.workunit.client.1.smithi176.stdout:9/438: read d5/de/d21/f39 [12094819,121174] 0 2024-07-16T09:55:37.247 INFO:tasks.workunit.client.0.smithi156.stdout:3/297: write d8/da/d13/f5c [346385,93552] 0 2024-07-16T09:55:37.248 INFO:tasks.workunit.client.1.smithi176.stdout:4/426: symlink da/d3d/d17/d54/d5e/d69/l7d 0 2024-07-16T09:55:37.248 INFO:tasks.workunit.client.1.smithi176.stdout:9/439: write d5/f10 [2203837,29654] 0 2024-07-16T09:55:37.248 INFO:tasks.workunit.client.0.smithi156.stdout:3/298: truncate d8/da/f5d 963047 0 2024-07-16T09:55:37.250 INFO:tasks.workunit.client.1.smithi176.stdout:3/325: mknod d3/da/d33/c5d 0 2024-07-16T09:55:37.251 INFO:tasks.workunit.client.1.smithi176.stdout:9/440: rmdir d5/de/d37 39 2024-07-16T09:55:37.251 INFO:tasks.workunit.client.1.smithi176.stdout:0/476: getdents d5/d1c/d5a 0 2024-07-16T09:55:37.254 INFO:tasks.workunit.client.0.smithi156.stdout:7/332: dread fb [0,4194304] 0 2024-07-16T09:55:37.255 INFO:tasks.workunit.client.1.smithi176.stdout:6/398: dwrite d1/d5/d47/f86 [0,4194304] 0 2024-07-16T09:55:37.255 INFO:tasks.workunit.client.1.smithi176.stdout:0/477: write d5/d1c/f21 [1354801,7561] 0 2024-07-16T09:55:37.256 INFO:tasks.workunit.client.0.smithi156.stdout:5/431: truncate d1b/d1f/f73 977 0 2024-07-16T09:55:37.258 INFO:tasks.workunit.client.0.smithi156.stdout:3/299: dwrite d8/da/d13/f17 [0,4194304] 0 2024-07-16T09:55:37.259 INFO:tasks.workunit.client.1.smithi176.stdout:4/427: rename da/d3d/d33/f6f to da/d3d/d17/d54/d5e/d69/f7e 0 2024-07-16T09:55:37.259 INFO:tasks.workunit.client.1.smithi176.stdout:9/441: unlink d5/d1d/l2e 0 2024-07-16T09:55:37.260 INFO:tasks.workunit.client.0.smithi156.stdout:5/432: write d1b/d1f/d45/d52/d43/d85/f75 [487671,45008] 0 2024-07-16T09:55:37.261 INFO:tasks.workunit.client.1.smithi176.stdout:3/326: mknod d3/d17/d1d/c5e 0 2024-07-16T09:55:37.261 INFO:tasks.workunit.client.0.smithi156.stdout:7/333: creat dc/d13/d15/f78 x:0 0 0 2024-07-16T09:55:37.263 INFO:tasks.workunit.client.1.smithi176.stdout:3/327: fdatasync d3/da/d33/f44 0 2024-07-16T09:55:37.264 INFO:tasks.workunit.client.0.smithi156.stdout:5/433: rename d1b/d1f/d45/d52/d2d/d86 to d1b/d1f/d45/d52/d2d/d96 0 2024-07-16T09:55:37.264 INFO:tasks.workunit.client.0.smithi156.stdout:5/434: readlink d1b/l4b 0 2024-07-16T09:55:37.266 INFO:tasks.workunit.client.1.smithi176.stdout:3/328: read - d3/d17/d1d/f4b zero size 2024-07-16T09:55:37.270 INFO:tasks.workunit.client.1.smithi176.stdout:9/442: mknod d5/de/d37/d58/c90 0 2024-07-16T09:55:37.271 INFO:tasks.workunit.client.0.smithi156.stdout:3/300: rename d8/da/d1d/d2c/d33/f50 to d8/f62 0 2024-07-16T09:55:37.271 INFO:tasks.workunit.client.1.smithi176.stdout:0/478: dread d5/f2a [0,4194304] 0 2024-07-16T09:55:37.272 INFO:tasks.workunit.client.0.smithi156.stdout:5/435: dread - d1b/d1e/d37/d88/d61/f69 zero size 2024-07-16T09:55:37.273 INFO:tasks.workunit.client.1.smithi176.stdout:9/443: readlink d5/de/d30/l63 0 2024-07-16T09:55:37.274 INFO:tasks.workunit.client.0.smithi156.stdout:3/301: mkdir d8/da/d28/d43/d63 0 2024-07-16T09:55:37.275 INFO:tasks.workunit.client.1.smithi176.stdout:4/428: dwrite da/d3d/d17/f18 [4194304,4194304] 0 2024-07-16T09:55:37.275 INFO:tasks.workunit.client.1.smithi176.stdout:0/479: readlink d5/d25/l5d 0 2024-07-16T09:55:37.276 INFO:tasks.workunit.client.0.smithi156.stdout:5/436: rename d1b/d1e to d1b/d1f/d45/d52/d43/d85/d97 0 2024-07-16T09:55:37.276 INFO:tasks.workunit.client.1.smithi176.stdout:6/399: dwrite d1/d5/d7/f6e [0,4194304] 0 2024-07-16T09:55:37.279 INFO:tasks.workunit.client.0.smithi156.stdout:5/437: rmdir d1b/d1f/d45/d7a 39 2024-07-16T09:55:37.280 INFO:tasks.workunit.client.1.smithi176.stdout:9/444: creat d5/d1d/d20/d8d/f91 x:0 0 0 2024-07-16T09:55:37.281 INFO:tasks.workunit.client.1.smithi176.stdout:0/480: unlink d5/d1c/d24/f34 0 2024-07-16T09:55:37.282 INFO:tasks.workunit.client.0.smithi156.stdout:3/302: creat d8/da/d1d/f64 x:0 0 0 2024-07-16T09:55:37.283 INFO:tasks.workunit.client.0.smithi156.stdout:5/438: stat d1b/d1f/d45/d7f/f84 0 2024-07-16T09:55:37.285 INFO:tasks.workunit.client.0.smithi156.stdout:5/439: dread - d1b/d1f/d42/f5c zero size 2024-07-16T09:55:37.285 INFO:tasks.workunit.client.1.smithi176.stdout:4/429: creat da/dc/d10/d28/f7f x:0 0 0 2024-07-16T09:55:37.286 INFO:tasks.workunit.client.0.smithi156.stdout:3/303: unlink d8/da/d28/d40/l46 0 2024-07-16T09:55:37.286 INFO:tasks.workunit.client.1.smithi176.stdout:6/400: symlink d1/d5/d7/d54/l93 0 2024-07-16T09:55:37.290 INFO:tasks.workunit.client.1.smithi176.stdout:4/430: creat da/d3d/d17/d48/f80 x:0 0 0 2024-07-16T09:55:37.290 INFO:tasks.workunit.client.0.smithi156.stdout:3/304: creat d8/da/d1d/d2c/d33/f65 x:0 0 0 2024-07-16T09:55:37.291 INFO:tasks.workunit.client.1.smithi176.stdout:9/445: rename d5/de/f88 to d5/de/f92 0 2024-07-16T09:55:37.293 INFO:tasks.workunit.client.1.smithi176.stdout:4/431: mknod da/dc/d10/d28/c81 0 2024-07-16T09:55:37.295 INFO:tasks.workunit.client.1.smithi176.stdout:9/446: truncate d5/de/d37/d58/d89/d8a/f4e 516050 0 2024-07-16T09:55:37.296 INFO:tasks.workunit.client.1.smithi176.stdout:4/432: mkdir da/d3d/d17/d48/d82 0 2024-07-16T09:55:37.297 INFO:tasks.workunit.client.1.smithi176.stdout:9/447: fsync d5/d1d/d20/d8d/f91 0 2024-07-16T09:55:37.298 INFO:tasks.workunit.client.0.smithi156.stdout:3/305: dwrite d8/da/d1d/d2c/d33/f65 [0,4194304] 0 2024-07-16T09:55:37.299 INFO:tasks.workunit.client.1.smithi176.stdout:9/448: read d5/de/f12 [3451872,127750] 0 2024-07-16T09:55:37.302 INFO:tasks.workunit.client.1.smithi176.stdout:3/329: truncate d3/d17/d1d/d4f/d59/f34 135946 0 2024-07-16T09:55:37.302 INFO:tasks.workunit.client.1.smithi176.stdout:0/481: write d5/d1c/d24/d35/d50/f7f [221684,29751] 0 2024-07-16T09:55:37.305 INFO:tasks.workunit.client.1.smithi176.stdout:6/401: write d1/f15 [923466,103041] 0 2024-07-16T09:55:37.305 INFO:tasks.workunit.client.0.smithi156.stdout:3/306: creat d8/da/d28/f66 x:0 0 0 2024-07-16T09:55:37.307 INFO:tasks.workunit.client.1.smithi176.stdout:6/402: write d1/f45 [1854876,116363] 0 2024-07-16T09:55:37.308 INFO:tasks.workunit.client.0.smithi156.stdout:5/440: dwrite d1b/d1f/d45/d52/f39 [0,4194304] 0 2024-07-16T09:55:37.308 INFO:tasks.workunit.client.0.smithi156.stdout:3/307: write d8/da/d13/f57 [289126,36970] 0 2024-07-16T09:55:37.309 INFO:tasks.workunit.client.1.smithi176.stdout:6/403: stat d1/d61/d65/d8f/f74 0 2024-07-16T09:55:37.311 INFO:tasks.workunit.client.1.smithi176.stdout:0/482: truncate f4 522449 0 2024-07-16T09:55:37.311 INFO:tasks.workunit.client.0.smithi156.stdout:5/441: stat d1b/d1f/d45/d52/d2d/l6c 0 2024-07-16T09:55:37.313 INFO:tasks.workunit.client.0.smithi156.stdout:3/308: truncate d8/da/f3d 1323979 0 2024-07-16T09:55:37.314 INFO:tasks.workunit.client.1.smithi176.stdout:3/330: rename d3/d17/d1d/d4f/d59/l35 to d3/da/d33/l5f 0 2024-07-16T09:55:37.314 INFO:tasks.workunit.client.1.smithi176.stdout:6/404: symlink d1/d5/d7/d1e/l94 0 2024-07-16T09:55:37.315 INFO:tasks.workunit.client.1.smithi176.stdout:9/449: chown d5/de/d37/d58/d89/d8a/f4e 24412593 1 2024-07-16T09:55:37.316 INFO:tasks.workunit.client.1.smithi176.stdout:0/483: fsync d5/d25/f4e 0 2024-07-16T09:55:37.317 INFO:tasks.workunit.client.1.smithi176.stdout:6/405: read d1/dc/d40/f4d [4704250,92007] 0 2024-07-16T09:55:37.317 INFO:tasks.workunit.client.1.smithi176.stdout:9/450: creat d5/d52/f93 x:0 0 0 2024-07-16T09:55:37.318 INFO:tasks.workunit.client.1.smithi176.stdout:3/331: symlink d3/d17/d1d/d4d/l60 0 2024-07-16T09:55:37.320 INFO:tasks.workunit.client.0.smithi156.stdout:3/309: rename d8/c5f to d8/da/d28/d43/d63/c67 0 2024-07-16T09:55:37.321 INFO:tasks.workunit.client.1.smithi176.stdout:3/332: dread d3/d17/f23 [0,4194304] 0 2024-07-16T09:55:37.321 INFO:tasks.workunit.client.0.smithi156.stdout:5/442: dwrite d1b/d1f/d45/d52/d43/d85/d97/d37/d88/d61/f69 [0,4194304] 0 2024-07-16T09:55:37.322 INFO:tasks.workunit.client.1.smithi176.stdout:3/333: readlink d3/d17/d1d/d4d/l36 0 2024-07-16T09:55:37.326 INFO:tasks.workunit.client.1.smithi176.stdout:0/484: rename d5/d2c/l72 to d5/d2c/d6c/d83/d84/l97 0 2024-07-16T09:55:37.326 INFO:tasks.workunit.client.0.smithi156.stdout:3/310: rename d8/da/d1d/d2c/l42 to d8/da/d21/l68 0 2024-07-16T09:55:37.328 INFO:tasks.workunit.client.1.smithi176.stdout:6/406: creat d1/d5/d73/f95 x:0 0 0 2024-07-16T09:55:37.330 INFO:tasks.workunit.client.1.smithi176.stdout:3/334: rmdir d3/da/d56 39 2024-07-16T09:55:37.331 INFO:tasks.workunit.client.0.smithi156.stdout:3/311: mkdir d8/da/d28/d43/d63/d69 0 2024-07-16T09:55:37.332 INFO:tasks.workunit.client.1.smithi176.stdout:0/485: rmdir d5/d1c/d24/d35/d50/d5b 39 2024-07-16T09:55:37.333 INFO:tasks.workunit.client.1.smithi176.stdout:0/486: chown d5/d39/c8d 784547391 1 2024-07-16T09:55:37.333 INFO:tasks.workunit.client.1.smithi176.stdout:0/487: stat d5/d1c/f89 0 2024-07-16T09:55:37.333 INFO:tasks.workunit.client.0.smithi156.stdout:3/312: symlink d8/da/d28/d40/l6a 0 2024-07-16T09:55:37.334 INFO:tasks.workunit.client.1.smithi176.stdout:6/407: creat d1/dc/d72/d6f/f96 x:0 0 0 2024-07-16T09:55:37.334 INFO:tasks.workunit.client.1.smithi176.stdout:0/488: write d5/d2c/f54 [1123755,51873] 0 2024-07-16T09:55:37.337 INFO:tasks.workunit.client.0.smithi156.stdout:3/313: rename d8/d14/f41 to d8/d14/f6b 0 2024-07-16T09:55:37.338 INFO:tasks.workunit.client.1.smithi176.stdout:9/451: dread d5/f1c [0,4194304] 0 2024-07-16T09:55:37.338 INFO:tasks.workunit.client.1.smithi176.stdout:6/408: creat d1/dc/f97 x:0 0 0 2024-07-16T09:55:37.339 INFO:tasks.workunit.client.0.smithi156.stdout:3/314: mknod d8/da/d28/d40/c6c 0 2024-07-16T09:55:37.341 INFO:tasks.workunit.client.1.smithi176.stdout:6/409: dread d1/f64 [0,4194304] 0 2024-07-16T09:55:37.342 INFO:tasks.workunit.client.1.smithi176.stdout:3/335: rename d3/d17/d1d/f4b to d3/f61 0 2024-07-16T09:55:37.343 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:55:36 smithi176 ceph-mon[36626]: from='client.14638 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-16T09:55:37.343 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:55:36 smithi176 ceph-mon[36626]: from='client.? 172.21.15.156:0/3729096851' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-07-16T09:55:37.344 INFO:tasks.workunit.client.0.smithi156.stdout:3/315: getdents d8/da/d28/d43/d63 0 2024-07-16T09:55:37.344 INFO:tasks.workunit.client.1.smithi176.stdout:0/489: mknod d5/d1c/d24/d26/c98 0 2024-07-16T09:55:37.345 INFO:tasks.workunit.client.0.smithi156.stdout:3/316: creat d8/da/d21/f6d x:0 0 0 2024-07-16T09:55:37.346 INFO:tasks.workunit.client.1.smithi176.stdout:3/336: symlink d3/d17/l62 0 2024-07-16T09:55:37.348 INFO:tasks.workunit.client.1.smithi176.stdout:0/490: symlink d5/d39/l99 0 2024-07-16T09:55:37.350 INFO:tasks.workunit.client.1.smithi176.stdout:6/410: creat d1/d5/f98 x:0 0 0 2024-07-16T09:55:37.350 INFO:tasks.workunit.client.0.smithi156.stdout:3/317: dwrite d8/da/d21/f3e [0,4194304] 0 2024-07-16T09:55:37.351 INFO:tasks.workunit.client.1.smithi176.stdout:9/452: dwrite d5/de/f15 [0,4194304] 0 2024-07-16T09:55:37.351 INFO:tasks.workunit.client.1.smithi176.stdout:6/411: read - d1/d5/d7/d1e/f79 zero size 2024-07-16T09:55:37.352 INFO:tasks.workunit.client.1.smithi176.stdout:6/412: fsync d1/f31 0 2024-07-16T09:55:37.352 INFO:tasks.workunit.client.1.smithi176.stdout:0/491: unlink d5/d1c/d24/d26/c98 0 2024-07-16T09:55:37.353 INFO:tasks.workunit.client.1.smithi176.stdout:3/337: creat d3/d17/d1d/f63 x:0 0 0 2024-07-16T09:55:37.353 INFO:tasks.workunit.client.1.smithi176.stdout:0/492: fsync d5/d39/f7e 0 2024-07-16T09:55:37.353 INFO:tasks.workunit.client.1.smithi176.stdout:6/413: chown d1/d5/f2c 39921637 1 2024-07-16T09:55:37.354 INFO:tasks.workunit.client.1.smithi176.stdout:3/338: stat d3/d17/f40 0 2024-07-16T09:55:37.356 INFO:tasks.workunit.client.1.smithi176.stdout:6/414: mknod d1/d61/c99 0 2024-07-16T09:55:37.357 INFO:tasks.workunit.client.0.smithi156.stdout:3/318: dwrite d8/da/f32 [0,4194304] 0 2024-07-16T09:55:37.358 INFO:tasks.workunit.client.1.smithi176.stdout:9/453: rename d5/d52/f55 to d5/de/d37/f94 0 2024-07-16T09:55:37.359 INFO:tasks.workunit.client.1.smithi176.stdout:3/339: mkdir d3/d17/d1d/d4d/d64 0 2024-07-16T09:55:37.359 INFO:tasks.workunit.client.0.smithi156.stdout:3/319: stat d8/da/d1d/d2c/d33/l45 0 2024-07-16T09:55:37.363 INFO:tasks.workunit.client.1.smithi176.stdout:0/493: dread d5/d2c/f4d [0,4194304] 0 2024-07-16T09:55:37.364 INFO:tasks.workunit.client.1.smithi176.stdout:9/454: symlink d5/d52/l95 0 2024-07-16T09:55:37.365 INFO:tasks.workunit.client.0.smithi156.stdout:3/320: dwrite d8/da/d21/f3e [4194304,4194304] 0 2024-07-16T09:55:37.371 INFO:tasks.workunit.client.1.smithi176.stdout:6/415: dwrite d1/d5/f2a [0,4194304] 0 2024-07-16T09:55:37.372 INFO:tasks.workunit.client.0.smithi156.stdout:3/321: creat d8/da/d28/f6e x:0 0 0 2024-07-16T09:55:37.372 INFO:tasks.workunit.client.1.smithi176.stdout:9/455: unlink d5/d52/l62 0 2024-07-16T09:55:37.372 INFO:tasks.workunit.client.1.smithi176.stdout:3/340: dwrite d3/f4a [0,4194304] 0 2024-07-16T09:55:37.373 INFO:tasks.workunit.client.1.smithi176.stdout:9/456: stat d5/d1d/d20 0 2024-07-16T09:55:37.373 INFO:tasks.workunit.client.1.smithi176.stdout:6/416: chown d1/dc/d40/c8c 4680 1 2024-07-16T09:55:37.374 INFO:tasks.workunit.client.1.smithi176.stdout:9/457: chown d5/de/d37 2 1 2024-07-16T09:55:37.375 INFO:tasks.workunit.client.1.smithi176.stdout:0/494: dread d5/f19 [0,4194304] 0 2024-07-16T09:55:37.376 INFO:tasks.workunit.client.0.smithi156.stdout:3/322: getdents d8/da/d13 0 2024-07-16T09:55:37.377 INFO:tasks.workunit.client.1.smithi176.stdout:6/417: creat d1/d5/d73/f9a x:0 0 0 2024-07-16T09:55:37.379 INFO:tasks.workunit.client.1.smithi176.stdout:0/495: creat d5/d25/d82/f9a x:0 0 0 2024-07-16T09:55:37.380 INFO:tasks.workunit.client.1.smithi176.stdout:9/458: link d5/d1d/d20/f25 d5/de/d21/f96 0 2024-07-16T09:55:37.380 INFO:tasks.workunit.client.0.smithi156.stdout:3/323: mkdir d8/da/d1d/d6f 0 2024-07-16T09:55:37.382 INFO:tasks.workunit.client.0.smithi156.stdout:3/324: readlink d8/da/d1d/d2c/d33/l45 0 2024-07-16T09:55:37.384 INFO:tasks.workunit.client.0.smithi156.stdout:3/325: fdatasync d8/f62 0 2024-07-16T09:55:37.385 INFO:tasks.workunit.client.1.smithi176.stdout:3/341: dwrite d3/da/d56/f5b [0,4194304] 0 2024-07-16T09:55:37.386 INFO:tasks.workunit.client.1.smithi176.stdout:3/342: readlink d3/d17/l1a 0 2024-07-16T09:55:37.386 INFO:tasks.workunit.client.1.smithi176.stdout:6/418: dread d1/d5/d7/f19 [0,4194304] 0 2024-07-16T09:55:37.388 INFO:tasks.workunit.client.1.smithi176.stdout:0/496: mkdir d5/d1c/d24/d35/d50/d9b 0 2024-07-16T09:55:37.388 INFO:tasks.workunit.client.0.smithi156.stdout:3/326: fdatasync d8/da/d13/f20 0 2024-07-16T09:55:37.389 INFO:tasks.workunit.client.1.smithi176.stdout:6/419: write d1/d5/d7/d2f/f85 [373638,61691] 0 2024-07-16T09:55:37.391 INFO:tasks.workunit.client.1.smithi176.stdout:0/497: dread d5/d1c/d5a/f75 [0,4194304] 0 2024-07-16T09:55:37.392 INFO:tasks.workunit.client.1.smithi176.stdout:6/420: rmdir d1/d5/d7/d54 39 2024-07-16T09:55:37.393 INFO:tasks.workunit.client.1.smithi176.stdout:6/421: dread - d1/dc/d5b/f6d zero size 2024-07-16T09:55:37.393 INFO:tasks.workunit.client.1.smithi176.stdout:0/498: symlink d5/d1c/d5a/l9c 0 2024-07-16T09:55:37.394 INFO:tasks.workunit.client.1.smithi176.stdout:0/499: chown d5/d1c 322 1 2024-07-16T09:55:37.396 INFO:tasks.workunit.client.1.smithi176.stdout:0/500: truncate d5/d1c/d24/d35/f4c 4308833 0 2024-07-16T09:55:37.396 INFO:tasks.workunit.client.1.smithi176.stdout:6/422: symlink d1/d5/d7/l9b 0 2024-07-16T09:55:37.396 INFO:tasks.workunit.client.1.smithi176.stdout:6/423: readlink d1/dc/l2b 0 2024-07-16T09:55:37.400 INFO:tasks.workunit.client.1.smithi176.stdout:0/501: mknod d5/d1c/d24/d35/d50/d5b/c9d 0 2024-07-16T09:55:37.403 INFO:tasks.workunit.client.1.smithi176.stdout:6/424: link d1/d5/d7/d54/c7d d1/d5/d7/d1e/c9c 0 2024-07-16T09:55:37.403 INFO:tasks.workunit.client.1.smithi176.stdout:6/425: stat d1/d61 0 2024-07-16T09:55:37.404 INFO:tasks.workunit.client.1.smithi176.stdout:0/502: getdents d5/d2c/d6c 0 2024-07-16T09:55:37.405 INFO:tasks.workunit.client.1.smithi176.stdout:6/426: creat d1/dc/d5b/f9d x:0 0 0 2024-07-16T09:55:37.406 INFO:tasks.workunit.client.1.smithi176.stdout:6/427: readlink d1/d5/d7/d1e/l4a 0 2024-07-16T09:55:37.408 INFO:tasks.workunit.client.1.smithi176.stdout:9/459: getdents d5/de/d21 0 2024-07-16T09:55:37.410 INFO:tasks.workunit.client.1.smithi176.stdout:3/343: write d3/d17/d1d/d4f/d59/f34 [437415,119828] 0 2024-07-16T09:55:37.410 INFO:tasks.workunit.client.1.smithi176.stdout:9/460: write d5/d1d/d20/f81 [361253,106196] 0 2024-07-16T09:55:37.411 INFO:tasks.workunit.client.1.smithi176.stdout:0/503: dread d5/d2c/f3f [0,4194304] 0 2024-07-16T09:55:37.411 INFO:tasks.workunit.client.1.smithi176.stdout:6/428: rename d1/d61/d65 to d1/d5d/d9e 0 2024-07-16T09:55:37.412 INFO:tasks.workunit.client.1.smithi176.stdout:0/504: chown d5/l45 782747473 1 2024-07-16T09:55:37.412 INFO:tasks.workunit.client.1.smithi176.stdout:6/429: readlink d1/d5/d7/d54/l93 0 2024-07-16T09:55:37.413 INFO:tasks.workunit.client.1.smithi176.stdout:3/344: symlink d3/da/d56/l65 0 2024-07-16T09:55:37.414 INFO:tasks.workunit.client.1.smithi176.stdout:9/461: mkdir d5/d1d/d77/d97 0 2024-07-16T09:55:37.414 INFO:tasks.workunit.client.1.smithi176.stdout:0/505: truncate d5/d25/d82/f8f 69851 0 2024-07-16T09:55:37.415 INFO:tasks.workunit.client.1.smithi176.stdout:3/345: mkdir d3/da/d66 0 2024-07-16T09:55:37.416 INFO:tasks.workunit.client.1.smithi176.stdout:0/506: mkdir d5/d1c/d5a/d9e 0 2024-07-16T09:55:37.416 INFO:tasks.workunit.client.1.smithi176.stdout:9/462: mkdir d5/d1d/d77/d98 0 2024-07-16T09:55:37.417 INFO:tasks.workunit.client.1.smithi176.stdout:3/346: creat d3/da/d33/f67 x:0 0 0 2024-07-16T09:55:37.420 INFO:tasks.workunit.client.1.smithi176.stdout:3/347: creat d3/d17/f68 x:0 0 0 2024-07-16T09:55:37.420 INFO:tasks.workunit.client.1.smithi176.stdout:6/430: dwrite d1/d5d/d9e/d8f/f8d [0,4194304] 0 2024-07-16T09:55:37.421 INFO:tasks.workunit.client.1.smithi176.stdout:3/348: fsync d3/f55 0 2024-07-16T09:55:37.421 INFO:tasks.workunit.client.1.smithi176.stdout:9/463: truncate d5/de/d30/f5e 967189 0 2024-07-16T09:55:37.423 INFO:tasks.workunit.client.1.smithi176.stdout:6/431: read d1/d5d/f53 [103837,95968] 0 2024-07-16T09:55:37.425 INFO:tasks.workunit.client.1.smithi176.stdout:3/349: rename d3/da/c3b to d3/da/d66/c69 0 2024-07-16T09:55:37.425 INFO:tasks.workunit.client.1.smithi176.stdout:6/432: truncate d1/dc/f97 780410 0 2024-07-16T09:55:37.427 INFO:tasks.workunit.client.1.smithi176.stdout:3/350: write d3/d17/f40 [1625068,83957] 0 2024-07-16T09:55:37.428 INFO:tasks.workunit.client.1.smithi176.stdout:0/507: write d5/d1c/f1f [977429,70922] 0 2024-07-16T09:55:37.429 INFO:tasks.workunit.client.1.smithi176.stdout:6/433: creat d1/dc/f9f x:0 0 0 2024-07-16T09:55:37.431 INFO:tasks.workunit.client.1.smithi176.stdout:3/351: mknod d3/d17/c6a 0 2024-07-16T09:55:37.432 INFO:tasks.workunit.client.1.smithi176.stdout:9/464: dwrite d5/de/d37/d58/d89/d8a/f82 [0,4194304] 0 2024-07-16T09:55:37.432 INFO:tasks.workunit.client.1.smithi176.stdout:0/508: write d5/d1c/d24/d35/f61 [3783999,96765] 0 2024-07-16T09:55:37.436 INFO:tasks.workunit.client.1.smithi176.stdout:3/352: fdatasync d3/d17/f23 0 2024-07-16T09:55:37.438 INFO:tasks.workunit.client.1.smithi176.stdout:0/509: chown d5/d1c/d24/c28 7152349 1 2024-07-16T09:55:37.443 INFO:tasks.workunit.client.1.smithi176.stdout:6/434: dwrite d1/d5d/d9e/d76/f5a [0,4194304] 0 2024-07-16T09:55:37.448 INFO:tasks.workunit.client.1.smithi176.stdout:0/510: write d5/f22 [1411300,20823] 0 2024-07-16T09:55:37.449 INFO:tasks.workunit.client.1.smithi176.stdout:9/465: dwrite d5/de/f40 [0,4194304] 0 2024-07-16T09:55:37.451 INFO:tasks.workunit.client.1.smithi176.stdout:6/435: symlink d1/d5/d47/d62/d77/la0 0 2024-07-16T09:55:37.452 INFO:tasks.workunit.client.1.smithi176.stdout:3/353: dwrite d3/d17/d1d/d4f/d59/f34 [0,4194304] 0 2024-07-16T09:55:37.452 INFO:tasks.workunit.client.1.smithi176.stdout:9/466: creat d5/d43/f99 x:0 0 0 2024-07-16T09:55:37.454 INFO:tasks.workunit.client.1.smithi176.stdout:0/511: truncate d5/d1c/d24/d35/d50/d58/f6f 945270 0 2024-07-16T09:55:37.456 INFO:tasks.workunit.client.1.smithi176.stdout:6/436: creat d1/d5/d47/d62/fa1 x:0 0 0 2024-07-16T09:55:37.456 INFO:tasks.workunit.client.1.smithi176.stdout:3/354: write d3/d17/d1d/d4d/f4e [1836116,40084] 0 2024-07-16T09:55:37.457 INFO:tasks.workunit.client.1.smithi176.stdout:9/467: creat d5/de/d30/f9a x:0 0 0 2024-07-16T09:55:37.458 INFO:tasks.workunit.client.1.smithi176.stdout:9/468: dread - d5/d43/f99 zero size 2024-07-16T09:55:37.459 INFO:tasks.workunit.client.1.smithi176.stdout:0/512: unlink d5/l48 0 2024-07-16T09:55:37.460 INFO:tasks.workunit.client.1.smithi176.stdout:0/513: chown d5/d44/l56 9587 1 2024-07-16T09:55:37.461 INFO:tasks.workunit.client.1.smithi176.stdout:6/437: creat d1/dc/d72/d6f/d7c/fa2 x:0 0 0 2024-07-16T09:55:37.461 INFO:tasks.workunit.client.1.smithi176.stdout:3/355: read d3/da/fe [897364,38731] 0 2024-07-16T09:55:37.462 INFO:tasks.workunit.client.1.smithi176.stdout:9/469: symlink d5/d1d/d67/l9b 0 2024-07-16T09:55:37.463 INFO:tasks.workunit.client.1.smithi176.stdout:3/356: stat d3/d17/d1d/d4d/l48 0 2024-07-16T09:55:37.464 INFO:tasks.workunit.client.1.smithi176.stdout:3/357: truncate d3/f61 607548 0 2024-07-16T09:55:37.465 INFO:tasks.workunit.client.1.smithi176.stdout:9/470: creat d5/d1d/d20/d8d/f9c x:0 0 0 2024-07-16T09:55:37.466 INFO:tasks.workunit.client.1.smithi176.stdout:6/438: rename d1/d5/la to d1/d5d/d9e/d8f/la3 0 2024-07-16T09:55:37.468 INFO:tasks.workunit.client.1.smithi176.stdout:3/358: rename d3/d17/d1d/d4d/l36 to d3/da/d33/l6b 0 2024-07-16T09:55:37.469 INFO:tasks.workunit.client.1.smithi176.stdout:3/359: write d3/f55 [246823,64054] 0 2024-07-16T09:55:37.470 INFO:tasks.workunit.client.1.smithi176.stdout:9/471: read d5/de/d37/d58/f5c [1546346,24063] 0 2024-07-16T09:55:37.470 INFO:tasks.workunit.client.1.smithi176.stdout:6/439: write d1/d5/d7/f19 [5660926,66229] 0 2024-07-16T09:55:37.471 INFO:tasks.workunit.client.1.smithi176.stdout:9/472: readlink d5/de/d37/d58/d89/d8a/l3d 0 2024-07-16T09:55:37.472 INFO:tasks.workunit.client.1.smithi176.stdout:3/360: mkdir d3/da/d6c 0 2024-07-16T09:55:37.472 INFO:tasks.workunit.client.1.smithi176.stdout:6/440: mknod d1/d5/d7/d2f/ca4 0 2024-07-16T09:55:37.473 INFO:tasks.workunit.client.1.smithi176.stdout:3/361: chown d3/d17/c1c 715 1 2024-07-16T09:55:37.475 INFO:tasks.workunit.client.1.smithi176.stdout:6/441: mkdir d1/d5d/d9e/d76/da5 0 2024-07-16T09:55:37.477 INFO:tasks.workunit.client.0.smithi156.stdout:1/416: sync 2024-07-16T09:55:37.477 INFO:tasks.workunit.client.0.smithi156.stdout:9/402: sync 2024-07-16T09:55:37.477 INFO:tasks.workunit.client.1.smithi176.stdout:9/473: rename d5/de/d37/d58/d89/d8a/f8e to d5/de/d30/f9d 0 2024-07-16T09:55:37.479 INFO:tasks.workunit.client.1.smithi176.stdout:6/442: unlink d1/d5d/d9e/d76/f58 0 2024-07-16T09:55:37.479 INFO:tasks.workunit.client.0.smithi156.stdout:9/403: write d2/d5/d16/d22/d36/f78 [890645,124777] 0 2024-07-16T09:55:37.480 INFO:tasks.workunit.client.1.smithi176.stdout:3/362: dwrite d3/d17/d1d/d4f/d59/f34 [0,4194304] 0 2024-07-16T09:55:37.481 INFO:tasks.workunit.client.0.smithi156.stdout:9/404: mknod d2/d5/d61/c7e 0 2024-07-16T09:55:37.482 INFO:tasks.workunit.client.1.smithi176.stdout:9/474: rename d5/f2d to d5/d43/f9e 0 2024-07-16T09:55:37.482 INFO:tasks.workunit.client.0.smithi156.stdout:1/417: rename dc/d14/d3e/d55/l6f to dc/d14/d1e/l8e 0 2024-07-16T09:55:37.483 INFO:tasks.workunit.client.0.smithi156.stdout:1/418: fdatasync dc/d14/f4b 0 2024-07-16T09:55:37.485 INFO:tasks.workunit.client.1.smithi176.stdout:9/475: mknod d5/d1d/d20/c9f 0 2024-07-16T09:55:37.485 INFO:tasks.workunit.client.0.smithi156.stdout:1/419: write dc/d14/d3e/f5d [846169,129879] 0 2024-07-16T09:55:37.487 INFO:tasks.workunit.client.1.smithi176.stdout:3/363: getdents d3/da/d56 0 2024-07-16T09:55:37.487 INFO:tasks.workunit.client.0.smithi156.stdout:9/405: rename d2/de to d2/d5/d16/d22/d2b/d3d/d3c/d7f 0 2024-07-16T09:55:37.488 INFO:tasks.workunit.client.1.smithi176.stdout:9/476: symlink d5/d1d/d67/la0 0 2024-07-16T09:55:37.489 INFO:tasks.workunit.client.1.smithi176.stdout:3/364: mkdir d3/d17/d1d/d4f/d6d 0 2024-07-16T09:55:37.490 INFO:tasks.workunit.client.1.smithi176.stdout:9/477: creat d5/d1d/d77/d97/fa1 x:0 0 0 2024-07-16T09:55:37.499 INFO:tasks.workunit.client.0.smithi156.stdout:1/420: dread f4 [0,4194304] 0 2024-07-16T09:55:37.508 INFO:tasks.workunit.client.1.smithi176.stdout:9/478: link d5/d1d/d67/d6d/f7a d5/de/d30/fa2 0 2024-07-16T09:55:37.509 INFO:tasks.workunit.client.1.smithi176.stdout:9/479: truncate d5/de/d37/f94 443646 0 2024-07-16T09:55:37.511 INFO:tasks.workunit.client.1.smithi176.stdout:9/480: mkdir d5/de/d37/d58/d89/d8a/da3 0 2024-07-16T09:55:37.518 INFO:tasks.workunit.client.0.smithi156.stdout:1/421: dread dc/f33 [0,4194304] 0 2024-07-16T09:55:37.520 INFO:tasks.workunit.client.0.smithi156.stdout:1/422: truncate dc/d14/f4b 1500163 0 2024-07-16T09:55:37.524 INFO:tasks.workunit.client.0.smithi156.stdout:1/423: link dc/d24/l40 dc/d14/d3e/d55/l8f 0 2024-07-16T09:55:37.525 INFO:tasks.workunit.client.1.smithi176.stdout:9/481: getdents d5/de/d37/d58/d89/d8a 0 2024-07-16T09:55:37.525 INFO:tasks.workunit.client.1.smithi176.stdout:9/482: chown d5/d1d/d20/d78 12683 1 2024-07-16T09:55:37.526 INFO:tasks.workunit.client.0.smithi156.stdout:1/424: symlink dc/d14/d1e/l90 0 2024-07-16T09:55:37.526 INFO:tasks.workunit.client.0.smithi156.stdout:1/425: chown dc/f56 555 1 2024-07-16T09:55:37.527 INFO:tasks.workunit.client.0.smithi156.stdout:1/426: dread - dc/d6b/f80 zero size 2024-07-16T09:55:37.527 INFO:tasks.workunit.client.1.smithi176.stdout:9/483: unlink d5/d1d/f3c 0 2024-07-16T09:55:37.529 INFO:tasks.workunit.client.0.smithi156.stdout:1/427: mknod dc/d24/c91 0 2024-07-16T09:55:37.529 INFO:tasks.workunit.client.1.smithi176.stdout:9/484: rename d5/de/d30/f4a to d5/d52/fa4 0 2024-07-16T09:55:37.530 INFO:tasks.workunit.client.0.smithi156.stdout:1/428: read dc/d14/d1e/f3d [1725658,119749] 0 2024-07-16T09:55:37.530 INFO:tasks.workunit.client.1.smithi176.stdout:9/485: read - d5/de/d21/f84 zero size 2024-07-16T09:55:37.531 INFO:tasks.workunit.client.0.smithi156.stdout:1/429: readlink dc/d14/d3e/d55/d57/l7f 0 2024-07-16T09:55:37.534 INFO:tasks.workunit.client.0.smithi156.stdout:1/430: rename dc/d24/c91 to dc/d14/d3e/d55/d83/c92 0 2024-07-16T09:55:37.535 INFO:tasks.workunit.client.0.smithi156.stdout:1/431: stat dc/d24/f7e 0 2024-07-16T09:55:37.536 INFO:tasks.workunit.client.1.smithi176.stdout:9/486: dwrite d5/de/d21/f84 [0,4194304] 0 2024-07-16T09:55:37.537 INFO:tasks.workunit.client.0.smithi156.stdout:1/432: creat dc/d14/d3e/d55/d57/f93 x:0 0 0 2024-07-16T09:55:37.539 INFO:tasks.workunit.client.0.smithi156.stdout:1/433: mknod dc/d6b/d8a/c94 0 2024-07-16T09:55:37.542 INFO:tasks.workunit.client.0.smithi156.stdout:1/434: creat dc/f95 x:0 0 0 2024-07-16T09:55:37.543 INFO:tasks.workunit.client.0.smithi156.stdout:1/435: mkdir dc/d6b/d96 0 2024-07-16T09:55:37.545 INFO:tasks.workunit.client.0.smithi156.stdout:1/436: mkdir dc/d14/d3e/d45/d97 0 2024-07-16T09:55:37.546 INFO:tasks.workunit.client.0.smithi156.stdout:1/437: mkdir dc/d14/d3e/d98 0 2024-07-16T09:55:37.547 INFO:tasks.workunit.client.0.smithi156.stdout:1/438: chown dc/d14/d3e/d55/d58/l79 32945551 1 2024-07-16T09:55:37.548 INFO:tasks.workunit.client.0.smithi156.stdout:1/439: write dc/d14/d3e/d55/d57/f67 [644492,6975] 0 2024-07-16T09:55:37.553 INFO:tasks.workunit.client.1.smithi176.stdout:5/434: sync 2024-07-16T09:55:37.554 INFO:tasks.workunit.client.0.smithi156.stdout:1/440: write dc/d6b/f80 [642759,70656] 0 2024-07-16T09:55:37.557 INFO:tasks.workunit.client.1.smithi176.stdout:5/435: write d4/d10/d17/f66 [58466,45751] 0 2024-07-16T09:55:37.557 INFO:tasks.workunit.client.0.smithi156.stdout:1/441: dread f4 [0,4194304] 0 2024-07-16T09:55:37.559 INFO:tasks.workunit.client.0.smithi156.stdout:1/442: mknod dc/d14/d3e/d45/c99 0 2024-07-16T09:55:37.560 INFO:tasks.workunit.client.1.smithi176.stdout:5/436: getdents d4/d37 0 2024-07-16T09:55:37.560 INFO:tasks.workunit.client.0.smithi156.stdout:1/443: write dc/d14/d3e/f5d [1179879,74211] 0 2024-07-16T09:55:37.563 INFO:tasks.workunit.client.1.smithi176.stdout:5/437: link d4/d10/f54 d4/f9e 0 2024-07-16T09:55:37.563 INFO:tasks.workunit.client.1.smithi176.stdout:5/438: readlink d4/l92 0 2024-07-16T09:55:37.564 INFO:tasks.workunit.client.0.smithi156.stdout:1/444: link dc/d14/d3e/d45/f53 dc/d14/d3e/d6c/f9a 0 2024-07-16T09:55:37.564 INFO:tasks.workunit.client.0.smithi156.stdout:1/445: stat dc/d14/d1e/f77 0 2024-07-16T09:55:37.565 INFO:tasks.workunit.client.1.smithi176.stdout:5/439: mkdir d4/d37/d72/d38/d39/d9f 0 2024-07-16T09:55:37.571 INFO:tasks.workunit.client.0.smithi156.stdout:1/446: dwrite f4 [0,4194304] 0 2024-07-16T09:55:37.573 INFO:tasks.workunit.client.1.smithi176.stdout:5/440: dwrite d4/d6c/f7c [0,4194304] 0 2024-07-16T09:55:37.576 INFO:tasks.workunit.client.1.smithi176.stdout:5/441: unlink d4/d37/d72/f5b 0 2024-07-16T09:55:37.577 INFO:tasks.workunit.client.1.smithi176.stdout:5/442: readlink d4/d10/l65 0 2024-07-16T09:55:37.578 INFO:tasks.workunit.client.1.smithi176.stdout:5/443: chown d4/d10/d17/f63 81422 1 2024-07-16T09:55:37.579 INFO:tasks.workunit.client.0.smithi156.stdout:1/447: link dc/d20/c6e dc/d14/d3e/d6c/c9b 0 2024-07-16T09:55:37.580 INFO:tasks.workunit.client.1.smithi176.stdout:5/444: symlink d4/d37/la0 0 2024-07-16T09:55:37.583 INFO:tasks.workunit.client.0.smithi156.stdout:1/448: link dc/d14/d1e/c26 dc/d14/d3e/d6c/c9c 0 2024-07-16T09:55:37.585 INFO:tasks.workunit.client.0.smithi156.stdout:1/449: creat dc/d20/f9d x:0 0 0 2024-07-16T09:55:37.587 INFO:tasks.workunit.client.0.smithi156.stdout:1/450: getdents dc/d14/d3e/d55/d58 0 2024-07-16T09:55:37.591 INFO:tasks.workunit.client.1.smithi176.stdout:5/445: creat d4/d37/d72/d38/d39/fa1 x:0 0 0 2024-07-16T09:55:37.591 INFO:tasks.workunit.client.0.smithi156.stdout:1/451: dread dc/f4a [0,4194304] 0 2024-07-16T09:55:37.592 INFO:tasks.workunit.client.0.smithi156.stdout:1/452: write dc/d14/d3e/d55/f70 [598454,112070] 0 2024-07-16T09:55:37.593 INFO:tasks.workunit.client.0.smithi156.stdout:1/453: stat dc/d14/d3e/d55/d57/f82 0 2024-07-16T09:55:37.595 INFO:tasks.workunit.client.1.smithi176.stdout:5/446: truncate d4/f86 2170116 0 2024-07-16T09:55:37.597 INFO:tasks.workunit.client.1.smithi176.stdout:5/447: symlink d4/d14/d19/d1b/la2 0 2024-07-16T09:55:37.601 INFO:tasks.workunit.client.0.smithi156.stdout:1/454: dwrite dc/d14/d1e/f3d [0,4194304] 0 2024-07-16T09:55:37.607 INFO:tasks.workunit.client.1.smithi176.stdout:5/448: dwrite d4/d37/d72/d38/d6e/f7b [0,4194304] 0 2024-07-16T09:55:37.608 INFO:tasks.workunit.client.1.smithi176.stdout:5/449: read - d4/d37/d72/d38/f97 zero size 2024-07-16T09:55:37.609 INFO:tasks.workunit.client.0.smithi156.stdout:1/455: dwrite dc/d14/d3e/d55/d57/f67 [0,4194304] 0 2024-07-16T09:55:37.611 INFO:tasks.workunit.client.0.smithi156.stdout:1/456: write dc/d14/d3e/d45/f69 [1005959,1897] 0 2024-07-16T09:55:37.623 INFO:tasks.workunit.client.1.smithi176.stdout:5/450: dwrite d4/fd [0,4194304] 0 2024-07-16T09:55:37.625 INFO:tasks.workunit.client.0.smithi156.stdout:1/457: dwrite dc/f13 [0,4194304] 0 2024-07-16T09:55:37.628 INFO:tasks.workunit.client.0.smithi156.stdout:1/458: symlink dc/d14/d3e/d6c/d7a/l9e 0 2024-07-16T09:55:37.631 INFO:tasks.workunit.client.1.smithi176.stdout:5/451: dwrite d4/d37/d72/d38/f7e [0,4194304] 0 2024-07-16T09:55:37.634 INFO:tasks.workunit.client.0.smithi156.stdout:1/459: truncate dc/d6b/f75 541475 0 2024-07-16T09:55:37.634 INFO:tasks.workunit.client.0.smithi156.stdout:1/460: chown dc/f4a 2 1 2024-07-16T09:55:37.635 INFO:tasks.workunit.client.1.smithi176.stdout:5/452: creat d4/d37/d72/d38/d39/fa3 x:0 0 0 2024-07-16T09:55:37.639 INFO:tasks.workunit.client.0.smithi156.stdout:1/461: rename dc/d14/c1b to dc/d14/d3e/d45/d97/c9f 0 2024-07-16T09:55:37.641 INFO:tasks.workunit.client.1.smithi176.stdout:5/453: creat d4/d37/d72/d38/fa4 x:0 0 0 2024-07-16T09:55:37.642 INFO:tasks.workunit.client.0.smithi156.stdout:1/462: rmdir dc/d14/d3e/d55/d57 39 2024-07-16T09:55:37.643 INFO:tasks.workunit.client.1.smithi176.stdout:5/454: creat d4/d37/d72/d38/d39/fa5 x:0 0 0 2024-07-16T09:55:37.646 INFO:tasks.workunit.client.1.smithi176.stdout:5/455: creat d4/d37/d72/d38/d39/d9f/fa6 x:0 0 0 2024-07-16T09:55:37.648 INFO:tasks.workunit.client.0.smithi156.stdout:1/463: dwrite dc/d14/d3e/d55/f70 [0,4194304] 0 2024-07-16T09:55:37.652 INFO:tasks.workunit.client.0.smithi156.stdout:1/464: creat dc/d24/fa0 x:0 0 0 2024-07-16T09:55:37.657 INFO:tasks.workunit.client.0.smithi156.stdout:1/465: dread dc/d14/d3e/d6c/d7a/f8c [0,4194304] 0 2024-07-16T09:55:37.660 INFO:tasks.workunit.client.0.smithi156.stdout:1/466: creat dc/d14/d3e/d55/d83/fa1 x:0 0 0 2024-07-16T09:55:37.660 INFO:tasks.workunit.client.1.smithi176.stdout:5/456: dwrite d4/d14/f45 [0,4194304] 0 2024-07-16T09:55:37.665 INFO:tasks.workunit.client.0.smithi156.stdout:1/467: dwrite dc/f28 [0,4194304] 0 2024-07-16T09:55:37.667 INFO:tasks.workunit.client.1.smithi176.stdout:5/457: dwrite d4/d6c/f6f [0,4194304] 0 2024-07-16T09:55:37.670 INFO:tasks.workunit.client.0.smithi156.stdout:1/468: rename dc/f51 to dc/d14/d3e/d55/d57/fa2 0 2024-07-16T09:55:37.672 INFO:tasks.workunit.client.1.smithi176.stdout:5/458: getdents d4/d14/d3d 0 2024-07-16T09:55:37.672 INFO:tasks.workunit.client.0.smithi156.stdout:1/469: rmdir dc/d14/d3e/d45/d97 39 2024-07-16T09:55:37.675 INFO:tasks.workunit.client.0.smithi156.stdout:1/470: creat dc/d24/fa3 x:0 0 0 2024-07-16T09:55:37.675 INFO:tasks.workunit.client.1.smithi176.stdout:5/459: creat d4/d9a/fa7 x:0 0 0 2024-07-16T09:55:37.678 INFO:tasks.workunit.client.1.smithi176.stdout:5/460: getdents d4/d37/d72/d82/d96 0 2024-07-16T09:55:37.680 INFO:tasks.workunit.client.0.smithi156.stdout:1/471: link dc/d24/f5e dc/d14/d3e/d55/d83/fa4 0 2024-07-16T09:55:37.680 INFO:tasks.workunit.client.1.smithi176.stdout:5/461: mknod d4/d6c/d94/ca8 0 2024-07-16T09:55:37.681 INFO:tasks.workunit.client.0.smithi156.stdout:1/472: write dc/d14/d3e/d55/d57/f67 [541096,17189] 0 2024-07-16T09:55:37.684 INFO:tasks.workunit.client.1.smithi176.stdout:5/462: rename d4/d14/d3d/d81 to d4/da9 0 2024-07-16T09:55:37.684 INFO:tasks.workunit.client.0.smithi156.stdout:1/473: getdents dc/d14/d8b 0 2024-07-16T09:55:37.685 INFO:tasks.workunit.client.0.smithi156.stdout:1/474: fsync dc/d14/d3e/d55/f70 0 2024-07-16T09:55:37.686 INFO:tasks.workunit.client.1.smithi176.stdout:5/463: rmdir d4/d37/d72/d38/d39 39 2024-07-16T09:55:37.687 INFO:tasks.workunit.client.0.smithi156.stdout:1/475: creat dc/d14/d3e/d6c/d7a/fa5 x:0 0 0 2024-07-16T09:55:37.688 INFO:tasks.workunit.client.1.smithi176.stdout:5/464: mknod d4/caa 0 2024-07-16T09:55:37.688 INFO:tasks.workunit.client.0.smithi156.stdout:1/476: write dc/d14/f84 [119601,49560] 0 2024-07-16T09:55:37.689 INFO:tasks.workunit.client.0.smithi156.stdout:1/477: chown dc/f19 13927830 1 2024-07-16T09:55:37.691 INFO:tasks.workunit.client.0.smithi156.stdout:1/478: creat dc/d14/d3e/d6c/d7a/fa6 x:0 0 0 2024-07-16T09:55:37.695 INFO:tasks.workunit.client.0.smithi156.stdout:1/479: rename dc/c11 to dc/d20/ca7 0 2024-07-16T09:55:37.696 INFO:tasks.workunit.client.1.smithi176.stdout:5/465: dwrite d4/d10/d17/f18 [0,4194304] 0 2024-07-16T09:55:37.696 INFO:tasks.workunit.client.0.smithi156.stdout:8/315: sync 2024-07-16T09:55:37.698 INFO:tasks.workunit.client.1.smithi176.stdout:5/466: mkdir d4/d14/d19/d1b/dab 0 2024-07-16T09:55:37.699 INFO:tasks.workunit.client.1.smithi176.stdout:5/467: chown d4/d37/d72/d82/f93 343 1 2024-07-16T09:55:37.699 INFO:tasks.workunit.client.1.smithi176.stdout:5/468: fsync d4/d9a/fa7 0 2024-07-16T09:55:37.700 INFO:tasks.workunit.client.0.smithi156.stdout:1/480: link dc/d14/d1e/f35 dc/d14/d3e/d98/fa8 0 2024-07-16T09:55:37.700 INFO:tasks.workunit.client.0.smithi156.stdout:8/316: mknod d0/dd/d1f/c75 0 2024-07-16T09:55:37.704 INFO:tasks.workunit.client.1.smithi176.stdout:5/469: dread d4/d37/f5a [0,4194304] 0 2024-07-16T09:55:37.705 INFO:tasks.workunit.client.0.smithi156.stdout:1/481: rename dc/d20/l71 to dc/d14/d3e/la9 0 2024-07-16T09:55:37.706 INFO:tasks.workunit.client.0.smithi156.stdout:8/317: dread d0/f6d [0,4194304] 0 2024-07-16T09:55:37.707 INFO:tasks.workunit.client.0.smithi156.stdout:8/318: write d0/dd/f17 [760804,95301] 0 2024-07-16T09:55:37.707 INFO:tasks.workunit.client.1.smithi176.stdout:5/470: creat d4/d14/fac x:0 0 0 2024-07-16T09:55:37.713 INFO:tasks.workunit.client.0.smithi156.stdout:8/319: dwrite d0/d11/d29/f5d [0,4194304] 0 2024-07-16T09:55:37.714 INFO:tasks.workunit.client.0.smithi156.stdout:8/320: chown d0/f12 1968961 1 2024-07-16T09:55:37.716 INFO:tasks.workunit.client.0.smithi156.stdout:8/321: write d0/d11/d13/d1c/d24/f42 [385643,118728] 0 2024-07-16T09:55:37.722 INFO:tasks.workunit.client.0.smithi156.stdout:8/322: dwrite d0/f2c [0,4194304] 0 2024-07-16T09:55:37.723 INFO:tasks.workunit.client.1.smithi176.stdout:5/471: dread d4/d37/d72/d82/f83 [0,4194304] 0 2024-07-16T09:55:37.725 INFO:tasks.workunit.client.0.smithi156.stdout:8/323: creat d0/d11/d13/f76 x:0 0 0 2024-07-16T09:55:37.727 INFO:tasks.workunit.client.0.smithi156.stdout:8/324: symlink d0/d11/d2e/l77 0 2024-07-16T09:55:37.730 INFO:tasks.workunit.client.0.smithi156.stdout:8/325: creat d0/f78 x:0 0 0 2024-07-16T09:55:37.733 INFO:tasks.workunit.client.1.smithi176.stdout:5/472: write d4/d10/f12 [990493,124259] 0 2024-07-16T09:55:37.735 INFO:tasks.workunit.client.0.smithi156.stdout:1/482: dread dc/d14/d1e/f29 [0,4194304] 0 2024-07-16T09:55:37.735 INFO:tasks.workunit.client.1.smithi176.stdout:5/473: chown d4/d37/d72/d38/d39/f56 32014 1 2024-07-16T09:55:37.738 INFO:tasks.workunit.client.1.smithi176.stdout:5/474: creat d4/d37/d72/fad x:0 0 0 2024-07-16T09:55:37.739 INFO:tasks.workunit.client.1.smithi176.stdout:5/475: creat d4/d37/d88/fae x:0 0 0 2024-07-16T09:55:37.740 INFO:tasks.workunit.client.1.smithi176.stdout:5/476: rmdir d4/d10/d17 39 2024-07-16T09:55:37.748 INFO:tasks.workunit.client.1.smithi176.stdout:1/271: sync 2024-07-16T09:55:37.749 INFO:tasks.workunit.client.0.smithi156.stdout:1/483: rename c8 to dc/d14/d3e/d6c/caa 0 2024-07-16T09:55:37.751 INFO:tasks.workunit.client.0.smithi156.stdout:8/326: truncate d0/f2c 8302935 0 2024-07-16T09:55:37.756 INFO:tasks.workunit.client.0.smithi156.stdout:1/484: dwrite dc/d14/d3e/d6c/d7a/f8c [0,4194304] 0 2024-07-16T09:55:37.778 INFO:tasks.workunit.client.0.smithi156.stdout:1/485: dread dc/d24/f3a [0,4194304] 0 2024-07-16T09:55:37.779 INFO:tasks.workunit.client.0.smithi156.stdout:1/486: mkdir dc/d14/d3e/d6c/dab 0 2024-07-16T09:55:37.780 INFO:tasks.workunit.client.0.smithi156.stdout:1/487: write dc/d20/f49 [1268631,103272] 0 2024-07-16T09:55:37.781 INFO:tasks.workunit.client.0.smithi156.stdout:1/488: chown dc/d6b/d96 53518 1 2024-07-16T09:55:37.782 INFO:tasks.workunit.client.0.smithi156.stdout:1/489: write dc/d14/f2d [3935599,58348] 0 2024-07-16T09:55:37.784 INFO:tasks.workunit.client.0.smithi156.stdout:1/490: mknod dc/d14/d3e/d6c/cac 0 2024-07-16T09:55:37.786 INFO:tasks.workunit.client.0.smithi156.stdout:1/491: creat dc/d14/d3e/d55/fad x:0 0 0 2024-07-16T09:55:37.787 INFO:tasks.workunit.client.0.smithi156.stdout:1/492: symlink dc/d14/d3e/d55/d57/lae 0 2024-07-16T09:55:37.789 INFO:tasks.workunit.client.0.smithi156.stdout:1/493: write dc/f4a [798552,42039] 0 2024-07-16T09:55:37.791 INFO:tasks.workunit.client.0.smithi156.stdout:1/494: dread dc/d14/d3e/d45/f53 [0,4194304] 0 2024-07-16T09:55:37.796 INFO:tasks.workunit.client.0.smithi156.stdout:1/495: dwrite dc/d14/d3e/f43 [0,4194304] 0 2024-07-16T09:55:37.798 INFO:tasks.workunit.client.0.smithi156.stdout:1/496: rmdir dc/d14/d1e 39 2024-07-16T09:55:37.806 INFO:tasks.workunit.client.0.smithi156.stdout:1/497: dwrite dc/d14/d3e/d55/fad [0,4194304] 0 2024-07-16T09:55:37.807 INFO:tasks.workunit.client.0.smithi156.stdout:1/498: dread dc/d14/d3e/d6c/f9a [0,4194304] 0 2024-07-16T09:55:37.815 INFO:tasks.workunit.client.0.smithi156.stdout:1/499: dwrite dc/d14/d3e/d6c/f73 [0,4194304] 0 2024-07-16T09:55:37.816 INFO:tasks.workunit.client.0.smithi156.stdout:1/500: chown dc/d14/d3e/d6c/f73 1405312 1 2024-07-16T09:55:37.818 INFO:tasks.workunit.client.0.smithi156.stdout:1/501: fsync dc/d20/f36 0 2024-07-16T09:55:37.822 INFO:tasks.workunit.client.0.smithi156.stdout:1/502: rename dc/d14/d3e/f5d to dc/d20/faf 0 2024-07-16T09:55:37.825 INFO:tasks.workunit.client.0.smithi156.stdout:1/503: rmdir dc/d14/d3e/d6c/dab 0 2024-07-16T09:55:37.827 INFO:tasks.workunit.client.0.smithi156.stdout:1/504: rename dc/d24/c44 to dc/d14/d8b/cb0 0 2024-07-16T09:55:37.827 INFO:tasks.workunit.client.0.smithi156.stdout:1/505: chown lb 21976875 1 2024-07-16T09:55:37.828 INFO:tasks.workunit.client.0.smithi156.stdout:1/506: stat dc/d14/d3e/d6c/c9b 0 2024-07-16T09:55:37.830 INFO:tasks.workunit.client.0.smithi156.stdout:1/507: creat dc/d24/fb1 x:0 0 0 2024-07-16T09:55:37.830 INFO:tasks.workunit.client.0.smithi156.stdout:1/508: fdatasync dc/d14/d3e/d55/d57/f67 0 2024-07-16T09:55:37.832 INFO:tasks.workunit.client.0.smithi156.stdout:1/509: creat dc/d14/d3e/d55/d83/fb2 x:0 0 0 2024-07-16T09:55:37.837 INFO:tasks.workunit.client.0.smithi156.stdout:1/510: dwrite dc/d14/d3e/d45/f86 [0,4194304] 0 2024-07-16T09:55:37.839 INFO:tasks.workunit.client.0.smithi156.stdout:1/511: rmdir dc/d14/d3e/d45 39 2024-07-16T09:55:37.839 INFO:tasks.workunit.client.0.smithi156.stdout:1/512: fsync dc/d6b/f74 0 2024-07-16T09:55:37.850 INFO:tasks.workunit.client.0.smithi156.stdout:1/513: dwrite dc/d14/d3e/d98/fa8 [0,4194304] 0 2024-07-16T09:55:37.853 INFO:tasks.workunit.client.0.smithi156.stdout:1/514: symlink dc/d14/d3e/lb3 0 2024-07-16T09:55:37.864 INFO:tasks.workunit.client.0.smithi156.stdout:1/515: dwrite dc/d14/d3e/d45/f53 [0,4194304] 0 2024-07-16T09:55:37.868 INFO:tasks.workunit.client.0.smithi156.stdout:1/516: getdents dc/d14/d3e/d55/d83 0 2024-07-16T09:55:37.869 INFO:tasks.workunit.client.0.smithi156.stdout:1/517: chown dc/d14/d3e/d6c/d7a 210 1 2024-07-16T09:55:37.869 INFO:tasks.workunit.client.0.smithi156.stdout:4/323: sync 2024-07-16T09:55:37.869 INFO:tasks.workunit.client.0.smithi156.stdout:2/501: sync 2024-07-16T09:55:37.871 INFO:tasks.workunit.client.0.smithi156.stdout:1/518: mkdir dc/d14/d8b/db4 0 2024-07-16T09:55:37.872 INFO:tasks.workunit.client.0.smithi156.stdout:1/519: readlink dc/d14/d3e/d55/d57/lae 0 2024-07-16T09:55:37.872 INFO:tasks.workunit.client.0.smithi156.stdout:2/502: chown d0/d5/d29/d2d/d5b/ca7 94 1 2024-07-16T09:55:37.872 INFO:tasks.workunit.client.0.smithi156.stdout:4/324: symlink d4/db/dc/d62/l6d 0 2024-07-16T09:55:37.873 INFO:tasks.workunit.client.1.smithi176.stdout:8/473: sync 2024-07-16T09:55:37.874 INFO:tasks.workunit.client.0.smithi156.stdout:4/325: stat d4/db/dc/f4d 0 2024-07-16T09:55:37.874 INFO:tasks.workunit.client.1.smithi176.stdout:8/474: write d8/d20/d2a/f75 [726650,55442] 0 2024-07-16T09:55:37.875 INFO:tasks.workunit.client.0.smithi156.stdout:1/520: rename dc/d14/d3e/d55/d58/l59 to dc/d14/d3e/d55/d58/lb5 0 2024-07-16T09:55:37.879 INFO:tasks.workunit.client.0.smithi156.stdout:1/521: dread dc/d14/d3e/d45/f86 [0,4194304] 0 2024-07-16T09:55:37.881 INFO:tasks.workunit.client.0.smithi156.stdout:4/326: truncate d4/d23/d30/d33/f3a 2450639 0 2024-07-16T09:55:37.882 INFO:tasks.workunit.client.0.smithi156.stdout:4/327: chown d4/db/dc/f2a 9392 1 2024-07-16T09:55:37.882 INFO:tasks.workunit.client.0.smithi156.stdout:1/522: rename dc/d14/d3e/d55/c62 to dc/d14/d1e/cb6 0 2024-07-16T09:55:37.883 INFO:tasks.workunit.client.0.smithi156.stdout:4/328: symlink d4/d23/d30/d33/l6e 0 2024-07-16T09:55:37.883 INFO:tasks.workunit.client.0.smithi156.stdout:1/523: read dc/d6b/f80 [856970,21009] 0 2024-07-16T09:55:37.886 INFO:tasks.workunit.client.0.smithi156.stdout:4/329: truncate d4/f9 6429714 0 2024-07-16T09:55:37.888 INFO:tasks.workunit.client.0.smithi156.stdout:1/524: creat dc/d14/d3e/d45/fb7 x:0 0 0 2024-07-16T09:55:37.889 INFO:tasks.workunit.client.0.smithi156.stdout:4/330: mknod d4/d23/d30/c6f 0 2024-07-16T09:55:37.891 INFO:tasks.workunit.client.0.smithi156.stdout:1/525: dread dc/d14/d3e/d6c/f9a [0,4194304] 0 2024-07-16T09:55:37.892 INFO:tasks.workunit.client.0.smithi156.stdout:4/331: creat d4/db/dc/f70 x:0 0 0 2024-07-16T09:55:37.893 INFO:tasks.workunit.client.0.smithi156.stdout:4/332: truncate d4/db/dc/f4e 125028 0 2024-07-16T09:55:37.896 INFO:tasks.workunit.client.0.smithi156.stdout:4/333: mkdir d4/d23/d52/d71 0 2024-07-16T09:55:37.896 INFO:tasks.workunit.client.0.smithi156.stdout:1/526: getdents dc/d14/d3e/d55/d58 0 2024-07-16T09:55:37.898 INFO:tasks.workunit.client.0.smithi156.stdout:4/334: symlink d4/db/dc/d67/l72 0 2024-07-16T09:55:37.901 INFO:tasks.workunit.client.0.smithi156.stdout:4/335: stat d4/d23/d30/f6c 0 2024-07-16T09:55:37.901 INFO:tasks.workunit.client.0.smithi156.stdout:1/527: truncate dc/d14/f48 369600 0 2024-07-16T09:55:37.902 INFO:tasks.workunit.client.0.smithi156.stdout:1/528: dread - dc/d14/d1e/f77 zero size 2024-07-16T09:55:37.903 INFO:tasks.workunit.client.0.smithi156.stdout:4/336: rename d4/db/c21 to d4/c73 0 2024-07-16T09:55:37.904 INFO:tasks.workunit.client.0.smithi156.stdout:4/337: chown d4/db/c1a 0 1 2024-07-16T09:55:37.905 INFO:tasks.workunit.client.0.smithi156.stdout:1/529: rename dc/d24/l39 to dc/lb8 0 2024-07-16T09:55:37.907 INFO:tasks.workunit.client.0.smithi156.stdout:1/530: write dc/d14/d3e/d55/d57/f82 [253671,81151] 0 2024-07-16T09:55:37.908 INFO:tasks.workunit.client.0.smithi156.stdout:4/338: rename d4/db/d47/d63/c4c to d4/db/dc/c74 0 2024-07-16T09:55:37.910 INFO:tasks.workunit.client.0.smithi156.stdout:4/339: creat d4/db/d47/d63/f75 x:0 0 0 2024-07-16T09:55:37.911 INFO:tasks.workunit.client.0.smithi156.stdout:1/531: getdents dc/d14/d1e 0 2024-07-16T09:55:37.911 INFO:tasks.workunit.client.0.smithi156.stdout:4/340: dread - d4/db/f61 zero size 2024-07-16T09:55:37.912 INFO:tasks.workunit.client.0.smithi156.stdout:4/341: read d4/db/dc/f4d [4780880,40199] 0 2024-07-16T09:55:37.913 INFO:tasks.workunit.client.0.smithi156.stdout:1/532: rename dc/d24/f3c to dc/d14/d3e/d6c/fb9 0 2024-07-16T09:55:37.917 INFO:tasks.workunit.client.0.smithi156.stdout:1/533: link dc/d6b/f75 dc/d14/d3e/d6c/d7a/fba 0 2024-07-16T09:55:37.922 INFO:tasks.workunit.client.0.smithi156.stdout:4/342: dread d4/db/dc/f68 [0,4194304] 0 2024-07-16T09:55:37.923 INFO:tasks.workunit.client.0.smithi156.stdout:1/534: dwrite dc/d14/d3e/d45/f53 [4194304,4194304] 0 2024-07-16T09:55:37.924 INFO:tasks.workunit.client.0.smithi156.stdout:4/343: fdatasync d4/d23/f39 0 2024-07-16T09:55:37.924 INFO:tasks.workunit.client.0.smithi156.stdout:1/535: rename dc/d14 to dc/d14/d8b/dbb 22 2024-07-16T09:55:37.925 INFO:tasks.workunit.client.0.smithi156.stdout:4/344: read d4/f9 [3765410,40807] 0 2024-07-16T09:55:37.926 INFO:tasks.workunit.client.0.smithi156.stdout:4/345: chown d4/db/d47/d4b 7659096 1 2024-07-16T09:55:37.932 INFO:tasks.workunit.client.0.smithi156.stdout:1/536: dwrite dc/f10 [4194304,4194304] 0 2024-07-16T09:55:37.933 INFO:tasks.workunit.client.0.smithi156.stdout:1/537: dread - dc/d14/d3e/d6c/d7a/fa6 zero size 2024-07-16T09:55:37.934 INFO:tasks.workunit.client.0.smithi156.stdout:1/538: chown dc/f28 310 1 2024-07-16T09:55:37.936 INFO:tasks.workunit.client.0.smithi156.stdout:1/539: read - dc/d14/d3e/d55/d83/fa1 zero size 2024-07-16T09:55:37.936 INFO:tasks.workunit.client.0.smithi156.stdout:4/346: dread d4/d23/d30/d33/f45 [0,4194304] 0 2024-07-16T09:55:37.937 INFO:tasks.workunit.client.0.smithi156.stdout:0/364: sync 2024-07-16T09:55:37.938 INFO:tasks.workunit.client.0.smithi156.stdout:4/347: write d4/d23/d52/f66 [930133,104212] 0 2024-07-16T09:55:37.940 INFO:tasks.workunit.client.0.smithi156.stdout:4/348: symlink d4/d23/d30/l76 0 2024-07-16T09:55:37.942 INFO:tasks.workunit.client.0.smithi156.stdout:0/365: dread d0/d22/f3c [0,4194304] 0 2024-07-16T09:55:37.943 INFO:tasks.workunit.client.0.smithi156.stdout:4/349: truncate d4/db/dc/f29 3143721 0 2024-07-16T09:55:37.944 INFO:tasks.workunit.client.0.smithi156.stdout:0/366: rmdir d0/d22/d41/d52 39 2024-07-16T09:55:37.946 INFO:tasks.workunit.client.0.smithi156.stdout:4/350: rename d4/d23/d30/d33/f45 to d4/d23/d52/d71/f77 0 2024-07-16T09:55:37.947 INFO:tasks.workunit.client.0.smithi156.stdout:1/540: truncate f4 1446559 0 2024-07-16T09:55:37.947 INFO:tasks.workunit.client.0.smithi156.stdout:4/351: chown d4/d23/d52/d71 0 1 2024-07-16T09:55:37.949 INFO:tasks.workunit.client.0.smithi156.stdout:4/352: write d4/db/dc/f55 [61958,55474] 0 2024-07-16T09:55:37.950 INFO:tasks.workunit.client.0.smithi156.stdout:0/367: dwrite d0/d22/d41/f4a [0,4194304] 0 2024-07-16T09:55:37.950 INFO:tasks.workunit.client.0.smithi156.stdout:1/541: truncate dc/d14/d3e/d6c/d7a/fa5 198701 0 2024-07-16T09:55:37.954 INFO:tasks.workunit.client.0.smithi156.stdout:0/368: mkdir d0/d1a/d4d/d9/d27/d2c/d6d 0 2024-07-16T09:55:37.955 INFO:tasks.workunit.client.0.smithi156.stdout:0/369: stat d0/d1a/d4d/d9/d27/d3d/f37 0 2024-07-16T09:55:37.957 INFO:tasks.workunit.client.0.smithi156.stdout:0/370: read d0/d1a/d4d/f6b [305629,53384] 0 2024-07-16T09:55:37.964 INFO:tasks.workunit.client.0.smithi156.stdout:4/353: truncate d4/db/dc/f68 2105846 0 2024-07-16T09:55:37.964 INFO:tasks.workunit.client.0.smithi156.stdout:1/542: truncate dc/d14/d3e/f43 3650836 0 2024-07-16T09:55:37.965 INFO:tasks.workunit.client.0.smithi156.stdout:0/371: getdents d0/d1a/d4d 0 2024-07-16T09:55:37.967 INFO:tasks.workunit.client.0.smithi156.stdout:0/372: write d0/d1a/d4d/f21 [232746,72372] 0 2024-07-16T09:55:37.967 INFO:tasks.workunit.client.0.smithi156.stdout:4/354: creat d4/d23/f78 x:0 0 0 2024-07-16T09:55:37.969 INFO:tasks.workunit.client.1.smithi176.stdout:7/467: sync 2024-07-16T09:55:37.971 INFO:tasks.workunit.client.0.smithi156.stdout:1/543: dwrite dc/d14/d1e/f41 [0,4194304] 0 2024-07-16T09:55:37.975 INFO:tasks.workunit.client.0.smithi156.stdout:4/355: rename d4/d23/d30/f6c to d4/f79 0 2024-07-16T09:55:37.979 INFO:tasks.workunit.client.0.smithi156.stdout:1/544: creat dc/d14/d3e/d45/d97/fbc x:0 0 0 2024-07-16T09:55:37.980 INFO:tasks.workunit.client.0.smithi156.stdout:1/545: write dc/d24/f7e [373024,89368] 0 2024-07-16T09:55:37.984 INFO:tasks.workunit.client.0.smithi156.stdout:1/546: getdents dc/d14/d3e/d45 0 2024-07-16T09:55:37.985 INFO:tasks.workunit.client.0.smithi156.stdout:1/547: rmdir dc/d6b/d8a 39 2024-07-16T09:55:37.988 INFO:tasks.workunit.client.0.smithi156.stdout:1/548: truncate dc/d14/d3e/d55/d57/f67 1078656 0 2024-07-16T09:55:37.991 INFO:tasks.workunit.client.0.smithi156.stdout:4/356: truncate d4/d23/f53 8116043 0 2024-07-16T09:55:37.992 INFO:tasks.workunit.client.0.smithi156.stdout:4/357: write d4/db/dc/d5f/f5c [351238,89458] 0 2024-07-16T09:55:37.993 INFO:tasks.workunit.client.0.smithi156.stdout:1/549: link dc/d6b/d8a/c94 dc/d14/d3e/d6c/d7a/cbd 0 2024-07-16T09:55:38.001 INFO:tasks.workunit.client.0.smithi156.stdout:1/550: dwrite dc/d24/f2a [0,4194304] 0 2024-07-16T09:55:38.004 INFO:tasks.workunit.client.0.smithi156.stdout:4/358: dwrite d4/f3d [0,4194304] 0 2024-07-16T09:55:38.006 INFO:tasks.workunit.client.0.smithi156.stdout:1/551: unlink dc/d14/d3e/d6c/c9c 0 2024-07-16T09:55:38.008 INFO:tasks.workunit.client.0.smithi156.stdout:4/359: read d4/f9 [2515541,55103] 0 2024-07-16T09:55:38.010 INFO:tasks.workunit.client.0.smithi156.stdout:1/552: creat dc/d20/fbe x:0 0 0 2024-07-16T09:55:38.013 INFO:tasks.workunit.client.0.smithi156.stdout:1/553: unlink dc/d14/d3e/d45/f69 0 2024-07-16T09:55:38.016 INFO:tasks.workunit.client.0.smithi156.stdout:4/360: dwrite d4/db/dc/d5f/f5c [0,4194304] 0 2024-07-16T09:55:38.019 INFO:tasks.workunit.client.0.smithi156.stdout:4/361: truncate d4/db/dc/d5f/f5c 4971203 0 2024-07-16T09:55:38.019 INFO:tasks.workunit.client.0.smithi156.stdout:4/362: write d4/d23/f17 [2393504,7993] 0 2024-07-16T09:55:38.023 INFO:tasks.workunit.client.0.smithi156.stdout:4/363: creat d4/db/d47/d63/d58/f7a x:0 0 0 2024-07-16T09:55:38.024 INFO:tasks.workunit.client.0.smithi156.stdout:4/364: write d4/d23/f17 [2180816,3907] 0 2024-07-16T09:55:38.028 INFO:tasks.workunit.client.0.smithi156.stdout:4/365: rename d4/db/f34 to d4/db/f7b 0 2024-07-16T09:55:38.029 INFO:tasks.workunit.client.0.smithi156.stdout:1/554: write dc/d14/d3e/d45/f86 [2859118,41656] 0 2024-07-16T09:55:38.029 INFO:tasks.workunit.client.0.smithi156.stdout:4/366: stat d4/d23/d30/d33/l6e 0 2024-07-16T09:55:38.032 INFO:tasks.workunit.client.0.smithi156.stdout:1/555: readlink dc/d14/d3e/d55/l8f 0 2024-07-16T09:55:38.033 INFO:tasks.workunit.client.0.smithi156.stdout:1/556: write dc/d24/f3a [1483302,23586] 0 2024-07-16T09:55:38.038 INFO:tasks.workunit.client.0.smithi156.stdout:1/557: dwrite dc/d14/d3e/d6c/f73 [0,4194304] 0 2024-07-16T09:55:38.039 INFO:tasks.workunit.client.0.smithi156.stdout:4/367: dwrite d4/db/f2c [0,4194304] 0 2024-07-16T09:55:38.041 INFO:tasks.workunit.client.0.smithi156.stdout:1/558: creat dc/d20/fbf x:0 0 0 2024-07-16T09:55:38.046 INFO:tasks.workunit.client.0.smithi156.stdout:1/559: link dc/d14/d3e/d55/d83/c92 dc/d14/d3e/d55/d83/cc0 0 2024-07-16T09:55:38.052 INFO:tasks.workunit.client.0.smithi156.stdout:4/368: write d4/f5 [2087290,32194] 0 2024-07-16T09:55:38.053 INFO:tasks.workunit.client.0.smithi156.stdout:1/560: write dc/d14/d3e/f43 [443391,8804] 0 2024-07-16T09:55:38.054 INFO:tasks.workunit.client.0.smithi156.stdout:1/561: rmdir dc/d14/d1e 39 2024-07-16T09:55:38.057 INFO:tasks.workunit.client.0.smithi156.stdout:4/369: rename d4/db/dc/d62 to d4/db/dc/d7c 0 2024-07-16T09:55:38.058 INFO:tasks.workunit.client.0.smithi156.stdout:4/370: chown d4/db/f18 0 1 2024-07-16T09:55:38.061 INFO:tasks.workunit.client.0.smithi156.stdout:1/562: dwrite dc/d24/f3a [0,4194304] 0 2024-07-16T09:55:38.065 INFO:tasks.workunit.client.0.smithi156.stdout:4/371: dwrite d4/db/f61 [0,4194304] 0 2024-07-16T09:55:38.066 INFO:tasks.workunit.client.0.smithi156.stdout:1/563: getdents dc/d6b 0 2024-07-16T09:55:38.070 INFO:tasks.workunit.client.0.smithi156.stdout:4/372: fdatasync d4/d23/d30/d33/f3a 0 2024-07-16T09:55:38.072 INFO:tasks.workunit.client.0.smithi156.stdout:6/379: sync 2024-07-16T09:55:38.074 INFO:tasks.workunit.client.0.smithi156.stdout:1/564: link dc/d6b/d8a/c94 dc/d14/d3e/d45/cc1 0 2024-07-16T09:55:38.075 INFO:tasks.workunit.client.0.smithi156.stdout:4/373: mknod d4/db/d47/c7d 0 2024-07-16T09:55:38.076 INFO:tasks.workunit.client.0.smithi156.stdout:6/380: creat d3/d13/d17/d4f/d30/f7f x:0 0 0 2024-07-16T09:55:38.076 INFO:tasks.workunit.client.0.smithi156.stdout:1/565: creat dc/d24/fc2 x:0 0 0 2024-07-16T09:55:38.077 INFO:tasks.workunit.client.0.smithi156.stdout:4/374: creat d4/db/d47/d4b/f7e x:0 0 0 2024-07-16T09:55:38.078 INFO:tasks.workunit.client.0.smithi156.stdout:6/381: creat d3/d13/d17/d4f/d5c/f80 x:0 0 0 2024-07-16T09:55:38.079 INFO:tasks.workunit.client.0.smithi156.stdout:6/382: stat d3/d13/d17/d4f/d4d 0 2024-07-16T09:55:38.079 INFO:tasks.workunit.client.0.smithi156.stdout:4/375: mkdir d4/db/dc/d7f 0 2024-07-16T09:55:38.080 INFO:tasks.workunit.client.0.smithi156.stdout:1/566: creat dc/d14/d3e/fc3 x:0 0 0 2024-07-16T09:55:38.080 INFO:tasks.workunit.client.0.smithi156.stdout:4/376: fsync d4/d23/d30/f46 0 2024-07-16T09:55:38.081 INFO:tasks.workunit.client.0.smithi156.stdout:4/377: dread - d4/d23/f78 zero size 2024-07-16T09:55:38.082 INFO:tasks.workunit.client.0.smithi156.stdout:6/383: rename d3/d13/c7d to d3/d13/d17/d4f/d4d/d3b/c81 0 2024-07-16T09:55:38.083 INFO:tasks.workunit.client.0.smithi156.stdout:4/378: creat d4/f80 x:0 0 0 2024-07-16T09:55:38.084 INFO:tasks.workunit.client.0.smithi156.stdout:6/384: mknod d3/d13/d17/d4f/d30/c82 0 2024-07-16T09:55:38.084 INFO:tasks.workunit.client.0.smithi156.stdout:4/379: symlink d4/d23/d52/d71/l81 0 2024-07-16T09:55:38.085 INFO:tasks.workunit.client.0.smithi156.stdout:1/567: fsync dc/f19 0 2024-07-16T09:55:38.085 INFO:tasks.workunit.client.0.smithi156.stdout:1/568: chown dc/d6b 232595 1 2024-07-16T09:55:38.085 INFO:tasks.workunit.client.0.smithi156.stdout:6/385: truncate d3/d13/d17/d4f/d30/f7f 999194 0 2024-07-16T09:55:38.088 INFO:tasks.workunit.client.0.smithi156.stdout:1/569: rename dc/d14/f84 to dc/d14/d8b/fc4 0 2024-07-16T09:55:38.089 INFO:tasks.workunit.client.0.smithi156.stdout:6/386: write d3/d13/d17/f7c [766820,44540] 0 2024-07-16T09:55:38.092 INFO:tasks.workunit.client.1.smithi176.stdout:4/433: sync 2024-07-16T09:55:38.092 INFO:tasks.workunit.client.1.smithi176.stdout:2/408: sync 2024-07-16T09:55:38.092 INFO:tasks.workunit.client.1.smithi176.stdout:2/409: chown dd/de 7790 1 2024-07-16T09:55:38.094 INFO:tasks.workunit.client.0.smithi156.stdout:6/387: creat d3/d13/d17/d4f/f83 x:0 0 0 2024-07-16T09:55:38.094 INFO:tasks.workunit.client.1.smithi176.stdout:2/410: chown dd/d49/l7b 3554234 1 2024-07-16T09:55:38.095 INFO:tasks.workunit.client.0.smithi156.stdout:1/570: getdents dc/d14/d3e/d55 0 2024-07-16T09:55:38.095 INFO:tasks.workunit.client.0.smithi156.stdout:6/388: write d3/d13/d75/f10 [1206915,19400] 0 2024-07-16T09:55:38.098 INFO:tasks.workunit.client.1.smithi176.stdout:4/434: dwrite da/d3d/f1b [0,4194304] 0 2024-07-16T09:55:38.099 INFO:tasks.workunit.client.0.smithi156.stdout:1/571: symlink dc/d14/d3e/d98/lc5 0 2024-07-16T09:55:38.103 INFO:tasks.workunit.client.0.smithi156.stdout:6/389: dwrite d3/d13/d17/f7c [0,4194304] 0 2024-07-16T09:55:38.104 INFO:tasks.workunit.client.0.smithi156.stdout:1/572: dread dc/d14/d8b/fc4 [0,4194304] 0 2024-07-16T09:55:38.104 INFO:tasks.workunit.client.1.smithi176.stdout:4/435: link da/d3d/f1b da/dc/f83 0 2024-07-16T09:55:38.104 INFO:tasks.workunit.client.0.smithi156.stdout:8/327: sync 2024-07-16T09:55:38.105 INFO:tasks.workunit.client.1.smithi176.stdout:4/436: read - da/dc/d10/d28/f79 zero size 2024-07-16T09:55:38.105 INFO:tasks.workunit.client.0.smithi156.stdout:6/390: fdatasync d3/d13/d75/d19/f66 0 2024-07-16T09:55:38.106 INFO:tasks.workunit.client.1.smithi176.stdout:2/411: dread dd/d49/f55 [0,4194304] 0 2024-07-16T09:55:38.107 INFO:tasks.workunit.client.1.smithi176.stdout:2/412: write dd/d16/d39/f6f [192041,57023] 0 2024-07-16T09:55:38.110 INFO:tasks.workunit.client.1.smithi176.stdout:4/437: rename da/dc/c25 to da/d41/d5b/c84 0 2024-07-16T09:55:38.111 INFO:tasks.workunit.client.0.smithi156.stdout:4/380: dwrite d4/db/f18 [0,4194304] 0 2024-07-16T09:55:38.116 INFO:tasks.workunit.client.0.smithi156.stdout:6/391: symlink d3/d13/d75/d19/d31/d35/l84 0 2024-07-16T09:55:38.117 INFO:tasks.workunit.client.1.smithi176.stdout:2/413: dwrite dd/d16/d39/f6f [0,4194304] 0 2024-07-16T09:55:38.119 INFO:tasks.workunit.client.1.smithi176.stdout:4/438: dwrite da/d3d/d17/d48/f65 [0,4194304] 0 2024-07-16T09:55:38.119 INFO:tasks.workunit.client.0.smithi156.stdout:6/392: creat d3/d13/d17/d4f/d5c/f85 x:0 0 0 2024-07-16T09:55:38.121 INFO:tasks.workunit.client.0.smithi156.stdout:1/573: rename dc/d14/d3e/d45/l5a to dc/d14/d3e/lc6 0 2024-07-16T09:55:38.122 INFO:tasks.workunit.client.0.smithi156.stdout:4/381: rename d4 to d4/db/d47/d4b/d82 22 2024-07-16T09:55:38.122 INFO:tasks.workunit.client.0.smithi156.stdout:4/382: chown d4/f54 0 1 2024-07-16T09:55:38.124 INFO:tasks.workunit.client.1.smithi176.stdout:4/439: creat da/d14/d40/d5d/f85 x:0 0 0 2024-07-16T09:55:38.126 INFO:tasks.workunit.client.0.smithi156.stdout:6/393: symlink d3/d13/d75/d19/d31/l86 0 2024-07-16T09:55:38.127 INFO:tasks.workunit.client.1.smithi176.stdout:3/365: sync 2024-07-16T09:55:38.127 INFO:tasks.workunit.client.1.smithi176.stdout:6/443: sync 2024-07-16T09:55:38.127 INFO:tasks.workunit.client.1.smithi176.stdout:0/514: sync 2024-07-16T09:55:38.129 INFO:tasks.workunit.client.1.smithi176.stdout:0/515: write d5/d25/d82/f8f [932653,83062] 0 2024-07-16T09:55:38.129 INFO:tasks.workunit.client.1.smithi176.stdout:6/444: chown d1/d5d/d9e/d76/f5a 0 1 2024-07-16T09:55:38.131 INFO:tasks.workunit.client.0.smithi156.stdout:1/574: creat dc/d14/d1e/fc7 x:0 0 0 2024-07-16T09:55:38.134 INFO:tasks.workunit.client.1.smithi176.stdout:6/445: creat d1/d5/d47/fa6 x:0 0 0 2024-07-16T09:55:38.135 INFO:tasks.workunit.client.1.smithi176.stdout:3/366: mkdir d3/d17/d1d/d6e 0 2024-07-16T09:55:38.137 INFO:tasks.workunit.client.0.smithi156.stdout:6/394: dwrite d3/d13/d17/d56/f64 [0,4194304] 0 2024-07-16T09:55:38.138 INFO:tasks.workunit.client.1.smithi176.stdout:0/516: dwrite d5/d1c/f89 [0,4194304] 0 2024-07-16T09:55:38.143 INFO:tasks.workunit.client.0.smithi156.stdout:1/575: dread dc/d14/d3e/d55/f70 [0,4194304] 0 2024-07-16T09:55:38.143 INFO:tasks.workunit.client.0.smithi156.stdout:6/395: symlink d3/d13/d17/d4f/d4d/d3b/l87 0 2024-07-16T09:55:38.148 INFO:tasks.workunit.client.0.smithi156.stdout:0/373: sync 2024-07-16T09:55:38.148 INFO:tasks.workunit.client.0.smithi156.stdout:5/443: sync 2024-07-16T09:55:38.148 INFO:tasks.workunit.client.0.smithi156.stdout:9/406: sync 2024-07-16T09:55:38.148 INFO:tasks.workunit.client.0.smithi156.stdout:3/327: sync 2024-07-16T09:55:38.148 INFO:tasks.workunit.client.0.smithi156.stdout:2/503: sync 2024-07-16T09:55:38.148 INFO:tasks.workunit.client.0.smithi156.stdout:7/334: sync 2024-07-16T09:55:38.148 INFO:tasks.workunit.client.0.smithi156.stdout:1/576: creat dc/d14/fc8 x:0 0 0 2024-07-16T09:55:38.149 INFO:tasks.workunit.client.0.smithi156.stdout:6/396: dread d3/d13/d17/d4f/d30/f7f [0,4194304] 0 2024-07-16T09:55:38.150 INFO:tasks.workunit.client.1.smithi176.stdout:0/517: write d5/d1c/d5a/f75 [397185,17420] 0 2024-07-16T09:55:38.150 INFO:tasks.workunit.client.0.smithi156.stdout:2/504: write d0/d5/d29/f32 [2705710,39331] 0 2024-07-16T09:55:38.152 INFO:tasks.workunit.client.1.smithi176.stdout:2/414: dwrite dd/d2e/d35/f3f [0,4194304] 0 2024-07-16T09:55:38.153 INFO:tasks.workunit.client.0.smithi156.stdout:1/577: write dc/d14/d3e/d55/d83/fa1 [218219,120394] 0 2024-07-16T09:55:38.154 INFO:tasks.workunit.client.0.smithi156.stdout:6/397: fdatasync d3/d13/d17/d4f/d4d/d3b/f59 0 2024-07-16T09:55:38.155 INFO:tasks.workunit.client.1.smithi176.stdout:0/518: creat d5/d39/f9f x:0 0 0 2024-07-16T09:55:38.156 INFO:tasks.workunit.client.0.smithi156.stdout:0/374: dread d0/d1a/d4d/f11 [0,4194304] 0 2024-07-16T09:55:38.157 INFO:tasks.workunit.client.1.smithi176.stdout:0/519: chown d5/fb 4950 1 2024-07-16T09:55:38.159 INFO:tasks.workunit.client.0.smithi156.stdout:8/328: dread d0/f2 [4194304,4194304] 0 2024-07-16T09:55:38.159 INFO:tasks.workunit.client.1.smithi176.stdout:2/415: link dd/d2e/d35/f64 dd/d2e/f83 0 2024-07-16T09:55:38.164 INFO:tasks.workunit.client.0.smithi156.stdout:3/328: symlink d8/l70 0 2024-07-16T09:55:38.165 INFO:tasks.workunit.client.0.smithi156.stdout:3/329: write d8/da/d28/f6e [527769,45312] 0 2024-07-16T09:55:38.167 INFO:tasks.workunit.client.0.smithi156.stdout:3/330: write d8/f62 [232811,61067] 0 2024-07-16T09:55:38.167 INFO:tasks.workunit.client.0.smithi156.stdout:9/407: creat d2/d5/d16/d22/d36/d33/f80 x:0 0 0 2024-07-16T09:55:38.170 INFO:tasks.workunit.client.1.smithi176.stdout:3/367: dread d3/d17/d1d/d4d/f3f [0,4194304] 0 2024-07-16T09:55:38.170 INFO:tasks.workunit.client.0.smithi156.stdout:5/444: mkdir d1b/d1f/d45/d7a/d98 0 2024-07-16T09:55:38.171 INFO:tasks.workunit.client.1.smithi176.stdout:3/368: chown d3/da/c16 2 1 2024-07-16T09:55:38.171 INFO:tasks.workunit.client.1.smithi176.stdout:2/416: dread dd/d16/d39/f4b [0,4194304] 0 2024-07-16T09:55:38.172 INFO:tasks.workunit.client.1.smithi176.stdout:4/440: dread da/dc/fe [0,4194304] 0 2024-07-16T09:55:38.172 INFO:tasks.workunit.client.0.smithi156.stdout:1/578: creat dc/d14/d3e/d55/d58/fc9 x:0 0 0 2024-07-16T09:55:38.173 INFO:tasks.workunit.client.1.smithi176.stdout:3/369: dread - d3/d17/d1d/f63 zero size 2024-07-16T09:55:38.173 INFO:tasks.workunit.client.0.smithi156.stdout:6/398: mknod d3/d13/d75/d19/d2e/c88 0 2024-07-16T09:55:38.173 INFO:tasks.workunit.client.0.smithi156.stdout:4/383: dwrite d4/d23/d30/d33/f3a [0,4194304] 0 2024-07-16T09:55:38.174 INFO:tasks.workunit.client.1.smithi176.stdout:4/441: readlink da/d3d/d17/d54/d5e/d69/l7d 0 2024-07-16T09:55:38.175 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:55:37 smithi156 ceph-mon[29302]: pgmap v250: 65 pgs: 65 active+clean; 1.4 GiB data, 5.4 GiB used, 531 GiB / 536 GiB avail; 22 MiB/s rd, 177 MiB/s wr, 380 op/s 2024-07-16T09:55:38.176 INFO:tasks.workunit.client.0.smithi156.stdout:8/329: readlink d0/d11/d13/d1c/l5a 0 2024-07-16T09:55:38.176 INFO:tasks.workunit.client.0.smithi156.stdout:3/331: mkdir d8/da/d28/d43/d71 0 2024-07-16T09:55:38.176 INFO:tasks.workunit.client.0.smithi156.stdout:0/375: read d0/f14 [566027,44863] 0 2024-07-16T09:55:38.176 INFO:tasks.workunit.client.1.smithi176.stdout:2/417: rename dd/f48 to dd/d16/d39/f84 0 2024-07-16T09:55:38.176 INFO:tasks.workunit.client.1.smithi176.stdout:3/370: read d3/d17/d1d/d4f/d59/f27 [723568,43150] 0 2024-07-16T09:55:38.178 INFO:tasks.workunit.client.0.smithi156.stdout:9/408: write d2/d5/d16/d22/d2b/d3d/d3c/f62 [2969874,49094] 0 2024-07-16T09:55:38.178 INFO:tasks.workunit.client.1.smithi176.stdout:3/371: read - d3/da/d33/f67 zero size 2024-07-16T09:55:38.179 INFO:tasks.workunit.client.1.smithi176.stdout:4/442: mkdir da/d3d/d17/d54/d5e/d69/d86 0 2024-07-16T09:55:38.180 INFO:tasks.workunit.client.1.smithi176.stdout:2/418: mknod dd/d49/c85 0 2024-07-16T09:55:38.181 INFO:tasks.workunit.client.1.smithi176.stdout:4/443: chown da/dc/d10/d28/f7f 0 1 2024-07-16T09:55:38.183 INFO:tasks.workunit.client.1.smithi176.stdout:3/372: rename d3/d17/d1d/d4d/d64 to d3/d17/d1d/d6e/d6f 0 2024-07-16T09:55:38.183 INFO:tasks.workunit.client.0.smithi156.stdout:9/409: write d2/f4 [3354102,89131] 0 2024-07-16T09:55:38.183 INFO:tasks.workunit.client.0.smithi156.stdout:5/445: mkdir d1b/d99 0 2024-07-16T09:55:38.185 INFO:tasks.workunit.client.1.smithi176.stdout:4/444: rmdir da/dc/d10 39 2024-07-16T09:55:38.186 INFO:tasks.workunit.client.0.smithi156.stdout:0/376: truncate d0/f36 29819 0 2024-07-16T09:55:38.186 INFO:tasks.workunit.client.0.smithi156.stdout:2/505: write d0/d5/d29/d2d/f31 [1254188,26388] 0 2024-07-16T09:55:38.186 INFO:tasks.workunit.client.1.smithi176.stdout:3/373: symlink d3/d17/d1d/d4f/l70 0 2024-07-16T09:55:38.188 INFO:tasks.workunit.client.1.smithi176.stdout:3/374: write d3/f61 [1530086,120730] 0 2024-07-16T09:55:38.190 INFO:tasks.workunit.client.1.smithi176.stdout:3/375: fdatasync d3/d17/f23 0 2024-07-16T09:55:38.190 INFO:tasks.workunit.client.0.smithi156.stdout:5/446: truncate d1b/d1f/d45/d52/d49/f53 208506 0 2024-07-16T09:55:38.191 INFO:tasks.workunit.client.0.smithi156.stdout:2/506: read - d0/d5/d29/d2d/f79 zero size 2024-07-16T09:55:38.192 INFO:tasks.workunit.client.1.smithi176.stdout:0/520: truncate d5/d1c/d24/d35/d50/f7f 116551 0 2024-07-16T09:55:38.192 INFO:tasks.workunit.client.0.smithi156.stdout:5/447: dread - d1b/d1f/d42/f5f zero size 2024-07-16T09:55:38.192 INFO:tasks.workunit.client.0.smithi156.stdout:6/399: rename d3/d13/d75/d19/d31/l6e to d3/d13/d17/d4f/d1e/l89 0 2024-07-16T09:55:38.193 INFO:tasks.workunit.client.0.smithi156.stdout:5/448: fsync d1b/d1f/d45/d52/f68 0 2024-07-16T09:55:38.193 INFO:tasks.workunit.client.1.smithi176.stdout:4/445: dwrite da/d3d/d17/f18 [0,4194304] 0 2024-07-16T09:55:38.194 INFO:tasks.workunit.client.1.smithi176.stdout:3/376: write d3/da/d33/f39 [249871,84694] 0 2024-07-16T09:55:38.194 INFO:tasks.workunit.client.1.smithi176.stdout:2/419: write dd/d16/d39/f4f [742661,15588] 0 2024-07-16T09:55:38.195 INFO:tasks.workunit.client.0.smithi156.stdout:9/410: fdatasync d2/f4d 0 2024-07-16T09:55:38.195 INFO:tasks.workunit.client.0.smithi156.stdout:7/335: truncate dc/d13/d20/f3c 437128 0 2024-07-16T09:55:38.195 INFO:tasks.workunit.client.0.smithi156.stdout:5/449: write d1b/d1f/d42/f5f [368795,13249] 0 2024-07-16T09:55:38.195 INFO:tasks.workunit.client.0.smithi156.stdout:6/400: dread - d3/d13/d17/d4f/d30/f67 zero size 2024-07-16T09:55:38.198 INFO:tasks.workunit.client.0.smithi156.stdout:0/377: mkdir d0/d1a/d4d/d6e 0 2024-07-16T09:55:38.198 INFO:tasks.workunit.client.1.smithi176.stdout:4/446: symlink da/d3d/l87 0 2024-07-16T09:55:38.199 INFO:tasks.workunit.client.1.smithi176.stdout:0/521: mkdir d5/d25/da0 0 2024-07-16T09:55:38.201 INFO:tasks.workunit.client.0.smithi156.stdout:2/507: rmdir d0/d13/d17/d1c/d51 39 2024-07-16T09:55:38.202 INFO:tasks.workunit.client.0.smithi156.stdout:8/330: write d0/d11/d13/d1c/d39/f73 [5062050,38171] 0 2024-07-16T09:55:38.202 INFO:tasks.workunit.client.1.smithi176.stdout:4/447: mknod da/d3d/d33/c88 0 2024-07-16T09:55:38.204 INFO:tasks.workunit.client.0.smithi156.stdout:5/450: write d1b/d1f/d54/f74 [1008405,58021] 0 2024-07-16T09:55:38.204 INFO:tasks.workunit.client.1.smithi176.stdout:4/448: write da/f19 [1252504,28354] 0 2024-07-16T09:55:38.205 INFO:tasks.workunit.client.1.smithi176.stdout:0/522: mkdir d5/da1 0 2024-07-16T09:55:38.205 INFO:tasks.workunit.client.0.smithi156.stdout:4/384: getdents d4/d23 0 2024-07-16T09:55:38.208 INFO:tasks.workunit.client.1.smithi176.stdout:4/449: dread da/d3d/d17/f64 [0,4194304] 0 2024-07-16T09:55:38.210 INFO:tasks.workunit.client.1.smithi176.stdout:2/420: getdents dd/d49/d52/d7c 0 2024-07-16T09:55:38.210 INFO:tasks.workunit.client.1.smithi176.stdout:4/450: chown da/d3d/d17/d54/d78 44654 1 2024-07-16T09:55:38.211 INFO:tasks.workunit.client.1.smithi176.stdout:0/523: mknod d5/d1c/d24/d35/d50/d9b/ca2 0 2024-07-16T09:55:38.213 INFO:tasks.workunit.client.0.smithi156.stdout:3/332: dread d8/fb [0,4194304] 0 2024-07-16T09:55:38.213 INFO:tasks.workunit.client.0.smithi156.stdout:1/579: truncate dc/d14/d3e/d45/f53 2858860 0 2024-07-16T09:55:38.213 INFO:tasks.workunit.client.0.smithi156.stdout:8/331: chown d0/dd/f3c 0 1 2024-07-16T09:55:38.214 INFO:tasks.workunit.client.0.smithi156.stdout:2/508: symlink d0/d5/lb1 0 2024-07-16T09:55:38.215 INFO:tasks.workunit.client.1.smithi176.stdout:0/524: rmdir d5/d1c/d24/d35/d50/d9b 39 2024-07-16T09:55:38.215 INFO:tasks.workunit.client.0.smithi156.stdout:4/385: creat d4/d23/d30/d33/f83 x:0 0 0 2024-07-16T09:55:38.216 INFO:tasks.workunit.client.1.smithi176.stdout:0/525: fsync d5/f7 0 2024-07-16T09:55:38.218 INFO:tasks.workunit.client.0.smithi156.stdout:8/332: unlink d0/d11/f48 0 2024-07-16T09:55:38.219 INFO:tasks.workunit.client.0.smithi156.stdout:7/336: dwrite dc/d44/f5c [0,4194304] 0 2024-07-16T09:55:38.219 INFO:tasks.workunit.client.0.smithi156.stdout:3/333: write d8/da/d13/f24 [3787933,125872] 0 2024-07-16T09:55:38.220 INFO:tasks.workunit.client.1.smithi176.stdout:2/421: dwrite dd/d16/f32 [0,4194304] 0 2024-07-16T09:55:38.222 INFO:tasks.workunit.client.1.smithi176.stdout:3/377: dread d3/f25 [0,4194304] 0 2024-07-16T09:55:38.223 INFO:tasks.workunit.client.1.smithi176.stdout:0/526: mkdir d5/d39/da3 0 2024-07-16T09:55:38.223 INFO:tasks.workunit.client.0.smithi156.stdout:8/333: fsync d0/d11/d13/f76 0 2024-07-16T09:55:38.224 INFO:tasks.workunit.client.0.smithi156.stdout:0/378: dwrite d0/d22/d41/d52/f5d [0,4194304] 0 2024-07-16T09:55:38.224 INFO:tasks.workunit.client.1.smithi176.stdout:3/378: stat d3/d17/d1d/d4d/f42 0 2024-07-16T09:55:38.225 INFO:tasks.workunit.client.0.smithi156.stdout:5/451: symlink d1b/d1f/d45/d52/d43/d85/d97/d37/d88/d61/l9a 0 2024-07-16T09:55:38.225 INFO:tasks.workunit.client.0.smithi156.stdout:9/411: dread d2/f4d [0,4194304] 0 2024-07-16T09:55:38.227 INFO:tasks.workunit.client.0.smithi156.stdout:4/386: truncate d4/db/d47/d4b/f7e 731090 0 2024-07-16T09:55:38.228 INFO:tasks.workunit.client.1.smithi176.stdout:4/451: dwrite da/d3d/d17/d54/d5e/d69/f7e [0,4194304] 0 2024-07-16T09:55:38.229 INFO:tasks.workunit.client.0.smithi156.stdout:1/580: dread - dc/d14/d3e/d45/fb7 zero size 2024-07-16T09:55:38.230 INFO:tasks.workunit.client.0.smithi156.stdout:6/401: dread d3/d13/d17/f55 [0,4194304] 0 2024-07-16T09:55:38.232 INFO:tasks.workunit.client.0.smithi156.stdout:8/334: write d0/d11/d13/d1c/d39/f73 [4770002,51751] 0 2024-07-16T09:55:38.233 INFO:tasks.workunit.client.0.smithi156.stdout:9/412: write d2/d5/d16/d22/d36/d33/f2c [3546604,7507] 0 2024-07-16T09:55:38.233 INFO:tasks.workunit.client.1.smithi176.stdout:2/422: rename dd/cf to dd/de/c86 0 2024-07-16T09:55:38.234 INFO:tasks.workunit.client.0.smithi156.stdout:4/387: chown d4/d23/c56 20331 1 2024-07-16T09:55:38.234 INFO:tasks.workunit.client.0.smithi156.stdout:4/388: fdatasync d4/f43 0 2024-07-16T09:55:38.237 INFO:tasks.workunit.client.0.smithi156.stdout:3/334: symlink d8/da/d28/d40/l72 0 2024-07-16T09:55:38.238 INFO:tasks.workunit.client.0.smithi156.stdout:5/452: rename d1b/d1f/f44 to d1b/d1f/d45/d52/d49/d4d/f9b 0 2024-07-16T09:55:38.238 INFO:tasks.workunit.client.1.smithi176.stdout:0/527: rmdir d5/d2c 39 2024-07-16T09:55:38.240 INFO:tasks.workunit.client.0.smithi156.stdout:7/337: mknod dc/d13/c79 0 2024-07-16T09:55:38.240 INFO:tasks.workunit.client.0.smithi156.stdout:4/389: write d4/db/f18 [2040183,107063] 0 2024-07-16T09:55:38.240 INFO:tasks.workunit.client.0.smithi156.stdout:6/402: chown d3/d13/d17/d4f/d1e 464158 1 2024-07-16T09:55:38.241 INFO:tasks.workunit.client.1.smithi176.stdout:4/452: dread - da/dc/d10/d28/f79 zero size 2024-07-16T09:55:38.243 INFO:tasks.workunit.client.1.smithi176.stdout:3/379: getdents d3/d17/d1d/d6e 0 2024-07-16T09:55:38.244 INFO:tasks.workunit.client.1.smithi176.stdout:0/528: mknod d5/d1c/d5a/d9e/ca4 0 2024-07-16T09:55:38.246 INFO:tasks.workunit.client.0.smithi156.stdout:8/335: fdatasync d0/d11/d13/d1c/f32 0 2024-07-16T09:55:38.247 INFO:tasks.workunit.client.1.smithi176.stdout:4/453: mknod da/d3d/d17/d54/d5e/d69/d86/c89 0 2024-07-16T09:55:38.247 INFO:tasks.workunit.client.1.smithi176.stdout:9/487: sync 2024-07-16T09:55:38.247 INFO:tasks.workunit.client.1.smithi176.stdout:8/475: sync 2024-07-16T09:55:38.247 INFO:tasks.workunit.client.1.smithi176.stdout:7/468: sync 2024-07-16T09:55:38.247 INFO:tasks.workunit.client.1.smithi176.stdout:1/272: sync 2024-07-16T09:55:38.247 INFO:tasks.workunit.client.1.smithi176.stdout:5/477: sync 2024-07-16T09:55:38.248 INFO:tasks.workunit.client.0.smithi156.stdout:1/581: write dc/d14/d3e/d45/d97/fbc [998281,127566] 0 2024-07-16T09:55:38.248 INFO:tasks.workunit.client.0.smithi156.stdout:2/509: getdents d0/d5/d29/d2d/d5b/d62 0 2024-07-16T09:55:38.248 INFO:tasks.workunit.client.1.smithi176.stdout:4/454: dread - da/f47 zero size 2024-07-16T09:55:38.248 INFO:tasks.workunit.client.0.smithi156.stdout:3/335: creat d8/da/d28/d43/d63/f73 x:0 0 0 2024-07-16T09:55:38.249 INFO:tasks.workunit.client.1.smithi176.stdout:0/529: creat d5/d1c/d5a/d9e/fa5 x:0 0 0 2024-07-16T09:55:38.249 INFO:tasks.workunit.client.0.smithi156.stdout:7/338: chown dc/d13/d15/c1b 69637 1 2024-07-16T09:55:38.250 INFO:tasks.workunit.client.1.smithi176.stdout:7/469: stat d4/dd/l71 0 2024-07-16T09:55:38.251 INFO:tasks.workunit.client.1.smithi176.stdout:4/455: chown da/d3d/d33 129724454 1 2024-07-16T09:55:38.253 INFO:tasks.workunit.client.1.smithi176.stdout:8/476: symlink d8/d20/d2a/d61/la0 0 2024-07-16T09:55:38.253 INFO:tasks.workunit.client.1.smithi176.stdout:1/273: chown d4/d8/d12/c53 25734 1 2024-07-16T09:55:38.254 INFO:tasks.workunit.client.0.smithi156.stdout:0/379: fsync d0/d1a/d4d/d9/f26 0 2024-07-16T09:55:38.254 INFO:tasks.workunit.client.1.smithi176.stdout:3/380: getdents d3/d17/d1d/d6e 0 2024-07-16T09:55:38.255 INFO:tasks.workunit.client.0.smithi156.stdout:8/336: truncate d0/d11/d13/f14 1506099 0 2024-07-16T09:55:38.256 INFO:tasks.workunit.client.0.smithi156.stdout:9/413: dwrite d2/d5/d16/d22/d2b/d3d/f68 [0,4194304] 0 2024-07-16T09:55:38.257 INFO:tasks.workunit.client.1.smithi176.stdout:4/456: dread - da/d3d/d17/d54/d78/f7b zero size 2024-07-16T09:55:38.257 INFO:tasks.workunit.client.0.smithi156.stdout:6/403: fdatasync d3/d13/d17/d4f/d4d/f61 0 2024-07-16T09:55:38.257 INFO:tasks.workunit.client.0.smithi156.stdout:5/453: unlink d1b/d1f/d45/d52/d43/d85/d97/f89 0 2024-07-16T09:55:38.258 INFO:tasks.workunit.client.0.smithi156.stdout:7/339: creat dc/d13/d36/f7a x:0 0 0 2024-07-16T09:55:38.259 INFO:tasks.workunit.client.1.smithi176.stdout:8/477: chown d8/d20/d2a/l3a 33929659 1 2024-07-16T09:55:38.260 INFO:tasks.workunit.client.1.smithi176.stdout:7/470: creat d4/d1f/d22/d80/d17/d1d/d8b/f9d x:0 0 0 2024-07-16T09:55:38.261 INFO:tasks.workunit.client.1.smithi176.stdout:0/530: fdatasync d5/d1c/d24/d35/f4c 0 2024-07-16T09:55:38.261 INFO:tasks.workunit.client.0.smithi156.stdout:3/336: symlink d8/l74 0 2024-07-16T09:55:38.263 INFO:tasks.workunit.client.1.smithi176.stdout:0/531: readlink d5/d1c/l1d 0 2024-07-16T09:55:38.264 INFO:tasks.workunit.client.0.smithi156.stdout:9/414: write d2/d5/d16/d22/d2b/d3d/f69 [4277039,26219] 0 2024-07-16T09:55:38.266 INFO:tasks.workunit.client.0.smithi156.stdout:3/337: write d8/da/d21/f3e [7211458,97729] 0 2024-07-16T09:55:38.266 INFO:tasks.workunit.client.0.smithi156.stdout:6/404: read d3/d13/d75/d19/d31/f4e [3258955,85956] 0 2024-07-16T09:55:38.266 INFO:tasks.workunit.client.1.smithi176.stdout:5/478: rename d4/d37/d72/d38/d6e/f7b to d4/d10/faf 0 2024-07-16T09:55:38.268 INFO:tasks.workunit.client.1.smithi176.stdout:7/471: chown d4/d47/l75 1 1 2024-07-16T09:55:38.269 INFO:tasks.workunit.client.1.smithi176.stdout:3/381: mkdir d3/d17/d1d/d4f/d6d/d71 0 2024-07-16T09:55:38.270 INFO:tasks.workunit.client.0.smithi156.stdout:4/390: getdents d4/db 0 2024-07-16T09:55:38.272 INFO:tasks.workunit.client.0.smithi156.stdout:1/582: getdents dc/d14/d1e 0 2024-07-16T09:55:38.274 INFO:tasks.workunit.client.1.smithi176.stdout:0/532: rename d5/f19 to d5/d2c/d6c/d83/fa6 0 2024-07-16T09:55:38.275 INFO:tasks.workunit.client.1.smithi176.stdout:2/423: dread dd/d2e/f37 [0,4194304] 0 2024-07-16T09:55:38.276 INFO:tasks.workunit.client.1.smithi176.stdout:4/457: dwrite da/dc/d4b/f52 [0,4194304] 0 2024-07-16T09:55:38.277 INFO:tasks.workunit.client.1.smithi176.stdout:7/472: mknod d4/d1f/d22/c9e 0 2024-07-16T09:55:38.279 INFO:tasks.workunit.client.0.smithi156.stdout:9/415: creat d2/d5/d16/d57/d63/f81 x:0 0 0 2024-07-16T09:55:38.279 INFO:tasks.workunit.client.0.smithi156.stdout:5/454: dwrite d1b/d1f/d45/d52/d43/d85/f8f [0,4194304] 0 2024-07-16T09:55:38.279 INFO:tasks.workunit.client.1.smithi176.stdout:7/473: read d4/d47/f88 [715108,50390] 0 2024-07-16T09:55:38.281 INFO:tasks.workunit.client.0.smithi156.stdout:3/338: rmdir d8/da/d1d/d2c/d33 39 2024-07-16T09:55:38.283 INFO:tasks.workunit.client.0.smithi156.stdout:7/340: link dc/d13/d15/l2a dc/d13/d24/l7b 0 2024-07-16T09:55:38.285 INFO:tasks.workunit.client.0.smithi156.stdout:9/416: readlink d2/l43 0 2024-07-16T09:55:38.285 INFO:tasks.workunit.client.0.smithi156.stdout:1/583: creat dc/d14/d3e/d55/fca x:0 0 0 2024-07-16T09:55:38.285 INFO:tasks.workunit.client.1.smithi176.stdout:1/274: dread d4/db/f1b [4194304,4194304] 0 2024-07-16T09:55:38.285 INFO:tasks.workunit.client.1.smithi176.stdout:1/275: fsync d4/d8/f26 0 2024-07-16T09:55:38.287 INFO:tasks.workunit.client.1.smithi176.stdout:3/382: symlink d3/da/l72 0 2024-07-16T09:55:38.288 INFO:tasks.workunit.client.0.smithi156.stdout:8/337: dread d0/d11/d13/d1c/d39/f73 [0,4194304] 0 2024-07-16T09:55:38.288 INFO:tasks.workunit.client.1.smithi176.stdout:5/479: truncate d4/d10/f33 2376436 0 2024-07-16T09:55:38.288 INFO:tasks.workunit.client.1.smithi176.stdout:2/424: readlink dd/d36/l4e 0 2024-07-16T09:55:38.288 INFO:tasks.workunit.client.1.smithi176.stdout:0/533: dread d5/d1c/d24/d35/d50/d58/d63/f66 [0,4194304] 0 2024-07-16T09:55:38.292 INFO:tasks.workunit.client.1.smithi176.stdout:3/383: dread - d3/da/f41 zero size 2024-07-16T09:55:38.292 INFO:tasks.workunit.client.0.smithi156.stdout:2/510: truncate d0/d5/d29/d2d/f31 4031888 0 2024-07-16T09:55:38.295 INFO:tasks.workunit.client.1.smithi176.stdout:9/488: rename d5/de/d37/d58/d89/d8a to d5/de/da5 0 2024-07-16T09:55:38.297 INFO:tasks.workunit.client.0.smithi156.stdout:4/391: dwrite d4/db/d47/d4b/f5e [0,4194304] 0 2024-07-16T09:55:38.297 INFO:tasks.workunit.client.1.smithi176.stdout:1/276: write d4/d8/d12/d21/d25/f60 [526866,111454] 0 2024-07-16T09:55:38.299 INFO:tasks.workunit.client.1.smithi176.stdout:3/384: symlink d3/da/d33/l73 0 2024-07-16T09:55:38.301 INFO:tasks.workunit.client.1.smithi176.stdout:4/458: dwrite da/dc/f83 [0,4194304] 0 2024-07-16T09:55:38.301 INFO:tasks.workunit.client.1.smithi176.stdout:7/474: dwrite d4/d1f/d22/d80/d17/d1d/f9b [4194304,4194304] 0 2024-07-16T09:55:38.302 INFO:tasks.workunit.client.0.smithi156.stdout:1/584: rename dc/d14/d3e/d45/d97 to dc/d14/d3e/d6c/d7a/dcb 0 2024-07-16T09:55:38.302 INFO:tasks.workunit.client.0.smithi156.stdout:4/392: read d4/db/dc/f4d [3811326,17479] 0 2024-07-16T09:55:38.302 INFO:tasks.workunit.client.0.smithi156.stdout:9/417: fsync d2/d5/d16/d57/d63/f81 0 2024-07-16T09:55:38.302 INFO:tasks.workunit.client.1.smithi176.stdout:0/534: rename d5/d1c/f21 to d5/d25/d82/fa7 0 2024-07-16T09:55:38.303 INFO:tasks.workunit.client.0.smithi156.stdout:5/455: chown d1b/d1f/d45/d52/d43/d85/d97/f48 208 1 2024-07-16T09:55:38.303 INFO:tasks.workunit.client.1.smithi176.stdout:9/489: creat d5/d1d/d77/d97/fa6 x:0 0 0 2024-07-16T09:55:38.304 INFO:tasks.workunit.client.1.smithi176.stdout:1/277: truncate d4/f24 4771037 0 2024-07-16T09:55:38.307 INFO:tasks.workunit.client.0.smithi156.stdout:5/456: chown d1b/d1f/d45/d52/d2d 987673229 1 2024-07-16T09:55:38.307 INFO:tasks.workunit.client.0.smithi156.stdout:1/585: chown dc/d14/d3e/lb3 689756 1 2024-07-16T09:55:38.311 INFO:tasks.workunit.client.0.smithi156.stdout:8/338: dwrite d0/dd/f33 [0,4194304] 0 2024-07-16T09:55:38.311 INFO:tasks.workunit.client.0.smithi156.stdout:7/341: rename dc/d13/d36/d56/f5a to dc/d48/f7c 0 2024-07-16T09:55:38.313 INFO:tasks.workunit.client.1.smithi176.stdout:1/278: mknod d4/d8/d12/d21/c62 0 2024-07-16T09:55:38.313 INFO:tasks.workunit.client.0.smithi156.stdout:4/393: mknod d4/d23/c84 0 2024-07-16T09:55:38.313 INFO:tasks.workunit.client.0.smithi156.stdout:5/457: fsync d1b/d1f/d45/d52/d43/d85/d97/f48 0 2024-07-16T09:55:38.313 INFO:tasks.workunit.client.1.smithi176.stdout:1/279: readlink d4/d8/lc 0 2024-07-16T09:55:38.317 INFO:tasks.workunit.client.0.smithi156.stdout:9/418: creat d2/d5/d16/d22/d36/d33/f82 x:0 0 0 2024-07-16T09:55:38.317 INFO:tasks.workunit.client.0.smithi156.stdout:3/339: dread d8/da/f32 [0,4194304] 0 2024-07-16T09:55:38.318 INFO:tasks.workunit.client.1.smithi176.stdout:0/535: dread d5/d1c/d24/d35/d50/d58/d63/f66 [0,4194304] 0 2024-07-16T09:55:38.319 INFO:tasks.workunit.client.1.smithi176.stdout:3/385: dwrite d3/d17/d1d/d4d/f3f [0,4194304] 0 2024-07-16T09:55:38.321 INFO:tasks.workunit.client.1.smithi176.stdout:0/536: fdatasync d5/d2c/d6c/f64 0 2024-07-16T09:55:38.321 INFO:tasks.workunit.client.0.smithi156.stdout:2/511: dread d0/d13/d17/f42 [0,4194304] 0 2024-07-16T09:55:38.322 INFO:tasks.workunit.client.1.smithi176.stdout:1/280: creat d4/d8/d12/d21/f63 x:0 0 0 2024-07-16T09:55:38.324 INFO:tasks.workunit.client.1.smithi176.stdout:9/490: dwrite d5/de/d37/d58/f5c [0,4194304] 0 2024-07-16T09:55:38.326 INFO:tasks.workunit.client.0.smithi156.stdout:0/380: dread d0/ff [0,4194304] 0 2024-07-16T09:55:38.327 INFO:tasks.workunit.client.0.smithi156.stdout:9/419: fdatasync d2/d5/d16/d57/d63/f6e 0 2024-07-16T09:55:38.331 INFO:tasks.workunit.client.0.smithi156.stdout:7/342: read dc/f14 [3735860,119709] 0 2024-07-16T09:55:38.333 INFO:tasks.workunit.client.0.smithi156.stdout:7/343: dread - dc/d13/d36/d56/f61 zero size 2024-07-16T09:55:38.334 INFO:tasks.workunit.client.1.smithi176.stdout:1/281: rmdir d4/d8/d12/d21/d25/d30 39 2024-07-16T09:55:38.335 INFO:tasks.workunit.client.0.smithi156.stdout:1/586: dread dc/d14/d3e/d55/fad [0,4194304] 0 2024-07-16T09:55:38.335 INFO:tasks.workunit.client.1.smithi176.stdout:9/491: symlink d5/d1d/d67/d6d/la7 0 2024-07-16T09:55:38.336 INFO:tasks.workunit.client.0.smithi156.stdout:5/458: rename d1b/c41 to d1b/d1f/d45/d52/d49/d4d/c9c 0 2024-07-16T09:55:38.337 INFO:tasks.workunit.client.1.smithi176.stdout:5/480: dread d4/d37/d72/d38/d6e/f74 [0,4194304] 0 2024-07-16T09:55:38.338 INFO:tasks.workunit.client.0.smithi156.stdout:3/340: creat d8/da/d13/f75 x:0 0 0 2024-07-16T09:55:38.338 INFO:tasks.workunit.client.0.smithi156.stdout:2/512: symlink d0/d5/d29/d2d/lb2 0 2024-07-16T09:55:38.338 INFO:tasks.workunit.client.0.smithi156.stdout:9/420: mknod d2/d5/d16/d22/d36/d33/d3a/c83 0 2024-07-16T09:55:38.342 INFO:tasks.workunit.client.0.smithi156.stdout:1/587: rename dc/d14 to dc/d14/dcc 22 2024-07-16T09:55:38.343 INFO:tasks.workunit.client.0.smithi156.stdout:7/344: write dc/d13/f43 [2046040,26756] 0 2024-07-16T09:55:38.343 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:55:37 smithi176 ceph-mon[36626]: pgmap v250: 65 pgs: 65 active+clean; 1.4 GiB data, 5.4 GiB used, 531 GiB / 536 GiB avail; 22 MiB/s rd, 177 MiB/s wr, 380 op/s 2024-07-16T09:55:38.344 INFO:tasks.workunit.client.0.smithi156.stdout:4/394: truncate d4/d23/d52/f69 424129 0 2024-07-16T09:55:38.344 INFO:tasks.workunit.client.0.smithi156.stdout:2/513: fdatasync d0/d13/d6e/fa2 0 2024-07-16T09:55:38.346 INFO:tasks.workunit.client.0.smithi156.stdout:8/339: getdents d0/d11/d13/d1c 0 2024-07-16T09:55:38.348 INFO:tasks.workunit.client.0.smithi156.stdout:6/405: dread d3/d13/d17/d4f/d1e/f20 [0,4194304] 0 2024-07-16T09:55:38.348 INFO:tasks.workunit.client.0.smithi156.stdout:2/514: readlink d0/d13/d17/d1c/l36 0 2024-07-16T09:55:38.349 INFO:tasks.workunit.client.0.smithi156.stdout:1/588: write dc/d14/d8b/fc4 [341333,25705] 0 2024-07-16T09:55:38.349 INFO:tasks.workunit.client.0.smithi156.stdout:4/395: creat d4/d23/d52/f85 x:0 0 0 2024-07-16T09:55:38.350 INFO:tasks.workunit.client.0.smithi156.stdout:5/459: dwrite d1b/d1f/d45/d52/f68 [0,4194304] 0 2024-07-16T09:55:38.350 INFO:tasks.workunit.client.0.smithi156.stdout:7/345: write dc/d13/d15/f50 [3508587,108335] 0 2024-07-16T09:55:38.351 INFO:tasks.workunit.client.0.smithi156.stdout:8/340: creat d0/d11/d29/f79 x:0 0 0 2024-07-16T09:55:38.355 INFO:tasks.workunit.client.0.smithi156.stdout:8/341: write d0/d11/f15 [1527460,38370] 0 2024-07-16T09:55:38.357 INFO:tasks.workunit.client.0.smithi156.stdout:7/346: truncate dc/d3e/d3f/d51/f53 1850796 0 2024-07-16T09:55:38.359 INFO:tasks.workunit.client.0.smithi156.stdout:4/396: creat d4/db/dc/d67/f86 x:0 0 0 2024-07-16T09:55:38.360 INFO:tasks.workunit.client.0.smithi156.stdout:5/460: mkdir d1b/d1f/d45/d9d 0 2024-07-16T09:55:38.360 INFO:tasks.workunit.client.0.smithi156.stdout:7/347: dread - dc/d13/d36/f7a zero size 2024-07-16T09:55:38.361 INFO:tasks.workunit.client.0.smithi156.stdout:8/342: write d0/fb [2460496,16727] 0 2024-07-16T09:55:38.363 INFO:tasks.workunit.client.0.smithi156.stdout:7/348: write dc/f40 [768693,27950] 0 2024-07-16T09:55:38.364 INFO:tasks.workunit.client.0.smithi156.stdout:1/589: dwrite dc/d14/d3e/d55/d58/fc9 [0,4194304] 0 2024-07-16T09:55:38.365 INFO:tasks.workunit.client.0.smithi156.stdout:3/341: truncate d8/da/d13/f5c 301793 0 2024-07-16T09:55:38.366 INFO:tasks.workunit.client.0.smithi156.stdout:5/461: truncate d1b/d1f/d45/d52/d43/f63 225990 0 2024-07-16T09:55:38.368 INFO:tasks.workunit.client.0.smithi156.stdout:8/343: rename d0/fb to d0/d11/d20/f7a 0 2024-07-16T09:55:38.369 INFO:tasks.workunit.client.0.smithi156.stdout:4/397: mkdir d4/db/dc/d7f/d87 0 2024-07-16T09:55:38.370 INFO:tasks.workunit.client.0.smithi156.stdout:9/421: truncate d2/d5/d16/d22/d2b/d3d/f69 6766175 0 2024-07-16T09:55:38.372 INFO:tasks.workunit.client.0.smithi156.stdout:2/515: write d0/d13/d17/f30 [1963870,123454] 0 2024-07-16T09:55:38.373 INFO:tasks.workunit.client.0.smithi156.stdout:5/462: symlink d1b/d1f/d45/d7a/l9e 0 2024-07-16T09:55:38.374 INFO:tasks.workunit.client.0.smithi156.stdout:8/344: mknod d0/d11/d4a/c7b 0 2024-07-16T09:55:38.374 INFO:tasks.workunit.client.0.smithi156.stdout:9/422: dread - d2/d5/d16/d22/d36/d33/d23/f60 zero size 2024-07-16T09:55:38.377 INFO:tasks.workunit.client.0.smithi156.stdout:1/590: dwrite dc/d24/fa0 [0,4194304] 0 2024-07-16T09:55:38.377 INFO:tasks.workunit.client.0.smithi156.stdout:5/463: read - d1b/d1f/d45/d52/d43/f8d zero size 2024-07-16T09:55:38.378 INFO:tasks.workunit.client.0.smithi156.stdout:5/464: fdatasync f19 0 2024-07-16T09:55:38.380 INFO:tasks.workunit.client.0.smithi156.stdout:3/342: dread d8/da/d1d/f53 [0,4194304] 0 2024-07-16T09:55:38.380 INFO:tasks.workunit.client.0.smithi156.stdout:9/423: write d2/d5/d16/d22/d36/f31 [3935776,74729] 0 2024-07-16T09:55:38.381 INFO:tasks.workunit.client.0.smithi156.stdout:1/591: read - dc/d14/d1e/fc7 zero size 2024-07-16T09:55:38.383 INFO:tasks.workunit.client.0.smithi156.stdout:8/345: rename d0/d74/f63 to d0/dd/d1f/d60/f7c 0 2024-07-16T09:55:38.384 INFO:tasks.workunit.client.0.smithi156.stdout:2/516: creat d0/d5/d29/d2d/d5b/d62/fb3 x:0 0 0 2024-07-16T09:55:38.385 INFO:tasks.workunit.client.1.smithi176.stdout:2/425: truncate dd/d16/f27 928248 0 2024-07-16T09:55:38.385 INFO:tasks.workunit.client.0.smithi156.stdout:4/398: dread d4/d23/f39 [0,4194304] 0 2024-07-16T09:55:38.387 INFO:tasks.workunit.client.0.smithi156.stdout:5/465: unlink d1b/d1f/d45/d52/d2d/f76 0 2024-07-16T09:55:38.389 INFO:tasks.workunit.client.0.smithi156.stdout:1/592: mkdir dc/d14/d8b/dcd 0 2024-07-16T09:55:38.390 INFO:tasks.workunit.client.0.smithi156.stdout:3/343: symlink d8/da/d13/l76 0 2024-07-16T09:55:38.391 INFO:tasks.workunit.client.1.smithi176.stdout:9/492: mknod d5/d1d/d20/d78/ca8 0 2024-07-16T09:55:38.392 INFO:tasks.workunit.client.1.smithi176.stdout:1/282: rmdir d4 39 2024-07-16T09:55:38.392 INFO:tasks.workunit.client.0.smithi156.stdout:6/406: dread d3/f25 [4194304,4194304] 0 2024-07-16T09:55:38.392 INFO:tasks.workunit.client.1.smithi176.stdout:4/459: dwrite da/dc/f45 [0,4194304] 0 2024-07-16T09:55:38.392 INFO:tasks.workunit.client.1.smithi176.stdout:3/386: creat d3/f74 x:0 0 0 2024-07-16T09:55:38.395 INFO:tasks.workunit.client.1.smithi176.stdout:4/460: chown da/d14/d40/d5d/f85 1105 1 2024-07-16T09:55:38.395 INFO:tasks.workunit.client.1.smithi176.stdout:3/387: write d3/d17/d1d/d4d/f4e [3596171,84272] 0 2024-07-16T09:55:38.396 INFO:tasks.workunit.client.1.smithi176.stdout:4/461: fsync da/f5f 0 2024-07-16T09:55:38.397 INFO:tasks.workunit.client.0.smithi156.stdout:3/344: write d8/da/d28/f6e [981525,97467] 0 2024-07-16T09:55:38.397 INFO:tasks.workunit.client.0.smithi156.stdout:6/407: stat d3/d13/d17/d4f/f83 0 2024-07-16T09:55:38.398 INFO:tasks.workunit.client.1.smithi176.stdout:0/537: truncate d5/d1c/d24/d35/d50/d58/d63/f66 3112517 0 2024-07-16T09:55:38.399 INFO:tasks.workunit.client.1.smithi176.stdout:5/481: truncate d4/f86 2432453 0 2024-07-16T09:55:38.400 INFO:tasks.workunit.client.0.smithi156.stdout:9/424: dwrite d2/d5/d16/d57/d63/f6e [0,4194304] 0 2024-07-16T09:55:38.401 INFO:tasks.workunit.client.1.smithi176.stdout:2/426: creat dd/d16/d39/d3d/f87 x:0 0 0 2024-07-16T09:55:38.401 INFO:tasks.workunit.client.0.smithi156.stdout:1/593: dread dc/f19 [0,4194304] 0 2024-07-16T09:55:38.406 INFO:tasks.workunit.client.1.smithi176.stdout:9/493: mknod d5/de/d37/ca9 0 2024-07-16T09:55:38.408 INFO:tasks.workunit.client.0.smithi156.stdout:5/466: rename d1b/d1f/d45/d7a/d94 to d1b/d1f/d42/d6d/d9f 0 2024-07-16T09:55:38.408 INFO:tasks.workunit.client.1.smithi176.stdout:5/482: dwrite d4/d37/d72/d38/d39/d9f/fa6 [0,4194304] 0 2024-07-16T09:55:38.409 INFO:tasks.workunit.client.1.smithi176.stdout:5/483: readlink d4/d14/d19/l4a 0 2024-07-16T09:55:38.409 INFO:tasks.workunit.client.0.smithi156.stdout:6/408: dread d3/d13/d17/d56/f64 [0,4194304] 0 2024-07-16T09:55:38.410 INFO:tasks.workunit.client.1.smithi176.stdout:1/283: unlink d4/d8/f29 0 2024-07-16T09:55:38.410 INFO:tasks.workunit.client.1.smithi176.stdout:4/462: mknod da/d3d/d17/d48/d82/c8a 0 2024-07-16T09:55:38.411 INFO:tasks.workunit.client.1.smithi176.stdout:6/446: sync 2024-07-16T09:55:38.411 INFO:tasks.workunit.client.1.smithi176.stdout:4/463: readlink da/dc/l70 0 2024-07-16T09:55:38.411 INFO:tasks.workunit.client.1.smithi176.stdout:0/538: symlink d5/d25/la8 0 2024-07-16T09:55:38.413 INFO:tasks.workunit.client.0.smithi156.stdout:1/594: dread dc/d14/d1e/f29 [0,4194304] 0 2024-07-16T09:55:38.413 INFO:tasks.workunit.client.1.smithi176.stdout:6/447: write d1/d5/f4f [32772,112544] 0 2024-07-16T09:55:38.413 INFO:tasks.workunit.client.0.smithi156.stdout:4/399: creat d4/db/dc/f88 x:0 0 0 2024-07-16T09:55:38.414 INFO:tasks.workunit.client.1.smithi176.stdout:4/464: chown da/d3d/d17/l5c 42724 1 2024-07-16T09:55:38.415 INFO:tasks.workunit.client.0.smithi156.stdout:9/425: getdents d2/d5/d16/d22/d6b 0 2024-07-16T09:55:38.416 INFO:tasks.workunit.client.0.smithi156.stdout:9/426: readlink d2/d5/d16/d22/d36/l28 0 2024-07-16T09:55:38.416 INFO:tasks.workunit.client.1.smithi176.stdout:6/448: dread - d1/dc/d40/f78 zero size 2024-07-16T09:55:38.417 INFO:tasks.workunit.client.1.smithi176.stdout:2/427: mkdir dd/d49/d52/d7c/d81/d88 0 2024-07-16T09:55:38.418 INFO:tasks.workunit.client.0.smithi156.stdout:2/517: creat d0/d5/d29/d2d/d5b/d62/d74/fb4 x:0 0 0 2024-07-16T09:55:38.422 INFO:tasks.workunit.client.0.smithi156.stdout:1/595: unlink dc/d14/d8b/fc4 0 2024-07-16T09:55:38.423 INFO:tasks.workunit.client.1.smithi176.stdout:9/494: dwrite d5/fb [0,4194304] 0 2024-07-16T09:55:38.423 INFO:tasks.workunit.client.0.smithi156.stdout:2/518: dread d0/d5/f93 [0,4194304] 0 2024-07-16T09:55:38.424 INFO:tasks.workunit.client.1.smithi176.stdout:0/539: rename d5/d44/l56 to d5/d1c/d24/d35/d59/la9 0 2024-07-16T09:55:38.424 INFO:tasks.workunit.client.0.smithi156.stdout:8/346: dread d0/dd/f1e [0,4194304] 0 2024-07-16T09:55:38.425 INFO:tasks.workunit.client.1.smithi176.stdout:9/495: fsync d5/d1d/d20/f81 0 2024-07-16T09:55:38.425 INFO:tasks.workunit.client.0.smithi156.stdout:2/519: write d0/d5/d29/f3f [4326,11741] 0 2024-07-16T09:55:38.427 INFO:tasks.workunit.client.0.smithi156.stdout:2/520: fdatasync d0/d5/f2a 0 2024-07-16T09:55:38.427 INFO:tasks.workunit.client.1.smithi176.stdout:3/388: truncate d3/d17/d1d/d4d/f3f 3449797 0 2024-07-16T09:55:38.429 INFO:tasks.workunit.client.0.smithi156.stdout:9/427: symlink d2/d5/d16/d22/l84 0 2024-07-16T09:55:38.430 INFO:tasks.workunit.client.1.smithi176.stdout:2/428: dread dd/d2e/f37 [0,4194304] 0 2024-07-16T09:55:38.430 INFO:tasks.workunit.client.1.smithi176.stdout:6/449: creat d1/d5/d47/fa7 x:0 0 0 2024-07-16T09:55:38.431 INFO:tasks.workunit.client.0.smithi156.stdout:5/467: link d1b/d1f/d45/d52/d49/f81 d1b/d1f/d45/d7f/fa0 0 2024-07-16T09:55:38.432 INFO:tasks.workunit.client.1.smithi176.stdout:9/496: symlink d5/d1d/d67/laa 0 2024-07-16T09:55:38.433 INFO:tasks.workunit.client.0.smithi156.stdout:6/409: creat d3/d13/d17/f8a x:0 0 0 2024-07-16T09:55:38.435 INFO:tasks.workunit.client.0.smithi156.stdout:8/347: creat d0/d11/d4a/d55/f7d x:0 0 0 2024-07-16T09:55:38.436 INFO:tasks.workunit.client.0.smithi156.stdout:3/345: truncate d8/da/d13/f5c 421164 0 2024-07-16T09:55:38.436 INFO:tasks.workunit.client.1.smithi176.stdout:3/389: creat d3/da/d33/f75 x:0 0 0 2024-07-16T09:55:38.437 INFO:tasks.workunit.client.1.smithi176.stdout:0/540: dwrite f3 [0,4194304] 0 2024-07-16T09:55:38.439 INFO:tasks.workunit.client.1.smithi176.stdout:6/450: creat d1/dc/d5b/fa8 x:0 0 0 2024-07-16T09:55:38.440 INFO:tasks.workunit.client.1.smithi176.stdout:3/390: write d3/da/d33/f39 [68601,13602] 0 2024-07-16T09:55:38.440 INFO:tasks.workunit.client.0.smithi156.stdout:6/410: write d3/d13/d75/fa [2289064,106225] 0 2024-07-16T09:55:38.441 INFO:tasks.workunit.client.0.smithi156.stdout:8/348: dread - d0/d11/d20/d3f/f69 zero size 2024-07-16T09:55:38.443 INFO:tasks.workunit.client.0.smithi156.stdout:3/346: creat d8/da/d28/d43/f77 x:0 0 0 2024-07-16T09:55:38.443 INFO:tasks.workunit.client.1.smithi176.stdout:6/451: fdatasync d1/d5/d47/d62/fa1 0 2024-07-16T09:55:38.443 INFO:tasks.workunit.client.1.smithi176.stdout:2/429: rename dd/d49/d52/d7c/d81/d88 to dd/d49/d52/d89 0 2024-07-16T09:55:38.443 INFO:tasks.workunit.client.0.smithi156.stdout:4/400: dwrite d4/d23/d30/f2d [0,4194304] 0 2024-07-16T09:55:38.444 INFO:tasks.workunit.client.1.smithi176.stdout:3/391: chown d3/da/cc 187756016 1 2024-07-16T09:55:38.444 INFO:tasks.workunit.client.1.smithi176.stdout:1/284: link d4/d8/ld d4/d8/d12/d21/d25/d30/l64 0 2024-07-16T09:55:38.445 INFO:tasks.workunit.client.1.smithi176.stdout:5/484: dwrite d4/d10/f5f [0,4194304] 0 2024-07-16T09:55:38.445 INFO:tasks.workunit.client.0.smithi156.stdout:9/428: creat d2/d5/d16/d22/d2b/d3d/d3c/d7f/d42/f85 x:0 0 0 2024-07-16T09:55:38.448 INFO:tasks.workunit.client.0.smithi156.stdout:5/468: unlink d1b/d1f/d45/d52/d43/d85/d97/f3d 0 2024-07-16T09:55:38.448 INFO:tasks.workunit.client.0.smithi156.stdout:6/411: unlink d3/d13/d17/d4f/d30/c82 0 2024-07-16T09:55:38.449 INFO:tasks.workunit.client.1.smithi176.stdout:1/285: fdatasync d4/d8/fa 0 2024-07-16T09:55:38.449 INFO:tasks.workunit.client.0.smithi156.stdout:8/349: chown d0/c4 1931 1 2024-07-16T09:55:38.449 INFO:tasks.workunit.client.0.smithi156.stdout:3/347: write d8/da/d28/d40/f56 [325819,2170] 0 2024-07-16T09:55:38.451 INFO:tasks.workunit.client.0.smithi156.stdout:5/469: chown d1b/d1f/d45/d52/d2d 53031 1 2024-07-16T09:55:38.452 INFO:tasks.workunit.client.1.smithi176.stdout:9/497: getdents d5/de/da5 0 2024-07-16T09:55:38.452 INFO:tasks.workunit.client.1.smithi176.stdout:1/286: fdatasync d4/db/f43 0 2024-07-16T09:55:38.453 INFO:tasks.workunit.client.0.smithi156.stdout:9/429: mkdir d2/d5/d16/d22/d6b/d86 0 2024-07-16T09:55:38.454 INFO:tasks.workunit.client.0.smithi156.stdout:4/401: symlink d4/d23/d52/d71/l89 0 2024-07-16T09:55:38.457 INFO:tasks.workunit.client.1.smithi176.stdout:2/430: rename dd/de/c1b to dd/d2e/d35/d4a/c8a 0 2024-07-16T09:55:38.457 INFO:tasks.workunit.client.1.smithi176.stdout:5/485: creat d4/d37/d72/d82/fb0 x:0 0 0 2024-07-16T09:55:38.459 INFO:tasks.workunit.client.1.smithi176.stdout:6/452: stat d1/d5/d7/d1e/f79 0 2024-07-16T09:55:38.459 INFO:tasks.workunit.client.1.smithi176.stdout:0/541: dwrite d5/f22 [0,4194304] 0 2024-07-16T09:55:38.459 INFO:tasks.workunit.client.1.smithi176.stdout:9/498: write d5/de/d37/d58/f60 [4947882,27044] 0 2024-07-16T09:55:38.460 INFO:tasks.workunit.client.0.smithi156.stdout:5/470: write f19 [2994562,82147] 0 2024-07-16T09:55:38.460 INFO:tasks.workunit.client.1.smithi176.stdout:1/287: stat d4/d8/ld 0 2024-07-16T09:55:38.460 INFO:tasks.workunit.client.1.smithi176.stdout:2/431: readlink dd/d16/l5b 0 2024-07-16T09:55:38.461 INFO:tasks.workunit.client.1.smithi176.stdout:4/465: dread da/d3d/d17/f35 [0,4194304] 0 2024-07-16T09:55:38.463 INFO:tasks.workunit.client.0.smithi156.stdout:8/350: creat d0/dd/d1f/d57/f7e x:0 0 0 2024-07-16T09:55:38.463 INFO:tasks.workunit.client.1.smithi176.stdout:0/542: write d5/d1c/d24/d35/d50/d58/f60 [1504809,117456] 0 2024-07-16T09:55:38.463 INFO:tasks.workunit.client.0.smithi156.stdout:2/521: dread d0/d13/d17/f30 [0,4194304] 0 2024-07-16T09:55:38.466 INFO:tasks.workunit.client.0.smithi156.stdout:5/471: chown d1b/d1f/d45/d52/d49/f53 2301789 1 2024-07-16T09:55:38.466 INFO:tasks.workunit.client.1.smithi176.stdout:2/432: truncate dd/d16/d39/f6d 585999 0 2024-07-16T09:55:38.466 INFO:tasks.workunit.client.1.smithi176.stdout:2/433: dread - dd/f77 zero size 2024-07-16T09:55:38.467 INFO:tasks.workunit.client.0.smithi156.stdout:1/596: dread dc/d14/d3e/d55/d83/fa4 [0,4194304] 0 2024-07-16T09:55:38.467 INFO:tasks.workunit.client.1.smithi176.stdout:6/453: read d1/dc/fd [450718,59446] 0 2024-07-16T09:55:38.467 INFO:tasks.workunit.client.1.smithi176.stdout:9/499: creat d5/d1d/d67/fab x:0 0 0 2024-07-16T09:55:38.469 INFO:tasks.workunit.client.0.smithi156.stdout:4/402: getdents d4/db/dc/d67 0 2024-07-16T09:55:38.471 INFO:tasks.workunit.client.0.smithi156.stdout:3/348: dwrite d8/da/d28/f5e [0,4194304] 0 2024-07-16T09:55:38.472 INFO:tasks.workunit.client.0.smithi156.stdout:8/351: truncate d0/dd/d1f/f47 485781 0 2024-07-16T09:55:38.472 INFO:tasks.workunit.client.0.smithi156.stdout:5/472: fsync d1b/d1f/d45/d52/d49/d4d/f80 0 2024-07-16T09:55:38.473 INFO:tasks.workunit.client.1.smithi176.stdout:0/543: rename d5/f6 to d5/d25/da0/faa 0 2024-07-16T09:55:38.474 INFO:tasks.workunit.client.0.smithi156.stdout:3/349: dread - d8/d14/f59 zero size 2024-07-16T09:55:38.474 INFO:tasks.workunit.client.0.smithi156.stdout:8/352: read - d0/d11/d2e/d35/f46 zero size 2024-07-16T09:55:38.476 INFO:tasks.workunit.client.1.smithi176.stdout:0/544: write d5/d44/f55 [572612,60860] 0 2024-07-16T09:55:38.477 INFO:tasks.workunit.client.0.smithi156.stdout:6/412: getdents d3/d13/d17/d4f/d30 0 2024-07-16T09:55:38.478 INFO:tasks.workunit.client.0.smithi156.stdout:8/353: chown d0/dd/l64 28121949 1 2024-07-16T09:55:38.478 INFO:tasks.workunit.client.0.smithi156.stdout:2/522: mknod d0/d13/cb5 0 2024-07-16T09:55:38.479 INFO:tasks.workunit.client.1.smithi176.stdout:5/486: creat d4/d10/fb1 x:0 0 0 2024-07-16T09:55:38.480 INFO:tasks.workunit.client.0.smithi156.stdout:4/403: mkdir d4/db/d8a 0 2024-07-16T09:55:38.484 INFO:tasks.workunit.client.0.smithi156.stdout:8/354: truncate d0/dd/f3c 1214194 0 2024-07-16T09:55:38.489 INFO:tasks.workunit.client.0.smithi156.stdout:2/523: chown d0/d5/d29/d2d/d5b/d5f/d89/f99 61579365 1 2024-07-16T09:55:38.489 INFO:tasks.workunit.client.0.smithi156.stdout:6/413: creat d3/d13/d17/d4f/d30/d45/d5f/f8b x:0 0 0 2024-07-16T09:55:38.489 INFO:tasks.workunit.client.1.smithi176.stdout:5/487: symlink d4/d14/d3d/lb2 0 2024-07-16T09:55:38.489 INFO:tasks.workunit.client.1.smithi176.stdout:2/434: dread dd/d16/f56 [0,4194304] 0 2024-07-16T09:55:38.490 INFO:tasks.workunit.client.1.smithi176.stdout:0/545: read d5/f18 [687558,24626] 0 2024-07-16T09:55:38.491 INFO:tasks.workunit.client.1.smithi176.stdout:6/454: creat d1/d5/fa9 x:0 0 0 2024-07-16T09:55:38.491 INFO:tasks.workunit.client.0.smithi156.stdout:5/473: dwrite d1b/d1f/d42/f5f [0,4194304] 0 2024-07-16T09:55:38.493 INFO:tasks.workunit.client.1.smithi176.stdout:5/488: creat d4/d37/d72/d38/d39/fb3 x:0 0 0 2024-07-16T09:55:38.493 INFO:tasks.workunit.client.1.smithi176.stdout:0/546: chown d5/d44/f55 2012 1 2024-07-16T09:55:38.493 INFO:tasks.workunit.client.0.smithi156.stdout:8/355: rmdir d0/d11/d13/d1c/d39 39 2024-07-16T09:55:38.494 INFO:tasks.workunit.client.1.smithi176.stdout:2/435: symlink dd/d2e/d35/d59/d66/l8b 0 2024-07-16T09:55:38.495 INFO:tasks.workunit.client.1.smithi176.stdout:3/392: dwrite d3/d17/d1d/d4f/d59/f27 [0,4194304] 0 2024-07-16T09:55:38.497 INFO:tasks.workunit.client.0.smithi156.stdout:9/430: dread d2/d5/d16/d22/d2b/d3d/d3c/f62 [0,4194304] 0 2024-07-16T09:55:38.498 INFO:tasks.workunit.client.0.smithi156.stdout:3/350: dwrite d8/da/d28/d43/f77 [0,4194304] 0 2024-07-16T09:55:38.500 INFO:tasks.workunit.client.1.smithi176.stdout:0/547: rmdir d5/d44 39 2024-07-16T09:55:38.501 INFO:tasks.workunit.client.0.smithi156.stdout:3/351: stat d8/da/d28/f5e 0 2024-07-16T09:55:38.504 INFO:tasks.workunit.client.1.smithi176.stdout:3/393: write d3/d17/d1d/d4d/f42 [2148886,101097] 0 2024-07-16T09:55:38.504 INFO:tasks.workunit.client.0.smithi156.stdout:8/356: mkdir d0/d11/d20/d7f 0 2024-07-16T09:55:38.504 INFO:tasks.workunit.client.0.smithi156.stdout:9/431: dread - d2/d5/d16/d22/d36/d33/d23/f60 zero size 2024-07-16T09:55:38.507 INFO:tasks.workunit.client.0.smithi156.stdout:6/414: creat d3/d13/d17/f8c x:0 0 0 2024-07-16T09:55:38.507 INFO:tasks.workunit.client.0.smithi156.stdout:2/524: dread d0/d13/d17/f30 [0,4194304] 0 2024-07-16T09:55:38.507 INFO:tasks.workunit.client.1.smithi176.stdout:0/548: dread d5/d39/f51 [0,4194304] 0 2024-07-16T09:55:38.507 INFO:tasks.workunit.client.1.smithi176.stdout:9/500: write d5/de/d21/f5f [539879,21437] 0 2024-07-16T09:55:38.508 INFO:tasks.workunit.client.1.smithi176.stdout:3/394: chown d3/da/d33/f67 1225 1 2024-07-16T09:55:38.509 INFO:tasks.workunit.client.0.smithi156.stdout:8/357: truncate d0/d11/d4a/f4f 972740 0 2024-07-16T09:55:38.511 INFO:tasks.workunit.client.1.smithi176.stdout:3/395: readlink d3/d17/d1d/d4d/l48 0 2024-07-16T09:55:38.511 INFO:tasks.workunit.client.1.smithi176.stdout:1/288: dwrite d4/ff [0,4194304] 0 2024-07-16T09:55:38.513 INFO:tasks.workunit.client.1.smithi176.stdout:9/501: truncate d5/de/da5/f5a 893286 0 2024-07-16T09:55:38.514 INFO:tasks.workunit.client.0.smithi156.stdout:9/432: symlink d2/d5/d61/l87 0 2024-07-16T09:55:38.514 INFO:tasks.workunit.client.0.smithi156.stdout:4/404: symlink d4/db/dc/d5f/d64/l8b 0 2024-07-16T09:55:38.515 INFO:tasks.workunit.client.0.smithi156.stdout:8/358: mkdir d0/d11/d29/d80 0 2024-07-16T09:55:38.516 INFO:tasks.workunit.client.0.smithi156.stdout:8/359: chown d0/d11/d29 264466292 1 2024-07-16T09:55:38.516 INFO:tasks.workunit.client.0.smithi156.stdout:5/474: mknod d1b/d1f/d45/d52/d43/d85/ca1 0 2024-07-16T09:55:38.517 INFO:tasks.workunit.client.0.smithi156.stdout:2/525: mknod d0/d13/d17/d1c/d51/cb6 0 2024-07-16T09:55:38.517 INFO:tasks.workunit.client.1.smithi176.stdout:5/489: write d4/d37/d72/f53 [831551,56197] 0 2024-07-16T09:55:38.518 INFO:tasks.workunit.client.1.smithi176.stdout:3/396: write d3/d17/f2f [2821770,36618] 0 2024-07-16T09:55:38.519 INFO:tasks.workunit.client.1.smithi176.stdout:5/490: dread - d4/d37/d72/d38/d39/fb3 zero size 2024-07-16T09:55:38.522 INFO:tasks.workunit.client.0.smithi156.stdout:4/405: rmdir d4/d23/d30/d33 39 2024-07-16T09:55:38.523 INFO:tasks.workunit.client.1.smithi176.stdout:6/455: truncate d1/d5/f6 6401101 0 2024-07-16T09:55:38.523 INFO:tasks.workunit.client.0.smithi156.stdout:8/360: symlink d0/d11/d4a/d55/l81 0 2024-07-16T09:55:38.526 INFO:tasks.workunit.client.1.smithi176.stdout:9/502: mknod d5/de/d37/d58/d89/cac 0 2024-07-16T09:55:38.526 INFO:tasks.workunit.client.0.smithi156.stdout:2/526: write d0/d13/d17/f42 [4633553,38186] 0 2024-07-16T09:55:38.526 INFO:tasks.workunit.client.0.smithi156.stdout:5/475: dwrite d1b/d1f/d45/d52/d49/d4d/f62 [0,4194304] 0 2024-07-16T09:55:38.528 INFO:tasks.workunit.client.0.smithi156.stdout:8/361: mknod d0/d11/d29/d5e/c82 0 2024-07-16T09:55:38.529 INFO:tasks.workunit.client.0.smithi156.stdout:9/433: getdents d2/d5 0 2024-07-16T09:55:38.529 INFO:tasks.workunit.client.0.smithi156.stdout:4/406: mkdir d4/db/dc/d7c/d8c 0 2024-07-16T09:55:38.530 INFO:tasks.workunit.client.1.smithi176.stdout:6/456: creat d1/d5d/d9e/faa x:0 0 0 2024-07-16T09:55:38.531 INFO:tasks.workunit.client.1.smithi176.stdout:6/457: truncate d1/f3b 1531493 0 2024-07-16T09:55:38.533 INFO:tasks.workunit.client.0.smithi156.stdout:2/527: creat d0/d13/d17/d1c/d86/fb7 x:0 0 0 2024-07-16T09:55:38.533 INFO:tasks.workunit.client.1.smithi176.stdout:2/436: rename dd/de/l28 to dd/d36/l8c 0 2024-07-16T09:55:38.534 INFO:tasks.workunit.client.1.smithi176.stdout:5/491: read d4/d10/d17/f67 [143570,75727] 0 2024-07-16T09:55:38.535 INFO:tasks.workunit.client.0.smithi156.stdout:9/434: creat d2/d5/d16/d22/d36/d33/d23/f88 x:0 0 0 2024-07-16T09:55:38.536 INFO:tasks.workunit.client.1.smithi176.stdout:6/458: creat d1/dc/fab x:0 0 0 2024-07-16T09:55:38.536 INFO:tasks.workunit.client.1.smithi176.stdout:0/549: rename d5/d1c/f20 to d5/d1c/d5a/fab 0 2024-07-16T09:55:38.537 INFO:tasks.workunit.client.0.smithi156.stdout:8/362: creat d0/d11/d62/f83 x:0 0 0 2024-07-16T09:55:38.537 INFO:tasks.workunit.client.0.smithi156.stdout:4/407: rmdir d4/db/dc/d7c/d8c 0 2024-07-16T09:55:38.538 INFO:tasks.workunit.client.0.smithi156.stdout:9/435: write d2/d5/d16/d22/d2b/d3d/f6d [784642,64520] 0 2024-07-16T09:55:38.541 INFO:tasks.workunit.client.0.smithi156.stdout:9/436: write d2/d5/d16/d22/d2b/d3d/d3c/d7f/f2f [1915476,116367] 0 2024-07-16T09:55:38.543 INFO:tasks.workunit.client.1.smithi176.stdout:6/459: rename d1/d5/d7/d2f/f44 to d1/d5/d47/fac 0 2024-07-16T09:55:38.543 INFO:tasks.workunit.client.1.smithi176.stdout:5/492: dread d4/d37/d72/d38/f7e [0,4194304] 0 2024-07-16T09:55:38.544 INFO:tasks.workunit.client.0.smithi156.stdout:9/437: chown d2/d5/d16/d22/d36/d33/f80 8754586 1 2024-07-16T09:55:38.545 INFO:tasks.workunit.client.1.smithi176.stdout:0/550: creat d5/d1c/d24/d35/d50/d9b/fac x:0 0 0 2024-07-16T09:55:38.546 INFO:tasks.workunit.client.0.smithi156.stdout:5/476: dread d1b/d1f/d45/d52/d43/d85/d97/f25 [0,4194304] 0 2024-07-16T09:55:38.546 INFO:tasks.workunit.client.0.smithi156.stdout:8/363: fsync d0/d11/d13/d1c/d24/f42 0 2024-07-16T09:55:38.547 INFO:tasks.workunit.client.0.smithi156.stdout:9/438: fdatasync d2/d5/d16/f20 0 2024-07-16T09:55:38.547 INFO:tasks.workunit.client.1.smithi176.stdout:6/460: chown d1/d5d/d9e/d8f/l89 325 1 2024-07-16T09:55:38.548 INFO:tasks.workunit.client.0.smithi156.stdout:5/477: dread - d1b/d1f/d45/d52/d43/f8d zero size 2024-07-16T09:55:38.548 INFO:tasks.workunit.client.1.smithi176.stdout:2/437: dwrite dd/d16/d39/d3d/f75 [0,4194304] 0 2024-07-16T09:55:38.549 INFO:tasks.workunit.client.0.smithi156.stdout:4/408: rmdir d4/db/d47/d4b 39 2024-07-16T09:55:38.550 INFO:tasks.workunit.client.1.smithi176.stdout:2/438: chown dd/de/c13 135 1 2024-07-16T09:55:38.554 INFO:tasks.workunit.client.0.smithi156.stdout:3/352: dread d8/da/d13/f20 [0,4194304] 0 2024-07-16T09:55:38.555 INFO:tasks.workunit.client.1.smithi176.stdout:0/551: mknod d5/d1c/d24/cad 0 2024-07-16T09:55:38.555 INFO:tasks.workunit.client.0.smithi156.stdout:4/409: symlink d4/db/dc/d7c/l8d 0 2024-07-16T09:55:38.556 INFO:tasks.workunit.client.0.smithi156.stdout:3/353: write d8/da/d21/f6d [365017,44261] 0 2024-07-16T09:55:38.558 INFO:tasks.workunit.client.0.smithi156.stdout:9/439: dwrite d2/d5/d16/d22/d36/d33/f2c [0,4194304] 0 2024-07-16T09:55:38.558 INFO:tasks.workunit.client.0.smithi156.stdout:9/440: stat d2/l43 0 2024-07-16T09:55:38.559 INFO:tasks.workunit.client.1.smithi176.stdout:1/289: truncate d4/f24 4068230 0 2024-07-16T09:55:38.560 INFO:tasks.workunit.client.1.smithi176.stdout:2/439: dwrite dd/d16/f7d [0,4194304] 0 2024-07-16T09:55:38.562 INFO:tasks.workunit.client.1.smithi176.stdout:9/503: truncate d5/f10 75259 0 2024-07-16T09:55:38.563 INFO:tasks.workunit.client.0.smithi156.stdout:5/478: link d1b/d1f/d45/d52/d43/d7c/f82 d1b/d1f/d45/d52/d43/d7c/fa2 0 2024-07-16T09:55:38.565 INFO:tasks.workunit.client.0.smithi156.stdout:4/410: chown d4/db/dc/l15 1103375140 1 2024-07-16T09:55:38.566 INFO:tasks.workunit.client.1.smithi176.stdout:2/440: read dd/d16/d39/f6d [344902,7208] 0 2024-07-16T09:55:38.567 INFO:tasks.workunit.client.1.smithi176.stdout:0/552: write d5/d1c/d24/d35/d50/d9b/fac [649213,6013] 0 2024-07-16T09:55:38.567 INFO:tasks.workunit.client.1.smithi176.stdout:9/504: write d5/d1d/d20/d8d/f91 [332183,126606] 0 2024-07-16T09:55:38.568 INFO:tasks.workunit.client.1.smithi176.stdout:1/290: unlink d4/d8/d12/d21/f63 0 2024-07-16T09:55:38.569 INFO:tasks.workunit.client.0.smithi156.stdout:5/479: fdatasync d1b/d1f/d45/d52/f3c 0 2024-07-16T09:55:38.572 INFO:tasks.workunit.client.1.smithi176.stdout:5/493: dread d4/fe [0,4194304] 0 2024-07-16T09:55:38.572 INFO:tasks.workunit.client.0.smithi156.stdout:9/441: dwrite d2/d5/d16/d22/d36/d33/d23/f60 [0,4194304] 0 2024-07-16T09:55:38.572 INFO:tasks.workunit.client.0.smithi156.stdout:4/411: write d4/f3d [1937053,12166] 0 2024-07-16T09:55:38.574 INFO:tasks.workunit.client.0.smithi156.stdout:3/354: getdents d8/da/d21 0 2024-07-16T09:55:38.575 INFO:tasks.workunit.client.1.smithi176.stdout:3/397: dread d3/d17/f2f [0,4194304] 0 2024-07-16T09:55:38.576 INFO:tasks.workunit.client.1.smithi176.stdout:0/553: symlink d5/d25/d82/lae 0 2024-07-16T09:55:38.577 INFO:tasks.workunit.client.0.smithi156.stdout:5/480: rmdir d1b/d1f/d45/d7a 39 2024-07-16T09:55:38.578 INFO:tasks.workunit.client.0.smithi156.stdout:9/442: fsync d2/d5/d16/d22/d36/f59 0 2024-07-16T09:55:38.578 INFO:tasks.workunit.client.1.smithi176.stdout:9/505: dread d5/f1e [0,4194304] 0 2024-07-16T09:55:38.578 INFO:tasks.workunit.client.0.smithi156.stdout:3/355: mknod d8/da/d28/c78 0 2024-07-16T09:55:38.580 INFO:tasks.workunit.client.1.smithi176.stdout:6/461: truncate d1/d5/d47/fac 450863 0 2024-07-16T09:55:38.582 INFO:tasks.workunit.client.0.smithi156.stdout:8/364: truncate d0/d11/d13/f14 278791 0 2024-07-16T09:55:38.582 INFO:tasks.workunit.client.0.smithi156.stdout:9/443: truncate d2/d5/d16/d22/f7c 251378 0 2024-07-16T09:55:38.583 INFO:tasks.workunit.client.0.smithi156.stdout:4/412: creat d4/db/dc/d7c/f8e x:0 0 0 2024-07-16T09:55:38.583 INFO:tasks.workunit.client.0.smithi156.stdout:8/365: readlink d0/dd/l2b 0 2024-07-16T09:55:38.584 INFO:tasks.workunit.client.1.smithi176.stdout:1/291: truncate d4/d8/d12/f4b 1125061 0 2024-07-16T09:55:38.584 INFO:tasks.workunit.client.0.smithi156.stdout:3/356: rename d8/f1a to d8/da/d28/d40/f79 0 2024-07-16T09:55:38.585 INFO:tasks.workunit.client.0.smithi156.stdout:9/444: fsync d2/d5/d16/d22/d2b/d3d/f6d 0 2024-07-16T09:55:38.586 INFO:tasks.workunit.client.0.smithi156.stdout:8/366: read d0/dd/d1f/f47 [457374,14774] 0 2024-07-16T09:55:38.586 INFO:tasks.workunit.client.1.smithi176.stdout:0/554: creat d5/d25/da0/faf x:0 0 0 2024-07-16T09:55:38.586 INFO:tasks.workunit.client.0.smithi156.stdout:5/481: mknod d1b/d1f/d45/d7a/d98/ca3 0 2024-07-16T09:55:38.587 INFO:tasks.workunit.client.1.smithi176.stdout:3/398: dwrite d3/da/d56/f5b [4194304,4194304] 0 2024-07-16T09:55:38.588 INFO:tasks.workunit.client.0.smithi156.stdout:5/482: truncate d1b/f55 833043 0 2024-07-16T09:55:38.589 INFO:tasks.workunit.client.1.smithi176.stdout:5/494: link d4/d37/d72/d38/d39/f3a d4/d14/fb4 0 2024-07-16T09:55:38.590 INFO:tasks.workunit.client.0.smithi156.stdout:5/483: stat d1b/d1f/d42/d6d 0 2024-07-16T09:55:38.590 INFO:tasks.workunit.client.1.smithi176.stdout:5/495: chown d4/d10/f15 176037 1 2024-07-16T09:55:38.591 INFO:tasks.workunit.client.1.smithi176.stdout:9/506: dwrite d5/de/da5/f82 [0,4194304] 0 2024-07-16T09:55:38.591 INFO:tasks.workunit.client.0.smithi156.stdout:4/413: rename d4/db/d47/d4b to d4/db/d47/d63/d58/d8f 0 2024-07-16T09:55:38.592 INFO:tasks.workunit.client.1.smithi176.stdout:9/507: chown d5/d52/l95 12427229 1 2024-07-16T09:55:38.593 INFO:tasks.workunit.client.0.smithi156.stdout:5/484: chown d1b/d1f/d45/d52/f39 784704 1 2024-07-16T09:55:38.597 INFO:tasks.workunit.client.0.smithi156.stdout:3/357: rename d8/da/d13/l76 to d8/da/d1d/d2c/d33/l7a 0 2024-07-16T09:55:38.597 INFO:tasks.workunit.client.0.smithi156.stdout:9/445: link d2/d5/d16/d22/d2b/d3d/d3c/d7f/c75 d2/d5/d16/d22/d6b/c89 0 2024-07-16T09:55:38.598 INFO:tasks.workunit.client.0.smithi156.stdout:8/367: dwrite d0/dd/f30 [0,4194304] 0 2024-07-16T09:55:38.598 INFO:tasks.workunit.client.1.smithi176.stdout:2/441: getdents dd/d36/d45 0 2024-07-16T09:55:38.598 INFO:tasks.workunit.client.0.smithi156.stdout:5/485: write d1b/d1f/d45/d52/d43/f5b [287735,98449] 0 2024-07-16T09:55:38.602 INFO:tasks.workunit.client.0.smithi156.stdout:4/414: truncate d4/d23/f78 803202 0 2024-07-16T09:55:38.603 INFO:tasks.workunit.client.1.smithi176.stdout:5/496: mknod d4/d6c/cb5 0 2024-07-16T09:55:38.604 INFO:tasks.workunit.client.1.smithi176.stdout:5/497: dread - d4/d37/d72/d38/f97 zero size 2024-07-16T09:55:38.606 INFO:tasks.workunit.client.1.smithi176.stdout:1/292: rmdir d4/d8/d12/d4c 39 2024-07-16T09:55:38.606 INFO:tasks.workunit.client.1.smithi176.stdout:5/498: dread - d4/d37/d72/d38/f87 zero size 2024-07-16T09:55:38.609 INFO:tasks.workunit.client.0.smithi156.stdout:3/358: mkdir d8/da/d1d/d7b 0 2024-07-16T09:55:38.610 INFO:tasks.workunit.client.1.smithi176.stdout:6/462: rmdir d1/d5/d7/d2f/d8e 0 2024-07-16T09:55:38.610 INFO:tasks.workunit.client.1.smithi176.stdout:1/293: fsync d4/d8/d12/d21/d25/d30/d44/f5f 0 2024-07-16T09:55:38.611 INFO:tasks.workunit.client.0.smithi156.stdout:9/446: symlink d2/d5/d16/d22/d36/d6a/l8a 0 2024-07-16T09:55:38.612 INFO:tasks.workunit.client.0.smithi156.stdout:4/415: dwrite d4/db/dc/f4e [0,4194304] 0 2024-07-16T09:55:38.613 INFO:tasks.workunit.client.0.smithi156.stdout:5/486: mkdir d1b/d1f/d45/d9d/da4 0 2024-07-16T09:55:38.614 INFO:tasks.workunit.client.0.smithi156.stdout:5/487: readlink d1b/l27 0 2024-07-16T09:55:38.614 INFO:tasks.workunit.client.1.smithi176.stdout:9/508: dwrite d5/f1c [4194304,4194304] 0 2024-07-16T09:55:38.614 INFO:tasks.workunit.client.1.smithi176.stdout:3/399: rmdir d3/d17/d1d/d6e/d6f 0 2024-07-16T09:55:38.616 INFO:tasks.workunit.client.0.smithi156.stdout:9/447: write d2/d5/d16/d22/d36/d33/d3a/f49 [1522679,32875] 0 2024-07-16T09:55:38.616 INFO:tasks.workunit.client.0.smithi156.stdout:4/416: mknod d4/d23/d52/c90 0 2024-07-16T09:55:38.619 INFO:tasks.workunit.client.0.smithi156.stdout:5/488: symlink d1b/d1f/d45/d52/d49/la5 0 2024-07-16T09:55:38.620 INFO:tasks.workunit.client.1.smithi176.stdout:2/442: dwrite dd/d36/f4d [4194304,4194304] 0 2024-07-16T09:55:38.647 INFO:tasks.workunit.client.0.smithi156.stdout:0/381: sync 2024-07-16T09:55:38.650 INFO:tasks.workunit.client.1.smithi176.stdout:1/294: getdents d4/d8/d12 0 2024-07-16T09:55:38.651 INFO:tasks.workunit.client.0.smithi156.stdout:9/448: unlink d2/d5/d61/l87 0 2024-07-16T09:55:38.651 INFO:tasks.workunit.client.0.smithi156.stdout:5/489: write d1b/d1f/d45/d52/f3c [4112415,121907] 0 2024-07-16T09:55:38.651 INFO:tasks.workunit.client.1.smithi176.stdout:1/295: read d4/d8/d12/d21/d25/f60 [612464,46703] 0 2024-07-16T09:55:38.653 INFO:tasks.workunit.client.1.smithi176.stdout:1/296: dread - d4/d8/d12/d21/d25/d30/f61 zero size 2024-07-16T09:55:38.655 INFO:tasks.workunit.client.0.smithi156.stdout:3/359: dwrite d8/da/d1d/d2c/f52 [0,4194304] 0 2024-07-16T09:55:38.656 INFO:tasks.workunit.client.1.smithi176.stdout:5/499: dread d4/f5 [0,4194304] 0 2024-07-16T09:55:38.656 INFO:tasks.workunit.client.0.smithi156.stdout:4/417: creat d4/db/d47/d63/d58/f91 x:0 0 0 2024-07-16T09:55:38.658 INFO:tasks.workunit.client.0.smithi156.stdout:3/360: write d8/da/d21/f3e [7835547,7916] 0 2024-07-16T09:55:38.659 INFO:tasks.workunit.client.1.smithi176.stdout:5/500: dread - d4/d37/d72/d38/d39/fb3 zero size 2024-07-16T09:55:38.659 INFO:tasks.workunit.client.0.smithi156.stdout:9/449: creat d2/d5/d16/d22/d2b/f8b x:0 0 0 2024-07-16T09:55:38.660 INFO:tasks.workunit.client.1.smithi176.stdout:6/463: dread d1/dc/fd [0,4194304] 0 2024-07-16T09:55:38.661 INFO:tasks.workunit.client.1.smithi176.stdout:1/297: dwrite d4/d8/d12/d21/d25/f60 [0,4194304] 0 2024-07-16T09:55:38.664 INFO:tasks.workunit.client.1.smithi176.stdout:2/443: write dd/d2e/f50 [386394,119005] 0 2024-07-16T09:55:38.666 INFO:tasks.workunit.client.1.smithi176.stdout:5/501: rmdir d4/d37/d72/d38/d6e 39 2024-07-16T09:55:38.670 INFO:tasks.workunit.client.1.smithi176.stdout:6/464: creat d1/d5/d47/fad x:0 0 0 2024-07-16T09:55:38.671 INFO:tasks.workunit.client.0.smithi156.stdout:4/418: creat d4/db/dc/d67/f92 x:0 0 0 2024-07-16T09:55:38.671 INFO:tasks.workunit.client.0.smithi156.stdout:9/450: rmdir d2/d5/d16/d22/d36/d33/d3a 39 2024-07-16T09:55:38.674 INFO:tasks.workunit.client.0.smithi156.stdout:9/451: write d2/d5/d16/d22/d2b/d3d/d3c/f53 [2211863,11291] 0 2024-07-16T09:55:38.674 INFO:tasks.workunit.client.1.smithi176.stdout:2/444: rename dd/d2e/d35/f3b to dd/d36/d82/f8d 0 2024-07-16T09:55:38.675 INFO:tasks.workunit.client.1.smithi176.stdout:5/502: write d4/d37/d72/d82/f93 [7958812,112122] 0 2024-07-16T09:55:38.676 INFO:tasks.workunit.client.0.smithi156.stdout:4/419: unlink d4/db/dc/d7c/l8d 0 2024-07-16T09:55:38.677 INFO:tasks.workunit.client.1.smithi176.stdout:3/400: dwrite d3/d17/d1d/d4d/f3f [0,4194304] 0 2024-07-16T09:55:38.677 INFO:tasks.workunit.client.0.smithi156.stdout:0/382: rename d0/d1a/d4d/c16 to d0/d1a/d4d/d9/d27/c6f 0 2024-07-16T09:55:38.678 INFO:tasks.workunit.client.1.smithi176.stdout:2/445: rename dd to dd/de/d8e 22 2024-07-16T09:55:38.679 INFO:tasks.workunit.client.1.smithi176.stdout:9/509: dwrite d5/de/d30/fa2 [0,4194304] 0 2024-07-16T09:55:38.679 INFO:tasks.workunit.client.0.smithi156.stdout:0/383: fdatasync d0/d1a/d4d/d9/d27/f6a 0 2024-07-16T09:55:38.686 INFO:tasks.workunit.client.1.smithi176.stdout:6/465: mkdir d1/d5/d7/dae 0 2024-07-16T09:55:38.689 INFO:tasks.workunit.client.0.smithi156.stdout:3/361: rename d8/da/d28/d40/f79 to d8/da/d28/f7c 0 2024-07-16T09:55:38.689 INFO:tasks.workunit.client.0.smithi156.stdout:9/452: dwrite d2/d5/d16/d22/d36/d33/d3a/f49 [0,4194304] 0 2024-07-16T09:55:38.690 INFO:tasks.workunit.client.1.smithi176.stdout:9/510: mknod d5/d1d/d77/d97/cad 0 2024-07-16T09:55:38.690 INFO:tasks.workunit.client.0.smithi156.stdout:4/420: rename d4/db/dc/d7f to d4/db/dc/d7f/d87/d93 22 2024-07-16T09:55:38.691 INFO:tasks.workunit.client.0.smithi156.stdout:9/453: write d2/d5/d16/d57/d63/f81 [826832,111277] 0 2024-07-16T09:55:38.691 INFO:tasks.workunit.client.0.smithi156.stdout:3/362: chown d8/f10 25605 1 2024-07-16T09:55:38.694 INFO:tasks.workunit.client.0.smithi156.stdout:9/454: chown d2/d5/d16/d22/d2b/d3d/d3c/d7f/l5d 902486 1 2024-07-16T09:55:38.694 INFO:tasks.workunit.client.0.smithi156.stdout:3/363: read d8/da/d13/f57 [128506,129160] 0 2024-07-16T09:55:38.695 INFO:tasks.workunit.client.0.smithi156.stdout:3/364: stat d8/da/d28 0 2024-07-16T09:55:38.696 INFO:tasks.workunit.client.1.smithi176.stdout:5/503: mknod d4/d37/d72/d38/cb6 0 2024-07-16T09:55:38.696 INFO:tasks.workunit.client.1.smithi176.stdout:1/298: write d4/d3a/d54/d5a/f5b [505929,19616] 0 2024-07-16T09:55:38.697 INFO:tasks.workunit.client.0.smithi156.stdout:0/384: rename d0/c2 to d0/d1a/d4d/d9/d27/d50/c70 0 2024-07-16T09:55:38.697 INFO:tasks.workunit.client.1.smithi176.stdout:6/466: unlink d1/d5/d7/l9b 0 2024-07-16T09:55:38.698 INFO:tasks.workunit.client.1.smithi176.stdout:9/511: symlink d5/d43/lae 0 2024-07-16T09:55:38.698 INFO:tasks.workunit.client.1.smithi176.stdout:5/504: truncate d4/d10/d17/f63 866769 0 2024-07-16T09:55:38.699 INFO:tasks.workunit.client.1.smithi176.stdout:3/401: creat d3/d17/d1d/f76 x:0 0 0 2024-07-16T09:55:38.702 INFO:tasks.workunit.client.1.smithi176.stdout:1/299: creat d4/f65 x:0 0 0 2024-07-16T09:55:38.702 INFO:tasks.workunit.client.0.smithi156.stdout:0/385: symlink d0/d1a/d4d/d9/d27/d3d/d62/l71 0 2024-07-16T09:55:38.703 INFO:tasks.workunit.client.0.smithi156.stdout:3/365: mkdir d8/da/d1d/d2c/d7d 0 2024-07-16T09:55:38.703 INFO:tasks.workunit.client.0.smithi156.stdout:5/490: dwrite d1b/d1f/d45/d7f/f90 [0,4194304] 0 2024-07-16T09:55:38.704 INFO:tasks.workunit.client.1.smithi176.stdout:2/446: dwrite dd/d16/f33 [0,4194304] 0 2024-07-16T09:55:38.704 INFO:tasks.workunit.client.1.smithi176.stdout:5/505: mknod d4/d14/d19/d1b/d61/cb7 0 2024-07-16T09:55:38.704 INFO:tasks.workunit.client.0.smithi156.stdout:9/455: creat d2/d5/d16/d22/d6b/d86/f8c x:0 0 0 2024-07-16T09:55:38.707 INFO:tasks.workunit.client.1.smithi176.stdout:1/300: mkdir d4/d3a/d66 0 2024-07-16T09:55:38.708 INFO:tasks.workunit.client.1.smithi176.stdout:2/447: creat dd/d2e/d35/d59/d66/f8f x:0 0 0 2024-07-16T09:55:38.709 INFO:tasks.workunit.client.1.smithi176.stdout:6/467: creat d1/d5/d7/faf x:0 0 0 2024-07-16T09:55:38.709 INFO:tasks.workunit.client.1.smithi176.stdout:2/448: chown dd/d16/f56 0 1 2024-07-16T09:55:38.710 INFO:tasks.workunit.client.1.smithi176.stdout:6/468: chown d1/dc/d40/f8b 0 1 2024-07-16T09:55:38.711 INFO:tasks.workunit.client.0.smithi156.stdout:9/456: symlink d2/d5/d16/d22/d2b/d74/l8d 0 2024-07-16T09:55:38.713 INFO:tasks.workunit.client.1.smithi176.stdout:9/512: rmdir d5/de/da5/da3 0 2024-07-16T09:55:38.714 INFO:tasks.workunit.client.1.smithi176.stdout:3/402: creat d3/da/f77 x:0 0 0 2024-07-16T09:55:38.715 INFO:tasks.workunit.client.1.smithi176.stdout:1/301: creat d4/d3a/d54/d5a/f67 x:0 0 0 2024-07-16T09:55:38.716 INFO:tasks.workunit.client.0.smithi156.stdout:5/491: rename d1b/d1f/d45/d52/d43/d85/f6a to d1b/d1f/d8a/fa6 0 2024-07-16T09:55:38.717 INFO:tasks.workunit.client.1.smithi176.stdout:6/469: creat d1/fb0 x:0 0 0 2024-07-16T09:55:38.718 INFO:tasks.workunit.client.1.smithi176.stdout:9/513: mkdir d5/de/d30/daf 0 2024-07-16T09:55:38.718 INFO:tasks.workunit.client.0.smithi156.stdout:0/386: dwrite d0/d1a/d4d/d9/d27/f6a [0,4194304] 0 2024-07-16T09:55:38.720 INFO:tasks.workunit.client.1.smithi176.stdout:3/403: creat d3/da/d33/f78 x:0 0 0 2024-07-16T09:55:38.720 INFO:tasks.workunit.client.1.smithi176.stdout:2/449: dwrite dd/d49/d52/f79 [0,4194304] 0 2024-07-16T09:55:38.720 INFO:tasks.workunit.client.1.smithi176.stdout:6/470: fdatasync d1/dc/d5b/fa8 0 2024-07-16T09:55:38.721 INFO:tasks.workunit.client.0.smithi156.stdout:3/366: write d8/da/d1d/f53 [1964720,97750] 0 2024-07-16T09:55:38.722 INFO:tasks.workunit.client.1.smithi176.stdout:3/404: write d3/d17/f58 [241441,74633] 0 2024-07-16T09:55:38.723 INFO:tasks.workunit.client.1.smithi176.stdout:1/302: write d4/d8/d12/d21/d25/d30/f61 [1034419,46528] 0 2024-07-16T09:55:38.727 INFO:tasks.workunit.client.0.smithi156.stdout:4/421: dwrite d4/db/f7b [0,4194304] 0 2024-07-16T09:55:38.727 INFO:tasks.workunit.client.0.smithi156.stdout:5/492: creat d1b/d1f/d45/d7f/d87/fa7 x:0 0 0 2024-07-16T09:55:38.727 INFO:tasks.workunit.client.1.smithi176.stdout:1/303: write d4/f65 [1025693,80466] 0 2024-07-16T09:55:38.728 INFO:tasks.workunit.client.1.smithi176.stdout:1/304: dread - d4/d8/d12/d21/d25/d30/d44/f5f zero size 2024-07-16T09:55:38.728 INFO:tasks.workunit.client.1.smithi176.stdout:2/450: fdatasync dd/d36/d82/f8d 0 2024-07-16T09:55:38.729 INFO:tasks.workunit.client.0.smithi156.stdout:0/387: truncate d0/d1a/d4d/d9/fe 4557710 0 2024-07-16T09:55:38.730 INFO:tasks.workunit.client.1.smithi176.stdout:2/451: dread - dd/d49/f78 zero size 2024-07-16T09:55:38.732 INFO:tasks.workunit.client.1.smithi176.stdout:6/471: dwrite d1/d5/d7/d1e/f79 [0,4194304] 0 2024-07-16T09:55:38.732 INFO:tasks.workunit.client.0.smithi156.stdout:5/493: symlink d1b/d1f/d45/d52/d43/d85/d97/la8 0 2024-07-16T09:55:38.734 INFO:tasks.workunit.client.1.smithi176.stdout:1/305: mkdir d4/db/d68 0 2024-07-16T09:55:38.734 INFO:tasks.workunit.client.0.smithi156.stdout:5/494: dread - d1b/f40 zero size 2024-07-16T09:55:38.735 INFO:tasks.workunit.client.1.smithi176.stdout:6/472: chown d1/d5/d47/d62/d77/la0 1056289656 1 2024-07-16T09:55:38.735 INFO:tasks.workunit.client.1.smithi176.stdout:2/452: creat dd/d2e/f90 x:0 0 0 2024-07-16T09:55:38.735 INFO:tasks.workunit.client.0.smithi156.stdout:9/457: link d2/d5/d16/d22/d2b/d3d/d3c/d7f/d42/l45 d2/d5/d16/d22/d2b/d74/l8e 0 2024-07-16T09:55:38.736 INFO:tasks.workunit.client.0.smithi156.stdout:4/422: chown d4/d23/d30/d33/c40 311718 1 2024-07-16T09:55:38.736 INFO:tasks.workunit.client.0.smithi156.stdout:0/388: mknod d0/d1a/d4d/d9/d27/d3d/d62/c72 0 2024-07-16T09:55:38.737 INFO:tasks.workunit.client.0.smithi156.stdout:0/389: read d0/ff [30589,69053] 0 2024-07-16T09:55:38.737 INFO:tasks.workunit.client.1.smithi176.stdout:6/473: write d1/d5/d47/f6c [755866,17538] 0 2024-07-16T09:55:38.740 INFO:tasks.workunit.client.1.smithi176.stdout:2/453: unlink dd/d2e/d35/d4a/c6b 0 2024-07-16T09:55:38.744 INFO:tasks.workunit.client.0.smithi156.stdout:3/367: rmdir d8/da/d1d/d7b 0 2024-07-16T09:55:38.746 INFO:tasks.workunit.client.1.smithi176.stdout:5/506: dread d4/d14/d3d/f41 [0,4194304] 0 2024-07-16T09:55:38.746 INFO:tasks.workunit.client.1.smithi176.stdout:6/474: dread d1/d5/f30 [0,4194304] 0 2024-07-16T09:55:38.746 INFO:tasks.workunit.client.1.smithi176.stdout:9/514: dread d5/de/d37/d58/f60 [4194304,4194304] 0 2024-07-16T09:55:38.747 INFO:tasks.workunit.client.0.smithi156.stdout:5/495: creat d1b/d1f/fa9 x:0 0 0 2024-07-16T09:55:38.747 INFO:tasks.workunit.client.1.smithi176.stdout:2/454: creat dd/d49/d52/f91 x:0 0 0 2024-07-16T09:55:38.747 INFO:tasks.workunit.client.1.smithi176.stdout:6/475: chown d1/d5/d7/dae 3 1 2024-07-16T09:55:38.748 INFO:tasks.workunit.client.0.smithi156.stdout:0/390: dwrite d0/d1a/f46 [0,4194304] 0 2024-07-16T09:55:38.749 INFO:tasks.workunit.client.1.smithi176.stdout:6/476: write d1/dc/d72/d6f/d7c/f90 [885614,117511] 0 2024-07-16T09:55:38.751 INFO:tasks.workunit.client.1.smithi176.stdout:1/306: creat d4/d8/f69 x:0 0 0 2024-07-16T09:55:38.751 INFO:tasks.workunit.client.0.smithi156.stdout:4/423: creat d4/db/f94 x:0 0 0 2024-07-16T09:55:38.751 INFO:tasks.workunit.client.0.smithi156.stdout:9/458: dread d2/d5/d16/d22/d2b/d3d/d3c/d7f/f5e [0,4194304] 0 2024-07-16T09:55:38.752 INFO:tasks.workunit.client.0.smithi156.stdout:4/424: dread - d4/f43 zero size 2024-07-16T09:55:38.753 INFO:tasks.workunit.client.1.smithi176.stdout:3/405: rmdir d3/da 39 2024-07-16T09:55:38.753 INFO:tasks.workunit.client.0.smithi156.stdout:3/368: rmdir d8/da/d28/d43/d63/d69 0 2024-07-16T09:55:38.754 INFO:tasks.workunit.client.1.smithi176.stdout:2/455: rmdir dd/d2e/d35/d59 39 2024-07-16T09:55:38.755 INFO:tasks.workunit.client.0.smithi156.stdout:5/496: mknod d1b/d1f/d45/d52/d49/d4d/caa 0 2024-07-16T09:55:38.755 INFO:tasks.workunit.client.0.smithi156.stdout:9/459: fsync d2/f4 0 2024-07-16T09:55:38.755 INFO:tasks.workunit.client.1.smithi176.stdout:6/477: symlink d1/d5d/d9e/lb1 0 2024-07-16T09:55:38.757 INFO:tasks.workunit.client.1.smithi176.stdout:9/515: mkdir d5/d1d/db0 0 2024-07-16T09:55:38.759 INFO:tasks.workunit.client.1.smithi176.stdout:3/406: stat d3/da/l72 0 2024-07-16T09:55:38.759 INFO:tasks.workunit.client.0.smithi156.stdout:4/425: rmdir d4/db/dc/d5f 39 2024-07-16T09:55:38.760 INFO:tasks.workunit.client.1.smithi176.stdout:2/456: unlink dd/de/f58 0 2024-07-16T09:55:38.761 INFO:tasks.workunit.client.0.smithi156.stdout:0/391: mkdir d0/d1a/d4d/d61/d73 0 2024-07-16T09:55:38.762 INFO:tasks.workunit.client.1.smithi176.stdout:9/516: dread - d5/d1d/d67/fab zero size 2024-07-16T09:55:38.762 INFO:tasks.workunit.client.1.smithi176.stdout:5/507: dwrite d4/d14/f45 [4194304,4194304] 0 2024-07-16T09:55:38.763 INFO:tasks.workunit.client.0.smithi156.stdout:5/497: creat d1b/d1f/d45/d9d/fab x:0 0 0 2024-07-16T09:55:38.763 INFO:tasks.workunit.client.1.smithi176.stdout:6/478: creat d1/d5d/d9e/d8f/fb2 x:0 0 0 2024-07-16T09:55:38.764 INFO:tasks.workunit.client.0.smithi156.stdout:9/460: unlink d2/d5/d16/c17 0 2024-07-16T09:55:38.765 INFO:tasks.workunit.client.1.smithi176.stdout:1/307: getdents d4/d8/d12 0 2024-07-16T09:55:38.769 INFO:tasks.workunit.client.1.smithi176.stdout:3/407: mknod d3/d17/d1d/d6e/c79 0 2024-07-16T09:55:38.770 INFO:tasks.workunit.client.0.smithi156.stdout:0/392: write d0/f5c [661510,113898] 0 2024-07-16T09:55:38.772 INFO:tasks.workunit.client.0.smithi156.stdout:9/461: unlink d2/d5/d16/d22/d2b/d3d/d3c/f53 0 2024-07-16T09:55:38.773 INFO:tasks.workunit.client.0.smithi156.stdout:4/426: creat d4/db/d47/d63/d58/f95 x:0 0 0 2024-07-16T09:55:38.775 INFO:tasks.workunit.client.1.smithi176.stdout:1/308: chown d4/db/c40 3 1 2024-07-16T09:55:38.776 INFO:tasks.workunit.client.1.smithi176.stdout:5/508: getdents d4/d9a 0 2024-07-16T09:55:38.778 INFO:tasks.workunit.client.0.smithi156.stdout:3/369: dwrite d8/da/d1d/d2c/d33/f49 [0,4194304] 0 2024-07-16T09:55:38.778 INFO:tasks.workunit.client.0.smithi156.stdout:4/427: write d4/db/f61 [3063434,36346] 0 2024-07-16T09:55:38.783 INFO:tasks.workunit.client.0.smithi156.stdout:4/428: creat d4/db/d8a/f96 x:0 0 0 2024-07-16T09:55:38.785 INFO:tasks.workunit.client.0.smithi156.stdout:9/462: link d2/d5/d16/d22/d2b/d3d/d3c/d7f/f19 d2/d5/d16/d22/f8f 0 2024-07-16T09:55:38.788 INFO:tasks.workunit.client.1.smithi176.stdout:5/509: readlink d4/l8c 0 2024-07-16T09:55:38.790 INFO:tasks.workunit.client.0.smithi156.stdout:4/429: mkdir d4/db/dc/d5f/d97 0 2024-07-16T09:55:38.790 INFO:tasks.workunit.client.1.smithi176.stdout:9/517: dwrite d5/de/f15 [0,4194304] 0 2024-07-16T09:55:38.792 INFO:tasks.workunit.client.1.smithi176.stdout:1/309: rename d4/d8/d12 to d4/d8/d12/d21/d25/d30/d44/d6a 22 2024-07-16T09:55:38.793 INFO:tasks.workunit.client.1.smithi176.stdout:2/457: dwrite dd/d16/d39/f4f [0,4194304] 0 2024-07-16T09:55:38.793 INFO:tasks.workunit.client.0.smithi156.stdout:4/430: creat d4/db/dc/d5f/f98 x:0 0 0 2024-07-16T09:55:38.794 INFO:tasks.workunit.client.0.smithi156.stdout:4/431: fdatasync d4/d23/f59 0 2024-07-16T09:55:38.795 INFO:tasks.workunit.client.0.smithi156.stdout:9/463: getdents d2/d5/d16/d22/d36 0 2024-07-16T09:55:38.795 INFO:tasks.workunit.client.0.smithi156.stdout:0/393: dread d0/d1a/d4d/d9/f15 [0,4194304] 0 2024-07-16T09:55:38.795 INFO:tasks.workunit.client.0.smithi156.stdout:3/370: truncate d8/da/d28/d40/f54 196352 0 2024-07-16T09:55:38.796 INFO:tasks.workunit.client.1.smithi176.stdout:6/479: dwrite d1/dc/d72/d6f/d7c/f90 [0,4194304] 0 2024-07-16T09:55:38.796 INFO:tasks.workunit.client.0.smithi156.stdout:4/432: chown d4/db/d8a 5111756 1 2024-07-16T09:55:38.796 INFO:tasks.workunit.client.0.smithi156.stdout:3/371: chown d8/da/d28/d43/d71 18 1 2024-07-16T09:55:38.798 INFO:tasks.workunit.client.0.smithi156.stdout:9/464: read d2/d5/d16/d22/d36/d33/d3a/f49 [3599630,13914] 0 2024-07-16T09:55:38.800 INFO:tasks.workunit.client.0.smithi156.stdout:9/465: truncate d2/d5/d16/f37 737940 0 2024-07-16T09:55:38.800 INFO:tasks.workunit.client.1.smithi176.stdout:9/518: dread d5/de/d37/d58/f5c [0,4194304] 0 2024-07-16T09:55:38.801 INFO:tasks.workunit.client.0.smithi156.stdout:4/433: creat d4/db/dc/d7f/f99 x:0 0 0 2024-07-16T09:55:38.802 INFO:tasks.workunit.client.1.smithi176.stdout:2/458: mknod dd/d49/d52/d7c/c92 0 2024-07-16T09:55:38.802 INFO:tasks.workunit.client.1.smithi176.stdout:9/519: dread - d5/d1d/d20/d78/f8f zero size 2024-07-16T09:55:38.802 INFO:tasks.workunit.client.0.smithi156.stdout:3/372: readlink d8/da/d1d/d2c/d33/l7a 0 2024-07-16T09:55:38.805 INFO:tasks.workunit.client.0.smithi156.stdout:3/373: mknod d8/da/d28/d43/d63/c7e 0 2024-07-16T09:55:38.805 INFO:tasks.workunit.client.1.smithi176.stdout:2/459: symlink dd/d16/d39/d3d/l93 0 2024-07-16T09:55:38.806 INFO:tasks.workunit.client.0.smithi156.stdout:4/434: mknod d4/d23/c9a 0 2024-07-16T09:55:38.806 INFO:tasks.workunit.client.1.smithi176.stdout:9/520: creat d5/d1d/d77/d98/fb1 x:0 0 0 2024-07-16T09:55:38.807 INFO:tasks.workunit.client.0.smithi156.stdout:9/466: dread d2/f4d [0,4194304] 0 2024-07-16T09:55:38.807 INFO:tasks.workunit.client.1.smithi176.stdout:9/521: chown d5/de/d21/f5f 449 1 2024-07-16T09:55:38.810 INFO:tasks.workunit.client.1.smithi176.stdout:2/460: creat dd/d36/f94 x:0 0 0 2024-07-16T09:55:38.811 INFO:tasks.workunit.client.0.smithi156.stdout:3/374: link d8/da/cd d8/da/d13/c7f 0 2024-07-16T09:55:38.813 INFO:tasks.workunit.client.0.smithi156.stdout:3/375: write d8/d14/f3f [131871,25136] 0 2024-07-16T09:55:38.814 INFO:tasks.workunit.client.0.smithi156.stdout:4/435: dwrite d4/db/f7b [4194304,4194304] 0 2024-07-16T09:55:38.814 INFO:tasks.workunit.client.1.smithi176.stdout:9/522: dread d5/de/da5/f4e [0,4194304] 0 2024-07-16T09:55:38.816 INFO:tasks.workunit.client.1.smithi176.stdout:2/461: rename dd/d2e/c74 to dd/d2e/d35/d59/d66/c95 0 2024-07-16T09:55:38.816 INFO:tasks.workunit.client.0.smithi156.stdout:3/376: mkdir d8/da/d21/d80 0 2024-07-16T09:55:38.818 INFO:tasks.workunit.client.0.smithi156.stdout:6/415: sync 2024-07-16T09:55:38.818 INFO:tasks.workunit.client.0.smithi156.stdout:7/349: sync 2024-07-16T09:55:38.819 INFO:tasks.workunit.client.0.smithi156.stdout:1/597: sync 2024-07-16T09:55:38.819 INFO:tasks.workunit.client.0.smithi156.stdout:7/350: fsync dc/d13/d36/f66 0 2024-07-16T09:55:38.819 INFO:tasks.workunit.client.0.smithi156.stdout:4/436: rename d4/f8 to d4/db/dc/d67/f9b 0 2024-07-16T09:55:38.820 INFO:tasks.workunit.client.1.smithi176.stdout:9/523: dread d5/f35 [0,4194304] 0 2024-07-16T09:55:38.820 INFO:tasks.workunit.client.1.smithi176.stdout:2/462: creat dd/d36/d82/f96 x:0 0 0 2024-07-16T09:55:38.820 INFO:tasks.workunit.client.0.smithi156.stdout:6/416: stat d3/d13/d17/d4f/f14 0 2024-07-16T09:55:38.822 INFO:tasks.workunit.client.0.smithi156.stdout:1/598: write dc/d14/d3e/f43 [3897967,9689] 0 2024-07-16T09:55:38.822 INFO:tasks.workunit.client.0.smithi156.stdout:0/394: truncate d0/d1a/d4d/d9/d27/f6a 2839989 0 2024-07-16T09:55:38.824 INFO:tasks.workunit.client.0.smithi156.stdout:3/377: symlink d8/da/d1d/d2c/l81 0 2024-07-16T09:55:38.824 INFO:tasks.workunit.client.1.smithi176.stdout:9/524: unlink d5/d1d/d20/d8d/f9c 0 2024-07-16T09:55:38.824 INFO:tasks.workunit.client.1.smithi176.stdout:2/463: creat dd/d16/d39/d3d/f97 x:0 0 0 2024-07-16T09:55:38.825 INFO:tasks.workunit.client.0.smithi156.stdout:4/437: symlink d4/db/dc/d5f/d64/l9c 0 2024-07-16T09:55:38.826 INFO:tasks.workunit.client.0.smithi156.stdout:3/378: write d8/da/d28/d43/f4b [1378930,116142] 0 2024-07-16T09:55:38.828 INFO:tasks.workunit.client.0.smithi156.stdout:7/351: creat dc/d3e/d3f/d45/f7d x:0 0 0 2024-07-16T09:55:38.829 INFO:tasks.workunit.client.1.smithi176.stdout:2/464: truncate dd/de/f10 1297681 0 2024-07-16T09:55:38.829 INFO:tasks.workunit.client.0.smithi156.stdout:6/417: symlink d3/d13/d17/d56/l8d 0 2024-07-16T09:55:38.830 INFO:tasks.workunit.client.0.smithi156.stdout:0/395: creat d0/d22/d41/d52/f74 x:0 0 0 2024-07-16T09:55:38.832 INFO:tasks.workunit.client.1.smithi176.stdout:2/465: symlink dd/de/l98 0 2024-07-16T09:55:38.833 INFO:tasks.workunit.client.1.smithi176.stdout:9/525: dwrite d5/d1d/d20/d78/f8f [0,4194304] 0 2024-07-16T09:55:38.834 INFO:tasks.workunit.client.1.smithi176.stdout:2/466: chown dd/d2e/f3e 3 1 2024-07-16T09:55:38.834 INFO:tasks.workunit.client.0.smithi156.stdout:1/599: rename dc/d14/d1e/f29 to dc/d14/d3e/d6c/d7a/dcb/fce 0 2024-07-16T09:55:38.836 INFO:tasks.workunit.client.0.smithi156.stdout:3/379: dwrite d8/da/f3d [0,4194304] 0 2024-07-16T09:55:38.836 INFO:tasks.workunit.client.0.smithi156.stdout:4/438: chown d4/db/l12 11998273 1 2024-07-16T09:55:38.840 INFO:tasks.workunit.client.0.smithi156.stdout:6/418: dread d3/d13/d17/d4f/d5c/f65 [0,4194304] 0 2024-07-16T09:55:38.842 INFO:tasks.workunit.client.0.smithi156.stdout:0/396: mknod d0/d22/d41/d52/c75 0 2024-07-16T09:55:38.843 INFO:tasks.workunit.client.1.smithi176.stdout:2/467: rename dd/d2e/c69 to dd/d49/d52/d89/c99 0 2024-07-16T09:55:38.846 INFO:tasks.workunit.client.1.smithi176.stdout:2/468: symlink dd/d49/d52/d89/l9a 0 2024-07-16T09:55:38.847 INFO:tasks.workunit.client.1.smithi176.stdout:2/469: chown dd/d49/d52/d7c/d81 110 1 2024-07-16T09:55:38.848 INFO:tasks.workunit.client.0.smithi156.stdout:1/600: write dc/d24/fa3 [742161,115322] 0 2024-07-16T09:55:38.848 INFO:tasks.workunit.client.1.smithi176.stdout:2/470: dread - dd/d16/d39/d3d/f80 zero size 2024-07-16T09:55:38.850 INFO:tasks.workunit.client.1.smithi176.stdout:2/471: truncate dd/d2e/d35/d59/d66/f6e 933187 0 2024-07-16T09:55:38.852 INFO:tasks.workunit.client.0.smithi156.stdout:9/467: dread d2/d5/d16/d22/d36/d33/d3a/f3b [0,4194304] 0 2024-07-16T09:55:38.852 INFO:tasks.workunit.client.1.smithi176.stdout:2/472: rmdir dd/d16/d39/d3d 39 2024-07-16T09:55:38.853 INFO:tasks.workunit.client.0.smithi156.stdout:0/397: mkdir d0/d1a/d76 0 2024-07-16T09:55:38.855 INFO:tasks.workunit.client.0.smithi156.stdout:7/352: dwrite dc/d13/d36/f6a [0,4194304] 0 2024-07-16T09:55:38.855 INFO:tasks.workunit.client.0.smithi156.stdout:9/468: stat d2/c24 0 2024-07-16T09:55:38.856 INFO:tasks.workunit.client.1.smithi176.stdout:2/473: mkdir dd/d9b 0 2024-07-16T09:55:38.857 INFO:tasks.workunit.client.0.smithi156.stdout:3/380: write d8/fb [2861973,44753] 0 2024-07-16T09:55:38.859 INFO:tasks.workunit.client.0.smithi156.stdout:0/398: write d0/d1a/d4d/d9/f26 [3860409,129756] 0 2024-07-16T09:55:38.859 INFO:tasks.workunit.client.0.smithi156.stdout:1/601: unlink dc/d14/d3e/d55/d58/l79 0 2024-07-16T09:55:38.859 INFO:tasks.workunit.client.0.smithi156.stdout:4/439: dwrite d4/db/dc/f88 [0,4194304] 0 2024-07-16T09:55:38.860 INFO:tasks.workunit.client.1.smithi176.stdout:2/474: rename dd/d49/c85 to dd/d49/d52/d7c/d81/c9c 0 2024-07-16T09:55:38.861 INFO:tasks.workunit.client.0.smithi156.stdout:7/353: unlink dc/d13/f12 0 2024-07-16T09:55:38.862 INFO:tasks.workunit.client.1.smithi176.stdout:9/526: fdatasync d5/d1d/d20/d78/f8f 0 2024-07-16T09:55:38.866 INFO:tasks.workunit.client.1.smithi176.stdout:9/527: creat d5/d52/fb2 x:0 0 0 2024-07-16T09:55:38.866 INFO:tasks.workunit.client.0.smithi156.stdout:0/399: write d0/f43 [3493481,68721] 0 2024-07-16T09:55:38.867 INFO:tasks.workunit.client.1.smithi176.stdout:2/475: rmdir dd/d16/d39/d3d 39 2024-07-16T09:55:38.869 INFO:tasks.workunit.client.1.smithi176.stdout:2/476: chown dd/d2e/d35/d4a/c8a 506 1 2024-07-16T09:55:38.871 INFO:tasks.workunit.client.0.smithi156.stdout:0/400: mknod d0/d1a/c77 0 2024-07-16T09:55:38.871 INFO:tasks.workunit.client.0.smithi156.stdout:1/602: creat dc/d14/d3e/d45/fcf x:0 0 0 2024-07-16T09:55:38.872 INFO:tasks.workunit.client.1.smithi176.stdout:2/477: chown dd/d16/d39/d3d/f87 62 1 2024-07-16T09:55:38.873 INFO:tasks.workunit.client.1.smithi176.stdout:9/528: rename d5/de/d37 to d5/d1d/db3 0 2024-07-16T09:55:38.873 INFO:tasks.workunit.client.0.smithi156.stdout:9/469: dwrite d2/d5/d16/d22/f7c [0,4194304] 0 2024-07-16T09:55:38.874 INFO:tasks.workunit.client.0.smithi156.stdout:9/470: chown d2/d5/d16/d22 15256712 1 2024-07-16T09:55:38.875 INFO:tasks.workunit.client.1.smithi176.stdout:9/529: dread d5/de/da5/f4e [0,4194304] 0 2024-07-16T09:55:38.876 INFO:tasks.workunit.client.0.smithi156.stdout:9/471: stat d2/d5/d16/f20 0 2024-07-16T09:55:38.876 INFO:tasks.workunit.client.0.smithi156.stdout:3/381: link d8/da/f2e d8/da/d21/d80/f82 0 2024-07-16T09:55:38.876 INFO:tasks.workunit.client.0.smithi156.stdout:0/401: mknod d0/d1a/d4d/d61/c78 0 2024-07-16T09:55:38.877 INFO:tasks.workunit.client.1.smithi176.stdout:2/478: dwrite dd/d16/f33 [0,4194304] 0 2024-07-16T09:55:38.879 INFO:tasks.workunit.client.1.smithi176.stdout:9/530: symlink d5/d1d/d67/d6d/lb4 0 2024-07-16T09:55:38.883 INFO:tasks.workunit.client.0.smithi156.stdout:1/603: dwrite dc/d20/f9d [0,4194304] 0 2024-07-16T09:55:38.887 INFO:tasks.workunit.client.0.smithi156.stdout:1/604: chown dc/d14/d3e/d6c/d7a/dcb/fce 93 1 2024-07-16T09:55:38.892 INFO:tasks.workunit.client.0.smithi156.stdout:9/472: dwrite d2/d5/d16/d22/d36/d33/f2c [0,4194304] 0 2024-07-16T09:55:38.893 INFO:tasks.workunit.client.1.smithi176.stdout:9/531: symlink d5/de/d30/lb5 0 2024-07-16T09:55:38.897 INFO:tasks.workunit.client.1.smithi176.stdout:9/532: symlink d5/d1d/d20/lb6 0 2024-07-16T09:55:38.900 INFO:tasks.workunit.client.1.smithi176.stdout:9/533: chown d5/d1d/d20/l59 2104114 1 2024-07-16T09:55:38.900 INFO:tasks.workunit.client.0.smithi156.stdout:7/354: dread dc/d48/f4b [0,4194304] 0 2024-07-16T09:55:38.902 INFO:tasks.workunit.client.1.smithi176.stdout:9/534: symlink d5/d52/lb7 0 2024-07-16T09:55:38.903 INFO:tasks.workunit.client.1.smithi176.stdout:9/535: readlink d5/d1d/d67/la0 0 2024-07-16T09:55:38.905 INFO:tasks.workunit.client.1.smithi176.stdout:2/479: dwrite dd/d2e/f83 [0,4194304] 0 2024-07-16T09:55:38.906 INFO:tasks.workunit.client.0.smithi156.stdout:0/402: rmdir d0/d1a/d4d/d61/d73 0 2024-07-16T09:55:38.906 INFO:tasks.workunit.client.1.smithi176.stdout:2/480: chown dd/d2e/f3e 294 1 2024-07-16T09:55:38.907 INFO:tasks.workunit.client.0.smithi156.stdout:9/473: link d2/d5/d16/d22/d2b/d3d/d3c/l40 d2/d5/d16/d22/d6b/l90 0 2024-07-16T09:55:38.909 INFO:tasks.workunit.client.0.smithi156.stdout:9/474: stat d2/d5/d16/d22/d36/l71 0 2024-07-16T09:55:38.909 INFO:tasks.workunit.client.1.smithi176.stdout:2/481: mknod dd/d49/c9d 0 2024-07-16T09:55:38.910 INFO:tasks.workunit.client.0.smithi156.stdout:1/605: dread dc/d14/d3e/d55/d57/f67 [0,4194304] 0 2024-07-16T09:55:38.910 INFO:tasks.workunit.client.1.smithi176.stdout:9/536: creat d5/fb8 x:0 0 0 2024-07-16T09:55:38.911 INFO:tasks.workunit.client.1.smithi176.stdout:9/537: dread - d5/d52/fb2 zero size 2024-07-16T09:55:38.912 INFO:tasks.workunit.client.1.smithi176.stdout:2/482: mknod dd/d2e/d35/d4a/c9e 0 2024-07-16T09:55:38.913 INFO:tasks.workunit.client.1.smithi176.stdout:9/538: readlink d5/de/l42 0 2024-07-16T09:55:38.913 INFO:tasks.workunit.client.0.smithi156.stdout:4/440: truncate d4/db/dc/d5f/f5c 772773 0 2024-07-16T09:55:38.914 INFO:tasks.workunit.client.1.smithi176.stdout:9/539: dread d5/f35 [0,4194304] 0 2024-07-16T09:55:38.914 INFO:tasks.workunit.client.1.smithi176.stdout:2/483: creat dd/d9b/f9f x:0 0 0 2024-07-16T09:55:38.914 INFO:tasks.workunit.client.0.smithi156.stdout:9/475: creat d2/d5/f91 x:0 0 0 2024-07-16T09:55:38.916 INFO:tasks.workunit.client.0.smithi156.stdout:7/355: rename dc/d3e/c47 to dc/d3e/c7e 0 2024-07-16T09:55:38.916 INFO:tasks.workunit.client.1.smithi176.stdout:2/484: write dd/d2e/d35/d59/d66/f6e [1358135,111007] 0 2024-07-16T09:55:38.916 INFO:tasks.workunit.client.1.smithi176.stdout:9/540: creat d5/d1d/db3/d58/d89/fb9 x:0 0 0 2024-07-16T09:55:38.917 INFO:tasks.workunit.client.0.smithi156.stdout:1/606: rmdir dc/d14/d3e/d45 39 2024-07-16T09:55:38.917 INFO:tasks.workunit.client.1.smithi176.stdout:9/541: readlink d5/l7 0 2024-07-16T09:55:38.919 INFO:tasks.workunit.client.1.smithi176.stdout:2/485: symlink dd/d16/d39/d3d/d7e/la0 0 2024-07-16T09:55:38.919 INFO:tasks.workunit.client.0.smithi156.stdout:3/382: write d8/da/d21/d80/f82 [1337339,114332] 0 2024-07-16T09:55:38.919 INFO:tasks.workunit.client.1.smithi176.stdout:9/542: mknod d5/de/d30/daf/cba 0 2024-07-16T09:55:38.920 INFO:tasks.workunit.client.0.smithi156.stdout:3/383: chown d8/da/d28/f66 8394 1 2024-07-16T09:55:38.921 INFO:tasks.workunit.client.1.smithi176.stdout:9/543: write d5/de/f47 [1409903,35354] 0 2024-07-16T09:55:38.921 INFO:tasks.workunit.client.0.smithi156.stdout:4/441: read d4/db/f2c [862070,69428] 0 2024-07-16T09:55:38.923 INFO:tasks.workunit.client.1.smithi176.stdout:9/544: mknod d5/de/d30/daf/cbb 0 2024-07-16T09:55:38.925 INFO:tasks.workunit.client.0.smithi156.stdout:1/607: rename dc/d14/d3e/d55/d57/l5f to dc/d14/d8b/db4/ld0 0 2024-07-16T09:55:38.927 INFO:tasks.workunit.client.1.smithi176.stdout:9/545: dread d5/d1d/d20/f25 [4194304,4194304] 0 2024-07-16T09:55:38.930 INFO:tasks.workunit.client.1.smithi176.stdout:2/486: dwrite dd/d16/d39/f4b [0,4194304] 0 2024-07-16T09:55:38.930 INFO:tasks.workunit.client.1.smithi176.stdout:9/546: readlink d5/lc 0 2024-07-16T09:55:38.930 INFO:tasks.workunit.client.0.smithi156.stdout:0/403: dread d0/d1a/d4d/f21 [0,4194304] 0 2024-07-16T09:55:38.931 INFO:tasks.workunit.client.0.smithi156.stdout:4/442: mknod d4/db/d47/d63/d58/c9d 0 2024-07-16T09:55:38.931 INFO:tasks.workunit.client.1.smithi176.stdout:2/487: dread - dd/d49/d52/f91 zero size 2024-07-16T09:55:38.932 INFO:tasks.workunit.client.1.smithi176.stdout:2/488: chown dd/d36/d82 4685907 1 2024-07-16T09:55:38.934 INFO:tasks.workunit.client.0.smithi156.stdout:1/608: chown dc/d14/d3e/d55/d83/c92 31 1 2024-07-16T09:55:38.936 INFO:tasks.workunit.client.1.smithi176.stdout:2/489: readlink dd/d16/d39/l72 0 2024-07-16T09:55:38.937 INFO:tasks.workunit.client.0.smithi156.stdout:7/356: creat dc/d3e/f7f x:0 0 0 2024-07-16T09:55:38.938 INFO:tasks.workunit.client.0.smithi156.stdout:4/443: symlink d4/d23/d52/d71/l9e 0 2024-07-16T09:55:38.939 INFO:tasks.workunit.client.0.smithi156.stdout:1/609: dread dc/f4a [0,4194304] 0 2024-07-16T09:55:38.941 INFO:tasks.workunit.client.1.smithi176.stdout:9/547: truncate d5/de/f47 1861522 0 2024-07-16T09:55:38.941 INFO:tasks.workunit.client.0.smithi156.stdout:1/610: stat dc/d24/c68 0 2024-07-16T09:55:38.941 INFO:tasks.workunit.client.0.smithi156.stdout:0/404: creat d0/d1a/d76/f79 x:0 0 0 2024-07-16T09:55:38.942 INFO:tasks.workunit.client.0.smithi156.stdout:9/476: unlink d2/d5/d16/f20 0 2024-07-16T09:55:38.942 INFO:tasks.workunit.client.0.smithi156.stdout:3/384: creat d8/da/d1d/f83 x:0 0 0 2024-07-16T09:55:38.942 INFO:tasks.workunit.client.1.smithi176.stdout:2/490: rename dd/de/l14 to dd/d36/la1 0 2024-07-16T09:55:38.943 INFO:tasks.workunit.client.1.smithi176.stdout:9/548: rmdir d5/d43 39 2024-07-16T09:55:38.944 INFO:tasks.workunit.client.0.smithi156.stdout:1/611: fsync dc/d14/fc8 0 2024-07-16T09:55:38.944 INFO:tasks.workunit.client.1.smithi176.stdout:2/491: symlink dd/d36/d45/la2 0 2024-07-16T09:55:38.945 INFO:tasks.workunit.client.1.smithi176.stdout:9/549: symlink d5/de/da5/lbc 0 2024-07-16T09:55:38.945 INFO:tasks.workunit.client.1.smithi176.stdout:2/492: chown dd/d16/f32 12122736 1 2024-07-16T09:55:38.946 INFO:tasks.workunit.client.0.smithi156.stdout:0/405: creat d0/d22/f7a x:0 0 0 2024-07-16T09:55:38.946 INFO:tasks.workunit.client.0.smithi156.stdout:4/444: dread d4/db/dc/f55 [0,4194304] 0 2024-07-16T09:55:38.946 INFO:tasks.workunit.client.1.smithi176.stdout:2/493: chown dd/d49/d52/f91 9097166 1 2024-07-16T09:55:38.949 INFO:tasks.workunit.client.0.smithi156.stdout:7/357: dwrite dc/d3e/d3f/d45/f7d [0,4194304] 0 2024-07-16T09:55:38.949 INFO:tasks.workunit.client.0.smithi156.stdout:9/477: write d2/d5/d16/d22/d2b/d3d/d3c/d7f/d42/f85 [1027944,118931] 0 2024-07-16T09:55:38.949 INFO:tasks.workunit.client.0.smithi156.stdout:3/385: creat d8/da/d1d/d2c/f84 x:0 0 0 2024-07-16T09:55:38.951 INFO:tasks.workunit.client.1.smithi176.stdout:2/494: unlink dd/d16/d39/f6f 0 2024-07-16T09:55:38.952 INFO:tasks.workunit.client.0.smithi156.stdout:1/612: unlink dc/d14/d3e/d55/l8f 0 2024-07-16T09:55:38.954 INFO:tasks.workunit.client.1.smithi176.stdout:9/550: rename d5/de/d30/c6e to d5/de/cbd 0 2024-07-16T09:55:38.955 INFO:tasks.workunit.client.0.smithi156.stdout:7/358: creat dc/d3e/d5d/f80 x:0 0 0 2024-07-16T09:55:38.955 INFO:tasks.workunit.client.0.smithi156.stdout:9/478: read d2/d5/d16/d22/d2b/d3d/f6d [614798,111948] 0 2024-07-16T09:55:38.957 INFO:tasks.workunit.client.1.smithi176.stdout:2/495: mknod dd/d16/d39/d3d/d7e/ca3 0 2024-07-16T09:55:38.957 INFO:tasks.workunit.client.0.smithi156.stdout:4/445: rename d4/d23/d30/l76 to d4/db/d47/d63/d58/d8f/l9f 0 2024-07-16T09:55:38.959 INFO:tasks.workunit.client.0.smithi156.stdout:7/359: read dc/d13/d15/f50 [2403839,110626] 0 2024-07-16T09:55:38.961 INFO:tasks.workunit.client.0.smithi156.stdout:0/406: dwrite d0/d1a/d4d/d9/f15 [0,4194304] 0 2024-07-16T09:55:38.962 INFO:tasks.workunit.client.0.smithi156.stdout:9/479: creat d2/d5/d16/d22/d6b/f92 x:0 0 0 2024-07-16T09:55:38.962 INFO:tasks.workunit.client.1.smithi176.stdout:9/551: dwrite d5/d1d/d20/d8d/f8c [0,4194304] 0 2024-07-16T09:55:38.964 INFO:tasks.workunit.client.0.smithi156.stdout:0/407: write d0/d1a/d4d/d9/f12 [363421,110828] 0 2024-07-16T09:55:38.964 INFO:tasks.workunit.client.0.smithi156.stdout:7/360: mkdir dc/d44/d81 0 2024-07-16T09:55:38.964 INFO:tasks.workunit.client.0.smithi156.stdout:9/480: chown d2/d5/d16/d22/d6b/f92 110328808 1 2024-07-16T09:55:38.964 INFO:tasks.workunit.client.0.smithi156.stdout:0/408: stat d0/ff 0 2024-07-16T09:55:38.966 INFO:tasks.workunit.client.0.smithi156.stdout:4/446: rename d4/f80 to d4/db/dc/d7f/d87/fa0 0 2024-07-16T09:55:38.967 INFO:tasks.workunit.client.0.smithi156.stdout:3/386: link l1 d8/da/d21/d80/l85 0 2024-07-16T09:55:38.968 INFO:tasks.workunit.client.1.smithi176.stdout:2/496: dwrite dd/d2e/d35/d59/d66/f8f [0,4194304] 0 2024-07-16T09:55:38.968 INFO:tasks.workunit.client.0.smithi156.stdout:3/387: fsync d8/da/d28/f66 0 2024-07-16T09:55:38.969 INFO:tasks.workunit.client.1.smithi176.stdout:7/475: sync 2024-07-16T09:55:38.969 INFO:tasks.workunit.client.1.smithi176.stdout:8/478: sync 2024-07-16T09:55:38.972 INFO:tasks.workunit.client.1.smithi176.stdout:7/476: dread - d4/d1f/d22/d80/d17/d1d/f73 zero size 2024-07-16T09:55:38.973 INFO:tasks.workunit.client.0.smithi156.stdout:7/361: symlink dc/d3e/d3f/d51/l82 0 2024-07-16T09:55:38.977 INFO:tasks.workunit.client.0.smithi156.stdout:3/388: mknod d8/da/d28/c86 0 2024-07-16T09:55:38.977 INFO:tasks.workunit.client.0.smithi156.stdout:0/409: getdents d0/d1a/d4d/d60 0 2024-07-16T09:55:38.977 INFO:tasks.workunit.client.0.smithi156.stdout:7/362: symlink dc/d3e/d3f/d45/l83 0 2024-07-16T09:55:38.977 INFO:tasks.workunit.client.1.smithi176.stdout:2/497: getdents dd/d16/d39/d3d/d7e 0 2024-07-16T09:55:38.978 INFO:tasks.workunit.client.1.smithi176.stdout:9/552: getdents d5/d1d/d67/d6d 0 2024-07-16T09:55:38.979 INFO:tasks.workunit.client.0.smithi156.stdout:7/363: fdatasync dc/d13/d36/f4c 0 2024-07-16T09:55:38.979 INFO:tasks.workunit.client.0.smithi156.stdout:1/613: rename dc/d14/d3e/d55/d57/l7f to dc/d14/d1e/ld1 0 2024-07-16T09:55:38.979 INFO:tasks.workunit.client.0.smithi156.stdout:7/364: dread - dc/d13/d36/f7a zero size 2024-07-16T09:55:38.980 INFO:tasks.workunit.client.0.smithi156.stdout:4/447: creat d4/d23/d30/fa1 x:0 0 0 2024-07-16T09:55:38.980 INFO:tasks.workunit.client.0.smithi156.stdout:1/614: chown dc/d24/fa3 0 1 2024-07-16T09:55:38.981 INFO:tasks.workunit.client.1.smithi176.stdout:9/553: symlink d5/de/da5/lbe 0 2024-07-16T09:55:38.981 INFO:tasks.workunit.client.0.smithi156.stdout:3/389: mknod d8/da/d28/d43/c87 0 2024-07-16T09:55:38.983 INFO:tasks.workunit.client.1.smithi176.stdout:9/554: chown d5/de/l18 768387433 1 2024-07-16T09:55:38.983 INFO:tasks.workunit.client.0.smithi156.stdout:9/481: rename d2/d5/d16/d22/d36/f50 to d2/d5/d16/d22/d36/d6a/f93 0 2024-07-16T09:55:38.984 INFO:tasks.workunit.client.1.smithi176.stdout:7/477: dwrite d4/dd/d26/f7b [0,4194304] 0 2024-07-16T09:55:38.986 INFO:tasks.workunit.client.1.smithi176.stdout:2/498: rename dd/de/c2b to dd/d2e/d35/ca4 0 2024-07-16T09:55:38.987 INFO:tasks.workunit.client.0.smithi156.stdout:4/448: unlink d4/db/dc/f4d 0 2024-07-16T09:55:38.988 INFO:tasks.workunit.client.1.smithi176.stdout:9/555: mkdir d5/de/da5/dbf 0 2024-07-16T09:55:38.988 INFO:tasks.workunit.client.0.smithi156.stdout:3/390: dwrite d8/da/d1d/f58 [0,4194304] 0 2024-07-16T09:55:38.989 INFO:tasks.workunit.client.0.smithi156.stdout:1/615: mkdir dc/d14/d8b/dcd/dd2 0 2024-07-16T09:55:38.990 INFO:tasks.workunit.client.1.smithi176.stdout:9/556: write d5/fd [3111839,128035] 0 2024-07-16T09:55:38.991 INFO:tasks.workunit.client.0.smithi156.stdout:3/391: write d8/da/d1d/d2c/f52 [4451891,43298] 0 2024-07-16T09:55:38.992 INFO:tasks.workunit.client.1.smithi176.stdout:9/557: dread - d5/d1d/d20/f7d zero size 2024-07-16T09:55:38.993 INFO:tasks.workunit.client.0.smithi156.stdout:3/392: write d8/d14/f3f [1053569,282] 0 2024-07-16T09:55:38.994 INFO:tasks.workunit.client.0.smithi156.stdout:4/449: unlink d4/db/f61 0 2024-07-16T09:55:38.996 INFO:tasks.workunit.client.1.smithi176.stdout:2/499: dwrite dd/d36/d82/f96 [0,4194304] 0 2024-07-16T09:55:38.998 INFO:tasks.workunit.client.1.smithi176.stdout:9/558: mknod d5/d1d/d20/d8d/cc0 0 2024-07-16T09:55:38.998 INFO:tasks.workunit.client.1.smithi176.stdout:8/479: write d8/d14/f27 [1086859,77594] 0 2024-07-16T09:55:38.999 INFO:tasks.workunit.client.0.smithi156.stdout:0/410: rename d0/d1a/d76/f79 to d0/d1a/d4d/d9/f7b 0 2024-07-16T09:55:39.001 INFO:tasks.workunit.client.1.smithi176.stdout:8/480: fsync d8/d23/d76/f98 0 2024-07-16T09:55:39.001 INFO:tasks.workunit.client.1.smithi176.stdout:2/500: chown dd/d16/d39/f6d 0 1 2024-07-16T09:55:39.001 INFO:tasks.workunit.client.0.smithi156.stdout:0/411: chown d0/d1a/d4d/d61/c78 1364296548 1 2024-07-16T09:55:39.001 INFO:tasks.workunit.client.1.smithi176.stdout:9/559: mkdir d5/de/da5/dc1 0 2024-07-16T09:55:39.005 INFO:tasks.workunit.client.1.smithi176.stdout:2/501: symlink dd/d49/d52/d89/la5 0 2024-07-16T09:55:39.006 INFO:tasks.workunit.client.0.smithi156.stdout:4/450: truncate d4/d23/d30/f25 72757 0 2024-07-16T09:55:39.009 INFO:tasks.workunit.client.0.smithi156.stdout:2/528: sync 2024-07-16T09:55:39.009 INFO:tasks.workunit.client.0.smithi156.stdout:8/368: sync 2024-07-16T09:55:39.010 INFO:tasks.workunit.client.0.smithi156.stdout:7/365: rmdir dc/d3e/d3f/d45 39 2024-07-16T09:55:39.011 INFO:tasks.workunit.client.1.smithi176.stdout:2/502: fsync dd/de/f10 0 2024-07-16T09:55:39.011 INFO:tasks.workunit.client.0.smithi156.stdout:8/369: write d0/d11/f72 [596737,82419] 0 2024-07-16T09:55:39.013 INFO:tasks.workunit.client.1.smithi176.stdout:7/478: dread d4/dd/d49/f55 [0,4194304] 0 2024-07-16T09:55:39.013 INFO:tasks.workunit.client.0.smithi156.stdout:8/370: stat d0/d11/d4a/c7b 0 2024-07-16T09:55:39.013 INFO:tasks.workunit.client.0.smithi156.stdout:0/412: unlink d0/d1a/d4d/d9/d27/d3d/d62/c72 0 2024-07-16T09:55:39.014 INFO:tasks.workunit.client.1.smithi176.stdout:7/479: chown d4/d5/f91 41973255 1 2024-07-16T09:55:39.015 INFO:tasks.workunit.client.1.smithi176.stdout:9/560: dread d5/d1d/d67/f6a [0,4194304] 0 2024-07-16T09:55:39.016 INFO:tasks.workunit.client.0.smithi156.stdout:3/393: link d8/da/d13/f24 d8/da/d1d/d2c/d33/f88 0 2024-07-16T09:55:39.016 INFO:tasks.workunit.client.1.smithi176.stdout:9/561: chown d5/l7 161116 1 2024-07-16T09:55:39.017 INFO:tasks.workunit.client.1.smithi176.stdout:2/503: link dd/d49/d52/d89/l9a dd/d2e/d35/la6 0 2024-07-16T09:55:39.019 INFO:tasks.workunit.client.0.smithi156.stdout:1/616: rename dc/d14/d3e/d45/f86 to dc/d14/d3e/d55/d58/fd3 0 2024-07-16T09:55:39.020 INFO:tasks.workunit.client.0.smithi156.stdout:3/394: read d8/fb [1377823,32250] 0 2024-07-16T09:55:39.023 INFO:tasks.workunit.client.1.smithi176.stdout:7/480: dwrite d4/d1f/d22/d80/d17/d1d/f9b [0,4194304] 0 2024-07-16T09:55:39.023 INFO:tasks.workunit.client.0.smithi156.stdout:2/529: dwrite d0/d13/d17/d1c/d86/fb7 [0,4194304] 0 2024-07-16T09:55:39.025 INFO:tasks.workunit.client.1.smithi176.stdout:7/481: stat d4/d47/f88 0 2024-07-16T09:55:39.026 INFO:tasks.workunit.client.1.smithi176.stdout:8/481: write d8/d14/f2d [2956934,101282] 0 2024-07-16T09:55:39.027 INFO:tasks.workunit.client.0.smithi156.stdout:7/366: dwrite dc/d44/f6d [0,4194304] 0 2024-07-16T09:55:39.031 INFO:tasks.workunit.client.1.smithi176.stdout:7/482: rename d4/d5/c95 to d4/d1f/d22/d51/c9f 0 2024-07-16T09:55:39.033 INFO:tasks.workunit.client.0.smithi156.stdout:6/419: sync 2024-07-16T09:55:39.033 INFO:tasks.workunit.client.0.smithi156.stdout:5/498: sync 2024-07-16T09:55:39.035 INFO:tasks.workunit.client.1.smithi176.stdout:9/562: dread d5/de/f1a [0,4194304] 0 2024-07-16T09:55:39.036 INFO:tasks.workunit.client.0.smithi156.stdout:1/617: unlink dc/d14/d3e/f43 0 2024-07-16T09:55:39.036 INFO:tasks.workunit.client.0.smithi156.stdout:4/451: dwrite d4/db/f16 [4194304,4194304] 0 2024-07-16T09:55:39.037 INFO:tasks.workunit.client.1.smithi176.stdout:2/504: dwrite dd/d16/d39/d3d/d7e/f7f [0,4194304] 0 2024-07-16T09:55:39.040 INFO:tasks.workunit.client.0.smithi156.stdout:3/395: fsync d8/da/d28/f7c 0 2024-07-16T09:55:39.041 INFO:tasks.workunit.client.1.smithi176.stdout:7/483: dread d4/d1f/f2a [4194304,4194304] 0 2024-07-16T09:55:39.041 INFO:tasks.workunit.client.0.smithi156.stdout:4/452: stat d4/db/dc/d5f 0 2024-07-16T09:55:39.042 INFO:tasks.workunit.client.1.smithi176.stdout:7/484: chown d4/d5 4 1 2024-07-16T09:55:39.042 INFO:tasks.workunit.client.0.smithi156.stdout:0/413: dwrite d0/d1a/d4d/d9/f26 [0,4194304] 0 2024-07-16T09:55:39.042 INFO:tasks.workunit.client.0.smithi156.stdout:2/530: symlink d0/d13/lb8 0 2024-07-16T09:55:39.045 INFO:tasks.workunit.client.0.smithi156.stdout:1/618: read f6 [1742973,34770] 0 2024-07-16T09:55:39.047 INFO:tasks.workunit.client.0.smithi156.stdout:9/482: dwrite d2/d5/d16/d22/d2b/d3d/d3c/d7f/f19 [4194304,4194304] 0 2024-07-16T09:55:39.051 INFO:tasks.workunit.client.0.smithi156.stdout:7/367: write dc/d48/f4b [2446914,48496] 0 2024-07-16T09:55:39.051 INFO:tasks.workunit.client.0.smithi156.stdout:5/499: chown d1b/d1f/d45/d52/d43/d85/ca1 11643 1 2024-07-16T09:55:39.051 INFO:tasks.workunit.client.0.smithi156.stdout:6/420: write d3/d13/d17/f78 [211782,2243] 0 2024-07-16T09:55:39.053 INFO:tasks.workunit.client.1.smithi176.stdout:9/563: getdents d5/d1d/d77/d98 0 2024-07-16T09:55:39.054 INFO:tasks.workunit.client.0.smithi156.stdout:5/500: stat d1b/d1f/d45/d52/d43/d85/d97/la8 0 2024-07-16T09:55:39.054 INFO:tasks.workunit.client.0.smithi156.stdout:3/396: symlink d8/da/d28/d43/l89 0 2024-07-16T09:55:39.055 INFO:tasks.workunit.client.0.smithi156.stdout:7/368: truncate dc/d3e/f7f 501514 0 2024-07-16T09:55:39.055 INFO:tasks.workunit.client.1.smithi176.stdout:8/482: dwrite d8/d23/f21 [0,4194304] 0 2024-07-16T09:55:39.056 INFO:tasks.workunit.client.0.smithi156.stdout:3/397: fdatasync d8/da/d28/d40/f56 0 2024-07-16T09:55:39.056 INFO:tasks.workunit.client.0.smithi156.stdout:5/501: chown d1b/d1f/d45/d52/d49/d4d 0 1 2024-07-16T09:55:39.058 INFO:tasks.workunit.client.0.smithi156.stdout:1/619: fdatasync dc/d14/d3e/d55/d83/fa1 0 2024-07-16T09:55:39.059 INFO:tasks.workunit.client.1.smithi176.stdout:7/485: dwrite d4/d1f/d22/d51/f96 [0,4194304] 0 2024-07-16T09:55:39.059 INFO:tasks.workunit.client.0.smithi156.stdout:9/483: symlink d2/d5/d16/d57/l94 0 2024-07-16T09:55:39.061 INFO:tasks.workunit.client.0.smithi156.stdout:0/414: mknod d0/d1a/d4d/d9/d27/d3d/d62/c7c 0 2024-07-16T09:55:39.062 INFO:tasks.workunit.client.0.smithi156.stdout:6/421: unlink d3/d13/d75/d19/d31/l58 0 2024-07-16T09:55:39.062 INFO:tasks.workunit.client.1.smithi176.stdout:7/486: write d4/dd/f1a [2736126,76596] 0 2024-07-16T09:55:39.065 INFO:tasks.workunit.client.0.smithi156.stdout:7/369: creat dc/d13/d20/f84 x:0 0 0 2024-07-16T09:55:39.067 INFO:tasks.workunit.client.0.smithi156.stdout:3/398: symlink d8/da/d1d/d2c/d33/l8a 0 2024-07-16T09:55:39.067 INFO:tasks.workunit.client.0.smithi156.stdout:4/453: link d4/l1c d4/la2 0 2024-07-16T09:55:39.067 INFO:tasks.workunit.client.1.smithi176.stdout:9/564: link d5/d52/l76 d5/de/da5/lc2 0 2024-07-16T09:55:39.068 INFO:tasks.workunit.client.0.smithi156.stdout:5/502: mknod d1b/d1f/d54/d72/cac 0 2024-07-16T09:55:39.068 INFO:tasks.workunit.client.1.smithi176.stdout:2/505: dread dd/d36/f61 [0,4194304] 0 2024-07-16T09:55:39.069 INFO:tasks.workunit.client.1.smithi176.stdout:9/565: stat d5/de/d21/c29 0 2024-07-16T09:55:39.070 INFO:tasks.workunit.client.1.smithi176.stdout:7/487: rename d4/d1f/d22/l60 to d4/dd/d63/la0 0 2024-07-16T09:55:39.070 INFO:tasks.workunit.client.1.smithi176.stdout:7/488: stat d4 0 2024-07-16T09:55:39.071 INFO:tasks.workunit.client.1.smithi176.stdout:8/483: link d8/d23/d2c/d34/d62/l97 d8/d14/d1e/d33/la1 0 2024-07-16T09:55:39.071 INFO:tasks.workunit.client.0.smithi156.stdout:9/484: unlink d2/d5/d16/d22/d2b/d3d/d3c/d7f/c5a 0 2024-07-16T09:55:39.072 INFO:tasks.workunit.client.1.smithi176.stdout:7/489: write d4/d1f/d22/d80/d17/d1d/f5b [966013,38186] 0 2024-07-16T09:55:39.075 INFO:tasks.workunit.client.1.smithi176.stdout:2/506: dread - dd/d36/f5d zero size 2024-07-16T09:55:39.076 INFO:tasks.workunit.client.1.smithi176.stdout:9/566: getdents d5/d1d/db0 0 2024-07-16T09:55:39.076 INFO:tasks.workunit.client.0.smithi156.stdout:0/415: mkdir d0/d22/d41/d66/d7d 0 2024-07-16T09:55:39.078 INFO:tasks.workunit.client.1.smithi176.stdout:4/466: sync 2024-07-16T09:55:39.078 INFO:tasks.workunit.client.1.smithi176.stdout:0/555: sync 2024-07-16T09:55:39.079 INFO:tasks.workunit.client.1.smithi176.stdout:7/490: dread d4/dd/d49/f7c [0,4194304] 0 2024-07-16T09:55:39.079 INFO:tasks.workunit.client.0.smithi156.stdout:6/422: link d3/d13/d17/d4f/d1e/f3d d3/d13/f8e 0 2024-07-16T09:55:39.079 INFO:tasks.workunit.client.0.smithi156.stdout:5/503: rename d1b/c3a to d1b/d1f/d54/d72/cad 0 2024-07-16T09:55:39.080 INFO:tasks.workunit.client.1.smithi176.stdout:8/484: dwrite d8/d23/f25 [0,4194304] 0 2024-07-16T09:55:39.081 INFO:tasks.workunit.client.0.smithi156.stdout:4/454: fdatasync d4/d23/d52/d71/f77 0 2024-07-16T09:55:39.082 INFO:tasks.workunit.client.0.smithi156.stdout:7/370: dread dc/d3e/d3f/d51/f53 [0,4194304] 0 2024-07-16T09:55:39.082 INFO:tasks.workunit.client.1.smithi176.stdout:0/556: write d5/d1c/d24/d35/d50/d7c/f93 [757352,61106] 0 2024-07-16T09:55:39.083 INFO:tasks.workunit.client.0.smithi156.stdout:9/485: symlink d2/d5/d61/l95 0 2024-07-16T09:55:39.084 INFO:tasks.workunit.client.1.smithi176.stdout:9/567: getdents d5/d1d/d77/d98 0 2024-07-16T09:55:39.085 INFO:tasks.workunit.client.1.smithi176.stdout:7/491: write d4/d5/f2f [9203751,88453] 0 2024-07-16T09:55:39.087 INFO:tasks.workunit.client.1.smithi176.stdout:8/485: creat d8/d23/d2c/d37/fa2 x:0 0 0 2024-07-16T09:55:39.088 INFO:tasks.workunit.client.1.smithi176.stdout:0/557: mknod d5/d1c/d24/d35/d50/d9b/cb0 0 2024-07-16T09:55:39.089 INFO:tasks.workunit.client.1.smithi176.stdout:7/492: write d4/d47/d6d/d89/d66/f8a [957640,130106] 0 2024-07-16T09:55:39.090 INFO:tasks.workunit.client.1.smithi176.stdout:8/486: read - d8/d20/d2a/d31/d46/d6a/f7a zero size 2024-07-16T09:55:39.090 INFO:tasks.workunit.client.1.smithi176.stdout:9/568: creat d5/d1d/d20/fc3 x:0 0 0 2024-07-16T09:55:39.093 INFO:tasks.workunit.client.0.smithi156.stdout:3/399: dwrite d8/da/d28/f66 [0,4194304] 0 2024-07-16T09:55:39.093 INFO:tasks.workunit.client.1.smithi176.stdout:4/467: link da/dc/d10/f59 da/d3d/d17/d48/f8b 0 2024-07-16T09:55:39.093 INFO:tasks.workunit.client.1.smithi176.stdout:9/569: write d5/d1d/d20/f74 [1184788,71948] 0 2024-07-16T09:55:39.094 INFO:tasks.workunit.client.1.smithi176.stdout:4/468: chown da/c20 91 1 2024-07-16T09:55:39.095 INFO:tasks.workunit.client.1.smithi176.stdout:9/570: chown d5/d1d/c68 23415348 1 2024-07-16T09:55:39.095 INFO:tasks.workunit.client.1.smithi176.stdout:4/469: stat da/d14/d40/d5d/f85 0 2024-07-16T09:55:39.097 INFO:tasks.workunit.client.1.smithi176.stdout:9/571: truncate d5/d1d/d67/f6a 1655826 0 2024-07-16T09:55:39.098 INFO:tasks.workunit.client.0.smithi156.stdout:8/371: dwrite d0/ff [0,4194304] 0 2024-07-16T09:55:39.098 INFO:tasks.workunit.client.0.smithi156.stdout:0/416: write d0/f34 [545380,44634] 0 2024-07-16T09:55:39.099 INFO:tasks.workunit.client.1.smithi176.stdout:2/507: truncate dd/d16/f56 48505 0 2024-07-16T09:55:39.099 INFO:tasks.workunit.client.0.smithi156.stdout:3/400: creat d8/da/d28/d43/f8b x:0 0 0 2024-07-16T09:55:39.101 INFO:tasks.workunit.client.1.smithi176.stdout:8/487: fsync d8/d14/d1e/d7c/f8f 0 2024-07-16T09:55:39.101 INFO:tasks.workunit.client.0.smithi156.stdout:1/620: dwrite dc/d14/d3e/d55/d57/fa2 [0,4194304] 0 2024-07-16T09:55:39.105 INFO:tasks.workunit.client.0.smithi156.stdout:6/423: link d3/f25 d3/d13/d17/d4f/d4d/d3b/f8f 0 2024-07-16T09:55:39.106 INFO:tasks.workunit.client.0.smithi156.stdout:3/401: write d8/d14/f59 [15398,21643] 0 2024-07-16T09:55:39.106 INFO:tasks.workunit.client.1.smithi176.stdout:4/470: dwrite da/d14/d4a/f4e [0,4194304] 0 2024-07-16T09:55:39.106 INFO:tasks.workunit.client.1.smithi176.stdout:8/488: fdatasync d8/d23/f25 0 2024-07-16T09:55:39.110 INFO:tasks.workunit.client.1.smithi176.stdout:8/489: chown d8/d14/d1e/d33/f47 428 1 2024-07-16T09:55:39.111 INFO:tasks.workunit.client.1.smithi176.stdout:9/572: getdents d5/d1d/d77/d97 0 2024-07-16T09:55:39.111 INFO:tasks.workunit.client.1.smithi176.stdout:4/471: write da/d3d/d17/d54/d5e/d69/f6d [653910,114343] 0 2024-07-16T09:55:39.116 INFO:tasks.workunit.client.1.smithi176.stdout:9/573: write d5/de/f40 [5358225,3732] 0 2024-07-16T09:55:39.117 INFO:tasks.workunit.client.0.smithi156.stdout:1/621: chown dc/d14/d3e/lc6 19765 1 2024-07-16T09:55:39.118 INFO:tasks.workunit.client.1.smithi176.stdout:4/472: mknod da/d14/d40/d5d/d73/c8c 0 2024-07-16T09:55:39.118 INFO:tasks.workunit.client.0.smithi156.stdout:7/371: dread dc/d13/d20/f3b [0,4194304] 0 2024-07-16T09:55:39.120 INFO:tasks.workunit.client.1.smithi176.stdout:4/473: fdatasync da/d14/d40/d5d/d73/f3c 0 2024-07-16T09:55:39.120 INFO:tasks.workunit.client.0.smithi156.stdout:7/372: fdatasync dc/d13/f43 0 2024-07-16T09:55:39.121 INFO:tasks.workunit.client.1.smithi176.stdout:4/474: chown da/dc/l46 38916764 1 2024-07-16T09:55:39.122 INFO:tasks.workunit.client.1.smithi176.stdout:4/475: fdatasync da/dc/d10/d28/f79 0 2024-07-16T09:55:39.124 INFO:tasks.workunit.client.0.smithi156.stdout:4/455: dwrite d4/db/d47/d63/d58/d8f/f7e [0,4194304] 0 2024-07-16T09:55:39.124 INFO:tasks.workunit.client.1.smithi176.stdout:9/574: mknod d5/de/da5/dbf/cc4 0 2024-07-16T09:55:39.124 INFO:tasks.workunit.client.0.smithi156.stdout:4/456: fdatasync f2 0 2024-07-16T09:55:39.128 INFO:tasks.workunit.client.1.smithi176.stdout:7/493: dread d4/d47/d6d/d89/d66/f8a [0,4194304] 0 2024-07-16T09:55:39.128 INFO:tasks.workunit.client.1.smithi176.stdout:9/575: rmdir d5/d52 39 2024-07-16T09:55:39.128 INFO:tasks.workunit.client.1.smithi176.stdout:9/576: stat d5 0 2024-07-16T09:55:39.131 INFO:tasks.workunit.client.1.smithi176.stdout:4/476: mknod da/d3d/d17/d54/d78/d7a/c8d 0 2024-07-16T09:55:39.132 INFO:tasks.workunit.client.1.smithi176.stdout:2/508: write dd/d16/d39/f6d [774022,112376] 0 2024-07-16T09:55:39.133 INFO:tasks.workunit.client.0.smithi156.stdout:9/486: write d2/d5/d16/d22/d2b/d3d/d3c/d7f/d42/f4c [2108675,123836] 0 2024-07-16T09:55:39.133 INFO:tasks.workunit.client.1.smithi176.stdout:7/494: write d4/d47/d6d/d89/d66/f8a [4665107,27525] 0 2024-07-16T09:55:39.134 INFO:tasks.workunit.client.0.smithi156.stdout:3/402: creat d8/da/d28/d43/d71/f8c x:0 0 0 2024-07-16T09:55:39.135 INFO:tasks.workunit.client.0.smithi156.stdout:3/403: chown d8/f10 73894 1 2024-07-16T09:55:39.136 INFO:tasks.workunit.client.0.smithi156.stdout:5/504: dwrite d1b/d1f/d45/d52/d43/d7c/f82 [0,4194304] 0 2024-07-16T09:55:39.136 INFO:tasks.workunit.client.1.smithi176.stdout:4/477: write da/dc/f50 [650025,102416] 0 2024-07-16T09:55:39.137 INFO:tasks.workunit.client.0.smithi156.stdout:4/457: link d4/db/d47/d63/d58/d8f/f5b d4/db/dc/fa3 0 2024-07-16T09:55:39.138 INFO:tasks.workunit.client.0.smithi156.stdout:9/487: dread - d2/d5/d16/d22/d2b/f8b zero size 2024-07-16T09:55:39.139 INFO:tasks.workunit.client.1.smithi176.stdout:2/509: creat dd/de/fa7 x:0 0 0 2024-07-16T09:55:39.141 INFO:tasks.workunit.client.1.smithi176.stdout:4/478: write da/dc/d10/f27 [659665,128797] 0 2024-07-16T09:55:39.142 INFO:tasks.workunit.client.0.smithi156.stdout:9/488: chown d2/d5/d16/d22/d36/d33/f2c 634447 1 2024-07-16T09:55:39.142 INFO:tasks.workunit.client.0.smithi156.stdout:5/505: mkdir d1b/d99/dae 0 2024-07-16T09:55:39.143 INFO:tasks.workunit.client.1.smithi176.stdout:2/510: fsync dd/d36/f4d 0 2024-07-16T09:55:39.143 INFO:tasks.workunit.client.1.smithi176.stdout:9/577: dread d5/d1d/d67/d6d/f7a [0,4194304] 0 2024-07-16T09:55:39.145 INFO:tasks.workunit.client.0.smithi156.stdout:3/404: getdents d8/da/d21 0 2024-07-16T09:55:39.146 INFO:tasks.workunit.client.1.smithi176.stdout:4/479: fsync da/d14/f26 0 2024-07-16T09:55:39.148 INFO:tasks.workunit.client.1.smithi176.stdout:7/495: link d4/f19 d4/d47/d6d/d89/d7e/fa1 0 2024-07-16T09:55:39.148 INFO:tasks.workunit.client.0.smithi156.stdout:6/424: dwrite d3/d13/d17/d4f/d1e/f3d [0,4194304] 0 2024-07-16T09:55:39.148 INFO:tasks.workunit.client.0.smithi156.stdout:5/506: fdatasync d1b/d1f/d45/d52/d43/f63 0 2024-07-16T09:55:39.149 INFO:tasks.workunit.client.1.smithi176.stdout:0/558: dread d5/f18 [0,4194304] 0 2024-07-16T09:55:39.151 INFO:tasks.workunit.client.1.smithi176.stdout:7/496: readlink d4/dd/d26/l40 0 2024-07-16T09:55:39.151 INFO:tasks.workunit.client.1.smithi176.stdout:9/578: creat d5/d1d/d20/fc5 x:0 0 0 2024-07-16T09:55:39.153 INFO:tasks.workunit.client.1.smithi176.stdout:0/559: write d5/d1c/f8e [649540,98041] 0 2024-07-16T09:55:39.153 INFO:tasks.workunit.client.1.smithi176.stdout:4/480: creat da/d3d/d17/f8e x:0 0 0 2024-07-16T09:55:39.154 INFO:tasks.workunit.client.1.smithi176.stdout:7/497: chown d4/dd/l35 54 1 2024-07-16T09:55:39.155 INFO:tasks.workunit.client.1.smithi176.stdout:2/511: creat dd/de/fa8 x:0 0 0 2024-07-16T09:55:39.155 INFO:tasks.workunit.client.0.smithi156.stdout:5/507: write d1b/d1f/d45/d52/f35 [3021943,72028] 0 2024-07-16T09:55:39.155 INFO:tasks.workunit.client.0.smithi156.stdout:6/425: dread - d3/d13/d17/d4f/d30/d45/d5f/f8b zero size 2024-07-16T09:55:39.155 INFO:tasks.workunit.client.0.smithi156.stdout:4/458: dwrite d4/db/dc/d67/f86 [0,4194304] 0 2024-07-16T09:55:39.156 INFO:tasks.workunit.client.0.smithi156.stdout:9/489: mknod d2/d5/d16/d22/d2b/d3d/c96 0 2024-07-16T09:55:39.157 INFO:tasks.workunit.client.1.smithi176.stdout:8/490: dwrite d8/fa [4194304,4194304] 0 2024-07-16T09:55:39.158 INFO:tasks.workunit.client.0.smithi156.stdout:9/490: fdatasync d2/f4 0 2024-07-16T09:55:39.159 INFO:tasks.workunit.client.1.smithi176.stdout:9/579: write d5/de/f5b [397237,58787] 0 2024-07-16T09:55:39.160 INFO:tasks.workunit.client.0.smithi156.stdout:6/426: dread - d3/d13/d17/f8c zero size 2024-07-16T09:55:39.160 INFO:tasks.workunit.client.0.smithi156.stdout:0/417: dread d0/d1a/d4d/d25/f38 [0,4194304] 0 2024-07-16T09:55:39.165 INFO:tasks.workunit.client.1.smithi176.stdout:4/481: mknod da/d3d/d17/d54/c8f 0 2024-07-16T09:55:39.166 INFO:tasks.workunit.client.0.smithi156.stdout:9/491: dread - d2/d5/d16/d22/d36/d33/d23/f88 zero size 2024-07-16T09:55:39.167 INFO:tasks.workunit.client.0.smithi156.stdout:3/405: dread d8/da/d28/d43/f77 [0,4194304] 0 2024-07-16T09:55:39.168 INFO:tasks.workunit.client.1.smithi176.stdout:2/512: unlink dd/d16/f7d 0 2024-07-16T09:55:39.168 INFO:tasks.workunit.client.1.smithi176.stdout:8/491: creat d8/d14/d1e/d55/fa3 x:0 0 0 2024-07-16T09:55:39.168 INFO:tasks.workunit.client.1.smithi176.stdout:0/560: chown d5/d1c/d24/f30 360679 1 2024-07-16T09:55:39.169 INFO:tasks.workunit.client.0.smithi156.stdout:6/427: mknod d3/d13/d17/d4f/d5c/c90 0 2024-07-16T09:55:39.170 INFO:tasks.workunit.client.0.smithi156.stdout:0/418: mknod d0/d1a/d4d/d9/c7e 0 2024-07-16T09:55:39.172 INFO:tasks.workunit.client.1.smithi176.stdout:0/561: mkdir d5/d1c/db1 0 2024-07-16T09:55:39.173 INFO:tasks.workunit.client.1.smithi176.stdout:7/498: dwrite d4/dd/d49/f55 [0,4194304] 0 2024-07-16T09:55:39.174 INFO:tasks.workunit.client.0.smithi156.stdout:3/406: write d8/d14/f1c [1286451,101878] 0 2024-07-16T09:55:39.176 INFO:tasks.workunit.client.1.smithi176.stdout:8/492: creat d8/d23/d2c/d34/d62/fa4 x:0 0 0 2024-07-16T09:55:39.177 INFO:tasks.workunit.client.1.smithi176.stdout:9/580: dread d5/de/d30/fa2 [0,4194304] 0 2024-07-16T09:55:39.177 INFO:tasks.workunit.client.0.smithi156.stdout:6/428: write d3/d13/d17/d4f/d4d/d3b/f59 [823400,65897] 0 2024-07-16T09:55:39.177 INFO:tasks.workunit.client.1.smithi176.stdout:2/513: write dd/d16/d39/f84 [703800,97341] 0 2024-07-16T09:55:39.178 INFO:tasks.workunit.client.0.smithi156.stdout:9/492: rename d2/d5/d16/d22/f8f to d2/d5/d16/d22/d6b/f97 0 2024-07-16T09:55:39.179 INFO:tasks.workunit.client.0.smithi156.stdout:4/459: dwrite d4/d23/f78 [0,4194304] 0 2024-07-16T09:55:39.182 INFO:tasks.workunit.client.0.smithi156.stdout:5/508: dwrite d1b/d1f/d42/f5f [0,4194304] 0 2024-07-16T09:55:39.182 INFO:tasks.workunit.client.1.smithi176.stdout:4/482: dwrite da/dc/f58 [0,4194304] 0 2024-07-16T09:55:39.182 INFO:tasks.workunit.client.0.smithi156.stdout:6/429: chown d3/d13/d17/d4f/d1e/l27 21865611 1 2024-07-16T09:55:39.183 INFO:tasks.workunit.client.1.smithi176.stdout:7/499: rename d4/d1f/d22/d80/d17/c93 to d4/d1f/d22/ca2 0 2024-07-16T09:55:39.185 INFO:tasks.workunit.client.1.smithi176.stdout:0/562: dread d5/d1c/d5a/f75 [0,4194304] 0 2024-07-16T09:55:39.185 INFO:tasks.workunit.client.1.smithi176.stdout:9/581: symlink d5/de/da5/lc6 0 2024-07-16T09:55:39.185 INFO:tasks.workunit.client.0.smithi156.stdout:9/493: mkdir d2/d5/d16/d22/d36/d6a/d98 0 2024-07-16T09:55:39.186 INFO:tasks.workunit.client.0.smithi156.stdout:6/430: fsync d3/d13/d75/d19/f66 0 2024-07-16T09:55:39.187 INFO:tasks.workunit.client.0.smithi156.stdout:3/407: creat d8/da/d1d/f8d x:0 0 0 2024-07-16T09:55:39.188 INFO:tasks.workunit.client.0.smithi156.stdout:0/419: dread d0/d1a/d4d/d9/d27/f6a [0,4194304] 0 2024-07-16T09:55:39.190 INFO:tasks.workunit.client.1.smithi176.stdout:2/514: creat dd/d36/d82/fa9 x:0 0 0 2024-07-16T09:55:39.192 INFO:tasks.workunit.client.1.smithi176.stdout:2/515: read dd/d2e/f37 [322998,117354] 0 2024-07-16T09:55:39.192 INFO:tasks.workunit.client.0.smithi156.stdout:5/509: creat d1b/d1f/d45/faf x:0 0 0 2024-07-16T09:55:39.194 INFO:tasks.workunit.client.0.smithi156.stdout:3/408: unlink d8/da/l29 0 2024-07-16T09:55:39.195 INFO:tasks.workunit.client.0.smithi156.stdout:9/494: mknod d2/d5/d16/d22/d36/d6a/c99 0 2024-07-16T09:55:39.196 INFO:tasks.workunit.client.1.smithi176.stdout:8/493: mknod d8/d20/d2a/d53/d9a/ca5 0 2024-07-16T09:55:39.198 INFO:tasks.workunit.client.1.smithi176.stdout:7/500: unlink d4/d1f/d22/d51/f58 0 2024-07-16T09:55:39.200 INFO:tasks.workunit.client.0.smithi156.stdout:5/510: unlink d1b/d1f/d45/d52/d49/la5 0 2024-07-16T09:55:39.200 INFO:tasks.workunit.client.1.smithi176.stdout:7/501: write d4/dd/d26/f7b [110463,43649] 0 2024-07-16T09:55:39.200 INFO:tasks.workunit.client.1.smithi176.stdout:0/563: creat d5/d1c/d24/d26/fb2 x:0 0 0 2024-07-16T09:55:39.200 INFO:tasks.workunit.client.1.smithi176.stdout:8/494: read d8/d20/d2a/f75 [486346,92899] 0 2024-07-16T09:55:39.201 INFO:tasks.workunit.client.0.smithi156.stdout:6/431: getdents d3/d13/d75/d19 0 2024-07-16T09:55:39.203 INFO:tasks.workunit.client.1.smithi176.stdout:7/502: read d4/f2b [560965,124592] 0 2024-07-16T09:55:39.204 INFO:tasks.workunit.client.1.smithi176.stdout:9/582: rename d5/d1d/db3/d58 to d5/d1d/dc7 0 2024-07-16T09:55:39.204 INFO:tasks.workunit.client.0.smithi156.stdout:3/409: dwrite d8/da/d1d/d2c/f84 [0,4194304] 0 2024-07-16T09:55:39.205 INFO:tasks.workunit.client.1.smithi176.stdout:9/583: chown d5/de/d30 28 1 2024-07-16T09:55:39.205 INFO:tasks.workunit.client.0.smithi156.stdout:5/511: write d1b/d1f/d45/d9d/fab [442004,84031] 0 2024-07-16T09:55:39.207 INFO:tasks.workunit.client.0.smithi156.stdout:6/432: rmdir d3/d13/d17/d4f/d30/d45 39 2024-07-16T09:55:39.207 INFO:tasks.workunit.client.0.smithi156.stdout:6/433: stat d3/d13/d17/d56/l8d 0 2024-07-16T09:55:39.208 INFO:tasks.workunit.client.1.smithi176.stdout:0/564: unlink d5/d25/l37 0 2024-07-16T09:55:39.209 INFO:tasks.workunit.client.0.smithi156.stdout:4/460: dread d4/f79 [0,4194304] 0 2024-07-16T09:55:39.209 INFO:tasks.workunit.client.1.smithi176.stdout:9/584: symlink d5/de/da5/dc1/lc8 0 2024-07-16T09:55:39.210 INFO:tasks.workunit.client.0.smithi156.stdout:5/512: unlink d1b/d1f/d45/d52/d43/d85/d97/d37/c4c 0 2024-07-16T09:55:39.211 INFO:tasks.workunit.client.1.smithi176.stdout:0/565: symlink d5/d1c/lb3 0 2024-07-16T09:55:39.211 INFO:tasks.workunit.client.1.smithi176.stdout:8/495: dread d8/d14/d1e/f1f [0,4194304] 0 2024-07-16T09:55:39.212 INFO:tasks.workunit.client.1.smithi176.stdout:9/585: write d5/de/da5/f82 [4608623,9511] 0 2024-07-16T09:55:39.212 INFO:tasks.workunit.client.0.smithi156.stdout:6/434: creat d3/d13/d75/d19/d31/d35/f91 x:0 0 0 2024-07-16T09:55:39.214 INFO:tasks.workunit.client.0.smithi156.stdout:5/513: chown d1b/d1f/c3b 73583 1 2024-07-16T09:55:39.215 INFO:tasks.workunit.client.0.smithi156.stdout:3/410: creat d8/da/d1d/d2c/f8e x:0 0 0 2024-07-16T09:55:39.215 INFO:tasks.workunit.client.1.smithi176.stdout:0/566: mkdir d5/d1c/d24/d35/d50/d58/d63/db4 0 2024-07-16T09:55:39.216 INFO:tasks.workunit.client.1.smithi176.stdout:8/496: truncate d8/d14/d1e/d55/f8a 4691265 0 2024-07-16T09:55:39.216 INFO:tasks.workunit.client.0.smithi156.stdout:6/435: mkdir d3/d13/d17/d4f/d92 0 2024-07-16T09:55:39.217 INFO:tasks.workunit.client.0.smithi156.stdout:3/411: write d8/da/d1d/d2c/f8e [932906,55943] 0 2024-07-16T09:55:39.218 INFO:tasks.workunit.client.1.smithi176.stdout:8/497: rmdir d8/d20/d2a/d31 39 2024-07-16T09:55:39.219 INFO:tasks.workunit.client.0.smithi156.stdout:5/514: creat d1b/d1f/d45/d52/fb0 x:0 0 0 2024-07-16T09:55:39.222 INFO:tasks.workunit.client.1.smithi176.stdout:0/567: symlink d5/d44/lb5 0 2024-07-16T09:55:39.223 INFO:tasks.workunit.client.0.smithi156.stdout:3/412: mknod d8/c8f 0 2024-07-16T09:55:39.225 INFO:tasks.workunit.client.0.smithi156.stdout:0/420: truncate d0/d1a/d4d/d9/d27/d3d/d62/f67 1038404 0 2024-07-16T09:55:39.226 INFO:tasks.workunit.client.1.smithi176.stdout:4/483: dwrite da/d3d/d17/f35 [0,4194304] 0 2024-07-16T09:55:39.229 INFO:tasks.workunit.client.1.smithi176.stdout:9/586: dwrite d5/d43/f9e [0,4194304] 0 2024-07-16T09:55:39.229 INFO:tasks.workunit.client.1.smithi176.stdout:8/498: mknod d8/d20/ca6 0 2024-07-16T09:55:39.230 INFO:tasks.workunit.client.0.smithi156.stdout:3/413: dwrite d8/da/d1d/d2c/f8e [0,4194304] 0 2024-07-16T09:55:39.231 INFO:tasks.workunit.client.0.smithi156.stdout:0/421: dread d0/d1a/d4d/d9/f15 [0,4194304] 0 2024-07-16T09:55:39.232 INFO:tasks.workunit.client.0.smithi156.stdout:3/414: dread - d8/da/d1d/f83 zero size 2024-07-16T09:55:39.232 INFO:tasks.workunit.client.0.smithi156.stdout:6/436: rmdir d3/d13/d75/d19/d31/d35 39 2024-07-16T09:55:39.232 INFO:tasks.workunit.client.0.smithi156.stdout:0/422: readlink d0/d22/l31 0 2024-07-16T09:55:39.233 INFO:tasks.workunit.client.0.smithi156.stdout:0/423: chown d0/d22/d41/f4c 2 1 2024-07-16T09:55:39.233 INFO:tasks.workunit.client.0.smithi156.stdout:3/415: dread - d8/da/d28/d43/d63/f73 zero size 2024-07-16T09:55:39.235 INFO:tasks.workunit.client.1.smithi176.stdout:9/587: read - d5/d1d/d67/fab zero size 2024-07-16T09:55:39.236 INFO:tasks.workunit.client.0.smithi156.stdout:5/515: truncate d1b/d1f/d45/d52/d43/d7c/fa2 202760 0 2024-07-16T09:55:39.238 INFO:tasks.workunit.client.1.smithi176.stdout:0/568: dwrite d5/f23 [0,4194304] 0 2024-07-16T09:55:39.238 INFO:tasks.workunit.client.0.smithi156.stdout:0/424: fsync d0/d22/d41/f47 0 2024-07-16T09:55:39.240 INFO:tasks.workunit.client.0.smithi156.stdout:0/425: read - d0/d22/f7a zero size 2024-07-16T09:55:39.241 INFO:tasks.workunit.client.0.smithi156.stdout:3/416: mkdir d8/da/d1d/d90 0 2024-07-16T09:55:39.242 INFO:tasks.workunit.client.0.smithi156.stdout:0/426: fsync d0/d1a/d4d/d9/f26 0 2024-07-16T09:55:39.243 INFO:tasks.workunit.client.0.smithi156.stdout:6/437: link d3/d13/d17/d4f/d30/f67 d3/d13/d17/d4f/d4d/d3b/f93 0 2024-07-16T09:55:39.248 INFO:tasks.workunit.client.0.smithi156.stdout:3/417: mknod d8/da/d28/d43/c91 0 2024-07-16T09:55:39.249 INFO:tasks.workunit.client.1.smithi176.stdout:4/484: rename da/dc/f55 to da/dc/f90 0 2024-07-16T09:55:39.249 INFO:tasks.workunit.client.0.smithi156.stdout:3/418: write d8/da/d13/f17 [6198914,8116] 0 2024-07-16T09:55:39.251 INFO:tasks.workunit.client.1.smithi176.stdout:0/569: getdents d5/d39/da3 0 2024-07-16T09:55:39.252 INFO:tasks.workunit.client.0.smithi156.stdout:3/419: fdatasync d8/da/d13/f20 0 2024-07-16T09:55:39.253 INFO:tasks.workunit.client.1.smithi176.stdout:9/588: write d5/de/da5/f4e [1339813,3451] 0 2024-07-16T09:55:39.254 INFO:tasks.workunit.client.1.smithi176.stdout:8/499: write d8/d14/d1e/d33/f5b [171283,89330] 0 2024-07-16T09:55:39.255 INFO:tasks.workunit.client.1.smithi176.stdout:0/570: rename f4 to d5/d1c/d24/d35/d50/d58/fb6 0 2024-07-16T09:55:39.255 INFO:tasks.workunit.client.1.smithi176.stdout:9/589: stat d5/de/d21/f39 0 2024-07-16T09:55:39.256 INFO:tasks.workunit.client.0.smithi156.stdout:0/427: getdents d0/d1a/d4d/d9/d27/d3d/d62 0 2024-07-16T09:55:39.257 INFO:tasks.workunit.client.1.smithi176.stdout:9/590: rmdir d5/d1d/d67 39 2024-07-16T09:55:39.258 INFO:tasks.workunit.client.0.smithi156.stdout:3/420: write d8/da/f32 [4606077,95015] 0 2024-07-16T09:55:39.258 INFO:tasks.workunit.client.0.smithi156.stdout:0/428: write d0/d22/d41/f4c [78344,32085] 0 2024-07-16T09:55:39.258 INFO:tasks.workunit.client.1.smithi176.stdout:9/591: read d5/de/f8b [2328343,32382] 0 2024-07-16T09:55:39.258 INFO:tasks.workunit.client.1.smithi176.stdout:0/571: symlink d5/d44/lb7 0 2024-07-16T09:55:39.259 INFO:tasks.workunit.client.0.smithi156.stdout:3/421: dread - d8/da/d1d/f8d zero size 2024-07-16T09:55:39.261 INFO:tasks.workunit.client.0.smithi156.stdout:3/422: fdatasync d8/da/fc 0 2024-07-16T09:55:39.261 INFO:tasks.workunit.client.0.smithi156.stdout:0/429: symlink d0/d1a/d4d/d9/d27/l7f 0 2024-07-16T09:55:39.262 INFO:tasks.workunit.client.0.smithi156.stdout:3/423: chown d8/da/d1d/l2f 3063617 1 2024-07-16T09:55:39.267 INFO:tasks.workunit.client.1.smithi176.stdout:4/485: dread da/d14/d40/d5d/d73/f3c [0,4194304] 0 2024-07-16T09:55:39.267 INFO:tasks.workunit.client.1.smithi176.stdout:9/592: dwrite d5/de/da5/f5a [0,4194304] 0 2024-07-16T09:55:39.267 INFO:tasks.workunit.client.0.smithi156.stdout:3/424: mknod d8/da/d1d/d2c/c92 0 2024-07-16T09:55:39.269 INFO:tasks.workunit.client.0.smithi156.stdout:3/425: readlink d8/da/d28/l51 0 2024-07-16T09:55:39.273 INFO:tasks.workunit.client.1.smithi176.stdout:9/593: dread d5/d1d/d20/f81 [0,4194304] 0 2024-07-16T09:55:39.273 INFO:tasks.workunit.client.0.smithi156.stdout:3/426: symlink d8/da/d28/d43/d71/l93 0 2024-07-16T09:55:39.275 INFO:tasks.workunit.client.1.smithi176.stdout:9/594: truncate d5/d1d/db3/f7c 4466421 0 2024-07-16T09:55:39.275 INFO:tasks.workunit.client.0.smithi156.stdout:3/427: write d8/da/d28/f6e [1748976,36962] 0 2024-07-16T09:55:39.277 INFO:tasks.workunit.client.0.smithi156.stdout:5/516: write d1b/d1f/d45/d52/d43/d85/d97/f21 [351799,31204] 0 2024-07-16T09:55:39.278 INFO:tasks.workunit.client.1.smithi176.stdout:9/595: write d5/d1d/d20/d8d/f91 [213526,126257] 0 2024-07-16T09:55:39.278 INFO:tasks.workunit.client.0.smithi156.stdout:3/428: chown d8/da/d1d/d2c/d33/f65 69423 1 2024-07-16T09:55:39.279 INFO:tasks.workunit.client.0.smithi156.stdout:5/517: chown d1b/d1f/d45/d52/d43/f5b 90000070 1 2024-07-16T09:55:39.282 INFO:tasks.workunit.client.0.smithi156.stdout:5/518: unlink d1b/d1f/d45/d52/d43/f63 0 2024-07-16T09:55:39.285 INFO:tasks.workunit.client.0.smithi156.stdout:5/519: mknod d1b/d1f/d8a/cb1 0 2024-07-16T09:55:39.286 INFO:tasks.workunit.client.0.smithi156.stdout:6/438: dread d3/f25 [4194304,4194304] 0 2024-07-16T09:55:39.287 INFO:tasks.workunit.client.0.smithi156.stdout:3/429: link d8/d14/f1c d8/da/f94 0 2024-07-16T09:55:39.292 INFO:tasks.workunit.client.0.smithi156.stdout:0/430: dwrite d0/d1a/d4d/f19 [0,4194304] 0 2024-07-16T09:55:39.294 INFO:tasks.workunit.client.0.smithi156.stdout:3/430: mknod d8/da/d28/d43/d63/c95 0 2024-07-16T09:55:39.295 INFO:tasks.workunit.client.0.smithi156.stdout:0/431: readlink d0/d1a/d4d/d9/d27/l4b 0 2024-07-16T09:55:39.299 INFO:tasks.workunit.client.0.smithi156.stdout:6/439: rename d3/d13/d75 to d3/d13/d17/d4f/d94 0 2024-07-16T09:55:39.301 INFO:tasks.workunit.client.1.smithi176.stdout:4/486: symlink da/d41/d5b/l91 0 2024-07-16T09:55:39.302 INFO:tasks.workunit.client.1.smithi176.stdout:4/487: chown da/d14/c2d 0 1 2024-07-16T09:55:39.303 INFO:tasks.workunit.client.1.smithi176.stdout:4/488: chown da/d3d/d17/f31 15898 1 2024-07-16T09:55:39.304 INFO:tasks.workunit.client.0.smithi156.stdout:5/520: dwrite d1b/d1f/d45/d52/d49/d4d/f80 [0,4194304] 0 2024-07-16T09:55:39.304 INFO:tasks.workunit.client.1.smithi176.stdout:4/489: chown da/d3d/d17/l5c 38132 1 2024-07-16T09:55:39.304 INFO:tasks.workunit.client.1.smithi176.stdout:8/500: write d8/d20/d2a/d31/f69 [135378,58250] 0 2024-07-16T09:55:39.305 INFO:tasks.workunit.client.0.smithi156.stdout:0/432: mknod d0/d1a/d4d/d9/d27/d2c/d6d/c80 0 2024-07-16T09:55:39.307 INFO:tasks.workunit.client.0.smithi156.stdout:5/521: chown d1b/d1f/d45/d52/d43/c77 1 1 2024-07-16T09:55:39.307 INFO:tasks.workunit.client.0.smithi156.stdout:0/433: rmdir d0/d1a/d4d/d25 39 2024-07-16T09:55:39.309 INFO:tasks.workunit.client.1.smithi176.stdout:9/596: rename d5/de/d21/l66 to d5/d1d/d77/d97/lc9 0 2024-07-16T09:55:39.311 INFO:tasks.workunit.client.1.smithi176.stdout:0/572: getdents d5/d1c/d24/d26 0 2024-07-16T09:55:39.311 INFO:tasks.workunit.client.1.smithi176.stdout:0/573: readlink d5/d44/lb7 0 2024-07-16T09:55:39.313 INFO:tasks.workunit.client.1.smithi176.stdout:4/490: creat da/d14/d40/f92 x:0 0 0 2024-07-16T09:55:39.314 INFO:tasks.workunit.client.1.smithi176.stdout:9/597: creat d5/d1d/dc7/fca x:0 0 0 2024-07-16T09:55:39.315 INFO:tasks.workunit.client.0.smithi156.stdout:6/440: getdents d3/d13/d17/d4f/d30/d45/d5f 0 2024-07-16T09:55:39.316 INFO:tasks.workunit.client.0.smithi156.stdout:0/434: creat d0/d1a/d4d/d9/d27/f81 x:0 0 0 2024-07-16T09:55:39.318 INFO:tasks.workunit.client.1.smithi176.stdout:9/598: write d5/d1d/d20/d8d/f75 [1092874,51187] 0 2024-07-16T09:55:39.319 INFO:tasks.workunit.client.0.smithi156.stdout:3/431: dwrite d8/da/d13/f20 [4194304,4194304] 0 2024-07-16T09:55:39.321 INFO:tasks.workunit.client.1.smithi176.stdout:4/491: dwrite da/dc/f50 [0,4194304] 0 2024-07-16T09:55:39.323 INFO:tasks.workunit.client.1.smithi176.stdout:4/492: write da/d3d/d17/d48/f80 [73333,24094] 0 2024-07-16T09:55:39.325 INFO:tasks.workunit.client.0.smithi156.stdout:5/522: dwrite d1b/d1f/d45/d52/d49/d4d/f9b [0,4194304] 0 2024-07-16T09:55:39.329 INFO:tasks.workunit.client.0.smithi156.stdout:5/523: write d1b/d1f/d45/d52/d43/d85/d97/d37/f5a [1956239,126430] 0 2024-07-16T09:55:39.329 INFO:tasks.workunit.client.1.smithi176.stdout:9/599: creat d5/d1d/db3/fcb x:0 0 0 2024-07-16T09:55:39.330 INFO:tasks.workunit.client.1.smithi176.stdout:4/493: dread da/d14/d4a/f4e [0,4194304] 0 2024-07-16T09:55:39.334 INFO:tasks.workunit.client.0.smithi156.stdout:5/524: mkdir d1b/d1f/d42/d6d/d9f/db2 0 2024-07-16T09:55:39.338 INFO:tasks.workunit.client.1.smithi176.stdout:4/494: creat da/d3d/d17/d54/d78/d7a/f93 x:0 0 0 2024-07-16T09:55:39.338 INFO:tasks.workunit.client.1.smithi176.stdout:4/495: dread - da/f47 zero size 2024-07-16T09:55:39.338 INFO:tasks.workunit.client.1.smithi176.stdout:0/574: dread d5/d1c/d24/d35/f4c [0,4194304] 0 2024-07-16T09:55:39.339 INFO:tasks.workunit.client.0.smithi156.stdout:5/525: symlink d1b/d1f/d45/d52/d43/d85/d97/d37/d88/lb3 0 2024-07-16T09:55:39.340 INFO:tasks.workunit.client.0.smithi156.stdout:0/435: write d0/d1a/d4d/d9/d27/d3d/d62/f63 [1435406,97177] 0 2024-07-16T09:55:39.341 INFO:tasks.workunit.client.1.smithi176.stdout:0/575: write d5/d1c/f1f [5871,34169] 0 2024-07-16T09:55:39.341 INFO:tasks.workunit.client.0.smithi156.stdout:0/436: stat d0/d1a/d4d/d9/f56 0 2024-07-16T09:55:39.342 INFO:tasks.workunit.client.0.smithi156.stdout:0/437: chown d0 15 1 2024-07-16T09:55:39.342 INFO:tasks.workunit.client.0.smithi156.stdout:5/526: creat d1b/d1f/d45/d52/d2d/fb4 x:0 0 0 2024-07-16T09:55:39.342 INFO:tasks.workunit.client.1.smithi176.stdout:4/496: mknod da/d3d/d17/d54/d5e/d69/d86/c94 0 2024-07-16T09:55:39.343 INFO:tasks.workunit.client.0.smithi156.stdout:5/527: stat d1b/d1f/d42/d6d/l92 0 2024-07-16T09:55:39.345 INFO:tasks.workunit.client.0.smithi156.stdout:5/528: symlink d1b/d1f/d45/d7a/lb5 0 2024-07-16T09:55:39.347 INFO:tasks.workunit.client.0.smithi156.stdout:5/529: symlink d1b/d1f/d45/d9d/lb6 0 2024-07-16T09:55:39.350 INFO:tasks.workunit.client.0.smithi156.stdout:5/530: dread d1b/d1f/d45/d52/d43/d85/f75 [0,4194304] 0 2024-07-16T09:55:39.351 INFO:tasks.workunit.client.0.smithi156.stdout:6/441: truncate d3/d13/d17/d4f/d94/fa 3315074 0 2024-07-16T09:55:39.352 INFO:tasks.workunit.client.0.smithi156.stdout:6/442: readlink d3/d13/d17/d4f/d1e/l4c 0 2024-07-16T09:55:39.352 INFO:tasks.workunit.client.1.smithi176.stdout:0/576: dwrite d5/f2a [0,4194304] 0 2024-07-16T09:55:39.354 INFO:tasks.workunit.client.1.smithi176.stdout:0/577: chown d5/d25/d82/f9a 410600 1 2024-07-16T09:55:39.359 INFO:tasks.workunit.client.1.smithi176.stdout:0/578: creat d5/d1c/fb8 x:0 0 0 2024-07-16T09:55:39.362 INFO:tasks.workunit.client.1.smithi176.stdout:0/579: creat d5/d1c/d24/d35/d50/d58/d63/fb9 x:0 0 0 2024-07-16T09:55:39.364 INFO:tasks.workunit.client.1.smithi176.stdout:0/580: chown d5/d2c/d6c/c90 174128281 1 2024-07-16T09:55:39.366 INFO:tasks.workunit.client.1.smithi176.stdout:0/581: creat d5/d2c/fba x:0 0 0 2024-07-16T09:55:39.369 INFO:tasks.workunit.client.1.smithi176.stdout:0/582: mknod d5/d1c/d24/d35/d50/d58/cbb 0 2024-07-16T09:55:39.379 INFO:tasks.workunit.client.0.smithi156.stdout:3/432: dread d8/da/d1d/d2c/d33/f65 [0,4194304] 0 2024-07-16T09:55:39.381 INFO:tasks.workunit.client.1.smithi176.stdout:4/497: dread da/fb [0,4194304] 0 2024-07-16T09:55:39.382 INFO:tasks.workunit.client.0.smithi156.stdout:0/438: dread d0/d1a/d4d/d9/d27/d3d/f37 [0,4194304] 0 2024-07-16T09:55:39.382 INFO:tasks.workunit.client.0.smithi156.stdout:3/433: symlink d8/da/d13/l96 0 2024-07-16T09:55:39.383 INFO:tasks.workunit.client.0.smithi156.stdout:6/443: write d3/d13/d17/d4f/f34 [2412781,83186] 0 2024-07-16T09:55:39.383 INFO:tasks.workunit.client.0.smithi156.stdout:5/531: write d1b/d1f/d45/f50 [1797169,112289] 0 2024-07-16T09:55:39.387 INFO:tasks.workunit.client.1.smithi176.stdout:8/501: rename d8/d20/d2a/d31/f69 to d8/d14/d1e/d33/d9d/fa7 0 2024-07-16T09:55:39.388 INFO:tasks.workunit.client.1.smithi176.stdout:4/498: dwrite da/d3d/d17/f2a [0,4194304] 0 2024-07-16T09:55:39.388 INFO:tasks.workunit.client.0.smithi156.stdout:6/444: creat d3/d13/d17/d4f/d1e/f95 x:0 0 0 2024-07-16T09:55:39.389 INFO:tasks.workunit.client.1.smithi176.stdout:4/499: chown da/d3d/d17/d54/d5e/d69/f6d 41070 1 2024-07-16T09:55:39.391 INFO:tasks.workunit.client.1.smithi176.stdout:9/600: rename d5/de/d30/f6b to d5/d1d/d20/fcc 0 2024-07-16T09:55:39.391 INFO:tasks.workunit.client.0.smithi156.stdout:5/532: dread d1b/d1f/d42/f5f [0,4194304] 0 2024-07-16T09:55:39.391 INFO:tasks.workunit.client.1.smithi176.stdout:4/500: chown da/dc/d4b/d62/c72 87991 1 2024-07-16T09:55:39.393 INFO:tasks.workunit.client.0.smithi156.stdout:5/533: chown d1b/d1f/d45/d52/d43/d85/d97/d37/d88/d61 29307 1 2024-07-16T09:55:39.394 INFO:tasks.workunit.client.0.smithi156.stdout:6/445: rename d3/d13/d17/d4f/f14 to d3/d13/d17/d4f/d4d/d3b/f96 0 2024-07-16T09:55:39.395 INFO:tasks.workunit.client.1.smithi176.stdout:4/501: read da/d3d/d17/d54/d5e/d69/f6d [608801,106627] 0 2024-07-16T09:55:39.395 INFO:tasks.workunit.client.1.smithi176.stdout:9/601: creat d5/d1d/d77/d98/fcd x:0 0 0 2024-07-16T09:55:39.396 INFO:tasks.workunit.client.0.smithi156.stdout:6/446: fdatasync d3/d13/d17/d4f/d30/d45/d5f/f8b 0 2024-07-16T09:55:39.396 INFO:tasks.workunit.client.0.smithi156.stdout:5/534: creat d1b/d1f/d45/d52/d43/d85/d97/d37/fb7 x:0 0 0 2024-07-16T09:55:39.397 INFO:tasks.workunit.client.1.smithi176.stdout:0/583: rename d5/d25/l5d to d5/d1c/d24/lbc 0 2024-07-16T09:55:39.397 INFO:tasks.workunit.client.1.smithi176.stdout:9/602: write d5/d1d/d20/f7d [391301,89301] 0 2024-07-16T09:55:39.397 INFO:tasks.workunit.client.0.smithi156.stdout:5/535: readlink d1b/d1f/d42/l83 0 2024-07-16T09:55:39.401 INFO:tasks.workunit.client.0.smithi156.stdout:5/536: mknod d1b/cb8 0 2024-07-16T09:55:39.402 INFO:tasks.workunit.client.1.smithi176.stdout:4/502: creat da/d14/f95 x:0 0 0 2024-07-16T09:55:39.405 INFO:tasks.workunit.client.0.smithi156.stdout:3/434: dwrite d8/da/d1d/d2c/d33/f88 [0,4194304] 0 2024-07-16T09:55:39.405 INFO:tasks.workunit.client.0.smithi156.stdout:5/537: mkdir d1b/d99/dae/db9 0 2024-07-16T09:55:39.406 INFO:tasks.workunit.client.1.smithi176.stdout:9/603: dwrite d5/de/d30/f9a [0,4194304] 0 2024-07-16T09:55:39.407 INFO:tasks.workunit.client.0.smithi156.stdout:3/435: write d8/da/d21/d80/f82 [4142256,27567] 0 2024-07-16T09:55:39.408 INFO:tasks.workunit.client.1.smithi176.stdout:4/503: rename da/d3d/d17/d54/d5e to da/dc/d10/d96 0 2024-07-16T09:55:39.408 INFO:tasks.workunit.client.1.smithi176.stdout:9/604: truncate d5/d1d/d20/d78/f8f 4920409 0 2024-07-16T09:55:39.408 INFO:tasks.workunit.client.0.smithi156.stdout:6/447: write d3/d13/d17/d4f/d94/fa [3954331,124112] 0 2024-07-16T09:55:39.413 INFO:tasks.workunit.client.1.smithi176.stdout:9/605: mknod d5/de/da5/dbf/cce 0 2024-07-16T09:55:39.414 INFO:tasks.workunit.client.0.smithi156.stdout:5/538: link d1b/d1f/d45/d52/d2d/fb4 d1b/d99/dae/fba 0 2024-07-16T09:55:39.414 INFO:tasks.workunit.client.1.smithi176.stdout:4/504: creat da/dc/d4b/f97 x:0 0 0 2024-07-16T09:55:39.415 INFO:tasks.workunit.client.1.smithi176.stdout:0/584: getdents d5/d1c/d24/d35/d50/d58 0 2024-07-16T09:55:39.417 INFO:tasks.workunit.client.0.smithi156.stdout:3/436: dwrite d8/da/d28/f6e [0,4194304] 0 2024-07-16T09:55:39.417 INFO:tasks.workunit.client.0.smithi156.stdout:6/448: dwrite d3/f1c [0,4194304] 0 2024-07-16T09:55:39.420 INFO:tasks.workunit.client.0.smithi156.stdout:3/437: read d8/da/f3d [2289965,77972] 0 2024-07-16T09:55:39.421 INFO:tasks.workunit.client.0.smithi156.stdout:0/439: dread d0/d1a/d4d/d9/fe [0,4194304] 0 2024-07-16T09:55:39.421 INFO:tasks.workunit.client.1.smithi176.stdout:4/505: mknod da/d41/d5b/c98 0 2024-07-16T09:55:39.421 INFO:tasks.workunit.client.1.smithi176.stdout:8/502: dread d8/d14/f27 [0,4194304] 0 2024-07-16T09:55:39.421 INFO:tasks.workunit.client.1.smithi176.stdout:0/585: mkdir d5/d1c/d24/d26/dbd 0 2024-07-16T09:55:39.422 INFO:tasks.workunit.client.1.smithi176.stdout:9/606: mknod d5/de/da5/dc1/ccf 0 2024-07-16T09:55:39.424 INFO:tasks.workunit.client.0.smithi156.stdout:5/539: rename d1b/d1f/d45/d7a/d98/ca3 to d1b/d1f/d45/d52/d43/d85/d97/d37/cbb 0 2024-07-16T09:55:39.424 INFO:tasks.workunit.client.1.smithi176.stdout:8/503: mkdir d8/d20/d2a/d31/d46/d6a/da8 0 2024-07-16T09:55:39.426 INFO:tasks.workunit.client.1.smithi176.stdout:9/607: mkdir d5/d1d/d20/d78/dd0 0 2024-07-16T09:55:39.426 INFO:tasks.workunit.client.0.smithi156.stdout:3/438: symlink d8/da/d1d/d2c/d33/l97 0 2024-07-16T09:55:39.426 INFO:tasks.workunit.client.0.smithi156.stdout:5/540: read d1b/d1f/d45/d52/d43/f5b [4131357,80166] 0 2024-07-16T09:55:39.428 INFO:tasks.workunit.client.1.smithi176.stdout:4/506: getdents da/d41 0 2024-07-16T09:55:39.429 INFO:tasks.workunit.client.0.smithi156.stdout:0/440: mknod d0/d1a/d4d/d9/d27/d3d/d62/c82 0 2024-07-16T09:55:39.429 INFO:tasks.workunit.client.1.smithi176.stdout:4/507: read - da/dc/d10/d28/f79 zero size 2024-07-16T09:55:39.430 INFO:tasks.workunit.client.0.smithi156.stdout:0/441: read - d0/d1a/d4d/d9/f56 zero size 2024-07-16T09:55:39.432 INFO:tasks.workunit.client.1.smithi176.stdout:8/504: mkdir d8/d20/d2a/d31/d2f/da9 0 2024-07-16T09:55:39.434 INFO:tasks.workunit.client.1.smithi176.stdout:4/508: creat da/d41/f99 x:0 0 0 2024-07-16T09:55:39.435 INFO:tasks.workunit.client.0.smithi156.stdout:5/541: dwrite d1b/d1f/d45/d7f/d87/fa7 [0,4194304] 0 2024-07-16T09:55:39.435 INFO:tasks.workunit.client.0.smithi156.stdout:0/442: mkdir d0/d1a/d76/d83 0 2024-07-16T09:55:39.436 INFO:tasks.workunit.client.1.smithi176.stdout:4/509: truncate da/f47 166569 0 2024-07-16T09:55:39.436 INFO:tasks.workunit.client.1.smithi176.stdout:9/608: rename d5/de/d21/f5f to d5/d1d/db3/fd1 0 2024-07-16T09:55:39.436 INFO:tasks.workunit.client.1.smithi176.stdout:8/505: write d8/d20/d2a/f75 [160918,110766] 0 2024-07-16T09:55:39.437 INFO:tasks.workunit.client.1.smithi176.stdout:8/506: readlink d8/l30 0 2024-07-16T09:55:39.439 INFO:tasks.workunit.client.0.smithi156.stdout:0/443: chown d0/d1a/d4d/c13 1958355237 1 2024-07-16T09:55:39.441 INFO:tasks.workunit.client.1.smithi176.stdout:8/507: creat d8/d20/d2a/d61/d90/faa x:0 0 0 2024-07-16T09:55:39.442 INFO:tasks.workunit.client.1.smithi176.stdout:4/510: symlink da/d3d/d17/d54/d78/d7a/l9a 0 2024-07-16T09:55:39.445 INFO:tasks.workunit.client.0.smithi156.stdout:5/542: dwrite d1b/f34 [0,4194304] 0 2024-07-16T09:55:39.445 INFO:tasks.workunit.client.0.smithi156.stdout:0/444: mknod d0/d1a/d4d/d9/d27/d2c/d6d/c84 0 2024-07-16T09:55:39.446 INFO:tasks.workunit.client.1.smithi176.stdout:9/609: rename d5/d1d/d20/d8d to d5/d1d/dd2 0 2024-07-16T09:55:39.446 INFO:tasks.workunit.client.1.smithi176.stdout:9/610: chown d5/de 1219680 1 2024-07-16T09:55:39.447 INFO:tasks.workunit.client.1.smithi176.stdout:4/511: mkdir da/dc/d10/d96/d9b 0 2024-07-16T09:55:39.449 INFO:tasks.workunit.client.0.smithi156.stdout:0/445: chown d0/d1a/d4d/d9/d27/d3d/d62/c82 23274189 1 2024-07-16T09:55:39.451 INFO:tasks.workunit.client.1.smithi176.stdout:4/512: symlink da/d3d/d17/d54/l9c 0 2024-07-16T09:55:39.456 INFO:tasks.workunit.client.0.smithi156.stdout:3/439: write d8/d14/f1c [2746084,31274] 0 2024-07-16T09:55:39.459 INFO:tasks.workunit.client.0.smithi156.stdout:0/446: dwrite d0/d1a/f46 [0,4194304] 0 2024-07-16T09:55:39.461 INFO:tasks.workunit.client.0.smithi156.stdout:6/449: dwrite d3/d13/d17/f2b [0,4194304] 0 2024-07-16T09:55:39.462 INFO:tasks.workunit.client.1.smithi176.stdout:0/586: dread d5/fb [0,4194304] 0 2024-07-16T09:55:39.464 INFO:tasks.workunit.client.0.smithi156.stdout:6/450: read - d3/d13/d17/d56/f60 zero size 2024-07-16T09:55:39.470 INFO:tasks.workunit.client.0.smithi156.stdout:6/451: write d3/d13/d17/d4f/d94/d19/d2e/f3e [1165872,88224] 0 2024-07-16T09:55:39.470 INFO:tasks.workunit.client.0.smithi156.stdout:0/447: mknod d0/d1a/d4d/c85 0 2024-07-16T09:55:39.471 INFO:tasks.workunit.client.1.smithi176.stdout:0/587: read d5/d1c/d24/d35/d50/f7f [54197,76152] 0 2024-07-16T09:55:39.471 INFO:tasks.workunit.client.0.smithi156.stdout:6/452: read d3/d13/d17/d4f/d94/fa [803161,57150] 0 2024-07-16T09:55:39.473 INFO:tasks.workunit.client.0.smithi156.stdout:6/453: dread - d3/d13/d17/d4f/d30/d45/d5f/f8b zero size 2024-07-16T09:55:39.474 INFO:tasks.workunit.client.1.smithi176.stdout:4/513: dread da/dc/fe [0,4194304] 0 2024-07-16T09:55:39.475 INFO:tasks.workunit.client.0.smithi156.stdout:3/440: getdents d8/da/d28/d43 0 2024-07-16T09:55:39.475 INFO:tasks.workunit.client.1.smithi176.stdout:8/508: truncate d8/d14/d1e/d33/f5b 72167 0 2024-07-16T09:55:39.476 INFO:tasks.workunit.client.1.smithi176.stdout:9/611: write d5/d43/f4f [972584,127776] 0 2024-07-16T09:55:39.477 INFO:tasks.workunit.client.0.smithi156.stdout:6/454: creat d3/d13/d17/d4f/d94/d19/d2e/f97 x:0 0 0 2024-07-16T09:55:39.479 INFO:tasks.workunit.client.0.smithi156.stdout:0/448: mkdir d0/d1a/d4d/d6e/d86 0 2024-07-16T09:55:39.479 INFO:tasks.workunit.client.1.smithi176.stdout:0/588: mkdir d5/d39/da3/dbe 0 2024-07-16T09:55:39.480 INFO:tasks.workunit.client.1.smithi176.stdout:8/509: mkdir d8/d23/d2c/d34/d62/dab 0 2024-07-16T09:55:39.481 INFO:tasks.workunit.client.0.smithi156.stdout:5/543: dread d1b/d1f/d42/f47 [0,4194304] 0 2024-07-16T09:55:39.481 INFO:tasks.workunit.client.0.smithi156.stdout:3/441: read d8/da/d13/f5c [379286,92476] 0 2024-07-16T09:55:39.481 INFO:tasks.workunit.client.1.smithi176.stdout:8/510: write d8/d14/d1e/f60 [2031735,45986] 0 2024-07-16T09:55:39.482 INFO:tasks.workunit.client.0.smithi156.stdout:3/442: stat d8/cf 0 2024-07-16T09:55:39.482 INFO:tasks.workunit.client.0.smithi156.stdout:0/449: chown d0/d1a/d4d/d25/f2a 14503 1 2024-07-16T09:55:39.482 INFO:tasks.workunit.client.1.smithi176.stdout:9/612: mkdir d5/d1d/db0/dd3 0 2024-07-16T09:55:39.483 INFO:tasks.workunit.client.1.smithi176.stdout:4/514: creat da/dc/f9d x:0 0 0 2024-07-16T09:55:39.484 INFO:tasks.workunit.client.0.smithi156.stdout:3/443: write d8/da/d28/d43/d63/f73 [655864,112115] 0 2024-07-16T09:55:39.484 INFO:tasks.workunit.client.1.smithi176.stdout:0/589: creat d5/d1c/db1/fbf x:0 0 0 2024-07-16T09:55:39.484 INFO:tasks.workunit.client.1.smithi176.stdout:4/515: write da/d14/d40/d5d/d73/f3c [341904,58448] 0 2024-07-16T09:55:39.485 INFO:tasks.workunit.client.1.smithi176.stdout:0/590: fsync d5/d1c/f1f 0 2024-07-16T09:55:39.486 INFO:tasks.workunit.client.1.smithi176.stdout:8/511: read f5 [1850375,85257] 0 2024-07-16T09:55:39.487 INFO:tasks.workunit.client.0.smithi156.stdout:0/450: rename d0/d1a/d4d/f59 to d0/d1a/d76/f87 0 2024-07-16T09:55:39.487 INFO:tasks.workunit.client.1.smithi176.stdout:9/613: read - d5/d52/fb2 zero size 2024-07-16T09:55:39.487 INFO:tasks.workunit.client.1.smithi176.stdout:4/516: mknod da/dc/d10/d28/c9e 0 2024-07-16T09:55:39.488 INFO:tasks.workunit.client.0.smithi156.stdout:5/544: mkdir d1b/d1f/d45/d52/d43/d85/d97/d37/d88/dbc 0 2024-07-16T09:55:39.489 INFO:tasks.workunit.client.1.smithi176.stdout:0/591: creat d5/d25/da0/fc0 x:0 0 0 2024-07-16T09:55:39.490 INFO:tasks.workunit.client.0.smithi156.stdout:5/545: truncate d1b/d1f/d45/d52/d43/f8d 147519 0 2024-07-16T09:55:39.490 INFO:tasks.workunit.client.0.smithi156.stdout:0/451: unlink d0/d1a/d4d/f19 0 2024-07-16T09:55:39.490 INFO:tasks.workunit.client.1.smithi176.stdout:3/408: sync 2024-07-16T09:55:39.491 INFO:tasks.workunit.client.0.smithi156.stdout:3/444: symlink d8/da/d21/d80/l98 0 2024-07-16T09:55:39.492 INFO:tasks.workunit.client.1.smithi176.stdout:0/592: mknod d5/d25/da0/cc1 0 2024-07-16T09:55:39.493 INFO:tasks.workunit.client.0.smithi156.stdout:5/546: rmdir d1b/d1f/d45/d9d 39 2024-07-16T09:55:39.493 INFO:tasks.workunit.client.1.smithi176.stdout:3/409: write d3/da/d56/f5b [2164682,13376] 0 2024-07-16T09:55:39.494 INFO:tasks.workunit.client.0.smithi156.stdout:0/452: fsync d0/d1a/d4d/d9/f7b 0 2024-07-16T09:55:39.495 INFO:tasks.workunit.client.1.smithi176.stdout:4/517: dread da/d14/f29 [0,4194304] 0 2024-07-16T09:55:39.495 INFO:tasks.workunit.client.1.smithi176.stdout:9/614: dread d5/de/f15 [0,4194304] 0 2024-07-16T09:55:39.495 INFO:tasks.workunit.client.1.smithi176.stdout:0/593: unlink f3 0 2024-07-16T09:55:39.496 INFO:tasks.workunit.client.0.smithi156.stdout:5/547: rename l17 to d1b/d1f/d54/lbd 0 2024-07-16T09:55:39.500 INFO:tasks.workunit.client.0.smithi156.stdout:3/445: dread d8/da/d1d/f53 [0,4194304] 0 2024-07-16T09:55:39.501 INFO:tasks.workunit.client.0.smithi156.stdout:0/453: creat d0/d1a/d4d/d9/f88 x:0 0 0 2024-07-16T09:55:39.502 INFO:tasks.workunit.client.0.smithi156.stdout:3/446: write d8/da/d28/d40/f56 [917622,117339] 0 2024-07-16T09:55:39.506 INFO:tasks.workunit.client.0.smithi156.stdout:3/447: mkdir d8/da/d21/d80/d99 0 2024-07-16T09:55:39.508 INFO:tasks.workunit.client.0.smithi156.stdout:3/448: truncate d8/da/d28/d43/f5b 530992 0 2024-07-16T09:55:39.510 INFO:tasks.workunit.client.1.smithi176.stdout:3/410: dwrite d3/da/d33/f53 [0,4194304] 0 2024-07-16T09:55:39.512 INFO:tasks.workunit.client.0.smithi156.stdout:3/449: rename d8/da/d28/d43/d71/l93 to d8/da/d28/d43/d63/l9a 0 2024-07-16T09:55:39.512 INFO:tasks.workunit.client.0.smithi156.stdout:5/548: write d1b/d1f/d45/d52/d43/d7c/fa2 [1126221,762] 0 2024-07-16T09:55:39.513 INFO:tasks.workunit.client.0.smithi156.stdout:0/454: write d0/d1a/d4d/d9/d27/d3d/d62/f67 [297371,74938] 0 2024-07-16T09:55:39.513 INFO:tasks.workunit.client.1.smithi176.stdout:0/594: dwrite d5/d25/da0/fc0 [0,4194304] 0 2024-07-16T09:55:39.514 INFO:tasks.workunit.client.0.smithi156.stdout:5/549: write d1b/d1f/d45/d52/d43/f58 [800431,111223] 0 2024-07-16T09:55:39.516 INFO:tasks.workunit.client.0.smithi156.stdout:0/455: mknod d0/d1a/d4d/d25/c89 0 2024-07-16T09:55:39.517 INFO:tasks.workunit.client.1.smithi176.stdout:8/512: dwrite d8/d20/d2a/f49 [4194304,4194304] 0 2024-07-16T09:55:39.517 INFO:tasks.workunit.client.0.smithi156.stdout:5/550: mkdir d1b/d1f/d54/dbe 0 2024-07-16T09:55:39.521 INFO:tasks.workunit.client.1.smithi176.stdout:0/595: creat d5/d2c/d6c/d83/d84/fc2 x:0 0 0 2024-07-16T09:55:39.524 INFO:tasks.workunit.client.1.smithi176.stdout:8/513: mknod d8/d14/d1e/d33/cac 0 2024-07-16T09:55:39.524 INFO:tasks.workunit.client.0.smithi156.stdout:0/456: creat d0/d22/d41/f8a x:0 0 0 2024-07-16T09:55:39.526 INFO:tasks.workunit.client.0.smithi156.stdout:0/457: fsync d0/f43 0 2024-07-16T09:55:39.527 INFO:tasks.workunit.client.0.smithi156.stdout:5/551: dwrite d1b/d1f/d45/d52/f68 [4194304,4194304] 0 2024-07-16T09:55:39.528 INFO:tasks.workunit.client.0.smithi156.stdout:2/531: sync 2024-07-16T09:55:39.528 INFO:tasks.workunit.client.1.smithi176.stdout:0/596: symlink d5/d1c/d5a/lc3 0 2024-07-16T09:55:39.528 INFO:tasks.workunit.client.1.smithi176.stdout:2/516: sync 2024-07-16T09:55:39.528 INFO:tasks.workunit.client.1.smithi176.stdout:7/503: sync 2024-07-16T09:55:39.529 INFO:tasks.workunit.client.1.smithi176.stdout:5/510: sync 2024-07-16T09:55:39.529 INFO:tasks.workunit.client.1.smithi176.stdout:6/480: sync 2024-07-16T09:55:39.529 INFO:tasks.workunit.client.1.smithi176.stdout:1/310: sync 2024-07-16T09:55:39.529 INFO:tasks.workunit.client.0.smithi156.stdout:5/552: truncate d1b/f40 203308 0 2024-07-16T09:55:39.532 INFO:tasks.workunit.client.1.smithi176.stdout:4/518: dwrite da/dc/d10/f59 [0,4194304] 0 2024-07-16T09:55:39.539 INFO:tasks.workunit.client.1.smithi176.stdout:0/597: write d5/d1c/d24/d35/d59/f8c [601596,91422] 0 2024-07-16T09:55:39.539 INFO:tasks.workunit.client.1.smithi176.stdout:3/411: dwrite d3/d17/f2f [0,4194304] 0 2024-07-16T09:55:39.540 INFO:tasks.workunit.client.1.smithi176.stdout:2/517: read dd/d16/d39/f84 [714366,112099] 0 2024-07-16T09:55:39.544 INFO:tasks.workunit.client.0.smithi156.stdout:0/458: rename d0/f5c to d0/d1a/d4d/d9/d27/d2c/d6d/f8b 0 2024-07-16T09:55:39.544 INFO:tasks.workunit.client.0.smithi156.stdout:5/553: readlink d1b/d1f/d45/d52/d43/d85/d97/l22 0 2024-07-16T09:55:39.545 INFO:tasks.workunit.client.1.smithi176.stdout:8/514: mkdir d8/d14/d1e/d33/d9d/dad 0 2024-07-16T09:55:39.550 INFO:tasks.workunit.client.0.smithi156.stdout:2/532: dwrite d0/d5/d29/f3f [0,4194304] 0 2024-07-16T09:55:39.551 INFO:tasks.workunit.client.0.smithi156.stdout:5/554: symlink d1b/d1f/d45/d52/d2d/d96/lbf 0 2024-07-16T09:55:39.552 INFO:tasks.workunit.client.0.smithi156.stdout:0/459: dread d0/d1a/d4d/d9/f15 [4194304,4194304] 0 2024-07-16T09:55:39.554 INFO:tasks.workunit.client.1.smithi176.stdout:6/481: rename d1/d5/d73 to d1/db3 0 2024-07-16T09:55:39.557 INFO:tasks.workunit.client.0.smithi156.stdout:0/460: unlink d0/f43 0 2024-07-16T09:55:39.561 INFO:tasks.workunit.client.0.smithi156.stdout:2/533: creat d0/d5/d29/d2d/d5b/d62/fb9 x:0 0 0 2024-07-16T09:55:39.563 INFO:tasks.workunit.client.0.smithi156.stdout:0/461: mknod d0/d1a/d4d/d9/d27/d3d/d62/c8c 0 2024-07-16T09:55:39.565 INFO:tasks.workunit.client.1.smithi176.stdout:7/504: mkdir d4/d47/d5d/d9a/da3 0 2024-07-16T09:55:39.566 INFO:tasks.workunit.client.1.smithi176.stdout:8/515: symlink d8/d14/d1e/d33/d9d/lae 0 2024-07-16T09:55:39.566 INFO:tasks.workunit.client.1.smithi176.stdout:3/412: rename d3/d17/d1d/d4f/d6d/d71 to d3/d17/d1d/d4d/d7a 0 2024-07-16T09:55:39.566 INFO:tasks.workunit.client.1.smithi176.stdout:5/511: dread d4/d10/d17/f2f [0,4194304] 0 2024-07-16T09:55:39.566 INFO:tasks.workunit.client.0.smithi156.stdout:5/555: getdents d1b/d1f/d45/d52/d43/d85/d97/d37 0 2024-07-16T09:55:39.566 INFO:tasks.workunit.client.1.smithi176.stdout:7/505: chown d4/d1f 2000549987 1 2024-07-16T09:55:39.567 INFO:tasks.workunit.client.1.smithi176.stdout:0/598: dwrite d5/d1c/d24/d35/d50/d58/f60 [0,4194304] 0 2024-07-16T09:55:39.567 INFO:tasks.workunit.client.0.smithi156.stdout:5/556: write d1b/d1f/d45/d52/d43/d85/d97/f21 [1813092,63796] 0 2024-07-16T09:55:39.567 INFO:tasks.workunit.client.0.smithi156.stdout:2/534: creat d0/d9b/fba x:0 0 0 2024-07-16T09:55:39.568 INFO:tasks.workunit.client.1.smithi176.stdout:9/615: dwrite d5/de/f17 [0,4194304] 0 2024-07-16T09:55:39.569 INFO:tasks.workunit.client.0.smithi156.stdout:0/462: mkdir d0/d1a/d4d/d9/d27/d8d 0 2024-07-16T09:55:39.570 INFO:tasks.workunit.client.1.smithi176.stdout:4/519: dwrite da/d14/d40/d5d/d73/f3c [0,4194304] 0 2024-07-16T09:55:39.572 INFO:tasks.workunit.client.0.smithi156.stdout:5/557: creat d1b/d1f/d45/d52/d43/d7c/fc0 x:0 0 0 2024-07-16T09:55:39.573 INFO:tasks.workunit.client.1.smithi176.stdout:8/516: chown d8/d14/f2d 47 1 2024-07-16T09:55:39.574 INFO:tasks.workunit.client.1.smithi176.stdout:3/413: readlink d3/l18 0 2024-07-16T09:55:39.574 INFO:tasks.workunit.client.1.smithi176.stdout:9/616: fsync d5/d1d/d20/f33 0 2024-07-16T09:55:39.575 INFO:tasks.workunit.client.1.smithi176.stdout:8/517: write d8/d14/f2d [2143233,69287] 0 2024-07-16T09:55:39.575 INFO:tasks.workunit.client.0.smithi156.stdout:2/535: chown d0/d5/d29/d2d/d5b/d62/d74/f8f 2842 1 2024-07-16T09:55:39.576 INFO:tasks.workunit.client.0.smithi156.stdout:0/463: creat d0/d1a/d4d/d9/d27/d3d/d62/f8e x:0 0 0 2024-07-16T09:55:39.578 INFO:tasks.workunit.client.0.smithi156.stdout:0/464: chown d0/d1a/d4d/d9/f56 3 1 2024-07-16T09:55:39.579 INFO:tasks.workunit.client.1.smithi176.stdout:0/599: creat d5/d1c/d24/d35/d50/d58/fc4 x:0 0 0 2024-07-16T09:55:39.580 INFO:tasks.workunit.client.0.smithi156.stdout:5/558: creat d1b/d1f/d45/fc1 x:0 0 0 2024-07-16T09:55:39.581 INFO:tasks.workunit.client.1.smithi176.stdout:0/600: read d5/d1c/f8e [651228,76443] 0 2024-07-16T09:55:39.581 INFO:tasks.workunit.client.1.smithi176.stdout:5/512: rename d4/d14/d19/d1b/l75 to d4/d9a/lb8 0 2024-07-16T09:55:39.581 INFO:tasks.workunit.client.0.smithi156.stdout:0/465: write d0/d22/d41/d52/f58 [622984,42162] 0 2024-07-16T09:55:39.581 INFO:tasks.workunit.client.0.smithi156.stdout:2/536: truncate d0/d13/d17/f21 2942673 0 2024-07-16T09:55:39.582 INFO:tasks.workunit.client.1.smithi176.stdout:5/513: dread - d4/d14/fac zero size 2024-07-16T09:55:39.582 INFO:tasks.workunit.client.0.smithi156.stdout:0/466: chown d0/d1a/d4d/c13 977975 1 2024-07-16T09:55:39.582 INFO:tasks.workunit.client.0.smithi156.stdout:5/559: chown d1b/d1f/d45/d52/d2d/fb4 16034783 1 2024-07-16T09:55:39.584 INFO:tasks.workunit.client.1.smithi176.stdout:3/414: mkdir d3/da/d7b 0 2024-07-16T09:55:39.584 INFO:tasks.workunit.client.0.smithi156.stdout:5/560: read d1b/d1f/d45/d52/f39 [1842625,30197] 0 2024-07-16T09:55:39.585 INFO:tasks.workunit.client.1.smithi176.stdout:8/518: mknod d8/d23/d2c/caf 0 2024-07-16T09:55:39.586 INFO:tasks.workunit.client.0.smithi156.stdout:5/561: stat d1b/d1f/d45/d52/d43 0 2024-07-16T09:55:39.587 INFO:tasks.workunit.client.0.smithi156.stdout:0/467: fdatasync d0/d1a/d4d/d9/d27/f6a 0 2024-07-16T09:55:39.589 INFO:tasks.workunit.client.0.smithi156.stdout:2/537: getdents d0/d5/d29/d2d/d5b 0 2024-07-16T09:55:39.590 INFO:tasks.workunit.client.1.smithi176.stdout:0/601: dwrite d5/d1c/d5a/f6d [4194304,4194304] 0 2024-07-16T09:55:39.591 INFO:tasks.workunit.client.1.smithi176.stdout:9/617: link d5/d1d/d77/c7e d5/d1d/db0/dd3/cd4 0 2024-07-16T09:55:39.591 INFO:tasks.workunit.client.1.smithi176.stdout:6/482: getdents d1/d5/d47/d62 0 2024-07-16T09:55:39.593 INFO:tasks.workunit.client.0.smithi156.stdout:5/562: mkdir d1b/d1f/d45/d52/d43/d85/d97/d37/d88/dbc/dc2 0 2024-07-16T09:55:39.594 INFO:tasks.workunit.client.1.smithi176.stdout:0/602: unlink d5/l45 0 2024-07-16T09:55:39.594 INFO:tasks.workunit.client.1.smithi176.stdout:0/603: stat d5/d39 0 2024-07-16T09:55:39.595 INFO:tasks.workunit.client.1.smithi176.stdout:5/514: creat d4/d10/fb9 x:0 0 0 2024-07-16T09:55:39.596 INFO:tasks.workunit.client.1.smithi176.stdout:6/483: rename d1/c14 to d1/d5/d47/d75/cb4 0 2024-07-16T09:55:39.597 INFO:tasks.workunit.client.1.smithi176.stdout:9/618: truncate d5/d1d/d20/f25 503384 0 2024-07-16T09:55:39.598 INFO:tasks.workunit.client.0.smithi156.stdout:0/468: getdents d0/d1a/d76 0 2024-07-16T09:55:39.599 INFO:tasks.workunit.client.0.smithi156.stdout:0/469: read d0/d1a/d4d/d9/d27/d3d/f37 [4110403,37258] 0 2024-07-16T09:55:39.600 INFO:tasks.workunit.client.1.smithi176.stdout:0/604: rename d5/d2c/f80 to d5/d1c/d5a/fc5 0 2024-07-16T09:55:39.600 INFO:tasks.workunit.client.1.smithi176.stdout:5/515: mknod d4/d37/d72/d38/d39/cba 0 2024-07-16T09:55:39.601 INFO:tasks.workunit.client.1.smithi176.stdout:4/520: fdatasync da/d14/d40/d5d/d73/f3c 0 2024-07-16T09:55:39.602 INFO:tasks.workunit.client.1.smithi176.stdout:0/605: dread - d5/d25/d82/f9a zero size 2024-07-16T09:55:39.603 INFO:tasks.workunit.client.1.smithi176.stdout:6/484: creat d1/d5/d47/d75/fb5 x:0 0 0 2024-07-16T09:55:39.605 INFO:tasks.workunit.client.1.smithi176.stdout:5/516: read - d4/d37/d72/f59 zero size 2024-07-16T09:55:39.605 INFO:tasks.workunit.client.0.smithi156.stdout:0/470: rename d0/d22/d41/d52/f58 to d0/d1a/d4d/d9/f8f 0 2024-07-16T09:55:39.608 INFO:tasks.workunit.client.1.smithi176.stdout:1/311: truncate d4/d3a/d54/d5a/f5b 87834 0 2024-07-16T09:55:39.609 INFO:tasks.workunit.client.1.smithi176.stdout:3/415: dread d3/d17/d1d/d4f/d59/f21 [0,4194304] 0 2024-07-16T09:55:39.609 INFO:tasks.workunit.client.1.smithi176.stdout:6/485: creat d1/d5/d7/d2f/fb6 x:0 0 0 2024-07-16T09:55:39.610 INFO:tasks.workunit.client.0.smithi156.stdout:0/471: dread d0/d1a/d4d/d9/d27/d3d/d62/f63 [0,4194304] 0 2024-07-16T09:55:39.611 INFO:tasks.workunit.client.1.smithi176.stdout:1/312: chown d4/d8/d12/d4c 0 1 2024-07-16T09:55:39.612 INFO:tasks.workunit.client.1.smithi176.stdout:4/521: creat da/dc/d10/d96/d69/f9f x:0 0 0 2024-07-16T09:55:39.612 INFO:tasks.workunit.client.1.smithi176.stdout:0/606: mknod d5/d1c/d24/d35/d59/cc6 0 2024-07-16T09:55:39.614 INFO:tasks.workunit.client.0.smithi156.stdout:0/472: mkdir d0/d1a/d4d/d9/d27/d50/d90 0 2024-07-16T09:55:39.616 INFO:tasks.workunit.client.1.smithi176.stdout:5/517: mkdir d4/d37/dbb 0 2024-07-16T09:55:39.616 INFO:tasks.workunit.client.0.smithi156.stdout:0/473: mknod d0/d1a/d4d/d9/d27/d2c/c91 0 2024-07-16T09:55:39.617 INFO:tasks.workunit.client.1.smithi176.stdout:6/486: write d1/db3/f9a [1011826,116208] 0 2024-07-16T09:55:39.617 INFO:tasks.workunit.client.1.smithi176.stdout:8/519: write d8/f17 [2368626,80769] 0 2024-07-16T09:55:39.620 INFO:tasks.workunit.client.1.smithi176.stdout:9/619: dwrite d5/de/f61 [0,4194304] 0 2024-07-16T09:55:39.621 INFO:tasks.workunit.client.0.smithi156.stdout:0/474: mkdir d0/d1a/d76/d92 0 2024-07-16T09:55:39.622 INFO:tasks.workunit.client.1.smithi176.stdout:2/518: dwrite dd/de/f10 [0,4194304] 0 2024-07-16T09:55:39.624 INFO:tasks.workunit.client.0.smithi156.stdout:2/538: dwrite d0/f7 [0,4194304] 0 2024-07-16T09:55:39.625 INFO:tasks.workunit.client.0.smithi156.stdout:5/563: dwrite fc [0,4194304] 0 2024-07-16T09:55:39.627 INFO:tasks.workunit.client.0.smithi156.stdout:0/475: write d0/d1a/d4d/d9/f15 [9518914,187] 0 2024-07-16T09:55:39.627 INFO:tasks.workunit.client.1.smithi176.stdout:4/522: symlink da/dc/la0 0 2024-07-16T09:55:39.627 INFO:tasks.workunit.client.1.smithi176.stdout:2/519: fsync dd/f77 0 2024-07-16T09:55:39.632 INFO:tasks.workunit.client.0.smithi156.stdout:5/564: creat d1b/d1f/d54/dbe/fc3 x:0 0 0 2024-07-16T09:55:39.634 INFO:tasks.workunit.client.1.smithi176.stdout:4/523: write da/dc/f58 [3885176,91476] 0 2024-07-16T09:55:39.635 INFO:tasks.workunit.client.1.smithi176.stdout:7/506: dwrite d4/d47/f46 [0,4194304] 0 2024-07-16T09:55:39.635 INFO:tasks.workunit.client.1.smithi176.stdout:0/607: symlink d5/d1c/d24/lc7 0 2024-07-16T09:55:39.636 INFO:tasks.workunit.client.0.smithi156.stdout:0/476: creat d0/d1a/d4d/d60/f93 x:0 0 0 2024-07-16T09:55:39.639 INFO:tasks.workunit.client.1.smithi176.stdout:8/520: mkdir d8/d20/d2a/d31/d2f/db0 0 2024-07-16T09:55:39.644 INFO:tasks.workunit.client.1.smithi176.stdout:6/487: creat d1/d5d/d9e/d76/da5/fb7 x:0 0 0 2024-07-16T09:55:39.644 INFO:tasks.workunit.client.0.smithi156.stdout:0/477: mknod d0/d1a/d4d/d6e/d86/c94 0 2024-07-16T09:55:39.645 INFO:tasks.workunit.client.1.smithi176.stdout:7/507: chown d4/lf 11 1 2024-07-16T09:55:39.645 INFO:tasks.workunit.client.0.smithi156.stdout:5/565: link d1b/cb8 d1b/d1f/d45/d9d/cc4 0 2024-07-16T09:55:39.646 INFO:tasks.workunit.client.1.smithi176.stdout:2/520: truncate f3 5630300 0 2024-07-16T09:55:39.647 INFO:tasks.workunit.client.1.smithi176.stdout:1/313: link d4/d8/l5d d4/d3a/l6b 0 2024-07-16T09:55:39.648 INFO:tasks.workunit.client.0.smithi156.stdout:5/566: mknod d1b/d1f/d45/d52/d43/d85/cc5 0 2024-07-16T09:55:39.648 INFO:tasks.workunit.client.1.smithi176.stdout:5/518: dwrite d4/d37/d72/f53 [0,4194304] 0 2024-07-16T09:55:39.650 INFO:tasks.workunit.client.1.smithi176.stdout:8/521: creat d8/d23/d2c/d34/d62/fb1 x:0 0 0 2024-07-16T09:55:39.651 INFO:tasks.workunit.client.0.smithi156.stdout:0/478: creat d0/d1a/d4d/f95 x:0 0 0 2024-07-16T09:55:39.651 INFO:tasks.workunit.client.0.smithi156.stdout:5/567: mkdir d1b/d1f/d54/dc6 0 2024-07-16T09:55:39.652 INFO:tasks.workunit.client.1.smithi176.stdout:3/416: link d3/d17/c1c d3/d17/d1d/d4f/d59/c7c 0 2024-07-16T09:55:39.653 INFO:tasks.workunit.client.0.smithi156.stdout:5/568: write d1b/d1f/d45/d9d/fab [1170938,40965] 0 2024-07-16T09:55:39.654 INFO:tasks.workunit.client.1.smithi176.stdout:0/608: creat d5/d1c/d24/d35/fc8 x:0 0 0 2024-07-16T09:55:39.654 INFO:tasks.workunit.client.1.smithi176.stdout:2/521: creat dd/d49/d52/faa x:0 0 0 2024-07-16T09:55:39.655 INFO:tasks.workunit.client.1.smithi176.stdout:1/314: dread - d4/d8/d12/f4a zero size 2024-07-16T09:55:39.656 INFO:tasks.workunit.client.0.smithi156.stdout:0/479: symlink d0/d22/d41/d52/l96 0 2024-07-16T09:55:39.656 INFO:tasks.workunit.client.1.smithi176.stdout:4/524: rename da/d3d/d17/d54/d78/d7a/f93 to da/d14/d40/fa1 0 2024-07-16T09:55:39.656 INFO:tasks.workunit.client.1.smithi176.stdout:6/488: dread d1/dc/f97 [0,4194304] 0 2024-07-16T09:55:39.657 INFO:tasks.workunit.client.1.smithi176.stdout:0/609: truncate d5/d44/f85 1329906 0 2024-07-16T09:55:39.658 INFO:tasks.workunit.client.0.smithi156.stdout:0/480: fdatasync d0/d22/d41/f47 0 2024-07-16T09:55:39.658 INFO:tasks.workunit.client.1.smithi176.stdout:4/525: write da/f47 [951415,76079] 0 2024-07-16T09:55:39.659 INFO:tasks.workunit.client.1.smithi176.stdout:1/315: chown d4/l4f 21161111 1 2024-07-16T09:55:39.659 INFO:tasks.workunit.client.1.smithi176.stdout:6/489: write d1/d5/f98 [1005149,79050] 0 2024-07-16T09:55:39.659 INFO:tasks.workunit.client.1.smithi176.stdout:3/417: symlink d3/da/d6c/l7d 0 2024-07-16T09:55:39.660 INFO:tasks.workunit.client.0.smithi156.stdout:0/481: mkdir d0/d1a/d4d/d61/d97 0 2024-07-16T09:55:39.662 INFO:tasks.workunit.client.1.smithi176.stdout:7/508: rename d4/f19 to d4/d47/d6d/d89/d66/fa4 0 2024-07-16T09:55:39.663 INFO:tasks.workunit.client.0.smithi156.stdout:0/482: rmdir d0/d1a/d4d/d9/d27/d3d/d62 39 2024-07-16T09:55:39.664 INFO:tasks.workunit.client.0.smithi156.stdout:0/483: write d0/d1a/d4d/f95 [377199,109303] 0 2024-07-16T09:55:39.665 INFO:tasks.workunit.client.1.smithi176.stdout:4/526: unlink da/d3d/d17/d54/c8f 0 2024-07-16T09:55:39.667 INFO:tasks.workunit.client.1.smithi176.stdout:3/418: getdents d3/da/d56 0 2024-07-16T09:55:39.667 INFO:tasks.workunit.client.0.smithi156.stdout:0/484: symlink d0/d1a/d76/d92/l98 0 2024-07-16T09:55:39.667 INFO:tasks.workunit.client.0.smithi156.stdout:0/485: chown d0/d22/d41/f47 5070 1 2024-07-16T09:55:39.668 INFO:tasks.workunit.client.1.smithi176.stdout:2/522: rename dd/d36/f5d to dd/d9b/fab 0 2024-07-16T09:55:39.672 INFO:tasks.workunit.client.1.smithi176.stdout:3/419: mkdir d3/d17/d7e 0 2024-07-16T09:55:39.673 INFO:tasks.workunit.client.0.smithi156.stdout:0/486: dwrite d0/d1a/d4d/d9/f26 [0,4194304] 0 2024-07-16T09:55:39.673 INFO:tasks.workunit.client.1.smithi176.stdout:5/519: rename d4/d14/d19/d1b/d61/c69 to d4/d37/d72/d78/cbc 0 2024-07-16T09:55:39.676 INFO:tasks.workunit.client.1.smithi176.stdout:6/490: link d1/dc/d40/l7f d1/d5d/d9e/d76/lb8 0 2024-07-16T09:55:39.676 INFO:tasks.workunit.client.1.smithi176.stdout:3/420: write d3/d17/d1d/d4f/d59/f34 [1491325,53207] 0 2024-07-16T09:55:39.676 INFO:tasks.workunit.client.1.smithi176.stdout:5/520: chown d4/d10/d17/c2b 750 1 2024-07-16T09:55:39.678 INFO:tasks.workunit.client.1.smithi176.stdout:2/523: dwrite dd/d16/f33 [4194304,4194304] 0 2024-07-16T09:55:39.679 INFO:tasks.workunit.client.0.smithi156.stdout:0/487: dread d0/d1a/d4d/d9/f15 [0,4194304] 0 2024-07-16T09:55:39.681 INFO:tasks.workunit.client.1.smithi176.stdout:1/316: rename d4/d8/d12/d21/d34/f51 to d4/d3a/f6c 0 2024-07-16T09:55:39.682 INFO:tasks.workunit.client.1.smithi176.stdout:4/527: link da/dc/fe da/d3d/fa2 0 2024-07-16T09:55:39.685 INFO:tasks.workunit.client.1.smithi176.stdout:6/491: rmdir d1/d5/d47 39 2024-07-16T09:55:39.686 INFO:tasks.workunit.client.1.smithi176.stdout:6/492: chown d1/dc/l3d 3 1 2024-07-16T09:55:39.687 INFO:tasks.workunit.client.0.smithi156.stdout:0/488: dwrite d0/d1a/d4d/d9/f26 [4194304,4194304] 0 2024-07-16T09:55:39.690 INFO:tasks.workunit.client.0.smithi156.stdout:0/489: rmdir d0/d1a/d4d/d9/d27/d2c 39 2024-07-16T09:55:39.691 INFO:tasks.workunit.client.1.smithi176.stdout:7/509: dread d4/d1f/d22/f52 [0,4194304] 0 2024-07-16T09:55:39.694 INFO:tasks.workunit.client.0.smithi156.stdout:0/490: symlink d0/d1a/d4d/d9/d27/d50/d90/l99 0 2024-07-16T09:55:39.695 INFO:tasks.workunit.client.0.smithi156.stdout:0/491: chown d0/d1a/l57 14258271 1 2024-07-16T09:55:39.696 INFO:tasks.workunit.client.0.smithi156.stdout:0/492: stat d0/c4e 0 2024-07-16T09:55:39.697 INFO:tasks.workunit.client.1.smithi176.stdout:5/521: mkdir d4/d37/d72/d82/d96/dbd 0 2024-07-16T09:55:39.698 INFO:tasks.workunit.client.1.smithi176.stdout:8/522: write d8/d20/d2a/d31/f3e [4114458,109370] 0 2024-07-16T09:55:39.698 INFO:tasks.workunit.client.1.smithi176.stdout:6/493: unlink d1/d5/d47/f86 0 2024-07-16T09:55:39.699 INFO:tasks.workunit.client.0.smithi156.stdout:0/493: creat d0/d1a/d4d/d9/d27/d50/f9a x:0 0 0 2024-07-16T09:55:39.702 INFO:tasks.workunit.client.1.smithi176.stdout:7/510: rename d4/d1f/d22/l29 to d4/d47/d6d/d89/d66/la5 0 2024-07-16T09:55:39.703 INFO:tasks.workunit.client.1.smithi176.stdout:7/511: read d4/f2b [467492,36835] 0 2024-07-16T09:55:39.705 INFO:tasks.workunit.client.1.smithi176.stdout:7/512: dread d4/d1f/d22/d80/d17/f4d [0,4194304] 0 2024-07-16T09:55:39.706 INFO:tasks.workunit.client.0.smithi156.stdout:0/494: dwrite d0/d1a/d4d/d9/f12 [0,4194304] 0 2024-07-16T09:55:39.708 INFO:tasks.workunit.client.1.smithi176.stdout:8/523: symlink d8/d20/d2a/d31/d2f/lb2 0 2024-07-16T09:55:39.709 INFO:tasks.workunit.client.1.smithi176.stdout:3/421: link d3/da/d33/l5f d3/d17/d1d/d4f/l7f 0 2024-07-16T09:55:39.709 INFO:tasks.workunit.client.0.smithi156.stdout:0/495: creat d0/d1a/f9b x:0 0 0 2024-07-16T09:55:39.710 INFO:tasks.workunit.client.0.smithi156.stdout:0/496: read d0/d1a/d4d/d9/f15 [9330830,23382] 0 2024-07-16T09:55:39.711 INFO:tasks.workunit.client.1.smithi176.stdout:7/513: dwrite d4/f11 [4194304,4194304] 0 2024-07-16T09:55:39.711 INFO:tasks.workunit.client.1.smithi176.stdout:6/494: rename d1/d5/d7/d2f to d1/d5d/db9 0 2024-07-16T09:55:39.715 INFO:tasks.workunit.client.1.smithi176.stdout:4/528: getdents da/dc/d4b/d62 0 2024-07-16T09:55:39.715 INFO:tasks.workunit.client.0.smithi156.stdout:0/497: dread d0/d1a/d4d/f95 [0,4194304] 0 2024-07-16T09:55:39.716 INFO:tasks.workunit.client.1.smithi176.stdout:5/522: mkdir d4/d37/d72/d38/d6e/dbe 0 2024-07-16T09:55:39.718 INFO:tasks.workunit.client.1.smithi176.stdout:4/529: write da/d3d/d17/d48/f80 [679016,9855] 0 2024-07-16T09:55:39.719 INFO:tasks.workunit.client.1.smithi176.stdout:3/422: mknod d3/da/d66/c80 0 2024-07-16T09:55:39.720 INFO:tasks.workunit.client.1.smithi176.stdout:5/523: fdatasync d4/d10/d17/f66 0 2024-07-16T09:55:39.720 INFO:tasks.workunit.client.0.smithi156.stdout:0/498: rename d0/d1a/d4d/d9/f7b to d0/d1a/f9c 0 2024-07-16T09:55:39.720 INFO:tasks.workunit.client.1.smithi176.stdout:4/530: write da/dc/f9d [816361,91973] 0 2024-07-16T09:55:39.721 INFO:tasks.workunit.client.1.smithi176.stdout:3/423: fsync d3/f55 0 2024-07-16T09:55:39.723 INFO:tasks.workunit.client.1.smithi176.stdout:3/424: readlink d3/l28 0 2024-07-16T09:55:39.725 INFO:tasks.workunit.client.0.smithi156.stdout:0/499: getdents d0 0 2024-07-16T09:55:39.725 INFO:tasks.workunit.client.1.smithi176.stdout:6/495: dwrite d1/dc/d72/d6f/d7c/f90 [0,4194304] 0 2024-07-16T09:55:39.725 INFO:tasks.workunit.client.1.smithi176.stdout:4/531: chown da/dc/f83 90420586 1 2024-07-16T09:55:39.725 INFO:tasks.workunit.client.0.smithi156.stdout:0/500: chown d0/d22/d41/f4a 3 1 2024-07-16T09:55:39.728 INFO:tasks.workunit.client.0.smithi156.stdout:0/501: getdents d0/d1a/d4d/d61/d97 0 2024-07-16T09:55:39.729 INFO:tasks.workunit.client.1.smithi176.stdout:8/524: mknod d8/d20/d2a/d31/d46/d6a/da8/cb3 0 2024-07-16T09:55:39.730 INFO:tasks.workunit.client.1.smithi176.stdout:2/524: dwrite dd/d16/f27 [0,4194304] 0 2024-07-16T09:55:39.730 INFO:tasks.workunit.client.0.smithi156.stdout:0/502: getdents d0/d1a/d76/d83 0 2024-07-16T09:55:39.731 INFO:tasks.workunit.client.1.smithi176.stdout:4/532: creat da/d3d/d17/d54/fa3 x:0 0 0 2024-07-16T09:55:39.731 INFO:tasks.workunit.client.1.smithi176.stdout:3/425: write d3/da/f13 [4018588,104225] 0 2024-07-16T09:55:39.736 INFO:tasks.workunit.client.1.smithi176.stdout:7/514: rename d4/dd/f99 to d4/d1f/d22/d80/fa6 0 2024-07-16T09:55:39.740 INFO:tasks.workunit.client.1.smithi176.stdout:6/496: dwrite d1/d5d/d9e/d8f/f74 [0,4194304] 0 2024-07-16T09:55:39.745 INFO:tasks.workunit.client.1.smithi176.stdout:3/426: mkdir d3/d17/d1d/d4f/d59/d81 0 2024-07-16T09:55:39.745 INFO:tasks.workunit.client.1.smithi176.stdout:7/515: mkdir d4/d47/d6d/da7 0 2024-07-16T09:55:39.746 INFO:tasks.workunit.client.1.smithi176.stdout:2/525: dwrite dd/d16/d39/d3d/f80 [0,4194304] 0 2024-07-16T09:55:39.748 INFO:tasks.workunit.client.1.smithi176.stdout:5/524: rename d4/d14/d19/d1b/c51 to d4/cbf 0 2024-07-16T09:55:39.751 INFO:tasks.workunit.client.1.smithi176.stdout:3/427: write d3/da/d33/f52 [4995557,94242] 0 2024-07-16T09:55:39.755 INFO:tasks.workunit.client.1.smithi176.stdout:4/533: rename da/d3d/d17/d48/d82 to da/dc/d10/d96/da4 0 2024-07-16T09:55:39.756 INFO:tasks.workunit.client.1.smithi176.stdout:4/534: read da/d3d/d17/f35 [877018,87068] 0 2024-07-16T09:55:39.760 INFO:tasks.workunit.client.1.smithi176.stdout:2/526: read dd/d2e/f50 [691194,121562] 0 2024-07-16T09:55:39.761 INFO:tasks.workunit.client.1.smithi176.stdout:3/428: symlink d3/d17/d1d/d6e/l82 0 2024-07-16T09:55:39.761 INFO:tasks.workunit.client.1.smithi176.stdout:5/525: mknod d4/d14/d19/d1b/dab/cc0 0 2024-07-16T09:55:39.762 INFO:tasks.workunit.client.1.smithi176.stdout:8/525: write d8/d14/d1e/d7c/f8f [247121,41408] 0 2024-07-16T09:55:39.763 INFO:tasks.workunit.client.1.smithi176.stdout:6/497: link d1/d5d/db9/c57 d1/d5d/cba 0 2024-07-16T09:55:39.764 INFO:tasks.workunit.client.1.smithi176.stdout:3/429: dread d3/da/f29 [0,4194304] 0 2024-07-16T09:55:39.765 INFO:tasks.workunit.client.1.smithi176.stdout:6/498: write d1/f31 [210183,118374] 0 2024-07-16T09:55:39.766 INFO:tasks.workunit.client.1.smithi176.stdout:8/526: rmdir d8/d20/d2a/d31/d46/d6a/da8 39 2024-07-16T09:55:39.766 INFO:tasks.workunit.client.1.smithi176.stdout:5/526: read - d4/d14/d19/d1b/d61/f80 zero size 2024-07-16T09:55:39.767 INFO:tasks.workunit.client.1.smithi176.stdout:4/535: creat da/d14/fa5 x:0 0 0 2024-07-16T09:55:39.767 INFO:tasks.workunit.client.1.smithi176.stdout:3/430: mkdir d3/d17/d83 0 2024-07-16T09:55:39.769 INFO:tasks.workunit.client.1.smithi176.stdout:4/536: stat da/dc/d10/d96/da4/c8a 0 2024-07-16T09:55:39.769 INFO:tasks.workunit.client.1.smithi176.stdout:4/537: stat da/d14/d40/d5d/d73/f3c 0 2024-07-16T09:55:39.770 INFO:tasks.workunit.client.1.smithi176.stdout:7/516: rename d4/d47/d5d/l65 to d4/d5/la8 0 2024-07-16T09:55:39.770 INFO:tasks.workunit.client.1.smithi176.stdout:6/499: symlink d1/dc/d72/d6f/lbb 0 2024-07-16T09:55:39.771 INFO:tasks.workunit.client.0.smithi156.stdout:0/503: dread d0/d1a/d4d/d25/f2a [0,4194304] 0 2024-07-16T09:55:39.772 INFO:tasks.workunit.client.0.smithi156.stdout:0/504: chown d0/d1a/d4d/d6e/d86/c94 59195 1 2024-07-16T09:55:39.773 INFO:tasks.workunit.client.1.smithi176.stdout:4/538: mknod da/dc/d10/d96/d69/ca6 0 2024-07-16T09:55:39.774 INFO:tasks.workunit.client.1.smithi176.stdout:7/517: rename d4/f27 to d4/d47/d5d/d9a/fa9 0 2024-07-16T09:55:39.775 INFO:tasks.workunit.client.1.smithi176.stdout:8/527: mkdir d8/d20/d2a/d31/db4 0 2024-07-16T09:55:39.776 INFO:tasks.workunit.client.1.smithi176.stdout:8/528: readlink d8/d20/d2a/d31/d2f/l7b 0 2024-07-16T09:55:39.776 INFO:tasks.workunit.client.1.smithi176.stdout:3/431: mkdir d3/da/d84 0 2024-07-16T09:55:39.777 INFO:tasks.workunit.client.0.smithi156.stdout:0/505: dread d0/d22/f3c [0,4194304] 0 2024-07-16T09:55:39.778 INFO:tasks.workunit.client.1.smithi176.stdout:4/539: creat da/dc/d4b/fa7 x:0 0 0 2024-07-16T09:55:39.779 INFO:tasks.workunit.client.1.smithi176.stdout:6/500: rename d1/d5/d7/d54/c7d to d1/d5/d7/d54/cbc 0 2024-07-16T09:55:39.779 INFO:tasks.workunit.client.0.smithi156.stdout:7/373: sync 2024-07-16T09:55:39.779 INFO:tasks.workunit.client.0.smithi156.stdout:4/461: sync 2024-07-16T09:55:39.779 INFO:tasks.workunit.client.0.smithi156.stdout:1/622: sync 2024-07-16T09:55:39.780 INFO:tasks.workunit.client.0.smithi156.stdout:8/372: sync 2024-07-16T09:55:39.780 INFO:tasks.workunit.client.0.smithi156.stdout:9/495: sync 2024-07-16T09:55:39.780 INFO:tasks.workunit.client.0.smithi156.stdout:0/506: mkdir d0/d1a/d4d/d9/d27/d2c/d6d/d9d 0 2024-07-16T09:55:39.780 INFO:tasks.workunit.client.1.smithi176.stdout:8/529: unlink d8/d20/d2a/l5e 0 2024-07-16T09:55:39.781 INFO:tasks.workunit.client.1.smithi176.stdout:8/530: stat d8/d14/d1e/f1f 0 2024-07-16T09:55:39.782 INFO:tasks.workunit.client.0.smithi156.stdout:7/374: write dc/d13/d36/f66 [898223,60207] 0 2024-07-16T09:55:39.782 INFO:tasks.workunit.client.0.smithi156.stdout:4/462: chown d4/db/d47/d63/d58/f91 330746 1 2024-07-16T09:55:39.783 INFO:tasks.workunit.client.0.smithi156.stdout:8/373: rmdir d0/d11 39 2024-07-16T09:55:39.785 INFO:tasks.workunit.client.0.smithi156.stdout:7/375: truncate dc/d3e/d3f/d51/f53 2382478 0 2024-07-16T09:55:39.785 INFO:tasks.workunit.client.0.smithi156.stdout:7/376: chown dc/d13/d15/f50 144085943 1 2024-07-16T09:55:39.787 INFO:tasks.workunit.client.0.smithi156.stdout:9/496: mknod d2/d5/d16/d22/d2b/d3d/c9a 0 2024-07-16T09:55:39.789 INFO:tasks.workunit.client.0.smithi156.stdout:0/507: link d0/d22/d41/f47 d0/d22/f9e 0 2024-07-16T09:55:39.790 INFO:tasks.workunit.client.0.smithi156.stdout:7/377: readlink dc/d3e/d3f/d45/l68 0 2024-07-16T09:55:39.792 INFO:tasks.workunit.client.0.smithi156.stdout:4/463: dread d4/db/dc/d5f/f5c [0,4194304] 0 2024-07-16T09:55:39.794 INFO:tasks.workunit.client.0.smithi156.stdout:9/497: write d2/d5/d16/d22/d2b/d3d/d3c/d7f/f19 [6004343,35732] 0 2024-07-16T09:55:39.795 INFO:tasks.workunit.client.0.smithi156.stdout:8/374: dwrite d0/f6d [0,4194304] 0 2024-07-16T09:55:39.798 INFO:tasks.workunit.client.0.smithi156.stdout:4/464: write d4/f79 [3591360,17302] 0 2024-07-16T09:55:39.803 INFO:tasks.workunit.client.0.smithi156.stdout:8/375: fsync d0/d11/f16 0 2024-07-16T09:55:39.804 INFO:tasks.workunit.client.0.smithi156.stdout:1/623: dwrite dc/d14/d3e/d45/f53 [0,4194304] 0 2024-07-16T09:55:39.808 INFO:tasks.workunit.client.0.smithi156.stdout:0/508: dwrite d0/ff [0,4194304] 0 2024-07-16T09:55:39.811 INFO:tasks.workunit.client.0.smithi156.stdout:8/376: mknod d0/d11/d29/d5e/c84 0 2024-07-16T09:55:39.812 INFO:tasks.workunit.client.0.smithi156.stdout:7/378: dwrite dc/d3e/f7f [0,4194304] 0 2024-07-16T09:55:39.812 INFO:tasks.workunit.client.0.smithi156.stdout:0/509: chown d0/d22/d41 2570 1 2024-07-16T09:55:39.814 INFO:tasks.workunit.client.0.smithi156.stdout:7/379: chown dc/d13/d24/l34 305780940 1 2024-07-16T09:55:39.818 INFO:tasks.workunit.client.1.smithi176.stdout:4/540: symlink da/dc/d10/d96/la8 0 2024-07-16T09:55:39.819 INFO:tasks.workunit.client.1.smithi176.stdout:4/541: fsync da/d3d/d17/d54/fa3 0 2024-07-16T09:55:39.820 INFO:tasks.workunit.client.1.smithi176.stdout:6/501: mknod d1/d5/d47/d62/cbd 0 2024-07-16T09:55:39.822 INFO:tasks.workunit.client.0.smithi156.stdout:0/510: symlink d0/d1a/d4d/d9/d68/l9f 0 2024-07-16T09:55:39.823 INFO:tasks.workunit.client.1.smithi176.stdout:7/518: link d4/d47/d5d/l6a d4/d47/d5d/d9a/laa 0 2024-07-16T09:55:39.824 INFO:tasks.workunit.client.0.smithi156.stdout:0/511: chown d0/d1a/c1c 63026170 1 2024-07-16T09:55:39.824 INFO:tasks.workunit.client.0.smithi156.stdout:1/624: dwrite f4 [0,4194304] 0 2024-07-16T09:55:39.825 INFO:tasks.workunit.client.1.smithi176.stdout:6/502: write d1/d5d/d9e/d8f/f74 [89120,47656] 0 2024-07-16T09:55:39.825 INFO:tasks.workunit.client.1.smithi176.stdout:3/432: rename d3/d17/d7e to d3/d85 0 2024-07-16T09:55:39.826 INFO:tasks.workunit.client.1.smithi176.stdout:8/531: mknod d8/d20/d2a/d31/db4/cb5 0 2024-07-16T09:55:39.828 INFO:tasks.workunit.client.1.smithi176.stdout:4/542: dwrite da/dc/d4b/f97 [0,4194304] 0 2024-07-16T09:55:39.831 INFO:tasks.workunit.client.0.smithi156.stdout:7/380: dwrite fb [0,4194304] 0 2024-07-16T09:55:39.834 INFO:tasks.workunit.client.0.smithi156.stdout:1/625: creat dc/d14/fd4 x:0 0 0 2024-07-16T09:55:39.835 INFO:tasks.workunit.client.1.smithi176.stdout:7/519: unlink d4/d1f/d22/l4f 0 2024-07-16T09:55:39.836 INFO:tasks.workunit.client.1.smithi176.stdout:2/527: truncate dd/d16/f32 1596417 0 2024-07-16T09:55:39.836 INFO:tasks.workunit.client.0.smithi156.stdout:9/498: dread d2/d5/d16/d22/d36/d33/d23/f2e [0,4194304] 0 2024-07-16T09:55:39.838 INFO:tasks.workunit.client.1.smithi176.stdout:6/503: creat d1/d5/d47/d75/fbe x:0 0 0 2024-07-16T09:55:39.840 INFO:tasks.workunit.client.1.smithi176.stdout:6/504: chown d1/d5d/d9e/d76/f7b 75691 1 2024-07-16T09:55:39.840 INFO:tasks.workunit.client.0.smithi156.stdout:9/499: mkdir d2/d9b 0 2024-07-16T09:55:39.841 INFO:tasks.workunit.client.0.smithi156.stdout:7/381: creat dc/d3e/d3f/d51/d6c/f85 x:0 0 0 2024-07-16T09:55:39.841 INFO:tasks.workunit.client.1.smithi176.stdout:4/543: creat da/d3d/d17/d54/fa9 x:0 0 0 2024-07-16T09:55:39.842 INFO:tasks.workunit.client.1.smithi176.stdout:5/527: write d4/d14/d3d/f79 [633579,88827] 0 2024-07-16T09:55:39.842 INFO:tasks.workunit.client.0.smithi156.stdout:4/465: dread d4/db/dc/f4e [0,4194304] 0 2024-07-16T09:55:39.844 INFO:tasks.workunit.client.1.smithi176.stdout:8/532: creat d8/d23/d76/d83/fb6 x:0 0 0 2024-07-16T09:55:39.845 INFO:tasks.workunit.client.1.smithi176.stdout:6/505: mkdir d1/d5/d47/d62/dbf 0 2024-07-16T09:55:39.846 INFO:tasks.workunit.client.0.smithi156.stdout:9/500: rename d2/d5/d16/d22/d36/d33/l65 to d2/d5/d16/d22/d6b/l9c 0 2024-07-16T09:55:39.847 INFO:tasks.workunit.client.0.smithi156.stdout:4/466: symlink d4/db/dc/d7c/la4 0 2024-07-16T09:55:39.847 INFO:tasks.workunit.client.1.smithi176.stdout:4/544: dread da/d14/d40/d5d/d73/f4f [0,4194304] 0 2024-07-16T09:55:39.847 INFO:tasks.workunit.client.1.smithi176.stdout:6/506: truncate d1/dc/d40/f8b 831051 0 2024-07-16T09:55:39.849 INFO:tasks.workunit.client.0.smithi156.stdout:4/467: write d4/db/dc/f2e [5417155,37785] 0 2024-07-16T09:55:39.849 INFO:tasks.workunit.client.0.smithi156.stdout:7/382: dread dc/f40 [0,4194304] 0 2024-07-16T09:55:39.849 INFO:tasks.workunit.client.0.smithi156.stdout:9/501: creat d2/d5/d16/f9d x:0 0 0 2024-07-16T09:55:39.851 INFO:tasks.workunit.client.1.smithi176.stdout:8/533: unlink d8/d23/d2c/d37/f4f 0 2024-07-16T09:55:39.851 INFO:tasks.workunit.client.0.smithi156.stdout:7/383: write dc/d13/d36/f4c [803860,101916] 0 2024-07-16T09:55:39.853 INFO:tasks.workunit.client.1.smithi176.stdout:2/528: rename dd/d16/d39/d3d/f97 to dd/d2e/d35/fac 0 2024-07-16T09:55:39.855 INFO:tasks.workunit.client.0.smithi156.stdout:4/468: dread d4/db/f3f [0,4194304] 0 2024-07-16T09:55:39.855 INFO:tasks.workunit.client.1.smithi176.stdout:2/529: stat dd/d9b/fab 0 2024-07-16T09:55:39.855 INFO:tasks.workunit.client.1.smithi176.stdout:7/520: link d4/d1f/d22/d80/d17/d1d/c28 d4/d47/d5d/cab 0 2024-07-16T09:55:39.856 INFO:tasks.workunit.client.0.smithi156.stdout:4/469: chown d4/d23/f78 8356 1 2024-07-16T09:55:39.856 INFO:tasks.workunit.client.1.smithi176.stdout:7/521: read d4/d47/f46 [1584027,42693] 0 2024-07-16T09:55:39.856 INFO:tasks.workunit.client.1.smithi176.stdout:2/530: chown dd/d36/f94 6415974 1 2024-07-16T09:55:39.857 INFO:tasks.workunit.client.0.smithi156.stdout:9/502: dread d2/d5/d16/d22/d36/f59 [0,4194304] 0 2024-07-16T09:55:39.857 INFO:tasks.workunit.client.1.smithi176.stdout:6/507: read d1/d5/d7/f9 [2165374,71479] 0 2024-07-16T09:55:39.858 INFO:tasks.workunit.client.1.smithi176.stdout:7/522: dread d4/dd/d49/f7c [0,4194304] 0 2024-07-16T09:55:39.858 INFO:tasks.workunit.client.0.smithi156.stdout:4/470: mknod d4/d23/d52/ca5 0 2024-07-16T09:55:39.859 INFO:tasks.workunit.client.0.smithi156.stdout:7/384: dwrite dc/d44/f6d [0,4194304] 0 2024-07-16T09:55:39.859 INFO:tasks.workunit.client.1.smithi176.stdout:4/545: creat da/d3d/d17/d48/faa x:0 0 0 2024-07-16T09:55:39.859 INFO:tasks.workunit.client.0.smithi156.stdout:1/626: dread dc/d14/f4b [0,4194304] 0 2024-07-16T09:55:39.860 INFO:tasks.workunit.client.1.smithi176.stdout:4/546: truncate da/dc/d10/d96/d69/f9f 86387 0 2024-07-16T09:55:39.862 INFO:tasks.workunit.client.0.smithi156.stdout:4/471: creat d4/db/d47/d63/d58/fa6 x:0 0 0 2024-07-16T09:55:39.862 INFO:tasks.workunit.client.0.smithi156.stdout:7/385: chown dc/d3e/d3f/d51/f53 12362873 1 2024-07-16T09:55:39.862 INFO:tasks.workunit.client.1.smithi176.stdout:8/534: creat d8/d20/d2a/d53/fb7 x:0 0 0 2024-07-16T09:55:39.863 INFO:tasks.workunit.client.1.smithi176.stdout:8/535: dread - d8/d20/d2a/d61/d90/faa zero size 2024-07-16T09:55:39.864 INFO:tasks.workunit.client.1.smithi176.stdout:8/536: truncate d8/d14/d1e/d55/fa3 108912 0 2024-07-16T09:55:39.866 INFO:tasks.workunit.client.1.smithi176.stdout:5/528: creat d4/d14/fc1 x:0 0 0 2024-07-16T09:55:39.867 INFO:tasks.workunit.client.1.smithi176.stdout:8/537: dread - d8/d20/d2a/d53/fb7 zero size 2024-07-16T09:55:39.868 INFO:tasks.workunit.client.1.smithi176.stdout:2/531: unlink dd/d16/d39/d3d/f80 0 2024-07-16T09:55:39.869 INFO:tasks.workunit.client.1.smithi176.stdout:2/532: read - dd/f77 zero size 2024-07-16T09:55:39.869 INFO:tasks.workunit.client.0.smithi156.stdout:9/503: dwrite d2/d5/d16/f9d [0,4194304] 0 2024-07-16T09:55:39.869 INFO:tasks.workunit.client.0.smithi156.stdout:4/472: rename d4/d23/d52/d71/f77 to d4/d23/d52/d71/fa7 0 2024-07-16T09:55:39.870 INFO:tasks.workunit.client.1.smithi176.stdout:6/508: rename d1/dc/d5b/fa8 to d1/d5d/d9e/d8f/fc0 0 2024-07-16T09:55:39.870 INFO:tasks.workunit.client.1.smithi176.stdout:8/538: write d8/d23/d2c/d34/d62/fb1 [586915,130943] 0 2024-07-16T09:55:39.871 INFO:tasks.workunit.client.1.smithi176.stdout:2/533: chown dd/d16/d39/d3d/c57 19609 1 2024-07-16T09:55:39.872 INFO:tasks.workunit.client.0.smithi156.stdout:4/473: readlink d4/db/dc/l4f 0 2024-07-16T09:55:39.872 INFO:tasks.workunit.client.1.smithi176.stdout:2/534: stat dd/d49/c9d 0 2024-07-16T09:55:39.873 INFO:tasks.workunit.client.1.smithi176.stdout:7/523: rename d4/d1f/d22/l5e to d4/d1f/d22/lac 0 2024-07-16T09:55:39.874 INFO:tasks.workunit.client.0.smithi156.stdout:9/504: fdatasync d2/d5/d16/d22/d36/f59 0 2024-07-16T09:55:39.874 INFO:tasks.workunit.client.0.smithi156.stdout:7/386: dwrite dc/d3e/d3f/d45/f4f [0,4194304] 0 2024-07-16T09:55:39.879 INFO:tasks.workunit.client.0.smithi156.stdout:4/474: dread - d4/db/dc/d7f/f99 zero size 2024-07-16T09:55:39.879 INFO:tasks.workunit.client.1.smithi176.stdout:2/535: write dd/d2e/d35/d59/d66/f6e [313971,122715] 0 2024-07-16T09:55:39.879 INFO:tasks.workunit.client.1.smithi176.stdout:5/529: symlink d4/da9/lc2 0 2024-07-16T09:55:39.880 INFO:tasks.workunit.client.1.smithi176.stdout:8/539: dwrite d8/d23/d2c/d34/d62/fa4 [0,4194304] 0 2024-07-16T09:55:39.881 INFO:tasks.workunit.client.1.smithi176.stdout:5/530: chown d4/d37/d72/d38/f87 359149 1 2024-07-16T09:55:39.882 INFO:tasks.workunit.client.1.smithi176.stdout:6/509: rename d1/f31 to d1/d5d/d9e/fc1 0 2024-07-16T09:55:39.883 INFO:tasks.workunit.client.0.smithi156.stdout:4/475: symlink d4/db/la8 0 2024-07-16T09:55:39.884 INFO:tasks.workunit.client.1.smithi176.stdout:6/510: fsync d1/db3/f95 0 2024-07-16T09:55:39.885 INFO:tasks.workunit.client.0.smithi156.stdout:9/505: dwrite d2/d5/d16/d22/d36/d33/d23/f60 [0,4194304] 0 2024-07-16T09:55:39.886 INFO:tasks.workunit.client.0.smithi156.stdout:7/387: getdents dc/d3e/d3f/d45 0 2024-07-16T09:55:39.887 INFO:tasks.workunit.client.0.smithi156.stdout:4/476: mkdir d4/d6a/da9 0 2024-07-16T09:55:39.893 INFO:tasks.workunit.client.0.smithi156.stdout:4/477: creat d4/faa x:0 0 0 2024-07-16T09:55:39.894 INFO:tasks.workunit.client.1.smithi176.stdout:8/540: creat d8/d14/d1e/d33/d9d/fb8 x:0 0 0 2024-07-16T09:55:39.895 INFO:tasks.workunit.client.0.smithi156.stdout:7/388: mknod dc/d3e/c86 0 2024-07-16T09:55:39.896 INFO:tasks.workunit.client.0.smithi156.stdout:7/389: read fb [656484,99035] 0 2024-07-16T09:55:39.896 INFO:tasks.workunit.client.1.smithi176.stdout:7/524: rename d4/d47/d5d/d9a/fa9 to d4/d47/d5d/fad 0 2024-07-16T09:55:39.898 INFO:tasks.workunit.client.0.smithi156.stdout:4/478: symlink d4/db/d47/d63/d58/d8f/lab 0 2024-07-16T09:55:39.898 INFO:tasks.workunit.client.0.smithi156.stdout:7/390: write dc/f42 [734539,30070] 0 2024-07-16T09:55:39.899 INFO:tasks.workunit.client.0.smithi156.stdout:9/506: read d2/d5/d16/d22/d2b/d3d/d3c/d7f/f2f [1965119,101671] 0 2024-07-16T09:55:39.902 INFO:tasks.workunit.client.0.smithi156.stdout:4/479: creat d4/d23/d52/fac x:0 0 0 2024-07-16T09:55:39.902 INFO:tasks.workunit.client.0.smithi156.stdout:7/391: read dc/d3e/d3f/d45/f7d [1525271,34394] 0 2024-07-16T09:55:39.903 INFO:tasks.workunit.client.0.smithi156.stdout:9/507: creat d2/d5/d16/d22/d2b/d74/f9e x:0 0 0 2024-07-16T09:55:39.904 INFO:tasks.workunit.client.0.smithi156.stdout:4/480: write d4/d23/d30/f46 [389545,117858] 0 2024-07-16T09:55:39.904 INFO:tasks.workunit.client.0.smithi156.stdout:7/392: symlink dc/d63/l87 0 2024-07-16T09:55:39.905 INFO:tasks.workunit.client.0.smithi156.stdout:9/508: stat d2/d5/f70 0 2024-07-16T09:55:39.907 INFO:tasks.workunit.client.1.smithi176.stdout:7/525: truncate d4/dd/d63/f64 1293872 0 2024-07-16T09:55:39.907 INFO:tasks.workunit.client.1.smithi176.stdout:5/531: dread d4/d37/d72/d38/d39/f56 [0,4194304] 0 2024-07-16T09:55:39.908 INFO:tasks.workunit.client.0.smithi156.stdout:4/481: dread d4/d23/f39 [0,4194304] 0 2024-07-16T09:55:39.908 INFO:tasks.workunit.client.1.smithi176.stdout:5/532: dread - d4/d10/d17/f8a zero size 2024-07-16T09:55:39.911 INFO:tasks.workunit.client.0.smithi156.stdout:4/482: creat d4/db/dc/d5f/fad x:0 0 0 2024-07-16T09:55:39.912 INFO:tasks.workunit.client.1.smithi176.stdout:6/511: rename d1/d5/d7/f52 to d1/dc/d40/d71/fc2 0 2024-07-16T09:55:39.912 INFO:tasks.workunit.client.0.smithi156.stdout:9/509: link d2/d5/d16/d22/d2b/d3d/d3c/d7f/d42/f4c d2/d5/d61/f9f 0 2024-07-16T09:55:39.917 INFO:tasks.workunit.client.1.smithi176.stdout:6/512: creat d1/dc/d72/d6f/d7c/fc3 x:0 0 0 2024-07-16T09:55:39.918 INFO:tasks.workunit.client.1.smithi176.stdout:2/536: dread dd/d16/d39/f4f [0,4194304] 0 2024-07-16T09:55:39.918 INFO:tasks.workunit.client.1.smithi176.stdout:7/526: symlink d4/d47/d6d/d89/d7e/d87/lae 0 2024-07-16T09:55:39.920 INFO:tasks.workunit.client.1.smithi176.stdout:6/513: symlink d1/lc4 0 2024-07-16T09:55:39.921 INFO:tasks.workunit.client.1.smithi176.stdout:8/541: truncate d8/d23/f21 27098 0 2024-07-16T09:55:39.922 INFO:tasks.workunit.client.0.smithi156.stdout:4/483: rename d4/db/d47/d63/d58/d8f/l9f to d4/db/dc/lae 0 2024-07-16T09:55:39.922 INFO:tasks.workunit.client.0.smithi156.stdout:7/393: truncate dc/d13/d36/f6a 4186240 0 2024-07-16T09:55:39.923 INFO:tasks.workunit.client.1.smithi176.stdout:8/542: write d8/d14/d1e/d55/f8a [3585775,87175] 0 2024-07-16T09:55:39.923 INFO:tasks.workunit.client.0.smithi156.stdout:7/394: stat dc/d3e/d3f/d45/l49 0 2024-07-16T09:55:39.928 INFO:tasks.workunit.client.1.smithi176.stdout:5/533: dwrite d4/d37/d72/f4f [0,4194304] 0 2024-07-16T09:55:39.928 INFO:tasks.workunit.client.0.smithi156.stdout:4/484: dread d4/d23/d52/f66 [0,4194304] 0 2024-07-16T09:55:39.929 INFO:tasks.workunit.client.1.smithi176.stdout:5/534: readlink d4/d14/d19/l48 0 2024-07-16T09:55:39.931 INFO:tasks.workunit.client.1.smithi176.stdout:6/514: symlink d1/dc/d81/lc5 0 2024-07-16T09:55:39.931 INFO:tasks.workunit.client.1.smithi176.stdout:2/537: link dd/de/l42 dd/d2e/lad 0 2024-07-16T09:55:39.931 INFO:tasks.workunit.client.0.smithi156.stdout:7/395: dwrite dc/d3e/d5d/f80 [0,4194304] 0 2024-07-16T09:55:39.932 INFO:tasks.workunit.client.0.smithi156.stdout:9/510: dwrite d2/d5/f44 [0,4194304] 0 2024-07-16T09:55:39.932 INFO:tasks.workunit.client.1.smithi176.stdout:8/543: mkdir d8/d20/d2a/db9 0 2024-07-16T09:55:39.932 INFO:tasks.workunit.client.0.smithi156.stdout:4/485: dread - d4/db/dc/d7f/d87/fa0 zero size 2024-07-16T09:55:39.936 INFO:tasks.workunit.client.1.smithi176.stdout:6/515: truncate d1/dc/d72/d6f/f96 553928 0 2024-07-16T09:55:39.938 INFO:tasks.workunit.client.1.smithi176.stdout:8/544: write d8/d23/d2c/d34/d62/fb1 [1391619,113716] 0 2024-07-16T09:55:39.941 INFO:tasks.workunit.client.0.smithi156.stdout:4/486: mkdir d4/db/dc/d7f/daf 0 2024-07-16T09:55:39.941 INFO:tasks.workunit.client.0.smithi156.stdout:7/396: creat dc/d63/f88 x:0 0 0 2024-07-16T09:55:39.942 INFO:tasks.workunit.client.0.smithi156.stdout:7/397: fdatasync dc/d48/f4b 0 2024-07-16T09:55:39.943 INFO:tasks.workunit.client.0.smithi156.stdout:4/487: dread - d4/d23/d30/d33/f83 zero size 2024-07-16T09:55:39.944 INFO:tasks.workunit.client.0.smithi156.stdout:7/398: dread - dc/d3e/d3f/d51/d6c/f85 zero size 2024-07-16T09:55:39.944 INFO:tasks.workunit.client.1.smithi176.stdout:6/516: truncate d1/d5/d47/fac 183662 0 2024-07-16T09:55:39.944 INFO:tasks.workunit.client.0.smithi156.stdout:7/399: chown dc/d3e/c86 426047066 1 2024-07-16T09:55:39.946 INFO:tasks.workunit.client.1.smithi176.stdout:7/527: dread d4/d5/f6c [0,4194304] 0 2024-07-16T09:55:39.947 INFO:tasks.workunit.client.0.smithi156.stdout:7/400: symlink dc/d13/d24/l89 0 2024-07-16T09:55:39.947 INFO:tasks.workunit.client.0.smithi156.stdout:4/488: fdatasync d4/db/f18 0 2024-07-16T09:55:39.948 INFO:tasks.workunit.client.0.smithi156.stdout:4/489: fsync d4/f9 0 2024-07-16T09:55:39.948 INFO:tasks.workunit.client.0.smithi156.stdout:7/401: mkdir dc/d3e/d3f/d51/d8a 0 2024-07-16T09:55:39.949 INFO:tasks.workunit.client.1.smithi176.stdout:8/545: symlink d8/d14/d1e/d33/d9d/dad/lba 0 2024-07-16T09:55:39.949 INFO:tasks.workunit.client.0.smithi156.stdout:4/490: fdatasync d4/db/d47/d63/d58/f95 0 2024-07-16T09:55:39.950 INFO:tasks.workunit.client.1.smithi176.stdout:7/528: mknod d4/dd/d26/caf 0 2024-07-16T09:55:39.952 INFO:tasks.workunit.client.1.smithi176.stdout:6/517: link d1/d5d/d9e/d76/f5c d1/d5/d47/d62/fc6 0 2024-07-16T09:55:39.954 INFO:tasks.workunit.client.1.smithi176.stdout:6/518: write d1/d5/d7/d1e/f66 [55597,54926] 0 2024-07-16T09:55:39.955 INFO:tasks.workunit.client.0.smithi156.stdout:4/491: link d4/db/d47/d63/c49 d4/db/dc/d7f/cb0 0 2024-07-16T09:55:39.956 INFO:tasks.workunit.client.0.smithi156.stdout:7/402: dwrite dc/f40 [0,4194304] 0 2024-07-16T09:55:39.957 INFO:tasks.workunit.client.1.smithi176.stdout:6/519: chown d1/dc/d5b/l4e 1 1 2024-07-16T09:55:39.958 INFO:tasks.workunit.client.0.smithi156.stdout:7/403: write dc/d13/d36/d56/f61 [144437,69829] 0 2024-07-16T09:55:39.958 INFO:tasks.workunit.client.1.smithi176.stdout:8/546: link d8/d14/d1e/d55/f6b d8/d20/d2a/d53/d93/fbb 0 2024-07-16T09:55:39.960 INFO:tasks.workunit.client.0.smithi156.stdout:7/404: stat dc/d3e/d3f/c55 0 2024-07-16T09:55:39.961 INFO:tasks.workunit.client.1.smithi176.stdout:6/520: symlink d1/d5d/d9e/d8f/lc7 0 2024-07-16T09:55:39.961 INFO:tasks.workunit.client.1.smithi176.stdout:7/529: dwrite d4/d5/f6c [0,4194304] 0 2024-07-16T09:55:39.962 INFO:tasks.workunit.client.0.smithi156.stdout:7/405: mkdir dc/d13/d15/d8b 0 2024-07-16T09:55:39.964 INFO:tasks.workunit.client.1.smithi176.stdout:6/521: write d1/dc/f17 [3787802,124442] 0 2024-07-16T09:55:39.964 INFO:tasks.workunit.client.1.smithi176.stdout:8/547: mknod d8/d20/d2a/d31/d46/d6a/da8/cbc 0 2024-07-16T09:55:39.965 INFO:tasks.workunit.client.0.smithi156.stdout:7/406: rmdir dc/d13/d36/d56 39 2024-07-16T09:55:39.965 INFO:tasks.workunit.client.1.smithi176.stdout:5/535: dread d4/d10/d17/f50 [0,4194304] 0 2024-07-16T09:55:39.970 INFO:tasks.workunit.client.1.smithi176.stdout:7/530: truncate d4/d1f/d22/d80/d17/d1d/f73 157170 0 2024-07-16T09:55:39.970 INFO:tasks.workunit.client.1.smithi176.stdout:7/531: chown d4/l56 3 1 2024-07-16T09:55:39.973 INFO:tasks.workunit.client.1.smithi176.stdout:6/522: symlink d1/d5/d47/d75/lc8 0 2024-07-16T09:55:39.974 INFO:tasks.workunit.client.1.smithi176.stdout:7/532: write d4/d1f/d22/d80/d17/d1d/f5b [1759863,114186] 0 2024-07-16T09:55:39.974 INFO:tasks.workunit.client.1.smithi176.stdout:6/523: readlink d1/d5/d7/d1e/l4a 0 2024-07-16T09:55:39.975 INFO:tasks.workunit.client.1.smithi176.stdout:6/524: chown d1/d5d 1055351520 1 2024-07-16T09:55:39.976 INFO:tasks.workunit.client.1.smithi176.stdout:6/525: fsync d1/d5d/d9e/d76/f7b 0 2024-07-16T09:55:39.977 INFO:tasks.workunit.client.0.smithi156.stdout:7/407: dread dc/d3e/d3f/d45/f7d [0,4194304] 0 2024-07-16T09:55:39.980 INFO:tasks.workunit.client.1.smithi176.stdout:6/526: mknod d1/d61/cc9 0 2024-07-16T09:55:39.981 INFO:tasks.workunit.client.1.smithi176.stdout:6/527: fdatasync d1/d5d/d9e/faa 0 2024-07-16T09:55:39.983 INFO:tasks.workunit.client.0.smithi156.stdout:9/511: dread d2/d5/d16/d22/d2b/d3d/d3c/d7f/f19 [4194304,4194304] 0 2024-07-16T09:55:39.983 INFO:tasks.workunit.client.1.smithi176.stdout:5/536: dwrite d4/d37/d72/d38/d39/d9f/fa6 [0,4194304] 0 2024-07-16T09:55:39.985 INFO:tasks.workunit.client.1.smithi176.stdout:5/537: unlink d4/caa 0 2024-07-16T09:55:39.987 INFO:tasks.workunit.client.1.smithi176.stdout:5/538: chown d4/d37/d88 354781810 1 2024-07-16T09:55:39.989 INFO:tasks.workunit.client.0.smithi156.stdout:9/512: truncate d2/d5/d16/d22/d2b/d3d/d3c/d7f/d42/f4c 781536 0 2024-07-16T09:55:39.992 INFO:tasks.workunit.client.0.smithi156.stdout:9/513: unlink d2/d5/d16/d22/d36/d33/c1f 0 2024-07-16T09:55:39.993 INFO:tasks.workunit.client.1.smithi176.stdout:5/539: link d4/d14/d19/d1b/f26 d4/d14/d19/d1b/dab/fc3 0 2024-07-16T09:55:39.995 INFO:tasks.workunit.client.0.smithi156.stdout:7/408: truncate dc/d3e/d3f/d45/f7d 3225989 0 2024-07-16T09:55:39.995 INFO:tasks.workunit.client.0.smithi156.stdout:4/492: dread d4/d23/f17 [0,4194304] 0 2024-07-16T09:55:39.995 INFO:tasks.workunit.client.1.smithi176.stdout:2/538: dread dd/d2e/d35/d59/d66/f8f [0,4194304] 0 2024-07-16T09:55:39.996 INFO:tasks.workunit.client.0.smithi156.stdout:9/514: mkdir d2/d5/d16/d22/d36/d33/d23/da0 0 2024-07-16T09:55:39.997 INFO:tasks.workunit.client.1.smithi176.stdout:5/540: mkdir d4/da9/dc4 0 2024-07-16T09:55:39.997 INFO:tasks.workunit.client.0.smithi156.stdout:4/493: mkdir d4/db/dc/d7f/db1 0 2024-07-16T09:55:39.998 INFO:tasks.workunit.client.0.smithi156.stdout:9/515: chown d2/d5/d16/d22/d36/d33/c46 2795 1 2024-07-16T09:55:39.998 INFO:tasks.workunit.client.0.smithi156.stdout:4/494: symlink d4/lb2 0 2024-07-16T09:55:39.999 INFO:tasks.workunit.client.0.smithi156.stdout:9/516: rmdir d2/d5 39 2024-07-16T09:55:40.000 INFO:tasks.workunit.client.1.smithi176.stdout:2/539: symlink dd/d49/d52/d7c/d81/lae 0 2024-07-16T09:55:40.000 INFO:tasks.workunit.client.1.smithi176.stdout:2/540: readlink dd/d49/l7b 0 2024-07-16T09:55:40.001 INFO:tasks.workunit.client.0.smithi156.stdout:9/517: write d2/d5/d16/d22/d36/f78 [479223,82093] 0 2024-07-16T09:55:40.002 INFO:tasks.workunit.client.0.smithi156.stdout:4/495: rename d4/db/d47/d63/d58/f95 to d4/db/dc/d5f/fb3 0 2024-07-16T09:55:40.003 INFO:tasks.workunit.client.0.smithi156.stdout:9/518: write d2/d5/d16/d22/f7c [4784578,124483] 0 2024-07-16T09:55:40.003 INFO:tasks.workunit.client.0.smithi156.stdout:4/496: dread - d4/d23/d30/d33/f83 zero size 2024-07-16T09:55:40.006 INFO:tasks.workunit.client.1.smithi176.stdout:7/533: write d4/dd/f15 [1580565,125946] 0 2024-07-16T09:55:40.006 INFO:tasks.workunit.client.0.smithi156.stdout:9/519: dread d2/d5/d16/d22/d36/f59 [0,4194304] 0 2024-07-16T09:55:40.008 INFO:tasks.workunit.client.0.smithi156.stdout:9/520: write d2/d5/d16/f9d [2532376,118326] 0 2024-07-16T09:55:40.008 INFO:tasks.workunit.client.0.smithi156.stdout:4/497: rename d4/db/dc/l15 to d4/db/dc/d7f/lb4 0 2024-07-16T09:55:40.011 INFO:tasks.workunit.client.0.smithi156.stdout:4/498: creat d4/db/dc/d7f/daf/fb5 x:0 0 0 2024-07-16T09:55:40.012 INFO:tasks.workunit.client.1.smithi176.stdout:2/541: truncate dd/de/f10 148140 0 2024-07-16T09:55:40.013 INFO:tasks.workunit.client.1.smithi176.stdout:5/541: write d4/d14/d19/d1b/f26 [335041,75886] 0 2024-07-16T09:55:40.014 INFO:tasks.workunit.client.1.smithi176.stdout:7/534: dread d4/d5/f6c [0,4194304] 0 2024-07-16T09:55:40.015 INFO:tasks.workunit.client.1.smithi176.stdout:6/528: dwrite d1/dc/f33 [0,4194304] 0 2024-07-16T09:55:40.016 INFO:tasks.workunit.client.1.smithi176.stdout:8/548: dread d8/d14/d1e/d55/f8a [0,4194304] 0 2024-07-16T09:55:40.017 INFO:tasks.workunit.client.0.smithi156.stdout:9/521: link d2/d5/d16/d22/d36/d33/d3a/c83 d2/d5/d16/d22/d6b/d86/ca1 0 2024-07-16T09:55:40.018 INFO:tasks.workunit.client.1.smithi176.stdout:5/542: write f2 [322259,20674] 0 2024-07-16T09:55:40.019 INFO:tasks.workunit.client.1.smithi176.stdout:7/535: rmdir d4/d1f/d22/d80/d17/d1d 39 2024-07-16T09:55:40.022 INFO:tasks.workunit.client.1.smithi176.stdout:7/536: rmdir d4 39 2024-07-16T09:55:40.023 INFO:tasks.workunit.client.1.smithi176.stdout:8/549: dread d8/d20/d2a/f49 [4194304,4194304] 0 2024-07-16T09:55:40.024 INFO:tasks.workunit.client.1.smithi176.stdout:2/542: rename dd/d49/d52/d89/la5 to dd/d36/d82/laf 0 2024-07-16T09:55:40.024 INFO:tasks.workunit.client.1.smithi176.stdout:5/543: mkdir d4/d14/dc5 0 2024-07-16T09:55:40.025 INFO:tasks.workunit.client.1.smithi176.stdout:2/543: read - dd/d36/f94 zero size 2024-07-16T09:55:40.026 INFO:tasks.workunit.client.0.smithi156.stdout:9/522: rename d2/d5/d16/d22/d36/d6a to d2/d5/d61/da2 0 2024-07-16T09:55:40.026 INFO:tasks.workunit.client.1.smithi176.stdout:8/550: creat d8/d23/d76/d83/fbd x:0 0 0 2024-07-16T09:55:40.028 INFO:tasks.workunit.client.1.smithi176.stdout:6/529: getdents d1/d5 0 2024-07-16T09:55:40.029 INFO:tasks.workunit.client.1.smithi176.stdout:2/544: write dd/d49/d52/f79 [1978985,77780] 0 2024-07-16T09:55:40.030 INFO:tasks.workunit.client.1.smithi176.stdout:8/551: unlink d8/l1d 0 2024-07-16T09:55:40.030 INFO:tasks.workunit.client.0.smithi156.stdout:9/523: mkdir d2/d5/d16/d57/da3 0 2024-07-16T09:55:40.030 INFO:tasks.workunit.client.1.smithi176.stdout:8/552: fsync d8/d23/d76/f98 0 2024-07-16T09:55:40.031 INFO:tasks.workunit.client.1.smithi176.stdout:7/537: rmdir d4/d47/d6d/d89/d7e/d8d 0 2024-07-16T09:55:40.031 INFO:tasks.workunit.client.1.smithi176.stdout:5/544: read d4/d14/f2c [7614458,104533] 0 2024-07-16T09:55:40.032 INFO:tasks.workunit.client.0.smithi156.stdout:9/524: mknod d2/d5/d16/d57/ca4 0 2024-07-16T09:55:40.037 INFO:tasks.workunit.client.0.smithi156.stdout:9/525: link d2/d5/d16/d22/d36/d33/c46 d2/d5/d16/d22/d36/ca5 0 2024-07-16T09:55:40.040 INFO:tasks.workunit.client.1.smithi176.stdout:6/530: dwrite d1/d5d/db9/f85 [0,4194304] 0 2024-07-16T09:55:40.040 INFO:tasks.workunit.client.1.smithi176.stdout:8/553: write d8/d20/d2a/d31/f5c [983248,31240] 0 2024-07-16T09:55:40.042 INFO:tasks.workunit.client.0.smithi156.stdout:9/526: mkdir d2/d5/d16/d22/d2b/d3d/da6 0 2024-07-16T09:55:40.043 INFO:tasks.workunit.client.1.smithi176.stdout:5/545: truncate d4/d10/f54 3514516 0 2024-07-16T09:55:40.043 INFO:tasks.workunit.client.1.smithi176.stdout:6/531: write d1/dc/d5b/f6d [819702,38791] 0 2024-07-16T09:55:40.044 INFO:tasks.workunit.client.1.smithi176.stdout:2/545: dread dd/d16/d39/f6d [0,4194304] 0 2024-07-16T09:55:40.045 INFO:tasks.workunit.client.0.smithi156.stdout:7/409: dread dc/f42 [0,4194304] 0 2024-07-16T09:55:40.045 INFO:tasks.workunit.client.1.smithi176.stdout:8/554: creat d8/d14/d1e/d7c/fbe x:0 0 0 2024-07-16T09:55:40.045 INFO:tasks.workunit.client.1.smithi176.stdout:6/532: fsync d1/d5d/d9e/faa 0 2024-07-16T09:55:40.047 INFO:tasks.workunit.client.1.smithi176.stdout:5/546: rename d4/c34 to d4/d14/d3d/cc6 0 2024-07-16T09:55:40.047 INFO:tasks.workunit.client.1.smithi176.stdout:6/533: chown d1/d5/d47/d62/d77 222304 1 2024-07-16T09:55:40.047 INFO:tasks.workunit.client.1.smithi176.stdout:7/538: link d4/d47/d6d/d89/l69 d4/d1f/d22/lb0 0 2024-07-16T09:55:40.047 INFO:tasks.workunit.client.1.smithi176.stdout:6/534: chown d1/l5f 144886 1 2024-07-16T09:55:40.049 INFO:tasks.workunit.client.0.smithi156.stdout:9/527: truncate d2/d5/ff 2809281 0 2024-07-16T09:55:40.050 INFO:tasks.workunit.client.1.smithi176.stdout:2/546: symlink dd/d36/d45/lb0 0 2024-07-16T09:55:40.050 INFO:tasks.workunit.client.1.smithi176.stdout:8/555: mknod d8/d14/cbf 0 2024-07-16T09:55:40.052 INFO:tasks.workunit.client.1.smithi176.stdout:6/535: unlink d1/d5/c25 0 2024-07-16T09:55:40.053 INFO:tasks.workunit.client.1.smithi176.stdout:5/547: mkdir d4/d14/d19/d1b/dab/dc7 0 2024-07-16T09:55:40.058 INFO:tasks.workunit.client.1.smithi176.stdout:5/548: mknod d4/d10/d17/cc8 0 2024-07-16T09:55:40.058 INFO:tasks.workunit.client.0.smithi156.stdout:7/410: dread dc/d13/f43 [0,4194304] 0 2024-07-16T09:55:40.059 INFO:tasks.workunit.client.1.smithi176.stdout:7/539: dwrite d4/d47/d6d/f6e [0,4194304] 0 2024-07-16T09:55:40.060 INFO:tasks.workunit.client.1.smithi176.stdout:5/549: chown d4/d37/d72/d82/f83 99471593 1 2024-07-16T09:55:40.060 INFO:tasks.workunit.client.0.smithi156.stdout:9/528: rename d2/d5/d16/d22/d2b/d3d/d3c/d7f/d42/f4c to d2/d5/d61/da2/d98/fa7 0 2024-07-16T09:55:40.062 INFO:tasks.workunit.client.0.smithi156.stdout:9/529: symlink d2/d5/d16/d22/d6b/la8 0 2024-07-16T09:55:40.063 INFO:tasks.workunit.client.0.smithi156.stdout:7/411: symlink dc/l8c 0 2024-07-16T09:55:40.064 INFO:tasks.workunit.client.0.smithi156.stdout:9/530: truncate d2/d5/d16/f37 1033087 0 2024-07-16T09:55:40.065 INFO:tasks.workunit.client.0.smithi156.stdout:4/499: dread d4/db/f1d [0,4194304] 0 2024-07-16T09:55:40.065 INFO:tasks.workunit.client.0.smithi156.stdout:9/531: read - d2/d5/d16/d22/d6b/f92 zero size 2024-07-16T09:55:40.069 INFO:tasks.workunit.client.1.smithi176.stdout:8/556: write d8/d23/d2c/f36 [1908407,110904] 0 2024-07-16T09:55:40.070 INFO:tasks.workunit.client.0.smithi156.stdout:9/532: write d2/d5/d16/d22/d2b/d3d/d3c/d7f/f2f [3260300,87541] 0 2024-07-16T09:55:40.070 INFO:tasks.workunit.client.1.smithi176.stdout:7/540: mkdir d4/dd/d49/d7a/db1 0 2024-07-16T09:55:40.072 INFO:tasks.workunit.client.0.smithi156.stdout:4/500: rmdir d4/db/dc/d5f/d97 0 2024-07-16T09:55:40.072 INFO:tasks.workunit.client.1.smithi176.stdout:6/536: dwrite d1/dc/fd [0,4194304] 0 2024-07-16T09:55:40.074 INFO:tasks.workunit.client.0.smithi156.stdout:9/533: creat d2/d5/d61/da2/d98/fa9 x:0 0 0 2024-07-16T09:55:40.074 INFO:tasks.workunit.client.1.smithi176.stdout:2/547: dwrite dd/d16/d39/d3d/f87 [0,4194304] 0 2024-07-16T09:55:40.074 INFO:tasks.workunit.client.1.smithi176.stdout:5/550: getdents d4/d14/d3d 0 2024-07-16T09:55:40.075 INFO:tasks.workunit.client.0.smithi156.stdout:4/501: symlink d4/db/dc/d7f/daf/lb6 0 2024-07-16T09:55:40.075 INFO:tasks.workunit.client.0.smithi156.stdout:6/455: sync 2024-07-16T09:55:40.076 INFO:tasks.workunit.client.1.smithi176.stdout:5/551: write d4/fd [4748467,70579] 0 2024-07-16T09:55:40.080 INFO:tasks.workunit.client.1.smithi176.stdout:5/552: readlink d4/d14/l2a 0 2024-07-16T09:55:40.081 INFO:tasks.workunit.client.1.smithi176.stdout:2/548: write dd/d16/d39/f4f [5231451,20324] 0 2024-07-16T09:55:40.090 INFO:tasks.workunit.client.1.smithi176.stdout:8/557: dwrite d8/d20/d2a/f75 [0,4194304] 0 2024-07-16T09:55:40.090 INFO:tasks.workunit.client.1.smithi176.stdout:6/537: dwrite d1/d5d/d9e/f82 [0,4194304] 0 2024-07-16T09:55:40.093 INFO:tasks.workunit.client.1.smithi176.stdout:7/541: dwrite d4/d5/f6c [0,4194304] 0 2024-07-16T09:55:40.093 INFO:tasks.workunit.client.1.smithi176.stdout:2/549: write dd/d36/f61 [145026,60620] 0 2024-07-16T09:55:40.098 INFO:tasks.workunit.client.1.smithi176.stdout:2/550: creat dd/d9b/fb1 x:0 0 0 2024-07-16T09:55:40.099 INFO:tasks.workunit.client.1.smithi176.stdout:7/542: mknod d4/d47/d5d/cb2 0 2024-07-16T09:55:40.099 INFO:tasks.workunit.client.1.smithi176.stdout:8/558: read d8/d20/f5a [1643717,69899] 0 2024-07-16T09:55:40.099 INFO:tasks.workunit.client.1.smithi176.stdout:2/551: dread - dd/de/fa8 zero size 2024-07-16T09:55:40.099 INFO:tasks.workunit.client.1.smithi176.stdout:8/559: chown d8/d20/d2a/f49 93 1 2024-07-16T09:55:40.100 INFO:tasks.workunit.client.1.smithi176.stdout:7/543: unlink f0 0 2024-07-16T09:55:40.101 INFO:tasks.workunit.client.1.smithi176.stdout:7/544: readlink d4/dd/l35 0 2024-07-16T09:55:40.102 INFO:tasks.workunit.client.1.smithi176.stdout:7/545: read - d4/d1f/f62 zero size 2024-07-16T09:55:40.102 INFO:tasks.workunit.client.1.smithi176.stdout:2/552: chown dd/d49/d52/d7c/d81/c9c 32 1 2024-07-16T09:55:40.105 INFO:tasks.workunit.client.1.smithi176.stdout:5/553: write d4/d14/d3d/f41 [4226661,100990] 0 2024-07-16T09:55:40.106 INFO:tasks.workunit.client.1.smithi176.stdout:7/546: unlink d4/d47/d6d/d89/d66/la5 0 2024-07-16T09:55:40.108 INFO:tasks.workunit.client.1.smithi176.stdout:2/553: creat dd/d36/fb2 x:0 0 0 2024-07-16T09:55:40.108 INFO:tasks.workunit.client.1.smithi176.stdout:5/554: symlink d4/d9a/lc9 0 2024-07-16T09:55:40.109 INFO:tasks.workunit.client.0.smithi156.stdout:9/534: chown d2/d5/d61/da2 1265 1 2024-07-16T09:55:40.110 INFO:tasks.workunit.client.1.smithi176.stdout:2/554: unlink c8 0 2024-07-16T09:55:40.111 INFO:tasks.workunit.client.0.smithi156.stdout:6/456: write d3/d13/d17/f55 [338990,12163] 0 2024-07-16T09:55:40.112 INFO:tasks.workunit.client.1.smithi176.stdout:7/547: creat d4/fb3 x:0 0 0 2024-07-16T09:55:40.113 INFO:tasks.workunit.client.1.smithi176.stdout:2/555: symlink dd/d49/d52/d7c/d81/lb3 0 2024-07-16T09:55:40.114 INFO:tasks.workunit.client.0.smithi156.stdout:6/457: creat d3/d13/d17/d4f/d1e/f98 x:0 0 0 2024-07-16T09:55:40.115 INFO:tasks.workunit.client.0.smithi156.stdout:6/458: read - d3/d13/d17/f8a zero size 2024-07-16T09:55:40.116 INFO:tasks.workunit.client.1.smithi176.stdout:7/548: symlink d4/d47/d6d/d89/d7e/d87/lb4 0 2024-07-16T09:55:40.121 INFO:tasks.workunit.client.1.smithi176.stdout:7/549: read d4/d47/f88 [431849,83875] 0 2024-07-16T09:55:40.121 INFO:tasks.workunit.client.1.smithi176.stdout:2/556: dwrite dd/d2e/f83 [4194304,4194304] 0 2024-07-16T09:55:40.124 INFO:tasks.workunit.client.1.smithi176.stdout:7/550: dread - d4/fb3 zero size 2024-07-16T09:55:40.125 INFO:tasks.workunit.client.1.smithi176.stdout:7/551: mkdir d4/dd/db5 0 2024-07-16T09:55:40.129 INFO:tasks.workunit.client.1.smithi176.stdout:7/552: fdatasync d4/d1f/d22/d80/d17/f7f 0 2024-07-16T09:55:40.130 INFO:tasks.workunit.client.1.smithi176.stdout:2/557: dwrite dd/d2e/d35/d59/d66/f6e [0,4194304] 0 2024-07-16T09:55:40.132 INFO:tasks.workunit.client.1.smithi176.stdout:2/558: dread - dd/d16/d39/d3d/f76 zero size 2024-07-16T09:55:40.139 INFO:tasks.workunit.client.1.smithi176.stdout:2/559: link dd/d2e/d35/d59/d66/f6e dd/d49/d52/d7c/fb4 0 2024-07-16T09:55:40.141 INFO:tasks.workunit.client.1.smithi176.stdout:2/560: chown dd/d49/d52/d7c/d81/c9c 6 1 2024-07-16T09:55:40.144 INFO:tasks.workunit.client.1.smithi176.stdout:5/555: dread d4/d37/d72/d82/f93 [4194304,4194304] 0 2024-07-16T09:55:40.145 INFO:tasks.workunit.client.1.smithi176.stdout:5/556: chown d4/d14/d19/f21 65967967 1 2024-07-16T09:55:40.145 INFO:tasks.workunit.client.1.smithi176.stdout:7/553: dread d4/d47/d6d/d89/d66/f8a [0,4194304] 0 2024-07-16T09:55:40.146 INFO:tasks.workunit.client.1.smithi176.stdout:5/557: chown d4/d10/d17/f67 18 1 2024-07-16T09:55:40.147 INFO:tasks.workunit.client.1.smithi176.stdout:5/558: write d4/d37/d72/d38/d39/f9b [44467,128655] 0 2024-07-16T09:55:40.148 INFO:tasks.workunit.client.1.smithi176.stdout:2/561: getdents dd/d36 0 2024-07-16T09:55:40.148 INFO:tasks.workunit.client.1.smithi176.stdout:7/554: mknod d4/d5/d97/cb6 0 2024-07-16T09:55:40.150 INFO:tasks.workunit.client.1.smithi176.stdout:5/559: unlink d4/d14/d19/l5e 0 2024-07-16T09:55:40.151 INFO:tasks.workunit.client.1.smithi176.stdout:2/562: creat dd/d49/fb5 x:0 0 0 2024-07-16T09:55:40.152 INFO:tasks.workunit.client.1.smithi176.stdout:5/560: symlink d4/d14/d19/d1b/dab/dc7/lca 0 2024-07-16T09:55:40.153 INFO:tasks.workunit.client.1.smithi176.stdout:2/563: creat dd/d49/fb6 x:0 0 0 2024-07-16T09:55:40.155 INFO:tasks.workunit.client.1.smithi176.stdout:2/564: truncate dd/d36/d82/fa9 944011 0 2024-07-16T09:55:40.155 INFO:tasks.workunit.client.1.smithi176.stdout:5/561: rename d4/d14/d19/d1b/dab/dc7/lca to d4/d37/d72/d38/d39/d9f/lcb 0 2024-07-16T09:55:40.157 INFO:tasks.workunit.client.1.smithi176.stdout:5/562: symlink d4/d37/d88/lcc 0 2024-07-16T09:55:40.160 INFO:tasks.workunit.client.1.smithi176.stdout:5/563: fdatasync d4/d37/d88/f8f 0 2024-07-16T09:55:40.163 INFO:tasks.workunit.client.1.smithi176.stdout:5/564: dread - d4/d37/d72/d38/d39/fa1 zero size 2024-07-16T09:55:40.163 INFO:tasks.workunit.client.1.smithi176.stdout:7/555: dwrite d4/d47/d6d/d89/d7e/fa1 [0,4194304] 0 2024-07-16T09:55:40.164 INFO:tasks.workunit.client.1.smithi176.stdout:2/565: dwrite dd/d2e/d35/fac [0,4194304] 0 2024-07-16T09:55:40.165 INFO:tasks.workunit.client.1.smithi176.stdout:2/566: chown dd/d16/d39/d3d/c57 1193222 1 2024-07-16T09:55:40.165 INFO:tasks.workunit.client.1.smithi176.stdout:5/565: chown d4/d37/d72/d38/d6e/f74 34364167 1 2024-07-16T09:55:40.168 INFO:tasks.workunit.client.1.smithi176.stdout:2/567: write dd/d16/d39/f6d [488667,91630] 0 2024-07-16T09:55:40.169 INFO:tasks.workunit.client.0.smithi156.stdout:3/450: sync 2024-07-16T09:55:40.173 INFO:tasks.workunit.client.1.smithi176.stdout:7/556: getdents d4/dd 0 2024-07-16T09:55:40.173 INFO:tasks.workunit.client.1.smithi176.stdout:2/568: creat dd/d49/d52/fb7 x:0 0 0 2024-07-16T09:55:40.174 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:55:39 smithi156 ceph-mon[29302]: pgmap v251: 65 pgs: 65 active+clean; 1.7 GiB data, 6.5 GiB used, 530 GiB / 536 GiB avail; 42 MiB/s rd, 238 MiB/s wr, 471 op/s 2024-07-16T09:55:40.174 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:55:39 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-16T09:55:40.175 INFO:tasks.workunit.client.1.smithi176.stdout:7/557: write d4/d47/d6d/d89/d7e/fa1 [4992345,58787] 0 2024-07-16T09:55:40.175 INFO:tasks.workunit.client.1.smithi176.stdout:5/566: creat d4/d14/fcd x:0 0 0 2024-07-16T09:55:40.177 INFO:tasks.workunit.client.0.smithi156.stdout:6/459: mkdir d3/d13/d17/d4f/d30/d45/d99 0 2024-07-16T09:55:40.178 INFO:tasks.workunit.client.0.smithi156.stdout:4/502: dread d4/db/dc/f2a [0,4194304] 0 2024-07-16T09:55:40.181 INFO:tasks.workunit.client.0.smithi156.stdout:4/503: creat d4/db/d47/d63/fb7 x:0 0 0 2024-07-16T09:55:40.181 INFO:tasks.workunit.client.0.smithi156.stdout:9/535: dread d2/d5/d16/d22/d36/d33/d3a/f49 [0,4194304] 0 2024-07-16T09:55:40.183 INFO:tasks.workunit.client.1.smithi176.stdout:7/558: fdatasync d4/d47/d6d/d89/d66/fa4 0 2024-07-16T09:55:40.184 INFO:tasks.workunit.client.0.smithi156.stdout:9/536: write d2/d5/d16/d22/d2b/d3d/f68 [805122,111763] 0 2024-07-16T09:55:40.184 INFO:tasks.workunit.client.1.smithi176.stdout:2/569: dwrite dd/d16/d39/d3d/f76 [0,4194304] 0 2024-07-16T09:55:40.188 INFO:tasks.workunit.client.0.smithi156.stdout:4/504: dwrite d4/db/d47/d63/d58/fa6 [0,4194304] 0 2024-07-16T09:55:40.188 INFO:tasks.workunit.client.0.smithi156.stdout:9/537: mknod d2/d5/d61/caa 0 2024-07-16T09:55:40.188 INFO:tasks.workunit.client.1.smithi176.stdout:5/567: dwrite d4/d10/d17/f66 [0,4194304] 0 2024-07-16T09:55:40.194 INFO:tasks.workunit.client.1.smithi176.stdout:5/568: read d4/d14/d3d/f55 [2858504,37569] 0 2024-07-16T09:55:40.197 INFO:tasks.workunit.client.1.smithi176.stdout:2/570: creat dd/d2e/d35/fb8 x:0 0 0 2024-07-16T09:55:40.199 INFO:tasks.workunit.client.1.smithi176.stdout:7/559: dwrite d4/d47/d6d/d89/d66/fa4 [4194304,4194304] 0 2024-07-16T09:55:40.203 INFO:tasks.workunit.client.0.smithi156.stdout:9/538: dwrite d2/d5/d16/d22/d2b/d3d/f6d [0,4194304] 0 2024-07-16T09:55:40.205 INFO:tasks.workunit.client.1.smithi176.stdout:7/560: truncate d4/dd/d63/f64 1052088 0 2024-07-16T09:55:40.207 INFO:tasks.workunit.client.0.smithi156.stdout:4/505: dwrite d4/db/dc/d7f/d87/fa0 [0,4194304] 0 2024-07-16T09:55:40.210 INFO:tasks.workunit.client.0.smithi156.stdout:6/460: dwrite d3/d13/d17/d4f/d4d/d3b/f93 [0,4194304] 0 2024-07-16T09:55:40.212 INFO:tasks.workunit.client.1.smithi176.stdout:2/571: rename dd/d2e/d35/d59/d66/l8b to dd/d2e/d35/d4a/lb9 0 2024-07-16T09:55:40.214 INFO:tasks.workunit.client.0.smithi156.stdout:9/539: fsync d2/d5/d16/d22/d36/d33/d3a/f49 0 2024-07-16T09:55:40.214 INFO:tasks.workunit.client.0.smithi156.stdout:4/506: symlink d4/db/dc/d7f/lb8 0 2024-07-16T09:55:40.217 INFO:tasks.workunit.client.0.smithi156.stdout:3/451: dread d8/da/d13/f20 [4194304,4194304] 0 2024-07-16T09:55:40.219 INFO:tasks.workunit.client.0.smithi156.stdout:4/507: truncate d4/d23/d52/f69 942713 0 2024-07-16T09:55:40.219 INFO:tasks.workunit.client.0.smithi156.stdout:3/452: dread - d8/da/d28/d43/f5a zero size 2024-07-16T09:55:40.222 INFO:tasks.workunit.client.1.smithi176.stdout:7/561: dwrite d4/d1f/d22/f52 [0,4194304] 0 2024-07-16T09:55:40.223 INFO:tasks.workunit.client.1.smithi176.stdout:5/569: dread d4/d14/d3d/f41 [0,4194304] 0 2024-07-16T09:55:40.223 INFO:tasks.workunit.client.0.smithi156.stdout:9/540: unlink d2/d5/d16/d22/d36/d33/d3a/c83 0 2024-07-16T09:55:40.226 INFO:tasks.workunit.client.1.smithi176.stdout:7/562: chown d4/d47/f88 1189161 1 2024-07-16T09:55:40.227 INFO:tasks.workunit.client.0.smithi156.stdout:3/453: symlink d8/da/l9b 0 2024-07-16T09:55:40.227 INFO:tasks.workunit.client.1.smithi176.stdout:5/570: mkdir d4/d14/dce 0 2024-07-16T09:55:40.228 INFO:tasks.workunit.client.0.smithi156.stdout:4/508: dwrite d4/f43 [0,4194304] 0 2024-07-16T09:55:40.229 INFO:tasks.workunit.client.0.smithi156.stdout:3/454: write d8/da/d28/f6e [3722773,14654] 0 2024-07-16T09:55:40.230 INFO:tasks.workunit.client.1.smithi176.stdout:5/571: chown d4/d10/l65 3956523 1 2024-07-16T09:55:40.231 INFO:tasks.workunit.client.1.smithi176.stdout:7/563: mkdir d4/d47/d6d/db7 0 2024-07-16T09:55:40.232 INFO:tasks.workunit.client.1.smithi176.stdout:7/564: dread - d4/d5/f91 zero size 2024-07-16T09:55:40.233 INFO:tasks.workunit.client.0.smithi156.stdout:3/455: chown d8/d14/l35 1 1 2024-07-16T09:55:40.233 INFO:tasks.workunit.client.1.smithi176.stdout:7/565: readlink d4/d1f/d22/d80/l18 0 2024-07-16T09:55:40.235 INFO:tasks.workunit.client.1.smithi176.stdout:7/566: write d4/dd/d49/f55 [102196,40134] 0 2024-07-16T09:55:40.236 INFO:tasks.workunit.client.1.smithi176.stdout:9/620: sync 2024-07-16T09:55:40.239 INFO:tasks.workunit.client.0.smithi156.stdout:4/509: dwrite d4/f5 [0,4194304] 0 2024-07-16T09:55:40.240 INFO:tasks.workunit.client.1.smithi176.stdout:5/572: dwrite d4/d37/f5a [0,4194304] 0 2024-07-16T09:55:40.240 INFO:tasks.workunit.client.1.smithi176.stdout:9/621: creat d5/d1d/db0/fd5 x:0 0 0 2024-07-16T09:55:40.240 INFO:tasks.workunit.client.0.smithi156.stdout:3/456: link d8/l70 d8/da/d1d/d2c/d33/l9c 0 2024-07-16T09:55:40.241 INFO:tasks.workunit.client.0.smithi156.stdout:3/457: chown d8/da/f3d 220734913 1 2024-07-16T09:55:40.242 INFO:tasks.workunit.client.1.smithi176.stdout:9/622: dread - d5/d52/fa4 zero size 2024-07-16T09:55:40.244 INFO:tasks.workunit.client.1.smithi176.stdout:7/567: mknod d4/d5/cb8 0 2024-07-16T09:55:40.245 INFO:tasks.workunit.client.0.smithi156.stdout:4/510: creat d4/db/dc/d7f/d87/fb9 x:0 0 0 2024-07-16T09:55:40.249 INFO:tasks.workunit.client.0.smithi156.stdout:3/458: rename d8/l70 to d8/da/d1d/d6f/l9d 0 2024-07-16T09:55:40.251 INFO:tasks.workunit.client.0.smithi156.stdout:4/511: symlink d4/d23/d30/lba 0 2024-07-16T09:55:40.252 INFO:tasks.workunit.client.0.smithi156.stdout:4/512: fdatasync d4/db/dc/d67/f92 0 2024-07-16T09:55:40.252 INFO:tasks.workunit.client.0.smithi156.stdout:3/459: creat d8/da/d21/d80/f9e x:0 0 0 2024-07-16T09:55:40.253 INFO:tasks.workunit.client.0.smithi156.stdout:9/541: dread d2/d5/d16/d22/d2b/d3d/d3c/d7f/f2f [0,4194304] 0 2024-07-16T09:55:40.255 INFO:tasks.workunit.client.0.smithi156.stdout:9/542: write d2/f4d [2873261,36671] 0 2024-07-16T09:55:40.255 INFO:tasks.workunit.client.0.smithi156.stdout:6/461: dread d3/d13/d17/f6c [0,4194304] 0 2024-07-16T09:55:40.258 INFO:tasks.workunit.client.0.smithi156.stdout:3/460: symlink d8/da/d21/d80/d99/l9f 0 2024-07-16T09:55:40.261 INFO:tasks.workunit.client.0.smithi156.stdout:9/543: rename d2/d5/d16/f37 to d2/d5/d16/d22/d36/d33/d3a/fab 0 2024-07-16T09:55:40.261 INFO:tasks.workunit.client.0.smithi156.stdout:4/513: dwrite d4/d23/d52/fac [0,4194304] 0 2024-07-16T09:55:40.264 INFO:tasks.workunit.client.0.smithi156.stdout:9/544: readlink d2/d5/d16/d22/l3e 0 2024-07-16T09:55:40.264 INFO:tasks.workunit.client.0.smithi156.stdout:9/545: readlink d2/d5/d16/d57/l94 0 2024-07-16T09:55:40.268 INFO:tasks.workunit.client.0.smithi156.stdout:9/546: chown d2/d5/d16/d22/c1e 490781765 1 2024-07-16T09:55:40.269 INFO:tasks.workunit.client.0.smithi156.stdout:3/461: link d8/da/d1d/d2c/c92 d8/da/d28/d43/ca0 0 2024-07-16T09:55:40.270 INFO:tasks.workunit.client.0.smithi156.stdout:3/462: dread - d8/da/d1d/f64 zero size 2024-07-16T09:55:40.275 INFO:tasks.workunit.client.0.smithi156.stdout:3/463: creat d8/da/d1d/d2c/d7d/fa1 x:0 0 0 2024-07-16T09:55:40.276 INFO:tasks.workunit.client.0.smithi156.stdout:4/514: write d4/db/f3f [834139,79619] 0 2024-07-16T09:55:40.277 INFO:tasks.workunit.client.0.smithi156.stdout:6/462: truncate d3/d13/d17/d4f/d94/d19/d31/d35/f62 3086114 0 2024-07-16T09:55:40.277 INFO:tasks.workunit.client.0.smithi156.stdout:4/515: chown d4/db/dc/d7c 3 1 2024-07-16T09:55:40.278 INFO:tasks.workunit.client.0.smithi156.stdout:3/464: mkdir d8/da/d28/d43/d71/da2 0 2024-07-16T09:55:40.281 INFO:tasks.workunit.client.0.smithi156.stdout:3/465: chown d8/da/d1d/d2c/d33/f65 108038477 1 2024-07-16T09:55:40.287 INFO:tasks.workunit.client.0.smithi156.stdout:3/466: dread - d8/da/d28/d43/f5a zero size 2024-07-16T09:55:40.287 INFO:tasks.workunit.client.0.smithi156.stdout:9/547: dwrite d2/d5/d16/d22/d6b/d86/f8c [0,4194304] 0 2024-07-16T09:55:40.287 INFO:tasks.workunit.client.0.smithi156.stdout:3/467: write d8/d14/f6b [321000,41290] 0 2024-07-16T09:55:40.287 INFO:tasks.workunit.client.0.smithi156.stdout:4/516: dwrite d4/db/f94 [0,4194304] 0 2024-07-16T09:55:40.288 INFO:tasks.workunit.client.0.smithi156.stdout:9/548: creat d2/d5/d16/d57/fac x:0 0 0 2024-07-16T09:55:40.290 INFO:tasks.workunit.client.0.smithi156.stdout:4/517: readlink d4/db/dc/d5f/d64/l9c 0 2024-07-16T09:55:40.290 INFO:tasks.workunit.client.0.smithi156.stdout:3/468: chown d8/da/f2e 14792 1 2024-07-16T09:55:40.293 INFO:tasks.workunit.client.0.smithi156.stdout:3/469: creat d8/da/d21/d80/d99/fa3 x:0 0 0 2024-07-16T09:55:40.297 INFO:tasks.workunit.client.0.smithi156.stdout:3/470: mknod d8/da/d1d/d2c/d7d/ca4 0 2024-07-16T09:55:40.298 INFO:tasks.workunit.client.0.smithi156.stdout:4/518: dwrite d4/db/dc/d7f/d87/fa0 [0,4194304] 0 2024-07-16T09:55:40.300 INFO:tasks.workunit.client.0.smithi156.stdout:9/549: truncate d2/d5/f70 2659253 0 2024-07-16T09:55:40.301 INFO:tasks.workunit.client.0.smithi156.stdout:9/550: dread - d2/d5/f91 zero size 2024-07-16T09:55:40.302 INFO:tasks.workunit.client.0.smithi156.stdout:4/519: mknod d4/db/dc/d5f/cbb 0 2024-07-16T09:55:40.304 INFO:tasks.workunit.client.0.smithi156.stdout:3/471: getdents d8/da/d28 0 2024-07-16T09:55:40.305 INFO:tasks.workunit.client.0.smithi156.stdout:4/520: symlink d4/d23/lbc 0 2024-07-16T09:55:40.306 INFO:tasks.workunit.client.0.smithi156.stdout:4/521: chown d4/f79 11 1 2024-07-16T09:55:40.308 INFO:tasks.workunit.client.0.smithi156.stdout:3/472: mknod d8/ca5 0 2024-07-16T09:55:40.308 INFO:tasks.workunit.client.0.smithi156.stdout:4/522: symlink d4/db/dc/d5f/d64/lbd 0 2024-07-16T09:55:40.310 INFO:tasks.workunit.client.0.smithi156.stdout:4/523: dread d4/db/dc/f2a [0,4194304] 0 2024-07-16T09:55:40.313 INFO:tasks.workunit.client.0.smithi156.stdout:9/551: truncate d2/d5/d16/d22/d6b/d86/f8c 2203062 0 2024-07-16T09:55:40.314 INFO:tasks.workunit.client.0.smithi156.stdout:3/473: dwrite d8/d14/f59 [0,4194304] 0 2024-07-16T09:55:40.315 INFO:tasks.workunit.client.0.smithi156.stdout:4/524: fsync d4/d23/d52/d71/fa7 0 2024-07-16T09:55:40.316 INFO:tasks.workunit.client.0.smithi156.stdout:4/525: chown d4/db/d47/d63/d58 13665894 1 2024-07-16T09:55:40.317 INFO:tasks.workunit.client.0.smithi156.stdout:4/526: stat d4/db/d47/d63/d58/d8f/lab 0 2024-07-16T09:55:40.319 INFO:tasks.workunit.client.0.smithi156.stdout:3/474: link d8/d14/f1c d8/da/d28/d43/fa6 0 2024-07-16T09:55:40.320 INFO:tasks.workunit.client.1.smithi176.stdout:2/572: rmdir dd/d2e/d35 39 2024-07-16T09:55:40.326 INFO:tasks.workunit.client.1.smithi176.stdout:5/573: dread d4/f5 [0,4194304] 0 2024-07-16T09:55:40.326 INFO:tasks.workunit.client.1.smithi176.stdout:9/623: link d5/de/f15 d5/d1d/d67/d6d/fd6 0 2024-07-16T09:55:40.327 INFO:tasks.workunit.client.0.smithi156.stdout:3/475: link d8/c8f d8/da/d28/d43/ca7 0 2024-07-16T09:55:40.327 INFO:tasks.workunit.client.0.smithi156.stdout:3/476: dread - d8/da/d21/d80/d99/fa3 zero size 2024-07-16T09:55:40.327 INFO:tasks.workunit.client.1.smithi176.stdout:9/624: stat d5/fb 0 2024-07-16T09:55:40.328 INFO:tasks.workunit.client.0.smithi156.stdout:4/527: write d4/d23/f53 [2214188,58109] 0 2024-07-16T09:55:40.332 INFO:tasks.workunit.client.1.smithi176.stdout:2/573: dwrite dd/d16/f1d [4194304,4194304] 0 2024-07-16T09:55:40.333 INFO:tasks.workunit.client.0.smithi156.stdout:4/528: dwrite d4/d23/f78 [0,4194304] 0 2024-07-16T09:55:40.335 INFO:tasks.workunit.client.0.smithi156.stdout:4/529: truncate d4/db/d47/d63/fb7 255019 0 2024-07-16T09:55:40.340 INFO:tasks.workunit.client.1.smithi176.stdout:7/568: dwrite d4/d1f/d22/d80/fa6 [0,4194304] 0 2024-07-16T09:55:40.340 INFO:tasks.workunit.client.1.smithi176.stdout:5/574: dread d4/d37/f5a [0,4194304] 0 2024-07-16T09:55:40.341 INFO:tasks.workunit.client.0.smithi156.stdout:4/530: mkdir d4/db/d47/dbe 0 2024-07-16T09:55:40.342 INFO:tasks.workunit.client.1.smithi176.stdout:5/575: chown d4/da9 287615 1 2024-07-16T09:55:40.343 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:55:39 smithi176 ceph-mon[36626]: pgmap v251: 65 pgs: 65 active+clean; 1.7 GiB data, 6.5 GiB used, 530 GiB / 536 GiB avail; 42 MiB/s rd, 238 MiB/s wr, 471 op/s 2024-07-16T09:55:40.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:55:39 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-16T09:55:40.346 INFO:tasks.workunit.client.0.smithi156.stdout:3/477: truncate d8/da/d1d/f58 2475296 0 2024-07-16T09:55:40.348 INFO:tasks.workunit.client.0.smithi156.stdout:3/478: fdatasync d8/da/d1d/f64 0 2024-07-16T09:55:40.349 INFO:tasks.workunit.client.1.smithi176.stdout:7/569: mknod d4/d1f/d22/d80/d17/cb9 0 2024-07-16T09:55:40.349 INFO:tasks.workunit.client.0.smithi156.stdout:3/479: fdatasync d8/da/d13/f17 0 2024-07-16T09:55:40.350 INFO:tasks.workunit.client.0.smithi156.stdout:4/531: creat d4/db/d47/fbf x:0 0 0 2024-07-16T09:55:40.350 INFO:tasks.workunit.client.1.smithi176.stdout:9/625: dwrite d5/d1d/dd2/f75 [0,4194304] 0 2024-07-16T09:55:40.353 INFO:tasks.workunit.client.1.smithi176.stdout:2/574: dwrite dd/d36/d82/f96 [4194304,4194304] 0 2024-07-16T09:55:40.354 INFO:tasks.workunit.client.0.smithi156.stdout:4/532: mknod d4/db/cc0 0 2024-07-16T09:55:40.356 INFO:tasks.workunit.client.0.smithi156.stdout:4/533: write d4/f43 [3396166,31480] 0 2024-07-16T09:55:40.360 INFO:tasks.workunit.client.1.smithi176.stdout:2/575: creat dd/d16/d39/d3d/d7e/fba x:0 0 0 2024-07-16T09:55:40.361 INFO:tasks.workunit.client.1.smithi176.stdout:9/626: creat d5/d1d/d20/d78/dd0/fd7 x:0 0 0 2024-07-16T09:55:40.362 INFO:tasks.workunit.client.1.smithi176.stdout:7/570: getdents d4/dd/d63 0 2024-07-16T09:55:40.362 INFO:tasks.workunit.client.1.smithi176.stdout:5/576: link d4/d37/d72/d38/d6e/l8b d4/d37/d72/d82/d96/dbd/lcf 0 2024-07-16T09:55:40.363 INFO:tasks.workunit.client.1.smithi176.stdout:2/576: mknod dd/d49/d52/cbb 0 2024-07-16T09:55:40.364 INFO:tasks.workunit.client.1.smithi176.stdout:7/571: chown d4/d1f/f6f 0 1 2024-07-16T09:55:40.365 INFO:tasks.workunit.client.1.smithi176.stdout:7/572: fsync d4/fb 0 2024-07-16T09:55:40.365 INFO:tasks.workunit.client.0.smithi156.stdout:4/534: getdents d4/db/d47/d63 0 2024-07-16T09:55:40.366 INFO:tasks.workunit.client.1.smithi176.stdout:9/627: truncate d5/f1e 674065 0 2024-07-16T09:55:40.368 INFO:tasks.workunit.client.1.smithi176.stdout:2/577: truncate dd/d49/d52/f63 4739965 0 2024-07-16T09:55:40.368 INFO:tasks.workunit.client.1.smithi176.stdout:7/573: stat d4/d47/d6d/d89/d7e 0 2024-07-16T09:55:40.370 INFO:tasks.workunit.client.0.smithi156.stdout:4/535: rename d4/db/dc/d5f/d64/l9c to d4/db/dc/d7f/d87/lc1 0 2024-07-16T09:55:40.371 INFO:tasks.workunit.client.0.smithi156.stdout:4/536: readlink d4/db/l5a 0 2024-07-16T09:55:40.372 INFO:tasks.workunit.client.1.smithi176.stdout:2/578: creat dd/d2e/d35/d59/fbc x:0 0 0 2024-07-16T09:55:40.373 INFO:tasks.workunit.client.0.smithi156.stdout:4/537: read d4/d23/d30/f2d [2383100,107159] 0 2024-07-16T09:55:40.373 INFO:tasks.workunit.client.1.smithi176.stdout:5/577: dwrite d4/f27 [0,4194304] 0 2024-07-16T09:55:40.373 INFO:tasks.workunit.client.1.smithi176.stdout:9/628: link d5/d1d/d20/c23 d5/d1d/cd8 0 2024-07-16T09:55:40.375 INFO:tasks.workunit.client.1.smithi176.stdout:2/579: mknod dd/d9b/cbd 0 2024-07-16T09:55:40.377 INFO:tasks.workunit.client.1.smithi176.stdout:5/578: fdatasync d4/d14/d19/d1b/d61/f80 0 2024-07-16T09:55:40.379 INFO:tasks.workunit.client.1.smithi176.stdout:5/579: fsync d4/d14/fc1 0 2024-07-16T09:55:40.381 INFO:tasks.workunit.client.1.smithi176.stdout:7/574: rename d4/d1f/d22/d80/d17/c6b to d4/d1f/d22/d51/cba 0 2024-07-16T09:55:40.382 INFO:tasks.workunit.client.1.smithi176.stdout:9/629: dwrite d5/de/d30/f9a [0,4194304] 0 2024-07-16T09:55:40.383 INFO:tasks.workunit.client.0.smithi156.stdout:4/538: dwrite d4/db/dc/d67/f92 [0,4194304] 0 2024-07-16T09:55:40.383 INFO:tasks.workunit.client.1.smithi176.stdout:9/630: stat d5/de/f1a 0 2024-07-16T09:55:40.384 INFO:tasks.workunit.client.1.smithi176.stdout:9/631: chown d5/d1d/d20/f81 45336295 1 2024-07-16T09:55:40.387 INFO:tasks.workunit.client.0.smithi156.stdout:4/539: mknod d4/db/d47/d63/d58/d8f/cc2 0 2024-07-16T09:55:40.389 INFO:tasks.workunit.client.1.smithi176.stdout:7/575: unlink d4/dd/d26/f7b 0 2024-07-16T09:55:40.389 INFO:tasks.workunit.client.1.smithi176.stdout:2/580: link dd/d49/c9d dd/d49/d52/d7c/d81/cbe 0 2024-07-16T09:55:40.389 INFO:tasks.workunit.client.1.smithi176.stdout:5/580: mknod d4/cd0 0 2024-07-16T09:55:40.396 INFO:tasks.workunit.client.1.smithi176.stdout:7/576: mknod d4/d1f/d22/d80/cbb 0 2024-07-16T09:55:40.397 INFO:tasks.workunit.client.0.smithi156.stdout:3/480: dread d8/da/d1d/d2c/f8e [0,4194304] 0 2024-07-16T09:55:40.400 INFO:tasks.workunit.client.0.smithi156.stdout:4/540: rmdir d4 39 2024-07-16T09:55:40.401 INFO:tasks.workunit.client.1.smithi176.stdout:7/577: mknod d4/d47/d6d/d89/d7e/cbc 0 2024-07-16T09:55:40.401 INFO:tasks.workunit.client.1.smithi176.stdout:5/581: dwrite d4/d14/f2c [0,4194304] 0 2024-07-16T09:55:40.402 INFO:tasks.workunit.client.1.smithi176.stdout:7/578: chown d4/d1f/l79 3 1 2024-07-16T09:55:40.402 INFO:tasks.workunit.client.0.smithi156.stdout:3/481: rename d8/da/d1d/f83 to d8/da/d21/d80/fa8 0 2024-07-16T09:55:40.407 INFO:tasks.workunit.client.0.smithi156.stdout:4/541: mknod d4/db/dc/d5f/d64/cc3 0 2024-07-16T09:55:40.408 INFO:tasks.workunit.client.0.smithi156.stdout:4/542: read d4/f54 [23995,121771] 0 2024-07-16T09:55:40.409 INFO:tasks.workunit.client.0.smithi156.stdout:4/543: chown d4/db/dc/f70 2 1 2024-07-16T09:55:40.411 INFO:tasks.workunit.client.0.smithi156.stdout:4/544: fdatasync d4/db/f1d 0 2024-07-16T09:55:40.414 INFO:tasks.workunit.client.1.smithi176.stdout:2/581: write dd/de/f10 [115572,63207] 0 2024-07-16T09:55:40.414 INFO:tasks.workunit.client.1.smithi176.stdout:5/582: dwrite d4/d14/fac [0,4194304] 0 2024-07-16T09:55:40.414 INFO:tasks.workunit.client.0.smithi156.stdout:4/545: rename d4/db/dc/d5f/f98 to d4/db/d8a/fc4 0 2024-07-16T09:55:40.416 INFO:tasks.workunit.client.1.smithi176.stdout:7/579: dwrite d4/d1f/d22/d80/fa6 [0,4194304] 0 2024-07-16T09:55:40.417 INFO:tasks.workunit.client.1.smithi176.stdout:2/582: mknod dd/cbf 0 2024-07-16T09:55:40.418 INFO:tasks.workunit.client.1.smithi176.stdout:2/583: readlink dd/d2e/l62 0 2024-07-16T09:55:40.418 INFO:tasks.workunit.client.1.smithi176.stdout:5/583: fsync d4/d37/d72/d38/d39/f3a 0 2024-07-16T09:55:40.420 INFO:tasks.workunit.client.1.smithi176.stdout:2/584: creat dd/de/fc0 x:0 0 0 2024-07-16T09:55:40.422 INFO:tasks.workunit.client.1.smithi176.stdout:7/580: mknod d4/dd/d49/d7a/db1/cbd 0 2024-07-16T09:55:40.423 INFO:tasks.workunit.client.1.smithi176.stdout:9/632: dread d5/de/f15 [0,4194304] 0 2024-07-16T09:55:40.424 INFO:tasks.workunit.client.0.smithi156.stdout:4/546: link d4/db/dc/d7f/d87/fb9 d4/db/dc/d7f/daf/fc5 0 2024-07-16T09:55:40.425 INFO:tasks.workunit.client.1.smithi176.stdout:2/585: creat dd/d2e/d35/d59/d66/fc1 x:0 0 0 2024-07-16T09:55:40.427 INFO:tasks.workunit.client.0.smithi156.stdout:4/547: symlink d4/db/d47/d63/d58/lc6 0 2024-07-16T09:55:40.429 INFO:tasks.workunit.client.1.smithi176.stdout:9/633: creat d5/d1d/d77/fd9 x:0 0 0 2024-07-16T09:55:40.430 INFO:tasks.workunit.client.1.smithi176.stdout:2/586: mkdir dd/d16/d39/d3d/dc2 0 2024-07-16T09:55:40.431 INFO:tasks.workunit.client.0.smithi156.stdout:4/548: symlink d4/db/d8a/lc7 0 2024-07-16T09:55:40.432 INFO:tasks.workunit.client.1.smithi176.stdout:2/587: write dd/d36/fb2 [323029,60194] 0 2024-07-16T09:55:40.433 INFO:tasks.workunit.client.1.smithi176.stdout:9/634: creat d5/d1d/d77/d98/fda x:0 0 0 2024-07-16T09:55:40.433 INFO:tasks.workunit.client.1.smithi176.stdout:7/581: mkdir d4/d1f/d22/d80/d17/d1d/d8b/dbe 0 2024-07-16T09:55:40.434 INFO:tasks.workunit.client.0.smithi156.stdout:4/549: mkdir d4/db/dc/d7f/db1/dc8 0 2024-07-16T09:55:40.435 INFO:tasks.workunit.client.1.smithi176.stdout:5/584: truncate d4/d37/f5a 1270760 0 2024-07-16T09:55:40.437 INFO:tasks.workunit.client.0.smithi156.stdout:3/482: dread d8/da/d28/d43/f77 [0,4194304] 0 2024-07-16T09:55:40.437 INFO:tasks.workunit.client.1.smithi176.stdout:5/585: truncate d4/d10/fb1 723627 0 2024-07-16T09:55:40.438 INFO:tasks.workunit.client.0.smithi156.stdout:4/550: creat d4/db/fc9 x:0 0 0 2024-07-16T09:55:40.438 INFO:tasks.workunit.client.1.smithi176.stdout:7/582: mknod d4/d47/d6d/db7/cbf 0 2024-07-16T09:55:40.438 INFO:tasks.workunit.client.0.smithi156.stdout:3/483: write d8/da/d28/d43/f8b [486642,71893] 0 2024-07-16T09:55:40.439 INFO:tasks.workunit.client.1.smithi176.stdout:2/588: link dd/d2e/f37 dd/d16/d39/d3d/d7e/fc3 0 2024-07-16T09:55:40.440 INFO:tasks.workunit.client.0.smithi156.stdout:2/539: sync 2024-07-16T09:55:40.441 INFO:tasks.workunit.client.1.smithi176.stdout:5/586: creat d4/d14/dc5/fd1 x:0 0 0 2024-07-16T09:55:40.442 INFO:tasks.workunit.client.0.smithi156.stdout:3/484: creat d8/da/d1d/d2c/d33/fa9 x:0 0 0 2024-07-16T09:55:40.443 INFO:tasks.workunit.client.1.smithi176.stdout:2/589: creat dd/d36/d45/fc4 x:0 0 0 2024-07-16T09:55:40.443 INFO:tasks.workunit.client.1.smithi176.stdout:7/583: dread d4/d1f/d22/d80/d17/f4d [0,4194304] 0 2024-07-16T09:55:40.443 INFO:tasks.workunit.client.0.smithi156.stdout:2/540: rmdir d0/d5/d8b 39 2024-07-16T09:55:40.444 INFO:tasks.workunit.client.1.smithi176.stdout:9/635: dwrite d5/de/d21/f39 [4194304,4194304] 0 2024-07-16T09:55:40.445 INFO:tasks.workunit.client.1.smithi176.stdout:5/587: symlink d4/ld2 0 2024-07-16T09:55:40.445 INFO:tasks.workunit.client.0.smithi156.stdout:4/551: dwrite d4/db/dc/d67/f92 [0,4194304] 0 2024-07-16T09:55:40.447 INFO:tasks.workunit.client.1.smithi176.stdout:2/590: truncate dd/d36/f2f 4254220 0 2024-07-16T09:55:40.447 INFO:tasks.workunit.client.0.smithi156.stdout:4/552: write d4/d23/d30/d33/f3a [1577923,21423] 0 2024-07-16T09:55:40.448 INFO:tasks.workunit.client.1.smithi176.stdout:9/636: write d5/d1d/d20/f7d [533352,117407] 0 2024-07-16T09:55:40.448 INFO:tasks.workunit.client.1.smithi176.stdout:7/584: symlink d4/d1f/d22/d80/lc0 0 2024-07-16T09:55:40.450 INFO:tasks.workunit.client.1.smithi176.stdout:9/637: chown d5/la 982 1 2024-07-16T09:55:40.454 INFO:tasks.workunit.client.0.smithi156.stdout:2/541: dwrite d0/d13/d17/d1c/faa [0,4194304] 0 2024-07-16T09:55:40.457 INFO:tasks.workunit.client.1.smithi176.stdout:7/585: rmdir d4/d47/d6d/da7 0 2024-07-16T09:55:40.457 INFO:tasks.workunit.client.1.smithi176.stdout:9/638: creat d5/d1d/d77/d97/fdb x:0 0 0 2024-07-16T09:55:40.459 INFO:tasks.workunit.client.0.smithi156.stdout:2/542: truncate d0/d5/f44 360462 0 2024-07-16T09:55:40.460 INFO:tasks.workunit.client.1.smithi176.stdout:9/639: symlink d5/d1d/d77/d98/ldc 0 2024-07-16T09:55:40.463 INFO:tasks.workunit.client.1.smithi176.stdout:7/586: rename d4/d5/f12 to d4/dd/d26/fc1 0 2024-07-16T09:55:40.463 INFO:tasks.workunit.client.0.smithi156.stdout:2/543: creat d0/d13/d17/d1c/fbb x:0 0 0 2024-07-16T09:55:40.463 INFO:tasks.workunit.client.1.smithi176.stdout:9/640: mknod d5/d1d/d20/cdd 0 2024-07-16T09:55:40.464 INFO:tasks.workunit.client.0.smithi156.stdout:2/544: dread - d0/d13/d17/d1c/f47 zero size 2024-07-16T09:55:40.467 INFO:tasks.workunit.client.0.smithi156.stdout:2/545: truncate d0/d5/d29/d2d/d5b/d62/fb3 804782 0 2024-07-16T09:55:40.482 INFO:tasks.workunit.client.0.smithi156.stdout:3/485: dread d8/da/d1d/d2c/d33/f88 [0,4194304] 0 2024-07-16T09:55:40.482 INFO:tasks.workunit.client.0.smithi156.stdout:2/546: rename d0/d5/f44 to d0/d5/d29/d2d/d5b/d62/fbc 0 2024-07-16T09:55:40.483 INFO:tasks.workunit.client.1.smithi176.stdout:5/588: dread d4/d14/f25 [0,4194304] 0 2024-07-16T09:55:40.484 INFO:tasks.workunit.client.1.smithi176.stdout:5/589: write d4/d14/dc5/fd1 [288850,25481] 0 2024-07-16T09:55:40.486 INFO:tasks.workunit.client.0.smithi156.stdout:4/553: dwrite d4/db/dc/d5f/f5c [0,4194304] 0 2024-07-16T09:55:40.489 INFO:tasks.workunit.client.1.smithi176.stdout:0/610: sync 2024-07-16T09:55:40.491 INFO:tasks.workunit.client.0.smithi156.stdout:3/486: dwrite d8/da/f5d [0,4194304] 0 2024-07-16T09:55:40.491 INFO:tasks.workunit.client.0.smithi156.stdout:2/547: creat d0/d13/d17/d1c/fbd x:0 0 0 2024-07-16T09:55:40.495 INFO:tasks.workunit.client.1.smithi176.stdout:7/587: dread d4/d5/f2f [0,4194304] 0 2024-07-16T09:55:40.496 INFO:tasks.workunit.client.1.smithi176.stdout:0/611: dwrite d5/fb [4194304,4194304] 0 2024-07-16T09:55:40.497 INFO:tasks.workunit.client.0.smithi156.stdout:2/548: creat d0/d5/d29/fbe x:0 0 0 2024-07-16T09:55:40.501 INFO:tasks.workunit.client.1.smithi176.stdout:0/612: rmdir d5/d1c/d5a 39 2024-07-16T09:55:40.501 INFO:tasks.workunit.client.0.smithi156.stdout:2/549: creat d0/d13/d17/fbf x:0 0 0 2024-07-16T09:55:40.503 INFO:tasks.workunit.client.1.smithi176.stdout:7/588: rename d4/d1f/d22/d80 to d4/dd/d26/dc2 0 2024-07-16T09:55:40.515 INFO:tasks.workunit.client.1.smithi176.stdout:0/613: rename d5/d39/d92 to d5/d1c/d24/d35/d59/dc9 0 2024-07-16T09:55:40.515 INFO:tasks.workunit.client.1.smithi176.stdout:5/590: write d4/d37/f3f [1308496,64348] 0 2024-07-16T09:55:40.518 INFO:tasks.workunit.client.0.smithi156.stdout:4/554: dread d4/db/d47/d63/d58/d8f/f5e [0,4194304] 0 2024-07-16T09:55:40.521 INFO:tasks.workunit.client.0.smithi156.stdout:4/555: creat d4/d23/d52/fca x:0 0 0 2024-07-16T09:55:40.523 INFO:tasks.workunit.client.1.smithi176.stdout:7/589: dwrite d4/d47/d6d/d89/d66/fa4 [0,4194304] 0 2024-07-16T09:55:40.527 INFO:tasks.workunit.client.0.smithi156.stdout:4/556: dwrite d4/db/dc/d7c/f8e [0,4194304] 0 2024-07-16T09:55:40.529 INFO:tasks.workunit.client.1.smithi176.stdout:7/590: dread d4/d47/f46 [0,4194304] 0 2024-07-16T09:55:40.531 INFO:tasks.workunit.client.1.smithi176.stdout:7/591: readlink d4/dd/d63/la0 0 2024-07-16T09:55:40.534 INFO:tasks.workunit.client.0.smithi156.stdout:4/557: creat d4/db/dc/fcb x:0 0 0 2024-07-16T09:55:40.535 INFO:tasks.workunit.client.1.smithi176.stdout:0/614: dread d5/d1c/d5a/fab [0,4194304] 0 2024-07-16T09:55:40.535 INFO:tasks.workunit.client.0.smithi156.stdout:4/558: stat d4/db/f18 0 2024-07-16T09:55:40.541 INFO:tasks.workunit.client.0.smithi156.stdout:4/559: dwrite d4/f3d [0,4194304] 0 2024-07-16T09:55:40.541 INFO:tasks.workunit.client.1.smithi176.stdout:0/615: dread d5/fb [4194304,4194304] 0 2024-07-16T09:55:40.543 INFO:tasks.workunit.client.1.smithi176.stdout:7/592: dread d4/dd/f15 [0,4194304] 0 2024-07-16T09:55:40.547 INFO:tasks.workunit.client.1.smithi176.stdout:0/616: truncate d5/d1c/d24/d26/fb2 733308 0 2024-07-16T09:55:40.547 INFO:tasks.workunit.client.1.smithi176.stdout:0/617: dread - d5/d25/da0/faf zero size 2024-07-16T09:55:40.549 INFO:tasks.workunit.client.1.smithi176.stdout:5/591: link d4/d37/f5a d4/d37/dbb/fd3 0 2024-07-16T09:55:40.550 INFO:tasks.workunit.client.1.smithi176.stdout:5/592: chown d4/d37/d72/d82/f83 1669832 1 2024-07-16T09:55:40.550 INFO:tasks.workunit.client.0.smithi156.stdout:4/560: dwrite d4/db/dc/d5f/fad [0,4194304] 0 2024-07-16T09:55:40.551 INFO:tasks.workunit.client.0.smithi156.stdout:2/550: dread d0/d5/f1e [0,4194304] 0 2024-07-16T09:55:40.551 INFO:tasks.workunit.client.1.smithi176.stdout:0/618: creat d5/d1c/d24/fca x:0 0 0 2024-07-16T09:55:40.553 INFO:tasks.workunit.client.1.smithi176.stdout:5/593: mknod d4/d6c/d94/cd4 0 2024-07-16T09:55:40.555 INFO:tasks.workunit.client.0.smithi156.stdout:4/561: unlink d4/db/dc/d5f/fad 0 2024-07-16T09:55:40.557 INFO:tasks.workunit.client.1.smithi176.stdout:0/619: getdents d5 0 2024-07-16T09:55:40.558 INFO:tasks.workunit.client.0.smithi156.stdout:2/551: rename d0/d13/d17/l67 to d0/d5/d29/d2d/d5b/d62/d74/lc0 0 2024-07-16T09:55:40.559 INFO:tasks.workunit.client.0.smithi156.stdout:2/552: chown d0/d13/d17/d1c/f46 4478 1 2024-07-16T09:55:40.560 INFO:tasks.workunit.client.1.smithi176.stdout:5/594: creat d4/d37/d72/d82/d96/dbd/fd5 x:0 0 0 2024-07-16T09:55:40.560 INFO:tasks.workunit.client.0.smithi156.stdout:2/553: dread - d0/d9b/fba zero size 2024-07-16T09:55:40.561 INFO:tasks.workunit.client.1.smithi176.stdout:0/620: creat d5/d1c/d24/d35/d50/d58/fcb x:0 0 0 2024-07-16T09:55:40.563 INFO:tasks.workunit.client.0.smithi156.stdout:4/562: rename d4/db/d47/d63 to d4/db/dc/dcc 0 2024-07-16T09:55:40.564 INFO:tasks.workunit.client.1.smithi176.stdout:5/595: unlink d4/d37/d72/d38/d39/d9f/fa6 0 2024-07-16T09:55:40.565 INFO:tasks.workunit.client.1.smithi176.stdout:5/596: chown d4/d37/d72/d82/d96/dbd 154010109 1 2024-07-16T09:55:40.566 INFO:tasks.workunit.client.1.smithi176.stdout:0/621: mkdir d5/d1c/d24/d35/d50/d58/d63/db4/dcc 0 2024-07-16T09:55:40.567 INFO:tasks.workunit.client.1.smithi176.stdout:5/597: rmdir d4/d14/dc5 39 2024-07-16T09:55:40.568 INFO:tasks.workunit.client.1.smithi176.stdout:0/622: creat d5/d1c/db1/fcd x:0 0 0 2024-07-16T09:55:40.571 INFO:tasks.workunit.client.1.smithi176.stdout:0/623: dread d5/d1c/d5a/f75 [0,4194304] 0 2024-07-16T09:55:40.573 INFO:tasks.workunit.client.0.smithi156.stdout:2/554: write d0/d5/d29/d2d/d5b/d62/fa0 [345825,108474] 0 2024-07-16T09:55:40.573 INFO:tasks.workunit.client.1.smithi176.stdout:7/593: dread d4/dd/d26/dc2/d17/f7f [0,4194304] 0 2024-07-16T09:55:40.573 INFO:tasks.workunit.client.0.smithi156.stdout:4/563: truncate d4/db/dc/dcc/d58/fa6 4101153 0 2024-07-16T09:55:40.574 INFO:tasks.workunit.client.1.smithi176.stdout:7/594: stat d4/dd/f2e 0 2024-07-16T09:55:40.574 INFO:tasks.workunit.client.0.smithi156.stdout:4/564: stat d4/db/dc/d5f/c51 0 2024-07-16T09:55:40.575 INFO:tasks.workunit.client.0.smithi156.stdout:4/565: chown d4/db/f16 3 1 2024-07-16T09:55:40.575 INFO:tasks.workunit.client.0.smithi156.stdout:2/555: write d0/d5/d76/f8a [833500,40741] 0 2024-07-16T09:55:40.577 INFO:tasks.workunit.client.0.smithi156.stdout:4/566: mknod d4/d6a/da9/ccd 0 2024-07-16T09:55:40.577 INFO:tasks.workunit.client.1.smithi176.stdout:0/624: write d5/d1c/d24/f30 [117207,34683] 0 2024-07-16T09:55:40.579 INFO:tasks.workunit.client.0.smithi156.stdout:4/567: symlink d4/d23/d30/d33/lce 0 2024-07-16T09:55:40.580 INFO:tasks.workunit.client.1.smithi176.stdout:5/598: dread d4/d37/d72/d82/f83 [0,4194304] 0 2024-07-16T09:55:40.580 INFO:tasks.workunit.client.1.smithi176.stdout:7/595: rename d4/d47/d6d/d89/d66/f68 to d4/dd/d26/dc2/fc3 0 2024-07-16T09:55:40.580 INFO:tasks.workunit.client.1.smithi176.stdout:0/625: fdatasync d5/d1c/d24/d35/d50/d5b/f65 0 2024-07-16T09:55:40.581 INFO:tasks.workunit.client.0.smithi156.stdout:2/556: link d0/d13/d17/d1c/faa d0/d5/d29/d2d/d5b/fc1 0 2024-07-16T09:55:40.582 INFO:tasks.workunit.client.1.smithi176.stdout:7/596: mkdir d4/d47/d6d/d89/d66/dc4 0 2024-07-16T09:55:40.582 INFO:tasks.workunit.client.0.smithi156.stdout:4/568: creat d4/db/dc/d7f/fcf x:0 0 0 2024-07-16T09:55:40.583 INFO:tasks.workunit.client.1.smithi176.stdout:0/626: mkdir d5/d1c/d24/d35/d59/dce 0 2024-07-16T09:55:40.585 INFO:tasks.workunit.client.0.smithi156.stdout:4/569: unlink d4/d6a/da9/ccd 0 2024-07-16T09:55:40.590 INFO:tasks.workunit.client.1.smithi176.stdout:1/317: sync 2024-07-16T09:55:40.591 INFO:tasks.workunit.client.1.smithi176.stdout:0/627: rename d5/d1c/d24/d35/d50/d5b/c9d to d5/d39/da3/dbe/ccf 0 2024-07-16T09:55:40.591 INFO:tasks.workunit.client.0.smithi156.stdout:4/570: dwrite d4/db/dc/d7f/fcf [0,4194304] 0 2024-07-16T09:55:40.593 INFO:tasks.workunit.client.1.smithi176.stdout:7/597: getdents d4/dd/d49/d7a/db1 0 2024-07-16T09:55:40.593 INFO:tasks.workunit.client.1.smithi176.stdout:7/598: dread - d4/fb3 zero size 2024-07-16T09:55:40.595 INFO:tasks.workunit.client.1.smithi176.stdout:0/628: read d5/f22 [1564492,56276] 0 2024-07-16T09:55:40.596 INFO:tasks.workunit.client.0.smithi156.stdout:4/571: mknod d4/db/dc/d5f/d64/cd0 0 2024-07-16T09:55:40.598 INFO:tasks.workunit.client.0.smithi156.stdout:2/557: dwrite d0/d5/d29/d2d/d5b/f6f [0,4194304] 0 2024-07-16T09:55:40.598 INFO:tasks.workunit.client.0.smithi156.stdout:4/572: truncate d4/d23/d30/d33/f3a 4570112 0 2024-07-16T09:55:40.599 INFO:tasks.workunit.client.1.smithi176.stdout:1/318: rename d4/d8/d12/d21/d25/d30/d44/d4d/f32 to d4/d3a/d54/f6d 0 2024-07-16T09:55:40.599 INFO:tasks.workunit.client.0.smithi156.stdout:4/573: chown d4/d23/d30 44131970 1 2024-07-16T09:55:40.599 INFO:tasks.workunit.client.1.smithi176.stdout:0/629: rename d5/d1c/d24 to d5/d1c/d24/d35/d50/d58/dd0 22 2024-07-16T09:55:40.600 INFO:tasks.workunit.client.0.smithi156.stdout:4/574: stat d4/db/dc/dcc/d58/f7a 0 2024-07-16T09:55:40.602 INFO:tasks.workunit.client.1.smithi176.stdout:1/319: chown d4/db/f49 1955950 1 2024-07-16T09:55:40.603 INFO:tasks.workunit.client.0.smithi156.stdout:2/558: unlink d0/d5/d8b/f8e 0 2024-07-16T09:55:40.604 INFO:tasks.workunit.client.1.smithi176.stdout:7/599: dwrite d4/fb3 [0,4194304] 0 2024-07-16T09:55:40.604 INFO:tasks.workunit.client.1.smithi176.stdout:5/599: dread d4/d10/d17/f18 [0,4194304] 0 2024-07-16T09:55:40.604 INFO:tasks.workunit.client.0.smithi156.stdout:4/575: creat d4/db/dc/d7f/db1/fd1 x:0 0 0 2024-07-16T09:55:40.606 INFO:tasks.workunit.client.1.smithi176.stdout:1/320: symlink d4/db/l6e 0 2024-07-16T09:55:40.607 INFO:tasks.workunit.client.0.smithi156.stdout:2/559: write d0/d5/d29/d2d/d5b/d62/fa0 [1365797,49502] 0 2024-07-16T09:55:40.607 INFO:tasks.workunit.client.1.smithi176.stdout:0/630: mknod d5/d1c/d24/d35/d50/d58/d63/db4/dcc/cd1 0 2024-07-16T09:55:40.609 INFO:tasks.workunit.client.0.smithi156.stdout:4/576: mkdir d4/d23/d30/dd2 0 2024-07-16T09:55:40.609 INFO:tasks.workunit.client.1.smithi176.stdout:7/600: mknod d4/d47/d5d/cc5 0 2024-07-16T09:55:40.610 INFO:tasks.workunit.client.0.smithi156.stdout:2/560: write d0/d13/d17/d1c/fbb [752664,106730] 0 2024-07-16T09:55:40.610 INFO:tasks.workunit.client.0.smithi156.stdout:4/577: fdatasync d4/f79 0 2024-07-16T09:55:40.613 INFO:tasks.workunit.client.1.smithi176.stdout:1/321: symlink d4/db/d68/l6f 0 2024-07-16T09:55:40.613 INFO:tasks.workunit.client.0.smithi156.stdout:4/578: mknod d4/d23/d52/d71/cd3 0 2024-07-16T09:55:40.614 INFO:tasks.workunit.client.1.smithi176.stdout:7/601: mknod d4/dd/d49/d7a/cc6 0 2024-07-16T09:55:40.615 INFO:tasks.workunit.client.0.smithi156.stdout:2/561: creat d0/d13/d17/d1c/d51/fc2 x:0 0 0 2024-07-16T09:55:40.616 INFO:tasks.workunit.client.1.smithi176.stdout:1/322: symlink d4/d3a/l70 0 2024-07-16T09:55:40.616 INFO:tasks.workunit.client.1.smithi176.stdout:0/631: dwrite d5/d1c/d24/d35/d50/d9b/fac [0,4194304] 0 2024-07-16T09:55:40.616 INFO:tasks.workunit.client.1.smithi176.stdout:7/602: symlink d4/dd/d49/d7a/db1/lc7 0 2024-07-16T09:55:40.617 INFO:tasks.workunit.client.0.smithi156.stdout:4/579: creat d4/db/dc/d7f/daf/fd4 x:0 0 0 2024-07-16T09:55:40.617 INFO:tasks.workunit.client.0.smithi156.stdout:4/580: write d4/d23/d52/fca [586199,122619] 0 2024-07-16T09:55:40.618 INFO:tasks.workunit.client.0.smithi156.stdout:2/562: unlink d0/d13/d17/d1c/f47 0 2024-07-16T09:55:40.619 INFO:tasks.workunit.client.1.smithi176.stdout:7/603: unlink d4/d47/d5d/f77 0 2024-07-16T09:55:40.620 INFO:tasks.workunit.client.1.smithi176.stdout:0/632: unlink d5/d1c/lb3 0 2024-07-16T09:55:40.622 INFO:tasks.workunit.client.1.smithi176.stdout:1/323: rename d4/db/c11 to d4/d8/d12/d21/d25/d30/d44/d4d/c71 0 2024-07-16T09:55:40.623 INFO:tasks.workunit.client.0.smithi156.stdout:4/581: unlink d4/db/dc/d7f/d87/lc1 0 2024-07-16T09:55:40.623 INFO:tasks.workunit.client.1.smithi176.stdout:7/604: creat d4/d47/d6d/d89/fc8 x:0 0 0 2024-07-16T09:55:40.624 INFO:tasks.workunit.client.1.smithi176.stdout:0/633: creat d5/d1c/d24/d35/d50/d58/d63/fd2 x:0 0 0 2024-07-16T09:55:40.624 INFO:tasks.workunit.client.1.smithi176.stdout:1/324: mkdir d4/db/d68/d72 0 2024-07-16T09:55:40.625 INFO:tasks.workunit.client.0.smithi156.stdout:4/582: chown d4/db/dc/d7f/f99 6 1 2024-07-16T09:55:40.626 INFO:tasks.workunit.client.1.smithi176.stdout:7/605: creat d4/d47/d5d/fc9 x:0 0 0 2024-07-16T09:55:40.626 INFO:tasks.workunit.client.1.smithi176.stdout:1/325: read d4/d3a/d54/f6d [1426223,241] 0 2024-07-16T09:55:40.626 INFO:tasks.workunit.client.1.smithi176.stdout:0/634: mknod d5/d39/da3/dbe/cd3 0 2024-07-16T09:55:40.627 INFO:tasks.workunit.client.0.smithi156.stdout:2/563: dwrite d0/d5/d29/fbe [0,4194304] 0 2024-07-16T09:55:40.628 INFO:tasks.workunit.client.1.smithi176.stdout:0/635: write d5/d39/f7e [4315879,122209] 0 2024-07-16T09:55:40.634 INFO:tasks.workunit.client.0.smithi156.stdout:4/583: rmdir d4/d23/d52 39 2024-07-16T09:55:40.639 INFO:tasks.workunit.client.1.smithi176.stdout:7/606: dwrite d4/d5/f6c [0,4194304] 0 2024-07-16T09:55:40.644 INFO:tasks.workunit.client.0.smithi156.stdout:4/584: dwrite d4/db/dc/f55 [0,4194304] 0 2024-07-16T09:55:40.647 INFO:tasks.workunit.client.0.smithi156.stdout:4/585: mknod d4/db/dc/d5f/cd5 0 2024-07-16T09:55:40.647 INFO:tasks.workunit.client.0.smithi156.stdout:4/586: write d4/f9 [4497386,75940] 0 2024-07-16T09:55:40.648 INFO:tasks.workunit.client.0.smithi156.stdout:5/569: sync 2024-07-16T09:55:40.653 INFO:tasks.workunit.client.1.smithi176.stdout:7/607: link d4/d47/d5d/cab d4/dd/d26/dc2/d17/d1d/cca 0 2024-07-16T09:55:40.653 INFO:tasks.workunit.client.0.smithi156.stdout:4/587: getdents d4/db/dc/d67 0 2024-07-16T09:55:40.656 INFO:tasks.workunit.client.0.smithi156.stdout:5/570: mkdir d1b/d1f/d45/d52/d43/d85/d97/d37/d88/dc7 0 2024-07-16T09:55:40.661 INFO:tasks.workunit.client.0.smithi156.stdout:2/564: dread d0/d5/d29/f38 [0,4194304] 0 2024-07-16T09:55:40.661 INFO:tasks.workunit.client.1.smithi176.stdout:7/608: dwrite d4/dd/d26/dc2/d17/d1d/d8b/f9d [0,4194304] 0 2024-07-16T09:55:40.661 INFO:tasks.workunit.client.1.smithi176.stdout:0/636: write d5/d2c/d6c/d83/fa6 [217776,84832] 0 2024-07-16T09:55:40.661 INFO:tasks.workunit.client.0.smithi156.stdout:4/588: rmdir d4/db/dc/d7f/db1/dc8 0 2024-07-16T09:55:40.662 INFO:tasks.workunit.client.1.smithi176.stdout:7/609: chown d4/d1f/f59 7371 1 2024-07-16T09:55:40.662 INFO:tasks.workunit.client.0.smithi156.stdout:4/589: write d4/db/dc/f2e [2881482,66851] 0 2024-07-16T09:55:40.663 INFO:tasks.workunit.client.1.smithi176.stdout:1/326: dread d4/f31 [0,4194304] 0 2024-07-16T09:55:40.665 INFO:tasks.workunit.client.1.smithi176.stdout:1/327: readlink d4/d3a/l70 0 2024-07-16T09:55:40.666 INFO:tasks.workunit.client.1.smithi176.stdout:0/637: symlink d5/d1c/d5a/d9e/ld4 0 2024-07-16T09:55:40.666 INFO:tasks.workunit.client.1.smithi176.stdout:7/610: mknod d4/d47/d6d/d89/d66/ccb 0 2024-07-16T09:55:40.667 INFO:tasks.workunit.client.0.smithi156.stdout:4/590: creat d4/d23/d30/fd6 x:0 0 0 2024-07-16T09:55:40.668 INFO:tasks.workunit.client.1.smithi176.stdout:1/328: chown d4/d8/d12/d21/d25/c46 116490117 1 2024-07-16T09:55:40.669 INFO:tasks.workunit.client.1.smithi176.stdout:7/611: symlink d4/dd/d49/d7a/lcc 0 2024-07-16T09:55:40.670 INFO:tasks.workunit.client.1.smithi176.stdout:0/638: mkdir d5/d1c/d24/d35/d50/dd5 0 2024-07-16T09:55:40.670 INFO:tasks.workunit.client.1.smithi176.stdout:1/329: symlink d4/db/l73 0 2024-07-16T09:55:40.673 INFO:tasks.workunit.client.0.smithi156.stdout:4/591: dwrite d4/db/dc/dcc/d58/f91 [0,4194304] 0 2024-07-16T09:55:40.674 INFO:tasks.workunit.client.1.smithi176.stdout:7/612: dread d4/d1f/d22/f52 [0,4194304] 0 2024-07-16T09:55:40.674 INFO:tasks.workunit.client.1.smithi176.stdout:0/639: rename d5/d1c/d24/d35/d59/l8a to d5/d44/ld6 0 2024-07-16T09:55:40.677 INFO:tasks.workunit.client.1.smithi176.stdout:1/330: rename d4/d8/f28 to d4/db/d68/d72/f74 0 2024-07-16T09:55:40.679 INFO:tasks.workunit.client.1.smithi176.stdout:7/613: creat d4/d47/d5d/d9a/fcd x:0 0 0 2024-07-16T09:55:40.679 INFO:tasks.workunit.client.1.smithi176.stdout:1/331: truncate d4/d8/f26 1346063 0 2024-07-16T09:55:40.680 INFO:tasks.workunit.client.1.smithi176.stdout:1/332: truncate d4/ff 4806173 0 2024-07-16T09:55:40.681 INFO:tasks.workunit.client.1.smithi176.stdout:0/640: dread d5/d2c/d6c/d83/fa6 [0,4194304] 0 2024-07-16T09:55:40.681 INFO:tasks.workunit.client.1.smithi176.stdout:7/614: mkdir d4/d1f/dce 0 2024-07-16T09:55:40.683 INFO:tasks.workunit.client.0.smithi156.stdout:4/592: write d4/d23/d52/d71/fa7 [932156,27575] 0 2024-07-16T09:55:40.684 INFO:tasks.workunit.client.1.smithi176.stdout:7/615: unlink d4/d47/d6d/d89/d7e/d87/lb4 0 2024-07-16T09:55:40.685 INFO:tasks.workunit.client.0.smithi156.stdout:4/593: write d4/db/dc/f88 [4399473,116325] 0 2024-07-16T09:55:40.687 INFO:tasks.workunit.client.0.smithi156.stdout:4/594: mkdir d4/db/dc/d67/dd7 0 2024-07-16T09:55:40.689 INFO:tasks.workunit.client.0.smithi156.stdout:4/595: write d4/db/dc/d7f/db1/fd1 [653266,39685] 0 2024-07-16T09:55:40.690 INFO:tasks.workunit.client.0.smithi156.stdout:4/596: write d4/db/dc/d7f/daf/fd4 [268046,83757] 0 2024-07-16T09:55:40.695 INFO:tasks.workunit.client.1.smithi176.stdout:1/333: dread d4/f17 [0,4194304] 0 2024-07-16T09:55:40.697 INFO:tasks.workunit.client.1.smithi176.stdout:1/334: rename d4/d8/d12/c53 to d4/d3a/d54/d5a/c75 0 2024-07-16T09:55:40.698 INFO:tasks.workunit.client.0.smithi156.stdout:4/597: dwrite d4/db/dc/f2a [0,4194304] 0 2024-07-16T09:55:40.698 INFO:tasks.workunit.client.1.smithi176.stdout:1/335: read d4/db/f43 [135201,75979] 0 2024-07-16T09:55:40.700 INFO:tasks.workunit.client.1.smithi176.stdout:1/336: getdents d4/d3a/d66 0 2024-07-16T09:55:40.703 INFO:tasks.workunit.client.1.smithi176.stdout:1/337: symlink d4/d8/d12/d21/d25/d30/l76 0 2024-07-16T09:55:40.704 INFO:tasks.workunit.client.1.smithi176.stdout:1/338: write d4/db/f16 [4652220,44422] 0 2024-07-16T09:55:40.710 INFO:tasks.workunit.client.1.smithi176.stdout:1/339: dwrite d4/db/f49 [0,4194304] 0 2024-07-16T09:55:40.714 INFO:tasks.workunit.client.1.smithi176.stdout:1/340: unlink d4/d8/d12/f4a 0 2024-07-16T09:55:40.715 INFO:tasks.workunit.client.1.smithi176.stdout:1/341: mkdir d4/d77 0 2024-07-16T09:55:40.718 INFO:tasks.workunit.client.1.smithi176.stdout:1/342: creat d4/d8/d12/d21/d25/d30/d44/f78 x:0 0 0 2024-07-16T09:55:40.719 INFO:tasks.workunit.client.1.smithi176.stdout:1/343: fdatasync d4/f31 0 2024-07-16T09:55:40.721 INFO:tasks.workunit.client.1.smithi176.stdout:1/344: write d4/d3a/d54/d5a/f67 [982737,70525] 0 2024-07-16T09:55:40.721 INFO:tasks.workunit.client.1.smithi176.stdout:1/345: readlink d4/l13 0 2024-07-16T09:55:40.723 INFO:tasks.workunit.client.1.smithi176.stdout:1/346: symlink d4/d3a/d54/l79 0 2024-07-16T09:55:40.725 INFO:tasks.workunit.client.1.smithi176.stdout:1/347: mkdir d4/d3a/d54/d5a/d7a 0 2024-07-16T09:55:40.727 INFO:tasks.workunit.client.1.smithi176.stdout:1/348: rename d4/d8/d12/l39 to d4/d8/d12/d4c/l7b 0 2024-07-16T09:55:40.729 INFO:tasks.workunit.client.1.smithi176.stdout:1/349: getdents d4/d8/d12/d21/d25/d30/d44/d4d 0 2024-07-16T09:55:40.742 INFO:tasks.workunit.client.1.smithi176.stdout:1/350: dread d4/db/d68/d72/f74 [0,4194304] 0 2024-07-16T09:55:40.749 INFO:tasks.workunit.client.1.smithi176.stdout:1/351: dwrite d4/f17 [0,4194304] 0 2024-07-16T09:55:40.751 INFO:tasks.workunit.client.1.smithi176.stdout:1/352: write d4/d8/d12/d21/d25/d30/d44/f5f [761651,121956] 0 2024-07-16T09:55:40.756 INFO:tasks.workunit.client.1.smithi176.stdout:1/353: dwrite d4/d8/d12/d21/d25/f55 [0,4194304] 0 2024-07-16T09:55:40.764 INFO:tasks.workunit.client.1.smithi176.stdout:1/354: dwrite d4/d8/d12/d21/d25/d30/d44/f5f [0,4194304] 0 2024-07-16T09:55:40.764 INFO:tasks.workunit.client.1.smithi176.stdout:1/355: stat d4/db/d68 0 2024-07-16T09:55:40.766 INFO:tasks.workunit.client.1.smithi176.stdout:3/433: sync 2024-07-16T09:55:40.767 INFO:tasks.workunit.client.1.smithi176.stdout:3/434: mkdir d3/da/d6c/d86 0 2024-07-16T09:55:40.770 INFO:tasks.workunit.client.1.smithi176.stdout:1/356: getdents d4/d8/d12/d21/d25/d30/d44 0 2024-07-16T09:55:40.771 INFO:tasks.workunit.client.1.smithi176.stdout:3/435: getdents d3/da/d56 0 2024-07-16T09:55:40.771 INFO:tasks.workunit.client.1.smithi176.stdout:3/436: fsync d3/da/d33/f78 0 2024-07-16T09:55:40.772 INFO:tasks.workunit.client.1.smithi176.stdout:3/437: write d3/d17/f68 [755536,2077] 0 2024-07-16T09:55:40.774 INFO:tasks.workunit.client.1.smithi176.stdout:3/438: write d3/d17/f2f [4259532,69098] 0 2024-07-16T09:55:40.774 INFO:tasks.workunit.client.1.smithi176.stdout:1/357: rename d4/d8/d2c/c3e to d4/d8/d12/d21/c7c 0 2024-07-16T09:55:40.776 INFO:tasks.workunit.client.1.smithi176.stdout:3/439: write d3/d17/d1d/d4f/d59/f27 [972886,107529] 0 2024-07-16T09:55:40.778 INFO:tasks.workunit.client.1.smithi176.stdout:3/440: dread d3/da/f29 [0,4194304] 0 2024-07-16T09:55:40.779 INFO:tasks.workunit.client.1.smithi176.stdout:1/358: getdents d4/d8/d12 0 2024-07-16T09:55:40.781 INFO:tasks.workunit.client.1.smithi176.stdout:3/441: mknod d3/da/d6c/c87 0 2024-07-16T09:55:40.782 INFO:tasks.workunit.client.1.smithi176.stdout:3/442: write d3/da/f13 [4380846,27255] 0 2024-07-16T09:55:40.783 INFO:tasks.workunit.client.1.smithi176.stdout:3/443: readlink d3/d17/d1d/d4d/l60 0 2024-07-16T09:55:40.786 INFO:tasks.workunit.client.0.smithi156.stdout:4/598: truncate d4/db/dc/dcc/d58/d8f/f5e 3891592 0 2024-07-16T09:55:40.786 INFO:tasks.workunit.client.1.smithi176.stdout:3/444: truncate d3/da/f41 863447 0 2024-07-16T09:55:40.786 INFO:tasks.workunit.client.1.smithi176.stdout:1/359: dwrite d4/d8/d12/d21/d34/f58 [0,4194304] 0 2024-07-16T09:55:40.790 INFO:tasks.workunit.client.0.smithi156.stdout:4/599: link d4/db/d47/fbf d4/db/dc/fd8 0 2024-07-16T09:55:40.791 INFO:tasks.workunit.client.1.smithi176.stdout:1/360: symlink d4/d77/l7d 0 2024-07-16T09:55:40.793 INFO:tasks.workunit.client.0.smithi156.stdout:4/600: mkdir d4/db/dc/dcc/d58/dd9 0 2024-07-16T09:55:40.794 INFO:tasks.workunit.client.1.smithi176.stdout:1/361: unlink d4/d8/fa 0 2024-07-16T09:55:40.794 INFO:tasks.workunit.client.0.smithi156.stdout:4/601: chown d4/db/f94 18 1 2024-07-16T09:55:40.797 INFO:tasks.workunit.client.0.smithi156.stdout:8/377: sync 2024-07-16T09:55:40.797 INFO:tasks.workunit.client.0.smithi156.stdout:8/378: stat d0/d11/d20 0 2024-07-16T09:55:40.803 INFO:tasks.workunit.client.1.smithi176.stdout:1/362: dwrite d4/db/d68/d72/f74 [0,4194304] 0 2024-07-16T09:55:40.807 INFO:tasks.workunit.client.1.smithi176.stdout:1/363: creat d4/d8/d12/d21/d25/d30/d44/d4d/f7e x:0 0 0 2024-07-16T09:55:40.808 INFO:tasks.workunit.client.1.smithi176.stdout:1/364: chown d4/d8/l5d 2442395 1 2024-07-16T09:55:40.810 INFO:tasks.workunit.client.1.smithi176.stdout:1/365: mknod d4/db/d68/c7f 0 2024-07-16T09:55:40.811 INFO:tasks.workunit.client.1.smithi176.stdout:1/366: chown d4/d8/f1e 11 1 2024-07-16T09:55:40.868 INFO:tasks.workunit.client.0.smithi156.stdout:7/412: sync 2024-07-16T09:55:40.868 INFO:tasks.workunit.client.0.smithi156.stdout:1/627: sync 2024-07-16T09:55:40.868 INFO:tasks.workunit.client.0.smithi156.stdout:0/512: sync 2024-07-16T09:55:40.869 INFO:tasks.workunit.client.0.smithi156.stdout:1/628: write dc/d24/f3a [2365398,35361] 0 2024-07-16T09:55:40.872 INFO:tasks.workunit.client.0.smithi156.stdout:1/629: dread dc/f37 [0,4194304] 0 2024-07-16T09:55:40.873 INFO:tasks.workunit.client.0.smithi156.stdout:1/630: chown dc/d14/d3e/d55/d57/f93 1283 1 2024-07-16T09:55:40.873 INFO:tasks.workunit.client.0.smithi156.stdout:1/631: chown dc/d14/d3e/d45/f53 28909 1 2024-07-16T09:55:40.874 INFO:tasks.workunit.client.0.smithi156.stdout:1/632: truncate dc/f95 552627 0 2024-07-16T09:55:40.878 INFO:tasks.workunit.client.0.smithi156.stdout:4/602: rename d4/db/dc/d5f/cbb to d4/d23/d30/d33/cda 0 2024-07-16T09:55:40.879 INFO:tasks.workunit.client.0.smithi156.stdout:8/379: fdatasync d0/dd/f1e 0 2024-07-16T09:55:40.880 INFO:tasks.workunit.client.0.smithi156.stdout:7/413: mkdir dc/d13/d15/d8d 0 2024-07-16T09:55:40.881 INFO:tasks.workunit.client.0.smithi156.stdout:0/513: symlink d0/d1a/d4d/d9/d27/d2c/d6d/la0 0 2024-07-16T09:55:40.882 INFO:tasks.workunit.client.0.smithi156.stdout:7/414: read dc/d44/f5c [2721001,38823] 0 2024-07-16T09:55:40.884 INFO:tasks.workunit.client.0.smithi156.stdout:4/603: dwrite d4/db/dc/d7f/daf/fd4 [0,4194304] 0 2024-07-16T09:55:40.886 INFO:tasks.workunit.client.0.smithi156.stdout:8/380: mknod d0/d11/d29/d5e/c85 0 2024-07-16T09:55:40.889 INFO:tasks.workunit.client.0.smithi156.stdout:0/514: rmdir d0/d1a/d4d/d9/d27/d2c/d6d 39 2024-07-16T09:55:40.891 INFO:tasks.workunit.client.0.smithi156.stdout:4/604: creat d4/db/d8a/fdb x:0 0 0 2024-07-16T09:55:40.892 INFO:tasks.workunit.client.0.smithi156.stdout:8/381: fdatasync d0/dd/f6a 0 2024-07-16T09:55:40.893 INFO:tasks.workunit.client.0.smithi156.stdout:7/415: symlink dc/d44/d81/l8e 0 2024-07-16T09:55:40.894 INFO:tasks.workunit.client.0.smithi156.stdout:8/382: write d0/f6f [448314,65765] 0 2024-07-16T09:55:40.896 INFO:tasks.workunit.client.0.smithi156.stdout:4/605: creat d4/db/dc/d7f/fdc x:0 0 0 2024-07-16T09:55:40.896 INFO:tasks.workunit.client.0.smithi156.stdout:1/633: getdents dc/d6b/d8a 0 2024-07-16T09:55:40.896 INFO:tasks.workunit.client.0.smithi156.stdout:0/515: dwrite d0/d22/d41/d52/f5d [0,4194304] 0 2024-07-16T09:55:40.897 INFO:tasks.workunit.client.0.smithi156.stdout:8/383: mkdir d0/d74/d86 0 2024-07-16T09:55:40.897 INFO:tasks.workunit.client.0.smithi156.stdout:4/606: dread - d4/d23/d30/fd6 zero size 2024-07-16T09:55:40.901 INFO:tasks.workunit.client.0.smithi156.stdout:5/571: sync 2024-07-16T09:55:40.901 INFO:tasks.workunit.client.0.smithi156.stdout:6/463: sync 2024-07-16T09:55:40.901 INFO:tasks.workunit.client.0.smithi156.stdout:9/552: sync 2024-07-16T09:55:40.901 INFO:tasks.workunit.client.0.smithi156.stdout:2/565: sync 2024-07-16T09:55:40.901 INFO:tasks.workunit.client.0.smithi156.stdout:3/487: sync 2024-07-16T09:55:40.902 INFO:tasks.workunit.client.0.smithi156.stdout:0/516: fsync d0/d22/d41/f47 0 2024-07-16T09:55:40.903 INFO:tasks.workunit.client.0.smithi156.stdout:4/607: mkdir d4/db/dc/dcc/d58/d8f/ddd 0 2024-07-16T09:55:40.905 INFO:tasks.workunit.client.0.smithi156.stdout:7/416: dread dc/d13/d20/f3b [0,4194304] 0 2024-07-16T09:55:40.905 INFO:tasks.workunit.client.0.smithi156.stdout:8/384: symlink d0/d11/d20/d7f/l87 0 2024-07-16T09:55:40.909 INFO:tasks.workunit.client.0.smithi156.stdout:0/517: stat d0/d1a/d4d/d9/d27/d2c/c91 0 2024-07-16T09:55:40.910 INFO:tasks.workunit.client.0.smithi156.stdout:7/417: chown dc/d3e/d3f/d51/d6c 6872955 1 2024-07-16T09:55:40.913 INFO:tasks.workunit.client.0.smithi156.stdout:2/566: dread d0/d13/d17/d1c/f24 [0,4194304] 0 2024-07-16T09:55:40.918 INFO:tasks.workunit.client.0.smithi156.stdout:1/634: dwrite dc/d14/d1e/fc7 [0,4194304] 0 2024-07-16T09:55:40.921 INFO:tasks.workunit.client.0.smithi156.stdout:4/608: rename d4/db/dc/c74 to d4/d23/d30/dd2/cde 0 2024-07-16T09:55:40.922 INFO:tasks.workunit.client.0.smithi156.stdout:7/418: mknod dc/c8f 0 2024-07-16T09:55:40.923 INFO:tasks.workunit.client.0.smithi156.stdout:5/572: dwrite d1b/d1f/d42/f5c [0,4194304] 0 2024-07-16T09:55:40.926 INFO:tasks.workunit.client.0.smithi156.stdout:3/488: dwrite d8/d14/f6b [0,4194304] 0 2024-07-16T09:55:40.930 INFO:tasks.workunit.client.0.smithi156.stdout:1/635: mkdir dc/d6b/dd5 0 2024-07-16T09:55:40.930 INFO:tasks.workunit.client.0.smithi156.stdout:3/489: write d8/d14/f6b [1888229,14085] 0 2024-07-16T09:55:40.930 INFO:tasks.workunit.client.0.smithi156.stdout:2/567: mkdir d0/d5/d29/d2d/d5b/d62/d74/dc3 0 2024-07-16T09:55:40.930 INFO:tasks.workunit.client.0.smithi156.stdout:1/636: stat dc/f28 0 2024-07-16T09:55:40.930 INFO:tasks.workunit.client.0.smithi156.stdout:9/553: dread d2/d5/d16/d22/d36/d33/f2c [0,4194304] 0 2024-07-16T09:55:40.931 INFO:tasks.workunit.client.0.smithi156.stdout:0/518: write d0/d1a/d4d/f6b [4152179,59395] 0 2024-07-16T09:55:40.935 INFO:tasks.workunit.client.0.smithi156.stdout:6/464: link d3/d13/d17/d4f/d5c/l79 d3/d13/d17/d4f/d94/d19/d31/l9a 0 2024-07-16T09:55:40.935 INFO:tasks.workunit.client.0.smithi156.stdout:3/490: fsync d8/d14/f30 0 2024-07-16T09:55:40.935 INFO:tasks.workunit.client.0.smithi156.stdout:5/573: fdatasync d1b/d1f/d45/d52/d49/f53 0 2024-07-16T09:55:40.936 INFO:tasks.workunit.client.0.smithi156.stdout:1/637: creat dc/d14/d8b/db4/fd6 x:0 0 0 2024-07-16T09:55:40.937 INFO:tasks.workunit.client.0.smithi156.stdout:7/419: mknod dc/c90 0 2024-07-16T09:55:40.937 INFO:tasks.workunit.client.0.smithi156.stdout:6/465: chown d3/d13/d17/d56/l8d 2089216 1 2024-07-16T09:55:40.938 INFO:tasks.workunit.client.0.smithi156.stdout:0/519: mkdir d0/d1a/d4d/d61/da1 0 2024-07-16T09:55:40.938 INFO:tasks.workunit.client.0.smithi156.stdout:7/420: readlink dc/l8c 0 2024-07-16T09:55:40.938 INFO:tasks.workunit.client.0.smithi156.stdout:8/385: dread d0/dd/f17 [0,4194304] 0 2024-07-16T09:55:40.940 INFO:tasks.workunit.client.0.smithi156.stdout:3/491: symlink d8/da/d28/laa 0 2024-07-16T09:55:40.940 INFO:tasks.workunit.client.0.smithi156.stdout:2/568: truncate d0/d5/d29/d2d/d5b/f9c 928375 0 2024-07-16T09:55:40.941 INFO:tasks.workunit.client.0.smithi156.stdout:4/609: creat d4/d23/fdf x:0 0 0 2024-07-16T09:55:40.941 INFO:tasks.workunit.client.0.smithi156.stdout:9/554: mknod d2/d5/d16/d22/d36/d33/d23/cad 0 2024-07-16T09:55:40.941 INFO:tasks.workunit.client.0.smithi156.stdout:5/574: mknod d1b/d1f/d45/d52/cc8 0 2024-07-16T09:55:40.942 INFO:tasks.workunit.client.0.smithi156.stdout:5/575: stat d1b/d1f/d42/l83 0 2024-07-16T09:55:40.943 INFO:tasks.workunit.client.1.smithi176.stdout:4/547: sync 2024-07-16T09:55:40.943 INFO:tasks.workunit.client.1.smithi176.stdout:5/600: sync 2024-07-16T09:55:40.943 INFO:tasks.workunit.client.1.smithi176.stdout:6/538: sync 2024-07-16T09:55:40.943 INFO:tasks.workunit.client.1.smithi176.stdout:2/591: sync 2024-07-16T09:55:40.943 INFO:tasks.workunit.client.1.smithi176.stdout:9/641: sync 2024-07-16T09:55:40.943 INFO:tasks.workunit.client.1.smithi176.stdout:8/560: sync 2024-07-16T09:55:40.944 INFO:tasks.workunit.client.0.smithi156.stdout:1/638: rename lb to dc/d14/d3e/d6c/ld7 0 2024-07-16T09:55:40.945 INFO:tasks.workunit.client.0.smithi156.stdout:1/639: write dc/d14/d3e/d6c/f73 [2416551,45971] 0 2024-07-16T09:55:40.946 INFO:tasks.workunit.client.1.smithi176.stdout:2/592: chown dd/d2e/f90 16274 1 2024-07-16T09:55:40.946 INFO:tasks.workunit.client.1.smithi176.stdout:9/642: chown d5/d1d/d77/d97/lc9 87131326 1 2024-07-16T09:55:40.946 INFO:tasks.workunit.client.0.smithi156.stdout:0/520: mknod d0/d22/ca2 0 2024-07-16T09:55:40.947 INFO:tasks.workunit.client.1.smithi176.stdout:8/561: creat d8/d20/d2a/d31/db4/fc0 x:0 0 0 2024-07-16T09:55:40.949 INFO:tasks.workunit.client.0.smithi156.stdout:2/569: dread - d0/d13/f6b zero size 2024-07-16T09:55:40.949 INFO:tasks.workunit.client.0.smithi156.stdout:4/610: mkdir d4/d23/d52/d71/de0 0 2024-07-16T09:55:40.950 INFO:tasks.workunit.client.1.smithi176.stdout:9/643: creat d5/d1d/db0/fde x:0 0 0 2024-07-16T09:55:40.951 INFO:tasks.workunit.client.1.smithi176.stdout:2/593: symlink dd/d49/d52/d7c/d81/lc5 0 2024-07-16T09:55:40.951 INFO:tasks.workunit.client.0.smithi156.stdout:5/576: unlink d1b/d1f/d45/d7a/f8b 0 2024-07-16T09:55:40.953 INFO:tasks.workunit.client.0.smithi156.stdout:1/640: dwrite dc/f95 [0,4194304] 0 2024-07-16T09:55:40.953 INFO:tasks.workunit.client.1.smithi176.stdout:6/539: truncate d1/d5/d47/fac 454525 0 2024-07-16T09:55:40.953 INFO:tasks.workunit.client.0.smithi156.stdout:8/386: creat d0/d11/d29/d80/f88 x:0 0 0 2024-07-16T09:55:40.955 INFO:tasks.workunit.client.0.smithi156.stdout:2/570: dread - d0/d5/d76/f7b zero size 2024-07-16T09:55:40.956 INFO:tasks.workunit.client.1.smithi176.stdout:6/540: creat d1/dc/d40/fca x:0 0 0 2024-07-16T09:55:40.958 INFO:tasks.workunit.client.1.smithi176.stdout:2/594: creat dd/d2e/d35/fc6 x:0 0 0 2024-07-16T09:55:40.960 INFO:tasks.workunit.client.1.smithi176.stdout:6/541: creat d1/d5d/db9/fcb x:0 0 0 2024-07-16T09:55:40.960 INFO:tasks.workunit.client.1.smithi176.stdout:2/595: read - dd/d16/d39/d3d/d7e/fba zero size 2024-07-16T09:55:40.961 INFO:tasks.workunit.client.0.smithi156.stdout:2/571: creat d0/d5/d76/fc4 x:0 0 0 2024-07-16T09:55:40.961 INFO:tasks.workunit.client.1.smithi176.stdout:5/601: write d4/d37/d72/d38/f87 [988839,85856] 0 2024-07-16T09:55:40.962 INFO:tasks.workunit.client.0.smithi156.stdout:5/577: symlink d1b/d1f/d54/dc6/lc9 0 2024-07-16T09:55:40.963 INFO:tasks.workunit.client.1.smithi176.stdout:4/548: truncate da/f47 751096 0 2024-07-16T09:55:40.963 INFO:tasks.workunit.client.1.smithi176.stdout:6/542: dread - d1/d5d/d9e/d76/da5/fb7 zero size 2024-07-16T09:55:40.964 INFO:tasks.workunit.client.1.smithi176.stdout:2/596: symlink dd/d49/d52/d89/lc7 0 2024-07-16T09:55:40.964 INFO:tasks.workunit.client.0.smithi156.stdout:1/641: read dc/d14/d3e/d6c/f73 [3195913,18029] 0 2024-07-16T09:55:40.964 INFO:tasks.workunit.client.0.smithi156.stdout:9/555: getdents d2/d5/d16/d22/d2b/d3d/d3c/d7f/d42 0 2024-07-16T09:55:40.965 INFO:tasks.workunit.client.1.smithi176.stdout:5/602: dread - d4/d9a/fa7 zero size 2024-07-16T09:55:40.965 INFO:tasks.workunit.client.0.smithi156.stdout:9/556: fdatasync d2/d5/f44 0 2024-07-16T09:55:40.966 INFO:tasks.workunit.client.0.smithi156.stdout:8/387: mknod d0/dd/d1f/d50/c89 0 2024-07-16T09:55:40.967 INFO:tasks.workunit.client.1.smithi176.stdout:2/597: write dd/d49/f78 [499942,12649] 0 2024-07-16T09:55:40.968 INFO:tasks.workunit.client.1.smithi176.stdout:4/549: symlink da/d3d/d17/d54/lab 0 2024-07-16T09:55:40.968 INFO:tasks.workunit.client.1.smithi176.stdout:5/603: read - d4/d10/fb9 zero size 2024-07-16T09:55:40.968 INFO:tasks.workunit.client.0.smithi156.stdout:2/572: chown d0/d5/d29/d2d/d5b/d5f/d89/f99 311495541 1 2024-07-16T09:55:40.970 INFO:tasks.workunit.client.0.smithi156.stdout:5/578: dwrite d1b/d1f/d45/d52/d43/d85/d97/d37/f5a [0,4194304] 0 2024-07-16T09:55:40.970 INFO:tasks.workunit.client.1.smithi176.stdout:6/543: read d1/d5/d7/d1e/f69 [3698104,103859] 0 2024-07-16T09:55:40.971 INFO:tasks.workunit.client.1.smithi176.stdout:4/550: readlink da/d3d/l67 0 2024-07-16T09:55:40.971 INFO:tasks.workunit.client.1.smithi176.stdout:2/598: write dd/d16/f27 [4815997,14363] 0 2024-07-16T09:55:40.971 INFO:tasks.workunit.client.1.smithi176.stdout:5/604: stat d4/d37/d88 0 2024-07-16T09:55:40.972 INFO:tasks.workunit.client.0.smithi156.stdout:7/421: write dc/d44/f5c [5093010,36345] 0 2024-07-16T09:55:40.973 INFO:tasks.workunit.client.0.smithi156.stdout:1/642: creat dc/d6b/fd8 x:0 0 0 2024-07-16T09:55:40.973 INFO:tasks.workunit.client.1.smithi176.stdout:4/551: fdatasync da/dc/d4b/f97 0 2024-07-16T09:55:40.974 INFO:tasks.workunit.client.1.smithi176.stdout:9/644: dwrite d5/d1d/dd2/f75 [4194304,4194304] 0 2024-07-16T09:55:40.976 INFO:tasks.workunit.client.1.smithi176.stdout:5/605: write d4/d37/d72/d38/d39/fa1 [608098,113577] 0 2024-07-16T09:55:40.977 INFO:tasks.workunit.client.1.smithi176.stdout:5/606: dread - d4/d37/d72/f59 zero size 2024-07-16T09:55:40.978 INFO:tasks.workunit.client.1.smithi176.stdout:6/544: mknod d1/d5/ccc 0 2024-07-16T09:55:40.979 INFO:tasks.workunit.client.0.smithi156.stdout:3/492: mknod d8/da/d28/cab 0 2024-07-16T09:55:40.980 INFO:tasks.workunit.client.1.smithi176.stdout:5/607: creat d4/d14/d19/fd6 x:0 0 0 2024-07-16T09:55:40.980 INFO:tasks.workunit.client.0.smithi156.stdout:6/466: dread d3/d13/d17/d4f/d94/d19/d31/f4e [0,4194304] 0 2024-07-16T09:55:40.981 INFO:tasks.workunit.client.1.smithi176.stdout:6/545: stat d1/d5/c3f 0 2024-07-16T09:55:40.983 INFO:tasks.workunit.client.0.smithi156.stdout:3/493: write d8/da/d1d/d2c/f84 [370088,85492] 0 2024-07-16T09:55:40.983 INFO:tasks.workunit.client.0.smithi156.stdout:6/467: truncate d3/d13/d17/f8a 930790 0 2024-07-16T09:55:40.984 INFO:tasks.workunit.client.1.smithi176.stdout:9/645: getdents d5/d1d/dc7/d89 0 2024-07-16T09:55:40.986 INFO:tasks.workunit.client.1.smithi176.stdout:5/608: dread d4/d37/dbb/fd3 [0,4194304] 0 2024-07-16T09:55:40.986 INFO:tasks.workunit.client.0.smithi156.stdout:1/643: rename dc/d14/d3e/d55/d83/fa4 to dc/d6b/dd5/fd9 0 2024-07-16T09:55:40.987 INFO:tasks.workunit.client.1.smithi176.stdout:6/546: creat d1/d5/d47/d62/d77/fcd x:0 0 0 2024-07-16T09:55:40.988 INFO:tasks.workunit.client.0.smithi156.stdout:2/573: dread d0/d5/d29/d2d/d5b/d5f/f7d [0,4194304] 0 2024-07-16T09:55:40.989 INFO:tasks.workunit.client.0.smithi156.stdout:0/521: truncate d0/d22/d41/f4a 2250578 0 2024-07-16T09:55:40.990 INFO:tasks.workunit.client.0.smithi156.stdout:9/557: dread d2/d5/fb [0,4194304] 0 2024-07-16T09:55:40.992 INFO:tasks.workunit.client.0.smithi156.stdout:8/388: rmdir d0/d74/d86 0 2024-07-16T09:55:40.992 INFO:tasks.workunit.client.0.smithi156.stdout:4/611: truncate d4/d23/d30/f2d 748799 0 2024-07-16T09:55:40.992 INFO:tasks.workunit.client.1.smithi176.stdout:6/547: symlink d1/d5/d47/d62/d80/lce 0 2024-07-16T09:55:40.995 INFO:tasks.workunit.client.0.smithi156.stdout:9/558: dread - d2/d5/d16/d22/d6b/f92 zero size 2024-07-16T09:55:40.996 INFO:tasks.workunit.client.1.smithi176.stdout:2/599: write dd/d2e/f37 [4025049,28756] 0 2024-07-16T09:55:40.996 INFO:tasks.workunit.client.0.smithi156.stdout:3/494: dwrite d8/da/d1d/d2c/d33/fa9 [0,4194304] 0 2024-07-16T09:55:40.998 INFO:tasks.workunit.client.1.smithi176.stdout:2/600: read - dd/d2e/d35/fc6 zero size 2024-07-16T09:55:40.998 INFO:tasks.workunit.client.1.smithi176.stdout:9/646: dwrite d5/de/f15 [4194304,4194304] 0 2024-07-16T09:55:40.999 INFO:tasks.workunit.client.0.smithi156.stdout:0/522: mkdir d0/d1a/da3 0 2024-07-16T09:55:41.000 INFO:tasks.workunit.client.0.smithi156.stdout:2/574: mkdir d0/d5/dc5 0 2024-07-16T09:55:41.000 INFO:tasks.workunit.client.0.smithi156.stdout:1/644: chown dc/d6b/f75 0 1 2024-07-16T09:55:41.001 INFO:tasks.workunit.client.1.smithi176.stdout:9/647: truncate d5/d1d/d67/fab 184208 0 2024-07-16T09:55:41.001 INFO:tasks.workunit.client.0.smithi156.stdout:2/575: chown d0 1282948205 1 2024-07-16T09:55:41.003 INFO:tasks.workunit.client.1.smithi176.stdout:2/601: fsync dd/d2e/d35/f47 0 2024-07-16T09:55:41.004 INFO:tasks.workunit.client.1.smithi176.stdout:9/648: creat d5/d1d/d67/fdf x:0 0 0 2024-07-16T09:55:41.008 INFO:tasks.workunit.client.1.smithi176.stdout:2/602: mkdir dd/d2e/d35/d59/d66/dc8 0 2024-07-16T09:55:41.008 INFO:tasks.workunit.client.0.smithi156.stdout:6/468: rename d3/d13/d17/d4f/d94/l3a to d3/d13/d17/d4f/d4d/l9b 0 2024-07-16T09:55:41.009 INFO:tasks.workunit.client.0.smithi156.stdout:3/495: symlink d8/lac 0 2024-07-16T09:55:41.010 INFO:tasks.workunit.client.0.smithi156.stdout:5/579: write d1b/d1f/d54/d72/f79 [419972,46888] 0 2024-07-16T09:55:41.010 INFO:tasks.workunit.client.0.smithi156.stdout:1/645: mknod dc/d24/cda 0 2024-07-16T09:55:41.012 INFO:tasks.workunit.client.0.smithi156.stdout:8/389: dwrite d0/f78 [0,4194304] 0 2024-07-16T09:55:41.012 INFO:tasks.workunit.client.1.smithi176.stdout:6/548: truncate d1/d5d/d9e/d8f/f8d 3340890 0 2024-07-16T09:55:41.014 INFO:tasks.workunit.client.0.smithi156.stdout:4/612: dwrite d4/db/dc/f68 [4194304,4194304] 0 2024-07-16T09:55:41.015 INFO:tasks.workunit.client.1.smithi176.stdout:9/649: dwrite d5/de/f61 [0,4194304] 0 2024-07-16T09:55:41.015 INFO:tasks.workunit.client.1.smithi176.stdout:6/549: fdatasync d1/f64 0 2024-07-16T09:55:41.016 INFO:tasks.workunit.client.0.smithi156.stdout:0/523: readlink d0/d1a/d4d/d9/d27/d3d/d62/l71 0 2024-07-16T09:55:41.018 INFO:tasks.workunit.client.0.smithi156.stdout:9/559: creat d2/d5/d16/fae x:0 0 0 2024-07-16T09:55:41.019 INFO:tasks.workunit.client.0.smithi156.stdout:9/560: stat d2/d5/d16/f9d 0 2024-07-16T09:55:41.019 INFO:tasks.workunit.client.1.smithi176.stdout:2/603: rename dd/d2e/f90 to dd/d16/d39/fc9 0 2024-07-16T09:55:41.019 INFO:tasks.workunit.client.1.smithi176.stdout:5/609: dread f2 [0,4194304] 0 2024-07-16T09:55:41.021 INFO:tasks.workunit.client.1.smithi176.stdout:9/650: symlink d5/d52/le0 0 2024-07-16T09:55:41.022 INFO:tasks.workunit.client.0.smithi156.stdout:8/390: fsync d0/dd/f17 0 2024-07-16T09:55:41.025 INFO:tasks.workunit.client.1.smithi176.stdout:5/610: dread d4/d14/f45 [0,4194304] 0 2024-07-16T09:55:41.025 INFO:tasks.workunit.client.0.smithi156.stdout:4/613: rmdir d4/db/dc/d7f 39 2024-07-16T09:55:41.026 INFO:tasks.workunit.client.1.smithi176.stdout:6/550: rename d1/dc/d81/lc5 to d1/d5/d7/d1e/lcf 0 2024-07-16T09:55:41.026 INFO:tasks.workunit.client.1.smithi176.stdout:2/604: symlink dd/d2e/d35/lca 0 2024-07-16T09:55:41.028 INFO:tasks.workunit.client.0.smithi156.stdout:5/580: fdatasync d1b/d1f/d8a/fa6 0 2024-07-16T09:55:41.029 INFO:tasks.workunit.client.0.smithi156.stdout:5/581: readlink d1b/d1f/l29 0 2024-07-16T09:55:41.035 INFO:tasks.workunit.client.1.smithi176.stdout:2/605: getdents dd/d49/d52/d7c/d81 0 2024-07-16T09:55:41.039 INFO:tasks.workunit.client.1.smithi176.stdout:6/551: dwrite d1/d5/d47/f51 [8388608,4194304] 0 2024-07-16T09:55:41.039 INFO:tasks.workunit.client.1.smithi176.stdout:2/606: write dd/d16/d39/d3d/d7e/fba [315030,111715] 0 2024-07-16T09:55:41.039 INFO:tasks.workunit.client.0.smithi156.stdout:5/582: mkdir d1b/d1f/d54/d72/dca 0 2024-07-16T09:55:41.042 INFO:tasks.workunit.client.1.smithi176.stdout:6/552: dread - d1/dc/d72/d6f/d7c/fa2 zero size 2024-07-16T09:55:41.042 INFO:tasks.workunit.client.0.smithi156.stdout:1/646: creat dc/fdb x:0 0 0 2024-07-16T09:55:41.044 INFO:tasks.workunit.client.0.smithi156.stdout:2/576: getdents d0/d5/d29 0 2024-07-16T09:55:41.045 INFO:tasks.workunit.client.1.smithi176.stdout:2/607: mknod dd/d2e/ccb 0 2024-07-16T09:55:41.045 INFO:tasks.workunit.client.1.smithi176.stdout:5/611: dwrite d4/d14/f25 [0,4194304] 0 2024-07-16T09:55:41.046 INFO:tasks.workunit.client.1.smithi176.stdout:6/553: mkdir d1/dc/d72/d6f/dd0 0 2024-07-16T09:55:41.046 INFO:tasks.workunit.client.0.smithi156.stdout:8/391: truncate d0/dd/f17 3913758 0 2024-07-16T09:55:41.047 INFO:tasks.workunit.client.1.smithi176.stdout:2/608: write dd/d9b/fb1 [663951,51168] 0 2024-07-16T09:55:41.047 INFO:tasks.workunit.client.0.smithi156.stdout:4/614: mknod d4/db/dc/dcc/d58/d8f/ddd/ce1 0 2024-07-16T09:55:41.048 INFO:tasks.workunit.client.0.smithi156.stdout:4/615: dread - d4/d23/d30/fd6 zero size 2024-07-16T09:55:41.049 INFO:tasks.workunit.client.1.smithi176.stdout:5/612: dread - d4/d37/d72/d38/d39/fa3 zero size 2024-07-16T09:55:41.049 INFO:tasks.workunit.client.0.smithi156.stdout:6/469: creat d3/d13/d17/d4f/d94/d19/d31/f9c x:0 0 0 2024-07-16T09:55:41.051 INFO:tasks.workunit.client.0.smithi156.stdout:1/647: mkdir dc/d6b/d8a/ddc 0 2024-07-16T09:55:41.052 INFO:tasks.workunit.client.1.smithi176.stdout:6/554: truncate d1/d5d/db9/f32 3138872 0 2024-07-16T09:55:41.057 INFO:tasks.workunit.client.0.smithi156.stdout:8/392: fdatasync d0/d11/d2e/f5c 0 2024-07-16T09:55:41.058 INFO:tasks.workunit.client.0.smithi156.stdout:8/393: read - d0/d11/d62/f83 zero size 2024-07-16T09:55:41.058 INFO:tasks.workunit.client.1.smithi176.stdout:6/555: getdents d1/d5/d47/d75 0 2024-07-16T09:55:41.059 INFO:tasks.workunit.client.1.smithi176.stdout:5/613: dwrite d4/d9a/fa7 [0,4194304] 0 2024-07-16T09:55:41.060 INFO:tasks.workunit.client.0.smithi156.stdout:1/648: readlink dc/d24/l40 0 2024-07-16T09:55:41.060 INFO:tasks.workunit.client.0.smithi156.stdout:9/561: link d2/d5/d16/d22/d6b/d86/f8c d2/d5/d16/d22/d36/d33/d23/da0/faf 0 2024-07-16T09:55:41.061 INFO:tasks.workunit.client.1.smithi176.stdout:6/556: fsync d1/dc/d72/d6f/d7c/fc3 0 2024-07-16T09:55:41.064 INFO:tasks.workunit.client.1.smithi176.stdout:5/614: creat d4/d37/fd7 x:0 0 0 2024-07-16T09:55:41.064 INFO:tasks.workunit.client.0.smithi156.stdout:5/583: rename d1b/d1f/f51 to d1b/d1f/fcb 0 2024-07-16T09:55:41.064 INFO:tasks.workunit.client.1.smithi176.stdout:6/557: creat d1/d5d/d9e/fd1 x:0 0 0 2024-07-16T09:55:41.065 INFO:tasks.workunit.client.1.smithi176.stdout:5/615: stat d4/d9a/lc9 0 2024-07-16T09:55:41.067 INFO:tasks.workunit.client.0.smithi156.stdout:1/649: dwrite dc/d14/d3e/d55/d83/fa1 [0,4194304] 0 2024-07-16T09:55:41.068 INFO:tasks.workunit.client.0.smithi156.stdout:1/650: chown dc/f28 24975604 1 2024-07-16T09:55:41.068 INFO:tasks.workunit.client.1.smithi176.stdout:5/616: write d4/d37/d72/d38/d6e/f74 [2220671,62660] 0 2024-07-16T09:55:41.071 INFO:tasks.workunit.client.1.smithi176.stdout:5/617: dread - d4/d37/d72/d38/f97 zero size 2024-07-16T09:55:41.072 INFO:tasks.workunit.client.0.smithi156.stdout:2/577: getdents d0/d5/d29/d2d 0 2024-07-16T09:55:41.073 INFO:tasks.workunit.client.1.smithi176.stdout:6/558: dwrite d1/f45 [0,4194304] 0 2024-07-16T09:55:41.073 INFO:tasks.workunit.client.0.smithi156.stdout:8/394: mkdir d0/d11/d13/d1c/d39/d8a 0 2024-07-16T09:55:41.074 INFO:tasks.workunit.client.1.smithi176.stdout:2/609: write dd/d36/d82/f8d [483066,6186] 0 2024-07-16T09:55:41.074 INFO:tasks.workunit.client.0.smithi156.stdout:9/562: dwrite d2/d5/d16/d22/d6b/f97 [0,4194304] 0 2024-07-16T09:55:41.076 INFO:tasks.workunit.client.0.smithi156.stdout:5/584: rename d1b/d1f/d42/d6d/d9f/db2 to d1b/d1f/d54/dc6/dcc 0 2024-07-16T09:55:41.078 INFO:tasks.workunit.client.1.smithi176.stdout:2/610: chown dd/d2e/d35/d59/d66/f8f 1115 1 2024-07-16T09:55:41.082 INFO:tasks.workunit.client.1.smithi176.stdout:5/618: symlink d4/d37/d72/d38/d6e/dbe/ld8 0 2024-07-16T09:55:41.082 INFO:tasks.workunit.client.0.smithi156.stdout:2/578: dwrite d0/d13/d6e/fa2 [0,4194304] 0 2024-07-16T09:55:41.083 INFO:tasks.workunit.client.0.smithi156.stdout:1/651: mknod dc/d14/d3e/d6c/d7a/cdd 0 2024-07-16T09:55:41.084 INFO:tasks.workunit.client.0.smithi156.stdout:4/616: link d4/d23/d30/d33/lce d4/d23/le2 0 2024-07-16T09:55:41.084 INFO:tasks.workunit.client.0.smithi156.stdout:2/579: fsync d0/d9b/fba 0 2024-07-16T09:55:41.086 INFO:tasks.workunit.client.1.smithi176.stdout:2/611: unlink dd/d2e/f83 0 2024-07-16T09:55:41.087 INFO:tasks.workunit.client.1.smithi176.stdout:6/559: getdents d1/dc/d40 0 2024-07-16T09:55:41.092 INFO:tasks.workunit.client.0.smithi156.stdout:2/580: mkdir d0/d13/d17/d1c/d51/dc6 0 2024-07-16T09:55:41.096 INFO:tasks.workunit.client.0.smithi156.stdout:9/563: link d2/d5/d16/d22/d2b/d74/l8e d2/d5/d61/lb0 0 2024-07-16T09:55:41.097 INFO:tasks.workunit.client.1.smithi176.stdout:2/612: dwrite dd/d36/f61 [0,4194304] 0 2024-07-16T09:55:41.098 INFO:tasks.workunit.client.0.smithi156.stdout:2/581: mknod d0/d9b/cc7 0 2024-07-16T09:55:41.101 INFO:tasks.workunit.client.1.smithi176.stdout:2/613: creat dd/d49/d52/d7c/fcc x:0 0 0 2024-07-16T09:55:41.103 INFO:tasks.workunit.client.0.smithi156.stdout:4/617: dwrite d4/d23/fdf [0,4194304] 0 2024-07-16T09:55:41.104 INFO:tasks.workunit.client.0.smithi156.stdout:6/470: dwrite d3/d13/d17/d4f/d4d/d3b/f5d [0,4194304] 0 2024-07-16T09:55:41.107 INFO:tasks.workunit.client.0.smithi156.stdout:8/395: dread d0/d11/d13/f14 [0,4194304] 0 2024-07-16T09:55:41.108 INFO:tasks.workunit.client.1.smithi176.stdout:2/614: dwrite dd/d2e/d35/d59/d66/f8f [4194304,4194304] 0 2024-07-16T09:55:41.110 INFO:tasks.workunit.client.0.smithi156.stdout:1/652: dwrite dc/d14/d3e/d6c/d7a/dcb/fce [4194304,4194304] 0 2024-07-16T09:55:41.111 INFO:tasks.workunit.client.0.smithi156.stdout:9/564: rmdir d2/d5/d16/d22/d2b/d3d/d3c/d7f 39 2024-07-16T09:55:41.112 INFO:tasks.workunit.client.0.smithi156.stdout:6/471: truncate d3/d13/d17/f55 4758391 0 2024-07-16T09:55:41.112 INFO:tasks.workunit.client.0.smithi156.stdout:5/585: dread d1b/d1f/d42/f5f [0,4194304] 0 2024-07-16T09:55:41.113 INFO:tasks.workunit.client.0.smithi156.stdout:8/396: symlink d0/d74/l8b 0 2024-07-16T09:55:41.117 INFO:tasks.workunit.client.1.smithi176.stdout:0/641: sync 2024-07-16T09:55:41.118 INFO:tasks.workunit.client.0.smithi156.stdout:1/653: dread dc/d14/d3e/d6c/d7a/fba [0,4194304] 0 2024-07-16T09:55:41.119 INFO:tasks.workunit.client.0.smithi156.stdout:2/582: rename d0/d5/d29/d2d/d5b/d62/fbc to d0/d5/d8b/fc8 0 2024-07-16T09:55:41.120 INFO:tasks.workunit.client.0.smithi156.stdout:2/583: chown d0/d13/d17/d1c/fbd 11 1 2024-07-16T09:55:41.124 INFO:tasks.workunit.client.0.smithi156.stdout:8/397: unlink d0/dd/d1f/d60/f7c 0 2024-07-16T09:55:41.126 INFO:tasks.workunit.client.1.smithi176.stdout:5/619: rmdir d4 39 2024-07-16T09:55:41.127 INFO:tasks.workunit.client.0.smithi156.stdout:5/586: rename c9 to d1b/d1f/d8a/ccd 0 2024-07-16T09:55:41.130 INFO:tasks.workunit.client.0.smithi156.stdout:9/565: mkdir d2/d5/d16/d22/d2b/d3d/db1 0 2024-07-16T09:55:41.132 INFO:tasks.workunit.client.0.smithi156.stdout:9/566: write d2/d5/d16/d22/d6b/f92 [249342,99390] 0 2024-07-16T09:55:41.134 INFO:tasks.workunit.client.0.smithi156.stdout:9/567: write d2/d5/d16/d22/d36/d33/d23/f88 [788261,87461] 0 2024-07-16T09:55:41.136 INFO:tasks.workunit.client.0.smithi156.stdout:6/472: rmdir d3/d13/d17/d4f/d92 0 2024-07-16T09:55:41.138 INFO:tasks.workunit.client.0.smithi156.stdout:8/398: creat d0/dd/d1f/f8c x:0 0 0 2024-07-16T09:55:41.138 INFO:tasks.workunit.client.0.smithi156.stdout:9/568: chown d2/d5/d16/d22/d2b/d3d/d3c/d7f/l18 100 1 2024-07-16T09:55:41.138 INFO:tasks.workunit.client.0.smithi156.stdout:2/584: dread d0/d5/d29/d2d/f31 [0,4194304] 0 2024-07-16T09:55:41.138 INFO:tasks.workunit.client.0.smithi156.stdout:5/587: rmdir d1b/d1f/d45/d9d/da4 0 2024-07-16T09:55:41.139 INFO:tasks.workunit.client.0.smithi156.stdout:6/473: mknod d3/d13/d17/d4f/d4d/d3b/c9d 0 2024-07-16T09:55:41.139 INFO:tasks.workunit.client.0.smithi156.stdout:2/585: chown d0/d13/d17/d1c 2092127 1 2024-07-16T09:55:41.139 INFO:tasks.workunit.client.0.smithi156.stdout:8/399: mkdir d0/dd/d1f/d50/d8d 0 2024-07-16T09:55:41.141 INFO:tasks.workunit.client.0.smithi156.stdout:5/588: mknod d1b/d1f/d45/d52/d2d/cce 0 2024-07-16T09:55:41.141 INFO:tasks.workunit.client.0.smithi156.stdout:6/474: chown d3/d13/d17/d4f/d94/d19/d31/f9c 1 1 2024-07-16T09:55:41.144 INFO:tasks.workunit.client.1.smithi176.stdout:6/560: dread d1/d5/d7/d1e/f79 [0,4194304] 0 2024-07-16T09:55:41.147 INFO:tasks.workunit.client.0.smithi156.stdout:6/475: getdents d3/d13/d17/d4f/d94 0 2024-07-16T09:55:41.149 INFO:tasks.workunit.client.1.smithi176.stdout:6/561: getdents d1/d5d/d9e/d76/da5 0 2024-07-16T09:55:41.150 INFO:tasks.workunit.client.1.smithi176.stdout:2/615: truncate dd/d16/d39/d3d/f75 2005859 0 2024-07-16T09:55:41.150 INFO:tasks.workunit.client.0.smithi156.stdout:6/476: symlink d3/d13/d17/d4f/d4d/d3b/d77/l9e 0 2024-07-16T09:55:41.150 INFO:tasks.workunit.client.1.smithi176.stdout:0/642: write d5/d1c/d24/d35/d50/d9b/fac [4622614,50038] 0 2024-07-16T09:55:41.151 INFO:tasks.workunit.client.0.smithi156.stdout:4/618: write d4/d23/d30/f2d [515161,12784] 0 2024-07-16T09:55:41.152 INFO:tasks.workunit.client.1.smithi176.stdout:2/616: stat dd/d2e/d35/ca4 0 2024-07-16T09:55:41.153 INFO:tasks.workunit.client.1.smithi176.stdout:0/643: chown d5/d1c/d24/d35/d50/d9b/fac 260921748 1 2024-07-16T09:55:41.153 INFO:tasks.workunit.client.0.smithi156.stdout:8/400: dread d0/f2c [0,4194304] 0 2024-07-16T09:55:41.154 INFO:tasks.workunit.client.1.smithi176.stdout:6/562: getdents d1/d5/d47 0 2024-07-16T09:55:41.154 INFO:tasks.workunit.client.1.smithi176.stdout:0/644: chown d5/d1c/d5a/f75 250 1 2024-07-16T09:55:41.155 INFO:tasks.workunit.client.0.smithi156.stdout:8/401: chown d0/dd/f58 10 1 2024-07-16T09:55:41.155 INFO:tasks.workunit.client.1.smithi176.stdout:2/617: write dd/d2e/d35/d59/d66/f8f [9200840,100199] 0 2024-07-16T09:55:41.155 INFO:tasks.workunit.client.0.smithi156.stdout:1/654: truncate dc/d24/f3a 2901629 0 2024-07-16T09:55:41.157 INFO:tasks.workunit.client.1.smithi176.stdout:0/645: fsync d5/d1c/db1/fcd 0 2024-07-16T09:55:41.158 INFO:tasks.workunit.client.0.smithi156.stdout:4/619: fdatasync d4/db/dc/d7f/daf/fd4 0 2024-07-16T09:55:41.158 INFO:tasks.workunit.client.1.smithi176.stdout:2/618: write dd/d16/d39/f4b [1945140,88397] 0 2024-07-16T09:55:41.158 INFO:tasks.workunit.client.1.smithi176.stdout:0/646: chown d5/d44/f85 24731453 1 2024-07-16T09:55:41.159 INFO:tasks.workunit.client.0.smithi156.stdout:6/477: dwrite d3/d13/d17/d4f/d4d/d3b/f47 [4194304,4194304] 0 2024-07-16T09:55:41.160 INFO:tasks.workunit.client.0.smithi156.stdout:4/620: write d4/d23/d52/f85 [720945,10524] 0 2024-07-16T09:55:41.162 INFO:tasks.workunit.client.1.smithi176.stdout:2/619: mkdir dd/dcd 0 2024-07-16T09:55:41.164 INFO:tasks.workunit.client.0.smithi156.stdout:7/422: sync 2024-07-16T09:55:41.165 INFO:tasks.workunit.client.0.smithi156.stdout:6/478: creat d3/d13/d17/d4f/d94/f9f x:0 0 0 2024-07-16T09:55:41.165 INFO:tasks.workunit.client.1.smithi176.stdout:5/620: dread d4/f13 [0,4194304] 0 2024-07-16T09:55:41.166 INFO:tasks.workunit.client.1.smithi176.stdout:0/647: creat d5/d1c/fd7 x:0 0 0 2024-07-16T09:55:41.167 INFO:tasks.workunit.client.1.smithi176.stdout:6/563: dread d1/d5/d47/d62/fc6 [0,4194304] 0 2024-07-16T09:55:41.167 INFO:tasks.workunit.client.1.smithi176.stdout:5/621: mknod d4/d9a/cd9 0 2024-07-16T09:55:41.168 INFO:tasks.workunit.client.0.smithi156.stdout:1/655: creat dc/d14/d3e/fde x:0 0 0 2024-07-16T09:55:41.168 INFO:tasks.workunit.client.1.smithi176.stdout:0/648: write d5/d1c/f95 [630002,107646] 0 2024-07-16T09:55:41.169 INFO:tasks.workunit.client.1.smithi176.stdout:6/564: chown d1/d5/l28 465 1 2024-07-16T09:55:41.170 INFO:tasks.workunit.client.0.smithi156.stdout:7/423: creat dc/d13/d15/f91 x:0 0 0 2024-07-16T09:55:41.170 INFO:tasks.workunit.client.0.smithi156.stdout:5/589: truncate d1b/d1f/d45/d52/d49/d4d/f62 1033485 0 2024-07-16T09:55:41.170 INFO:tasks.workunit.client.1.smithi176.stdout:0/649: write d5/d1c/d24/d35/d59/f74 [103540,76981] 0 2024-07-16T09:55:41.171 INFO:tasks.workunit.client.0.smithi156.stdout:7/424: readlink dc/d13/d15/l19 0 2024-07-16T09:55:41.171 INFO:tasks.workunit.client.0.smithi156.stdout:5/590: fdatasync d1b/f55 0 2024-07-16T09:55:41.172 INFO:tasks.workunit.client.0.smithi156.stdout:1/656: creat dc/d14/d3e/d6c/d7a/dcb/fdf x:0 0 0 2024-07-16T09:55:41.178 INFO:tasks.workunit.client.0.smithi156.stdout:7/425: creat dc/d48/f92 x:0 0 0 2024-07-16T09:55:41.180 INFO:tasks.workunit.client.0.smithi156.stdout:2/586: dwrite d0/d13/d17/d1c/f46 [0,4194304] 0 2024-07-16T09:55:41.180 INFO:tasks.workunit.client.0.smithi156.stdout:9/569: dread d2/d5/d16/d22/d36/f31 [0,4194304] 0 2024-07-16T09:55:41.190 INFO:tasks.workunit.client.0.smithi156.stdout:1/657: creat dc/d14/d8b/db4/fe0 x:0 0 0 2024-07-16T09:55:41.191 INFO:tasks.workunit.client.0.smithi156.stdout:2/587: readlink d0/l6 0 2024-07-16T09:55:41.192 INFO:tasks.workunit.client.0.smithi156.stdout:9/570: readlink d2/d5/d61/lb0 0 2024-07-16T09:55:41.193 INFO:tasks.workunit.client.0.smithi156.stdout:6/479: link d3/d13/d17/d4f/d5c/f65 d3/fa0 0 2024-07-16T09:55:41.194 INFO:tasks.workunit.client.0.smithi156.stdout:5/591: symlink d1b/d1f/lcf 0 2024-07-16T09:55:41.195 INFO:tasks.workunit.client.0.smithi156.stdout:1/658: creat dc/d14/d3e/d6c/d7a/dcb/fe1 x:0 0 0 2024-07-16T09:55:41.195 INFO:tasks.workunit.client.0.smithi156.stdout:2/588: truncate d0/d5/d29/f38 827655 0 2024-07-16T09:55:41.198 INFO:tasks.workunit.client.0.smithi156.stdout:5/592: creat d1b/d1f/d8a/fd0 x:0 0 0 2024-07-16T09:55:41.199 INFO:tasks.workunit.client.0.smithi156.stdout:5/593: chown d1b/d1f/d45/d52/f3c 12318 1 2024-07-16T09:55:41.200 INFO:tasks.workunit.client.0.smithi156.stdout:7/426: getdents dc/d44 0 2024-07-16T09:55:41.201 INFO:tasks.workunit.client.0.smithi156.stdout:7/427: readlink dc/d13/d24/l6e 0 2024-07-16T09:55:41.201 INFO:tasks.workunit.client.0.smithi156.stdout:1/659: symlink dc/d14/le2 0 2024-07-16T09:55:41.202 INFO:tasks.workunit.client.0.smithi156.stdout:2/589: fdatasync d0/d5/d8b/fc8 0 2024-07-16T09:55:41.206 INFO:tasks.workunit.client.0.smithi156.stdout:2/590: mkdir d0/d13/dc9 0 2024-07-16T09:55:41.207 INFO:tasks.workunit.client.0.smithi156.stdout:1/660: symlink dc/d6b/d96/le3 0 2024-07-16T09:55:41.207 INFO:tasks.workunit.client.0.smithi156.stdout:2/591: readlink d0/d5/d29/la6 0 2024-07-16T09:55:41.207 INFO:tasks.workunit.client.0.smithi156.stdout:6/480: creat d3/d13/d17/d4f/d94/d19/d31/fa1 x:0 0 0 2024-07-16T09:55:41.209 INFO:tasks.workunit.client.0.smithi156.stdout:1/661: creat dc/d14/d3e/d55/d83/fe4 x:0 0 0 2024-07-16T09:55:41.210 INFO:tasks.workunit.client.0.smithi156.stdout:5/594: dread d1b/d1f/d45/d52/d43/d85/d97/f25 [0,4194304] 0 2024-07-16T09:55:41.210 INFO:tasks.workunit.client.0.smithi156.stdout:1/662: chown dc/d24/c46 44395 1 2024-07-16T09:55:41.213 INFO:tasks.workunit.client.0.smithi156.stdout:1/663: read dc/d14/f34 [435740,77623] 0 2024-07-16T09:55:41.213 INFO:tasks.workunit.client.0.smithi156.stdout:9/571: write d2/d5/d16/d22/d6b/d86/f8c [2584377,67238] 0 2024-07-16T09:55:41.217 INFO:tasks.workunit.client.0.smithi156.stdout:5/595: truncate d1b/d1f/d45/d52/d43/d85/d97/f48 4727265 0 2024-07-16T09:55:41.217 INFO:tasks.workunit.client.0.smithi156.stdout:6/481: dwrite d3/d13/d17/d4f/d94/d19/d2e/f74 [0,4194304] 0 2024-07-16T09:55:41.218 INFO:tasks.workunit.client.0.smithi156.stdout:5/596: chown d1b/f40 129656 1 2024-07-16T09:55:41.219 INFO:tasks.workunit.client.0.smithi156.stdout:1/664: rename dc/d24/f5e to dc/d6b/dd5/fe5 0 2024-07-16T09:55:41.219 INFO:tasks.workunit.client.0.smithi156.stdout:5/597: readlink d1b/d1f/d45/d9d/lb6 0 2024-07-16T09:55:41.222 INFO:tasks.workunit.client.0.smithi156.stdout:6/482: unlink d3/l24 0 2024-07-16T09:55:41.223 INFO:tasks.workunit.client.0.smithi156.stdout:1/665: readlink dc/d20/l61 0 2024-07-16T09:55:41.224 INFO:tasks.workunit.client.0.smithi156.stdout:1/666: stat dc/d14/d3e/d6c/d7a/dcb/fe1 0 2024-07-16T09:55:41.224 INFO:tasks.workunit.client.0.smithi156.stdout:7/428: dread dc/d13/d20/f2b [4194304,4194304] 0 2024-07-16T09:55:41.224 INFO:tasks.workunit.client.0.smithi156.stdout:5/598: creat d1b/d1f/d45/d7f/fd1 x:0 0 0 2024-07-16T09:55:41.225 INFO:tasks.workunit.client.0.smithi156.stdout:7/429: chown dc/d13/f35 127032 1 2024-07-16T09:55:41.225 INFO:tasks.workunit.client.0.smithi156.stdout:9/572: mknod d2/d5/d16/d22/d2b/d3d/db1/cb2 0 2024-07-16T09:55:41.225 INFO:tasks.workunit.client.0.smithi156.stdout:5/599: write d1b/f31 [3346485,26201] 0 2024-07-16T09:55:41.226 INFO:tasks.workunit.client.0.smithi156.stdout:7/430: chown ca 0 1 2024-07-16T09:55:41.227 INFO:tasks.workunit.client.0.smithi156.stdout:5/600: fsync d1b/d1f/d54/d72/f79 0 2024-07-16T09:55:41.228 INFO:tasks.workunit.client.0.smithi156.stdout:1/667: mknod dc/d6b/dd5/ce6 0 2024-07-16T09:55:41.228 INFO:tasks.workunit.client.0.smithi156.stdout:7/431: dread - dc/d3e/d3f/d51/d6c/f85 zero size 2024-07-16T09:55:41.229 INFO:tasks.workunit.client.0.smithi156.stdout:6/483: unlink d3/d13/d17/d4f/d4d/l9b 0 2024-07-16T09:55:41.229 INFO:tasks.workunit.client.0.smithi156.stdout:9/573: write d2/d5/d16/d22/d36/f78 [1540978,14394] 0 2024-07-16T09:55:41.231 INFO:tasks.workunit.client.0.smithi156.stdout:5/601: symlink d1b/d1f/d42/d6d/ld2 0 2024-07-16T09:55:41.232 INFO:tasks.workunit.client.0.smithi156.stdout:1/668: creat dc/d14/d3e/d6c/d7a/fe7 x:0 0 0 2024-07-16T09:55:41.235 INFO:tasks.workunit.client.0.smithi156.stdout:5/602: mknod d1b/d1f/d45/d52/d43/d7c/cd3 0 2024-07-16T09:55:41.236 INFO:tasks.workunit.client.0.smithi156.stdout:9/574: symlink d2/d5/d16/d22/lb3 0 2024-07-16T09:55:41.236 INFO:tasks.workunit.client.1.smithi176.stdout:6/565: mkdir d1/d5/d47/d62/d77/dd2 0 2024-07-16T09:55:41.237 INFO:tasks.workunit.client.1.smithi176.stdout:0/650: creat d5/d1c/d24/d35/d50/d58/d63/fd8 x:0 0 0 2024-07-16T09:55:41.238 INFO:tasks.workunit.client.0.smithi156.stdout:1/669: unlink dc/d14/d3e/d6c/d7a/fa5 0 2024-07-16T09:55:41.239 INFO:tasks.workunit.client.1.smithi176.stdout:6/566: unlink d1/d5d/db9/ca4 0 2024-07-16T09:55:41.240 INFO:tasks.workunit.client.0.smithi156.stdout:6/484: rmdir d3/d13/d17/d4f/d4d 39 2024-07-16T09:55:41.240 INFO:tasks.workunit.client.1.smithi176.stdout:0/651: creat d5/d39/da3/fd9 x:0 0 0 2024-07-16T09:55:41.240 INFO:tasks.workunit.client.0.smithi156.stdout:6/485: fdatasync d3/d13/d17/d4f/d94/d19/d2e/f97 0 2024-07-16T09:55:41.241 INFO:tasks.workunit.client.1.smithi176.stdout:5/622: dwrite d4/d14/d19/d1b/d61/f76 [0,4194304] 0 2024-07-16T09:55:41.242 INFO:tasks.workunit.client.0.smithi156.stdout:6/486: write d3/d13/d17/d4f/d94/d19/d31/d35/f91 [497324,22332] 0 2024-07-16T09:55:41.242 INFO:tasks.workunit.client.1.smithi176.stdout:6/567: creat d1/d5/d7/d54/fd3 x:0 0 0 2024-07-16T09:55:41.243 INFO:tasks.workunit.client.1.smithi176.stdout:0/652: read d5/d2c/f4d [3914372,86905] 0 2024-07-16T09:55:41.244 INFO:tasks.workunit.client.0.smithi156.stdout:9/575: rename d2/d5/d16/d57/l94 to d2/d5/d16/d57/da3/lb4 0 2024-07-16T09:55:41.249 INFO:tasks.workunit.client.0.smithi156.stdout:7/432: link dc/d3e/d3f/d45/l49 dc/d3e/d3f/d51/d8a/l93 0 2024-07-16T09:55:41.255 INFO:tasks.workunit.client.1.smithi176.stdout:5/623: dwrite d4/d37/d72/d38/d39/f3a [0,4194304] 0 2024-07-16T09:55:41.255 INFO:tasks.workunit.client.0.smithi156.stdout:6/487: creat d3/d13/d17/d4f/d94/d19/d31/fa2 x:0 0 0 2024-07-16T09:55:41.256 INFO:tasks.workunit.client.0.smithi156.stdout:9/576: dread d2/d5/f44 [0,4194304] 0 2024-07-16T09:55:41.257 INFO:tasks.workunit.client.1.smithi176.stdout:0/653: dwrite d5/d25/d82/f87 [0,4194304] 0 2024-07-16T09:55:41.257 INFO:tasks.workunit.client.0.smithi156.stdout:9/577: read d2/d5/f44 [1107462,92659] 0 2024-07-16T09:55:41.257 INFO:tasks.workunit.client.0.smithi156.stdout:7/433: creat dc/f94 x:0 0 0 2024-07-16T09:55:41.258 INFO:tasks.workunit.client.1.smithi176.stdout:6/568: dwrite d1/d5d/db9/fb6 [0,4194304] 0 2024-07-16T09:55:41.260 INFO:tasks.workunit.client.1.smithi176.stdout:6/569: read - d1/d5/d47/d75/fb5 zero size 2024-07-16T09:55:41.261 INFO:tasks.workunit.client.0.smithi156.stdout:6/488: dread d3/d13/d17/d4f/d94/d19/d31/f4e [0,4194304] 0 2024-07-16T09:55:41.263 INFO:tasks.workunit.client.1.smithi176.stdout:0/654: creat d5/d1c/d24/d35/d50/dd5/fda x:0 0 0 2024-07-16T09:55:41.265 INFO:tasks.workunit.client.1.smithi176.stdout:5/624: link d4/d6c/d94/ca8 d4/d14/cda 0 2024-07-16T09:55:41.266 INFO:tasks.workunit.client.0.smithi156.stdout:9/578: getdents d2/d5/d16/d57/da3 0 2024-07-16T09:55:41.267 INFO:tasks.workunit.client.0.smithi156.stdout:1/670: dwrite dc/d6b/f80 [0,4194304] 0 2024-07-16T09:55:41.268 INFO:tasks.workunit.client.1.smithi176.stdout:5/625: dread d4/d10/d17/f2f [0,4194304] 0 2024-07-16T09:55:41.268 INFO:tasks.workunit.client.1.smithi176.stdout:0/655: symlink d5/d44/ldb 0 2024-07-16T09:55:41.270 INFO:tasks.workunit.client.0.smithi156.stdout:7/434: dwrite dc/d3e/d3f/d45/f4f [0,4194304] 0 2024-07-16T09:55:41.272 INFO:tasks.workunit.client.0.smithi156.stdout:9/579: readlink d2/d5/d16/d22/d6b/l9c 0 2024-07-16T09:55:41.273 INFO:tasks.workunit.client.1.smithi176.stdout:0/656: unlink d5/d1c/d24/d35/d50/d9b/ca2 0 2024-07-16T09:55:41.274 INFO:tasks.workunit.client.0.smithi156.stdout:7/435: symlink dc/d44/l95 0 2024-07-16T09:55:41.274 INFO:tasks.workunit.client.0.smithi156.stdout:6/489: link d3/d13/d17/d4f/d1e/f20 d3/d13/d17/d4f/d30/fa3 0 2024-07-16T09:55:41.276 INFO:tasks.workunit.client.0.smithi156.stdout:7/436: fsync dc/d63/f6b 0 2024-07-16T09:55:41.276 INFO:tasks.workunit.client.1.smithi176.stdout:0/657: dread d5/f2f [0,4194304] 0 2024-07-16T09:55:41.278 INFO:tasks.workunit.client.0.smithi156.stdout:1/671: getdents dc/d24 0 2024-07-16T09:55:41.279 INFO:tasks.workunit.client.0.smithi156.stdout:6/490: creat d3/d13/d17/d56/fa4 x:0 0 0 2024-07-16T09:55:41.279 INFO:tasks.workunit.client.0.smithi156.stdout:7/437: rmdir dc/d3e/d5d 39 2024-07-16T09:55:41.279 INFO:tasks.workunit.client.1.smithi176.stdout:0/658: rmdir d5/d1c/d24/d35/d59 39 2024-07-16T09:55:41.280 INFO:tasks.workunit.client.0.smithi156.stdout:7/438: fsync dc/f94 0 2024-07-16T09:55:41.283 INFO:tasks.workunit.client.1.smithi176.stdout:0/659: mknod d5/d1c/d5a/d9e/cdc 0 2024-07-16T09:55:41.283 INFO:tasks.workunit.client.1.smithi176.stdout:6/570: truncate d1/dc/d5b/f6d 59251 0 2024-07-16T09:55:41.284 INFO:tasks.workunit.client.0.smithi156.stdout:9/580: rename d2/d5/d16/d22/lb3 to d2/d5/d16/lb5 0 2024-07-16T09:55:41.284 INFO:tasks.workunit.client.1.smithi176.stdout:5/626: truncate d4/d10/faf 2219565 0 2024-07-16T09:55:41.285 INFO:tasks.workunit.client.0.smithi156.stdout:1/672: write dc/d6b/dd5/fe5 [184444,110551] 0 2024-07-16T09:55:41.286 INFO:tasks.workunit.client.1.smithi176.stdout:6/571: readlink d1/d5/d7/d1e/lcf 0 2024-07-16T09:55:41.287 INFO:tasks.workunit.client.0.smithi156.stdout:6/491: mkdir d3/d13/d17/d4f/d4d/d3b/da5 0 2024-07-16T09:55:41.289 INFO:tasks.workunit.client.0.smithi156.stdout:9/581: symlink d2/d5/d61/da2/lb6 0 2024-07-16T09:55:41.291 INFO:tasks.workunit.client.0.smithi156.stdout:7/439: creat dc/d3e/d5d/f96 x:0 0 0 2024-07-16T09:55:41.292 INFO:tasks.workunit.client.0.smithi156.stdout:7/440: read - dc/d13/d36/f7a zero size 2024-07-16T09:55:41.292 INFO:tasks.workunit.client.1.smithi176.stdout:6/572: rename d1/d5/d47/fac to d1/d5/d47/fd4 0 2024-07-16T09:55:41.292 INFO:tasks.workunit.client.0.smithi156.stdout:1/673: creat dc/d14/d3e/d98/fe8 x:0 0 0 2024-07-16T09:55:41.293 INFO:tasks.workunit.client.1.smithi176.stdout:5/627: dwrite d4/d14/fcd [0,4194304] 0 2024-07-16T09:55:41.293 INFO:tasks.workunit.client.0.smithi156.stdout:7/441: stat dc/d13/d36/f66 0 2024-07-16T09:55:41.294 INFO:tasks.workunit.client.1.smithi176.stdout:0/660: dread d5/d39/f51 [0,4194304] 0 2024-07-16T09:55:41.295 INFO:tasks.workunit.client.1.smithi176.stdout:6/573: write d1/dc/d72/d6f/d7c/f90 [3351144,40707] 0 2024-07-16T09:55:41.295 INFO:tasks.workunit.client.0.smithi156.stdout:7/442: chown dc/d63/f88 133 1 2024-07-16T09:55:41.296 INFO:tasks.workunit.client.1.smithi176.stdout:6/574: chown d1/d5d/db9/fcb 7380 1 2024-07-16T09:55:41.297 INFO:tasks.workunit.client.1.smithi176.stdout:6/575: write d1/d5d/d9e/d76/f5a [2836860,100416] 0 2024-07-16T09:55:41.298 INFO:tasks.workunit.client.1.smithi176.stdout:0/661: creat d5/d25/da0/fdd x:0 0 0 2024-07-16T09:55:41.298 INFO:tasks.workunit.client.0.smithi156.stdout:9/582: fsync d2/d5/f70 0 2024-07-16T09:55:41.298 INFO:tasks.workunit.client.0.smithi156.stdout:6/492: creat d3/d13/d17/d4f/d4d/fa6 x:0 0 0 2024-07-16T09:55:41.299 INFO:tasks.workunit.client.1.smithi176.stdout:5/628: fsync d4/f86 0 2024-07-16T09:55:41.301 INFO:tasks.workunit.client.1.smithi176.stdout:0/662: mkdir d5/d39/dde 0 2024-07-16T09:55:41.302 INFO:tasks.workunit.client.0.smithi156.stdout:7/443: dread dc/d3e/d3f/d45/f4f [0,4194304] 0 2024-07-16T09:55:41.306 INFO:tasks.workunit.client.0.smithi156.stdout:9/583: readlink d2/d5/d16/d22/l6c 0 2024-07-16T09:55:41.308 INFO:tasks.workunit.client.1.smithi176.stdout:6/576: unlink d1/d5/d7/l5e 0 2024-07-16T09:55:41.311 INFO:tasks.workunit.client.0.smithi156.stdout:6/493: dwrite d3/d13/d17/f2b [4194304,4194304] 0 2024-07-16T09:55:41.311 INFO:tasks.workunit.client.1.smithi176.stdout:0/663: dwrite d5/d1c/d24/d35/d50/d5b/f65 [0,4194304] 0 2024-07-16T09:55:41.312 INFO:tasks.workunit.client.0.smithi156.stdout:6/494: truncate d3/d13/d17/f8c 441717 0 2024-07-16T09:55:41.313 INFO:tasks.workunit.client.0.smithi156.stdout:9/584: link d2/d5/d16/d22/d36/ca5 d2/d5/d16/d22/d36/cb7 0 2024-07-16T09:55:41.315 INFO:tasks.workunit.client.0.smithi156.stdout:6/495: mknod d3/ca7 0 2024-07-16T09:55:41.316 INFO:tasks.workunit.client.1.smithi176.stdout:0/664: mkdir d5/d1c/d24/ddf 0 2024-07-16T09:55:41.317 INFO:tasks.workunit.client.0.smithi156.stdout:1/674: truncate dc/d14/d3e/d6c/d7a/dcb/fce 1675393 0 2024-07-16T09:55:41.320 INFO:tasks.workunit.client.1.smithi176.stdout:5/629: truncate d4/d10/d17/f18 3417433 0 2024-07-16T09:55:41.321 INFO:tasks.workunit.client.1.smithi176.stdout:0/665: stat d5/d1c/d24/d35/d50/d58/d63/f66 0 2024-07-16T09:55:41.322 INFO:tasks.workunit.client.0.smithi156.stdout:1/675: chown dc/d24/f3a 731912 1 2024-07-16T09:55:41.322 INFO:tasks.workunit.client.0.smithi156.stdout:6/496: mkdir d3/d13/d17/d4f/d30/d45/d5f/da8 0 2024-07-16T09:55:41.323 INFO:tasks.workunit.client.0.smithi156.stdout:9/585: dwrite d2/d5/d16/d57/d63/f6e [0,4194304] 0 2024-07-16T09:55:41.325 INFO:tasks.workunit.client.0.smithi156.stdout:1/676: stat dc/d14/d3e/d55/d83/fb2 0 2024-07-16T09:55:41.328 INFO:tasks.workunit.client.1.smithi176.stdout:5/630: creat d4/d14/dce/fdb x:0 0 0 2024-07-16T09:55:41.328 INFO:tasks.workunit.client.1.smithi176.stdout:0/666: symlink d5/d1c/d24/ddf/le0 0 2024-07-16T09:55:41.328 INFO:tasks.workunit.client.0.smithi156.stdout:0/524: sync 2024-07-16T09:55:41.328 INFO:tasks.workunit.client.0.smithi156.stdout:7/444: dwrite dc/f14 [0,4194304] 0 2024-07-16T09:55:41.328 INFO:tasks.workunit.client.0.smithi156.stdout:3/496: sync 2024-07-16T09:55:41.330 INFO:tasks.workunit.client.0.smithi156.stdout:9/586: chown d2/d5/d16/d22/d36/d33/d23/da0/faf 368440 1 2024-07-16T09:55:41.330 INFO:tasks.workunit.client.0.smithi156.stdout:9/587: stat d2/d5/f70 0 2024-07-16T09:55:41.331 INFO:tasks.workunit.client.1.smithi176.stdout:6/577: dread d1/dc/d72/d6f/d7c/f90 [0,4194304] 0 2024-07-16T09:55:41.332 INFO:tasks.workunit.client.1.smithi176.stdout:5/631: mkdir d4/d14/ddc 0 2024-07-16T09:55:41.333 INFO:tasks.workunit.client.0.smithi156.stdout:9/588: write d2/d5/f44 [4044499,83387] 0 2024-07-16T09:55:41.334 INFO:tasks.workunit.client.0.smithi156.stdout:6/497: mknod d3/d13/d17/d4f/d4d/d3b/d5a/ca9 0 2024-07-16T09:55:41.335 INFO:tasks.workunit.client.0.smithi156.stdout:3/497: write d8/da/d28/d43/d63/f73 [1607549,90850] 0 2024-07-16T09:55:41.337 INFO:tasks.workunit.client.1.smithi176.stdout:0/667: link d5/lc d5/d1c/d24/d26/dbd/le1 0 2024-07-16T09:55:41.339 INFO:tasks.workunit.client.0.smithi156.stdout:0/525: creat d0/d1a/d4d/d9/d27/d8d/fa4 x:0 0 0 2024-07-16T09:55:41.340 INFO:tasks.workunit.client.1.smithi176.stdout:6/578: link d1/d5/d7/d1e/f66 d1/d5d/d9e/d8f/fd5 0 2024-07-16T09:55:41.343 INFO:tasks.workunit.client.0.smithi156.stdout:7/445: dread dc/d48/f7c [0,4194304] 0 2024-07-16T09:55:41.345 INFO:tasks.workunit.client.0.smithi156.stdout:3/498: dread d8/da/d13/f5c [0,4194304] 0 2024-07-16T09:55:41.346 INFO:tasks.workunit.client.0.smithi156.stdout:0/526: chown d0/d1a/d4d/d9/f15 15168 1 2024-07-16T09:55:41.347 INFO:tasks.workunit.client.0.smithi156.stdout:1/677: truncate dc/d20/f49 1337515 0 2024-07-16T09:55:41.348 INFO:tasks.workunit.client.0.smithi156.stdout:7/446: write dc/f40 [445257,103784] 0 2024-07-16T09:55:41.350 INFO:tasks.workunit.client.0.smithi156.stdout:9/589: dwrite d2/d5/d16/d22/d36/d33/d23/f88 [0,4194304] 0 2024-07-16T09:55:41.350 INFO:tasks.workunit.client.0.smithi156.stdout:6/498: dwrite d3/d13/d17/d4f/d1e/f95 [0,4194304] 0 2024-07-16T09:55:41.353 INFO:tasks.workunit.client.0.smithi156.stdout:3/499: write d8/da/d1d/f64 [648145,72979] 0 2024-07-16T09:55:41.358 INFO:tasks.workunit.client.0.smithi156.stdout:0/527: symlink d0/d1a/d4d/d60/la5 0 2024-07-16T09:55:41.358 INFO:tasks.workunit.client.1.smithi176.stdout:5/632: truncate d4/d10/faf 1334521 0 2024-07-16T09:55:41.360 INFO:tasks.workunit.client.1.smithi176.stdout:0/668: dread d5/d1c/d24/d35/d50/d58/f79 [4194304,4194304] 0 2024-07-16T09:55:41.363 INFO:tasks.workunit.client.0.smithi156.stdout:0/528: mknod d0/d22/ca6 0 2024-07-16T09:55:41.364 INFO:tasks.workunit.client.1.smithi176.stdout:5/633: creat d4/d37/d72/d38/fdd x:0 0 0 2024-07-16T09:55:41.364 INFO:tasks.workunit.client.0.smithi156.stdout:6/499: mkdir d3/d13/d17/d4f/d94/daa 0 2024-07-16T09:55:41.366 INFO:tasks.workunit.client.0.smithi156.stdout:6/500: write d3/d13/d17/d4f/d4d/d3b/f93 [4329622,57615] 0 2024-07-16T09:55:41.367 INFO:tasks.workunit.client.1.smithi176.stdout:6/579: dwrite d1/d5/f27 [0,4194304] 0 2024-07-16T09:55:41.369 INFO:tasks.workunit.client.1.smithi176.stdout:5/634: creat d4/d37/d72/d38/d39/d9f/fde x:0 0 0 2024-07-16T09:55:41.370 INFO:tasks.workunit.client.0.smithi156.stdout:9/590: rename d2/d5/d16/d22/l84 to d2/d5/d16/d22/d36/d33/lb8 0 2024-07-16T09:55:41.370 INFO:tasks.workunit.client.0.smithi156.stdout:1/678: getdents dc/d14/d3e/d55/d57/d76 0 2024-07-16T09:55:41.372 INFO:tasks.workunit.client.1.smithi176.stdout:5/635: truncate d4/d14/dce/fdb 333825 0 2024-07-16T09:55:41.372 INFO:tasks.workunit.client.1.smithi176.stdout:0/669: dwrite d5/d1c/d24/d35/d50/d58/f79 [0,4194304] 0 2024-07-16T09:55:41.374 INFO:tasks.workunit.client.0.smithi156.stdout:1/679: rmdir dc/d14/d3e/d55/d83 39 2024-07-16T09:55:41.376 INFO:tasks.workunit.client.0.smithi156.stdout:9/591: creat d2/d5/d16/d22/d2b/d3d/d3c/d7f/fb9 x:0 0 0 2024-07-16T09:55:41.377 INFO:tasks.workunit.client.0.smithi156.stdout:7/447: dread dc/d3e/f7f [0,4194304] 0 2024-07-16T09:55:41.378 INFO:tasks.workunit.client.1.smithi176.stdout:0/670: write d5/d25/da0/faa [2618646,110926] 0 2024-07-16T09:55:41.378 INFO:tasks.workunit.client.0.smithi156.stdout:1/680: chown dc/d14/d1e/f77 32707 1 2024-07-16T09:55:41.379 INFO:tasks.workunit.client.1.smithi176.stdout:0/671: write d5/d2c/d6c/d83/d84/fc2 [376167,77221] 0 2024-07-16T09:55:41.381 INFO:tasks.workunit.client.1.smithi176.stdout:5/636: link d4/d14/f25 d4/d37/d88/fdf 0 2024-07-16T09:55:41.381 INFO:tasks.workunit.client.0.smithi156.stdout:9/592: mknod d2/d5/d16/d22/d2b/d3d/cba 0 2024-07-16T09:55:41.382 INFO:tasks.workunit.client.1.smithi176.stdout:0/672: creat d5/d39/da3/fe2 x:0 0 0 2024-07-16T09:55:41.382 INFO:tasks.workunit.client.1.smithi176.stdout:5/637: chown d4/d37/d72/f4e 10930 1 2024-07-16T09:55:41.386 INFO:tasks.workunit.client.0.smithi156.stdout:7/448: mknod dc/d48/c97 0 2024-07-16T09:55:41.386 INFO:tasks.workunit.client.0.smithi156.stdout:1/681: getdents dc/d6b/d8a/ddc 0 2024-07-16T09:55:41.386 INFO:tasks.workunit.client.1.smithi176.stdout:0/673: rmdir d5/d1c/d5a/d9e 39 2024-07-16T09:55:41.387 INFO:tasks.workunit.client.0.smithi156.stdout:7/449: truncate dc/d13/d36/f4c 4730128 0 2024-07-16T09:55:41.389 INFO:tasks.workunit.client.1.smithi176.stdout:5/638: dread d4/d14/fac [0,4194304] 0 2024-07-16T09:55:41.389 INFO:tasks.workunit.client.0.smithi156.stdout:1/682: symlink dc/d14/d3e/d55/d58/le9 0 2024-07-16T09:55:41.389 INFO:tasks.workunit.client.1.smithi176.stdout:6/580: fdatasync d1/d5/f27 0 2024-07-16T09:55:41.390 INFO:tasks.workunit.client.1.smithi176.stdout:5/639: rename d4/l1e to d4/d6c/le0 0 2024-07-16T09:55:41.391 INFO:tasks.workunit.client.1.smithi176.stdout:6/581: chown d1/d5d/d9e/fd1 766106 1 2024-07-16T09:55:41.392 INFO:tasks.workunit.client.0.smithi156.stdout:9/593: getdents d2/d5 0 2024-07-16T09:55:41.392 INFO:tasks.workunit.client.0.smithi156.stdout:7/450: unlink dc/d13/d24/l6e 0 2024-07-16T09:55:41.393 INFO:tasks.workunit.client.0.smithi156.stdout:1/683: mknod dc/d14/d3e/d6c/cea 0 2024-07-16T09:55:41.393 INFO:tasks.workunit.client.0.smithi156.stdout:3/500: dread d8/da/fc [0,4194304] 0 2024-07-16T09:55:41.394 INFO:tasks.workunit.client.1.smithi176.stdout:6/582: rename d1/d5/l28 to d1/dc/d72/ld6 0 2024-07-16T09:55:41.395 INFO:tasks.workunit.client.0.smithi156.stdout:7/451: mkdir dc/d63/d98 0 2024-07-16T09:55:41.396 INFO:tasks.workunit.client.1.smithi176.stdout:5/640: mknod d4/d14/ddc/ce1 0 2024-07-16T09:55:41.398 INFO:tasks.workunit.client.1.smithi176.stdout:6/583: mknod d1/d5/d47/d62/dbf/cd7 0 2024-07-16T09:55:41.398 INFO:tasks.workunit.client.0.smithi156.stdout:7/452: write dc/d13/d20/f3b [2349352,3217] 0 2024-07-16T09:55:41.398 INFO:tasks.workunit.client.0.smithi156.stdout:0/529: dread d0/d1a/d4d/d9/d27/d3d/f37 [0,4194304] 0 2024-07-16T09:55:41.399 INFO:tasks.workunit.client.0.smithi156.stdout:7/453: dread - dc/d13/d36/f7a zero size 2024-07-16T09:55:41.399 INFO:tasks.workunit.client.0.smithi156.stdout:0/530: chown d0/d1a/d4d/d6e/d86/c94 386955230 1 2024-07-16T09:55:41.400 INFO:tasks.workunit.client.0.smithi156.stdout:0/531: chown d0/d1a/f46 3 1 2024-07-16T09:55:41.400 INFO:tasks.workunit.client.1.smithi176.stdout:7/616: sync 2024-07-16T09:55:41.401 INFO:tasks.workunit.client.0.smithi156.stdout:0/532: chown d0/d1a/d4d/d9/f15 234070 1 2024-07-16T09:55:41.403 INFO:tasks.workunit.client.0.smithi156.stdout:0/533: mkdir d0/d1a/d76/da7 0 2024-07-16T09:55:41.406 INFO:tasks.workunit.client.0.smithi156.stdout:0/534: symlink d0/d1a/d4d/d9/d27/d2c/la8 0 2024-07-16T09:55:41.408 INFO:tasks.workunit.client.0.smithi156.stdout:0/535: mkdir d0/d22/d41/da9 0 2024-07-16T09:55:41.410 INFO:tasks.workunit.client.0.smithi156.stdout:1/684: dwrite dc/d14/f34 [0,4194304] 0 2024-07-16T09:55:41.415 INFO:tasks.workunit.client.0.smithi156.stdout:6/501: dread d3/d13/d17/f7c [0,4194304] 0 2024-07-16T09:55:41.415 INFO:tasks.workunit.client.0.smithi156.stdout:9/594: dread d2/d5/f55 [4194304,4194304] 0 2024-07-16T09:55:41.417 INFO:tasks.workunit.client.0.smithi156.stdout:1/685: truncate dc/d14/d3e/d55/d83/fb2 966155 0 2024-07-16T09:55:41.418 INFO:tasks.workunit.client.0.smithi156.stdout:9/595: fdatasync d2/d5/d16/d22/d2b/d3d/d3c/d7f/f5e 0 2024-07-16T09:55:41.420 INFO:tasks.workunit.client.0.smithi156.stdout:3/501: dread d8/da/d28/f7c [0,4194304] 0 2024-07-16T09:55:41.422 INFO:tasks.workunit.client.0.smithi156.stdout:1/686: creat dc/d14/d8b/dcd/dd2/feb x:0 0 0 2024-07-16T09:55:41.425 INFO:tasks.workunit.client.1.smithi176.stdout:6/584: mkdir d1/dc/dd8 0 2024-07-16T09:55:41.426 INFO:tasks.workunit.client.1.smithi176.stdout:5/641: write d4/d37/d72/d38/f7e [558288,50433] 0 2024-07-16T09:55:41.426 INFO:tasks.workunit.client.1.smithi176.stdout:7/617: fdatasync d4/dd/f1a 0 2024-07-16T09:55:41.427 INFO:tasks.workunit.client.1.smithi176.stdout:7/618: stat d4/dd/d49/d7a/db1 0 2024-07-16T09:55:41.428 INFO:tasks.workunit.client.0.smithi156.stdout:1/687: mkdir dc/d14/d3e/d55/dec 0 2024-07-16T09:55:41.428 INFO:tasks.workunit.client.0.smithi156.stdout:0/536: dread d0/d1a/f46 [0,4194304] 0 2024-07-16T09:55:41.429 INFO:tasks.workunit.client.1.smithi176.stdout:5/642: creat d4/d14/d3d/fe2 x:0 0 0 2024-07-16T09:55:41.429 INFO:tasks.workunit.client.0.smithi156.stdout:9/596: write d2/d5/f55 [6876624,107485] 0 2024-07-16T09:55:41.429 INFO:tasks.workunit.client.0.smithi156.stdout:1/688: readlink dc/d14/d3e/d55/d57/lae 0 2024-07-16T09:55:41.430 INFO:tasks.workunit.client.0.smithi156.stdout:6/502: truncate d3/d13/d17/d4f/d94/d19/d2e/f74 90341 0 2024-07-16T09:55:41.431 INFO:tasks.workunit.client.0.smithi156.stdout:3/502: write d8/da/d28/d40/f54 [456116,77129] 0 2024-07-16T09:55:41.432 INFO:tasks.workunit.client.0.smithi156.stdout:0/537: unlink d0/d1a/d4d/d25/f3a 0 2024-07-16T09:55:41.433 INFO:tasks.workunit.client.1.smithi176.stdout:5/643: symlink d4/d14/ddc/le3 0 2024-07-16T09:55:41.433 INFO:tasks.workunit.client.0.smithi156.stdout:6/503: read d3/d13/d17/d4f/d30/f7f [915165,32780] 0 2024-07-16T09:55:41.433 INFO:tasks.workunit.client.0.smithi156.stdout:1/689: chown dc/d14/d3e/d55/f5c 542731 1 2024-07-16T09:55:41.435 INFO:tasks.workunit.client.1.smithi176.stdout:6/585: dwrite d1/d5/d7/faf [0,4194304] 0 2024-07-16T09:55:41.438 INFO:tasks.workunit.client.1.smithi176.stdout:6/586: write d1/d5d/db9/fcb [1030102,103006] 0 2024-07-16T09:55:41.439 INFO:tasks.workunit.client.1.smithi176.stdout:7/619: dwrite d4/dd/d49/f7c [0,4194304] 0 2024-07-16T09:55:41.439 INFO:tasks.workunit.client.1.smithi176.stdout:5/644: readlink d4/d37/d72/d38/d39/d9f/lcb 0 2024-07-16T09:55:41.440 INFO:tasks.workunit.client.1.smithi176.stdout:6/587: dread - d1/d5/d47/d62/fa1 zero size 2024-07-16T09:55:41.440 INFO:tasks.workunit.client.1.smithi176.stdout:7/620: chown d4/lf 508997 1 2024-07-16T09:55:41.444 INFO:tasks.workunit.client.1.smithi176.stdout:6/588: readlink d1/d5/l11 0 2024-07-16T09:55:41.444 INFO:tasks.workunit.client.0.smithi156.stdout:3/503: dwrite d8/da/d1d/d2c/d33/f49 [4194304,4194304] 0 2024-07-16T09:55:41.447 INFO:tasks.workunit.client.1.smithi176.stdout:5/645: dread d4/d37/f84 [0,4194304] 0 2024-07-16T09:55:41.447 INFO:tasks.workunit.client.1.smithi176.stdout:7/621: stat d4/d1f/d22/d51/f67 0 2024-07-16T09:55:41.447 INFO:tasks.workunit.client.0.smithi156.stdout:9/597: dwrite d2/d5/d16/d22/d36/d33/d3a/f3b [0,4194304] 0 2024-07-16T09:55:41.451 INFO:tasks.workunit.client.1.smithi176.stdout:6/589: rename d1/dc/d72 to d1/d5d/d9e/dd9 0 2024-07-16T09:55:41.452 INFO:tasks.workunit.client.1.smithi176.stdout:7/622: mknod d4/d1f/d22/ccf 0 2024-07-16T09:55:41.454 INFO:tasks.workunit.client.1.smithi176.stdout:5/646: creat d4/d14/ddc/fe4 x:0 0 0 2024-07-16T09:55:41.454 INFO:tasks.workunit.client.0.smithi156.stdout:3/504: chown d8/da/d1d/f53 57739489 1 2024-07-16T09:55:41.454 INFO:tasks.workunit.client.0.smithi156.stdout:1/690: dread dc/d20/f49 [0,4194304] 0 2024-07-16T09:55:41.454 INFO:tasks.workunit.client.1.smithi176.stdout:7/623: write d4/dd/f37 [539232,41650] 0 2024-07-16T09:55:41.457 INFO:tasks.workunit.client.0.smithi156.stdout:9/598: rename d2/d5/d61/da2/l8a to d2/d5/d16/d22/d36/lbb 0 2024-07-16T09:55:41.461 INFO:tasks.workunit.client.0.smithi156.stdout:3/505: dwrite d8/da/d28/d40/f56 [0,4194304] 0 2024-07-16T09:55:41.461 INFO:tasks.workunit.client.0.smithi156.stdout:1/691: dread dc/f56 [0,4194304] 0 2024-07-16T09:55:41.464 INFO:tasks.workunit.client.0.smithi156.stdout:9/599: creat d2/d5/d16/d22/d36/d33/d23/fbc x:0 0 0 2024-07-16T09:55:41.471 INFO:tasks.workunit.client.0.smithi156.stdout:9/600: creat d2/d5/d61/da2/d98/fbd x:0 0 0 2024-07-16T09:55:41.472 INFO:tasks.workunit.client.0.smithi156.stdout:1/692: dwrite dc/d20/fbe [0,4194304] 0 2024-07-16T09:55:41.472 INFO:tasks.workunit.client.0.smithi156.stdout:6/504: dread d3/d13/d17/d4f/d1e/f20 [0,4194304] 0 2024-07-16T09:55:41.473 INFO:tasks.workunit.client.0.smithi156.stdout:3/506: dwrite d8/da/d1d/d2c/d7d/fa1 [0,4194304] 0 2024-07-16T09:55:41.478 INFO:tasks.workunit.client.0.smithi156.stdout:9/601: rename d2/d5/d16/d22/d36/f78 to d2/d5/d16/d22/d2b/d74/fbe 0 2024-07-16T09:55:41.491 INFO:tasks.workunit.client.1.smithi176.stdout:7/624: dwrite d4/dd/d63/f64 [0,4194304] 0 2024-07-16T09:55:41.492 INFO:tasks.workunit.client.1.smithi176.stdout:5/647: dwrite d4/d37/d88/f8f [0,4194304] 0 2024-07-16T09:55:41.494 INFO:tasks.workunit.client.0.smithi156.stdout:1/693: write dc/d24/f3a [197386,66576] 0 2024-07-16T09:55:41.497 INFO:tasks.workunit.client.1.smithi176.stdout:5/648: unlink d4/d10/f22 0 2024-07-16T09:55:41.500 INFO:tasks.workunit.client.0.smithi156.stdout:1/694: creat dc/d14/d3e/d45/fed x:0 0 0 2024-07-16T09:55:41.503 INFO:tasks.workunit.client.1.smithi176.stdout:7/625: dwrite d4/d5/f2f [4194304,4194304] 0 2024-07-16T09:55:41.507 INFO:tasks.workunit.client.1.smithi176.stdout:5/649: dwrite d4/d14/f2c [4194304,4194304] 0 2024-07-16T09:55:41.509 INFO:tasks.workunit.client.0.smithi156.stdout:1/695: dwrite dc/d14/d3e/d45/f53 [0,4194304] 0 2024-07-16T09:55:41.511 INFO:tasks.workunit.client.1.smithi176.stdout:5/650: symlink d4/da9/le5 0 2024-07-16T09:55:41.512 INFO:tasks.workunit.client.0.smithi156.stdout:9/602: write d2/d5/d16/d22/d2b/d3d/d3c/d7f/f5e [1260491,96272] 0 2024-07-16T09:55:41.513 INFO:tasks.workunit.client.1.smithi176.stdout:5/651: readlink d4/d37/d72/d38/l7f 0 2024-07-16T09:55:41.515 INFO:tasks.workunit.client.0.smithi156.stdout:6/505: dwrite d3/d13/d17/d4f/d4d/d3b/f96 [0,4194304] 0 2024-07-16T09:55:41.517 INFO:tasks.workunit.client.0.smithi156.stdout:9/603: chown d2/d5/d16/d22/d36/ca5 124824798 1 2024-07-16T09:55:41.518 INFO:tasks.workunit.client.0.smithi156.stdout:6/506: chown d3/d13/d17/d4f/d4d/d3b/da5 76 1 2024-07-16T09:55:41.526 INFO:tasks.workunit.client.0.smithi156.stdout:6/507: dread d3/d13/d17/d4f/d4d/d3b/f5d [0,4194304] 0 2024-07-16T09:55:41.527 INFO:tasks.workunit.client.0.smithi156.stdout:9/604: mkdir d2/d5/d61/da2/d98/dbf 0 2024-07-16T09:55:41.527 INFO:tasks.workunit.client.0.smithi156.stdout:1/696: read f6 [1239251,109969] 0 2024-07-16T09:55:41.531 INFO:tasks.workunit.client.0.smithi156.stdout:6/508: creat d3/d13/fab x:0 0 0 2024-07-16T09:55:41.531 INFO:tasks.workunit.client.0.smithi156.stdout:1/697: rmdir dc/d14/d8b/dcd 39 2024-07-16T09:55:41.531 INFO:tasks.workunit.client.0.smithi156.stdout:1/698: fsync dc/d14/fc8 0 2024-07-16T09:55:41.532 INFO:tasks.workunit.client.0.smithi156.stdout:6/509: fdatasync d3/d13/d17/d4f/d94/d19/f66 0 2024-07-16T09:55:41.532 INFO:tasks.workunit.client.1.smithi176.stdout:5/652: mknod d4/d37/d72/d38/ce6 0 2024-07-16T09:55:41.533 INFO:tasks.workunit.client.1.smithi176.stdout:5/653: chown d4/d6c/f7c 1898205 1 2024-07-16T09:55:41.535 INFO:tasks.workunit.client.0.smithi156.stdout:1/699: symlink dc/d14/d3e/d98/lee 0 2024-07-16T09:55:41.535 INFO:tasks.workunit.client.0.smithi156.stdout:1/700: fsync dc/d14/d1e/f77 0 2024-07-16T09:55:41.536 INFO:tasks.workunit.client.0.smithi156.stdout:6/510: readlink d3/d13/d17/d4f/d94/l33 0 2024-07-16T09:55:41.537 INFO:tasks.workunit.client.0.smithi156.stdout:1/701: write dc/d14/d3e/d55/d83/fa1 [1556177,101390] 0 2024-07-16T09:55:41.539 INFO:tasks.workunit.client.0.smithi156.stdout:1/702: rmdir dc/d24 39 2024-07-16T09:55:41.540 INFO:tasks.workunit.client.1.smithi176.stdout:7/626: dread d4/d47/d6d/d89/f50 [0,4194304] 0 2024-07-16T09:55:41.541 INFO:tasks.workunit.client.0.smithi156.stdout:1/703: truncate dc/d24/f7e 852082 0 2024-07-16T09:55:41.542 INFO:tasks.workunit.client.1.smithi176.stdout:7/627: creat d4/dd/d8e/fd0 x:0 0 0 2024-07-16T09:55:41.543 INFO:tasks.workunit.client.0.smithi156.stdout:1/704: write dc/d14/d3e/d55/f70 [3025664,121730] 0 2024-07-16T09:55:41.543 INFO:tasks.workunit.client.1.smithi176.stdout:7/628: truncate d4/d1f/f62 76089 0 2024-07-16T09:55:41.544 INFO:tasks.workunit.client.1.smithi176.stdout:7/629: chown d4/dd/l71 149979745 1 2024-07-16T09:55:41.551 INFO:tasks.workunit.client.1.smithi176.stdout:7/630: dwrite d4/d47/d5d/d9a/fcd [0,4194304] 0 2024-07-16T09:55:41.559 INFO:tasks.workunit.client.0.smithi156.stdout:1/705: rename dc/d14/d3e/d98 to dc/d14/d3e/d55/d57/def 0 2024-07-16T09:55:41.560 INFO:tasks.workunit.client.0.smithi156.stdout:1/706: rmdir dc/d6b/d96 39 2024-07-16T09:55:41.562 INFO:tasks.workunit.client.1.smithi176.stdout:3/445: sync 2024-07-16T09:55:41.563 INFO:tasks.workunit.client.1.smithi176.stdout:5/654: write d4/f5 [1649206,37226] 0 2024-07-16T09:55:41.564 INFO:tasks.workunit.client.1.smithi176.stdout:7/631: write d4/d1f/d22/d51/f98 [794350,106788] 0 2024-07-16T09:55:41.567 INFO:tasks.workunit.client.1.smithi176.stdout:3/446: rename d3/d17/f58 to d3/d17/f88 0 2024-07-16T09:55:41.568 INFO:tasks.workunit.client.1.smithi176.stdout:5/655: mkdir d4/d14/d19/d1b/d61/de7 0 2024-07-16T09:55:41.568 INFO:tasks.workunit.client.1.smithi176.stdout:3/447: fdatasync d3/da/d56/f5b 0 2024-07-16T09:55:41.569 INFO:tasks.workunit.client.1.smithi176.stdout:7/632: rename d4/dd/d49/d7a/cc6 to d4/d47/d6d/db7/cd1 0 2024-07-16T09:55:41.570 INFO:tasks.workunit.client.1.smithi176.stdout:3/448: chown d3/d17/d1d 2042 1 2024-07-16T09:55:41.570 INFO:tasks.workunit.client.1.smithi176.stdout:5/656: read d4/d37/d72/d38/d39/f3a [2545966,24668] 0 2024-07-16T09:55:41.571 INFO:tasks.workunit.client.1.smithi176.stdout:3/449: write d3/d17/f68 [136785,82373] 0 2024-07-16T09:55:41.572 INFO:tasks.workunit.client.1.smithi176.stdout:7/633: write d4/d1f/d22/d51/f98 [1844512,54181] 0 2024-07-16T09:55:41.575 INFO:tasks.workunit.client.1.smithi176.stdout:5/657: dread d4/d14/f45 [0,4194304] 0 2024-07-16T09:55:41.575 INFO:tasks.workunit.client.1.smithi176.stdout:3/450: creat d3/da/d56/f89 x:0 0 0 2024-07-16T09:55:41.577 INFO:tasks.workunit.client.1.smithi176.stdout:7/634: rename d4/d5/cb8 to d4/dd/d26/dc2/cd2 0 2024-07-16T09:55:41.579 INFO:tasks.workunit.client.1.smithi176.stdout:3/451: creat d3/d17/d83/f8a x:0 0 0 2024-07-16T09:55:41.580 INFO:tasks.workunit.client.1.smithi176.stdout:3/452: write d3/d17/f23 [3478268,113583] 0 2024-07-16T09:55:41.581 INFO:tasks.workunit.client.1.smithi176.stdout:7/635: mknod d4/dd/db5/cd3 0 2024-07-16T09:55:41.584 INFO:tasks.workunit.client.1.smithi176.stdout:3/453: unlink d3/da/d33/f75 0 2024-07-16T09:55:41.584 INFO:tasks.workunit.client.1.smithi176.stdout:7/636: stat d4/dd/d26/dc2/d17/d1d/c28 0 2024-07-16T09:55:41.585 INFO:tasks.workunit.client.1.smithi176.stdout:3/454: stat d3/d17/d1d/d4f/d59/d81 0 2024-07-16T09:55:41.592 INFO:tasks.workunit.client.1.smithi176.stdout:7/637: symlink d4/d47/ld4 0 2024-07-16T09:55:41.595 INFO:tasks.workunit.client.1.smithi176.stdout:7/638: mkdir d4/dd/d49/dd5 0 2024-07-16T09:55:41.597 INFO:tasks.workunit.client.1.smithi176.stdout:3/455: write d3/da/f29 [839670,63558] 0 2024-07-16T09:55:41.599 INFO:tasks.workunit.client.1.smithi176.stdout:5/658: dwrite d4/d10/faf [0,4194304] 0 2024-07-16T09:55:41.599 INFO:tasks.workunit.client.1.smithi176.stdout:3/456: dread - d3/f74 zero size 2024-07-16T09:55:41.601 INFO:tasks.workunit.client.1.smithi176.stdout:5/659: read d4/d10/faf [4110881,22983] 0 2024-07-16T09:55:41.606 INFO:tasks.workunit.client.1.smithi176.stdout:3/457: rename d3/d17/d1d/d4d/l60 to d3/da/d7b/l8b 0 2024-07-16T09:55:41.608 INFO:tasks.workunit.client.1.smithi176.stdout:5/660: dwrite d4/d9a/fa7 [0,4194304] 0 2024-07-16T09:55:41.613 INFO:tasks.workunit.client.1.smithi176.stdout:3/458: symlink d3/l8c 0 2024-07-16T09:55:41.613 INFO:tasks.workunit.client.1.smithi176.stdout:3/459: fsync d3/da/d33/f53 0 2024-07-16T09:55:41.613 INFO:tasks.workunit.client.1.smithi176.stdout:5/661: mkdir d4/d37/d72/d38/d39/d9f/de8 0 2024-07-16T09:55:41.616 INFO:tasks.workunit.client.1.smithi176.stdout:3/460: symlink d3/d17/d1d/d4f/d59/d81/l8d 0 2024-07-16T09:55:41.620 INFO:tasks.workunit.client.1.smithi176.stdout:3/461: mknod d3/d17/d1d/d4f/d6d/c8e 0 2024-07-16T09:55:41.622 INFO:tasks.workunit.client.1.smithi176.stdout:5/662: link d4/d10/f33 d4/d37/d72/d38/d39/d9f/fe9 0 2024-07-16T09:55:41.622 INFO:tasks.workunit.client.1.smithi176.stdout:5/663: chown d4/d37/d72/d38/f97 260788 1 2024-07-16T09:55:41.622 INFO:tasks.workunit.client.1.smithi176.stdout:3/462: symlink d3/d17/d1d/d4f/l8f 0 2024-07-16T09:55:41.623 INFO:tasks.workunit.client.1.smithi176.stdout:3/463: write d3/da/d56/f89 [18048,78386] 0 2024-07-16T09:55:41.625 INFO:tasks.workunit.client.0.smithi156.stdout:1/707: dwrite dc/d14/d3e/d6c/d7a/dcb/fce [0,4194304] 0 2024-07-16T09:55:41.629 INFO:tasks.workunit.client.1.smithi176.stdout:5/664: creat d4/fea x:0 0 0 2024-07-16T09:55:41.632 INFO:tasks.workunit.client.1.smithi176.stdout:5/665: mkdir d4/da9/dc4/deb 0 2024-07-16T09:55:41.633 INFO:tasks.workunit.client.0.smithi156.stdout:1/708: creat dc/d6b/d8a/ddc/ff0 x:0 0 0 2024-07-16T09:55:41.634 INFO:tasks.workunit.client.1.smithi176.stdout:5/666: chown d4/d10/d17/f8a 101 1 2024-07-16T09:55:41.640 INFO:tasks.workunit.client.0.smithi156.stdout:1/709: rename dc/d14/f34 to dc/d14/d8b/db4/ff1 0 2024-07-16T09:55:41.641 INFO:tasks.workunit.client.1.smithi176.stdout:5/667: creat d4/d14/fec x:0 0 0 2024-07-16T09:55:41.642 INFO:tasks.workunit.client.1.smithi176.stdout:5/668: dread - d4/d37/d72/f59 zero size 2024-07-16T09:55:41.643 INFO:tasks.workunit.client.1.smithi176.stdout:5/669: write d4/d14/d19/d1b/f26 [1742747,30925] 0 2024-07-16T09:55:41.645 INFO:tasks.workunit.client.1.smithi176.stdout:1/367: sync 2024-07-16T09:55:41.647 INFO:tasks.workunit.client.0.smithi156.stdout:1/710: dwrite dc/d14/d3e/fde [0,4194304] 0 2024-07-16T09:55:41.647 INFO:tasks.workunit.client.1.smithi176.stdout:5/670: rename d4/d37/d88 to d4/d9a/ded 0 2024-07-16T09:55:41.648 INFO:tasks.workunit.client.1.smithi176.stdout:1/368: truncate d4/d8/d12/f2e 71755 0 2024-07-16T09:55:41.650 INFO:tasks.workunit.client.1.smithi176.stdout:1/369: write d4/f65 [1376076,24314] 0 2024-07-16T09:55:41.652 INFO:tasks.workunit.client.0.smithi156.stdout:1/711: rename dc/d20 to dc/d14/d8b/db4/df2 0 2024-07-16T09:55:41.653 INFO:tasks.workunit.client.0.smithi156.stdout:1/712: fsync dc/d14/d8b/db4/df2/f9d 0 2024-07-16T09:55:41.655 INFO:tasks.workunit.client.1.smithi176.stdout:1/370: unlink d4/ff 0 2024-07-16T09:55:41.657 INFO:tasks.workunit.client.0.smithi156.stdout:1/713: mkdir dc/d14/d3e/d6c/d7a/df3 0 2024-07-16T09:55:41.657 INFO:tasks.workunit.client.1.smithi176.stdout:1/371: symlink d4/d8/d12/d4c/l80 0 2024-07-16T09:55:41.661 INFO:tasks.workunit.client.0.smithi156.stdout:1/714: rename dc/d14/d3e/d55/d57/def/lc5 to dc/d14/d3e/d55/d57/def/lf4 0 2024-07-16T09:55:41.662 INFO:tasks.workunit.client.0.smithi156.stdout:1/715: chown dc/d6b/dd5/ce6 1 1 2024-07-16T09:55:41.664 INFO:tasks.workunit.client.1.smithi176.stdout:3/464: dread d3/d17/f40 [0,4194304] 0 2024-07-16T09:55:41.667 INFO:tasks.workunit.client.0.smithi156.stdout:1/716: dwrite dc/d14/d3e/d55/d83/fa1 [0,4194304] 0 2024-07-16T09:55:41.668 INFO:tasks.workunit.client.0.smithi156.stdout:1/717: stat dc/d6b/dd5/ce6 0 2024-07-16T09:55:41.669 INFO:tasks.workunit.client.1.smithi176.stdout:1/372: dwrite d4/db/f43 [0,4194304] 0 2024-07-16T09:55:41.669 INFO:tasks.workunit.client.1.smithi176.stdout:3/465: creat d3/d17/f90 x:0 0 0 2024-07-16T09:55:41.673 INFO:tasks.workunit.client.1.smithi176.stdout:5/671: dwrite d4/d37/d72/f59 [0,4194304] 0 2024-07-16T09:55:41.678 INFO:tasks.workunit.client.1.smithi176.stdout:3/466: creat d3/d17/d83/f91 x:0 0 0 2024-07-16T09:55:41.678 INFO:tasks.workunit.client.0.smithi156.stdout:1/718: creat dc/d6b/d96/ff5 x:0 0 0 2024-07-16T09:55:41.679 INFO:tasks.workunit.client.1.smithi176.stdout:1/373: creat d4/d3a/d54/f81 x:0 0 0 2024-07-16T09:55:41.679 INFO:tasks.workunit.client.0.smithi156.stdout:1/719: read - dc/d14/d3e/d55/d57/f93 zero size 2024-07-16T09:55:41.680 INFO:tasks.workunit.client.1.smithi176.stdout:5/672: mknod d4/d6c/cee 0 2024-07-16T09:55:41.680 INFO:tasks.workunit.client.1.smithi176.stdout:3/467: chown d3/da/d33/f39 2543 1 2024-07-16T09:55:41.681 INFO:tasks.workunit.client.1.smithi176.stdout:1/374: dread - d4/db/f5c zero size 2024-07-16T09:55:41.686 INFO:tasks.workunit.client.1.smithi176.stdout:5/673: rename d4/d6c/d94 to d4/d14/dc5/def 0 2024-07-16T09:55:41.686 INFO:tasks.workunit.client.0.smithi156.stdout:2/592: sync 2024-07-16T09:55:41.686 INFO:tasks.workunit.client.0.smithi156.stdout:4/621: sync 2024-07-16T09:55:41.686 INFO:tasks.workunit.client.0.smithi156.stdout:5/603: sync 2024-07-16T09:55:41.686 INFO:tasks.workunit.client.0.smithi156.stdout:8/402: sync 2024-07-16T09:55:41.687 INFO:tasks.workunit.client.1.smithi176.stdout:1/375: mkdir d4/d82 0 2024-07-16T09:55:41.687 INFO:tasks.workunit.client.1.smithi176.stdout:5/674: write d4/d6c/f6f [789274,47459] 0 2024-07-16T09:55:41.688 INFO:tasks.workunit.client.0.smithi156.stdout:4/622: stat d4/d23/d30 0 2024-07-16T09:55:41.689 INFO:tasks.workunit.client.0.smithi156.stdout:1/720: truncate dc/d14/f48 1013312 0 2024-07-16T09:55:41.691 INFO:tasks.workunit.client.0.smithi156.stdout:2/593: chown d0/l3a 12 1 2024-07-16T09:55:41.692 INFO:tasks.workunit.client.0.smithi156.stdout:8/403: rename d0/d11/d62/f83 to d0/d11/d29/f8e 0 2024-07-16T09:55:41.693 INFO:tasks.workunit.client.1.smithi176.stdout:3/468: dwrite d3/da/d56/f5b [4194304,4194304] 0 2024-07-16T09:55:41.695 INFO:tasks.workunit.client.0.smithi156.stdout:1/721: unlink dc/d14/d3e/d45/fcf 0 2024-07-16T09:55:41.697 INFO:tasks.workunit.client.0.smithi156.stdout:4/623: dwrite d4/d23/f39 [0,4194304] 0 2024-07-16T09:55:41.698 INFO:tasks.workunit.client.0.smithi156.stdout:1/722: stat dc/d6b 0 2024-07-16T09:55:41.698 INFO:tasks.workunit.client.0.smithi156.stdout:4/624: chown d4/db/dc/d67/l72 2018208 1 2024-07-16T09:55:41.699 INFO:tasks.workunit.client.1.smithi176.stdout:5/675: link d4/d10/l65 d4/d37/d72/d38/lf0 0 2024-07-16T09:55:41.699 INFO:tasks.workunit.client.1.smithi176.stdout:1/376: getdents d4/d3a/d54/d5a 0 2024-07-16T09:55:41.699 INFO:tasks.workunit.client.1.smithi176.stdout:3/469: creat d3/da/d6c/d86/f92 x:0 0 0 2024-07-16T09:55:41.699 INFO:tasks.workunit.client.1.smithi176.stdout:1/377: rmdir d4/db/d68/d72 39 2024-07-16T09:55:41.700 INFO:tasks.workunit.client.0.smithi156.stdout:8/404: creat d0/dd/d1f/d57/f8f x:0 0 0 2024-07-16T09:55:41.700 INFO:tasks.workunit.client.1.smithi176.stdout:1/378: chown d4/d3a/d54/d5a/f67 6220672 1 2024-07-16T09:55:41.700 INFO:tasks.workunit.client.1.smithi176.stdout:5/676: mknod d4/d9a/ded/cf1 0 2024-07-16T09:55:41.701 INFO:tasks.workunit.client.1.smithi176.stdout:1/379: fsync d4/d8/f26 0 2024-07-16T09:55:41.701 INFO:tasks.workunit.client.0.smithi156.stdout:2/594: truncate d0/d13/d17/d1c/f1f 1491262 0 2024-07-16T09:55:41.703 INFO:tasks.workunit.client.0.smithi156.stdout:8/405: creat d0/d11/d62/f90 x:0 0 0 2024-07-16T09:55:41.704 INFO:tasks.workunit.client.0.smithi156.stdout:5/604: dread f14 [0,4194304] 0 2024-07-16T09:55:41.704 INFO:tasks.workunit.client.0.smithi156.stdout:4/625: symlink d4/db/dc/d7f/db1/le3 0 2024-07-16T09:55:41.706 INFO:tasks.workunit.client.0.smithi156.stdout:2/595: rmdir d0/d5/d29/d2d/d5b/d5f 39 2024-07-16T09:55:41.707 INFO:tasks.workunit.client.0.smithi156.stdout:1/723: symlink dc/d14/d8b/dcd/lf6 0 2024-07-16T09:55:41.708 INFO:tasks.workunit.client.0.smithi156.stdout:1/724: write dc/d24/f7e [918143,43183] 0 2024-07-16T09:55:41.709 INFO:tasks.workunit.client.1.smithi176.stdout:3/470: creat d3/d17/d1d/d4f/d59/f93 x:0 0 0 2024-07-16T09:55:41.711 INFO:tasks.workunit.client.0.smithi156.stdout:5/605: creat d1b/d99/dae/fd4 x:0 0 0 2024-07-16T09:55:41.713 INFO:tasks.workunit.client.0.smithi156.stdout:1/725: creat dc/d6b/dd5/ff7 x:0 0 0 2024-07-16T09:55:41.713 INFO:tasks.workunit.client.1.smithi176.stdout:5/677: dwrite d4/d37/d72/d38/fdd [0,4194304] 0 2024-07-16T09:55:41.714 INFO:tasks.workunit.client.0.smithi156.stdout:4/626: mknod d4/d23/ce4 0 2024-07-16T09:55:41.714 INFO:tasks.workunit.client.1.smithi176.stdout:1/380: getdents d4/d8 0 2024-07-16T09:55:41.715 INFO:tasks.workunit.client.0.smithi156.stdout:5/606: symlink d1b/d1f/d45/d52/ld5 0 2024-07-16T09:55:41.716 INFO:tasks.workunit.client.1.smithi176.stdout:1/381: write d4/d8/d12/d21/d25/f55 [2457242,108579] 0 2024-07-16T09:55:41.716 INFO:tasks.workunit.client.0.smithi156.stdout:1/726: mkdir dc/d6b/d8a/df8 0 2024-07-16T09:55:41.718 INFO:tasks.workunit.client.0.smithi156.stdout:4/627: creat d4/d23/d52/fe5 x:0 0 0 2024-07-16T09:55:41.719 INFO:tasks.workunit.client.1.smithi176.stdout:1/382: chown d4/d8/d12/c18 181 1 2024-07-16T09:55:41.719 INFO:tasks.workunit.client.1.smithi176.stdout:1/383: write d4/d8/d12/d21/d25/d30/f61 [654595,31891] 0 2024-07-16T09:55:41.722 INFO:tasks.workunit.client.1.smithi176.stdout:5/678: symlink d4/d37/d72/d78/lf2 0 2024-07-16T09:55:41.722 INFO:tasks.workunit.client.1.smithi176.stdout:3/471: rename d3/da/cc to d3/d17/d1d/d4f/d59/c94 0 2024-07-16T09:55:41.724 INFO:tasks.workunit.client.0.smithi156.stdout:4/628: unlink d4/db/dc/d5f/d64/cc3 0 2024-07-16T09:55:41.728 INFO:tasks.workunit.client.1.smithi176.stdout:5/679: creat d4/d10/ff3 x:0 0 0 2024-07-16T09:55:41.728 INFO:tasks.workunit.client.1.smithi176.stdout:3/472: dwrite d3/da/d56/f5b [4194304,4194304] 0 2024-07-16T09:55:41.729 INFO:tasks.workunit.client.1.smithi176.stdout:3/473: write d3/d17/f68 [1053456,8430] 0 2024-07-16T09:55:41.730 INFO:tasks.workunit.client.1.smithi176.stdout:5/680: read - d4/d37/d72/d82/d96/dbd/fd5 zero size 2024-07-16T09:55:41.730 INFO:tasks.workunit.client.0.smithi156.stdout:4/629: dread d4/d23/d52/d71/fa7 [0,4194304] 0 2024-07-16T09:55:41.730 INFO:tasks.workunit.client.1.smithi176.stdout:3/474: write d3/da/f41 [1198213,34276] 0 2024-07-16T09:55:41.732 INFO:tasks.workunit.client.1.smithi176.stdout:3/475: readlink d3/da/d6c/l7d 0 2024-07-16T09:55:41.732 INFO:tasks.workunit.client.1.smithi176.stdout:5/681: write d4/d37/d72/d38/d39/fa3 [592035,61942] 0 2024-07-16T09:55:41.740 INFO:tasks.workunit.client.1.smithi176.stdout:5/682: dwrite d4/d37/d72/d38/d39/fa3 [0,4194304] 0 2024-07-16T09:55:41.742 INFO:tasks.workunit.client.1.smithi176.stdout:1/384: fdatasync d4/d8/d12/d21/d25/d30/f61 0 2024-07-16T09:55:41.742 INFO:tasks.workunit.client.1.smithi176.stdout:3/476: fsync d3/da/f41 0 2024-07-16T09:55:41.743 INFO:tasks.workunit.client.1.smithi176.stdout:3/477: read - d3/f74 zero size 2024-07-16T09:55:41.744 INFO:tasks.workunit.client.0.smithi156.stdout:8/406: write d0/dd/f26 [4243896,116546] 0 2024-07-16T09:55:41.745 INFO:tasks.workunit.client.0.smithi156.stdout:8/407: rename d0/d11/d2e to d0/d11/d2e/d35/d91 22 2024-07-16T09:55:41.746 INFO:tasks.workunit.client.1.smithi176.stdout:5/683: chown d4/d37/d72/f4f 16 1 2024-07-16T09:55:41.747 INFO:tasks.workunit.client.1.smithi176.stdout:3/478: write d3/d17/d1d/d4d/f3f [934326,11229] 0 2024-07-16T09:55:41.749 INFO:tasks.workunit.client.0.smithi156.stdout:1/727: write dc/d14/d3e/d6c/d7a/fba [758414,58434] 0 2024-07-16T09:55:41.749 INFO:tasks.workunit.client.0.smithi156.stdout:5/607: truncate d1b/d1f/d45/d9d/fab 1005595 0 2024-07-16T09:55:41.751 INFO:tasks.workunit.client.0.smithi156.stdout:2/596: dwrite d0/d13/d17/f34 [4194304,4194304] 0 2024-07-16T09:55:41.751 INFO:tasks.workunit.client.1.smithi176.stdout:5/684: mkdir d4/d9a/ded/df4 0 2024-07-16T09:55:41.751 INFO:tasks.workunit.client.1.smithi176.stdout:3/479: rmdir d3/d17/d1d/d4d 39 2024-07-16T09:55:41.752 INFO:tasks.workunit.client.1.smithi176.stdout:1/385: dwrite d4/d8/f26 [0,4194304] 0 2024-07-16T09:55:41.752 INFO:tasks.workunit.client.0.smithi156.stdout:8/408: mkdir d0/d11/d13/d1c/d39/d8a/d92 0 2024-07-16T09:55:41.755 INFO:tasks.workunit.client.0.smithi156.stdout:1/728: mkdir dc/d14/d8b/dcd/dd2/df9 0 2024-07-16T09:55:41.756 INFO:tasks.workunit.client.1.smithi176.stdout:5/685: mkdir d4/d14/dc5/def/df5 0 2024-07-16T09:55:41.757 INFO:tasks.workunit.client.0.smithi156.stdout:8/409: mkdir d0/d11/d4a/d55/d93 0 2024-07-16T09:55:41.757 INFO:tasks.workunit.client.0.smithi156.stdout:2/597: symlink d0/d13/d17/d1c/lca 0 2024-07-16T09:55:41.761 INFO:tasks.workunit.client.0.smithi156.stdout:2/598: mknod d0/d9b/ccb 0 2024-07-16T09:55:41.761 INFO:tasks.workunit.client.1.smithi176.stdout:3/480: link d3/c57 d3/da/d7b/c95 0 2024-07-16T09:55:41.762 INFO:tasks.workunit.client.1.smithi176.stdout:1/386: dread d4/db/d68/d72/f74 [0,4194304] 0 2024-07-16T09:55:41.765 INFO:tasks.workunit.client.0.smithi156.stdout:1/729: dread dc/d6b/f75 [0,4194304] 0 2024-07-16T09:55:41.766 INFO:tasks.workunit.client.1.smithi176.stdout:1/387: read d4/d3a/d54/d5a/f67 [750740,123307] 0 2024-07-16T09:55:41.766 INFO:tasks.workunit.client.0.smithi156.stdout:2/599: getdents d0/d5/d76 0 2024-07-16T09:55:41.768 INFO:tasks.workunit.client.0.smithi156.stdout:4/630: dread d4/db/dc/d67/f92 [0,4194304] 0 2024-07-16T09:55:41.768 INFO:tasks.workunit.client.0.smithi156.stdout:1/730: read dc/d14/d1e/f3d [4082612,51839] 0 2024-07-16T09:55:41.768 INFO:tasks.workunit.client.1.smithi176.stdout:3/481: mknod d3/d17/d1d/d4d/d7a/c96 0 2024-07-16T09:55:41.769 INFO:tasks.workunit.client.1.smithi176.stdout:1/388: rmdir d4/d8/d12/d4c 39 2024-07-16T09:55:41.770 INFO:tasks.workunit.client.0.smithi156.stdout:1/731: write dc/d14/d3e/d45/f53 [7197,30436] 0 2024-07-16T09:55:41.771 INFO:tasks.workunit.client.1.smithi176.stdout:3/482: symlink d3/d17/d1d/d4f/l97 0 2024-07-16T09:55:41.771 INFO:tasks.workunit.client.1.smithi176.stdout:3/483: stat l0 0 2024-07-16T09:55:41.772 INFO:tasks.workunit.client.0.smithi156.stdout:1/732: stat dc/d14/d3e/d55/d58/fd3 0 2024-07-16T09:55:41.772 INFO:tasks.workunit.client.0.smithi156.stdout:2/600: read - d0/d5/d29/d2d/d5b/d62/fb9 zero size 2024-07-16T09:55:41.773 INFO:tasks.workunit.client.1.smithi176.stdout:1/389: symlink d4/db/d68/d72/l83 0 2024-07-16T09:55:41.775 INFO:tasks.workunit.client.0.smithi156.stdout:4/631: rename d4/d23/d30/d33/c40 to d4/d23/d52/d71/ce6 0 2024-07-16T09:55:41.775 INFO:tasks.workunit.client.0.smithi156.stdout:8/410: rmdir d0 39 2024-07-16T09:55:41.775 INFO:tasks.workunit.client.0.smithi156.stdout:4/632: chown d4/d23/d30/c6f 7 1 2024-07-16T09:55:41.777 INFO:tasks.workunit.client.0.smithi156.stdout:1/733: read dc/f4a [175093,42488] 0 2024-07-16T09:55:41.777 INFO:tasks.workunit.client.1.smithi176.stdout:3/484: rename d3/c43 to d3/c98 0 2024-07-16T09:55:41.777 INFO:tasks.workunit.client.1.smithi176.stdout:1/390: symlink d4/d82/l84 0 2024-07-16T09:55:41.778 INFO:tasks.workunit.client.0.smithi156.stdout:2/601: stat d0/d13/d17/d1c/f1f 0 2024-07-16T09:55:41.778 INFO:tasks.workunit.client.1.smithi176.stdout:5/686: truncate d4/f27 3535900 0 2024-07-16T09:55:41.779 INFO:tasks.workunit.client.0.smithi156.stdout:4/633: symlink d4/db/dc/d7c/le7 0 2024-07-16T09:55:41.780 INFO:tasks.workunit.client.1.smithi176.stdout:1/391: mknod d4/d3a/c85 0 2024-07-16T09:55:41.780 INFO:tasks.workunit.client.1.smithi176.stdout:5/687: mkdir d4/d6c/df6 0 2024-07-16T09:55:41.783 INFO:tasks.workunit.client.0.smithi156.stdout:2/602: rename d0/d5/cf to d0/d5/d76/da1/ccc 0 2024-07-16T09:55:41.784 INFO:tasks.workunit.client.1.smithi176.stdout:3/485: dread d3/d17/d1d/d4f/d59/f3a [0,4194304] 0 2024-07-16T09:55:41.784 INFO:tasks.workunit.client.1.smithi176.stdout:5/688: rename d4/d14/l24 to d4/d6c/df6/lf7 0 2024-07-16T09:55:41.785 INFO:tasks.workunit.client.0.smithi156.stdout:8/411: mknod d0/d11/d29/d5e/c94 0 2024-07-16T09:55:41.786 INFO:tasks.workunit.client.1.smithi176.stdout:3/486: stat d3/c57 0 2024-07-16T09:55:41.786 INFO:tasks.workunit.client.0.smithi156.stdout:1/734: creat dc/d14/d3e/d6c/d7a/df3/ffa x:0 0 0 2024-07-16T09:55:41.786 INFO:tasks.workunit.client.1.smithi176.stdout:1/392: dread d4/d8/d12/d21/d25/d30/d44/f5f [0,4194304] 0 2024-07-16T09:55:41.788 INFO:tasks.workunit.client.0.smithi156.stdout:2/603: creat d0/d5/d29/d2d/d5b/d62/d74/fcd x:0 0 0 2024-07-16T09:55:41.793 INFO:tasks.workunit.client.1.smithi176.stdout:1/393: getdents d4/d3a/d54 0 2024-07-16T09:55:41.794 INFO:tasks.workunit.client.0.smithi156.stdout:4/634: dwrite d4/db/d8a/fc4 [0,4194304] 0 2024-07-16T09:55:41.795 INFO:tasks.workunit.client.1.smithi176.stdout:1/394: rmdir d4/d77 39 2024-07-16T09:55:41.796 INFO:tasks.workunit.client.1.smithi176.stdout:5/689: truncate d4/d37/d72/d38/d39/f3a 1994241 0 2024-07-16T09:55:41.797 INFO:tasks.workunit.client.0.smithi156.stdout:1/735: dwrite dc/d14/d3e/d55/d57/fa2 [0,4194304] 0 2024-07-16T09:55:41.799 INFO:tasks.workunit.client.0.smithi156.stdout:2/604: rename d0/d5/d29/d2d/d5b/d5f/d89/l96 to d0/d5/d76/da1/lce 0 2024-07-16T09:55:41.799 INFO:tasks.workunit.client.1.smithi176.stdout:5/690: mkdir d4/d37/dbb/df8 0 2024-07-16T09:55:41.800 INFO:tasks.workunit.client.0.smithi156.stdout:8/412: truncate d0/d11/f72 122207 0 2024-07-16T09:55:41.801 INFO:tasks.workunit.client.1.smithi176.stdout:5/691: write d4/d6c/f7c [1545706,34364] 0 2024-07-16T09:55:41.801 INFO:tasks.workunit.client.0.smithi156.stdout:1/736: mkdir dc/d6b/dd5/dfb 0 2024-07-16T09:55:41.801 INFO:tasks.workunit.client.0.smithi156.stdout:8/413: write d0/dd/f3b [3124124,101943] 0 2024-07-16T09:55:41.802 INFO:tasks.workunit.client.0.smithi156.stdout:1/737: readlink dc/d6b/d96/le3 0 2024-07-16T09:55:41.803 INFO:tasks.workunit.client.1.smithi176.stdout:1/395: creat d4/d8/d12/d21/d25/f86 x:0 0 0 2024-07-16T09:55:41.803 INFO:tasks.workunit.client.0.smithi156.stdout:2/605: read d0/d5/d29/d2d/f73 [99518,43501] 0 2024-07-16T09:55:41.805 INFO:tasks.workunit.client.1.smithi176.stdout:5/692: mkdir d4/d14/dc5/def/df9 0 2024-07-16T09:55:41.805 INFO:tasks.workunit.client.0.smithi156.stdout:2/606: stat d0/d13/d17/d1c/d51/fc2 0 2024-07-16T09:55:41.806 INFO:tasks.workunit.client.0.smithi156.stdout:1/738: mkdir dc/d14/d8b/dfc 0 2024-07-16T09:55:41.807 INFO:tasks.workunit.client.0.smithi156.stdout:2/607: chown d0/d5/d76/da1 6 1 2024-07-16T09:55:41.810 INFO:tasks.workunit.client.0.smithi156.stdout:8/414: dwrite d0/d11/d13/f76 [0,4194304] 0 2024-07-16T09:55:41.810 INFO:tasks.workunit.client.0.smithi156.stdout:4/635: write d4/f54 [1459510,517] 0 2024-07-16T09:55:41.812 INFO:tasks.workunit.client.0.smithi156.stdout:1/739: symlink dc/d14/d3e/d55/d83/lfd 0 2024-07-16T09:55:41.812 INFO:tasks.workunit.client.0.smithi156.stdout:8/415: write d0/d11/d29/d80/f88 [250172,50450] 0 2024-07-16T09:55:41.815 INFO:tasks.workunit.client.0.smithi156.stdout:1/740: write dc/d14/d8b/dcd/dd2/feb [440666,152] 0 2024-07-16T09:55:41.815 INFO:tasks.workunit.client.0.smithi156.stdout:2/608: creat d0/d5/d76/da1/fcf x:0 0 0 2024-07-16T09:55:41.817 INFO:tasks.workunit.client.0.smithi156.stdout:4/636: creat d4/db/dc/d7c/fe8 x:0 0 0 2024-07-16T09:55:41.818 INFO:tasks.workunit.client.0.smithi156.stdout:1/741: stat c3 0 2024-07-16T09:55:41.820 INFO:tasks.workunit.client.0.smithi156.stdout:2/609: dread d0/d5/d76/f8a [0,4194304] 0 2024-07-16T09:55:41.820 INFO:tasks.workunit.client.0.smithi156.stdout:1/742: chown dc/d14/d3e/fc3 137 1 2024-07-16T09:55:41.822 INFO:tasks.workunit.client.0.smithi156.stdout:8/416: getdents d0/d11/d29 0 2024-07-16T09:55:41.824 INFO:tasks.workunit.client.0.smithi156.stdout:1/743: unlink dc/d14/fc8 0 2024-07-16T09:55:41.824 INFO:tasks.workunit.client.0.smithi156.stdout:8/417: chown d0/c4 0 1 2024-07-16T09:55:41.826 INFO:tasks.workunit.client.0.smithi156.stdout:2/610: link d0/l60 d0/d13/d17/ld0 0 2024-07-16T09:55:41.827 INFO:tasks.workunit.client.0.smithi156.stdout:4/637: dwrite d4/db/dc/d7f/fdc [0,4194304] 0 2024-07-16T09:55:41.827 INFO:tasks.workunit.client.0.smithi156.stdout:8/418: chown d0/d11/d13/c51 71535 1 2024-07-16T09:55:41.828 INFO:tasks.workunit.client.0.smithi156.stdout:1/744: symlink dc/d14/d8b/dcd/dd2/df9/lfe 0 2024-07-16T09:55:41.829 INFO:tasks.workunit.client.0.smithi156.stdout:1/745: chown dc/d6b/dd5/fe5 109 1 2024-07-16T09:55:41.835 INFO:tasks.workunit.client.1.smithi176.stdout:5/693: dread d4/d37/d72/d38/d39/d9f/fe9 [0,4194304] 0 2024-07-16T09:55:41.835 INFO:tasks.workunit.client.1.smithi176.stdout:1/396: dwrite d4/d8/d12/f2e [0,4194304] 0 2024-07-16T09:55:41.837 INFO:tasks.workunit.client.1.smithi176.stdout:1/397: write d4/f65 [964961,91512] 0 2024-07-16T09:55:41.838 INFO:tasks.workunit.client.1.smithi176.stdout:1/398: write d4/d8/d2c/f2f [1842594,4771] 0 2024-07-16T09:55:41.847 INFO:tasks.workunit.client.0.smithi156.stdout:7/454: sync 2024-07-16T09:55:41.847 INFO:tasks.workunit.client.0.smithi156.stdout:0/538: sync 2024-07-16T09:55:41.848 INFO:tasks.workunit.client.1.smithi176.stdout:5/694: creat d4/d37/d72/d38/d39/ffa x:0 0 0 2024-07-16T09:55:41.850 INFO:tasks.workunit.client.0.smithi156.stdout:8/419: truncate d0/d74/f4e 656279 0 2024-07-16T09:55:41.851 INFO:tasks.workunit.client.1.smithi176.stdout:5/695: mkdir d4/d14/d3d/dfb 0 2024-07-16T09:55:41.852 INFO:tasks.workunit.client.1.smithi176.stdout:1/399: symlink d4/l87 0 2024-07-16T09:55:41.853 INFO:tasks.workunit.client.1.smithi176.stdout:5/696: mknod d4/d14/d3d/cfc 0 2024-07-16T09:55:41.855 INFO:tasks.workunit.client.1.smithi176.stdout:5/697: write d4/d37/d72/d38/d39/fa5 [90232,47191] 0 2024-07-16T09:55:41.857 INFO:tasks.workunit.client.1.smithi176.stdout:5/698: write d4/d37/d72/d38/d6e/f74 [4113231,117777] 0 2024-07-16T09:55:41.858 INFO:tasks.workunit.client.0.smithi156.stdout:0/539: mknod d0/d1a/d4d/d61/caa 0 2024-07-16T09:55:41.859 INFO:tasks.workunit.client.1.smithi176.stdout:5/699: symlink d4/d37/d72/d38/d39/d9f/lfd 0 2024-07-16T09:55:41.861 INFO:tasks.workunit.client.0.smithi156.stdout:8/420: symlink d0/d11/d13/l95 0 2024-07-16T09:55:41.862 INFO:tasks.workunit.client.0.smithi156.stdout:8/421: chown d0/d11/d2e/f5c 1896 1 2024-07-16T09:55:41.863 INFO:tasks.workunit.client.0.smithi156.stdout:1/746: rmdir dc/d14 39 2024-07-16T09:55:41.864 INFO:tasks.workunit.client.1.smithi176.stdout:5/700: rename d4/d37/d72/d38/d39/fb3 to d4/d14/dc5/def/df9/ffe 0 2024-07-16T09:55:41.865 INFO:tasks.workunit.client.0.smithi156.stdout:8/422: mkdir d0/d11/d20/d3f/d96 0 2024-07-16T09:55:41.865 INFO:tasks.workunit.client.1.smithi176.stdout:1/400: dwrite d4/f57 [0,4194304] 0 2024-07-16T09:55:41.869 INFO:tasks.workunit.client.0.smithi156.stdout:2/611: write d0/d5/d29/d2d/d5b/d5f/f7d [1390071,52262] 0 2024-07-16T09:55:41.869 INFO:tasks.workunit.client.1.smithi176.stdout:5/701: rename d4/d14/d19/d1b/dab to d4/d37/d72/d82/d96/dff 0 2024-07-16T09:55:41.870 INFO:tasks.workunit.client.1.smithi176.stdout:1/401: write d4/db/d68/d72/f74 [4147461,48503] 0 2024-07-16T09:55:41.871 INFO:tasks.workunit.client.0.smithi156.stdout:8/423: mkdir d0/d11/d29/d80/d97 0 2024-07-16T09:55:41.873 INFO:tasks.workunit.client.1.smithi176.stdout:5/702: mkdir d4/d37/d72/d38/d39/d9f/d100 0 2024-07-16T09:55:41.874 INFO:tasks.workunit.client.1.smithi176.stdout:1/402: rename d4/db/c27 to d4/d3a/d54/c88 0 2024-07-16T09:55:41.875 INFO:tasks.workunit.client.1.smithi176.stdout:5/703: rmdir d4/d6c 39 2024-07-16T09:55:41.877 INFO:tasks.workunit.client.0.smithi156.stdout:8/424: rename d0/dd/f33 to d0/dd/d1f/d60/f98 0 2024-07-16T09:55:41.877 INFO:tasks.workunit.client.1.smithi176.stdout:1/403: rename d4/d3a/f6c to d4/d8/d12/d21/f89 0 2024-07-16T09:55:41.879 INFO:tasks.workunit.client.0.smithi156.stdout:8/425: chown d0/c1b 9 1 2024-07-16T09:55:41.880 INFO:tasks.workunit.client.1.smithi176.stdout:5/704: dread d4/d37/f84 [0,4194304] 0 2024-07-16T09:55:41.880 INFO:tasks.workunit.client.0.smithi156.stdout:8/426: fdatasync d0/d11/d13/f76 0 2024-07-16T09:55:41.881 INFO:tasks.workunit.client.1.smithi176.stdout:1/404: rename d4/d8/d12/d21/c62 to d4/d8/d12/d21/c8a 0 2024-07-16T09:55:41.883 INFO:tasks.workunit.client.0.smithi156.stdout:2/612: dread d0/d13/f50 [0,4194304] 0 2024-07-16T09:55:41.883 INFO:tasks.workunit.client.1.smithi176.stdout:1/405: creat d4/d3a/d54/f8b x:0 0 0 2024-07-16T09:55:41.884 INFO:tasks.workunit.client.0.smithi156.stdout:8/427: symlink d0/d11/d20/d3f/d96/l99 0 2024-07-16T09:55:41.884 INFO:tasks.workunit.client.0.smithi156.stdout:2/613: chown d0/d13/d17 104197 1 2024-07-16T09:55:41.884 INFO:tasks.workunit.client.0.smithi156.stdout:1/747: dread dc/d14/d1e/f35 [0,4194304] 0 2024-07-16T09:55:41.885 INFO:tasks.workunit.client.0.smithi156.stdout:0/540: dwrite d0/d1a/d4d/d25/f2a [0,4194304] 0 2024-07-16T09:55:41.886 INFO:tasks.workunit.client.1.smithi176.stdout:1/406: creat d4/d8/f8c x:0 0 0 2024-07-16T09:55:41.888 INFO:tasks.workunit.client.0.smithi156.stdout:0/541: write d0/d1a/f54 [1706592,16642] 0 2024-07-16T09:55:41.888 INFO:tasks.workunit.client.0.smithi156.stdout:1/748: rmdir dc/d14/d8b/dcd/dd2 39 2024-07-16T09:55:41.889 INFO:tasks.workunit.client.0.smithi156.stdout:2/614: write d0/d5/d76/f8a [1772478,116338] 0 2024-07-16T09:55:41.889 INFO:tasks.workunit.client.0.smithi156.stdout:1/749: readlink dc/l12 0 2024-07-16T09:55:41.894 INFO:tasks.workunit.client.0.smithi156.stdout:0/542: rename d0/d22/d41/f4c to d0/d1a/d4d/d61/d97/fab 0 2024-07-16T09:55:41.894 INFO:tasks.workunit.client.0.smithi156.stdout:2/615: creat d0/d13/dc9/fd1 x:0 0 0 2024-07-16T09:55:41.896 INFO:tasks.workunit.client.0.smithi156.stdout:2/616: dread - d0/d5/d76/f7b zero size 2024-07-16T09:55:41.896 INFO:tasks.workunit.client.1.smithi176.stdout:5/705: dwrite d4/d14/d3d/f55 [0,4194304] 0 2024-07-16T09:55:41.899 INFO:tasks.workunit.client.1.smithi176.stdout:5/706: write d4/d14/d19/fd6 [358064,127501] 0 2024-07-16T09:55:41.899 INFO:tasks.workunit.client.0.smithi156.stdout:1/750: rename dc/d14/d3e/d6c/d7a/fa6 to dc/fff 0 2024-07-16T09:55:41.900 INFO:tasks.workunit.client.1.smithi176.stdout:5/707: chown d4/d37/d72/d38/d39/f5d 15900 1 2024-07-16T09:55:41.902 INFO:tasks.workunit.client.1.smithi176.stdout:1/407: link c2 d4/d8/d12/d21/d25/d30/d44/c8d 0 2024-07-16T09:55:41.904 INFO:tasks.workunit.client.0.smithi156.stdout:2/617: creat d0/fd2 x:0 0 0 2024-07-16T09:55:41.910 INFO:tasks.workunit.client.0.smithi156.stdout:1/751: link dc/d14/d3e/d45/cc1 dc/d14/d3e/c100 0 2024-07-16T09:55:41.910 INFO:tasks.workunit.client.0.smithi156.stdout:2/618: write d0/d13/d17/d1c/d51/fc2 [683133,126941] 0 2024-07-16T09:55:41.910 INFO:tasks.workunit.client.0.smithi156.stdout:0/543: dread d0/d1a/d4d/d9/f26 [0,4194304] 0 2024-07-16T09:55:41.912 INFO:tasks.workunit.client.0.smithi156.stdout:2/619: write d0/f7 [1106512,54347] 0 2024-07-16T09:55:41.914 INFO:tasks.workunit.client.0.smithi156.stdout:2/620: write d0/d13/d17/d1c/fbb [198821,38820] 0 2024-07-16T09:55:41.917 INFO:tasks.workunit.client.0.smithi156.stdout:0/544: creat d0/d1a/d4d/d9/d27/d2c/d6d/d9d/fac x:0 0 0 2024-07-16T09:55:41.919 INFO:tasks.workunit.client.1.smithi176.stdout:5/708: dwrite d4/f9e [0,4194304] 0 2024-07-16T09:55:41.919 INFO:tasks.workunit.client.0.smithi156.stdout:1/752: link dc/d14/d8b/db4/df2/faf dc/d14/d8b/db4/df2/f101 0 2024-07-16T09:55:41.920 INFO:tasks.workunit.client.0.smithi156.stdout:0/545: write d0/d1a/f46 [1939153,130425] 0 2024-07-16T09:55:41.920 INFO:tasks.workunit.client.1.smithi176.stdout:1/408: dwrite d4/db/f1b [4194304,4194304] 0 2024-07-16T09:55:41.921 INFO:tasks.workunit.client.0.smithi156.stdout:1/753: fdatasync dc/d14/d3e/d6c/d7a/dcb/fdf 0 2024-07-16T09:55:41.921 INFO:tasks.workunit.client.0.smithi156.stdout:2/621: read d0/d5/d29/d2d/d5b/f6f [4093828,65536] 0 2024-07-16T09:55:41.924 INFO:tasks.workunit.client.1.smithi176.stdout:1/409: write d4/d8/f8c [795089,98344] 0 2024-07-16T09:55:41.924 INFO:tasks.workunit.client.0.smithi156.stdout:2/622: creat d0/d13/d6e/fd3 x:0 0 0 2024-07-16T09:55:41.925 INFO:tasks.workunit.client.0.smithi156.stdout:0/546: truncate d0/d1a/d4d/d9/d27/d3d/d62/f8e 444669 0 2024-07-16T09:55:41.927 INFO:tasks.workunit.client.0.smithi156.stdout:0/547: write d0/d22/d41/d52/f74 [808025,27105] 0 2024-07-16T09:55:41.927 INFO:tasks.workunit.client.1.smithi176.stdout:5/709: symlink d4/d9a/ded/l101 0 2024-07-16T09:55:41.928 INFO:tasks.workunit.client.1.smithi176.stdout:1/410: unlink d4/db/l73 0 2024-07-16T09:55:41.929 INFO:tasks.workunit.client.1.smithi176.stdout:1/411: fsync d4/db/f49 0 2024-07-16T09:55:41.932 INFO:tasks.workunit.client.1.smithi176.stdout:1/412: truncate d4/d8/f8c 1129277 0 2024-07-16T09:55:41.934 INFO:tasks.workunit.client.0.smithi156.stdout:1/754: creat dc/d14/d3e/d55/f102 x:0 0 0 2024-07-16T09:55:41.936 INFO:tasks.workunit.client.0.smithi156.stdout:1/755: dread - dc/d24/fb1 zero size 2024-07-16T09:55:41.936 INFO:tasks.workunit.client.1.smithi176.stdout:5/710: dwrite d4/d37/d72/f4f [4194304,4194304] 0 2024-07-16T09:55:41.937 INFO:tasks.workunit.client.1.smithi176.stdout:1/413: mkdir d4/d3a/d8e 0 2024-07-16T09:55:41.937 INFO:tasks.workunit.client.0.smithi156.stdout:2/623: creat d0/d13/d17/d1c/d51/dc6/fd4 x:0 0 0 2024-07-16T09:55:41.939 INFO:tasks.workunit.client.0.smithi156.stdout:1/756: write dc/d6b/d96/ff5 [600146,105238] 0 2024-07-16T09:55:41.942 INFO:tasks.workunit.client.1.smithi176.stdout:1/414: getdents d4/d3a/d8e 0 2024-07-16T09:55:41.942 INFO:tasks.workunit.client.1.smithi176.stdout:1/415: dread - d4/d8/d12/d21/f89 zero size 2024-07-16T09:55:41.943 INFO:tasks.workunit.client.0.smithi156.stdout:0/548: dread d0/d1a/d4d/d9/d27/f6a [0,4194304] 0 2024-07-16T09:55:41.943 INFO:tasks.workunit.client.0.smithi156.stdout:2/624: rename d0/d13/d17/d1c/f27 to d0/d5/dc5/fd5 0 2024-07-16T09:55:41.944 INFO:tasks.workunit.client.1.smithi176.stdout:5/711: rename d4/d37/d72/d82/d96 to d4/d37/d72/d38/d102 0 2024-07-16T09:55:41.945 INFO:tasks.workunit.client.1.smithi176.stdout:1/416: chown d4/d8/d12/d21/c8a 701152318 1 2024-07-16T09:55:41.946 INFO:tasks.workunit.client.1.smithi176.stdout:5/712: write d4/d37/d72/d38/d102/dff/fc3 [3296731,76719] 0 2024-07-16T09:55:41.946 INFO:tasks.workunit.client.0.smithi156.stdout:1/757: write dc/d14/d1e/f3d [116436,62102] 0 2024-07-16T09:55:41.946 INFO:tasks.workunit.client.0.smithi156.stdout:2/625: read - d0/d5/d29/d2d/d5b/d62/fb9 zero size 2024-07-16T09:55:41.949 INFO:tasks.workunit.client.1.smithi176.stdout:1/417: unlink d4/d3a/l5e 0 2024-07-16T09:55:41.950 INFO:tasks.workunit.client.1.smithi176.stdout:5/713: creat d4/d37/d72/d38/d102/f103 x:0 0 0 2024-07-16T09:55:41.950 INFO:tasks.workunit.client.0.smithi156.stdout:1/758: creat dc/d14/d3e/d6c/d7a/dcb/f103 x:0 0 0 2024-07-16T09:55:41.951 INFO:tasks.workunit.client.0.smithi156.stdout:2/626: creat d0/d5/d76/da1/fd6 x:0 0 0 2024-07-16T09:55:41.952 INFO:tasks.workunit.client.0.smithi156.stdout:1/759: fsync dc/d14/d3e/d45/fed 0 2024-07-16T09:55:41.954 INFO:tasks.workunit.client.0.smithi156.stdout:1/760: chown dc/d14/d3e/d55/d83/cc0 2542738 1 2024-07-16T09:55:41.955 INFO:tasks.workunit.client.0.smithi156.stdout:0/549: dwrite d0/d1a/d4d/d9/d27/d2c/d6d/f8b [0,4194304] 0 2024-07-16T09:55:41.958 INFO:tasks.workunit.client.1.smithi176.stdout:5/714: dwrite d4/d10/d17/f28 [0,4194304] 0 2024-07-16T09:55:41.962 INFO:tasks.workunit.client.1.smithi176.stdout:5/715: mknod d4/d37/d72/d38/d102/c104 0 2024-07-16T09:55:41.965 INFO:tasks.workunit.client.1.smithi176.stdout:5/716: chown d4/d14/dc5/def/df5 15 1 2024-07-16T09:55:41.967 INFO:tasks.workunit.client.1.smithi176.stdout:1/418: dwrite d4/d8/d12/d21/d34/f3d [0,4194304] 0 2024-07-16T09:55:41.970 INFO:tasks.workunit.client.1.smithi176.stdout:5/717: unlink d4/f86 0 2024-07-16T09:55:41.983 INFO:tasks.workunit.client.1.smithi176.stdout:1/419: write d4/d8/f26 [2802226,15754] 0 2024-07-16T09:55:41.984 INFO:tasks.workunit.client.1.smithi176.stdout:5/718: rmdir d4/da9 39 2024-07-16T09:55:41.986 INFO:tasks.workunit.client.1.smithi176.stdout:1/420: fsync d4/f24 0 2024-07-16T09:55:41.989 INFO:tasks.workunit.client.1.smithi176.stdout:1/421: mkdir d4/d8/d12/d8f 0 2024-07-16T09:55:41.996 INFO:tasks.workunit.client.1.smithi176.stdout:1/422: dwrite d4/d3a/d54/f6d [4194304,4194304] 0 2024-07-16T09:55:41.996 INFO:tasks.workunit.client.1.smithi176.stdout:5/719: truncate d4/d37/d72/d38/d39/fa3 1151465 0 2024-07-16T09:55:41.997 INFO:tasks.workunit.client.0.smithi156.stdout:3/507: sync 2024-07-16T09:55:41.998 INFO:tasks.workunit.client.1.smithi176.stdout:5/720: dread - d4/d37/fd7 zero size 2024-07-16T09:55:42.005 INFO:tasks.workunit.client.1.smithi176.stdout:1/423: dwrite d4/d8/d12/d21/d34/f58 [0,4194304] 0 2024-07-16T09:55:42.022 INFO:tasks.workunit.client.0.smithi156.stdout:2/627: truncate d0/d13/f8d 691911 0 2024-07-16T09:55:42.023 INFO:tasks.workunit.client.0.smithi156.stdout:2/628: chown d0/d13/d17/d1c/d86/fb7 29633819 1 2024-07-16T09:55:42.024 INFO:tasks.workunit.client.0.smithi156.stdout:2/629: write d0/d13/d6e/fd3 [668109,98641] 0 2024-07-16T09:55:42.035 INFO:tasks.workunit.client.1.smithi176.stdout:5/721: mkdir d4/d37/d72/d38/d102/d105 0 2024-07-16T09:55:42.035 INFO:tasks.workunit.client.1.smithi176.stdout:1/424: creat d4/d8/d12/d21/d25/f90 x:0 0 0 2024-07-16T09:55:42.038 INFO:tasks.workunit.client.1.smithi176.stdout:1/425: creat d4/d8/d12/d21/d25/d30/d44/d4d/f91 x:0 0 0 2024-07-16T09:55:42.043 INFO:tasks.workunit.client.0.smithi156.stdout:1/761: unlink dc/fdb 0 2024-07-16T09:55:42.044 INFO:tasks.workunit.client.0.smithi156.stdout:3/508: mkdir d8/da/d1d/d2c/d33/dad 0 2024-07-16T09:55:42.045 INFO:tasks.workunit.client.1.smithi176.stdout:5/722: write d4/d14/f9c [467598,63957] 0 2024-07-16T09:55:42.046 INFO:tasks.workunit.client.1.smithi176.stdout:1/426: symlink d4/d77/l92 0 2024-07-16T09:55:42.047 INFO:tasks.workunit.client.1.smithi176.stdout:1/427: chown d4/db/c40 28 1 2024-07-16T09:55:42.048 INFO:tasks.workunit.client.0.smithi156.stdout:0/550: creat d0/d1a/d76/d83/fad x:0 0 0 2024-07-16T09:55:42.048 INFO:tasks.workunit.client.1.smithi176.stdout:5/723: chown d4/d37/d72/d38/d102/d105 4681 1 2024-07-16T09:55:42.048 INFO:tasks.workunit.client.1.smithi176.stdout:1/428: stat d4/l14 0 2024-07-16T09:55:42.049 INFO:tasks.workunit.client.0.smithi156.stdout:2/630: truncate d0/f91 993617 0 2024-07-16T09:55:42.050 INFO:tasks.workunit.client.1.smithi176.stdout:1/429: write d4/d8/f69 [219847,30142] 0 2024-07-16T09:55:42.050 INFO:tasks.workunit.client.1.smithi176.stdout:5/724: mkdir d4/d37/d72/d106 0 2024-07-16T09:55:42.051 INFO:tasks.workunit.client.0.smithi156.stdout:3/509: rename d8/l18 to d8/da/d1d/d2c/d33/lae 0 2024-07-16T09:55:42.051 INFO:tasks.workunit.client.0.smithi156.stdout:3/510: readlink d8/da/l4c 0 2024-07-16T09:55:42.051 INFO:tasks.workunit.client.0.smithi156.stdout:2/631: rmdir d0/d13/d17/d1c 39 2024-07-16T09:55:42.052 INFO:tasks.workunit.client.0.smithi156.stdout:3/511: symlink d8/d14/laf 0 2024-07-16T09:55:42.056 INFO:tasks.workunit.client.0.smithi156.stdout:0/551: dwrite d0/d1a/d4d/f21 [0,4194304] 0 2024-07-16T09:55:42.056 INFO:tasks.workunit.client.0.smithi156.stdout:2/632: mknod d0/d5/d29/d2d/d5b/d62/d74/cd7 0 2024-07-16T09:55:42.056 INFO:tasks.workunit.client.0.smithi156.stdout:3/512: creat d8/da/d1d/d2c/d33/fb0 x:0 0 0 2024-07-16T09:55:42.057 INFO:tasks.workunit.client.1.smithi176.stdout:1/430: dwrite d4/d8/d12/d21/d25/f86 [0,4194304] 0 2024-07-16T09:55:42.058 INFO:tasks.workunit.client.0.smithi156.stdout:0/552: read d0/d1a/d4d/d9/f8f [347151,109825] 0 2024-07-16T09:55:42.060 INFO:tasks.workunit.client.1.smithi176.stdout:5/725: write d4/d6c/f6f [2579515,123495] 0 2024-07-16T09:55:42.063 INFO:tasks.workunit.client.0.smithi156.stdout:3/513: mknod d8/da/cb1 0 2024-07-16T09:55:42.064 INFO:tasks.workunit.client.0.smithi156.stdout:0/553: creat d0/d1a/d4d/d9/d27/d50/fae x:0 0 0 2024-07-16T09:55:42.065 INFO:tasks.workunit.client.0.smithi156.stdout:2/633: dwrite d0/d13/f9e [0,4194304] 0 2024-07-16T09:55:42.067 INFO:tasks.workunit.client.0.smithi156.stdout:3/514: mkdir d8/da/d1d/db2 0 2024-07-16T09:55:42.068 INFO:tasks.workunit.client.1.smithi176.stdout:1/431: truncate d4/d8/d12/d21/d34/f3d 1274655 0 2024-07-16T09:55:42.069 INFO:tasks.workunit.client.0.smithi156.stdout:0/554: truncate d0/d1a/d4d/d9/d27/d50/fae 446605 0 2024-07-16T09:55:42.073 INFO:tasks.workunit.client.0.smithi156.stdout:0/555: write d0/d1a/d76/f87 [164621,22846] 0 2024-07-16T09:55:42.074 INFO:tasks.workunit.client.0.smithi156.stdout:2/634: dwrite d0/f7 [4194304,4194304] 0 2024-07-16T09:55:42.084 INFO:tasks.workunit.client.0.smithi156.stdout:2/635: dread d0/d13/f59 [0,4194304] 0 2024-07-16T09:55:42.086 INFO:tasks.workunit.client.1.smithi176.stdout:5/726: dread d4/d10/d17/f50 [0,4194304] 0 2024-07-16T09:55:42.086 INFO:tasks.workunit.client.0.smithi156.stdout:2/636: mknod d0/d5/d29/d2d/d5b/d62/cd8 0 2024-07-16T09:55:42.087 INFO:tasks.workunit.client.0.smithi156.stdout:2/637: chown d0/d13/d6e/fd3 4 1 2024-07-16T09:55:42.088 INFO:tasks.workunit.client.0.smithi156.stdout:2/638: creat d0/d5/d29/fd9 x:0 0 0 2024-07-16T09:55:42.090 INFO:tasks.workunit.client.0.smithi156.stdout:2/639: chown d0/f91 60988196 1 2024-07-16T09:55:42.096 INFO:tasks.workunit.client.1.smithi176.stdout:0/674: sync 2024-07-16T09:55:42.096 INFO:tasks.workunit.client.1.smithi176.stdout:4/552: sync 2024-07-16T09:55:42.097 INFO:tasks.workunit.client.1.smithi176.stdout:9/651: sync 2024-07-16T09:55:42.097 INFO:tasks.workunit.client.1.smithi176.stdout:2/620: sync 2024-07-16T09:55:42.097 INFO:tasks.workunit.client.1.smithi176.stdout:7/639: sync 2024-07-16T09:55:42.097 INFO:tasks.workunit.client.1.smithi176.stdout:3/487: sync 2024-07-16T09:55:42.097 INFO:tasks.workunit.client.1.smithi176.stdout:6/590: sync 2024-07-16T09:55:42.098 INFO:tasks.workunit.client.1.smithi176.stdout:8/562: sync 2024-07-16T09:55:42.098 INFO:tasks.workunit.client.1.smithi176.stdout:1/432: truncate d4/f17 3995368 0 2024-07-16T09:55:42.099 INFO:tasks.workunit.client.1.smithi176.stdout:2/621: chown f9 1 1 2024-07-16T09:55:42.102 INFO:tasks.workunit.client.1.smithi176.stdout:6/591: write d1/d5/d47/fa6 [660344,79926] 0 2024-07-16T09:55:42.103 INFO:tasks.workunit.client.1.smithi176.stdout:7/640: rmdir d4/dd/d26 39 2024-07-16T09:55:42.103 INFO:tasks.workunit.client.1.smithi176.stdout:0/675: read d5/d1c/f8e [303933,91183] 0 2024-07-16T09:55:42.105 INFO:tasks.workunit.client.1.smithi176.stdout:9/652: creat d5/de/da5/dbf/fe1 x:0 0 0 2024-07-16T09:55:42.105 INFO:tasks.workunit.client.1.smithi176.stdout:3/488: rename d3/d17/d1d/d4d/f42 to d3/d85/f99 0 2024-07-16T09:55:42.106 INFO:tasks.workunit.client.1.smithi176.stdout:4/553: chown da/d14/d40/d5d/d73/f61 1895054 1 2024-07-16T09:55:42.106 INFO:tasks.workunit.client.0.smithi156.stdout:3/515: write d8/da/d21/d80/f82 [136357,72661] 0 2024-07-16T09:55:42.107 INFO:tasks.workunit.client.1.smithi176.stdout:8/563: mkdir d8/d20/d2a/d53/d9a/dc1 0 2024-07-16T09:55:42.108 INFO:tasks.workunit.client.1.smithi176.stdout:7/641: stat d4/d47/d6d/d89/d66/ccb 0 2024-07-16T09:55:42.108 INFO:tasks.workunit.client.1.smithi176.stdout:4/554: read - da/dc/d4b/fa7 zero size 2024-07-16T09:55:42.108 INFO:tasks.workunit.client.1.smithi176.stdout:8/564: stat d8/d14/d1e/d55 0 2024-07-16T09:55:42.109 INFO:tasks.workunit.client.1.smithi176.stdout:6/592: symlink d1/d5/d47/d62/d80/lda 0 2024-07-16T09:55:42.110 INFO:tasks.workunit.client.1.smithi176.stdout:9/653: symlink d5/de/da5/dc1/le2 0 2024-07-16T09:55:42.111 INFO:tasks.workunit.client.1.smithi176.stdout:0/676: truncate d5/d39/da3/fd9 1040560 0 2024-07-16T09:55:42.112 INFO:tasks.workunit.client.0.smithi156.stdout:2/640: dwrite d0/d5/d29/d2d/d5b/d5f/f66 [0,4194304] 0 2024-07-16T09:55:42.113 INFO:tasks.workunit.client.0.smithi156.stdout:3/516: dread d8/da/d1d/d2c/d33/f49 [4194304,4194304] 0 2024-07-16T09:55:42.115 INFO:tasks.workunit.client.1.smithi176.stdout:0/677: stat d5/d1c/d24/d35/d50/c6a 0 2024-07-16T09:55:42.115 INFO:tasks.workunit.client.1.smithi176.stdout:1/433: rename d4/d8/d12/d21/d25/d30/l33 to d4/d3a/d54/d5a/d7a/l93 0 2024-07-16T09:55:42.118 INFO:tasks.workunit.client.0.smithi156.stdout:3/517: mkdir d8/da/d1d/d6f/db3 0 2024-07-16T09:55:42.118 INFO:tasks.workunit.client.1.smithi176.stdout:1/434: chown d4/d3a/d54/f8b 2 1 2024-07-16T09:55:42.119 INFO:tasks.workunit.client.1.smithi176.stdout:6/593: creat d1/fdb x:0 0 0 2024-07-16T09:55:42.120 INFO:tasks.workunit.client.1.smithi176.stdout:3/489: dwrite d3/da/f13 [0,4194304] 0 2024-07-16T09:55:42.123 INFO:tasks.workunit.client.0.smithi156.stdout:3/518: dwrite d8/da/d28/d43/f5b [0,4194304] 0 2024-07-16T09:55:42.125 INFO:tasks.workunit.client.0.smithi156.stdout:3/519: chown d8/da/d28/d43/d63/c95 613695 1 2024-07-16T09:55:42.125 INFO:tasks.workunit.client.1.smithi176.stdout:4/555: dwrite da/d3d/d17/d48/faa [0,4194304] 0 2024-07-16T09:55:42.126 INFO:tasks.workunit.client.1.smithi176.stdout:0/678: write d5/d1c/d24/d35/d50/dd5/fda [566053,123078] 0 2024-07-16T09:55:42.129 INFO:tasks.workunit.client.0.smithi156.stdout:2/641: dwrite d0/d13/d17/d1c/f5e [0,4194304] 0 2024-07-16T09:55:42.133 INFO:tasks.workunit.client.0.smithi156.stdout:3/520: rename d8/da/d1d/d2c/d33/l97 to d8/d14/lb4 0 2024-07-16T09:55:42.133 INFO:tasks.workunit.client.0.smithi156.stdout:2/642: write d0/d5/d29/d2d/d5b/d5f/d89/f9a [805864,114835] 0 2024-07-16T09:55:42.136 INFO:tasks.workunit.client.1.smithi176.stdout:9/654: mkdir d5/d1d/dd2/de3 0 2024-07-16T09:55:42.136 INFO:tasks.workunit.client.1.smithi176.stdout:6/594: rmdir d1/d5d 39 2024-07-16T09:55:42.138 INFO:tasks.workunit.client.0.smithi156.stdout:2/643: chown d0/d5/d29/d2d/d5b/d62/d74/lc0 215574 1 2024-07-16T09:55:42.139 INFO:tasks.workunit.client.0.smithi156.stdout:3/521: read d8/f10 [3080194,50948] 0 2024-07-16T09:55:42.140 INFO:tasks.workunit.client.0.smithi156.stdout:2/644: readlink d0/l15 0 2024-07-16T09:55:42.144 INFO:tasks.workunit.client.1.smithi176.stdout:0/679: mkdir d5/d1c/d24/d35/d50/d9b/de3 0 2024-07-16T09:55:42.144 INFO:tasks.workunit.client.1.smithi176.stdout:8/565: creat d8/d20/d2a/db9/fc2 x:0 0 0 2024-07-16T09:55:42.146 INFO:tasks.workunit.client.1.smithi176.stdout:6/595: write d1/d5d/d9e/faa [336606,6345] 0 2024-07-16T09:55:42.147 INFO:tasks.workunit.client.1.smithi176.stdout:4/556: mkdir da/d3d/dac 0 2024-07-16T09:55:42.148 INFO:tasks.workunit.client.0.smithi156.stdout:2/645: dwrite d0/d5/d29/d2d/f5a [0,4194304] 0 2024-07-16T09:55:42.148 INFO:tasks.workunit.client.1.smithi176.stdout:4/557: read - da/d41/f99 zero size 2024-07-16T09:55:42.150 INFO:tasks.workunit.client.1.smithi176.stdout:2/622: link dd/d36/d82/f8d dd/d2e/d35/d4a/fce 0 2024-07-16T09:55:42.152 INFO:tasks.workunit.client.1.smithi176.stdout:9/655: fdatasync d5/f10 0 2024-07-16T09:55:42.153 INFO:tasks.workunit.client.1.smithi176.stdout:7/642: link d4/dd/l71 d4/d47/ld6 0 2024-07-16T09:55:42.155 INFO:tasks.workunit.client.1.smithi176.stdout:9/656: readlink d5/d1d/dc7/l6c 0 2024-07-16T09:55:42.162 INFO:tasks.workunit.client.1.smithi176.stdout:8/566: creat d8/d23/fc3 x:0 0 0 2024-07-16T09:55:42.163 INFO:tasks.workunit.client.1.smithi176.stdout:0/680: dwrite d5/d1c/d24/d35/d50/d58/d63/fd2 [0,4194304] 0 2024-07-16T09:55:42.163 INFO:tasks.workunit.client.1.smithi176.stdout:6/596: dwrite d1/d5/d47/f51 [4194304,4194304] 0 2024-07-16T09:55:42.164 INFO:tasks.workunit.client.1.smithi176.stdout:9/657: chown d5/de/da5/dc1/ccf 9419 1 2024-07-16T09:55:42.164 INFO:tasks.workunit.client.0.smithi156.stdout:3/522: dwrite d8/da/fc [0,4194304] 0 2024-07-16T09:55:42.165 INFO:tasks.workunit.client.1.smithi176.stdout:1/435: getdents d4/d8/d2c 0 2024-07-16T09:55:42.165 INFO:tasks.workunit.client.1.smithi176.stdout:4/558: creat da/d14/fad x:0 0 0 2024-07-16T09:55:42.167 INFO:tasks.workunit.client.1.smithi176.stdout:7/643: mknod d4/d47/d6d/db7/cd7 0 2024-07-16T09:55:42.167 INFO:tasks.workunit.client.1.smithi176.stdout:3/490: truncate d3/da/d33/f52 2958275 0 2024-07-16T09:55:42.171 INFO:tasks.workunit.client.1.smithi176.stdout:7/644: dread - d4/d47/d5d/fc9 zero size 2024-07-16T09:55:42.171 INFO:tasks.workunit.client.1.smithi176.stdout:0/681: read d5/d1c/d24/d35/d50/d5b/f65 [1506890,26788] 0 2024-07-16T09:55:42.172 INFO:tasks.workunit.client.0.smithi156.stdout:3/523: dread d8/da/d28/f7c [0,4194304] 0 2024-07-16T09:55:42.174 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:55:41 smithi156 ceph-mon[29302]: pgmap v252: 65 pgs: 65 active+clean; 2.1 GiB data, 7.7 GiB used, 529 GiB / 536 GiB avail; 61 MiB/s rd, 280 MiB/s wr, 528 op/s 2024-07-16T09:55:42.177 INFO:tasks.workunit.client.1.smithi176.stdout:2/623: dread dd/d36/f61 [0,4194304] 0 2024-07-16T09:55:42.182 INFO:tasks.workunit.client.1.smithi176.stdout:8/567: stat d8/d14/d1e/d33/l7f 0 2024-07-16T09:55:42.189 INFO:tasks.workunit.client.1.smithi176.stdout:6/597: dread d1/d5/d47/fd4 [0,4194304] 0 2024-07-16T09:55:42.190 INFO:tasks.workunit.client.1.smithi176.stdout:4/559: rename da/f5f to da/dc/d10/d96/d69/fae 0 2024-07-16T09:55:42.194 INFO:tasks.workunit.client.1.smithi176.stdout:0/682: mknod d5/d1c/d24/d35/d50/d7c/ce4 0 2024-07-16T09:55:42.196 INFO:tasks.workunit.client.1.smithi176.stdout:7/645: dread d4/d47/d6d/d89/d66/fa4 [0,4194304] 0 2024-07-16T09:55:42.196 INFO:tasks.workunit.client.1.smithi176.stdout:2/624: unlink dd/d16/d39/f6d 0 2024-07-16T09:55:42.198 INFO:tasks.workunit.client.1.smithi176.stdout:8/568: read d8/d20/d2a/f3f [1847721,85219] 0 2024-07-16T09:55:42.199 INFO:tasks.workunit.client.0.smithi156.stdout:6/511: sync 2024-07-16T09:55:42.200 INFO:tasks.workunit.client.0.smithi156.stdout:9/605: sync 2024-07-16T09:55:42.202 INFO:tasks.workunit.client.0.smithi156.stdout:6/512: write d3/d13/d17/d4f/d30/d45/d5f/f8b [184158,70719] 0 2024-07-16T09:55:42.203 INFO:tasks.workunit.client.0.smithi156.stdout:3/524: dread f3 [4194304,4194304] 0 2024-07-16T09:55:42.204 INFO:tasks.workunit.client.0.smithi156.stdout:6/513: dread - d3/d13/d17/d4f/d5c/f80 zero size 2024-07-16T09:55:42.204 INFO:tasks.workunit.client.1.smithi176.stdout:2/625: symlink dd/d49/lcf 0 2024-07-16T09:55:42.205 INFO:tasks.workunit.client.1.smithi176.stdout:4/560: link da/dc/d4b/f97 da/dc/d10/d96/faf 0 2024-07-16T09:55:42.205 INFO:tasks.workunit.client.0.smithi156.stdout:9/606: unlink d2/d5/d16/d22/d36/f31 0 2024-07-16T09:55:42.206 INFO:tasks.workunit.client.0.smithi156.stdout:3/525: rmdir d8/da/d21 39 2024-07-16T09:55:42.207 INFO:tasks.workunit.client.0.smithi156.stdout:6/514: dread d3/d13/d17/d4f/d94/d19/d31/d35/f91 [0,4194304] 0 2024-07-16T09:55:42.208 INFO:tasks.workunit.client.0.smithi156.stdout:6/515: chown d3/f39 2395 1 2024-07-16T09:55:42.208 INFO:tasks.workunit.client.1.smithi176.stdout:1/436: link d4/f24 d4/d8/d12/d21/d25/d30/d44/d4d/f94 0 2024-07-16T09:55:42.209 INFO:tasks.workunit.client.1.smithi176.stdout:0/683: mknod d5/d1c/d24/d35/d50/d9b/de3/ce5 0 2024-07-16T09:55:42.209 INFO:tasks.workunit.client.0.smithi156.stdout:1/762: mkdir dc/d6b/d104 0 2024-07-16T09:55:42.210 INFO:tasks.workunit.client.0.smithi156.stdout:9/607: unlink d2/d5/d16/d22/l2d 0 2024-07-16T09:55:42.210 INFO:tasks.workunit.client.1.smithi176.stdout:3/491: dread d3/da/fe [0,4194304] 0 2024-07-16T09:55:42.211 INFO:tasks.workunit.client.0.smithi156.stdout:3/526: read d8/da/d13/f17 [2040994,31014] 0 2024-07-16T09:55:42.211 INFO:tasks.workunit.client.1.smithi176.stdout:8/569: dwrite d8/d14/d1e/d55/fa3 [0,4194304] 0 2024-07-16T09:55:42.212 INFO:tasks.workunit.client.1.smithi176.stdout:6/598: creat d1/d5/d7/fdc x:0 0 0 2024-07-16T09:55:42.212 INFO:tasks.workunit.client.1.smithi176.stdout:7/646: write d4/dd/d26/dc2/d17/d1d/f73 [448521,42066] 0 2024-07-16T09:55:42.213 INFO:tasks.workunit.client.0.smithi156.stdout:6/516: dread d3/d13/d17/d4f/d30/fa3 [0,4194304] 0 2024-07-16T09:55:42.216 INFO:tasks.workunit.client.0.smithi156.stdout:6/517: fdatasync d3/d13/d17/d4f/d4d/d3b/f8f 0 2024-07-16T09:55:42.217 INFO:tasks.workunit.client.1.smithi176.stdout:3/492: chown d3/d17/l3e 1754071684 1 2024-07-16T09:55:42.218 INFO:tasks.workunit.client.0.smithi156.stdout:3/527: dwrite d8/da/d1d/d2c/d7d/fa1 [0,4194304] 0 2024-07-16T09:55:42.220 INFO:tasks.workunit.client.1.smithi176.stdout:7/647: mkdir d4/dd/d26/dd8 0 2024-07-16T09:55:42.221 INFO:tasks.workunit.client.1.smithi176.stdout:6/599: read - d1/dc/fab zero size 2024-07-16T09:55:42.222 INFO:tasks.workunit.client.1.smithi176.stdout:4/561: getdents da/d14/d40 0 2024-07-16T09:55:42.223 INFO:tasks.workunit.client.1.smithi176.stdout:2/626: link dd/d2e/d35/lca dd/d49/d52/d7c/ld0 0 2024-07-16T09:55:42.224 INFO:tasks.workunit.client.0.smithi156.stdout:3/528: creat d8/da/d1d/d6f/fb5 x:0 0 0 2024-07-16T09:55:42.224 INFO:tasks.workunit.client.1.smithi176.stdout:3/493: fdatasync d3/d17/d1d/d4f/d59/f3a 0 2024-07-16T09:55:42.225 INFO:tasks.workunit.client.0.smithi156.stdout:6/518: getdents d3/d13/d17/d4f/d4d/d3b/da5 0 2024-07-16T09:55:42.225 INFO:tasks.workunit.client.1.smithi176.stdout:9/658: dread d5/fb [0,4194304] 0 2024-07-16T09:55:42.226 INFO:tasks.workunit.client.0.smithi156.stdout:6/519: write d3/d13/d17/f8a [1483702,11033] 0 2024-07-16T09:55:42.229 INFO:tasks.workunit.client.0.smithi156.stdout:3/529: rename d8/da/d13/f24 to d8/da/d13/fb6 0 2024-07-16T09:55:42.230 INFO:tasks.workunit.client.1.smithi176.stdout:1/437: creat d4/d8/d12/d21/d25/d30/d44/d4d/f95 x:0 0 0 2024-07-16T09:55:42.231 INFO:tasks.workunit.client.0.smithi156.stdout:6/520: dread d3/d13/d17/d4f/d4d/d3b/f96 [0,4194304] 0 2024-07-16T09:55:42.237 INFO:tasks.workunit.client.0.smithi156.stdout:3/530: dwrite d8/da/d28/d43/f5a [0,4194304] 0 2024-07-16T09:55:42.237 INFO:tasks.workunit.client.1.smithi176.stdout:6/600: truncate d1/dc/d5b/f9d 285302 0 2024-07-16T09:55:42.239 INFO:tasks.workunit.client.0.smithi156.stdout:6/521: chown d3/d13/d17/d4f/d94/fe 23 1 2024-07-16T09:55:42.240 INFO:tasks.workunit.client.0.smithi156.stdout:3/531: creat d8/d14/fb7 x:0 0 0 2024-07-16T09:55:42.243 INFO:tasks.workunit.client.1.smithi176.stdout:0/684: dwrite d5/d1c/d24/f30 [0,4194304] 0 2024-07-16T09:55:42.244 INFO:tasks.workunit.client.0.smithi156.stdout:1/763: truncate dc/d6b/f80 1290599 0 2024-07-16T09:55:42.246 INFO:tasks.workunit.client.0.smithi156.stdout:1/764: unlink dc/d14/d3e/d55/d58/l81 0 2024-07-16T09:55:42.247 INFO:tasks.workunit.client.0.smithi156.stdout:3/532: symlink d8/da/d1d/db2/lb8 0 2024-07-16T09:55:42.248 INFO:tasks.workunit.client.0.smithi156.stdout:6/522: getdents d3/d13/d17/d4f/d94 0 2024-07-16T09:55:42.248 INFO:tasks.workunit.client.0.smithi156.stdout:1/765: read dc/f10 [2691424,42226] 0 2024-07-16T09:55:42.249 INFO:tasks.workunit.client.0.smithi156.stdout:3/533: dread - d8/da/d1d/d2c/d33/fb0 zero size 2024-07-16T09:55:42.250 INFO:tasks.workunit.client.0.smithi156.stdout:6/523: chown d3/d13/d17/d4f/d1e/l27 896576 1 2024-07-16T09:55:42.250 INFO:tasks.workunit.client.0.smithi156.stdout:3/534: fdatasync d8/da/d13/f75 0 2024-07-16T09:55:42.251 INFO:tasks.workunit.client.0.smithi156.stdout:9/608: getdents d2/d5/d16/d22/d36 0 2024-07-16T09:55:42.254 INFO:tasks.workunit.client.0.smithi156.stdout:1/766: rmdir dc/d14/d8b/db4 39 2024-07-16T09:55:42.255 INFO:tasks.workunit.client.1.smithi176.stdout:4/562: write da/dc/d10/d96/d69/fae [354438,758] 0 2024-07-16T09:55:42.255 INFO:tasks.workunit.client.1.smithi176.stdout:4/563: chown da/dc/l1e 6919 1 2024-07-16T09:55:42.255 INFO:tasks.workunit.client.1.smithi176.stdout:3/494: dwrite d3/da/f77 [0,4194304] 0 2024-07-16T09:55:42.256 INFO:tasks.workunit.client.0.smithi156.stdout:6/524: symlink d3/d13/d17/d4f/d4d/d3b/d77/lac 0 2024-07-16T09:55:42.257 INFO:tasks.workunit.client.1.smithi176.stdout:9/659: dwrite d5/d1d/dd2/f91 [0,4194304] 0 2024-07-16T09:55:42.258 INFO:tasks.workunit.client.0.smithi156.stdout:3/535: rename d8/da/d28/f7c to d8/da/d13/fb9 0 2024-07-16T09:55:42.258 INFO:tasks.workunit.client.0.smithi156.stdout:1/767: dread - dc/d14/d8b/db4/fd6 zero size 2024-07-16T09:55:42.260 INFO:tasks.workunit.client.1.smithi176.stdout:7/648: getdents d4/d5 0 2024-07-16T09:55:42.261 INFO:tasks.workunit.client.0.smithi156.stdout:9/609: dwrite d2/d5/d16/d22/d36/d33/f82 [0,4194304] 0 2024-07-16T09:55:42.261 INFO:tasks.workunit.client.0.smithi156.stdout:1/768: dread - dc/d14/d8b/db4/fe0 zero size 2024-07-16T09:55:42.261 INFO:tasks.workunit.client.0.smithi156.stdout:6/525: readlink d3/d13/d17/d4f/d94/d19/d31/d35/l84 0 2024-07-16T09:55:42.262 INFO:tasks.workunit.client.1.smithi176.stdout:9/660: dread - d5/d1d/d67/fdf zero size 2024-07-16T09:55:42.264 INFO:tasks.workunit.client.0.smithi156.stdout:9/610: chown d2/d5/d16/d22/d36/lbb 20831 1 2024-07-16T09:55:42.265 INFO:tasks.workunit.client.1.smithi176.stdout:1/438: rename d4/d8/f1e to d4/d8/d12/d21/d25/d30/d44/d4d/f96 0 2024-07-16T09:55:42.265 INFO:tasks.workunit.client.0.smithi156.stdout:3/536: readlink d8/da/d1d/d2c/d33/lae 0 2024-07-16T09:55:42.267 INFO:tasks.workunit.client.0.smithi156.stdout:6/526: unlink d3/d13/d17/d56/f64 0 2024-07-16T09:55:42.268 INFO:tasks.workunit.client.0.smithi156.stdout:3/537: read d8/da/d1d/d2c/d33/f88 [2485977,81228] 0 2024-07-16T09:55:42.269 INFO:tasks.workunit.client.0.smithi156.stdout:6/527: chown d3/d13/d17/d4f/d1e/f3d 824083 1 2024-07-16T09:55:42.270 INFO:tasks.workunit.client.0.smithi156.stdout:1/769: read dc/d14/d1e/f21 [1318369,35486] 0 2024-07-16T09:55:42.270 INFO:tasks.workunit.client.1.smithi176.stdout:0/685: mkdir d5/d1c/d24/d26/dbd/de6 0 2024-07-16T09:55:42.271 INFO:tasks.workunit.client.0.smithi156.stdout:9/611: rmdir d2/d5/d16/d57/da3 39 2024-07-16T09:55:42.271 INFO:tasks.workunit.client.1.smithi176.stdout:8/570: dwrite d8/d23/d2c/d34/f84 [0,4194304] 0 2024-07-16T09:55:42.273 INFO:tasks.workunit.client.1.smithi176.stdout:1/439: creat d4/d82/f97 x:0 0 0 2024-07-16T09:55:42.273 INFO:tasks.workunit.client.1.smithi176.stdout:4/564: fdatasync da/d3d/d17/f31 0 2024-07-16T09:55:42.273 INFO:tasks.workunit.client.0.smithi156.stdout:3/538: mknod d8/da/d28/d43/cba 0 2024-07-16T09:55:42.277 INFO:tasks.workunit.client.0.smithi156.stdout:9/612: symlink d2/d5/d16/d22/d2b/d3d/lc0 0 2024-07-16T09:55:42.278 INFO:tasks.workunit.client.1.smithi176.stdout:3/495: dwrite d3/d17/d83/f8a [0,4194304] 0 2024-07-16T09:55:42.279 INFO:tasks.workunit.client.0.smithi156.stdout:3/539: symlink d8/da/d1d/db2/lbb 0 2024-07-16T09:55:42.279 INFO:tasks.workunit.client.1.smithi176.stdout:2/627: dread dd/d16/f32 [0,4194304] 0 2024-07-16T09:55:42.280 INFO:tasks.workunit.client.1.smithi176.stdout:3/496: stat d3/d17/f88 0 2024-07-16T09:55:42.281 INFO:tasks.workunit.client.1.smithi176.stdout:9/661: write d5/d1d/d20/fcc [1810300,14224] 0 2024-07-16T09:55:42.282 INFO:tasks.workunit.client.0.smithi156.stdout:6/528: dwrite d3/d13/d17/d4f/d4d/d3b/f47 [4194304,4194304] 0 2024-07-16T09:55:42.282 INFO:tasks.workunit.client.0.smithi156.stdout:9/613: chown d2/d5/d16/d22/l58 530017 1 2024-07-16T09:55:42.285 INFO:tasks.workunit.client.0.smithi156.stdout:1/770: dwrite dc/d14/d3e/d6c/d7a/dcb/fce [0,4194304] 0 2024-07-16T09:55:42.287 INFO:tasks.workunit.client.1.smithi176.stdout:8/571: creat d8/d20/d2a/d31/db4/fc4 x:0 0 0 2024-07-16T09:55:42.288 INFO:tasks.workunit.client.1.smithi176.stdout:8/572: chown d8/c94 0 1 2024-07-16T09:55:42.290 INFO:tasks.workunit.client.1.smithi176.stdout:4/565: creat da/dc/d10/d96/d69/fb0 x:0 0 0 2024-07-16T09:55:42.292 INFO:tasks.workunit.client.0.smithi156.stdout:6/529: chown d3/d13/d17/d4f/d30/d45/d5f 403317142 1 2024-07-16T09:55:42.292 INFO:tasks.workunit.client.1.smithi176.stdout:9/662: dwrite d5/d1d/dc7/d89/fb9 [0,4194304] 0 2024-07-16T09:55:42.293 INFO:tasks.workunit.client.0.smithi156.stdout:3/540: write d8/da/d1d/d2c/d33/f88 [1217186,51212] 0 2024-07-16T09:55:42.294 INFO:tasks.workunit.client.1.smithi176.stdout:4/566: chown da/dc/d10/f59 120676549 1 2024-07-16T09:55:42.294 INFO:tasks.workunit.client.1.smithi176.stdout:2/628: creat dd/d36/d45/fd1 x:0 0 0 2024-07-16T09:55:42.294 INFO:tasks.workunit.client.0.smithi156.stdout:1/771: creat dc/d6b/d8a/df8/f105 x:0 0 0 2024-07-16T09:55:42.295 INFO:tasks.workunit.client.0.smithi156.stdout:6/530: chown d3/d13/d17/c28 0 1 2024-07-16T09:55:42.296 INFO:tasks.workunit.client.1.smithi176.stdout:4/567: chown da/dc/la0 318057992 1 2024-07-16T09:55:42.297 INFO:tasks.workunit.client.1.smithi176.stdout:0/686: mknod d5/d1c/d24/d26/ce7 0 2024-07-16T09:55:42.298 INFO:tasks.workunit.client.0.smithi156.stdout:1/772: symlink dc/d6b/d8a/l106 0 2024-07-16T09:55:42.302 INFO:tasks.workunit.client.0.smithi156.stdout:1/773: creat dc/d14/d3e/d55/d83/f107 x:0 0 0 2024-07-16T09:55:42.302 INFO:tasks.workunit.client.0.smithi156.stdout:1/774: stat dc/d14/d3e/fc3 0 2024-07-16T09:55:42.303 INFO:tasks.workunit.client.0.smithi156.stdout:6/531: mknod d3/d13/d17/d4f/d30/d45/cad 0 2024-07-16T09:55:42.305 INFO:tasks.workunit.client.1.smithi176.stdout:9/663: fsync d5/de/f8b 0 2024-07-16T09:55:42.305 INFO:tasks.workunit.client.1.smithi176.stdout:0/687: mknod d5/d2c/ce8 0 2024-07-16T09:55:42.305 INFO:tasks.workunit.client.1.smithi176.stdout:2/629: creat dd/d16/d39/d3d/fd2 x:0 0 0 2024-07-16T09:55:42.305 INFO:tasks.workunit.client.0.smithi156.stdout:1/775: truncate dc/d14/d3e/d55/f5c 1799835 0 2024-07-16T09:55:42.306 INFO:tasks.workunit.client.0.smithi156.stdout:1/776: read - dc/d6b/dd5/ff7 zero size 2024-07-16T09:55:42.308 INFO:tasks.workunit.client.1.smithi176.stdout:9/664: rmdir d5/d1d/d20/d78 39 2024-07-16T09:55:42.309 INFO:tasks.workunit.client.1.smithi176.stdout:0/688: truncate d5/d2c/d6c/d83/fa6 1546342 0 2024-07-16T09:55:42.310 INFO:tasks.workunit.client.1.smithi176.stdout:0/689: readlink d5/d44/lb5 0 2024-07-16T09:55:42.310 INFO:tasks.workunit.client.0.smithi156.stdout:6/532: fsync d3/d13/d17/d4f/d5c/f65 0 2024-07-16T09:55:42.310 INFO:tasks.workunit.client.1.smithi176.stdout:7/649: truncate d4/d47/d6d/f6e 967727 0 2024-07-16T09:55:42.312 INFO:tasks.workunit.client.0.smithi156.stdout:1/777: getdents dc/d14/d3e/d6c/d7a 0 2024-07-16T09:55:42.313 INFO:tasks.workunit.client.0.smithi156.stdout:6/533: mkdir d3/d13/d17/dae 0 2024-07-16T09:55:42.316 INFO:tasks.workunit.client.0.smithi156.stdout:1/778: creat dc/d14/d3e/d55/d83/f108 x:0 0 0 2024-07-16T09:55:42.316 INFO:tasks.workunit.client.1.smithi176.stdout:1/440: dread d4/d8/d12/d21/d25/d30/f61 [0,4194304] 0 2024-07-16T09:55:42.316 INFO:tasks.workunit.client.1.smithi176.stdout:2/630: getdents dd/d49 0 2024-07-16T09:55:42.317 INFO:tasks.workunit.client.1.smithi176.stdout:0/690: creat d5/d1c/d24/d35/d50/fe9 x:0 0 0 2024-07-16T09:55:42.318 INFO:tasks.workunit.client.0.smithi156.stdout:6/534: rename d3/fa0 to d3/d13/d17/d4f/faf 0 2024-07-16T09:55:42.318 INFO:tasks.workunit.client.0.smithi156.stdout:1/779: creat dc/d14/d3e/d55/f109 x:0 0 0 2024-07-16T09:55:42.319 INFO:tasks.workunit.client.1.smithi176.stdout:1/441: read d4/d8/d12/d21/d34/f58 [4398961,39391] 0 2024-07-16T09:55:42.319 INFO:tasks.workunit.client.0.smithi156.stdout:6/535: write d3/d13/d17/d4f/d94/d19/d2e/f3e [2473972,97931] 0 2024-07-16T09:55:42.320 INFO:tasks.workunit.client.0.smithi156.stdout:1/780: fsync dc/d14/d3e/d45/fb7 0 2024-07-16T09:55:42.321 INFO:tasks.workunit.client.0.smithi156.stdout:6/536: fdatasync d3/d13/d17/d4f/d4d/f61 0 2024-07-16T09:55:42.321 INFO:tasks.workunit.client.0.smithi156.stdout:8/428: sync 2024-07-16T09:55:42.321 INFO:tasks.workunit.client.0.smithi156.stdout:7/455: sync 2024-07-16T09:55:42.321 INFO:tasks.workunit.client.0.smithi156.stdout:5/608: sync 2024-07-16T09:55:42.321 INFO:tasks.workunit.client.0.smithi156.stdout:4/638: sync 2024-07-16T09:55:42.321 INFO:tasks.workunit.client.0.smithi156.stdout:0/556: sync 2024-07-16T09:55:42.322 INFO:tasks.workunit.client.1.smithi176.stdout:9/665: dwrite d5/de/d30/f9a [0,4194304] 0 2024-07-16T09:55:42.322 INFO:tasks.workunit.client.1.smithi176.stdout:2/631: chown dd/d49/d52/d7c/fb4 28 1 2024-07-16T09:55:42.323 INFO:tasks.workunit.client.0.smithi156.stdout:5/609: chown d1b/d1f/d45/d52/d2d 36852 1 2024-07-16T09:55:42.324 INFO:tasks.workunit.client.0.smithi156.stdout:1/781: rmdir dc/d14/d3e/d6c/d7a/dcb 39 2024-07-16T09:55:42.324 INFO:tasks.workunit.client.0.smithi156.stdout:5/610: truncate d1b/f55 1746761 0 2024-07-16T09:55:42.324 INFO:tasks.workunit.client.0.smithi156.stdout:7/456: rmdir dc/d3e 39 2024-07-16T09:55:42.325 INFO:tasks.workunit.client.1.smithi176.stdout:2/632: dread - dd/d9b/fab zero size 2024-07-16T09:55:42.326 INFO:tasks.workunit.client.0.smithi156.stdout:9/614: getdents d2/d5/d16/d22/d2b/d3d 0 2024-07-16T09:55:42.330 INFO:tasks.workunit.client.1.smithi176.stdout:9/666: fdatasync d5/d1d/dc7/f5c 0 2024-07-16T09:55:42.330 INFO:tasks.workunit.client.1.smithi176.stdout:0/691: rename d5/d1c/d24/d35/d50/fe9 to d5/d39/dde/fea 0 2024-07-16T09:55:42.330 INFO:tasks.workunit.client.1.smithi176.stdout:9/667: write d5/d1d/d20/f74 [589528,76976] 0 2024-07-16T09:55:42.330 INFO:tasks.workunit.client.0.smithi156.stdout:3/541: write d8/da/d28/f60 [813379,51451] 0 2024-07-16T09:55:42.331 INFO:tasks.workunit.client.0.smithi156.stdout:4/639: chown d4/d23/d52/d71/ce6 2940095 1 2024-07-16T09:55:42.331 INFO:tasks.workunit.client.0.smithi156.stdout:1/782: mknod dc/d6b/dd5/dfb/c10a 0 2024-07-16T09:55:42.333 INFO:tasks.workunit.client.1.smithi176.stdout:7/650: dread d4/f11 [4194304,4194304] 0 2024-07-16T09:55:42.333 INFO:tasks.workunit.client.1.smithi176.stdout:9/668: stat d5/d1d/dd2/f8c 0 2024-07-16T09:55:42.333 INFO:tasks.workunit.client.1.smithi176.stdout:0/692: write d5/d1c/d24/d35/d50/d58/f6f [1450765,52335] 0 2024-07-16T09:55:42.333 INFO:tasks.workunit.client.0.smithi156.stdout:9/615: write d2/d5/d16/d22/d2b/d3d/d3c/d7f/fb9 [918169,2429] 0 2024-07-16T09:55:42.335 INFO:tasks.workunit.client.0.smithi156.stdout:5/611: write f14 [2634671,21961] 0 2024-07-16T09:55:42.336 INFO:tasks.workunit.client.0.smithi156.stdout:4/640: chown d4/db/dc/d7f/fdc 6610779 1 2024-07-16T09:55:42.336 INFO:tasks.workunit.client.0.smithi156.stdout:7/457: creat dc/d13/d20/f99 x:0 0 0 2024-07-16T09:55:42.337 INFO:tasks.workunit.client.0.smithi156.stdout:8/429: mkdir d0/d11/d13/d1c/d9a 0 2024-07-16T09:55:42.338 INFO:tasks.workunit.client.0.smithi156.stdout:1/783: write dc/d14/d1e/f3d [3630757,126698] 0 2024-07-16T09:55:42.339 INFO:tasks.workunit.client.0.smithi156.stdout:6/537: dwrite d3/d13/d17/d4f/d1e/f98 [0,4194304] 0 2024-07-16T09:55:42.339 INFO:tasks.workunit.client.0.smithi156.stdout:5/612: write fc [3776030,15940] 0 2024-07-16T09:55:42.339 INFO:tasks.workunit.client.1.smithi176.stdout:0/693: dread - d5/d1c/d24/d35/fc8 zero size 2024-07-16T09:55:42.342 INFO:tasks.workunit.client.1.smithi176.stdout:1/442: getdents d4/d3a/d54/d5a/d7a 0 2024-07-16T09:55:42.343 INFO:tasks.workunit.client.1.smithi176.stdout:8/573: dwrite d8/d14/d1e/f1f [0,4194304] 0 2024-07-16T09:55:42.343 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:55:41 smithi176 ceph-mon[36626]: pgmap v252: 65 pgs: 65 active+clean; 2.1 GiB data, 7.7 GiB used, 529 GiB / 536 GiB avail; 61 MiB/s rd, 280 MiB/s wr, 528 op/s 2024-07-16T09:55:42.344 INFO:tasks.workunit.client.1.smithi176.stdout:9/669: creat d5/d1d/d67/d6d/fe4 x:0 0 0 2024-07-16T09:55:42.348 INFO:tasks.workunit.client.0.smithi156.stdout:1/784: fsync dc/d14/d8b/dcd/dd2/feb 0 2024-07-16T09:55:42.349 INFO:tasks.workunit.client.0.smithi156.stdout:7/458: fdatasync dc/d3e/d3f/d51/d6c/f85 0 2024-07-16T09:55:42.351 INFO:tasks.workunit.client.0.smithi156.stdout:0/557: creat d0/d1a/d4d/d9/d27/d3d/faf x:0 0 0 2024-07-16T09:55:42.352 INFO:tasks.workunit.client.0.smithi156.stdout:3/542: dread d8/da/d28/d43/f5b [0,4194304] 0 2024-07-16T09:55:42.352 INFO:tasks.workunit.client.0.smithi156.stdout:6/538: truncate d3/d13/d17/d4f/d5c/f65 439297 0 2024-07-16T09:55:42.356 INFO:tasks.workunit.client.1.smithi176.stdout:2/633: dread dd/d16/f33 [4194304,4194304] 0 2024-07-16T09:55:42.357 INFO:tasks.workunit.client.1.smithi176.stdout:9/670: link d5/d1d/l79 d5/de/da5/dbf/le5 0 2024-07-16T09:55:42.359 INFO:tasks.workunit.client.0.smithi156.stdout:1/785: creat dc/d6b/d96/f10b x:0 0 0 2024-07-16T09:55:42.359 INFO:tasks.workunit.client.0.smithi156.stdout:4/641: dread d4/f79 [0,4194304] 0 2024-07-16T09:55:42.360 INFO:tasks.workunit.client.0.smithi156.stdout:8/430: dwrite d0/dd/f3b [0,4194304] 0 2024-07-16T09:55:42.362 INFO:tasks.workunit.client.1.smithi176.stdout:9/671: creat d5/d1d/d77/d97/fe6 x:0 0 0 2024-07-16T09:55:42.364 INFO:tasks.workunit.client.0.smithi156.stdout:6/539: creat d3/d13/d17/d4f/d30/d45/fb0 x:0 0 0 2024-07-16T09:55:42.364 INFO:tasks.workunit.client.1.smithi176.stdout:7/651: dread d4/dd/f2e [0,4194304] 0 2024-07-16T09:55:42.365 INFO:tasks.workunit.client.1.smithi176.stdout:2/634: dread dd/de/f10 [0,4194304] 0 2024-07-16T09:55:42.365 INFO:tasks.workunit.client.0.smithi156.stdout:6/540: dread - d3/d13/d17/d4f/d94/d19/f5b zero size 2024-07-16T09:55:42.366 INFO:tasks.workunit.client.1.smithi176.stdout:9/672: mknod d5/de/d21/ce7 0 2024-07-16T09:55:42.367 INFO:tasks.workunit.client.1.smithi176.stdout:2/635: fdatasync dd/d16/f71 0 2024-07-16T09:55:42.368 INFO:tasks.workunit.client.0.smithi156.stdout:8/431: unlink d0/dd/d1f/l3d 0 2024-07-16T09:55:42.369 INFO:tasks.workunit.client.0.smithi156.stdout:5/613: dread d1b/d1f/d45/d52/d49/f81 [0,4194304] 0 2024-07-16T09:55:42.369 INFO:tasks.workunit.client.0.smithi156.stdout:8/432: stat d0/d11/d2e/l77 0 2024-07-16T09:55:42.371 INFO:tasks.workunit.client.0.smithi156.stdout:5/614: readlink d1b/d1f/d45/d52/d2d/d96/lbf 0 2024-07-16T09:55:42.371 INFO:tasks.workunit.client.0.smithi156.stdout:7/459: truncate dc/d48/f4b 3603053 0 2024-07-16T09:55:42.372 INFO:tasks.workunit.client.1.smithi176.stdout:8/574: dread d8/d23/d2c/f36 [0,4194304] 0 2024-07-16T09:55:42.372 INFO:tasks.workunit.client.1.smithi176.stdout:2/636: mkdir dd/d36/d45/dd3 0 2024-07-16T09:55:42.372 INFO:tasks.workunit.client.0.smithi156.stdout:0/558: link d0/d1a/c77 d0/d1a/d4d/d9/d27/d50/cb0 0 2024-07-16T09:55:42.373 INFO:tasks.workunit.client.0.smithi156.stdout:3/543: dwrite d8/da/d21/d80/fa8 [0,4194304] 0 2024-07-16T09:55:42.373 INFO:tasks.workunit.client.0.smithi156.stdout:7/460: chown dc/d3e/d3f/c70 241 1 2024-07-16T09:55:42.374 INFO:tasks.workunit.client.1.smithi176.stdout:3/497: dread d3/da/d33/f52 [0,4194304] 0 2024-07-16T09:55:42.374 INFO:tasks.workunit.client.0.smithi156.stdout:7/461: write dc/d3e/d3f/d51/d6c/f85 [203429,89972] 0 2024-07-16T09:55:42.376 INFO:tasks.workunit.client.0.smithi156.stdout:8/433: fsync d0/dd/f4c 0 2024-07-16T09:55:42.377 INFO:tasks.workunit.client.1.smithi176.stdout:8/575: chown d8/d23/d76/c89 1322 1 2024-07-16T09:55:42.377 INFO:tasks.workunit.client.1.smithi176.stdout:3/498: readlink d3/d17/d1d/d4f/d59/d81/l8d 0 2024-07-16T09:55:42.379 INFO:tasks.workunit.client.0.smithi156.stdout:5/615: unlink d1b/d1f/d45/d52/fb0 0 2024-07-16T09:55:42.380 INFO:tasks.workunit.client.1.smithi176.stdout:9/673: dwrite d5/d1d/d67/f6a [0,4194304] 0 2024-07-16T09:55:42.383 INFO:tasks.workunit.client.0.smithi156.stdout:3/544: creat d8/da/d1d/d2c/d33/fbc x:0 0 0 2024-07-16T09:55:42.384 INFO:tasks.workunit.client.0.smithi156.stdout:3/545: chown d8/da/d28/f5e 357 1 2024-07-16T09:55:42.384 INFO:tasks.workunit.client.0.smithi156.stdout:7/462: creat dc/d3e/d3f/d45/f9a x:0 0 0 2024-07-16T09:55:42.385 INFO:tasks.workunit.client.0.smithi156.stdout:7/463: chown dc/d3e/d3f/d51/d8a 12 1 2024-07-16T09:55:42.386 INFO:tasks.workunit.client.0.smithi156.stdout:9/616: truncate d2/d5/d16/d22/d36/d33/d23/f60 1980034 0 2024-07-16T09:55:42.389 INFO:tasks.workunit.client.0.smithi156.stdout:5/616: creat d1b/d1f/d42/fd6 x:0 0 0 2024-07-16T09:55:42.390 INFO:tasks.workunit.client.0.smithi156.stdout:3/546: creat d8/da/d28/d43/d63/fbd x:0 0 0 2024-07-16T09:55:42.390 INFO:tasks.workunit.client.0.smithi156.stdout:8/434: mknod d0/d11/d4a/d55/d93/c9b 0 2024-07-16T09:55:42.392 INFO:tasks.workunit.client.0.smithi156.stdout:9/617: fdatasync d2/d5/d16/d22/d36/d33/d23/f88 0 2024-07-16T09:55:42.393 INFO:tasks.workunit.client.0.smithi156.stdout:8/435: write d0/d11/d29/f79 [906174,83349] 0 2024-07-16T09:55:42.393 INFO:tasks.workunit.client.0.smithi156.stdout:9/618: chown d2/l10 33 1 2024-07-16T09:55:42.395 INFO:tasks.workunit.client.0.smithi156.stdout:2/646: sync 2024-07-16T09:55:42.395 INFO:tasks.workunit.client.0.smithi156.stdout:3/547: fsync d8/da/d13/f57 0 2024-07-16T09:55:42.397 INFO:tasks.workunit.client.0.smithi156.stdout:7/464: dwrite dc/d3e/d3f/d45/f9a [0,4194304] 0 2024-07-16T09:55:42.399 INFO:tasks.workunit.client.1.smithi176.stdout:8/576: dread d8/d23/f25 [0,4194304] 0 2024-07-16T09:55:42.400 INFO:tasks.workunit.client.0.smithi156.stdout:9/619: read d2/d5/d16/d22/d36/d33/f2c [3356653,88410] 0 2024-07-16T09:55:42.402 INFO:tasks.workunit.client.0.smithi156.stdout:8/436: creat d0/d11/d4a/d55/d93/f9c x:0 0 0 2024-07-16T09:55:42.404 INFO:tasks.workunit.client.0.smithi156.stdout:3/548: creat d8/da/d28/d43/fbe x:0 0 0 2024-07-16T09:55:42.405 INFO:tasks.workunit.client.0.smithi156.stdout:2/647: dread d0/d13/d17/f34 [4194304,4194304] 0 2024-07-16T09:55:42.407 INFO:tasks.workunit.client.0.smithi156.stdout:5/617: dread d1b/d1f/d45/f50 [0,4194304] 0 2024-07-16T09:55:42.410 INFO:tasks.workunit.client.0.smithi156.stdout:2/648: write d0/d5/d29/d2d/d5b/d62/d74/fb4 [454266,39343] 0 2024-07-16T09:55:42.410 INFO:tasks.workunit.client.0.smithi156.stdout:0/559: dread d0/ff [0,4194304] 0 2024-07-16T09:55:42.410 INFO:tasks.workunit.client.0.smithi156.stdout:9/620: dwrite d2/d5/d16/d22/d36/d33/f82 [0,4194304] 0 2024-07-16T09:55:42.412 INFO:tasks.workunit.client.0.smithi156.stdout:8/437: creat d0/d11/d29/d80/f9d x:0 0 0 2024-07-16T09:55:42.416 INFO:tasks.workunit.client.0.smithi156.stdout:9/621: write d2/d5/d16/d22/d2b/d3d/d3c/d7f/d42/f85 [1092330,129533] 0 2024-07-16T09:55:42.416 INFO:tasks.workunit.client.0.smithi156.stdout:0/560: write d0/d1a/d76/f87 [328735,6338] 0 2024-07-16T09:55:42.416 INFO:tasks.workunit.client.0.smithi156.stdout:3/549: creat d8/da/d1d/d2c/d33/fbf x:0 0 0 2024-07-16T09:55:42.417 INFO:tasks.workunit.client.0.smithi156.stdout:5/618: mknod d1b/d1f/d45/d52/d43/d85/cd7 0 2024-07-16T09:55:42.418 INFO:tasks.workunit.client.0.smithi156.stdout:2/649: write d0/d5/d29/d2d/d5b/f9c [1508583,106932] 0 2024-07-16T09:55:42.421 INFO:tasks.workunit.client.0.smithi156.stdout:9/622: symlink d2/d5/d16/d22/d36/lc1 0 2024-07-16T09:55:42.423 INFO:tasks.workunit.client.0.smithi156.stdout:9/623: stat d2/d5/d16/d22/d6b 0 2024-07-16T09:55:42.425 INFO:tasks.workunit.client.0.smithi156.stdout:0/561: dwrite d0/d1a/d4d/d9/d27/d8d/fa4 [0,4194304] 0 2024-07-16T09:55:42.426 INFO:tasks.workunit.client.0.smithi156.stdout:7/465: write dc/d3e/d3f/d51/f5e [5752793,64056] 0 2024-07-16T09:55:42.428 INFO:tasks.workunit.client.0.smithi156.stdout:9/624: mkdir d2/d5/d16/d22/d2b/d74/dc2 0 2024-07-16T09:55:42.430 INFO:tasks.workunit.client.0.smithi156.stdout:0/562: symlink d0/d1a/d4d/d9/d27/d8d/lb1 0 2024-07-16T09:55:42.431 INFO:tasks.workunit.client.0.smithi156.stdout:7/466: symlink dc/d13/d15/l9b 0 2024-07-16T09:55:42.433 INFO:tasks.workunit.client.0.smithi156.stdout:9/625: fsync d2/d5/d16/d22/d36/d33/d3a/f3b 0 2024-07-16T09:55:42.435 INFO:tasks.workunit.client.0.smithi156.stdout:7/467: write dc/d13/d36/f66 [1249431,14122] 0 2024-07-16T09:55:42.435 INFO:tasks.workunit.client.0.smithi156.stdout:8/438: dwrite d0/d11/d13/d1c/d24/f42 [0,4194304] 0 2024-07-16T09:55:42.436 INFO:tasks.workunit.client.0.smithi156.stdout:0/563: mknod d0/d1a/d4d/d61/da1/cb2 0 2024-07-16T09:55:42.438 INFO:tasks.workunit.client.0.smithi156.stdout:9/626: chown d2/d5/d16/d22/l6c 76969698 1 2024-07-16T09:55:42.439 INFO:tasks.workunit.client.1.smithi176.stdout:2/637: creat dd/fd4 x:0 0 0 2024-07-16T09:55:42.441 INFO:tasks.workunit.client.1.smithi176.stdout:2/638: chown dd/d49/d52/f79 256023951 1 2024-07-16T09:55:42.442 INFO:tasks.workunit.client.0.smithi156.stdout:0/564: creat d0/d1a/d4d/d9/d27/d2c/d6d/fb3 x:0 0 0 2024-07-16T09:55:42.442 INFO:tasks.workunit.client.1.smithi176.stdout:8/577: mknod d8/d23/d2c/d34/d62/cc5 0 2024-07-16T09:55:42.443 INFO:tasks.workunit.client.0.smithi156.stdout:8/439: unlink d0/d11/d29/d5e/c94 0 2024-07-16T09:55:42.444 INFO:tasks.workunit.client.0.smithi156.stdout:7/468: rename dc/d3e/d3f/d45/l83 to dc/d3e/d3f/d45/l9c 0 2024-07-16T09:55:42.444 INFO:tasks.workunit.client.0.smithi156.stdout:3/550: dread f3 [0,4194304] 0 2024-07-16T09:55:42.445 INFO:tasks.workunit.client.1.smithi176.stdout:9/674: symlink d5/de/le8 0 2024-07-16T09:55:42.446 INFO:tasks.workunit.client.1.smithi176.stdout:2/639: stat dd/d2e/d35/lca 0 2024-07-16T09:55:42.447 INFO:tasks.workunit.client.0.smithi156.stdout:8/440: creat d0/d11/d20/d7f/f9e x:0 0 0 2024-07-16T09:55:42.448 INFO:tasks.workunit.client.1.smithi176.stdout:9/675: creat d5/de/d30/fe9 x:0 0 0 2024-07-16T09:55:42.448 INFO:tasks.workunit.client.0.smithi156.stdout:7/469: mknod dc/d13/d15/d8d/c9d 0 2024-07-16T09:55:42.449 INFO:tasks.workunit.client.1.smithi176.stdout:3/499: getdents d3/da/d66 0 2024-07-16T09:55:42.450 INFO:tasks.workunit.client.0.smithi156.stdout:9/627: rename d2/d5/d16/d22/d36/ca5 to d2/d5/d16/d22/d36/d33/cc3 0 2024-07-16T09:55:42.451 INFO:tasks.workunit.client.1.smithi176.stdout:2/640: creat dd/d16/d39/d3d/d7e/fd5 x:0 0 0 2024-07-16T09:55:42.451 INFO:tasks.workunit.client.0.smithi156.stdout:8/441: chown d0/dd/l66 12600 1 2024-07-16T09:55:42.451 INFO:tasks.workunit.client.0.smithi156.stdout:9/628: write d2/d5/d61/da2/d98/fbd [573292,91080] 0 2024-07-16T09:55:42.452 INFO:tasks.workunit.client.1.smithi176.stdout:8/578: link d8/d14/d1e/d33/d9d/lae d8/d23/d2c/d34/d62/lc6 0 2024-07-16T09:55:42.452 INFO:tasks.workunit.client.1.smithi176.stdout:2/641: fsync dd/d2e/f50 0 2024-07-16T09:55:42.453 INFO:tasks.workunit.client.0.smithi156.stdout:0/565: rename d0/d1a/d4d/f6b to d0/d22/d41/d52/fb4 0 2024-07-16T09:55:42.454 INFO:tasks.workunit.client.1.smithi176.stdout:9/676: creat d5/d1d/d20/d78/dd0/fea x:0 0 0 2024-07-16T09:55:42.454 INFO:tasks.workunit.client.0.smithi156.stdout:8/442: symlink d0/d11/d20/d3f/l9f 0 2024-07-16T09:55:42.455 INFO:tasks.workunit.client.0.smithi156.stdout:8/443: dread - d0/d11/d29/d80/f9d zero size 2024-07-16T09:55:42.456 INFO:tasks.workunit.client.1.smithi176.stdout:9/677: write d5/fb8 [811727,121637] 0 2024-07-16T09:55:42.456 INFO:tasks.workunit.client.0.smithi156.stdout:9/629: chown d2/d5/d16/d22/d2b/d3d/d3c/d7f/c75 45996926 1 2024-07-16T09:55:42.457 INFO:tasks.workunit.client.0.smithi156.stdout:7/470: rename dc/d3e/d3f/d51/l82 to dc/d3e/d3f/d45/l9e 0 2024-07-16T09:55:42.458 INFO:tasks.workunit.client.1.smithi176.stdout:2/642: rename dd/d2e/d35/f47 to dd/fd6 0 2024-07-16T09:55:42.458 INFO:tasks.workunit.client.1.smithi176.stdout:9/678: dread - d5/d1d/db0/fde zero size 2024-07-16T09:55:42.460 INFO:tasks.workunit.client.1.smithi176.stdout:2/643: readlink dd/de/l2c 0 2024-07-16T09:55:42.461 INFO:tasks.workunit.client.1.smithi176.stdout:9/679: symlink d5/d52/leb 0 2024-07-16T09:55:42.462 INFO:tasks.workunit.client.1.smithi176.stdout:8/579: dwrite d8/d20/d2a/db9/fc2 [0,4194304] 0 2024-07-16T09:55:42.464 INFO:tasks.workunit.client.1.smithi176.stdout:8/580: dread - d8/d23/d76/f98 zero size 2024-07-16T09:55:42.468 INFO:tasks.workunit.client.1.smithi176.stdout:9/680: unlink d5/d1d/l79 0 2024-07-16T09:55:42.469 INFO:tasks.workunit.client.1.smithi176.stdout:2/644: dread dd/d2e/d35/d4a/fce [0,4194304] 0 2024-07-16T09:55:42.470 INFO:tasks.workunit.client.1.smithi176.stdout:2/645: stat dd/d36/l26 0 2024-07-16T09:55:42.470 INFO:tasks.workunit.client.0.smithi156.stdout:8/444: creat d0/d11/d13/d1c/d39/fa0 x:0 0 0 2024-07-16T09:55:42.472 INFO:tasks.workunit.client.0.smithi156.stdout:8/445: chown d0/d11/d13 8031 1 2024-07-16T09:55:42.472 INFO:tasks.workunit.client.0.smithi156.stdout:9/630: rename d2/d5/d16/d22/d36/d33 to d2/d5/d16/d22/d2b/d74/dc2/dc4 0 2024-07-16T09:55:42.473 INFO:tasks.workunit.client.1.smithi176.stdout:9/681: symlink d5/d1d/dd2/de3/lec 0 2024-07-16T09:55:42.474 INFO:tasks.workunit.client.0.smithi156.stdout:7/471: dwrite dc/d13/d36/d56/f61 [0,4194304] 0 2024-07-16T09:55:42.475 INFO:tasks.workunit.client.1.smithi176.stdout:8/581: rename d8/f17 to d8/d20/d2a/d31/d46/fc7 0 2024-07-16T09:55:42.477 INFO:tasks.workunit.client.1.smithi176.stdout:9/682: mknod d5/d1d/d77/d98/ced 0 2024-07-16T09:55:42.477 INFO:tasks.workunit.client.1.smithi176.stdout:2/646: dread dd/d9b/fb1 [0,4194304] 0 2024-07-16T09:55:42.478 INFO:tasks.workunit.client.0.smithi156.stdout:7/472: mknod dc/d13/d15/d8d/c9f 0 2024-07-16T09:55:42.478 INFO:tasks.workunit.client.1.smithi176.stdout:8/582: dread - d8/d20/d2a/d53/d93/fbb zero size 2024-07-16T09:55:42.479 INFO:tasks.workunit.client.0.smithi156.stdout:9/631: rename d2/l21 to d2/lc5 0 2024-07-16T09:55:42.480 INFO:tasks.workunit.client.1.smithi176.stdout:2/647: write dd/d49/d52/d7c/fcc [684926,108596] 0 2024-07-16T09:55:42.481 INFO:tasks.workunit.client.1.smithi176.stdout:8/583: creat d8/d20/d2a/d31/db4/fc8 x:0 0 0 2024-07-16T09:55:42.481 INFO:tasks.workunit.client.0.smithi156.stdout:0/566: dwrite d0/d22/f9e [0,4194304] 0 2024-07-16T09:55:42.482 INFO:tasks.workunit.client.1.smithi176.stdout:9/683: creat d5/d1d/fee x:0 0 0 2024-07-16T09:55:42.483 INFO:tasks.workunit.client.0.smithi156.stdout:0/567: dread - d0/d1a/d4d/d9/d27/f81 zero size 2024-07-16T09:55:42.483 INFO:tasks.workunit.client.0.smithi156.stdout:8/446: rename d0/d74/f70 to d0/d11/d13/fa1 0 2024-07-16T09:55:42.484 INFO:tasks.workunit.client.0.smithi156.stdout:7/473: creat dc/d3e/d3f/d51/fa0 x:0 0 0 2024-07-16T09:55:42.484 INFO:tasks.workunit.client.0.smithi156.stdout:0/568: readlink d0/d1a/d4d/d25/l29 0 2024-07-16T09:55:42.488 INFO:tasks.workunit.client.0.smithi156.stdout:9/632: creat d2/d5/d61/da2/d98/fc6 x:0 0 0 2024-07-16T09:55:42.488 INFO:tasks.workunit.client.0.smithi156.stdout:7/474: symlink dc/d44/d81/la1 0 2024-07-16T09:55:42.489 INFO:tasks.workunit.client.0.smithi156.stdout:0/569: rename d0/d1a/d4d/d9/d27 to d0/d1a/d4d/d9/d27/d3d/db5 22 2024-07-16T09:55:42.490 INFO:tasks.workunit.client.0.smithi156.stdout:8/447: truncate d0/d11/d13/d1c/d39/f73 5403429 0 2024-07-16T09:55:42.493 INFO:tasks.workunit.client.0.smithi156.stdout:7/475: truncate dc/f69 261786 0 2024-07-16T09:55:42.494 INFO:tasks.workunit.client.0.smithi156.stdout:7/476: fdatasync dc/d13/f43 0 2024-07-16T09:55:42.495 INFO:tasks.workunit.client.1.smithi176.stdout:2/648: getdents dd/d2e/d35/d59/d66 0 2024-07-16T09:55:42.495 INFO:tasks.workunit.client.1.smithi176.stdout:9/684: getdents d5/d1d/db0 0 2024-07-16T09:55:42.496 INFO:tasks.workunit.client.0.smithi156.stdout:0/570: creat d0/d1a/d4d/d9/d27/d8d/fb6 x:0 0 0 2024-07-16T09:55:42.496 INFO:tasks.workunit.client.1.smithi176.stdout:9/685: dread - d5/d1d/d77/d98/fda zero size 2024-07-16T09:55:42.498 INFO:tasks.workunit.client.1.smithi176.stdout:9/686: write d5/d1d/d20/f7d [1556980,105346] 0 2024-07-16T09:55:42.499 INFO:tasks.workunit.client.1.smithi176.stdout:2/649: rename dd/de/l73 to dd/d36/d82/ld7 0 2024-07-16T09:55:42.499 INFO:tasks.workunit.client.0.smithi156.stdout:8/448: dwrite d0/d11/d29/d80/f88 [0,4194304] 0 2024-07-16T09:55:42.500 INFO:tasks.workunit.client.1.smithi176.stdout:9/687: write d5/d1d/d77/d98/fda [1012480,24869] 0 2024-07-16T09:55:42.502 INFO:tasks.workunit.client.1.smithi176.stdout:9/688: fsync d5/de/d21/f39 0 2024-07-16T09:55:42.502 INFO:tasks.workunit.client.0.smithi156.stdout:0/571: mkdir d0/d1a/d4d/d5e/db7 0 2024-07-16T09:55:42.503 INFO:tasks.workunit.client.1.smithi176.stdout:9/689: fdatasync d5/d43/f9e 0 2024-07-16T09:55:42.503 INFO:tasks.workunit.client.1.smithi176.stdout:2/650: mknod dd/d49/d52/cd8 0 2024-07-16T09:55:42.503 INFO:tasks.workunit.client.1.smithi176.stdout:9/690: fsync d5/d1d/d77/d98/fb1 0 2024-07-16T09:55:42.505 INFO:tasks.workunit.client.0.smithi156.stdout:7/477: dwrite dc/d13/d36/d56/f61 [0,4194304] 0 2024-07-16T09:55:42.506 INFO:tasks.workunit.client.1.smithi176.stdout:9/691: write d5/d1d/d77/fd9 [814725,55866] 0 2024-07-16T09:55:42.508 INFO:tasks.workunit.client.1.smithi176.stdout:9/692: fdatasync d5/d1d/d77/d97/fa6 0 2024-07-16T09:55:42.509 INFO:tasks.workunit.client.0.smithi156.stdout:0/572: truncate d0/d1a/d4d/d9/d27/d8d/fb6 784477 0 2024-07-16T09:55:42.511 INFO:tasks.workunit.client.0.smithi156.stdout:8/449: mknod d0/dd/d1f/d50/d8d/ca2 0 2024-07-16T09:55:42.513 INFO:tasks.workunit.client.1.smithi176.stdout:8/584: dwrite d8/d20/d2a/d53/d93/fbb [0,4194304] 0 2024-07-16T09:55:42.513 INFO:tasks.workunit.client.1.smithi176.stdout:2/651: creat dd/d2e/d35/d4a/fd9 x:0 0 0 2024-07-16T09:55:42.514 INFO:tasks.workunit.client.0.smithi156.stdout:7/478: creat dc/d13/d36/d56/fa2 x:0 0 0 2024-07-16T09:55:42.515 INFO:tasks.workunit.client.1.smithi176.stdout:8/585: chown d8/d14/d1e/d33/d9d/fa7 66 1 2024-07-16T09:55:42.515 INFO:tasks.workunit.client.1.smithi176.stdout:9/693: link d5/de/f8b d5/d1d/dc7/d89/fef 0 2024-07-16T09:55:42.516 INFO:tasks.workunit.client.1.smithi176.stdout:2/652: stat dd/d16/d39/c44 0 2024-07-16T09:55:42.517 INFO:tasks.workunit.client.0.smithi156.stdout:0/573: creat d0/d1a/d4d/d25/fb8 x:0 0 0 2024-07-16T09:55:42.517 INFO:tasks.workunit.client.1.smithi176.stdout:2/653: stat dd/d49/l53 0 2024-07-16T09:55:42.519 INFO:tasks.workunit.client.1.smithi176.stdout:2/654: write dd/d49/fb5 [319793,42492] 0 2024-07-16T09:55:42.519 INFO:tasks.workunit.client.1.smithi176.stdout:9/694: unlink d5/d1d/d67/d6d/fe4 0 2024-07-16T09:55:42.521 INFO:tasks.workunit.client.0.smithi156.stdout:7/479: creat dc/d48/fa3 x:0 0 0 2024-07-16T09:55:42.521 INFO:tasks.workunit.client.0.smithi156.stdout:0/574: mknod d0/d22/d41/d66/cb9 0 2024-07-16T09:55:42.522 INFO:tasks.workunit.client.0.smithi156.stdout:7/480: dread - dc/d63/f6b zero size 2024-07-16T09:55:42.523 INFO:tasks.workunit.client.0.smithi156.stdout:0/575: fdatasync d0/d22/d41/f47 0 2024-07-16T09:55:42.525 INFO:tasks.workunit.client.0.smithi156.stdout:0/576: readlink d0/d1a/d4d/d9/d68/l9f 0 2024-07-16T09:55:42.528 INFO:tasks.workunit.client.1.smithi176.stdout:8/586: mknod d8/d23/cc9 0 2024-07-16T09:55:42.528 INFO:tasks.workunit.client.1.smithi176.stdout:9/695: chown d5/de/d30/lb5 439305781 1 2024-07-16T09:55:42.530 INFO:tasks.workunit.client.1.smithi176.stdout:8/587: chown d8/d20/d2a/d31/d46/d6a/l70 0 1 2024-07-16T09:55:42.531 INFO:tasks.workunit.client.0.smithi156.stdout:9/633: dread d2/d5/f55 [4194304,4194304] 0 2024-07-16T09:55:42.532 INFO:tasks.workunit.client.0.smithi156.stdout:0/577: getdents d0/d1a/d4d/d9/d27/d3d 0 2024-07-16T09:55:42.532 INFO:tasks.workunit.client.0.smithi156.stdout:7/481: dwrite dc/d3e/d3f/d51/fa0 [0,4194304] 0 2024-07-16T09:55:42.534 INFO:tasks.workunit.client.0.smithi156.stdout:0/578: chown d0/d1a/d4d/d6e/d86 365258 1 2024-07-16T09:55:42.534 INFO:tasks.workunit.client.0.smithi156.stdout:7/482: chown dc/d48/c97 64680 1 2024-07-16T09:55:42.535 INFO:tasks.workunit.client.0.smithi156.stdout:7/483: chown dc/d3e/d3f 51395069 1 2024-07-16T09:55:42.536 INFO:tasks.workunit.client.0.smithi156.stdout:0/579: write d0/d1a/d4d/d9/d27/d3d/d62/f8e [1476122,113342] 0 2024-07-16T09:55:42.538 INFO:tasks.workunit.client.0.smithi156.stdout:9/634: symlink d2/d5/d16/d22/d2b/d3d/d3c/lc7 0 2024-07-16T09:55:42.540 INFO:tasks.workunit.client.0.smithi156.stdout:9/635: chown d2/d5/d16/d22/d2b/d3d/da6 155523 1 2024-07-16T09:55:42.540 INFO:tasks.workunit.client.0.smithi156.stdout:0/580: read d0/d1a/f54 [502513,316] 0 2024-07-16T09:55:42.542 INFO:tasks.workunit.client.1.smithi176.stdout:2/655: mknod dd/d16/d39/d3d/dc2/cda 0 2024-07-16T09:55:42.543 INFO:tasks.workunit.client.1.smithi176.stdout:9/696: unlink d5/de/d21/f96 0 2024-07-16T09:55:42.545 INFO:tasks.workunit.client.1.smithi176.stdout:8/588: rmdir d8/d20/d2a/d31/d2f/db0 0 2024-07-16T09:55:42.546 INFO:tasks.workunit.client.0.smithi156.stdout:0/581: write d0/d22/d41/f4a [2711854,120742] 0 2024-07-16T09:55:42.546 INFO:tasks.workunit.client.0.smithi156.stdout:7/484: dwrite dc/d48/fa3 [0,4194304] 0 2024-07-16T09:55:42.547 INFO:tasks.workunit.client.0.smithi156.stdout:8/450: truncate d0/dd/f3b 1788342 0 2024-07-16T09:55:42.547 INFO:tasks.workunit.client.1.smithi176.stdout:2/656: rename dd/de/c13 to dd/d9b/cdb 0 2024-07-16T09:55:42.548 INFO:tasks.workunit.client.1.smithi176.stdout:8/589: truncate f5 3862688 0 2024-07-16T09:55:42.549 INFO:tasks.workunit.client.0.smithi156.stdout:7/485: symlink dc/d48/la4 0 2024-07-16T09:55:42.549 INFO:tasks.workunit.client.1.smithi176.stdout:2/657: chown dd/d36/l8c 339463636 1 2024-07-16T09:55:42.550 INFO:tasks.workunit.client.1.smithi176.stdout:2/658: read - dd/d16/d39/fc9 zero size 2024-07-16T09:55:42.550 INFO:tasks.workunit.client.1.smithi176.stdout:9/697: rename d5/d43/f69 to d5/d52/ff0 0 2024-07-16T09:55:42.551 INFO:tasks.workunit.client.0.smithi156.stdout:8/451: mkdir d0/dd/da3 0 2024-07-16T09:55:42.553 INFO:tasks.workunit.client.0.smithi156.stdout:7/486: unlink dc/d13/d36/l62 0 2024-07-16T09:55:42.554 INFO:tasks.workunit.client.1.smithi176.stdout:9/698: creat d5/d52/ff1 x:0 0 0 2024-07-16T09:55:42.554 INFO:tasks.workunit.client.1.smithi176.stdout:2/659: creat dd/d16/d39/d3d/d7e/fdc x:0 0 0 2024-07-16T09:55:42.554 INFO:tasks.workunit.client.0.smithi156.stdout:0/582: dwrite d0/d1a/d4d/d9/d27/d8d/fb6 [0,4194304] 0 2024-07-16T09:55:42.555 INFO:tasks.workunit.client.0.smithi156.stdout:9/636: creat d2/d5/d16/d22/d2b/d74/dc2/dc4/fc8 x:0 0 0 2024-07-16T09:55:42.559 INFO:tasks.workunit.client.1.smithi176.stdout:8/590: rename d8/d20/d2a/d61/l73 to d8/d23/d2c/d34/d62/d88/lca 0 2024-07-16T09:55:42.559 INFO:tasks.workunit.client.1.smithi176.stdout:2/660: symlink dd/d36/d45/ldd 0 2024-07-16T09:55:42.560 INFO:tasks.workunit.client.0.smithi156.stdout:0/583: symlink d0/d1a/d4d/d9/d27/d50/d90/lba 0 2024-07-16T09:55:42.560 INFO:tasks.workunit.client.0.smithi156.stdout:8/452: getdents d0/d11/d29/d80/d97 0 2024-07-16T09:55:42.561 INFO:tasks.workunit.client.1.smithi176.stdout:9/699: mkdir d5/d1d/d20/d78/df2 0 2024-07-16T09:55:42.564 INFO:tasks.workunit.client.0.smithi156.stdout:7/487: dwrite dc/d44/f5c [4194304,4194304] 0 2024-07-16T09:55:42.564 INFO:tasks.workunit.client.1.smithi176.stdout:9/700: rmdir d5/d1d/d20/d78 39 2024-07-16T09:55:42.565 INFO:tasks.workunit.client.0.smithi156.stdout:0/584: fsync d0/d1a/d4d/d9/d27/d8d/fa4 0 2024-07-16T09:55:42.568 INFO:tasks.workunit.client.1.smithi176.stdout:8/591: dwrite d8/d23/d2c/d34/d62/fb1 [0,4194304] 0 2024-07-16T09:55:42.570 INFO:tasks.workunit.client.0.smithi156.stdout:9/637: dwrite d2/d5/d16/d22/d2b/d74/fbe [0,4194304] 0 2024-07-16T09:55:42.578 INFO:tasks.workunit.client.0.smithi156.stdout:0/585: mkdir d0/d1a/d4d/d9/d27/d2c/d6d/d9d/dbb 0 2024-07-16T09:55:42.578 INFO:tasks.workunit.client.0.smithi156.stdout:9/638: mknod d2/d5/d61/da2/d98/dbf/cc9 0 2024-07-16T09:55:42.579 INFO:tasks.workunit.client.0.smithi156.stdout:1/786: sync 2024-07-16T09:55:42.581 INFO:tasks.workunit.client.0.smithi156.stdout:1/787: readlink dc/d14/d3e/d55/d57/def/lee 0 2024-07-16T09:55:42.581 INFO:tasks.workunit.client.0.smithi156.stdout:9/639: write d2/d5/d16/d57/d63/f6e [2636862,125358] 0 2024-07-16T09:55:42.582 INFO:tasks.workunit.client.0.smithi156.stdout:0/586: mkdir d0/d1a/d76/dbc 0 2024-07-16T09:55:42.582 INFO:tasks.workunit.client.0.smithi156.stdout:7/488: creat dc/d13/fa5 x:0 0 0 2024-07-16T09:55:42.584 INFO:tasks.workunit.client.0.smithi156.stdout:1/788: fsync dc/d6b/dd5/fd9 0 2024-07-16T09:55:42.585 INFO:tasks.workunit.client.0.smithi156.stdout:7/489: mknod dc/d3e/d3f/d51/ca6 0 2024-07-16T09:55:42.588 INFO:tasks.workunit.client.0.smithi156.stdout:8/453: write d0/dd/f1a [2902541,27428] 0 2024-07-16T09:55:42.589 INFO:tasks.workunit.client.0.smithi156.stdout:9/640: mknod d2/d5/d16/d22/d6b/cca 0 2024-07-16T09:55:42.590 INFO:tasks.workunit.client.0.smithi156.stdout:7/490: fsync dc/d13/d20/f2b 0 2024-07-16T09:55:42.590 INFO:tasks.workunit.client.0.smithi156.stdout:0/587: dread d0/d1a/d4d/f21 [0,4194304] 0 2024-07-16T09:55:42.592 INFO:tasks.workunit.client.1.smithi176.stdout:9/701: getdents d5/de/d30/daf 0 2024-07-16T09:55:42.592 INFO:tasks.workunit.client.0.smithi156.stdout:8/454: chown d0/la 1960578 1 2024-07-16T09:55:42.596 INFO:tasks.workunit.client.1.smithi176.stdout:9/702: creat d5/d1d/d77/d97/ff3 x:0 0 0 2024-07-16T09:55:42.597 INFO:tasks.workunit.client.1.smithi176.stdout:9/703: chown d5/d52/leb 194 1 2024-07-16T09:55:42.598 INFO:tasks.workunit.client.0.smithi156.stdout:7/491: creat dc/d13/d15/d8b/fa7 x:0 0 0 2024-07-16T09:55:42.600 INFO:tasks.workunit.client.1.smithi176.stdout:8/592: rename d8/d23/d2c to d8/d20/d2a/db9/dcb 0 2024-07-16T09:55:42.603 INFO:tasks.workunit.client.1.smithi176.stdout:8/593: creat d8/d20/d2a/db9/dcb/d34/fcc x:0 0 0 2024-07-16T09:55:42.603 INFO:tasks.workunit.client.0.smithi156.stdout:0/588: dwrite d0/d22/f7a [0,4194304] 0 2024-07-16T09:55:42.604 INFO:tasks.workunit.client.1.smithi176.stdout:8/594: stat d8/d20/d2a/db9/dcb/d34/fcc 0 2024-07-16T09:55:42.605 INFO:tasks.workunit.client.0.smithi156.stdout:9/641: dwrite d2/d5/d16/d22/d2b/d74/dc2/dc4/f2c [0,4194304] 0 2024-07-16T09:55:42.607 INFO:tasks.workunit.client.0.smithi156.stdout:9/642: readlink d2/l77 0 2024-07-16T09:55:42.607 INFO:tasks.workunit.client.1.smithi176.stdout:2/661: write dd/d2e/d35/d4a/fce [555832,102628] 0 2024-07-16T09:55:42.607 INFO:tasks.workunit.client.1.smithi176.stdout:8/595: write d8/d14/d1e/d55/fa3 [4933373,98556] 0 2024-07-16T09:55:42.610 INFO:tasks.workunit.client.0.smithi156.stdout:0/589: mkdir d0/d1a/d4d/d9/d27/d8d/dbd 0 2024-07-16T09:55:42.610 INFO:tasks.workunit.client.1.smithi176.stdout:9/704: link d5/d43/f99 d5/d1d/d77/d98/ff4 0 2024-07-16T09:55:42.612 INFO:tasks.workunit.client.1.smithi176.stdout:8/596: write d8/d20/d2a/f49 [1556343,66918] 0 2024-07-16T09:55:42.612 INFO:tasks.workunit.client.1.smithi176.stdout:9/705: write d5/d1d/d77/d98/fb1 [264252,4003] 0 2024-07-16T09:55:42.614 INFO:tasks.workunit.client.1.smithi176.stdout:8/597: stat d8/d20/d2a/d31/d46/d6a/f7a 0 2024-07-16T09:55:42.615 INFO:tasks.workunit.client.1.smithi176.stdout:9/706: write d5/d1d/d77/d98/fda [53176,97451] 0 2024-07-16T09:55:42.616 INFO:tasks.workunit.client.1.smithi176.stdout:8/598: rename d8/d20 to d8/d20/d2a/d31/db4/dcd 22 2024-07-16T09:55:42.616 INFO:tasks.workunit.client.1.smithi176.stdout:2/662: dread dd/d36/d82/f8d [0,4194304] 0 2024-07-16T09:55:42.617 INFO:tasks.workunit.client.1.smithi176.stdout:5/727: sync 2024-07-16T09:55:42.618 INFO:tasks.workunit.client.1.smithi176.stdout:2/663: fdatasync dd/d16/d39/f4b 0 2024-07-16T09:55:42.619 INFO:tasks.workunit.client.1.smithi176.stdout:9/707: mknod d5/d1d/dc7/cf5 0 2024-07-16T09:55:42.621 INFO:tasks.workunit.client.0.smithi156.stdout:7/492: dwrite dc/f69 [0,4194304] 0 2024-07-16T09:55:42.622 INFO:tasks.workunit.client.0.smithi156.stdout:0/590: getdents d0/d1a/d4d/d25 0 2024-07-16T09:55:42.622 INFO:tasks.workunit.client.1.smithi176.stdout:8/599: read d8/d20/d2a/db9/dcb/d34/d62/fa4 [2222408,114968] 0 2024-07-16T09:55:42.623 INFO:tasks.workunit.client.1.smithi176.stdout:5/728: creat d4/d14/dce/f107 x:0 0 0 2024-07-16T09:55:42.624 INFO:tasks.workunit.client.0.smithi156.stdout:0/591: chown d0/d1a/d4d/d61/da1 0 1 2024-07-16T09:55:42.628 INFO:tasks.workunit.client.0.smithi156.stdout:8/455: dread d0/dd/f1e [0,4194304] 0 2024-07-16T09:55:42.629 INFO:tasks.workunit.client.0.smithi156.stdout:9/643: truncate d2/d5/d16/f9d 2029544 0 2024-07-16T09:55:42.632 INFO:tasks.workunit.client.1.smithi176.stdout:9/708: rename d5/d1d/dc7/f5c to d5/d1d/db3/ff6 0 2024-07-16T09:55:42.632 INFO:tasks.workunit.client.0.smithi156.stdout:8/456: write d0/f6f [1042131,77976] 0 2024-07-16T09:55:42.633 INFO:tasks.workunit.client.1.smithi176.stdout:9/709: write d5/de/f5b [354771,90863] 0 2024-07-16T09:55:42.634 INFO:tasks.workunit.client.1.smithi176.stdout:5/729: mknod d4/d10/c108 0 2024-07-16T09:55:42.634 INFO:tasks.workunit.client.0.smithi156.stdout:7/493: dwrite dc/d13/d36/d56/fa2 [0,4194304] 0 2024-07-16T09:55:42.637 INFO:tasks.workunit.client.0.smithi156.stdout:9/644: truncate d2/d5/d61/da2/d98/fc6 227324 0 2024-07-16T09:55:42.637 INFO:tasks.workunit.client.0.smithi156.stdout:0/592: rename d0/d1a/d4d/f11 to d0/d1a/d4d/d9/d27/d50/fbe 0 2024-07-16T09:55:42.637 INFO:tasks.workunit.client.0.smithi156.stdout:8/457: write d0/d11/d13/d1c/d39/fa0 [749048,56833] 0 2024-07-16T09:55:42.637 INFO:tasks.workunit.client.0.smithi156.stdout:7/494: stat dc/d13/d15/l57 0 2024-07-16T09:55:42.639 INFO:tasks.workunit.client.0.smithi156.stdout:8/458: read - d0/d11/d29/d80/f9d zero size 2024-07-16T09:55:42.639 INFO:tasks.workunit.client.1.smithi176.stdout:8/600: rename d8/f3d to d8/d14/d1e/d33/d9d/fce 0 2024-07-16T09:55:42.640 INFO:tasks.workunit.client.1.smithi176.stdout:9/710: read d5/de/d21/f84 [1645675,8017] 0 2024-07-16T09:55:42.640 INFO:tasks.workunit.client.0.smithi156.stdout:9/645: mknod d2/d5/d16/d22/d2b/d74/ccb 0 2024-07-16T09:55:42.641 INFO:tasks.workunit.client.1.smithi176.stdout:5/730: symlink d4/d37/d72/d106/l109 0 2024-07-16T09:55:42.642 INFO:tasks.workunit.client.1.smithi176.stdout:8/601: chown d8/d14/d1e/d33/c38 265122548 1 2024-07-16T09:55:42.642 INFO:tasks.workunit.client.0.smithi156.stdout:9/646: readlink d2/d5/d61/l72 0 2024-07-16T09:55:42.643 INFO:tasks.workunit.client.0.smithi156.stdout:0/593: readlink d0/d1a/d4d/l24 0 2024-07-16T09:55:42.643 INFO:tasks.workunit.client.1.smithi176.stdout:5/731: mkdir d4/d9a/d10a 0 2024-07-16T09:55:42.644 INFO:tasks.workunit.client.1.smithi176.stdout:2/664: truncate dd/d2e/f37 3880892 0 2024-07-16T09:55:42.645 INFO:tasks.workunit.client.0.smithi156.stdout:9/647: write d2/d5/d16/fae [57339,23605] 0 2024-07-16T09:55:42.645 INFO:tasks.workunit.client.1.smithi176.stdout:9/711: dread - d5/d1d/d77/d97/fa1 zero size 2024-07-16T09:55:42.647 INFO:tasks.workunit.client.0.smithi156.stdout:8/459: dwrite d0/dd/d1f/f8c [0,4194304] 0 2024-07-16T09:55:42.648 INFO:tasks.workunit.client.0.smithi156.stdout:0/594: write d0/d1a/d4d/d9/d27/d3d/faf [643358,113762] 0 2024-07-16T09:55:42.648 INFO:tasks.workunit.client.1.smithi176.stdout:5/732: write d4/d37/d72/d82/f64 [1983191,53077] 0 2024-07-16T09:55:42.649 INFO:tasks.workunit.client.1.smithi176.stdout:9/712: fsync d5/d1d/dc7/d89/fb9 0 2024-07-16T09:55:42.650 INFO:tasks.workunit.client.0.smithi156.stdout:9/648: creat d2/d5/d16/d22/d6b/fcc x:0 0 0 2024-07-16T09:55:42.650 INFO:tasks.workunit.client.0.smithi156.stdout:0/595: chown d0/d22/d41/d52/f74 26507995 1 2024-07-16T09:55:42.652 INFO:tasks.workunit.client.0.smithi156.stdout:8/460: write d0/f1 [8965218,78288] 0 2024-07-16T09:55:42.652 INFO:tasks.workunit.client.0.smithi156.stdout:0/596: dread - d0/d1a/d76/d83/fad zero size 2024-07-16T09:55:42.652 INFO:tasks.workunit.client.1.smithi176.stdout:8/602: dwrite d8/f2e [0,4194304] 0 2024-07-16T09:55:42.654 INFO:tasks.workunit.client.1.smithi176.stdout:9/713: symlink d5/de/d21/lf7 0 2024-07-16T09:55:42.657 INFO:tasks.workunit.client.0.smithi156.stdout:0/597: dread d0/d22/d41/f47 [0,4194304] 0 2024-07-16T09:55:42.659 INFO:tasks.workunit.client.0.smithi156.stdout:0/598: write d0/d1a/d4d/d9/d27/d3d/d62/f67 [1057523,126065] 0 2024-07-16T09:55:42.660 INFO:tasks.workunit.client.1.smithi176.stdout:8/603: creat d8/d14/fcf x:0 0 0 2024-07-16T09:55:42.660 INFO:tasks.workunit.client.0.smithi156.stdout:8/461: link d0/d11/d13/d1c/d39/f73 d0/d11/d62/fa4 0 2024-07-16T09:55:42.663 INFO:tasks.workunit.client.0.smithi156.stdout:0/599: symlink d0/d22/d41/d66/lbf 0 2024-07-16T09:55:42.665 INFO:tasks.workunit.client.0.smithi156.stdout:0/600: write d0/d1a/d4d/d25/f2a [1694046,117113] 0 2024-07-16T09:55:42.665 INFO:tasks.workunit.client.1.smithi176.stdout:5/733: dwrite d4/d14/fc1 [0,4194304] 0 2024-07-16T09:55:42.668 INFO:tasks.workunit.client.0.smithi156.stdout:8/462: getdents d0/d11/d2e/d35 0 2024-07-16T09:55:42.669 INFO:tasks.workunit.client.1.smithi176.stdout:9/714: link d5/d1d/cd8 d5/d1d/d67/d6d/cf8 0 2024-07-16T09:55:42.669 INFO:tasks.workunit.client.0.smithi156.stdout:0/601: mknod d0/d22/d41/d52/cc0 0 2024-07-16T09:55:42.671 INFO:tasks.workunit.client.1.smithi176.stdout:9/715: write d5/d1d/d77/d97/fe6 [422103,2667] 0 2024-07-16T09:55:42.672 INFO:tasks.workunit.client.1.smithi176.stdout:5/734: link d4/d37/d72/d38/d102/c104 d4/d14/dce/c10b 0 2024-07-16T09:55:42.672 INFO:tasks.workunit.client.1.smithi176.stdout:9/716: chown d5/de/f15 1 1 2024-07-16T09:55:42.674 INFO:tasks.workunit.client.1.smithi176.stdout:5/735: write d4/f5 [1358370,74649] 0 2024-07-16T09:55:42.675 INFO:tasks.workunit.client.1.smithi176.stdout:9/717: write d5/d1d/d77/d98/fb1 [357545,101639] 0 2024-07-16T09:55:42.675 INFO:tasks.workunit.client.0.smithi156.stdout:8/463: unlink d0/dd/d1f/d50/l4d 0 2024-07-16T09:55:42.677 INFO:tasks.workunit.client.0.smithi156.stdout:8/464: write d0/d11/d29/f8e [681845,87194] 0 2024-07-16T09:55:42.680 INFO:tasks.workunit.client.1.smithi176.stdout:9/718: rmdir d5/d52 39 2024-07-16T09:55:42.682 INFO:tasks.workunit.client.1.smithi176.stdout:5/736: truncate d4/d37/d72/d38/d39/f5d 2782663 0 2024-07-16T09:55:42.683 INFO:tasks.workunit.client.1.smithi176.stdout:5/737: chown d4/d37/d72/d82 386 1 2024-07-16T09:55:42.685 INFO:tasks.workunit.client.0.smithi156.stdout:8/465: dwrite d0/d11/d62/f90 [0,4194304] 0 2024-07-16T09:55:42.690 INFO:tasks.workunit.client.1.smithi176.stdout:5/738: mkdir d4/d10/d17/d10c 0 2024-07-16T09:55:42.691 INFO:tasks.workunit.client.1.smithi176.stdout:8/604: write d8/d20/f5a [7801921,38180] 0 2024-07-16T09:55:42.691 INFO:tasks.workunit.client.0.smithi156.stdout:7/495: dread dc/f42 [0,4194304] 0 2024-07-16T09:55:42.692 INFO:tasks.workunit.client.1.smithi176.stdout:9/719: dwrite d5/d1d/d77/d97/fdb [0,4194304] 0 2024-07-16T09:55:42.692 INFO:tasks.workunit.client.0.smithi156.stdout:7/496: fsync dc/d3e/d3f/d51/f53 0 2024-07-16T09:55:42.696 INFO:tasks.workunit.client.1.smithi176.stdout:2/665: dread f9 [4194304,4194304] 0 2024-07-16T09:55:42.697 INFO:tasks.workunit.client.1.smithi176.stdout:8/605: creat d8/d20/d2a/d53/d9a/dc1/fd0 x:0 0 0 2024-07-16T09:55:42.699 INFO:tasks.workunit.client.1.smithi176.stdout:8/606: chown d8/d20/d2a/db9/dcb/d37 2165188 1 2024-07-16T09:55:42.701 INFO:tasks.workunit.client.1.smithi176.stdout:2/666: fsync dd/d16/d39/d3d/f75 0 2024-07-16T09:55:42.701 INFO:tasks.workunit.client.0.smithi156.stdout:0/602: write d0/d1a/d4d/d9/d27/f6a [2683363,126991] 0 2024-07-16T09:55:42.702 INFO:tasks.workunit.client.1.smithi176.stdout:9/720: dread d5/d1d/d77/fd9 [0,4194304] 0 2024-07-16T09:55:42.702 INFO:tasks.workunit.client.1.smithi176.stdout:2/667: chown dd/d16 26 1 2024-07-16T09:55:42.703 INFO:tasks.workunit.client.0.smithi156.stdout:0/603: stat d0 0 2024-07-16T09:55:42.703 INFO:tasks.workunit.client.0.smithi156.stdout:8/466: write d0/dd/d1f/f6b [33242,107343] 0 2024-07-16T09:55:42.704 INFO:tasks.workunit.client.0.smithi156.stdout:0/604: fdatasync d0/d22/d41/f47 0 2024-07-16T09:55:42.705 INFO:tasks.workunit.client.0.smithi156.stdout:8/467: chown d0/d11/d20 26 1 2024-07-16T09:55:42.706 INFO:tasks.workunit.client.0.smithi156.stdout:8/468: readlink d0/d11/d4a/d55/l81 0 2024-07-16T09:55:42.706 INFO:tasks.workunit.client.0.smithi156.stdout:0/605: readlink d0/d1a/d4d/d9/d27/l4b 0 2024-07-16T09:55:42.707 INFO:tasks.workunit.client.0.smithi156.stdout:7/497: truncate dc/d13/d20/f3c 1198110 0 2024-07-16T09:55:42.707 INFO:tasks.workunit.client.0.smithi156.stdout:0/606: read - d0/d1a/f9b zero size 2024-07-16T09:55:42.708 INFO:tasks.workunit.client.1.smithi176.stdout:8/607: unlink d8/d20/d2a/db9/dcb/d34/d62/lc6 0 2024-07-16T09:55:42.709 INFO:tasks.workunit.client.0.smithi156.stdout:8/469: creat d0/d11/d2e/fa5 x:0 0 0 2024-07-16T09:55:42.710 INFO:tasks.workunit.client.1.smithi176.stdout:2/668: mknod dd/d2e/cde 0 2024-07-16T09:55:42.711 INFO:tasks.workunit.client.0.smithi156.stdout:8/470: truncate d0/d11/d20/d7f/f9e 971304 0 2024-07-16T09:55:42.711 INFO:tasks.workunit.client.0.smithi156.stdout:0/607: rename d0/d1a/d4d/d25/l69 to d0/d1a/d76/lc1 0 2024-07-16T09:55:42.711 INFO:tasks.workunit.client.1.smithi176.stdout:9/721: mkdir d5/de/df9 0 2024-07-16T09:55:42.712 INFO:tasks.workunit.client.1.smithi176.stdout:8/608: rmdir d8/d20/d2a/db9/dcb/d37 39 2024-07-16T09:55:42.713 INFO:tasks.workunit.client.1.smithi176.stdout:9/722: fsync d5/d1d/db3/fcb 0 2024-07-16T09:55:42.713 INFO:tasks.workunit.client.1.smithi176.stdout:2/669: rmdir dd/d36/d45 39 2024-07-16T09:55:42.714 INFO:tasks.workunit.client.0.smithi156.stdout:8/471: rmdir d0/d11/d20/d3f 39 2024-07-16T09:55:42.715 INFO:tasks.workunit.client.1.smithi176.stdout:8/609: read - d8/d14/d1e/d7c/fbe zero size 2024-07-16T09:55:42.715 INFO:tasks.workunit.client.0.smithi156.stdout:8/472: chown d0/dd/f4c 0 1 2024-07-16T09:55:42.715 INFO:tasks.workunit.client.0.smithi156.stdout:0/608: mknod d0/d1a/d4d/d61/cc2 0 2024-07-16T09:55:42.717 INFO:tasks.workunit.client.1.smithi176.stdout:8/610: write d8/d14/d1e/f60 [97429,111800] 0 2024-07-16T09:55:42.717 INFO:tasks.workunit.client.0.smithi156.stdout:0/609: symlink d0/d1a/d4d/d61/lc3 0 2024-07-16T09:55:42.718 INFO:tasks.workunit.client.0.smithi156.stdout:0/610: dread - d0/d1a/d4d/d9/d27/f81 zero size 2024-07-16T09:55:42.721 INFO:tasks.workunit.client.1.smithi176.stdout:9/723: rename d5/d1d/d20/c23 to d5/de/d30/cfa 0 2024-07-16T09:55:42.722 INFO:tasks.workunit.client.0.smithi156.stdout:0/611: creat d0/d1a/d4d/d9/d27/d2c/d6d/d9d/dbb/fc4 x:0 0 0 2024-07-16T09:55:42.723 INFO:tasks.workunit.client.1.smithi176.stdout:2/670: rename dd/d36/d45/fd1 to dd/d16/d39/d3d/d7e/fdf 0 2024-07-16T09:55:42.724 INFO:tasks.workunit.client.1.smithi176.stdout:2/671: fdatasync dd/d2e/d35/d59/fbc 0 2024-07-16T09:55:42.725 INFO:tasks.workunit.client.0.smithi156.stdout:0/612: rename d0/d1a/d4d/d25/c89 to d0/d22/d41/d66/cc5 0 2024-07-16T09:55:42.725 INFO:tasks.workunit.client.0.smithi156.stdout:0/613: stat d0/d1a/d4d/d25/fb8 0 2024-07-16T09:55:42.726 INFO:tasks.workunit.client.0.smithi156.stdout:8/473: dread d0/f12 [0,4194304] 0 2024-07-16T09:55:42.727 INFO:tasks.workunit.client.1.smithi176.stdout:9/724: rename d5/d52/c4d to d5/d1d/d67/cfb 0 2024-07-16T09:55:42.728 INFO:tasks.workunit.client.1.smithi176.stdout:2/672: mknod dd/d36/d82/ce0 0 2024-07-16T09:55:42.728 INFO:tasks.workunit.client.0.smithi156.stdout:8/474: mkdir d0/dd/d1f/d57/da6 0 2024-07-16T09:55:42.732 INFO:tasks.workunit.client.1.smithi176.stdout:8/611: write d8/d14/d1e/d33/d9d/fce [236018,57688] 0 2024-07-16T09:55:42.733 INFO:tasks.workunit.client.0.smithi156.stdout:0/614: dwrite d0/d1a/d4d/d9/d27/d2c/d6d/f8b [4194304,4194304] 0 2024-07-16T09:55:42.737 INFO:tasks.workunit.client.1.smithi176.stdout:9/725: dwrite d5/d1d/d77/d97/fdb [0,4194304] 0 2024-07-16T09:55:42.739 INFO:tasks.workunit.client.0.smithi156.stdout:0/615: mknod d0/d1a/d4d/d9/d27/cc6 0 2024-07-16T09:55:42.741 INFO:tasks.workunit.client.1.smithi176.stdout:8/612: dread d8/d23/f25 [0,4194304] 0 2024-07-16T09:55:42.745 INFO:tasks.workunit.client.0.smithi156.stdout:0/616: dread d0/d1a/d4d/d9/d27/d8d/fb6 [0,4194304] 0 2024-07-16T09:55:42.746 INFO:tasks.workunit.client.0.smithi156.stdout:8/475: dwrite d0/d11/f72 [0,4194304] 0 2024-07-16T09:55:42.750 INFO:tasks.workunit.client.0.smithi156.stdout:8/476: creat d0/dd/fa7 x:0 0 0 2024-07-16T09:55:42.754 INFO:tasks.workunit.client.1.smithi176.stdout:2/673: dread dd/d36/f2f [0,4194304] 0 2024-07-16T09:55:42.755 INFO:tasks.workunit.client.0.smithi156.stdout:0/617: getdents d0/d1a/d4d/d9/d27/d3d/d62 0 2024-07-16T09:55:42.756 INFO:tasks.workunit.client.0.smithi156.stdout:0/618: dread - d0/d1a/d4d/d9/d27/f81 zero size 2024-07-16T09:55:42.757 INFO:tasks.workunit.client.0.smithi156.stdout:0/619: stat d0/d1a/d4d/d9/d27/d8d/fa4 0 2024-07-16T09:55:42.759 INFO:tasks.workunit.client.0.smithi156.stdout:0/620: truncate d0/f14 2330966 0 2024-07-16T09:55:42.761 INFO:tasks.workunit.client.0.smithi156.stdout:0/621: getdents d0 0 2024-07-16T09:55:42.763 INFO:tasks.workunit.client.1.smithi176.stdout:2/674: dwrite dd/d2e/d35/d4a/fd9 [0,4194304] 0 2024-07-16T09:55:42.765 INFO:tasks.workunit.client.0.smithi156.stdout:0/622: creat d0/d1a/d4d/d9/fc7 x:0 0 0 2024-07-16T09:55:42.770 INFO:tasks.workunit.client.0.smithi156.stdout:4/642: sync 2024-07-16T09:55:42.772 INFO:tasks.workunit.client.0.smithi156.stdout:8/477: truncate d0/f2c 4671497 0 2024-07-16T09:55:42.773 INFO:tasks.workunit.client.0.smithi156.stdout:4/643: truncate d4/db/dc/f42 798536 0 2024-07-16T09:55:42.773 INFO:tasks.workunit.client.1.smithi176.stdout:9/726: dread d5/d1d/d67/d6d/f7a [0,4194304] 0 2024-07-16T09:55:42.774 INFO:tasks.workunit.client.1.smithi176.stdout:2/675: dwrite dd/d2e/f50 [0,4194304] 0 2024-07-16T09:55:42.775 INFO:tasks.workunit.client.1.smithi176.stdout:2/676: dread - dd/de/fa7 zero size 2024-07-16T09:55:42.775 INFO:tasks.workunit.client.0.smithi156.stdout:8/478: mkdir d0/d11/d29/da8 0 2024-07-16T09:55:42.777 INFO:tasks.workunit.client.0.smithi156.stdout:8/479: rmdir d0/d11/d29 39 2024-07-16T09:55:42.778 INFO:tasks.workunit.client.1.smithi176.stdout:2/677: readlink dd/d36/la1 0 2024-07-16T09:55:42.781 INFO:tasks.workunit.client.0.smithi156.stdout:4/644: dwrite d4/db/f16 [0,4194304] 0 2024-07-16T09:55:42.783 INFO:tasks.workunit.client.0.smithi156.stdout:0/623: dwrite d0/d1a/d4d/d9/d27/d50/f5f [0,4194304] 0 2024-07-16T09:55:42.783 INFO:tasks.workunit.client.1.smithi176.stdout:9/727: dwrite d5/f1c [4194304,4194304] 0 2024-07-16T09:55:42.784 INFO:tasks.workunit.client.0.smithi156.stdout:4/645: readlink d4/db/dc/l13 0 2024-07-16T09:55:42.785 INFO:tasks.workunit.client.0.smithi156.stdout:8/480: rename d0/dd/d1f/d50/c65 to d0/d11/d20/ca9 0 2024-07-16T09:55:42.785 INFO:tasks.workunit.client.1.smithi176.stdout:2/678: creat dd/d36/d45/dd3/fe1 x:0 0 0 2024-07-16T09:55:42.786 INFO:tasks.workunit.client.1.smithi176.stdout:2/679: dread - dd/d16/f71 zero size 2024-07-16T09:55:42.787 INFO:tasks.workunit.client.0.smithi156.stdout:0/624: symlink d0/d1a/d76/d92/lc8 0 2024-07-16T09:55:42.787 INFO:tasks.workunit.client.1.smithi176.stdout:9/728: unlink d5/d1d/db0/fde 0 2024-07-16T09:55:42.787 INFO:tasks.workunit.client.1.smithi176.stdout:2/680: chown dd/d49/d52/d89/lc7 9217 1 2024-07-16T09:55:42.788 INFO:tasks.workunit.client.0.smithi156.stdout:4/646: write d4/d23/d30/f2d [1339286,62820] 0 2024-07-16T09:55:42.791 INFO:tasks.workunit.client.1.smithi176.stdout:9/729: fdatasync d5/d1d/dc7/f60 0 2024-07-16T09:55:42.796 INFO:tasks.workunit.client.1.smithi176.stdout:9/730: dread - d5/d43/f99 zero size 2024-07-16T09:55:42.797 INFO:tasks.workunit.client.0.smithi156.stdout:0/625: mkdir d0/d22/d41/da9/dc9 0 2024-07-16T09:55:42.798 INFO:tasks.workunit.client.0.smithi156.stdout:8/481: dwrite d0/d11/d20/d3f/f69 [0,4194304] 0 2024-07-16T09:55:42.800 INFO:tasks.workunit.client.0.smithi156.stdout:0/626: stat d0/d1a/d4d/d9/d27/d8d/lb1 0 2024-07-16T09:55:42.802 INFO:tasks.workunit.client.0.smithi156.stdout:4/647: dwrite d4/db/dc/f70 [0,4194304] 0 2024-07-16T09:55:42.802 INFO:tasks.workunit.client.0.smithi156.stdout:8/482: write d0/ff [4771119,73617] 0 2024-07-16T09:55:42.803 INFO:tasks.workunit.client.0.smithi156.stdout:4/648: chown d4/db 176389 1 2024-07-16T09:55:42.804 INFO:tasks.workunit.client.0.smithi156.stdout:0/627: write d0/d1a/d4d/d9/d27/d2c/d6d/f8b [1369203,113813] 0 2024-07-16T09:55:42.804 INFO:tasks.workunit.client.0.smithi156.stdout:4/649: chown d4/d23/f59 246 1 2024-07-16T09:55:42.805 INFO:tasks.workunit.client.1.smithi176.stdout:9/731: dread d5/f1e [0,4194304] 0 2024-07-16T09:55:42.807 INFO:tasks.workunit.client.0.smithi156.stdout:4/650: write d4/d23/d30/d33/f3a [2128744,4160] 0 2024-07-16T09:55:42.808 INFO:tasks.workunit.client.0.smithi156.stdout:4/651: stat d4/d23/d52/d71/fa7 0 2024-07-16T09:55:42.808 INFO:tasks.workunit.client.1.smithi176.stdout:9/732: symlink d5/d1d/lfc 0 2024-07-16T09:55:42.809 INFO:tasks.workunit.client.0.smithi156.stdout:4/652: chown d4/db/d47/fbf 2251788 1 2024-07-16T09:55:42.809 INFO:tasks.workunit.client.1.smithi176.stdout:9/733: dread - d5/d1d/d20/f33 zero size 2024-07-16T09:55:42.810 INFO:tasks.workunit.client.0.smithi156.stdout:0/628: write d0/d1a/d4d/d9/d27/d8d/fb6 [3263584,118437] 0 2024-07-16T09:55:42.811 INFO:tasks.workunit.client.0.smithi156.stdout:8/483: mknod d0/d11/d29/d80/d97/caa 0 2024-07-16T09:55:42.813 INFO:tasks.workunit.client.0.smithi156.stdout:4/653: read d4/d23/d30/f25 [4403,109931] 0 2024-07-16T09:55:42.815 INFO:tasks.workunit.client.0.smithi156.stdout:0/629: unlink d0/d1a/d4d/d9/d27/d8d/lb1 0 2024-07-16T09:55:42.816 INFO:tasks.workunit.client.1.smithi176.stdout:9/734: dwrite d5/de/f61 [0,4194304] 0 2024-07-16T09:55:42.818 INFO:tasks.workunit.client.0.smithi156.stdout:8/484: read d0/dd/d1f/d60/f98 [2894100,86362] 0 2024-07-16T09:55:42.820 INFO:tasks.workunit.client.1.smithi176.stdout:9/735: getdents d5/d1d/dd2/de3 0 2024-07-16T09:55:42.830 INFO:tasks.workunit.client.1.smithi176.stdout:9/736: dwrite d5/de/f40 [0,4194304] 0 2024-07-16T09:55:42.833 INFO:tasks.workunit.client.1.smithi176.stdout:2/681: dwrite dd/d2e/d35/f64 [4194304,4194304] 0 2024-07-16T09:55:42.837 INFO:tasks.workunit.client.1.smithi176.stdout:9/737: rename d5/d1d/dc7/c80 to d5/de/df9/cfd 0 2024-07-16T09:55:42.838 INFO:tasks.workunit.client.1.smithi176.stdout:2/682: creat dd/d36/d45/fe2 x:0 0 0 2024-07-16T09:55:42.841 INFO:tasks.workunit.client.1.smithi176.stdout:2/683: write dd/d49/d52/f79 [4565310,91935] 0 2024-07-16T09:55:42.845 INFO:tasks.workunit.client.1.smithi176.stdout:2/684: getdents dd/d36 0 2024-07-16T09:55:42.848 INFO:tasks.workunit.client.1.smithi176.stdout:9/738: dread d5/de/f2f [0,4194304] 0 2024-07-16T09:55:42.849 INFO:tasks.workunit.client.1.smithi176.stdout:2/685: dread dd/d49/d52/d7c/fcc [0,4194304] 0 2024-07-16T09:55:42.850 INFO:tasks.workunit.client.0.smithi156.stdout:4/654: truncate d4/db/dc/dcc/d58/d8f/f7e 2088785 0 2024-07-16T09:55:42.850 INFO:tasks.workunit.client.1.smithi176.stdout:9/739: read d5/d1d/db3/fd1 [100863,5653] 0 2024-07-16T09:55:42.851 INFO:tasks.workunit.client.0.smithi156.stdout:4/655: chown d4/db/dc/d7f/daf/fb5 69489567 1 2024-07-16T09:55:42.852 INFO:tasks.workunit.client.0.smithi156.stdout:4/656: readlink d4/d23/l28 0 2024-07-16T09:55:42.853 INFO:tasks.workunit.client.1.smithi176.stdout:9/740: symlink d5/d1d/d77/d98/lfe 0 2024-07-16T09:55:42.854 INFO:tasks.workunit.client.0.smithi156.stdout:4/657: creat d4/fe9 x:0 0 0 2024-07-16T09:55:42.855 INFO:tasks.workunit.client.1.smithi176.stdout:9/741: symlink d5/d1d/dc7/d89/lff 0 2024-07-16T09:55:42.856 INFO:tasks.workunit.client.0.smithi156.stdout:0/630: dread d0/d1a/d4d/d25/f2a [0,4194304] 0 2024-07-16T09:55:42.857 INFO:tasks.workunit.client.1.smithi176.stdout:9/742: unlink d5/d1d/d20/l26 0 2024-07-16T09:55:42.858 INFO:tasks.workunit.client.0.smithi156.stdout:0/631: rename d0/d1a/c1c to d0/cca 0 2024-07-16T09:55:42.859 INFO:tasks.workunit.client.0.smithi156.stdout:4/658: link d4/db/dc/d7f/daf/lb6 d4/db/dc/d5f/lea 0 2024-07-16T09:55:42.859 INFO:tasks.workunit.client.1.smithi176.stdout:9/743: mknod d5/de/c100 0 2024-07-16T09:55:42.860 INFO:tasks.workunit.client.0.smithi156.stdout:0/632: mkdir d0/d1a/d4d/d9/d68/dcb 0 2024-07-16T09:55:42.861 INFO:tasks.workunit.client.0.smithi156.stdout:4/659: creat d4/d23/d30/dd2/feb x:0 0 0 2024-07-16T09:55:42.861 INFO:tasks.workunit.client.1.smithi176.stdout:9/744: unlink d5/d1d/d67/d6d/c85 0 2024-07-16T09:55:42.862 INFO:tasks.workunit.client.0.smithi156.stdout:0/633: creat d0/d1a/d76/d92/fcc x:0 0 0 2024-07-16T09:55:42.863 INFO:tasks.workunit.client.1.smithi176.stdout:9/745: write d5/d1d/d77/d97/fdb [622785,63495] 0 2024-07-16T09:55:42.864 INFO:tasks.workunit.client.0.smithi156.stdout:0/634: creat d0/d1a/d76/d83/fcd x:0 0 0 2024-07-16T09:55:42.865 INFO:tasks.workunit.client.1.smithi176.stdout:2/686: write dd/d16/d39/d3d/f75 [1545432,10817] 0 2024-07-16T09:55:42.865 INFO:tasks.workunit.client.0.smithi156.stdout:8/485: dread d0/dd/f17 [0,4194304] 0 2024-07-16T09:55:42.867 INFO:tasks.workunit.client.1.smithi176.stdout:2/687: rmdir dd/d49/d52/d7c 39 2024-07-16T09:55:42.868 INFO:tasks.workunit.client.1.smithi176.stdout:9/746: truncate d5/d1d/d20/f25 525357 0 2024-07-16T09:55:42.869 INFO:tasks.workunit.client.0.smithi156.stdout:4/660: rmdir d4/d6a/da9 0 2024-07-16T09:55:42.869 INFO:tasks.workunit.client.0.smithi156.stdout:4/661: stat d4/db/dc/d7c/le7 0 2024-07-16T09:55:42.870 INFO:tasks.workunit.client.1.smithi176.stdout:2/688: mkdir dd/d36/d45/de3 0 2024-07-16T09:55:42.870 INFO:tasks.workunit.client.0.smithi156.stdout:0/635: truncate d0/f14 269314 0 2024-07-16T09:55:42.871 INFO:tasks.workunit.client.1.smithi176.stdout:9/747: write d5/d52/fa4 [626229,98598] 0 2024-07-16T09:55:42.872 INFO:tasks.workunit.client.0.smithi156.stdout:0/636: chown d0/d1a/d4d/d9/d27/d2c/d6d/d9d/dbb 119614468 1 2024-07-16T09:55:42.872 INFO:tasks.workunit.client.1.smithi176.stdout:2/689: chown dd/d49/d52/d7c/d81/lae 22463 1 2024-07-16T09:55:42.872 INFO:tasks.workunit.client.1.smithi176.stdout:9/748: read d5/d1d/d67/d6d/f7a [2874524,110562] 0 2024-07-16T09:55:42.873 INFO:tasks.workunit.client.0.smithi156.stdout:0/637: chown d0/d1a/d4d/d9/d27/d2c/d6d/f8b 332930670 1 2024-07-16T09:55:42.874 INFO:tasks.workunit.client.0.smithi156.stdout:8/486: rename d0/d11/d13/d1c/d39/c6e to d0/dd/d1f/cab 0 2024-07-16T09:55:42.875 INFO:tasks.workunit.client.0.smithi156.stdout:0/638: write d0/d1a/d4d/d9/d27/f6a [2727337,15938] 0 2024-07-16T09:55:42.876 INFO:tasks.workunit.client.0.smithi156.stdout:0/639: write d0/d22/f9e [682742,28938] 0 2024-07-16T09:55:42.877 INFO:tasks.workunit.client.0.smithi156.stdout:4/662: rename d4/db/dc/dcc/d58/d8f/ddd to d4/db/dc/dcc/dec 0 2024-07-16T09:55:42.877 INFO:tasks.workunit.client.1.smithi176.stdout:9/749: creat d5/d1d/d20/f101 x:0 0 0 2024-07-16T09:55:42.880 INFO:tasks.workunit.client.0.smithi156.stdout:4/663: stat d4/db/dc/f42 0 2024-07-16T09:55:42.880 INFO:tasks.workunit.client.1.smithi176.stdout:2/690: dwrite dd/d36/f94 [0,4194304] 0 2024-07-16T09:55:42.880 INFO:tasks.workunit.client.1.smithi176.stdout:9/750: dread d5/f10 [0,4194304] 0 2024-07-16T09:55:42.881 INFO:tasks.workunit.client.0.smithi156.stdout:4/664: stat d4/db/dc/dcc/dec 0 2024-07-16T09:55:42.881 INFO:tasks.workunit.client.1.smithi176.stdout:6/601: sync 2024-07-16T09:55:42.881 INFO:tasks.workunit.client.1.smithi176.stdout:6/602: dread - d1/dc/d40/f78 zero size 2024-07-16T09:55:42.882 INFO:tasks.workunit.client.0.smithi156.stdout:0/640: rename d0/d1a/f54 to d0/d22/d41/fce 0 2024-07-16T09:55:42.883 INFO:tasks.workunit.client.0.smithi156.stdout:0/641: dread - d0/d1a/f9c zero size 2024-07-16T09:55:42.884 INFO:tasks.workunit.client.1.smithi176.stdout:2/691: unlink dd/d16/c68 0 2024-07-16T09:55:42.884 INFO:tasks.workunit.client.0.smithi156.stdout:4/665: dread d4/db/dc/d7f/db1/fd1 [0,4194304] 0 2024-07-16T09:55:42.885 INFO:tasks.workunit.client.1.smithi176.stdout:2/692: write dd/d36/d45/fe2 [646154,38236] 0 2024-07-16T09:55:42.886 INFO:tasks.workunit.client.1.smithi176.stdout:9/751: unlink d5/d1d/d67/fdf 0 2024-07-16T09:55:42.886 INFO:tasks.workunit.client.0.smithi156.stdout:4/666: readlink d4/d23/l27 0 2024-07-16T09:55:42.886 INFO:tasks.workunit.client.1.smithi176.stdout:6/603: creat d1/dc/d81/fdd x:0 0 0 2024-07-16T09:55:42.887 INFO:tasks.workunit.client.1.smithi176.stdout:2/693: write dd/d16/f33 [1022581,7360] 0 2024-07-16T09:55:42.889 INFO:tasks.workunit.client.0.smithi156.stdout:4/667: write d4/db/dc/d7c/f8e [4855168,98607] 0 2024-07-16T09:55:42.889 INFO:tasks.workunit.client.1.smithi176.stdout:6/604: fdatasync d1/d5/f2c 0 2024-07-16T09:55:42.891 INFO:tasks.workunit.client.1.smithi176.stdout:2/694: creat dd/d16/fe4 x:0 0 0 2024-07-16T09:55:42.892 INFO:tasks.workunit.client.0.smithi156.stdout:0/642: dwrite d0/d22/f3c [0,4194304] 0 2024-07-16T09:55:42.893 INFO:tasks.workunit.client.1.smithi176.stdout:9/752: dread d5/d1d/d77/d98/fda [0,4194304] 0 2024-07-16T09:55:42.894 INFO:tasks.workunit.client.0.smithi156.stdout:4/668: rename d4/db/dc/d7f to d4/db/dc/dcc/dec/ded 0 2024-07-16T09:55:42.895 INFO:tasks.workunit.client.1.smithi176.stdout:2/695: chown dd/d16/f56 3751 1 2024-07-16T09:55:42.897 INFO:tasks.workunit.client.1.smithi176.stdout:9/753: creat d5/d1d/db0/f102 x:0 0 0 2024-07-16T09:55:42.899 INFO:tasks.workunit.client.1.smithi176.stdout:9/754: readlink d5/d1d/d20/lb6 0 2024-07-16T09:55:42.900 INFO:tasks.workunit.client.0.smithi156.stdout:4/669: truncate d4/d23/d30/f25 289774 0 2024-07-16T09:55:42.902 INFO:tasks.workunit.client.0.smithi156.stdout:0/643: dwrite d0/d22/d41/d52/fb4 [4194304,4194304] 0 2024-07-16T09:55:42.902 INFO:tasks.workunit.client.0.smithi156.stdout:4/670: write d4/d23/d30/fa1 [585084,52536] 0 2024-07-16T09:55:42.905 INFO:tasks.workunit.client.1.smithi176.stdout:2/696: dwrite dd/de/fa8 [0,4194304] 0 2024-07-16T09:55:42.908 INFO:tasks.workunit.client.0.smithi156.stdout:0/644: creat d0/d22/d41/d66/fcf x:0 0 0 2024-07-16T09:55:42.910 INFO:tasks.workunit.client.0.smithi156.stdout:0/645: truncate d0/d1a/d76/d83/fad 108373 0 2024-07-16T09:55:42.910 INFO:tasks.workunit.client.1.smithi176.stdout:9/755: dwrite d5/de/f61 [4194304,4194304] 0 2024-07-16T09:55:42.911 INFO:tasks.workunit.client.0.smithi156.stdout:4/671: creat d4/db/fee x:0 0 0 2024-07-16T09:55:42.912 INFO:tasks.workunit.client.1.smithi176.stdout:9/756: write d5/de/f5b [551140,116341] 0 2024-07-16T09:55:42.914 INFO:tasks.workunit.client.0.smithi156.stdout:4/672: mkdir d4/d23/d52/d71/def 0 2024-07-16T09:55:42.915 INFO:tasks.workunit.client.1.smithi176.stdout:9/757: dread - d5/d1d/db0/fd5 zero size 2024-07-16T09:55:42.916 INFO:tasks.workunit.client.0.smithi156.stdout:0/646: mknod d0/d1a/d4d/d9/d27/d8d/dbd/cd0 0 2024-07-16T09:55:42.917 INFO:tasks.workunit.client.1.smithi176.stdout:2/697: dwrite dd/d49/d52/f63 [0,4194304] 0 2024-07-16T09:55:42.918 INFO:tasks.workunit.client.0.smithi156.stdout:4/673: write d4/db/dc/d5f/fb3 [199702,13943] 0 2024-07-16T09:55:42.919 INFO:tasks.workunit.client.0.smithi156.stdout:0/647: mknod d0/d22/cd1 0 2024-07-16T09:55:42.920 INFO:tasks.workunit.client.0.smithi156.stdout:4/674: write d4/d23/d52/fac [3322167,70064] 0 2024-07-16T09:55:42.921 INFO:tasks.workunit.client.1.smithi176.stdout:9/758: unlink d5/d1d/db3/fcb 0 2024-07-16T09:55:42.922 INFO:tasks.workunit.client.0.smithi156.stdout:0/648: write d0/d1a/d4d/d9/d27/f6a [324731,109787] 0 2024-07-16T09:55:42.923 INFO:tasks.workunit.client.0.smithi156.stdout:0/649: readlink d0/d1a/d4d/d60/la5 0 2024-07-16T09:55:42.924 INFO:tasks.workunit.client.1.smithi176.stdout:2/698: rename dd/d49/l7b to dd/d16/d39/d3d/le5 0 2024-07-16T09:55:42.925 INFO:tasks.workunit.client.0.smithi156.stdout:4/675: symlink d4/d23/d52/d71/def/lf0 0 2024-07-16T09:55:42.925 INFO:tasks.workunit.client.1.smithi176.stdout:9/759: mkdir d5/d1d/dd2/de3/d103 0 2024-07-16T09:55:42.927 INFO:tasks.workunit.client.1.smithi176.stdout:2/699: mknod dd/dcd/ce6 0 2024-07-16T09:55:42.929 INFO:tasks.workunit.client.0.smithi156.stdout:4/676: mknod d4/db/dc/dcc/dec/ded/daf/cf1 0 2024-07-16T09:55:42.929 INFO:tasks.workunit.client.1.smithi176.stdout:9/760: symlink d5/d1d/db0/dd3/l104 0 2024-07-16T09:55:42.930 INFO:tasks.workunit.client.1.smithi176.stdout:9/761: chown d5/d1d/db0/dd3 1332231 1 2024-07-16T09:55:42.930 INFO:tasks.workunit.client.0.smithi156.stdout:0/650: link d0/d1a/d4d/d9/d27/d3d/d62/f67 d0/d22/d41/da9/fd2 0 2024-07-16T09:55:42.931 INFO:tasks.workunit.client.0.smithi156.stdout:4/677: chown d4/db/dc/d5f/fb3 3 1 2024-07-16T09:55:42.931 INFO:tasks.workunit.client.1.smithi176.stdout:2/700: mknod dd/d2e/d35/d59/d66/dc8/ce7 0 2024-07-16T09:55:42.933 INFO:tasks.workunit.client.0.smithi156.stdout:4/678: write d4/db/dc/dcc/dec/ded/daf/fb5 [87498,28118] 0 2024-07-16T09:55:42.934 INFO:tasks.workunit.client.1.smithi176.stdout:2/701: symlink dd/d49/d52/d89/le8 0 2024-07-16T09:55:42.935 INFO:tasks.workunit.client.1.smithi176.stdout:2/702: stat dd/f3c 0 2024-07-16T09:55:42.937 INFO:tasks.workunit.client.0.smithi156.stdout:0/651: dwrite d0/d22/f7a [0,4194304] 0 2024-07-16T09:55:42.937 INFO:tasks.workunit.client.1.smithi176.stdout:2/703: chown dd/d16/d39/d3d/d7e/fc3 107 1 2024-07-16T09:55:42.940 INFO:tasks.workunit.client.1.smithi176.stdout:2/704: unlink dd/d16/d39/d3d/fd2 0 2024-07-16T09:55:42.942 INFO:tasks.workunit.client.0.smithi156.stdout:4/679: rmdir d4/db/dc/d67/dd7 0 2024-07-16T09:55:42.943 INFO:tasks.workunit.client.1.smithi176.stdout:2/705: mknod dd/d2e/d35/d59/d66/dc8/ce9 0 2024-07-16T09:55:42.946 INFO:tasks.workunit.client.0.smithi156.stdout:0/652: dwrite d0/d1a/d4d/d9/f56 [0,4194304] 0 2024-07-16T09:55:42.950 INFO:tasks.workunit.client.0.smithi156.stdout:0/653: mkdir d0/d1a/d76/da7/dd3 0 2024-07-16T09:55:42.950 INFO:tasks.workunit.client.0.smithi156.stdout:4/680: dwrite d4/d23/f53 [0,4194304] 0 2024-07-16T09:55:42.952 INFO:tasks.workunit.client.1.smithi176.stdout:2/706: dwrite dd/d16/f27 [0,4194304] 0 2024-07-16T09:55:42.953 INFO:tasks.workunit.client.1.smithi176.stdout:9/762: dwrite d5/d52/ff0 [0,4194304] 0 2024-07-16T09:55:42.953 INFO:tasks.workunit.client.0.smithi156.stdout:4/681: chown d4/db/dc/d5f/f5c 61053873 1 2024-07-16T09:55:42.954 INFO:tasks.workunit.client.0.smithi156.stdout:4/682: chown d4/db/dc/d7c/le7 30996 1 2024-07-16T09:55:42.960 INFO:tasks.workunit.client.0.smithi156.stdout:0/654: creat d0/d1a/d4d/d9/d68/dcb/fd4 x:0 0 0 2024-07-16T09:55:42.962 INFO:tasks.workunit.client.1.smithi176.stdout:2/707: dwrite dd/d2e/d35/fac [0,4194304] 0 2024-07-16T09:55:42.966 INFO:tasks.workunit.client.1.smithi176.stdout:9/763: getdents d5/de/da5/dc1 0 2024-07-16T09:55:42.968 INFO:tasks.workunit.client.0.smithi156.stdout:4/683: dwrite d4/db/dc/dcc/dec/ded/db1/fd1 [0,4194304] 0 2024-07-16T09:55:42.968 INFO:tasks.workunit.client.1.smithi176.stdout:2/708: write dd/d9b/fb1 [394060,8131] 0 2024-07-16T09:55:42.972 INFO:tasks.workunit.client.0.smithi156.stdout:0/655: getdents d0/d1a/d4d/d6e 0 2024-07-16T09:55:42.974 INFO:tasks.workunit.client.1.smithi176.stdout:9/764: getdents d5/d1d/dd2/de3 0 2024-07-16T09:55:42.975 INFO:tasks.workunit.client.0.smithi156.stdout:4/684: mkdir d4/db/dc/dcc/dec/ded/d87/df2 0 2024-07-16T09:55:42.975 INFO:tasks.workunit.client.0.smithi156.stdout:0/656: truncate d0/d22/d41/f8a 1046041 0 2024-07-16T09:55:42.976 INFO:tasks.workunit.client.0.smithi156.stdout:4/685: write d4/fe9 [805928,79133] 0 2024-07-16T09:55:42.978 INFO:tasks.workunit.client.0.smithi156.stdout:0/657: truncate d0/d1a/d4d/d25/fb8 704417 0 2024-07-16T09:55:42.981 INFO:tasks.workunit.client.0.smithi156.stdout:4/686: dread d4/f9 [0,4194304] 0 2024-07-16T09:55:42.981 INFO:tasks.workunit.client.0.smithi156.stdout:0/658: truncate d0/d1a/d4d/d9/d27/f81 203235 0 2024-07-16T09:55:42.981 INFO:tasks.workunit.client.1.smithi176.stdout:9/765: link d5/d1d/db0/f102 d5/de/da5/dbf/f105 0 2024-07-16T09:55:42.981 INFO:tasks.workunit.client.0.smithi156.stdout:4/687: stat d4/db/dc/dcc/d58/f91 0 2024-07-16T09:55:42.982 INFO:tasks.workunit.client.1.smithi176.stdout:9/766: rmdir d5/d1d/d77/d98 39 2024-07-16T09:55:42.983 INFO:tasks.workunit.client.0.smithi156.stdout:0/659: creat d0/d1a/d76/da7/dd3/fd5 x:0 0 0 2024-07-16T09:55:42.984 INFO:tasks.workunit.client.0.smithi156.stdout:4/688: mkdir d4/db/dc/dcc/df3 0 2024-07-16T09:55:42.984 INFO:tasks.workunit.client.0.smithi156.stdout:0/660: chown d0/d1a/d76/d92/l98 20024119 1 2024-07-16T09:55:42.985 INFO:tasks.workunit.client.0.smithi156.stdout:4/689: rename d4/d23 to d4/d23/d30/dd2/df4 22 2024-07-16T09:55:42.985 INFO:tasks.workunit.client.0.smithi156.stdout:0/661: chown d0/d1a/d4d/d6e 161545838 1 2024-07-16T09:55:42.987 INFO:tasks.workunit.client.0.smithi156.stdout:4/690: unlink d4/db/f16 0 2024-07-16T09:55:42.989 INFO:tasks.workunit.client.0.smithi156.stdout:4/691: mknod d4/db/d8a/cf5 0 2024-07-16T09:55:42.990 INFO:tasks.workunit.client.0.smithi156.stdout:4/692: symlink d4/db/d47/dbe/lf6 0 2024-07-16T09:55:42.992 INFO:tasks.workunit.client.0.smithi156.stdout:4/693: creat d4/d23/ff7 x:0 0 0 2024-07-16T09:55:42.994 INFO:tasks.workunit.client.1.smithi176.stdout:2/709: write dd/d16/d39/d3d/d7e/fc3 [1518300,96815] 0 2024-07-16T09:55:42.995 INFO:tasks.workunit.client.0.smithi156.stdout:4/694: creat d4/db/dc/dcc/dec/ded/ff8 x:0 0 0 2024-07-16T09:55:42.997 INFO:tasks.workunit.client.1.smithi176.stdout:2/710: chown dd/d2e/lad 2778 1 2024-07-16T09:55:42.997 INFO:tasks.workunit.client.0.smithi156.stdout:4/695: read d4/d23/d30/fa1 [138682,52702] 0 2024-07-16T09:55:42.997 INFO:tasks.workunit.client.0.smithi156.stdout:4/696: chown d4/db/d47/dbe/lf6 761 1 2024-07-16T09:55:42.999 INFO:tasks.workunit.client.1.smithi176.stdout:2/711: mkdir dd/d9b/dea 0 2024-07-16T09:55:42.999 INFO:tasks.workunit.client.0.smithi156.stdout:1/789: sync 2024-07-16T09:55:42.999 INFO:tasks.workunit.client.0.smithi156.stdout:9/649: sync 2024-07-16T09:55:42.999 INFO:tasks.workunit.client.0.smithi156.stdout:8/487: sync 2024-07-16T09:55:42.999 INFO:tasks.workunit.client.0.smithi156.stdout:6/541: sync 2024-07-16T09:55:42.999 INFO:tasks.workunit.client.0.smithi156.stdout:3/551: sync 2024-07-16T09:55:43.000 INFO:tasks.workunit.client.0.smithi156.stdout:5/619: sync 2024-07-16T09:55:43.000 INFO:tasks.workunit.client.0.smithi156.stdout:2/650: sync 2024-07-16T09:55:43.000 INFO:tasks.workunit.client.0.smithi156.stdout:7/498: sync 2024-07-16T09:55:43.000 INFO:tasks.workunit.client.1.smithi176.stdout:2/712: stat dd/d49/fb6 0 2024-07-16T09:55:43.000 INFO:tasks.workunit.client.1.smithi176.stdout:9/767: dwrite d5/de/d30/f5e [0,4194304] 0 2024-07-16T09:55:43.002 INFO:tasks.workunit.client.0.smithi156.stdout:4/697: creat d4/d23/ff9 x:0 0 0 2024-07-16T09:55:43.003 INFO:tasks.workunit.client.0.smithi156.stdout:1/790: dread - dc/d14/d8b/db4/fd6 zero size 2024-07-16T09:55:43.003 INFO:tasks.workunit.client.1.smithi176.stdout:2/713: symlink dd/d2e/d35/d4a/leb 0 2024-07-16T09:55:43.006 INFO:tasks.workunit.client.1.smithi176.stdout:2/714: creat dd/d9b/fec x:0 0 0 2024-07-16T09:55:43.008 INFO:tasks.workunit.client.0.smithi156.stdout:1/791: read dc/f28 [4152259,10322] 0 2024-07-16T09:55:43.010 INFO:tasks.workunit.client.0.smithi156.stdout:0/662: dwrite d0/d1a/d4d/d9/d27/d3d/d62/f67 [0,4194304] 0 2024-07-16T09:55:43.019 INFO:tasks.workunit.client.0.smithi156.stdout:1/792: symlink dc/d14/d8b/dcd/l10c 0 2024-07-16T09:55:43.021 INFO:tasks.workunit.client.0.smithi156.stdout:2/651: mknod d0/d5/d29/d2d/d5b/cda 0 2024-07-16T09:55:43.023 INFO:tasks.workunit.client.0.smithi156.stdout:4/698: mknod d4/db/cfa 0 2024-07-16T09:55:43.023 INFO:tasks.workunit.client.0.smithi156.stdout:8/488: dwrite d0/dd/f30 [0,4194304] 0 2024-07-16T09:55:43.025 INFO:tasks.workunit.client.0.smithi156.stdout:9/650: dwrite d2/d5/d61/da2/d98/fc6 [0,4194304] 0 2024-07-16T09:55:43.026 INFO:tasks.workunit.client.0.smithi156.stdout:3/552: creat d8/da/d1d/fc0 x:0 0 0 2024-07-16T09:55:43.026 INFO:tasks.workunit.client.0.smithi156.stdout:7/499: link dc/d44/c60 dc/d3e/d5d/ca8 0 2024-07-16T09:55:43.028 INFO:tasks.workunit.client.1.smithi176.stdout:2/715: dread dd/d16/f3a [4194304,4194304] 0 2024-07-16T09:55:43.035 INFO:tasks.workunit.client.0.smithi156.stdout:6/542: dread d3/d13/d17/d4f/d30/f67 [0,4194304] 0 2024-07-16T09:55:43.035 INFO:tasks.workunit.client.0.smithi156.stdout:1/793: getdents dc/d14/d3e/d55/d58 0 2024-07-16T09:55:43.036 INFO:tasks.workunit.client.1.smithi176.stdout:2/716: dwrite dd/d16/f27 [0,4194304] 0 2024-07-16T09:55:43.039 INFO:tasks.workunit.client.1.smithi176.stdout:2/717: truncate dd/d36/d45/fc4 433837 0 2024-07-16T09:55:43.039 INFO:tasks.workunit.client.0.smithi156.stdout:5/620: dwrite d1b/d99/dae/fba [0,4194304] 0 2024-07-16T09:55:43.039 INFO:tasks.workunit.client.0.smithi156.stdout:0/663: dwrite d0/d1a/d4d/d9/d27/d3d/d62/f63 [4194304,4194304] 0 2024-07-16T09:55:43.040 INFO:tasks.workunit.client.0.smithi156.stdout:2/652: mkdir d0/d5/d29/d2d/d5b/ddb 0 2024-07-16T09:55:43.042 INFO:tasks.workunit.client.1.smithi176.stdout:2/718: unlink dd/d36/d82/f96 0 2024-07-16T09:55:43.044 INFO:tasks.workunit.client.1.smithi176.stdout:2/719: mkdir dd/d36/ded 0 2024-07-16T09:55:43.051 INFO:tasks.workunit.client.1.smithi176.stdout:2/720: link dd/f77 dd/d49/d52/d89/fee 0 2024-07-16T09:55:43.051 INFO:tasks.workunit.client.0.smithi156.stdout:3/553: rename d8/da/d1d/d2c/c27 to d8/da/d28/d43/d71/cc1 0 2024-07-16T09:55:43.053 INFO:tasks.workunit.client.0.smithi156.stdout:4/699: dread d4/f43 [0,4194304] 0 2024-07-16T09:55:43.053 INFO:tasks.workunit.client.0.smithi156.stdout:1/794: fdatasync dc/d14/d3e/d6c/d7a/dcb/fe1 0 2024-07-16T09:55:43.053 INFO:tasks.workunit.client.0.smithi156.stdout:0/664: unlink d0/d1a/d4d/d9/d27/l4b 0 2024-07-16T09:55:43.053 INFO:tasks.workunit.client.1.smithi176.stdout:2/721: write dd/d16/d39/d3d/d7e/fc3 [2326506,66596] 0 2024-07-16T09:55:43.054 INFO:tasks.workunit.client.0.smithi156.stdout:9/651: read d2/d5/d61/f9f [526675,34004] 0 2024-07-16T09:55:43.057 INFO:tasks.workunit.client.0.smithi156.stdout:5/621: rmdir d1b/d1f/d54 39 2024-07-16T09:55:43.058 INFO:tasks.workunit.client.0.smithi156.stdout:3/554: rename d8/da/d28/d40/l72 to d8/da/d21/d80/lc2 0 2024-07-16T09:55:43.058 INFO:tasks.workunit.client.0.smithi156.stdout:4/700: dread - d4/db/dc/dcc/dec/ded/ff8 zero size 2024-07-16T09:55:43.058 INFO:tasks.workunit.client.0.smithi156.stdout:1/795: stat dc/d14/d3e/d55/d57 0 2024-07-16T09:55:43.059 INFO:tasks.workunit.client.0.smithi156.stdout:6/543: symlink d3/d13/d17/d4f/d94/daa/lb1 0 2024-07-16T09:55:43.060 INFO:tasks.workunit.client.0.smithi156.stdout:3/555: chown d8/da/d28/d43/fbe 128 1 2024-07-16T09:55:43.060 INFO:tasks.workunit.client.0.smithi156.stdout:0/665: mknod d0/d1a/d4d/d9/d68/cd6 0 2024-07-16T09:55:43.061 INFO:tasks.workunit.client.0.smithi156.stdout:7/500: link dc/d63/l64 dc/d13/d36/la9 0 2024-07-16T09:55:43.063 INFO:tasks.workunit.client.0.smithi156.stdout:3/556: stat d8/da/d1d/d2c/d7d/fa1 0 2024-07-16T09:55:43.067 INFO:tasks.workunit.client.0.smithi156.stdout:6/544: dread d3/d13/d17/f8c [0,4194304] 0 2024-07-16T09:55:43.067 INFO:tasks.workunit.client.0.smithi156.stdout:5/622: rename d1b/d1f/fcb to d1b/d1f/d45/d52/d2d/d96/fd8 0 2024-07-16T09:55:43.068 INFO:tasks.workunit.client.0.smithi156.stdout:7/501: mkdir dc/d13/d20/d4d/daa 0 2024-07-16T09:55:43.068 INFO:tasks.workunit.client.0.smithi156.stdout:3/557: creat d8/da/d21/d80/d99/fc3 x:0 0 0 2024-07-16T09:55:43.070 INFO:tasks.workunit.client.0.smithi156.stdout:9/652: dwrite d2/d5/d61/da2/d98/fc6 [0,4194304] 0 2024-07-16T09:55:43.071 INFO:tasks.workunit.client.0.smithi156.stdout:5/623: chown d1b/d1f/d45/d52/d43/c77 1 1 2024-07-16T09:55:43.072 INFO:tasks.workunit.client.1.smithi176.stdout:2/722: dwrite dd/d16/f32 [0,4194304] 0 2024-07-16T09:55:43.074 INFO:tasks.workunit.client.0.smithi156.stdout:7/502: creat dc/d3e/d3f/fab x:0 0 0 2024-07-16T09:55:43.075 INFO:tasks.workunit.client.1.smithi176.stdout:2/723: fdatasync f3 0 2024-07-16T09:55:43.077 INFO:tasks.workunit.client.1.smithi176.stdout:2/724: fsync dd/d16/d39/d3d/d7e/fc3 0 2024-07-16T09:55:43.078 INFO:tasks.workunit.client.1.smithi176.stdout:2/725: fdatasync dd/d2e/d35/d4a/fd9 0 2024-07-16T09:55:43.078 INFO:tasks.workunit.client.0.smithi156.stdout:6/545: creat d3/d13/d17/d4f/d4d/d3b/d77/fb2 x:0 0 0 2024-07-16T09:55:43.079 INFO:tasks.workunit.client.0.smithi156.stdout:4/701: dread d4/d23/d30/d33/f50 [0,4194304] 0 2024-07-16T09:55:43.079 INFO:tasks.workunit.client.0.smithi156.stdout:0/666: dwrite d0/d1a/d4d/d9/fe [0,4194304] 0 2024-07-16T09:55:43.080 INFO:tasks.workunit.client.0.smithi156.stdout:3/558: mkdir d8/da/d1d/d6f/db3/dc4 0 2024-07-16T09:55:43.083 INFO:tasks.workunit.client.1.smithi176.stdout:2/726: creat dd/fef x:0 0 0 2024-07-16T09:55:43.083 INFO:tasks.workunit.client.0.smithi156.stdout:9/653: creat d2/d5/d16/d22/d2b/d74/dc2/fcd x:0 0 0 2024-07-16T09:55:43.087 INFO:tasks.workunit.client.0.smithi156.stdout:2/653: truncate d0/d5/d29/d2d/d5b/f9c 232915 0 2024-07-16T09:55:43.087 INFO:tasks.workunit.client.0.smithi156.stdout:1/796: dwrite dc/d14/f48 [0,4194304] 0 2024-07-16T09:55:43.087 INFO:tasks.workunit.client.0.smithi156.stdout:3/559: symlink d8/da/d21/lc5 0 2024-07-16T09:55:43.089 INFO:tasks.workunit.client.0.smithi156.stdout:0/667: write d0/d1a/d4d/d9/d27/d3d/d62/f8e [1105375,6267] 0 2024-07-16T09:55:43.092 INFO:tasks.workunit.client.1.smithi176.stdout:2/727: dwrite dd/de/fa8 [0,4194304] 0 2024-07-16T09:55:43.096 INFO:tasks.workunit.client.0.smithi156.stdout:9/654: write d2/d5/d16/d22/d2b/d3d/f6d [4574509,5160] 0 2024-07-16T09:55:43.096 INFO:tasks.workunit.client.0.smithi156.stdout:4/702: symlink d4/db/dc/dcc/d58/dd9/lfb 0 2024-07-16T09:55:43.096 INFO:tasks.workunit.client.0.smithi156.stdout:2/654: write d0/d13/d6e/fd3 [619161,88027] 0 2024-07-16T09:55:43.096 INFO:tasks.workunit.client.0.smithi156.stdout:1/797: rmdir dc/d6b/d8a 39 2024-07-16T09:55:43.100 INFO:tasks.workunit.client.0.smithi156.stdout:5/624: dwrite d1b/d1f/d45/d52/d43/d85/f75 [0,4194304] 0 2024-07-16T09:55:43.103 INFO:tasks.workunit.client.0.smithi156.stdout:5/625: fsync d1b/d1f/d45/d52/d43/f58 0 2024-07-16T09:55:43.103 INFO:tasks.workunit.client.0.smithi156.stdout:9/655: chown d2/d5/d16/d22/d2b/d74/dc2/dc4/d3a/fab 103675 1 2024-07-16T09:55:43.104 INFO:tasks.workunit.client.0.smithi156.stdout:1/798: truncate dc/d14/d1e/f77 714711 0 2024-07-16T09:55:43.106 INFO:tasks.workunit.client.0.smithi156.stdout:9/656: fdatasync d2/d5/d16/d22/d2b/d3d/d3c/d7f/f5e 0 2024-07-16T09:55:43.109 INFO:tasks.workunit.client.0.smithi156.stdout:1/799: fdatasync dc/d24/fc2 0 2024-07-16T09:55:43.111 INFO:tasks.workunit.client.0.smithi156.stdout:7/503: fsync dc/d3e/d3f/fab 0 2024-07-16T09:55:43.112 INFO:tasks.workunit.client.0.smithi156.stdout:2/655: rmdir d0/d5/d29 39 2024-07-16T09:55:43.112 INFO:tasks.workunit.client.0.smithi156.stdout:0/668: rename d0/d1a/d4d/d9/d27/d8d/dbd to d0/d1a/d4d/d9/d68/dcb/dd7 0 2024-07-16T09:55:43.113 INFO:tasks.workunit.client.0.smithi156.stdout:3/560: creat d8/da/d1d/fc6 x:0 0 0 2024-07-16T09:55:43.116 INFO:tasks.workunit.client.0.smithi156.stdout:4/703: getdents d4 0 2024-07-16T09:55:43.116 INFO:tasks.workunit.client.0.smithi156.stdout:5/626: dwrite d1b/d1f/d45/d52/d49/f60 [4194304,4194304] 0 2024-07-16T09:55:43.117 INFO:tasks.workunit.client.0.smithi156.stdout:0/669: rmdir d0/d1a/d76/d83 39 2024-07-16T09:55:43.118 INFO:tasks.workunit.client.0.smithi156.stdout:7/504: mkdir dc/d3e/dac 0 2024-07-16T09:55:43.119 INFO:tasks.workunit.client.0.smithi156.stdout:2/656: unlink d0/d5/d29/d2d/c4f 0 2024-07-16T09:55:43.120 INFO:tasks.workunit.client.0.smithi156.stdout:0/670: fsync d0/d1a/d4d/d9/fe 0 2024-07-16T09:55:43.122 INFO:tasks.workunit.client.0.smithi156.stdout:2/657: stat d0/d13/d6e 0 2024-07-16T09:55:43.122 INFO:tasks.workunit.client.0.smithi156.stdout:4/704: chown d4/d23/d30/f25 6370488 1 2024-07-16T09:55:43.123 INFO:tasks.workunit.client.1.smithi176.stdout:2/728: fdatasync dd/de/fa8 0 2024-07-16T09:55:43.123 INFO:tasks.workunit.client.0.smithi156.stdout:9/657: getdents d2/d5/d16 0 2024-07-16T09:55:43.124 INFO:tasks.workunit.client.1.smithi176.stdout:2/729: chown dd/d49/d52/f91 427 1 2024-07-16T09:55:43.124 INFO:tasks.workunit.client.0.smithi156.stdout:1/800: truncate dc/d14/d3e/d6c/f73 2579148 0 2024-07-16T09:55:43.126 INFO:tasks.workunit.client.0.smithi156.stdout:7/505: mknod dc/d13/cad 0 2024-07-16T09:55:43.126 INFO:tasks.workunit.client.0.smithi156.stdout:4/705: chown d4/db/dc/dcc/dec/ded/d87/fa0 18004606 1 2024-07-16T09:55:43.130 INFO:tasks.workunit.client.0.smithi156.stdout:4/706: write d4/d23/f53 [7151706,104208] 0 2024-07-16T09:55:43.132 INFO:tasks.workunit.client.0.smithi156.stdout:0/671: write d0/d1a/d4d/d9/f8f [271065,68283] 0 2024-07-16T09:55:43.132 INFO:tasks.workunit.client.0.smithi156.stdout:2/658: rename d0/d5/d29/d2d/d5b/d62/d74 to d0/d5/d29/d2d/d5b/d5f/ddc 0 2024-07-16T09:55:43.133 INFO:tasks.workunit.client.0.smithi156.stdout:3/561: dwrite d8/da/d1d/d2c/d33/fbf [0,4194304] 0 2024-07-16T09:55:43.134 INFO:tasks.workunit.client.0.smithi156.stdout:3/562: write d8/f62 [925095,108738] 0 2024-07-16T09:55:43.139 INFO:tasks.workunit.client.1.smithi176.stdout:2/730: dread dd/d16/d39/d3d/f75 [0,4194304] 0 2024-07-16T09:55:43.140 INFO:tasks.workunit.client.0.smithi156.stdout:1/801: truncate dc/d14/f2d 5549434 0 2024-07-16T09:55:43.140 INFO:tasks.workunit.client.1.smithi176.stdout:2/731: write dd/d49/d52/f79 [729692,36700] 0 2024-07-16T09:55:43.142 INFO:tasks.workunit.client.0.smithi156.stdout:3/563: read d8/da/d28/d43/fa6 [2034666,22917] 0 2024-07-16T09:55:43.142 INFO:tasks.workunit.client.0.smithi156.stdout:7/506: mkdir dc/d3e/d3f/d51/d8a/dae 0 2024-07-16T09:55:43.143 INFO:tasks.workunit.client.0.smithi156.stdout:4/707: symlink d4/d23/d52/d71/lfc 0 2024-07-16T09:55:43.144 INFO:tasks.workunit.client.0.smithi156.stdout:0/672: rmdir d0/d1a/d4d/d61 39 2024-07-16T09:55:43.146 INFO:tasks.workunit.client.0.smithi156.stdout:5/627: dread fc [0,4194304] 0 2024-07-16T09:55:43.146 INFO:tasks.workunit.client.0.smithi156.stdout:1/802: truncate dc/d14/d8b/db4/fd6 430 0 2024-07-16T09:55:43.148 INFO:tasks.workunit.client.0.smithi156.stdout:2/659: rename d0/d13/lb8 to d0/d5/d29/d2d/d5b/d5f/d89/ldd 0 2024-07-16T09:55:43.149 INFO:tasks.workunit.client.0.smithi156.stdout:5/628: stat d1b/d1f/d42/d6d/c6f 0 2024-07-16T09:55:43.149 INFO:tasks.workunit.client.0.smithi156.stdout:9/658: dread d2/d5/f70 [0,4194304] 0 2024-07-16T09:55:43.149 INFO:tasks.workunit.client.1.smithi176.stdout:2/732: dwrite dd/d36/f94 [0,4194304] 0 2024-07-16T09:55:43.152 INFO:tasks.workunit.client.0.smithi156.stdout:1/803: mkdir dc/d14/d8b/db4/df2/d10d 0 2024-07-16T09:55:43.154 INFO:tasks.workunit.client.1.smithi176.stdout:2/733: symlink dd/d36/ded/lf0 0 2024-07-16T09:55:43.155 INFO:tasks.workunit.client.1.smithi176.stdout:2/734: write dd/d2e/d35/f3f [1251649,33758] 0 2024-07-16T09:55:43.158 INFO:tasks.workunit.client.0.smithi156.stdout:4/708: dwrite d4/db/dc/dcc/dec/ded/ff8 [0,4194304] 0 2024-07-16T09:55:43.158 INFO:tasks.workunit.client.1.smithi176.stdout:2/735: truncate dd/d16/d39/f84 109491 0 2024-07-16T09:55:43.161 INFO:tasks.workunit.client.0.smithi156.stdout:1/804: creat dc/d6b/d8a/f10e x:0 0 0 2024-07-16T09:55:43.161 INFO:tasks.workunit.client.0.smithi156.stdout:4/709: chown d4/d23/d52/fe5 1985776 1 2024-07-16T09:55:43.163 INFO:tasks.workunit.client.0.smithi156.stdout:1/805: fsync dc/d24/fb1 0 2024-07-16T09:55:43.164 INFO:tasks.workunit.client.0.smithi156.stdout:5/629: dread f14 [0,4194304] 0 2024-07-16T09:55:43.164 INFO:tasks.workunit.client.0.smithi156.stdout:4/710: readlink d4/db/dc/dcc/d58/d8f/lab 0 2024-07-16T09:55:43.165 INFO:tasks.workunit.client.1.smithi176.stdout:2/736: mknod dd/d36/d45/de3/cf1 0 2024-07-16T09:55:43.165 INFO:tasks.workunit.client.0.smithi156.stdout:9/659: dread d2/d5/d61/da2/d98/fc6 [0,4194304] 0 2024-07-16T09:55:43.165 INFO:tasks.workunit.client.0.smithi156.stdout:2/660: dread d0/d5/d76/f8a [0,4194304] 0 2024-07-16T09:55:43.167 INFO:tasks.workunit.client.0.smithi156.stdout:1/806: symlink dc/d14/d8b/db4/l10f 0 2024-07-16T09:55:43.168 INFO:tasks.workunit.client.0.smithi156.stdout:3/564: truncate d8/da/d28/d43/f5b 3568618 0 2024-07-16T09:55:43.170 INFO:tasks.workunit.client.1.smithi176.stdout:2/737: link dd/d36/la1 dd/d49/lf2 0 2024-07-16T09:55:43.172 INFO:tasks.workunit.client.0.smithi156.stdout:1/807: dread - dc/d14/d3e/d55/d83/f108 zero size 2024-07-16T09:55:43.172 INFO:tasks.workunit.client.0.smithi156.stdout:3/565: write d8/da/d13/fb6 [4451939,40078] 0 2024-07-16T09:55:43.173 INFO:tasks.workunit.client.0.smithi156.stdout:9/660: mknod d2/d5/d16/d22/d2b/d74/dc2/dc4/d66/cce 0 2024-07-16T09:55:43.174 INFO:tasks.workunit.client.1.smithi176.stdout:2/738: creat dd/d2e/ff3 x:0 0 0 2024-07-16T09:55:43.176 INFO:tasks.workunit.client.1.smithi176.stdout:2/739: fdatasync dd/d36/f61 0 2024-07-16T09:55:43.177 INFO:tasks.workunit.client.1.smithi176.stdout:2/740: fdatasync dd/d2e/d35/fac 0 2024-07-16T09:55:43.177 INFO:tasks.workunit.client.0.smithi156.stdout:1/808: rmdir dc/d24 39 2024-07-16T09:55:43.180 INFO:tasks.workunit.client.0.smithi156.stdout:4/711: dwrite d4/db/dc/fcb [0,4194304] 0 2024-07-16T09:55:43.192 INFO:tasks.workunit.client.0.smithi156.stdout:5/630: dwrite d1b/d1f/d54/dbe/fc3 [0,4194304] 0 2024-07-16T09:55:43.192 INFO:tasks.workunit.client.1.smithi176.stdout:2/741: dwrite dd/d36/f2f [0,4194304] 0 2024-07-16T09:55:43.194 INFO:tasks.workunit.client.0.smithi156.stdout:4/712: chown d4/db/d8a/cf5 1902 1 2024-07-16T09:55:43.200 INFO:tasks.workunit.client.1.smithi176.stdout:2/742: link dd/d2e/d35/d59/d66/dc8/ce7 dd/d16/d39/cf4 0 2024-07-16T09:55:43.200 INFO:tasks.workunit.client.1.smithi176.stdout:2/743: dread - dd/d16/fe4 zero size 2024-07-16T09:55:43.201 INFO:tasks.workunit.client.1.smithi176.stdout:2/744: write dd/d2e/f37 [4672740,42627] 0 2024-07-16T09:55:43.204 INFO:tasks.workunit.client.0.smithi156.stdout:2/661: dwrite d0/d13/f8d [0,4194304] 0 2024-07-16T09:55:43.204 INFO:tasks.workunit.client.0.smithi156.stdout:9/661: dwrite d2/d5/f44 [0,4194304] 0 2024-07-16T09:55:43.207 INFO:tasks.workunit.client.0.smithi156.stdout:1/809: readlink dc/d14/d8b/dcd/dd2/df9/lfe 0 2024-07-16T09:55:43.207 INFO:tasks.workunit.client.1.smithi176.stdout:2/745: creat dd/d49/d52/d89/ff5 x:0 0 0 2024-07-16T09:55:43.208 INFO:tasks.workunit.client.0.smithi156.stdout:3/566: dwrite d8/d14/f30 [4194304,4194304] 0 2024-07-16T09:55:43.211 INFO:tasks.workunit.client.0.smithi156.stdout:4/713: rename d4/f5 to d4/db/d8a/ffd 0 2024-07-16T09:55:43.211 INFO:tasks.workunit.client.0.smithi156.stdout:3/567: stat d8/da/f94 0 2024-07-16T09:55:43.216 INFO:tasks.workunit.client.0.smithi156.stdout:3/568: creat d8/d14/fc7 x:0 0 0 2024-07-16T09:55:43.217 INFO:tasks.workunit.client.1.smithi176.stdout:2/746: creat dd/d49/ff6 x:0 0 0 2024-07-16T09:55:43.218 INFO:tasks.workunit.client.0.smithi156.stdout:5/631: rename d1b/d99/dae/db9 to d1b/d1f/d45/d52/d43/d85/d97/d37/d88/dc7/dd9 0 2024-07-16T09:55:43.220 INFO:tasks.workunit.client.1.smithi176.stdout:2/747: mknod dd/d2e/d35/d59/cf7 0 2024-07-16T09:55:43.220 INFO:tasks.workunit.client.0.smithi156.stdout:2/662: truncate d0/d13/d6e/fd3 728496 0 2024-07-16T09:55:43.221 INFO:tasks.workunit.client.0.smithi156.stdout:3/569: stat d8/da/d1d/db2/lbb 0 2024-07-16T09:55:43.222 INFO:tasks.workunit.client.0.smithi156.stdout:9/662: dwrite d2/d5/d16/d22/d6b/fcc [0,4194304] 0 2024-07-16T09:55:43.226 INFO:tasks.workunit.client.0.smithi156.stdout:2/663: mkdir d0/d13/d17/d1c/d86/dde 0 2024-07-16T09:55:43.227 INFO:tasks.workunit.client.0.smithi156.stdout:1/810: rename dc/d6b/d8a/c94 to dc/d14/d8b/db4/df2/c110 0 2024-07-16T09:55:43.228 INFO:tasks.workunit.client.0.smithi156.stdout:4/714: dwrite f2 [0,4194304] 0 2024-07-16T09:55:43.228 INFO:tasks.workunit.client.1.smithi176.stdout:2/748: dwrite dd/d49/ff6 [0,4194304] 0 2024-07-16T09:55:43.231 INFO:tasks.workunit.client.0.smithi156.stdout:9/663: mknod d2/d5/d61/da2/d98/dbf/ccf 0 2024-07-16T09:55:43.234 INFO:tasks.workunit.client.1.smithi176.stdout:2/749: readlink dd/d36/l26 0 2024-07-16T09:55:43.235 INFO:tasks.workunit.client.0.smithi156.stdout:5/632: dwrite d1b/d1f/fa9 [0,4194304] 0 2024-07-16T09:55:43.235 INFO:tasks.workunit.client.0.smithi156.stdout:9/664: fdatasync d2/d5/d16/d22/d2b/f8b 0 2024-07-16T09:55:43.240 INFO:tasks.workunit.client.0.smithi156.stdout:3/570: dwrite d8/da/f32 [0,4194304] 0 2024-07-16T09:55:43.244 INFO:tasks.workunit.client.1.smithi176.stdout:2/750: rename dd/d2e/ccb to dd/d2e/d35/d4a/cf8 0 2024-07-16T09:55:43.244 INFO:tasks.workunit.client.0.smithi156.stdout:4/715: symlink d4/db/dc/dcc/dec/ded/lfe 0 2024-07-16T09:55:43.245 INFO:tasks.workunit.client.1.smithi176.stdout:2/751: write dd/d36/d82/fa9 [1002949,89811] 0 2024-07-16T09:55:43.246 INFO:tasks.workunit.client.0.smithi156.stdout:5/633: truncate d1b/d1f/d45/d7f/fa0 1483814 0 2024-07-16T09:55:43.249 INFO:tasks.workunit.client.0.smithi156.stdout:5/634: write f19 [431854,100319] 0 2024-07-16T09:55:43.251 INFO:tasks.workunit.client.0.smithi156.stdout:4/716: creat d4/db/dc/dcc/fff x:0 0 0 2024-07-16T09:55:43.252 INFO:tasks.workunit.client.0.smithi156.stdout:2/664: creat d0/d13/d17/fdf x:0 0 0 2024-07-16T09:55:43.252 INFO:tasks.workunit.client.0.smithi156.stdout:3/571: mknod d8/da/d1d/d90/cc8 0 2024-07-16T09:55:43.255 INFO:tasks.workunit.client.0.smithi156.stdout:8/489: sync 2024-07-16T09:55:43.256 INFO:tasks.workunit.client.1.smithi176.stdout:2/752: dwrite dd/d49/d52/f79 [0,4194304] 0 2024-07-16T09:55:43.256 INFO:tasks.workunit.client.0.smithi156.stdout:4/717: mkdir d4/db/dc/dcc/d58/d100 0 2024-07-16T09:55:43.257 INFO:tasks.workunit.client.0.smithi156.stdout:5/635: creat d1b/d1f/d42/d6d/d9f/fda x:0 0 0 2024-07-16T09:55:43.258 INFO:tasks.workunit.client.0.smithi156.stdout:1/811: dwrite dc/d14/d8b/db4/df2/f101 [0,4194304] 0 2024-07-16T09:55:43.258 INFO:tasks.workunit.client.0.smithi156.stdout:8/490: dread - d0/d11/d4a/d55/d93/f9c zero size 2024-07-16T09:55:43.258 INFO:tasks.workunit.client.1.smithi176.stdout:2/753: write dd/d49/fb6 [590486,81820] 0 2024-07-16T09:55:43.261 INFO:tasks.workunit.client.1.smithi176.stdout:2/754: rmdir dd/d36/d45 39 2024-07-16T09:55:43.262 INFO:tasks.workunit.client.0.smithi156.stdout:3/572: chown d8/da/d28/d43/d71/cc1 82 1 2024-07-16T09:55:43.264 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:55:42 smithi176 ceph-mon[36626]: fsmap cephfs:2 {0=cephfs.smithi156.rdibys=up:active,1=cephfs.smithi176.dvvbud=up:active} 2 up:standby-replay 2024-07-16T09:55:43.264 INFO:tasks.workunit.client.1.smithi176.stdout:2/755: read dd/d36/d45/fc4 [426927,53426] 0 2024-07-16T09:55:43.265 INFO:tasks.workunit.client.0.smithi156.stdout:5/636: fsync d1b/d1f/d45/d52/f39 0 2024-07-16T09:55:43.265 INFO:tasks.workunit.client.0.smithi156.stdout:3/573: chown d8/da/d21/d80/d99/fa3 312887 1 2024-07-16T09:55:43.266 INFO:tasks.workunit.client.0.smithi156.stdout:2/665: unlink d0/d13/d6e/fd3 0 2024-07-16T09:55:43.267 INFO:tasks.workunit.client.0.smithi156.stdout:8/491: mknod d0/d11/d62/cac 0 2024-07-16T09:55:43.267 INFO:tasks.workunit.client.1.smithi176.stdout:2/756: unlink dd/d49/d52/f91 0 2024-07-16T09:55:43.268 INFO:tasks.workunit.client.1.smithi176.stdout:2/757: stat dd/d16/d39/f4f 0 2024-07-16T09:55:43.272 INFO:tasks.workunit.client.1.smithi176.stdout:2/758: mkdir dd/d2e/d35/d4a/df9 0 2024-07-16T09:55:43.272 INFO:tasks.workunit.client.1.smithi176.stdout:4/568: sync 2024-07-16T09:55:43.272 INFO:tasks.workunit.client.1.smithi176.stdout:0/694: sync 2024-07-16T09:55:43.273 INFO:tasks.workunit.client.1.smithi176.stdout:0/695: fsync d5/d1c/db1/fcd 0 2024-07-16T09:55:43.274 INFO:tasks.workunit.client.0.smithi156.stdout:1/812: rename dc/d14/d3e/d45 to dc/d14/d8b/db4/d111 0 2024-07-16T09:55:43.275 INFO:tasks.workunit.client.1.smithi176.stdout:2/759: truncate dd/f3c 9070944 0 2024-07-16T09:55:43.275 INFO:tasks.workunit.client.0.smithi156.stdout:1/813: readlink dc/d14/le2 0 2024-07-16T09:55:43.276 INFO:tasks.workunit.client.1.smithi176.stdout:4/569: read da/d14/d40/d5d/d73/f61 [2392388,78937] 0 2024-07-16T09:55:43.277 INFO:tasks.workunit.client.0.smithi156.stdout:2/666: dread d0/d5/d29/d2d/d5b/d62/fb3 [0,4194304] 0 2024-07-16T09:55:43.277 INFO:tasks.workunit.client.1.smithi176.stdout:0/696: symlink d5/d1c/d5a/leb 0 2024-07-16T09:55:43.278 INFO:tasks.workunit.client.0.smithi156.stdout:4/718: rename d4/db/dc/dcc/dec/ded/daf/fb5 to d4/db/dc/dcc/d58/d8f/f101 0 2024-07-16T09:55:43.278 INFO:tasks.workunit.client.1.smithi176.stdout:2/760: rmdir dd/d49/d52/d89 39 2024-07-16T09:55:43.279 INFO:tasks.workunit.client.0.smithi156.stdout:5/637: dwrite d1b/d1f/d45/faf [0,4194304] 0 2024-07-16T09:55:43.282 INFO:tasks.workunit.client.0.smithi156.stdout:3/574: dread f3 [0,4194304] 0 2024-07-16T09:55:43.282 INFO:tasks.workunit.client.0.smithi156.stdout:4/719: chown d4/db/dc/dcc/d58/dd9/lfb 23805 1 2024-07-16T09:55:43.282 INFO:tasks.workunit.client.1.smithi176.stdout:4/570: unlink da/d3d/d17/d48/f80 0 2024-07-16T09:55:43.282 INFO:tasks.workunit.client.0.smithi156.stdout:5/638: fsync d1b/d1f/d45/d52/d49/f60 0 2024-07-16T09:55:43.285 INFO:tasks.workunit.client.0.smithi156.stdout:3/575: write d8/d14/fb7 [138679,79703] 0 2024-07-16T09:55:43.287 INFO:tasks.workunit.client.1.smithi176.stdout:2/761: mkdir dd/dcd/dfa 0 2024-07-16T09:55:43.288 INFO:tasks.workunit.client.1.smithi176.stdout:2/762: chown dd/d2e/d35 234110 1 2024-07-16T09:55:43.290 INFO:tasks.workunit.client.1.smithi176.stdout:4/571: creat da/d14/d40/d5d/d73/fb1 x:0 0 0 2024-07-16T09:55:43.290 INFO:tasks.workunit.client.0.smithi156.stdout:1/814: creat dc/d14/d3e/f112 x:0 0 0 2024-07-16T09:55:43.291 INFO:tasks.workunit.client.0.smithi156.stdout:1/815: chown dc/d14/d8b/db4/fe0 1 1 2024-07-16T09:55:43.291 INFO:tasks.workunit.client.1.smithi176.stdout:0/697: dwrite d5/d25/d82/fa7 [0,4194304] 0 2024-07-16T09:55:43.293 INFO:tasks.workunit.client.1.smithi176.stdout:4/572: write da/d3d/d17/d48/faa [3777992,108271] 0 2024-07-16T09:55:43.295 INFO:tasks.workunit.client.0.smithi156.stdout:1/816: symlink dc/d14/d3e/d55/l113 0 2024-07-16T09:55:43.296 INFO:tasks.workunit.client.1.smithi176.stdout:2/763: fdatasync dd/d49/f55 0 2024-07-16T09:55:43.297 INFO:tasks.workunit.client.1.smithi176.stdout:0/698: rename d5/d39/da3/dbe to d5/d25/d82/dec 0 2024-07-16T09:55:43.298 INFO:tasks.workunit.client.1.smithi176.stdout:2/764: truncate f3 31571 0 2024-07-16T09:55:43.298 INFO:tasks.workunit.client.1.smithi176.stdout:0/699: stat d5/d1c/d24/d35/d50/d58 0 2024-07-16T09:55:43.300 INFO:tasks.workunit.client.1.smithi176.stdout:4/573: link da/dc/d4b/fa7 da/dc/d10/d96/d69/fb2 0 2024-07-16T09:55:43.301 INFO:tasks.workunit.client.0.smithi156.stdout:1/817: dread dc/d14/d3e/d55/f5c [0,4194304] 0 2024-07-16T09:55:43.301 INFO:tasks.workunit.client.1.smithi176.stdout:4/574: read da/d3d/fa2 [1940850,126437] 0 2024-07-16T09:55:43.302 INFO:tasks.workunit.client.0.smithi156.stdout:1/818: write dc/d14/d3e/d55/f102 [925542,83363] 0 2024-07-16T09:55:43.303 INFO:tasks.workunit.client.1.smithi176.stdout:0/700: mknod d5/d39/dde/ced 0 2024-07-16T09:55:43.303 INFO:tasks.workunit.client.0.smithi156.stdout:1/819: chown dc/d14/d8b/c8d 13849 1 2024-07-16T09:55:43.304 INFO:tasks.workunit.client.1.smithi176.stdout:0/701: fsync d5/d1c/d24/d35/d50/d58/f60 0 2024-07-16T09:55:43.305 INFO:tasks.workunit.client.1.smithi176.stdout:4/575: mknod da/d41/d5b/cb3 0 2024-07-16T09:55:43.308 INFO:tasks.workunit.client.0.smithi156.stdout:1/820: mkdir dc/d14/d3e/d114 0 2024-07-16T09:55:43.310 INFO:tasks.workunit.client.1.smithi176.stdout:4/576: getdents da/d3d/dac 0 2024-07-16T09:55:43.311 INFO:tasks.workunit.client.1.smithi176.stdout:0/702: link d5/f2f d5/d39/da3/fee 0 2024-07-16T09:55:43.312 INFO:tasks.workunit.client.0.smithi156.stdout:1/821: truncate dc/d14/d3e/d55/f5c 207775 0 2024-07-16T09:55:43.312 INFO:tasks.workunit.client.1.smithi176.stdout:0/703: fsync d5/d39/f9f 0 2024-07-16T09:55:43.313 INFO:tasks.workunit.client.1.smithi176.stdout:0/704: read d5/d2c/d6c/f5e [271170,62808] 0 2024-07-16T09:55:43.315 INFO:tasks.workunit.client.0.smithi156.stdout:8/492: dread d0/dd/f10 [4194304,4194304] 0 2024-07-16T09:55:43.315 INFO:tasks.workunit.client.1.smithi176.stdout:4/577: rename da/d14 to da/dc/d10/d96/d9b/db4 0 2024-07-16T09:55:43.316 INFO:tasks.workunit.client.0.smithi156.stdout:1/822: dread dc/d14/d8b/db4/df2/f101 [0,4194304] 0 2024-07-16T09:55:43.317 INFO:tasks.workunit.client.1.smithi176.stdout:2/765: write dd/d49/d52/d89/fee [428905,13994] 0 2024-07-16T09:55:43.322 INFO:tasks.workunit.client.1.smithi176.stdout:4/578: dread - da/dc/d10/d28/f7f zero size 2024-07-16T09:55:43.323 INFO:tasks.workunit.client.1.smithi176.stdout:2/766: unlink dd/d49/d52/d7c/fb4 0 2024-07-16T09:55:43.325 INFO:tasks.workunit.client.1.smithi176.stdout:4/579: mknod da/d41/d5b/cb5 0 2024-07-16T09:55:43.328 INFO:tasks.workunit.client.1.smithi176.stdout:4/580: symlink da/dc/d4b/lb6 0 2024-07-16T09:55:43.331 INFO:tasks.workunit.client.1.smithi176.stdout:2/767: write dd/d16/d39/f84 [146902,64899] 0 2024-07-16T09:55:43.332 INFO:tasks.workunit.client.0.smithi156.stdout:6/546: sync 2024-07-16T09:55:43.333 INFO:tasks.workunit.client.1.smithi176.stdout:2/768: unlink dd/d36/l25 0 2024-07-16T09:55:43.334 INFO:tasks.workunit.client.1.smithi176.stdout:2/769: chown dd/d16 37525 1 2024-07-16T09:55:43.334 INFO:tasks.workunit.client.1.smithi176.stdout:4/581: dwrite da/d3d/d33/f3a [0,4194304] 0 2024-07-16T09:55:43.335 INFO:tasks.workunit.client.1.smithi176.stdout:0/705: dread d5/f10 [0,4194304] 0 2024-07-16T09:55:43.336 INFO:tasks.workunit.client.1.smithi176.stdout:4/582: truncate da/d3d/d17/d54/fa3 780529 0 2024-07-16T09:55:43.337 INFO:tasks.workunit.client.1.smithi176.stdout:2/770: creat dd/d2e/d35/d59/d66/ffb x:0 0 0 2024-07-16T09:55:43.338 INFO:tasks.workunit.client.1.smithi176.stdout:4/583: chown da/dc/d10/d96/d69/fb0 266381 1 2024-07-16T09:55:43.338 INFO:tasks.workunit.client.1.smithi176.stdout:0/706: truncate d5/f2f 1195011 0 2024-07-16T09:55:43.339 INFO:tasks.workunit.client.1.smithi176.stdout:0/707: write d5/d39/da3/fd9 [1896000,122323] 0 2024-07-16T09:55:43.340 INFO:tasks.workunit.client.1.smithi176.stdout:4/584: creat da/dc/d10/d28/fb7 x:0 0 0 2024-07-16T09:55:43.341 INFO:tasks.workunit.client.1.smithi176.stdout:0/708: chown d5/d1c/d24/d35/d50/d58/d63/db4/dcc/cd1 105193 1 2024-07-16T09:55:43.342 INFO:tasks.workunit.client.1.smithi176.stdout:0/709: dread - d5/d1c/d24/d35/fc8 zero size 2024-07-16T09:55:43.344 INFO:tasks.workunit.client.1.smithi176.stdout:0/710: readlink d5/d2c/d6c/d83/d84/l97 0 2024-07-16T09:55:43.346 INFO:tasks.workunit.client.1.smithi176.stdout:4/585: getdents da/d3d 0 2024-07-16T09:55:43.350 INFO:tasks.workunit.client.1.smithi176.stdout:4/586: chown da/dc/d10/d96/d9b/db4/d4a/l60 1960831 1 2024-07-16T09:55:43.350 INFO:tasks.workunit.client.1.smithi176.stdout:2/771: truncate dd/d2e/d35/d59/d66/f6e 1962809 0 2024-07-16T09:55:43.350 INFO:tasks.workunit.client.1.smithi176.stdout:4/587: mknod da/dc/d10/cb8 0 2024-07-16T09:55:43.350 INFO:tasks.workunit.client.1.smithi176.stdout:2/772: unlink dd/d16/d39/d3d/f87 0 2024-07-16T09:55:43.352 INFO:tasks.workunit.client.1.smithi176.stdout:0/711: truncate d5/d39/f7e 3651769 0 2024-07-16T09:55:43.352 INFO:tasks.workunit.client.1.smithi176.stdout:2/773: readlink dd/d16/d39/d3d/l93 0 2024-07-16T09:55:43.356 INFO:tasks.workunit.client.1.smithi176.stdout:2/774: creat dd/dcd/dfa/ffc x:0 0 0 2024-07-16T09:55:43.356 INFO:tasks.workunit.client.1.smithi176.stdout:4/588: dwrite da/dc/d10/d96/d9b/db4/d40/d5d/f85 [0,4194304] 0 2024-07-16T09:55:43.358 INFO:tasks.workunit.client.1.smithi176.stdout:2/775: write dd/d49/f78 [587173,51356] 0 2024-07-16T09:55:43.359 INFO:tasks.workunit.client.1.smithi176.stdout:4/589: fsync da/dc/d10/f59 0 2024-07-16T09:55:43.359 INFO:tasks.workunit.client.1.smithi176.stdout:2/776: write dd/fef [586013,104393] 0 2024-07-16T09:55:43.361 INFO:tasks.workunit.client.1.smithi176.stdout:2/777: chown dd/d2e/l62 6722890 1 2024-07-16T09:55:43.366 INFO:tasks.workunit.client.1.smithi176.stdout:4/590: creat da/d3d/dac/fb9 x:0 0 0 2024-07-16T09:55:43.367 INFO:tasks.workunit.client.1.smithi176.stdout:2/778: creat dd/d9b/dea/ffd x:0 0 0 2024-07-16T09:55:43.367 INFO:tasks.workunit.client.1.smithi176.stdout:0/712: write d5/d1c/d24/d26/f4f [986827,56449] 0 2024-07-16T09:55:43.369 INFO:tasks.workunit.client.1.smithi176.stdout:0/713: chown d5/d1c/d24/d35/d50/d58/d63/db4/dcc/cd1 77290 1 2024-07-16T09:55:43.369 INFO:tasks.workunit.client.1.smithi176.stdout:2/779: stat dd/d49/d52/d89/c99 0 2024-07-16T09:55:43.371 INFO:tasks.workunit.client.1.smithi176.stdout:0/714: creat d5/d39/fef x:0 0 0 2024-07-16T09:55:43.372 INFO:tasks.workunit.client.1.smithi176.stdout:2/780: fsync dd/d49/f60 0 2024-07-16T09:55:43.373 INFO:tasks.workunit.client.0.smithi156.stdout:7/507: sync 2024-07-16T09:55:43.373 INFO:tasks.workunit.client.0.smithi156.stdout:0/673: sync 2024-07-16T09:55:43.373 INFO:tasks.workunit.client.1.smithi176.stdout:0/715: write d5/d1c/d24/d35/d50/d58/fcb [454008,88030] 0 2024-07-16T09:55:43.374 INFO:tasks.workunit.client.0.smithi156.stdout:0/674: chown d0/d1a/d4d/d25/f2a 699378 1 2024-07-16T09:55:43.374 INFO:tasks.workunit.client.1.smithi176.stdout:2/781: truncate dd/de/fc0 883572 0 2024-07-16T09:55:43.376 INFO:tasks.workunit.client.1.smithi176.stdout:2/782: mknod dd/de/cfe 0 2024-07-16T09:55:43.376 INFO:tasks.workunit.client.1.smithi176.stdout:0/716: read - d5/d1c/d24/d35/f94 zero size 2024-07-16T09:55:43.379 INFO:tasks.workunit.client.0.smithi156.stdout:7/508: dwrite dc/f94 [0,4194304] 0 2024-07-16T09:55:43.381 INFO:tasks.workunit.client.1.smithi176.stdout:2/783: link dd/c67 dd/d16/d39/cff 0 2024-07-16T09:55:43.384 INFO:tasks.workunit.client.1.smithi176.stdout:2/784: mknod dd/d36/d45/c100 0 2024-07-16T09:55:43.384 INFO:tasks.workunit.client.1.smithi176.stdout:0/717: dwrite d5/d25/d82/f8f [0,4194304] 0 2024-07-16T09:55:43.384 INFO:tasks.workunit.client.1.smithi176.stdout:4/591: dread da/fb [0,4194304] 0 2024-07-16T09:55:43.385 INFO:tasks.workunit.client.0.smithi156.stdout:7/509: dwrite dc/d3e/d5d/f80 [4194304,4194304] 0 2024-07-16T09:55:43.386 INFO:tasks.workunit.client.1.smithi176.stdout:0/718: dread - d5/d1c/fd7 zero size 2024-07-16T09:55:43.388 INFO:tasks.workunit.client.1.smithi176.stdout:0/719: write d5/d1c/d24/d35/d50/d58/d63/fb9 [935492,90691] 0 2024-07-16T09:55:43.391 INFO:tasks.workunit.client.1.smithi176.stdout:2/785: rename dd/d2e/d35/ca4 to dd/d36/d45/dd3/c101 0 2024-07-16T09:55:43.392 INFO:tasks.workunit.client.1.smithi176.stdout:4/592: mkdir da/d41/d5b/dba 0 2024-07-16T09:55:43.398 INFO:tasks.workunit.client.0.smithi156.stdout:4/720: dread - d4/db/dc/dcc/d58/f7a zero size 2024-07-16T09:55:43.400 INFO:tasks.workunit.client.0.smithi156.stdout:9/665: dwrite d2/d5/d16/d22/d2b/d74/dc2/dc4/d3a/f49 [0,4194304] 0 2024-07-16T09:55:43.400 INFO:tasks.workunit.client.1.smithi176.stdout:2/786: dread dd/d36/f94 [0,4194304] 0 2024-07-16T09:55:43.400 INFO:tasks.workunit.client.1.smithi176.stdout:0/720: dread d5/d1c/d24/d35/f88 [0,4194304] 0 2024-07-16T09:55:43.401 INFO:tasks.workunit.client.0.smithi156.stdout:5/639: symlink d1b/d1f/d54/dbe/ldb 0 2024-07-16T09:55:43.401 INFO:tasks.workunit.client.0.smithi156.stdout:9/666: chown d2/d5/d16/d22/d36/c79 31846214 1 2024-07-16T09:55:43.402 INFO:tasks.workunit.client.1.smithi176.stdout:4/593: dwrite da/dc/d10/d96/d69/fb0 [0,4194304] 0 2024-07-16T09:55:43.402 INFO:tasks.workunit.client.1.smithi176.stdout:0/721: dread - d5/d1c/db1/fcd zero size 2024-07-16T09:55:43.404 INFO:tasks.workunit.client.0.smithi156.stdout:3/576: rename d8/d14/l1b to d8/da/d1d/d2c/d7d/lc9 0 2024-07-16T09:55:43.405 INFO:tasks.workunit.client.1.smithi176.stdout:4/594: read da/d3d/d17/f31 [1696843,78459] 0 2024-07-16T09:55:43.406 INFO:tasks.workunit.client.0.smithi156.stdout:8/493: creat d0/d11/d20/d3f/d96/fad x:0 0 0 2024-07-16T09:55:43.406 INFO:tasks.workunit.client.1.smithi176.stdout:0/722: mkdir d5/d39/dde/df0 0 2024-07-16T09:55:43.408 INFO:tasks.workunit.client.0.smithi156.stdout:0/675: creat d0/d1a/d4d/d9/d27/d50/d90/fd8 x:0 0 0 2024-07-16T09:55:43.411 INFO:tasks.workunit.client.1.smithi176.stdout:0/723: truncate d5/d25/da0/fdd 915213 0 2024-07-16T09:55:43.411 INFO:tasks.workunit.client.0.smithi156.stdout:7/510: symlink dc/d3e/d3f/d51/d8a/laf 0 2024-07-16T09:55:43.414 INFO:tasks.workunit.client.0.smithi156.stdout:9/667: dwrite d2/d5/d16/d22/d36/f59 [0,4194304] 0 2024-07-16T09:55:43.415 INFO:tasks.workunit.client.0.smithi156.stdout:5/640: creat d1b/d1f/d45/d52/d49/d4d/fdc x:0 0 0 2024-07-16T09:55:43.417 INFO:tasks.workunit.client.0.smithi156.stdout:6/547: symlink d3/d13/d17/d4f/d4d/d3b/da5/lb3 0 2024-07-16T09:55:43.418 INFO:tasks.workunit.client.0.smithi156.stdout:8/494: fsync d0/dd/d1f/f47 0 2024-07-16T09:55:43.418 INFO:tasks.workunit.client.0.smithi156.stdout:9/668: truncate d2/d5/d16/d22/d36/f59 4696193 0 2024-07-16T09:55:43.419 INFO:tasks.workunit.client.0.smithi156.stdout:1/823: unlink dc/l2c 0 2024-07-16T09:55:43.423 INFO:tasks.workunit.client.0.smithi156.stdout:1/824: fsync dc/f56 0 2024-07-16T09:55:43.423 INFO:tasks.workunit.client.0.smithi156.stdout:0/676: dread d0/d1a/d4d/d25/f55 [0,4194304] 0 2024-07-16T09:55:43.424 INFO:tasks.workunit.client.0.smithi156.stdout:7/511: chown dc/d13/d20/f3c 33 1 2024-07-16T09:55:43.424 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:55:42 smithi156 ceph-mon[29302]: fsmap cephfs:2 {0=cephfs.smithi156.rdibys=up:active,1=cephfs.smithi176.dvvbud=up:active} 2 up:standby-replay 2024-07-16T09:55:43.425 INFO:tasks.workunit.client.0.smithi156.stdout:2/667: truncate d0/d5/d29/d2d/d5b/d5f/ddc/fb4 173304 0 2024-07-16T09:55:43.426 INFO:tasks.workunit.client.0.smithi156.stdout:3/577: dread d8/da/d21/f3e [4194304,4194304] 0 2024-07-16T09:55:43.427 INFO:tasks.workunit.client.0.smithi156.stdout:5/641: mknod d1b/d1f/d54/d72/cdd 0 2024-07-16T09:55:43.428 INFO:tasks.workunit.client.0.smithi156.stdout:1/825: truncate dc/d24/f2a 4669291 0 2024-07-16T09:55:43.431 INFO:tasks.workunit.client.0.smithi156.stdout:0/677: dwrite d0/d22/d41/d66/fcf [0,4194304] 0 2024-07-16T09:55:43.431 INFO:tasks.workunit.client.1.smithi176.stdout:0/724: dwrite d5/fb [4194304,4194304] 0 2024-07-16T09:55:43.431 INFO:tasks.workunit.client.1.smithi176.stdout:2/787: dwrite dd/d49/f60 [0,4194304] 0 2024-07-16T09:55:43.431 INFO:tasks.workunit.client.1.smithi176.stdout:0/725: fdatasync d5/d1c/fb8 0 2024-07-16T09:55:43.432 INFO:tasks.workunit.client.0.smithi156.stdout:6/548: rename d3/d13/d17/d56/f60 to d3/d13/d17/d4f/d30/d45/d5f/fb4 0 2024-07-16T09:55:43.433 INFO:tasks.workunit.client.0.smithi156.stdout:1/826: unlink dc/d14/d3e/d55/d57/f93 0 2024-07-16T09:55:43.435 INFO:tasks.workunit.client.0.smithi156.stdout:7/512: creat dc/d3e/d3f/d51/fb0 x:0 0 0 2024-07-16T09:55:43.436 INFO:tasks.workunit.client.0.smithi156.stdout:1/827: truncate dc/d14/d1e/f21 4334332 0 2024-07-16T09:55:43.436 INFO:tasks.workunit.client.0.smithi156.stdout:2/668: dread - d0/d9b/fba zero size 2024-07-16T09:55:43.437 INFO:tasks.workunit.client.0.smithi156.stdout:3/578: chown d8/da/d13/l2b 7980317 1 2024-07-16T09:55:43.438 INFO:tasks.workunit.client.0.smithi156.stdout:7/513: chown dc/d13/l31 127301 1 2024-07-16T09:55:43.440 INFO:tasks.workunit.client.0.smithi156.stdout:8/495: mkdir d0/dd/da3/dae 0 2024-07-16T09:55:43.441 INFO:tasks.workunit.client.0.smithi156.stdout:6/549: dwrite d3/d13/d17/f2b [0,4194304] 0 2024-07-16T09:55:43.441 INFO:tasks.workunit.client.0.smithi156.stdout:4/721: getdents d4/d23/d30/dd2 0 2024-07-16T09:55:43.442 INFO:tasks.workunit.client.0.smithi156.stdout:8/496: chown d0/c4 7 1 2024-07-16T09:55:43.443 INFO:tasks.workunit.client.1.smithi176.stdout:0/726: creat d5/d1c/d24/d35/d59/dce/ff1 x:0 0 0 2024-07-16T09:55:43.444 INFO:tasks.workunit.client.0.smithi156.stdout:7/514: rmdir dc/d13/d15/d8b 39 2024-07-16T09:55:43.445 INFO:tasks.workunit.client.0.smithi156.stdout:8/497: write d0/d11/d4a/f4f [460754,52890] 0 2024-07-16T09:55:43.445 INFO:tasks.workunit.client.0.smithi156.stdout:0/678: getdents d0/d1a/da3 0 2024-07-16T09:55:43.445 INFO:tasks.workunit.client.0.smithi156.stdout:2/669: fsync d0/d13/f81 0 2024-07-16T09:55:43.446 INFO:tasks.workunit.client.0.smithi156.stdout:3/579: mkdir d8/da/d13/dca 0 2024-07-16T09:55:43.446 INFO:tasks.workunit.client.0.smithi156.stdout:5/642: truncate d1b/d1f/d45/d9d/fab 570795 0 2024-07-16T09:55:43.447 INFO:tasks.workunit.client.0.smithi156.stdout:1/828: link dc/d14/d3e/d55/d83/fb2 dc/d14/d3e/d55/d57/d76/f115 0 2024-07-16T09:55:43.448 INFO:tasks.workunit.client.1.smithi176.stdout:1/443: sync 2024-07-16T09:55:43.449 INFO:tasks.workunit.client.0.smithi156.stdout:2/670: chown d0/d5/d29/la6 19 1 2024-07-16T09:55:43.450 INFO:tasks.workunit.client.0.smithi156.stdout:1/829: read dc/d14/d8b/db4/fd6 [98,71805] 0 2024-07-16T09:55:43.450 INFO:tasks.workunit.client.0.smithi156.stdout:2/671: chown d0/d13/d6e 80475758 1 2024-07-16T09:55:43.451 INFO:tasks.workunit.client.1.smithi176.stdout:1/444: write d4/d8/d12/d21/d25/d30/f61 [1932844,92949] 0 2024-07-16T09:55:43.452 INFO:tasks.workunit.client.1.smithi176.stdout:0/727: rename d5/d39/da3/fd9 to d5/d2c/d6c/d83/ff2 0 2024-07-16T09:55:43.452 INFO:tasks.workunit.client.0.smithi156.stdout:6/550: creat d3/d13/d17/d4f/d30/d45/d99/fb5 x:0 0 0 2024-07-16T09:55:43.452 INFO:tasks.workunit.client.0.smithi156.stdout:0/679: fdatasync d0/d1a/d4d/f21 0 2024-07-16T09:55:43.453 INFO:tasks.workunit.client.0.smithi156.stdout:1/830: mknod dc/d14/d3e/d55/d83/c116 0 2024-07-16T09:55:43.455 INFO:tasks.workunit.client.1.smithi176.stdout:0/728: write d5/d2c/d6c/d83/d84/fc2 [446465,65041] 0 2024-07-16T09:55:43.455 INFO:tasks.workunit.client.0.smithi156.stdout:8/498: mkdir d0/d11/d20/d7f/daf 0 2024-07-16T09:55:43.455 INFO:tasks.workunit.client.0.smithi156.stdout:7/515: mkdir dc/d44/db1 0 2024-07-16T09:55:43.456 INFO:tasks.workunit.client.0.smithi156.stdout:5/643: mkdir d1b/d1f/d54/dbe/dde 0 2024-07-16T09:55:43.456 INFO:tasks.workunit.client.0.smithi156.stdout:8/499: chown d0/d11/d4a 29692 1 2024-07-16T09:55:43.457 INFO:tasks.workunit.client.0.smithi156.stdout:7/516: chown dc/d13/d20/d4d/daa 87178574 1 2024-07-16T09:55:43.458 INFO:tasks.workunit.client.0.smithi156.stdout:5/644: dread - d1b/d99/dae/fd4 zero size 2024-07-16T09:55:43.458 INFO:tasks.workunit.client.1.smithi176.stdout:1/445: fdatasync d4/f24 0 2024-07-16T09:55:43.459 INFO:tasks.workunit.client.1.smithi176.stdout:4/595: dread da/dc/d10/d96/d69/fae [0,4194304] 0 2024-07-16T09:55:43.459 INFO:tasks.workunit.client.0.smithi156.stdout:3/580: symlink d8/lcb 0 2024-07-16T09:55:43.460 INFO:tasks.workunit.client.1.smithi176.stdout:0/729: creat d5/d1c/db1/ff3 x:0 0 0 2024-07-16T09:55:43.460 INFO:tasks.workunit.client.0.smithi156.stdout:1/831: unlink dc/d6b/f80 0 2024-07-16T09:55:43.460 INFO:tasks.workunit.client.0.smithi156.stdout:6/551: symlink d3/d13/lb6 0 2024-07-16T09:55:43.461 INFO:tasks.workunit.client.1.smithi176.stdout:1/446: mknod d4/d82/c98 0 2024-07-16T09:55:43.461 INFO:tasks.workunit.client.1.smithi176.stdout:0/730: write d5/d25/d82/f8f [1643833,59478] 0 2024-07-16T09:55:43.463 INFO:tasks.workunit.client.0.smithi156.stdout:8/500: creat d0/d74/fb0 x:0 0 0 2024-07-16T09:55:43.464 INFO:tasks.workunit.client.1.smithi176.stdout:0/731: write d5/d1c/f1f [2593934,32014] 0 2024-07-16T09:55:43.464 INFO:tasks.workunit.client.0.smithi156.stdout:7/517: dread - dc/d13/d15/f78 zero size 2024-07-16T09:55:43.465 INFO:tasks.workunit.client.1.smithi176.stdout:4/596: link da/dc/d10/d96/d69/fae da/d3d/d17/fbb 0 2024-07-16T09:55:43.465 INFO:tasks.workunit.client.0.smithi156.stdout:2/672: symlink d0/d13/d17/d1c/d86/dde/le0 0 2024-07-16T09:55:43.466 INFO:tasks.workunit.client.1.smithi176.stdout:4/597: chown da/fb 459423502 1 2024-07-16T09:55:43.466 INFO:tasks.workunit.client.0.smithi156.stdout:1/832: truncate dc/d14/d8b/db4/df2/f36 5235923 0 2024-07-16T09:55:43.467 INFO:tasks.workunit.client.1.smithi176.stdout:4/598: readlink da/dc/l1e 0 2024-07-16T09:55:43.467 INFO:tasks.workunit.client.0.smithi156.stdout:0/680: symlink d0/d1a/d4d/d9/d27/d2c/ld9 0 2024-07-16T09:55:43.468 INFO:tasks.workunit.client.0.smithi156.stdout:3/581: symlink d8/da/d28/d40/lcc 0 2024-07-16T09:55:43.468 INFO:tasks.workunit.client.0.smithi156.stdout:0/681: chown d0/d1a/d4d/d9/d27/c35 109801 1 2024-07-16T09:55:43.469 INFO:tasks.workunit.client.1.smithi176.stdout:4/599: write da/dc/d10/d96/d9b/db4/f26 [7343725,71627] 0 2024-07-16T09:55:43.470 INFO:tasks.workunit.client.0.smithi156.stdout:5/645: mkdir d1b/d1f/d45/d52/d43/d85/d97/d37/d88/dc7/ddf 0 2024-07-16T09:55:43.471 INFO:tasks.workunit.client.0.smithi156.stdout:3/582: write d8/da/d28/d43/f5a [3776777,28129] 0 2024-07-16T09:55:43.473 INFO:tasks.workunit.client.0.smithi156.stdout:6/552: read d3/d13/d17/d4f/d4d/f57 [2980559,34684] 0 2024-07-16T09:55:43.474 INFO:tasks.workunit.client.0.smithi156.stdout:3/583: readlink d8/d14/laf 0 2024-07-16T09:55:43.476 INFO:tasks.workunit.client.0.smithi156.stdout:9/669: dwrite d2/d5/d61/da2/d98/fa7 [0,4194304] 0 2024-07-16T09:55:43.476 INFO:tasks.workunit.client.1.smithi176.stdout:2/788: dread dd/d2e/d35/f3f [0,4194304] 0 2024-07-16T09:55:43.477 INFO:tasks.workunit.client.0.smithi156.stdout:5/646: write d1b/f40 [402174,117921] 0 2024-07-16T09:55:43.480 INFO:tasks.workunit.client.0.smithi156.stdout:3/584: write d8/da/d28/d40/f54 [248881,24047] 0 2024-07-16T09:55:43.481 INFO:tasks.workunit.client.0.smithi156.stdout:6/553: fsync d3/d13/d17/d4f/d30/d45/f49 0 2024-07-16T09:55:43.481 INFO:tasks.workunit.client.0.smithi156.stdout:9/670: mkdir d2/d5/d16/d22/d2b/dd0 0 2024-07-16T09:55:43.482 INFO:tasks.workunit.client.0.smithi156.stdout:1/833: dwrite dc/d14/d3e/d55/d57/d76/f115 [0,4194304] 0 2024-07-16T09:55:43.483 INFO:tasks.workunit.client.0.smithi156.stdout:5/647: write d1b/d1f/d42/d6d/d9f/fda [531332,120282] 0 2024-07-16T09:55:43.484 INFO:tasks.workunit.client.0.smithi156.stdout:0/682: dwrite d0/d1a/f46 [0,4194304] 0 2024-07-16T09:55:43.485 INFO:tasks.workunit.client.0.smithi156.stdout:1/834: chown dc/d24/f7e 183759 1 2024-07-16T09:55:43.485 INFO:tasks.workunit.client.0.smithi156.stdout:3/585: read - d8/da/d21/d80/d99/fa3 zero size 2024-07-16T09:55:43.488 INFO:tasks.workunit.client.0.smithi156.stdout:1/835: stat dc/d24/c46 0 2024-07-16T09:55:43.493 INFO:tasks.workunit.client.0.smithi156.stdout:1/836: readlink dc/d14/le2 0 2024-07-16T09:55:43.495 INFO:tasks.workunit.client.0.smithi156.stdout:4/722: write d4/db/dc/dcc/d58/d8f/f5e [4793102,95609] 0 2024-07-16T09:55:43.497 INFO:tasks.workunit.client.0.smithi156.stdout:1/837: write dc/d14/fd4 [530023,12867] 0 2024-07-16T09:55:43.498 INFO:tasks.workunit.client.1.smithi176.stdout:7/652: sync 2024-07-16T09:55:43.499 INFO:tasks.workunit.client.0.smithi156.stdout:5/648: dwrite d1b/d1f/d45/d52/d43/f8d [0,4194304] 0 2024-07-16T09:55:43.503 INFO:tasks.workunit.client.0.smithi156.stdout:4/723: mkdir d4/d23/d52/d102 0 2024-07-16T09:55:43.503 INFO:tasks.workunit.client.0.smithi156.stdout:3/586: creat d8/da/d28/d43/d63/fcd x:0 0 0 2024-07-16T09:55:43.505 INFO:tasks.workunit.client.0.smithi156.stdout:5/649: write d1b/d1f/d54/dbe/fc3 [2928228,25713] 0 2024-07-16T09:55:43.505 INFO:tasks.workunit.client.0.smithi156.stdout:8/501: write d0/d11/d20/f21 [2603901,71371] 0 2024-07-16T09:55:43.506 INFO:tasks.workunit.client.0.smithi156.stdout:6/554: dwrite d3/d13/d17/d4f/d30/f67 [0,4194304] 0 2024-07-16T09:55:43.506 INFO:tasks.workunit.client.0.smithi156.stdout:0/683: rename d0/f34 to d0/d1a/d4d/fda 0 2024-07-16T09:55:43.509 INFO:tasks.workunit.client.0.smithi156.stdout:7/518: write dc/d13/f1d [176608,99889] 0 2024-07-16T09:55:43.510 INFO:tasks.workunit.client.0.smithi156.stdout:8/502: readlink d0/d11/d13/l95 0 2024-07-16T09:55:43.510 INFO:tasks.workunit.client.1.smithi176.stdout:7/653: fdatasync d4/dd/d49/d7a/f92 0 2024-07-16T09:55:43.511 INFO:tasks.workunit.client.0.smithi156.stdout:4/724: mknod d4/d6a/c103 0 2024-07-16T09:55:43.511 INFO:tasks.workunit.client.0.smithi156.stdout:0/684: mkdir d0/d22/d41/d52/ddb 0 2024-07-16T09:55:43.512 INFO:tasks.workunit.client.1.smithi176.stdout:1/447: truncate d4/d8/d12/d21/d34/f58 1852167 0 2024-07-16T09:55:43.515 INFO:tasks.workunit.client.1.smithi176.stdout:0/732: dwrite d5/d1c/d24/d35/f94 [0,4194304] 0 2024-07-16T09:55:43.516 INFO:tasks.workunit.client.1.smithi176.stdout:2/789: truncate dd/d2e/f3e 7559820 0 2024-07-16T09:55:43.516 INFO:tasks.workunit.client.0.smithi156.stdout:3/587: creat d8/da/d1d/d6f/fce x:0 0 0 2024-07-16T09:55:43.517 INFO:tasks.workunit.client.1.smithi176.stdout:0/733: fsync d5/d39/fef 0 2024-07-16T09:55:43.517 INFO:tasks.workunit.client.0.smithi156.stdout:6/555: rmdir d3/d13/d17/d4f/d30/d45/d5f 39 2024-07-16T09:55:43.518 INFO:tasks.workunit.client.1.smithi176.stdout:4/600: dread da/dc/d10/d96/d9b/db4/d40/d5d/d73/f61 [0,4194304] 0 2024-07-16T09:55:43.519 INFO:tasks.workunit.client.0.smithi156.stdout:0/685: rmdir d0/d1a/d4d/d9/d27/d2c/d6d/d9d/dbb 39 2024-07-16T09:55:43.521 INFO:tasks.workunit.client.1.smithi176.stdout:2/790: mknod dd/d36/d45/c102 0 2024-07-16T09:55:43.523 INFO:tasks.workunit.client.1.smithi176.stdout:0/734: dread d5/d1c/d24/d35/f88 [0,4194304] 0 2024-07-16T09:55:43.524 INFO:tasks.workunit.client.1.smithi176.stdout:1/448: dwrite d4/f65 [0,4194304] 0 2024-07-16T09:55:43.526 INFO:tasks.workunit.client.0.smithi156.stdout:7/519: dwrite dc/d3e/d3f/d51/fb0 [0,4194304] 0 2024-07-16T09:55:43.529 INFO:tasks.workunit.client.0.smithi156.stdout:9/671: write d2/d5/f70 [819895,96203] 0 2024-07-16T09:55:43.530 INFO:tasks.workunit.client.0.smithi156.stdout:6/556: chown d3/d13/d17/d4f/l16 0 1 2024-07-16T09:55:43.532 INFO:tasks.workunit.client.1.smithi176.stdout:4/601: symlink da/d3d/d17/d54/lbc 0 2024-07-16T09:55:43.533 INFO:tasks.workunit.client.1.smithi176.stdout:0/735: symlink d5/d44/lf4 0 2024-07-16T09:55:43.533 INFO:tasks.workunit.client.0.smithi156.stdout:7/520: dread - dc/d48/f92 zero size 2024-07-16T09:55:43.533 INFO:tasks.workunit.client.0.smithi156.stdout:4/725: dwrite d4/d23/f39 [4194304,4194304] 0 2024-07-16T09:55:43.534 INFO:tasks.workunit.client.1.smithi176.stdout:1/449: chown d4/d8/f69 75012 1 2024-07-16T09:55:43.534 INFO:tasks.workunit.client.1.smithi176.stdout:4/602: chown da/dc/d10/d28/c53 1067710357 1 2024-07-16T09:55:43.536 INFO:tasks.workunit.client.0.smithi156.stdout:8/503: link d0/dd/f17 d0/d11/d20/d3f/fb1 0 2024-07-16T09:55:43.541 INFO:tasks.workunit.client.0.smithi156.stdout:9/672: fsync d2/d5/d16/d22/d2b/d74/dc2/dc4/d23/da0/faf 0 2024-07-16T09:55:43.544 INFO:tasks.workunit.client.0.smithi156.stdout:7/521: mkdir dc/d48/db2 0 2024-07-16T09:55:43.544 INFO:tasks.workunit.client.0.smithi156.stdout:8/504: read d0/d11/d20/d3f/f69 [1243393,122172] 0 2024-07-16T09:55:43.544 INFO:tasks.workunit.client.0.smithi156.stdout:4/726: creat d4/db/dc/dcc/d58/dd9/f104 x:0 0 0 2024-07-16T09:55:43.546 INFO:tasks.workunit.client.0.smithi156.stdout:6/557: mknod d3/d13/d17/d4f/d94/cb7 0 2024-07-16T09:55:43.547 INFO:tasks.workunit.client.0.smithi156.stdout:8/505: readlink d0/dd/d1f/l28 0 2024-07-16T09:55:43.548 INFO:tasks.workunit.client.0.smithi156.stdout:0/686: dwrite d0/d1a/d4d/d9/f8f [0,4194304] 0 2024-07-16T09:55:43.550 INFO:tasks.workunit.client.0.smithi156.stdout:7/522: chown dc/d48/c97 248797120 1 2024-07-16T09:55:43.550 INFO:tasks.workunit.client.0.smithi156.stdout:2/673: dread d0/d5/d29/d2d/d5b/d5f/f7e [0,4194304] 0 2024-07-16T09:55:43.554 INFO:tasks.workunit.client.0.smithi156.stdout:8/506: symlink d0/dd/d1f/d57/lb2 0 2024-07-16T09:55:43.554 INFO:tasks.workunit.client.0.smithi156.stdout:4/727: creat d4/db/d47/dbe/f105 x:0 0 0 2024-07-16T09:55:43.555 INFO:tasks.workunit.client.0.smithi156.stdout:0/687: truncate d0/d22/d41/f8a 1166314 0 2024-07-16T09:55:43.558 INFO:tasks.workunit.client.0.smithi156.stdout:6/558: mknod d3/d13/d17/d4f/d4d/cb8 0 2024-07-16T09:55:43.558 INFO:tasks.workunit.client.0.smithi156.stdout:9/673: dread d2/d5/d16/d22/d2b/d3d/d3c/d7f/fb9 [0,4194304] 0 2024-07-16T09:55:43.559 INFO:tasks.workunit.client.1.smithi176.stdout:2/791: fsync dd/d2e/d35/f64 0 2024-07-16T09:55:43.560 INFO:tasks.workunit.client.0.smithi156.stdout:7/523: unlink dc/d3e/d5d/ca8 0 2024-07-16T09:55:43.562 INFO:tasks.workunit.client.1.smithi176.stdout:1/450: symlink d4/d8/d12/d21/d25/d30/d44/d4d/l99 0 2024-07-16T09:55:43.563 INFO:tasks.workunit.client.1.smithi176.stdout:1/451: dread - d4/d3a/d54/f8b zero size 2024-07-16T09:55:43.563 INFO:tasks.workunit.client.1.smithi176.stdout:2/792: creat dd/d2e/d35/d59/d66/dc8/f103 x:0 0 0 2024-07-16T09:55:43.564 INFO:tasks.workunit.client.0.smithi156.stdout:4/728: write d4/db/d47/dbe/f105 [111451,1660] 0 2024-07-16T09:55:43.564 INFO:tasks.workunit.client.1.smithi176.stdout:1/452: fdatasync d4/d8/d12/d21/f89 0 2024-07-16T09:55:43.565 INFO:tasks.workunit.client.0.smithi156.stdout:1/838: dwrite dc/d14/d8b/db4/df2/faf [0,4194304] 0 2024-07-16T09:55:43.565 INFO:tasks.workunit.client.0.smithi156.stdout:8/507: write d0/dd/d1f/d57/f8f [523654,107873] 0 2024-07-16T09:55:43.566 INFO:tasks.workunit.client.1.smithi176.stdout:4/603: getdents da/dc/d10/d96/d9b/db4 0 2024-07-16T09:55:43.566 INFO:tasks.workunit.client.1.smithi176.stdout:1/453: read - d4/d8/d12/d21/d25/d30/d44/d4d/f7e zero size 2024-07-16T09:55:43.569 INFO:tasks.workunit.client.0.smithi156.stdout:3/588: dwrite d8/d14/f55 [0,4194304] 0 2024-07-16T09:55:43.569 INFO:tasks.workunit.client.0.smithi156.stdout:7/524: write dc/d13/d36/f66 [1930230,52287] 0 2024-07-16T09:55:43.569 INFO:tasks.workunit.client.1.smithi176.stdout:2/793: mkdir dd/d2e/d104 0 2024-07-16T09:55:43.571 INFO:tasks.workunit.client.0.smithi156.stdout:7/525: write dc/d48/fa3 [4577262,120538] 0 2024-07-16T09:55:43.572 INFO:tasks.workunit.client.1.smithi176.stdout:2/794: truncate dd/d49/d52/d89/ff5 131178 0 2024-07-16T09:55:43.573 INFO:tasks.workunit.client.1.smithi176.stdout:0/736: dread d5/d2c/d6c/f64 [0,4194304] 0 2024-07-16T09:55:43.573 INFO:tasks.workunit.client.0.smithi156.stdout:6/559: creat d3/d13/d17/d4f/d4d/d3b/d5a/fb9 x:0 0 0 2024-07-16T09:55:43.574 INFO:tasks.workunit.client.0.smithi156.stdout:3/589: chown d8/lcb 175 1 2024-07-16T09:55:43.575 INFO:tasks.workunit.client.0.smithi156.stdout:1/839: creat dc/d14/d3e/d55/d57/f117 x:0 0 0 2024-07-16T09:55:43.578 INFO:tasks.workunit.client.0.smithi156.stdout:2/674: dwrite d0/d13/dc9/fd1 [0,4194304] 0 2024-07-16T09:55:43.578 INFO:tasks.workunit.client.0.smithi156.stdout:8/508: mkdir d0/dd/d1f/d57/db3 0 2024-07-16T09:55:43.581 INFO:tasks.workunit.client.0.smithi156.stdout:3/590: write d8/da/d28/d43/f5a [2340816,68581] 0 2024-07-16T09:55:43.581 INFO:tasks.workunit.client.1.smithi176.stdout:0/737: rmdir d5/d1c/d24/d35/d50/d5b 39 2024-07-16T09:55:43.582 INFO:tasks.workunit.client.1.smithi176.stdout:4/604: dwrite da/dc/d10/d96/d9b/db4/d40/d5d/d73/f4f [0,4194304] 0 2024-07-16T09:55:43.585 INFO:tasks.workunit.client.1.smithi176.stdout:0/738: fdatasync d5/d1c/d24/d35/d50/d58/fcb 0 2024-07-16T09:55:43.585 INFO:tasks.workunit.client.0.smithi156.stdout:4/729: dread d4/db/d8a/ffd [0,4194304] 0 2024-07-16T09:55:43.585 INFO:tasks.workunit.client.1.smithi176.stdout:4/605: readlink da/d3d/d17/d54/l9c 0 2024-07-16T09:55:43.586 INFO:tasks.workunit.client.0.smithi156.stdout:1/840: symlink dc/d14/d8b/dcd/dd2/df9/l118 0 2024-07-16T09:55:43.586 INFO:tasks.workunit.client.0.smithi156.stdout:6/560: creat d3/d13/d17/d4f/d4d/d3b/fba x:0 0 0 2024-07-16T09:55:43.586 INFO:tasks.workunit.client.1.smithi176.stdout:0/739: write d5/d25/d82/f9a [34510,121618] 0 2024-07-16T09:55:43.588 INFO:tasks.workunit.client.1.smithi176.stdout:4/606: stat da/dc/d10/d96/d9b/db4/d40/d5d/d73/fb1 0 2024-07-16T09:55:43.588 INFO:tasks.workunit.client.1.smithi176.stdout:0/740: dread - d5/d39/f9f zero size 2024-07-16T09:55:43.589 INFO:tasks.workunit.client.1.smithi176.stdout:2/795: creat dd/d16/d39/f105 x:0 0 0 2024-07-16T09:55:43.591 INFO:tasks.workunit.client.0.smithi156.stdout:9/674: link d2/d5/d16/d22/d2b/d3d/d3c/d7f/f2f d2/d5/d16/d22/d2b/d3d/fd1 0 2024-07-16T09:55:43.592 INFO:tasks.workunit.client.0.smithi156.stdout:1/841: dread dc/f56 [0,4194304] 0 2024-07-16T09:55:43.594 INFO:tasks.workunit.client.0.smithi156.stdout:1/842: dread - dc/d14/d3e/d55/d57/def/fe8 zero size 2024-07-16T09:55:43.594 INFO:tasks.workunit.client.1.smithi176.stdout:1/454: dread d4/d3a/d54/f6d [0,4194304] 0 2024-07-16T09:55:43.595 INFO:tasks.workunit.client.0.smithi156.stdout:1/843: fdatasync dc/d14/d3e/f112 0 2024-07-16T09:55:43.595 INFO:tasks.workunit.client.0.smithi156.stdout:8/509: creat d0/d11/d13/d1c/d24/fb4 x:0 0 0 2024-07-16T09:55:43.596 INFO:tasks.workunit.client.0.smithi156.stdout:1/844: readlink dc/d14/d8b/db4/l10f 0 2024-07-16T09:55:43.597 INFO:tasks.workunit.client.0.smithi156.stdout:8/510: write d0/dd/f1a [3605076,64310] 0 2024-07-16T09:55:43.598 INFO:tasks.workunit.client.0.smithi156.stdout:4/730: unlink d4/d23/fdf 0 2024-07-16T09:55:43.599 INFO:tasks.workunit.client.1.smithi176.stdout:0/741: creat d5/d1c/d24/d35/d50/d58/d63/db4/dcc/ff5 x:0 0 0 2024-07-16T09:55:43.601 INFO:tasks.workunit.client.0.smithi156.stdout:2/675: write d0/d5/d29/f56 [596285,12688] 0 2024-07-16T09:55:43.601 INFO:tasks.workunit.client.0.smithi156.stdout:7/526: link dc/d48/f4b dc/d3e/d3f/d45/fb3 0 2024-07-16T09:55:43.601 INFO:tasks.workunit.client.1.smithi176.stdout:4/607: mknod da/dc/d10/d96/d9b/db4/d4a/cbd 0 2024-07-16T09:55:43.602 INFO:tasks.workunit.client.0.smithi156.stdout:3/591: mkdir d8/da/d28/d43/d71/da2/dcf 0 2024-07-16T09:55:43.602 INFO:tasks.workunit.client.1.smithi176.stdout:4/608: chown da/dc/d10/d96/d69/fb0 472015 1 2024-07-16T09:55:43.603 INFO:tasks.workunit.client.0.smithi156.stdout:1/845: rmdir dc/d6b/d8a/df8 39 2024-07-16T09:55:43.604 INFO:tasks.workunit.client.0.smithi156.stdout:7/527: write dc/d3e/d3f/d51/d6c/f85 [76606,66033] 0 2024-07-16T09:55:43.604 INFO:tasks.workunit.client.1.smithi176.stdout:4/609: fsync da/dc/f9d 0 2024-07-16T09:55:43.604 INFO:tasks.workunit.client.0.smithi156.stdout:8/511: write d0/f1 [9292371,80303] 0 2024-07-16T09:55:43.607 INFO:tasks.workunit.client.0.smithi156.stdout:2/676: mkdir d0/d5/d29/d2d/de1 0 2024-07-16T09:55:43.607 INFO:tasks.workunit.client.1.smithi176.stdout:4/610: symlink da/dc/d10/d96/da4/lbe 0 2024-07-16T09:55:43.610 INFO:tasks.workunit.client.0.smithi156.stdout:7/528: symlink dc/d13/d36/d56/lb4 0 2024-07-16T09:55:43.611 INFO:tasks.workunit.client.0.smithi156.stdout:4/731: creat d4/db/dc/f106 x:0 0 0 2024-07-16T09:55:43.611 INFO:tasks.workunit.client.1.smithi176.stdout:2/796: write dd/d16/f56 [484865,31690] 0 2024-07-16T09:55:43.612 INFO:tasks.workunit.client.1.smithi176.stdout:0/742: write d5/d44/f96 [113857,18042] 0 2024-07-16T09:55:43.613 INFO:tasks.workunit.client.0.smithi156.stdout:9/675: link d2/d5/ld d2/d5/d16/d22/d2b/d3d/da6/ld2 0 2024-07-16T09:55:43.613 INFO:tasks.workunit.client.0.smithi156.stdout:6/561: truncate d3/d13/d17/d4f/d94/f10 5314684 0 2024-07-16T09:55:43.616 INFO:tasks.workunit.client.0.smithi156.stdout:4/732: symlink d4/db/dc/d67/l107 0 2024-07-16T09:55:43.616 INFO:tasks.workunit.client.0.smithi156.stdout:6/562: read d3/d13/d17/f55 [201644,545] 0 2024-07-16T09:55:43.616 INFO:tasks.workunit.client.1.smithi176.stdout:2/797: creat dd/d2e/d35/d4a/df9/f106 x:0 0 0 2024-07-16T09:55:43.616 INFO:tasks.workunit.client.1.smithi176.stdout:4/611: dwrite da/dc/d10/d96/d9b/db4/d40/d5d/f85 [0,4194304] 0 2024-07-16T09:55:43.617 INFO:tasks.workunit.client.0.smithi156.stdout:7/529: rename dc/l8c to dc/d13/d20/d4d/daa/lb5 0 2024-07-16T09:55:43.619 INFO:tasks.workunit.client.1.smithi176.stdout:2/798: chown f3 49738347 1 2024-07-16T09:55:43.620 INFO:tasks.workunit.client.0.smithi156.stdout:1/846: dread dc/d14/d3e/d6c/f73 [0,4194304] 0 2024-07-16T09:55:43.620 INFO:tasks.workunit.client.0.smithi156.stdout:2/677: creat d0/d5/d29/d2d/de1/fe2 x:0 0 0 2024-07-16T09:55:43.621 INFO:tasks.workunit.client.1.smithi176.stdout:4/612: mknod da/dc/d10/d96/d9b/db4/d40/d5d/cbf 0 2024-07-16T09:55:43.623 INFO:tasks.workunit.client.1.smithi176.stdout:0/743: dwrite d5/d1c/d24/d35/d50/dd5/fda [0,4194304] 0 2024-07-16T09:55:43.624 INFO:tasks.workunit.client.1.smithi176.stdout:4/613: mknod da/dc/d10/d96/d69/cc0 0 2024-07-16T09:55:43.626 INFO:tasks.workunit.client.0.smithi156.stdout:8/512: rmdir d0/d11/d13/d1c/d39/d8a/d92 0 2024-07-16T09:55:43.629 INFO:tasks.workunit.client.0.smithi156.stdout:7/530: dwrite dc/f94 [0,4194304] 0 2024-07-16T09:55:43.632 INFO:tasks.workunit.client.0.smithi156.stdout:2/678: symlink d0/d13/d17/d1c/d51/le3 0 2024-07-16T09:55:43.634 INFO:tasks.workunit.client.0.smithi156.stdout:1/847: dwrite dc/f95 [4194304,4194304] 0 2024-07-16T09:55:43.638 INFO:tasks.workunit.client.0.smithi156.stdout:8/513: creat d0/dd/fb5 x:0 0 0 2024-07-16T09:55:43.640 INFO:tasks.workunit.client.0.smithi156.stdout:7/531: unlink dc/d48/f7c 0 2024-07-16T09:55:43.641 INFO:tasks.workunit.client.0.smithi156.stdout:9/676: dread d2/d5/d16/d22/d2b/d3d/d3c/d7f/f5e [0,4194304] 0 2024-07-16T09:55:43.641 INFO:tasks.workunit.client.0.smithi156.stdout:3/592: truncate d8/d14/f30 3984500 0 2024-07-16T09:55:43.647 INFO:tasks.workunit.client.0.smithi156.stdout:3/593: chown d8/da/d28/d40/f56 0 1 2024-07-16T09:55:43.647 INFO:tasks.workunit.client.0.smithi156.stdout:2/679: getdents d0/d13/d17/d1c/d86/dde 0 2024-07-16T09:55:43.648 INFO:tasks.workunit.client.0.smithi156.stdout:8/514: dwrite d0/d11/d13/f14 [4194304,4194304] 0 2024-07-16T09:55:43.658 INFO:tasks.workunit.client.0.smithi156.stdout:3/594: mknod d8/da/d28/cd0 0 2024-07-16T09:55:43.659 INFO:tasks.workunit.client.0.smithi156.stdout:7/532: getdents dc/d3e/d5d 0 2024-07-16T09:55:43.659 INFO:tasks.workunit.client.0.smithi156.stdout:9/677: dwrite d2/d5/d16/d22/d2b/d74/dc2/dc4/d3a/fab [0,4194304] 0 2024-07-16T09:55:43.661 INFO:tasks.workunit.client.0.smithi156.stdout:7/533: fdatasync dc/d48/fa3 0 2024-07-16T09:55:43.662 INFO:tasks.workunit.client.0.smithi156.stdout:4/733: write d4/db/dc/dcc/d58/d8f/f7e [1281646,35279] 0 2024-07-16T09:55:43.664 INFO:tasks.workunit.client.0.smithi156.stdout:6/563: truncate d3/d13/d17/d4f/d4d/d3b/f96 3016074 0 2024-07-16T09:55:43.665 INFO:tasks.workunit.client.0.smithi156.stdout:9/678: mkdir d2/d5/d16/d57/d63/dd3 0 2024-07-16T09:55:43.666 INFO:tasks.workunit.client.0.smithi156.stdout:9/679: stat d2/d5/d61/l95 0 2024-07-16T09:55:43.672 INFO:tasks.workunit.client.0.smithi156.stdout:6/564: dread d3/d13/d17/d4f/d30/f7f [0,4194304] 0 2024-07-16T09:55:43.674 INFO:tasks.workunit.client.1.smithi176.stdout:4/614: mkdir da/dc/d10/d96/d9b/db4/d4a/dc1 0 2024-07-16T09:55:43.675 INFO:tasks.workunit.client.1.smithi176.stdout:4/615: stat da/dc/d10 0 2024-07-16T09:55:43.677 INFO:tasks.workunit.client.0.smithi156.stdout:1/848: rename dc/d6b/dd5/fe5 to dc/d14/d3e/d55/dec/f119 0 2024-07-16T09:55:43.678 INFO:tasks.workunit.client.1.smithi176.stdout:4/616: mkdir da/dc/d10/d96/da4/dc2 0 2024-07-16T09:55:43.680 INFO:tasks.workunit.client.0.smithi156.stdout:8/515: dwrite d0/f2c [0,4194304] 0 2024-07-16T09:55:43.681 INFO:tasks.workunit.client.1.smithi176.stdout:0/744: write d5/d25/f4e [3466781,33426] 0 2024-07-16T09:55:43.684 INFO:tasks.workunit.client.0.smithi156.stdout:7/534: rename dc/d44/d81/la1 to dc/d13/d24/lb6 0 2024-07-16T09:55:43.686 INFO:tasks.workunit.client.0.smithi156.stdout:6/565: fsync d3/d13/d17/d4f/d5c/f72 0 2024-07-16T09:55:43.687 INFO:tasks.workunit.client.0.smithi156.stdout:2/680: dwrite d0/d13/d17/d1c/d86/f94 [0,4194304] 0 2024-07-16T09:55:43.687 INFO:tasks.workunit.client.1.smithi176.stdout:4/617: rename da/d3d/d17/d48 to da/d41/dc3 0 2024-07-16T09:55:43.689 INFO:tasks.workunit.client.1.smithi176.stdout:0/745: creat d5/d1c/d24/d35/d50/dd5/ff6 x:0 0 0 2024-07-16T09:55:43.690 INFO:tasks.workunit.client.1.smithi176.stdout:0/746: chown d5/d2c/fba 986 1 2024-07-16T09:55:43.691 INFO:tasks.workunit.client.1.smithi176.stdout:0/747: fsync d5/d1c/d24/d35/d50/d58/f79 0 2024-07-16T09:55:43.692 INFO:tasks.workunit.client.0.smithi156.stdout:4/734: dwrite d4/d23/d30/f2d [0,4194304] 0 2024-07-16T09:55:43.693 INFO:tasks.workunit.client.1.smithi176.stdout:4/618: creat da/dc/d10/d28/fc4 x:0 0 0 2024-07-16T09:55:43.696 INFO:tasks.workunit.client.0.smithi156.stdout:1/849: mkdir dc/d6b/d96/d11a 0 2024-07-16T09:55:43.697 INFO:tasks.workunit.client.1.smithi176.stdout:4/619: mknod da/dc/d10/d28/cc5 0 2024-07-16T09:55:43.697 INFO:tasks.workunit.client.1.smithi176.stdout:0/748: getdents d5/d1c/d24/d26/dbd/de6 0 2024-07-16T09:55:43.698 INFO:tasks.workunit.client.0.smithi156.stdout:4/735: mkdir d4/db/dc/dcc/dec/ded/d87/d108 0 2024-07-16T09:55:43.700 INFO:tasks.workunit.client.0.smithi156.stdout:9/680: dread d2/d5/d16/f9d [0,4194304] 0 2024-07-16T09:55:43.700 INFO:tasks.workunit.client.0.smithi156.stdout:3/595: dread d8/da/f5d [0,4194304] 0 2024-07-16T09:55:43.700 INFO:tasks.workunit.client.0.smithi156.stdout:2/681: rmdir d0/d13/d17 39 2024-07-16T09:55:43.702 INFO:tasks.workunit.client.1.smithi176.stdout:0/749: rename d5/d25/c7a to d5/d1c/d24/d35/d59/cf7 0 2024-07-16T09:55:43.703 INFO:tasks.workunit.client.1.smithi176.stdout:4/620: unlink da/dc/d10/d96/d9b/db4/d40/d5d/d73/fb1 0 2024-07-16T09:55:43.703 INFO:tasks.workunit.client.0.smithi156.stdout:7/535: dwrite dc/f94 [0,4194304] 0 2024-07-16T09:55:43.704 INFO:tasks.workunit.client.0.smithi156.stdout:9/681: symlink d2/d5/d61/ld4 0 2024-07-16T09:55:43.705 INFO:tasks.workunit.client.1.smithi176.stdout:0/750: fdatasync d5/d2c/d6c/f64 0 2024-07-16T09:55:43.705 INFO:tasks.workunit.client.0.smithi156.stdout:8/516: link d0/dd/d1f/d50/c89 d0/d11/d29/d5e/cb6 0 2024-07-16T09:55:43.707 INFO:tasks.workunit.client.0.smithi156.stdout:9/682: chown d2/d5/d16/d22/d2b/d74/dc2/dc4/d66/cce 31 1 2024-07-16T09:55:43.707 INFO:tasks.workunit.client.1.smithi176.stdout:4/621: link da/d41/d5b/c98 da/d41/cc6 0 2024-07-16T09:55:43.708 INFO:tasks.workunit.client.0.smithi156.stdout:2/682: chown d0/d13/d17/d1c/f24 216075 1 2024-07-16T09:55:43.709 INFO:tasks.workunit.client.0.smithi156.stdout:8/517: rmdir d0/d11/d20/d3f 39 2024-07-16T09:55:43.709 INFO:tasks.workunit.client.0.smithi156.stdout:7/536: write dc/f42 [5243174,38898] 0 2024-07-16T09:55:43.710 INFO:tasks.workunit.client.0.smithi156.stdout:8/518: write d0/dd/f1a [466017,40109] 0 2024-07-16T09:55:43.710 INFO:tasks.workunit.client.1.smithi176.stdout:0/751: rename d5/d1c/d24/d35/d59/dce to d5/d1c/d24/d26/dbd/de6/df8 0 2024-07-16T09:55:43.712 INFO:tasks.workunit.client.1.smithi176.stdout:0/752: stat d5/d44/ldb 0 2024-07-16T09:55:43.714 INFO:tasks.workunit.client.1.smithi176.stdout:0/753: fdatasync d5/d25/d82/f9a 0 2024-07-16T09:55:43.715 INFO:tasks.workunit.client.0.smithi156.stdout:9/683: creat d2/d5/d16/d57/d63/dd3/fd5 x:0 0 0 2024-07-16T09:55:43.716 INFO:tasks.workunit.client.0.smithi156.stdout:8/519: dwrite d0/dd/f26 [0,4194304] 0 2024-07-16T09:55:43.717 INFO:tasks.workunit.client.0.smithi156.stdout:4/736: dread d4/db/dc/dcc/d58/d8f/f101 [0,4194304] 0 2024-07-16T09:55:43.717 INFO:tasks.workunit.client.0.smithi156.stdout:9/684: fsync d2/d5/d16/d22/d2b/d74/dc2/dc4/d3a/f3b 0 2024-07-16T09:55:43.718 INFO:tasks.workunit.client.1.smithi176.stdout:4/622: dwrite da/dc/d10/d96/d9b/db4/f95 [0,4194304] 0 2024-07-16T09:55:43.719 INFO:tasks.workunit.client.0.smithi156.stdout:2/683: mknod d0/d5/d29/d2d/d5b/ce4 0 2024-07-16T09:55:43.719 INFO:tasks.workunit.client.1.smithi176.stdout:0/754: creat d5/d25/ff9 x:0 0 0 2024-07-16T09:55:43.721 INFO:tasks.workunit.client.0.smithi156.stdout:6/566: write d3/d13/d17/d4f/d5c/f65 [1149470,80159] 0 2024-07-16T09:55:43.721 INFO:tasks.workunit.client.1.smithi176.stdout:0/755: dread - d5/d39/fef zero size 2024-07-16T09:55:43.721 INFO:tasks.workunit.client.0.smithi156.stdout:8/520: mkdir d0/d11/d20/d7f/db7 0 2024-07-16T09:55:43.723 INFO:tasks.workunit.client.0.smithi156.stdout:6/567: write d3/d13/d17/d4f/d4d/d3b/f93 [2231989,58649] 0 2024-07-16T09:55:43.723 INFO:tasks.workunit.client.1.smithi176.stdout:0/756: fsync d5/d1c/d24/d35/d50/d58/d63/fd8 0 2024-07-16T09:55:43.725 INFO:tasks.workunit.client.0.smithi156.stdout:4/737: mkdir d4/d23/d52/d71/def/d109 0 2024-07-16T09:55:43.725 INFO:tasks.workunit.client.1.smithi176.stdout:4/623: getdents da/dc/d10/d96/da4 0 2024-07-16T09:55:43.729 INFO:tasks.workunit.client.0.smithi156.stdout:7/537: rename dc/d48/f92 to dc/d3e/fb7 0 2024-07-16T09:55:43.729 INFO:tasks.workunit.client.0.smithi156.stdout:1/850: truncate dc/d14/d3e/d55/d58/fc9 712858 0 2024-07-16T09:55:43.730 INFO:tasks.workunit.client.0.smithi156.stdout:7/538: chown dc/d63/d98 3218144 1 2024-07-16T09:55:43.731 INFO:tasks.workunit.client.0.smithi156.stdout:9/685: symlink d2/d5/d16/d57/d63/ld6 0 2024-07-16T09:55:43.732 INFO:tasks.workunit.client.0.smithi156.stdout:2/684: mknod d0/d13/dc9/ce5 0 2024-07-16T09:55:43.733 INFO:tasks.workunit.client.1.smithi176.stdout:0/757: dwrite d5/d25/d82/f9a [0,4194304] 0 2024-07-16T09:55:43.736 INFO:tasks.workunit.client.0.smithi156.stdout:1/851: rename dc/d14/d8b/db4/df2/fbf to dc/d14/d8b/dcd/f11b 0 2024-07-16T09:55:43.736 INFO:tasks.workunit.client.0.smithi156.stdout:2/685: chown d0/d5/d29/f56 7 1 2024-07-16T09:55:43.737 INFO:tasks.workunit.client.1.smithi176.stdout:0/758: rmdir d5/d1c/d24/d35/d50/d9b/de3 39 2024-07-16T09:55:43.738 INFO:tasks.workunit.client.0.smithi156.stdout:3/596: dwrite d8/da/d28/f2a [0,4194304] 0 2024-07-16T09:55:43.738 INFO:tasks.workunit.client.0.smithi156.stdout:1/852: fdatasync dc/d24/fc2 0 2024-07-16T09:55:43.738 INFO:tasks.workunit.client.0.smithi156.stdout:2/686: readlink d0/d13/d17/d1c/d51/le3 0 2024-07-16T09:55:43.742 INFO:tasks.workunit.client.1.smithi176.stdout:0/759: mknod d5/d1c/d24/d35/d50/d9b/de3/cfa 0 2024-07-16T09:55:43.744 INFO:tasks.workunit.client.0.smithi156.stdout:7/539: rmdir dc/d3e/d3f/d51/d6c 39 2024-07-16T09:55:43.745 INFO:tasks.workunit.client.0.smithi156.stdout:9/686: mknod d2/d5/d61/da2/cd7 0 2024-07-16T09:55:43.745 INFO:tasks.workunit.client.1.smithi176.stdout:0/760: creat d5/d1c/d24/d35/d59/dc9/ffb x:0 0 0 2024-07-16T09:55:43.747 INFO:tasks.workunit.client.0.smithi156.stdout:6/568: fsync d3/d13/d17/d4f/d94/f10 0 2024-07-16T09:55:43.749 INFO:tasks.workunit.client.0.smithi156.stdout:3/597: dwrite d8/d14/f59 [0,4194304] 0 2024-07-16T09:55:43.754 INFO:tasks.workunit.client.1.smithi176.stdout:0/761: dwrite d5/d39/f9f [0,4194304] 0 2024-07-16T09:55:43.757 INFO:tasks.workunit.client.0.smithi156.stdout:4/738: dread d4/db/d8a/fc4 [0,4194304] 0 2024-07-16T09:55:43.758 INFO:tasks.workunit.client.0.smithi156.stdout:2/687: dwrite d0/d5/d29/fd9 [0,4194304] 0 2024-07-16T09:55:43.758 INFO:tasks.workunit.client.0.smithi156.stdout:1/853: rename dc/f19 to dc/d14/d8b/dcd/f11c 0 2024-07-16T09:55:43.759 INFO:tasks.workunit.client.0.smithi156.stdout:3/598: write d8/da/d28/d43/d63/fbd [335458,102578] 0 2024-07-16T09:55:43.759 INFO:tasks.workunit.client.0.smithi156.stdout:2/688: readlink d0/d5/d8b/l98 0 2024-07-16T09:55:43.761 INFO:tasks.workunit.client.0.smithi156.stdout:1/854: chown dc/d14/d8b/dcd/dd2/df9 57 1 2024-07-16T09:55:43.761 INFO:tasks.workunit.client.1.smithi176.stdout:0/762: getdents d5/d44 0 2024-07-16T09:55:43.762 INFO:tasks.workunit.client.0.smithi156.stdout:9/687: fsync d2/d5/d16/d22/d2b/d3d/f69 0 2024-07-16T09:55:43.763 INFO:tasks.workunit.client.0.smithi156.stdout:1/855: creat dc/d14/d3e/d55/d57/d76/f11d x:0 0 0 2024-07-16T09:55:43.764 INFO:tasks.workunit.client.1.smithi176.stdout:4/624: dwrite da/dc/d10/d96/d9b/db4/d40/d5d/d73/f61 [0,4194304] 0 2024-07-16T09:55:43.764 INFO:tasks.workunit.client.0.smithi156.stdout:6/569: dwrite d3/d13/d17/d4f/d30/f67 [0,4194304] 0 2024-07-16T09:55:43.766 INFO:tasks.workunit.client.0.smithi156.stdout:3/599: creat d8/da/d1d/d6f/fd1 x:0 0 0 2024-07-16T09:55:43.769 INFO:tasks.workunit.client.1.smithi176.stdout:4/625: chown da/dc/d10/d28 107 1 2024-07-16T09:55:43.770 INFO:tasks.workunit.client.0.smithi156.stdout:8/521: dwrite d0/d11/d29/f5d [0,4194304] 0 2024-07-16T09:55:43.774 INFO:tasks.workunit.client.0.smithi156.stdout:9/688: creat d2/d5/d16/d22/d2b/d74/dc2/dc4/d3a/fd8 x:0 0 0 2024-07-16T09:55:43.775 INFO:tasks.workunit.client.0.smithi156.stdout:3/600: rmdir d8/da/d13 39 2024-07-16T09:55:43.776 INFO:tasks.workunit.client.1.smithi176.stdout:0/763: dwrite d5/d1c/fd7 [0,4194304] 0 2024-07-16T09:55:43.777 INFO:tasks.workunit.client.1.smithi176.stdout:8/613: sync 2024-07-16T09:55:43.778 INFO:tasks.workunit.client.1.smithi176.stdout:9/768: sync 2024-07-16T09:55:43.778 INFO:tasks.workunit.client.1.smithi176.stdout:3/500: sync 2024-07-16T09:55:43.778 INFO:tasks.workunit.client.1.smithi176.stdout:6/605: sync 2024-07-16T09:55:43.778 INFO:tasks.workunit.client.1.smithi176.stdout:5/739: sync 2024-07-16T09:55:43.778 INFO:tasks.workunit.client.0.smithi156.stdout:2/689: creat d0/d13/d17/d1c/fe6 x:0 0 0 2024-07-16T09:55:43.781 INFO:tasks.workunit.client.0.smithi156.stdout:6/570: mkdir d3/d13/d17/d4f/d4d/dbb 0 2024-07-16T09:55:43.784 INFO:tasks.workunit.client.1.smithi176.stdout:0/764: creat d5/d25/da0/ffc x:0 0 0 2024-07-16T09:55:43.788 INFO:tasks.workunit.client.0.smithi156.stdout:3/601: symlink d8/da/d21/d80/ld2 0 2024-07-16T09:55:43.789 INFO:tasks.workunit.client.1.smithi176.stdout:9/769: write d5/d1d/d67/d6d/fd6 [5511809,33367] 0 2024-07-16T09:55:43.790 INFO:tasks.workunit.client.1.smithi176.stdout:6/606: read - d1/d5/fa9 zero size 2024-07-16T09:55:43.791 INFO:tasks.workunit.client.1.smithi176.stdout:0/765: readlink d5/d1c/d24/ddf/le0 0 2024-07-16T09:55:43.794 INFO:tasks.workunit.client.1.smithi176.stdout:4/626: dwrite da/d3d/f1b [4194304,4194304] 0 2024-07-16T09:55:43.794 INFO:tasks.workunit.client.1.smithi176.stdout:3/501: creat d3/d17/d1d/d6e/f9a x:0 0 0 2024-07-16T09:55:43.795 INFO:tasks.workunit.client.0.smithi156.stdout:7/540: rename dc/d13/d20/c28 to dc/d13/cb8 0 2024-07-16T09:55:43.795 INFO:tasks.workunit.client.1.smithi176.stdout:5/740: write d4/d37/d72/d38/f97 [846683,76380] 0 2024-07-16T09:55:43.795 INFO:tasks.workunit.client.1.smithi176.stdout:6/607: mkdir d1/d5/d47/d62/d77/dde 0 2024-07-16T09:55:43.797 INFO:tasks.workunit.client.0.smithi156.stdout:2/690: fdatasync d0/d5/d29/d2d/d5b/d62/fb3 0 2024-07-16T09:55:43.799 INFO:tasks.workunit.client.1.smithi176.stdout:6/608: fdatasync d1/d5/d47/d75/fbe 0 2024-07-16T09:55:43.799 INFO:tasks.workunit.client.1.smithi176.stdout:4/627: write da/d3d/d17/f8e [415111,66411] 0 2024-07-16T09:55:43.799 INFO:tasks.workunit.client.0.smithi156.stdout:6/571: mkdir d3/d13/d17/d4f/d4d/d3b/da5/dbc 0 2024-07-16T09:55:43.799 INFO:tasks.workunit.client.0.smithi156.stdout:9/689: mkdir d2/d5/d16/d22/d2b/d3d/da6/dd9 0 2024-07-16T09:55:43.803 INFO:tasks.workunit.client.1.smithi176.stdout:5/741: creat d4/d14/dc5/f10d x:0 0 0 2024-07-16T09:55:43.804 INFO:tasks.workunit.client.0.smithi156.stdout:7/541: dwrite dc/d13/d20/f99 [0,4194304] 0 2024-07-16T09:55:43.804 INFO:tasks.workunit.client.1.smithi176.stdout:8/614: dread d8/d20/d2a/db9/dcb/d34/d62/f7d [0,4194304] 0 2024-07-16T09:55:43.806 INFO:tasks.workunit.client.1.smithi176.stdout:5/742: dread - d4/d37/fd7 zero size 2024-07-16T09:55:43.806 INFO:tasks.workunit.client.1.smithi176.stdout:0/766: read d5/d1c/d5a/f6d [949897,45961] 0 2024-07-16T09:55:43.807 INFO:tasks.workunit.client.0.smithi156.stdout:8/522: rename d0/dd/d1f/d50/c89 to d0/dd/d1f/d60/cb8 0 2024-07-16T09:55:43.807 INFO:tasks.workunit.client.1.smithi176.stdout:4/628: read da/dc/d10/d96/d69/f9f [73311,95930] 0 2024-07-16T09:55:43.809 INFO:tasks.workunit.client.0.smithi156.stdout:2/691: unlink d0/d13/d17/d1c/lca 0 2024-07-16T09:55:43.812 INFO:tasks.workunit.client.1.smithi176.stdout:6/609: creat d1/d5d/d9e/d76/da5/fdf x:0 0 0 2024-07-16T09:55:43.815 INFO:tasks.workunit.client.0.smithi156.stdout:9/690: fdatasync d2/d5/d16/d22/d2b/d3d/f69 0 2024-07-16T09:55:43.816 INFO:tasks.workunit.client.1.smithi176.stdout:4/629: mknod da/dc/d10/d96/d9b/db4/cc7 0 2024-07-16T09:55:43.820 INFO:tasks.workunit.client.0.smithi156.stdout:7/542: dwrite dc/f14 [0,4194304] 0 2024-07-16T09:55:43.820 INFO:tasks.workunit.client.0.smithi156.stdout:1/856: write dc/d14/d3e/f85 [555499,75200] 0 2024-07-16T09:55:43.820 INFO:tasks.workunit.client.0.smithi156.stdout:8/523: dwrite d0/f78 [4194304,4194304] 0 2024-07-16T09:55:43.820 INFO:tasks.workunit.client.0.smithi156.stdout:6/572: link d3/d13/d17/d4f/d1e/f3d d3/d13/d17/d4f/d4d/d3b/fbd 0 2024-07-16T09:55:43.820 INFO:tasks.workunit.client.1.smithi176.stdout:6/610: readlink d1/d5d/d9e/d8f/la3 0 2024-07-16T09:55:43.821 INFO:tasks.workunit.client.0.smithi156.stdout:6/573: fdatasync d3/f25 0 2024-07-16T09:55:43.821 INFO:tasks.workunit.client.1.smithi176.stdout:9/770: getdents d5/d1d/d67/d6d 0 2024-07-16T09:55:43.822 INFO:tasks.workunit.client.0.smithi156.stdout:8/524: fsync d0/d11/d13/f76 0 2024-07-16T09:55:43.822 INFO:tasks.workunit.client.1.smithi176.stdout:0/767: symlink d5/d1c/d24/d26/lfd 0 2024-07-16T09:55:43.823 INFO:tasks.workunit.client.1.smithi176.stdout:0/768: write d5/d25/f4e [3055755,86705] 0 2024-07-16T09:55:43.824 INFO:tasks.workunit.client.0.smithi156.stdout:3/602: getdents d8/da/d13 0 2024-07-16T09:55:43.825 INFO:tasks.workunit.client.1.smithi176.stdout:5/743: mkdir d4/d9a/d10a/d10e 0 2024-07-16T09:55:43.826 INFO:tasks.workunit.client.1.smithi176.stdout:0/769: write d5/d1c/d24/d35/d50/d58/d63/db4/dcc/ff5 [595577,112583] 0 2024-07-16T09:55:43.827 INFO:tasks.workunit.client.0.smithi156.stdout:2/692: creat d0/d13/d17/d1c/fe7 x:0 0 0 2024-07-16T09:55:43.827 INFO:tasks.workunit.client.0.smithi156.stdout:8/525: dread - d0/d11/d4a/d55/d93/f9c zero size 2024-07-16T09:55:43.827 INFO:tasks.workunit.client.1.smithi176.stdout:8/615: link d8/d20/d2a/db9/dcb/d34/d62/fb1 d8/d14/d1e/d33/fd1 0 2024-07-16T09:55:43.828 INFO:tasks.workunit.client.0.smithi156.stdout:3/603: chown d8/da/d1d/fc6 0 1 2024-07-16T09:55:43.828 INFO:tasks.workunit.client.1.smithi176.stdout:8/616: chown d8/d14/d1e/d7c 4500864 1 2024-07-16T09:55:43.830 INFO:tasks.workunit.client.0.smithi156.stdout:2/693: readlink d0/d13/d17/d1c/d51/la9 0 2024-07-16T09:55:43.832 INFO:tasks.workunit.client.1.smithi176.stdout:6/611: creat d1/dc/d81/fe0 x:0 0 0 2024-07-16T09:55:43.832 INFO:tasks.workunit.client.0.smithi156.stdout:9/691: dwrite d2/d5/d16/d22/d2b/d74/dc2/dc4/d3a/fd8 [0,4194304] 0 2024-07-16T09:55:43.834 INFO:tasks.workunit.client.1.smithi176.stdout:8/617: symlink d8/d20/d2a/d31/d2f/ld2 0 2024-07-16T09:55:43.835 INFO:tasks.workunit.client.0.smithi156.stdout:8/526: creat d0/d11/d13/d1c/d24/fb9 x:0 0 0 2024-07-16T09:55:43.837 INFO:tasks.workunit.client.0.smithi156.stdout:2/694: dread d0/d13/f8d [0,4194304] 0 2024-07-16T09:55:43.837 INFO:tasks.workunit.client.1.smithi176.stdout:6/612: symlink d1/d5d/le1 0 2024-07-16T09:55:43.837 INFO:tasks.workunit.client.1.smithi176.stdout:9/771: creat d5/d1d/d20/d78/dd0/f106 x:0 0 0 2024-07-16T09:55:43.838 INFO:tasks.workunit.client.0.smithi156.stdout:3/604: creat d8/da/d1d/d90/fd3 x:0 0 0 2024-07-16T09:55:43.840 INFO:tasks.workunit.client.1.smithi176.stdout:8/618: write d8/d20/d2a/db9/dcb/d34/f84 [157449,38785] 0 2024-07-16T09:55:43.841 INFO:tasks.workunit.client.1.smithi176.stdout:6/613: chown d1/d5d/db9/fb6 5429 1 2024-07-16T09:55:43.841 INFO:tasks.workunit.client.0.smithi156.stdout:8/527: symlink d0/d11/d4a/lba 0 2024-07-16T09:55:43.842 INFO:tasks.workunit.client.0.smithi156.stdout:8/528: write d0/dd/f10 [9555458,80111] 0 2024-07-16T09:55:43.842 INFO:tasks.workunit.client.1.smithi176.stdout:0/770: dwrite d5/d1c/d24/d35/d50/d58/d63/fb9 [0,4194304] 0 2024-07-16T09:55:43.845 INFO:tasks.workunit.client.1.smithi176.stdout:9/772: symlink d5/l107 0 2024-07-16T09:55:43.845 INFO:tasks.workunit.client.1.smithi176.stdout:6/614: readlink d1/d5/d47/d75/lc8 0 2024-07-16T09:55:43.845 INFO:tasks.workunit.client.0.smithi156.stdout:9/692: creat d2/d5/d16/d22/d2b/d3d/fda x:0 0 0 2024-07-16T09:55:43.846 INFO:tasks.workunit.client.1.smithi176.stdout:8/619: dread - d8/d20/d2a/db9/dcb/d34/fcc zero size 2024-07-16T09:55:43.846 INFO:tasks.workunit.client.0.smithi156.stdout:8/529: dread d0/dd/f10 [4194304,4194304] 0 2024-07-16T09:55:43.847 INFO:tasks.workunit.client.0.smithi156.stdout:3/605: link d8/da/f2e d8/da/d28/d43/d71/da2/fd4 0 2024-07-16T09:55:43.848 INFO:tasks.workunit.client.1.smithi176.stdout:9/773: write d5/d1d/dc7/d89/fb9 [2210185,34451] 0 2024-07-16T09:55:43.848 INFO:tasks.workunit.client.1.smithi176.stdout:6/615: rmdir d1/d5/d7 39 2024-07-16T09:55:43.849 INFO:tasks.workunit.client.1.smithi176.stdout:8/620: readlink d8/d20/d2a/db9/dcb/d34/l51 0 2024-07-16T09:55:43.851 INFO:tasks.workunit.client.1.smithi176.stdout:9/774: chown d5/d1d/dd2/de3/d103 344 1 2024-07-16T09:55:43.852 INFO:tasks.workunit.client.1.smithi176.stdout:9/775: chown d5/d1d/d20/d78 13218 1 2024-07-16T09:55:43.852 INFO:tasks.workunit.client.1.smithi176.stdout:6/616: getdents d1/d5/d47/d62/d77/dde 0 2024-07-16T09:55:43.853 INFO:tasks.workunit.client.1.smithi176.stdout:9/776: chown d5/d1d/d20/lb6 12413724 1 2024-07-16T09:55:43.853 INFO:tasks.workunit.client.1.smithi176.stdout:8/621: creat d8/d20/d2a/db9/dcb/d34/d62/d88/fd3 x:0 0 0 2024-07-16T09:55:43.854 INFO:tasks.workunit.client.0.smithi156.stdout:8/530: mknod d0/d11/d20/d7f/daf/cbb 0 2024-07-16T09:55:43.854 INFO:tasks.workunit.client.1.smithi176.stdout:6/617: mknod d1/dc/d40/d71/ce2 0 2024-07-16T09:55:43.856 INFO:tasks.workunit.client.0.smithi156.stdout:8/531: creat d0/d11/d62/fbc x:0 0 0 2024-07-16T09:55:43.856 INFO:tasks.workunit.client.1.smithi176.stdout:8/622: creat d8/d20/d2a/d53/d93/fd4 x:0 0 0 2024-07-16T09:55:43.857 INFO:tasks.workunit.client.0.smithi156.stdout:8/532: write d0/f78 [5766454,91509] 0 2024-07-16T09:55:43.858 INFO:tasks.workunit.client.0.smithi156.stdout:3/606: link d8/d14/l36 d8/da/d28/ld5 0 2024-07-16T09:55:43.858 INFO:tasks.workunit.client.0.smithi156.stdout:8/533: chown d0/ff 3213 1 2024-07-16T09:55:43.864 INFO:tasks.workunit.client.0.smithi156.stdout:8/534: dwrite d0/d11/d4a/d55/d93/f9c [0,4194304] 0 2024-07-16T09:55:43.865 INFO:tasks.workunit.client.0.smithi156.stdout:8/535: write d0/d11/d13/f76 [1898389,46590] 0 2024-07-16T09:55:43.866 INFO:tasks.workunit.client.0.smithi156.stdout:8/536: chown d0/d11/d13/c51 3819 1 2024-07-16T09:55:43.867 INFO:tasks.workunit.client.0.smithi156.stdout:8/537: fsync d0/d11/d29/f79 0 2024-07-16T09:55:43.882 INFO:tasks.workunit.client.1.smithi176.stdout:3/502: truncate d3/d17/d83/f8a 1382490 0 2024-07-16T09:55:43.883 INFO:tasks.workunit.client.1.smithi176.stdout:4/630: read - da/dc/d10/d96/d9b/db4/fa5 zero size 2024-07-16T09:55:43.883 INFO:tasks.workunit.client.0.smithi156.stdout:7/543: truncate dc/f14 1872425 0 2024-07-16T09:55:43.884 INFO:tasks.workunit.client.0.smithi156.stdout:6/574: write d3/d13/d17/d4f/d5c/f72 [1407410,76194] 0 2024-07-16T09:55:43.885 INFO:tasks.workunit.client.1.smithi176.stdout:3/503: creat d3/da/f9b x:0 0 0 2024-07-16T09:55:43.885 INFO:tasks.workunit.client.1.smithi176.stdout:4/631: symlink da/dc/d10/d96/d69/lc8 0 2024-07-16T09:55:43.886 INFO:tasks.workunit.client.1.smithi176.stdout:3/504: chown d3/d17/d1d/d6e 13 1 2024-07-16T09:55:43.888 INFO:tasks.workunit.client.0.smithi156.stdout:6/575: mkdir d3/d13/d17/d4f/d4d/d3b/d5a/dbe 0 2024-07-16T09:55:43.888 INFO:tasks.workunit.client.1.smithi176.stdout:3/505: chown d3/d17/d1d/d4f/l70 89103 1 2024-07-16T09:55:43.888 INFO:tasks.workunit.client.1.smithi176.stdout:4/632: creat da/d41/fc9 x:0 0 0 2024-07-16T09:55:43.889 INFO:tasks.workunit.client.0.smithi156.stdout:2/695: write d0/d5/d8b/fc8 [352029,112793] 0 2024-07-16T09:55:43.890 INFO:tasks.workunit.client.1.smithi176.stdout:3/506: symlink d3/d17/d83/l9c 0 2024-07-16T09:55:43.891 INFO:tasks.workunit.client.0.smithi156.stdout:9/693: rename d2/d5/d16/d22/d2b to d2/d5/d16/d57/d63/dd3/ddb 0 2024-07-16T09:55:43.892 INFO:tasks.workunit.client.0.smithi156.stdout:5/650: sync 2024-07-16T09:55:43.892 INFO:tasks.workunit.client.1.smithi176.stdout:3/507: mknod d3/d17/d83/c9d 0 2024-07-16T09:55:43.893 INFO:tasks.workunit.client.0.smithi156.stdout:3/607: truncate d8/da/d1d/f58 1789627 0 2024-07-16T09:55:43.893 INFO:tasks.workunit.client.1.smithi176.stdout:5/744: truncate d4/d37/d72/d82/f93 2748224 0 2024-07-16T09:55:43.895 INFO:tasks.workunit.client.1.smithi176.stdout:3/508: creat d3/d17/d1d/d4d/f9e x:0 0 0 2024-07-16T09:55:43.895 INFO:tasks.workunit.client.0.smithi156.stdout:6/576: dwrite d3/d13/d17/d4f/d30/d45/f49 [0,4194304] 0 2024-07-16T09:55:43.895 INFO:tasks.workunit.client.1.smithi176.stdout:3/509: chown l0 2503 1 2024-07-16T09:55:43.896 INFO:tasks.workunit.client.0.smithi156.stdout:2/696: mkdir d0/d13/de8 0 2024-07-16T09:55:43.897 INFO:tasks.workunit.client.1.smithi176.stdout:0/771: truncate d5/d1c/d24/d35/d50/dd5/fda 2311736 0 2024-07-16T09:55:43.897 INFO:tasks.workunit.client.1.smithi176.stdout:5/745: link d4/d10/d17/cc8 d4/d10/d17/d10c/c10f 0 2024-07-16T09:55:43.898 INFO:tasks.workunit.client.0.smithi156.stdout:2/697: dread - d0/d13/d17/d1c/fbd zero size 2024-07-16T09:55:43.899 INFO:tasks.workunit.client.0.smithi156.stdout:3/608: getdents d8/da/d13/dca 0 2024-07-16T09:55:43.899 INFO:tasks.workunit.client.0.smithi156.stdout:8/538: write d0/dd/f3b [49592,75654] 0 2024-07-16T09:55:43.900 INFO:tasks.workunit.client.1.smithi176.stdout:3/510: dread d3/da/f41 [0,4194304] 0 2024-07-16T09:55:43.901 INFO:tasks.workunit.client.0.smithi156.stdout:3/609: write d8/da/d28/d43/d71/f8c [806242,90332] 0 2024-07-16T09:55:43.902 INFO:tasks.workunit.client.1.smithi176.stdout:5/746: unlink d4/d10/ff3 0 2024-07-16T09:55:43.904 INFO:tasks.workunit.client.1.smithi176.stdout:0/772: mkdir d5/d1c/d24/dfe 0 2024-07-16T09:55:43.905 INFO:tasks.workunit.client.0.smithi156.stdout:2/698: creat d0/d5/d76/da1/fe9 x:0 0 0 2024-07-16T09:55:43.907 INFO:tasks.workunit.client.0.smithi156.stdout:6/577: mkdir d3/d13/d17/d4f/dbf 0 2024-07-16T09:55:43.907 INFO:tasks.workunit.client.1.smithi176.stdout:8/623: dwrite d8/d23/f1b [0,4194304] 0 2024-07-16T09:55:43.908 INFO:tasks.workunit.client.0.smithi156.stdout:8/539: dwrite d0/d11/d2e/d35/f46 [0,4194304] 0 2024-07-16T09:55:43.909 INFO:tasks.workunit.client.1.smithi176.stdout:0/773: write d5/d1c/d24/d35/d50/d58/f79 [760739,39899] 0 2024-07-16T09:55:43.909 INFO:tasks.workunit.client.0.smithi156.stdout:9/694: getdents d2/d5/d16/d57/d63/dd3/ddb/d3d 0 2024-07-16T09:55:43.909 INFO:tasks.workunit.client.1.smithi176.stdout:8/624: stat d8/d20/d2a/d31/db4/cb5 0 2024-07-16T09:55:43.912 INFO:tasks.workunit.client.1.smithi176.stdout:0/774: fsync d5/d2c/fba 0 2024-07-16T09:55:43.912 INFO:tasks.workunit.client.1.smithi176.stdout:3/511: creat d3/d17/d1d/d4f/d6d/f9f x:0 0 0 2024-07-16T09:55:43.914 INFO:tasks.workunit.client.0.smithi156.stdout:2/699: unlink d0/d13/d17/f21 0 2024-07-16T09:55:43.915 INFO:tasks.workunit.client.1.smithi176.stdout:5/747: fsync d4/d37/d72/d38/d39/d9f/fe9 0 2024-07-16T09:55:43.915 INFO:tasks.workunit.client.1.smithi176.stdout:3/512: chown d3/da/d66 8134 1 2024-07-16T09:55:43.918 INFO:tasks.workunit.client.0.smithi156.stdout:7/544: dwrite dc/d3e/d3f/d45/fb3 [0,4194304] 0 2024-07-16T09:55:43.920 INFO:tasks.workunit.client.1.smithi176.stdout:8/625: dwrite d8/d20/d2a/f49 [4194304,4194304] 0 2024-07-16T09:55:43.924 INFO:tasks.workunit.client.0.smithi156.stdout:2/700: truncate d0/d5/d29/f32 1311964 0 2024-07-16T09:55:43.926 INFO:tasks.workunit.client.0.smithi156.stdout:6/578: dwrite d3/d13/d17/d4f/d1e/f98 [0,4194304] 0 2024-07-16T09:55:43.926 INFO:tasks.workunit.client.1.smithi176.stdout:5/748: creat d4/d14/dce/f110 x:0 0 0 2024-07-16T09:55:43.927 INFO:tasks.workunit.client.1.smithi176.stdout:4/633: dread da/dc/f50 [0,4194304] 0 2024-07-16T09:55:43.928 INFO:tasks.workunit.client.0.smithi156.stdout:8/540: creat d0/d11/d13/d1c/fbd x:0 0 0 2024-07-16T09:55:43.929 INFO:tasks.workunit.client.0.smithi156.stdout:2/701: chown d0/d5/d29/d2d/d5b/ce4 3634361 1 2024-07-16T09:55:43.933 INFO:tasks.workunit.client.0.smithi156.stdout:8/541: readlink d0/d11/d20/l44 0 2024-07-16T09:55:43.938 INFO:tasks.workunit.client.1.smithi176.stdout:8/626: fdatasync d8/d14/d1e/d33/f5b 0 2024-07-16T09:55:43.939 INFO:tasks.workunit.client.0.smithi156.stdout:7/545: dwrite dc/d13/d20/f3b [0,4194304] 0 2024-07-16T09:55:43.939 INFO:tasks.workunit.client.0.smithi156.stdout:2/702: mkdir d0/d5/d29/dea 0 2024-07-16T09:55:43.939 INFO:tasks.workunit.client.0.smithi156.stdout:8/542: chown d0/dd/d1f/d50/d8d/ca2 1124690073 1 2024-07-16T09:55:43.940 INFO:tasks.workunit.client.0.smithi156.stdout:2/703: chown d0/d13/d17 0 1 2024-07-16T09:55:43.941 INFO:tasks.workunit.client.0.smithi156.stdout:7/546: fsync dc/d44/f5c 0 2024-07-16T09:55:43.943 INFO:tasks.workunit.client.1.smithi176.stdout:3/513: rename d3/d17/d1d/d4d to d3/d17/d1d/d4f/d59/da0 0 2024-07-16T09:55:43.943 INFO:tasks.workunit.client.0.smithi156.stdout:8/543: write d0/d11/d2e/d35/f46 [624976,98558] 0 2024-07-16T09:55:43.944 INFO:tasks.workunit.client.0.smithi156.stdout:2/704: readlink d0/d5/d76/da1/la4 0 2024-07-16T09:55:43.946 INFO:tasks.workunit.client.0.smithi156.stdout:3/610: dwrite d8/da/d21/d80/f82 [0,4194304] 0 2024-07-16T09:55:43.948 INFO:tasks.workunit.client.0.smithi156.stdout:6/579: link d3/d13/d17/d4f/d4d/d3b/d77/fb2 d3/d13/d17/d4f/d94/d19/fc0 0 2024-07-16T09:55:43.949 INFO:tasks.workunit.client.1.smithi176.stdout:4/634: dwrite da/dc/d10/d96/d9b/db4/d40/d5d/f85 [0,4194304] 0 2024-07-16T09:55:43.954 INFO:tasks.workunit.client.1.smithi176.stdout:0/775: write d5/d1c/d24/d35/d50/d5b/f65 [4967263,74068] 0 2024-07-16T09:55:43.955 INFO:tasks.workunit.client.0.smithi156.stdout:2/705: creat d0/d5/d8b/feb x:0 0 0 2024-07-16T09:55:43.958 INFO:tasks.workunit.client.1.smithi176.stdout:5/749: truncate d4/d10/d17/f66 414495 0 2024-07-16T09:55:43.961 INFO:tasks.workunit.client.0.smithi156.stdout:8/544: mkdir d0/d11/d20/d3f/dbe 0 2024-07-16T09:55:43.963 INFO:tasks.workunit.client.1.smithi176.stdout:0/776: mknod d5/d1c/d24/d35/d50/d58/cff 0 2024-07-16T09:55:43.963 INFO:tasks.workunit.client.0.smithi156.stdout:5/651: dread d1b/d1f/d45/d52/d43/d85/d97/d37/f5a [0,4194304] 0 2024-07-16T09:55:43.963 INFO:tasks.workunit.client.0.smithi156.stdout:3/611: dwrite d8/da/d1d/d6f/fd1 [0,4194304] 0 2024-07-16T09:55:43.964 INFO:tasks.workunit.client.0.smithi156.stdout:2/706: readlink d0/d5/d76/da1/la8 0 2024-07-16T09:55:43.964 INFO:tasks.workunit.client.1.smithi176.stdout:0/777: write d5/d1c/d24/d35/d50/d58/f6f [1029943,82832] 0 2024-07-16T09:55:43.965 INFO:tasks.workunit.client.0.smithi156.stdout:9/695: truncate d2/d5/d16/d57/d63/dd3/ddb/d3d/d3c/d7f/f19 3492228 0 2024-07-16T09:55:43.966 INFO:tasks.workunit.client.0.smithi156.stdout:2/707: readlink d0/l15 0 2024-07-16T09:55:43.966 INFO:tasks.workunit.client.1.smithi176.stdout:5/750: rename d4/d37/d72/d38/d39/d9f/d100 to d4/d37/d72/d38/d102/d105/d111 0 2024-07-16T09:55:43.966 INFO:tasks.workunit.client.0.smithi156.stdout:0/688: sync 2024-07-16T09:55:43.966 INFO:tasks.workunit.client.0.smithi156.stdout:4/739: sync 2024-07-16T09:55:43.967 INFO:tasks.workunit.client.0.smithi156.stdout:5/652: mkdir d1b/d1f/d42/de0 0 2024-07-16T09:55:43.967 INFO:tasks.workunit.client.1.smithi176.stdout:4/635: creat da/fca x:0 0 0 2024-07-16T09:55:43.971 INFO:tasks.workunit.client.1.smithi176.stdout:0/778: creat d5/d1c/d24/d35/d50/d9b/de3/f100 x:0 0 0 2024-07-16T09:55:43.971 INFO:tasks.workunit.client.0.smithi156.stdout:8/545: link d0/d11/d2e/fa5 d0/dd/d1f/d57/fbf 0 2024-07-16T09:55:43.972 INFO:tasks.workunit.client.0.smithi156.stdout:5/653: truncate d1b/d99/dae/fd4 177912 0 2024-07-16T09:55:43.972 INFO:tasks.workunit.client.0.smithi156.stdout:2/708: write d0/d13/d17/d1c/d86/fb7 [4985911,92895] 0 2024-07-16T09:55:43.973 INFO:tasks.workunit.client.1.smithi176.stdout:4/636: mknod da/dc/d10/d96/d9b/db4/d40/ccb 0 2024-07-16T09:55:43.973 INFO:tasks.workunit.client.0.smithi156.stdout:4/740: rmdir d4/db/dc/d67 39 2024-07-16T09:55:43.973 INFO:tasks.workunit.client.0.smithi156.stdout:9/696: truncate d2/d5/ff 3730788 0 2024-07-16T09:55:43.974 INFO:tasks.workunit.client.0.smithi156.stdout:8/546: stat d0/d11/d20/d3f/d96/l99 0 2024-07-16T09:55:43.974 INFO:tasks.workunit.client.0.smithi156.stdout:4/741: fdatasync d4/db/dc/dcc/d58/f91 0 2024-07-16T09:55:43.974 INFO:tasks.workunit.client.0.smithi156.stdout:9/697: fsync d2/d5/f44 0 2024-07-16T09:55:43.976 INFO:tasks.workunit.client.0.smithi156.stdout:5/654: fdatasync f14 0 2024-07-16T09:55:43.977 INFO:tasks.workunit.client.1.smithi176.stdout:5/751: dwrite d4/fea [0,4194304] 0 2024-07-16T09:55:43.978 INFO:tasks.workunit.client.0.smithi156.stdout:9/698: rmdir d2 39 2024-07-16T09:55:43.979 INFO:tasks.workunit.client.1.smithi176.stdout:4/637: write da/dc/d10/d96/d9b/db4/f26 [1256278,117992] 0 2024-07-16T09:55:43.981 INFO:tasks.workunit.client.1.smithi176.stdout:4/638: dread - da/dc/d10/d28/f7f zero size 2024-07-16T09:55:43.981 INFO:tasks.workunit.client.0.smithi156.stdout:0/689: mkdir d0/d1a/d4d/d9/d27/d2c/d6d/d9d/ddc 0 2024-07-16T09:55:43.982 INFO:tasks.workunit.client.0.smithi156.stdout:8/547: mkdir d0/dd/d1f/d57/dc0 0 2024-07-16T09:55:43.983 INFO:tasks.workunit.client.0.smithi156.stdout:2/709: truncate d0/d5/d29/d2d/f40 1509385 0 2024-07-16T09:55:43.983 INFO:tasks.workunit.client.1.smithi176.stdout:0/779: dread d5/d39/f51 [0,4194304] 0 2024-07-16T09:55:43.984 INFO:tasks.workunit.client.1.smithi176.stdout:5/752: dread d4/d10/fb1 [0,4194304] 0 2024-07-16T09:55:43.988 INFO:tasks.workunit.client.1.smithi176.stdout:4/639: creat da/fcc x:0 0 0 2024-07-16T09:55:43.990 INFO:tasks.workunit.client.1.smithi176.stdout:5/753: mknod d4/c112 0 2024-07-16T09:55:43.992 INFO:tasks.workunit.client.1.smithi176.stdout:5/754: stat d4/d14/ddc/ce1 0 2024-07-16T09:55:43.994 INFO:tasks.workunit.client.0.smithi156.stdout:5/655: getdents d1b/d1f/d54/d72 0 2024-07-16T09:55:43.994 INFO:tasks.workunit.client.0.smithi156.stdout:4/742: dwrite d4/db/dc/dcc/d58/d8f/f7e [0,4194304] 0 2024-07-16T09:55:43.994 INFO:tasks.workunit.client.0.smithi156.stdout:6/580: write d3/d13/d17/d4f/d4d/d3b/d5a/f69 [1896523,86855] 0 2024-07-16T09:55:43.997 INFO:tasks.workunit.client.0.smithi156.stdout:9/699: write d2/d5/d61/da2/f93 [8968554,67018] 0 2024-07-16T09:55:43.998 INFO:tasks.workunit.client.1.smithi176.stdout:4/640: creat da/d3d/d33/fcd x:0 0 0 2024-07-16T09:55:43.998 INFO:tasks.workunit.client.1.smithi176.stdout:3/514: dwrite d3/d17/f51 [0,4194304] 0 2024-07-16T09:55:43.999 INFO:tasks.workunit.client.0.smithi156.stdout:8/548: symlink d0/d11/d20/d3f/dbe/lc1 0 2024-07-16T09:55:44.000 INFO:tasks.workunit.client.0.smithi156.stdout:3/612: truncate d8/da/d1d/d2c/f84 529570 0 2024-07-16T09:55:44.002 INFO:tasks.workunit.client.1.smithi176.stdout:3/515: fsync d3/d17/d1d/d4f/d59/da0/f3f 0 2024-07-16T09:55:44.003 INFO:tasks.workunit.client.1.smithi176.stdout:4/641: creat da/d3d/d17/d54/fce x:0 0 0 2024-07-16T09:55:44.004 INFO:tasks.workunit.client.0.smithi156.stdout:6/581: readlink d3/d13/d17/d4f/d1e/l89 0 2024-07-16T09:55:44.005 INFO:tasks.workunit.client.0.smithi156.stdout:8/549: mkdir d0/d11/d62/dc2 0 2024-07-16T09:55:44.007 INFO:tasks.workunit.client.0.smithi156.stdout:3/613: chown d8/da/d1d/db2/lbb 47241623 1 2024-07-16T09:55:44.007 INFO:tasks.workunit.client.0.smithi156.stdout:7/547: dread dc/d13/f35 [0,4194304] 0 2024-07-16T09:55:44.008 INFO:tasks.workunit.client.0.smithi156.stdout:1/857: sync 2024-07-16T09:55:44.008 INFO:tasks.workunit.client.1.smithi176.stdout:8/627: dread d8/d20/d2a/f3f [0,4194304] 0 2024-07-16T09:55:44.009 INFO:tasks.workunit.client.1.smithi176.stdout:4/642: truncate da/d3d/d17/f64 1781676 0 2024-07-16T09:55:44.009 INFO:tasks.workunit.client.0.smithi156.stdout:6/582: mkdir d3/d13/d17/d4f/d4d/d3b/d5a/dc1 0 2024-07-16T09:55:44.012 INFO:tasks.workunit.client.0.smithi156.stdout:2/710: write d0/d13/f50 [935999,123079] 0 2024-07-16T09:55:44.013 INFO:tasks.workunit.client.1.smithi176.stdout:4/643: chown da/dc/d10/d96 548 1 2024-07-16T09:55:44.013 INFO:tasks.workunit.client.1.smithi176.stdout:4/644: write da/dc/d10/d96/d69/fb0 [3544547,96005] 0 2024-07-16T09:55:44.014 INFO:tasks.workunit.client.0.smithi156.stdout:5/656: dwrite d1b/d1f/d45/d52/f68 [0,4194304] 0 2024-07-16T09:55:44.014 INFO:tasks.workunit.client.1.smithi176.stdout:5/755: dread d4/d37/dbb/fd3 [0,4194304] 0 2024-07-16T09:55:44.014 INFO:tasks.workunit.client.1.smithi176.stdout:8/628: symlink d8/d20/d2a/d31/d46/d6a/ld5 0 2024-07-16T09:55:44.016 INFO:tasks.workunit.client.1.smithi176.stdout:8/629: dread - d8/d20/d2a/d31/db4/fc8 zero size 2024-07-16T09:55:44.016 INFO:tasks.workunit.client.0.smithi156.stdout:8/550: chown d0/d11/d62/fa4 1230055977 1 2024-07-16T09:55:44.017 INFO:tasks.workunit.client.0.smithi156.stdout:0/690: dread d0/d1a/d4d/fda [0,4194304] 0 2024-07-16T09:55:44.017 INFO:tasks.workunit.client.1.smithi176.stdout:5/756: mkdir d4/d14/d3d/dfb/d113 0 2024-07-16T09:55:44.018 INFO:tasks.workunit.client.0.smithi156.stdout:9/700: creat d2/d5/d16/d57/d63/dd3/ddb/dd0/fdc x:0 0 0 2024-07-16T09:55:44.018 INFO:tasks.workunit.client.0.smithi156.stdout:4/743: truncate d4/db/dc/dcc/d58/fa6 351058 0 2024-07-16T09:55:44.019 INFO:tasks.workunit.client.0.smithi156.stdout:1/858: mknod dc/d14/d3e/c11e 0 2024-07-16T09:55:44.019 INFO:tasks.workunit.client.1.smithi176.stdout:5/757: read d4/d14/d3d/f41 [2589338,21074] 0 2024-07-16T09:55:44.020 INFO:tasks.workunit.client.0.smithi156.stdout:8/551: fsync d0/f78 0 2024-07-16T09:55:44.022 INFO:tasks.workunit.client.0.smithi156.stdout:7/548: creat dc/d13/d20/d4d/fb9 x:0 0 0 2024-07-16T09:55:44.023 INFO:tasks.workunit.client.1.smithi176.stdout:3/516: truncate d3/d17/f2f 1104570 0 2024-07-16T09:55:44.023 INFO:tasks.workunit.client.1.smithi176.stdout:5/758: rmdir d4/d9a/ded 39 2024-07-16T09:55:44.024 INFO:tasks.workunit.client.0.smithi156.stdout:5/657: creat d1b/d1f/d45/d52/d49/d4d/fe1 x:0 0 0 2024-07-16T09:55:44.024 INFO:tasks.workunit.client.1.smithi176.stdout:8/630: dwrite d8/d23/f1b [0,4194304] 0 2024-07-16T09:55:44.025 INFO:tasks.workunit.client.0.smithi156.stdout:5/658: stat f14 0 2024-07-16T09:55:44.025 INFO:tasks.workunit.client.0.smithi156.stdout:4/744: rename d4/db/dc to d4/db/dc/d10a 22 2024-07-16T09:55:44.026 INFO:tasks.workunit.client.1.smithi176.stdout:5/759: dread - d4/d14/dc5/def/df9/ffe zero size 2024-07-16T09:55:44.027 INFO:tasks.workunit.client.0.smithi156.stdout:5/659: write d1b/d1f/d45/d52/d2d/fb4 [154393,70308] 0 2024-07-16T09:55:44.027 INFO:tasks.workunit.client.1.smithi176.stdout:3/517: rmdir d3/d17/d1d/d4f 39 2024-07-16T09:55:44.028 INFO:tasks.workunit.client.0.smithi156.stdout:9/701: mkdir d2/d5/d16/d57/da3/ddd 0 2024-07-16T09:55:44.033 INFO:tasks.workunit.client.0.smithi156.stdout:7/549: link dc/d13/f1d dc/d48/fba 0 2024-07-16T09:55:44.034 INFO:tasks.workunit.client.1.smithi176.stdout:5/760: rename d4/d37/d72/d78 to d4/d37/d72/d38/d39/d9f/de8/d114 0 2024-07-16T09:55:44.034 INFO:tasks.workunit.client.0.smithi156.stdout:5/660: creat d1b/d1f/d45/d52/d43/fe2 x:0 0 0 2024-07-16T09:55:44.035 INFO:tasks.workunit.client.0.smithi156.stdout:4/745: dwrite d4/db/dc/dcc/d58/d8f/f5e [4194304,4194304] 0 2024-07-16T09:55:44.038 INFO:tasks.workunit.client.1.smithi176.stdout:5/761: symlink d4/d37/d72/d38/d102/d105/l115 0 2024-07-16T09:55:44.039 INFO:tasks.workunit.client.1.smithi176.stdout:5/762: chown d4/d10/d17/c77 108782 1 2024-07-16T09:55:44.040 INFO:tasks.workunit.client.0.smithi156.stdout:5/661: creat d1b/d1f/d42/d6d/d9f/fe3 x:0 0 0 2024-07-16T09:55:44.041 INFO:tasks.workunit.client.1.smithi176.stdout:8/631: link d8/d20/d2a/db9/dcb/d34/l8b d8/d20/d2a/d53/d9a/ld6 0 2024-07-16T09:55:44.042 INFO:tasks.workunit.client.0.smithi156.stdout:9/702: getdents d2/d5/d16/d22/d36 0 2024-07-16T09:55:44.043 INFO:tasks.workunit.client.0.smithi156.stdout:7/550: symlink dc/d13/d15/lbb 0 2024-07-16T09:55:44.045 INFO:tasks.workunit.client.1.smithi176.stdout:5/763: rename d4/d14/d3d/cfc to d4/d37/d72/d82/c116 0 2024-07-16T09:55:44.045 INFO:tasks.workunit.client.0.smithi156.stdout:9/703: mknod d2/d5/d61/cde 0 2024-07-16T09:55:44.046 INFO:tasks.workunit.client.1.smithi176.stdout:8/632: creat d8/d20/d2a/d31/d2f/da9/fd7 x:0 0 0 2024-07-16T09:55:44.047 INFO:tasks.workunit.client.1.smithi176.stdout:8/633: write d8/d14/f2d [820907,57018] 0 2024-07-16T09:55:44.048 INFO:tasks.workunit.client.0.smithi156.stdout:6/583: dwrite d3/d13/f8e [0,4194304] 0 2024-07-16T09:55:44.048 INFO:tasks.workunit.client.0.smithi156.stdout:7/551: creat dc/d48/fbc x:0 0 0 2024-07-16T09:55:44.051 INFO:tasks.workunit.client.1.smithi176.stdout:5/764: creat d4/f117 x:0 0 0 2024-07-16T09:55:44.053 INFO:tasks.workunit.client.0.smithi156.stdout:5/662: getdents d1b/d1f/d45/d52/d43/d85/d97/d37/d88 0 2024-07-16T09:55:44.054 INFO:tasks.workunit.client.0.smithi156.stdout:6/584: read d3/d13/d17/d4f/d1e/f20 [605047,42647] 0 2024-07-16T09:55:44.057 INFO:tasks.workunit.client.0.smithi156.stdout:8/552: truncate d0/dd/f26 3375585 0 2024-07-16T09:55:44.058 INFO:tasks.workunit.client.0.smithi156.stdout:0/691: truncate d0/d1a/d4d/d9/f15 820983 0 2024-07-16T09:55:44.058 INFO:tasks.workunit.client.1.smithi176.stdout:5/765: read d4/d14/d3d/f55 [322035,104637] 0 2024-07-16T09:55:44.059 INFO:tasks.workunit.client.0.smithi156.stdout:5/663: chown d1b/d1f/d54 3082 1 2024-07-16T09:55:44.060 INFO:tasks.workunit.client.0.smithi156.stdout:3/614: dread d8/da/d1d/d2c/f52 [0,4194304] 0 2024-07-16T09:55:44.061 INFO:tasks.workunit.client.0.smithi156.stdout:6/585: unlink d3/c53 0 2024-07-16T09:55:44.061 INFO:tasks.workunit.client.1.smithi176.stdout:5/766: mkdir d4/d10/d118 0 2024-07-16T09:55:44.062 INFO:tasks.workunit.client.0.smithi156.stdout:1/859: dread dc/d14/d8b/dcd/f11c [0,4194304] 0 2024-07-16T09:55:44.063 INFO:tasks.workunit.client.1.smithi176.stdout:5/767: creat d4/d14/dce/f119 x:0 0 0 2024-07-16T09:55:44.064 INFO:tasks.workunit.client.0.smithi156.stdout:0/692: mkdir d0/d1a/d4d/d5e/ddd 0 2024-07-16T09:55:44.064 INFO:tasks.workunit.client.0.smithi156.stdout:9/704: dwrite d2/d5/d16/d57/d63/dd3/ddb/d3d/f68 [4194304,4194304] 0 2024-07-16T09:55:44.066 INFO:tasks.workunit.client.1.smithi176.stdout:5/768: mknod d4/d37/d72/d38/d39/d9f/c11a 0 2024-07-16T09:55:44.068 INFO:tasks.workunit.client.1.smithi176.stdout:5/769: readlink d4/d37/d72/d38/d39/d9f/lcb 0 2024-07-16T09:55:44.070 INFO:tasks.workunit.client.0.smithi156.stdout:2/711: dread d0/d5/d29/f3c [0,4194304] 0 2024-07-16T09:55:44.070 INFO:tasks.workunit.client.0.smithi156.stdout:6/586: symlink d3/d13/d17/d4f/d4d/d3b/d5a/lc2 0 2024-07-16T09:55:44.070 INFO:tasks.workunit.client.1.smithi176.stdout:3/518: truncate d3/d17/f2f 1226814 0 2024-07-16T09:55:44.073 INFO:tasks.workunit.client.0.smithi156.stdout:1/860: read dc/d14/d3e/f85 [70071,113105] 0 2024-07-16T09:55:44.074 INFO:tasks.workunit.client.0.smithi156.stdout:9/705: write d2/d5/d16/d57/d63/dd3/ddb/f8b [962145,43955] 0 2024-07-16T09:55:44.075 INFO:tasks.workunit.client.0.smithi156.stdout:0/693: mkdir d0/d1a/d4d/d6e/dde 0 2024-07-16T09:55:44.075 INFO:tasks.workunit.client.0.smithi156.stdout:8/553: dread d0/d11/d13/d1c/d39/fa0 [0,4194304] 0 2024-07-16T09:55:44.075 INFO:tasks.workunit.client.1.smithi176.stdout:8/634: dread d8/d20/d2a/db9/dcb/d34/f84 [0,4194304] 0 2024-07-16T09:55:44.077 INFO:tasks.workunit.client.1.smithi176.stdout:5/770: getdents d4/d6c 0 2024-07-16T09:55:44.077 INFO:tasks.workunit.client.0.smithi156.stdout:6/587: fdatasync d3/d13/d17/d4f/d94/d19/d2e/f3e 0 2024-07-16T09:55:44.078 INFO:tasks.workunit.client.1.smithi176.stdout:5/771: chown d4/d14/dc5/fd1 3319709 1 2024-07-16T09:55:44.078 INFO:tasks.workunit.client.1.smithi176.stdout:5/772: readlink d4/d10/d17/l1d 0 2024-07-16T09:55:44.079 INFO:tasks.workunit.client.0.smithi156.stdout:8/554: readlink d0/d11/d4a/lba 0 2024-07-16T09:55:44.079 INFO:tasks.workunit.client.1.smithi176.stdout:8/635: mknod d8/d20/d2a/db9/dcb/d34/d62/d88/cd8 0 2024-07-16T09:55:44.081 INFO:tasks.workunit.client.0.smithi156.stdout:3/615: truncate d8/da/d1d/d2c/f84 1264797 0 2024-07-16T09:55:44.081 INFO:tasks.workunit.client.1.smithi176.stdout:5/773: creat d4/d37/d72/d38/d102/dff/dc7/f11b x:0 0 0 2024-07-16T09:55:44.084 INFO:tasks.workunit.client.0.smithi156.stdout:5/664: link d1b/d1f/d45/d52/d43/c77 d1b/d1f/ce4 0 2024-07-16T09:55:44.084 INFO:tasks.workunit.client.0.smithi156.stdout:7/552: dwrite dc/d3e/d3f/d45/f7d [0,4194304] 0 2024-07-16T09:55:44.085 INFO:tasks.workunit.client.0.smithi156.stdout:6/588: read d3/d13/d17/d4f/d30/d45/f49 [434377,36970] 0 2024-07-16T09:55:44.087 INFO:tasks.workunit.client.1.smithi176.stdout:5/774: unlink d4/d9a/ded/l101 0 2024-07-16T09:55:44.088 INFO:tasks.workunit.client.0.smithi156.stdout:6/589: stat d3/f39 0 2024-07-16T09:55:44.088 INFO:tasks.workunit.client.0.smithi156.stdout:8/555: creat d0/d11/d4a/fc3 x:0 0 0 2024-07-16T09:55:44.089 INFO:tasks.workunit.client.1.smithi176.stdout:3/519: dwrite d3/d17/d1d/d4f/d59/f3a [0,4194304] 0 2024-07-16T09:55:44.090 INFO:tasks.workunit.client.1.smithi176.stdout:5/775: dread d4/d10/d17/f2f [0,4194304] 0 2024-07-16T09:55:44.092 INFO:tasks.workunit.client.0.smithi156.stdout:9/706: unlink d2/d5/d16/d57/d63/dd3/ddb/d3d/d3c/d7f/l18 0 2024-07-16T09:55:44.095 INFO:tasks.workunit.client.1.smithi176.stdout:5/776: fsync d4/d37/d72/d82/fb0 0 2024-07-16T09:55:44.099 INFO:tasks.workunit.client.0.smithi156.stdout:0/694: dwrite d0/d22/d41/fce [4194304,4194304] 0 2024-07-16T09:55:44.099 INFO:tasks.workunit.client.0.smithi156.stdout:2/712: dwrite d0/d13/d17/f34 [4194304,4194304] 0 2024-07-16T09:55:44.100 INFO:tasks.workunit.client.1.smithi176.stdout:3/520: dwrite d3/f55 [0,4194304] 0 2024-07-16T09:55:44.100 INFO:tasks.workunit.client.1.smithi176.stdout:8/636: dwrite d8/d14/d1e/d33/f5b [0,4194304] 0 2024-07-16T09:55:44.102 INFO:tasks.workunit.client.0.smithi156.stdout:1/861: link dc/d24/l40 dc/d14/d1e/l11f 0 2024-07-16T09:55:44.102 INFO:tasks.workunit.client.0.smithi156.stdout:3/616: creat d8/da/d1d/d90/fd6 x:0 0 0 2024-07-16T09:55:44.108 INFO:tasks.workunit.client.0.smithi156.stdout:5/665: unlink d1b/d1f/d45/d52/c56 0 2024-07-16T09:55:44.110 INFO:tasks.workunit.client.1.smithi176.stdout:3/521: dwrite d3/da/f9b [0,4194304] 0 2024-07-16T09:55:44.111 INFO:tasks.workunit.client.0.smithi156.stdout:6/590: creat d3/d13/d17/d4f/d4d/d3b/d77/fc3 x:0 0 0 2024-07-16T09:55:44.113 INFO:tasks.workunit.client.0.smithi156.stdout:3/617: rmdir d8/da/d28/d43/d71/da2 39 2024-07-16T09:55:44.116 INFO:tasks.workunit.client.0.smithi156.stdout:2/713: mknod d0/d13/d17/d1c/d51/dc6/cec 0 2024-07-16T09:55:44.117 INFO:tasks.workunit.client.0.smithi156.stdout:0/695: symlink d0/d22/ldf 0 2024-07-16T09:55:44.118 INFO:tasks.workunit.client.0.smithi156.stdout:9/707: symlink d2/d5/d16/d57/d63/dd3/ddb/d74/dc2/dc4/d23/ldf 0 2024-07-16T09:55:44.119 INFO:tasks.workunit.client.1.smithi176.stdout:3/522: rename d3/d17/d1d/d4f/d59/f27 to d3/da/d66/fa1 0 2024-07-16T09:55:44.119 INFO:tasks.workunit.client.1.smithi176.stdout:8/637: link d8/d20/d2a/db9/dcb/d34/l54 d8/d20/d2a/d31/d2f/da9/ld9 0 2024-07-16T09:55:44.120 INFO:tasks.workunit.client.0.smithi156.stdout:7/553: dwrite dc/f69 [0,4194304] 0 2024-07-16T09:55:44.120 INFO:tasks.workunit.client.0.smithi156.stdout:6/591: creat d3/d13/fc4 x:0 0 0 2024-07-16T09:55:44.122 INFO:tasks.workunit.client.1.smithi176.stdout:3/523: fsync d3/da/d33/f52 0 2024-07-16T09:55:44.124 INFO:tasks.workunit.client.0.smithi156.stdout:5/666: mkdir d1b/d1f/d42/de0/de5 0 2024-07-16T09:55:44.124 INFO:tasks.workunit.client.0.smithi156.stdout:2/714: chown d0/d13/d17/d1c/l36 2192 1 2024-07-16T09:55:44.124 INFO:tasks.workunit.client.1.smithi176.stdout:8/638: creat d8/d20/d2a/db9/dcb/d34/d62/dab/fda x:0 0 0 2024-07-16T09:55:44.125 INFO:tasks.workunit.client.0.smithi156.stdout:7/554: read dc/d44/f6d [2979859,50260] 0 2024-07-16T09:55:44.127 INFO:tasks.workunit.client.0.smithi156.stdout:1/862: creat dc/d14/d8b/db4/df2/f120 x:0 0 0 2024-07-16T09:55:44.127 INFO:tasks.workunit.client.1.smithi176.stdout:3/524: mknod d3/d17/d1d/d4f/d59/d81/ca2 0 2024-07-16T09:55:44.129 INFO:tasks.workunit.client.1.smithi176.stdout:8/639: creat d8/d20/d2a/d31/d2f/da9/fdb x:0 0 0 2024-07-16T09:55:44.130 INFO:tasks.workunit.client.1.smithi176.stdout:8/640: dread - d8/d20/d2a/d53/fb7 zero size 2024-07-16T09:55:44.133 INFO:tasks.workunit.client.0.smithi156.stdout:3/618: creat d8/da/fd7 x:0 0 0 2024-07-16T09:55:44.134 INFO:tasks.workunit.client.1.smithi176.stdout:3/525: link d3/da/f9b d3/d17/d1d/d4f/fa3 0 2024-07-16T09:55:44.135 INFO:tasks.workunit.client.0.smithi156.stdout:5/667: symlink d1b/d1f/d54/dbe/dde/le6 0 2024-07-16T09:55:44.136 INFO:tasks.workunit.client.0.smithi156.stdout:6/592: creat d3/d13/d17/d4f/d94/fc5 x:0 0 0 2024-07-16T09:55:44.136 INFO:tasks.workunit.client.1.smithi176.stdout:8/641: rename d8/d14/d1e/d55/f6b to d8/d20/d2a/fdc 0 2024-07-16T09:55:44.137 INFO:tasks.workunit.client.0.smithi156.stdout:6/593: dread - d3/d13/d17/d4f/d4d/d3b/d5a/fb9 zero size 2024-07-16T09:55:44.138 INFO:tasks.workunit.client.1.smithi176.stdout:3/526: creat d3/d17/d1d/d6e/fa4 x:0 0 0 2024-07-16T09:55:44.138 INFO:tasks.workunit.client.0.smithi156.stdout:3/619: getdents d8 0 2024-07-16T09:55:44.139 INFO:tasks.workunit.client.0.smithi156.stdout:1/863: rename dc/d14/d8b/db4/df2/l61 to dc/d14/d3e/d6c/d7a/l121 0 2024-07-16T09:55:44.141 INFO:tasks.workunit.client.1.smithi176.stdout:8/642: symlink d8/d20/d2a/db9/dcb/d34/ldd 0 2024-07-16T09:55:44.141 INFO:tasks.workunit.client.1.smithi176.stdout:8/643: stat d8/d14/d1e/d33 0 2024-07-16T09:55:44.142 INFO:tasks.workunit.client.0.smithi156.stdout:1/864: symlink dc/d6b/d104/l122 0 2024-07-16T09:55:44.142 INFO:tasks.workunit.client.0.smithi156.stdout:3/620: mkdir d8/da/d28/d43/dd8 0 2024-07-16T09:55:44.144 INFO:tasks.workunit.client.0.smithi156.stdout:3/621: dread - d8/da/d21/d80/d99/fa3 zero size 2024-07-16T09:55:44.144 INFO:tasks.workunit.client.1.smithi176.stdout:5/777: dread d4/d37/d72/f53 [0,4194304] 0 2024-07-16T09:55:44.145 INFO:tasks.workunit.client.1.smithi176.stdout:5/778: readlink d4/d37/d72/d38/d39/d9f/lcb 0 2024-07-16T09:55:44.146 INFO:tasks.workunit.client.0.smithi156.stdout:1/865: unlink dc/d14/d3e/c100 0 2024-07-16T09:55:44.148 INFO:tasks.workunit.client.1.smithi176.stdout:8/644: getdents d8/d20/d2a/d53/d9a 0 2024-07-16T09:55:44.149 INFO:tasks.workunit.client.0.smithi156.stdout:7/555: fsync dc/f69 0 2024-07-16T09:55:44.149 INFO:tasks.workunit.client.0.smithi156.stdout:3/622: rename d8/da/d28/f5e to d8/da/d28/d43/fd9 0 2024-07-16T09:55:44.149 INFO:tasks.workunit.client.1.smithi176.stdout:5/779: creat d4/d37/d72/d38/d39/d9f/de8/f11c x:0 0 0 2024-07-16T09:55:44.152 INFO:tasks.workunit.client.0.smithi156.stdout:1/866: chown dc/d14/d3e/d6c/d7a/dcb/fe1 17 1 2024-07-16T09:55:44.152 INFO:tasks.workunit.client.0.smithi156.stdout:8/556: dread d0/d11/d20/d3f/fb1 [0,4194304] 0 2024-07-16T09:55:44.153 INFO:tasks.workunit.client.0.smithi156.stdout:5/668: dread d1b/d1f/f20 [4194304,4194304] 0 2024-07-16T09:55:44.153 INFO:tasks.workunit.client.1.smithi176.stdout:8/645: unlink d8/d14/d1e/d33/d9d/dad/lba 0 2024-07-16T09:55:44.153 INFO:tasks.workunit.client.1.smithi176.stdout:3/527: dread d3/da/d33/f53 [0,4194304] 0 2024-07-16T09:55:44.155 INFO:tasks.workunit.client.1.smithi176.stdout:3/528: chown d3/d17/d83/c9d 30 1 2024-07-16T09:55:44.155 INFO:tasks.workunit.client.0.smithi156.stdout:3/623: mkdir d8/da/d28/d43/dd8/dda 0 2024-07-16T09:55:44.156 INFO:tasks.workunit.client.0.smithi156.stdout:2/715: write d0/f91 [651424,88767] 0 2024-07-16T09:55:44.156 INFO:tasks.workunit.client.0.smithi156.stdout:1/867: creat dc/d14/d8b/dfc/f123 x:0 0 0 2024-07-16T09:55:44.157 INFO:tasks.workunit.client.1.smithi176.stdout:3/529: dread - d3/da/d33/f67 zero size 2024-07-16T09:55:44.157 INFO:tasks.workunit.client.0.smithi156.stdout:5/669: mknod d1b/d1f/d54/dbe/dde/ce7 0 2024-07-16T09:55:44.159 INFO:tasks.workunit.client.1.smithi176.stdout:3/530: chown d3/d17/d1d/d4f/l8f 13 1 2024-07-16T09:55:44.159 INFO:tasks.workunit.client.0.smithi156.stdout:2/716: truncate d0/d5/d29/f56 1335383 0 2024-07-16T09:55:44.161 INFO:tasks.workunit.client.0.smithi156.stdout:6/594: write d3/d13/d17/d4f/d94/f10 [5926490,128164] 0 2024-07-16T09:55:44.162 INFO:tasks.workunit.client.1.smithi176.stdout:8/646: dwrite d8/d20/d2a/f75 [4194304,4194304] 0 2024-07-16T09:55:44.163 INFO:tasks.workunit.client.0.smithi156.stdout:2/717: chown d0/d13/d17 1 1 2024-07-16T09:55:44.163 INFO:tasks.workunit.client.1.smithi176.stdout:8/647: write d8/fa [2824913,22543] 0 2024-07-16T09:55:44.165 INFO:tasks.workunit.client.0.smithi156.stdout:7/556: truncate dc/f40 643239 0 2024-07-16T09:55:44.165 INFO:tasks.workunit.client.0.smithi156.stdout:3/624: dread d8/da/d28/f2a [0,4194304] 0 2024-07-16T09:55:44.165 INFO:tasks.workunit.client.1.smithi176.stdout:8/648: readlink d8/d20/d2a/d61/la0 0 2024-07-16T09:55:44.167 INFO:tasks.workunit.client.0.smithi156.stdout:8/557: link d0/d11/d13/d1c/l5a d0/d11/d20/d3f/dbe/lc4 0 2024-07-16T09:55:44.169 INFO:tasks.workunit.client.1.smithi176.stdout:3/531: getdents d3/d17/d1d/d4f/d6d 0 2024-07-16T09:55:44.169 INFO:tasks.workunit.client.0.smithi156.stdout:7/557: fsync dc/d44/f6d 0 2024-07-16T09:55:44.170 INFO:tasks.workunit.client.1.smithi176.stdout:8/649: read d8/d14/d1e/d33/f78 [1941026,57896] 0 2024-07-16T09:55:44.172 INFO:tasks.workunit.client.1.smithi176.stdout:3/532: unlink d3/da/c16 0 2024-07-16T09:55:44.173 INFO:tasks.workunit.client.1.smithi176.stdout:3/533: chown d3/d17/d1d/d4f/d6d 11409 1 2024-07-16T09:55:44.173 INFO:tasks.workunit.client.0.smithi156.stdout:2/718: rename d0/d5/d29/d2d/d5b/f9c to d0/d13/d17/d1c/d51/fed 0 2024-07-16T09:55:44.174 INFO:tasks.workunit.client.1.smithi176.stdout:5/780: dread d4/d14/d3d/f55 [0,4194304] 0 2024-07-16T09:55:44.174 INFO:tasks.workunit.client.0.smithi156.stdout:7/558: mknod dc/d3e/d3f/d51/d8a/cbd 0 2024-07-16T09:55:44.175 INFO:tasks.workunit.client.1.smithi176.stdout:8/650: creat d8/d20/d2a/db9/dcb/d34/fde x:0 0 0 2024-07-16T09:55:44.175 INFO:tasks.workunit.client.1.smithi176.stdout:3/534: readlink d3/da/d33/l6b 0 2024-07-16T09:55:44.175 INFO:tasks.workunit.client.0.smithi156.stdout:3/625: dread d8/da/d13/f57 [0,4194304] 0 2024-07-16T09:55:44.175 INFO:tasks.workunit.client.0.smithi156.stdout:2/719: chown d0/d5/f93 0 1 2024-07-16T09:55:44.176 INFO:tasks.workunit.client.1.smithi176.stdout:8/651: readlink d8/d20/d2a/db9/dcb/l9b 0 2024-07-16T09:55:44.178 INFO:tasks.workunit.client.1.smithi176.stdout:3/535: creat d3/da/d84/fa5 x:0 0 0 2024-07-16T09:55:44.178 INFO:tasks.workunit.client.0.smithi156.stdout:7/559: read dc/d44/f5c [4448412,21673] 0 2024-07-16T09:55:44.180 INFO:tasks.workunit.client.1.smithi176.stdout:8/652: symlink d8/d20/d2a/d31/db4/ldf 0 2024-07-16T09:55:44.181 INFO:tasks.workunit.client.0.smithi156.stdout:3/626: symlink d8/da/ldb 0 2024-07-16T09:55:44.182 INFO:tasks.workunit.client.0.smithi156.stdout:1/868: dwrite dc/d14/d8b/dcd/f11b [0,4194304] 0 2024-07-16T09:55:44.183 INFO:tasks.workunit.client.1.smithi176.stdout:5/781: dwrite f2 [0,4194304] 0 2024-07-16T09:55:44.185 INFO:tasks.workunit.client.0.smithi156.stdout:7/560: chown dc/d3e/d3f/d51/fa0 4090 1 2024-07-16T09:55:44.186 INFO:tasks.workunit.client.1.smithi176.stdout:8/653: chown d8/d14/d1e/d33/cac 0 1 2024-07-16T09:55:44.186 INFO:tasks.workunit.client.0.smithi156.stdout:8/558: dread d0/d11/d13/d1c/d39/f73 [0,4194304] 0 2024-07-16T09:55:44.186 INFO:tasks.workunit.client.0.smithi156.stdout:3/627: dread - d8/da/d1d/f8d zero size 2024-07-16T09:55:44.188 INFO:tasks.workunit.client.1.smithi176.stdout:3/536: dwrite d3/da/d56/f5b [0,4194304] 0 2024-07-16T09:55:44.188 INFO:tasks.workunit.client.1.smithi176.stdout:5/782: creat d4/d37/d72/d38/d102/dbd/f11d x:0 0 0 2024-07-16T09:55:44.189 INFO:tasks.workunit.client.1.smithi176.stdout:8/654: mkdir d8/d14/d1e/d7c/d82/de0 0 2024-07-16T09:55:44.190 INFO:tasks.workunit.client.0.smithi156.stdout:1/869: write dc/d14/d3e/d6c/d7a/dcb/fe1 [188471,128475] 0 2024-07-16T09:55:44.192 INFO:tasks.workunit.client.1.smithi176.stdout:8/655: rmdir d8/d20/d2a/db9/dcb/d34/d62 39 2024-07-16T09:55:44.193 INFO:tasks.workunit.client.1.smithi176.stdout:8/656: chown d8/d20/ca6 20 1 2024-07-16T09:55:44.197 INFO:tasks.workunit.client.1.smithi176.stdout:5/783: creat d4/d37/d72/f11e x:0 0 0 2024-07-16T09:55:44.198 INFO:tasks.workunit.client.1.smithi176.stdout:8/657: symlink d8/d20/d2a/db9/le1 0 2024-07-16T09:55:44.200 INFO:tasks.workunit.client.0.smithi156.stdout:7/561: write dc/d63/f88 [105207,75729] 0 2024-07-16T09:55:44.200 INFO:tasks.workunit.client.1.smithi176.stdout:5/784: mkdir d4/d14/dc5/def/d11f 0 2024-07-16T09:55:44.200 INFO:tasks.workunit.client.0.smithi156.stdout:5/670: dread f19 [0,4194304] 0 2024-07-16T09:55:44.201 INFO:tasks.workunit.client.0.smithi156.stdout:2/720: dwrite d0/d5/d29/d2d/d5b/d5f/f7d [4194304,4194304] 0 2024-07-16T09:55:44.202 INFO:tasks.workunit.client.1.smithi176.stdout:5/785: chown d4/d37/d72/d38/d39/d9f/de8 5479553 1 2024-07-16T09:55:44.203 INFO:tasks.workunit.client.0.smithi156.stdout:1/870: dread - dc/d14/d3e/d6c/d7a/dcb/f103 zero size 2024-07-16T09:55:44.203 INFO:tasks.workunit.client.0.smithi156.stdout:3/628: chown d8/da/d1d/d2c/d33/l7a 0 1 2024-07-16T09:55:44.205 INFO:tasks.workunit.client.0.smithi156.stdout:1/871: chown dc/f95 1607971 1 2024-07-16T09:55:44.206 INFO:tasks.workunit.client.1.smithi176.stdout:5/786: unlink d4/d10/d17/f18 0 2024-07-16T09:55:44.207 INFO:tasks.workunit.client.0.smithi156.stdout:2/721: fdatasync d0/d13/d17/d1c/d86/f94 0 2024-07-16T09:55:44.207 INFO:tasks.workunit.client.0.smithi156.stdout:1/872: mkdir dc/d24/d124 0 2024-07-16T09:55:44.208 INFO:tasks.workunit.client.1.smithi176.stdout:8/658: dwrite d8/d23/d76/d83/fb6 [0,4194304] 0 2024-07-16T09:55:44.209 INFO:tasks.workunit.client.0.smithi156.stdout:5/671: truncate d1b/d1f/d45/d52/d43/d85/d97/f25 4673383 0 2024-07-16T09:55:44.212 INFO:tasks.workunit.client.0.smithi156.stdout:1/873: chown dc/d14/d3e/d6c/d7a/fe7 1022983237 1 2024-07-16T09:55:44.213 INFO:tasks.workunit.client.0.smithi156.stdout:8/559: getdents d0/dd/d1f/d57/da6 0 2024-07-16T09:55:44.215 INFO:tasks.workunit.client.1.smithi176.stdout:8/659: read - d8/d20/d2a/db9/dcb/d34/d62/d88/fd3 zero size 2024-07-16T09:55:44.215 INFO:tasks.workunit.client.0.smithi156.stdout:3/629: dwrite d8/da/f32 [0,4194304] 0 2024-07-16T09:55:44.216 INFO:tasks.workunit.client.0.smithi156.stdout:5/672: symlink d1b/d1f/d54/dbe/le8 0 2024-07-16T09:55:44.216 INFO:tasks.workunit.client.0.smithi156.stdout:2/722: dread d0/d5/d29/f3c [0,4194304] 0 2024-07-16T09:55:44.217 INFO:tasks.workunit.client.0.smithi156.stdout:7/562: rename dc/d3e/fb7 to dc/d13/d15/d8b/fbe 0 2024-07-16T09:55:44.219 INFO:tasks.workunit.client.0.smithi156.stdout:3/630: fdatasync d8/da/d13/f75 0 2024-07-16T09:55:44.220 INFO:tasks.workunit.client.1.smithi176.stdout:8/660: symlink d8/d14/d1e/d55/le2 0 2024-07-16T09:55:44.220 INFO:tasks.workunit.client.1.smithi176.stdout:3/537: dread d3/da/d56/f5b [4194304,4194304] 0 2024-07-16T09:55:44.222 INFO:tasks.workunit.client.0.smithi156.stdout:5/673: truncate d1b/d1f/d45/d52/d49/d4d/fdc 462047 0 2024-07-16T09:55:44.222 INFO:tasks.workunit.client.1.smithi176.stdout:5/787: read d4/d14/d19/d1b/d61/f76 [2227480,19396] 0 2024-07-16T09:55:44.223 INFO:tasks.workunit.client.0.smithi156.stdout:3/631: fdatasync d8/da/f5d 0 2024-07-16T09:55:44.224 INFO:tasks.workunit.client.0.smithi156.stdout:8/560: dread d0/d11/d13/f14 [4194304,4194304] 0 2024-07-16T09:55:44.224 INFO:tasks.workunit.client.1.smithi176.stdout:8/661: mknod d8/d20/d2a/db9/dcb/d34/d62/d88/ce3 0 2024-07-16T09:55:44.225 INFO:tasks.workunit.client.0.smithi156.stdout:5/674: creat d1b/d1f/d54/d72/fe9 x:0 0 0 2024-07-16T09:55:44.226 INFO:tasks.workunit.client.0.smithi156.stdout:5/675: stat d1b/d1f/d45/d52/d49 0 2024-07-16T09:55:44.226 INFO:tasks.workunit.client.1.smithi176.stdout:3/538: dread d3/da/d56/f5b [4194304,4194304] 0 2024-07-16T09:55:44.227 INFO:tasks.workunit.client.1.smithi176.stdout:8/662: chown d8/d20/d2a/d31/d2f/l6d 1587 1 2024-07-16T09:55:44.227 INFO:tasks.workunit.client.0.smithi156.stdout:8/561: creat d0/d11/d62/fc5 x:0 0 0 2024-07-16T09:55:44.228 INFO:tasks.workunit.client.0.smithi156.stdout:2/723: rename d0/d13/d17/d1c/d86/fae to d0/fee 0 2024-07-16T09:55:44.229 INFO:tasks.workunit.client.1.smithi176.stdout:3/539: symlink d3/d17/d83/la6 0 2024-07-16T09:55:44.229 INFO:tasks.workunit.client.1.smithi176.stdout:8/663: creat d8/d20/d2a/d31/d46/d6a/da8/fe4 x:0 0 0 2024-07-16T09:55:44.229 INFO:tasks.workunit.client.0.smithi156.stdout:5/676: fsync d1b/d1f/d45/d7f/f84 0 2024-07-16T09:55:44.231 INFO:tasks.workunit.client.1.smithi176.stdout:3/540: mknod d3/d17/d1d/d4f/d6d/ca7 0 2024-07-16T09:55:44.231 INFO:tasks.workunit.client.1.smithi176.stdout:3/541: dread - d3/da/d84/fa5 zero size 2024-07-16T09:55:44.233 INFO:tasks.workunit.client.0.smithi156.stdout:7/563: rename dc/d13/d20/f84 to dc/d3e/d3f/fbf 0 2024-07-16T09:55:44.235 INFO:tasks.workunit.client.1.smithi176.stdout:8/664: read d8/d14/d1e/d7c/f85 [144562,96949] 0 2024-07-16T09:55:44.236 INFO:tasks.workunit.client.0.smithi156.stdout:5/677: write d1b/d1f/f73 [2484295,81763] 0 2024-07-16T09:55:44.236 INFO:tasks.workunit.client.0.smithi156.stdout:8/562: dwrite d0/dd/fb5 [0,4194304] 0 2024-07-16T09:55:44.237 INFO:tasks.workunit.client.0.smithi156.stdout:5/678: stat d1b/d1f/d54/dbe/dde/ce7 0 2024-07-16T09:55:44.242 INFO:tasks.workunit.client.1.smithi176.stdout:3/542: dwrite d3/d17/f88 [0,4194304] 0 2024-07-16T09:55:44.243 INFO:tasks.workunit.client.1.smithi176.stdout:8/665: mknod d8/d20/d2a/d31/ce5 0 2024-07-16T09:55:44.243 INFO:tasks.workunit.client.0.smithi156.stdout:8/563: creat d0/dd/d1f/fc6 x:0 0 0 2024-07-16T09:55:44.245 INFO:tasks.workunit.client.1.smithi176.stdout:3/543: creat d3/da/d7b/fa8 x:0 0 0 2024-07-16T09:55:44.245 INFO:tasks.workunit.client.1.smithi176.stdout:5/788: write d4/d14/fb4 [1737336,52133] 0 2024-07-16T09:55:44.246 INFO:tasks.workunit.client.0.smithi156.stdout:5/679: dwrite d1b/d1f/d45/d52/d43/d7c/fc0 [0,4194304] 0 2024-07-16T09:55:44.247 INFO:tasks.workunit.client.0.smithi156.stdout:5/680: fdatasync d1b/d1f/d42/f5c 0 2024-07-16T09:55:44.252 INFO:tasks.workunit.client.1.smithi176.stdout:5/789: unlink d4/d14/fc1 0 2024-07-16T09:55:44.255 INFO:tasks.workunit.client.1.smithi176.stdout:3/544: dread d3/da/f29 [0,4194304] 0 2024-07-16T09:55:44.257 INFO:tasks.workunit.client.1.smithi176.stdout:3/545: dread - d3/da/d33/f44 zero size 2024-07-16T09:55:44.258 INFO:tasks.workunit.client.0.smithi156.stdout:8/564: link d0/d11/d4a/d55/f7d d0/d11/fc7 0 2024-07-16T09:55:44.259 INFO:tasks.workunit.client.0.smithi156.stdout:8/565: chown d0/d11/d62/fc5 4950 1 2024-07-16T09:55:44.260 INFO:tasks.workunit.client.1.smithi176.stdout:3/546: write d3/d17/d1d/d4f/d6d/f9f [111186,10979] 0 2024-07-16T09:55:44.260 INFO:tasks.workunit.client.1.smithi176.stdout:5/790: link d4/d37/d72/d38/d39/fa3 d4/d14/d19/d1b/d61/de7/f120 0 2024-07-16T09:55:44.260 INFO:tasks.workunit.client.0.smithi156.stdout:8/566: readlink d0/d11/d4a/d55/l81 0 2024-07-16T09:55:44.261 INFO:tasks.workunit.client.0.smithi156.stdout:5/681: dread d1b/d1f/d45/d9d/fab [0,4194304] 0 2024-07-16T09:55:44.262 INFO:tasks.workunit.client.1.smithi176.stdout:3/547: rename d3/d17/d1d/d4f/d59 to d3/d17/d1d/d4f/d59/da0/d7a/da9 22 2024-07-16T09:55:44.262 INFO:tasks.workunit.client.0.smithi156.stdout:8/567: creat d0/d11/d20/d7f/fc8 x:0 0 0 2024-07-16T09:55:44.263 INFO:tasks.workunit.client.1.smithi176.stdout:3/548: truncate d3/d85/f99 5093691 0 2024-07-16T09:55:44.264 INFO:tasks.workunit.client.0.smithi156.stdout:2/724: truncate d0/d5/d29/d2d/f5a 504528 0 2024-07-16T09:55:44.268 INFO:tasks.workunit.client.0.smithi156.stdout:5/682: rename d1b/d1f/d45/d52/d49/d4d/f9b to d1b/d1f/d42/d6d/d9f/fea 0 2024-07-16T09:55:44.269 INFO:tasks.workunit.client.0.smithi156.stdout:2/725: creat d0/d5/d29/d2d/d5b/d5f/fef x:0 0 0 2024-07-16T09:55:44.270 INFO:tasks.workunit.client.0.smithi156.stdout:8/568: creat d0/d11/d62/dc2/fc9 x:0 0 0 2024-07-16T09:55:44.273 INFO:tasks.workunit.client.0.smithi156.stdout:2/726: creat d0/d13/d17/d1c/d51/dc6/ff0 x:0 0 0 2024-07-16T09:55:44.273 INFO:tasks.workunit.client.0.smithi156.stdout:8/569: symlink d0/dd/da3/dae/lca 0 2024-07-16T09:55:44.275 INFO:tasks.workunit.client.0.smithi156.stdout:2/727: dread d0/d5/d29/d2d/f73 [0,4194304] 0 2024-07-16T09:55:44.276 INFO:tasks.workunit.client.0.smithi156.stdout:8/570: mknod d0/dd/d1f/d57/da6/ccb 0 2024-07-16T09:55:44.276 INFO:tasks.workunit.client.0.smithi156.stdout:2/728: chown d0/d9b/cc7 239 1 2024-07-16T09:55:44.277 INFO:tasks.workunit.client.0.smithi156.stdout:2/729: dread - d0/d5/d76/f7b zero size 2024-07-16T09:55:44.279 INFO:tasks.workunit.client.0.smithi156.stdout:5/683: rmdir d1b/d1f/d45/d52/d49/d4d 39 2024-07-16T09:55:44.281 INFO:tasks.workunit.client.0.smithi156.stdout:5/684: readlink d1b/d1f/d45/d7a/lb5 0 2024-07-16T09:55:44.282 INFO:tasks.workunit.client.0.smithi156.stdout:5/685: write d1b/f40 [1464560,106494] 0 2024-07-16T09:55:44.285 INFO:tasks.workunit.client.0.smithi156.stdout:2/730: truncate d0/d13/dc9/fd1 700876 0 2024-07-16T09:55:44.291 INFO:tasks.workunit.client.0.smithi156.stdout:8/571: dwrite d0/d11/d20/d3f/fb1 [0,4194304] 0 2024-07-16T09:55:44.294 INFO:tasks.workunit.client.0.smithi156.stdout:8/572: write d0/d11/d4a/fc3 [766086,55014] 0 2024-07-16T09:55:44.298 INFO:tasks.workunit.client.0.smithi156.stdout:2/731: dwrite d0/d5/d29/d2d/d5b/d5f/fef [0,4194304] 0 2024-07-16T09:55:44.301 INFO:tasks.workunit.client.0.smithi156.stdout:2/732: chown d0/d5/d29/d2d/d5b/d5f/d89 1075799 1 2024-07-16T09:55:44.303 INFO:tasks.workunit.client.0.smithi156.stdout:8/573: truncate d0/d11/fc7 420913 0 2024-07-16T09:55:44.307 INFO:tasks.workunit.client.0.smithi156.stdout:8/574: symlink d0/d11/d13/d1c/lcc 0 2024-07-16T09:55:44.309 INFO:tasks.workunit.client.0.smithi156.stdout:8/575: write d0/d11/d20/d3f/fb1 [4870361,83724] 0 2024-07-16T09:55:44.310 INFO:tasks.workunit.client.0.smithi156.stdout:2/733: getdents d0/d13/d17/d1c/d51/dc6 0 2024-07-16T09:55:44.311 INFO:tasks.workunit.client.0.smithi156.stdout:2/734: chown d0/f7 96976866 1 2024-07-16T09:55:44.312 INFO:tasks.workunit.client.0.smithi156.stdout:2/735: write d0/f91 [2020857,35885] 0 2024-07-16T09:55:44.314 INFO:tasks.workunit.client.1.smithi176.stdout:3/549: write d3/da/f29 [1855836,125067] 0 2024-07-16T09:55:44.315 INFO:tasks.workunit.client.1.smithi176.stdout:3/550: chown l0 1114 1 2024-07-16T09:55:44.320 INFO:tasks.workunit.client.0.smithi156.stdout:2/736: write d0/d5/d8b/f92 [232464,38073] 0 2024-07-16T09:55:44.320 INFO:tasks.workunit.client.1.smithi176.stdout:3/551: getdents d3/d85 0 2024-07-16T09:55:44.321 INFO:tasks.workunit.client.1.smithi176.stdout:3/552: chown d3/d17/d1d/d4f/d59/f93 78134 1 2024-07-16T09:55:44.323 INFO:tasks.workunit.client.0.smithi156.stdout:2/737: unlink d0/d5/d8b/f92 0 2024-07-16T09:55:44.325 INFO:tasks.workunit.client.1.smithi176.stdout:3/553: dread d3/d17/f88 [0,4194304] 0 2024-07-16T09:55:44.325 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:55:43 smithi156 ceph-mon[29302]: pgmap v253: 65 pgs: 65 active+clean; 2.3 GiB data, 8.5 GiB used, 528 GiB / 536 GiB avail; 75 MiB/s rd, 300 MiB/s wr, 534 op/s 2024-07-16T09:55:44.326 INFO:tasks.workunit.client.0.smithi156.stdout:8/576: dwrite d0/d11/f40 [0,4194304] 0 2024-07-16T09:55:44.331 INFO:tasks.workunit.client.0.smithi156.stdout:2/738: link d0/d13/d17/d1c/d51/dc6/cec d0/d13/d17/d1c/d51/dc6/cf1 0 2024-07-16T09:55:44.334 INFO:tasks.workunit.client.0.smithi156.stdout:2/739: unlink d0/d5/d8b/l98 0 2024-07-16T09:55:44.336 INFO:tasks.workunit.client.0.smithi156.stdout:2/740: mkdir d0/d5/d76/da1/df2 0 2024-07-16T09:55:44.339 INFO:tasks.workunit.client.0.smithi156.stdout:2/741: chown d0/d13/d17/d1c/d51/dc6/ff0 1 1 2024-07-16T09:55:44.342 INFO:tasks.workunit.client.0.smithi156.stdout:8/577: dwrite d0/d11/d29/d80/f88 [4194304,4194304] 0 2024-07-16T09:55:44.343 INFO:tasks.workunit.client.0.smithi156.stdout:2/742: write d0/d5/d29/d2d/d5b/d5f/fef [4013192,28244] 0 2024-07-16T09:55:44.343 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:55:43 smithi176 ceph-mon[36626]: pgmap v253: 65 pgs: 65 active+clean; 2.3 GiB data, 8.5 GiB used, 528 GiB / 536 GiB avail; 75 MiB/s rd, 300 MiB/s wr, 534 op/s 2024-07-16T09:55:44.345 INFO:tasks.workunit.client.0.smithi156.stdout:2/743: chown d0/d13/d17/f30 95950 1 2024-07-16T09:55:44.347 INFO:tasks.workunit.client.0.smithi156.stdout:2/744: dread - d0/d5/d29/d2d/d5b/d5f/ddc/fcd zero size 2024-07-16T09:55:44.351 INFO:tasks.workunit.client.1.smithi176.stdout:7/654: sync 2024-07-16T09:55:44.352 INFO:tasks.workunit.client.0.smithi156.stdout:8/578: dwrite d0/d11/d20/f7a [0,4194304] 0 2024-07-16T09:55:44.352 INFO:tasks.workunit.client.0.smithi156.stdout:2/745: mkdir d0/d5/d29/d2d/d5b/df3 0 2024-07-16T09:55:44.361 INFO:tasks.workunit.client.0.smithi156.stdout:2/746: getdents d0/d13/d17/d1c 0 2024-07-16T09:55:44.361 INFO:tasks.workunit.client.1.smithi176.stdout:5/791: dwrite d4/d37/d72/d38/d39/d9f/fe9 [0,4194304] 0 2024-07-16T09:55:44.364 INFO:tasks.workunit.client.0.smithi156.stdout:8/579: dwrite d0/dd/d1f/d60/f98 [0,4194304] 0 2024-07-16T09:55:44.370 INFO:tasks.workunit.client.1.smithi176.stdout:7/655: rmdir d4/dd/d26 39 2024-07-16T09:55:44.370 INFO:tasks.workunit.client.1.smithi176.stdout:3/554: dwrite d3/da/d33/f39 [0,4194304] 0 2024-07-16T09:55:44.370 INFO:tasks.workunit.client.1.smithi176.stdout:7/656: dread - d4/d47/d5d/fc9 zero size 2024-07-16T09:55:44.370 INFO:tasks.workunit.client.1.smithi176.stdout:3/555: read - d3/f74 zero size 2024-07-16T09:55:44.371 INFO:tasks.workunit.client.0.smithi156.stdout:2/747: read d0/f8 [355409,38058] 0 2024-07-16T09:55:44.371 INFO:tasks.workunit.client.1.smithi176.stdout:7/657: stat d4/d47/d6d/d89/d7e/cbc 0 2024-07-16T09:55:44.372 INFO:tasks.workunit.client.0.smithi156.stdout:8/580: dwrite d0/d11/d62/fc5 [0,4194304] 0 2024-07-16T09:55:44.372 INFO:tasks.workunit.client.1.smithi176.stdout:7/658: stat d4/d1f/d22/c41 0 2024-07-16T09:55:44.376 INFO:tasks.workunit.client.0.smithi156.stdout:8/581: mkdir d0/d11/d62/dc2/dcd 0 2024-07-16T09:55:44.388 INFO:tasks.workunit.client.1.smithi176.stdout:5/792: dwrite d4/d10/d17/f50 [0,4194304] 0 2024-07-16T09:55:44.393 INFO:tasks.workunit.client.1.smithi176.stdout:5/793: symlink d4/d37/l121 0 2024-07-16T09:55:44.394 INFO:tasks.workunit.client.1.smithi176.stdout:5/794: dread - d4/d37/d72/d38/f70 zero size 2024-07-16T09:55:44.396 INFO:tasks.workunit.client.1.smithi176.stdout:3/556: dread d3/da/d66/fa1 [0,4194304] 0 2024-07-16T09:55:44.397 INFO:tasks.workunit.client.1.smithi176.stdout:7/659: write d4/dd/f2e [3270653,81333] 0 2024-07-16T09:55:44.398 INFO:tasks.workunit.client.1.smithi176.stdout:3/557: write d3/da/d33/f78 [525342,16080] 0 2024-07-16T09:55:44.398 INFO:tasks.workunit.client.1.smithi176.stdout:5/795: mkdir d4/da9/dc4/deb/d122 0 2024-07-16T09:55:44.398 INFO:tasks.workunit.client.1.smithi176.stdout:7/660: rmdir d4/d5/d97 39 2024-07-16T09:55:44.400 INFO:tasks.workunit.client.1.smithi176.stdout:3/558: write d3/d17/f90 [554744,113075] 0 2024-07-16T09:55:44.402 INFO:tasks.workunit.client.1.smithi176.stdout:3/559: write d3/da/f29 [367216,60377] 0 2024-07-16T09:55:44.404 INFO:tasks.workunit.client.1.smithi176.stdout:7/661: getdents d4/d47/d6d 0 2024-07-16T09:55:44.404 INFO:tasks.workunit.client.1.smithi176.stdout:3/560: mkdir d3/d17/d1d/d4f/daa 0 2024-07-16T09:55:44.408 INFO:tasks.workunit.client.0.smithi156.stdout:2/748: dwrite d0/d5/d29/d2d/d5b/d5f/d89/f9f [0,4194304] 0 2024-07-16T09:55:44.409 INFO:tasks.workunit.client.0.smithi156.stdout:8/582: dwrite d0/dd/f26 [0,4194304] 0 2024-07-16T09:55:44.409 INFO:tasks.workunit.client.1.smithi176.stdout:5/796: dread d4/f5 [0,4194304] 0 2024-07-16T09:55:44.409 INFO:tasks.workunit.client.0.smithi156.stdout:2/749: readlink d0/d13/d17/d1c/l1d 0 2024-07-16T09:55:44.410 INFO:tasks.workunit.client.0.smithi156.stdout:8/583: chown d0/d11/d29/d80/f9d 32 1 2024-07-16T09:55:44.411 INFO:tasks.workunit.client.1.smithi176.stdout:3/561: dwrite d3/d17/d1d/d4f/d59/da0/f9e [0,4194304] 0 2024-07-16T09:55:44.411 INFO:tasks.workunit.client.0.smithi156.stdout:2/750: truncate d0/d5/d8b/feb 1017347 0 2024-07-16T09:55:44.414 INFO:tasks.workunit.client.0.smithi156.stdout:8/584: readlink d0/d11/d20/d3f/dbe/lc4 0 2024-07-16T09:55:44.415 INFO:tasks.workunit.client.0.smithi156.stdout:2/751: mkdir d0/d13/d17/d1c/d86/dde/df4 0 2024-07-16T09:55:44.416 INFO:tasks.workunit.client.0.smithi156.stdout:2/752: chown d0/d13/d6e 3863 1 2024-07-16T09:55:44.418 INFO:tasks.workunit.client.0.smithi156.stdout:8/585: fsync d0/d11/d20/d3f/f69 0 2024-07-16T09:55:44.418 INFO:tasks.workunit.client.0.smithi156.stdout:2/753: fsync d0/d13/d17/d1c/fe7 0 2024-07-16T09:55:44.421 INFO:tasks.workunit.client.0.smithi156.stdout:8/586: unlink d0/f12 0 2024-07-16T09:55:44.422 INFO:tasks.workunit.client.1.smithi176.stdout:3/562: dwrite d3/d17/d1d/d6e/fa4 [0,4194304] 0 2024-07-16T09:55:44.423 INFO:tasks.workunit.client.0.smithi156.stdout:2/754: unlink d0/d5/f1e 0 2024-07-16T09:55:44.425 INFO:tasks.workunit.client.0.smithi156.stdout:2/755: chown d0/c61 14 1 2024-07-16T09:55:44.428 INFO:tasks.workunit.client.0.smithi156.stdout:2/756: creat d0/d13/d17/d1c/d86/ff5 x:0 0 0 2024-07-16T09:55:44.431 INFO:tasks.workunit.client.0.smithi156.stdout:2/757: mknod d0/d13/d17/cf6 0 2024-07-16T09:55:44.432 INFO:tasks.workunit.client.0.smithi156.stdout:4/746: sync 2024-07-16T09:55:44.433 INFO:tasks.workunit.client.0.smithi156.stdout:4/747: chown d4/db/dc/l10 146333 1 2024-07-16T09:55:44.434 INFO:tasks.workunit.client.0.smithi156.stdout:2/758: creat d0/d5/dc5/ff7 x:0 0 0 2024-07-16T09:55:44.434 INFO:tasks.workunit.client.0.smithi156.stdout:4/748: chown d4/d23/d52/d71/l89 838 1 2024-07-16T09:55:44.435 INFO:tasks.workunit.client.0.smithi156.stdout:4/749: read d4/db/dc/dcc/fb7 [74087,84709] 0 2024-07-16T09:55:44.436 INFO:tasks.workunit.client.1.smithi176.stdout:5/797: dread d4/d10/f54 [0,4194304] 0 2024-07-16T09:55:44.437 INFO:tasks.workunit.client.1.smithi176.stdout:5/798: dread - d4/d14/dce/f110 zero size 2024-07-16T09:55:44.439 INFO:tasks.workunit.client.0.smithi156.stdout:4/750: fsync d4/db/dc/f42 0 2024-07-16T09:55:44.440 INFO:tasks.workunit.client.0.smithi156.stdout:4/751: rmdir d4/d6a 39 2024-07-16T09:55:44.446 INFO:tasks.workunit.client.0.smithi156.stdout:4/752: dwrite d4/db/d8a/fc4 [0,4194304] 0 2024-07-16T09:55:44.447 INFO:tasks.workunit.client.0.smithi156.stdout:4/753: dread - d4/d23/d30/d33/f83 zero size 2024-07-16T09:55:44.449 INFO:tasks.workunit.client.0.smithi156.stdout:4/754: symlink d4/db/dc/d7c/l10b 0 2024-07-16T09:55:44.450 INFO:tasks.workunit.client.1.smithi176.stdout:5/799: write d4/d37/d72/d38/f99 [1561521,84501] 0 2024-07-16T09:55:44.452 INFO:tasks.workunit.client.0.smithi156.stdout:4/755: dread d4/d23/d30/f25 [0,4194304] 0 2024-07-16T09:55:44.455 INFO:tasks.workunit.client.0.smithi156.stdout:2/759: truncate d0/d5/d29/d2d/d5b/d5f/f66 2290864 0 2024-07-16T09:55:44.455 INFO:tasks.workunit.client.0.smithi156.stdout:2/760: fsync d0/fd2 0 2024-07-16T09:55:44.456 INFO:tasks.workunit.client.0.smithi156.stdout:4/756: symlink d4/d23/d52/d102/l10c 0 2024-07-16T09:55:44.459 INFO:tasks.workunit.client.0.smithi156.stdout:2/761: mkdir d0/d5/d29/d2d/d5b/d62/df8 0 2024-07-16T09:55:44.459 INFO:tasks.workunit.client.1.smithi176.stdout:3/563: dread d3/da/f13 [0,4194304] 0 2024-07-16T09:55:44.461 INFO:tasks.workunit.client.0.smithi156.stdout:2/762: mkdir d0/d5/d29/d2d/d5b/ddb/df9 0 2024-07-16T09:55:44.461 INFO:tasks.workunit.client.0.smithi156.stdout:8/587: dread d0/dd/d1f/f8c [0,4194304] 0 2024-07-16T09:55:44.462 INFO:tasks.workunit.client.1.smithi176.stdout:3/564: creat d3/d17/d1d/d6e/fab x:0 0 0 2024-07-16T09:55:44.462 INFO:tasks.workunit.client.1.smithi176.stdout:3/565: dread - d3/da/d33/f44 zero size 2024-07-16T09:55:44.463 INFO:tasks.workunit.client.1.smithi176.stdout:3/566: mkdir d3/d17/d1d/d4f/d59/da0/d7a/dac 0 2024-07-16T09:55:44.464 INFO:tasks.workunit.client.0.smithi156.stdout:8/588: symlink d0/dd/d1f/d57/db3/lce 0 2024-07-16T09:55:44.464 INFO:tasks.workunit.client.1.smithi176.stdout:3/567: fdatasync d3/d17/d1d/d4f/d59/da0/f3f 0 2024-07-16T09:55:44.465 INFO:tasks.workunit.client.0.smithi156.stdout:8/589: mknod d0/d11/d62/dc2/ccf 0 2024-07-16T09:55:44.466 INFO:tasks.workunit.client.1.smithi176.stdout:3/568: mknod d3/d17/d1d/d4f/cad 0 2024-07-16T09:55:44.468 INFO:tasks.workunit.client.1.smithi176.stdout:3/569: creat d3/d17/d1d/d4f/daa/fae x:0 0 0 2024-07-16T09:55:44.469 INFO:tasks.workunit.client.1.smithi176.stdout:3/570: read d3/da/d33/f39 [1717273,15456] 0 2024-07-16T09:55:44.470 INFO:tasks.workunit.client.0.smithi156.stdout:8/590: getdents d0/d11/d13/d1c/d9a 0 2024-07-16T09:55:44.471 INFO:tasks.workunit.client.0.smithi156.stdout:2/763: write d0/d5/d29/d2d/d5b/d5f/f66 [1254200,39262] 0 2024-07-16T09:55:44.472 INFO:tasks.workunit.client.0.smithi156.stdout:8/591: mknod d0/d11/d20/d3f/cd0 0 2024-07-16T09:55:44.473 INFO:tasks.workunit.client.1.smithi176.stdout:3/571: getdents d3/da/d84 0 2024-07-16T09:55:44.474 INFO:tasks.workunit.client.0.smithi156.stdout:2/764: chown d0/d5/d29/d2d/d5b/f6d 959 1 2024-07-16T09:55:44.474 INFO:tasks.workunit.client.1.smithi176.stdout:3/572: read d3/f55 [1566085,34384] 0 2024-07-16T09:55:44.476 INFO:tasks.workunit.client.0.smithi156.stdout:2/765: fsync d0/d13/d17/f30 0 2024-07-16T09:55:44.479 INFO:tasks.workunit.client.1.smithi176.stdout:3/573: dread d3/f61 [0,4194304] 0 2024-07-16T09:55:44.481 INFO:tasks.workunit.client.1.smithi176.stdout:3/574: rmdir d3/d17/d1d/d4f/d59/d81 39 2024-07-16T09:55:44.482 INFO:tasks.workunit.client.1.smithi176.stdout:3/575: write d3/d17/d83/f91 [874607,112865] 0 2024-07-16T09:55:44.484 INFO:tasks.workunit.client.0.smithi156.stdout:2/766: dwrite d0/d5/d29/d2d/d5b/f6f [0,4194304] 0 2024-07-16T09:55:44.484 INFO:tasks.workunit.client.0.smithi156.stdout:8/592: dread d0/ff [0,4194304] 0 2024-07-16T09:55:44.486 INFO:tasks.workunit.client.0.smithi156.stdout:8/593: creat d0/d11/d2e/fd1 x:0 0 0 2024-07-16T09:55:44.487 INFO:tasks.workunit.client.1.smithi176.stdout:3/576: dwrite d3/d17/f90 [0,4194304] 0 2024-07-16T09:55:44.490 INFO:tasks.workunit.client.0.smithi156.stdout:8/594: mkdir d0/d11/d4a/dd2 0 2024-07-16T09:55:44.491 INFO:tasks.workunit.client.0.smithi156.stdout:2/767: write d0/d5/d29/d2d/d5b/d5f/f7e [1809535,117415] 0 2024-07-16T09:55:44.493 INFO:tasks.workunit.client.0.smithi156.stdout:8/595: mkdir d0/d74/dd3 0 2024-07-16T09:55:44.494 INFO:tasks.workunit.client.0.smithi156.stdout:8/596: stat d0/d11/d13/l95 0 2024-07-16T09:55:44.495 INFO:tasks.workunit.client.0.smithi156.stdout:2/768: rename d0/d5/d29/d2d/d5b/f6f to d0/d5/d29/d2d/d5b/d5f/d89/ffa 0 2024-07-16T09:55:44.496 INFO:tasks.workunit.client.1.smithi176.stdout:3/577: write d3/da/d33/f52 [2851267,58900] 0 2024-07-16T09:55:44.497 INFO:tasks.workunit.client.0.smithi156.stdout:8/597: mkdir d0/d11/d20/d3f/dd4 0 2024-07-16T09:55:44.498 INFO:tasks.workunit.client.0.smithi156.stdout:2/769: creat d0/d5/dc5/ffb x:0 0 0 2024-07-16T09:55:44.499 INFO:tasks.workunit.client.0.smithi156.stdout:8/598: symlink d0/d74/dd3/ld5 0 2024-07-16T09:55:44.503 INFO:tasks.workunit.client.0.smithi156.stdout:2/770: rename d0/d5/d29/d2d/d5b/d62/df8 to d0/d13/de8/dfc 0 2024-07-16T09:55:44.504 INFO:tasks.workunit.client.0.smithi156.stdout:8/599: dread d0/d11/d4a/d55/d93/f9c [0,4194304] 0 2024-07-16T09:55:44.504 INFO:tasks.workunit.client.0.smithi156.stdout:2/771: dread - d0/d13/d17/d1c/d51/dc6/fd4 zero size 2024-07-16T09:55:44.508 INFO:tasks.workunit.client.0.smithi156.stdout:2/772: dread d0/d13/d17/d1c/f71 [0,4194304] 0 2024-07-16T09:55:44.513 INFO:tasks.workunit.client.0.smithi156.stdout:8/600: truncate d0/d11/d20/f7a 1458542 0 2024-07-16T09:55:44.514 INFO:tasks.workunit.client.0.smithi156.stdout:2/773: write d0/d5/d29/d2d/d5b/f6d [1341179,116074] 0 2024-07-16T09:55:44.517 INFO:tasks.workunit.client.1.smithi176.stdout:3/578: dread d3/da/fe [0,4194304] 0 2024-07-16T09:55:44.518 INFO:tasks.workunit.client.0.smithi156.stdout:8/601: rename d0/d11/d29/f5d to d0/d11/d4a/d55/fd6 0 2024-07-16T09:55:44.521 INFO:tasks.workunit.client.0.smithi156.stdout:8/602: rename d0/dd/da3/dae to d0/d11/d13/d1c/d39/d8a/dd7 0 2024-07-16T09:55:44.523 INFO:tasks.workunit.client.0.smithi156.stdout:8/603: symlink d0/d11/d29/ld8 0 2024-07-16T09:55:44.523 INFO:tasks.workunit.client.0.smithi156.stdout:2/774: dwrite d0/d13/d17/d1c/f48 [0,4194304] 0 2024-07-16T09:55:44.523 INFO:tasks.workunit.client.0.smithi156.stdout:8/604: readlink d0/d74/dd3/ld5 0 2024-07-16T09:55:44.525 INFO:tasks.workunit.client.0.smithi156.stdout:8/605: creat d0/d11/d4a/fd9 x:0 0 0 2024-07-16T09:55:44.526 INFO:tasks.workunit.client.0.smithi156.stdout:2/775: creat d0/d5/d29/d2d/d5b/d5f/d89/ffd x:0 0 0 2024-07-16T09:55:44.527 INFO:tasks.workunit.client.0.smithi156.stdout:2/776: chown d0/d13/de8 1023 1 2024-07-16T09:55:44.527 INFO:tasks.workunit.client.0.smithi156.stdout:2/777: readlink d0/d13/l58 0 2024-07-16T09:55:44.528 INFO:tasks.workunit.client.0.smithi156.stdout:2/778: dread - d0/d13/d17/d1c/fbd zero size 2024-07-16T09:55:44.531 INFO:tasks.workunit.client.0.smithi156.stdout:8/606: dwrite d0/dd/f10 [0,4194304] 0 2024-07-16T09:55:44.535 INFO:tasks.workunit.client.1.smithi176.stdout:1/455: sync 2024-07-16T09:55:44.552 INFO:tasks.workunit.client.0.smithi156.stdout:2/779: truncate d0/d5/d29/d2d/d5b/fc1 606216 0 2024-07-16T09:55:44.554 INFO:tasks.workunit.client.0.smithi156.stdout:8/607: write d0/d11/d4a/d55/fd6 [3698989,91136] 0 2024-07-16T09:55:44.558 INFO:tasks.workunit.client.0.smithi156.stdout:8/608: truncate d0/d11/d4a/d55/f7d 27566 0 2024-07-16T09:55:44.558 INFO:tasks.workunit.client.0.smithi156.stdout:2/780: link d0/d5/d29/d2d/d5b/d62/fb3 d0/d5/d76/ffe 0 2024-07-16T09:55:44.561 INFO:tasks.workunit.client.0.smithi156.stdout:2/781: fdatasync d0/d5/d29/f38 0 2024-07-16T09:55:44.565 INFO:tasks.workunit.client.0.smithi156.stdout:2/782: unlink d0/d5/d29/d2d/cad 0 2024-07-16T09:55:44.566 INFO:tasks.workunit.client.1.smithi176.stdout:1/456: mknod d4/d8/d12/d21/d34/c9a 0 2024-07-16T09:55:44.566 INFO:tasks.workunit.client.1.smithi176.stdout:1/457: chown d4/d8 380 1 2024-07-16T09:55:44.567 INFO:tasks.workunit.client.1.smithi176.stdout:1/458: chown d4/db/f1b 1230 1 2024-07-16T09:55:44.569 INFO:tasks.workunit.client.0.smithi156.stdout:2/783: truncate d0/d5/d29/d2d/f40 1122086 0 2024-07-16T09:55:44.569 INFO:tasks.workunit.client.1.smithi176.stdout:1/459: mknod d4/d3a/d54/d5a/d7a/c9b 0 2024-07-16T09:55:44.570 INFO:tasks.workunit.client.0.smithi156.stdout:2/784: readlink d0/d5/d76/da1/la8 0 2024-07-16T09:55:44.574 INFO:tasks.workunit.client.1.smithi176.stdout:1/460: creat d4/d8/d12/d21/f9c x:0 0 0 2024-07-16T09:55:44.574 INFO:tasks.workunit.client.1.smithi176.stdout:1/461: creat d4/d82/f9d x:0 0 0 2024-07-16T09:55:44.575 INFO:tasks.workunit.client.0.smithi156.stdout:2/785: creat d0/d13/fff x:0 0 0 2024-07-16T09:55:44.575 INFO:tasks.workunit.client.0.smithi156.stdout:8/609: dwrite d0/d11/d2e/f5c [0,4194304] 0 2024-07-16T09:55:44.577 INFO:tasks.workunit.client.0.smithi156.stdout:8/610: creat d0/d11/d2e/fda x:0 0 0 2024-07-16T09:55:44.579 INFO:tasks.workunit.client.1.smithi176.stdout:1/462: truncate d4/d8/d12/d21/d25/d30/d44/d4d/f96 2077916 0 2024-07-16T09:55:44.581 INFO:tasks.workunit.client.0.smithi156.stdout:8/611: symlink d0/dd/da3/ldb 0 2024-07-16T09:55:44.582 INFO:tasks.workunit.client.0.smithi156.stdout:2/786: getdents d0/d13/de8 0 2024-07-16T09:55:44.582 INFO:tasks.workunit.client.0.smithi156.stdout:2/787: readlink d0/d5/lb1 0 2024-07-16T09:55:44.583 INFO:tasks.workunit.client.0.smithi156.stdout:8/612: mknod d0/dd/d1f/d57/da6/cdc 0 2024-07-16T09:55:44.584 INFO:tasks.workunit.client.1.smithi176.stdout:1/463: link d4/d8/d12/d21/d34/f3d d4/d3a/f9e 0 2024-07-16T09:55:44.587 INFO:tasks.workunit.client.0.smithi156.stdout:8/613: link d0/d11/d62/cac d0/d11/d29/da8/cdd 0 2024-07-16T09:55:44.588 INFO:tasks.workunit.client.1.smithi176.stdout:1/464: rename d4/d8/d12/d21/d25/c47 to d4/db/c9f 0 2024-07-16T09:55:44.590 INFO:tasks.workunit.client.0.smithi156.stdout:2/788: dwrite d0/d13/fff [0,4194304] 0 2024-07-16T09:55:44.590 INFO:tasks.workunit.client.1.smithi176.stdout:1/465: creat d4/d8/d12/d8f/fa0 x:0 0 0 2024-07-16T09:55:44.592 INFO:tasks.workunit.client.1.smithi176.stdout:1/466: fsync d4/db/f5c 0 2024-07-16T09:55:44.594 INFO:tasks.workunit.client.0.smithi156.stdout:2/789: write d0/d13/d17/d1c/d51/fc2 [199560,51990] 0 2024-07-16T09:55:44.596 INFO:tasks.workunit.client.0.smithi156.stdout:8/614: dwrite d0/dd/fa7 [0,4194304] 0 2024-07-16T09:55:44.597 INFO:tasks.workunit.client.1.smithi176.stdout:1/467: link d4/l13 d4/d3a/d66/la1 0 2024-07-16T09:55:44.598 INFO:tasks.workunit.client.0.smithi156.stdout:8/615: dread - d0/d11/d62/fbc zero size 2024-07-16T09:55:44.602 INFO:tasks.workunit.client.0.smithi156.stdout:8/616: write d0/d11/d13/f76 [2103843,74308] 0 2024-07-16T09:55:44.604 INFO:tasks.workunit.client.1.smithi176.stdout:1/468: link d4/d8/d12/d21/d25/d30/d44/d4d/f7e d4/d8/d12/d21/fa2 0 2024-07-16T09:55:44.605 INFO:tasks.workunit.client.0.smithi156.stdout:2/790: dwrite d0/d9b/fba [0,4194304] 0 2024-07-16T09:55:44.607 INFO:tasks.workunit.client.0.smithi156.stdout:2/791: write d0/d13/d17/d1c/d86/fb7 [2216148,85037] 0 2024-07-16T09:55:44.610 INFO:tasks.workunit.client.0.smithi156.stdout:2/792: stat d0/d13/d17/d1c/d86/fb7 0 2024-07-16T09:55:44.612 INFO:tasks.workunit.client.1.smithi176.stdout:1/469: rename d4/d3a/d66/la1 to d4/d8/d12/d21/la3 0 2024-07-16T09:55:44.614 INFO:tasks.workunit.client.1.smithi176.stdout:1/470: write d4/d8/d12/d21/d25/d30/d44/f78 [352716,26927] 0 2024-07-16T09:55:44.615 INFO:tasks.workunit.client.1.smithi176.stdout:1/471: chown d4/d8/d12/c18 3122 1 2024-07-16T09:55:44.615 INFO:tasks.workunit.client.0.smithi156.stdout:2/793: unlink d0/d5/d29/d2d/d5b/d62/f6a 0 2024-07-16T09:55:44.616 INFO:tasks.workunit.client.1.smithi176.stdout:1/472: dread - d4/d8/d12/d21/d25/d30/d44/d4d/f7e zero size 2024-07-16T09:55:44.618 INFO:tasks.workunit.client.0.smithi156.stdout:8/617: dread d0/f6d [0,4194304] 0 2024-07-16T09:55:44.618 INFO:tasks.workunit.client.1.smithi176.stdout:1/473: dread d4/d3a/d54/d5a/f5b [0,4194304] 0 2024-07-16T09:55:44.618 INFO:tasks.workunit.client.0.smithi156.stdout:2/794: mknod d0/d5/d29/dea/c100 0 2024-07-16T09:55:44.624 INFO:tasks.workunit.client.1.smithi176.stdout:1/474: dread d4/d8/d12/d21/d25/d30/f61 [0,4194304] 0 2024-07-16T09:55:44.627 INFO:tasks.workunit.client.0.smithi156.stdout:0/696: sync 2024-07-16T09:55:44.628 INFO:tasks.workunit.client.0.smithi156.stdout:0/697: chown d0/ff 457 1 2024-07-16T09:55:44.629 INFO:tasks.workunit.client.1.smithi176.stdout:1/475: mkdir d4/d8/d12/d4c/da4 0 2024-07-16T09:55:44.629 INFO:tasks.workunit.client.0.smithi156.stdout:0/698: chown d0/d1a/d4d/d9/d27/d2c/d6d/f8b 80872711 1 2024-07-16T09:55:44.630 INFO:tasks.workunit.client.1.smithi176.stdout:1/476: write d4/db/f49 [4815883,127391] 0 2024-07-16T09:55:44.633 INFO:tasks.workunit.client.1.smithi176.stdout:1/477: truncate d4/d8/d12/d21/d25/d30/d44/f78 1078658 0 2024-07-16T09:55:44.635 INFO:tasks.workunit.client.0.smithi156.stdout:0/699: getdents d0/d22/d41/d52 0 2024-07-16T09:55:44.638 INFO:tasks.workunit.client.0.smithi156.stdout:0/700: getdents d0/d1a/d76/da7/dd3 0 2024-07-16T09:55:44.639 INFO:tasks.workunit.client.0.smithi156.stdout:0/701: chown d0/d22/d41/d52/f74 297469 1 2024-07-16T09:55:44.640 INFO:tasks.workunit.client.1.smithi176.stdout:1/478: truncate d4/d8/d12/d21/d25/d30/d44/d4d/f96 816841 0 2024-07-16T09:55:44.640 INFO:tasks.workunit.client.1.smithi176.stdout:1/479: stat d4/d3a/d54/d5a 0 2024-07-16T09:55:44.643 INFO:tasks.workunit.client.0.smithi156.stdout:0/702: truncate d0/d1a/d4d/d9/f26 4398313 0 2024-07-16T09:55:44.647 INFO:tasks.workunit.client.0.smithi156.stdout:0/703: rmdir d0/d1a/d4d/d5e/ddd 0 2024-07-16T09:55:44.649 INFO:tasks.workunit.client.0.smithi156.stdout:0/704: symlink d0/d1a/d76/da7/dd3/le0 0 2024-07-16T09:55:44.651 INFO:tasks.workunit.client.0.smithi156.stdout:0/705: mkdir d0/d1a/d4d/d61/d97/de1 0 2024-07-16T09:55:44.652 INFO:tasks.workunit.client.0.smithi156.stdout:0/706: write d0/d1a/d4d/d9/d27/d8d/fb6 [4390910,93919] 0 2024-07-16T09:55:44.653 INFO:tasks.workunit.client.0.smithi156.stdout:0/707: rmdir d0/d22 39 2024-07-16T09:55:44.655 INFO:tasks.workunit.client.0.smithi156.stdout:0/708: mknod d0/d1a/d4d/d9/d27/d3d/ce2 0 2024-07-16T09:55:44.658 INFO:tasks.workunit.client.0.smithi156.stdout:0/709: rename d0/d1a/d4d/d9/d68/dcb/fd4 to d0/d22/d41/d52/ddb/fe3 0 2024-07-16T09:55:44.659 INFO:tasks.workunit.client.0.smithi156.stdout:0/710: truncate d0/d1a/d4d/d9/d27/d50/f5f 4403073 0 2024-07-16T09:55:44.666 INFO:tasks.workunit.client.1.smithi176.stdout:2/799: sync 2024-07-16T09:55:44.666 INFO:tasks.workunit.client.1.smithi176.stdout:8/666: sync 2024-07-16T09:55:44.666 INFO:tasks.workunit.client.1.smithi176.stdout:9/777: sync 2024-07-16T09:55:44.666 INFO:tasks.workunit.client.1.smithi176.stdout:0/780: sync 2024-07-16T09:55:44.666 INFO:tasks.workunit.client.1.smithi176.stdout:4/645: sync 2024-07-16T09:55:44.666 INFO:tasks.workunit.client.1.smithi176.stdout:6/618: sync 2024-07-16T09:55:44.667 INFO:tasks.workunit.client.1.smithi176.stdout:2/800: chown dd/d2e/d35/fc6 24 1 2024-07-16T09:55:44.668 INFO:tasks.workunit.client.1.smithi176.stdout:8/667: read - d8/d20/d2a/d31/d46/d6a/da8/fe4 zero size 2024-07-16T09:55:44.668 INFO:tasks.workunit.client.1.smithi176.stdout:0/781: chown d5/d1c/d24/d35/d50/d58/d63/f66 92559 1 2024-07-16T09:55:44.668 INFO:tasks.workunit.client.1.smithi176.stdout:0/782: readlink d5/d39/l99 0 2024-07-16T09:55:44.668 INFO:tasks.workunit.client.1.smithi176.stdout:8/668: chown d8/f2e 130638 1 2024-07-16T09:55:44.669 INFO:tasks.workunit.client.1.smithi176.stdout:9/778: unlink d5/d52/leb 0 2024-07-16T09:55:44.669 INFO:tasks.workunit.client.1.smithi176.stdout:2/801: read - dd/d2e/d35/d4a/df9/f106 zero size 2024-07-16T09:55:44.669 INFO:tasks.workunit.client.1.smithi176.stdout:6/619: creat d1/d5d/db9/fe3 x:0 0 0 2024-07-16T09:55:44.670 INFO:tasks.workunit.client.1.smithi176.stdout:4/646: fdatasync da/dc/d10/d96/d69/f6d 0 2024-07-16T09:55:44.673 INFO:tasks.workunit.client.1.smithi176.stdout:9/779: truncate d5/d1d/d77/d97/fe6 1466839 0 2024-07-16T09:55:44.673 INFO:tasks.workunit.client.1.smithi176.stdout:0/783: read d5/d1c/d24/d35/d50/d5b/f65 [1567017,56261] 0 2024-07-16T09:55:44.673 INFO:tasks.workunit.client.1.smithi176.stdout:2/802: mknod dd/d36/d45/dd3/c107 0 2024-07-16T09:55:44.677 INFO:tasks.workunit.client.1.smithi176.stdout:4/647: dread da/dc/f83 [4194304,4194304] 0 2024-07-16T09:55:44.679 INFO:tasks.workunit.client.1.smithi176.stdout:8/669: rename d8/d20/f2b to d8/d20/d2a/d31/d2f/fe6 0 2024-07-16T09:55:44.680 INFO:tasks.workunit.client.1.smithi176.stdout:2/803: dwrite dd/d2e/f37 [0,4194304] 0 2024-07-16T09:55:44.681 INFO:tasks.workunit.client.1.smithi176.stdout:8/670: write d8/d23/f1b [2282646,85825] 0 2024-07-16T09:55:44.681 INFO:tasks.workunit.client.1.smithi176.stdout:0/784: creat d5/d1c/d24/d35/d50/d7c/f101 x:0 0 0 2024-07-16T09:55:44.681 INFO:tasks.workunit.client.1.smithi176.stdout:2/804: write dd/d49/d52/fb7 [701216,30422] 0 2024-07-16T09:55:44.687 INFO:tasks.workunit.client.1.smithi176.stdout:9/780: creat d5/d1d/d20/f108 x:0 0 0 2024-07-16T09:55:44.689 INFO:tasks.workunit.client.1.smithi176.stdout:0/785: symlink d5/d1c/d5a/d9e/l102 0 2024-07-16T09:55:44.690 INFO:tasks.workunit.client.1.smithi176.stdout:4/648: creat da/fcf x:0 0 0 2024-07-16T09:55:44.696 INFO:tasks.workunit.client.1.smithi176.stdout:4/649: creat da/d3d/d17/d54/fd0 x:0 0 0 2024-07-16T09:55:44.697 INFO:tasks.workunit.client.1.smithi176.stdout:2/805: dwrite dd/d16/d39/d3d/d7e/fdc [0,4194304] 0 2024-07-16T09:55:44.698 INFO:tasks.workunit.client.1.smithi176.stdout:6/620: dwrite d1/d5/d7/d54/f70 [0,4194304] 0 2024-07-16T09:55:44.699 INFO:tasks.workunit.client.1.smithi176.stdout:0/786: symlink d5/d1c/d24/d35/d50/l103 0 2024-07-16T09:55:44.700 INFO:tasks.workunit.client.1.smithi176.stdout:8/671: dwrite d8/d14/d1e/d33/d9d/fce [0,4194304] 0 2024-07-16T09:55:44.701 INFO:tasks.workunit.client.1.smithi176.stdout:0/787: readlink d5/d1c/d5a/d9e/l102 0 2024-07-16T09:55:44.703 INFO:tasks.workunit.client.1.smithi176.stdout:9/781: rename d5/de/da5/dbf/cc4 to d5/d1d/db0/dd3/c109 0 2024-07-16T09:55:44.704 INFO:tasks.workunit.client.1.smithi176.stdout:8/672: chown d8/d14/d1e/f60 1084 1 2024-07-16T09:55:44.709 INFO:tasks.workunit.client.1.smithi176.stdout:2/806: creat dd/d36/d82/f108 x:0 0 0 2024-07-16T09:55:44.710 INFO:tasks.workunit.client.1.smithi176.stdout:6/621: symlink d1/d5d/d9e/le4 0 2024-07-16T09:55:44.711 INFO:tasks.workunit.client.1.smithi176.stdout:0/788: rmdir d5/d1c/d24/d35/d50/d58/d63/db4 39 2024-07-16T09:55:44.715 INFO:tasks.workunit.client.1.smithi176.stdout:6/622: creat d1/d5/d47/d62/d80/fe5 x:0 0 0 2024-07-16T09:55:44.715 INFO:tasks.workunit.client.1.smithi176.stdout:0/789: symlink d5/d1c/db1/l104 0 2024-07-16T09:55:44.716 INFO:tasks.workunit.client.1.smithi176.stdout:9/782: fsync d5/d1d/d77/d98/fcd 0 2024-07-16T09:55:44.717 INFO:tasks.workunit.client.1.smithi176.stdout:2/807: mkdir dd/d2e/d109 0 2024-07-16T09:55:44.720 INFO:tasks.workunit.client.1.smithi176.stdout:2/808: write dd/d16/d39/d3d/f75 [1673861,20986] 0 2024-07-16T09:55:44.720 INFO:tasks.workunit.client.1.smithi176.stdout:8/673: getdents d8/d23/d76 0 2024-07-16T09:55:44.720 INFO:tasks.workunit.client.1.smithi176.stdout:6/623: creat d1/d5/d7/fe6 x:0 0 0 2024-07-16T09:55:44.723 INFO:tasks.workunit.client.1.smithi176.stdout:2/809: mkdir dd/d49/d52/d10a 0 2024-07-16T09:55:44.724 INFO:tasks.workunit.client.1.smithi176.stdout:6/624: truncate d1/d5/d47/fa7 1047469 0 2024-07-16T09:55:44.727 INFO:tasks.workunit.client.1.smithi176.stdout:4/650: write da/dc/d10/d96/d9b/db4/f29 [790730,35887] 0 2024-07-16T09:55:44.728 INFO:tasks.workunit.client.1.smithi176.stdout:2/810: symlink dd/d2e/l10b 0 2024-07-16T09:55:44.730 INFO:tasks.workunit.client.1.smithi176.stdout:8/674: getdents d8/d14/d1e/d33 0 2024-07-16T09:55:44.732 INFO:tasks.workunit.client.1.smithi176.stdout:2/811: chown dd/d2e/d35/d4a/lb9 249 1 2024-07-16T09:55:44.733 INFO:tasks.workunit.client.1.smithi176.stdout:4/651: truncate da/d3d/d17/f64 482198 0 2024-07-16T09:55:44.733 INFO:tasks.workunit.client.1.smithi176.stdout:2/812: chown dd/d16/d39/d3d/d7e/fd5 1032144062 1 2024-07-16T09:55:44.734 INFO:tasks.workunit.client.1.smithi176.stdout:6/625: dwrite d1/d5/d7/d1e/f79 [4194304,4194304] 0 2024-07-16T09:55:44.739 INFO:tasks.workunit.client.1.smithi176.stdout:2/813: mkdir dd/d2e/d104/d10c 0 2024-07-16T09:55:44.740 INFO:tasks.workunit.client.1.smithi176.stdout:2/814: readlink dd/d49/d52/d7c/d81/lae 0 2024-07-16T09:55:44.741 INFO:tasks.workunit.client.1.smithi176.stdout:0/790: dread d5/d2c/f2d [0,4194304] 0 2024-07-16T09:55:44.741 INFO:tasks.workunit.client.1.smithi176.stdout:2/815: fsync dd/d2e/d35/fc6 0 2024-07-16T09:55:44.742 INFO:tasks.workunit.client.1.smithi176.stdout:8/675: write d8/d20/d2a/d31/d2f/fe6 [971565,121881] 0 2024-07-16T09:55:44.744 INFO:tasks.workunit.client.1.smithi176.stdout:6/626: dread d1/d5d/db9/fcb [0,4194304] 0 2024-07-16T09:55:44.744 INFO:tasks.workunit.client.1.smithi176.stdout:2/816: write dd/d9b/fab [532528,985] 0 2024-07-16T09:55:44.745 INFO:tasks.workunit.client.1.smithi176.stdout:0/791: read d5/d1c/d24/d35/d50/d58/d63/f66 [1567356,43470] 0 2024-07-16T09:55:44.745 INFO:tasks.workunit.client.1.smithi176.stdout:8/676: dread - d8/d20/d2a/d31/d46/d6a/f7a zero size 2024-07-16T09:55:44.749 INFO:tasks.workunit.client.1.smithi176.stdout:8/677: creat d8/d20/d2a/d31/d2f/fe7 x:0 0 0 2024-07-16T09:55:44.750 INFO:tasks.workunit.client.1.smithi176.stdout:4/652: write da/d3d/d17/d54/d78/f7b [574420,97072] 0 2024-07-16T09:55:44.755 INFO:tasks.workunit.client.1.smithi176.stdout:0/792: getdents d5/d1c/d24/d26/dbd/de6/df8 0 2024-07-16T09:55:44.755 INFO:tasks.workunit.client.1.smithi176.stdout:8/678: write d8/d14/d1e/d33/d9d/fb8 [263814,130075] 0 2024-07-16T09:55:44.756 INFO:tasks.workunit.client.1.smithi176.stdout:4/653: creat da/dc/d10/d96/d69/fd1 x:0 0 0 2024-07-16T09:55:44.757 INFO:tasks.workunit.client.1.smithi176.stdout:6/627: dwrite d1/d5d/d9e/d76/f7b [0,4194304] 0 2024-07-16T09:55:44.757 INFO:tasks.workunit.client.1.smithi176.stdout:2/817: dwrite dd/dcd/dfa/ffc [0,4194304] 0 2024-07-16T09:55:44.757 INFO:tasks.workunit.client.0.smithi156.stdout:9/708: sync 2024-07-16T09:55:44.759 INFO:tasks.workunit.client.1.smithi176.stdout:2/818: stat dd/d49/d52 0 2024-07-16T09:55:44.760 INFO:tasks.workunit.client.1.smithi176.stdout:4/654: write da/d3d/d33/f3a [2273845,113951] 0 2024-07-16T09:55:44.761 INFO:tasks.workunit.client.1.smithi176.stdout:6/628: read d1/dc/f33 [211301,8802] 0 2024-07-16T09:55:44.761 INFO:tasks.workunit.client.1.smithi176.stdout:0/793: creat d5/d1c/d24/ddf/f105 x:0 0 0 2024-07-16T09:55:44.761 INFO:tasks.workunit.client.0.smithi156.stdout:9/709: mknod d2/d5/d16/d22/d6b/d86/ce0 0 2024-07-16T09:55:44.762 INFO:tasks.workunit.client.1.smithi176.stdout:8/679: write d8/d20/d2a/d31/d46/d6a/da8/fe4 [839738,68677] 0 2024-07-16T09:55:44.763 INFO:tasks.workunit.client.1.smithi176.stdout:0/794: readlink d5/d44/ld6 0 2024-07-16T09:55:44.765 INFO:tasks.workunit.client.1.smithi176.stdout:0/795: chown d5/d39/f43 749064 1 2024-07-16T09:55:44.765 INFO:tasks.workunit.client.1.smithi176.stdout:6/629: mkdir d1/d5d/d9e/dd9/de7 0 2024-07-16T09:55:44.766 INFO:tasks.workunit.client.1.smithi176.stdout:8/680: chown d8/d20/d2a/d31/d46/d6a/l9e 498434953 1 2024-07-16T09:55:44.767 INFO:tasks.workunit.client.1.smithi176.stdout:2/819: creat dd/f10d x:0 0 0 2024-07-16T09:55:44.768 INFO:tasks.workunit.client.1.smithi176.stdout:4/655: fdatasync da/dc/d10/d96/d9b/db4/fad 0 2024-07-16T09:55:44.768 INFO:tasks.workunit.client.1.smithi176.stdout:8/681: chown d8/d20/d2a/db9/fc2 491039418 1 2024-07-16T09:55:44.769 INFO:tasks.workunit.client.1.smithi176.stdout:4/656: chown da/dc/d10/d96/d69/f9f 214126 1 2024-07-16T09:55:44.769 INFO:tasks.workunit.client.1.smithi176.stdout:8/682: chown d8/d20/d2a/d31/d2f/l81 51280 1 2024-07-16T09:55:44.770 INFO:tasks.workunit.client.1.smithi176.stdout:8/683: chown d8/d14/d1e 28 1 2024-07-16T09:55:44.772 INFO:tasks.workunit.client.1.smithi176.stdout:2/820: creat dd/d49/d52/d89/f10e x:0 0 0 2024-07-16T09:55:44.773 INFO:tasks.workunit.client.1.smithi176.stdout:4/657: creat da/d3d/d33/fd2 x:0 0 0 2024-07-16T09:55:44.774 INFO:tasks.workunit.client.1.smithi176.stdout:8/684: creat d8/d20/d2a/d61/fe8 x:0 0 0 2024-07-16T09:55:44.774 INFO:tasks.workunit.client.1.smithi176.stdout:4/658: stat da/dc/d10/d96/d9b/db4/d40/d5d/d73/c77 0 2024-07-16T09:55:44.774 INFO:tasks.workunit.client.1.smithi176.stdout:0/796: creat d5/d1c/f106 x:0 0 0 2024-07-16T09:55:44.775 INFO:tasks.workunit.client.1.smithi176.stdout:2/821: symlink dd/d36/ded/l10f 0 2024-07-16T09:55:44.775 INFO:tasks.workunit.client.1.smithi176.stdout:2/822: fdatasync dd/d36/d82/fa9 0 2024-07-16T09:55:44.776 INFO:tasks.workunit.client.1.smithi176.stdout:8/685: rmdir d8/d20/d2a/d53/d9a/dc1 39 2024-07-16T09:55:44.777 INFO:tasks.workunit.client.1.smithi176.stdout:0/797: mknod d5/c107 0 2024-07-16T09:55:44.779 INFO:tasks.workunit.client.0.smithi156.stdout:9/710: dread d2/d5/d16/d57/d63/dd3/ddb/d3d/d3c/d7f/f19 [0,4194304] 0 2024-07-16T09:55:44.779 INFO:tasks.workunit.client.1.smithi176.stdout:6/630: rmdir d1 39 2024-07-16T09:55:44.782 INFO:tasks.workunit.client.0.smithi156.stdout:9/711: creat d2/d5/d16/d57/d63/dd3/ddb/d74/dc2/fe1 x:0 0 0 2024-07-16T09:55:44.783 INFO:tasks.workunit.client.1.smithi176.stdout:4/659: dwrite da/fcc [0,4194304] 0 2024-07-16T09:55:44.783 INFO:tasks.workunit.client.1.smithi176.stdout:8/686: creat d8/d20/d2a/d31/d46/fe9 x:0 0 0 2024-07-16T09:55:44.784 INFO:tasks.workunit.client.0.smithi156.stdout:9/712: mkdir d2/d5/d16/de2 0 2024-07-16T09:55:44.787 INFO:tasks.workunit.client.1.smithi176.stdout:2/823: dwrite dd/fef [0,4194304] 0 2024-07-16T09:55:44.787 INFO:tasks.workunit.client.1.smithi176.stdout:0/798: rename d5/d1c/d24/d35/d59/la9 to d5/d25/d82/l108 0 2024-07-16T09:55:44.789 INFO:tasks.workunit.client.1.smithi176.stdout:2/824: chown dd/d36/d82/f108 117399 1 2024-07-16T09:55:44.790 INFO:tasks.workunit.client.1.smithi176.stdout:4/660: mknod da/dc/cd3 0 2024-07-16T09:55:44.791 INFO:tasks.workunit.client.1.smithi176.stdout:3/579: sync 2024-07-16T09:55:44.791 INFO:tasks.workunit.client.1.smithi176.stdout:5/800: sync 2024-07-16T09:55:44.791 INFO:tasks.workunit.client.1.smithi176.stdout:7/662: sync 2024-07-16T09:55:44.794 INFO:tasks.workunit.client.1.smithi176.stdout:8/687: rmdir d8/d20/d2a/db9/dcb/d34 39 2024-07-16T09:55:44.797 INFO:tasks.workunit.client.1.smithi176.stdout:5/801: write d4/fd [4196650,128540] 0 2024-07-16T09:55:44.797 INFO:tasks.workunit.client.1.smithi176.stdout:0/799: creat d5/d1c/d24/d35/d50/d7c/f109 x:0 0 0 2024-07-16T09:55:44.798 INFO:tasks.workunit.client.1.smithi176.stdout:4/661: unlink da/d3d/d17/d54/fd0 0 2024-07-16T09:55:44.800 INFO:tasks.workunit.client.1.smithi176.stdout:2/825: mknod dd/d36/d45/de3/c110 0 2024-07-16T09:55:44.800 INFO:tasks.workunit.client.1.smithi176.stdout:5/802: chown d4/d10/f33 16526979 1 2024-07-16T09:55:44.800 INFO:tasks.workunit.client.1.smithi176.stdout:8/688: mkdir d8/d20/d2a/d31/d46/d6a/da8/dea 0 2024-07-16T09:55:44.801 INFO:tasks.workunit.client.0.smithi156.stdout:9/713: chown d2/d5/d16/d57/d63/dd3/ddb/d74/dc2/dc4/cc3 444 1 2024-07-16T09:55:44.801 INFO:tasks.workunit.client.1.smithi176.stdout:7/663: creat d4/dd/d81/fd9 x:0 0 0 2024-07-16T09:55:44.802 INFO:tasks.workunit.client.1.smithi176.stdout:0/800: creat d5/d25/d82/dec/f10a x:0 0 0 2024-07-16T09:55:44.802 INFO:tasks.workunit.client.1.smithi176.stdout:3/580: creat d3/d17/d1d/faf x:0 0 0 2024-07-16T09:55:44.803 INFO:tasks.workunit.client.0.smithi156.stdout:9/714: rmdir d2/d5/d16/d57/d63/dd3/ddb/d74/dc2/dc4/d23/da0 39 2024-07-16T09:55:44.804 INFO:tasks.workunit.client.1.smithi176.stdout:2/826: rmdir dd/d16/d39/d3d 39 2024-07-16T09:55:44.804 INFO:tasks.workunit.client.1.smithi176.stdout:5/803: write d4/d37/f84 [1587011,95494] 0 2024-07-16T09:55:44.804 INFO:tasks.workunit.client.1.smithi176.stdout:7/664: fdatasync d4/d47/f88 0 2024-07-16T09:55:44.805 INFO:tasks.workunit.client.1.smithi176.stdout:4/662: rename da/d3d/d33/f3a to da/d41/dc3/fd4 0 2024-07-16T09:55:44.805 INFO:tasks.workunit.client.0.smithi156.stdout:9/715: symlink d2/le3 0 2024-07-16T09:55:44.806 INFO:tasks.workunit.client.0.smithi156.stdout:9/716: stat d2/d5/d16/d57/d63/dd3/ddb/d3d/f69 0 2024-07-16T09:55:44.809 INFO:tasks.workunit.client.1.smithi176.stdout:7/665: symlink d4/d47/d5d/d9a/lda 0 2024-07-16T09:55:44.810 INFO:tasks.workunit.client.0.smithi156.stdout:9/717: creat d2/d5/d16/d22/d6b/d86/fe4 x:0 0 0 2024-07-16T09:55:44.811 INFO:tasks.workunit.client.1.smithi176.stdout:6/631: dread d1/dc/d40/f4d [0,4194304] 0 2024-07-16T09:55:44.813 INFO:tasks.workunit.client.1.smithi176.stdout:6/632: read - d1/d5/d47/d62/fa1 zero size 2024-07-16T09:55:44.814 INFO:tasks.workunit.client.0.smithi156.stdout:9/718: link d2/d5/d61/da2/d98/dbf/cc9 d2/d5/d61/da2/d98/dbf/ce5 0 2024-07-16T09:55:44.815 INFO:tasks.workunit.client.1.smithi176.stdout:0/801: dwrite d5/d1c/d24/d35/d50/d58/d63/db4/dcc/ff5 [0,4194304] 0 2024-07-16T09:55:44.816 INFO:tasks.workunit.client.0.smithi156.stdout:9/719: mkdir d2/d5/d16/d57/d63/dd3/ddb/d74/dc2/de6 0 2024-07-16T09:55:44.817 INFO:tasks.workunit.client.0.smithi156.stdout:9/720: write d2/d5/d16/d22/d6b/d86/f8c [3590427,83724] 0 2024-07-16T09:55:44.821 INFO:tasks.workunit.client.0.smithi156.stdout:9/721: mkdir d2/d5/d16/d57/d63/dd3/ddb/d74/dc2/dc4/d3a/de7 0 2024-07-16T09:55:44.822 INFO:tasks.workunit.client.0.smithi156.stdout:9/722: rmdir d2/d5/d61/da2/d98/dbf 39 2024-07-16T09:55:44.825 INFO:tasks.workunit.client.0.smithi156.stdout:9/723: symlink d2/d5/d16/d57/d63/dd3/ddb/d3d/da6/dd9/le8 0 2024-07-16T09:55:44.825 INFO:tasks.workunit.client.0.smithi156.stdout:9/724: chown d2/f15 1633441812 1 2024-07-16T09:55:44.827 INFO:tasks.workunit.client.0.smithi156.stdout:9/725: creat d2/d5/d16/d22/d36/fe9 x:0 0 0 2024-07-16T09:55:44.827 INFO:tasks.workunit.client.0.smithi156.stdout:9/726: stat d2/d5/d61/da2/f93 0 2024-07-16T09:55:44.836 INFO:tasks.workunit.client.0.smithi156.stdout:9/727: write d2/d5/d16/d57/d63/dd3/ddb/d3d/d3c/d7f/f2f [4363777,116661] 0 2024-07-16T09:55:44.838 INFO:tasks.workunit.client.0.smithi156.stdout:9/728: write d2/d5/d16/d57/d63/dd3/ddb/d74/dc2/dc4/f2c [2673703,56388] 0 2024-07-16T09:55:44.838 INFO:tasks.workunit.client.0.smithi156.stdout:9/729: fdatasync d2/d5/f44 0 2024-07-16T09:55:44.840 INFO:tasks.workunit.client.0.smithi156.stdout:9/730: creat d2/d5/fea x:0 0 0 2024-07-16T09:55:44.841 INFO:tasks.workunit.client.0.smithi156.stdout:9/731: chown d2/d5/d61/da2/d98/dbf/ccf 41 1 2024-07-16T09:55:44.843 INFO:tasks.workunit.client.0.smithi156.stdout:9/732: mkdir d2/d5/d16/d22/d36/deb 0 2024-07-16T09:55:44.857 INFO:tasks.workunit.client.0.smithi156.stdout:9/733: dread d2/d5/d61/da2/d98/fa7 [0,4194304] 0 2024-07-16T09:55:44.860 INFO:tasks.workunit.client.0.smithi156.stdout:9/734: dread d2/d5/d61/da2/d98/fbd [0,4194304] 0 2024-07-16T09:55:44.862 INFO:tasks.workunit.client.0.smithi156.stdout:9/735: mknod d2/d5/d16/d57/d63/dd3/ddb/d74/dc2/cec 0 2024-07-16T09:55:44.863 INFO:tasks.workunit.client.0.smithi156.stdout:9/736: stat d2/d5/ff 0 2024-07-16T09:55:44.864 INFO:tasks.workunit.client.0.smithi156.stdout:9/737: dread d2/d5/d61/da2/d98/fbd [0,4194304] 0 2024-07-16T09:55:44.867 INFO:tasks.workunit.client.0.smithi156.stdout:9/738: fdatasync d2/d5/d16/d57/d63/dd3/ddb/d74/dc2/dc4/d23/f60 0 2024-07-16T09:55:44.868 INFO:tasks.workunit.client.0.smithi156.stdout:9/739: chown d2/d5/d16/d22/d6b/d86 458 1 2024-07-16T09:55:44.877 INFO:tasks.workunit.client.0.smithi156.stdout:9/740: dread d2/d5/d16/d57/d63/dd3/ddb/d74/dc2/dc4/d23/f88 [0,4194304] 0 2024-07-16T09:55:44.880 INFO:tasks.workunit.client.0.smithi156.stdout:9/741: write d2/d5/d16/d57/d63/dd3/ddb/d74/dc2/dc4/d23/f60 [13498,119753] 0 2024-07-16T09:55:44.881 INFO:tasks.workunit.client.0.smithi156.stdout:9/742: stat d2/d5/d16/d57/d63/dd3/ddb/d3d/da6 0 2024-07-16T09:55:44.882 INFO:tasks.workunit.client.0.smithi156.stdout:9/743: creat d2/d5/d16/d57/d63/dd3/fed x:0 0 0 2024-07-16T09:55:44.884 INFO:tasks.workunit.client.0.smithi156.stdout:9/744: write d2/d5/d16/d57/d63/dd3/ddb/d3d/fd1 [3037584,36811] 0 2024-07-16T09:55:44.890 INFO:tasks.workunit.client.0.smithi156.stdout:9/745: dwrite d2/d5/d16/d57/d63/dd3/ddb/d74/dc2/dc4/d23/fbc [0,4194304] 0 2024-07-16T09:55:44.901 INFO:tasks.workunit.client.0.smithi156.stdout:9/746: dread d2/d5/f44 [0,4194304] 0 2024-07-16T09:55:44.902 INFO:tasks.workunit.client.0.smithi156.stdout:9/747: chown d2/d5/f91 467197407 1 2024-07-16T09:55:44.905 INFO:tasks.workunit.client.1.smithi176.stdout:1/480: sync 2024-07-16T09:55:44.908 INFO:tasks.workunit.client.0.smithi156.stdout:9/748: dwrite d2/d5/d16/d57/d63/dd3/ddb/d3d/d3c/d7f/d42/f85 [0,4194304] 0 2024-07-16T09:55:44.914 INFO:tasks.workunit.client.0.smithi156.stdout:9/749: dread d2/d5/d16/d57/d63/dd3/ddb/d3d/d3c/d7f/d42/f85 [0,4194304] 0 2024-07-16T09:55:44.917 INFO:tasks.workunit.client.0.smithi156.stdout:9/750: write d2/d5/d16/d57/d63/dd3/ddb/d74/dc2/dc4/d23/f2e [1939187,34875] 0 2024-07-16T09:55:44.918 INFO:tasks.workunit.client.0.smithi156.stdout:9/751: chown d2/d5/d16/d57/d63/dd3/ddb/d3d/d3c/lc7 3 1 2024-07-16T09:55:44.920 INFO:tasks.workunit.client.0.smithi156.stdout:9/752: fdatasync d2/d5/d16/d57/d63/dd3/ddb/d74/dc2/dc4/d3a/fd8 0 2024-07-16T09:55:44.921 INFO:tasks.workunit.client.0.smithi156.stdout:9/753: fdatasync d2/d5/d16/d57/d63/dd3/ddb/d74/dc2/dc4/d23/f88 0 2024-07-16T09:55:44.922 INFO:tasks.workunit.client.0.smithi156.stdout:9/754: write d2/d5/d16/d57/d63/dd3/ddb/d3d/fd1 [4564155,9862] 0 2024-07-16T09:55:44.926 INFO:tasks.workunit.client.0.smithi156.stdout:9/755: creat d2/d5/d16/d57/d63/dd3/ddb/d74/dc2/dc4/fee x:0 0 0 2024-07-16T09:55:44.927 INFO:tasks.workunit.client.0.smithi156.stdout:9/756: fsync d2/d5/d16/d57/d63/dd3/ddb/d3d/fda 0 2024-07-16T09:55:44.931 INFO:tasks.workunit.client.0.smithi156.stdout:9/757: dwrite d2/d5/f44 [0,4194304] 0 2024-07-16T09:55:44.934 INFO:tasks.workunit.client.0.smithi156.stdout:9/758: unlink d2/d5/d16/d57/d63/dd3/ddb/d3d/d3c/d7f/f19 0 2024-07-16T09:55:44.935 INFO:tasks.workunit.client.0.smithi156.stdout:9/759: chown d2/d5/d16/d22/d6b/l9c 9100116 1 2024-07-16T09:55:44.937 INFO:tasks.workunit.client.0.smithi156.stdout:9/760: mkdir d2/d5/d16/d57/d63/dd3/ddb/dd0/def 0 2024-07-16T09:55:44.939 INFO:tasks.workunit.client.0.smithi156.stdout:9/761: readlink d2/d5/d16/d22/l30 0 2024-07-16T09:55:44.943 INFO:tasks.workunit.client.0.smithi156.stdout:9/762: truncate d2/d5/d16/d57/d63/dd3/ddb/d74/dc2/dc4/d3a/fd8 2505101 0 2024-07-16T09:55:44.949 INFO:tasks.workunit.client.0.smithi156.stdout:9/763: link d2/d5/d16/d57/d63/f6e d2/d5/d16/d57/d63/dd3/ddb/d74/dc2/de6/ff0 0 2024-07-16T09:55:44.950 INFO:tasks.workunit.client.0.smithi156.stdout:9/764: fsync d2/d5/d16/d57/d63/dd3/ddb/d74/dc2/dc4/d3a/f3b 0 2024-07-16T09:55:44.951 INFO:tasks.workunit.client.0.smithi156.stdout:9/765: truncate d2/d5/f91 620964 0 2024-07-16T09:55:44.953 INFO:tasks.workunit.client.0.smithi156.stdout:9/766: chown d2/d5/d16/d57/d63/dd3/ddb/d3d/d3c/d7f/l51 153372144 1 2024-07-16T09:55:44.955 INFO:tasks.workunit.client.1.smithi176.stdout:5/804: rename d4/ff to d4/da9/dc4/deb/f123 0 2024-07-16T09:55:44.956 INFO:tasks.workunit.client.1.smithi176.stdout:5/805: write d4/d10/f33 [4546952,25026] 0 2024-07-16T09:55:44.956 INFO:tasks.workunit.client.1.smithi176.stdout:4/663: mknod da/dc/d10/d96/cd5 0 2024-07-16T09:55:44.958 INFO:tasks.workunit.client.1.smithi176.stdout:8/689: link d8/d23/f1b d8/d20/d2a/d61/d90/feb 0 2024-07-16T09:55:44.959 INFO:tasks.workunit.client.0.smithi156.stdout:9/767: dwrite d2/d5/d16/d57/d63/dd3/ddb/d74/dc2/dc4/d23/f60 [0,4194304] 0 2024-07-16T09:55:44.964 INFO:tasks.workunit.client.1.smithi176.stdout:6/633: mkdir d1/d5/d47/d62/de8 0 2024-07-16T09:55:44.966 INFO:tasks.workunit.client.1.smithi176.stdout:6/634: write d1/d5d/d9e/d76/da5/fb7 [1023461,61245] 0 2024-07-16T09:55:44.968 INFO:tasks.workunit.client.1.smithi176.stdout:4/664: truncate da/d3d/d17/f18 364746 0 2024-07-16T09:55:44.968 INFO:tasks.workunit.client.1.smithi176.stdout:6/635: dread d1/d5d/d9e/dd9/d6f/f96 [0,4194304] 0 2024-07-16T09:55:44.969 INFO:tasks.workunit.client.1.smithi176.stdout:6/636: chown d1/lf 52318 1 2024-07-16T09:55:44.970 INFO:tasks.workunit.client.1.smithi176.stdout:8/690: creat d8/d14/d1e/d7c/fec x:0 0 0 2024-07-16T09:55:44.970 INFO:tasks.workunit.client.1.smithi176.stdout:4/665: fdatasync da/dc/d10/d96/d9b/db4/f26 0 2024-07-16T09:55:44.970 INFO:tasks.workunit.client.1.smithi176.stdout:3/581: link d3/d17/d1d/f76 d3/d17/d83/fb0 0 2024-07-16T09:55:44.974 INFO:tasks.workunit.client.1.smithi176.stdout:5/806: truncate d4/d37/d72/d38/d39/f5d 175651 0 2024-07-16T09:55:44.977 INFO:tasks.workunit.client.1.smithi176.stdout:7/666: getdents d4/dd/d26/dd8 0 2024-07-16T09:55:44.977 INFO:tasks.workunit.client.0.smithi156.stdout:6/595: sync 2024-07-16T09:55:44.978 INFO:tasks.workunit.client.1.smithi176.stdout:6/637: dwrite d1/d5/d47/d62/fa1 [0,4194304] 0 2024-07-16T09:55:44.979 INFO:tasks.workunit.client.1.smithi176.stdout:8/691: dwrite d8/d20/d2a/d31/db4/fc0 [0,4194304] 0 2024-07-16T09:55:44.981 INFO:tasks.workunit.client.1.smithi176.stdout:5/807: creat d4/da9/dc4/deb/f124 x:0 0 0 2024-07-16T09:55:44.982 INFO:tasks.workunit.client.1.smithi176.stdout:8/692: read - d8/d20/d2a/d31/db4/fc8 zero size 2024-07-16T09:55:44.985 INFO:tasks.workunit.client.1.smithi176.stdout:1/481: getdents d4/d3a/d54 0 2024-07-16T09:55:44.988 INFO:tasks.workunit.client.1.smithi176.stdout:5/808: unlink d4/d10/d17/f28 0 2024-07-16T09:55:44.992 INFO:tasks.workunit.client.1.smithi176.stdout:6/638: creat d1/d5/d47/d62/d77/dd2/fe9 x:0 0 0 2024-07-16T09:55:44.992 INFO:tasks.workunit.client.1.smithi176.stdout:4/666: write da/f47 [1102,16431] 0 2024-07-16T09:55:44.995 INFO:tasks.workunit.client.1.smithi176.stdout:5/809: mknod d4/d14/d19/d1b/c125 0 2024-07-16T09:55:44.997 INFO:tasks.workunit.client.1.smithi176.stdout:3/582: write d3/d17/f2f [707180,48446] 0 2024-07-16T09:55:44.997 INFO:tasks.workunit.client.0.smithi156.stdout:9/768: dwrite d2/d5/d16/d57/d63/dd3/ddb/d74/dc2/dc4/d3a/fab [0,4194304] 0 2024-07-16T09:55:45.000 INFO:tasks.workunit.client.1.smithi176.stdout:6/639: symlink d1/d5/d47/d62/d80/lea 0 2024-07-16T09:55:45.000 INFO:tasks.workunit.client.0.smithi156.stdout:9/769: mkdir d2/d5/d61/df1 0 2024-07-16T09:55:45.001 INFO:tasks.workunit.client.0.smithi156.stdout:9/770: write d2/f15 [2781857,1338] 0 2024-07-16T09:55:45.002 INFO:tasks.workunit.client.1.smithi176.stdout:3/583: read d3/da/d33/f52 [2125722,42827] 0 2024-07-16T09:55:45.004 INFO:tasks.workunit.client.1.smithi176.stdout:7/667: rename d4/d47/ld6 to d4/d47/ldb 0 2024-07-16T09:55:45.008 INFO:tasks.workunit.client.1.smithi176.stdout:9/783: sync 2024-07-16T09:55:45.010 INFO:tasks.workunit.client.0.smithi156.stdout:9/771: dwrite d2/d5/d16/d22/d36/fe9 [0,4194304] 0 2024-07-16T09:55:45.010 INFO:tasks.workunit.client.1.smithi176.stdout:5/810: creat d4/da9/dc4/deb/d122/f126 x:0 0 0 2024-07-16T09:55:45.012 INFO:tasks.workunit.client.1.smithi176.stdout:3/584: symlink d3/d17/lb1 0 2024-07-16T09:55:45.012 INFO:tasks.workunit.client.0.smithi156.stdout:6/596: dwrite d3/d13/d17/d4f/d30/f7f [0,4194304] 0 2024-07-16T09:55:45.012 INFO:tasks.workunit.client.0.smithi156.stdout:9/772: fdatasync d2/d5/d16/d22/d36/f59 0 2024-07-16T09:55:45.013 INFO:tasks.workunit.client.1.smithi176.stdout:9/784: fdatasync d5/d52/fa4 0 2024-07-16T09:55:45.014 INFO:tasks.workunit.client.1.smithi176.stdout:6/640: dwrite d1/d5/d47/d62/d77/dd2/fe9 [0,4194304] 0 2024-07-16T09:55:45.015 INFO:tasks.workunit.client.0.smithi156.stdout:6/597: dread - d3/d13/d17/d4f/d4d/f70 zero size 2024-07-16T09:55:45.017 INFO:tasks.workunit.client.0.smithi156.stdout:9/773: rmdir d2/d5/d16/d57/d63/dd3/ddb/d74/dc2/dc4 39 2024-07-16T09:55:45.020 INFO:tasks.workunit.client.1.smithi176.stdout:5/811: chown d4/d14/d19/d1b/d95 184058054 1 2024-07-16T09:55:45.020 INFO:tasks.workunit.client.1.smithi176.stdout:1/482: dwrite d4/d8/d12/d21/d25/d30/d44/d4d/f7e [0,4194304] 0 2024-07-16T09:55:45.027 INFO:tasks.workunit.client.1.smithi176.stdout:8/693: dwrite d8/d20/d2a/db9/dcb/d37/f3c [0,4194304] 0 2024-07-16T09:55:45.029 INFO:tasks.workunit.client.0.smithi156.stdout:6/598: unlink d3/d13/d17/d4f/d30/d45/d5f/f8b 0 2024-07-16T09:55:45.030 INFO:tasks.workunit.client.1.smithi176.stdout:9/785: symlink d5/d1d/dc7/d89/l10a 0 2024-07-16T09:55:45.030 INFO:tasks.workunit.client.0.smithi156.stdout:9/774: symlink d2/d5/d16/d57/d63/dd3/ddb/d74/dc2/dc4/d66/lf2 0 2024-07-16T09:55:45.031 INFO:tasks.workunit.client.1.smithi176.stdout:3/585: rename d3/f25 to d3/da/d84/fb2 0 2024-07-16T09:55:45.033 INFO:tasks.workunit.client.1.smithi176.stdout:4/667: dwrite da/d3d/d17/f35 [4194304,4194304] 0 2024-07-16T09:55:45.034 INFO:tasks.workunit.client.1.smithi176.stdout:6/641: truncate d1/dc/d40/f4d 6620836 0 2024-07-16T09:55:45.035 INFO:tasks.workunit.client.0.smithi156.stdout:6/599: creat d3/d13/d17/d4f/d4d/dbb/fc6 x:0 0 0 2024-07-16T09:55:45.038 INFO:tasks.workunit.client.0.smithi156.stdout:6/600: mknod d3/d13/d17/d56/cc7 0 2024-07-16T09:55:45.040 INFO:tasks.workunit.client.1.smithi176.stdout:1/483: dread - d4/d8/d12/d21/d25/d30/d44/d4d/f95 zero size 2024-07-16T09:55:45.041 INFO:tasks.workunit.client.1.smithi176.stdout:7/668: dwrite d4/d47/d5d/fad [0,4194304] 0 2024-07-16T09:55:45.043 INFO:tasks.workunit.client.0.smithi156.stdout:6/601: symlink d3/d13/d17/d4f/d94/daa/lc8 0 2024-07-16T09:55:45.043 INFO:tasks.workunit.client.1.smithi176.stdout:5/812: creat d4/d37/d72/d38/d6e/dbe/f127 x:0 0 0 2024-07-16T09:55:45.044 INFO:tasks.workunit.client.1.smithi176.stdout:2/827: sync 2024-07-16T09:55:45.044 INFO:tasks.workunit.client.1.smithi176.stdout:0/802: sync 2024-07-16T09:55:45.044 INFO:tasks.workunit.client.1.smithi176.stdout:3/586: mknod d3/d17/d1d/d4f/cb3 0 2024-07-16T09:55:45.045 INFO:tasks.workunit.client.1.smithi176.stdout:9/786: truncate d5/d1d/d20/d78/dd0/fd7 1015169 0 2024-07-16T09:55:45.046 INFO:tasks.workunit.client.0.smithi156.stdout:6/602: mknod d3/d13/d17/d4f/d4d/d3b/cc9 0 2024-07-16T09:55:45.047 INFO:tasks.workunit.client.1.smithi176.stdout:8/694: creat d8/d20/d2a/d31/d46/d6a/fed x:0 0 0 2024-07-16T09:55:45.049 INFO:tasks.workunit.client.1.smithi176.stdout:1/484: symlink d4/d8/d12/d21/la5 0 2024-07-16T09:55:45.053 INFO:tasks.workunit.client.1.smithi176.stdout:1/485: write d4/d8/d12/d21/d25/f55 [4971828,116697] 0 2024-07-16T09:55:45.053 INFO:tasks.workunit.client.1.smithi176.stdout:0/803: rmdir d5/d1c/d24/d35 39 2024-07-16T09:55:45.055 INFO:tasks.workunit.client.1.smithi176.stdout:7/669: dread d4/fb [0,4194304] 0 2024-07-16T09:55:45.055 INFO:tasks.workunit.client.1.smithi176.stdout:6/642: link d1/l3a d1/d5d/d9e/dd9/d6f/d7c/leb 0 2024-07-16T09:55:45.056 INFO:tasks.workunit.client.1.smithi176.stdout:3/587: dread d3/d17/d1d/d6e/fa4 [0,4194304] 0 2024-07-16T09:55:45.058 INFO:tasks.workunit.client.1.smithi176.stdout:3/588: fdatasync d3/d17/d1d/faf 0 2024-07-16T09:55:45.058 INFO:tasks.workunit.client.1.smithi176.stdout:7/670: read d4/fb3 [490744,14751] 0 2024-07-16T09:55:45.059 INFO:tasks.workunit.client.1.smithi176.stdout:1/486: creat d4/d3a/d54/d5a/d7a/fa6 x:0 0 0 2024-07-16T09:55:45.060 INFO:tasks.workunit.client.1.smithi176.stdout:9/787: creat d5/d1d/dd2/de3/d103/f10b x:0 0 0 2024-07-16T09:55:45.062 INFO:tasks.workunit.client.1.smithi176.stdout:6/643: fdatasync d1/d5/d7/f19 0 2024-07-16T09:55:45.062 INFO:tasks.workunit.client.1.smithi176.stdout:9/788: readlink d5/d43/lae 0 2024-07-16T09:55:45.063 INFO:tasks.workunit.client.1.smithi176.stdout:8/695: truncate d8/d20/d2a/f67 1371984 0 2024-07-16T09:55:45.063 INFO:tasks.workunit.client.1.smithi176.stdout:0/804: dwrite d5/d1c/f1f [0,4194304] 0 2024-07-16T09:55:45.066 INFO:tasks.workunit.client.1.smithi176.stdout:5/813: link d4/f27 d4/d14/d3d/f128 0 2024-07-16T09:55:45.067 INFO:tasks.workunit.client.1.smithi176.stdout:4/668: dread da/dc/d10/d96/d69/fb0 [0,4194304] 0 2024-07-16T09:55:45.068 INFO:tasks.workunit.client.1.smithi176.stdout:2/828: getdents dd/d2e/d35/d4a/df9 0 2024-07-16T09:55:45.068 INFO:tasks.workunit.client.1.smithi176.stdout:6/644: readlink d1/l3a 0 2024-07-16T09:55:45.069 INFO:tasks.workunit.client.1.smithi176.stdout:8/696: readlink d8/d14/d1e/d33/d9d/lae 0 2024-07-16T09:55:45.069 INFO:tasks.workunit.client.1.smithi176.stdout:3/589: dread d3/d17/f51 [0,4194304] 0 2024-07-16T09:55:45.069 INFO:tasks.workunit.client.1.smithi176.stdout:0/805: creat d5/d1c/db1/f10b x:0 0 0 2024-07-16T09:55:45.069 INFO:tasks.workunit.client.1.smithi176.stdout:7/671: truncate d4/d1f/d22/d51/f96 5257 0 2024-07-16T09:55:45.070 INFO:tasks.workunit.client.0.smithi156.stdout:9/775: truncate d2/d5/d16/d57/d63/dd3/ddb/d74/fbe 3516093 0 2024-07-16T09:55:45.072 INFO:tasks.workunit.client.1.smithi176.stdout:9/789: read d5/d1d/dc7/d89/fb9 [405321,5869] 0 2024-07-16T09:55:45.073 INFO:tasks.workunit.client.0.smithi156.stdout:9/776: symlink d2/d5/d16/d22/d36/deb/lf3 0 2024-07-16T09:55:45.074 INFO:tasks.workunit.client.1.smithi176.stdout:0/806: symlink d5/d39/da3/l10c 0 2024-07-16T09:55:45.075 INFO:tasks.workunit.client.1.smithi176.stdout:0/807: chown d5/d1c/d24/fca 406494782 1 2024-07-16T09:55:45.076 INFO:tasks.workunit.client.0.smithi156.stdout:6/603: dwrite d3/d13/d17/d4f/d1e/f20 [0,4194304] 0 2024-07-16T09:55:45.078 INFO:tasks.workunit.client.1.smithi176.stdout:5/814: rename d4/f13 to d4/d37/d72/d38/d39/f129 0 2024-07-16T09:55:45.078 INFO:tasks.workunit.client.1.smithi176.stdout:5/815: chown d4/l8c 727 1 2024-07-16T09:55:45.079 INFO:tasks.workunit.client.1.smithi176.stdout:4/669: creat da/d3d/d17/d54/d78/d7a/fd6 x:0 0 0 2024-07-16T09:55:45.081 INFO:tasks.workunit.client.1.smithi176.stdout:8/697: mknod d8/d20/d2a/d61/cee 0 2024-07-16T09:55:45.082 INFO:tasks.workunit.client.1.smithi176.stdout:9/790: unlink d5/d1d/d77/d97/ff3 0 2024-07-16T09:55:45.086 INFO:tasks.workunit.client.1.smithi176.stdout:9/791: truncate d5/d1d/d77/d97/fa6 389087 0 2024-07-16T09:55:45.087 INFO:tasks.workunit.client.1.smithi176.stdout:5/816: creat d4/d6c/df6/f12a x:0 0 0 2024-07-16T09:55:45.087 INFO:tasks.workunit.client.1.smithi176.stdout:7/672: mkdir d4/dd/d26/dc2/d17/d1d/ddc 0 2024-07-16T09:55:45.089 INFO:tasks.workunit.client.1.smithi176.stdout:4/670: symlink da/d3d/ld7 0 2024-07-16T09:55:45.091 INFO:tasks.workunit.client.1.smithi176.stdout:2/829: dwrite dd/d16/d39/d3d/d7e/fdf [0,4194304] 0 2024-07-16T09:55:45.092 INFO:tasks.workunit.client.1.smithi176.stdout:1/487: dread d4/f24 [0,4194304] 0 2024-07-16T09:55:45.094 INFO:tasks.workunit.client.0.smithi156.stdout:6/604: truncate d3/d13/d17/d4f/d4d/d3b/d5a/f69 753069 0 2024-07-16T09:55:45.095 INFO:tasks.workunit.client.0.smithi156.stdout:6/605: write d3/d13/d17/f2b [360190,108584] 0 2024-07-16T09:55:45.095 INFO:tasks.workunit.client.1.smithi176.stdout:3/590: truncate d3/d17/d1d/d4f/d59/da0/f3f 2590939 0 2024-07-16T09:55:45.097 INFO:tasks.workunit.client.1.smithi176.stdout:0/808: mknod d5/d1c/d5a/c10d 0 2024-07-16T09:55:45.097 INFO:tasks.workunit.client.1.smithi176.stdout:6/645: dread d1/dc/d40/f4d [0,4194304] 0 2024-07-16T09:55:45.097 INFO:tasks.workunit.client.0.smithi156.stdout:6/606: write d3/d13/d17/d4f/d4d/d3b/f93 [1613892,124632] 0 2024-07-16T09:55:45.100 INFO:tasks.workunit.client.1.smithi176.stdout:0/809: chown d5/d2c/d6c/c90 14 1 2024-07-16T09:55:45.101 INFO:tasks.workunit.client.1.smithi176.stdout:7/673: fsync d4/d47/d5d/f90 0 2024-07-16T09:55:45.102 INFO:tasks.workunit.client.1.smithi176.stdout:4/671: creat da/dc/d10/d96/fd8 x:0 0 0 2024-07-16T09:55:45.103 INFO:tasks.workunit.client.1.smithi176.stdout:2/830: dwrite dd/d49/d52/d89/ff5 [0,4194304] 0 2024-07-16T09:55:45.103 INFO:tasks.workunit.client.1.smithi176.stdout:7/674: chown d4/dd/d8e 3247 1 2024-07-16T09:55:45.105 INFO:tasks.workunit.client.1.smithi176.stdout:7/675: truncate d4/d47/d5d/fc9 272866 0 2024-07-16T09:55:45.106 INFO:tasks.workunit.client.0.smithi156.stdout:6/607: dwrite d3/d13/d17/d4f/faf [0,4194304] 0 2024-07-16T09:55:45.107 INFO:tasks.workunit.client.1.smithi176.stdout:8/698: creat d8/d20/d2a/d53/d9a/dc1/fef x:0 0 0 2024-07-16T09:55:45.107 INFO:tasks.workunit.client.1.smithi176.stdout:1/488: creat d4/d8/d12/d4c/fa7 x:0 0 0 2024-07-16T09:55:45.109 INFO:tasks.workunit.client.1.smithi176.stdout:5/817: creat d4/d9a/ded/df4/f12b x:0 0 0 2024-07-16T09:55:45.111 INFO:tasks.workunit.client.1.smithi176.stdout:3/591: rename d3/d17/c1b to d3/da/d7b/cb4 0 2024-07-16T09:55:45.116 INFO:tasks.workunit.client.1.smithi176.stdout:6/646: read d1/d5d/d9e/d8f/f8d [1397146,46140] 0 2024-07-16T09:55:45.118 INFO:tasks.workunit.client.1.smithi176.stdout:6/647: fsync d1/d5/d47/d62/d77/dd2/fe9 0 2024-07-16T09:55:45.119 INFO:tasks.workunit.client.1.smithi176.stdout:9/792: dread d5/de/d30/f9a [0,4194304] 0 2024-07-16T09:55:45.120 INFO:tasks.workunit.client.1.smithi176.stdout:2/831: chown dd/d36/la1 78932843 1 2024-07-16T09:55:45.120 INFO:tasks.workunit.client.1.smithi176.stdout:6/648: write d1/d5/d47/f51 [8776866,85028] 0 2024-07-16T09:55:45.121 INFO:tasks.workunit.client.1.smithi176.stdout:7/676: fdatasync d4/dd/d26/dc2/d17/f4d 0 2024-07-16T09:55:45.122 INFO:tasks.workunit.client.0.smithi156.stdout:6/608: dread d3/d13/f8e [0,4194304] 0 2024-07-16T09:55:45.124 INFO:tasks.workunit.client.1.smithi176.stdout:5/818: mknod d4/d37/d72/d38/d6e/dbe/c12c 0 2024-07-16T09:55:45.125 INFO:tasks.workunit.client.1.smithi176.stdout:8/699: dread d8/d23/f25 [0,4194304] 0 2024-07-16T09:55:45.125 INFO:tasks.workunit.client.1.smithi176.stdout:3/592: rename d3/d17/d1d/c47 to d3/d17/d83/cb5 0 2024-07-16T09:55:45.126 INFO:tasks.workunit.client.1.smithi176.stdout:0/810: truncate d5/d39/f7e 3587197 0 2024-07-16T09:55:45.129 INFO:tasks.workunit.client.0.smithi156.stdout:6/609: dwrite d3/d13/d17/d4f/d1e/f95 [0,4194304] 0 2024-07-16T09:55:45.131 INFO:tasks.workunit.client.1.smithi176.stdout:9/793: rmdir d5/d1d/db0/dd3 39 2024-07-16T09:55:45.132 INFO:tasks.workunit.client.0.smithi156.stdout:6/610: readlink d3/d13/d17/d4f/d4d/d3b/d77/lac 0 2024-07-16T09:55:45.135 INFO:tasks.workunit.client.0.smithi156.stdout:6/611: creat d3/d13/d17/d4f/d4d/d3b/da5/dbc/fca x:0 0 0 2024-07-16T09:55:45.137 INFO:tasks.workunit.client.0.smithi156.stdout:6/612: write d3/d13/d17/d4f/d1e/f95 [4807651,87612] 0 2024-07-16T09:55:45.137 INFO:tasks.workunit.client.1.smithi176.stdout:3/593: symlink d3/da/d6c/lb6 0 2024-07-16T09:55:45.140 INFO:tasks.workunit.client.0.smithi156.stdout:6/613: mknod d3/d13/d17/d56/ccb 0 2024-07-16T09:55:45.141 INFO:tasks.workunit.client.1.smithi176.stdout:4/672: creat da/fd9 x:0 0 0 2024-07-16T09:55:45.142 INFO:tasks.workunit.client.1.smithi176.stdout:4/673: dread - da/dc/d10/d28/fc4 zero size 2024-07-16T09:55:45.143 INFO:tasks.workunit.client.0.smithi156.stdout:6/614: mkdir d3/d13/d17/d4f/d4d/d3b/dcc 0 2024-07-16T09:55:45.144 INFO:tasks.workunit.client.1.smithi176.stdout:5/819: dread d4/d37/d72/d38/d102/dff/fc3 [0,4194304] 0 2024-07-16T09:55:45.146 INFO:tasks.workunit.client.1.smithi176.stdout:7/677: truncate d4/d5/f2f 10305877 0 2024-07-16T09:55:45.148 INFO:tasks.workunit.client.1.smithi176.stdout:8/700: creat d8/d14/d1e/d7c/d82/de0/ff0 x:0 0 0 2024-07-16T09:55:45.148 INFO:tasks.workunit.client.1.smithi176.stdout:0/811: mknod d5/d1c/d24/dfe/c10e 0 2024-07-16T09:55:45.148 INFO:tasks.workunit.client.0.smithi156.stdout:6/615: getdents d3/d13/d17/d4f/d94 0 2024-07-16T09:55:45.150 INFO:tasks.workunit.client.1.smithi176.stdout:6/649: link d1/dc/l2b d1/d5d/d9e/dd9/d6f/d7c/lec 0 2024-07-16T09:55:45.150 INFO:tasks.workunit.client.0.smithi156.stdout:6/616: read d3/d13/d17/d4f/d4d/d3b/fbd [2402988,93032] 0 2024-07-16T09:55:45.150 INFO:tasks.workunit.client.1.smithi176.stdout:2/832: rename dd/d36/d45/lb0 to dd/d2e/d35/l111 0 2024-07-16T09:55:45.152 INFO:tasks.workunit.client.1.smithi176.stdout:1/489: dread d4/d8/f26 [0,4194304] 0 2024-07-16T09:55:45.153 INFO:tasks.workunit.client.1.smithi176.stdout:9/794: getdents d5/d1d/dd2/de3 0 2024-07-16T09:55:45.154 INFO:tasks.workunit.client.1.smithi176.stdout:8/701: creat d8/d14/d1e/ff1 x:0 0 0 2024-07-16T09:55:45.159 INFO:tasks.workunit.client.1.smithi176.stdout:4/674: link da/dc/d10/d96/fd8 da/dc/d10/d96/da4/dc2/fda 0 2024-07-16T09:55:45.159 INFO:tasks.workunit.client.1.smithi176.stdout:6/650: dwrite d1/d5/f98 [0,4194304] 0 2024-07-16T09:55:45.160 INFO:tasks.workunit.client.1.smithi176.stdout:2/833: mkdir dd/d16/d39/d3d/d112 0 2024-07-16T09:55:45.160 INFO:tasks.workunit.client.1.smithi176.stdout:7/678: symlink d4/dd/d26/dc2/d17/ldd 0 2024-07-16T09:55:45.164 INFO:tasks.workunit.client.1.smithi176.stdout:0/812: write d5/d25/da0/fc0 [130456,76815] 0 2024-07-16T09:55:45.165 INFO:tasks.workunit.client.1.smithi176.stdout:8/702: symlink d8/d20/d2a/db9/lf2 0 2024-07-16T09:55:45.168 INFO:tasks.workunit.client.1.smithi176.stdout:4/675: unlink da/c20 0 2024-07-16T09:55:45.168 INFO:tasks.workunit.client.1.smithi176.stdout:3/594: getdents d3 0 2024-07-16T09:55:45.169 INFO:tasks.workunit.client.1.smithi176.stdout:8/703: chown d8/l29 2847 1 2024-07-16T09:55:45.171 INFO:tasks.workunit.client.1.smithi176.stdout:9/795: unlink d5/de/d30/f9d 0 2024-07-16T09:55:45.171 INFO:tasks.workunit.client.1.smithi176.stdout:1/490: rename d4/d82 to d4/d8/d12/d21/d25/da8 0 2024-07-16T09:55:45.171 INFO:tasks.workunit.client.1.smithi176.stdout:7/679: write d4/d1f/d22/f52 [2453334,42446] 0 2024-07-16T09:55:45.173 INFO:tasks.workunit.client.0.smithi156.stdout:6/617: dread d3/d13/d17/d4f/d30/f67 [0,4194304] 0 2024-07-16T09:55:45.177 INFO:tasks.workunit.client.1.smithi176.stdout:6/651: dwrite d1/d5d/d9e/d76/da5/fdf [0,4194304] 0 2024-07-16T09:55:45.177 INFO:tasks.workunit.client.0.smithi156.stdout:6/618: rename d3/d13/d17/d4f/d30/d45/c7e to d3/d13/d17/d56/ccd 0 2024-07-16T09:55:45.178 INFO:tasks.workunit.client.1.smithi176.stdout:3/595: write d3/d17/d1d/d4f/daa/fae [235177,22363] 0 2024-07-16T09:55:45.178 INFO:tasks.workunit.client.1.smithi176.stdout:4/676: creat da/dc/d10/d96/d9b/fdb x:0 0 0 2024-07-16T09:55:45.178 INFO:tasks.workunit.client.1.smithi176.stdout:5/820: dwrite d4/d37/d72/d82/f93 [0,4194304] 0 2024-07-16T09:55:45.186 INFO:tasks.workunit.client.0.smithi156.stdout:6/619: dwrite d3/d13/d17/f2b [8388608,4194304] 0 2024-07-16T09:55:45.187 INFO:tasks.workunit.client.1.smithi176.stdout:8/704: symlink d8/d20/d2a/db9/lf3 0 2024-07-16T09:55:45.187 INFO:tasks.workunit.client.1.smithi176.stdout:2/834: dwrite dd/d16/f1d [4194304,4194304] 0 2024-07-16T09:55:45.187 INFO:tasks.workunit.client.1.smithi176.stdout:6/652: dread - d1/d5/fa9 zero size 2024-07-16T09:55:45.187 INFO:tasks.workunit.client.1.smithi176.stdout:7/680: unlink d4/dd/d26/dc2/d17/ldd 0 2024-07-16T09:55:45.188 INFO:tasks.workunit.client.0.smithi156.stdout:6/620: chown d3/d13/d17/d4f/d30/d45/fb0 254485913 1 2024-07-16T09:55:45.188 INFO:tasks.workunit.client.0.smithi156.stdout:6/621: chown d3/d13 121 1 2024-07-16T09:55:45.189 INFO:tasks.workunit.client.1.smithi176.stdout:5/821: creat d4/d37/d72/d38/d102/d105/f12d x:0 0 0 2024-07-16T09:55:45.189 INFO:tasks.workunit.client.1.smithi176.stdout:3/596: mknod d3/d85/cb7 0 2024-07-16T09:55:45.189 INFO:tasks.workunit.client.1.smithi176.stdout:0/813: dread d5/d1c/d24/d35/d50/d58/fcb [0,4194304] 0 2024-07-16T09:55:45.190 INFO:tasks.workunit.client.1.smithi176.stdout:1/491: mkdir d4/d8/d12/d4c/da4/da9 0 2024-07-16T09:55:45.191 INFO:tasks.workunit.client.1.smithi176.stdout:8/705: read - d8/d20/d2a/d31/db4/fc4 zero size 2024-07-16T09:55:45.192 INFO:tasks.workunit.client.1.smithi176.stdout:5/822: read - d4/d14/dc5/def/df9/ffe zero size 2024-07-16T09:55:45.193 INFO:tasks.workunit.client.1.smithi176.stdout:6/653: read d1/f45 [2979944,83958] 0 2024-07-16T09:55:45.193 INFO:tasks.workunit.client.0.smithi156.stdout:6/622: mkdir d3/d13/d17/d4f/d4d/d3b/d5a/dc1/dce 0 2024-07-16T09:55:45.196 INFO:tasks.workunit.client.1.smithi176.stdout:0/814: write d5/d2c/fba [635974,84055] 0 2024-07-16T09:55:45.196 INFO:tasks.workunit.client.1.smithi176.stdout:9/796: write d5/de/da5/dbf/f105 [116891,10424] 0 2024-07-16T09:55:45.197 INFO:tasks.workunit.client.0.smithi156.stdout:6/623: symlink d3/d13/d17/d4f/d94/d19/d31/d35/lcf 0 2024-07-16T09:55:45.198 INFO:tasks.workunit.client.1.smithi176.stdout:0/815: stat d5/ce 0 2024-07-16T09:55:45.198 INFO:tasks.workunit.client.0.smithi156.stdout:6/624: chown d3/d13/d17/d4f/d4d/d3b/dcc 3 1 2024-07-16T09:55:45.199 INFO:tasks.workunit.client.1.smithi176.stdout:2/835: symlink dd/d16/d39/l113 0 2024-07-16T09:55:45.200 INFO:tasks.workunit.client.1.smithi176.stdout:5/823: write d4/d14/dc5/fd1 [269964,125940] 0 2024-07-16T09:55:45.203 INFO:tasks.workunit.client.1.smithi176.stdout:8/706: write d8/d14/fcf [1001196,14712] 0 2024-07-16T09:55:45.203 INFO:tasks.workunit.client.1.smithi176.stdout:6/654: fdatasync d1/d5d/d9e/d8f/fb2 0 2024-07-16T09:55:45.204 INFO:tasks.workunit.client.1.smithi176.stdout:4/677: creat da/dc/d10/d96/d9b/db4/fdc x:0 0 0 2024-07-16T09:55:45.205 INFO:tasks.workunit.client.1.smithi176.stdout:0/816: rename d5/d39/l4a to d5/d1c/db1/l10f 0 2024-07-16T09:55:45.208 INFO:tasks.workunit.client.1.smithi176.stdout:6/655: fsync d1/db3/f9a 0 2024-07-16T09:55:45.211 INFO:tasks.workunit.client.1.smithi176.stdout:0/817: chown d5/d1c/d5a/d9e 122254 1 2024-07-16T09:55:45.212 INFO:tasks.workunit.client.1.smithi176.stdout:3/597: dwrite d3/da/d33/f78 [0,4194304] 0 2024-07-16T09:55:45.213 INFO:tasks.workunit.client.1.smithi176.stdout:4/678: fsync da/dc/d10/d96/d69/f7e 0 2024-07-16T09:55:45.216 INFO:tasks.workunit.client.1.smithi176.stdout:6/656: chown d1/d5/d7/d1e/c59 5467 1 2024-07-16T09:55:45.218 INFO:tasks.workunit.client.0.smithi156.stdout:6/625: unlink d3/d13/d17/d4f/d94/d19/f5b 0 2024-07-16T09:55:45.219 INFO:tasks.workunit.client.1.smithi176.stdout:9/797: getdents d5/d1d/db0 0 2024-07-16T09:55:45.222 INFO:tasks.workunit.client.0.smithi156.stdout:6/626: mkdir d3/d13/d17/d4f/d94/daa/dd0 0 2024-07-16T09:55:45.222 INFO:tasks.workunit.client.1.smithi176.stdout:5/824: dwrite d4/d6c/f6f [0,4194304] 0 2024-07-16T09:55:45.224 INFO:tasks.workunit.client.0.smithi156.stdout:6/627: write d3/d13/d17/d4f/d30/d45/d5f/fb4 [1047903,43314] 0 2024-07-16T09:55:45.226 INFO:tasks.workunit.client.1.smithi176.stdout:9/798: rmdir d5/d1d/d77/d98 39 2024-07-16T09:55:45.229 INFO:tasks.workunit.client.1.smithi176.stdout:8/707: link d8/d20/d2a/d31/d46/d6a/l70 d8/d14/d1e/d55/lf4 0 2024-07-16T09:55:45.230 INFO:tasks.workunit.client.1.smithi176.stdout:2/836: dwrite dd/d16/d39/fc9 [0,4194304] 0 2024-07-16T09:55:45.231 INFO:tasks.workunit.client.0.smithi156.stdout:6/628: rename d3/d13/d17/d4f/d4d/d3b/d77/c7b to d3/d13/d17/d4f/d4d/d3b/d5a/dc1/cd1 0 2024-07-16T09:55:45.231 INFO:tasks.workunit.client.1.smithi176.stdout:4/679: creat da/d3d/d33/fdd x:0 0 0 2024-07-16T09:55:45.233 INFO:tasks.workunit.client.1.smithi176.stdout:9/799: read d5/fb8 [891578,52643] 0 2024-07-16T09:55:45.234 INFO:tasks.workunit.client.1.smithi176.stdout:7/681: dread d4/d1f/f59 [0,4194304] 0 2024-07-16T09:55:45.235 INFO:tasks.workunit.client.1.smithi176.stdout:5/825: truncate d4/d14/d19/f21 3254863 0 2024-07-16T09:55:45.238 INFO:tasks.workunit.client.1.smithi176.stdout:3/598: dwrite d3/da/d33/f78 [0,4194304] 0 2024-07-16T09:55:45.238 INFO:tasks.workunit.client.1.smithi176.stdout:1/492: dwrite d4/d8/d12/d21/d34/f3d [0,4194304] 0 2024-07-16T09:55:45.239 INFO:tasks.workunit.client.0.smithi156.stdout:6/629: dwrite d3/d13/d17/d4f/d94/d19/d31/f9c [0,4194304] 0 2024-07-16T09:55:45.241 INFO:tasks.workunit.client.0.smithi156.stdout:6/630: dread - d3/d13/d17/d4f/d94/d19/d2e/f97 zero size 2024-07-16T09:55:45.244 INFO:tasks.workunit.client.0.smithi156.stdout:6/631: write d3/d13/d17/d4f/d94/d19/d31/f4e [3070444,50310] 0 2024-07-16T09:55:45.246 INFO:tasks.workunit.client.0.smithi156.stdout:3/632: sync 2024-07-16T09:55:45.246 INFO:tasks.workunit.client.0.smithi156.stdout:1/874: sync 2024-07-16T09:55:45.247 INFO:tasks.workunit.client.1.smithi176.stdout:9/800: creat d5/de/da5/dc1/f10c x:0 0 0 2024-07-16T09:55:45.251 INFO:tasks.workunit.client.0.smithi156.stdout:1/875: read dc/d14/d3e/fde [3986838,63202] 0 2024-07-16T09:55:45.252 INFO:tasks.workunit.client.1.smithi176.stdout:0/818: dread d5/d25/da0/fc0 [0,4194304] 0 2024-07-16T09:55:45.253 INFO:tasks.workunit.client.1.smithi176.stdout:5/826: fdatasync d4/d37/d72/d38/d39/f56 0 2024-07-16T09:55:45.253 INFO:tasks.workunit.client.1.smithi176.stdout:9/801: chown d5/d1d/dd2/de3/d103 147517 1 2024-07-16T09:55:45.254 INFO:tasks.workunit.client.1.smithi176.stdout:8/708: dwrite d8/d20/d2a/db9/dcb/d34/d62/d88/fd3 [0,4194304] 0 2024-07-16T09:55:45.256 INFO:tasks.workunit.client.0.smithi156.stdout:6/632: dread d3/d13/d17/d4f/d30/d45/d5f/fb4 [0,4194304] 0 2024-07-16T09:55:45.258 INFO:tasks.workunit.client.1.smithi176.stdout:1/493: mkdir d4/d8/d12/d21/d34/daa 0 2024-07-16T09:55:45.259 INFO:tasks.workunit.client.1.smithi176.stdout:4/680: write da/d3d/fa2 [4305397,97200] 0 2024-07-16T09:55:45.260 INFO:tasks.workunit.client.1.smithi176.stdout:7/682: mknod d4/dd/d26/dc2/d17/d1d/cde 0 2024-07-16T09:55:45.260 INFO:tasks.workunit.client.0.smithi156.stdout:1/876: dwrite dc/d14/d3e/d55/d83/f108 [0,4194304] 0 2024-07-16T09:55:45.262 INFO:tasks.workunit.client.1.smithi176.stdout:9/802: write d5/d1d/d20/d78/dd0/fd7 [379444,1978] 0 2024-07-16T09:55:45.267 INFO:tasks.workunit.client.1.smithi176.stdout:8/709: rename d8/d20/d2a/d31/d2f/da9 to d8/d14/d1e/d7c/d82/df5 0 2024-07-16T09:55:45.267 INFO:tasks.workunit.client.0.smithi156.stdout:1/877: stat dc/f37 0 2024-07-16T09:55:45.268 INFO:tasks.workunit.client.1.smithi176.stdout:1/494: write d4/d8/d12/d21/d25/f90 [522103,126596] 0 2024-07-16T09:55:45.269 INFO:tasks.workunit.client.1.smithi176.stdout:9/803: read d5/d1d/d77/fd9 [635261,46466] 0 2024-07-16T09:55:45.269 INFO:tasks.workunit.client.1.smithi176.stdout:0/819: symlink d5/d2c/d6c/l110 0 2024-07-16T09:55:45.272 INFO:tasks.workunit.client.1.smithi176.stdout:7/683: mknod d4/dd/d26/cdf 0 2024-07-16T09:55:45.273 INFO:tasks.workunit.client.1.smithi176.stdout:2/837: dwrite f3 [0,4194304] 0 2024-07-16T09:55:45.273 INFO:tasks.workunit.client.1.smithi176.stdout:8/710: fdatasync d8/d20/d2a/d31/db4/fc8 0 2024-07-16T09:55:45.274 INFO:tasks.workunit.client.1.smithi176.stdout:5/827: chown d4/d10/d17/f67 786024 1 2024-07-16T09:55:45.275 INFO:tasks.workunit.client.1.smithi176.stdout:0/820: truncate d5/d25/ff9 63872 0 2024-07-16T09:55:45.275 INFO:tasks.workunit.client.1.smithi176.stdout:6/657: dread d1/d5d/db9/fb6 [0,4194304] 0 2024-07-16T09:55:45.276 INFO:tasks.workunit.client.1.smithi176.stdout:4/681: truncate da/d3d/d17/d54/fa9 708929 0 2024-07-16T09:55:45.276 INFO:tasks.workunit.client.0.smithi156.stdout:1/878: dwrite dc/d6b/dd5/fd9 [0,4194304] 0 2024-07-16T09:55:45.279 INFO:tasks.workunit.client.1.smithi176.stdout:1/495: rename d4/f65 to d4/d8/d12/d21/d34/daa/fab 0 2024-07-16T09:55:45.280 INFO:tasks.workunit.client.1.smithi176.stdout:6/658: stat d1/d5/d47/fa6 0 2024-07-16T09:55:45.280 INFO:tasks.workunit.client.1.smithi176.stdout:7/684: mkdir d4/d47/d5d/d9a/de0 0 2024-07-16T09:55:45.281 INFO:tasks.workunit.client.1.smithi176.stdout:5/828: mkdir d4/d6c/d12e 0 2024-07-16T09:55:45.286 INFO:tasks.workunit.client.1.smithi176.stdout:0/821: mknod d5/d1c/d24/d26/dbd/de6/c111 0 2024-07-16T09:55:45.287 INFO:tasks.workunit.client.1.smithi176.stdout:4/682: dread - da/dc/d10/d96/d9b/db4/fad zero size 2024-07-16T09:55:45.287 INFO:tasks.workunit.client.1.smithi176.stdout:9/804: dwrite d5/d1d/d20/f74 [0,4194304] 0 2024-07-16T09:55:45.288 INFO:tasks.workunit.client.1.smithi176.stdout:7/685: rmdir d4/dd/d26/dc2 39 2024-07-16T09:55:45.292 INFO:tasks.workunit.client.1.smithi176.stdout:7/686: unlink d4/d47/d5d/fc9 0 2024-07-16T09:55:45.293 INFO:tasks.workunit.client.1.smithi176.stdout:9/805: write d5/d1d/d67/f6a [2369958,108234] 0 2024-07-16T09:55:45.294 INFO:tasks.workunit.client.1.smithi176.stdout:8/711: getdents d8/d14/d1e/d33 0 2024-07-16T09:55:45.295 INFO:tasks.workunit.client.1.smithi176.stdout:2/838: rename dd/d16/d39/d3d/f75 to dd/d49/d52/f114 0 2024-07-16T09:55:45.296 INFO:tasks.workunit.client.1.smithi176.stdout:9/806: stat d5/de/f2f 0 2024-07-16T09:55:45.297 INFO:tasks.workunit.client.1.smithi176.stdout:0/822: dread d5/d1c/f95 [0,4194304] 0 2024-07-16T09:55:45.298 INFO:tasks.workunit.client.1.smithi176.stdout:7/687: rmdir d4/d47/d6d/d89/d66 39 2024-07-16T09:55:45.299 INFO:tasks.workunit.client.1.smithi176.stdout:6/659: link d1/d5/d47/d75/c88 d1/d5d/d9e/dd9/d6f/d7c/ced 0 2024-07-16T09:55:45.299 INFO:tasks.workunit.client.1.smithi176.stdout:9/807: write d5/de/f15 [8055891,99757] 0 2024-07-16T09:55:45.299 INFO:tasks.workunit.client.1.smithi176.stdout:4/683: dread da/fcc [0,4194304] 0 2024-07-16T09:55:45.301 INFO:tasks.workunit.client.1.smithi176.stdout:8/712: mkdir d8/d14/d1e/d33/df6 0 2024-07-16T09:55:45.301 INFO:tasks.workunit.client.1.smithi176.stdout:2/839: fdatasync dd/d36/fb2 0 2024-07-16T09:55:45.303 INFO:tasks.workunit.client.1.smithi176.stdout:7/688: readlink d4/dd/d26/dc2/l18 0 2024-07-16T09:55:45.303 INFO:tasks.workunit.client.1.smithi176.stdout:6/660: write d1/d5d/d9e/fd1 [683262,81779] 0 2024-07-16T09:55:45.303 INFO:tasks.workunit.client.1.smithi176.stdout:0/823: creat d5/d1c/d24/d26/dbd/de6/f112 x:0 0 0 2024-07-16T09:55:45.303 INFO:tasks.workunit.client.1.smithi176.stdout:4/684: symlink da/d41/lde 0 2024-07-16T09:55:45.304 INFO:tasks.workunit.client.1.smithi176.stdout:5/829: dread d4/d14/d3d/f55 [0,4194304] 0 2024-07-16T09:55:45.305 INFO:tasks.workunit.client.1.smithi176.stdout:2/840: chown dd/d49/d52/cbb 205 1 2024-07-16T09:55:45.305 INFO:tasks.workunit.client.1.smithi176.stdout:7/689: read d4/d47/d5d/fad [545899,36184] 0 2024-07-16T09:55:45.306 INFO:tasks.workunit.client.1.smithi176.stdout:0/824: write d5/d1c/fb8 [310160,103516] 0 2024-07-16T09:55:45.307 INFO:tasks.workunit.client.1.smithi176.stdout:9/808: truncate d5/d1d/d20/d78/dd0/fd7 1298092 0 2024-07-16T09:55:45.308 INFO:tasks.workunit.client.1.smithi176.stdout:5/830: creat d4/da9/dc4/f12f x:0 0 0 2024-07-16T09:55:45.309 INFO:tasks.workunit.client.1.smithi176.stdout:6/661: truncate d1/d5/f2c 344546 0 2024-07-16T09:55:45.310 INFO:tasks.workunit.client.1.smithi176.stdout:0/825: chown d5/d2c/d6c/d83/d84/l97 11158780 1 2024-07-16T09:55:45.311 INFO:tasks.workunit.client.1.smithi176.stdout:7/690: creat d4/dd/db5/fe1 x:0 0 0 2024-07-16T09:55:45.312 INFO:tasks.workunit.client.1.smithi176.stdout:9/809: unlink d5/d1d/dd2/de3/lec 0 2024-07-16T09:55:45.314 INFO:tasks.workunit.client.1.smithi176.stdout:0/826: creat d5/d39/f113 x:0 0 0 2024-07-16T09:55:45.315 INFO:tasks.workunit.client.1.smithi176.stdout:6/662: mknod d1/d5/d47/d62/dbf/cee 0 2024-07-16T09:55:45.315 INFO:tasks.workunit.client.1.smithi176.stdout:0/827: readlink d5/d44/lb7 0 2024-07-16T09:55:45.317 INFO:tasks.workunit.client.1.smithi176.stdout:7/691: symlink d4/d47/d6d/db7/le2 0 2024-07-16T09:55:45.317 INFO:tasks.workunit.client.1.smithi176.stdout:8/713: dread d8/d23/f4a [0,4194304] 0 2024-07-16T09:55:45.323 INFO:tasks.workunit.client.1.smithi176.stdout:7/692: dwrite d4/d47/d5d/d9a/fcd [0,4194304] 0 2024-07-16T09:55:45.325 INFO:tasks.workunit.client.1.smithi176.stdout:0/828: mknod d5/da1/c114 0 2024-07-16T09:55:45.327 INFO:tasks.workunit.client.1.smithi176.stdout:0/829: truncate d5/d1c/db1/fcd 636376 0 2024-07-16T09:55:45.327 INFO:tasks.workunit.client.1.smithi176.stdout:8/714: mkdir d8/df7 0 2024-07-16T09:55:45.330 INFO:tasks.workunit.client.1.smithi176.stdout:7/693: write d4/dd/f15 [1564841,128064] 0 2024-07-16T09:55:45.331 INFO:tasks.workunit.client.1.smithi176.stdout:6/663: link d1/dc/d40/c8c d1/dc/d40/d71/cef 0 2024-07-16T09:55:45.332 INFO:tasks.workunit.client.1.smithi176.stdout:1/496: dwrite d4/d8/d12/f4b [0,4194304] 0 2024-07-16T09:55:45.333 INFO:tasks.workunit.client.1.smithi176.stdout:9/810: getdents d5/de/d30 0 2024-07-16T09:55:45.336 INFO:tasks.workunit.client.0.smithi156.stdout:3/633: creat d8/da/d28/d43/dd8/fdc x:0 0 0 2024-07-16T09:55:45.340 INFO:tasks.workunit.client.0.smithi156.stdout:3/634: read d8/da/d28/d43/fa6 [650471,56709] 0 2024-07-16T09:55:45.340 INFO:tasks.workunit.client.1.smithi176.stdout:9/811: write d5/d1d/d20/f108 [813339,120752] 0 2024-07-16T09:55:45.340 INFO:tasks.workunit.client.1.smithi176.stdout:6/664: creat d1/dc/d40/d71/ff0 x:0 0 0 2024-07-16T09:55:45.342 INFO:tasks.workunit.client.1.smithi176.stdout:8/715: write d8/d14/d1e/d7c/d82/df5/fd7 [702231,73917] 0 2024-07-16T09:55:45.342 INFO:tasks.workunit.client.1.smithi176.stdout:1/497: creat d4/db/fac x:0 0 0 2024-07-16T09:55:45.343 INFO:tasks.workunit.client.1.smithi176.stdout:4/685: truncate da/dc/d10/d96/d9b/db4/d40/d5d/f85 3804928 0 2024-07-16T09:55:45.344 INFO:tasks.workunit.client.0.smithi156.stdout:3/635: dread d8/da/d28/d43/f8b [0,4194304] 0 2024-07-16T09:55:45.344 INFO:tasks.workunit.client.1.smithi176.stdout:7/694: unlink d4/d47/d6d/d89/d66/fa4 0 2024-07-16T09:55:45.348 INFO:tasks.workunit.client.1.smithi176.stdout:6/665: symlink d1/d5/d47/lf1 0 2024-07-16T09:55:45.349 INFO:tasks.workunit.client.1.smithi176.stdout:0/830: dwrite d5/d25/da0/faf [0,4194304] 0 2024-07-16T09:55:45.351 INFO:tasks.workunit.client.1.smithi176.stdout:4/686: read da/d3d/d17/d54/fa9 [321228,86922] 0 2024-07-16T09:55:45.352 INFO:tasks.workunit.client.1.smithi176.stdout:9/812: mknod d5/d1d/d67/d6d/c10d 0 2024-07-16T09:55:45.352 INFO:tasks.workunit.client.0.smithi156.stdout:3/636: dwrite d8/da/d1d/d6f/fb5 [0,4194304] 0 2024-07-16T09:55:45.352 INFO:tasks.workunit.client.1.smithi176.stdout:1/498: rename d4/db/d68 to d4/d77/dad 0 2024-07-16T09:55:45.353 INFO:tasks.workunit.client.1.smithi176.stdout:5/831: dwrite d4/d37/d72/d38/d39/f5d [0,4194304] 0 2024-07-16T09:55:45.354 INFO:tasks.workunit.client.1.smithi176.stdout:4/687: stat da/d3d/d33/fdd 0 2024-07-16T09:55:45.355 INFO:tasks.workunit.client.1.smithi176.stdout:9/813: write d5/d1d/d20/f33 [589195,83392] 0 2024-07-16T09:55:45.356 INFO:tasks.workunit.client.0.smithi156.stdout:3/637: rmdir d8 39 2024-07-16T09:55:45.359 INFO:tasks.workunit.client.0.smithi156.stdout:3/638: chown d8/da/d21/d80/d99 283368499 1 2024-07-16T09:55:45.359 INFO:tasks.workunit.client.1.smithi176.stdout:4/688: dread - da/dc/d10/d96/d69/fb2 zero size 2024-07-16T09:55:45.360 INFO:tasks.workunit.client.1.smithi176.stdout:0/831: creat d5/d1c/d24/d26/dbd/de6/df8/f115 x:0 0 0 2024-07-16T09:55:45.361 INFO:tasks.workunit.client.1.smithi176.stdout:9/814: chown d5/d52/l95 5 1 2024-07-16T09:55:45.362 INFO:tasks.workunit.client.1.smithi176.stdout:6/666: creat d1/d61/ff2 x:0 0 0 2024-07-16T09:55:45.364 INFO:tasks.workunit.client.1.smithi176.stdout:8/716: mknod d8/d20/d2a/db9/dcb/d34/cf8 0 2024-07-16T09:55:45.365 INFO:tasks.workunit.client.1.smithi176.stdout:5/832: mknod d4/d37/d72/d38/d39/d9f/de8/d114/c130 0 2024-07-16T09:55:45.369 INFO:tasks.workunit.client.1.smithi176.stdout:8/717: dread - d8/d20/d2a/d53/d9a/dc1/fef zero size 2024-07-16T09:55:45.369 INFO:tasks.workunit.client.0.smithi156.stdout:6/633: dwrite d3/d13/d17/d4f/d94/d19/d31/d35/f62 [0,4194304] 0 2024-07-16T09:55:45.369 INFO:tasks.workunit.client.1.smithi176.stdout:7/695: dread d4/dd/d26/dc2/d17/d1d/f5b [0,4194304] 0 2024-07-16T09:55:45.370 INFO:tasks.workunit.client.1.smithi176.stdout:4/689: dread - da/d3d/dac/fb9 zero size 2024-07-16T09:55:45.371 INFO:tasks.workunit.client.1.smithi176.stdout:0/832: truncate d5/f18 3639422 0 2024-07-16T09:55:45.373 INFO:tasks.workunit.client.1.smithi176.stdout:7/696: chown d4/dd/f15 1530156 1 2024-07-16T09:55:45.373 INFO:tasks.workunit.client.1.smithi176.stdout:1/499: dwrite d4/d8/d12/d21/d25/d30/d44/d4d/f7e [0,4194304] 0 2024-07-16T09:55:45.375 INFO:tasks.workunit.client.1.smithi176.stdout:5/833: creat d4/d14/dc5/def/d11f/f131 x:0 0 0 2024-07-16T09:55:45.375 INFO:tasks.workunit.client.1.smithi176.stdout:6/667: dread d1/db3/f9a [0,4194304] 0 2024-07-16T09:55:45.375 INFO:tasks.workunit.client.1.smithi176.stdout:8/718: chown d8/d20/d2a/d31/d46/d6a/da8/cb3 25776 1 2024-07-16T09:55:45.376 INFO:tasks.workunit.client.0.smithi156.stdout:6/634: truncate d3/d13/d17/d4f/d94/d19/d2e/f3e 2081660 0 2024-07-16T09:55:45.377 INFO:tasks.workunit.client.0.smithi156.stdout:6/635: chown d3/d13/d17/d4f/d94/d19/d2e/l42 53371116 1 2024-07-16T09:55:45.382 INFO:tasks.workunit.client.1.smithi176.stdout:4/690: truncate da/d3d/d17/f2a 5056627 0 2024-07-16T09:55:45.382 INFO:tasks.workunit.client.0.smithi156.stdout:6/636: chown d3/d13/d17/d4f/d94/d19/d2e/f74 77394 1 2024-07-16T09:55:45.383 INFO:tasks.workunit.client.0.smithi156.stdout:6/637: chown d3/d13/d17/d4f/d4d/d3b/d5a/fb9 30991 1 2024-07-16T09:55:45.384 INFO:tasks.workunit.client.1.smithi176.stdout:7/697: link d4/dd/f15 d4/dd/fe3 0 2024-07-16T09:55:45.384 INFO:tasks.workunit.client.1.smithi176.stdout:9/815: dread d5/d1d/d20/f33 [0,4194304] 0 2024-07-16T09:55:45.386 INFO:tasks.workunit.client.1.smithi176.stdout:5/834: symlink d4/d37/dbb/l132 0 2024-07-16T09:55:45.386 INFO:tasks.workunit.client.1.smithi176.stdout:7/698: chown d4/d47/d6d 26 1 2024-07-16T09:55:45.387 INFO:tasks.workunit.client.1.smithi176.stdout:6/668: mknod d1/d5/d47/d62/d77/cf3 0 2024-07-16T09:55:45.387 INFO:tasks.workunit.client.1.smithi176.stdout:4/691: write da/dc/d10/d96/d9b/fdb [463110,30200] 0 2024-07-16T09:55:45.389 INFO:tasks.workunit.client.0.smithi156.stdout:6/638: link d3/d13/d17/d4f/d4d/d3b/d77/fc3 d3/d13/d17/d4f/d4d/d3b/d77/fd2 0 2024-07-16T09:55:45.391 INFO:tasks.workunit.client.1.smithi176.stdout:0/833: dwrite d5/d1c/d24/d35/d50/d58/f6f [0,4194304] 0 2024-07-16T09:55:45.392 INFO:tasks.workunit.client.0.smithi156.stdout:6/639: readlink d3/d13/d17/d4f/d94/d19/d31/l9a 0 2024-07-16T09:55:45.394 INFO:tasks.workunit.client.0.smithi156.stdout:6/640: read - d3/d13/d17/d4f/d30/d45/f6d zero size 2024-07-16T09:55:45.395 INFO:tasks.workunit.client.0.smithi156.stdout:6/641: readlink d3/d13/d17/d4f/d1e/l89 0 2024-07-16T09:55:45.396 INFO:tasks.workunit.client.0.smithi156.stdout:6/642: fsync d3/d13/fab 0 2024-07-16T09:55:45.399 INFO:tasks.workunit.client.1.smithi176.stdout:9/816: chown d5/d1d/d77/d98/ldc 783 1 2024-07-16T09:55:45.400 INFO:tasks.workunit.client.1.smithi176.stdout:8/719: dwrite d8/d14/d1e/d33/d9d/fce [0,4194304] 0 2024-07-16T09:55:45.401 INFO:tasks.workunit.client.0.smithi156.stdout:6/643: dread d3/d13/d17/d4f/d94/d19/d31/d35/f62 [0,4194304] 0 2024-07-16T09:55:45.401 INFO:tasks.workunit.client.0.smithi156.stdout:3/639: dread d8/f10 [0,4194304] 0 2024-07-16T09:55:45.401 INFO:tasks.workunit.client.1.smithi176.stdout:5/835: read d4/d37/d72/d38/f99 [707526,28899] 0 2024-07-16T09:55:45.404 INFO:tasks.workunit.client.0.smithi156.stdout:6/644: truncate d3/d13/d17/d4f/d94/d19/d31/f4e 4527618 0 2024-07-16T09:55:45.404 INFO:tasks.workunit.client.0.smithi156.stdout:3/640: symlink d8/da/d1d/d2c/ldd 0 2024-07-16T09:55:45.405 INFO:tasks.workunit.client.0.smithi156.stdout:3/641: write d8/d14/fc7 [264446,74957] 0 2024-07-16T09:55:45.406 INFO:tasks.workunit.client.0.smithi156.stdout:3/642: stat d8/da/le 0 2024-07-16T09:55:45.409 INFO:tasks.workunit.client.0.smithi156.stdout:6/645: rmdir d3/d13/d17/d4f/dbf 0 2024-07-16T09:55:45.410 INFO:tasks.workunit.client.0.smithi156.stdout:3/643: mknod d8/d14/cde 0 2024-07-16T09:55:45.411 INFO:tasks.workunit.client.0.smithi156.stdout:3/644: dread - d8/da/d28/d43/dd8/fdc zero size 2024-07-16T09:55:45.413 INFO:tasks.workunit.client.0.smithi156.stdout:6/646: symlink d3/d13/d17/d4f/d30/d45/ld3 0 2024-07-16T09:55:45.415 INFO:tasks.workunit.client.0.smithi156.stdout:6/647: mknod d3/d13/d17/d4f/d30/d45/d99/cd4 0 2024-07-16T09:55:45.415 INFO:tasks.workunit.client.0.smithi156.stdout:6/648: dread - d3/d13/d17/d4f/d94/d19/fc0 zero size 2024-07-16T09:55:45.418 INFO:tasks.workunit.client.0.smithi156.stdout:6/649: creat d3/d13/d17/d4f/d4d/d3b/da5/fd5 x:0 0 0 2024-07-16T09:55:45.421 INFO:tasks.workunit.client.0.smithi156.stdout:6/650: mknod d3/d13/d17/d4f/d94/d19/d31/cd6 0 2024-07-16T09:55:45.422 INFO:tasks.workunit.client.0.smithi156.stdout:3/645: dwrite d8/da/f94 [0,4194304] 0 2024-07-16T09:55:45.425 INFO:tasks.workunit.client.0.smithi156.stdout:3/646: mkdir d8/ddf 0 2024-07-16T09:55:45.431 INFO:tasks.workunit.client.0.smithi156.stdout:6/651: truncate d3/d13/d17/f6c 4076078 0 2024-07-16T09:55:45.433 INFO:tasks.workunit.client.0.smithi156.stdout:3/647: chown d8/da/d1d/d90/fd3 2780789 1 2024-07-16T09:55:45.437 INFO:tasks.workunit.client.0.smithi156.stdout:3/648: rename d8/da/d1d/d2c/d33 to d8/da/d1d/d2c/de0 0 2024-07-16T09:55:45.439 INFO:tasks.workunit.client.0.smithi156.stdout:6/652: dwrite d3/d13/d17/d4f/d4d/d3b/da5/fd5 [0,4194304] 0 2024-07-16T09:55:45.443 INFO:tasks.workunit.client.0.smithi156.stdout:3/649: rename d8/da/d1d/d2c/c92 to d8/da/d1d/d6f/db3/dc4/ce1 0 2024-07-16T09:55:45.445 INFO:tasks.workunit.client.0.smithi156.stdout:3/650: fsync d8/da/d28/f2a 0 2024-07-16T09:55:45.446 INFO:tasks.workunit.client.0.smithi156.stdout:6/653: dwrite d3/d13/d17/d4f/d94/d19/d2e/f97 [0,4194304] 0 2024-07-16T09:55:45.446 INFO:tasks.workunit.client.0.smithi156.stdout:3/651: chown d8/da/d28/d43 1 1 2024-07-16T09:55:45.449 INFO:tasks.workunit.client.0.smithi156.stdout:3/652: unlink d8/da/d1d/d6f/fce 0 2024-07-16T09:55:45.453 INFO:tasks.workunit.client.0.smithi156.stdout:3/653: getdents d8/da/d13 0 2024-07-16T09:55:45.455 INFO:tasks.workunit.client.0.smithi156.stdout:6/654: fdatasync d3/d13/d17/d4f/d4d/d3b/da5/fd5 0 2024-07-16T09:55:45.461 INFO:tasks.workunit.client.0.smithi156.stdout:3/654: dwrite d8/da/d28/f6e [0,4194304] 0 2024-07-16T09:55:45.462 INFO:tasks.workunit.client.0.smithi156.stdout:6/655: dwrite d3/d13/d17/f2b [0,4194304] 0 2024-07-16T09:55:45.464 INFO:tasks.workunit.client.0.smithi156.stdout:8/618: sync 2024-07-16T09:55:45.464 INFO:tasks.workunit.client.0.smithi156.stdout:4/757: sync 2024-07-16T09:55:45.464 INFO:tasks.workunit.client.0.smithi156.stdout:7/564: sync 2024-07-16T09:55:45.464 INFO:tasks.workunit.client.0.smithi156.stdout:2/795: sync 2024-07-16T09:55:45.464 INFO:tasks.workunit.client.0.smithi156.stdout:5/686: sync 2024-07-16T09:55:45.464 INFO:tasks.workunit.client.0.smithi156.stdout:3/655: creat d8/da/d28/d43/fe2 x:0 0 0 2024-07-16T09:55:45.466 INFO:tasks.workunit.client.0.smithi156.stdout:7/565: stat dc/d13/d24/l89 0 2024-07-16T09:55:45.466 INFO:tasks.workunit.client.0.smithi156.stdout:8/619: fdatasync d0/d11/d13/d1c/d24/f42 0 2024-07-16T09:55:45.466 INFO:tasks.workunit.client.0.smithi156.stdout:5/687: fdatasync d1b/d1f/d45/f50 0 2024-07-16T09:55:45.466 INFO:tasks.workunit.client.0.smithi156.stdout:2/796: chown d0/d5/d29/d2d/l41 341919866 1 2024-07-16T09:55:45.468 INFO:tasks.workunit.client.0.smithi156.stdout:2/797: stat d0/d13/cb5 0 2024-07-16T09:55:45.469 INFO:tasks.workunit.client.0.smithi156.stdout:4/758: mknod d4/db/dc/dcc/dec/c10d 0 2024-07-16T09:55:45.471 INFO:tasks.workunit.client.0.smithi156.stdout:2/798: mkdir d0/d5/d8b/d101 0 2024-07-16T09:55:45.474 INFO:tasks.workunit.client.0.smithi156.stdout:3/656: dwrite d8/da/d28/d43/fa6 [0,4194304] 0 2024-07-16T09:55:45.476 INFO:tasks.workunit.client.0.smithi156.stdout:5/688: creat d1b/d1f/d45/d52/d43/d85/d97/d37/d88/dbc/dc2/feb x:0 0 0 2024-07-16T09:55:45.479 INFO:tasks.workunit.client.0.smithi156.stdout:3/657: mknod d8/da/d13/dca/ce3 0 2024-07-16T09:55:45.479 INFO:tasks.workunit.client.0.smithi156.stdout:5/689: readlink d1b/d1f/d45/d52/d43/d85/d97/d37/d88/d61/l9a 0 2024-07-16T09:55:45.488 INFO:tasks.workunit.client.0.smithi156.stdout:5/690: dwrite d1b/d1f/d42/d6d/d9f/fe3 [0,4194304] 0 2024-07-16T09:55:45.488 INFO:tasks.workunit.client.0.smithi156.stdout:2/799: dread d0/d13/f8d [0,4194304] 0 2024-07-16T09:55:45.489 INFO:tasks.workunit.client.0.smithi156.stdout:2/800: fdatasync d0/d13/d17/d1c/f5e 0 2024-07-16T09:55:45.492 INFO:tasks.workunit.client.0.smithi156.stdout:2/801: stat d0/d5/d29/d2d/f31 0 2024-07-16T09:55:45.498 INFO:tasks.workunit.client.0.smithi156.stdout:8/620: dwrite d0/d11/f15 [0,4194304] 0 2024-07-16T09:55:45.498 INFO:tasks.workunit.client.0.smithi156.stdout:4/759: write d4/db/d47/fbf [728682,88332] 0 2024-07-16T09:55:45.499 INFO:tasks.workunit.client.0.smithi156.stdout:3/658: truncate d8/da/d21/d80/f82 2046897 0 2024-07-16T09:55:45.500 INFO:tasks.workunit.client.0.smithi156.stdout:5/691: write d1b/f34 [467650,86295] 0 2024-07-16T09:55:45.501 INFO:tasks.workunit.client.0.smithi156.stdout:8/621: dread - d0/d11/d2e/fd1 zero size 2024-07-16T09:55:45.502 INFO:tasks.workunit.client.0.smithi156.stdout:4/760: fdatasync d4/db/dc/f42 0 2024-07-16T09:55:45.502 INFO:tasks.workunit.client.0.smithi156.stdout:3/659: read d8/da/d28/d43/f8b [254265,18759] 0 2024-07-16T09:55:45.502 INFO:tasks.workunit.client.0.smithi156.stdout:2/802: truncate d0/d5/d76/f8a 1782004 0 2024-07-16T09:55:45.503 INFO:tasks.workunit.client.0.smithi156.stdout:8/622: stat d0/dd/d1f/c75 0 2024-07-16T09:55:45.505 INFO:tasks.workunit.client.0.smithi156.stdout:4/761: unlink d4/d23/d52/d71/l81 0 2024-07-16T09:55:45.508 INFO:tasks.workunit.client.0.smithi156.stdout:2/803: readlink d0/d5/d29/d2d/d5b/d62/l8c 0 2024-07-16T09:55:45.509 INFO:tasks.workunit.client.0.smithi156.stdout:2/804: write d0/d13/d17/f34 [6574610,129362] 0 2024-07-16T09:55:45.515 INFO:tasks.workunit.client.1.smithi176.stdout:7/699: truncate d4/dd/d26/dc2/d17/d1d/f73 1534827 0 2024-07-16T09:55:45.516 INFO:tasks.workunit.client.0.smithi156.stdout:5/692: truncate d1b/d1f/d54/d72/f79 317296 0 2024-07-16T09:55:45.517 INFO:tasks.workunit.client.1.smithi176.stdout:0/834: dread - d5/d1c/d24/d26/dbd/de6/f112 zero size 2024-07-16T09:55:45.517 INFO:tasks.workunit.client.1.smithi176.stdout:1/500: dwrite d4/d8/d12/d21/d25/d30/f61 [0,4194304] 0 2024-07-16T09:55:45.519 INFO:tasks.workunit.client.0.smithi156.stdout:3/660: creat d8/da/d28/d43/d71/fe4 x:0 0 0 2024-07-16T09:55:45.519 INFO:tasks.workunit.client.0.smithi156.stdout:3/661: readlink d8/da/l2d 0 2024-07-16T09:55:45.520 INFO:tasks.workunit.client.1.smithi176.stdout:9/817: creat d5/de/d30/f10e x:0 0 0 2024-07-16T09:55:45.521 INFO:tasks.workunit.client.0.smithi156.stdout:3/662: write d8/da/d1d/fc0 [688865,1563] 0 2024-07-16T09:55:45.523 INFO:tasks.workunit.client.0.smithi156.stdout:8/623: getdents d0/dd/d1f/d57 0 2024-07-16T09:55:45.523 INFO:tasks.workunit.client.0.smithi156.stdout:4/762: link d4/db/dc/dcc/fb7 d4/db/dc/dcc/dec/f10e 0 2024-07-16T09:55:45.525 INFO:tasks.workunit.client.0.smithi156.stdout:8/624: chown d0/d11/d13/d1c/d24/f42 25 1 2024-07-16T09:55:45.526 INFO:tasks.workunit.client.1.smithi176.stdout:5/836: rename d4/d14/dce/f119 to d4/d6c/f133 0 2024-07-16T09:55:45.526 INFO:tasks.workunit.client.1.smithi176.stdout:6/669: dwrite d1/d5/d7/fe6 [0,4194304] 0 2024-07-16T09:55:45.527 INFO:tasks.workunit.client.1.smithi176.stdout:7/700: symlink d4/d47/d6d/d89/le4 0 2024-07-16T09:55:45.527 INFO:tasks.workunit.client.0.smithi156.stdout:2/805: creat d0/d13/d17/f102 x:0 0 0 2024-07-16T09:55:45.528 INFO:tasks.workunit.client.1.smithi176.stdout:5/837: chown d4/l92 1 1 2024-07-16T09:55:45.529 INFO:tasks.workunit.client.1.smithi176.stdout:9/818: creat d5/d1d/d20/d78/f10f x:0 0 0 2024-07-16T09:55:45.529 INFO:tasks.workunit.client.1.smithi176.stdout:4/692: dwrite da/d3d/fa2 [0,4194304] 0 2024-07-16T09:55:45.530 INFO:tasks.workunit.client.1.smithi176.stdout:5/838: chown d4/d37/la0 0 1 2024-07-16T09:55:45.531 INFO:tasks.workunit.client.0.smithi156.stdout:0/711: sync 2024-07-16T09:55:45.531 INFO:tasks.workunit.client.1.smithi176.stdout:1/501: unlink d4/d3a/d54/f8b 0 2024-07-16T09:55:45.534 INFO:tasks.workunit.client.1.smithi176.stdout:6/670: unlink d1/f15 0 2024-07-16T09:55:45.535 INFO:tasks.workunit.client.1.smithi176.stdout:7/701: dread d4/fb [0,4194304] 0 2024-07-16T09:55:45.535 INFO:tasks.workunit.client.1.smithi176.stdout:5/839: fdatasync d4/f6 0 2024-07-16T09:55:45.537 INFO:tasks.workunit.client.0.smithi156.stdout:8/625: rename d0/dd/d1f/c75 to d0/d11/d20/d7f/cde 0 2024-07-16T09:55:45.544 INFO:tasks.workunit.client.0.smithi156.stdout:2/806: mkdir d0/d5/d29/d2d/d5b/d5f/ddc/d103 0 2024-07-16T09:55:45.544 INFO:tasks.workunit.client.1.smithi176.stdout:9/819: unlink d5/de/d21/ce7 0 2024-07-16T09:55:45.545 INFO:tasks.workunit.client.1.smithi176.stdout:4/693: creat da/d41/d5b/dba/fdf x:0 0 0 2024-07-16T09:55:45.545 INFO:tasks.workunit.client.0.smithi156.stdout:5/693: getdents d1b/d1f/d45/d52/d43 0 2024-07-16T09:55:45.546 INFO:tasks.workunit.client.1.smithi176.stdout:7/702: chown d4/d1f/d22/l84 1550 1 2024-07-16T09:55:45.547 INFO:tasks.workunit.client.0.smithi156.stdout:0/712: mkdir d0/d1a/d4d/d9/d27/d2c/d6d/de4 0 2024-07-16T09:55:45.548 INFO:tasks.workunit.client.0.smithi156.stdout:4/763: symlink d4/db/dc/dcc/dec/ded/d87/df2/l10f 0 2024-07-16T09:55:45.549 INFO:tasks.workunit.client.0.smithi156.stdout:0/713: dread - d0/d1a/d4d/d9/d27/d50/d90/fd8 zero size 2024-07-16T09:55:45.550 INFO:tasks.workunit.client.0.smithi156.stdout:3/663: link d8/da/d28/d40/f56 d8/da/d1d/d2c/fe5 0 2024-07-16T09:55:45.552 INFO:tasks.workunit.client.0.smithi156.stdout:8/626: rename d0/d11/d20/d3f/dbe/lc1 to d0/d11/d20/d7f/daf/ldf 0 2024-07-16T09:55:45.554 INFO:tasks.workunit.client.0.smithi156.stdout:5/694: truncate d1b/d1f/d45/d52/d43/d85/d97/d37/fb7 470619 0 2024-07-16T09:55:45.556 INFO:tasks.workunit.client.0.smithi156.stdout:0/714: dwrite d0/d22/d41/f8a [0,4194304] 0 2024-07-16T09:55:45.557 INFO:tasks.workunit.client.1.smithi176.stdout:6/671: write d1/f45 [4218297,36237] 0 2024-07-16T09:55:45.558 INFO:tasks.workunit.client.0.smithi156.stdout:0/715: chown d0/d1a/d4d/d9/d68/l9f 7 1 2024-07-16T09:55:45.560 INFO:tasks.workunit.client.0.smithi156.stdout:3/664: getdents d8/da/d13 0 2024-07-16T09:55:45.560 INFO:tasks.workunit.client.0.smithi156.stdout:0/716: fsync d0/d1a/d4d/d25/f55 0 2024-07-16T09:55:45.562 INFO:tasks.workunit.client.1.smithi176.stdout:5/840: dread d4/fe [0,4194304] 0 2024-07-16T09:55:45.563 INFO:tasks.workunit.client.0.smithi156.stdout:3/665: mknod d8/da/d28/ce6 0 2024-07-16T09:55:45.565 INFO:tasks.workunit.client.0.smithi156.stdout:0/717: creat d0/d1a/d4d/d5e/db7/fe5 x:0 0 0 2024-07-16T09:55:45.568 INFO:tasks.workunit.client.0.smithi156.stdout:0/718: mkdir d0/d1a/de6 0 2024-07-16T09:55:45.570 INFO:tasks.workunit.client.0.smithi156.stdout:0/719: truncate d0/d1a/d4d/d61/d97/fab 1152786 0 2024-07-16T09:55:45.572 INFO:tasks.workunit.client.0.smithi156.stdout:2/807: write d0/d5/d29/d2d/d5b/fc1 [1009450,54977] 0 2024-07-16T09:55:45.572 INFO:tasks.workunit.client.1.smithi176.stdout:1/502: fsync d4/d8/d12/d21/d34/f58 0 2024-07-16T09:55:45.577 INFO:tasks.workunit.client.0.smithi156.stdout:2/808: rmdir d0/d13 39 2024-07-16T09:55:45.578 INFO:tasks.workunit.client.1.smithi176.stdout:5/841: creat d4/d37/dbb/f134 x:0 0 0 2024-07-16T09:55:45.579 INFO:tasks.workunit.client.0.smithi156.stdout:3/666: write d8/da/d13/f57 [624266,116918] 0 2024-07-16T09:55:45.579 INFO:tasks.workunit.client.0.smithi156.stdout:8/627: dwrite d0/dd/f19 [0,4194304] 0 2024-07-16T09:55:45.580 INFO:tasks.workunit.client.0.smithi156.stdout:0/720: dwrite d0/d1a/d4d/d9/d27/d50/f5f [0,4194304] 0 2024-07-16T09:55:45.580 INFO:tasks.workunit.client.1.smithi176.stdout:1/503: mknod d4/d3a/cae 0 2024-07-16T09:55:45.581 INFO:tasks.workunit.client.0.smithi156.stdout:2/809: write d0/d5/d29/d2d/d5b/d62/fb9 [573270,75138] 0 2024-07-16T09:55:45.582 INFO:tasks.workunit.client.0.smithi156.stdout:3/667: chown d8/da/d28/d43/d71/f8c 0 1 2024-07-16T09:55:45.582 INFO:tasks.workunit.client.0.smithi156.stdout:8/628: chown d0/d11/c18 4 1 2024-07-16T09:55:45.584 INFO:tasks.workunit.client.1.smithi176.stdout:4/694: unlink da/d3d/c43 0 2024-07-16T09:55:45.584 INFO:tasks.workunit.client.1.smithi176.stdout:9/820: mknod d5/d1d/d20/c110 0 2024-07-16T09:55:45.584 INFO:tasks.workunit.client.0.smithi156.stdout:0/721: write d0/d22/f3c [1357226,44212] 0 2024-07-16T09:55:45.588 INFO:tasks.workunit.client.0.smithi156.stdout:0/722: write d0/d1a/d4d/d9/d27/d8d/fa4 [4849469,66921] 0 2024-07-16T09:55:45.589 INFO:tasks.workunit.client.0.smithi156.stdout:8/629: symlink d0/dd/d1f/d57/dc0/le0 0 2024-07-16T09:55:45.590 INFO:tasks.workunit.client.0.smithi156.stdout:0/723: stat d0/d1a/d4d/d61/cc2 0 2024-07-16T09:55:45.594 INFO:tasks.workunit.client.1.smithi176.stdout:5/842: creat d4/d10/d118/f135 x:0 0 0 2024-07-16T09:55:45.594 INFO:tasks.workunit.client.0.smithi156.stdout:2/810: link d0/d5/d8b/feb d0/d13/d17/f104 0 2024-07-16T09:55:45.596 INFO:tasks.workunit.client.0.smithi156.stdout:2/811: mknod d0/d5/d29/d2d/d5b/d5f/ddc/c105 0 2024-07-16T09:55:45.596 INFO:tasks.workunit.client.0.smithi156.stdout:2/812: fdatasync d0/d5/d29/d2d/d5b/d5f/f7d 0 2024-07-16T09:55:45.597 INFO:tasks.workunit.client.0.smithi156.stdout:2/813: fsync d0/d5/d29/d2d/d5b/d62/fb9 0 2024-07-16T09:55:45.599 INFO:tasks.workunit.client.0.smithi156.stdout:0/724: mkdir d0/d1a/d4d/d61/d97/de1/de7 0 2024-07-16T09:55:45.602 INFO:tasks.workunit.client.0.smithi156.stdout:8/630: link d0/d11/d4a/d55/l81 d0/d11/d13/d1c/d39/d8a/dd7/le1 0 2024-07-16T09:55:45.603 INFO:tasks.workunit.client.1.smithi176.stdout:5/843: dwrite d4/d14/f9c [0,4194304] 0 2024-07-16T09:55:45.603 INFO:tasks.workunit.client.0.smithi156.stdout:8/631: fsync d0/f1 0 2024-07-16T09:55:45.603 INFO:tasks.workunit.client.0.smithi156.stdout:0/725: rmdir d0/d22/d41 39 2024-07-16T09:55:45.603 INFO:tasks.workunit.client.0.smithi156.stdout:3/668: link d8/da/d28/l51 d8/da/d28/d43/d71/da2/le7 0 2024-07-16T09:55:45.604 INFO:tasks.workunit.client.0.smithi156.stdout:8/632: write d0/dd/f3b [245257,126037] 0 2024-07-16T09:55:45.605 INFO:tasks.workunit.client.0.smithi156.stdout:0/726: fdatasync d0/d1a/d4d/d9/d27/d50/fae 0 2024-07-16T09:55:45.605 INFO:tasks.workunit.client.0.smithi156.stdout:8/633: chown d0/dd/d1f/f47 1 1 2024-07-16T09:55:45.606 INFO:tasks.workunit.client.0.smithi156.stdout:0/727: stat d0/d1a/d76/d92 0 2024-07-16T09:55:45.606 INFO:tasks.workunit.client.0.smithi156.stdout:2/814: creat d0/d5/d29/d2d/d5b/df3/f106 x:0 0 0 2024-07-16T09:55:45.608 INFO:tasks.workunit.client.0.smithi156.stdout:2/815: write d0/d13/d17/d1c/d86/fb7 [5844892,25474] 0 2024-07-16T09:55:45.608 INFO:tasks.workunit.client.0.smithi156.stdout:0/728: read d0/d1a/d76/f87 [11652,30287] 0 2024-07-16T09:55:45.609 INFO:tasks.workunit.client.1.smithi176.stdout:9/821: dread d5/f1c [0,4194304] 0 2024-07-16T09:55:45.609 INFO:tasks.workunit.client.0.smithi156.stdout:8/634: stat d0/dd/d1f/cab 0 2024-07-16T09:55:45.610 INFO:tasks.workunit.client.0.smithi156.stdout:8/635: chown d0/dd/d1f/l28 3 1 2024-07-16T09:55:45.614 INFO:tasks.workunit.client.0.smithi156.stdout:3/669: mkdir d8/d14/de8 0 2024-07-16T09:55:45.617 INFO:tasks.workunit.client.0.smithi156.stdout:3/670: dread d8/da/d28/d40/f54 [0,4194304] 0 2024-07-16T09:55:45.619 INFO:tasks.workunit.client.0.smithi156.stdout:3/671: truncate d8/da/d21/d80/d99/fc3 251705 0 2024-07-16T09:55:45.620 INFO:tasks.workunit.client.0.smithi156.stdout:8/636: rename d0/d74/fb0 to d0/d11/d2e/d35/fe2 0 2024-07-16T09:55:45.621 INFO:tasks.workunit.client.0.smithi156.stdout:3/672: truncate d8/da/d1d/d90/fd3 359306 0 2024-07-16T09:55:45.623 INFO:tasks.workunit.client.0.smithi156.stdout:2/816: mknod d0/d13/c107 0 2024-07-16T09:55:45.625 INFO:tasks.workunit.client.0.smithi156.stdout:8/637: chown d0/c1d 16384240 1 2024-07-16T09:55:45.626 INFO:tasks.workunit.client.1.smithi176.stdout:9/822: dread d5/d52/ff0 [0,4194304] 0 2024-07-16T09:55:45.636 INFO:tasks.workunit.client.0.smithi156.stdout:0/729: dwrite d0/f36 [0,4194304] 0 2024-07-16T09:55:45.637 INFO:tasks.workunit.client.0.smithi156.stdout:2/817: truncate d0/d5/d29/fbe 506911 0 2024-07-16T09:55:45.638 INFO:tasks.workunit.client.1.smithi176.stdout:5/844: rename d4/d37/d72/d38/d102/d105/f12d to d4/d37/d72/d38/d102/d105/d111/f136 0 2024-07-16T09:55:45.638 INFO:tasks.workunit.client.0.smithi156.stdout:8/638: dwrite d0/d11/d13/f14 [0,4194304] 0 2024-07-16T09:55:45.642 INFO:tasks.workunit.client.1.smithi176.stdout:9/823: creat d5/d1d/dc7/d89/f111 x:0 0 0 2024-07-16T09:55:45.642 INFO:tasks.workunit.client.0.smithi156.stdout:3/673: dread d8/da/d28/f66 [0,4194304] 0 2024-07-16T09:55:45.643 INFO:tasks.workunit.client.0.smithi156.stdout:2/818: truncate d0/d5/d8b/fc8 536644 0 2024-07-16T09:55:45.645 INFO:tasks.workunit.client.0.smithi156.stdout:8/639: mknod d0/d11/d13/d1c/d39/ce3 0 2024-07-16T09:55:45.647 INFO:tasks.workunit.client.1.smithi176.stdout:4/695: write da/d3d/d17/fbb [4146499,43078] 0 2024-07-16T09:55:45.648 INFO:tasks.workunit.client.1.smithi176.stdout:6/672: rmdir d1/db3 39 2024-07-16T09:55:45.649 INFO:tasks.workunit.client.0.smithi156.stdout:2/819: rename d0/d13/d17/d1c/faa to d0/d5/d76/f108 0 2024-07-16T09:55:45.650 INFO:tasks.workunit.client.1.smithi176.stdout:9/824: dread d5/d1d/d20/f108 [0,4194304] 0 2024-07-16T09:55:45.651 INFO:tasks.workunit.client.0.smithi156.stdout:2/820: chown d0/d5/d8b/c9d 1665 1 2024-07-16T09:55:45.652 INFO:tasks.workunit.client.0.smithi156.stdout:3/674: getdents d8/da/d1d/db2 0 2024-07-16T09:55:45.654 INFO:tasks.workunit.client.1.smithi176.stdout:6/673: creat d1/dc/ff4 x:0 0 0 2024-07-16T09:55:45.655 INFO:tasks.workunit.client.0.smithi156.stdout:3/675: stat d8/d14/f30 0 2024-07-16T09:55:45.655 INFO:tasks.workunit.client.0.smithi156.stdout:0/730: dwrite d0/d1a/d4d/d25/f38 [0,4194304] 0 2024-07-16T09:55:45.656 INFO:tasks.workunit.client.1.smithi176.stdout:4/696: dwrite da/d3d/d33/fdd [0,4194304] 0 2024-07-16T09:55:45.661 INFO:tasks.workunit.client.1.smithi176.stdout:6/674: rename d1/d5/d7/fe6 to d1/d5/d47/ff5 0 2024-07-16T09:55:45.666 INFO:tasks.workunit.client.0.smithi156.stdout:0/731: write d0/d1a/d4d/f95 [72378,106273] 0 2024-07-16T09:55:45.667 INFO:tasks.workunit.client.1.smithi176.stdout:4/697: write da/f19 [6855483,68061] 0 2024-07-16T09:55:45.667 INFO:tasks.workunit.client.0.smithi156.stdout:3/676: truncate d8/da/d1d/f64 653366 0 2024-07-16T09:55:45.669 INFO:tasks.workunit.client.1.smithi176.stdout:6/675: creat d1/d5d/d9e/dd9/d6f/d7c/ff6 x:0 0 0 2024-07-16T09:55:45.670 INFO:tasks.workunit.client.0.smithi156.stdout:2/821: mkdir d0/d13/d17/d1c/d86/dde/d109 0 2024-07-16T09:55:45.670 INFO:tasks.workunit.client.1.smithi176.stdout:4/698: creat da/dc/d4b/fe0 x:0 0 0 2024-07-16T09:55:45.671 INFO:tasks.workunit.client.0.smithi156.stdout:0/732: chown d0/d1a/d76/d83/fad 13 1 2024-07-16T09:55:45.673 INFO:tasks.workunit.client.1.smithi176.stdout:4/699: chown da/dc/d10/d96/d9b/db4/fa5 0 1 2024-07-16T09:55:45.674 INFO:tasks.workunit.client.1.smithi176.stdout:6/676: rename d1/d5/d47/d62/d77/cf3 to d1/d5/d47/cf7 0 2024-07-16T09:55:45.674 INFO:tasks.workunit.client.0.smithi156.stdout:0/733: write d0/d1a/d4d/d9/d27/f81 [1210023,109958] 0 2024-07-16T09:55:45.675 INFO:tasks.workunit.client.1.smithi176.stdout:4/700: creat da/dc/d10/fe1 x:0 0 0 2024-07-16T09:55:45.676 INFO:tasks.workunit.client.1.smithi176.stdout:9/825: dread d5/de/da5/f5a [0,4194304] 0 2024-07-16T09:55:45.676 INFO:tasks.workunit.client.0.smithi156.stdout:3/677: dwrite d8/da/d28/d43/d63/f73 [0,4194304] 0 2024-07-16T09:55:45.677 INFO:tasks.workunit.client.1.smithi176.stdout:4/701: fsync da/dc/d10/d28/fc4 0 2024-07-16T09:55:45.677 INFO:tasks.workunit.client.1.smithi176.stdout:6/677: truncate d1/d5d/d9e/d8f/fc0 394462 0 2024-07-16T09:55:45.681 INFO:tasks.workunit.client.1.smithi176.stdout:9/826: rename d5/d1d/d77 to d5/de/da5/dc1/d112 0 2024-07-16T09:55:45.681 INFO:tasks.workunit.client.0.smithi156.stdout:2/822: dwrite d0/d5/d29/d2d/d5b/d5f/d89/f9a [0,4194304] 0 2024-07-16T09:55:45.685 INFO:tasks.workunit.client.1.smithi176.stdout:6/678: rename d1/d5d/d9e/dd9/c38 to d1/d5d/d9e/d8f/cf8 0 2024-07-16T09:55:45.686 INFO:tasks.workunit.client.1.smithi176.stdout:6/679: chown d1/dc/d40/f78 11 1 2024-07-16T09:55:45.688 INFO:tasks.workunit.client.0.smithi156.stdout:3/678: truncate d8/da/d13/f17 1052047 0 2024-07-16T09:55:45.690 INFO:tasks.workunit.client.0.smithi156.stdout:0/734: dwrite d0/d1a/d76/f87 [0,4194304] 0 2024-07-16T09:55:45.690 INFO:tasks.workunit.client.0.smithi156.stdout:2/823: truncate d0/d5/d29/fbe 593221 0 2024-07-16T09:55:45.693 INFO:tasks.workunit.client.0.smithi156.stdout:2/824: chown d0/d5/d76/da1/lce 2061197341 1 2024-07-16T09:55:45.693 INFO:tasks.workunit.client.0.smithi156.stdout:3/679: dread d8/da/d28/d43/d63/f73 [0,4194304] 0 2024-07-16T09:55:45.694 INFO:tasks.workunit.client.0.smithi156.stdout:0/735: read - d0/d1a/d4d/d9/f88 zero size 2024-07-16T09:55:45.698 INFO:tasks.workunit.client.0.smithi156.stdout:0/736: creat d0/d1a/d4d/d9/d68/dcb/fe8 x:0 0 0 2024-07-16T09:55:45.700 INFO:tasks.workunit.client.0.smithi156.stdout:2/825: mknod d0/d13/d17/d1c/d86/dde/df4/c10a 0 2024-07-16T09:55:45.702 INFO:tasks.workunit.client.0.smithi156.stdout:3/680: dwrite d8/d14/fc7 [0,4194304] 0 2024-07-16T09:55:45.704 INFO:tasks.workunit.client.0.smithi156.stdout:2/826: mkdir d0/d13/d17/d1c/d10b 0 2024-07-16T09:55:45.706 INFO:tasks.workunit.client.0.smithi156.stdout:3/681: symlink d8/da/d1d/d2c/le9 0 2024-07-16T09:55:45.707 INFO:tasks.workunit.client.0.smithi156.stdout:2/827: dread - d0/d13/d17/d1c/d51/dc6/fd4 zero size 2024-07-16T09:55:45.708 INFO:tasks.workunit.client.0.smithi156.stdout:3/682: creat d8/da/d21/d80/fea x:0 0 0 2024-07-16T09:55:45.709 INFO:tasks.workunit.client.0.smithi156.stdout:2/828: symlink d0/d5/d29/d2d/de1/l10c 0 2024-07-16T09:55:45.710 INFO:tasks.workunit.client.0.smithi156.stdout:2/829: dread - d0/d5/d29/d2d/de1/fe2 zero size 2024-07-16T09:55:45.711 INFO:tasks.workunit.client.0.smithi156.stdout:2/830: read - d0/d13/d17/d1c/d51/dc6/fd4 zero size 2024-07-16T09:55:45.713 INFO:tasks.workunit.client.0.smithi156.stdout:2/831: creat d0/d5/d8b/d101/f10d x:0 0 0 2024-07-16T09:55:45.735 INFO:tasks.workunit.client.0.smithi156.stdout:0/737: dwrite d0/d1a/d4d/d60/f93 [0,4194304] 0 2024-07-16T09:55:45.736 INFO:tasks.workunit.client.0.smithi156.stdout:2/832: dwrite d0/d13/dc9/fd1 [0,4194304] 0 2024-07-16T09:55:45.738 INFO:tasks.workunit.client.0.smithi156.stdout:2/833: mkdir d0/d9b/d10e 0 2024-07-16T09:55:45.742 INFO:tasks.workunit.client.0.smithi156.stdout:2/834: readlink d0/d5/d29/d2d/d5b/d62/lb0 0 2024-07-16T09:55:45.745 INFO:tasks.workunit.client.0.smithi156.stdout:0/738: dwrite d0/d1a/d4d/d25/f38 [4194304,4194304] 0 2024-07-16T09:55:45.747 INFO:tasks.workunit.client.0.smithi156.stdout:3/683: dread d8/da/d13/f20 [4194304,4194304] 0 2024-07-16T09:55:45.748 INFO:tasks.workunit.client.0.smithi156.stdout:2/835: mkdir d0/d5/d76/d10f 0 2024-07-16T09:55:45.749 INFO:tasks.workunit.client.0.smithi156.stdout:2/836: dread - d0/fd2 zero size 2024-07-16T09:55:45.753 INFO:tasks.workunit.client.0.smithi156.stdout:3/684: creat d8/da/d1d/d2c/de0/dad/feb x:0 0 0 2024-07-16T09:55:45.756 INFO:tasks.workunit.client.0.smithi156.stdout:0/739: dwrite d0/d1a/d4d/d9/d27/d2c/d6d/d9d/fac [0,4194304] 0 2024-07-16T09:55:45.757 INFO:tasks.workunit.client.0.smithi156.stdout:3/685: read - d8/da/d1d/d2c/de0/fbc zero size 2024-07-16T09:55:45.760 INFO:tasks.workunit.client.0.smithi156.stdout:3/686: mkdir d8/da/dec 0 2024-07-16T09:55:45.764 INFO:tasks.workunit.client.0.smithi156.stdout:3/687: creat d8/da/d1d/d90/fed x:0 0 0 2024-07-16T09:55:45.765 INFO:tasks.workunit.client.0.smithi156.stdout:3/688: chown d8/da/d1d/d2c/de0/f49 6446733 1 2024-07-16T09:55:45.766 INFO:tasks.workunit.client.0.smithi156.stdout:0/740: dwrite d0/d1a/d4d/f95 [0,4194304] 0 2024-07-16T09:55:45.766 INFO:tasks.workunit.client.0.smithi156.stdout:3/689: chown d8/da/d28/d43/d63 1070279 1 2024-07-16T09:55:45.767 INFO:tasks.workunit.client.0.smithi156.stdout:0/741: chown d0/d1a/d4d/d9/c7e 8 1 2024-07-16T09:55:45.770 INFO:tasks.workunit.client.0.smithi156.stdout:0/742: chown d0/d1a/d4d/d9/d27/c6f 15 1 2024-07-16T09:55:45.771 INFO:tasks.workunit.client.0.smithi156.stdout:0/743: write d0/d1a/d4d/d5e/db7/fe5 [705939,14094] 0 2024-07-16T09:55:45.776 INFO:tasks.workunit.client.0.smithi156.stdout:3/690: dwrite d8/da/d28/d43/d63/f73 [0,4194304] 0 2024-07-16T09:55:45.779 INFO:tasks.workunit.client.0.smithi156.stdout:2/837: dwrite d0/fee [0,4194304] 0 2024-07-16T09:55:45.780 INFO:tasks.workunit.client.0.smithi156.stdout:2/838: chown d0/d5/d8b/d101 579 1 2024-07-16T09:55:45.783 INFO:tasks.workunit.client.0.smithi156.stdout:0/744: dwrite d0/d22/d41/d52/ddb/fe3 [0,4194304] 0 2024-07-16T09:55:45.796 INFO:tasks.workunit.client.1.smithi176.stdout:6/680: unlink d1/d5d/f53 0 2024-07-16T09:55:45.796 INFO:tasks.workunit.client.0.smithi156.stdout:0/745: dwrite d0/d22/d41/d52/ddb/fe3 [0,4194304] 0 2024-07-16T09:55:45.798 INFO:tasks.workunit.client.1.smithi176.stdout:4/702: getdents da/d3d/d33 0 2024-07-16T09:55:45.801 INFO:tasks.workunit.client.0.smithi156.stdout:0/746: rename d0/d1a/l57 to d0/d1a/d4d/d5e/db7/le9 0 2024-07-16T09:55:45.803 INFO:tasks.workunit.client.0.smithi156.stdout:0/747: chown d0/d22/d41/d66/cc5 532853 1 2024-07-16T09:55:45.810 INFO:tasks.workunit.client.0.smithi156.stdout:0/748: rename d0/d1a/d4d/fda to d0/d1a/d4d/d9/d27/d2c/d6d/d9d/fea 0 2024-07-16T09:55:45.810 INFO:tasks.workunit.client.0.smithi156.stdout:3/691: dread d8/d14/f30 [0,4194304] 0 2024-07-16T09:55:45.811 INFO:tasks.workunit.client.0.smithi156.stdout:0/749: write d0/d22/f3c [4217202,105455] 0 2024-07-16T09:55:45.812 INFO:tasks.workunit.client.1.smithi176.stdout:3/599: sync 2024-07-16T09:55:45.817 INFO:tasks.workunit.client.0.smithi156.stdout:3/692: dwrite d8/d14/fb7 [0,4194304] 0 2024-07-16T09:55:45.819 INFO:tasks.workunit.client.0.smithi156.stdout:0/750: dread d0/d1a/d4d/d9/d27/d2c/d6d/d9d/fac [0,4194304] 0 2024-07-16T09:55:45.821 INFO:tasks.workunit.client.0.smithi156.stdout:0/751: chown d0/d1a/d76/d83 901466 1 2024-07-16T09:55:45.826 INFO:tasks.workunit.client.0.smithi156.stdout:3/693: dwrite d8/da/d1d/fc6 [0,4194304] 0 2024-07-16T09:55:45.827 INFO:tasks.workunit.client.0.smithi156.stdout:0/752: mknod d0/d1a/d4d/d9/d27/d2c/d6d/d9d/dbb/ceb 0 2024-07-16T09:55:45.829 INFO:tasks.workunit.client.0.smithi156.stdout:3/694: write d8/da/d13/fb6 [3210955,116154] 0 2024-07-16T09:55:45.834 INFO:tasks.workunit.client.0.smithi156.stdout:3/695: mkdir d8/da/d1d/d2c/dee 0 2024-07-16T09:55:45.837 INFO:tasks.workunit.client.0.smithi156.stdout:0/753: dwrite d0/d22/d41/d52/f74 [0,4194304] 0 2024-07-16T09:55:45.837 INFO:tasks.workunit.client.0.smithi156.stdout:3/696: creat d8/da/d1d/d6f/db3/dc4/fef x:0 0 0 2024-07-16T09:55:45.840 INFO:tasks.workunit.client.0.smithi156.stdout:0/754: creat d0/d1a/d4d/d9/d27/d2c/d6d/fec x:0 0 0 2024-07-16T09:55:45.840 INFO:tasks.workunit.client.0.smithi156.stdout:3/697: creat d8/da/d1d/d6f/db3/dc4/ff0 x:0 0 0 2024-07-16T09:55:45.841 INFO:tasks.workunit.client.0.smithi156.stdout:0/755: write d0/d22/d41/f8a [3725753,28383] 0 2024-07-16T09:55:45.843 INFO:tasks.workunit.client.0.smithi156.stdout:3/698: truncate d8/da/f3d 1954353 0 2024-07-16T09:55:45.845 INFO:tasks.workunit.client.0.smithi156.stdout:3/699: fsync d8/da/d1d/d2c/de0/f88 0 2024-07-16T09:55:45.845 INFO:tasks.workunit.client.0.smithi156.stdout:0/756: truncate d0/d1a/d4d/d9/d27/d50/f9a 264370 0 2024-07-16T09:55:45.846 INFO:tasks.workunit.client.0.smithi156.stdout:3/700: chown d8/da/d28/d43/dd8 0 1 2024-07-16T09:55:45.847 INFO:tasks.workunit.client.0.smithi156.stdout:3/701: write d8/da/d1d/d6f/fd1 [3388445,121284] 0 2024-07-16T09:55:45.848 INFO:tasks.workunit.client.0.smithi156.stdout:0/757: symlink d0/d1a/d4d/d9/d27/d50/led 0 2024-07-16T09:55:45.852 INFO:tasks.workunit.client.0.smithi156.stdout:3/702: getdents d8/da/d28 0 2024-07-16T09:55:45.860 INFO:tasks.workunit.client.0.smithi156.stdout:3/703: dwrite d8/da/f5d [4194304,4194304] 0 2024-07-16T09:55:45.860 INFO:tasks.workunit.client.0.smithi156.stdout:0/758: dwrite d0/d1a/d4d/d25/f55 [0,4194304] 0 2024-07-16T09:55:45.877 INFO:tasks.workunit.client.0.smithi156.stdout:9/777: sync 2024-07-16T09:55:45.878 INFO:tasks.workunit.client.0.smithi156.stdout:9/778: write d2/d5/d16/d57/d63/dd3/ddb/d74/dc2/dc4/d23/fbc [4430233,26075] 0 2024-07-16T09:55:45.879 INFO:tasks.workunit.client.0.smithi156.stdout:9/779: stat d2/d5/d16/d57/fac 0 2024-07-16T09:55:45.882 INFO:tasks.workunit.client.1.smithi176.stdout:6/681: stat d1/f45 0 2024-07-16T09:55:45.883 INFO:tasks.workunit.client.0.smithi156.stdout:0/759: fsync d0/d1a/d4d/d25/f55 0 2024-07-16T09:55:45.883 INFO:tasks.workunit.client.1.smithi176.stdout:9/827: link d5/d43/f99 d5/d1d/d67/f113 0 2024-07-16T09:55:45.884 INFO:tasks.workunit.client.1.smithi176.stdout:9/828: dread - d5/de/da5/dc1/d112/d97/fa1 zero size 2024-07-16T09:55:45.885 INFO:tasks.workunit.client.1.smithi176.stdout:4/703: rmdir da/d3d/d17/d54/d78 39 2024-07-16T09:55:45.886 INFO:tasks.workunit.client.0.smithi156.stdout:0/760: fdatasync d0/d1a/d4d/d9/f12 0 2024-07-16T09:55:45.887 INFO:tasks.workunit.client.1.smithi176.stdout:6/682: write d1/dc/f33 [1821609,51783] 0 2024-07-16T09:55:45.888 INFO:tasks.workunit.client.0.smithi156.stdout:9/780: dwrite d2/d5/d16/d57/d63/dd3/ddb/d74/fbe [0,4194304] 0 2024-07-16T09:55:45.889 INFO:tasks.workunit.client.1.smithi176.stdout:9/829: symlink d5/d1d/dd2/de3/d103/l114 0 2024-07-16T09:55:45.893 INFO:tasks.workunit.client.1.smithi176.stdout:9/830: chown d5/de/d30/daf/cba 30439212 1 2024-07-16T09:55:45.895 INFO:tasks.workunit.client.1.smithi176.stdout:4/704: dwrite da/fca [0,4194304] 0 2024-07-16T09:55:45.895 INFO:tasks.workunit.client.1.smithi176.stdout:3/600: write d3/d17/d1d/f76 [999404,3165] 0 2024-07-16T09:55:45.896 INFO:tasks.workunit.client.0.smithi156.stdout:0/761: link d0/d1a/d76/da7/dd3/le0 d0/d1a/d76/da7/lee 0 2024-07-16T09:55:45.897 INFO:tasks.workunit.client.1.smithi176.stdout:3/601: chown d3/da/ld 56452 1 2024-07-16T09:55:45.898 INFO:tasks.workunit.client.1.smithi176.stdout:4/705: symlink da/d3d/dac/le2 0 2024-07-16T09:55:45.899 INFO:tasks.workunit.client.1.smithi176.stdout:3/602: fdatasync d3/d17/d1d/d4f/d59/da0/f9e 0 2024-07-16T09:55:45.900 INFO:tasks.workunit.client.0.smithi156.stdout:0/762: symlink d0/d1a/d4d/lef 0 2024-07-16T09:55:45.904 INFO:tasks.workunit.client.1.smithi176.stdout:6/683: dread d1/d5d/d9e/d8f/fd5 [0,4194304] 0 2024-07-16T09:55:45.905 INFO:tasks.workunit.client.0.smithi156.stdout:9/781: truncate d2/d5/d16/d57/d63/dd3/ddb/d74/dc2/dc4/d3a/f49 3794743 0 2024-07-16T09:55:45.907 INFO:tasks.workunit.client.0.smithi156.stdout:9/782: rmdir d2/d5/d16/d57/d63/dd3/ddb/d3d/db1 39 2024-07-16T09:55:45.908 INFO:tasks.workunit.client.0.smithi156.stdout:9/783: fdatasync d2/d5/d16/d57/d63/dd3/ddb/f8b 0 2024-07-16T09:55:45.909 INFO:tasks.workunit.client.0.smithi156.stdout:0/763: link d0/d1a/d4d/d9/d27/l3f d0/d22/d41/lf0 0 2024-07-16T09:55:45.911 INFO:tasks.workunit.client.0.smithi156.stdout:0/764: stat d0/d1a/d4d/d9/d27/d3d/ce2 0 2024-07-16T09:55:45.911 INFO:tasks.workunit.client.1.smithi176.stdout:9/831: dwrite d5/d1d/dc7/d89/fb9 [4194304,4194304] 0 2024-07-16T09:55:45.911 INFO:tasks.workunit.client.1.smithi176.stdout:3/603: dwrite d3/d17/d83/fb0 [0,4194304] 0 2024-07-16T09:55:45.914 INFO:tasks.workunit.client.0.smithi156.stdout:0/765: creat d0/d22/d41/da9/ff1 x:0 0 0 2024-07-16T09:55:45.914 INFO:tasks.workunit.client.1.smithi176.stdout:9/832: mkdir d5/d1d/d20/d115 0 2024-07-16T09:55:45.915 INFO:tasks.workunit.client.0.smithi156.stdout:9/784: truncate d2/d5/d16/d57/d63/dd3/ddb/d74/dc2/dc4/d23/f88 1999176 0 2024-07-16T09:55:45.916 INFO:tasks.workunit.client.1.smithi176.stdout:9/833: write d5/d1d/d67/d6d/fd6 [296619,8920] 0 2024-07-16T09:55:45.916 INFO:tasks.workunit.client.0.smithi156.stdout:0/766: mknod d0/d22/d41/d66/cf2 0 2024-07-16T09:55:45.921 INFO:tasks.workunit.client.1.smithi176.stdout:3/604: truncate d3/d17/f51 1880681 0 2024-07-16T09:55:45.925 INFO:tasks.workunit.client.1.smithi176.stdout:9/834: dwrite d5/d52/fa4 [0,4194304] 0 2024-07-16T09:55:45.928 INFO:tasks.workunit.client.1.smithi176.stdout:6/684: dwrite d1/d5d/d9e/d8f/f8d [0,4194304] 0 2024-07-16T09:55:45.928 INFO:tasks.workunit.client.1.smithi176.stdout:3/605: write d3/d17/d1d/d4f/d59/f93 [998968,8275] 0 2024-07-16T09:55:45.933 INFO:tasks.workunit.client.1.smithi176.stdout:9/835: unlink d5/de/d30/daf/cbb 0 2024-07-16T09:55:45.935 INFO:tasks.workunit.client.1.smithi176.stdout:6/685: unlink d1/d5d/db9/l60 0 2024-07-16T09:55:45.936 INFO:tasks.workunit.client.1.smithi176.stdout:4/706: dread da/dc/d10/d96/faf [0,4194304] 0 2024-07-16T09:55:45.936 INFO:tasks.workunit.client.1.smithi176.stdout:3/606: creat d3/da/d7b/fb8 x:0 0 0 2024-07-16T09:55:45.939 INFO:tasks.workunit.client.1.smithi176.stdout:4/707: symlink da/d3d/d17/le3 0 2024-07-16T09:55:45.943 INFO:tasks.workunit.client.1.smithi176.stdout:4/708: truncate da/dc/d10/d28/f7f 402786 0 2024-07-16T09:55:45.943 INFO:tasks.workunit.client.1.smithi176.stdout:9/836: link d5/de/df9/cfd d5/d1d/d20/d78/c116 0 2024-07-16T09:55:45.948 INFO:tasks.workunit.client.1.smithi176.stdout:4/709: creat da/dc/d10/d96/d9b/db4/d4a/fe4 x:0 0 0 2024-07-16T09:55:45.950 INFO:tasks.workunit.client.1.smithi176.stdout:9/837: getdents d5/de/da5/dc1/d112 0 2024-07-16T09:55:45.950 INFO:tasks.workunit.client.1.smithi176.stdout:4/710: fsync da/dc/d10/d28/fb7 0 2024-07-16T09:55:45.951 INFO:tasks.workunit.client.1.smithi176.stdout:4/711: readlink da/d3d/d17/l2b 0 2024-07-16T09:55:45.954 INFO:tasks.workunit.client.1.smithi176.stdout:4/712: mkdir da/de5 0 2024-07-16T09:55:45.954 INFO:tasks.workunit.client.1.smithi176.stdout:3/607: dwrite d3/d17/d83/fb0 [4194304,4194304] 0 2024-07-16T09:55:45.956 INFO:tasks.workunit.client.1.smithi176.stdout:6/686: dread d1/d5/d7/f19 [0,4194304] 0 2024-07-16T09:55:45.959 INFO:tasks.workunit.client.1.smithi176.stdout:4/713: symlink da/dc/d10/d96/da4/le6 0 2024-07-16T09:55:45.959 INFO:tasks.workunit.client.1.smithi176.stdout:3/608: creat d3/d17/fb9 x:0 0 0 2024-07-16T09:55:45.961 INFO:tasks.workunit.client.1.smithi176.stdout:9/838: dwrite d5/de/f92 [0,4194304] 0 2024-07-16T09:55:45.962 INFO:tasks.workunit.client.1.smithi176.stdout:9/839: write d5/d1d/d20/f101 [28939,14935] 0 2024-07-16T09:55:45.963 INFO:tasks.workunit.client.1.smithi176.stdout:9/840: chown d5/d1d/d20/f7d 241308452 1 2024-07-16T09:55:45.965 INFO:tasks.workunit.client.1.smithi176.stdout:6/687: symlink d1/d5/d47/d62/de8/lf9 0 2024-07-16T09:55:45.965 INFO:tasks.workunit.client.1.smithi176.stdout:4/714: creat da/dc/d10/d96/d9b/fe7 x:0 0 0 2024-07-16T09:55:45.967 INFO:tasks.workunit.client.1.smithi176.stdout:6/688: fsync d1/d5d/d9e/dd9/d6f/d7c/fa2 0 2024-07-16T09:55:45.970 INFO:tasks.workunit.client.1.smithi176.stdout:4/715: mknod da/dc/d10/d96/d9b/db4/d40/d5d/d73/ce8 0 2024-07-16T09:55:45.972 INFO:tasks.workunit.client.1.smithi176.stdout:6/689: dread d1/d5/d47/ff5 [0,4194304] 0 2024-07-16T09:55:45.973 INFO:tasks.workunit.client.1.smithi176.stdout:4/716: mknod da/d41/dc3/ce9 0 2024-07-16T09:55:45.974 INFO:tasks.workunit.client.0.smithi156.stdout:9/785: creat d2/d5/ff4 x:0 0 0 2024-07-16T09:55:45.975 INFO:tasks.workunit.client.1.smithi176.stdout:9/841: write d5/d1d/d67/f113 [318162,371] 0 2024-07-16T09:55:45.977 INFO:tasks.workunit.client.1.smithi176.stdout:3/609: dwrite d3/f61 [0,4194304] 0 2024-07-16T09:55:45.979 INFO:tasks.workunit.client.0.smithi156.stdout:9/786: mkdir d2/df5 0 2024-07-16T09:55:45.979 INFO:tasks.workunit.client.1.smithi176.stdout:9/842: rmdir d5/de/da5/dc1 39 2024-07-16T09:55:45.980 INFO:tasks.workunit.client.1.smithi176.stdout:9/843: chown d5/d43/f9e 1403853 1 2024-07-16T09:55:45.981 INFO:tasks.workunit.client.0.smithi156.stdout:9/787: symlink d2/d5/d16/de2/lf6 0 2024-07-16T09:55:45.981 INFO:tasks.workunit.client.1.smithi176.stdout:9/844: chown d5/de/d30/f10e 1630 1 2024-07-16T09:55:45.983 INFO:tasks.workunit.client.1.smithi176.stdout:9/845: dread - d5/d1d/d20/fc5 zero size 2024-07-16T09:55:45.984 INFO:tasks.workunit.client.1.smithi176.stdout:9/846: stat d5/de/f17 0 2024-07-16T09:55:45.985 INFO:tasks.workunit.client.0.smithi156.stdout:9/788: getdents d2/d5/d16/d22/d36/deb 0 2024-07-16T09:55:45.986 INFO:tasks.workunit.client.1.smithi176.stdout:9/847: symlink d5/d1d/db3/l117 0 2024-07-16T09:55:45.989 INFO:tasks.workunit.client.1.smithi176.stdout:9/848: getdents d5/d1d/dd2/de3 0 2024-07-16T09:55:45.991 INFO:tasks.workunit.client.1.smithi176.stdout:9/849: mkdir d5/d1d/d20/d78/df2/d118 0 2024-07-16T09:55:45.994 INFO:tasks.workunit.client.1.smithi176.stdout:6/690: write d1/f64 [1006361,20662] 0 2024-07-16T09:55:45.997 INFO:tasks.workunit.client.1.smithi176.stdout:6/691: symlink d1/d5d/d9e/dd9/d6f/d7c/lfa 0 2024-07-16T09:55:45.998 INFO:tasks.workunit.client.1.smithi176.stdout:9/850: write d5/de/f8b [4247895,16617] 0 2024-07-16T09:55:45.999 INFO:tasks.workunit.client.0.smithi156.stdout:9/789: dread d2/d5/d16/d57/d63/dd3/ddb/d74/dc2/dc4/f82 [0,4194304] 0 2024-07-16T09:55:46.000 INFO:tasks.workunit.client.1.smithi176.stdout:6/692: creat d1/d5d/d9e/d76/da5/ffb x:0 0 0 2024-07-16T09:55:46.002 INFO:tasks.workunit.client.1.smithi176.stdout:9/851: creat d5/d1d/d20/f119 x:0 0 0 2024-07-16T09:55:46.003 INFO:tasks.workunit.client.0.smithi156.stdout:0/767: dread d0/d22/d41/da9/fd2 [0,4194304] 0 2024-07-16T09:55:46.004 INFO:tasks.workunit.client.1.smithi176.stdout:9/852: write d5/de/d30/fe9 [604948,119208] 0 2024-07-16T09:55:46.004 INFO:tasks.workunit.client.1.smithi176.stdout:3/610: dread d3/d17/d1d/d4f/d59/da0/f9e [0,4194304] 0 2024-07-16T09:55:46.004 INFO:tasks.workunit.client.1.smithi176.stdout:6/693: rename d1/dc/d5b/l4e to d1/dc/d40/d71/lfc 0 2024-07-16T09:55:46.004 INFO:tasks.workunit.client.1.smithi176.stdout:9/853: fdatasync d5/fb8 0 2024-07-16T09:55:46.006 INFO:tasks.workunit.client.1.smithi176.stdout:6/694: creat d1/d5/d47/ffd x:0 0 0 2024-07-16T09:55:46.007 INFO:tasks.workunit.client.1.smithi176.stdout:6/695: chown d1/dc/d40/f78 0 1 2024-07-16T09:55:46.007 INFO:tasks.workunit.client.0.smithi156.stdout:0/768: getdents d0/d1a/d4d/d9/d68 0 2024-07-16T09:55:46.008 INFO:tasks.workunit.client.1.smithi176.stdout:9/854: rename d5/d1d/fee to d5/d1d/d20/d78/f11a 0 2024-07-16T09:55:46.008 INFO:tasks.workunit.client.1.smithi176.stdout:6/696: truncate d1/d5/d47/ffd 301863 0 2024-07-16T09:55:46.011 INFO:tasks.workunit.client.1.smithi176.stdout:6/697: rename d1/d5/d47/ffd to d1/d5d/d9e/d8f/ffe 0 2024-07-16T09:55:46.012 INFO:tasks.workunit.client.1.smithi176.stdout:6/698: readlink d1/d5/l8 0 2024-07-16T09:55:46.014 INFO:tasks.workunit.client.0.smithi156.stdout:0/769: dwrite d0/d22/f9e [4194304,4194304] 0 2024-07-16T09:55:46.016 INFO:tasks.workunit.client.0.smithi156.stdout:9/790: dread d2/f4d [0,4194304] 0 2024-07-16T09:55:46.018 INFO:tasks.workunit.client.0.smithi156.stdout:0/770: mknod d0/d1a/d4d/d9/d27/d2c/cf3 0 2024-07-16T09:55:46.019 INFO:tasks.workunit.client.0.smithi156.stdout:9/791: fdatasync d2/d5/d16/d57/d63/dd3/ddb/d74/dc2/dc4/d23/f60 0 2024-07-16T09:55:46.022 INFO:tasks.workunit.client.0.smithi156.stdout:0/771: mknod d0/d1a/d4d/d9/d27/d3d/d62/cf4 0 2024-07-16T09:55:46.022 INFO:tasks.workunit.client.1.smithi176.stdout:6/699: truncate d1/d5d/d9e/d76/f5c 237938 0 2024-07-16T09:55:46.024 INFO:tasks.workunit.client.0.smithi156.stdout:9/792: symlink d2/d5/d16/d57/d63/dd3/ddb/d74/dc2/dc4/lf7 0 2024-07-16T09:55:46.025 INFO:tasks.workunit.client.0.smithi156.stdout:9/793: fsync d2/d5/d16/d22/d36/fe9 0 2024-07-16T09:55:46.026 INFO:tasks.workunit.client.1.smithi176.stdout:6/700: creat d1/d5/d47/d75/fff x:0 0 0 2024-07-16T09:55:46.026 INFO:tasks.workunit.client.0.smithi156.stdout:0/772: dread d0/d1a/d4d/d25/f38 [0,4194304] 0 2024-07-16T09:55:46.027 INFO:tasks.workunit.client.0.smithi156.stdout:9/794: chown d2/d5/d16/d22/d36/l38 16152678 1 2024-07-16T09:55:46.027 INFO:tasks.workunit.client.1.smithi176.stdout:9/855: dread d5/d1d/db3/f7c [0,4194304] 0 2024-07-16T09:55:46.028 INFO:tasks.workunit.client.1.smithi176.stdout:6/701: symlink d1/d5d/d9e/dd9/l100 0 2024-07-16T09:55:46.029 INFO:tasks.workunit.client.0.smithi156.stdout:4/764: sync 2024-07-16T09:55:46.030 INFO:tasks.workunit.client.1.smithi176.stdout:9/856: creat d5/d1d/d20/f11b x:0 0 0 2024-07-16T09:55:46.031 INFO:tasks.workunit.client.0.smithi156.stdout:4/765: read d4/d23/f78 [899470,47486] 0 2024-07-16T09:55:46.031 INFO:tasks.workunit.client.0.smithi156.stdout:0/773: creat d0/d1a/d4d/d9/d27/d3d/ff5 x:0 0 0 2024-07-16T09:55:46.031 INFO:tasks.workunit.client.1.smithi176.stdout:6/702: chown d1/d5d/d9e/dd9/d6f/d7c/ced 580942 1 2024-07-16T09:55:46.033 INFO:tasks.workunit.client.0.smithi156.stdout:0/774: write d0/d22/d41/d52/ddb/fe3 [2157442,78908] 0 2024-07-16T09:55:46.033 INFO:tasks.workunit.client.1.smithi176.stdout:6/703: mkdir d1/d5d/db9/d101 0 2024-07-16T09:55:46.034 INFO:tasks.workunit.client.0.smithi156.stdout:9/795: rmdir d2/d5/d16/d57/d63/dd3/ddb/d3d/da6 39 2024-07-16T09:55:46.036 INFO:tasks.workunit.client.1.smithi176.stdout:9/857: dread d5/d1d/d20/d78/dd0/fd7 [0,4194304] 0 2024-07-16T09:55:46.037 INFO:tasks.workunit.client.1.smithi176.stdout:6/704: getdents d1/d5d/d9e/dd9/d6f 0 2024-07-16T09:55:46.037 INFO:tasks.workunit.client.0.smithi156.stdout:0/775: dread d0/d1a/d4d/d25/f38 [0,4194304] 0 2024-07-16T09:55:46.037 INFO:tasks.workunit.client.1.smithi176.stdout:9/858: fdatasync d5/de/d30/f9a 0 2024-07-16T09:55:46.038 INFO:tasks.workunit.client.1.smithi176.stdout:6/705: chown d1/d5/d47/ff5 0 1 2024-07-16T09:55:46.039 INFO:tasks.workunit.client.1.smithi176.stdout:6/706: fsync d1/d5/d47/f51 0 2024-07-16T09:55:46.040 INFO:tasks.workunit.client.1.smithi176.stdout:9/859: mkdir d5/d1d/d20/d78/df2/d118/d11c 0 2024-07-16T09:55:46.042 INFO:tasks.workunit.client.1.smithi176.stdout:9/860: rename d5/de/da5/dc1/d112/d97 to d5/de/da5/dc1/d112/d97/d11d 22 2024-07-16T09:55:46.043 INFO:tasks.workunit.client.0.smithi156.stdout:9/796: link d2/d5/d16/d57/d63/dd3/ddb/d74/dc2/dc4/cc3 d2/d5/d61/da2/cf8 0 2024-07-16T09:55:46.044 INFO:tasks.workunit.client.0.smithi156.stdout:4/766: truncate d4/db/dc/d5f/f5c 866194 0 2024-07-16T09:55:46.046 INFO:tasks.workunit.client.0.smithi156.stdout:9/797: dread - d2/d5/d16/d57/d63/dd3/ddb/d3d/fda zero size 2024-07-16T09:55:46.046 INFO:tasks.workunit.client.0.smithi156.stdout:0/776: write d0/d1a/d4d/d9/f26 [4482706,44851] 0 2024-07-16T09:55:46.049 INFO:tasks.workunit.client.0.smithi156.stdout:9/798: chown d2/d5/d16/d57/d63/dd3/ddb/d3d/f68 27 1 2024-07-16T09:55:46.051 INFO:tasks.workunit.client.1.smithi176.stdout:6/707: dwrite d1/d5/d7/d54/f55 [0,4194304] 0 2024-07-16T09:55:46.053 INFO:tasks.workunit.client.0.smithi156.stdout:4/767: getdents d4/db/dc/d67 0 2024-07-16T09:55:46.054 INFO:tasks.workunit.client.1.smithi176.stdout:6/708: truncate d1/d5/d47/f6c 1463299 0 2024-07-16T09:55:46.056 INFO:tasks.workunit.client.0.smithi156.stdout:9/799: symlink d2/d5/d16/d22/d6b/lf9 0 2024-07-16T09:55:46.057 INFO:tasks.workunit.client.0.smithi156.stdout:0/777: truncate d0/d1a/d4d/d25/f2a 1009575 0 2024-07-16T09:55:46.059 INFO:tasks.workunit.client.1.smithi176.stdout:6/709: getdents d1/d5/d47/d62/d77/dd2 0 2024-07-16T09:55:46.061 INFO:tasks.workunit.client.0.smithi156.stdout:0/778: creat d0/d1a/d4d/d9/d27/d2c/d6d/ff6 x:0 0 0 2024-07-16T09:55:46.061 INFO:tasks.workunit.client.0.smithi156.stdout:4/768: dwrite d4/db/dc/dcc/d58/f91 [0,4194304] 0 2024-07-16T09:55:46.061 INFO:tasks.workunit.client.1.smithi176.stdout:6/710: truncate d1/dc/d40/f8b 98038 0 2024-07-16T09:55:46.065 INFO:tasks.workunit.client.0.smithi156.stdout:0/779: creat d0/d1a/d4d/d6e/dde/ff7 x:0 0 0 2024-07-16T09:55:46.065 INFO:tasks.workunit.client.1.smithi176.stdout:6/711: unlink d1/d5/d7/l21 0 2024-07-16T09:55:46.067 INFO:tasks.workunit.client.1.smithi176.stdout:6/712: fdatasync d1/d5/d47/d62/d80/fe5 0 2024-07-16T09:55:46.079 INFO:tasks.workunit.client.1.smithi176.stdout:6/713: write d1/f3b [2241002,47815] 0 2024-07-16T09:55:46.082 INFO:tasks.workunit.client.1.smithi176.stdout:6/714: readlink d1/l3 0 2024-07-16T09:55:46.084 INFO:tasks.workunit.client.1.smithi176.stdout:6/715: symlink d1/db3/l102 0 2024-07-16T09:55:46.086 INFO:tasks.workunit.client.1.smithi176.stdout:6/716: unlink d1/d5d/d9e/d76/f5a 0 2024-07-16T09:55:46.087 INFO:tasks.workunit.client.0.smithi156.stdout:0/780: dread d0/d22/d41/f47 [4194304,4194304] 0 2024-07-16T09:55:46.088 INFO:tasks.workunit.client.1.smithi176.stdout:6/717: unlink d1/d5/d7/c23 0 2024-07-16T09:55:46.090 INFO:tasks.workunit.client.1.smithi176.stdout:6/718: chown d1/d5d/d9e/dd9/c7e 24 1 2024-07-16T09:55:46.092 INFO:tasks.workunit.client.1.smithi176.stdout:6/719: truncate d1/d5/d47/fad 280563 0 2024-07-16T09:55:46.094 INFO:tasks.workunit.client.1.smithi176.stdout:6/720: creat d1/d5d/d9e/d76/f103 x:0 0 0 2024-07-16T09:55:46.097 INFO:tasks.workunit.client.0.smithi156.stdout:0/781: dwrite d0/d1a/d4d/d9/f15 [0,4194304] 0 2024-07-16T09:55:46.097 INFO:tasks.workunit.client.0.smithi156.stdout:4/769: dread d4/db/dc/f70 [0,4194304] 0 2024-07-16T09:55:46.101 INFO:tasks.workunit.client.0.smithi156.stdout:0/782: rename d0/d1a/d4d/d9/d27/d50/c49 to d0/d22/d41/d66/d7d/cf8 0 2024-07-16T09:55:46.103 INFO:tasks.workunit.client.0.smithi156.stdout:0/783: readlink d0/d1a/d4d/l24 0 2024-07-16T09:55:46.105 INFO:tasks.workunit.client.0.smithi156.stdout:4/770: dwrite d4/db/d47/dbe/f105 [0,4194304] 0 2024-07-16T09:55:46.106 INFO:tasks.workunit.client.0.smithi156.stdout:0/784: creat d0/d1a/d4d/d9/d27/d8d/ff9 x:0 0 0 2024-07-16T09:55:46.108 INFO:tasks.workunit.client.0.smithi156.stdout:4/771: truncate d4/faa 95964 0 2024-07-16T09:55:46.108 INFO:tasks.workunit.client.0.smithi156.stdout:0/785: write d0/d1a/d76/f87 [1664699,102696] 0 2024-07-16T09:55:46.110 INFO:tasks.workunit.client.0.smithi156.stdout:4/772: stat d4/l1c 0 2024-07-16T09:55:46.113 INFO:tasks.workunit.client.0.smithi156.stdout:4/773: rename d4/d23/d52/d71/fa7 to d4/d23/d52/d71/def/f110 0 2024-07-16T09:55:46.113 INFO:tasks.workunit.client.0.smithi156.stdout:0/786: dwrite d0/d1a/d4d/d60/f93 [0,4194304] 0 2024-07-16T09:55:46.115 INFO:tasks.workunit.client.0.smithi156.stdout:0/787: write d0/d1a/d4d/d9/fc7 [416068,43443] 0 2024-07-16T09:55:46.116 INFO:tasks.workunit.client.1.smithi176.stdout:6/721: dread d1/d5d/db9/f85 [0,4194304] 0 2024-07-16T09:55:46.117 INFO:tasks.workunit.client.1.smithi176.stdout:6/722: write d1/dc/d40/d71/ff0 [108684,48997] 0 2024-07-16T09:55:46.118 INFO:tasks.workunit.client.1.smithi176.stdout:6/723: stat d1/dc/f97 0 2024-07-16T09:55:46.121 INFO:tasks.workunit.client.0.smithi156.stdout:4/774: creat d4/d23/d52/d71/def/f111 x:0 0 0 2024-07-16T09:55:46.123 INFO:tasks.workunit.client.1.smithi176.stdout:6/724: creat d1/f104 x:0 0 0 2024-07-16T09:55:46.125 INFO:tasks.workunit.client.1.smithi176.stdout:6/725: mkdir d1/d5d/db9/d101/d105 0 2024-07-16T09:55:46.139 INFO:tasks.workunit.client.0.smithi156.stdout:0/788: truncate d0/d22/d41/d52/ddb/fe3 1622271 0 2024-07-16T09:55:46.143 INFO:tasks.workunit.client.0.smithi156.stdout:0/789: link d0/d1a/d4d/d9/d27/d2c/d6d/f8b d0/d1a/d4d/d60/ffa 0 2024-07-16T09:55:46.143 INFO:tasks.workunit.client.0.smithi156.stdout:0/790: stat d0/d1a/d76/d83/fcd 0 2024-07-16T09:55:46.144 INFO:tasks.workunit.client.0.smithi156.stdout:0/791: write d0/d1a/d4d/d9/d27/f81 [392347,19324] 0 2024-07-16T09:55:46.145 INFO:tasks.workunit.client.1.smithi176.stdout:6/726: dwrite d1/dc/f9f [0,4194304] 0 2024-07-16T09:55:46.147 INFO:tasks.workunit.client.0.smithi156.stdout:0/792: write d0/d1a/d4d/d9/d27/d3d/d62/f67 [3299066,58805] 0 2024-07-16T09:55:46.148 INFO:tasks.workunit.client.0.smithi156.stdout:0/793: fdatasync d0/d22/d41/f8a 0 2024-07-16T09:55:46.148 INFO:tasks.workunit.client.1.smithi176.stdout:6/727: rename d1/d5/d47/d62/c87 to d1/d5d/d9e/c106 0 2024-07-16T09:55:46.150 INFO:tasks.workunit.client.1.smithi176.stdout:6/728: dread - d1/d5d/d9e/dd9/d6f/d7c/fa2 zero size 2024-07-16T09:55:46.153 INFO:tasks.workunit.client.0.smithi156.stdout:0/794: truncate d0/d1a/d4d/d9/d27/d50/fae 93558 0 2024-07-16T09:55:46.153 INFO:tasks.workunit.client.1.smithi176.stdout:6/729: mknod d1/d5/d7/c107 0 2024-07-16T09:55:46.154 INFO:tasks.workunit.client.0.smithi156.stdout:0/795: readlink d0/d1a/d4d/d25/l65 0 2024-07-16T09:55:46.154 INFO:tasks.workunit.client.1.smithi176.stdout:6/730: stat d1/d5d/db9/f85 0 2024-07-16T09:55:46.155 INFO:tasks.workunit.client.1.smithi176.stdout:6/731: symlink d1/dc/d40/l108 0 2024-07-16T09:55:46.155 INFO:tasks.workunit.client.1.smithi176.stdout:6/732: readlink d1/dc/l3d 0 2024-07-16T09:55:46.158 INFO:tasks.workunit.client.0.smithi156.stdout:0/796: rename d0/d1a/d4d/d61/d97/fab to d0/d1a/d4d/d9/d27/d2c/d6d/d9d/ffb 0 2024-07-16T09:55:46.158 INFO:tasks.workunit.client.1.smithi176.stdout:6/733: creat d1/d5/d7/f109 x:0 0 0 2024-07-16T09:55:46.159 INFO:tasks.workunit.client.0.smithi156.stdout:0/797: dread - d0/d1a/d76/da7/dd3/fd5 zero size 2024-07-16T09:55:46.161 INFO:tasks.workunit.client.1.smithi176.stdout:6/734: rmdir d1/d5d/d9e/dd9/d6f/d7c 39 2024-07-16T09:55:46.162 INFO:tasks.workunit.client.1.smithi176.stdout:6/735: fdatasync d1/d5/d7/d54/f70 0 2024-07-16T09:55:46.164 INFO:tasks.workunit.client.1.smithi176.stdout:6/736: mkdir d1/dc/d81/d10a 0 2024-07-16T09:55:46.166 INFO:tasks.workunit.client.1.smithi176.stdout:6/737: creat d1/d5/d7/d54/f10b x:0 0 0 2024-07-16T09:55:46.166 INFO:tasks.workunit.client.0.smithi156.stdout:0/798: write d0/d1a/d4d/d9/d27/d50/f9a [273088,43719] 0 2024-07-16T09:55:46.167 INFO:tasks.workunit.client.1.smithi176.stdout:6/738: rmdir d1/d5d/d9e/dd9 39 2024-07-16T09:55:46.169 INFO:tasks.workunit.client.0.smithi156.stdout:0/799: mkdir d0/d22/d41/dfc 0 2024-07-16T09:55:46.170 INFO:tasks.workunit.client.1.smithi176.stdout:2/841: sync 2024-07-16T09:55:46.172 INFO:tasks.workunit.client.0.smithi156.stdout:0/800: truncate d0/d1a/d4d/d25/f55 2581560 0 2024-07-16T09:55:46.172 INFO:tasks.workunit.client.1.smithi176.stdout:2/842: readlink dd/d36/d82/ld7 0 2024-07-16T09:55:46.173 INFO:tasks.workunit.client.1.smithi176.stdout:6/739: truncate d1/d5d/db9/fb6 1745855 0 2024-07-16T09:55:46.173 INFO:tasks.workunit.client.0.smithi156.stdout:0/801: chown d0/d1a/d4d/d9/d27/d2c/d6d/ff6 287650 1 2024-07-16T09:55:46.174 INFO:tasks.workunit.client.1.smithi176.stdout:2/843: mknod dd/d16/c115 0 2024-07-16T09:55:46.175 INFO:tasks.workunit.client.1.smithi176.stdout:2/844: write dd/d2e/d35/d4a/df9/f106 [893031,1110] 0 2024-07-16T09:55:46.175 INFO:tasks.workunit.client.0.smithi156.stdout:0/802: truncate d0/d1a/d4d/d9/d27/d2c/d6d/ff6 150280 0 2024-07-16T09:55:46.177 INFO:tasks.workunit.client.0.smithi156.stdout:0/803: write d0/d1a/d4d/d9/d27/d2c/d6d/fec [760949,126355] 0 2024-07-16T09:55:46.179 INFO:tasks.workunit.client.0.smithi156.stdout:0/804: mkdir d0/d1a/da3/dfd 0 2024-07-16T09:55:46.179 INFO:tasks.workunit.client.0.smithi156.stdout:1/879: sync 2024-07-16T09:55:46.180 INFO:tasks.workunit.client.0.smithi156.stdout:6/656: sync 2024-07-16T09:55:46.181 INFO:tasks.workunit.client.0.smithi156.stdout:0/805: unlink d0/d22/d41/f8a 0 2024-07-16T09:55:46.182 INFO:tasks.workunit.client.1.smithi176.stdout:6/740: dwrite d1/dc/fd [0,4194304] 0 2024-07-16T09:55:46.183 INFO:tasks.workunit.client.0.smithi156.stdout:1/880: dread dc/d24/f7e [0,4194304] 0 2024-07-16T09:55:46.184 INFO:tasks.workunit.client.1.smithi176.stdout:6/741: stat d1/dc/d40/d71/cef 0 2024-07-16T09:55:46.186 INFO:tasks.workunit.client.0.smithi156.stdout:1/881: chown dc/d14/d1e/f21 2003741 1 2024-07-16T09:55:46.187 INFO:tasks.workunit.client.0.smithi156.stdout:1/882: stat dc/d14/d1e/f21 0 2024-07-16T09:55:46.193 INFO:tasks.workunit.client.0.smithi156.stdout:1/883: link dc/d14/d3e/d6c/d7a/cbd dc/d14/d8b/db4/d111/c125 0 2024-07-16T09:55:46.196 INFO:tasks.workunit.client.0.smithi156.stdout:1/884: symlink dc/d14/d8b/l126 0 2024-07-16T09:55:46.196 INFO:tasks.workunit.client.0.smithi156.stdout:0/806: dread d0/d22/d41/f47 [0,4194304] 0 2024-07-16T09:55:46.198 INFO:tasks.workunit.client.1.smithi176.stdout:6/742: fdatasync d1/dc/fd 0 2024-07-16T09:55:46.199 INFO:tasks.workunit.client.0.smithi156.stdout:0/807: rename d0/d1a/da3/dfd to d0/d1a/d4d/d61/d97/dfe 0 2024-07-16T09:55:46.200 INFO:tasks.workunit.client.1.smithi176.stdout:6/743: rmdir d1/db3 39 2024-07-16T09:55:46.200 INFO:tasks.workunit.client.0.smithi156.stdout:0/808: write d0/d1a/d76/f87 [3130812,1557] 0 2024-07-16T09:55:46.201 INFO:tasks.workunit.client.1.smithi176.stdout:6/744: creat d1/d5d/d9e/d76/da5/f10c x:0 0 0 2024-07-16T09:55:46.203 INFO:tasks.workunit.client.0.smithi156.stdout:1/885: truncate f6 2909112 0 2024-07-16T09:55:46.203 INFO:tasks.workunit.client.1.smithi176.stdout:6/745: mkdir d1/d5d/d9e/d76/d10d 0 2024-07-16T09:55:46.205 INFO:tasks.workunit.client.1.smithi176.stdout:6/746: write d1/d5/d47/d62/d77/fcd [123406,78841] 0 2024-07-16T09:55:46.210 INFO:tasks.workunit.client.1.smithi176.stdout:6/747: creat d1/d5/d47/d62/de8/f10e x:0 0 0 2024-07-16T09:55:46.211 INFO:tasks.workunit.client.1.smithi176.stdout:6/748: creat d1/d61/f10f x:0 0 0 2024-07-16T09:55:46.211 INFO:tasks.workunit.client.0.smithi156.stdout:0/809: rename d0/d1a/d4d/d9/d27/d8d to d0/d1a/d4d/d9/d68/dff 0 2024-07-16T09:55:46.211 INFO:tasks.workunit.client.0.smithi156.stdout:1/886: readlink dc/lb8 0 2024-07-16T09:55:46.211 INFO:tasks.workunit.client.0.smithi156.stdout:0/810: fdatasync d0/d1a/d4d/d5e/db7/fe5 0 2024-07-16T09:55:46.212 INFO:tasks.workunit.client.0.smithi156.stdout:0/811: fdatasync d0/d22/d41/da9/ff1 0 2024-07-16T09:55:46.214 INFO:tasks.workunit.client.1.smithi176.stdout:6/749: rename d1/d5d/d9e/dd9/d6f/d7c/ff6 to d1/d5d/d9e/f110 0 2024-07-16T09:55:46.217 INFO:tasks.workunit.client.1.smithi176.stdout:6/750: mkdir d1/d5/d47/d62/de8/d111 0 2024-07-16T09:55:46.217 INFO:tasks.workunit.client.0.smithi156.stdout:1/887: truncate dc/d14/d3e/d55/d83/fa1 3158486 0 2024-07-16T09:55:46.223 INFO:tasks.workunit.client.0.smithi156.stdout:1/888: dwrite dc/d14/d1e/f35 [0,4194304] 0 2024-07-16T09:55:46.225 INFO:tasks.workunit.client.0.smithi156.stdout:1/889: creat dc/d14/d3e/d55/d83/f127 x:0 0 0 2024-07-16T09:55:46.226 INFO:tasks.workunit.client.0.smithi156.stdout:1/890: fdatasync dc/d6b/d8a/f10e 0 2024-07-16T09:55:46.227 INFO:tasks.workunit.client.0.smithi156.stdout:0/812: rename d0/d1a/d4d/d9/d27/d2c/d6d/de4 to d0/d1a/d4d/d9/d68/dcb/dd7/d100 0 2024-07-16T09:55:46.228 INFO:tasks.workunit.client.0.smithi156.stdout:0/813: readlink d0/d1a/d4d/d9/d27/l7f 0 2024-07-16T09:55:46.232 INFO:tasks.workunit.client.1.smithi176.stdout:6/751: dread d1/d5/d7/f6e [0,4194304] 0 2024-07-16T09:55:46.233 INFO:tasks.workunit.client.1.smithi176.stdout:6/752: fsync d1/dc/d40/fca 0 2024-07-16T09:55:46.234 INFO:tasks.workunit.client.1.smithi176.stdout:6/753: fdatasync d1/d5d/d9e/d76/da5/fb7 0 2024-07-16T09:55:46.236 INFO:tasks.workunit.client.1.smithi176.stdout:6/754: creat d1/d5/d47/d62/d77/f112 x:0 0 0 2024-07-16T09:55:46.238 INFO:tasks.workunit.client.1.smithi176.stdout:6/755: creat d1/d5/d7/d54/f113 x:0 0 0 2024-07-16T09:55:46.238 INFO:tasks.workunit.client.1.smithi176.stdout:6/756: readlink d1/d5/d47/d75/lc8 0 2024-07-16T09:55:46.240 INFO:tasks.workunit.client.1.smithi176.stdout:6/757: mkdir d1/d5/d47/d62/de8/d114 0 2024-07-16T09:55:46.241 INFO:tasks.workunit.client.1.smithi176.stdout:6/758: mkdir d1/dc/d81/d10a/d115 0 2024-07-16T09:55:46.242 INFO:tasks.workunit.client.0.smithi156.stdout:0/814: dread d0/d1a/d4d/d9/d27/d50/fbe [0,4194304] 0 2024-07-16T09:55:46.243 INFO:tasks.workunit.client.1.smithi176.stdout:6/759: creat d1/d5d/f116 x:0 0 0 2024-07-16T09:55:46.244 INFO:tasks.workunit.client.0.smithi156.stdout:0/815: mkdir d0/d22/d41/d52/ddb/d101 0 2024-07-16T09:55:46.245 INFO:tasks.workunit.client.0.smithi156.stdout:0/816: write d0/d1a/d76/d83/fcd [502496,14336] 0 2024-07-16T09:55:46.251 INFO:tasks.workunit.client.0.smithi156.stdout:0/817: dwrite d0/d22/d41/d52/f74 [0,4194304] 0 2024-07-16T09:55:46.254 INFO:tasks.workunit.client.0.smithi156.stdout:0/818: write d0/d1a/d4d/d9/d27/d2c/d6d/ff6 [57058,82092] 0 2024-07-16T09:55:46.255 INFO:tasks.workunit.client.0.smithi156.stdout:0/819: read - d0/d1a/d4d/d9/d27/d50/d90/fd8 zero size 2024-07-16T09:55:46.258 INFO:tasks.workunit.client.1.smithi176.stdout:6/760: dread d1/d5d/d9e/dd9/d6f/d7c/f90 [0,4194304] 0 2024-07-16T09:55:46.260 INFO:tasks.workunit.client.0.smithi156.stdout:0/820: dwrite d0/d1a/d76/da7/dd3/fd5 [0,4194304] 0 2024-07-16T09:55:46.264 INFO:tasks.workunit.client.0.smithi156.stdout:0/821: creat d0/d1a/d4d/d9/d27/d50/d90/f102 x:0 0 0 2024-07-16T09:55:46.265 INFO:tasks.workunit.client.1.smithi176.stdout:6/761: dread d1/dc/d40/f4d [4194304,4194304] 0 2024-07-16T09:55:46.268 INFO:tasks.workunit.client.1.smithi176.stdout:6/762: symlink d1/d5/d47/d62/de8/d114/l117 0 2024-07-16T09:55:46.268 INFO:tasks.workunit.client.0.smithi156.stdout:0/822: write d0/d1a/d4d/d25/f38 [8750445,69598] 0 2024-07-16T09:55:46.270 INFO:tasks.workunit.client.1.smithi176.stdout:6/763: read d1/d5d/d9e/f82 [1795228,35606] 0 2024-07-16T09:55:46.272 INFO:tasks.workunit.client.1.smithi176.stdout:6/764: symlink d1/d5/d47/d62/l118 0 2024-07-16T09:55:46.273 INFO:tasks.workunit.client.1.smithi176.stdout:6/765: readlink d1/d5/l11 0 2024-07-16T09:55:46.273 INFO:tasks.workunit.client.1.smithi176.stdout:6/766: chown d1/d5d/db9/l43 8314 1 2024-07-16T09:55:46.274 INFO:tasks.workunit.client.1.smithi176.stdout:6/767: symlink d1/d5d/db9/l119 0 2024-07-16T09:55:46.276 INFO:tasks.workunit.client.1.smithi176.stdout:6/768: rmdir d1/d5/d7/d1e 39 2024-07-16T09:55:46.276 INFO:tasks.workunit.client.0.smithi156.stdout:0/823: write d0/d22/d41/d52/f5d [3353719,14292] 0 2024-07-16T09:55:46.278 INFO:tasks.workunit.client.0.smithi156.stdout:0/824: mknod d0/d22/d41/d66/c103 0 2024-07-16T09:55:46.279 INFO:tasks.workunit.client.1.smithi176.stdout:6/769: rmdir d1/d5d/d9e/d76/d10d 0 2024-07-16T09:55:46.280 INFO:tasks.workunit.client.0.smithi156.stdout:0/825: write d0/d1a/d4d/d9/d27/d50/f5f [1213189,45077] 0 2024-07-16T09:55:46.280 INFO:tasks.workunit.client.1.smithi176.stdout:6/770: write d1/d5d/d9e/faa [540452,60216] 0 2024-07-16T09:55:46.282 INFO:tasks.workunit.client.1.smithi176.stdout:6/771: rmdir d1/d61 39 2024-07-16T09:55:46.283 INFO:tasks.workunit.client.0.smithi156.stdout:0/826: symlink d0/d1a/d4d/d9/d27/d2c/d6d/d9d/l104 0 2024-07-16T09:55:46.285 INFO:tasks.workunit.client.0.smithi156.stdout:0/827: mknod d0/d1a/d4d/c105 0 2024-07-16T09:55:46.293 INFO:tasks.workunit.client.1.smithi176.stdout:8/720: sync 2024-07-16T09:55:46.294 INFO:tasks.workunit.client.1.smithi176.stdout:6/772: dread d1/dc/f33 [0,4194304] 0 2024-07-16T09:55:46.295 INFO:tasks.workunit.client.1.smithi176.stdout:6/773: dread d1/dc/d40/d71/ff0 [0,4194304] 0 2024-07-16T09:55:46.295 INFO:tasks.workunit.client.1.smithi176.stdout:8/721: creat d8/d14/d1e/d7c/d82/df5/ff9 x:0 0 0 2024-07-16T09:55:46.307 INFO:tasks.workunit.client.0.smithi156.stdout:8/640: sync 2024-07-16T09:55:46.307 INFO:tasks.workunit.client.0.smithi156.stdout:2/839: sync 2024-07-16T09:55:46.307 INFO:tasks.workunit.client.0.smithi156.stdout:5/695: sync 2024-07-16T09:55:46.308 INFO:tasks.workunit.client.0.smithi156.stdout:3/704: sync 2024-07-16T09:55:46.308 INFO:tasks.workunit.client.0.smithi156.stdout:7/566: sync 2024-07-16T09:55:46.310 INFO:tasks.workunit.client.0.smithi156.stdout:7/567: symlink dc/d13/d20/d4d/lc0 0 2024-07-16T09:55:46.310 INFO:tasks.workunit.client.0.smithi156.stdout:5/696: read d1b/d1f/d45/d52/d43/f5b [937848,112462] 0 2024-07-16T09:55:46.311 INFO:tasks.workunit.client.0.smithi156.stdout:3/705: rename d8/da/d1d/d90 to d8/da/dec/df1 0 2024-07-16T09:55:46.312 INFO:tasks.workunit.client.0.smithi156.stdout:2/840: creat d0/d5/d76/d10f/f110 x:0 0 0 2024-07-16T09:55:46.314 INFO:tasks.workunit.client.0.smithi156.stdout:7/568: chown dc/d3e/d3f/d45/l49 216 1 2024-07-16T09:55:46.316 INFO:tasks.workunit.client.0.smithi156.stdout:3/706: mknod d8/da/d28/d43/cf2 0 2024-07-16T09:55:46.317 INFO:tasks.workunit.client.0.smithi156.stdout:5/697: truncate d1b/d1f/d45/d52/d49/f53 1018220 0 2024-07-16T09:55:46.318 INFO:tasks.workunit.client.0.smithi156.stdout:8/641: dwrite d0/d11/d13/d1c/d24/fb4 [0,4194304] 0 2024-07-16T09:55:46.320 INFO:tasks.workunit.client.0.smithi156.stdout:3/707: unlink d8/da/d1d/d2c/f52 0 2024-07-16T09:55:46.320 INFO:tasks.workunit.client.0.smithi156.stdout:8/642: write d0/dd/f30 [607456,7548] 0 2024-07-16T09:55:46.322 INFO:tasks.workunit.client.0.smithi156.stdout:3/708: fdatasync d8/f62 0 2024-07-16T09:55:46.325 INFO:tasks.workunit.client.0.smithi156.stdout:2/841: dwrite d0/d5/d29/f56 [0,4194304] 0 2024-07-16T09:55:46.326 INFO:tasks.workunit.client.0.smithi156.stdout:8/643: creat d0/d11/d13/d1c/d39/fe4 x:0 0 0 2024-07-16T09:55:46.331 INFO:tasks.workunit.client.0.smithi156.stdout:8/644: mknod d0/d11/d13/d1c/ce5 0 2024-07-16T09:55:46.333 INFO:tasks.workunit.client.0.smithi156.stdout:7/569: dread dc/d13/d20/f2b [0,4194304] 0 2024-07-16T09:55:46.334 INFO:tasks.workunit.client.0.smithi156.stdout:8/645: symlink d0/d11/d20/d3f/dbe/le6 0 2024-07-16T09:55:46.334 INFO:tasks.workunit.client.0.smithi156.stdout:2/842: dread d0/d13/d17/d1c/d51/fc2 [0,4194304] 0 2024-07-16T09:55:46.337 INFO:tasks.workunit.client.0.smithi156.stdout:7/570: creat dc/d3e/fc1 x:0 0 0 2024-07-16T09:55:46.337 INFO:tasks.workunit.client.0.smithi156.stdout:8/646: read d0/dd/f17 [2414029,103208] 0 2024-07-16T09:55:46.338 INFO:tasks.workunit.client.0.smithi156.stdout:8/647: stat d0/d11/d29/c5f 0 2024-07-16T09:55:46.340 INFO:tasks.workunit.client.0.smithi156.stdout:2/843: link d0/d5/d29/fd9 d0/d5/d29/f111 0 2024-07-16T09:55:46.342 INFO:tasks.workunit.client.0.smithi156.stdout:2/844: mknod d0/d5/d8b/d101/c112 0 2024-07-16T09:55:46.343 INFO:tasks.workunit.client.0.smithi156.stdout:5/698: truncate d1b/d1f/d42/d6d/d9f/fea 4184936 0 2024-07-16T09:55:46.343 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:55:45 smithi176 ceph-mon[36626]: pgmap v254: 65 pgs: 65 active+clean; 2.7 GiB data, 9.3 GiB used, 527 GiB / 536 GiB avail; 97 MiB/s rd, 327 MiB/s wr, 608 op/s 2024-07-16T09:55:46.344 INFO:tasks.workunit.client.0.smithi156.stdout:7/571: rename dc/d3e/d3f/c55 to dc/d3e/d3f/d51/cc2 0 2024-07-16T09:55:46.344 INFO:tasks.workunit.client.0.smithi156.stdout:2/845: symlink d0/d13/d17/d1c/d51/dc6/l113 0 2024-07-16T09:55:46.347 INFO:tasks.workunit.client.0.smithi156.stdout:8/648: rename d0/d11/d4a/d55 to d0/d11/d29/d80/d97/de7 0 2024-07-16T09:55:46.349 INFO:tasks.workunit.client.0.smithi156.stdout:2/846: symlink d0/d5/d29/d2d/d5b/d5f/d89/l114 0 2024-07-16T09:55:46.349 INFO:tasks.workunit.client.0.smithi156.stdout:8/649: write d0/dd/d1f/d60/f98 [1271602,41374] 0 2024-07-16T09:55:46.350 INFO:tasks.workunit.client.0.smithi156.stdout:7/572: dread dc/f42 [4194304,4194304] 0 2024-07-16T09:55:46.351 INFO:tasks.workunit.client.0.smithi156.stdout:5/699: dwrite d1b/d1f/d45/d7f/fd1 [0,4194304] 0 2024-07-16T09:55:46.356 INFO:tasks.workunit.client.0.smithi156.stdout:7/573: symlink dc/d13/d15/d8d/lc3 0 2024-07-16T09:55:46.357 INFO:tasks.workunit.client.0.smithi156.stdout:5/700: read d1b/d1f/d54/f74 [605235,75496] 0 2024-07-16T09:55:46.357 INFO:tasks.workunit.client.0.smithi156.stdout:8/650: dread d0/d11/fc7 [0,4194304] 0 2024-07-16T09:55:46.358 INFO:tasks.workunit.client.0.smithi156.stdout:5/701: chown d1b/d1f/d42/d6d/d9f/fda 4571441 1 2024-07-16T09:55:46.359 INFO:tasks.workunit.client.0.smithi156.stdout:5/702: write d1b/f31 [4751417,114150] 0 2024-07-16T09:55:46.360 INFO:tasks.workunit.client.0.smithi156.stdout:5/703: readlink l16 0 2024-07-16T09:55:46.363 INFO:tasks.workunit.client.0.smithi156.stdout:8/651: link d0/d74/f4e d0/dd/d1f/d57/dc0/fe8 0 2024-07-16T09:55:46.367 INFO:tasks.workunit.client.0.smithi156.stdout:2/847: dwrite d0/d5/d29/d2d/d5b/d5f/d89/f99 [0,4194304] 0 2024-07-16T09:55:46.369 INFO:tasks.workunit.client.0.smithi156.stdout:8/652: creat d0/d11/d20/d3f/fe9 x:0 0 0 2024-07-16T09:55:46.370 INFO:tasks.workunit.client.0.smithi156.stdout:5/704: dwrite d1b/d1f/d45/d52/d43/d85/f75 [0,4194304] 0 2024-07-16T09:55:46.374 INFO:tasks.workunit.client.0.smithi156.stdout:8/653: symlink d0/d11/d13/d1c/d39/d8a/dd7/lea 0 2024-07-16T09:55:46.376 INFO:tasks.workunit.client.0.smithi156.stdout:7/574: dread dc/d13/d20/f3b [0,4194304] 0 2024-07-16T09:55:46.380 INFO:tasks.workunit.client.0.smithi156.stdout:7/575: symlink dc/d13/d36/d56/lc4 0 2024-07-16T09:55:46.380 INFO:tasks.workunit.client.0.smithi156.stdout:8/654: dread d0/dd/f58 [0,4194304] 0 2024-07-16T09:55:46.382 INFO:tasks.workunit.client.0.smithi156.stdout:2/848: truncate d0/d13/d17/d1c/d86/fb7 3608814 0 2024-07-16T09:55:46.383 INFO:tasks.workunit.client.0.smithi156.stdout:8/655: mknod d0/d11/d29/d80/ceb 0 2024-07-16T09:55:46.383 INFO:tasks.workunit.client.0.smithi156.stdout:2/849: fdatasync d0/d5/d29/d2d/de1/fe2 0 2024-07-16T09:55:46.384 INFO:tasks.workunit.client.0.smithi156.stdout:8/656: fsync d0/dd/d1f/f6b 0 2024-07-16T09:55:46.386 INFO:tasks.workunit.client.0.smithi156.stdout:8/657: truncate d0/d11/d2e/d35/fe2 112135 0 2024-07-16T09:55:46.386 INFO:tasks.workunit.client.0.smithi156.stdout:2/850: creat d0/d5/d29/d2d/d5b/ddb/f115 x:0 0 0 2024-07-16T09:55:46.391 INFO:tasks.workunit.client.0.smithi156.stdout:8/658: rename d0/d11/d13/f14 to d0/d11/d62/dc2/dcd/fec 0 2024-07-16T09:55:46.392 INFO:tasks.workunit.client.0.smithi156.stdout:2/851: getdents d0 0 2024-07-16T09:55:46.396 INFO:tasks.workunit.client.0.smithi156.stdout:5/705: dread d1b/d1f/d42/f47 [4194304,4194304] 0 2024-07-16T09:55:46.397 INFO:tasks.workunit.client.0.smithi156.stdout:5/706: chown d1b/d1f/d45/d52/ld5 0 1 2024-07-16T09:55:46.398 INFO:tasks.workunit.client.0.smithi156.stdout:5/707: write d1b/f34 [1975737,61853] 0 2024-07-16T09:55:46.398 INFO:tasks.workunit.client.0.smithi156.stdout:8/659: rename d0/d11/d13/d1c/d39/d8a to d0/d11/ded 0 2024-07-16T09:55:46.401 INFO:tasks.workunit.client.0.smithi156.stdout:5/708: mknod d1b/d1f/d45/d52/cec 0 2024-07-16T09:55:46.401 INFO:tasks.workunit.client.0.smithi156.stdout:8/660: mkdir d0/dd/d1f/dee 0 2024-07-16T09:55:46.402 INFO:tasks.workunit.client.0.smithi156.stdout:7/576: dread dc/d13/d36/f6a [0,4194304] 0 2024-07-16T09:55:46.404 INFO:tasks.workunit.client.0.smithi156.stdout:7/577: fdatasync dc/d13/d20/f3b 0 2024-07-16T09:55:46.405 INFO:tasks.workunit.client.0.smithi156.stdout:2/852: dwrite d0/d5/d76/f8a [0,4194304] 0 2024-07-16T09:55:46.406 INFO:tasks.workunit.client.0.smithi156.stdout:7/578: write dc/d13/d36/d56/f61 [2515832,42602] 0 2024-07-16T09:55:46.409 INFO:tasks.workunit.client.0.smithi156.stdout:2/853: unlink d0/d5/d29/f38 0 2024-07-16T09:55:46.414 INFO:tasks.workunit.client.0.smithi156.stdout:5/709: dwrite d1b/d1f/d45/d9d/fab [0,4194304] 0 2024-07-16T09:55:46.415 INFO:tasks.workunit.client.0.smithi156.stdout:5/710: chown d1b/d1f/d54 49 1 2024-07-16T09:55:46.417 INFO:tasks.workunit.client.0.smithi156.stdout:7/579: dwrite dc/d3e/d3f/d51/f53 [0,4194304] 0 2024-07-16T09:55:46.421 INFO:tasks.workunit.client.0.smithi156.stdout:2/854: dwrite d0/f8 [4194304,4194304] 0 2024-07-16T09:55:46.424 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:55:45 smithi156 ceph-mon[29302]: pgmap v254: 65 pgs: 65 active+clean; 2.7 GiB data, 9.3 GiB used, 527 GiB / 536 GiB avail; 97 MiB/s rd, 327 MiB/s wr, 608 op/s 2024-07-16T09:55:46.427 INFO:tasks.workunit.client.0.smithi156.stdout:7/580: link dc/d13/d15/f50 dc/d3e/d76/fc5 0 2024-07-16T09:55:46.428 INFO:tasks.workunit.client.0.smithi156.stdout:7/581: write dc/d3e/d3f/fab [138702,3968] 0 2024-07-16T09:55:46.429 INFO:tasks.workunit.client.1.smithi176.stdout:8/722: write d8/d20/d2a/f3f [3247972,98675] 0 2024-07-16T09:55:46.431 INFO:tasks.workunit.client.1.smithi176.stdout:6/774: dwrite d1/d5/d47/d62/d80/fe5 [0,4194304] 0 2024-07-16T09:55:46.432 INFO:tasks.workunit.client.0.smithi156.stdout:5/711: dread d1b/d1f/d45/d52/d43/f5b [0,4194304] 0 2024-07-16T09:55:46.432 INFO:tasks.workunit.client.0.smithi156.stdout:7/582: mknod dc/d13/d20/d4d/daa/cc6 0 2024-07-16T09:55:46.434 INFO:tasks.workunit.client.0.smithi156.stdout:5/712: chown d1b/d1f/d42/d6d/d9f/fe3 9 1 2024-07-16T09:55:46.436 INFO:tasks.workunit.client.1.smithi176.stdout:8/723: creat d8/d14/d1e/d55/ffa x:0 0 0 2024-07-16T09:55:46.438 INFO:tasks.workunit.client.1.smithi176.stdout:6/775: rename d1/d5d/d9e/dd9/c7e to d1/d5/d47/d62/d77/dd2/c11a 0 2024-07-16T09:55:46.439 INFO:tasks.workunit.client.0.smithi156.stdout:5/713: creat d1b/d1f/d45/d52/d43/d85/d97/d37/d88/dbc/fed x:0 0 0 2024-07-16T09:55:46.441 INFO:tasks.workunit.client.0.smithi156.stdout:5/714: chown d1b/cb8 1347218 1 2024-07-16T09:55:46.442 INFO:tasks.workunit.client.1.smithi176.stdout:6/776: dread d1/d5d/db9/fcb [0,4194304] 0 2024-07-16T09:55:46.442 INFO:tasks.workunit.client.0.smithi156.stdout:5/715: chown d1b/d1f/d45/d52/d43/d85/d97 7838493 1 2024-07-16T09:55:46.443 INFO:tasks.workunit.client.1.smithi176.stdout:6/777: chown d1/d5d/d9e/d8f/c84 55131 1 2024-07-16T09:55:46.444 INFO:tasks.workunit.client.0.smithi156.stdout:4/775: sync 2024-07-16T09:55:46.444 INFO:tasks.workunit.client.0.smithi156.stdout:9/800: sync 2024-07-16T09:55:46.444 INFO:tasks.workunit.client.0.smithi156.stdout:5/716: rmdir d1b/d1f/d45/d7f 39 2024-07-16T09:55:46.445 INFO:tasks.workunit.client.1.smithi176.stdout:6/778: truncate d1/d5d/d9e/dd9/d6f/d7c/fa2 885203 0 2024-07-16T09:55:46.447 INFO:tasks.workunit.client.0.smithi156.stdout:9/801: creat d2/d5/d16/de2/ffa x:0 0 0 2024-07-16T09:55:46.447 INFO:tasks.workunit.client.0.smithi156.stdout:5/717: mknod d1b/d1f/d42/de0/cee 0 2024-07-16T09:55:46.449 INFO:tasks.workunit.client.1.smithi176.stdout:6/779: mknod d1/d5d/d9e/dd9/d6f/c11b 0 2024-07-16T09:55:46.450 INFO:tasks.workunit.client.0.smithi156.stdout:9/802: rmdir d2/d5/d61/da2 39 2024-07-16T09:55:46.450 INFO:tasks.workunit.client.1.smithi176.stdout:6/780: stat d1/d5/d7/d1e 0 2024-07-16T09:55:46.451 INFO:tasks.workunit.client.0.smithi156.stdout:5/718: chown d1b/d1f/d45/d52/d43/d85/d97/l22 1445912 1 2024-07-16T09:55:46.452 INFO:tasks.workunit.client.0.smithi156.stdout:9/803: rmdir d2/d5/d16/d57 39 2024-07-16T09:55:46.453 INFO:tasks.workunit.client.1.smithi176.stdout:8/724: dwrite d8/d20/d2a/db9/dcb/d34/d62/fb1 [0,4194304] 0 2024-07-16T09:55:46.453 INFO:tasks.workunit.client.0.smithi156.stdout:2/855: dwrite d0/d13/f1a [0,4194304] 0 2024-07-16T09:55:46.454 INFO:tasks.workunit.client.1.smithi176.stdout:6/781: unlink d1/d5/d47/d62/dbf/cd7 0 2024-07-16T09:55:46.456 INFO:tasks.workunit.client.0.smithi156.stdout:4/776: write d4/faa [885541,23112] 0 2024-07-16T09:55:46.456 INFO:tasks.workunit.client.1.smithi176.stdout:8/725: creat d8/d20/d2a/d53/d9a/dc1/ffb x:0 0 0 2024-07-16T09:55:46.457 INFO:tasks.workunit.client.0.smithi156.stdout:5/719: creat d1b/d1f/d45/d52/d43/d85/d97/d37/d88/dc7/dd9/fef x:0 0 0 2024-07-16T09:55:46.459 INFO:tasks.workunit.client.1.smithi176.stdout:8/726: read d8/d23/d76/d83/fb6 [1149782,71573] 0 2024-07-16T09:55:46.462 INFO:tasks.workunit.client.0.smithi156.stdout:2/856: dread d0/d13/d17/d1c/f71 [0,4194304] 0 2024-07-16T09:55:46.464 INFO:tasks.workunit.client.0.smithi156.stdout:4/777: symlink d4/d23/d52/d71/def/d109/l112 0 2024-07-16T09:55:46.467 INFO:tasks.workunit.client.0.smithi156.stdout:5/720: rename d1b/d1f/d54/dbe/ldb to d1b/d1f/d45/d52/d43/d85/d97/d37/lf0 0 2024-07-16T09:55:46.468 INFO:tasks.workunit.client.1.smithi176.stdout:8/727: dwrite d8/d14/d1e/d55/f8a [0,4194304] 0 2024-07-16T09:55:46.469 INFO:tasks.workunit.client.0.smithi156.stdout:7/583: dread dc/d3e/d3f/d45/fb3 [0,4194304] 0 2024-07-16T09:55:46.471 INFO:tasks.workunit.client.0.smithi156.stdout:2/857: dwrite d0/d13/d17/f102 [0,4194304] 0 2024-07-16T09:55:46.471 INFO:tasks.workunit.client.0.smithi156.stdout:4/778: rename d4/db/dc/fa3 to d4/db/dc/d67/f113 0 2024-07-16T09:55:46.473 INFO:tasks.workunit.client.0.smithi156.stdout:2/858: chown d0/d5/d29/d2d/de1 1 1 2024-07-16T09:55:46.475 INFO:tasks.workunit.client.0.smithi156.stdout:9/804: rename d2/d5/d16/d57/d63/dd3/ddb/d3d/d3c/c5b to d2/d5/d16/de2/cfb 0 2024-07-16T09:55:46.476 INFO:tasks.workunit.client.1.smithi176.stdout:6/782: dwrite d1/d5/d47/fa7 [0,4194304] 0 2024-07-16T09:55:46.476 INFO:tasks.workunit.client.0.smithi156.stdout:4/779: write d4/f79 [4977640,71979] 0 2024-07-16T09:55:46.478 INFO:tasks.workunit.client.0.smithi156.stdout:7/584: mknod dc/d3e/d3f/d51/d8a/dae/cc7 0 2024-07-16T09:55:46.478 INFO:tasks.workunit.client.1.smithi176.stdout:6/783: write d1/f104 [937921,31666] 0 2024-07-16T09:55:46.480 INFO:tasks.workunit.client.0.smithi156.stdout:7/585: read dc/d3e/f7f [676127,41702] 0 2024-07-16T09:55:46.482 INFO:tasks.workunit.client.1.smithi176.stdout:6/784: symlink d1/d5/d47/d75/l11c 0 2024-07-16T09:55:46.484 INFO:tasks.workunit.client.1.smithi176.stdout:8/728: dwrite d8/d20/d2a/db9/dcb/d34/d62/fa4 [0,4194304] 0 2024-07-16T09:55:46.484 INFO:tasks.workunit.client.0.smithi156.stdout:9/805: symlink d2/d9b/lfc 0 2024-07-16T09:55:46.485 INFO:tasks.workunit.client.0.smithi156.stdout:5/721: rename d1b/d1f/d54/dbe/fc3 to d1b/d1f/d45/d52/d49/ff1 0 2024-07-16T09:55:46.486 INFO:tasks.workunit.client.1.smithi176.stdout:6/785: mkdir d1/dc/d5b/d11d 0 2024-07-16T09:55:46.488 INFO:tasks.workunit.client.0.smithi156.stdout:5/722: read d1b/d1f/d45/d52/d49/f60 [15668,105776] 0 2024-07-16T09:55:46.492 INFO:tasks.workunit.client.1.smithi176.stdout:6/786: mkdir d1/d5/d7/d1e/d11e 0 2024-07-16T09:55:46.492 INFO:tasks.workunit.client.0.smithi156.stdout:9/806: symlink d2/d5/d16/d57/d63/dd3/ddb/d3d/d3c/d7f/d42/lfd 0 2024-07-16T09:55:46.495 INFO:tasks.workunit.client.0.smithi156.stdout:5/723: chown d1b/d1f/d45/d52/d43/d85/cd7 23123044 1 2024-07-16T09:55:46.496 INFO:tasks.workunit.client.1.smithi176.stdout:8/729: getdents d8/d20/d2a/d31/d46/d6a/da8/dea 0 2024-07-16T09:55:46.498 INFO:tasks.workunit.client.1.smithi176.stdout:8/730: creat d8/d20/d2a/d53/d93/ffc x:0 0 0 2024-07-16T09:55:46.498 INFO:tasks.workunit.client.0.smithi156.stdout:9/807: creat d2/d5/d16/d57/d63/dd3/ddb/dd0/def/ffe x:0 0 0 2024-07-16T09:55:46.499 INFO:tasks.workunit.client.1.smithi176.stdout:8/731: read - d8/d14/d1e/d55/ffa zero size 2024-07-16T09:55:46.500 INFO:tasks.workunit.client.0.smithi156.stdout:5/724: fsync d1b/d99/dae/fd4 0 2024-07-16T09:55:46.500 INFO:tasks.workunit.client.0.smithi156.stdout:7/586: dread dc/d3e/d3f/d51/f5e [4194304,4194304] 0 2024-07-16T09:55:46.500 INFO:tasks.workunit.client.0.smithi156.stdout:9/808: chown d2/d9b/lfc 2620 1 2024-07-16T09:55:46.501 INFO:tasks.workunit.client.0.smithi156.stdout:4/780: dread d4/db/dc/f2e [4194304,4194304] 0 2024-07-16T09:55:46.503 INFO:tasks.workunit.client.0.smithi156.stdout:7/587: creat dc/d3e/d3f/d45/fc8 x:0 0 0 2024-07-16T09:55:46.503 INFO:tasks.workunit.client.0.smithi156.stdout:9/809: dread - d2/d5/d16/d57/d63/dd3/ddb/d74/dc2/dc4/fee zero size 2024-07-16T09:55:46.504 INFO:tasks.workunit.client.0.smithi156.stdout:7/588: write dc/d48/fa3 [3989538,35006] 0 2024-07-16T09:55:46.506 INFO:tasks.workunit.client.0.smithi156.stdout:4/781: creat d4/db/dc/dcc/dec/ded/d87/f114 x:0 0 0 2024-07-16T09:55:46.509 INFO:tasks.workunit.client.1.smithi176.stdout:1/504: sync 2024-07-16T09:55:46.509 INFO:tasks.workunit.client.1.smithi176.stdout:2/845: sync 2024-07-16T09:55:46.510 INFO:tasks.workunit.client.1.smithi176.stdout:5/845: sync 2024-07-16T09:55:46.510 INFO:tasks.workunit.client.1.smithi176.stdout:4/717: sync 2024-07-16T09:55:46.510 INFO:tasks.workunit.client.1.smithi176.stdout:7/703: sync 2024-07-16T09:55:46.510 INFO:tasks.workunit.client.1.smithi176.stdout:9/861: sync 2024-07-16T09:55:46.510 INFO:tasks.workunit.client.1.smithi176.stdout:3/611: sync 2024-07-16T09:55:46.510 INFO:tasks.workunit.client.1.smithi176.stdout:0/835: sync 2024-07-16T09:55:46.516 INFO:tasks.workunit.client.0.smithi156.stdout:9/810: rename d2/d5/d16/d57/d63/dd3/ddb/d3d/d3c/l54 to d2/d5/d16/d57/d63/dd3/ddb/d3d/db1/lff 0 2024-07-16T09:55:46.516 INFO:tasks.workunit.client.1.smithi176.stdout:6/787: write d1/dc/d40/d71/ff0 [729358,19774] 0 2024-07-16T09:55:46.516 INFO:tasks.workunit.client.1.smithi176.stdout:5/846: dread - d4/da9/dc4/deb/f124 zero size 2024-07-16T09:55:46.517 INFO:tasks.workunit.client.1.smithi176.stdout:3/612: rmdir d3/d17/d1d/d4f/d6d 39 2024-07-16T09:55:46.519 INFO:tasks.workunit.client.0.smithi156.stdout:5/725: dread d1b/d1f/d45/d52/d43/d85/d97/d37/f5a [0,4194304] 0 2024-07-16T09:55:46.519 INFO:tasks.workunit.client.0.smithi156.stdout:7/589: dwrite dc/d3e/d3f/d45/f9a [0,4194304] 0 2024-07-16T09:55:46.520 INFO:tasks.workunit.client.1.smithi176.stdout:7/704: creat d4/d5/fe5 x:0 0 0 2024-07-16T09:55:46.520 INFO:tasks.workunit.client.1.smithi176.stdout:2/846: mknod dd/d36/d82/c116 0 2024-07-16T09:55:46.521 INFO:tasks.workunit.client.0.smithi156.stdout:4/782: dwrite d4/db/dc/f106 [0,4194304] 0 2024-07-16T09:55:46.522 INFO:tasks.workunit.client.1.smithi176.stdout:0/836: truncate d5/d2c/d6c/d83/ff2 2244065 0 2024-07-16T09:55:46.523 INFO:tasks.workunit.client.1.smithi176.stdout:6/788: write d1/d5/d7/d54/f70 [82131,121853] 0 2024-07-16T09:55:46.524 INFO:tasks.workunit.client.1.smithi176.stdout:9/862: creat d5/d1d/f11e x:0 0 0 2024-07-16T09:55:46.524 INFO:tasks.workunit.client.1.smithi176.stdout:8/732: dread d8/d20/d2a/db9/dcb/d37/f57 [0,4194304] 0 2024-07-16T09:55:46.524 INFO:tasks.workunit.client.1.smithi176.stdout:3/613: fsync d3/da/d66/fa1 0 2024-07-16T09:55:46.525 INFO:tasks.workunit.client.0.smithi156.stdout:5/726: mknod d1b/d99/dae/cf2 0 2024-07-16T09:55:46.525 INFO:tasks.workunit.client.0.smithi156.stdout:9/811: getdents d2 0 2024-07-16T09:55:46.526 INFO:tasks.workunit.client.1.smithi176.stdout:5/847: symlink d4/d14/l137 0 2024-07-16T09:55:46.527 INFO:tasks.workunit.client.0.smithi156.stdout:7/590: symlink dc/d13/d20/d4d/lc9 0 2024-07-16T09:55:46.528 INFO:tasks.workunit.client.0.smithi156.stdout:5/727: write d1b/d1f/fa9 [38856,20359] 0 2024-07-16T09:55:46.529 INFO:tasks.workunit.client.0.smithi156.stdout:4/783: mkdir d4/d23/d52/d71/def/d115 0 2024-07-16T09:55:46.529 INFO:tasks.workunit.client.1.smithi176.stdout:2/847: chown dd/d36/l4e 162941181 1 2024-07-16T09:55:46.529 INFO:tasks.workunit.client.1.smithi176.stdout:5/848: readlink d4/l8 0 2024-07-16T09:55:46.529 INFO:tasks.workunit.client.1.smithi176.stdout:8/733: stat f5 0 2024-07-16T09:55:46.532 INFO:tasks.workunit.client.0.smithi156.stdout:9/812: read d2/d5/f55 [4723568,49924] 0 2024-07-16T09:55:46.533 INFO:tasks.workunit.client.1.smithi176.stdout:1/505: dwrite d4/d8/d12/d4c/fa7 [0,4194304] 0 2024-07-16T09:55:46.533 INFO:tasks.workunit.client.0.smithi156.stdout:4/784: symlink d4/db/dc/dcc/dec/ded/l116 0 2024-07-16T09:55:46.535 INFO:tasks.workunit.client.1.smithi176.stdout:6/789: rmdir d1/d5/d47/d62/de8 39 2024-07-16T09:55:46.536 INFO:tasks.workunit.client.1.smithi176.stdout:3/614: write d3/da/f41 [434135,2132] 0 2024-07-16T09:55:46.539 INFO:tasks.workunit.client.0.smithi156.stdout:9/813: unlink d2/d5/c12 0 2024-07-16T09:55:46.540 INFO:tasks.workunit.client.1.smithi176.stdout:7/705: truncate d4/dd/d63/f64 2936557 0 2024-07-16T09:55:46.543 INFO:tasks.workunit.client.0.smithi156.stdout:5/728: write d1b/d1f/d45/d52/f39 [4712121,66817] 0 2024-07-16T09:55:46.543 INFO:tasks.workunit.client.0.smithi156.stdout:9/814: rmdir d2/d5/d16/d57/d63/dd3/ddb/d74/dc2/dc4/d3a 39 2024-07-16T09:55:46.545 INFO:tasks.workunit.client.0.smithi156.stdout:5/729: fsync d1b/d1f/d45/d52/d43/f64 0 2024-07-16T09:55:46.546 INFO:tasks.workunit.client.0.smithi156.stdout:4/785: read d4/db/dc/d67/f113 [47254,107657] 0 2024-07-16T09:55:46.546 INFO:tasks.workunit.client.0.smithi156.stdout:9/815: rmdir d2/d5/d16/d57/d63/dd3/ddb/dd0/def 39 2024-07-16T09:55:46.547 INFO:tasks.workunit.client.0.smithi156.stdout:4/786: stat d4/db/dc/d7c/le7 0 2024-07-16T09:55:46.549 INFO:tasks.workunit.client.1.smithi176.stdout:0/837: symlink d5/d1c/d24/d35/d50/d7c/l116 0 2024-07-16T09:55:46.550 INFO:tasks.workunit.client.0.smithi156.stdout:5/730: symlink d1b/d1f/d45/d7a/d98/lf3 0 2024-07-16T09:55:46.550 INFO:tasks.workunit.client.0.smithi156.stdout:9/816: truncate d2/d5/d16/d57/d63/dd3/ddb/d3d/fda 496322 0 2024-07-16T09:55:46.550 INFO:tasks.workunit.client.1.smithi176.stdout:3/615: stat d3/da/d33/f39 0 2024-07-16T09:55:46.551 INFO:tasks.workunit.client.0.smithi156.stdout:4/787: rename d4/d23/d30/fa1 to d4/db/dc/d5f/f117 0 2024-07-16T09:55:46.551 INFO:tasks.workunit.client.0.smithi156.stdout:5/731: write d1b/d1f/d42/d6d/d9f/fe3 [3020855,68480] 0 2024-07-16T09:55:46.553 INFO:tasks.workunit.client.0.smithi156.stdout:9/817: write d2/d5/d16/d57/d63/dd3/fed [766111,20829] 0 2024-07-16T09:55:46.555 INFO:tasks.workunit.client.0.smithi156.stdout:5/732: mknod d1b/d1f/d42/d6d/cf4 0 2024-07-16T09:55:46.555 INFO:tasks.workunit.client.0.smithi156.stdout:5/733: stat d1b/d1f/d42/f47 0 2024-07-16T09:55:46.556 INFO:tasks.workunit.client.0.smithi156.stdout:9/818: readlink d2/d5/d16/d22/d36/l71 0 2024-07-16T09:55:46.558 INFO:tasks.workunit.client.0.smithi156.stdout:9/819: write d2/d5/d16/d57/d63/dd3/ddb/f8b [742552,22703] 0 2024-07-16T09:55:46.558 INFO:tasks.workunit.client.0.smithi156.stdout:5/734: chown d1b/d1f/d45/d52/d49/d4d/fdc 398140 1 2024-07-16T09:55:46.559 INFO:tasks.workunit.client.1.smithi176.stdout:4/718: dread da/dc/d10/d96/d9b/db4/d40/d5d/d73/f4f [0,4194304] 0 2024-07-16T09:55:46.560 INFO:tasks.workunit.client.1.smithi176.stdout:5/849: dwrite d4/f27 [0,4194304] 0 2024-07-16T09:55:46.563 INFO:tasks.workunit.client.1.smithi176.stdout:2/848: dwrite dd/de/fa7 [0,4194304] 0 2024-07-16T09:55:46.563 INFO:tasks.workunit.client.1.smithi176.stdout:9/863: dwrite d5/d1d/dc7/d89/f111 [0,4194304] 0 2024-07-16T09:55:46.564 INFO:tasks.workunit.client.0.smithi156.stdout:9/820: creat d2/d9b/f100 x:0 0 0 2024-07-16T09:55:46.565 INFO:tasks.workunit.client.1.smithi176.stdout:2/849: dread - dd/f10d zero size 2024-07-16T09:55:46.566 INFO:tasks.workunit.client.1.smithi176.stdout:4/719: write da/d3d/d17/d54/fce [37133,79486] 0 2024-07-16T09:55:46.567 INFO:tasks.workunit.client.1.smithi176.stdout:8/734: dwrite d8/d23/d76/f98 [0,4194304] 0 2024-07-16T09:55:46.575 INFO:tasks.workunit.client.1.smithi176.stdout:6/790: symlink d1/d5/d7/l11f 0 2024-07-16T09:55:46.578 INFO:tasks.workunit.client.0.smithi156.stdout:6/657: sync 2024-07-16T09:55:46.579 INFO:tasks.workunit.client.1.smithi176.stdout:5/850: unlink d4/l92 0 2024-07-16T09:55:46.580 INFO:tasks.workunit.client.0.smithi156.stdout:6/658: chown d3/d13/d17/d4f/d94/d19 59 1 2024-07-16T09:55:46.580 INFO:tasks.workunit.client.0.smithi156.stdout:6/659: dread - d3/d13/d17/d4f/d5c/f80 zero size 2024-07-16T09:55:46.581 INFO:tasks.workunit.client.1.smithi176.stdout:6/791: dread d1/d5d/d9e/d8f/ffe [0,4194304] 0 2024-07-16T09:55:46.583 INFO:tasks.workunit.client.1.smithi176.stdout:2/850: symlink dd/d2e/d35/d59/d66/l117 0 2024-07-16T09:55:46.585 INFO:tasks.workunit.client.1.smithi176.stdout:9/864: write d5/d1d/d20/f33 [267389,76372] 0 2024-07-16T09:55:46.585 INFO:tasks.workunit.client.1.smithi176.stdout:1/506: link d4/db/f43 d4/d8/d12/d21/d25/faf 0 2024-07-16T09:55:46.587 INFO:tasks.workunit.client.1.smithi176.stdout:6/792: read d1/d5/d47/fa6 [597089,74079] 0 2024-07-16T09:55:46.589 INFO:tasks.workunit.client.1.smithi176.stdout:0/838: dwrite d5/d25/da0/fc0 [0,4194304] 0 2024-07-16T09:55:46.592 INFO:tasks.workunit.client.1.smithi176.stdout:9/865: dread - d5/d1d/d20/fc5 zero size 2024-07-16T09:55:46.596 INFO:tasks.workunit.client.1.smithi176.stdout:1/507: mknod d4/d8/d12/d21/d25/d30/cb0 0 2024-07-16T09:55:46.597 INFO:tasks.workunit.client.1.smithi176.stdout:0/839: write d5/d1c/d24/d35/d50/d58/f6f [1735121,117086] 0 2024-07-16T09:55:46.600 INFO:tasks.workunit.client.1.smithi176.stdout:6/793: symlink d1/d5d/d9e/l120 0 2024-07-16T09:55:46.601 INFO:tasks.workunit.client.1.smithi176.stdout:3/616: link d3/d17/d1d/d4f/d59/c94 d3/d17/d1d/d4f/d59/cba 0 2024-07-16T09:55:46.601 INFO:tasks.workunit.client.1.smithi176.stdout:9/866: write d5/de/da5/dc1/d112/d98/fb1 [384025,7866] 0 2024-07-16T09:55:46.606 INFO:tasks.workunit.client.1.smithi176.stdout:2/851: truncate dd/d36/d82/fa9 323154 0 2024-07-16T09:55:46.610 INFO:tasks.workunit.client.1.smithi176.stdout:6/794: rename d1/d5/d47/d62/dbf to d1/dc/d81/d10a/d115/d121 0 2024-07-16T09:55:46.615 INFO:tasks.workunit.client.1.smithi176.stdout:2/852: symlink dd/d49/d52/d89/l118 0 2024-07-16T09:55:46.615 INFO:tasks.workunit.client.1.smithi176.stdout:5/851: dread d4/d10/f15 [0,4194304] 0 2024-07-16T09:55:46.616 INFO:tasks.workunit.client.1.smithi176.stdout:2/853: chown dd/d16/f1d 70514 1 2024-07-16T09:55:46.616 INFO:tasks.workunit.client.1.smithi176.stdout:9/867: mkdir d5/d1d/db3/d11f 0 2024-07-16T09:55:46.617 INFO:tasks.workunit.client.1.smithi176.stdout:3/617: creat d3/d17/d1d/d4f/d59/da0/d7a/dac/fbb x:0 0 0 2024-07-16T09:55:46.617 INFO:tasks.workunit.client.1.smithi176.stdout:2/854: stat dd/d36/d45/c102 0 2024-07-16T09:55:46.617 INFO:tasks.workunit.client.0.smithi156.stdout:9/821: creat d2/d5/d16/d22/d36/deb/f101 x:0 0 0 2024-07-16T09:55:46.619 INFO:tasks.workunit.client.1.smithi176.stdout:7/706: dread d4/dd/d26/dc2/d17/f7f [0,4194304] 0 2024-07-16T09:55:46.620 INFO:tasks.workunit.client.1.smithi176.stdout:3/618: mkdir d3/da/d6c/dbc 0 2024-07-16T09:55:46.621 INFO:tasks.workunit.client.1.smithi176.stdout:3/619: chown d3/d17/d1d/d4f/c5a 435373925 1 2024-07-16T09:55:46.622 INFO:tasks.workunit.client.0.smithi156.stdout:6/660: symlink d3/d13/d17/d4f/d4d/d3b/da5/ld7 0 2024-07-16T09:55:46.622 INFO:tasks.workunit.client.0.smithi156.stdout:9/822: fsync d2/d9b/f100 0 2024-07-16T09:55:46.622 INFO:tasks.workunit.client.1.smithi176.stdout:5/852: mkdir d4/d14/dc5/def/df5/d138 0 2024-07-16T09:55:46.624 INFO:tasks.workunit.client.0.smithi156.stdout:4/788: dwrite d4/db/dc/f2e [0,4194304] 0 2024-07-16T09:55:46.628 INFO:tasks.workunit.client.1.smithi176.stdout:2/855: creat dd/d16/d39/d3d/f119 x:0 0 0 2024-07-16T09:55:46.631 INFO:tasks.workunit.client.0.smithi156.stdout:6/661: rename d3/d13/d17/d4f/d94/d19/d2e/f74 to d3/d13/d17/d4f/d94/d19/d31/d35/fd8 0 2024-07-16T09:55:46.632 INFO:tasks.workunit.client.0.smithi156.stdout:4/789: dread d4/db/dc/dcc/d58/f91 [0,4194304] 0 2024-07-16T09:55:46.634 INFO:tasks.workunit.client.1.smithi176.stdout:9/868: rename d5/de/da5/dc1/d112/d97/fe6 to d5/d1d/d20/d78/f120 0 2024-07-16T09:55:46.635 INFO:tasks.workunit.client.1.smithi176.stdout:7/707: write d4/dd/f1a [30289,56256] 0 2024-07-16T09:55:46.636 INFO:tasks.workunit.client.0.smithi156.stdout:9/823: dwrite d2/f4d [0,4194304] 0 2024-07-16T09:55:46.638 INFO:tasks.workunit.client.0.smithi156.stdout:6/662: creat d3/d13/d17/d4f/d4d/d3b/d5a/fd9 x:0 0 0 2024-07-16T09:55:46.638 INFO:tasks.workunit.client.1.smithi176.stdout:3/620: mkdir d3/da/d33/dbd 0 2024-07-16T09:55:46.640 INFO:tasks.workunit.client.0.smithi156.stdout:4/790: symlink d4/db/dc/dcc/df3/l118 0 2024-07-16T09:55:46.641 INFO:tasks.workunit.client.1.smithi176.stdout:3/621: dread - d3/d17/d1d/faf zero size 2024-07-16T09:55:46.644 INFO:tasks.workunit.client.0.smithi156.stdout:4/791: chown d4/db/dc/dcc/d58/fa6 2555 1 2024-07-16T09:55:46.646 INFO:tasks.workunit.client.0.smithi156.stdout:6/663: rmdir d3/d13/d17/d4f/d94/daa/dd0 0 2024-07-16T09:55:46.647 INFO:tasks.workunit.client.1.smithi176.stdout:9/869: creat d5/de/da5/dc1/d112/f121 x:0 0 0 2024-07-16T09:55:46.647 INFO:tasks.workunit.client.0.smithi156.stdout:4/792: read f2 [1238084,77808] 0 2024-07-16T09:55:46.648 INFO:tasks.workunit.client.1.smithi176.stdout:5/853: dread d4/fd [0,4194304] 0 2024-07-16T09:55:46.649 INFO:tasks.workunit.client.1.smithi176.stdout:3/622: mknod d3/da/d33/cbe 0 2024-07-16T09:55:46.651 INFO:tasks.workunit.client.1.smithi176.stdout:2/856: creat dd/d2e/d35/f11a x:0 0 0 2024-07-16T09:55:46.651 INFO:tasks.workunit.client.1.smithi176.stdout:8/735: write d8/d20/d2a/db9/dcb/d37/f57 [2800916,24874] 0 2024-07-16T09:55:46.653 INFO:tasks.workunit.client.1.smithi176.stdout:8/736: dread - d8/d14/d1e/ff1 zero size 2024-07-16T09:55:46.654 INFO:tasks.workunit.client.1.smithi176.stdout:5/854: dread - d4/d10/d17/f8a zero size 2024-07-16T09:55:46.656 INFO:tasks.workunit.client.1.smithi176.stdout:2/857: unlink dd/d36/d45/la2 0 2024-07-16T09:55:46.657 INFO:tasks.workunit.client.1.smithi176.stdout:3/623: creat d3/da/d6c/dbc/fbf x:0 0 0 2024-07-16T09:55:46.657 INFO:tasks.workunit.client.1.smithi176.stdout:2/858: stat dd/d36/d45/c102 0 2024-07-16T09:55:46.659 INFO:tasks.workunit.client.1.smithi176.stdout:8/737: write d8/d20/d2a/db9/dcb/f36 [2465016,52811] 0 2024-07-16T09:55:46.659 INFO:tasks.workunit.client.1.smithi176.stdout:5/855: truncate d4/d14/f45 2305407 0 2024-07-16T09:55:46.661 INFO:tasks.workunit.client.1.smithi176.stdout:3/624: creat d3/d17/d1d/d6e/fc0 x:0 0 0 2024-07-16T09:55:46.663 INFO:tasks.workunit.client.1.smithi176.stdout:8/738: write d8/d20/d2a/d53/d9a/dc1/fef [707239,15989] 0 2024-07-16T09:55:46.664 INFO:tasks.workunit.client.1.smithi176.stdout:2/859: rename dd/d16/f1d to dd/d2e/d35/d4a/df9/f11b 0 2024-07-16T09:55:46.665 INFO:tasks.workunit.client.1.smithi176.stdout:5/856: rmdir d4/d37/d72/d38/d102/d105 39 2024-07-16T09:55:46.667 INFO:tasks.workunit.client.1.smithi176.stdout:8/739: stat d8/d20/d2a/db9/dcb/f3b 0 2024-07-16T09:55:46.667 INFO:tasks.workunit.client.1.smithi176.stdout:3/625: creat d3/da/d6c/fc1 x:0 0 0 2024-07-16T09:55:46.670 INFO:tasks.workunit.client.1.smithi176.stdout:8/740: stat d8/d20/d2a/db9/dcb/d34/d62/fb1 0 2024-07-16T09:55:46.670 INFO:tasks.workunit.client.1.smithi176.stdout:1/508: truncate d4/d8/d12/d21/d25/d30/f61 1023719 0 2024-07-16T09:55:46.671 INFO:tasks.workunit.client.1.smithi176.stdout:0/840: dwrite d5/f13 [4194304,4194304] 0 2024-07-16T09:55:46.675 INFO:tasks.workunit.client.1.smithi176.stdout:5/857: creat d4/d37/d72/d38/d102/dff/dc7/f139 x:0 0 0 2024-07-16T09:55:46.675 INFO:tasks.workunit.client.0.smithi156.stdout:5/735: truncate f19 5348428 0 2024-07-16T09:55:46.675 INFO:tasks.workunit.client.1.smithi176.stdout:2/860: read dd/d2e/d35/d59/d66/f6e [1532442,102662] 0 2024-07-16T09:55:46.677 INFO:tasks.workunit.client.1.smithi176.stdout:1/509: truncate d4/d8/d12/d21/d34/f3d 4605032 0 2024-07-16T09:55:46.680 INFO:tasks.workunit.client.0.smithi156.stdout:5/736: creat d1b/d1f/d45/d52/d43/d85/d97/d37/d88/dc7/ff5 x:0 0 0 2024-07-16T09:55:46.685 INFO:tasks.workunit.client.0.smithi156.stdout:5/737: unlink d1b/d1f/d54/c91 0 2024-07-16T09:55:46.685 INFO:tasks.workunit.client.1.smithi176.stdout:0/841: rename d5/d1c/d24/d26 to d5/d1c/d24/d35/d50/d9b/de3/d117 0 2024-07-16T09:55:46.686 INFO:tasks.workunit.client.1.smithi176.stdout:5/858: rmdir d4/d14/dc5/def 39 2024-07-16T09:55:46.686 INFO:tasks.workunit.client.1.smithi176.stdout:6/795: dwrite d1/d5/d7/f6e [0,4194304] 0 2024-07-16T09:55:46.688 INFO:tasks.workunit.client.1.smithi176.stdout:9/870: truncate d5/de/d30/fa2 1892114 0 2024-07-16T09:55:46.689 INFO:tasks.workunit.client.1.smithi176.stdout:0/842: chown d5/d1c/d5a/d9e/ld4 485 1 2024-07-16T09:55:46.690 INFO:tasks.workunit.client.1.smithi176.stdout:6/796: fdatasync d1/d5/d47/d75/fff 0 2024-07-16T09:55:46.690 INFO:tasks.workunit.client.0.smithi156.stdout:5/738: creat d1b/d1f/d45/d52/d43/d85/d97/d37/d88/d61/ff6 x:0 0 0 2024-07-16T09:55:46.691 INFO:tasks.workunit.client.0.smithi156.stdout:5/739: write f14 [6922201,131071] 0 2024-07-16T09:55:46.693 INFO:tasks.workunit.client.1.smithi176.stdout:2/861: symlink dd/d16/d39/d3d/d112/l11c 0 2024-07-16T09:55:46.694 INFO:tasks.workunit.client.1.smithi176.stdout:7/708: dwrite d4/dd/d49/f55 [4194304,4194304] 0 2024-07-16T09:55:46.694 INFO:tasks.workunit.client.0.smithi156.stdout:4/793: write d4/d23/d52/f66 [2047386,124045] 0 2024-07-16T09:55:46.695 INFO:tasks.workunit.client.1.smithi176.stdout:6/797: write d1/d5d/d9e/faa [235020,57908] 0 2024-07-16T09:55:46.695 INFO:tasks.workunit.client.1.smithi176.stdout:2/862: readlink dd/d2e/l10b 0 2024-07-16T09:55:46.696 INFO:tasks.workunit.client.0.smithi156.stdout:5/740: mknod d1b/d1f/d45/d52/d43/d85/d97/d37/d88/dbc/dc2/cf7 0 2024-07-16T09:55:46.697 INFO:tasks.workunit.client.0.smithi156.stdout:5/741: fsync d1b/f55 0 2024-07-16T09:55:46.698 INFO:tasks.workunit.client.1.smithi176.stdout:1/510: dread d4/f24 [0,4194304] 0 2024-07-16T09:55:46.698 INFO:tasks.workunit.client.1.smithi176.stdout:8/741: link d8/f2e d8/d20/d2a/d31/d46/d6a/ffd 0 2024-07-16T09:55:46.698 INFO:tasks.workunit.client.1.smithi176.stdout:0/843: mkdir d5/d1c/d24/d35/d50/dd5/d118 0 2024-07-16T09:55:46.700 INFO:tasks.workunit.client.0.smithi156.stdout:6/664: dwrite d3/d13/d17/d4f/d4d/d3b/f5d [0,4194304] 0 2024-07-16T09:55:46.700 INFO:tasks.workunit.client.1.smithi176.stdout:1/511: readlink d4/d77/l7d 0 2024-07-16T09:55:46.702 INFO:tasks.workunit.client.0.smithi156.stdout:4/794: symlink d4/d23/d52/d71/de0/l119 0 2024-07-16T09:55:46.703 INFO:tasks.workunit.client.1.smithi176.stdout:3/626: truncate d3/f61 1561785 0 2024-07-16T09:55:46.704 INFO:tasks.workunit.client.0.smithi156.stdout:6/665: rmdir d3 39 2024-07-16T09:55:46.705 INFO:tasks.workunit.client.1.smithi176.stdout:3/627: chown d3/d17/d83/c9d 1073409591 1 2024-07-16T09:55:46.707 INFO:tasks.workunit.client.0.smithi156.stdout:5/742: dwrite d1b/d1f/f73 [0,4194304] 0 2024-07-16T09:55:46.707 INFO:tasks.workunit.client.1.smithi176.stdout:2/863: creat dd/d2e/d104/f11d x:0 0 0 2024-07-16T09:55:46.708 INFO:tasks.workunit.client.0.smithi156.stdout:4/795: mkdir d4/db/dc/dcc/dec/ded/d11a 0 2024-07-16T09:55:46.709 INFO:tasks.workunit.client.0.smithi156.stdout:4/796: stat d4/db/d47/dbe/lf6 0 2024-07-16T09:55:46.710 INFO:tasks.workunit.client.0.smithi156.stdout:5/743: chown d1b/d1f/d42/f5c 53348299 1 2024-07-16T09:55:46.710 INFO:tasks.workunit.client.1.smithi176.stdout:1/512: unlink d4/d8/d12/f2e 0 2024-07-16T09:55:46.711 INFO:tasks.workunit.client.1.smithi176.stdout:1/513: chown d4/l14 2 1 2024-07-16T09:55:46.717 INFO:tasks.workunit.client.1.smithi176.stdout:3/628: creat d3/d85/fc2 x:0 0 0 2024-07-16T09:55:46.717 INFO:tasks.workunit.client.0.smithi156.stdout:6/666: creat d3/d13/d17/d4f/d4d/d3b/d5a/dc1/dce/fda x:0 0 0 2024-07-16T09:55:46.718 INFO:tasks.workunit.client.1.smithi176.stdout:7/709: creat d4/d5/d97/fe6 x:0 0 0 2024-07-16T09:55:46.720 INFO:tasks.workunit.client.0.smithi156.stdout:4/797: unlink d4/db/dc/d5f/f5c 0 2024-07-16T09:55:46.722 INFO:tasks.workunit.client.0.smithi156.stdout:1/891: sync 2024-07-16T09:55:46.722 INFO:tasks.workunit.client.0.smithi156.stdout:0/828: sync 2024-07-16T09:55:46.722 INFO:tasks.workunit.client.1.smithi176.stdout:8/742: dread d8/d20/f5a [0,4194304] 0 2024-07-16T09:55:46.722 INFO:tasks.workunit.client.1.smithi176.stdout:2/864: rename dd/c70 to dd/d9b/c11e 0 2024-07-16T09:55:46.723 INFO:tasks.workunit.client.1.smithi176.stdout:2/865: chown dd/d16/d39/d3d/d7e/ca3 29358 1 2024-07-16T09:55:46.724 INFO:tasks.workunit.client.0.smithi156.stdout:6/667: link d3/d13/d17/d4f/d1e/l27 d3/d13/d17/d4f/d4d/d3b/d77/ldb 0 2024-07-16T09:55:46.724 INFO:tasks.workunit.client.0.smithi156.stdout:4/798: read d4/db/dc/f4e [2523689,117645] 0 2024-07-16T09:55:46.725 INFO:tasks.workunit.client.1.smithi176.stdout:2/866: write dd/d16/fe4 [1020956,74008] 0 2024-07-16T09:55:46.728 INFO:tasks.workunit.client.1.smithi176.stdout:5/859: getdents d4/d14/d19/d1b/d61/de7 0 2024-07-16T09:55:46.729 INFO:tasks.workunit.client.1.smithi176.stdout:9/871: link d5/d1d/dc7/d89/cac d5/d1d/dd2/de3/c122 0 2024-07-16T09:55:46.729 INFO:tasks.workunit.client.0.smithi156.stdout:0/829: rename d0/d1a/d4d/d61/d97/dfe to d0/d1a/d4d/d9/d27/d2c/d6d/d9d/dbb/d106 0 2024-07-16T09:55:46.729 INFO:tasks.workunit.client.0.smithi156.stdout:1/892: read f6 [1577714,10156] 0 2024-07-16T09:55:46.730 INFO:tasks.workunit.client.1.smithi176.stdout:3/629: unlink d3/c15 0 2024-07-16T09:55:46.730 INFO:tasks.workunit.client.0.smithi156.stdout:4/799: mknod d4/d23/d52/d71/def/c11b 0 2024-07-16T09:55:46.731 INFO:tasks.workunit.client.1.smithi176.stdout:8/743: dwrite d8/d20/d2a/d53/d93/ffc [0,4194304] 0 2024-07-16T09:55:46.732 INFO:tasks.workunit.client.0.smithi156.stdout:0/830: symlink d0/d1a/d4d/d61/d97/de1/de7/l107 0 2024-07-16T09:55:46.733 INFO:tasks.workunit.client.0.smithi156.stdout:4/800: truncate d4/db/dc/fd8 1831560 0 2024-07-16T09:55:46.735 INFO:tasks.workunit.client.1.smithi176.stdout:6/798: mkdir d1/d5/d47/d62/de8/d111/d122 0 2024-07-16T09:55:46.735 INFO:tasks.workunit.client.0.smithi156.stdout:1/893: rename dc/d14/d8b/db4/df2/f101 to dc/d14/d3e/d6c/f128 0 2024-07-16T09:55:46.735 INFO:tasks.workunit.client.0.smithi156.stdout:1/894: stat dc/d6b/dd5/dfb 0 2024-07-16T09:55:46.736 INFO:tasks.workunit.client.0.smithi156.stdout:0/831: creat d0/d1a/d4d/d60/f108 x:0 0 0 2024-07-16T09:55:46.736 INFO:tasks.workunit.client.1.smithi176.stdout:2/867: symlink dd/d2e/d35/d59/d66/l11f 0 2024-07-16T09:55:46.740 INFO:tasks.workunit.client.1.smithi176.stdout:5/860: write d4/f5 [477435,36283] 0 2024-07-16T09:55:46.741 INFO:tasks.workunit.client.1.smithi176.stdout:7/710: creat d4/dd/d26/dc2/fe7 x:0 0 0 2024-07-16T09:55:46.741 INFO:tasks.workunit.client.1.smithi176.stdout:8/744: chown d8/d20/d2a/db9/dcb/d37/c91 8037055 1 2024-07-16T09:55:46.742 INFO:tasks.workunit.client.0.smithi156.stdout:4/801: mkdir d4/db/d11c 0 2024-07-16T09:55:46.743 INFO:tasks.workunit.client.1.smithi176.stdout:7/711: dread - d4/d47/d5d/f90 zero size 2024-07-16T09:55:46.743 INFO:tasks.workunit.client.1.smithi176.stdout:0/844: dwrite d5/d39/f51 [0,4194304] 0 2024-07-16T09:55:46.744 INFO:tasks.workunit.client.0.smithi156.stdout:0/832: dwrite d0/d1a/d4d/d9/d68/dff/fb6 [4194304,4194304] 0 2024-07-16T09:55:46.744 INFO:tasks.workunit.client.0.smithi156.stdout:1/895: symlink dc/d14/d3e/d55/d58/l129 0 2024-07-16T09:55:46.746 INFO:tasks.workunit.client.1.smithi176.stdout:6/799: dread d1/dc/d40/f4d [4194304,4194304] 0 2024-07-16T09:55:46.747 INFO:tasks.workunit.client.1.smithi176.stdout:8/745: chown d8/d20/d2a/db9/dcb/d34/d62/dab/fda 26293576 1 2024-07-16T09:55:46.750 INFO:tasks.workunit.client.0.smithi156.stdout:1/896: write dc/d24/f3a [2094656,65480] 0 2024-07-16T09:55:46.750 INFO:tasks.workunit.client.1.smithi176.stdout:7/712: fdatasync d4/dd/d26/dc2/fa6 0 2024-07-16T09:55:46.754 INFO:tasks.workunit.client.0.smithi156.stdout:0/833: dwrite d0/d1a/d4d/d9/d27/d50/f5f [4194304,4194304] 0 2024-07-16T09:55:46.754 INFO:tasks.workunit.client.1.smithi176.stdout:9/872: link d5/de/d30/l63 d5/d1d/d20/d78/df2/l123 0 2024-07-16T09:55:46.755 INFO:tasks.workunit.client.0.smithi156.stdout:0/834: readlink d0/d1a/d4d/d9/d27/d2c/la8 0 2024-07-16T09:55:46.755 INFO:tasks.workunit.client.0.smithi156.stdout:9/824: sync 2024-07-16T09:55:46.756 INFO:tasks.workunit.client.0.smithi156.stdout:8/661: sync 2024-07-16T09:55:46.756 INFO:tasks.workunit.client.0.smithi156.stdout:7/591: sync 2024-07-16T09:55:46.756 INFO:tasks.workunit.client.0.smithi156.stdout:2/859: sync 2024-07-16T09:55:46.756 INFO:tasks.workunit.client.0.smithi156.stdout:3/709: sync 2024-07-16T09:55:46.760 INFO:tasks.workunit.client.1.smithi176.stdout:8/746: creat d8/d20/d2a/d31/d46/d6a/da8/ffe x:0 0 0 2024-07-16T09:55:46.760 INFO:tasks.workunit.client.0.smithi156.stdout:3/710: fsync d8/da/d28/d43/fbe 0 2024-07-16T09:55:46.761 INFO:tasks.workunit.client.1.smithi176.stdout:8/747: truncate d8/d20/d2a/f75 8812628 0 2024-07-16T09:55:46.763 INFO:tasks.workunit.client.0.smithi156.stdout:4/802: write d4/d23/f59 [4552760,101636] 0 2024-07-16T09:55:46.764 INFO:tasks.workunit.client.0.smithi156.stdout:8/662: fdatasync d0/d11/d20/d7f/fc8 0 2024-07-16T09:55:46.764 INFO:tasks.workunit.client.0.smithi156.stdout:5/744: write f19 [3677643,64232] 0 2024-07-16T09:55:46.764 INFO:tasks.workunit.client.0.smithi156.stdout:4/803: fsync d4/d23/ff9 0 2024-07-16T09:55:46.766 INFO:tasks.workunit.client.0.smithi156.stdout:0/835: getdents d0/d1a/d4d/d9/d27/d2c/d6d/d9d/dbb/d106 0 2024-07-16T09:55:46.767 INFO:tasks.workunit.client.0.smithi156.stdout:8/663: chown d0/d11/d2e/d35 4052451 1 2024-07-16T09:55:46.767 INFO:tasks.workunit.client.0.smithi156.stdout:7/592: truncate dc/d13/f1d 3875901 0 2024-07-16T09:55:46.768 INFO:tasks.workunit.client.0.smithi156.stdout:5/745: chown d1b/d1f/d54 816213 1 2024-07-16T09:55:46.769 INFO:tasks.workunit.client.0.smithi156.stdout:0/836: dread - d0/d1a/d4d/d9/d68/dcb/fe8 zero size 2024-07-16T09:55:46.770 INFO:tasks.workunit.client.0.smithi156.stdout:1/897: rename dc/d14/d8b/db4/df2/c7b to dc/d14/d3e/d6c/c12a 0 2024-07-16T09:55:46.770 INFO:tasks.workunit.client.1.smithi176.stdout:3/630: chown d3/d17/d1d/f76 41110 1 2024-07-16T09:55:46.771 INFO:tasks.workunit.client.0.smithi156.stdout:2/860: symlink d0/d5/d29/d2d/d5b/d5f/ddc/d103/l116 0 2024-07-16T09:55:46.771 INFO:tasks.workunit.client.1.smithi176.stdout:9/873: creat d5/de/d21/f124 x:0 0 0 2024-07-16T09:55:46.771 INFO:tasks.workunit.client.0.smithi156.stdout:3/711: truncate d8/d14/f6b 3406843 0 2024-07-16T09:55:46.772 INFO:tasks.workunit.client.0.smithi156.stdout:7/593: fdatasync dc/d13/d15/d8b/fa7 0 2024-07-16T09:55:46.773 INFO:tasks.workunit.client.1.smithi176.stdout:7/713: creat d4/d1f/dce/fe8 x:0 0 0 2024-07-16T09:55:46.774 INFO:tasks.workunit.client.0.smithi156.stdout:4/804: creat d4/db/dc/dcc/dec/ded/daf/f11d x:0 0 0 2024-07-16T09:55:46.774 INFO:tasks.workunit.client.1.smithi176.stdout:6/800: creat d1/f123 x:0 0 0 2024-07-16T09:55:46.775 INFO:tasks.workunit.client.1.smithi176.stdout:2/868: dread dd/fd6 [0,4194304] 0 2024-07-16T09:55:46.775 INFO:tasks.workunit.client.1.smithi176.stdout:3/631: write d3/d17/d1d/d4f/fa3 [1528822,91986] 0 2024-07-16T09:55:46.775 INFO:tasks.workunit.client.1.smithi176.stdout:0/845: creat d5/d1c/d24/d35/d50/d9b/de3/d117/f119 x:0 0 0 2024-07-16T09:55:46.775 INFO:tasks.workunit.client.0.smithi156.stdout:9/825: rename d2/f4d to d2/d5/d16/d57/da3/ddd/f102 0 2024-07-16T09:55:46.775 INFO:tasks.workunit.client.0.smithi156.stdout:5/746: read d1b/d1f/f20 [3290714,111279] 0 2024-07-16T09:55:46.776 INFO:tasks.workunit.client.0.smithi156.stdout:8/664: mknod d0/d11/d20/cef 0 2024-07-16T09:55:46.776 INFO:tasks.workunit.client.1.smithi176.stdout:9/874: mkdir d5/d1d/dd2/de3/d103/d125 0 2024-07-16T09:55:46.777 INFO:tasks.workunit.client.1.smithi176.stdout:0/846: chown d5/d39/f9f 6700359 1 2024-07-16T09:55:46.777 INFO:tasks.workunit.client.1.smithi176.stdout:8/748: symlink d8/lff 0 2024-07-16T09:55:46.779 INFO:tasks.workunit.client.0.smithi156.stdout:3/712: write d8/da/d28/d43/d71/fe4 [964384,113888] 0 2024-07-16T09:55:46.779 INFO:tasks.workunit.client.1.smithi176.stdout:7/714: creat d4/dd/db5/fe9 x:0 0 0 2024-07-16T09:55:46.780 INFO:tasks.workunit.client.0.smithi156.stdout:6/668: dread d3/d13/d17/f6c [0,4194304] 0 2024-07-16T09:55:46.781 INFO:tasks.workunit.client.1.smithi176.stdout:0/847: stat d5/d25/c49 0 2024-07-16T09:55:46.782 INFO:tasks.workunit.client.1.smithi176.stdout:7/715: write d4/d5/fe5 [262647,111203] 0 2024-07-16T09:55:46.782 INFO:tasks.workunit.client.1.smithi176.stdout:5/861: dread d4/d37/d72/d38/f7e [0,4194304] 0 2024-07-16T09:55:46.783 INFO:tasks.workunit.client.0.smithi156.stdout:8/665: symlink d0/d11/d20/d7f/daf/lf0 0 2024-07-16T09:55:46.785 INFO:tasks.workunit.client.1.smithi176.stdout:5/862: fdatasync d4/d14/dce/f107 0 2024-07-16T09:55:46.786 INFO:tasks.workunit.client.1.smithi176.stdout:2/869: rename dd/dcd/dfa/ffc to dd/d2e/d35/d4a/f120 0 2024-07-16T09:55:46.786 INFO:tasks.workunit.client.0.smithi156.stdout:3/713: rename d8/lac to d8/da/d28/lf3 0 2024-07-16T09:55:46.788 INFO:tasks.workunit.client.0.smithi156.stdout:1/898: truncate dc/d14/d3e/d6c/d7a/dcb/fbc 1890700 0 2024-07-16T09:55:46.789 INFO:tasks.workunit.client.1.smithi176.stdout:9/875: creat d5/de/da5/dc1/d112/d97/f126 x:0 0 0 2024-07-16T09:55:46.790 INFO:tasks.workunit.client.1.smithi176.stdout:3/632: dread d3/d17/d83/f91 [0,4194304] 0 2024-07-16T09:55:46.791 INFO:tasks.workunit.client.0.smithi156.stdout:7/594: dwrite dc/d13/d36/d56/fa2 [0,4194304] 0 2024-07-16T09:55:46.791 INFO:tasks.workunit.client.1.smithi176.stdout:7/716: mknod d4/d1f/cea 0 2024-07-16T09:55:46.792 INFO:tasks.workunit.client.1.smithi176.stdout:3/633: chown d3/d17/d1d/d4f/l70 2 1 2024-07-16T09:55:46.793 INFO:tasks.workunit.client.1.smithi176.stdout:0/848: truncate d5/f2f 923933 0 2024-07-16T09:55:46.793 INFO:tasks.workunit.client.0.smithi156.stdout:1/899: stat dc/d6b/d96/f10b 0 2024-07-16T09:55:46.793 INFO:tasks.workunit.client.0.smithi156.stdout:4/805: getdents d4/db/dc/dcc 0 2024-07-16T09:55:46.796 INFO:tasks.workunit.client.1.smithi176.stdout:2/870: creat dd/d16/d39/d3d/dc2/f121 x:0 0 0 2024-07-16T09:55:46.797 INFO:tasks.workunit.client.0.smithi156.stdout:7/595: chown dc/c8f 35737 1 2024-07-16T09:55:46.798 INFO:tasks.workunit.client.1.smithi176.stdout:5/863: dread d4/d37/d72/d38/d39/f9b [0,4194304] 0 2024-07-16T09:55:46.800 INFO:tasks.workunit.client.0.smithi156.stdout:4/806: rmdir d4/d23/d30/dd2 39 2024-07-16T09:55:46.801 INFO:tasks.workunit.client.1.smithi176.stdout:6/801: write d1/d5/d47/fa6 [1398381,95774] 0 2024-07-16T09:55:46.801 INFO:tasks.workunit.client.0.smithi156.stdout:8/666: dwrite d0/d11/d2e/f5c [0,4194304] 0 2024-07-16T09:55:46.803 INFO:tasks.workunit.client.1.smithi176.stdout:3/634: dread - d3/f74 zero size 2024-07-16T09:55:46.804 INFO:tasks.workunit.client.0.smithi156.stdout:1/900: read - dc/d6b/fd8 zero size 2024-07-16T09:55:46.805 INFO:tasks.workunit.client.1.smithi176.stdout:9/876: creat d5/d1d/d20/d78/df2/d118/d11c/f127 x:0 0 0 2024-07-16T09:55:46.805 INFO:tasks.workunit.client.1.smithi176.stdout:5/864: symlink d4/d9a/d10a/l13a 0 2024-07-16T09:55:46.806 INFO:tasks.workunit.client.0.smithi156.stdout:9/826: creat d2/d5/d16/d57/da3/f103 x:0 0 0 2024-07-16T09:55:46.806 INFO:tasks.workunit.client.0.smithi156.stdout:5/747: dread d1b/d1f/d45/d52/d49/ff1 [0,4194304] 0 2024-07-16T09:55:46.807 INFO:tasks.workunit.client.0.smithi156.stdout:7/596: symlink dc/d3e/d3f/d51/d8a/dae/lca 0 2024-07-16T09:55:46.809 INFO:tasks.workunit.client.0.smithi156.stdout:3/714: creat d8/ff4 x:0 0 0 2024-07-16T09:55:46.810 INFO:tasks.workunit.client.0.smithi156.stdout:2/861: dwrite d0/d5/d8b/feb [0,4194304] 0 2024-07-16T09:55:46.810 INFO:tasks.workunit.client.0.smithi156.stdout:1/901: chown dc/d14/fd4 3654 1 2024-07-16T09:55:46.811 INFO:tasks.workunit.client.1.smithi176.stdout:8/749: rename d8/d20/d2a to d8/d23/d100 0 2024-07-16T09:55:46.811 INFO:tasks.workunit.client.0.smithi156.stdout:4/807: truncate d4/d23/f78 3173187 0 2024-07-16T09:55:46.812 INFO:tasks.workunit.client.0.smithi156.stdout:5/748: stat d1b/d1f/d45/d52/cec 0 2024-07-16T09:55:46.813 INFO:tasks.workunit.client.0.smithi156.stdout:8/667: truncate d0/d11/d20/d7f/fc8 968255 0 2024-07-16T09:55:46.813 INFO:tasks.workunit.client.1.smithi176.stdout:6/802: mkdir d1/d5d/db9/d124 0 2024-07-16T09:55:46.814 INFO:tasks.workunit.client.0.smithi156.stdout:2/862: chown d0/l3a 879544479 1 2024-07-16T09:55:46.815 INFO:tasks.workunit.client.1.smithi176.stdout:7/717: rmdir d4/dd/d26/dd8 0 2024-07-16T09:55:46.815 INFO:tasks.workunit.client.0.smithi156.stdout:8/668: readlink d0/d11/d20/l31 0 2024-07-16T09:55:46.816 INFO:tasks.workunit.client.0.smithi156.stdout:4/808: readlink d4/db/l60 0 2024-07-16T09:55:46.816 INFO:tasks.workunit.client.0.smithi156.stdout:2/863: chown d0/d13/d17/d1c/d10b 695 1 2024-07-16T09:55:46.817 INFO:tasks.workunit.client.0.smithi156.stdout:8/669: chown d0/d11/d13/d1c/d24/fb4 6854 1 2024-07-16T09:55:46.817 INFO:tasks.workunit.client.0.smithi156.stdout:5/749: chown d1b/d1f/d42/c6b 982895 1 2024-07-16T09:55:46.818 INFO:tasks.workunit.client.0.smithi156.stdout:5/750: fdatasync f19 0 2024-07-16T09:55:46.818 INFO:tasks.workunit.client.1.smithi176.stdout:9/877: creat d5/d1d/d20/d78/df2/d118/d11c/f128 x:0 0 0 2024-07-16T09:55:46.821 INFO:tasks.workunit.client.1.smithi176.stdout:0/849: dread d5/d1c/d24/f30 [0,4194304] 0 2024-07-16T09:55:46.821 INFO:tasks.workunit.client.0.smithi156.stdout:5/751: chown d1b/d1f/d54/d72 1282 1 2024-07-16T09:55:46.821 INFO:tasks.workunit.client.0.smithi156.stdout:4/809: mknod d4/d23/d52/c11e 0 2024-07-16T09:55:46.823 INFO:tasks.workunit.client.1.smithi176.stdout:8/750: creat d8/d23/d100/db9/dcb/d34/d62/f101 x:0 0 0 2024-07-16T09:55:46.824 INFO:tasks.workunit.client.0.smithi156.stdout:3/715: mknod d8/da/cf5 0 2024-07-16T09:55:46.824 INFO:tasks.workunit.client.1.smithi176.stdout:0/850: stat d5/d1c/d24/d35/d50/d58/cff 0 2024-07-16T09:55:46.824 INFO:tasks.workunit.client.1.smithi176.stdout:6/803: dread d1/d5/d47/ff5 [0,4194304] 0 2024-07-16T09:55:46.825 INFO:tasks.workunit.client.0.smithi156.stdout:2/864: mkdir d0/d13/d17/d1c/d117 0 2024-07-16T09:55:46.826 INFO:tasks.workunit.client.0.smithi156.stdout:2/865: fsync d0/f7 0 2024-07-16T09:55:46.826 INFO:tasks.workunit.client.0.smithi156.stdout:8/670: rename d0/d11/d20/f21 to d0/d11/d20/d3f/d96/ff1 0 2024-07-16T09:55:46.827 INFO:tasks.workunit.client.1.smithi176.stdout:3/635: dwrite d3/da/d33/f78 [0,4194304] 0 2024-07-16T09:55:46.828 INFO:tasks.workunit.client.0.smithi156.stdout:5/752: creat d1b/d1f/d45/d52/d2d/d96/ff8 x:0 0 0 2024-07-16T09:55:46.829 INFO:tasks.workunit.client.0.smithi156.stdout:0/837: dread d0/d1a/d4d/d9/d27/d2c/d6d/f8b [4194304,4194304] 0 2024-07-16T09:55:46.829 INFO:tasks.workunit.client.0.smithi156.stdout:3/716: read d8/da/fc [4160703,47585] 0 2024-07-16T09:55:46.830 INFO:tasks.workunit.client.1.smithi176.stdout:2/871: rename dd/d2e/d104/f11d to dd/d2e/d35/d4a/f122 0 2024-07-16T09:55:46.830 INFO:tasks.workunit.client.0.smithi156.stdout:9/827: dwrite d2/d5/d16/d57/d63/dd3/ddb/d3d/f6d [0,4194304] 0 2024-07-16T09:55:46.832 INFO:tasks.workunit.client.1.smithi176.stdout:8/751: creat d8/d14/d1e/d7c/f102 x:0 0 0 2024-07-16T09:55:46.832 INFO:tasks.workunit.client.1.smithi176.stdout:0/851: dread d5/d25/da0/fdd [0,4194304] 0 2024-07-16T09:55:46.833 INFO:tasks.workunit.client.1.smithi176.stdout:6/804: symlink d1/d5/d47/d62/d77/l125 0 2024-07-16T09:55:46.835 INFO:tasks.workunit.client.1.smithi176.stdout:2/872: fdatasync dd/d36/fb2 0 2024-07-16T09:55:46.835 INFO:tasks.workunit.client.0.smithi156.stdout:5/753: write d1b/d1f/d45/d52/d43/d85/d97/d37/d88/dc7/ff5 [537970,25083] 0 2024-07-16T09:55:46.836 INFO:tasks.workunit.client.1.smithi176.stdout:7/718: getdents d4/d47 0 2024-07-16T09:55:46.838 INFO:tasks.workunit.client.0.smithi156.stdout:9/828: chown d2/d5/d16/d57/d63/dd3/ddb/d74/f9e 432474 1 2024-07-16T09:55:46.839 INFO:tasks.workunit.client.1.smithi176.stdout:6/805: dread - d1/d5d/d9e/d76/da5/ffb zero size 2024-07-16T09:55:46.839 INFO:tasks.workunit.client.0.smithi156.stdout:1/902: truncate dc/f95 21525 0 2024-07-16T09:55:46.842 INFO:tasks.workunit.client.1.smithi176.stdout:2/873: chown dd/d2e/d104/d10c 124858 1 2024-07-16T09:55:46.842 INFO:tasks.workunit.client.1.smithi176.stdout:9/878: dwrite d5/d1d/db3/fd1 [0,4194304] 0 2024-07-16T09:55:46.842 INFO:tasks.workunit.client.1.smithi176.stdout:5/865: truncate f2 1220178 0 2024-07-16T09:55:46.843 INFO:tasks.workunit.client.0.smithi156.stdout:5/754: dread d1b/d1f/d45/d52/d49/d4d/fdc [0,4194304] 0 2024-07-16T09:55:46.845 INFO:tasks.workunit.client.0.smithi156.stdout:7/597: truncate dc/d48/f4b 2885824 0 2024-07-16T09:55:46.846 INFO:tasks.workunit.client.1.smithi176.stdout:8/752: write d8/d14/d1e/d7c/d82/de0/ff0 [93255,11024] 0 2024-07-16T09:55:46.847 INFO:tasks.workunit.client.0.smithi156.stdout:7/598: chown dc/d13/d15/l2e 4 1 2024-07-16T09:55:46.847 INFO:tasks.workunit.client.0.smithi156.stdout:3/717: mknod d8/da/d28/d40/cf6 0 2024-07-16T09:55:46.848 INFO:tasks.workunit.client.1.smithi176.stdout:6/806: mknod d1/d5/d47/d62/c126 0 2024-07-16T09:55:46.849 INFO:tasks.workunit.client.1.smithi176.stdout:2/874: symlink dd/dcd/l123 0 2024-07-16T09:55:46.850 INFO:tasks.workunit.client.0.smithi156.stdout:1/903: dwrite dc/d6b/d96/f10b [0,4194304] 0 2024-07-16T09:55:46.851 INFO:tasks.workunit.client.0.smithi156.stdout:2/866: link d0/d13/d17/d1c/fe7 d0/d5/d76/f118 0 2024-07-16T09:55:46.852 INFO:tasks.workunit.client.1.smithi176.stdout:0/852: dwrite d5/d1c/fd7 [0,4194304] 0 2024-07-16T09:55:46.852 INFO:tasks.workunit.client.1.smithi176.stdout:6/807: read - d1/d5/d7/fdc zero size 2024-07-16T09:55:46.852 INFO:tasks.workunit.client.1.smithi176.stdout:9/879: unlink d5/de/da5/lbe 0 2024-07-16T09:55:46.854 INFO:tasks.workunit.client.0.smithi156.stdout:2/867: write d0/d5/d76/da1/fcf [18528,15697] 0 2024-07-16T09:55:46.856 INFO:tasks.workunit.client.1.smithi176.stdout:5/866: rename d4/f117 to d4/d6c/df6/f13b 0 2024-07-16T09:55:46.857 INFO:tasks.workunit.client.1.smithi176.stdout:9/880: dread - d5/d52/ff1 zero size 2024-07-16T09:55:46.859 INFO:tasks.workunit.client.0.smithi156.stdout:8/671: link d0/d11/d62/fc5 d0/d11/d13/d1c/ff2 0 2024-07-16T09:55:46.859 INFO:tasks.workunit.client.0.smithi156.stdout:1/904: rename dc/d6b/d104/l122 to dc/d14/d8b/db4/l12b 0 2024-07-16T09:55:46.860 INFO:tasks.workunit.client.1.smithi176.stdout:7/719: dwrite d4/dd/d26/dc2/d17/d1d/f9b [0,4194304] 0 2024-07-16T09:55:46.862 INFO:tasks.workunit.client.0.smithi156.stdout:5/755: truncate d1b/d1f/d54/f74 946947 0 2024-07-16T09:55:46.865 INFO:tasks.workunit.client.0.smithi156.stdout:7/599: symlink dc/d13/d20/lcb 0 2024-07-16T09:55:46.869 INFO:tasks.workunit.client.1.smithi176.stdout:3/636: truncate d3/d17/d1d/f76 6242355 0 2024-07-16T09:55:46.870 INFO:tasks.workunit.client.0.smithi156.stdout:0/838: getdents d0/d1a/d4d 0 2024-07-16T09:55:46.871 INFO:tasks.workunit.client.1.smithi176.stdout:8/753: getdents d8/df7 0 2024-07-16T09:55:46.871 INFO:tasks.workunit.client.1.smithi176.stdout:3/637: write d3/da/d33/f78 [3629047,89038] 0 2024-07-16T09:55:46.872 INFO:tasks.workunit.client.0.smithi156.stdout:5/756: write d1b/d1f/d45/d52/d43/d85/d97/f25 [4814498,8515] 0 2024-07-16T09:55:46.872 INFO:tasks.workunit.client.0.smithi156.stdout:2/868: rmdir d0/d13/dc9 39 2024-07-16T09:55:46.873 INFO:tasks.workunit.client.0.smithi156.stdout:8/672: dwrite d0/d11/d4a/fc3 [0,4194304] 0 2024-07-16T09:55:46.873 INFO:tasks.workunit.client.1.smithi176.stdout:2/875: mkdir dd/d49/d52/d10a/d124 0 2024-07-16T09:55:46.874 INFO:tasks.workunit.client.1.smithi176.stdout:3/638: write d3/d17/d1d/d4f/d59/da0/f9e [3912772,124454] 0 2024-07-16T09:55:46.874 INFO:tasks.workunit.client.1.smithi176.stdout:6/808: mknod d1/d5/d47/d75/c127 0 2024-07-16T09:55:46.877 INFO:tasks.workunit.client.0.smithi156.stdout:1/905: stat dc/d14/d8b/db4/df2/l72 0 2024-07-16T09:55:46.878 INFO:tasks.workunit.client.1.smithi176.stdout:9/881: rename d5/d1d/dc7/d89/lff to d5/d52/l129 0 2024-07-16T09:55:46.878 INFO:tasks.workunit.client.1.smithi176.stdout:7/720: mkdir d4/dd/d49/d7a/db1/deb 0 2024-07-16T09:55:46.878 INFO:tasks.workunit.client.0.smithi156.stdout:7/600: read dc/d13/d36/d56/fa2 [3673046,74109] 0 2024-07-16T09:55:46.880 INFO:tasks.workunit.client.1.smithi176.stdout:8/754: mkdir d8/d23/d76/d83/d103 0 2024-07-16T09:55:46.880 INFO:tasks.workunit.client.1.smithi176.stdout:3/639: mknod d3/da/d66/cc3 0 2024-07-16T09:55:46.881 INFO:tasks.workunit.client.0.smithi156.stdout:1/906: stat dc/d6b/f74 0 2024-07-16T09:55:46.882 INFO:tasks.workunit.client.1.smithi176.stdout:0/853: dread d5/d2c/f3f [0,4194304] 0 2024-07-16T09:55:46.883 INFO:tasks.workunit.client.0.smithi156.stdout:4/810: dwrite d4/db/dc/dcc/d58/fa6 [0,4194304] 0 2024-07-16T09:55:46.883 INFO:tasks.workunit.client.1.smithi176.stdout:5/867: creat d4/d10/f13c x:0 0 0 2024-07-16T09:55:46.887 INFO:tasks.workunit.client.1.smithi176.stdout:7/721: dread d4/d5/f2f [0,4194304] 0 2024-07-16T09:55:46.887 INFO:tasks.workunit.client.1.smithi176.stdout:8/755: fdatasync d8/d23/d76/d83/fb6 0 2024-07-16T09:55:46.887 INFO:tasks.workunit.client.1.smithi176.stdout:3/640: readlink d3/d17/d1d/d4f/l7f 0 2024-07-16T09:55:46.888 INFO:tasks.workunit.client.0.smithi156.stdout:0/839: creat d0/d1a/d4d/d61/d97/de1/f109 x:0 0 0 2024-07-16T09:55:46.889 INFO:tasks.workunit.client.1.smithi176.stdout:0/854: chown d5/d1c/d24/d35/d50/d9b/de3/d117/l5c 83 1 2024-07-16T09:55:46.891 INFO:tasks.workunit.client.0.smithi156.stdout:8/673: creat d0/d11/d20/ff3 x:0 0 0 2024-07-16T09:55:46.896 INFO:tasks.workunit.client.1.smithi176.stdout:7/722: creat d4/d1f/d22/fec x:0 0 0 2024-07-16T09:55:46.896 INFO:tasks.workunit.client.1.smithi176.stdout:8/756: dwrite d8/d14/d1e/d7c/d82/df5/fd7 [0,4194304] 0 2024-07-16T09:55:46.896 INFO:tasks.workunit.client.1.smithi176.stdout:3/641: rmdir d3/da/d84 39 2024-07-16T09:55:46.896 INFO:tasks.workunit.client.0.smithi156.stdout:8/674: fsync d0/d11/f15 0 2024-07-16T09:55:46.896 INFO:tasks.workunit.client.0.smithi156.stdout:7/601: dwrite dc/d3e/d3f/d51/f53 [4194304,4194304] 0 2024-07-16T09:55:46.902 INFO:tasks.workunit.client.0.smithi156.stdout:7/602: mkdir dc/d13/d15/d8b/dcc 0 2024-07-16T09:55:46.903 INFO:tasks.workunit.client.1.smithi176.stdout:7/723: fsync d4/d5/f2f 0 2024-07-16T09:55:46.903 INFO:tasks.workunit.client.0.smithi156.stdout:3/718: dread d8/da/f32 [0,4194304] 0 2024-07-16T09:55:46.904 INFO:tasks.workunit.client.0.smithi156.stdout:1/907: rename dc/d14/d3e/d6c/c9b to dc/d14/d1e/c12c 0 2024-07-16T09:55:46.908 INFO:tasks.workunit.client.1.smithi176.stdout:8/757: dwrite d8/d23/d100/d53/d93/ffc [0,4194304] 0 2024-07-16T09:55:46.909 INFO:tasks.workunit.client.0.smithi156.stdout:1/908: mkdir dc/d14/d8b/dcd/d12d 0 2024-07-16T09:55:46.913 INFO:tasks.workunit.client.0.smithi156.stdout:7/603: mknod dc/d13/ccd 0 2024-07-16T09:55:46.914 INFO:tasks.workunit.client.0.smithi156.stdout:7/604: fsync dc/d13/d20/f99 0 2024-07-16T09:55:46.918 INFO:tasks.workunit.client.1.smithi176.stdout:7/724: mkdir d4/dd/d49/d7a/db1/deb/ded 0 2024-07-16T09:55:46.918 INFO:tasks.workunit.client.0.smithi156.stdout:8/675: getdents d0/dd/d1f/d50/d8d 0 2024-07-16T09:55:46.919 INFO:tasks.workunit.client.0.smithi156.stdout:8/676: write d0/d11/d4a/fc3 [2409592,118479] 0 2024-07-16T09:55:46.919 INFO:tasks.workunit.client.1.smithi176.stdout:6/809: dread d1/d5/d47/d62/fa1 [0,4194304] 0 2024-07-16T09:55:46.920 INFO:tasks.workunit.client.1.smithi176.stdout:8/758: mknod d8/d23/d76/d83/d103/c104 0 2024-07-16T09:55:46.920 INFO:tasks.workunit.client.1.smithi176.stdout:6/810: chown d1/dc/d81 400 1 2024-07-16T09:55:46.922 INFO:tasks.workunit.client.1.smithi176.stdout:6/811: dread - d1/d5/d7/fdc zero size 2024-07-16T09:55:46.922 INFO:tasks.workunit.client.1.smithi176.stdout:9/882: write d5/d1d/db3/f94 [744968,101456] 0 2024-07-16T09:55:46.923 INFO:tasks.workunit.client.0.smithi156.stdout:3/719: truncate d8/d14/f6b 2138707 0 2024-07-16T09:55:46.925 INFO:tasks.workunit.client.0.smithi156.stdout:1/909: rmdir dc/d14/d3e/d6c 39 2024-07-16T09:55:46.926 INFO:tasks.workunit.client.1.smithi176.stdout:8/759: creat d8/d14/d1e/d33/f105 x:0 0 0 2024-07-16T09:55:46.926 INFO:tasks.workunit.client.0.smithi156.stdout:9/829: truncate d2/d5/f44 3221144 0 2024-07-16T09:55:46.926 INFO:tasks.workunit.client.1.smithi176.stdout:9/883: creat d5/de/da5/dc1/f12a x:0 0 0 2024-07-16T09:55:46.926 INFO:tasks.workunit.client.0.smithi156.stdout:8/677: creat d0/d11/d20/d3f/dbe/ff4 x:0 0 0 2024-07-16T09:55:46.927 INFO:tasks.workunit.client.1.smithi176.stdout:0/855: write d5/d1c/d24/d35/d50/f7f [364232,56629] 0 2024-07-16T09:55:46.927 INFO:tasks.workunit.client.1.smithi176.stdout:9/884: fdatasync d5/d1d/f11e 0 2024-07-16T09:55:46.929 INFO:tasks.workunit.client.1.smithi176.stdout:6/812: mkdir d1/d5d/d9e/dd9/d6f/d7c/d128 0 2024-07-16T09:55:46.930 INFO:tasks.workunit.client.1.smithi176.stdout:7/725: getdents d4/dd/d49/d7a/db1 0 2024-07-16T09:55:46.932 INFO:tasks.workunit.client.1.smithi176.stdout:0/856: mkdir d5/d39/da3/d11a 0 2024-07-16T09:55:46.933 INFO:tasks.workunit.client.0.smithi156.stdout:1/910: chown dc/d14/d3e/d6c/d7a/cbd 57476622 1 2024-07-16T09:55:46.934 INFO:tasks.workunit.client.1.smithi176.stdout:6/813: fsync d1/d5d/db9/f85 0 2024-07-16T09:55:46.936 INFO:tasks.workunit.client.1.smithi176.stdout:0/857: mknod d5/d1c/d24/d35/d50/dd5/c11b 0 2024-07-16T09:55:46.936 INFO:tasks.workunit.client.0.smithi156.stdout:7/605: getdents dc/d13/d36 0 2024-07-16T09:55:46.936 INFO:tasks.workunit.client.1.smithi176.stdout:9/885: getdents d5/d1d/db0 0 2024-07-16T09:55:46.937 INFO:tasks.workunit.client.0.smithi156.stdout:8/678: dwrite d0/d11/d2e/f5c [0,4194304] 0 2024-07-16T09:55:46.937 INFO:tasks.workunit.client.0.smithi156.stdout:3/720: rmdir d8/d14/de8 0 2024-07-16T09:55:46.938 INFO:tasks.workunit.client.1.smithi176.stdout:6/814: truncate d1/d5d/d9e/dd9/d6f/f96 764773 0 2024-07-16T09:55:46.939 INFO:tasks.workunit.client.1.smithi176.stdout:0/858: rename d5/d25/f4e to d5/d44/f11c 0 2024-07-16T09:55:46.940 INFO:tasks.workunit.client.0.smithi156.stdout:9/830: unlink d2/d5/d16/d57/d63/dd3/ddb/d74/dc2/dc4/c46 0 2024-07-16T09:55:46.941 INFO:tasks.workunit.client.0.smithi156.stdout:0/840: write d0/d1a/d4d/d9/f88 [645400,51734] 0 2024-07-16T09:55:46.941 INFO:tasks.workunit.client.1.smithi176.stdout:4/720: sync 2024-07-16T09:55:46.942 INFO:tasks.workunit.client.0.smithi156.stdout:1/911: chown dc/d14/d8b/db4/fd6 3 1 2024-07-16T09:55:46.943 INFO:tasks.workunit.client.1.smithi176.stdout:0/859: read d5/d1c/d24/d35/d50/f7f [61527,116737] 0 2024-07-16T09:55:46.945 INFO:tasks.workunit.client.1.smithi176.stdout:0/860: chown d5/d1c/d24/d35/f61 961 1 2024-07-16T09:55:46.946 INFO:tasks.workunit.client.0.smithi156.stdout:9/831: stat d2/d5/d61 0 2024-07-16T09:55:46.947 INFO:tasks.workunit.client.1.smithi176.stdout:6/815: mknod d1/dc/d5b/d11d/c129 0 2024-07-16T09:55:46.948 INFO:tasks.workunit.client.1.smithi176.stdout:4/721: fsync da/dc/d10/d96/d69/f6d 0 2024-07-16T09:55:46.949 INFO:tasks.workunit.client.0.smithi156.stdout:2/869: dwrite d0/d5/d29/d2d/f73 [0,4194304] 0 2024-07-16T09:55:46.949 INFO:tasks.workunit.client.1.smithi176.stdout:6/816: chown d1/d5/d47/d62/l118 15678162 1 2024-07-16T09:55:46.950 INFO:tasks.workunit.client.0.smithi156.stdout:5/757: dwrite d1b/d1f/d45/d52/d43/d7c/fa2 [0,4194304] 0 2024-07-16T09:55:46.951 INFO:tasks.workunit.client.1.smithi176.stdout:0/861: rmdir d5/d1c/d24/d35/d50/d5b 39 2024-07-16T09:55:46.951 INFO:tasks.workunit.client.0.smithi156.stdout:9/832: fdatasync d2/d5/d16/de2/ffa 0 2024-07-16T09:55:46.952 INFO:tasks.workunit.client.1.smithi176.stdout:9/886: dread d5/d1d/d67/d6d/f7a [0,4194304] 0 2024-07-16T09:55:46.953 INFO:tasks.workunit.client.0.smithi156.stdout:4/811: dwrite d4/d23/d52/f69 [0,4194304] 0 2024-07-16T09:55:46.954 INFO:tasks.workunit.client.0.smithi156.stdout:1/912: chown dc/d14/d3e/d6c/d7a/f8c 707804 1 2024-07-16T09:55:46.956 INFO:tasks.workunit.client.0.smithi156.stdout:6/669: sync 2024-07-16T09:55:46.957 INFO:tasks.workunit.client.1.smithi176.stdout:4/722: write da/dc/d10/d96/d69/fb0 [2747490,70148] 0 2024-07-16T09:55:46.959 INFO:tasks.workunit.client.0.smithi156.stdout:3/721: mkdir d8/da/d28/d43/d63/df7 0 2024-07-16T09:55:46.964 INFO:tasks.workunit.client.0.smithi156.stdout:9/833: symlink d2/d5/d16/d57/d63/dd3/ddb/l104 0 2024-07-16T09:55:46.964 INFO:tasks.workunit.client.1.smithi176.stdout:0/862: fdatasync d5/d1c/d5a/f75 0 2024-07-16T09:55:46.966 INFO:tasks.workunit.client.0.smithi156.stdout:0/841: chown d0/d1a/d4d/d9/d27/l3f 118497 1 2024-07-16T09:55:46.967 INFO:tasks.workunit.client.0.smithi156.stdout:6/670: rename d3/d13/d17/d4f/c12 to d3/d13/d17/d56/cdc 0 2024-07-16T09:55:46.967 INFO:tasks.workunit.client.0.smithi156.stdout:2/870: creat d0/d13/d17/d1c/d10b/f119 x:0 0 0 2024-07-16T09:55:46.967 INFO:tasks.workunit.client.0.smithi156.stdout:9/834: fdatasync d2/d5/d16/de2/ffa 0 2024-07-16T09:55:46.970 INFO:tasks.workunit.client.0.smithi156.stdout:5/758: rmdir d1b/d1f/d54/d72/dca 0 2024-07-16T09:55:46.971 INFO:tasks.workunit.client.1.smithi176.stdout:4/723: write da/d3d/f1b [2602046,70005] 0 2024-07-16T09:55:46.971 INFO:tasks.workunit.client.1.smithi176.stdout:6/817: link d1/d5d/d9e/dd9/d6f/d7c/leb d1/d5/d47/d62/d77/dde/l12a 0 2024-07-16T09:55:46.971 INFO:tasks.workunit.client.0.smithi156.stdout:3/722: getdents d8/da/d1d/d2c/dee 0 2024-07-16T09:55:46.972 INFO:tasks.workunit.client.1.smithi176.stdout:6/818: chown d1/d5/d47/d62/d77 14 1 2024-07-16T09:55:46.972 INFO:tasks.workunit.client.1.smithi176.stdout:4/724: truncate da/dc/d10/fe1 57322 0 2024-07-16T09:55:46.974 INFO:tasks.workunit.client.0.smithi156.stdout:0/842: unlink d0/d1a/d4d/d9/d68/cd6 0 2024-07-16T09:55:46.976 INFO:tasks.workunit.client.1.smithi176.stdout:4/725: write da/dc/d10/d96/d9b/db4/d40/d5d/d73/f4f [4003826,62337] 0 2024-07-16T09:55:46.980 INFO:tasks.workunit.client.0.smithi156.stdout:1/913: truncate dc/d14/d3e/d55/d83/f108 307576 0 2024-07-16T09:55:46.981 INFO:tasks.workunit.client.1.smithi176.stdout:6/819: dwrite d1/f3b [0,4194304] 0 2024-07-16T09:55:46.981 INFO:tasks.workunit.client.0.smithi156.stdout:7/606: write dc/d13/f1d [3343528,102851] 0 2024-07-16T09:55:46.983 INFO:tasks.workunit.client.0.smithi156.stdout:5/759: rmdir d1b/d1f/d45/d52/d43/d85/d97/d37/d88/d61 39 2024-07-16T09:55:46.985 INFO:tasks.workunit.client.0.smithi156.stdout:8/679: dwrite d0/d11/d20/d3f/f69 [4194304,4194304] 0 2024-07-16T09:55:46.985 INFO:tasks.workunit.client.0.smithi156.stdout:5/760: stat d1b/d1f/d54/dbe/dde 0 2024-07-16T09:55:46.987 INFO:tasks.workunit.client.1.smithi176.stdout:6/820: write d1/d61/ff2 [880221,64410] 0 2024-07-16T09:55:46.992 INFO:tasks.workunit.client.1.smithi176.stdout:0/863: dread d5/d1c/d5a/f6d [4194304,4194304] 0 2024-07-16T09:55:46.992 INFO:tasks.workunit.client.1.smithi176.stdout:6/821: chown d1/fb0 368895 1 2024-07-16T09:55:46.993 INFO:tasks.workunit.client.1.smithi176.stdout:0/864: chown d5/d1c/fd7 45918459 1 2024-07-16T09:55:46.995 INFO:tasks.workunit.client.1.smithi176.stdout:4/726: dread da/dc/d10/d96/faf [0,4194304] 0 2024-07-16T09:55:46.996 INFO:tasks.workunit.client.0.smithi156.stdout:1/914: dwrite f6 [0,4194304] 0 2024-07-16T09:55:46.997 INFO:tasks.workunit.client.1.smithi176.stdout:0/865: creat d5/d1c/d24/d35/d59/f11d x:0 0 0 2024-07-16T09:55:46.997 INFO:tasks.workunit.client.0.smithi156.stdout:4/812: dwrite d4/db/f2c [0,4194304] 0 2024-07-16T09:55:46.998 INFO:tasks.workunit.client.0.smithi156.stdout:2/871: symlink d0/d13/d17/d1c/d86/dde/d109/l11a 0 2024-07-16T09:55:46.999 INFO:tasks.workunit.client.1.smithi176.stdout:4/727: symlink da/dc/d10/d96/lea 0 2024-07-16T09:55:47.000 INFO:tasks.workunit.client.0.smithi156.stdout:1/915: mkdir dc/d12e 0 2024-07-16T09:55:47.001 INFO:tasks.workunit.client.0.smithi156.stdout:3/723: truncate d8/da/d13/f17 1390694 0 2024-07-16T09:55:47.001 INFO:tasks.workunit.client.0.smithi156.stdout:4/813: read - d4/db/dc/dcc/dec/ded/d87/fb9 zero size 2024-07-16T09:55:47.001 INFO:tasks.workunit.client.0.smithi156.stdout:0/843: mkdir d0/d1a/d76/dbc/d10a 0 2024-07-16T09:55:47.002 INFO:tasks.workunit.client.0.smithi156.stdout:1/916: stat dc/d14/d3e/f85 0 2024-07-16T09:55:47.003 INFO:tasks.workunit.client.0.smithi156.stdout:0/844: write d0/d1a/d4d/d60/f108 [810540,118528] 0 2024-07-16T09:55:47.003 INFO:tasks.workunit.client.0.smithi156.stdout:1/917: stat dc/d24/l40 0 2024-07-16T09:55:47.004 INFO:tasks.workunit.client.0.smithi156.stdout:7/607: rename dc/d63/l72 to dc/d13/d15/d8d/lce 0 2024-07-16T09:55:47.007 INFO:tasks.workunit.client.1.smithi176.stdout:4/728: mknod da/dc/d10/d96/d9b/ceb 0 2024-07-16T09:55:47.007 INFO:tasks.workunit.client.1.smithi176.stdout:6/822: dwrite d1/d5/d47/ff5 [4194304,4194304] 0 2024-07-16T09:55:47.008 INFO:tasks.workunit.client.1.smithi176.stdout:9/887: dread d5/d1d/dd2/f75 [4194304,4194304] 0 2024-07-16T09:55:47.013 INFO:tasks.workunit.client.1.smithi176.stdout:9/888: mknod d5/d1d/db0/dd3/c12b 0 2024-07-16T09:55:47.014 INFO:tasks.workunit.client.1.smithi176.stdout:0/866: dwrite d5/d1c/f1f [0,4194304] 0 2024-07-16T09:55:47.015 INFO:tasks.workunit.client.0.smithi156.stdout:8/680: write d0/dd/d1f/d57/fbf [214317,88226] 0 2024-07-16T09:55:47.017 INFO:tasks.workunit.client.0.smithi156.stdout:4/814: mkdir d4/db/dc/dcc/d58/d8f/d11f 0 2024-07-16T09:55:47.019 INFO:tasks.workunit.client.0.smithi156.stdout:4/815: stat d4/db/dc/d67/f92 0 2024-07-16T09:55:47.020 INFO:tasks.workunit.client.1.smithi176.stdout:0/867: mkdir d5/d1c/d24/d35/d50/d7c/d11e 0 2024-07-16T09:55:47.020 INFO:tasks.workunit.client.1.smithi176.stdout:6/823: dwrite d1/d5/d47/d62/d80/fe5 [0,4194304] 0 2024-07-16T09:55:47.021 INFO:tasks.workunit.client.0.smithi156.stdout:5/761: chown d1b/d1f/d45/d52/d43/d85/d97/d37/d88/dbc/dc2 4 1 2024-07-16T09:55:47.022 INFO:tasks.workunit.client.0.smithi156.stdout:6/671: getdents d3/d13/d17/d4f/d30/d45/d99 0 2024-07-16T09:55:47.022 INFO:tasks.workunit.client.1.smithi176.stdout:9/889: creat d5/d1d/f12c x:0 0 0 2024-07-16T09:55:47.024 INFO:tasks.workunit.client.0.smithi156.stdout:5/762: write d1b/d1f/d45/d52/d43/d7c/fc0 [4287559,6910] 0 2024-07-16T09:55:47.029 INFO:tasks.workunit.client.0.smithi156.stdout:8/681: symlink d0/d74/dd3/lf5 0 2024-07-16T09:55:47.030 INFO:tasks.workunit.client.0.smithi156.stdout:6/672: dread - d3/d13/d17/d4f/d94/d19/d31/fa1 zero size 2024-07-16T09:55:47.030 INFO:tasks.workunit.client.0.smithi156.stdout:2/872: rename d0/d13/d17/d1c/d86/fb7 to d0/d13/d17/d1c/d51/dc6/f11b 0 2024-07-16T09:55:47.030 INFO:tasks.workunit.client.0.smithi156.stdout:5/763: readlink d1b/d1f/d45/d52/ld5 0 2024-07-16T09:55:47.030 INFO:tasks.workunit.client.1.smithi176.stdout:9/890: rename d5/d1d/dc7/d89/fef to d5/de/da5/dc1/f12d 0 2024-07-16T09:55:47.031 INFO:tasks.workunit.client.1.smithi176.stdout:6/824: mkdir d1/d5/d47/d62/de8/d111/d122/d12b 0 2024-07-16T09:55:47.031 INFO:tasks.workunit.client.0.smithi156.stdout:7/608: mkdir dc/d48/db2/dcf 0 2024-07-16T09:55:47.032 INFO:tasks.workunit.client.1.smithi176.stdout:0/868: rmdir d5/d39/dde/df0 0 2024-07-16T09:55:47.032 INFO:tasks.workunit.client.0.smithi156.stdout:5/764: symlink d1b/d1f/d42/d6d/d9f/lf9 0 2024-07-16T09:55:47.035 INFO:tasks.workunit.client.0.smithi156.stdout:4/816: mknod d4/db/dc/c120 0 2024-07-16T09:55:47.035 INFO:tasks.workunit.client.0.smithi156.stdout:7/609: readlink dc/d13/d15/l57 0 2024-07-16T09:55:47.036 INFO:tasks.workunit.client.1.smithi176.stdout:0/869: readlink d5/d1c/d24/d35/d50/d9b/de3/d117/l5c 0 2024-07-16T09:55:47.037 INFO:tasks.workunit.client.0.smithi156.stdout:4/817: chown d4/db/d47/c7d 252 1 2024-07-16T09:55:47.037 INFO:tasks.workunit.client.0.smithi156.stdout:9/835: link d2/d5/d16/d57/d63/dd3/ddb/d3d/da6/ld2 d2/d5/d61/da2/d98/dbf/l105 0 2024-07-16T09:55:47.037 INFO:tasks.workunit.client.0.smithi156.stdout:7/610: chown dc/d3e/d3f/d51/fb0 31 1 2024-07-16T09:55:47.037 INFO:tasks.workunit.client.0.smithi156.stdout:3/724: getdents d8/da/d21 0 2024-07-16T09:55:47.038 INFO:tasks.workunit.client.0.smithi156.stdout:0/845: rename d0/d1a/d4d/d9/d27/d3d/d62/f8e to d0/d1a/d76/d92/f10b 0 2024-07-16T09:55:47.038 INFO:tasks.workunit.client.1.smithi176.stdout:9/891: dwrite d5/d1d/db3/fd1 [4194304,4194304] 0 2024-07-16T09:55:47.039 INFO:tasks.workunit.client.1.smithi176.stdout:4/729: write da/dc/d10/d96/d69/f7e [419309,53867] 0 2024-07-16T09:55:47.041 INFO:tasks.workunit.client.1.smithi176.stdout:0/870: write d5/d1c/d24/d35/d50/d58/fb6 [1583579,65734] 0 2024-07-16T09:55:47.041 INFO:tasks.workunit.client.1.smithi176.stdout:9/892: write d5/de/f5b [694488,9069] 0 2024-07-16T09:55:47.042 INFO:tasks.workunit.client.0.smithi156.stdout:8/682: write d0/d11/d62/dc2/dcd/fec [8998195,105509] 0 2024-07-16T09:55:47.042 INFO:tasks.workunit.client.0.smithi156.stdout:3/725: dread - d8/da/d1d/d2c/de0/fb0 zero size 2024-07-16T09:55:47.043 INFO:tasks.workunit.client.0.smithi156.stdout:5/765: mknod d1b/d1f/d45/d7a/d98/cfa 0 2024-07-16T09:55:47.046 INFO:tasks.workunit.client.0.smithi156.stdout:5/766: chown d1b/d1f/d45/d52/d43/d7c 51385113 1 2024-07-16T09:55:47.046 INFO:tasks.workunit.client.1.smithi176.stdout:6/825: dread d1/d5/f3e [0,4194304] 0 2024-07-16T09:55:47.046 INFO:tasks.workunit.client.1.smithi176.stdout:4/730: dread da/dc/d10/d96/faf [0,4194304] 0 2024-07-16T09:55:47.048 INFO:tasks.workunit.client.1.smithi176.stdout:6/826: chown d1/dc/d81/fdd 52334506 1 2024-07-16T09:55:47.048 INFO:tasks.workunit.client.1.smithi176.stdout:0/871: rename d5/d1c/db1/fcd to d5/d1c/d24/d35/d50/d58/f11f 0 2024-07-16T09:55:47.049 INFO:tasks.workunit.client.0.smithi156.stdout:8/683: readlink d0/la 0 2024-07-16T09:55:47.050 INFO:tasks.workunit.client.0.smithi156.stdout:5/767: write d1b/d1f/d45/d52/d43/f8d [3782050,124405] 0 2024-07-16T09:55:47.053 INFO:tasks.workunit.client.0.smithi156.stdout:1/918: dwrite f4 [0,4194304] 0 2024-07-16T09:55:47.053 INFO:tasks.workunit.client.1.smithi176.stdout:9/893: rename d5/d1d/d20/d78/dd0 to d5/d1d/dc7/d89/d12e 0 2024-07-16T09:55:47.054 INFO:tasks.workunit.client.1.smithi176.stdout:0/872: dread d5/d1c/d24/d35/d50/d58/fb6 [0,4194304] 0 2024-07-16T09:55:47.055 INFO:tasks.workunit.client.0.smithi156.stdout:0/846: creat d0/d1a/d76/dbc/f10c x:0 0 0 2024-07-16T09:55:47.055 INFO:tasks.workunit.client.1.smithi176.stdout:6/827: mkdir d1/d5d/db9/d101/d12c 0 2024-07-16T09:55:47.055 INFO:tasks.workunit.client.1.smithi176.stdout:0/873: stat d5/d1c/d24/d35/d50/dd5/ff6 0 2024-07-16T09:55:47.058 INFO:tasks.workunit.client.0.smithi156.stdout:0/847: write d0/d1a/d4d/d9/fc7 [673541,84867] 0 2024-07-16T09:55:47.060 INFO:tasks.workunit.client.1.smithi176.stdout:4/731: mknod da/d3d/d17/d54/d78/d7a/cec 0 2024-07-16T09:55:47.061 INFO:tasks.workunit.client.0.smithi156.stdout:3/726: mkdir d8/da/d1d/d2c/de0/df8 0 2024-07-16T09:55:47.062 INFO:tasks.workunit.client.1.smithi176.stdout:4/732: chown da/d41/dc3/f65 7573 1 2024-07-16T09:55:47.064 INFO:tasks.workunit.client.1.smithi176.stdout:4/733: chown da/dc/d4b/c4c 1 1 2024-07-16T09:55:47.064 INFO:tasks.workunit.client.0.smithi156.stdout:5/768: fsync d1b/d1f/d8a/fa6 0 2024-07-16T09:55:47.064 INFO:tasks.workunit.client.1.smithi176.stdout:0/874: dwrite d5/d1c/d24/ddf/f105 [0,4194304] 0 2024-07-16T09:55:47.065 INFO:tasks.workunit.client.0.smithi156.stdout:2/873: getdents d0/d9b 0 2024-07-16T09:55:47.065 INFO:tasks.workunit.client.1.smithi176.stdout:9/894: symlink d5/d1d/dd2/de3/d103/d125/l12f 0 2024-07-16T09:55:47.066 INFO:tasks.workunit.client.0.smithi156.stdout:3/727: chown d8/d14/c22 263211 1 2024-07-16T09:55:47.067 INFO:tasks.workunit.client.1.smithi176.stdout:9/895: chown d5/d1d/d20/d78/f8f 39165796 1 2024-07-16T09:55:47.067 INFO:tasks.workunit.client.1.smithi176.stdout:6/828: link d1/d5d/d9e/dd9/d6f/d7c/f90 d1/d5d/d9e/dd9/d6f/dd0/f12d 0 2024-07-16T09:55:47.068 INFO:tasks.workunit.client.0.smithi156.stdout:7/611: dwrite dc/f14 [0,4194304] 0 2024-07-16T09:55:47.068 INFO:tasks.workunit.client.1.smithi176.stdout:6/829: readlink d1/d5/d47/lf1 0 2024-07-16T09:55:47.069 INFO:tasks.workunit.client.1.smithi176.stdout:0/875: rename d5/d39/f52 to d5/d1c/d24/ddf/f120 0 2024-07-16T09:55:47.071 INFO:tasks.workunit.client.1.smithi176.stdout:0/876: read d5/d2c/d6c/f5e [103403,87999] 0 2024-07-16T09:55:47.073 INFO:tasks.workunit.client.0.smithi156.stdout:1/919: dwrite dc/d6b/dd5/ff7 [0,4194304] 0 2024-07-16T09:55:47.073 INFO:tasks.workunit.client.1.smithi176.stdout:4/734: symlink da/de5/led 0 2024-07-16T09:55:47.075 INFO:tasks.workunit.client.1.smithi176.stdout:9/896: creat d5/d1d/d67/d6d/f130 x:0 0 0 2024-07-16T09:55:47.076 INFO:tasks.workunit.client.0.smithi156.stdout:7/612: chown dc/d48/db2 1 1 2024-07-16T09:55:47.076 INFO:tasks.workunit.client.0.smithi156.stdout:3/728: symlink d8/da/d1d/d2c/de0/lf9 0 2024-07-16T09:55:47.076 INFO:tasks.workunit.client.1.smithi176.stdout:6/830: mkdir d1/d5/d47/d62/de8/d111/d122/d12e 0 2024-07-16T09:55:47.078 INFO:tasks.workunit.client.0.smithi156.stdout:6/673: link d3/d13/d17/d4f/d94/d19/d31/d35/c7a d3/d13/d17/d4f/d94/d19/d31/d35/cdd 0 2024-07-16T09:55:47.078 INFO:tasks.workunit.client.0.smithi156.stdout:0/848: write d0/d1a/d4d/d60/ffa [3504493,2400] 0 2024-07-16T09:55:47.081 INFO:tasks.workunit.client.1.smithi176.stdout:1/514: sync 2024-07-16T09:55:47.081 INFO:tasks.workunit.client.0.smithi156.stdout:9/836: getdents d2/d5/d16/d57/d63/dd3 0 2024-07-16T09:55:47.082 INFO:tasks.workunit.client.1.smithi176.stdout:1/515: chown d4/d8/d12/d21/d34 125606 1 2024-07-16T09:55:47.085 INFO:tasks.workunit.client.1.smithi176.stdout:4/735: mknod da/dc/d10/d96/d9b/db4/d4a/dc1/cee 0 2024-07-16T09:55:47.085 INFO:tasks.workunit.client.1.smithi176.stdout:6/831: mknod d1/d5/d47/d62/c12f 0 2024-07-16T09:55:47.087 INFO:tasks.workunit.client.0.smithi156.stdout:1/920: mkdir dc/d14/d3e/d6c/d12f 0 2024-07-16T09:55:47.087 INFO:tasks.workunit.client.1.smithi176.stdout:1/516: truncate d4/db/f43 2287699 0 2024-07-16T09:55:47.087 INFO:tasks.workunit.client.0.smithi156.stdout:4/818: dwrite d4/db/dc/dcc/dec/ded/daf/fd4 [4194304,4194304] 0 2024-07-16T09:55:47.091 INFO:tasks.workunit.client.0.smithi156.stdout:2/874: write d0/d13/d17/f30 [1481789,115450] 0 2024-07-16T09:55:47.091 INFO:tasks.workunit.client.0.smithi156.stdout:0/849: write d0/d1a/d4d/d9/f8f [442382,122439] 0 2024-07-16T09:55:47.093 INFO:tasks.workunit.client.0.smithi156.stdout:8/684: truncate d0/d11/d62/fa4 684025 0 2024-07-16T09:55:47.093 INFO:tasks.workunit.client.1.smithi176.stdout:4/736: truncate da/dc/d10/d96/d9b/db4/d40/f92 917830 0 2024-07-16T09:55:47.095 INFO:tasks.workunit.client.0.smithi156.stdout:4/819: dread - d4/d23/d30/d33/f83 zero size 2024-07-16T09:55:47.096 INFO:tasks.workunit.client.0.smithi156.stdout:3/729: dwrite d8/da/dec/df1/fd3 [0,4194304] 0 2024-07-16T09:55:47.096 INFO:tasks.workunit.client.0.smithi156.stdout:7/613: mkdir dc/d13/d15/d8d/dd0 0 2024-07-16T09:55:47.098 INFO:tasks.workunit.client.0.smithi156.stdout:2/875: stat d0/d5/d76/da1/la8 0 2024-07-16T09:55:47.098 INFO:tasks.workunit.client.1.smithi176.stdout:4/737: mknod da/dc/d10/d96/d69/cef 0 2024-07-16T09:55:47.100 INFO:tasks.workunit.client.1.smithi176.stdout:4/738: chown da/dc/f83 911430731 1 2024-07-16T09:55:47.100 INFO:tasks.workunit.client.0.smithi156.stdout:1/921: mkdir dc/d14/d1e/d130 0 2024-07-16T09:55:47.101 INFO:tasks.workunit.client.0.smithi156.stdout:0/850: symlink d0/d1a/d76/d92/l10d 0 2024-07-16T09:55:47.103 INFO:tasks.workunit.client.0.smithi156.stdout:8/685: symlink d0/d11/d29/d80/d97/de7/d93/lf6 0 2024-07-16T09:55:47.103 INFO:tasks.workunit.client.0.smithi156.stdout:0/851: read - d0/d1a/d76/dbc/f10c zero size 2024-07-16T09:55:47.104 INFO:tasks.workunit.client.0.smithi156.stdout:1/922: symlink dc/d14/d1e/l131 0 2024-07-16T09:55:47.104 INFO:tasks.workunit.client.0.smithi156.stdout:7/614: creat dc/d13/d20/d4d/fd1 x:0 0 0 2024-07-16T09:55:47.106 INFO:tasks.workunit.client.0.smithi156.stdout:8/686: dread - d0/d11/d20/d3f/fe9 zero size 2024-07-16T09:55:47.107 INFO:tasks.workunit.client.1.smithi176.stdout:9/897: dread d5/d52/fa4 [0,4194304] 0 2024-07-16T09:55:47.108 INFO:tasks.workunit.client.0.smithi156.stdout:9/837: rename d2/d5/d16/d57/d63/dd3/ddb/d74/dc2/dc4/d66/cce to d2/d5/d16/d57/da3/c106 0 2024-07-16T09:55:47.109 INFO:tasks.workunit.client.1.smithi176.stdout:4/739: dwrite da/d3d/d33/fdd [0,4194304] 0 2024-07-16T09:55:47.110 INFO:tasks.workunit.client.0.smithi156.stdout:4/820: truncate d4/db/dc/dcc/dec/ded/d87/fa0 4407350 0 2024-07-16T09:55:47.110 INFO:tasks.workunit.client.0.smithi156.stdout:8/687: truncate d0/dd/d1f/fc6 914467 0 2024-07-16T09:55:47.110 INFO:tasks.workunit.client.0.smithi156.stdout:5/769: getdents d1b/d1f/d45/d52/d43/d85/d97/d37/d88/dc7 0 2024-07-16T09:55:47.111 INFO:tasks.workunit.client.0.smithi156.stdout:2/876: dwrite d0/d5/d29/d2d/d5b/d5f/f7e [0,4194304] 0 2024-07-16T09:55:47.112 INFO:tasks.workunit.client.1.smithi176.stdout:1/517: dread d4/f17 [0,4194304] 0 2024-07-16T09:55:47.112 INFO:tasks.workunit.client.0.smithi156.stdout:0/852: symlink d0/d1a/d4d/d61/d97/de1/l10e 0 2024-07-16T09:55:47.113 INFO:tasks.workunit.client.0.smithi156.stdout:7/615: symlink dc/d13/d15/d8b/ld2 0 2024-07-16T09:55:47.114 INFO:tasks.workunit.client.0.smithi156.stdout:2/877: stat d0/d5/d8b 0 2024-07-16T09:55:47.115 INFO:tasks.workunit.client.1.smithi176.stdout:9/898: mkdir d5/d1d/db3/d11f/d131 0 2024-07-16T09:55:47.115 INFO:tasks.workunit.client.0.smithi156.stdout:3/730: creat d8/da/d28/ffa x:0 0 0 2024-07-16T09:55:47.116 INFO:tasks.workunit.client.0.smithi156.stdout:5/770: unlink d1b/f40 0 2024-07-16T09:55:47.119 INFO:tasks.workunit.client.0.smithi156.stdout:0/853: chown d0/d1a/d4d/d61/d97/de1/de7/l107 2020181 1 2024-07-16T09:55:47.122 INFO:tasks.workunit.client.0.smithi156.stdout:7/616: truncate dc/d13/d36/f66 2476722 0 2024-07-16T09:55:47.123 INFO:tasks.workunit.client.0.smithi156.stdout:9/838: dread - d2/d5/d16/d57/d63/dd3/ddb/d74/dc2/dc4/fc8 zero size 2024-07-16T09:55:47.123 INFO:tasks.workunit.client.0.smithi156.stdout:8/688: mkdir d0/dd/d1f/d57/db3/df7 0 2024-07-16T09:55:47.124 INFO:tasks.workunit.client.0.smithi156.stdout:3/731: symlink d8/da/d28/d43/lfb 0 2024-07-16T09:55:47.124 INFO:tasks.workunit.client.1.smithi176.stdout:1/518: dwrite d4/d3a/d54/f6d [8388608,4194304] 0 2024-07-16T09:55:47.125 INFO:tasks.workunit.client.1.smithi176.stdout:9/899: getdents d5/de/da5/dbf 0 2024-07-16T09:55:47.126 INFO:tasks.workunit.client.1.smithi176.stdout:9/900: chown d5/d1d/d67/fab 12369 1 2024-07-16T09:55:47.127 INFO:tasks.workunit.client.1.smithi176.stdout:0/877: write d5/f1e [4274994,48404] 0 2024-07-16T09:55:47.127 INFO:tasks.workunit.client.1.smithi176.stdout:1/519: read d4/d77/dad/d72/f74 [598248,81584] 0 2024-07-16T09:55:47.128 INFO:tasks.workunit.client.0.smithi156.stdout:5/771: mkdir d1b/d1f/d45/dfb 0 2024-07-16T09:55:47.130 INFO:tasks.workunit.client.1.smithi176.stdout:9/901: rmdir d5/d1d/db0/dd3 39 2024-07-16T09:55:47.131 INFO:tasks.workunit.client.1.smithi176.stdout:9/902: write d5/d1d/d67/d6d/f130 [550678,36639] 0 2024-07-16T09:55:47.132 INFO:tasks.workunit.client.1.smithi176.stdout:9/903: dread - d5/de/da5/dc1/d112/d97/f126 zero size 2024-07-16T09:55:47.133 INFO:tasks.workunit.client.0.smithi156.stdout:0/854: rename d0/d1a/d4d/d9/d27/cc6 to d0/d1a/d4d/d9/d27/d2c/d6d/d9d/dbb/c10f 0 2024-07-16T09:55:47.133 INFO:tasks.workunit.client.0.smithi156.stdout:2/878: rmdir d0/d13/dc9 39 2024-07-16T09:55:47.137 INFO:tasks.workunit.client.0.smithi156.stdout:8/689: symlink d0/d11/d20/lf8 0 2024-07-16T09:55:47.137 INFO:tasks.workunit.client.0.smithi156.stdout:7/617: dwrite dc/d13/d36/f66 [0,4194304] 0 2024-07-16T09:55:47.141 INFO:tasks.workunit.client.1.smithi176.stdout:6/832: dwrite d1/dc/d40/f4d [4194304,4194304] 0 2024-07-16T09:55:47.141 INFO:tasks.workunit.client.0.smithi156.stdout:9/839: write d2/d5/d61/da2/d98/fc6 [4030831,12557] 0 2024-07-16T09:55:47.143 INFO:tasks.workunit.client.0.smithi156.stdout:0/855: mkdir d0/d1a/d76/d92/d110 0 2024-07-16T09:55:47.147 INFO:tasks.workunit.client.0.smithi156.stdout:7/618: read dc/d13/d36/f6a [2251957,1662] 0 2024-07-16T09:55:47.148 INFO:tasks.workunit.client.1.smithi176.stdout:0/878: write d5/d1c/d24/d35/d50/d58/d63/db4/dcc/ff5 [4379411,53936] 0 2024-07-16T09:55:47.148 INFO:tasks.workunit.client.0.smithi156.stdout:1/923: truncate dc/d14/d8b/dcd/f11b 1426649 0 2024-07-16T09:55:47.149 INFO:tasks.workunit.client.0.smithi156.stdout:6/674: dread d3/d13/d17/d4f/d30/fa3 [0,4194304] 0 2024-07-16T09:55:47.150 INFO:tasks.workunit.client.0.smithi156.stdout:2/879: dwrite d0/d5/d76/da1/fd6 [0,4194304] 0 2024-07-16T09:55:47.153 INFO:tasks.workunit.client.0.smithi156.stdout:3/732: dread d8/da/f3d [0,4194304] 0 2024-07-16T09:55:47.153 INFO:tasks.workunit.client.0.smithi156.stdout:0/856: mknod d0/d1a/d4d/d61/da1/c111 0 2024-07-16T09:55:47.154 INFO:tasks.workunit.client.1.smithi176.stdout:6/833: creat d1/d5d/d9e/dd9/f130 x:0 0 0 2024-07-16T09:55:47.154 INFO:tasks.workunit.client.0.smithi156.stdout:5/772: dread d1b/d1f/d45/d52/d2d/d96/fd8 [0,4194304] 0 2024-07-16T09:55:47.156 INFO:tasks.workunit.client.1.smithi176.stdout:0/879: creat d5/d2c/d6c/d83/f121 x:0 0 0 2024-07-16T09:55:47.156 INFO:tasks.workunit.client.1.smithi176.stdout:4/740: dread da/dc/f58 [0,4194304] 0 2024-07-16T09:55:47.157 INFO:tasks.workunit.client.0.smithi156.stdout:4/821: write d4/d23/d30/fd6 [24322,31371] 0 2024-07-16T09:55:47.157 INFO:tasks.workunit.client.1.smithi176.stdout:4/741: chown l9 707213 1 2024-07-16T09:55:47.157 INFO:tasks.workunit.client.1.smithi176.stdout:6/834: fsync d1/d5d/d9e/dd9/d6f/d7c/f90 0 2024-07-16T09:55:47.158 INFO:tasks.workunit.client.0.smithi156.stdout:6/675: rmdir d3/d13/d17 39 2024-07-16T09:55:47.159 INFO:tasks.workunit.client.0.smithi156.stdout:8/690: write d0/d11/d62/fc5 [3069773,129669] 0 2024-07-16T09:55:47.159 INFO:tasks.workunit.client.1.smithi176.stdout:0/880: creat d5/d1c/d24/dfe/f122 x:0 0 0 2024-07-16T09:55:47.161 INFO:tasks.workunit.client.0.smithi156.stdout:7/619: unlink dc/d13/d15/d8b/ld2 0 2024-07-16T09:55:47.162 INFO:tasks.workunit.client.1.smithi176.stdout:0/881: write d5/d1c/db1/ff3 [790055,39773] 0 2024-07-16T09:55:47.162 INFO:tasks.workunit.client.1.smithi176.stdout:6/835: dread - d1/fdb zero size 2024-07-16T09:55:47.162 INFO:tasks.workunit.client.1.smithi176.stdout:0/882: stat d5/fb 0 2024-07-16T09:55:47.163 INFO:tasks.workunit.client.1.smithi176.stdout:0/883: chown d5/f10 809 1 2024-07-16T09:55:47.163 INFO:tasks.workunit.client.0.smithi156.stdout:1/924: creat dc/d14/d3e/d6c/d7a/f132 x:0 0 0 2024-07-16T09:55:47.163 INFO:tasks.workunit.client.1.smithi176.stdout:4/742: rmdir da/dc/d10/d96/d9b/db4/d40 39 2024-07-16T09:55:47.164 INFO:tasks.workunit.client.0.smithi156.stdout:0/857: symlink d0/d1a/d4d/d9/d68/dff/l112 0 2024-07-16T09:55:47.164 INFO:tasks.workunit.client.0.smithi156.stdout:5/773: fsync d1b/d1f/d45/d52/d43/d85/d97/d37/f5a 0 2024-07-16T09:55:47.164 INFO:tasks.workunit.client.1.smithi176.stdout:6/836: creat d1/dc/d5b/d11d/f131 x:0 0 0 2024-07-16T09:55:47.165 INFO:tasks.workunit.client.0.smithi156.stdout:8/691: symlink d0/dd/d1f/d57/da6/lf9 0 2024-07-16T09:55:47.166 INFO:tasks.workunit.client.1.smithi176.stdout:4/743: chown da/dc/d10/d96/d9b/db4/d4a/dc1/cee 31 1 2024-07-16T09:55:47.167 INFO:tasks.workunit.client.0.smithi156.stdout:9/840: creat d2/d5/d16/d57/d63/dd3/ddb/d3d/d3c/f107 x:0 0 0 2024-07-16T09:55:47.167 INFO:tasks.workunit.client.1.smithi176.stdout:4/744: readlink da/de5/led 0 2024-07-16T09:55:47.167 INFO:tasks.workunit.client.0.smithi156.stdout:8/692: write d0/d11/d2e/fda [187207,12949] 0 2024-07-16T09:55:47.168 INFO:tasks.workunit.client.1.smithi176.stdout:6/837: creat d1/d5d/db9/d101/f132 x:0 0 0 2024-07-16T09:55:47.170 INFO:tasks.workunit.client.0.smithi156.stdout:1/925: mkdir dc/d14/d133 0 2024-07-16T09:55:47.170 INFO:tasks.workunit.client.1.smithi176.stdout:6/838: chown d1/d5/d47/cf7 41062 1 2024-07-16T09:55:47.171 INFO:tasks.workunit.client.0.smithi156.stdout:4/822: creat d4/d23/f121 x:0 0 0 2024-07-16T09:55:47.171 INFO:tasks.workunit.client.1.smithi176.stdout:6/839: chown d1/dc/d40/d71/ce2 163191379 1 2024-07-16T09:55:47.171 INFO:tasks.workunit.client.0.smithi156.stdout:3/733: creat d8/da/d1d/d2c/ffc x:0 0 0 2024-07-16T09:55:47.172 INFO:tasks.workunit.client.1.smithi176.stdout:4/745: unlink da/d3d/l22 0 2024-07-16T09:55:47.175 INFO:tasks.workunit.client.1.smithi176.stdout:1/520: dread d4/d8/f26 [0,4194304] 0 2024-07-16T09:55:47.175 INFO:tasks.workunit.client.0.smithi156.stdout:3/734: readlink d8/da/d1d/db2/lbb 0 2024-07-16T09:55:47.176 INFO:tasks.workunit.client.0.smithi156.stdout:4/823: dread - d4/d23/f121 zero size 2024-07-16T09:55:47.177 INFO:tasks.workunit.client.1.smithi176.stdout:6/840: dread d1/d5/f30 [0,4194304] 0 2024-07-16T09:55:47.179 INFO:tasks.workunit.client.1.smithi176.stdout:1/521: mknod d4/d8/d12/d21/d25/d30/d44/cb1 0 2024-07-16T09:55:47.180 INFO:tasks.workunit.client.1.smithi176.stdout:6/841: chown d1/dc/d5b/f6d 4006159 1 2024-07-16T09:55:47.180 INFO:tasks.workunit.client.0.smithi156.stdout:2/880: getdents d0/d13/d17/d1c 0 2024-07-16T09:55:47.180 INFO:tasks.workunit.client.0.smithi156.stdout:1/926: dread dc/d14/d3e/d6c/d7a/dcb/fe1 [0,4194304] 0 2024-07-16T09:55:47.181 INFO:tasks.workunit.client.0.smithi156.stdout:7/620: getdents dc/d13 0 2024-07-16T09:55:47.181 INFO:tasks.workunit.client.1.smithi176.stdout:6/842: truncate d1/dc/fab 279573 0 2024-07-16T09:55:47.182 INFO:tasks.workunit.client.1.smithi176.stdout:1/522: chown d4/d8/d12/d21/d25/d30/d44/d4d/f96 1708550 1 2024-07-16T09:55:47.182 INFO:tasks.workunit.client.1.smithi176.stdout:6/843: chown d1/d5/d47/d62/c12f 0 1 2024-07-16T09:55:47.183 INFO:tasks.workunit.client.1.smithi176.stdout:0/884: write d5/d1c/d24/d35/d50/d9b/de3/d117/fb2 [1071990,77602] 0 2024-07-16T09:55:47.183 INFO:tasks.workunit.client.0.smithi156.stdout:0/858: dread d0/d22/f3c [0,4194304] 0 2024-07-16T09:55:47.183 INFO:tasks.workunit.client.0.smithi156.stdout:3/735: write d8/da/d28/d43/d63/fbd [885687,62481] 0 2024-07-16T09:55:47.184 INFO:tasks.workunit.client.0.smithi156.stdout:7/621: fdatasync dc/d13/d15/f78 0 2024-07-16T09:55:47.185 INFO:tasks.workunit.client.1.smithi176.stdout:4/746: rmdir da/d3d 39 2024-07-16T09:55:47.187 INFO:tasks.workunit.client.0.smithi156.stdout:9/841: creat d2/d5/d16/d57/d63/dd3/ddb/d74/dc2/dc4/d3a/de7/f108 x:0 0 0 2024-07-16T09:55:47.189 INFO:tasks.workunit.client.1.smithi176.stdout:0/885: symlink d5/d2c/d6c/d83/d84/l123 0 2024-07-16T09:55:47.190 INFO:tasks.workunit.client.0.smithi156.stdout:0/859: mkdir d0/d1a/d4d/d5e/db7/d113 0 2024-07-16T09:55:47.190 INFO:tasks.workunit.client.0.smithi156.stdout:2/881: mknod d0/d5/d29/d2d/de1/c11c 0 2024-07-16T09:55:47.192 INFO:tasks.workunit.client.1.smithi176.stdout:4/747: rename da/dc/d10 to da/dc/d4b/df0 0 2024-07-16T09:55:47.193 INFO:tasks.workunit.client.1.smithi176.stdout:4/748: fsync da/f47 0 2024-07-16T09:55:47.197 INFO:tasks.workunit.client.1.smithi176.stdout:6/844: dwrite d1/d5/d7/f63 [0,4194304] 0 2024-07-16T09:55:47.197 INFO:tasks.workunit.client.1.smithi176.stdout:0/886: mkdir d5/d25/d82/dec/d124 0 2024-07-16T09:55:47.198 INFO:tasks.workunit.client.1.smithi176.stdout:6/845: chown d1/d5d/le1 968 1 2024-07-16T09:55:47.199 INFO:tasks.workunit.client.0.smithi156.stdout:2/882: write d0/d13/d17/fdf [269912,16963] 0 2024-07-16T09:55:47.199 INFO:tasks.workunit.client.0.smithi156.stdout:5/774: dread d1b/f2f [0,4194304] 0 2024-07-16T09:55:47.199 INFO:tasks.workunit.client.0.smithi156.stdout:9/842: chown d2/d5/d61/da2/d98/fa9 384 1 2024-07-16T09:55:47.200 INFO:tasks.workunit.client.0.smithi156.stdout:3/736: chown d8/da/d28/d43/d71/da2/le7 0 1 2024-07-16T09:55:47.200 INFO:tasks.workunit.client.1.smithi176.stdout:4/749: mknod da/d41/d5b/cf1 0 2024-07-16T09:55:47.202 INFO:tasks.workunit.client.1.smithi176.stdout:1/523: truncate d4/d8/f69 84952 0 2024-07-16T09:55:47.203 INFO:tasks.workunit.client.0.smithi156.stdout:1/927: creat dc/d6b/f134 x:0 0 0 2024-07-16T09:55:47.204 INFO:tasks.workunit.client.0.smithi156.stdout:6/676: dread d3/f1c [0,4194304] 0 2024-07-16T09:55:47.204 INFO:tasks.workunit.client.1.smithi176.stdout:0/887: creat d5/d1c/d24/d35/d50/d9b/de3/f125 x:0 0 0 2024-07-16T09:55:47.205 INFO:tasks.workunit.client.0.smithi156.stdout:4/824: dwrite d4/db/dc/d5f/fb3 [0,4194304] 0 2024-07-16T09:55:47.205 INFO:tasks.workunit.client.1.smithi176.stdout:0/888: dread - d5/d1c/f106 zero size 2024-07-16T09:55:47.205 INFO:tasks.workunit.client.0.smithi156.stdout:2/883: chown d0/d13/d17/d1c/d51/fc2 14 1 2024-07-16T09:55:47.206 INFO:tasks.workunit.client.1.smithi176.stdout:6/846: rename d1/dc/d5b/f9d to d1/d5/d47/d62/d77/dde/f133 0 2024-07-16T09:55:47.207 INFO:tasks.workunit.client.0.smithi156.stdout:9/843: mkdir d2/d9b/d109 0 2024-07-16T09:55:47.207 INFO:tasks.workunit.client.1.smithi176.stdout:6/847: fsync d1/dc/d40/fca 0 2024-07-16T09:55:47.207 INFO:tasks.workunit.client.1.smithi176.stdout:4/750: mknod da/dc/d4b/df0/d28/cf2 0 2024-07-16T09:55:47.209 INFO:tasks.workunit.client.1.smithi176.stdout:1/524: rmdir d4/d3a/d54 39 2024-07-16T09:55:47.209 INFO:tasks.workunit.client.1.smithi176.stdout:1/525: stat d4/f24 0 2024-07-16T09:55:47.214 INFO:tasks.workunit.client.1.smithi176.stdout:0/889: mknod d5/d25/d82/c126 0 2024-07-16T09:55:47.216 INFO:tasks.workunit.client.1.smithi176.stdout:6/848: read d1/d5d/d9e/d76/f5c [159303,55144] 0 2024-07-16T09:55:47.217 INFO:tasks.workunit.client.1.smithi176.stdout:0/890: read - d5/d1c/d24/d35/d50/d9b/de3/d117/dbd/de6/df8/ff1 zero size 2024-07-16T09:55:47.217 INFO:tasks.workunit.client.0.smithi156.stdout:8/693: dread d0/dd/fb5 [0,4194304] 0 2024-07-16T09:55:47.218 INFO:tasks.workunit.client.1.smithi176.stdout:1/526: creat d4/d8/d12/d21/d25/da8/fb2 x:0 0 0 2024-07-16T09:55:47.219 INFO:tasks.workunit.client.0.smithi156.stdout:0/860: truncate d0/d1a/d4d/d9/d27/d50/f5f 7640348 0 2024-07-16T09:55:47.219 INFO:tasks.workunit.client.0.smithi156.stdout:7/622: dwrite dc/d3e/d3f/d45/f7d [0,4194304] 0 2024-07-16T09:55:47.219 INFO:tasks.workunit.client.1.smithi176.stdout:1/527: dread - d4/db/f5c zero size 2024-07-16T09:55:47.219 INFO:tasks.workunit.client.0.smithi156.stdout:3/737: truncate d8/da/d1d/fc0 1217152 0 2024-07-16T09:55:47.220 INFO:tasks.workunit.client.1.smithi176.stdout:1/528: dread - d4/db/f5c zero size 2024-07-16T09:55:47.224 INFO:tasks.workunit.client.0.smithi156.stdout:8/694: chown d0/d11/d13/c49 221460 1 2024-07-16T09:55:47.224 INFO:tasks.workunit.client.0.smithi156.stdout:4/825: creat d4/db/dc/dcc/dec/ded/daf/f122 x:0 0 0 2024-07-16T09:55:47.226 INFO:tasks.workunit.client.0.smithi156.stdout:9/844: chown d2/d5/d16/d57/d63/dd3/ddb/d74/dc2/dc4/d3a/de7 1125 1 2024-07-16T09:55:47.227 INFO:tasks.workunit.client.0.smithi156.stdout:7/623: creat dc/d63/d98/fd3 x:0 0 0 2024-07-16T09:55:47.230 INFO:tasks.workunit.client.1.smithi176.stdout:6/849: creat d1/dc/dd8/f134 x:0 0 0 2024-07-16T09:55:47.230 INFO:tasks.workunit.client.0.smithi156.stdout:5/775: dwrite d1b/d99/dae/fd4 [0,4194304] 0 2024-07-16T09:55:47.231 INFO:tasks.workunit.client.0.smithi156.stdout:2/884: creat d0/d5/d29/d2d/d5b/ddb/df9/f11d x:0 0 0 2024-07-16T09:55:47.232 INFO:tasks.workunit.client.0.smithi156.stdout:3/738: symlink d8/da/lfd 0 2024-07-16T09:55:47.233 INFO:tasks.workunit.client.0.smithi156.stdout:7/624: mkdir dc/d13/d36/dd4 0 2024-07-16T09:55:47.237 INFO:tasks.workunit.client.0.smithi156.stdout:5/776: mknod d1b/d1f/d54/dc6/cfc 0 2024-07-16T09:55:47.238 INFO:tasks.workunit.client.0.smithi156.stdout:7/625: chown dc/c32 77993802 1 2024-07-16T09:55:47.238 INFO:tasks.workunit.client.1.smithi176.stdout:6/850: symlink d1/dc/d81/d10a/d115/l135 0 2024-07-16T09:55:47.238 INFO:tasks.workunit.client.0.smithi156.stdout:4/826: rename d4/db/dc/f88 to d4/d23/d30/f123 0 2024-07-16T09:55:47.239 INFO:tasks.workunit.client.1.smithi176.stdout:1/529: getdents d4/d3a/d8e 0 2024-07-16T09:55:47.243 INFO:tasks.workunit.client.0.smithi156.stdout:3/739: creat d8/ffe x:0 0 0 2024-07-16T09:55:47.246 INFO:tasks.workunit.client.1.smithi176.stdout:0/891: rename d5/d2c/d6c/d83 to d5/d1c/d24/d127 0 2024-07-16T09:55:47.246 INFO:tasks.workunit.client.1.smithi176.stdout:4/751: link da/d3d/d33/c4d da/dc/d4b/df0/d96/d9b/db4/d40/d5d/d73/cf3 0 2024-07-16T09:55:47.249 INFO:tasks.workunit.client.1.smithi176.stdout:6/851: rename d1/d5d/l39 to d1/d5d/d9e/d8f/l136 0 2024-07-16T09:55:47.251 INFO:tasks.workunit.client.1.smithi176.stdout:4/752: mkdir da/d41/d5b/df4 0 2024-07-16T09:55:47.251 INFO:tasks.workunit.client.1.smithi176.stdout:0/892: dread d5/d2c/d6c/f64 [0,4194304] 0 2024-07-16T09:55:47.251 INFO:tasks.workunit.client.0.smithi156.stdout:5/777: rmdir d1b/d1f/d54/dc6/dcc 0 2024-07-16T09:55:47.252 INFO:tasks.workunit.client.0.smithi156.stdout:9/845: dwrite d2/d5/d16/d57/da3/ddd/f102 [0,4194304] 0 2024-07-16T09:55:47.253 INFO:tasks.workunit.client.1.smithi176.stdout:0/893: chown d5/d1c/d24/d35/d50/d9b/de3/d117/dbd/de6/df8 729713 1 2024-07-16T09:55:47.253 INFO:tasks.workunit.client.1.smithi176.stdout:6/852: rmdir d1/d5/d47/d62/d77 39 2024-07-16T09:55:47.255 INFO:tasks.workunit.client.0.smithi156.stdout:0/861: dread d0/d1a/d4d/d9/d27/d50/fbe [0,4194304] 0 2024-07-16T09:55:47.255 INFO:tasks.workunit.client.0.smithi156.stdout:3/740: chown d8/da/d1d/c31 777 1 2024-07-16T09:55:47.256 INFO:tasks.workunit.client.0.smithi156.stdout:6/677: write d3/d13/d17/d4f/d4d/d3b/d5a/f69 [690224,78291] 0 2024-07-16T09:55:47.258 INFO:tasks.workunit.client.0.smithi156.stdout:1/928: getdents dc/d14/d8b/dcd/dd2/df9 0 2024-07-16T09:55:47.260 INFO:tasks.workunit.client.1.smithi176.stdout:0/894: dwrite d5/d25/d82/dec/f10a [0,4194304] 0 2024-07-16T09:55:47.260 INFO:tasks.workunit.client.0.smithi156.stdout:8/695: dread d0/dd/fa7 [0,4194304] 0 2024-07-16T09:55:47.261 INFO:tasks.workunit.client.1.smithi176.stdout:0/895: stat d5/d44/lb7 0 2024-07-16T09:55:47.261 INFO:tasks.workunit.client.0.smithi156.stdout:2/885: dread d0/d5/d29/d2d/d5b/d5f/d89/f99 [0,4194304] 0 2024-07-16T09:55:47.262 INFO:tasks.workunit.client.0.smithi156.stdout:7/626: dwrite dc/d3e/d3f/d45/f4f [0,4194304] 0 2024-07-16T09:55:47.264 INFO:tasks.workunit.client.0.smithi156.stdout:9/846: chown d2/d5/d16/d57/d63/dd3/ddb/d74/fbe 19423091 1 2024-07-16T09:55:47.265 INFO:tasks.workunit.client.0.smithi156.stdout:9/847: fdatasync d2/d5/f70 0 2024-07-16T09:55:47.268 INFO:tasks.workunit.client.0.smithi156.stdout:4/827: dread d4/db/dc/d67/f86 [0,4194304] 0 2024-07-16T09:55:47.268 INFO:tasks.workunit.client.0.smithi156.stdout:6/678: dread - d3/d13/d17/d4f/d94/f9f zero size 2024-07-16T09:55:47.269 INFO:tasks.workunit.client.0.smithi156.stdout:1/929: write dc/d14/d8b/db4/ff1 [2429393,62110] 0 2024-07-16T09:55:47.270 INFO:tasks.workunit.client.1.smithi176.stdout:4/753: link da/dc/d4b/df0/d96/d9b/db4/d40/d5d/d73/c8c da/d41/d5b/cf5 0 2024-07-16T09:55:47.273 INFO:tasks.workunit.client.0.smithi156.stdout:3/741: creat d8/fff x:0 0 0 2024-07-16T09:55:47.273 INFO:tasks.workunit.client.0.smithi156.stdout:4/828: write d4/f3d [1721247,4654] 0 2024-07-16T09:55:47.274 INFO:tasks.workunit.client.0.smithi156.stdout:7/627: chown dc/d48/f4b 138 1 2024-07-16T09:55:47.275 INFO:tasks.workunit.client.0.smithi156.stdout:6/679: read d3/d13/d17/d4f/d4d/f61 [110013,88581] 0 2024-07-16T09:55:47.277 INFO:tasks.workunit.client.1.smithi176.stdout:1/530: truncate d4/d8/d12/d21/d34/daa/fab 994882 0 2024-07-16T09:55:47.278 INFO:tasks.workunit.client.0.smithi156.stdout:2/886: dwrite d0/d13/d17/f102 [0,4194304] 0 2024-07-16T09:55:47.281 INFO:tasks.workunit.client.1.smithi176.stdout:4/754: rename da/dc/d4b/df0/d96/d69/fb2 to da/dc/d4b/df0/d96/d9b/db4/ff6 0 2024-07-16T09:55:47.281 INFO:tasks.workunit.client.0.smithi156.stdout:7/628: fdatasync dc/d44/f5c 0 2024-07-16T09:55:47.282 INFO:tasks.workunit.client.1.smithi176.stdout:1/531: mknod d4/d8/d12/d21/d25/d30/d44/d4d/cb3 0 2024-07-16T09:55:47.283 INFO:tasks.workunit.client.1.smithi176.stdout:4/755: stat da/d3d/l67 0 2024-07-16T09:55:47.283 INFO:tasks.workunit.client.0.smithi156.stdout:8/696: symlink d0/d11/d20/d7f/db7/lfa 0 2024-07-16T09:55:47.283 INFO:tasks.workunit.client.0.smithi156.stdout:4/829: write d4/db/dc/dcc/d58/fa6 [360792,122686] 0 2024-07-16T09:55:47.284 INFO:tasks.workunit.client.1.smithi176.stdout:6/853: rmdir d1 39 2024-07-16T09:55:47.285 INFO:tasks.workunit.client.0.smithi156.stdout:5/778: write d1b/d1f/d45/d52/d49/f53 [778615,6523] 0 2024-07-16T09:55:47.285 INFO:tasks.workunit.client.0.smithi156.stdout:6/680: creat d3/d13/d17/d4f/d4d/d3b/fde x:0 0 0 2024-07-16T09:55:47.287 INFO:tasks.workunit.client.0.smithi156.stdout:9/848: write d2/f4 [2040143,18705] 0 2024-07-16T09:55:47.289 INFO:tasks.workunit.client.0.smithi156.stdout:9/849: dread - d2/d9b/f100 zero size 2024-07-16T09:55:47.289 INFO:tasks.workunit.client.1.smithi176.stdout:1/532: creat d4/d3a/d54/fb4 x:0 0 0 2024-07-16T09:55:47.289 INFO:tasks.workunit.client.0.smithi156.stdout:2/887: symlink d0/d13/d17/d1c/d10b/l11e 0 2024-07-16T09:55:47.290 INFO:tasks.workunit.client.0.smithi156.stdout:3/742: read d8/d14/f6b [1635383,4545] 0 2024-07-16T09:55:47.291 INFO:tasks.workunit.client.1.smithi176.stdout:6/854: write d1/d5d/db9/d101/f132 [718863,123264] 0 2024-07-16T09:55:47.292 INFO:tasks.workunit.client.0.smithi156.stdout:7/629: rmdir dc/d63/d98 39 2024-07-16T09:55:47.292 INFO:tasks.workunit.client.1.smithi176.stdout:6/855: chown d1/dc/f33 0 1 2024-07-16T09:55:47.292 INFO:tasks.workunit.client.0.smithi156.stdout:8/697: rename d0/d11/d20/cef to d0/dd/d1f/d57/da6/cfb 0 2024-07-16T09:55:47.294 INFO:tasks.workunit.client.0.smithi156.stdout:9/850: creat d2/d5/d16/d57/d63/dd3/ddb/d3d/da6/f10a x:0 0 0 2024-07-16T09:55:47.294 INFO:tasks.workunit.client.1.smithi176.stdout:4/756: creat da/dc/d4b/df0/d96/da4/dc2/ff7 x:0 0 0 2024-07-16T09:55:47.294 INFO:tasks.workunit.client.1.smithi176.stdout:6/856: rmdir d1/d5d/d9e/d8f 39 2024-07-16T09:55:47.295 INFO:tasks.workunit.client.0.smithi156.stdout:1/930: link dc/d14/f48 dc/d14/d3e/d6c/d7a/dcb/f135 0 2024-07-16T09:55:47.295 INFO:tasks.workunit.client.0.smithi156.stdout:2/888: chown d0/d5/d29/d2d/d5b/df3/f106 4 1 2024-07-16T09:55:47.296 INFO:tasks.workunit.client.1.smithi176.stdout:0/896: dread d5/f1e [0,4194304] 0 2024-07-16T09:55:47.297 INFO:tasks.workunit.client.1.smithi176.stdout:1/533: creat d4/d8/fb5 x:0 0 0 2024-07-16T09:55:47.298 INFO:tasks.workunit.client.1.smithi176.stdout:0/897: stat d5/d1c/d24/d35/d59/dc9/ffb 0 2024-07-16T09:55:47.298 INFO:tasks.workunit.client.1.smithi176.stdout:4/757: creat da/d41/d5b/ff8 x:0 0 0 2024-07-16T09:55:47.299 INFO:tasks.workunit.client.1.smithi176.stdout:4/758: chown da/dc/f58 1015563 1 2024-07-16T09:55:47.299 INFO:tasks.workunit.client.1.smithi176.stdout:0/898: truncate d5/d39/f9f 4790640 0 2024-07-16T09:55:47.301 INFO:tasks.workunit.client.0.smithi156.stdout:0/862: write d0/d1a/d4d/d9/d27/d50/f5f [4858551,95080] 0 2024-07-16T09:55:47.304 INFO:tasks.workunit.client.0.smithi156.stdout:8/698: dwrite d0/dd/f1a [0,4194304] 0 2024-07-16T09:55:47.305 INFO:tasks.workunit.client.1.smithi176.stdout:0/899: symlink d5/d1c/d24/d35/d50/d58/d63/db4/dcc/l128 0 2024-07-16T09:55:47.305 INFO:tasks.workunit.client.0.smithi156.stdout:7/630: creat dc/d13/d15/d8d/dd0/fd5 x:0 0 0 2024-07-16T09:55:47.306 INFO:tasks.workunit.client.0.smithi156.stdout:3/743: rename d8/da/d1d/d2c/d7d/lc9 to d8/da/d1d/d2c/d7d/l100 0 2024-07-16T09:55:47.306 INFO:tasks.workunit.client.1.smithi176.stdout:4/759: dread da/dc/d4b/df0/d96/d9b/db4/d40/f92 [0,4194304] 0 2024-07-16T09:55:47.307 INFO:tasks.workunit.client.0.smithi156.stdout:5/779: link d1b/d1f/d45/d52/d43/d85/d97/c8e d1b/d1f/d8a/cfd 0 2024-07-16T09:55:47.307 INFO:tasks.workunit.client.0.smithi156.stdout:0/863: readlink d0/d1a/d4d/d9/d27/d2c/l39 0 2024-07-16T09:55:47.310 INFO:tasks.workunit.client.0.smithi156.stdout:8/699: fdatasync d0/d11/d2e/d35/f46 0 2024-07-16T09:55:47.310 INFO:tasks.workunit.client.0.smithi156.stdout:9/851: fdatasync d2/d5/d16/d22/d6b/d86/fe4 0 2024-07-16T09:55:47.310 INFO:tasks.workunit.client.1.smithi176.stdout:4/760: symlink da/dc/d4b/df0/d96/d9b/db4/d40/d5d/d73/lf9 0 2024-07-16T09:55:47.311 INFO:tasks.workunit.client.0.smithi156.stdout:7/631: mknod dc/d13/d15/d8b/cd6 0 2024-07-16T09:55:47.317 INFO:tasks.workunit.client.1.smithi176.stdout:6/857: truncate d1/d5d/d9e/faa 417085 0 2024-07-16T09:55:47.318 INFO:tasks.workunit.client.1.smithi176.stdout:6/858: fsync d1/d5d/d9e/d76/f103 0 2024-07-16T09:55:47.319 INFO:tasks.workunit.client.0.smithi156.stdout:1/931: write dc/d14/d8b/db4/df2/f36 [5958591,81601] 0 2024-07-16T09:55:47.321 INFO:tasks.workunit.client.0.smithi156.stdout:0/864: rename d0/d1a/d4d/d9/d27/d3d/faf to d0/d22/d41/d52/ddb/f114 0 2024-07-16T09:55:47.322 INFO:tasks.workunit.client.0.smithi156.stdout:7/632: truncate dc/d63/f88 735402 0 2024-07-16T09:55:47.323 INFO:tasks.workunit.client.1.smithi176.stdout:6/859: rename d1/d5d/db9/fcb to d1/d5/d47/d62/de8/d114/f137 0 2024-07-16T09:55:47.324 INFO:tasks.workunit.client.1.smithi176.stdout:1/534: dwrite d4/db/f43 [0,4194304] 0 2024-07-16T09:55:47.324 INFO:tasks.workunit.client.0.smithi156.stdout:9/852: truncate d2/d5/d16/d22/d6b/f97 1461968 0 2024-07-16T09:55:47.324 INFO:tasks.workunit.client.0.smithi156.stdout:6/681: dwrite d3/d13/d17/d4f/d4d/d3b/f40 [0,4194304] 0 2024-07-16T09:55:47.325 INFO:tasks.workunit.client.1.smithi176.stdout:0/900: dread d5/d25/da0/faa [0,4194304] 0 2024-07-16T09:55:47.325 INFO:tasks.workunit.client.0.smithi156.stdout:7/633: rmdir dc/d3e/d76 39 2024-07-16T09:55:47.327 INFO:tasks.workunit.client.1.smithi176.stdout:1/535: mknod d4/d3a/d54/d5a/cb6 0 2024-07-16T09:55:47.327 INFO:tasks.workunit.client.0.smithi156.stdout:7/634: fsync dc/d13/fa5 0 2024-07-16T09:55:47.329 INFO:tasks.workunit.client.0.smithi156.stdout:9/853: mkdir d2/d5/d61/da2/d98/dbf/d10b 0 2024-07-16T09:55:47.329 INFO:tasks.workunit.client.1.smithi176.stdout:1/536: write d4/d8/f8c [1752867,17791] 0 2024-07-16T09:55:47.334 INFO:tasks.workunit.client.0.smithi156.stdout:5/780: write d1b/d1f/d45/d52/d49/f81 [2277503,72535] 0 2024-07-16T09:55:47.334 INFO:tasks.workunit.client.1.smithi176.stdout:1/537: write d4/d8/d12/d21/d25/d30/d44/d4d/f91 [902256,70240] 0 2024-07-16T09:55:47.335 INFO:tasks.workunit.client.1.smithi176.stdout:1/538: dread - d4/d3a/d54/fb4 zero size 2024-07-16T09:55:47.338 INFO:tasks.workunit.client.1.smithi176.stdout:1/539: truncate d4/d8/d12/d21/d25/f55 5206725 0 2024-07-16T09:55:47.341 INFO:tasks.workunit.client.1.smithi176.stdout:1/540: stat d4/d8/d12/d21/d25/da8/c98 0 2024-07-16T09:55:47.341 INFO:tasks.workunit.client.0.smithi156.stdout:4/830: dread d4/db/f18 [0,4194304] 0 2024-07-16T09:55:47.342 INFO:tasks.workunit.client.0.smithi156.stdout:9/854: chown d2/d5/d16/d57/d63/dd3/ddb/d74/dc2/dc4/d23/cad 319470 1 2024-07-16T09:55:47.342 INFO:tasks.workunit.client.0.smithi156.stdout:8/700: dread d0/d11/d13/d1c/d24/fb4 [0,4194304] 0 2024-07-16T09:55:47.342 INFO:tasks.workunit.client.0.smithi156.stdout:6/682: link d3/d13/d17/d4f/d4d/d3b/l87 d3/d13/d17/d4f/d94/d19/d31/d35/ldf 0 2024-07-16T09:55:47.343 INFO:tasks.workunit.client.0.smithi156.stdout:1/932: write dc/d14/d3e/d55/d57/fa2 [3962966,57262] 0 2024-07-16T09:55:47.343 INFO:tasks.workunit.client.1.smithi176.stdout:4/761: dwrite da/dc/d4b/fa7 [0,4194304] 0 2024-07-16T09:55:47.345 INFO:tasks.workunit.client.0.smithi156.stdout:1/933: readlink dc/d6b/d8a/l106 0 2024-07-16T09:55:47.345 INFO:tasks.workunit.client.0.smithi156.stdout:4/831: dread - d4/db/dc/dcc/dec/ded/d87/f114 zero size 2024-07-16T09:55:47.345 INFO:tasks.workunit.client.0.smithi156.stdout:6/683: readlink d3/d13/d17/d4f/d4d/l2d 0 2024-07-16T09:55:47.346 INFO:tasks.workunit.client.1.smithi176.stdout:1/541: write d4/d8/f26 [1930964,117677] 0 2024-07-16T09:55:47.348 INFO:tasks.workunit.client.1.smithi176.stdout:4/762: dread da/d3d/d17/d54/fa9 [0,4194304] 0 2024-07-16T09:55:47.349 INFO:tasks.workunit.client.0.smithi156.stdout:8/701: write d0/dd/fa7 [4645303,89298] 0 2024-07-16T09:55:47.352 INFO:tasks.workunit.client.0.smithi156.stdout:9/855: creat d2/d5/d16/d22/f10c x:0 0 0 2024-07-16T09:55:47.352 INFO:tasks.workunit.client.0.smithi156.stdout:5/781: write d1b/d1f/d45/d52/d43/d85/f8f [1999897,46194] 0 2024-07-16T09:55:47.353 INFO:tasks.workunit.client.0.smithi156.stdout:7/635: write dc/d13/d15/f91 [826568,61607] 0 2024-07-16T09:55:47.355 INFO:tasks.workunit.client.1.smithi176.stdout:1/542: link d4/d77/dad/d72/f74 d4/fb7 0 2024-07-16T09:55:47.355 INFO:tasks.workunit.client.1.smithi176.stdout:1/543: dread - d4/d8/fb5 zero size 2024-07-16T09:55:47.356 INFO:tasks.workunit.client.0.smithi156.stdout:4/832: creat d4/db/dc/d5f/f124 x:0 0 0 2024-07-16T09:55:47.356 INFO:tasks.workunit.client.0.smithi156.stdout:6/684: getdents d3/d13/d17/d4f/d4d/d3b/dcc 0 2024-07-16T09:55:47.357 INFO:tasks.workunit.client.1.smithi176.stdout:0/901: write d5/d1c/d24/d35/d50/dd5/fda [2017280,85539] 0 2024-07-16T09:55:47.358 INFO:tasks.workunit.client.1.smithi176.stdout:1/544: truncate d4/f17 752564 0 2024-07-16T09:55:47.359 INFO:tasks.workunit.client.1.smithi176.stdout:6/860: truncate d1/d5/d7/d54/f55 3175881 0 2024-07-16T09:55:47.360 INFO:tasks.workunit.client.1.smithi176.stdout:6/861: read - d1/d5/d7/f109 zero size 2024-07-16T09:55:47.361 INFO:tasks.workunit.client.0.smithi156.stdout:4/833: fsync d4/d23/f59 0 2024-07-16T09:55:47.362 INFO:tasks.workunit.client.1.smithi176.stdout:1/545: fdatasync d4/d8/d12/d21/d25/d30/d44/d4d/f94 0 2024-07-16T09:55:47.362 INFO:tasks.workunit.client.0.smithi156.stdout:1/934: dwrite dc/d14/d1e/f35 [0,4194304] 0 2024-07-16T09:55:47.364 INFO:tasks.workunit.client.0.smithi156.stdout:8/702: chown d0/d11/d62/dc2/dcd/fec 40835 1 2024-07-16T09:55:47.364 INFO:tasks.workunit.client.0.smithi156.stdout:6/685: symlink d3/d13/d17/le0 0 2024-07-16T09:55:47.365 INFO:tasks.workunit.client.0.smithi156.stdout:7/636: chown dc/d13/d15/d8b/fbe 1710454 1 2024-07-16T09:55:47.366 INFO:tasks.workunit.client.0.smithi156.stdout:0/865: dread d0/d1a/d4d/d9/d27/d3d/d62/f63 [0,4194304] 0 2024-07-16T09:55:47.366 INFO:tasks.workunit.client.0.smithi156.stdout:3/744: dread d8/da/d1d/d2c/de0/fa9 [0,4194304] 0 2024-07-16T09:55:47.372 INFO:tasks.workunit.client.0.smithi156.stdout:1/935: symlink dc/d14/d8b/db4/df2/d10d/l136 0 2024-07-16T09:55:47.375 INFO:tasks.workunit.client.0.smithi156.stdout:0/866: truncate d0/d1a/d4d/d9/d27/d50/d90/fd8 444556 0 2024-07-16T09:55:47.375 INFO:tasks.workunit.client.0.smithi156.stdout:1/936: write dc/d14/d1e/f35 [3508779,88439] 0 2024-07-16T09:55:47.376 INFO:tasks.workunit.client.0.smithi156.stdout:9/856: dwrite d2/d5/d16/d57/d63/dd3/ddb/d74/dc2/dc4/d3a/de7/f108 [0,4194304] 0 2024-07-16T09:55:47.378 INFO:tasks.workunit.client.0.smithi156.stdout:1/937: write dc/d6b/d96/f10b [3716848,67084] 0 2024-07-16T09:55:47.381 INFO:tasks.workunit.client.0.smithi156.stdout:1/938: getdents dc/d14/d3e/d6c/d12f 0 2024-07-16T09:55:47.381 INFO:tasks.workunit.client.1.smithi176.stdout:6/862: readlink d1/d5d/d9e/dd9/d6f/d7c/lec 0 2024-07-16T09:55:47.382 INFO:tasks.workunit.client.0.smithi156.stdout:5/782: creat d1b/d1f/d45/d52/d43/d85/d97/d37/d88/dc7/ffe x:0 0 0 2024-07-16T09:55:47.383 INFO:tasks.workunit.client.0.smithi156.stdout:4/834: fsync d4/db/f94 0 2024-07-16T09:55:47.384 INFO:tasks.workunit.client.1.smithi176.stdout:1/546: mkdir d4/d8/d12/d21/d34/daa/db8 0 2024-07-16T09:55:47.385 INFO:tasks.workunit.client.0.smithi156.stdout:7/637: mkdir dc/d48/dd7 0 2024-07-16T09:55:47.387 INFO:tasks.workunit.client.0.smithi156.stdout:6/686: dread - d3/d13/d17/d4f/d4d/f76 zero size 2024-07-16T09:55:47.387 INFO:tasks.workunit.client.1.smithi176.stdout:1/547: creat d4/d8/d12/d21/fb9 x:0 0 0 2024-07-16T09:55:47.388 INFO:tasks.workunit.client.0.smithi156.stdout:9/857: dwrite d2/d5/d16/d22/f10c [0,4194304] 0 2024-07-16T09:55:47.389 INFO:tasks.workunit.client.0.smithi156.stdout:7/638: readlink dc/d44/l95 0 2024-07-16T09:55:47.390 INFO:tasks.workunit.client.0.smithi156.stdout:5/783: symlink d1b/d1f/d45/d9d/lff 0 2024-07-16T09:55:47.393 INFO:tasks.workunit.client.0.smithi156.stdout:4/835: creat d4/db/dc/dcc/dec/ded/daf/f125 x:0 0 0 2024-07-16T09:55:47.393 INFO:tasks.workunit.client.0.smithi156.stdout:6/687: symlink d3/d13/d17/d4f/d4d/dbb/le1 0 2024-07-16T09:55:47.393 INFO:tasks.workunit.client.0.smithi156.stdout:3/745: link d8/d14/f3f d8/da/d21/d80/d99/f101 0 2024-07-16T09:55:47.393 INFO:tasks.workunit.client.0.smithi156.stdout:5/784: dread - d1b/d1f/d45/d52/d43/d85/d97/d37/d88/dc7/ffe zero size 2024-07-16T09:55:47.394 INFO:tasks.workunit.client.1.smithi176.stdout:1/548: symlink d4/d8/d12/d21/d25/lba 0 2024-07-16T09:55:47.394 INFO:tasks.workunit.client.0.smithi156.stdout:4/836: fsync d4/d23/d52/f69 0 2024-07-16T09:55:47.396 INFO:tasks.workunit.client.0.smithi156.stdout:8/703: getdents d0 0 2024-07-16T09:55:47.396 INFO:tasks.workunit.client.1.smithi176.stdout:1/549: rmdir d4/d3a/d54/d5a/d7a 39 2024-07-16T09:55:47.396 INFO:tasks.workunit.client.1.smithi176.stdout:1/550: readlink d4/d77/dad/l6f 0 2024-07-16T09:55:47.397 INFO:tasks.workunit.client.0.smithi156.stdout:4/837: truncate d4/d23/d30/fd6 791168 0 2024-07-16T09:55:47.397 INFO:tasks.workunit.client.0.smithi156.stdout:7/639: mkdir dc/d3e/d3f/d51/d8a/dae/dd8 0 2024-07-16T09:55:47.398 INFO:tasks.workunit.client.0.smithi156.stdout:9/858: creat d2/d9b/d109/f10d x:0 0 0 2024-07-16T09:55:47.398 INFO:tasks.workunit.client.1.smithi176.stdout:1/551: fsync d4/d8/d12/d21/d25/f55 0 2024-07-16T09:55:47.400 INFO:tasks.workunit.client.1.smithi176.stdout:1/552: readlink d4/d8/d12/d21/la3 0 2024-07-16T09:55:47.401 INFO:tasks.workunit.client.1.smithi176.stdout:6/863: dread d1/d5d/db9/f32 [0,4194304] 0 2024-07-16T09:55:47.401 INFO:tasks.workunit.client.0.smithi156.stdout:6/688: mknod d3/d13/d17/d4f/d5c/ce2 0 2024-07-16T09:55:47.401 INFO:tasks.workunit.client.0.smithi156.stdout:4/838: write d4/db/dc/dcc/d58/d8f/f7e [506466,67644] 0 2024-07-16T09:55:47.401 INFO:tasks.workunit.client.0.smithi156.stdout:5/785: symlink d1b/d1f/d45/d52/d43/d85/l100 0 2024-07-16T09:55:47.402 INFO:tasks.workunit.client.1.smithi176.stdout:1/553: write d4/d8/d12/d21/f9c [667261,69253] 0 2024-07-16T09:55:47.403 INFO:tasks.workunit.client.0.smithi156.stdout:7/640: mkdir dc/d13/d36/dd4/dd9 0 2024-07-16T09:55:47.406 INFO:tasks.workunit.client.1.smithi176.stdout:6/864: rename d1/d5d/db9/l3c to d1/dc/d40/l138 0 2024-07-16T09:55:47.407 INFO:tasks.workunit.client.0.smithi156.stdout:6/689: dread - d3/d13/d17/d4f/d4d/d3b/fba zero size 2024-07-16T09:55:47.407 INFO:tasks.workunit.client.0.smithi156.stdout:5/786: readlink d1b/d1f/d45/d52/d2d/l30 0 2024-07-16T09:55:47.407 INFO:tasks.workunit.client.0.smithi156.stdout:8/704: dwrite d0/d11/d20/d7f/fc8 [0,4194304] 0 2024-07-16T09:55:47.411 INFO:tasks.workunit.client.0.smithi156.stdout:7/641: chown dc/f40 222 1 2024-07-16T09:55:47.412 INFO:tasks.workunit.client.1.smithi176.stdout:0/902: write d5/d1c/d24/d35/f61 [4101823,65304] 0 2024-07-16T09:55:47.414 INFO:tasks.workunit.client.1.smithi176.stdout:6/865: dwrite d1/d5d/db9/d101/f132 [0,4194304] 0 2024-07-16T09:55:47.415 INFO:tasks.workunit.client.0.smithi156.stdout:5/787: mknod d1b/d1f/d45/d9d/c101 0 2024-07-16T09:55:47.421 INFO:tasks.workunit.client.0.smithi156.stdout:4/839: rename d4/d23/l26 to d4/db/dc/dcc/dec/l126 0 2024-07-16T09:55:47.421 INFO:tasks.workunit.client.0.smithi156.stdout:5/788: chown d1b/d1f/d45/d52/d49/d4d/fdc 0 1 2024-07-16T09:55:47.421 INFO:tasks.workunit.client.0.smithi156.stdout:9/859: dread d2/d5/d16/d22/d6b/f97 [0,4194304] 0 2024-07-16T09:55:47.422 INFO:tasks.workunit.client.1.smithi176.stdout:0/903: creat d5/d1c/d24/d35/d50/d58/d63/f129 x:0 0 0 2024-07-16T09:55:47.423 INFO:tasks.workunit.client.1.smithi176.stdout:0/904: fdatasync d5/d1c/d24/d35/d50/dd5/ff6 0 2024-07-16T09:55:47.424 INFO:tasks.workunit.client.0.smithi156.stdout:6/690: dwrite d3/d13/d17/d4f/d4d/d3b/d5a/dc1/dce/fda [0,4194304] 0 2024-07-16T09:55:47.424 INFO:tasks.workunit.client.0.smithi156.stdout:7/642: creat dc/d3e/d3f/d51/d8a/fda x:0 0 0 2024-07-16T09:55:47.424 INFO:tasks.workunit.client.0.smithi156.stdout:5/789: stat d1b/d1f/d45/d7a 0 2024-07-16T09:55:47.424 INFO:tasks.workunit.client.1.smithi176.stdout:1/554: write d4/d77/dad/d72/f74 [4003102,122940] 0 2024-07-16T09:55:47.427 INFO:tasks.workunit.client.1.smithi176.stdout:0/905: fsync d5/f18 0 2024-07-16T09:55:47.430 INFO:tasks.workunit.client.1.smithi176.stdout:0/906: mknod d5/d44/c12a 0 2024-07-16T09:55:47.436 INFO:tasks.workunit.client.0.smithi156.stdout:1/939: dwrite dc/d14/d3e/d55/f70 [0,4194304] 0 2024-07-16T09:55:47.438 INFO:tasks.workunit.client.0.smithi156.stdout:3/746: dwrite d8/da/d1d/d2c/f84 [0,4194304] 0 2024-07-16T09:55:47.442 INFO:tasks.workunit.client.0.smithi156.stdout:5/790: write d1b/d1f/d45/d52/d43/d85/f8f [2861743,56662] 0 2024-07-16T09:55:47.442 INFO:tasks.workunit.client.1.smithi176.stdout:0/907: link d5/d1c/d24/d35/d50/d7c/f93 d5/d1c/d24/d35/d50/d9b/de3/d117/f12b 0 2024-07-16T09:55:47.444 INFO:tasks.workunit.client.1.smithi176.stdout:6/866: dwrite d1/d5/d47/f83 [0,4194304] 0 2024-07-16T09:55:47.444 INFO:tasks.workunit.client.0.smithi156.stdout:9/860: mkdir d2/d5/d16/d57/d63/dd3/ddb/d3d/d3c/d7f/d10e 0 2024-07-16T09:55:47.445 INFO:tasks.workunit.client.1.smithi176.stdout:0/908: rename d5/d1c/d5a/f6d to d5/d39/da3/d11a/f12c 0 2024-07-16T09:55:47.446 INFO:tasks.workunit.client.1.smithi176.stdout:6/867: rmdir d1/d5/d47 39 2024-07-16T09:55:47.446 INFO:tasks.workunit.client.0.smithi156.stdout:3/747: mknod d8/da/d13/dca/c102 0 2024-07-16T09:55:47.448 INFO:tasks.workunit.client.0.smithi156.stdout:5/791: truncate d1b/d1f/d45/d52/d49/d4d/fe1 471718 0 2024-07-16T09:55:47.448 INFO:tasks.workunit.client.0.smithi156.stdout:1/940: creat dc/d6b/d96/d11a/f137 x:0 0 0 2024-07-16T09:55:47.449 INFO:tasks.workunit.client.0.smithi156.stdout:4/840: dwrite d4/db/dc/fcb [4194304,4194304] 0 2024-07-16T09:55:47.450 INFO:tasks.workunit.client.1.smithi176.stdout:6/868: creat d1/d5d/db9/d101/f139 x:0 0 0 2024-07-16T09:55:47.451 INFO:tasks.workunit.client.0.smithi156.stdout:1/941: write f4 [1622275,77185] 0 2024-07-16T09:55:47.457 INFO:tasks.workunit.client.0.smithi156.stdout:4/841: chown d4/db/dc/d67/l72 6616 1 2024-07-16T09:55:47.457 INFO:tasks.workunit.client.0.smithi156.stdout:9/861: write d2/d5/d16/d22/d6b/fcc [1479040,49696] 0 2024-07-16T09:55:47.460 INFO:tasks.workunit.client.0.smithi156.stdout:6/691: dwrite d3/d13/d17/d4f/d94/d19/d31/fa2 [0,4194304] 0 2024-07-16T09:55:47.462 INFO:tasks.workunit.client.1.smithi176.stdout:6/869: dwrite d1/d5d/d9e/d8f/f8d [0,4194304] 0 2024-07-16T09:55:47.464 INFO:tasks.workunit.client.0.smithi156.stdout:6/692: read - d3/d13/d17/d4f/d4d/dbb/fc6 zero size 2024-07-16T09:55:47.469 INFO:tasks.workunit.client.0.smithi156.stdout:9/862: creat d2/d5/d16/d57/d63/f10f x:0 0 0 2024-07-16T09:55:47.469 INFO:tasks.workunit.client.0.smithi156.stdout:4/842: symlink d4/db/dc/dcc/dec/ded/d87/d108/l127 0 2024-07-16T09:55:47.469 INFO:tasks.workunit.client.0.smithi156.stdout:1/942: link dc/d14/d3e/d55/f5c dc/d14/d8b/dcd/d12d/f138 0 2024-07-16T09:55:47.471 INFO:tasks.workunit.client.0.smithi156.stdout:6/693: creat d3/d13/d17/d4f/d1e/fe3 x:0 0 0 2024-07-16T09:55:47.473 INFO:tasks.workunit.client.0.smithi156.stdout:4/843: mkdir d4/db/dc/d5f/d128 0 2024-07-16T09:55:47.475 INFO:tasks.workunit.client.1.smithi176.stdout:6/870: rename d1/d5/f4f to d1/d5/d47/d62/d77/f13a 0 2024-07-16T09:55:47.475 INFO:tasks.workunit.client.0.smithi156.stdout:6/694: stat d3/d13/d17/d4f/d4d/d3b/f8f 0 2024-07-16T09:55:47.476 INFO:tasks.workunit.client.1.smithi176.stdout:6/871: fdatasync d1/d5/d47/d62/d77/dd2/fe9 0 2024-07-16T09:55:47.477 INFO:tasks.workunit.client.0.smithi156.stdout:1/943: creat dc/d14/d8b/db4/df2/f139 x:0 0 0 2024-07-16T09:55:47.478 INFO:tasks.workunit.client.0.smithi156.stdout:4/844: symlink d4/db/dc/d5f/d64/l129 0 2024-07-16T09:55:47.481 INFO:tasks.workunit.client.0.smithi156.stdout:9/863: dwrite d2/d5/d16/d57/d63/dd3/ddb/dd0/fdc [0,4194304] 0 2024-07-16T09:55:47.481 INFO:tasks.workunit.client.1.smithi176.stdout:6/872: dread d1/d5/d47/ff5 [0,4194304] 0 2024-07-16T09:55:47.482 INFO:tasks.workunit.client.0.smithi156.stdout:1/944: read dc/d14/d3e/d6c/f128 [3114484,117263] 0 2024-07-16T09:55:47.482 INFO:tasks.workunit.client.0.smithi156.stdout:6/695: rmdir d3/d13/d17/d4f/d4d/d3b/d5a/dc1/dce 39 2024-07-16T09:55:47.483 INFO:tasks.workunit.client.0.smithi156.stdout:6/696: readlink d3/d13/d17/d4f/d94/l33 0 2024-07-16T09:55:47.485 INFO:tasks.workunit.client.1.smithi176.stdout:6/873: symlink d1/d5/d47/d62/de8/d114/l13b 0 2024-07-16T09:55:47.485 INFO:tasks.workunit.client.0.smithi156.stdout:4/845: dread - d4/db/fc9 zero size 2024-07-16T09:55:47.485 INFO:tasks.workunit.client.0.smithi156.stdout:6/697: write d3/d13/d17/d4f/d4d/d3b/fba [1016899,50627] 0 2024-07-16T09:55:47.486 INFO:tasks.workunit.client.1.smithi176.stdout:0/909: dread d5/d44/f11c [0,4194304] 0 2024-07-16T09:55:47.487 INFO:tasks.workunit.client.0.smithi156.stdout:5/792: write d1b/d1f/d45/d7f/d87/fa7 [1928193,50083] 0 2024-07-16T09:55:47.489 INFO:tasks.workunit.client.1.smithi176.stdout:6/874: symlink d1/d5/d47/d62/de8/d114/l13c 0 2024-07-16T09:55:47.491 INFO:tasks.workunit.client.0.smithi156.stdout:4/846: symlink d4/d23/d30/l12a 0 2024-07-16T09:55:47.493 INFO:tasks.workunit.client.1.smithi176.stdout:6/875: mkdir d1/d5/d47/d62/d80/d13d 0 2024-07-16T09:55:47.494 INFO:tasks.workunit.client.0.smithi156.stdout:5/793: creat d1b/d1f/d45/d52/f102 x:0 0 0 2024-07-16T09:55:47.494 INFO:tasks.workunit.client.1.smithi176.stdout:0/910: getdents d5/d1c/d24/d35/d50/d58/d63 0 2024-07-16T09:55:47.497 INFO:tasks.workunit.client.1.smithi176.stdout:0/911: fdatasync d5/f2a 0 2024-07-16T09:55:47.497 INFO:tasks.workunit.client.0.smithi156.stdout:9/864: dwrite d2/f15 [0,4194304] 0 2024-07-16T09:55:47.497 INFO:tasks.workunit.client.0.smithi156.stdout:5/794: chown d1b/d1f/d42/d6d/c6f 141449284 1 2024-07-16T09:55:47.499 INFO:tasks.workunit.client.0.smithi156.stdout:3/748: dwrite d8/da/d13/f5c [0,4194304] 0 2024-07-16T09:55:47.500 INFO:tasks.workunit.client.0.smithi156.stdout:4/847: rename d4/db/dc/dcc/fb7 to d4/d23/d52/f12b 0 2024-07-16T09:55:47.502 INFO:tasks.workunit.client.1.smithi176.stdout:6/876: dwrite d1/d5d/d9e/fc1 [4194304,4194304] 0 2024-07-16T09:55:47.505 INFO:tasks.workunit.client.0.smithi156.stdout:9/865: creat d2/d5/d16/d57/d63/f110 x:0 0 0 2024-07-16T09:55:47.506 INFO:tasks.workunit.client.0.smithi156.stdout:3/749: rename d8/da/d28/d43/l89 to d8/da/d1d/d2c/dee/l103 0 2024-07-16T09:55:47.506 INFO:tasks.workunit.client.0.smithi156.stdout:6/698: write d3/d13/d17/d4f/d4d/d3b/f96 [1346012,78883] 0 2024-07-16T09:55:47.507 INFO:tasks.workunit.client.0.smithi156.stdout:3/750: write d8/d14/fc7 [1780107,90148] 0 2024-07-16T09:55:47.508 INFO:tasks.workunit.client.1.smithi176.stdout:6/877: mknod d1/dc/d5b/d11d/c13e 0 2024-07-16T09:55:47.509 INFO:tasks.workunit.client.0.smithi156.stdout:6/699: read d3/f1c [2034581,21352] 0 2024-07-16T09:55:47.509 INFO:tasks.workunit.client.1.smithi176.stdout:6/878: read - d1/dc/d81/fdd zero size 2024-07-16T09:55:47.510 INFO:tasks.workunit.client.1.smithi176.stdout:6/879: fdatasync d1/d5/d7/d54/f70 0 2024-07-16T09:55:47.512 INFO:tasks.workunit.client.0.smithi156.stdout:5/795: mkdir d1b/d1f/d45/d52/d43/d85/d97/d37/d88/d61/d103 0 2024-07-16T09:55:47.513 INFO:tasks.workunit.client.0.smithi156.stdout:4/848: creat d4/db/dc/dcc/dec/f12c x:0 0 0 2024-07-16T09:55:47.514 INFO:tasks.workunit.client.1.smithi176.stdout:6/880: write d1/d5d/d9e/d76/da5/f10c [783586,19976] 0 2024-07-16T09:55:47.516 INFO:tasks.workunit.client.1.smithi176.stdout:0/912: dwrite d5/d2c/f2d [4194304,4194304] 0 2024-07-16T09:55:47.519 INFO:tasks.workunit.client.1.smithi176.stdout:0/913: dread d5/d25/da0/fdd [0,4194304] 0 2024-07-16T09:55:47.519 INFO:tasks.workunit.client.1.smithi176.stdout:6/881: creat d1/d5d/d9e/dd9/de7/f13f x:0 0 0 2024-07-16T09:55:47.522 INFO:tasks.workunit.client.1.smithi176.stdout:0/914: mkdir d5/d44/d12d 0 2024-07-16T09:55:47.523 INFO:tasks.workunit.client.1.smithi176.stdout:0/915: chown d5/d1c/d24/ddf/le0 31199 1 2024-07-16T09:55:47.526 INFO:tasks.workunit.client.1.smithi176.stdout:0/916: mknod d5/d25/d82/c12e 0 2024-07-16T09:55:47.526 INFO:tasks.workunit.client.1.smithi176.stdout:6/882: dwrite d1/d5d/d9e/d76/f103 [0,4194304] 0 2024-07-16T09:55:47.529 INFO:tasks.workunit.client.1.smithi176.stdout:0/917: rmdir d5/d1c/d24/d35/d50/d5b 39 2024-07-16T09:55:47.530 INFO:tasks.workunit.client.1.smithi176.stdout:6/883: mknod d1/d5/d47/d62/c140 0 2024-07-16T09:55:47.555 INFO:tasks.workunit.client.1.smithi176.stdout:6/884: dwrite d1/d5d/db9/fb6 [0,4194304] 0 2024-07-16T09:55:47.560 INFO:tasks.workunit.client.0.smithi156.stdout:3/751: creat d8/da/d1d/d2c/de0/dad/f104 x:0 0 0 2024-07-16T09:55:47.560 INFO:tasks.workunit.client.0.smithi156.stdout:6/700: mkdir d3/d13/d17/d4f/d94/de4 0 2024-07-16T09:55:47.562 INFO:tasks.workunit.client.0.smithi156.stdout:5/796: creat d1b/d1f/d45/d7a/d98/f104 x:0 0 0 2024-07-16T09:55:47.563 INFO:tasks.workunit.client.0.smithi156.stdout:3/752: stat d8/c8f 0 2024-07-16T09:55:47.565 INFO:tasks.workunit.client.0.smithi156.stdout:9/866: getdents d2/d5/d16/de2 0 2024-07-16T09:55:47.566 INFO:tasks.workunit.client.0.smithi156.stdout:5/797: read - d1b/d1f/d45/d7a/d98/f104 zero size 2024-07-16T09:55:47.566 INFO:tasks.workunit.client.0.smithi156.stdout:3/753: fsync d8/da/d1d/d6f/db3/dc4/ff0 0 2024-07-16T09:55:47.568 INFO:tasks.workunit.client.0.smithi156.stdout:9/867: chown d2/d5/d61/l72 21034868 1 2024-07-16T09:55:47.570 INFO:tasks.workunit.client.0.smithi156.stdout:3/754: read d8/d14/f1c [2117972,119670] 0 2024-07-16T09:55:47.570 INFO:tasks.workunit.client.0.smithi156.stdout:5/798: chown d1b/d1f/d45/d52/d43/d85/d97/d37/d88/dc7/dd9 2104473920 1 2024-07-16T09:55:47.570 INFO:tasks.workunit.client.0.smithi156.stdout:4/849: dwrite d4/db/dc/d67/f9b [0,4194304] 0 2024-07-16T09:55:47.572 INFO:tasks.workunit.client.0.smithi156.stdout:6/701: chown d3/d13/d17/d4f/d94/d19/d31/d35/f62 1319038897 1 2024-07-16T09:55:47.575 INFO:tasks.workunit.client.0.smithi156.stdout:5/799: truncate f19 1282237 0 2024-07-16T09:55:47.576 INFO:tasks.workunit.client.0.smithi156.stdout:4/850: creat d4/d23/d52/d71/def/d115/f12d x:0 0 0 2024-07-16T09:55:47.579 INFO:tasks.workunit.client.0.smithi156.stdout:4/851: chown d4/d23/d30/d33/c4a 9763671 1 2024-07-16T09:55:47.580 INFO:tasks.workunit.client.0.smithi156.stdout:9/868: rename d2/d5/d61 to d2/d5/d16/d57/d63/dd3/ddb/d74/dc2/dc4/d23/d111 0 2024-07-16T09:55:47.582 INFO:tasks.workunit.client.0.smithi156.stdout:5/800: getdents d1b/d1f/d45/d52/d43/d7c 0 2024-07-16T09:55:47.583 INFO:tasks.workunit.client.0.smithi156.stdout:3/755: dwrite d8/da/d28/d43/f77 [0,4194304] 0 2024-07-16T09:55:47.587 INFO:tasks.workunit.client.0.smithi156.stdout:9/869: symlink d2/d5/d16/d57/d63/dd3/ddb/d3d/db1/l112 0 2024-07-16T09:55:47.588 INFO:tasks.workunit.client.0.smithi156.stdout:4/852: dread d4/db/dc/d67/f86 [0,4194304] 0 2024-07-16T09:55:47.591 INFO:tasks.workunit.client.0.smithi156.stdout:3/756: getdents d8/da/d1d/d6f 0 2024-07-16T09:55:47.592 INFO:tasks.workunit.client.0.smithi156.stdout:3/757: fdatasync d8/da/d28/d43/d71/fe4 0 2024-07-16T09:55:47.593 INFO:tasks.workunit.client.0.smithi156.stdout:6/702: dwrite d3/d13/d17/f6c [4194304,4194304] 0 2024-07-16T09:55:47.593 INFO:tasks.workunit.client.0.smithi156.stdout:9/870: symlink d2/d5/d16/l113 0 2024-07-16T09:55:47.598 INFO:tasks.workunit.client.0.smithi156.stdout:6/703: fdatasync d3/d13/d17/d4f/d30/d45/d99/fb5 0 2024-07-16T09:55:47.598 INFO:tasks.workunit.client.0.smithi156.stdout:5/801: link d1b/d1f/d45/d52/d43/d85/d97/d37/d88/d61/l9a d1b/d1f/d42/de0/de5/l105 0 2024-07-16T09:55:47.598 INFO:tasks.workunit.client.0.smithi156.stdout:9/871: mkdir d2/d5/d16/d57/d63/dd3/ddb/d3d/d3c/d7f/d114 0 2024-07-16T09:55:47.599 INFO:tasks.workunit.client.0.smithi156.stdout:6/704: mkdir d3/d13/d17/d4f/d94/d19/de5 0 2024-07-16T09:55:47.600 INFO:tasks.workunit.client.0.smithi156.stdout:3/758: rename d8/da/d1d/d6f/db3/dc4/ce1 to d8/da/d1d/d6f/db3/dc4/c105 0 2024-07-16T09:55:47.601 INFO:tasks.workunit.client.0.smithi156.stdout:3/759: chown d8/da/f94 70 1 2024-07-16T09:55:47.604 INFO:tasks.workunit.client.0.smithi156.stdout:3/760: symlink d8/da/d1d/d2c/de0/l106 0 2024-07-16T09:55:47.606 INFO:tasks.workunit.client.0.smithi156.stdout:4/853: write d4/d23/d30/f123 [2458178,5598] 0 2024-07-16T09:55:47.607 INFO:tasks.workunit.client.0.smithi156.stdout:9/872: dwrite d2/d9b/f100 [0,4194304] 0 2024-07-16T09:55:47.608 INFO:tasks.workunit.client.0.smithi156.stdout:9/873: fsync d2/f15 0 2024-07-16T09:55:47.615 INFO:tasks.workunit.client.0.smithi156.stdout:4/854: rename d4/db/c48 to d4/db/c12e 0 2024-07-16T09:55:47.619 INFO:tasks.workunit.client.0.smithi156.stdout:4/855: symlink d4/db/dc/dcc/dec/ded/d87/df2/l12f 0 2024-07-16T09:55:47.622 INFO:tasks.workunit.client.0.smithi156.stdout:5/802: dread d1b/d1f/d45/d52/d43/d7c/f82 [0,4194304] 0 2024-07-16T09:55:47.622 INFO:tasks.workunit.client.0.smithi156.stdout:4/856: symlink d4/db/dc/dcc/dec/ded/d87/df2/l130 0 2024-07-16T09:55:47.624 INFO:tasks.workunit.client.0.smithi156.stdout:3/761: dread d8/da/d28/d43/d63/f73 [0,4194304] 0 2024-07-16T09:55:47.625 INFO:tasks.workunit.client.1.smithi176.stdout:6/885: getdents d1/d5/d47/d62/d77/dde 0 2024-07-16T09:55:47.628 INFO:tasks.workunit.client.0.smithi156.stdout:9/874: dread d2/d5/d16/d57/d63/dd3/ddb/d74/dc2/de6/ff0 [0,4194304] 0 2024-07-16T09:55:47.628 INFO:tasks.workunit.client.0.smithi156.stdout:5/803: getdents d1b/d1f/d45 0 2024-07-16T09:55:47.629 INFO:tasks.workunit.client.1.smithi176.stdout:6/886: rename d1/dc/d5b/f6d to d1/d5/d47/d75/f141 0 2024-07-16T09:55:47.629 INFO:tasks.workunit.client.0.smithi156.stdout:4/857: write d4/db/f3f [673544,102363] 0 2024-07-16T09:55:47.631 INFO:tasks.workunit.client.1.smithi176.stdout:6/887: creat d1/d5/d47/f142 x:0 0 0 2024-07-16T09:55:47.631 INFO:tasks.workunit.client.1.smithi176.stdout:6/888: stat d1/d5/d47/d62/d77/dde 0 2024-07-16T09:55:47.633 INFO:tasks.workunit.client.1.smithi176.stdout:6/889: write d1/f3b [43010,58917] 0 2024-07-16T09:55:47.633 INFO:tasks.workunit.client.0.smithi156.stdout:9/875: chown d2/d5/d16/d22/d6b/d86/ce0 835 1 2024-07-16T09:55:47.633 INFO:tasks.workunit.client.0.smithi156.stdout:4/858: mknod d4/db/dc/d5f/d64/c131 0 2024-07-16T09:55:47.634 INFO:tasks.workunit.client.1.smithi176.stdout:6/890: dread - d1/d5d/d9e/dd9/d6f/d7c/fc3 zero size 2024-07-16T09:55:47.636 INFO:tasks.workunit.client.0.smithi156.stdout:9/876: stat d2/d5/d16/d22/d36/lbb 0 2024-07-16T09:55:47.636 INFO:tasks.workunit.client.0.smithi156.stdout:3/762: dwrite d8/da/d1d/f53 [0,4194304] 0 2024-07-16T09:55:47.637 INFO:tasks.workunit.client.0.smithi156.stdout:4/859: mknod d4/db/d47/dbe/c132 0 2024-07-16T09:55:47.638 INFO:tasks.workunit.client.0.smithi156.stdout:9/877: mkdir d2/d5/d16/de2/d115 0 2024-07-16T09:55:47.639 INFO:tasks.workunit.client.1.smithi176.stdout:6/891: creat d1/d5/d47/d62/f143 x:0 0 0 2024-07-16T09:55:47.640 INFO:tasks.workunit.client.1.smithi176.stdout:6/892: dread - d1/dc/ff4 zero size 2024-07-16T09:55:47.641 INFO:tasks.workunit.client.0.smithi156.stdout:3/763: symlink d8/da/d1d/d2c/de0/dad/l107 0 2024-07-16T09:55:47.641 INFO:tasks.workunit.client.0.smithi156.stdout:5/804: link d1b/d1f/d45/d52/d43/d85/d97/c8e d1b/c106 0 2024-07-16T09:55:47.644 INFO:tasks.workunit.client.0.smithi156.stdout:4/860: creat d4/d23/d30/d33/f133 x:0 0 0 2024-07-16T09:55:47.645 INFO:tasks.workunit.client.0.smithi156.stdout:3/764: write d8/da/fc [3869421,31855] 0 2024-07-16T09:55:47.651 INFO:tasks.workunit.client.1.smithi176.stdout:6/893: truncate d1/d5/d47/d75/f141 29894 0 2024-07-16T09:55:47.656 INFO:tasks.workunit.client.0.smithi156.stdout:5/805: dread d1b/d1f/d45/d52/d43/f64 [0,4194304] 0 2024-07-16T09:55:47.657 INFO:tasks.workunit.client.1.smithi176.stdout:6/894: dwrite d1/d5/d47/d62/f143 [0,4194304] 0 2024-07-16T09:55:47.658 INFO:tasks.workunit.client.0.smithi156.stdout:5/806: write d1b/d1f/d42/d6d/d9f/fda [1161891,31893] 0 2024-07-16T09:55:47.658 INFO:tasks.workunit.client.1.smithi176.stdout:6/895: readlink d1/dc/d40/l108 0 2024-07-16T09:55:47.661 INFO:tasks.workunit.client.0.smithi156.stdout:5/807: creat d1b/d1f/d45/d52/d43/d85/d97/d37/d88/dc7/f107 x:0 0 0 2024-07-16T09:55:47.662 INFO:tasks.workunit.client.1.smithi176.stdout:6/896: creat d1/d5d/d9e/dd9/d6f/d7c/f144 x:0 0 0 2024-07-16T09:55:47.662 INFO:tasks.workunit.client.0.smithi156.stdout:5/808: chown d1b/d1f/d45 39835 1 2024-07-16T09:55:47.663 INFO:tasks.workunit.client.0.smithi156.stdout:5/809: readlink d1b/d1f/d45/d52/d2d/l6c 0 2024-07-16T09:55:47.667 INFO:tasks.workunit.client.0.smithi156.stdout:9/878: dread d2/d5/d16/d57/d63/dd3/ddb/d74/dc2/dc4/d23/d111/da2/d98/fa7 [0,4194304] 0 2024-07-16T09:55:47.668 INFO:tasks.workunit.client.1.smithi176.stdout:6/897: rename d1/d5d/d9e/d8f/f74 to d1/d5d/d9e/dd9/d6f/dd0/f145 0 2024-07-16T09:55:47.668 INFO:tasks.workunit.client.0.smithi156.stdout:5/810: unlink d1b/d1f/d45/d52/d43/d85/c93 0 2024-07-16T09:55:47.670 INFO:tasks.workunit.client.1.smithi176.stdout:6/898: symlink d1/d5/d47/d62/de8/l146 0 2024-07-16T09:55:47.671 INFO:tasks.workunit.client.1.smithi176.stdout:6/899: write d1/d5/f98 [477408,8379] 0 2024-07-16T09:55:47.671 INFO:tasks.workunit.client.1.smithi176.stdout:2/876: sync 2024-07-16T09:55:47.673 INFO:tasks.workunit.client.0.smithi156.stdout:4/861: dread d4/db/dc/dcc/dec/ded/fdc [0,4194304] 0 2024-07-16T09:55:47.674 INFO:tasks.workunit.client.1.smithi176.stdout:2/877: readlink dd/l31 0 2024-07-16T09:55:47.675 INFO:tasks.workunit.client.0.smithi156.stdout:3/765: dread d8/da/d28/d43/fd9 [0,4194304] 0 2024-07-16T09:55:47.675 INFO:tasks.workunit.client.1.smithi176.stdout:6/900: creat d1/d5/d47/d62/de8/f147 x:0 0 0 2024-07-16T09:55:47.676 INFO:tasks.workunit.client.1.smithi176.stdout:2/878: mknod dd/dcd/c125 0 2024-07-16T09:55:47.677 INFO:tasks.workunit.client.0.smithi156.stdout:4/862: chown d4/d23/d52/d71/def/lf0 5 1 2024-07-16T09:55:47.680 INFO:tasks.workunit.client.0.smithi156.stdout:5/811: dwrite d1b/d1f/d45/d52/d43/d85/d97/d37/d88/d61/ff6 [0,4194304] 0 2024-07-16T09:55:47.681 INFO:tasks.workunit.client.1.smithi176.stdout:6/901: dwrite d1/d5/d7/f6e [0,4194304] 0 2024-07-16T09:55:47.683 INFO:tasks.workunit.client.1.smithi176.stdout:2/879: dwrite dd/d16/d39/d3d/d7e/fdf [0,4194304] 0 2024-07-16T09:55:47.683 INFO:tasks.workunit.client.0.smithi156.stdout:4/863: truncate d4/d23/ff7 339121 0 2024-07-16T09:55:47.687 INFO:tasks.workunit.client.0.smithi156.stdout:9/879: dwrite d2/d5/fb [0,4194304] 0 2024-07-16T09:55:47.687 INFO:tasks.workunit.client.0.smithi156.stdout:3/766: dwrite d8/ff4 [0,4194304] 0 2024-07-16T09:55:47.689 INFO:tasks.workunit.client.1.smithi176.stdout:2/880: symlink dd/d36/d45/dd3/l126 0 2024-07-16T09:55:47.692 INFO:tasks.workunit.client.1.smithi176.stdout:6/902: creat d1/d5d/d9e/dd9/d6f/f148 x:0 0 0 2024-07-16T09:55:47.697 INFO:tasks.workunit.client.1.smithi176.stdout:2/881: mkdir dd/d36/d127 0 2024-07-16T09:55:47.698 INFO:tasks.workunit.client.0.smithi156.stdout:3/767: creat d8/da/d1d/d2c/d7d/f108 x:0 0 0 2024-07-16T09:55:47.699 INFO:tasks.workunit.client.0.smithi156.stdout:3/768: chown d8/da/d13/f57 10 1 2024-07-16T09:55:47.703 INFO:tasks.workunit.client.0.smithi156.stdout:4/864: truncate d4/db/dc/dcc/dec/f10e 577969 0 2024-07-16T09:55:47.704 INFO:tasks.workunit.client.0.smithi156.stdout:3/769: chown d8/d14/f59 419589920 1 2024-07-16T09:55:47.705 INFO:tasks.workunit.client.1.smithi176.stdout:6/903: dwrite d1/db3/f9a [0,4194304] 0 2024-07-16T09:55:47.707 INFO:tasks.workunit.client.0.smithi156.stdout:4/865: fdatasync d4/db/dc/dcc/d58/f7a 0 2024-07-16T09:55:47.710 INFO:tasks.workunit.client.1.smithi176.stdout:6/904: creat d1/d5d/db9/d124/f149 x:0 0 0 2024-07-16T09:55:47.710 INFO:tasks.workunit.client.0.smithi156.stdout:9/880: dread d2/d5/d16/d22/d6b/f92 [0,4194304] 0 2024-07-16T09:55:47.710 INFO:tasks.workunit.client.0.smithi156.stdout:4/866: mkdir d4/d134 0 2024-07-16T09:55:47.713 INFO:tasks.workunit.client.0.smithi156.stdout:9/881: unlink d2/d5/d16/d57/d63/dd3/ddb/d74/dc2/cec 0 2024-07-16T09:55:47.713 INFO:tasks.workunit.client.0.smithi156.stdout:3/770: getdents d8/da/d28/d43/d63/df7 0 2024-07-16T09:55:47.715 INFO:tasks.workunit.client.1.smithi176.stdout:6/905: getdents d1/d5d/d9e 0 2024-07-16T09:55:47.717 INFO:tasks.workunit.client.0.smithi156.stdout:9/882: rmdir d2/d5/d16/d57/d63/dd3/ddb/d3d/da6 39 2024-07-16T09:55:47.719 INFO:tasks.workunit.client.0.smithi156.stdout:4/867: getdents d4/db/dc/dcc/dec/ded/db1 0 2024-07-16T09:55:47.719 INFO:tasks.workunit.client.1.smithi176.stdout:6/906: getdents d1/dc/d81/d10a/d115/d121 0 2024-07-16T09:55:47.720 INFO:tasks.workunit.client.0.smithi156.stdout:3/771: link d8/da/d28/d43/f5a d8/da/d21/d80/d99/f109 0 2024-07-16T09:55:47.722 INFO:tasks.workunit.client.1.smithi176.stdout:6/907: symlink d1/d5/d47/d62/de8/d111/d122/d12b/l14a 0 2024-07-16T09:55:47.722 INFO:tasks.workunit.client.1.smithi176.stdout:2/882: dread dd/d16/d39/d3d/d7e/f7f [0,4194304] 0 2024-07-16T09:55:47.723 INFO:tasks.workunit.client.0.smithi156.stdout:9/883: mknod d2/d5/d16/d57/d63/dd3/ddb/d74/dc2/dc4/c116 0 2024-07-16T09:55:47.727 INFO:tasks.workunit.client.0.smithi156.stdout:5/812: dwrite d1b/d1f/d45/d52/d43/d85/d97/f21 [0,4194304] 0 2024-07-16T09:55:47.727 INFO:tasks.workunit.client.1.smithi176.stdout:6/908: dread d1/d5/f2c [0,4194304] 0 2024-07-16T09:55:47.730 INFO:tasks.workunit.client.0.smithi156.stdout:3/772: rename d8/da/d1d/d2c/d7d/fa1 to d8/da/d1d/d2c/de0/df8/f10a 0 2024-07-16T09:55:47.730 INFO:tasks.workunit.client.1.smithi176.stdout:2/883: dwrite dd/d16/d39/d3d/d7e/fdc [0,4194304] 0 2024-07-16T09:55:47.733 INFO:tasks.workunit.client.0.smithi156.stdout:4/868: dread d4/d23/d52/f66 [0,4194304] 0 2024-07-16T09:55:47.736 INFO:tasks.workunit.client.0.smithi156.stdout:5/813: write d1b/d1f/d42/f5f [3909983,89816] 0 2024-07-16T09:55:47.738 INFO:tasks.workunit.client.0.smithi156.stdout:3/773: creat d8/da/d28/d43/d71/da2/f10b x:0 0 0 2024-07-16T09:55:47.739 INFO:tasks.workunit.client.0.smithi156.stdout:4/869: symlink d4/d23/d30/d33/l135 0 2024-07-16T09:55:47.739 INFO:tasks.workunit.client.1.smithi176.stdout:2/884: truncate dd/d49/d52/f79 172930 0 2024-07-16T09:55:47.741 INFO:tasks.workunit.client.1.smithi176.stdout:2/885: symlink dd/dcd/dfa/l128 0 2024-07-16T09:55:47.742 INFO:tasks.workunit.client.0.smithi156.stdout:5/814: readlink d1b/d1f/d45/d52/d2d/d96/lbf 0 2024-07-16T09:55:47.742 INFO:tasks.workunit.client.0.smithi156.stdout:4/870: chown d4/l1c 6435 1 2024-07-16T09:55:47.742 INFO:tasks.workunit.client.0.smithi156.stdout:9/884: dwrite d2/d5/d16/d57/d63/dd3/ddb/d74/dc2/dc4/d23/da0/faf [0,4194304] 0 2024-07-16T09:55:47.743 INFO:tasks.workunit.client.0.smithi156.stdout:9/885: rmdir d2/d5/d16/d22/d6b 39 2024-07-16T09:55:47.744 INFO:tasks.workunit.client.1.smithi176.stdout:2/886: creat dd/d49/d52/d7c/d81/f129 x:0 0 0 2024-07-16T09:55:47.745 INFO:tasks.workunit.client.0.smithi156.stdout:5/815: write d1b/d1f/d45/d52/f3c [9208686,117661] 0 2024-07-16T09:55:47.745 INFO:tasks.workunit.client.1.smithi176.stdout:2/887: stat dd/d49/l53 0 2024-07-16T09:55:47.745 INFO:tasks.workunit.client.0.smithi156.stdout:3/774: rename d8/da/f2e to d8/da/d1d/d6f/db3/f10c 0 2024-07-16T09:55:47.746 INFO:tasks.workunit.client.0.smithi156.stdout:3/775: fdatasync d8/da/fd7 0 2024-07-16T09:55:47.747 INFO:tasks.workunit.client.0.smithi156.stdout:4/871: creat d4/d23/d30/f136 x:0 0 0 2024-07-16T09:55:47.750 INFO:tasks.workunit.client.0.smithi156.stdout:5/816: creat d1b/d1f/d54/dc6/f108 x:0 0 0 2024-07-16T09:55:47.752 INFO:tasks.workunit.client.1.smithi176.stdout:2/888: dwrite dd/d36/d45/dd3/fe1 [0,4194304] 0 2024-07-16T09:55:47.752 INFO:tasks.workunit.client.0.smithi156.stdout:5/817: write d1b/d1f/d45/d52/d43/d85/d97/d37/f5a [1835205,13193] 0 2024-07-16T09:55:47.753 INFO:tasks.workunit.client.1.smithi176.stdout:2/889: chown dd/d36/ded/l10f 2007 1 2024-07-16T09:55:47.755 INFO:tasks.workunit.client.0.smithi156.stdout:3/776: mknod d8/da/d28/d43/d71/c10d 0 2024-07-16T09:55:47.757 INFO:tasks.workunit.client.1.smithi176.stdout:2/890: write dd/d2e/d35/d4a/fce [10108,114888] 0 2024-07-16T09:55:47.759 INFO:tasks.workunit.client.1.smithi176.stdout:2/891: write dd/d49/d52/d89/fee [47686,73755] 0 2024-07-16T09:55:47.762 INFO:tasks.workunit.client.0.smithi156.stdout:3/777: read f3 [6626357,106176] 0 2024-07-16T09:55:47.763 INFO:tasks.workunit.client.0.smithi156.stdout:5/818: getdents d1b/d1f/d45/d52/d43/d85 0 2024-07-16T09:55:47.765 INFO:tasks.workunit.client.1.smithi176.stdout:2/892: dwrite dd/d2e/d35/d59/fbc [0,4194304] 0 2024-07-16T09:55:47.766 INFO:tasks.workunit.client.1.smithi176.stdout:2/893: chown dd/d49/f78 204961 1 2024-07-16T09:55:47.770 INFO:tasks.workunit.client.0.smithi156.stdout:9/886: dwrite d2/d5/d16/f2a [0,4194304] 0 2024-07-16T09:55:47.770 INFO:tasks.workunit.client.1.smithi176.stdout:2/894: rmdir dd/d16 39 2024-07-16T09:55:47.772 INFO:tasks.workunit.client.1.smithi176.stdout:2/895: write dd/d16/fe4 [1021270,121748] 0 2024-07-16T09:55:47.773 INFO:tasks.workunit.client.0.smithi156.stdout:4/872: dwrite d4/db/f18 [0,4194304] 0 2024-07-16T09:55:47.777 INFO:tasks.workunit.client.0.smithi156.stdout:9/887: truncate d2/d5/ff4 911900 0 2024-07-16T09:55:47.778 INFO:tasks.workunit.client.1.smithi176.stdout:2/896: rename dd/d2e/d104 to dd/d36/d82/d12a 0 2024-07-16T09:55:47.779 INFO:tasks.workunit.client.1.smithi176.stdout:2/897: chown dd/d49/l53 2761 1 2024-07-16T09:55:47.779 INFO:tasks.workunit.client.0.smithi156.stdout:3/778: dwrite d8/da/d13/f57 [0,4194304] 0 2024-07-16T09:55:47.784 INFO:tasks.workunit.client.0.smithi156.stdout:4/873: write d4/db/dc/dcc/dec/ded/daf/f11d [188576,93304] 0 2024-07-16T09:55:47.784 INFO:tasks.workunit.client.0.smithi156.stdout:5/819: truncate d1b/d1f/d42/d6d/d9f/fda 878855 0 2024-07-16T09:55:47.786 INFO:tasks.workunit.client.0.smithi156.stdout:4/874: readlink d4/d23/l27 0 2024-07-16T09:55:47.787 INFO:tasks.workunit.client.1.smithi176.stdout:2/898: dwrite dd/d2e/d35/d59/d66/dc8/f103 [0,4194304] 0 2024-07-16T09:55:47.787 INFO:tasks.workunit.client.0.smithi156.stdout:9/888: rmdir d2/d5/d16/d57/d63/dd3/ddb/d74/dc2/de6 39 2024-07-16T09:55:47.790 INFO:tasks.workunit.client.0.smithi156.stdout:5/820: unlink d1b/l27 0 2024-07-16T09:55:47.791 INFO:tasks.workunit.client.0.smithi156.stdout:3/779: dwrite d8/d14/fb7 [0,4194304] 0 2024-07-16T09:55:47.791 INFO:tasks.workunit.client.1.smithi176.stdout:2/899: mkdir dd/d16/d39/d3d/d12b 0 2024-07-16T09:55:47.793 INFO:tasks.workunit.client.0.smithi156.stdout:9/889: symlink d2/d5/d16/de2/l117 0 2024-07-16T09:55:47.796 INFO:tasks.workunit.client.0.smithi156.stdout:3/780: symlink d8/l10e 0 2024-07-16T09:55:47.797 INFO:tasks.workunit.client.1.smithi176.stdout:2/900: rename c1 to dd/d49/d52/c12c 0 2024-07-16T09:55:47.798 INFO:tasks.workunit.client.0.smithi156.stdout:9/890: rmdir d2/d5/d16/d57/d63/dd3/ddb/d3d/da6/dd9 39 2024-07-16T09:55:47.801 INFO:tasks.workunit.client.0.smithi156.stdout:3/781: link d8/da/d28/f60 d8/da/dec/f10f 0 2024-07-16T09:55:47.803 INFO:tasks.workunit.client.0.smithi156.stdout:3/782: fdatasync d8/da/d1d/d2c/de0/fbc 0 2024-07-16T09:55:47.803 INFO:tasks.workunit.client.0.smithi156.stdout:9/891: creat d2/d5/d16/d57/f118 x:0 0 0 2024-07-16T09:55:47.804 INFO:tasks.workunit.client.0.smithi156.stdout:3/783: write d8/da/d28/ffa [795301,6548] 0 2024-07-16T09:55:47.805 INFO:tasks.workunit.client.0.smithi156.stdout:9/892: write d2/d5/d16/d57/d63/dd3/ddb/d74/dc2/dc4/d23/d111/da2/f93 [7377849,57606] 0 2024-07-16T09:55:47.806 INFO:tasks.workunit.client.0.smithi156.stdout:3/784: creat d8/da/d13/f110 x:0 0 0 2024-07-16T09:55:47.808 INFO:tasks.workunit.client.0.smithi156.stdout:9/893: rmdir d2/d5/d16/d57/d63/dd3/ddb/d74/dc2/dc4/d23/d111/da2 39 2024-07-16T09:55:47.809 INFO:tasks.workunit.client.0.smithi156.stdout:9/894: chown d2/d5/d16/d57/d63/dd3/ddb/dd0/def 6 1 2024-07-16T09:55:47.810 INFO:tasks.workunit.client.0.smithi156.stdout:9/895: stat d2/d5/d16/d57/d63/dd3/ddb/d3d/d3c/d7f/d42/c48 0 2024-07-16T09:55:47.811 INFO:tasks.workunit.client.0.smithi156.stdout:9/896: chown d2/d5/d16/d57/d63/dd3/ddb/dd0/fdc 12 1 2024-07-16T09:55:47.812 INFO:tasks.workunit.client.0.smithi156.stdout:9/897: write d2/d5/d16/d57/d63/dd3/ddb/dd0/def/ffe [493549,44388] 0 2024-07-16T09:55:47.819 INFO:tasks.workunit.client.0.smithi156.stdout:4/875: write d4/db/dc/d67/f92 [4733651,80175] 0 2024-07-16T09:55:47.820 INFO:tasks.workunit.client.1.smithi176.stdout:2/901: write dd/fd6 [155862,38337] 0 2024-07-16T09:55:47.823 INFO:tasks.workunit.client.1.smithi176.stdout:2/902: mkdir dd/d36/d45/dd3/d12d 0 2024-07-16T09:55:47.825 INFO:tasks.workunit.client.0.smithi156.stdout:4/876: chown d4/db/dc/d5f/d64/l65 59221919 1 2024-07-16T09:55:47.825 INFO:tasks.workunit.client.1.smithi176.stdout:2/903: creat dd/d36/f12e x:0 0 0 2024-07-16T09:55:47.826 INFO:tasks.workunit.client.0.smithi156.stdout:9/898: dread d2/d5/d16/d57/d63/dd3/ddb/d3d/d3c/f62 [0,4194304] 0 2024-07-16T09:55:47.828 INFO:tasks.workunit.client.0.smithi156.stdout:5/821: dwrite d1b/d1f/d45/d52/d43/d85/d97/d37/fb7 [0,4194304] 0 2024-07-16T09:55:47.828 INFO:tasks.workunit.client.0.smithi156.stdout:4/877: mkdir d4/db/d8a/d137 0 2024-07-16T09:55:47.829 INFO:tasks.workunit.client.1.smithi176.stdout:2/904: rename dd/d36/d82/f8d to dd/d16/d39/d3d/d12b/f12f 0 2024-07-16T09:55:47.829 INFO:tasks.workunit.client.1.smithi176.stdout:2/905: write dd/d16/d39/d3d/d7e/fc3 [5306891,63130] 0 2024-07-16T09:55:47.830 INFO:tasks.workunit.client.0.smithi156.stdout:3/785: dwrite d8/da/d28/f66 [0,4194304] 0 2024-07-16T09:55:47.835 INFO:tasks.workunit.client.0.smithi156.stdout:5/822: rmdir d1b/d1f/d45/dfb 0 2024-07-16T09:55:47.837 INFO:tasks.workunit.client.0.smithi156.stdout:3/786: dread d8/da/d28/d40/f54 [0,4194304] 0 2024-07-16T09:55:47.837 INFO:tasks.workunit.client.1.smithi176.stdout:2/906: dread dd/d16/d39/d3d/d12b/f12f [0,4194304] 0 2024-07-16T09:55:47.838 INFO:tasks.workunit.client.0.smithi156.stdout:4/878: link d4/d23/f59 d4/db/dc/dcc/dec/ded/d87/d108/f138 0 2024-07-16T09:55:47.840 INFO:tasks.workunit.client.0.smithi156.stdout:9/899: creat d2/d5/d16/d57/d63/dd3/ddb/d74/dc2/dc4/d23/d111/da2/d98/dbf/d10b/f119 x:0 0 0 2024-07-16T09:55:47.841 INFO:tasks.workunit.client.0.smithi156.stdout:5/823: mknod d1b/d99/dae/c109 0 2024-07-16T09:55:47.841 INFO:tasks.workunit.client.1.smithi176.stdout:2/907: rename dd/d16/f33 to dd/d9b/f130 0 2024-07-16T09:55:47.841 INFO:tasks.workunit.client.0.smithi156.stdout:3/787: rmdir d8/da/d1d/d2c/de0/df8 39 2024-07-16T09:55:47.844 INFO:tasks.workunit.client.1.smithi176.stdout:2/908: rename dd/d49/d52/d7c/d81/lc5 to dd/d36/d45/l131 0 2024-07-16T09:55:47.846 INFO:tasks.workunit.client.0.smithi156.stdout:9/900: fsync d2/d5/d16/d57/d63/dd3/ddb/d74/dc2/dc4/f82 0 2024-07-16T09:55:47.852 INFO:tasks.workunit.client.0.smithi156.stdout:3/788: dwrite d8/da/d1d/d2c/de0/df8/f10a [0,4194304] 0 2024-07-16T09:55:47.853 INFO:tasks.workunit.client.0.smithi156.stdout:9/901: dwrite d2/d5/d16/d57/d63/dd3/ddb/d3d/d3c/f107 [0,4194304] 0 2024-07-16T09:55:47.856 INFO:tasks.workunit.client.0.smithi156.stdout:3/789: unlink d8/da/d13/f5c 0 2024-07-16T09:55:47.856 INFO:tasks.workunit.client.0.smithi156.stdout:3/790: fsync d8/da/f32 0 2024-07-16T09:55:47.860 INFO:tasks.workunit.client.0.smithi156.stdout:3/791: truncate d8/da/d1d/d6f/db3/dc4/fef 639567 0 2024-07-16T09:55:47.860 INFO:tasks.workunit.client.0.smithi156.stdout:9/902: rename d2/d5/d16/d57/d63/dd3/ddb/d3d/d3c/d7f/d42/l45 to d2/l11a 0 2024-07-16T09:55:47.866 INFO:tasks.workunit.client.0.smithi156.stdout:4/879: write d4/d23/f78 [2858416,106517] 0 2024-07-16T09:55:47.867 INFO:tasks.workunit.client.0.smithi156.stdout:4/880: dread - d4/db/fee zero size 2024-07-16T09:55:47.868 INFO:tasks.workunit.client.0.smithi156.stdout:5/824: creat d1b/d1f/d45/d52/d2d/d96/f10a x:0 0 0 2024-07-16T09:55:47.869 INFO:tasks.workunit.client.1.smithi176.stdout:2/909: dwrite dd/d2e/d35/d59/d66/f6e [0,4194304] 0 2024-07-16T09:55:47.869 INFO:tasks.workunit.client.0.smithi156.stdout:3/792: dwrite d8/da/d1d/f8d [0,4194304] 0 2024-07-16T09:55:47.870 INFO:tasks.workunit.client.1.smithi176.stdout:5/868: sync 2024-07-16T09:55:47.871 INFO:tasks.workunit.client.0.smithi156.stdout:9/903: rmdir d2/d5/d16/d57/d63/dd3/ddb/d3d/d3c/d7f/d10e 0 2024-07-16T09:55:47.871 INFO:tasks.workunit.client.1.smithi176.stdout:2/910: dread - dd/d2e/d35/d59/d66/ffb zero size 2024-07-16T09:55:47.871 INFO:tasks.workunit.client.1.smithi176.stdout:5/869: chown d4/fea 650082 1 2024-07-16T09:55:47.872 INFO:tasks.workunit.client.1.smithi176.stdout:2/911: fdatasync dd/d16/fe4 0 2024-07-16T09:55:47.873 INFO:tasks.workunit.client.0.smithi156.stdout:9/904: write d2/d5/f70 [3018806,49298] 0 2024-07-16T09:55:47.873 INFO:tasks.workunit.client.0.smithi156.stdout:4/881: chown d4/db/dc/dcc/dec/ded/daf/lb6 185 1 2024-07-16T09:55:47.875 INFO:tasks.workunit.client.0.smithi156.stdout:9/905: chown d2/d5/d16/d57/d63/dd3/ddb/d74 0 1 2024-07-16T09:55:47.877 INFO:tasks.workunit.client.1.smithi176.stdout:2/912: mknod dd/d2e/d35/c132 0 2024-07-16T09:55:47.880 INFO:tasks.workunit.client.0.smithi156.stdout:4/882: chown d4/db/dc/l4f 5256 1 2024-07-16T09:55:47.880 INFO:tasks.workunit.client.0.smithi156.stdout:4/883: readlink d4/db/dc/dcc/dec/ded/l116 0 2024-07-16T09:55:47.882 INFO:tasks.workunit.client.0.smithi156.stdout:3/793: truncate d8/d14/fc7 2490926 0 2024-07-16T09:55:47.884 INFO:tasks.workunit.client.1.smithi176.stdout:2/913: unlink dd/d49/d52/faa 0 2024-07-16T09:55:47.884 INFO:tasks.workunit.client.1.smithi176.stdout:2/914: fsync dd/d2e/f3e 0 2024-07-16T09:55:47.885 INFO:tasks.workunit.client.0.smithi156.stdout:4/884: creat d4/db/dc/dcc/dec/ded/daf/f139 x:0 0 0 2024-07-16T09:55:47.885 INFO:tasks.workunit.client.0.smithi156.stdout:5/825: dwrite d1b/d1f/d45/d52/d43/d85/d97/d37/f5a [0,4194304] 0 2024-07-16T09:55:47.886 INFO:tasks.workunit.client.0.smithi156.stdout:5/826: chown d1b/d1f 3439559 1 2024-07-16T09:55:47.886 INFO:tasks.workunit.client.0.smithi156.stdout:3/794: truncate d8/da/d28/d43/d71/da2/f10b 228875 0 2024-07-16T09:55:47.887 INFO:tasks.workunit.client.1.smithi176.stdout:2/915: creat dd/d49/d52/d10a/f133 x:0 0 0 2024-07-16T09:55:47.889 INFO:tasks.workunit.client.1.smithi176.stdout:2/916: rmdir dd/dcd 39 2024-07-16T09:55:47.890 INFO:tasks.workunit.client.0.smithi156.stdout:3/795: mkdir d8/da/dec/d111 0 2024-07-16T09:55:47.890 INFO:tasks.workunit.client.1.smithi176.stdout:2/917: write dd/d16/d39/f4b [3747678,4383] 0 2024-07-16T09:55:47.890 INFO:tasks.workunit.client.1.smithi176.stdout:2/918: fdatasync dd/de/fa8 0 2024-07-16T09:55:47.891 INFO:tasks.workunit.client.1.smithi176.stdout:2/919: stat dd/d2e 0 2024-07-16T09:55:47.892 INFO:tasks.workunit.client.1.smithi176.stdout:0/918: sync 2024-07-16T09:55:47.892 INFO:tasks.workunit.client.1.smithi176.stdout:9/904: sync 2024-07-16T09:55:47.892 INFO:tasks.workunit.client.1.smithi176.stdout:8/760: sync 2024-07-16T09:55:47.892 INFO:tasks.workunit.client.1.smithi176.stdout:4/763: sync 2024-07-16T09:55:47.892 INFO:tasks.workunit.client.1.smithi176.stdout:7/726: sync 2024-07-16T09:55:47.892 INFO:tasks.workunit.client.1.smithi176.stdout:1/555: sync 2024-07-16T09:55:47.893 INFO:tasks.workunit.client.1.smithi176.stdout:3/642: sync 2024-07-16T09:55:47.893 INFO:tasks.workunit.client.1.smithi176.stdout:6/909: sync 2024-07-16T09:55:47.893 INFO:tasks.workunit.client.0.smithi156.stdout:3/796: truncate d8/da/d1d/d2c/de0/fbf 4356129 0 2024-07-16T09:55:47.895 INFO:tasks.workunit.client.0.smithi156.stdout:5/827: link d1b/d1f/d45/d52/d43/d85/d97/f25 d1b/f10b 0 2024-07-16T09:55:47.896 INFO:tasks.workunit.client.1.smithi176.stdout:2/920: chown dd/dcd 21 1 2024-07-16T09:55:47.897 INFO:tasks.workunit.client.0.smithi156.stdout:3/797: creat d8/da/d28/d43/dd8/f112 x:0 0 0 2024-07-16T09:55:47.897 INFO:tasks.workunit.client.0.smithi156.stdout:5/828: readlink d1b/d1f/d45/d52/d2d/d96/lbf 0 2024-07-16T09:55:47.898 INFO:tasks.workunit.client.1.smithi176.stdout:3/643: dread - d3/d17/fb9 zero size 2024-07-16T09:55:47.898 INFO:tasks.workunit.client.1.smithi176.stdout:2/921: chown dd/d49/l53 0 1 2024-07-16T09:55:47.899 INFO:tasks.workunit.client.1.smithi176.stdout:9/905: rmdir d5/de/da5 39 2024-07-16T09:55:47.899 INFO:tasks.workunit.client.1.smithi176.stdout:0/919: creat d5/d39/dde/f12f x:0 0 0 2024-07-16T09:55:47.899 INFO:tasks.workunit.client.0.smithi156.stdout:4/885: truncate d4/db/dc/f106 1580224 0 2024-07-16T09:55:47.902 INFO:tasks.workunit.client.0.smithi156.stdout:4/886: creat d4/db/d47/dbe/f13a x:0 0 0 2024-07-16T09:55:47.903 INFO:tasks.workunit.client.1.smithi176.stdout:6/910: chown d1/d5d/d9e/d76/da5/fb7 40 1 2024-07-16T09:55:47.903 INFO:tasks.workunit.client.1.smithi176.stdout:1/556: write d4/d8/fb5 [1031918,107225] 0 2024-07-16T09:55:47.904 INFO:tasks.workunit.client.0.smithi156.stdout:5/829: truncate d1b/d1f/d45/d7f/f84 947889 0 2024-07-16T09:55:47.904 INFO:tasks.workunit.client.1.smithi176.stdout:9/906: chown d5/de/f17 1954979198 1 2024-07-16T09:55:47.905 INFO:tasks.workunit.client.0.smithi156.stdout:4/887: mknod d4/db/dc/dcc/df3/c13b 0 2024-07-16T09:55:47.906 INFO:tasks.workunit.client.1.smithi176.stdout:7/727: write d4/dd/f15 [835875,27709] 0 2024-07-16T09:55:47.907 INFO:tasks.workunit.client.1.smithi176.stdout:3/644: mkdir d3/da/d66/dc4 0 2024-07-16T09:55:47.907 INFO:tasks.workunit.client.1.smithi176.stdout:5/870: dread d4/d37/d72/d82/f64 [0,4194304] 0 2024-07-16T09:55:47.907 INFO:tasks.workunit.client.1.smithi176.stdout:4/764: truncate da/dc/d4b/df0/d96/da4/dc2/ff7 822155 0 2024-07-16T09:55:47.908 INFO:tasks.workunit.client.1.smithi176.stdout:2/922: rmdir dd/d16/d39/d3d/d12b 39 2024-07-16T09:55:47.908 INFO:tasks.workunit.client.0.smithi156.stdout:4/888: mknod d4/c13c 0 2024-07-16T09:55:47.908 INFO:tasks.workunit.client.0.smithi156.stdout:9/906: dread d2/d5/d16/d57/d63/dd3/ddb/d74/dc2/dc4/d23/d111/da2/d98/fc6 [0,4194304] 0 2024-07-16T09:55:47.910 INFO:tasks.workunit.client.1.smithi176.stdout:6/911: write d1/dc/fd [2543089,106952] 0 2024-07-16T09:55:47.911 INFO:tasks.workunit.client.1.smithi176.stdout:0/920: mkdir d5/d1c/d24/d35/d130 0 2024-07-16T09:55:47.913 INFO:tasks.workunit.client.1.smithi176.stdout:1/557: mkdir d4/d8/d12/d21/d34/daa/dbb 0 2024-07-16T09:55:47.913 INFO:tasks.workunit.client.1.smithi176.stdout:4/765: write da/dc/d4b/df0/d96/d69/fb0 [3893670,101218] 0 2024-07-16T09:55:47.915 INFO:tasks.workunit.client.1.smithi176.stdout:5/871: readlink d4/d37/d72/d82/l8e 0 2024-07-16T09:55:47.915 INFO:tasks.workunit.client.1.smithi176.stdout:3/645: chown d3/d17/d1d/d4f/d59/da0/d7a/c96 4 1 2024-07-16T09:55:47.916 INFO:tasks.workunit.client.0.smithi156.stdout:5/830: link d1b/d1f/d45/fc1 d1b/d1f/d45/d52/d43/f10c 0 2024-07-16T09:55:47.916 INFO:tasks.workunit.client.0.smithi156.stdout:0/867: sync 2024-07-16T09:55:47.916 INFO:tasks.workunit.client.0.smithi156.stdout:2/889: sync 2024-07-16T09:55:47.916 INFO:tasks.workunit.client.1.smithi176.stdout:8/761: dwrite d8/d23/d100/db9/dcb/d34/d62/fa4 [0,4194304] 0 2024-07-16T09:55:47.916 INFO:tasks.workunit.client.1.smithi176.stdout:9/907: creat d5/de/f132 x:0 0 0 2024-07-16T09:55:47.916 INFO:tasks.workunit.client.1.smithi176.stdout:7/728: creat d4/d5/d97/fee x:0 0 0 2024-07-16T09:55:47.917 INFO:tasks.workunit.client.0.smithi156.stdout:3/798: write d8/da/d1d/f58 [2494948,59831] 0 2024-07-16T09:55:47.918 INFO:tasks.workunit.client.1.smithi176.stdout:6/912: fdatasync d1/d5/d47/fd4 0 2024-07-16T09:55:47.918 INFO:tasks.workunit.client.1.smithi176.stdout:0/921: write d5/f13 [2648723,37319] 0 2024-07-16T09:55:47.921 INFO:tasks.workunit.client.1.smithi176.stdout:8/762: fsync d8/d23/d76/f98 0 2024-07-16T09:55:47.923 INFO:tasks.workunit.client.1.smithi176.stdout:4/766: mknod da/d3d/d17/d54/d78/d7a/cfa 0 2024-07-16T09:55:47.923 INFO:tasks.workunit.client.0.smithi156.stdout:3/799: truncate d8/da/d1d/d2c/ffc 973926 0 2024-07-16T09:55:47.924 INFO:tasks.workunit.client.1.smithi176.stdout:4/767: chown da/d3d/d33/fcd 15379 1 2024-07-16T09:55:47.924 INFO:tasks.workunit.client.1.smithi176.stdout:4/768: dread - da/dc/d4b/df0/d28/fb7 zero size 2024-07-16T09:55:47.930 INFO:tasks.workunit.client.0.smithi156.stdout:5/831: dwrite d1b/d1f/d45/d52/d43/d7c/fc0 [0,4194304] 0 2024-07-16T09:55:47.934 INFO:tasks.workunit.client.1.smithi176.stdout:1/558: rmdir d4/d8/d12 39 2024-07-16T09:55:47.934 INFO:tasks.workunit.client.0.smithi156.stdout:4/889: dwrite d4/db/dc/dcc/dec/ded/fcf [0,4194304] 0 2024-07-16T09:55:47.937 INFO:tasks.workunit.client.0.smithi156.stdout:0/868: creat d0/d1a/d4d/d9/f115 x:0 0 0 2024-07-16T09:55:47.937 INFO:tasks.workunit.client.1.smithi176.stdout:2/923: creat dd/d16/d39/d3d/f134 x:0 0 0 2024-07-16T09:55:47.938 INFO:tasks.workunit.client.0.smithi156.stdout:9/907: dread d2/d5/d16/d57/d63/dd3/ddb/d74/dc2/dc4/d3a/fd8 [0,4194304] 0 2024-07-16T09:55:47.939 INFO:tasks.workunit.client.1.smithi176.stdout:0/922: write d5/d25/d82/f87 [3288076,78638] 0 2024-07-16T09:55:47.943 INFO:tasks.workunit.client.0.smithi156.stdout:2/890: dwrite d0/d5/d29/d2d/d5b/d5f/ddc/f8f [0,4194304] 0 2024-07-16T09:55:47.943 INFO:tasks.workunit.client.0.smithi156.stdout:5/832: truncate d1b/d1f/d45/d52/d43/d7c/f82 4394227 0 2024-07-16T09:55:47.943 INFO:tasks.workunit.client.1.smithi176.stdout:9/908: rename d5/de/d30/cfa to d5/d1d/db3/d11f/c133 0 2024-07-16T09:55:47.943 INFO:tasks.workunit.client.1.smithi176.stdout:1/559: truncate d4/f24 2458900 0 2024-07-16T09:55:47.944 INFO:tasks.workunit.client.0.smithi156.stdout:4/890: mkdir d4/db/dc/dcc/dec/ded/db1/d13d 0 2024-07-16T09:55:47.946 INFO:tasks.workunit.client.1.smithi176.stdout:3/646: creat d3/d17/d1d/d4f/fc5 x:0 0 0 2024-07-16T09:55:47.947 INFO:tasks.workunit.client.0.smithi156.stdout:4/891: dread - d4/db/d47/dbe/f13a zero size 2024-07-16T09:55:47.947 INFO:tasks.workunit.client.0.smithi156.stdout:3/800: dwrite d8/da/f32 [0,4194304] 0 2024-07-16T09:55:47.947 INFO:tasks.workunit.client.1.smithi176.stdout:4/769: symlink da/dc/lfb 0 2024-07-16T09:55:47.949 INFO:tasks.workunit.client.1.smithi176.stdout:5/872: dwrite d4/d10/fb9 [0,4194304] 0 2024-07-16T09:55:47.951 INFO:tasks.workunit.client.0.smithi156.stdout:0/869: rename d0/d1a/de6 to d0/d1a/d4d/d5e/db7/d116 0 2024-07-16T09:55:47.951 INFO:tasks.workunit.client.0.smithi156.stdout:5/833: write d1b/d1f/d45/d52/d2d/fb4 [605197,110326] 0 2024-07-16T09:55:47.953 INFO:tasks.workunit.client.1.smithi176.stdout:4/770: write da/dc/d4b/fa7 [4241232,30367] 0 2024-07-16T09:55:47.957 INFO:tasks.workunit.client.0.smithi156.stdout:2/891: stat d0/d5/d29/d2d/de1/l10c 0 2024-07-16T09:55:47.957 INFO:tasks.workunit.client.0.smithi156.stdout:3/801: write d8/da/d13/f57 [1464990,27066] 0 2024-07-16T09:55:47.957 INFO:tasks.workunit.client.1.smithi176.stdout:4/771: write da/d3d/d33/fdd [4767472,7141] 0 2024-07-16T09:55:47.958 INFO:tasks.workunit.client.1.smithi176.stdout:8/763: getdents d8/d23/d76 0 2024-07-16T09:55:47.960 INFO:tasks.workunit.client.1.smithi176.stdout:3/647: creat d3/d17/d1d/d4f/d59/da0/d7a/fc6 x:0 0 0 2024-07-16T09:55:47.962 INFO:tasks.workunit.client.1.smithi176.stdout:7/729: rename d4/dd/db5/cd3 to d4/dd/d26/dc2/d17/d1d/d8b/cef 0 2024-07-16T09:55:47.962 INFO:tasks.workunit.client.0.smithi156.stdout:4/892: rename d4/d23/d30/c6f to d4/db/dc/dcc/d58/d8f/c13e 0 2024-07-16T09:55:47.964 INFO:tasks.workunit.client.1.smithi176.stdout:4/772: creat da/dc/d4b/df0/d28/ffc x:0 0 0 2024-07-16T09:55:47.964 INFO:tasks.workunit.client.0.smithi156.stdout:9/908: dwrite d2/d9b/d109/f10d [0,4194304] 0 2024-07-16T09:55:47.965 INFO:tasks.workunit.client.1.smithi176.stdout:9/909: creat d5/de/da5/dc1/f134 x:0 0 0 2024-07-16T09:55:47.967 INFO:tasks.workunit.client.0.smithi156.stdout:3/802: fdatasync d8/da/d13/fb9 0 2024-07-16T09:55:47.967 INFO:tasks.workunit.client.0.smithi156.stdout:1/945: sync 2024-07-16T09:55:47.967 INFO:tasks.workunit.client.0.smithi156.stdout:7/643: sync 2024-07-16T09:55:47.967 INFO:tasks.workunit.client.0.smithi156.stdout:8/705: sync 2024-07-16T09:55:47.967 INFO:tasks.workunit.client.0.smithi156.stdout:6/705: sync 2024-07-16T09:55:47.967 INFO:tasks.workunit.client.1.smithi176.stdout:1/560: creat d4/d8/d12/d4c/da4/fbc x:0 0 0 2024-07-16T09:55:47.969 INFO:tasks.workunit.client.1.smithi176.stdout:8/764: truncate d8/f4c 349114 0 2024-07-16T09:55:47.971 INFO:tasks.workunit.client.0.smithi156.stdout:8/706: stat d0/ce 0 2024-07-16T09:55:47.971 INFO:tasks.workunit.client.0.smithi156.stdout:4/893: fsync d4/db/dc/dcc/dec/f12c 0 2024-07-16T09:55:47.972 INFO:tasks.workunit.client.0.smithi156.stdout:3/803: fdatasync d8/da/d1d/f58 0 2024-07-16T09:55:47.973 INFO:tasks.workunit.client.1.smithi176.stdout:6/913: rename d1/dc/d81/fdd to d1/db3/f14b 0 2024-07-16T09:55:47.973 INFO:tasks.workunit.client.1.smithi176.stdout:2/924: getdents dd/d2e/d35/d4a/df9 0 2024-07-16T09:55:47.974 INFO:tasks.workunit.client.0.smithi156.stdout:5/834: creat d1b/d1f/d45/d52/d43/d85/d97/d37/d88/dbc/f10d x:0 0 0 2024-07-16T09:55:47.974 INFO:tasks.workunit.client.0.smithi156.stdout:1/946: chown dc/d14/d3e/d6c/d7a/f132 11134 1 2024-07-16T09:55:47.975 INFO:tasks.workunit.client.1.smithi176.stdout:2/925: chown dd/d49/d52/d7c/ld0 0 1 2024-07-16T09:55:47.975 INFO:tasks.workunit.client.1.smithi176.stdout:7/730: mknod d4/dd/d49/d7a/db1/cf0 0 2024-07-16T09:55:47.977 INFO:tasks.workunit.client.0.smithi156.stdout:3/804: write d8/da/d1d/f53 [3664326,21721] 0 2024-07-16T09:55:47.978 INFO:tasks.workunit.client.0.smithi156.stdout:6/706: getdents d3/d13/d17/d4f/d4d/dbb 0 2024-07-16T09:55:47.980 INFO:tasks.workunit.client.0.smithi156.stdout:2/892: creat d0/d5/d29/d2d/d5b/d5f/ddc/dc3/f11f x:0 0 0 2024-07-16T09:55:47.980 INFO:tasks.workunit.client.0.smithi156.stdout:9/909: truncate d2/d5/d16/d57/d63/dd3/ddb/f8b 1974467 0 2024-07-16T09:55:47.980 INFO:tasks.workunit.client.0.smithi156.stdout:4/894: dread d4/d23/d30/fd6 [0,4194304] 0 2024-07-16T09:55:47.980 INFO:tasks.workunit.client.0.smithi156.stdout:8/707: creat d0/d11/d20/d3f/dbe/ffc x:0 0 0 2024-07-16T09:55:47.981 INFO:tasks.workunit.client.0.smithi156.stdout:2/893: stat d0/d5/d76/da1/fd6 0 2024-07-16T09:55:47.981 INFO:tasks.workunit.client.0.smithi156.stdout:5/835: mkdir d1b/d1f/d54/dc6/d10e 0 2024-07-16T09:55:47.983 INFO:tasks.workunit.client.0.smithi156.stdout:8/708: chown d0/d11/d2e/l77 24 1 2024-07-16T09:55:47.983 INFO:tasks.workunit.client.0.smithi156.stdout:4/895: fsync d4/db/dc/dcc/dec/ded/daf/f139 0 2024-07-16T09:55:47.985 INFO:tasks.workunit.client.0.smithi156.stdout:5/836: chown d1b/d1f/d45/d52/d43/d85/d97/d37/l3f 5156 1 2024-07-16T09:55:47.985 INFO:tasks.workunit.client.0.smithi156.stdout:8/709: fsync d0/d11/d13/d1c/fbd 0 2024-07-16T09:55:47.985 INFO:tasks.workunit.client.0.smithi156.stdout:9/910: creat d2/d5/d16/d22/d36/deb/f11b x:0 0 0 2024-07-16T09:55:47.986 INFO:tasks.workunit.client.0.smithi156.stdout:2/894: chown d0/d5/d29/d2d/d5b/d62/fb9 1873505 1 2024-07-16T09:55:47.986 INFO:tasks.workunit.client.0.smithi156.stdout:6/707: creat d3/d13/d17/d4f/d4d/d3b/d5a/dbe/fe6 x:0 0 0 2024-07-16T09:55:47.988 INFO:tasks.workunit.client.0.smithi156.stdout:4/896: creat d4/d23/d52/d71/f13f x:0 0 0 2024-07-16T09:55:47.992 INFO:tasks.workunit.client.0.smithi156.stdout:3/805: dwrite d8/da/d1d/f53 [4194304,4194304] 0 2024-07-16T09:55:47.992 INFO:tasks.workunit.client.0.smithi156.stdout:2/895: creat d0/d5/d29/d2d/d5b/d5f/ddc/f120 x:0 0 0 2024-07-16T09:55:47.993 INFO:tasks.workunit.client.0.smithi156.stdout:4/897: unlink d4/d23/d52/c11e 0 2024-07-16T09:55:47.997 INFO:tasks.workunit.client.0.smithi156.stdout:8/710: symlink d0/d11/d29/d80/d97/de7/lfd 0 2024-07-16T09:55:48.001 INFO:tasks.workunit.client.0.smithi156.stdout:2/896: symlink d0/d5/d29/d2d/de1/l121 0 2024-07-16T09:55:48.001 INFO:tasks.workunit.client.0.smithi156.stdout:0/870: truncate d0/d1a/d4d/d9/d27/f6a 2454410 0 2024-07-16T09:55:48.002 INFO:tasks.workunit.client.0.smithi156.stdout:4/898: fdatasync d4/db/d8a/ffd 0 2024-07-16T09:55:48.003 INFO:tasks.workunit.client.0.smithi156.stdout:5/837: dwrite d1b/d1f/d45/d52/d43/d7c/fa2 [4194304,4194304] 0 2024-07-16T09:55:48.003 INFO:tasks.workunit.client.0.smithi156.stdout:0/871: chown d0/d1a/d4d/d9/d27 55194 1 2024-07-16T09:55:48.004 INFO:tasks.workunit.client.0.smithi156.stdout:2/897: chown d0/d13/d17/d1c/d86/ff5 58 1 2024-07-16T09:55:48.006 INFO:tasks.workunit.client.0.smithi156.stdout:4/899: write d4/db/dc/dcc/dec/ded/daf/f139 [226622,15858] 0 2024-07-16T09:55:48.007 INFO:tasks.workunit.client.0.smithi156.stdout:9/911: creat d2/d5/d16/d57/d63/dd3/ddb/d74/dc2/dc4/d23/f11c x:0 0 0 2024-07-16T09:55:48.007 INFO:tasks.workunit.client.0.smithi156.stdout:0/872: truncate d0/d1a/d76/d83/fad 778957 0 2024-07-16T09:55:48.007 INFO:tasks.workunit.client.0.smithi156.stdout:3/806: creat d8/da/d28/f113 x:0 0 0 2024-07-16T09:55:48.008 INFO:tasks.workunit.client.0.smithi156.stdout:2/898: symlink d0/d5/d29/d2d/d5b/d5f/ddc/l122 0 2024-07-16T09:55:48.008 INFO:tasks.workunit.client.0.smithi156.stdout:4/900: chown d4/d23/d52/d71/l89 7966 1 2024-07-16T09:55:48.010 INFO:tasks.workunit.client.0.smithi156.stdout:3/807: chown d8/l12 12516442 1 2024-07-16T09:55:48.013 INFO:tasks.workunit.client.0.smithi156.stdout:9/912: symlink d2/d5/d16/d57/d63/dd3/ddb/d74/dc2/de6/l11d 0 2024-07-16T09:55:48.015 INFO:tasks.workunit.client.0.smithi156.stdout:2/899: mknod d0/d5/d29/d2d/d5b/d5f/ddc/dc3/c123 0 2024-07-16T09:55:48.016 INFO:tasks.workunit.client.0.smithi156.stdout:8/711: getdents d0/dd/d1f/d57 0 2024-07-16T09:55:48.018 INFO:tasks.workunit.client.0.smithi156.stdout:3/808: rename d8/da/d28/d43/dd8/dda to d8/da/d28/d43/d71/da2/d114 0 2024-07-16T09:55:48.018 INFO:tasks.workunit.client.0.smithi156.stdout:0/873: getdents d0/d1a/d4d/d9/d27/d2c/d6d/d9d/dbb/d106 0 2024-07-16T09:55:48.021 INFO:tasks.workunit.client.0.smithi156.stdout:6/708: truncate d3/d13/d17/d4f/d4d/d3b/f47 3583437 0 2024-07-16T09:55:48.022 INFO:tasks.workunit.client.0.smithi156.stdout:6/709: fsync d3/d13/d17/d4f/d30/f7f 0 2024-07-16T09:55:48.023 INFO:tasks.workunit.client.0.smithi156.stdout:8/712: symlink d0/dd/d1f/d50/d8d/lfe 0 2024-07-16T09:55:48.024 INFO:tasks.workunit.client.0.smithi156.stdout:4/901: rename d4/d23/d30/l12a to d4/d6a/l140 0 2024-07-16T09:55:48.026 INFO:tasks.workunit.client.0.smithi156.stdout:3/809: write d8/da/d28/d43/fd9 [2670249,67481] 0 2024-07-16T09:55:48.026 INFO:tasks.workunit.client.0.smithi156.stdout:4/902: write d4/db/dc/dcc/dec/ded/daf/f125 [846432,118244] 0 2024-07-16T09:55:48.026 INFO:tasks.workunit.client.0.smithi156.stdout:9/913: mknod d2/d5/d16/d57/d63/dd3/ddb/d3d/d3c/d7f/d114/c11e 0 2024-07-16T09:55:48.027 INFO:tasks.workunit.client.0.smithi156.stdout:2/900: mknod d0/d13/dc9/c124 0 2024-07-16T09:55:48.031 INFO:tasks.workunit.client.0.smithi156.stdout:8/713: mkdir d0/dd/d1f/d57/dc0/dff 0 2024-07-16T09:55:48.033 INFO:tasks.workunit.client.0.smithi156.stdout:0/874: rename d0/d1a/d4d/d9/d27/d50/d90 to d0/d1a/d4d/d5e/d117 0 2024-07-16T09:55:48.037 INFO:tasks.workunit.client.0.smithi156.stdout:2/901: symlink d0/d5/d76/l125 0 2024-07-16T09:55:48.040 INFO:tasks.workunit.client.0.smithi156.stdout:2/902: chown d0/d13/d17/d1c/d51/fc2 6 1 2024-07-16T09:55:48.040 INFO:tasks.workunit.client.0.smithi156.stdout:4/903: dread d4/d23/d52/d71/def/f110 [0,4194304] 0 2024-07-16T09:55:48.043 INFO:tasks.workunit.client.0.smithi156.stdout:4/904: write d4/d23/ff7 [261177,19763] 0 2024-07-16T09:55:48.043 INFO:tasks.workunit.client.0.smithi156.stdout:2/903: dread - d0/d5/d29/d2d/d5b/ddb/df9/f11d zero size 2024-07-16T09:55:48.045 INFO:tasks.workunit.client.0.smithi156.stdout:3/810: getdents d8/da/d21/d80 0 2024-07-16T09:55:48.046 INFO:tasks.workunit.client.0.smithi156.stdout:8/714: dread d0/dd/f58 [0,4194304] 0 2024-07-16T09:55:48.048 INFO:tasks.workunit.client.0.smithi156.stdout:2/904: chown d0/d5/d76/da1/la8 179 1 2024-07-16T09:55:48.049 INFO:tasks.workunit.client.0.smithi156.stdout:9/914: dread d2/d5/d16/d57/d63/dd3/ddb/d74/dc2/dc4/d23/fbc [0,4194304] 0 2024-07-16T09:55:48.049 INFO:tasks.workunit.client.0.smithi156.stdout:3/811: fdatasync d8/da/d28/d43/fbe 0 2024-07-16T09:55:48.049 INFO:tasks.workunit.client.1.smithi176.stdout:4/773: mknod da/dc/d4b/df0/d96/d9b/db4/d40/d5d/d73/cfd 0 2024-07-16T09:55:48.050 INFO:tasks.workunit.client.1.smithi176.stdout:4/774: write da/fcf [881515,76310] 0 2024-07-16T09:55:48.050 INFO:tasks.workunit.client.0.smithi156.stdout:4/905: creat d4/db/dc/d5f/f141 x:0 0 0 2024-07-16T09:55:48.050 INFO:tasks.workunit.client.1.smithi176.stdout:9/910: creat d5/d1d/dd2/f135 x:0 0 0 2024-07-16T09:55:48.052 INFO:tasks.workunit.client.0.smithi156.stdout:6/710: dwrite d3/f1c [0,4194304] 0 2024-07-16T09:55:48.052 INFO:tasks.workunit.client.0.smithi156.stdout:2/905: mkdir d0/d9b/d126 0 2024-07-16T09:55:48.053 INFO:tasks.workunit.client.1.smithi176.stdout:8/765: rmdir d8/d20 39 2024-07-16T09:55:48.054 INFO:tasks.workunit.client.0.smithi156.stdout:1/947: mkdir dc/d6b/d104/d13a 0 2024-07-16T09:55:48.058 INFO:tasks.workunit.client.1.smithi176.stdout:8/766: fsync d8/d23/d100/d53/fb7 0 2024-07-16T09:55:48.058 INFO:tasks.workunit.client.1.smithi176.stdout:8/767: dread d8/d14/d1e/d7c/f85 [0,4194304] 0 2024-07-16T09:55:48.058 INFO:tasks.workunit.client.0.smithi156.stdout:8/715: truncate d0/d11/d62/dc2/fc9 32952 0 2024-07-16T09:55:48.058 INFO:tasks.workunit.client.0.smithi156.stdout:8/716: readlink d0/dd/l66 0 2024-07-16T09:55:48.059 INFO:tasks.workunit.client.0.smithi156.stdout:0/875: dread d0/d1a/d4d/d25/f55 [0,4194304] 0 2024-07-16T09:55:48.059 INFO:tasks.workunit.client.0.smithi156.stdout:3/812: creat d8/da/d1d/d2c/dee/f115 x:0 0 0 2024-07-16T09:55:48.059 INFO:tasks.workunit.client.0.smithi156.stdout:2/906: mknod d0/d9b/d126/c127 0 2024-07-16T09:55:48.059 INFO:tasks.workunit.client.0.smithi156.stdout:6/711: chown d3/d13/d17/d4f/d4d/d3b/d77/l9e 3 1 2024-07-16T09:55:48.059 INFO:tasks.workunit.client.1.smithi176.stdout:8/768: chown d8/d23/d100/d31/d46/d6a/l9e 474 1 2024-07-16T09:55:48.060 INFO:tasks.workunit.client.0.smithi156.stdout:4/906: mknod d4/db/dc/dcc/d58/d8f/d11f/c142 0 2024-07-16T09:55:48.061 INFO:tasks.workunit.client.0.smithi156.stdout:0/876: write d0/d1a/d4d/d5e/d117/fd8 [714179,35630] 0 2024-07-16T09:55:48.061 INFO:tasks.workunit.client.0.smithi156.stdout:8/717: chown d0/d11/d13/d1c/d39/f73 13 1 2024-07-16T09:55:48.064 INFO:tasks.workunit.client.0.smithi156.stdout:4/907: fdatasync d4/db/dc/f68 0 2024-07-16T09:55:48.064 INFO:tasks.workunit.client.0.smithi156.stdout:2/907: dread d0/d13/d17/f23 [0,4194304] 0 2024-07-16T09:55:48.065 INFO:tasks.workunit.client.0.smithi156.stdout:8/718: mknod d0/c100 0 2024-07-16T09:55:48.066 INFO:tasks.workunit.client.0.smithi156.stdout:2/908: chown d0/d5/d29/dea/c100 738363 1 2024-07-16T09:55:48.066 INFO:tasks.workunit.client.0.smithi156.stdout:9/915: creat d2/d5/d16/d57/d63/dd3/ddb/d74/dc2/dc4/d23/d111/da2/d98/f11f x:0 0 0 2024-07-16T09:55:48.067 INFO:tasks.workunit.client.0.smithi156.stdout:4/908: write d4/d23/d30/f136 [426567,89681] 0 2024-07-16T09:55:48.067 INFO:tasks.workunit.client.0.smithi156.stdout:6/712: mknod d3/d13/d17/d4f/d94/de4/ce7 0 2024-07-16T09:55:48.069 INFO:tasks.workunit.client.0.smithi156.stdout:4/909: chown d4/db/d47 22906646 1 2024-07-16T09:55:48.071 INFO:tasks.workunit.client.0.smithi156.stdout:6/713: creat d3/d13/d17/d4f/d1e/fe8 x:0 0 0 2024-07-16T09:55:48.071 INFO:tasks.workunit.client.0.smithi156.stdout:2/909: dread d0/d13/d17/d1c/f24 [0,4194304] 0 2024-07-16T09:55:48.074 INFO:tasks.workunit.client.0.smithi156.stdout:4/910: truncate d4/d23/d52/d71/def/f111 723687 0 2024-07-16T09:55:48.074 INFO:tasks.workunit.client.0.smithi156.stdout:0/877: truncate d0/d1a/d4d/d9/fc7 117440 0 2024-07-16T09:55:48.074 INFO:tasks.workunit.client.0.smithi156.stdout:2/910: fsync d0/d5/d29/d2d/d5b/d5f/f66 0 2024-07-16T09:55:48.075 INFO:tasks.workunit.client.0.smithi156.stdout:9/916: symlink d2/d5/d16/d57/d63/dd3/ddb/d74/dc2/dc4/d23/d111/da2/l120 0 2024-07-16T09:55:48.075 INFO:tasks.workunit.client.0.smithi156.stdout:3/813: dread d8/da/d1d/fc6 [0,4194304] 0 2024-07-16T09:55:48.078 INFO:tasks.workunit.client.0.smithi156.stdout:2/911: chown d0/d5/d76/ffe 53776 1 2024-07-16T09:55:48.079 INFO:tasks.workunit.client.0.smithi156.stdout:8/719: dread d0/d11/d20/d3f/fb1 [4194304,4194304] 0 2024-07-16T09:55:48.079 INFO:tasks.workunit.client.0.smithi156.stdout:4/911: dread d4/d23/d52/f12b [0,4194304] 0 2024-07-16T09:55:48.080 INFO:tasks.workunit.client.0.smithi156.stdout:6/714: rename d3/d13/d17/d4f/d4d/d3b/da5/dbc/fca to d3/d13/d17/d4f/fe9 0 2024-07-16T09:55:48.081 INFO:tasks.workunit.client.0.smithi156.stdout:3/814: rmdir d8/ddf 0 2024-07-16T09:55:48.082 INFO:tasks.workunit.client.0.smithi156.stdout:6/715: chown d3/d13/d17/d4f/d94/d19/de5 10303 1 2024-07-16T09:55:48.082 INFO:tasks.workunit.client.0.smithi156.stdout:0/878: getdents d0/d1a/d4d/d5e 0 2024-07-16T09:55:48.083 INFO:tasks.workunit.client.0.smithi156.stdout:9/917: rename d2/d5/d16/d57/d63/dd3/ddb/d3d/d3c to d2/d5/d16/de2/d121 0 2024-07-16T09:55:48.084 INFO:tasks.workunit.client.0.smithi156.stdout:6/716: chown d3/d13/d17/d4f/d1e/fe3 3229126 1 2024-07-16T09:55:48.084 INFO:tasks.workunit.client.0.smithi156.stdout:8/720: mknod d0/d11/d20/d7f/c101 0 2024-07-16T09:55:48.084 INFO:tasks.workunit.client.0.smithi156.stdout:2/912: mkdir d0/d13/d17/d128 0 2024-07-16T09:55:48.086 INFO:tasks.workunit.client.1.smithi176.stdout:5/873: link d4/d37/d72/d38/d39/f9b d4/d9a/ded/df4/f13d 0 2024-07-16T09:55:48.087 INFO:tasks.workunit.client.0.smithi156.stdout:8/721: dread - d0/d11/d29/d80/f9d zero size 2024-07-16T09:55:48.088 INFO:tasks.workunit.client.0.smithi156.stdout:0/879: mknod d0/d1a/d76/da7/c118 0 2024-07-16T09:55:48.088 INFO:tasks.workunit.client.0.smithi156.stdout:9/918: chown d2/d5/d16/d57/ca4 45 1 2024-07-16T09:55:48.089 INFO:tasks.workunit.client.0.smithi156.stdout:1/948: chown dc/d14/d3e/d6c/d7a/df3 29 1 2024-07-16T09:55:48.089 INFO:tasks.workunit.client.1.smithi176.stdout:3/648: mknod d3/da/d33/dbd/cc7 0 2024-07-16T09:55:48.089 INFO:tasks.workunit.client.0.smithi156.stdout:2/913: fdatasync d0/d5/d29/d2d/d5b/f6d 0 2024-07-16T09:55:48.090 INFO:tasks.workunit.client.0.smithi156.stdout:3/815: creat d8/da/d1d/d2c/de0/df8/f116 x:0 0 0 2024-07-16T09:55:48.090 INFO:tasks.workunit.client.1.smithi176.stdout:3/649: dread - d3/d17/fb9 zero size 2024-07-16T09:55:48.090 INFO:tasks.workunit.client.1.smithi176.stdout:3/650: chown d3 34443 1 2024-07-16T09:55:48.091 INFO:tasks.workunit.client.0.smithi156.stdout:6/717: symlink d3/d13/d17/d4f/d1e/lea 0 2024-07-16T09:55:48.091 INFO:tasks.workunit.client.0.smithi156.stdout:4/912: dwrite d4/db/dc/dcc/dec/ded/daf/fd4 [4194304,4194304] 0 2024-07-16T09:55:48.091 INFO:tasks.workunit.client.1.smithi176.stdout:2/926: mknod dd/d16/d39/d3d/dc2/c135 0 2024-07-16T09:55:48.093 INFO:tasks.workunit.client.1.smithi176.stdout:4/775: fsync da/fb 0 2024-07-16T09:55:48.094 INFO:tasks.workunit.client.0.smithi156.stdout:2/914: write d0/d13/d17/d1c/fbd [838915,84366] 0 2024-07-16T09:55:48.094 INFO:tasks.workunit.client.1.smithi176.stdout:4/776: write da/f19 [13616740,104196] 0 2024-07-16T09:55:48.096 INFO:tasks.workunit.client.0.smithi156.stdout:1/949: mkdir dc/d6b/dd5/d13b 0 2024-07-16T09:55:48.097 INFO:tasks.workunit.client.0.smithi156.stdout:0/880: symlink d0/d1a/d4d/d61/da1/l119 0 2024-07-16T09:55:48.097 INFO:tasks.workunit.client.1.smithi176.stdout:9/911: creat d5/d52/f136 x:0 0 0 2024-07-16T09:55:48.098 INFO:tasks.workunit.client.1.smithi176.stdout:9/912: read - d5/d1d/d20/d78/df2/d118/d11c/f127 zero size 2024-07-16T09:55:48.099 INFO:tasks.workunit.client.0.smithi156.stdout:2/915: mkdir d0/d129 0 2024-07-16T09:55:48.101 INFO:tasks.workunit.client.1.smithi176.stdout:8/769: creat d8/d23/d100/d53/f106 x:0 0 0 2024-07-16T09:55:48.103 INFO:tasks.workunit.client.1.smithi176.stdout:6/914: mkdir d1/d5/d47/d62/d14c 0 2024-07-16T09:55:48.104 INFO:tasks.workunit.client.0.smithi156.stdout:2/916: write d0/d5/d76/f8a [84661,121284] 0 2024-07-16T09:55:48.104 INFO:tasks.workunit.client.1.smithi176.stdout:6/915: dread - d1/d5/d47/d75/fbe zero size 2024-07-16T09:55:48.105 INFO:tasks.workunit.client.0.smithi156.stdout:8/722: dwrite d0/d11/d4a/fd9 [0,4194304] 0 2024-07-16T09:55:48.105 INFO:tasks.workunit.client.0.smithi156.stdout:9/919: dread d2/f15 [0,4194304] 0 2024-07-16T09:55:48.106 INFO:tasks.workunit.client.1.smithi176.stdout:8/770: truncate d8/d14/d1e/d7c/d82/de0/ff0 703138 0 2024-07-16T09:55:48.106 INFO:tasks.workunit.client.1.smithi176.stdout:7/731: truncate d4/fb 1374599 0 2024-07-16T09:55:48.108 INFO:tasks.workunit.client.1.smithi176.stdout:2/927: unlink dd/d9b/fec 0 2024-07-16T09:55:48.108 INFO:tasks.workunit.client.0.smithi156.stdout:3/816: dwrite d8/da/d1d/d2c/de0/dad/feb [0,4194304] 0 2024-07-16T09:55:48.111 INFO:tasks.workunit.client.1.smithi176.stdout:9/913: creat d5/d1d/dd2/de3/d103/f137 x:0 0 0 2024-07-16T09:55:48.111 INFO:tasks.workunit.client.1.smithi176.stdout:4/777: dread da/d41/dc3/fd4 [0,4194304] 0 2024-07-16T09:55:48.112 INFO:tasks.workunit.client.0.smithi156.stdout:6/718: truncate d3/d13/f8e 4113539 0 2024-07-16T09:55:48.113 INFO:tasks.workunit.client.1.smithi176.stdout:4/778: stat da/dc/d4b/df0/c5a 0 2024-07-16T09:55:48.114 INFO:tasks.workunit.client.1.smithi176.stdout:0/923: rename d5/d25/da0 to d5/d1c/d24/d35/d50/d9b/de3/d117/d131 0 2024-07-16T09:55:48.115 INFO:tasks.workunit.client.0.smithi156.stdout:9/920: read d2/d5/d16/d22/d6b/f92 [99028,4786] 0 2024-07-16T09:55:48.115 INFO:tasks.workunit.client.1.smithi176.stdout:3/651: truncate d3/d17/d1d/f76 6436716 0 2024-07-16T09:55:48.118 INFO:tasks.workunit.client.1.smithi176.stdout:6/916: mkdir d1/dc/dd8/d14d 0 2024-07-16T09:55:48.118 INFO:tasks.workunit.client.0.smithi156.stdout:2/917: rename d0/d13/f8d to d0/d5/d29/dea/f12a 0 2024-07-16T09:55:48.120 INFO:tasks.workunit.client.1.smithi176.stdout:4/779: rmdir da/d3d/d33 39 2024-07-16T09:55:48.120 INFO:tasks.workunit.client.1.smithi176.stdout:2/928: dwrite dd/d2e/d35/fac [4194304,4194304] 0 2024-07-16T09:55:48.121 INFO:tasks.workunit.client.1.smithi176.stdout:7/732: rename d4/d5/d97/cb6 to d4/dd/d26/dc2/d17/d1d/d8b/cf1 0 2024-07-16T09:55:48.121 INFO:tasks.workunit.client.1.smithi176.stdout:0/924: fsync d5/d2c/f54 0 2024-07-16T09:55:48.122 INFO:tasks.workunit.client.1.smithi176.stdout:8/771: truncate d8/f2e 3231556 0 2024-07-16T09:55:48.123 INFO:tasks.workunit.client.0.smithi156.stdout:8/723: mknod d0/dd/d1f/d57/c102 0 2024-07-16T09:55:48.127 INFO:tasks.workunit.client.0.smithi156.stdout:9/921: rmdir d2/d5/d16/d57/d63/dd3/ddb/dd0 39 2024-07-16T09:55:48.130 INFO:tasks.workunit.client.0.smithi156.stdout:2/918: write d0/d13/d17/d1c/d51/fc2 [194792,35097] 0 2024-07-16T09:55:48.130 INFO:tasks.workunit.client.0.smithi156.stdout:0/881: getdents d0/d1a/d4d 0 2024-07-16T09:55:48.131 INFO:tasks.workunit.client.1.smithi176.stdout:2/929: creat dd/d36/d82/d12a/d10c/f136 x:0 0 0 2024-07-16T09:55:48.132 INFO:tasks.workunit.client.1.smithi176.stdout:6/917: dwrite d1/d5/d7/d54/f113 [0,4194304] 0 2024-07-16T09:55:48.132 INFO:tasks.workunit.client.0.smithi156.stdout:8/724: dread d0/d11/d13/d1c/d39/f73 [0,4194304] 0 2024-07-16T09:55:48.133 INFO:tasks.workunit.client.1.smithi176.stdout:2/930: write dd/d49/f60 [3317134,128162] 0 2024-07-16T09:55:48.133 INFO:tasks.workunit.client.1.smithi176.stdout:4/780: fdatasync da/dc/d4b/df0/d96/da4/dc2/fda 0 2024-07-16T09:55:48.136 INFO:tasks.workunit.client.1.smithi176.stdout:6/918: read - d1/db3/f95 zero size 2024-07-16T09:55:48.137 INFO:tasks.workunit.client.0.smithi156.stdout:2/919: truncate d0/d5/d29/d2d/f31 3690923 0 2024-07-16T09:55:48.139 INFO:tasks.workunit.client.0.smithi156.stdout:9/922: dwrite d2/f4 [0,4194304] 0 2024-07-16T09:55:48.140 INFO:tasks.workunit.client.1.smithi176.stdout:6/919: truncate d1/f64 1756119 0 2024-07-16T09:55:48.140 INFO:tasks.workunit.client.1.smithi176.stdout:6/920: chown d1/dc 108407 1 2024-07-16T09:55:48.142 INFO:tasks.workunit.client.1.smithi176.stdout:8/772: getdents d8/d23/d100/d31/db4 0 2024-07-16T09:55:48.143 INFO:tasks.workunit.client.1.smithi176.stdout:7/733: getdents d4/dd/d26/dc2/d17/d1d 0 2024-07-16T09:55:48.143 INFO:tasks.workunit.client.1.smithi176.stdout:2/931: mkdir dd/d49/d52/d137 0 2024-07-16T09:55:48.144 INFO:tasks.workunit.client.0.smithi156.stdout:2/920: unlink d0/d5/d29/l84 0 2024-07-16T09:55:48.145 INFO:tasks.workunit.client.0.smithi156.stdout:2/921: chown d0/d13/d17/d1c/c45 53515786 1 2024-07-16T09:55:48.146 INFO:tasks.workunit.client.1.smithi176.stdout:8/773: truncate d8/d14/f27 2287441 0 2024-07-16T09:55:48.147 INFO:tasks.workunit.client.1.smithi176.stdout:2/932: dread dd/d16/d39/d3d/d12b/f12f [0,4194304] 0 2024-07-16T09:55:48.149 INFO:tasks.workunit.client.0.smithi156.stdout:6/719: write d3/d13/d17/d4f/d4d/f76 [999482,68029] 0 2024-07-16T09:55:48.149 INFO:tasks.workunit.client.1.smithi176.stdout:2/933: read dd/d49/fb6 [184959,6956] 0 2024-07-16T09:55:48.150 INFO:tasks.workunit.client.0.smithi156.stdout:6/720: dread - d3/d13/fc4 zero size 2024-07-16T09:55:48.151 INFO:tasks.workunit.client.1.smithi176.stdout:8/774: creat d8/d23/d100/d61/d90/f107 x:0 0 0 2024-07-16T09:55:48.152 INFO:tasks.workunit.client.0.smithi156.stdout:2/922: rename d0/d5/d29/d2d/d5b/ddb/df9 to d0/d5/d29/d2d/d5b/ddb/d12b 0 2024-07-16T09:55:48.153 INFO:tasks.workunit.client.0.smithi156.stdout:6/721: write d3/d13/d17/d4f/d94/d19/d31/d35/f62 [3005277,19090] 0 2024-07-16T09:55:48.157 INFO:tasks.workunit.client.1.smithi176.stdout:8/775: mknod d8/d14/d1e/d7c/d82/de0/c108 0 2024-07-16T09:55:48.158 INFO:tasks.workunit.client.1.smithi176.stdout:3/652: write d3/d17/d1d/d4f/d59/f21 [1703626,61828] 0 2024-07-16T09:55:48.161 INFO:tasks.workunit.client.1.smithi176.stdout:2/934: rename dd/d16/fe4 to dd/d2e/d35/d4a/f138 0 2024-07-16T09:55:48.162 INFO:tasks.workunit.client.1.smithi176.stdout:3/653: fdatasync d3/d17/d1d/d4f/d6d/f9f 0 2024-07-16T09:55:48.163 INFO:tasks.workunit.client.1.smithi176.stdout:0/925: truncate d5/d25/d82/f9a 1813148 0 2024-07-16T09:55:48.164 INFO:tasks.workunit.client.1.smithi176.stdout:8/776: creat d8/d23/d100/d31/d46/d6a/da8/dea/f109 x:0 0 0 2024-07-16T09:55:48.166 INFO:tasks.workunit.client.1.smithi176.stdout:8/777: symlink d8/d23/d100/d61/l10a 0 2024-07-16T09:55:48.169 INFO:tasks.workunit.client.0.smithi156.stdout:3/817: write d8/da/d28/d43/f5b [555452,64383] 0 2024-07-16T09:55:48.171 INFO:tasks.workunit.client.1.smithi176.stdout:4/781: write da/d41/f99 [268558,90005] 0 2024-07-16T09:55:48.173 INFO:tasks.workunit.client.1.smithi176.stdout:0/926: dwrite d5/f1e [0,4194304] 0 2024-07-16T09:55:48.173 INFO:tasks.workunit.client.1.smithi176.stdout:6/921: write d1/d5/d7/d54/fd3 [963547,88867] 0 2024-07-16T09:55:48.174 INFO:tasks.workunit.client.0.smithi156.stdout:0/882: dread d0/d1a/d4d/d60/ffa [0,4194304] 0 2024-07-16T09:55:48.174 INFO:tasks.workunit.client.0.smithi156.stdout:8/725: truncate d0/dd/f26 2184391 0 2024-07-16T09:55:48.174 INFO:tasks.workunit.client.0.smithi156.stdout:3/818: chown d8/f10 139 1 2024-07-16T09:55:48.176 INFO:tasks.workunit.client.1.smithi176.stdout:4/782: rmdir da/dc/d4b/d62 39 2024-07-16T09:55:48.176 INFO:tasks.workunit.client.1.smithi176.stdout:7/734: write d4/d47/f88 [234552,123900] 0 2024-07-16T09:55:48.181 INFO:tasks.workunit.client.0.smithi156.stdout:8/726: dread d0/d11/d2e/fda [0,4194304] 0 2024-07-16T09:55:48.182 INFO:tasks.workunit.client.1.smithi176.stdout:6/922: mkdir d1/d5d/db9/d101/d105/d14e 0 2024-07-16T09:55:48.185 INFO:tasks.workunit.client.0.smithi156.stdout:0/883: unlink d0/d1a/d4d/d9/d27/d50/f9a 0 2024-07-16T09:55:48.186 INFO:tasks.workunit.client.0.smithi156.stdout:2/923: dwrite d0/d13/d17/d1c/f46 [0,4194304] 0 2024-07-16T09:55:48.188 INFO:tasks.workunit.client.1.smithi176.stdout:8/778: rmdir d8/d23/d100/d31/d46/d6a/da8/dea 39 2024-07-16T09:55:48.189 INFO:tasks.workunit.client.0.smithi156.stdout:9/923: dwrite d2/d5/d16/d57/d63/dd3/ddb/d74/dc2/dc4/d23/d111/da2/d98/fa9 [0,4194304] 0 2024-07-16T09:55:48.190 INFO:tasks.workunit.client.1.smithi176.stdout:6/923: creat d1/d5d/d9e/d8f/f14f x:0 0 0 2024-07-16T09:55:48.192 INFO:tasks.workunit.client.0.smithi156.stdout:3/819: rename d8/da/d21/lc5 to d8/da/d1d/l117 0 2024-07-16T09:55:48.195 INFO:tasks.workunit.client.1.smithi176.stdout:4/783: link da/dc/d4b/df0/d96/d9b/db4/d40/d5d/d73/ce8 da/dc/d4b/df0/d96/d69/d86/cfe 0 2024-07-16T09:55:48.197 INFO:tasks.workunit.client.1.smithi176.stdout:8/779: dread d8/d14/d1e/d33/d9d/fb8 [0,4194304] 0 2024-07-16T09:55:48.197 INFO:tasks.workunit.client.0.smithi156.stdout:8/727: dwrite d0/d11/d2e/d35/f46 [0,4194304] 0 2024-07-16T09:55:48.201 INFO:tasks.workunit.client.0.smithi156.stdout:2/924: truncate d0/d5/f2a 1064738 0 2024-07-16T09:55:48.201 INFO:tasks.workunit.client.1.smithi176.stdout:4/784: chown da/dc/d4b/df0/d96/lea 9585502 1 2024-07-16T09:55:48.201 INFO:tasks.workunit.client.1.smithi176.stdout:3/654: dwrite d3/da/d33/f44 [0,4194304] 0 2024-07-16T09:55:48.204 INFO:tasks.workunit.client.1.smithi176.stdout:6/924: link d1/dc/d40/fca d1/dc/d81/d10a/d115/d121/f150 0 2024-07-16T09:55:48.207 INFO:tasks.workunit.client.0.smithi156.stdout:0/884: dread d0/d22/d41/f4a [0,4194304] 0 2024-07-16T09:55:48.207 INFO:tasks.workunit.client.0.smithi156.stdout:3/820: dwrite d8/da/d28/ffa [0,4194304] 0 2024-07-16T09:55:48.209 INFO:tasks.workunit.client.0.smithi156.stdout:8/728: mknod d0/dd/d1f/d50/d8d/c103 0 2024-07-16T09:55:48.210 INFO:tasks.workunit.client.0.smithi156.stdout:6/722: dwrite d3/d13/d17/d4f/d4d/d3b/fbd [4194304,4194304] 0 2024-07-16T09:55:48.211 INFO:tasks.workunit.client.1.smithi176.stdout:0/927: dwrite d5/f2f [0,4194304] 0 2024-07-16T09:55:48.211 INFO:tasks.workunit.client.0.smithi156.stdout:8/729: stat d0/d11/ded/dd7/lea 0 2024-07-16T09:55:48.213 INFO:tasks.workunit.client.0.smithi156.stdout:8/730: chown d0/d11/d20/d7f/fc8 56210 1 2024-07-16T09:55:48.213 INFO:tasks.workunit.client.0.smithi156.stdout:6/723: chown d3/d13/d17/d4f/d94/d19/d31/fa1 256 1 2024-07-16T09:55:48.213 INFO:tasks.workunit.client.0.smithi156.stdout:2/925: symlink d0/d5/l12c 0 2024-07-16T09:55:48.215 INFO:tasks.workunit.client.1.smithi176.stdout:6/925: rmdir d1/d5/d47/d62/d80 39 2024-07-16T09:55:48.217 INFO:tasks.workunit.client.1.smithi176.stdout:8/780: creat d8/d23/d100/d31/f10b x:0 0 0 2024-07-16T09:55:48.217 INFO:tasks.workunit.client.1.smithi176.stdout:6/926: write d1/d5/d47/fa6 [125709,106964] 0 2024-07-16T09:55:48.218 INFO:tasks.workunit.client.0.smithi156.stdout:9/924: rename d2/d5/d16/d22/d36/f59 to d2/d5/d16/d57/d63/dd3/ddb/d74/dc2/dc4/d23/f122 0 2024-07-16T09:55:48.219 INFO:tasks.workunit.client.1.smithi176.stdout:0/928: symlink d5/d1c/d24/d127/l132 0 2024-07-16T09:55:48.222 INFO:tasks.workunit.client.1.smithi176.stdout:0/929: fsync d5/d1c/d24/d35/d50/d58/d63/f129 0 2024-07-16T09:55:48.226 INFO:tasks.workunit.client.1.smithi176.stdout:0/930: truncate d5/d1c/d24/d35/d50/d7c/f101 157410 0 2024-07-16T09:55:48.227 INFO:tasks.workunit.client.0.smithi156.stdout:2/926: chown d0/d5/d76/d10f/f110 11140047 1 2024-07-16T09:55:48.228 INFO:tasks.workunit.client.0.smithi156.stdout:6/724: chown d3/d13/d17/d4f/d4d/d3b/d77 416 1 2024-07-16T09:55:48.235 INFO:tasks.workunit.client.0.smithi156.stdout:8/731: dwrite d0/d11/d13/d1c/d39/fe4 [0,4194304] 0 2024-07-16T09:55:48.240 INFO:tasks.workunit.client.0.smithi156.stdout:0/885: dread d0/d1a/d4d/d9/d27/d50/f5f [0,4194304] 0 2024-07-16T09:55:48.254 INFO:tasks.workunit.client.0.smithi156.stdout:2/927: dread d0/d5/d76/f108 [0,4194304] 0 2024-07-16T09:55:48.255 INFO:tasks.workunit.client.0.smithi156.stdout:2/928: readlink d0/d5/d29/d2d/laf 0 2024-07-16T09:55:48.255 INFO:tasks.workunit.client.0.smithi156.stdout:2/929: chown d0/d5/d29/l33 1 1 2024-07-16T09:55:48.271 INFO:tasks.workunit.client.0.smithi156.stdout:3/821: dread d8/da/d21/d80/d99/f109 [0,4194304] 0 2024-07-16T09:55:48.272 INFO:tasks.workunit.client.0.smithi156.stdout:3/822: write d8/da/d21/d80/fea [74649,61507] 0 2024-07-16T09:55:48.273 INFO:tasks.workunit.client.0.smithi156.stdout:3/823: stat d8/da/d1d/d2c/d7d/f108 0 2024-07-16T09:55:48.277 INFO:tasks.workunit.client.1.smithi176.stdout:4/785: rename da/dc/d4b/df0/d96/lea to da/dc/d4b/df0/d96/d9b/db4/lff 0 2024-07-16T09:55:48.278 INFO:tasks.workunit.client.1.smithi176.stdout:8/781: dwrite d8/d23/d100/d31/db4/fc4 [0,4194304] 0 2024-07-16T09:55:48.278 INFO:tasks.workunit.client.1.smithi176.stdout:1/561: sync 2024-07-16T09:55:48.281 INFO:tasks.workunit.client.1.smithi176.stdout:2/935: dread dd/d36/f94 [0,4194304] 0 2024-07-16T09:55:48.281 INFO:tasks.workunit.client.1.smithi176.stdout:0/931: chown d5/d1c/d24/d35/d50/d9b/de3/d117/dbd/de6/df8/f115 7 1 2024-07-16T09:55:48.281 INFO:tasks.workunit.client.1.smithi176.stdout:8/782: chown d8/d23/d100/d31/d2f/l6d 0 1 2024-07-16T09:55:48.282 INFO:tasks.workunit.client.1.smithi176.stdout:0/932: fdatasync d5/d25/ff9 0 2024-07-16T09:55:48.282 INFO:tasks.workunit.client.0.smithi156.stdout:8/732: dread d0/dd/d1f/d60/f98 [0,4194304] 0 2024-07-16T09:55:48.283 INFO:tasks.workunit.client.0.smithi156.stdout:7/644: sync 2024-07-16T09:55:48.283 INFO:tasks.workunit.client.0.smithi156.stdout:8/733: readlink d0/d11/d13/l95 0 2024-07-16T09:55:48.284 INFO:tasks.workunit.client.1.smithi176.stdout:2/936: creat dd/de/f139 x:0 0 0 2024-07-16T09:55:48.287 INFO:tasks.workunit.client.1.smithi176.stdout:4/786: rename da/dc/d4b/df0/d28/fc4 to da/dc/d4b/df0/d96/d9b/db4/d40/d5d/f100 0 2024-07-16T09:55:48.289 INFO:tasks.workunit.client.1.smithi176.stdout:0/933: creat d5/d39/f133 x:0 0 0 2024-07-16T09:55:48.289 INFO:tasks.workunit.client.1.smithi176.stdout:4/787: write da/fd9 [67965,75879] 0 2024-07-16T09:55:48.291 INFO:tasks.workunit.client.1.smithi176.stdout:8/783: creat d8/d14/d1e/d33/df6/f10c x:0 0 0 2024-07-16T09:55:48.295 INFO:tasks.workunit.client.1.smithi176.stdout:2/937: dwrite dd/d49/f60 [0,4194304] 0 2024-07-16T09:55:48.298 INFO:tasks.workunit.client.1.smithi176.stdout:0/934: rename d5/d1c/d24/d35/d50/d9b/fac to d5/da1/f134 0 2024-07-16T09:55:48.298 INFO:tasks.workunit.client.1.smithi176.stdout:8/784: symlink d8/d14/d1e/d7c/d82/de0/l10d 0 2024-07-16T09:55:48.301 INFO:tasks.workunit.client.1.smithi176.stdout:1/562: dread d4/d3a/f9e [0,4194304] 0 2024-07-16T09:55:48.301 INFO:tasks.workunit.client.1.smithi176.stdout:2/938: mkdir dd/d16/d39/d13a 0 2024-07-16T09:55:48.303 INFO:tasks.workunit.client.1.smithi176.stdout:0/935: mknod d5/d1c/d24/d35/d50/d58/d63/db4/c135 0 2024-07-16T09:55:48.303 INFO:tasks.workunit.client.1.smithi176.stdout:1/563: dread - d4/d8/d12/d21/d25/da8/f9d zero size 2024-07-16T09:55:48.304 INFO:tasks.workunit.client.1.smithi176.stdout:2/939: creat dd/d2e/d35/d59/f13b x:0 0 0 2024-07-16T09:55:48.305 INFO:tasks.workunit.client.1.smithi176.stdout:2/940: fdatasync dd/d49/f55 0 2024-07-16T09:55:48.306 INFO:tasks.workunit.client.1.smithi176.stdout:4/788: write da/dc/d4b/df0/d96/d9b/db4/d40/d5d/f85 [586697,33158] 0 2024-07-16T09:55:48.307 INFO:tasks.workunit.client.1.smithi176.stdout:8/785: getdents d8/d23/d100/d53/d9a 0 2024-07-16T09:55:48.308 INFO:tasks.workunit.client.1.smithi176.stdout:0/936: symlink d5/d1c/l136 0 2024-07-16T09:55:48.308 INFO:tasks.workunit.client.0.smithi156.stdout:9/925: stat d2/d5/d16/f9d 0 2024-07-16T09:55:48.309 INFO:tasks.workunit.client.1.smithi176.stdout:2/941: write dd/d9b/f130 [2643776,64867] 0 2024-07-16T09:55:48.311 INFO:tasks.workunit.client.0.smithi156.stdout:0/886: mknod d0/d1a/d4d/d60/c11a 0 2024-07-16T09:55:48.311 INFO:tasks.workunit.client.0.smithi156.stdout:0/887: readlink d0/d22/d41/lf0 0 2024-07-16T09:55:48.312 INFO:tasks.workunit.client.0.smithi156.stdout:3/824: mknod d8/da/d21/c118 0 2024-07-16T09:55:48.314 INFO:tasks.workunit.client.0.smithi156.stdout:3/825: write d8/da/d28/d43/d71/da2/f10b [679375,77552] 0 2024-07-16T09:55:48.314 INFO:tasks.workunit.client.0.smithi156.stdout:8/734: creat d0/d11/d20/d7f/f104 x:0 0 0 2024-07-16T09:55:48.316 INFO:tasks.workunit.client.0.smithi156.stdout:6/725: symlink d3/d13/d17/d4f/d30/d45/d5f/da8/leb 0 2024-07-16T09:55:48.317 INFO:tasks.workunit.client.0.smithi156.stdout:9/926: mknod d2/d5/d16/de2/d121/d7f/d114/c123 0 2024-07-16T09:55:48.319 INFO:tasks.workunit.client.0.smithi156.stdout:2/930: mkdir d0/d5/d76/da1/df2/d12d 0 2024-07-16T09:55:48.321 INFO:tasks.workunit.client.1.smithi176.stdout:0/937: dwrite d5/d25/ff9 [0,4194304] 0 2024-07-16T09:55:48.321 INFO:tasks.workunit.client.1.smithi176.stdout:8/786: dwrite d8/d23/d100/d61/d90/f107 [0,4194304] 0 2024-07-16T09:55:48.321 INFO:tasks.workunit.client.0.smithi156.stdout:0/888: mkdir d0/d1a/d4d/d9/d27/d50/d11b 0 2024-07-16T09:55:48.323 INFO:tasks.workunit.client.0.smithi156.stdout:0/889: read - d0/d1a/d4d/d61/d97/de1/f109 zero size 2024-07-16T09:55:48.324 INFO:tasks.workunit.client.0.smithi156.stdout:9/927: dwrite d2/d5/d16/de2/d121/f62 [0,4194304] 0 2024-07-16T09:55:48.328 INFO:tasks.workunit.client.1.smithi176.stdout:8/787: write d8/d23/d100/db9/dcb/d34/fcc [962572,38608] 0 2024-07-16T09:55:48.329 INFO:tasks.workunit.client.0.smithi156.stdout:2/931: write d0/d5/d29/d2d/d5b/d5f/d89/f99 [2714820,24013] 0 2024-07-16T09:55:48.329 INFO:tasks.workunit.client.1.smithi176.stdout:1/564: dwrite d4/d8/d12/d21/d25/d30/d44/d4d/f94 [0,4194304] 0 2024-07-16T09:55:48.329 INFO:tasks.workunit.client.1.smithi176.stdout:2/942: dwrite dd/d2e/d35/fb8 [0,4194304] 0 2024-07-16T09:55:48.333 INFO:tasks.workunit.client.1.smithi176.stdout:8/788: stat d8/d23/d100/db9/dcb/d34/fde 0 2024-07-16T09:55:48.333 INFO:tasks.workunit.client.1.smithi176.stdout:1/565: chown d4/d8/d12/d21/d25/d30 74931617 1 2024-07-16T09:55:48.334 INFO:tasks.workunit.client.1.smithi176.stdout:2/943: read dd/d16/f3a [6951915,121108] 0 2024-07-16T09:55:48.335 INFO:tasks.workunit.client.1.smithi176.stdout:2/944: chown dd/d49/d52/d10a/d124 43625 1 2024-07-16T09:55:48.336 INFO:tasks.workunit.client.1.smithi176.stdout:1/566: write d4/d8/d12/d4c/da4/fbc [432899,62846] 0 2024-07-16T09:55:48.339 INFO:tasks.workunit.client.1.smithi176.stdout:8/789: creat d8/d23/d76/f10e x:0 0 0 2024-07-16T09:55:48.339 INFO:tasks.workunit.client.0.smithi156.stdout:2/932: mknod d0/d5/d29/d2d/d5b/ddb/d12b/c12e 0 2024-07-16T09:55:48.343 INFO:tasks.workunit.client.0.smithi156.stdout:0/890: mkdir d0/d1a/d4d/d9/d27/d50/d11b/d11c 0 2024-07-16T09:55:48.343 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:55:47 smithi176 ceph-mon[36626]: pgmap v255: 65 pgs: 65 active+clean; 2.9 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 106 MiB/s rd, 302 MiB/s wr, 611 op/s 2024-07-16T09:55:48.344 INFO:tasks.workunit.client.0.smithi156.stdout:9/928: mknod d2/d5/d16/d22/d6b/d86/c124 0 2024-07-16T09:55:48.345 INFO:tasks.workunit.client.0.smithi156.stdout:8/735: getdents d0/d11/d13/d1c/d39 0 2024-07-16T09:55:48.346 INFO:tasks.workunit.client.0.smithi156.stdout:2/933: symlink d0/d9b/d126/l12f 0 2024-07-16T09:55:48.348 INFO:tasks.workunit.client.0.smithi156.stdout:0/891: creat d0/d22/d41/d52/ddb/f11d x:0 0 0 2024-07-16T09:55:48.348 INFO:tasks.workunit.client.0.smithi156.stdout:7/645: getdents dc/d13 0 2024-07-16T09:55:48.349 INFO:tasks.workunit.client.0.smithi156.stdout:9/929: mknod d2/d5/d16/d57/d63/dd3/ddb/d74/dc2/dc4/d23/d111/da2/c125 0 2024-07-16T09:55:48.350 INFO:tasks.workunit.client.0.smithi156.stdout:8/736: creat d0/d11/d62/dc2/f105 x:0 0 0 2024-07-16T09:55:48.352 INFO:tasks.workunit.client.0.smithi156.stdout:9/930: readlink d2/d5/d16/d57/d63/dd3/ddb/d74/dc2/dc4/lf7 0 2024-07-16T09:55:48.353 INFO:tasks.workunit.client.0.smithi156.stdout:2/934: mkdir d0/d129/d130 0 2024-07-16T09:55:48.354 INFO:tasks.workunit.client.0.smithi156.stdout:9/931: read d2/d5/f55 [925740,52695] 0 2024-07-16T09:55:48.354 INFO:tasks.workunit.client.0.smithi156.stdout:2/935: dread - d0/d5/d76/da1/fe9 zero size 2024-07-16T09:55:48.357 INFO:tasks.workunit.client.0.smithi156.stdout:9/932: mkdir d2/df5/d126 0 2024-07-16T09:55:48.358 INFO:tasks.workunit.client.0.smithi156.stdout:8/737: link d0/d11/d20/f7a d0/d11/d20/d7f/db7/f106 0 2024-07-16T09:55:48.360 INFO:tasks.workunit.client.0.smithi156.stdout:8/738: creat d0/d11/ded/f107 x:0 0 0 2024-07-16T09:55:48.360 INFO:tasks.workunit.client.0.smithi156.stdout:6/726: dwrite d3/d13/d17/d4f/d94/d19/d31/d35/fd8 [0,4194304] 0 2024-07-16T09:55:48.360 INFO:tasks.workunit.client.0.smithi156.stdout:2/936: getdents d0/d5/d76/da1/df2 0 2024-07-16T09:55:48.364 INFO:tasks.workunit.client.0.smithi156.stdout:9/933: mkdir d2/d5/d16/d57/d127 0 2024-07-16T09:55:48.364 INFO:tasks.workunit.client.0.smithi156.stdout:8/739: mkdir d0/d11/d62/dc2/dcd/d108 0 2024-07-16T09:55:48.365 INFO:tasks.workunit.client.1.smithi176.stdout:2/945: write f9 [8981584,62843] 0 2024-07-16T09:55:48.366 INFO:tasks.workunit.client.0.smithi156.stdout:9/934: chown d2/d5/d16/d57/da3/f103 418 1 2024-07-16T09:55:48.366 INFO:tasks.workunit.client.0.smithi156.stdout:8/740: write d0/d11/d4a/fd9 [2480809,86035] 0 2024-07-16T09:55:48.366 INFO:tasks.workunit.client.1.smithi176.stdout:2/946: dread - dd/d16/d39/d3d/dc2/f121 zero size 2024-07-16T09:55:48.369 INFO:tasks.workunit.client.0.smithi156.stdout:6/727: mkdir d3/d13/d17/d4f/d94/dec 0 2024-07-16T09:55:48.371 INFO:tasks.workunit.client.1.smithi176.stdout:2/947: rename dd/d2e/f3e to dd/d49/d52/d10a/f13c 0 2024-07-16T09:55:48.372 INFO:tasks.workunit.client.1.smithi176.stdout:2/948: chown dd/d49/d52/d7c 13684 1 2024-07-16T09:55:48.372 INFO:tasks.workunit.client.0.smithi156.stdout:2/937: getdents d0/d13/de8 0 2024-07-16T09:55:48.373 INFO:tasks.workunit.client.0.smithi156.stdout:8/741: write d0/f6d [3051449,75654] 0 2024-07-16T09:55:48.373 INFO:tasks.workunit.client.1.smithi176.stdout:2/949: truncate dd/d49/d52/d10a/f133 538113 0 2024-07-16T09:55:48.377 INFO:tasks.workunit.client.0.smithi156.stdout:2/938: symlink d0/d5/d29/dea/l131 0 2024-07-16T09:55:48.377 INFO:tasks.workunit.client.0.smithi156.stdout:9/935: dread d2/d5/d16/d57/d63/dd3/ddb/dd0/def/ffe [0,4194304] 0 2024-07-16T09:55:48.377 INFO:tasks.workunit.client.0.smithi156.stdout:8/742: mknod d0/dd/d1f/dee/c109 0 2024-07-16T09:55:48.377 INFO:tasks.workunit.client.1.smithi176.stdout:2/950: rename dd/d49/l7a to dd/d16/d39/d3d/d7e/l13d 0 2024-07-16T09:55:48.377 INFO:tasks.workunit.client.1.smithi176.stdout:8/790: dread d8/d23/d76/f98 [0,4194304] 0 2024-07-16T09:55:48.378 INFO:tasks.workunit.client.1.smithi176.stdout:8/791: stat d8/d23/d100/d53 0 2024-07-16T09:55:48.379 INFO:tasks.workunit.client.1.smithi176.stdout:2/951: unlink dd/d2e/d35/f64 0 2024-07-16T09:55:48.383 INFO:tasks.workunit.client.0.smithi156.stdout:2/939: creat d0/d13/d17/d1c/d86/dde/d109/f132 x:0 0 0 2024-07-16T09:55:48.384 INFO:tasks.workunit.client.0.smithi156.stdout:6/728: dwrite d3/d13/d17/d4f/d94/d19/d31/d35/f62 [4194304,4194304] 0 2024-07-16T09:55:48.384 INFO:tasks.workunit.client.0.smithi156.stdout:7/646: dread dc/d3e/d3f/d51/f53 [0,4194304] 0 2024-07-16T09:55:48.387 INFO:tasks.workunit.client.0.smithi156.stdout:9/936: dread d2/d9b/d109/f10d [0,4194304] 0 2024-07-16T09:55:48.388 INFO:tasks.workunit.client.0.smithi156.stdout:8/743: dread d0/dd/d1f/d57/fbf [0,4194304] 0 2024-07-16T09:55:48.388 INFO:tasks.workunit.client.0.smithi156.stdout:6/729: write d3/d13/d17/d4f/d30/f7f [3966497,47884] 0 2024-07-16T09:55:48.389 INFO:tasks.workunit.client.1.smithi176.stdout:2/952: truncate dd/d16/d39/d3d/d7e/fdc 234903 0 2024-07-16T09:55:48.390 INFO:tasks.workunit.client.0.smithi156.stdout:2/940: mknod d0/d13/d17/d1c/d117/c133 0 2024-07-16T09:55:48.391 INFO:tasks.workunit.client.0.smithi156.stdout:7/647: creat dc/d13/d15/d8b/dcc/fdb x:0 0 0 2024-07-16T09:55:48.391 INFO:tasks.workunit.client.0.smithi156.stdout:2/941: chown d0/d13/f26 156352818 1 2024-07-16T09:55:48.393 INFO:tasks.workunit.client.0.smithi156.stdout:8/744: dread d0/dd/d1f/fc6 [0,4194304] 0 2024-07-16T09:55:48.393 INFO:tasks.workunit.client.1.smithi176.stdout:2/953: creat dd/d49/d52/d7c/d81/f13e x:0 0 0 2024-07-16T09:55:48.396 INFO:tasks.workunit.client.0.smithi156.stdout:2/942: fsync d0/d5/d29/d2d/d5b/d5f/d89/ffa 0 2024-07-16T09:55:48.396 INFO:tasks.workunit.client.1.smithi176.stdout:8/792: dread d8/d23/d100/db9/fc2 [0,4194304] 0 2024-07-16T09:55:48.397 INFO:tasks.workunit.client.1.smithi176.stdout:8/793: write d8/d23/d100/d31/d46/d6a/da8/ffe [86182,64349] 0 2024-07-16T09:55:48.398 INFO:tasks.workunit.client.0.smithi156.stdout:7/648: rename dc/d63/l73 to dc/d44/ldc 0 2024-07-16T09:55:48.398 INFO:tasks.workunit.client.0.smithi156.stdout:8/745: rmdir d0/dd/d1f/d60 39 2024-07-16T09:55:48.399 INFO:tasks.workunit.client.0.smithi156.stdout:8/746: readlink d0/l53 0 2024-07-16T09:55:48.400 INFO:tasks.workunit.client.1.smithi176.stdout:8/794: read d8/d20/f5a [3356740,126150] 0 2024-07-16T09:55:48.401 INFO:tasks.workunit.client.1.smithi176.stdout:8/795: stat d8/d23/d100/d31/d2f/fe6 0 2024-07-16T09:55:48.402 INFO:tasks.workunit.client.0.smithi156.stdout:6/730: write d3/d13/d17/d4f/d94/fe [231921,125029] 0 2024-07-16T09:55:48.403 INFO:tasks.workunit.client.1.smithi176.stdout:8/796: write d8/d23/d100/db9/dcb/d37/f57 [1818214,122305] 0 2024-07-16T09:55:48.403 INFO:tasks.workunit.client.0.smithi156.stdout:8/747: mkdir d0/d11/ded/dd7/d10a 0 2024-07-16T09:55:48.405 INFO:tasks.workunit.client.0.smithi156.stdout:9/937: rename d2/d5/d16/d57/d63/dd3/ddb/d74/dc2/dc4/d23/f88 to d2/d5/d16/d57/d63/dd3/f128 0 2024-07-16T09:55:48.407 INFO:tasks.workunit.client.0.smithi156.stdout:7/649: write dc/d63/f6b [256330,96591] 0 2024-07-16T09:55:48.407 INFO:tasks.workunit.client.1.smithi176.stdout:2/954: rename dd/d16 to dd/d2e/d35/d13f 0 2024-07-16T09:55:48.408 INFO:tasks.workunit.client.0.smithi156.stdout:6/731: creat d3/d13/d17/d4f/d30/d45/d5f/da8/fed x:0 0 0 2024-07-16T09:55:48.409 INFO:tasks.workunit.client.0.smithi156.stdout:2/943: getdents d0/d5/d29/d2d/d5b 0 2024-07-16T09:55:48.411 INFO:tasks.workunit.client.1.smithi176.stdout:2/955: symlink dd/d2e/d35/l140 0 2024-07-16T09:55:48.411 INFO:tasks.workunit.client.0.smithi156.stdout:7/650: creat dc/d3e/d3f/d51/d8a/fdd x:0 0 0 2024-07-16T09:55:48.412 INFO:tasks.workunit.client.1.smithi176.stdout:2/956: dread - dd/de/f139 zero size 2024-07-16T09:55:48.412 INFO:tasks.workunit.client.0.smithi156.stdout:8/748: link d0/dd/d1f/d57/dc0/le0 d0/l10b 0 2024-07-16T09:55:48.413 INFO:tasks.workunit.client.0.smithi156.stdout:2/944: mknod d0/d5/d76/d10f/c134 0 2024-07-16T09:55:48.414 INFO:tasks.workunit.client.0.smithi156.stdout:7/651: stat dc/d3e/d3f/d51/d8a/l93 0 2024-07-16T09:55:48.415 INFO:tasks.workunit.client.1.smithi176.stdout:2/957: mknod dd/d36/d45/c141 0 2024-07-16T09:55:48.415 INFO:tasks.workunit.client.0.smithi156.stdout:8/749: fdatasync d0/dd/d1f/f6b 0 2024-07-16T09:55:48.415 INFO:tasks.workunit.client.0.smithi156.stdout:2/945: chown d0/d13/d17/f104 51627 1 2024-07-16T09:55:48.415 INFO:tasks.workunit.client.0.smithi156.stdout:9/938: dread d2/d5/d16/d57/da3/ddd/f102 [0,4194304] 0 2024-07-16T09:55:48.416 INFO:tasks.workunit.client.0.smithi156.stdout:7/652: write dc/f14 [4078478,54947] 0 2024-07-16T09:55:48.416 INFO:tasks.workunit.client.0.smithi156.stdout:6/732: truncate d3/d13/d17/d4f/d30/d45/d5f/da8/fed 914245 0 2024-07-16T09:55:48.416 INFO:tasks.workunit.client.1.smithi176.stdout:2/958: fsync dd/d2e/d35/d4a/f120 0 2024-07-16T09:55:48.417 INFO:tasks.workunit.client.1.smithi176.stdout:2/959: stat dd/d49/d52/c5a 0 2024-07-16T09:55:48.418 INFO:tasks.workunit.client.1.smithi176.stdout:8/797: dwrite d8/d23/f1b [0,4194304] 0 2024-07-16T09:55:48.419 INFO:tasks.workunit.client.0.smithi156.stdout:7/653: mkdir dc/d13/d15/d8b/dcc/dde 0 2024-07-16T09:55:48.420 INFO:tasks.workunit.client.0.smithi156.stdout:8/750: creat d0/d11/ded/f10c x:0 0 0 2024-07-16T09:55:48.421 INFO:tasks.workunit.client.0.smithi156.stdout:6/733: fdatasync d3/d13/d17/d4f/d94/f2c 0 2024-07-16T09:55:48.422 INFO:tasks.workunit.client.1.smithi176.stdout:2/960: creat dd/d36/d82/d12a/d10c/f142 x:0 0 0 2024-07-16T09:55:48.422 INFO:tasks.workunit.client.0.smithi156.stdout:9/939: creat d2/d5/d16/d57/d127/f129 x:0 0 0 2024-07-16T09:55:48.423 INFO:tasks.workunit.client.0.smithi156.stdout:8/751: dread d0/dd/d1f/f47 [0,4194304] 0 2024-07-16T09:55:48.424 INFO:tasks.workunit.client.1.smithi176.stdout:2/961: read - dd/d2e/d35/d59/f13b zero size 2024-07-16T09:55:48.424 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:55:47 smithi156 ceph-mon[29302]: pgmap v255: 65 pgs: 65 active+clean; 2.9 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 106 MiB/s rd, 302 MiB/s wr, 611 op/s 2024-07-16T09:55:48.425 INFO:tasks.workunit.client.0.smithi156.stdout:9/940: chown d2/d5/d16/d57/d63/dd3/ddb/d74/dc2/dc4/d23/da0/faf 7 1 2024-07-16T09:55:48.426 INFO:tasks.workunit.client.1.smithi176.stdout:2/962: write dd/d2e/d35/d13f/d39/d3d/d7e/fc3 [6067010,89661] 0 2024-07-16T09:55:48.426 INFO:tasks.workunit.client.0.smithi156.stdout:6/734: mkdir d3/d13/dee 0 2024-07-16T09:55:48.426 INFO:tasks.workunit.client.0.smithi156.stdout:7/654: link dc/d3e/d3f/d51/d8a/dae/lca dc/d3e/dac/ldf 0 2024-07-16T09:55:48.427 INFO:tasks.workunit.client.1.smithi176.stdout:2/963: readlink dd/dcd/l123 0 2024-07-16T09:55:48.427 INFO:tasks.workunit.client.0.smithi156.stdout:8/752: mknod d0/d11/d29/d80/d97/de7/d93/c10d 0 2024-07-16T09:55:48.429 INFO:tasks.workunit.client.0.smithi156.stdout:9/941: creat d2/d5/f12a x:0 0 0 2024-07-16T09:55:48.431 INFO:tasks.workunit.client.1.smithi176.stdout:2/964: dread dd/d2e/d35/d59/d66/f6e [0,4194304] 0 2024-07-16T09:55:48.431 INFO:tasks.workunit.client.0.smithi156.stdout:8/753: symlink d0/d11/ded/l10e 0 2024-07-16T09:55:48.431 INFO:tasks.workunit.client.0.smithi156.stdout:6/735: dread d3/f1c [0,4194304] 0 2024-07-16T09:55:48.433 INFO:tasks.workunit.client.0.smithi156.stdout:8/754: stat d0/f2c 0 2024-07-16T09:55:48.433 INFO:tasks.workunit.client.0.smithi156.stdout:8/755: fsync d0/dd/f19 0 2024-07-16T09:55:48.434 INFO:tasks.workunit.client.1.smithi176.stdout:2/965: creat dd/d2e/d35/d13f/d39/d3d/d7e/f143 x:0 0 0 2024-07-16T09:55:48.434 INFO:tasks.workunit.client.0.smithi156.stdout:9/942: write d2/d5/d16/d57/d63/dd3/ddb/d74/dc2/dc4/d23/d111/da2/d98/fa7 [3450245,20489] 0 2024-07-16T09:55:48.437 INFO:tasks.workunit.client.0.smithi156.stdout:6/736: truncate d3/d13/d17/d4f/f34 4531561 0 2024-07-16T09:55:48.439 INFO:tasks.workunit.client.1.smithi176.stdout:8/798: dwrite d8/d23/d100/d31/d46/d6a/ffd [0,4194304] 0 2024-07-16T09:55:48.440 INFO:tasks.workunit.client.0.smithi156.stdout:8/756: creat d0/dd/d1f/d57/db3/f10f x:0 0 0 2024-07-16T09:55:48.440 INFO:tasks.workunit.client.0.smithi156.stdout:8/757: chown d0/dd/da3 708295306 1 2024-07-16T09:55:48.441 INFO:tasks.workunit.client.0.smithi156.stdout:8/758: truncate d0/dd/d1f/f6b 182112 0 2024-07-16T09:55:48.442 INFO:tasks.workunit.client.0.smithi156.stdout:5/838: sync 2024-07-16T09:55:48.442 INFO:tasks.workunit.client.0.smithi156.stdout:4/913: sync 2024-07-16T09:55:48.442 INFO:tasks.workunit.client.0.smithi156.stdout:1/950: sync 2024-07-16T09:55:48.443 INFO:tasks.workunit.client.0.smithi156.stdout:8/759: write d0/d11/f15 [227035,118222] 0 2024-07-16T09:55:48.443 INFO:tasks.workunit.client.0.smithi156.stdout:7/655: rename dc/d13/d36/la9 to dc/d3e/d3f/d51/d6c/le0 0 2024-07-16T09:55:48.444 INFO:tasks.workunit.client.1.smithi176.stdout:8/799: rmdir d8/d23/d100/d31/d46/d6a 39 2024-07-16T09:55:48.445 INFO:tasks.workunit.client.1.smithi176.stdout:2/966: dwrite dd/d2e/d35/d4a/f122 [0,4194304] 0 2024-07-16T09:55:48.445 INFO:tasks.workunit.client.0.smithi156.stdout:8/760: fdatasync d0/d11/d62/fc5 0 2024-07-16T09:55:48.446 INFO:tasks.workunit.client.1.smithi176.stdout:2/967: dread - dd/d36/d82/f108 zero size 2024-07-16T09:55:48.447 INFO:tasks.workunit.client.0.smithi156.stdout:9/943: fsync d2/d5/d16/d57/d63/dd3/ddb/d74/dc2/dc4/f80 0 2024-07-16T09:55:48.447 INFO:tasks.workunit.client.0.smithi156.stdout:1/951: mknod dc/d6b/d96/c13c 0 2024-07-16T09:55:48.451 INFO:tasks.workunit.client.1.smithi176.stdout:2/968: creat dd/d2e/d35/f144 x:0 0 0 2024-07-16T09:55:48.452 INFO:tasks.workunit.client.1.smithi176.stdout:8/800: getdents d8/d23/d100/db9/dcb/d34/d62/dab 0 2024-07-16T09:55:48.452 INFO:tasks.workunit.client.1.smithi176.stdout:2/969: chown dd/d49/d52/d89/lc7 751461 1 2024-07-16T09:55:48.453 INFO:tasks.workunit.client.0.smithi156.stdout:7/656: readlink dc/d13/d24/lb6 0 2024-07-16T09:55:48.454 INFO:tasks.workunit.client.0.smithi156.stdout:6/737: unlink d3/d13/d17/d4f/d94/d19/d31/d35/c7a 0 2024-07-16T09:55:48.455 INFO:tasks.workunit.client.1.smithi176.stdout:2/970: mkdir dd/dcd/d145 0 2024-07-16T09:55:48.456 INFO:tasks.workunit.client.1.smithi176.stdout:8/801: mkdir d8/d10f 0 2024-07-16T09:55:48.457 INFO:tasks.workunit.client.1.smithi176.stdout:2/971: truncate dd/d36/fb2 1425315 0 2024-07-16T09:55:48.458 INFO:tasks.workunit.client.0.smithi156.stdout:8/761: rename d0/dd to d0/d11/d20/d7f/db7/d110 0 2024-07-16T09:55:48.459 INFO:tasks.workunit.client.1.smithi176.stdout:8/802: chown d8/d23/l56 449949860 1 2024-07-16T09:55:48.459 INFO:tasks.workunit.client.0.smithi156.stdout:6/738: write d3/d13/d17/d4f/d4d/d3b/f5d [2638294,77517] 0 2024-07-16T09:55:48.459 INFO:tasks.workunit.client.1.smithi176.stdout:2/972: chown dd/d2e/d35/la6 1302025 1 2024-07-16T09:55:48.460 INFO:tasks.workunit.client.0.smithi156.stdout:7/657: symlink dc/d13/d20/d4d/le1 0 2024-07-16T09:55:48.461 INFO:tasks.workunit.client.0.smithi156.stdout:5/839: dwrite d1b/d1f/d45/d52/d43/f8d [0,4194304] 0 2024-07-16T09:55:48.461 INFO:tasks.workunit.client.1.smithi176.stdout:8/803: mknod d8/d14/d1e/d7c/d82/c110 0 2024-07-16T09:55:48.462 INFO:tasks.workunit.client.1.smithi176.stdout:8/804: stat d8/d14/d1e/d33/f105 0 2024-07-16T09:55:48.462 INFO:tasks.workunit.client.1.smithi176.stdout:2/973: chown dd/d2e/d35/d13f/d39/c54 1970 1 2024-07-16T09:55:48.462 INFO:tasks.workunit.client.0.smithi156.stdout:9/944: write d2/d5/d16/d57/d63/dd3/ddb/d74/dc2/dc4/d23/d111/da2/d98/fc6 [4080075,67190] 0 2024-07-16T09:55:48.466 INFO:tasks.workunit.client.1.smithi176.stdout:8/805: creat d8/d23/d76/d83/d103/f111 x:0 0 0 2024-07-16T09:55:48.466 INFO:tasks.workunit.client.1.smithi176.stdout:2/974: mknod dd/d2e/d35/d13f/d39/d3d/dc2/c146 0 2024-07-16T09:55:48.467 INFO:tasks.workunit.client.0.smithi156.stdout:7/658: dread - dc/d3e/d5d/f96 zero size 2024-07-16T09:55:48.467 INFO:tasks.workunit.client.0.smithi156.stdout:6/739: symlink d3/d13/d17/d4f/d30/lef 0 2024-07-16T09:55:48.467 INFO:tasks.workunit.client.0.smithi156.stdout:7/659: chown dc/f40 644540 1 2024-07-16T09:55:48.467 INFO:tasks.workunit.client.0.smithi156.stdout:9/945: stat d2/d5/d16/d57/d63/dd3/ddb/d3d/da6 0 2024-07-16T09:55:48.467 INFO:tasks.workunit.client.1.smithi176.stdout:2/975: chown dd/d36/d82/ld7 5 1 2024-07-16T09:55:48.469 INFO:tasks.workunit.client.0.smithi156.stdout:4/914: write d4/d23/d52/f85 [37101,5290] 0 2024-07-16T09:55:48.470 INFO:tasks.workunit.client.1.smithi176.stdout:8/806: rmdir d8/d14/d1e/d33/d9d/dad 0 2024-07-16T09:55:48.474 INFO:tasks.workunit.client.1.smithi176.stdout:8/807: truncate d8/d23/d100/d31/f5c 832094 0 2024-07-16T09:55:48.474 INFO:tasks.workunit.client.0.smithi156.stdout:5/840: mkdir d1b/d1f/d45/d52/d43/d85/d97/d37/d88/d10f 0 2024-07-16T09:55:48.474 INFO:tasks.workunit.client.0.smithi156.stdout:8/762: link d0/d11/d20/d7f/db7/d110/d1f/d57/da6/ccb d0/d11/d29/d80/c111 0 2024-07-16T09:55:48.476 INFO:tasks.workunit.client.1.smithi176.stdout:8/808: rmdir d8/d23/d100/db9/dcb/d37 39 2024-07-16T09:55:48.477 INFO:tasks.workunit.client.0.smithi156.stdout:4/915: fdatasync d4/db/f1d 0 2024-07-16T09:55:48.477 INFO:tasks.workunit.client.0.smithi156.stdout:8/763: chown d0/d11/d20/lf8 249277680 1 2024-07-16T09:55:48.479 INFO:tasks.workunit.client.0.smithi156.stdout:1/952: dwrite dc/d14/d8b/db4/df2/f49 [0,4194304] 0 2024-07-16T09:55:48.480 INFO:tasks.workunit.client.0.smithi156.stdout:9/946: creat d2/f12b x:0 0 0 2024-07-16T09:55:48.482 INFO:tasks.workunit.client.0.smithi156.stdout:6/740: dread d3/d13/f8e [0,4194304] 0 2024-07-16T09:55:48.483 INFO:tasks.workunit.client.1.smithi176.stdout:8/809: dwrite d8/d23/d100/d31/db4/fc8 [0,4194304] 0 2024-07-16T09:55:48.484 INFO:tasks.workunit.client.1.smithi176.stdout:8/810: fsync d8/d14/d1e/d55/fa3 0 2024-07-16T09:55:48.484 INFO:tasks.workunit.client.0.smithi156.stdout:1/953: mkdir dc/d14/d3e/d55/dec/d13d 0 2024-07-16T09:55:48.485 INFO:tasks.workunit.client.0.smithi156.stdout:7/660: write dc/f40 [985969,41626] 0 2024-07-16T09:55:48.486 INFO:tasks.workunit.client.0.smithi156.stdout:1/954: chown dc/d14/d8b/dcd/dd2/feb 53 1 2024-07-16T09:55:48.487 INFO:tasks.workunit.client.0.smithi156.stdout:7/661: dread - dc/d13/d20/d4d/fb9 zero size 2024-07-16T09:55:48.488 INFO:tasks.workunit.client.1.smithi176.stdout:8/811: readlink d8/d14/d1e/d33/la1 0 2024-07-16T09:55:48.490 INFO:tasks.workunit.client.0.smithi156.stdout:1/955: creat dc/d6b/d104/f13e x:0 0 0 2024-07-16T09:55:48.491 INFO:tasks.workunit.client.0.smithi156.stdout:8/764: mknod d0/d11/d13/d1c/c112 0 2024-07-16T09:55:48.491 INFO:tasks.workunit.client.0.smithi156.stdout:1/956: chown dc/d14/c6d 3279458 1 2024-07-16T09:55:48.494 INFO:tasks.workunit.client.0.smithi156.stdout:6/741: creat d3/d13/dee/ff0 x:0 0 0 2024-07-16T09:55:48.495 INFO:tasks.workunit.client.0.smithi156.stdout:6/742: dread - d3/d13/d17/d4f/fe9 zero size 2024-07-16T09:55:48.495 INFO:tasks.workunit.client.0.smithi156.stdout:8/765: mkdir d0/d11/d13/d113 0 2024-07-16T09:55:48.496 INFO:tasks.workunit.client.0.smithi156.stdout:5/841: truncate d1b/d1f/f20 3594814 0 2024-07-16T09:55:48.497 INFO:tasks.workunit.client.0.smithi156.stdout:4/916: link d4/la2 d4/db/dc/dcc/d58/d100/l143 0 2024-07-16T09:55:48.498 INFO:tasks.workunit.client.0.smithi156.stdout:9/947: creat d2/d5/d16/de2/d121/f12c x:0 0 0 2024-07-16T09:55:48.498 INFO:tasks.workunit.client.0.smithi156.stdout:7/662: getdents dc/d48/dd7 0 2024-07-16T09:55:48.500 INFO:tasks.workunit.client.0.smithi156.stdout:6/743: write d3/d13/d17/d4f/d30/fa3 [3633756,36891] 0 2024-07-16T09:55:48.500 INFO:tasks.workunit.client.0.smithi156.stdout:1/957: write dc/d14/d1e/f77 [1621617,91266] 0 2024-07-16T09:55:48.501 INFO:tasks.workunit.client.0.smithi156.stdout:5/842: unlink d1b/d1f/d45/d52/d43/d85/d97/d37/d88/dc7/dd9/fef 0 2024-07-16T09:55:48.504 INFO:tasks.workunit.client.0.smithi156.stdout:7/663: symlink dc/d3e/d76/le2 0 2024-07-16T09:55:48.504 INFO:tasks.workunit.client.1.smithi176.stdout:8/812: dwrite d8/d14/d1e/d33/f78 [0,4194304] 0 2024-07-16T09:55:48.505 INFO:tasks.workunit.client.0.smithi156.stdout:6/744: creat d3/d13/d17/d4f/d5c/ff1 x:0 0 0 2024-07-16T09:55:48.506 INFO:tasks.workunit.client.0.smithi156.stdout:1/958: symlink dc/d14/d3e/d55/dec/d13d/l13f 0 2024-07-16T09:55:48.506 INFO:tasks.workunit.client.1.smithi176.stdout:8/813: chown d8/d23/d100/db9/dcb/d34/d62/dab/fda 44 1 2024-07-16T09:55:48.507 INFO:tasks.workunit.client.1.smithi176.stdout:8/814: truncate d8/d23/d100/d53/f106 825376 0 2024-07-16T09:55:48.508 INFO:tasks.workunit.client.0.smithi156.stdout:1/959: write f6 [426079,98715] 0 2024-07-16T09:55:48.510 INFO:tasks.workunit.client.0.smithi156.stdout:7/664: chown dc/c26 1566619366 1 2024-07-16T09:55:48.511 INFO:tasks.workunit.client.1.smithi176.stdout:8/815: symlink d8/d14/d1e/d33/df6/l112 0 2024-07-16T09:55:48.511 INFO:tasks.workunit.client.0.smithi156.stdout:9/948: dwrite d2/d5/d16/de2/d121/f107 [0,4194304] 0 2024-07-16T09:55:48.519 INFO:tasks.workunit.client.0.smithi156.stdout:4/917: link d4/db/cc0 d4/db/dc/dcc/dec/ded/daf/c144 0 2024-07-16T09:55:48.519 INFO:tasks.workunit.client.0.smithi156.stdout:5/843: symlink d1b/d1f/d45/l110 0 2024-07-16T09:55:48.520 INFO:tasks.workunit.client.0.smithi156.stdout:6/745: dwrite d3/d13/d17/d4f/d94/d19/d31/fa2 [0,4194304] 0 2024-07-16T09:55:48.521 INFO:tasks.workunit.client.1.smithi176.stdout:8/816: dwrite d8/d23/d100/db9/dcb/d34/f84 [0,4194304] 0 2024-07-16T09:55:48.529 INFO:tasks.workunit.client.0.smithi156.stdout:4/918: fsync d4/db/dc/d7c/fe8 0 2024-07-16T09:55:48.530 INFO:tasks.workunit.client.0.smithi156.stdout:5/844: write d1b/d1f/d45/d52/d49/d4d/fe1 [498105,58662] 0 2024-07-16T09:55:48.534 INFO:tasks.workunit.client.0.smithi156.stdout:7/665: rename dc/d44 to dc/d3e/d3f/d51/de3 0 2024-07-16T09:55:48.535 INFO:tasks.workunit.client.0.smithi156.stdout:8/766: dread d0/d11/d20/d3f/d96/ff1 [0,4194304] 0 2024-07-16T09:55:48.536 INFO:tasks.workunit.client.1.smithi176.stdout:8/817: rename d8/d23/d100/d61/cee to d8/d23/d100/db9/dcb/c113 0 2024-07-16T09:55:48.537 INFO:tasks.workunit.client.1.smithi176.stdout:8/818: chown d8/d23/d100/f67 49661671 1 2024-07-16T09:55:48.538 INFO:tasks.workunit.client.0.smithi156.stdout:6/746: creat d3/d13/d17/d4f/d4d/d3b/d5a/dc1/ff2 x:0 0 0 2024-07-16T09:55:48.538 INFO:tasks.workunit.client.0.smithi156.stdout:4/919: mknod d4/c145 0 2024-07-16T09:55:48.538 INFO:tasks.workunit.client.0.smithi156.stdout:5/845: dread - d1b/d1f/d45/fc1 zero size 2024-07-16T09:55:48.539 INFO:tasks.workunit.client.0.smithi156.stdout:8/767: mkdir d0/d11/d20/d7f/daf/d114 0 2024-07-16T09:55:48.540 INFO:tasks.workunit.client.0.smithi156.stdout:4/920: stat d4/db/f3f 0 2024-07-16T09:55:48.544 INFO:tasks.workunit.client.0.smithi156.stdout:7/666: link dc/d3e/d3f/d45/fc8 dc/d3e/d3f/d51/d8a/fe4 0 2024-07-16T09:55:48.547 INFO:tasks.workunit.client.1.smithi176.stdout:8/819: write d8/d14/d1e/d7c/f8f [1151563,57440] 0 2024-07-16T09:55:48.548 INFO:tasks.workunit.client.0.smithi156.stdout:8/768: mkdir d0/d74/d115 0 2024-07-16T09:55:48.549 INFO:tasks.workunit.client.0.smithi156.stdout:9/949: dwrite d2/d5/d16/d57/d63/dd3/ddb/d74/dc2/dc4/d3a/f49 [4194304,4194304] 0 2024-07-16T09:55:48.551 INFO:tasks.workunit.client.0.smithi156.stdout:6/747: fdatasync d3/d13/d17/d4f/d94/d19/d2e/f3e 0 2024-07-16T09:55:48.555 INFO:tasks.workunit.client.0.smithi156.stdout:5/846: dwrite d1b/d1f/d45/d7f/f90 [0,4194304] 0 2024-07-16T09:55:48.556 INFO:tasks.workunit.client.0.smithi156.stdout:6/748: dread - d3/d13/d17/d4f/d4d/f70 zero size 2024-07-16T09:55:48.556 INFO:tasks.workunit.client.0.smithi156.stdout:1/960: dread dc/d14/d3e/d6c/f128 [0,4194304] 0 2024-07-16T09:55:48.557 INFO:tasks.workunit.client.0.smithi156.stdout:9/950: creat d2/d5/f12d x:0 0 0 2024-07-16T09:55:48.560 INFO:tasks.workunit.client.0.smithi156.stdout:5/847: chown d1b/d1f/d54/lbd 1428941 1 2024-07-16T09:55:48.561 INFO:tasks.workunit.client.0.smithi156.stdout:7/667: write dc/d13/d15/f78 [70139,89449] 0 2024-07-16T09:55:48.563 INFO:tasks.workunit.client.0.smithi156.stdout:4/921: write d4/db/dc/dcc/dec/f10e [1378087,57106] 0 2024-07-16T09:55:48.564 INFO:tasks.workunit.client.0.smithi156.stdout:9/951: rmdir d2/d5/d16/d22 39 2024-07-16T09:55:48.565 INFO:tasks.workunit.client.0.smithi156.stdout:8/769: dwrite d0/d11/d29/d80/f88 [0,4194304] 0 2024-07-16T09:55:48.565 INFO:tasks.workunit.client.0.smithi156.stdout:6/749: mkdir d3/d13/d17/d4f/d94/df3 0 2024-07-16T09:55:48.568 INFO:tasks.workunit.client.0.smithi156.stdout:4/922: readlink d4/db/dc/d67/l72 0 2024-07-16T09:55:48.569 INFO:tasks.workunit.client.0.smithi156.stdout:1/961: rename dc/d24/c46 to dc/d12e/c140 0 2024-07-16T09:55:48.569 INFO:tasks.workunit.client.0.smithi156.stdout:7/668: creat dc/d13/d36/dd4/fe5 x:0 0 0 2024-07-16T09:55:48.570 INFO:tasks.workunit.client.0.smithi156.stdout:8/770: readlink d0/d11/d2e/l77 0 2024-07-16T09:55:48.573 INFO:tasks.workunit.client.1.smithi176.stdout:8/820: dread d8/d23/d100/db9/dcb/d37/f57 [0,4194304] 0 2024-07-16T09:55:48.574 INFO:tasks.workunit.client.0.smithi156.stdout:4/923: mkdir d4/db/dc/d7c/d146 0 2024-07-16T09:55:48.574 INFO:tasks.workunit.client.0.smithi156.stdout:1/962: stat dc/d6b/d96/ff5 0 2024-07-16T09:55:48.575 INFO:tasks.workunit.client.0.smithi156.stdout:9/952: chown d2/d5/d16/d57/d63/dd3/ddb/d74/dc2/dc4/d23/d111/da2/d98/fc6 977 1 2024-07-16T09:55:48.576 INFO:tasks.workunit.client.0.smithi156.stdout:8/771: rmdir d0/d11/d13/d1c 39 2024-07-16T09:55:48.576 INFO:tasks.workunit.client.0.smithi156.stdout:4/924: chown d4/d23/d30/d33/l135 10475 1 2024-07-16T09:55:48.577 INFO:tasks.workunit.client.0.smithi156.stdout:5/848: getdents d1b/d1f/d42/de0/de5 0 2024-07-16T09:55:48.578 INFO:tasks.workunit.client.0.smithi156.stdout:7/669: creat dc/d63/fe6 x:0 0 0 2024-07-16T09:55:48.579 INFO:tasks.workunit.client.0.smithi156.stdout:8/772: creat d0/d74/dd3/f116 x:0 0 0 2024-07-16T09:55:48.579 INFO:tasks.workunit.client.0.smithi156.stdout:4/925: chown d4/db/dc/d7c/f8e 55724394 1 2024-07-16T09:55:48.581 INFO:tasks.workunit.client.0.smithi156.stdout:7/670: creat dc/d48/db2/fe7 x:0 0 0 2024-07-16T09:55:48.582 INFO:tasks.workunit.client.0.smithi156.stdout:3/826: sync 2024-07-16T09:55:48.583 INFO:tasks.workunit.client.0.smithi156.stdout:5/849: getdents d1b/d1f/d45/d52/d43/d85/d97/d37/d88/d10f 0 2024-07-16T09:55:48.585 INFO:tasks.workunit.client.0.smithi156.stdout:4/926: creat d4/db/dc/dcc/df3/f147 x:0 0 0 2024-07-16T09:55:48.586 INFO:tasks.workunit.client.0.smithi156.stdout:7/671: dread - dc/d63/d98/fd3 zero size 2024-07-16T09:55:48.586 INFO:tasks.workunit.client.0.smithi156.stdout:3/827: truncate d8/fff 1011102 0 2024-07-16T09:55:48.588 INFO:tasks.workunit.client.0.smithi156.stdout:9/953: dwrite d2/d5/d16/d57/d63/dd3/ddb/d74/dc2/dc4/d23/fbc [4194304,4194304] 0 2024-07-16T09:55:48.590 INFO:tasks.workunit.client.0.smithi156.stdout:4/927: mknod d4/d23/d52/d71/c148 0 2024-07-16T09:55:48.592 INFO:tasks.workunit.client.0.smithi156.stdout:8/773: dread d0/d11/d20/d7f/db7/d110/f58 [4194304,4194304] 0 2024-07-16T09:55:48.593 INFO:tasks.workunit.client.1.smithi176.stdout:8/821: dread d8/d23/d100/d31/d46/d6a/da8/fe4 [0,4194304] 0 2024-07-16T09:55:48.594 INFO:tasks.workunit.client.0.smithi156.stdout:7/672: creat dc/d3e/d3f/d51/d8a/dae/dd8/fe8 x:0 0 0 2024-07-16T09:55:48.595 INFO:tasks.workunit.client.0.smithi156.stdout:9/954: fsync d2/d5/d16/d57/d63/dd3/ddb/d74/dc2/dc4/d23/d111/da2/f93 0 2024-07-16T09:55:48.595 INFO:tasks.workunit.client.0.smithi156.stdout:7/673: fsync dc/d48/fba 0 2024-07-16T09:55:48.597 INFO:tasks.workunit.client.0.smithi156.stdout:3/828: dread d8/da/d1d/f8d [0,4194304] 0 2024-07-16T09:55:48.598 INFO:tasks.workunit.client.0.smithi156.stdout:7/674: dread - dc/d13/d36/dd4/fe5 zero size 2024-07-16T09:55:48.598 INFO:tasks.workunit.client.1.smithi176.stdout:8/822: unlink d8/d14/d1e/d7c/d82/de0/l10d 0 2024-07-16T09:55:48.599 INFO:tasks.workunit.client.0.smithi156.stdout:5/850: read d1b/d1f/d42/d6d/d9f/fda [778436,124826] 0 2024-07-16T09:55:48.599 INFO:tasks.workunit.client.0.smithi156.stdout:4/928: rename d4/db/dc/dcc/dec/ded/fcf to d4/db/dc/dcc/dec/ded/f149 0 2024-07-16T09:55:48.600 INFO:tasks.workunit.client.1.smithi176.stdout:8/823: symlink d8/d23/d100/db9/dcb/d37/l114 0 2024-07-16T09:55:48.601 INFO:tasks.workunit.client.0.smithi156.stdout:9/955: chown d2/d5/d16/d57/d63/dd3/ddb/d74/dc2/dc4/d23/d111/l95 3752 1 2024-07-16T09:55:48.602 INFO:tasks.workunit.client.0.smithi156.stdout:4/929: fsync d4/d23/d30/f2d 0 2024-07-16T09:55:48.602 INFO:tasks.workunit.client.0.smithi156.stdout:1/963: write dc/f95 [688390,88492] 0 2024-07-16T09:55:48.603 INFO:tasks.workunit.client.0.smithi156.stdout:3/829: rmdir d8/da/d1d/d6f/db3/dc4 39 2024-07-16T09:55:48.605 INFO:tasks.workunit.client.0.smithi156.stdout:1/964: chown dc/d14/d1e/f77 265320609 1 2024-07-16T09:55:48.607 INFO:tasks.workunit.client.0.smithi156.stdout:1/965: fdatasync f4 0 2024-07-16T09:55:48.607 INFO:tasks.workunit.client.0.smithi156.stdout:8/774: dwrite d0/d11/d13/d1c/ff2 [0,4194304] 0 2024-07-16T09:55:48.607 INFO:tasks.workunit.client.0.smithi156.stdout:3/830: mkdir d8/da/d21/d80/d119 0 2024-07-16T09:55:48.612 INFO:tasks.workunit.client.0.smithi156.stdout:4/930: creat d4/db/d8a/d137/f14a x:0 0 0 2024-07-16T09:55:48.612 INFO:tasks.workunit.client.0.smithi156.stdout:3/831: write f3 [237066,118215] 0 2024-07-16T09:55:48.616 INFO:tasks.workunit.client.0.smithi156.stdout:8/775: unlink d0/d11/d20/d3f/d96/ff1 0 2024-07-16T09:55:48.616 INFO:tasks.workunit.client.0.smithi156.stdout:7/675: dwrite dc/d3e/d3f/d51/d8a/fda [0,4194304] 0 2024-07-16T09:55:48.618 INFO:tasks.workunit.client.0.smithi156.stdout:1/966: write dc/d6b/fd8 [111062,102399] 0 2024-07-16T09:55:48.618 INFO:tasks.workunit.client.0.smithi156.stdout:4/931: write d4/db/d47/fbf [965719,60964] 0 2024-07-16T09:55:48.622 INFO:tasks.workunit.client.0.smithi156.stdout:5/851: dread d1b/d1f/d45/f50 [0,4194304] 0 2024-07-16T09:55:48.624 INFO:tasks.workunit.client.0.smithi156.stdout:0/892: sync 2024-07-16T09:55:48.626 INFO:tasks.workunit.client.0.smithi156.stdout:4/932: unlink d4/db/cfa 0 2024-07-16T09:55:48.627 INFO:tasks.workunit.client.0.smithi156.stdout:1/967: write dc/d14/d3e/d55/d83/fb2 [1663439,47588] 0 2024-07-16T09:55:48.627 INFO:tasks.workunit.client.0.smithi156.stdout:9/956: dwrite d2/d5/d16/d57/d63/dd3/ddb/d74/dc2/dc4/d23/d111/da2/d98/dbf/d10b/f119 [0,4194304] 0 2024-07-16T09:55:48.628 INFO:tasks.workunit.client.0.smithi156.stdout:7/676: write dc/d3e/d3f/d51/d8a/dae/dd8/fe8 [322104,97959] 0 2024-07-16T09:55:48.633 INFO:tasks.workunit.client.0.smithi156.stdout:3/832: rename d8/da/d28/d40/c6c to d8/da/d1d/d6f/db3/dc4/c11a 0 2024-07-16T09:55:48.633 INFO:tasks.workunit.client.0.smithi156.stdout:5/852: mknod d1b/d1f/d45/d7a/d98/c111 0 2024-07-16T09:55:48.634 INFO:tasks.workunit.client.0.smithi156.stdout:9/957: symlink d2/d5/d16/d57/da3/l12e 0 2024-07-16T09:55:48.636 INFO:tasks.workunit.client.0.smithi156.stdout:5/853: write d1b/d1f/d45/d52/f102 [946976,2601] 0 2024-07-16T09:55:48.639 INFO:tasks.workunit.client.0.smithi156.stdout:1/968: creat dc/d14/d8b/db4/df2/f141 x:0 0 0 2024-07-16T09:55:48.639 INFO:tasks.workunit.client.0.smithi156.stdout:9/958: stat d2/d5/d16/d57/d63/dd3/ddb/d74/dc2/dc4/d23/cad 0 2024-07-16T09:55:48.639 INFO:tasks.workunit.client.0.smithi156.stdout:3/833: rmdir d8/da/d21/d80/d99 39 2024-07-16T09:55:48.641 INFO:tasks.workunit.client.0.smithi156.stdout:1/969: symlink dc/d12e/l142 0 2024-07-16T09:55:48.642 INFO:tasks.workunit.client.0.smithi156.stdout:9/959: dread - d2/d5/d16/d57/d63/dd3/fd5 zero size 2024-07-16T09:55:48.644 INFO:tasks.workunit.client.0.smithi156.stdout:8/776: dread d0/d11/d20/d7f/db7/d110/f30 [0,4194304] 0 2024-07-16T09:55:48.646 INFO:tasks.workunit.client.0.smithi156.stdout:1/970: creat dc/d6b/dd5/dfb/f143 x:0 0 0 2024-07-16T09:55:48.646 INFO:tasks.workunit.client.0.smithi156.stdout:5/854: rename d1b/d1f/d45/d52/d43/d85/d97/d37/d88/d61/d103 to d1b/d1f/d45/d52/d43/d85/d97/d37/d88/dc7/d112 0 2024-07-16T09:55:48.646 INFO:tasks.workunit.client.0.smithi156.stdout:9/960: chown d2/d5/d16/d22/d6b/cca 160129001 1 2024-07-16T09:55:48.646 INFO:tasks.workunit.client.0.smithi156.stdout:8/777: mkdir d0/d11/d2e/d117 0 2024-07-16T09:55:48.648 INFO:tasks.workunit.client.0.smithi156.stdout:1/971: unlink dc/d6b/f134 0 2024-07-16T09:55:48.648 INFO:tasks.workunit.client.0.smithi156.stdout:5/855: rmdir d1b/d1f/d45/d52/d2d/d96 39 2024-07-16T09:55:48.650 INFO:tasks.workunit.client.0.smithi156.stdout:5/856: chown d1b/d1f/lcf 0 1 2024-07-16T09:55:48.650 INFO:tasks.workunit.client.0.smithi156.stdout:8/778: rmdir d0/d11/d29/d80/d97/de7 39 2024-07-16T09:55:48.650 INFO:tasks.workunit.client.0.smithi156.stdout:1/972: write dc/d14/d1e/f35 [2522469,91224] 0 2024-07-16T09:55:48.651 INFO:tasks.workunit.client.0.smithi156.stdout:9/961: creat d2/d5/d16/f12f x:0 0 0 2024-07-16T09:55:48.652 INFO:tasks.workunit.client.0.smithi156.stdout:1/973: chown dc/l12 164730 1 2024-07-16T09:55:48.653 INFO:tasks.workunit.client.0.smithi156.stdout:5/857: read d1b/d1f/d45/d52/d43/d85/f75 [2646973,112752] 0 2024-07-16T09:55:48.653 INFO:tasks.workunit.client.0.smithi156.stdout:4/933: dread d4/db/dc/dcc/dec/ded/db1/fd1 [0,4194304] 0 2024-07-16T09:55:48.655 INFO:tasks.workunit.client.0.smithi156.stdout:5/858: creat d1b/d1f/d45/d52/f113 x:0 0 0 2024-07-16T09:55:48.656 INFO:tasks.workunit.client.0.smithi156.stdout:1/974: chown dc/d14/d8b/db4/df2/c31 58571883 1 2024-07-16T09:55:48.656 INFO:tasks.workunit.client.0.smithi156.stdout:9/962: dread d2/d5/d16/d57/d63/dd3/ddb/dd0/def/ffe [0,4194304] 0 2024-07-16T09:55:48.656 INFO:tasks.workunit.client.0.smithi156.stdout:8/779: truncate d0/d11/d29/d80/d97/de7/f7d 28434 0 2024-07-16T09:55:48.657 INFO:tasks.workunit.client.0.smithi156.stdout:5/859: fdatasync d1b/d1f/d42/d6d/d9f/fe3 0 2024-07-16T09:55:48.658 INFO:tasks.workunit.client.0.smithi156.stdout:1/975: creat dc/d6b/dd5/dfb/f144 x:0 0 0 2024-07-16T09:55:48.659 INFO:tasks.workunit.client.0.smithi156.stdout:1/976: stat dc/d24/c68 0 2024-07-16T09:55:48.659 INFO:tasks.workunit.client.0.smithi156.stdout:8/780: creat d0/d11/d29/d80/d97/de7/f118 x:0 0 0 2024-07-16T09:55:48.660 INFO:tasks.workunit.client.0.smithi156.stdout:5/860: mkdir d1b/d1f/d45/d114 0 2024-07-16T09:55:48.660 INFO:tasks.workunit.client.0.smithi156.stdout:8/781: stat d0/d11/d13/f76 0 2024-07-16T09:55:48.662 INFO:tasks.workunit.client.0.smithi156.stdout:8/782: fdatasync d0/d11/d62/fa4 0 2024-07-16T09:55:48.662 INFO:tasks.workunit.client.0.smithi156.stdout:5/861: write d1b/d1f/d45/d52/d43/d85/d97/d37/d88/dc7/f107 [395736,56619] 0 2024-07-16T09:55:48.665 INFO:tasks.workunit.client.0.smithi156.stdout:5/862: fsync d1b/d1f/d8a/fd0 0 2024-07-16T09:55:48.666 INFO:tasks.workunit.client.0.smithi156.stdout:8/783: creat d0/d11/d20/d7f/db7/d110/d1f/d50/d8d/f119 x:0 0 0 2024-07-16T09:55:48.667 INFO:tasks.workunit.client.0.smithi156.stdout:8/784: rmdir d0/d11/d13/d1c 39 2024-07-16T09:55:48.668 INFO:tasks.workunit.client.0.smithi156.stdout:5/863: mkdir d1b/d1f/d45/d52/d43/d85/d97/d115 0 2024-07-16T09:55:48.668 INFO:tasks.workunit.client.0.smithi156.stdout:8/785: rmdir d0/d11/d62/dc2/dcd 39 2024-07-16T09:55:48.670 INFO:tasks.workunit.client.0.smithi156.stdout:1/977: dread dc/d14/d3e/d6c/fb9 [0,4194304] 0 2024-07-16T09:55:48.671 INFO:tasks.workunit.client.0.smithi156.stdout:8/786: dread d0/d11/d20/d7f/db7/d110/d1f/d57/fbf [0,4194304] 0 2024-07-16T09:55:48.673 INFO:tasks.workunit.client.0.smithi156.stdout:1/978: truncate dc/d6b/dd5/dfb/f143 671925 0 2024-07-16T09:55:48.675 INFO:tasks.workunit.client.0.smithi156.stdout:5/864: dwrite d1b/d1f/d45/d52/d2d/d96/ff8 [0,4194304] 0 2024-07-16T09:55:48.678 INFO:tasks.workunit.client.0.smithi156.stdout:5/865: creat d1b/d1f/d45/d52/d43/d85/d97/d37/d88/dc7/dd9/f116 x:0 0 0 2024-07-16T09:55:48.700 INFO:tasks.workunit.client.1.smithi176.stdout:9/914: sync 2024-07-16T09:55:48.700 INFO:tasks.workunit.client.1.smithi176.stdout:5/874: sync 2024-07-16T09:55:48.702 INFO:tasks.workunit.client.1.smithi176.stdout:9/915: write d5/de/d30/fe9 [26658,78493] 0 2024-07-16T09:55:48.702 INFO:tasks.workunit.client.0.smithi156.stdout:7/677: dread dc/d13/d15/f78 [0,4194304] 0 2024-07-16T09:55:48.703 INFO:tasks.workunit.client.1.smithi176.stdout:5/875: symlink d4/d10/d118/l13e 0 2024-07-16T09:55:48.704 INFO:tasks.workunit.client.0.smithi156.stdout:7/678: fsync dc/d3e/d3f/d51/fb0 0 2024-07-16T09:55:48.708 INFO:tasks.workunit.client.0.smithi156.stdout:7/679: dread dc/d3e/d3f/d51/f53 [4194304,4194304] 0 2024-07-16T09:55:48.708 INFO:tasks.workunit.client.1.smithi176.stdout:9/916: rename d5/lc to d5/d1d/db3/l138 0 2024-07-16T09:55:48.709 INFO:tasks.workunit.client.0.smithi156.stdout:7/680: dread - dc/d63/d98/fd3 zero size 2024-07-16T09:55:48.709 INFO:tasks.workunit.client.0.smithi156.stdout:8/787: dread d0/d11/d20/d7f/db7/d110/d1f/d57/f8f [0,4194304] 0 2024-07-16T09:55:48.710 INFO:tasks.workunit.client.1.smithi176.stdout:9/917: chown d5/de/da5/dc1/d112/d97/lc9 126906 1 2024-07-16T09:55:48.710 INFO:tasks.workunit.client.1.smithi176.stdout:5/876: link d4/d9a/lc9 d4/d10/l13f 0 2024-07-16T09:55:48.713 INFO:tasks.workunit.client.1.smithi176.stdout:5/877: mknod d4/d10/d118/c140 0 2024-07-16T09:55:48.714 INFO:tasks.workunit.client.1.smithi176.stdout:9/918: mkdir d5/de/da5/dc1/d112/d98/d139 0 2024-07-16T09:55:48.715 INFO:tasks.workunit.client.0.smithi156.stdout:8/788: unlink d0/d11/d13/d1c/d24/fb4 0 2024-07-16T09:55:48.717 INFO:tasks.workunit.client.0.smithi156.stdout:7/681: getdents dc/d3e/d3f/d51/d8a/dae/dd8 0 2024-07-16T09:55:48.718 INFO:tasks.workunit.client.1.smithi176.stdout:9/919: dread d5/d1d/d20/d78/f120 [0,4194304] 0 2024-07-16T09:55:48.718 INFO:tasks.workunit.client.0.smithi156.stdout:3/834: getdents d8/da/d1d/d6f/db3/dc4 0 2024-07-16T09:55:48.719 INFO:tasks.workunit.client.0.smithi156.stdout:7/682: write dc/d13/d15/d8d/dd0/fd5 [680967,96972] 0 2024-07-16T09:55:48.722 INFO:tasks.workunit.client.0.smithi156.stdout:8/789: rename d0/c1b to d0/d11/d62/dc2/dcd/d108/c11a 0 2024-07-16T09:55:48.722 INFO:tasks.workunit.client.0.smithi156.stdout:3/835: mkdir d8/da/d21/d80/d119/d11b 0 2024-07-16T09:55:48.723 INFO:tasks.workunit.client.0.smithi156.stdout:7/683: mknod dc/d48/db2/ce9 0 2024-07-16T09:55:48.723 INFO:tasks.workunit.client.0.smithi156.stdout:7/684: stat dc/d13/cad 0 2024-07-16T09:55:48.725 INFO:tasks.workunit.client.1.smithi176.stdout:9/920: dwrite d5/de/da5/dbf/f105 [0,4194304] 0 2024-07-16T09:55:48.725 INFO:tasks.workunit.client.0.smithi156.stdout:8/790: mkdir d0/d11/d20/d7f/db7/d110/d1f/d57/d11b 0 2024-07-16T09:55:48.728 INFO:tasks.workunit.client.0.smithi156.stdout:7/685: mkdir dc/d13/d20/dea 0 2024-07-16T09:55:48.730 INFO:tasks.workunit.client.0.smithi156.stdout:3/836: mknod d8/da/d21/d80/c11c 0 2024-07-16T09:55:48.731 INFO:tasks.workunit.client.0.smithi156.stdout:8/791: symlink d0/d11/ded/dd7/l11c 0 2024-07-16T09:55:48.732 INFO:tasks.workunit.client.0.smithi156.stdout:8/792: chown d0/d11/d62/dc2 757215 1 2024-07-16T09:55:48.733 INFO:tasks.workunit.client.0.smithi156.stdout:7/686: creat dc/d63/d98/feb x:0 0 0 2024-07-16T09:55:48.735 INFO:tasks.workunit.client.0.smithi156.stdout:3/837: chown d8/da/d21/d80/d99/fa3 1 1 2024-07-16T09:55:48.736 INFO:tasks.workunit.client.0.smithi156.stdout:8/793: fsync d0/d11/d20/d7f/db7/d110/d1f/d50/d8d/f119 0 2024-07-16T09:55:48.736 INFO:tasks.workunit.client.0.smithi156.stdout:6/750: dread d3/d13/d17/d4f/d94/d19/d31/f4e [0,4194304] 0 2024-07-16T09:55:48.739 INFO:tasks.workunit.client.0.smithi156.stdout:7/687: unlink dc/d13/d24/l7b 0 2024-07-16T09:55:48.741 INFO:tasks.workunit.client.1.smithi176.stdout:9/921: getdents d5/de/da5/dc1/d112/d98 0 2024-07-16T09:55:48.741 INFO:tasks.workunit.client.1.smithi176.stdout:9/922: stat d5 0 2024-07-16T09:55:48.741 INFO:tasks.workunit.client.0.smithi156.stdout:8/794: chown d0/d11/d13/l95 11392 1 2024-07-16T09:55:48.742 INFO:tasks.workunit.client.0.smithi156.stdout:6/751: rmdir d3/d13/d17/d4f/d94 39 2024-07-16T09:55:48.743 INFO:tasks.workunit.client.1.smithi176.stdout:9/923: creat d5/de/da5/dc1/d112/d97/f13a x:0 0 0 2024-07-16T09:55:48.743 INFO:tasks.workunit.client.1.smithi176.stdout:9/924: stat d5/d1d/db3/fd1 0 2024-07-16T09:55:48.744 INFO:tasks.workunit.client.0.smithi156.stdout:4/934: dwrite d4/db/dc/dcc/dec/ded/daf/fc5 [0,4194304] 0 2024-07-16T09:55:48.746 INFO:tasks.workunit.client.0.smithi156.stdout:7/688: read dc/f40 [380937,27953] 0 2024-07-16T09:55:48.748 INFO:tasks.workunit.client.0.smithi156.stdout:4/935: chown d4/d23/d52/ca5 4 1 2024-07-16T09:55:48.749 INFO:tasks.workunit.client.0.smithi156.stdout:5/866: dwrite d1b/d1f/d45/d52/d49/d4d/f80 [0,4194304] 0 2024-07-16T09:55:48.751 INFO:tasks.workunit.client.1.smithi176.stdout:9/925: write d5/d1d/d20/f25 [1365393,108435] 0 2024-07-16T09:55:48.752 INFO:tasks.workunit.client.0.smithi156.stdout:7/689: dread - dc/d13/fa5 zero size 2024-07-16T09:55:48.755 INFO:tasks.workunit.client.1.smithi176.stdout:9/926: symlink d5/de/da5/dc1/d112/d98/d139/l13b 0 2024-07-16T09:55:48.756 INFO:tasks.workunit.client.0.smithi156.stdout:3/838: dwrite d8/da/d28/d43/d63/f73 [0,4194304] 0 2024-07-16T09:55:48.757 INFO:tasks.workunit.client.0.smithi156.stdout:4/936: creat d4/db/dc/dcc/d58/d8f/f14b x:0 0 0 2024-07-16T09:55:48.758 INFO:tasks.workunit.client.0.smithi156.stdout:5/867: dread - d1b/d1f/d45/d52/d43/d85/d97/d37/d88/dbc/dc2/feb zero size 2024-07-16T09:55:48.758 INFO:tasks.workunit.client.0.smithi156.stdout:7/690: creat dc/d3e/d3f/d45/fec x:0 0 0 2024-07-16T09:55:48.762 INFO:tasks.workunit.client.0.smithi156.stdout:6/752: dwrite d3/d13/d17/d4f/d4d/f61 [0,4194304] 0 2024-07-16T09:55:48.762 INFO:tasks.workunit.client.0.smithi156.stdout:4/937: fdatasync d4/f3d 0 2024-07-16T09:55:48.767 INFO:tasks.workunit.client.0.smithi156.stdout:3/839: mknod d8/da/dec/d111/c11d 0 2024-07-16T09:55:48.768 INFO:tasks.workunit.client.0.smithi156.stdout:5/868: rename d1b/d1f/d8a/cb1 to d1b/d1f/d45/d52/d43/d85/d97/c117 0 2024-07-16T09:55:48.769 INFO:tasks.workunit.client.0.smithi156.stdout:6/753: readlink d3/d13/d17/d4f/d94/daa/lb1 0 2024-07-16T09:55:48.774 INFO:tasks.workunit.client.0.smithi156.stdout:4/938: rename d4/db/dc/dcc/d58/d100 to d4/d23/d52/d71/def/d14c 0 2024-07-16T09:55:48.775 INFO:tasks.workunit.client.0.smithi156.stdout:3/840: mkdir d8/da/d28/d43/d11e 0 2024-07-16T09:55:48.775 INFO:tasks.workunit.client.0.smithi156.stdout:2/946: sync 2024-07-16T09:55:48.775 INFO:tasks.workunit.client.0.smithi156.stdout:2/947: chown d0/f16 1870 1 2024-07-16T09:55:48.777 INFO:tasks.workunit.client.0.smithi156.stdout:2/948: rename d0/c2b to d0/d13/d17/d1c/d51/c135 0 2024-07-16T09:55:48.778 INFO:tasks.workunit.client.0.smithi156.stdout:2/949: read d0/d13/fff [2107730,63358] 0 2024-07-16T09:55:48.782 INFO:tasks.workunit.client.0.smithi156.stdout:2/950: dread d0/d13/f43 [0,4194304] 0 2024-07-16T09:55:48.786 INFO:tasks.workunit.client.0.smithi156.stdout:8/795: truncate d0/d11/d20/d7f/db7/d110/d1f/d57/fbf 75850 0 2024-07-16T09:55:48.788 INFO:tasks.workunit.client.0.smithi156.stdout:8/796: read d0/d11/d20/d7f/db7/d110/d1f/f8c [3706597,13101] 0 2024-07-16T09:55:48.790 INFO:tasks.workunit.client.0.smithi156.stdout:8/797: read d0/d11/d20/d3f/fb1 [3107009,84909] 0 2024-07-16T09:55:48.791 INFO:tasks.workunit.client.0.smithi156.stdout:8/798: write d0/d11/d13/d1c/d39/fe4 [2093114,124010] 0 2024-07-16T09:55:48.792 INFO:tasks.workunit.client.0.smithi156.stdout:8/799: chown d0/d11/d20/d7f/db7/d110/fa7 496022 1 2024-07-16T09:55:48.794 INFO:tasks.workunit.client.0.smithi156.stdout:8/800: getdents d0/d11/d20/d7f 0 2024-07-16T09:55:48.796 INFO:tasks.workunit.client.0.smithi156.stdout:8/801: unlink d0/c4 0 2024-07-16T09:55:48.797 INFO:tasks.workunit.client.0.smithi156.stdout:8/802: fdatasync d0/d11/d20/d7f/db7/d110/f1a 0 2024-07-16T09:55:48.800 INFO:tasks.workunit.client.0.smithi156.stdout:8/803: link d0/d11/d20/d7f/fc8 d0/d11/d29/d80/d97/de7/f11d 0 2024-07-16T09:55:48.803 INFO:tasks.workunit.client.0.smithi156.stdout:7/691: dread fb [0,4194304] 0 2024-07-16T09:55:48.806 INFO:tasks.workunit.client.0.smithi156.stdout:7/692: link dc/d48/fbc dc/d13/fed 0 2024-07-16T09:55:48.806 INFO:tasks.workunit.client.0.smithi156.stdout:6/754: write d3/d13/d17/d4f/d5c/f85 [924672,53635] 0 2024-07-16T09:55:48.807 INFO:tasks.workunit.client.0.smithi156.stdout:5/869: truncate f14 117319 0 2024-07-16T09:55:48.807 INFO:tasks.workunit.client.0.smithi156.stdout:3/841: write d8/da/d28/ffa [4216657,36771] 0 2024-07-16T09:55:48.808 INFO:tasks.workunit.client.0.smithi156.stdout:4/939: truncate d4/db/dc/fcb 5897080 0 2024-07-16T09:55:48.810 INFO:tasks.workunit.client.0.smithi156.stdout:7/693: chown dc/d13/d15/l2e 172902413 1 2024-07-16T09:55:48.810 INFO:tasks.workunit.client.0.smithi156.stdout:2/951: write d0/d13/d17/d1c/d51/dc6/f11b [4240859,5809] 0 2024-07-16T09:55:48.813 INFO:tasks.workunit.client.0.smithi156.stdout:7/694: write dc/d48/db2/fe7 [686828,126068] 0 2024-07-16T09:55:48.814 INFO:tasks.workunit.client.0.smithi156.stdout:2/952: dread - d0/d5/d29/d2d/d5b/d5f/ddc/fcd zero size 2024-07-16T09:55:48.815 INFO:tasks.workunit.client.0.smithi156.stdout:8/804: write d0/d11/d20/d7f/db7/d110/f26 [618681,26443] 0 2024-07-16T09:55:48.816 INFO:tasks.workunit.client.0.smithi156.stdout:7/695: stat dc/d13/d36/f6a 0 2024-07-16T09:55:48.818 INFO:tasks.workunit.client.0.smithi156.stdout:2/953: dread - d0/d13/d17/d1c/d86/ff5 zero size 2024-07-16T09:55:48.818 INFO:tasks.workunit.client.0.smithi156.stdout:3/842: dread d8/da/d1d/f64 [0,4194304] 0 2024-07-16T09:55:48.821 INFO:tasks.workunit.client.0.smithi156.stdout:7/696: creat dc/d13/d20/d4d/fee x:0 0 0 2024-07-16T09:55:48.821 INFO:tasks.workunit.client.0.smithi156.stdout:8/805: write d0/d11/d13/d1c/d39/fa0 [598033,70519] 0 2024-07-16T09:55:48.822 INFO:tasks.workunit.client.0.smithi156.stdout:6/755: write d3/d13/d17/d4f/d94/d19/d2e/f3e [1863302,37283] 0 2024-07-16T09:55:48.826 INFO:tasks.workunit.client.0.smithi156.stdout:3/843: truncate d8/da/d21/d80/d99/fa3 1008245 0 2024-07-16T09:55:48.826 INFO:tasks.workunit.client.0.smithi156.stdout:4/940: dread d4/db/dc/d67/f86 [0,4194304] 0 2024-07-16T09:55:48.829 INFO:tasks.workunit.client.0.smithi156.stdout:7/697: write dc/d3e/d3f/d51/d8a/fe4 [295925,47540] 0 2024-07-16T09:55:48.832 INFO:tasks.workunit.client.0.smithi156.stdout:4/941: dread - d4/db/dc/dcc/fff zero size 2024-07-16T09:55:48.832 INFO:tasks.workunit.client.0.smithi156.stdout:5/870: dwrite d1b/d1f/d45/d52/d49/d4d/f62 [0,4194304] 0 2024-07-16T09:55:48.837 INFO:tasks.workunit.client.0.smithi156.stdout:3/844: link d8/da/c26 d8/da/d28/d43/d63/c11f 0 2024-07-16T09:55:48.838 INFO:tasks.workunit.client.0.smithi156.stdout:2/954: dwrite d0/d13/d17/d1c/fe7 [0,4194304] 0 2024-07-16T09:55:48.839 INFO:tasks.workunit.client.0.smithi156.stdout:4/942: rmdir d4/db/dc/dcc/dec/ded/daf 39 2024-07-16T09:55:48.840 INFO:tasks.workunit.client.0.smithi156.stdout:6/756: dread d3/d13/d17/d4f/d4d/d3b/fbd [4194304,4194304] 0 2024-07-16T09:55:48.841 INFO:tasks.workunit.client.0.smithi156.stdout:3/845: rmdir d8/da/d1d 39 2024-07-16T09:55:48.842 INFO:tasks.workunit.client.0.smithi156.stdout:2/955: read d0/d13/d17/f34 [2965832,9694] 0 2024-07-16T09:55:48.844 INFO:tasks.workunit.client.0.smithi156.stdout:5/871: creat d1b/d1f/d45/d52/d43/d85/d97/d37/d88/d61/f118 x:0 0 0 2024-07-16T09:55:48.845 INFO:tasks.workunit.client.0.smithi156.stdout:5/872: stat d1b/d1f/d42/de0 0 2024-07-16T09:55:48.846 INFO:tasks.workunit.client.0.smithi156.stdout:3/846: mkdir d8/d120 0 2024-07-16T09:55:48.846 INFO:tasks.workunit.client.0.smithi156.stdout:2/956: write d0/d5/d76/d10f/f110 [540380,45560] 0 2024-07-16T09:55:48.847 INFO:tasks.workunit.client.0.smithi156.stdout:7/698: dwrite dc/d3e/d3f/d45/fb3 [0,4194304] 0 2024-07-16T09:55:48.847 INFO:tasks.workunit.client.0.smithi156.stdout:6/757: fdatasync d3/d13/d17/d4f/d4d/d3b/d77/fb2 0 2024-07-16T09:55:48.852 INFO:tasks.workunit.client.0.smithi156.stdout:6/758: dread - d3/d13/d17/d4f/fe9 zero size 2024-07-16T09:55:48.854 INFO:tasks.workunit.client.0.smithi156.stdout:4/943: dwrite d4/d23/d52/d71/def/d115/f12d [0,4194304] 0 2024-07-16T09:55:48.856 INFO:tasks.workunit.client.0.smithi156.stdout:7/699: mkdir dc/d13/d15/def 0 2024-07-16T09:55:48.857 INFO:tasks.workunit.client.0.smithi156.stdout:2/957: truncate d0/f14 2677737 0 2024-07-16T09:55:48.859 INFO:tasks.workunit.client.0.smithi156.stdout:4/944: write d4/d23/d30/d33/f133 [1048441,126516] 0 2024-07-16T09:55:48.859 INFO:tasks.workunit.client.0.smithi156.stdout:5/873: truncate d1b/d1f/d45/d52/d43/d85/d97/f48 2312376 0 2024-07-16T09:55:48.860 INFO:tasks.workunit.client.0.smithi156.stdout:6/759: creat d3/d13/d17/d4f/d4d/d3b/d5a/ff4 x:0 0 0 2024-07-16T09:55:48.862 INFO:tasks.workunit.client.0.smithi156.stdout:2/958: creat d0/f136 x:0 0 0 2024-07-16T09:55:48.863 INFO:tasks.workunit.client.0.smithi156.stdout:4/945: mknod d4/db/dc/d67/c14d 0 2024-07-16T09:55:48.864 INFO:tasks.workunit.client.0.smithi156.stdout:8/806: dread d0/d11/d20/d7f/db7/d110/f1e [0,4194304] 0 2024-07-16T09:55:48.864 INFO:tasks.workunit.client.0.smithi156.stdout:7/700: mknod dc/d3e/d3f/d51/cf0 0 2024-07-16T09:55:48.866 INFO:tasks.workunit.client.0.smithi156.stdout:4/946: write d4/d23/d52/f12b [327229,6179] 0 2024-07-16T09:55:48.866 INFO:tasks.workunit.client.0.smithi156.stdout:5/874: unlink d1b/d1f/d8a/ccd 0 2024-07-16T09:55:48.866 INFO:tasks.workunit.client.0.smithi156.stdout:2/959: mknod d0/c137 0 2024-07-16T09:55:48.867 INFO:tasks.workunit.client.0.smithi156.stdout:6/760: mknod d3/d13/d17/d4f/d94/d19/de5/cf5 0 2024-07-16T09:55:48.873 INFO:tasks.workunit.client.0.smithi156.stdout:2/960: write d0/d13/d17/d1c/f46 [2584195,28322] 0 2024-07-16T09:55:48.874 INFO:tasks.workunit.client.0.smithi156.stdout:3/847: dread d8/da/d28/f2a [0,4194304] 0 2024-07-16T09:55:48.874 INFO:tasks.workunit.client.0.smithi156.stdout:6/761: getdents d3/d13 0 2024-07-16T09:55:48.874 INFO:tasks.workunit.client.0.smithi156.stdout:2/961: rmdir d0/d5/d76/d10f 39 2024-07-16T09:55:48.874 INFO:tasks.workunit.client.0.smithi156.stdout:3/848: write d8/da/fd7 [674439,101189] 0 2024-07-16T09:55:48.876 INFO:tasks.workunit.client.0.smithi156.stdout:6/762: write d3/d13/d17/d4f/d94/f2c [8358376,91869] 0 2024-07-16T09:55:48.880 INFO:tasks.workunit.client.0.smithi156.stdout:7/701: dread dc/d13/d36/f66 [0,4194304] 0 2024-07-16T09:55:48.882 INFO:tasks.workunit.client.0.smithi156.stdout:7/702: dread - dc/d13/d36/f7a zero size 2024-07-16T09:55:48.883 INFO:tasks.workunit.client.0.smithi156.stdout:6/763: dread d3/d13/d17/d4f/d4d/d3b/d5a/f69 [0,4194304] 0 2024-07-16T09:55:48.884 INFO:tasks.workunit.client.0.smithi156.stdout:6/764: read d3/d13/d17/d4f/d4d/f76 [799982,4451] 0 2024-07-16T09:55:48.886 INFO:tasks.workunit.client.0.smithi156.stdout:6/765: read d3/d13/d17/d4f/d4d/d3b/f93 [2894651,125841] 0 2024-07-16T09:55:48.888 INFO:tasks.workunit.client.0.smithi156.stdout:7/703: dwrite dc/f42 [0,4194304] 0 2024-07-16T09:55:48.890 INFO:tasks.workunit.client.0.smithi156.stdout:7/704: mkdir dc/d3e/d76/df1 0 2024-07-16T09:55:48.891 INFO:tasks.workunit.client.0.smithi156.stdout:7/705: creat dc/d3e/d3f/d51/d8a/dae/ff2 x:0 0 0 2024-07-16T09:55:48.892 INFO:tasks.workunit.client.0.smithi156.stdout:9/963: sync 2024-07-16T09:55:48.892 INFO:tasks.workunit.client.0.smithi156.stdout:1/979: sync 2024-07-16T09:55:48.892 INFO:tasks.workunit.client.0.smithi156.stdout:0/893: sync 2024-07-16T09:55:48.893 INFO:tasks.workunit.client.0.smithi156.stdout:6/766: link d3/d13/d17/d4f/d94/d19/d2e/l42 d3/lf6 0 2024-07-16T09:55:48.894 INFO:tasks.workunit.client.0.smithi156.stdout:1/980: chown dc/d14/d3e/d55/d83/fb2 12589367 1 2024-07-16T09:55:48.894 INFO:tasks.workunit.client.0.smithi156.stdout:6/767: fsync d3/d13/d17/d4f/d30/d45/d5f/da8/fed 0 2024-07-16T09:55:48.895 INFO:tasks.workunit.client.0.smithi156.stdout:7/706: symlink dc/d13/d15/lf3 0 2024-07-16T09:55:48.897 INFO:tasks.workunit.client.0.smithi156.stdout:7/707: chown dc/d48/db2/dcf 8429 1 2024-07-16T09:55:48.897 INFO:tasks.workunit.client.1.smithi176.stdout:7/735: sync 2024-07-16T09:55:48.897 INFO:tasks.workunit.client.0.smithi156.stdout:9/964: truncate d2/d5/d16/de2/d121/d7f/fb9 1575580 0 2024-07-16T09:55:48.898 INFO:tasks.workunit.client.0.smithi156.stdout:6/768: creat d3/d13/d17/d56/ff7 x:0 0 0 2024-07-16T09:55:48.901 INFO:tasks.workunit.client.0.smithi156.stdout:9/965: chown d2/d5/d16/d22/d36/l28 45532 1 2024-07-16T09:55:48.903 INFO:tasks.workunit.client.0.smithi156.stdout:0/894: dwrite d0/d1a/d4d/d5e/d117/f102 [0,4194304] 0 2024-07-16T09:55:48.925 INFO:tasks.workunit.client.1.smithi176.stdout:7/736: write d4/dd/d26/dc2/fc3 [760347,47330] 0 2024-07-16T09:55:48.928 INFO:tasks.workunit.client.1.smithi176.stdout:7/737: unlink d4/dd/d26/dc2/d17/d1d/l39 0 2024-07-16T09:55:48.930 INFO:tasks.workunit.client.1.smithi176.stdout:7/738: rmdir d4/dd/d81 39 2024-07-16T09:55:48.931 INFO:tasks.workunit.client.1.smithi176.stdout:7/739: read d4/f2b [118491,23718] 0 2024-07-16T09:55:48.931 INFO:tasks.workunit.client.0.smithi156.stdout:9/966: unlink d2/d5/f12a 0 2024-07-16T09:55:48.931 INFO:tasks.workunit.client.0.smithi156.stdout:1/981: rename dc/d14/d8b/dcd/d12d/f138 to dc/d6b/f145 0 2024-07-16T09:55:48.933 INFO:tasks.workunit.client.0.smithi156.stdout:5/875: dread d1b/d1f/d45/d52/f3c [0,4194304] 0 2024-07-16T09:55:48.935 INFO:tasks.workunit.client.1.smithi176.stdout:7/740: creat d4/d47/d5d/d9a/de0/ff2 x:0 0 0 2024-07-16T09:55:48.935 INFO:tasks.workunit.client.0.smithi156.stdout:8/807: write d0/d11/fc7 [917489,22273] 0 2024-07-16T09:55:48.936 INFO:tasks.workunit.client.0.smithi156.stdout:6/769: dread d3/d13/d17/d4f/d4d/f61 [0,4194304] 0 2024-07-16T09:55:48.936 INFO:tasks.workunit.client.0.smithi156.stdout:1/982: mknod dc/d14/d3e/d55/dec/d13d/c146 0 2024-07-16T09:55:48.937 INFO:tasks.workunit.client.1.smithi176.stdout:7/741: creat d4/d47/d5d/d9a/da3/ff3 x:0 0 0 2024-07-16T09:55:48.937 INFO:tasks.workunit.client.0.smithi156.stdout:9/967: read d2/d5/d16/d57/d63/dd3/ddb/d74/dc2/dc4/d23/f60 [3659098,21232] 0 2024-07-16T09:55:48.939 INFO:tasks.workunit.client.1.smithi176.stdout:7/742: creat d4/dd/d26/dc2/d17/d1d/ddc/ff4 x:0 0 0 2024-07-16T09:55:48.941 INFO:tasks.workunit.client.1.smithi176.stdout:7/743: truncate d4/d1f/d22/d51/f98 2870280 0 2024-07-16T09:55:48.942 INFO:tasks.workunit.client.0.smithi156.stdout:1/983: readlink dc/d14/d3e/d55/d58/le9 0 2024-07-16T09:55:48.944 INFO:tasks.workunit.client.0.smithi156.stdout:9/968: chown d2/d5/d16/d22/l30 1700 1 2024-07-16T09:55:48.944 INFO:tasks.workunit.client.0.smithi156.stdout:0/895: mkdir d0/d1a/d4d/d9/d68/dcb/dd7/d11e 0 2024-07-16T09:55:48.945 INFO:tasks.workunit.client.0.smithi156.stdout:6/770: creat d3/d13/d17/d56/ff8 x:0 0 0 2024-07-16T09:55:48.946 INFO:tasks.workunit.client.0.smithi156.stdout:5/876: creat d1b/d1f/d45/d52/d43/d85/d97/d37/d88/dc7/f119 x:0 0 0 2024-07-16T09:55:48.948 INFO:tasks.workunit.client.0.smithi156.stdout:4/947: dwrite d4/db/d8a/fdb [0,4194304] 0 2024-07-16T09:55:48.950 INFO:tasks.workunit.client.0.smithi156.stdout:2/962: dwrite d0/d13/d17/d1c/fbb [0,4194304] 0 2024-07-16T09:55:48.950 INFO:tasks.workunit.client.0.smithi156.stdout:8/808: creat d0/d11/d13/d113/f11e x:0 0 0 2024-07-16T09:55:48.951 INFO:tasks.workunit.client.1.smithi176.stdout:7/744: truncate d4/dd/f1a 2750393 0 2024-07-16T09:55:48.952 INFO:tasks.workunit.client.0.smithi156.stdout:3/849: dwrite d8/fb [0,4194304] 0 2024-07-16T09:55:48.952 INFO:tasks.workunit.client.0.smithi156.stdout:0/896: chown d0/d1a/d4d/d9/f115 2372063 1 2024-07-16T09:55:48.953 INFO:tasks.workunit.client.0.smithi156.stdout:1/984: getdents dc/d6b/d104/d13a 0 2024-07-16T09:55:48.953 INFO:tasks.workunit.client.1.smithi176.stdout:7/745: dread - d4/dd/d81/fd9 zero size 2024-07-16T09:55:48.960 INFO:tasks.workunit.client.0.smithi156.stdout:1/985: chown dc/d6b/d96/d11a 12 1 2024-07-16T09:55:48.963 INFO:tasks.workunit.client.0.smithi156.stdout:8/809: creat d0/d11/d20/d3f/dd4/f11f x:0 0 0 2024-07-16T09:55:48.965 INFO:tasks.workunit.client.0.smithi156.stdout:5/877: mknod d1b/d1f/d45/d52/d43/d85/d97/d37/d88/dc7/c11a 0 2024-07-16T09:55:48.966 INFO:tasks.workunit.client.0.smithi156.stdout:9/969: truncate d2/d5/d16/d57/d63/dd3/ddb/d74/dc2/fe1 852389 0 2024-07-16T09:55:48.967 INFO:tasks.workunit.client.0.smithi156.stdout:0/897: mkdir d0/d1a/d4d/d9/d27/d50/d11b/d11c/d11f 0 2024-07-16T09:55:48.968 INFO:tasks.workunit.client.0.smithi156.stdout:3/850: getdents d8/da/d1d/d2c/de0/dad 0 2024-07-16T09:55:48.968 INFO:tasks.workunit.client.0.smithi156.stdout:8/810: chown d0/d11/d20/d3f/fe9 446796 1 2024-07-16T09:55:48.968 INFO:tasks.workunit.client.0.smithi156.stdout:6/771: dwrite d3/d13/d17/d4f/d94/f2c [4194304,4194304] 0 2024-07-16T09:55:48.969 INFO:tasks.workunit.client.0.smithi156.stdout:1/986: chown dc/d14/d3e/d6c/d7a/l121 10 1 2024-07-16T09:55:48.971 INFO:tasks.workunit.client.0.smithi156.stdout:0/898: mknod d0/d1a/d4d/d6e/c120 0 2024-07-16T09:55:48.973 INFO:tasks.workunit.client.0.smithi156.stdout:3/851: fsync d8/da/d21/f6d 0 2024-07-16T09:55:48.973 INFO:tasks.workunit.client.0.smithi156.stdout:1/987: mknod dc/d12e/c147 0 2024-07-16T09:55:48.974 INFO:tasks.workunit.client.0.smithi156.stdout:5/878: creat d1b/d1f/d45/d52/d43/d85/d97/d37/d88/d61/f11b x:0 0 0 2024-07-16T09:55:48.976 INFO:tasks.workunit.client.0.smithi156.stdout:1/988: truncate dc/d24/fc2 219866 0 2024-07-16T09:55:48.976 INFO:tasks.workunit.client.0.smithi156.stdout:3/852: mkdir d8/d14/d121 0 2024-07-16T09:55:48.977 INFO:tasks.workunit.client.0.smithi156.stdout:6/772: mkdir d3/d13/d17/d4f/d4d/d3b/d5a/dc1/df9 0 2024-07-16T09:55:48.977 INFO:tasks.workunit.client.0.smithi156.stdout:1/989: chown dc/d24/fb1 7 1 2024-07-16T09:55:48.978 INFO:tasks.workunit.client.0.smithi156.stdout:1/990: readlink dc/d14/d8b/dcd/lf6 0 2024-07-16T09:55:48.979 INFO:tasks.workunit.client.0.smithi156.stdout:1/991: write f4 [1277658,57512] 0 2024-07-16T09:55:48.980 INFO:tasks.workunit.client.0.smithi156.stdout:6/773: symlink d3/d13/d17/d4f/d1e/lfa 0 2024-07-16T09:55:48.981 INFO:tasks.workunit.client.0.smithi156.stdout:0/899: link d0/d1a/d4d/lef d0/d1a/d4d/d9/d27/d2c/d6d/d9d/ddc/l121 0 2024-07-16T09:55:48.984 INFO:tasks.workunit.client.0.smithi156.stdout:6/774: truncate d3/d13/d17/d4f/d5c/ff1 159069 0 2024-07-16T09:55:48.984 INFO:tasks.workunit.client.0.smithi156.stdout:1/992: rmdir dc/d14/d3e/d6c/d7a 39 2024-07-16T09:55:48.987 INFO:tasks.workunit.client.0.smithi156.stdout:6/775: mkdir d3/d13/d17/dae/dfb 0 2024-07-16T09:55:48.991 INFO:tasks.workunit.client.0.smithi156.stdout:0/900: dwrite d0/d1a/d76/f87 [0,4194304] 0 2024-07-16T09:55:48.992 INFO:tasks.workunit.client.0.smithi156.stdout:6/776: write d3/d13/f8e [2513555,29229] 0 2024-07-16T09:55:48.993 INFO:tasks.workunit.client.1.smithi176.stdout:7/746: dread d4/dd/f23 [0,4194304] 0 2024-07-16T09:55:48.997 INFO:tasks.workunit.client.0.smithi156.stdout:3/853: dread d8/da/d28/f66 [0,4194304] 0 2024-07-16T09:55:48.997 INFO:tasks.workunit.client.0.smithi156.stdout:4/948: truncate d4/db/d8a/fdb 1824876 0 2024-07-16T09:55:48.998 INFO:tasks.workunit.client.1.smithi176.stdout:7/747: rename d4/dd/d26/l43 to d4/dd/d8e/lf5 0 2024-07-16T09:55:48.998 INFO:tasks.workunit.client.0.smithi156.stdout:2/963: write d0/d5/d29/fbe [765315,97736] 0 2024-07-16T09:55:48.999 INFO:tasks.workunit.client.0.smithi156.stdout:0/901: unlink d0/d1a/d76/d92/l98 0 2024-07-16T09:55:49.001 INFO:tasks.workunit.client.0.smithi156.stdout:9/970: truncate d2/d5/d16/d57/d63/dd3/ddb/dd0/def/ffe 90413 0 2024-07-16T09:55:49.001 INFO:tasks.workunit.client.0.smithi156.stdout:5/879: dread d1b/d1f/d45/d52/d49/ff1 [0,4194304] 0 2024-07-16T09:55:49.002 INFO:tasks.workunit.client.0.smithi156.stdout:4/949: rmdir d4/db/dc/d67 39 2024-07-16T09:55:49.005 INFO:tasks.workunit.client.1.smithi176.stdout:7/748: dwrite d4/dd/f2e [0,4194304] 0 2024-07-16T09:55:49.005 INFO:tasks.workunit.client.0.smithi156.stdout:3/854: dread - d8/da/d1d/d2c/de0/df8/f116 zero size 2024-07-16T09:55:49.006 INFO:tasks.workunit.client.0.smithi156.stdout:6/777: mknod d3/d13/d17/d4f/d94/d19/d31/d35/cfc 0 2024-07-16T09:55:49.008 INFO:tasks.workunit.client.0.smithi156.stdout:1/993: dwrite dc/d14/d3e/d55/f109 [0,4194304] 0 2024-07-16T09:55:49.009 INFO:tasks.workunit.client.0.smithi156.stdout:0/902: unlink d0/d1a/d4d/d9/d27/d2c/d6d/la0 0 2024-07-16T09:55:49.009 INFO:tasks.workunit.client.0.smithi156.stdout:2/964: write d0/d5/d29/d2d/d5b/d5f/d89/ffd [771583,47167] 0 2024-07-16T09:55:49.015 INFO:tasks.workunit.client.1.smithi176.stdout:3/655: sync 2024-07-16T09:55:49.018 INFO:tasks.workunit.client.0.smithi156.stdout:4/950: creat d4/db/dc/dcc/d58/d8f/d11f/f14e x:0 0 0 2024-07-16T09:55:49.023 INFO:tasks.workunit.client.0.smithi156.stdout:8/811: dread d0/d11/d20/d7f/db7/d110/f3b [0,4194304] 0 2024-07-16T09:55:49.024 INFO:tasks.workunit.client.0.smithi156.stdout:1/994: symlink dc/d14/d8b/db4/df2/d10d/l148 0 2024-07-16T09:55:49.024 INFO:tasks.workunit.client.1.smithi176.stdout:3/656: write d3/d17/d1d/f63 [986829,33445] 0 2024-07-16T09:55:49.024 INFO:tasks.workunit.client.0.smithi156.stdout:4/951: creat d4/db/dc/d5f/f14f x:0 0 0 2024-07-16T09:55:49.026 INFO:tasks.workunit.client.0.smithi156.stdout:8/812: chown d0/d11/d20/d3f/dd4/f11f 580 1 2024-07-16T09:55:49.026 INFO:tasks.workunit.client.0.smithi156.stdout:9/971: dwrite d2/d5/d16/d57/d63/dd3/ddb/d74/dc2/dc4/d23/fbc [4194304,4194304] 0 2024-07-16T09:55:49.027 INFO:tasks.workunit.client.0.smithi156.stdout:3/855: creat d8/da/d21/d80/f122 x:0 0 0 2024-07-16T09:55:49.028 INFO:tasks.workunit.client.0.smithi156.stdout:4/952: mkdir d4/db/dc/dcc/d150 0 2024-07-16T09:55:49.028 INFO:tasks.workunit.client.1.smithi176.stdout:3/657: creat d3/da/d84/fc8 x:0 0 0 2024-07-16T09:55:49.030 INFO:tasks.workunit.client.1.smithi176.stdout:3/658: chown d3/da/d6c/d86/f92 9 1 2024-07-16T09:55:49.030 INFO:tasks.workunit.client.0.smithi156.stdout:1/995: mknod dc/d6b/d104/c149 0 2024-07-16T09:55:49.031 INFO:tasks.workunit.client.1.smithi176.stdout:7/749: dwrite d4/d1f/f62 [0,4194304] 0 2024-07-16T09:55:49.031 INFO:tasks.workunit.client.0.smithi156.stdout:3/856: readlink d8/da/d28/d43/d63/l9a 0 2024-07-16T09:55:49.032 INFO:tasks.workunit.client.0.smithi156.stdout:1/996: fdatasync dc/d6b/d104/f13e 0 2024-07-16T09:55:49.033 INFO:tasks.workunit.client.0.smithi156.stdout:9/972: fsync d2/d5/d16/d57/d63/dd3/ddb/d3d/f69 0 2024-07-16T09:55:49.034 INFO:tasks.workunit.client.0.smithi156.stdout:3/857: rmdir d8/da/d13/dca 39 2024-07-16T09:55:49.036 INFO:tasks.workunit.client.1.smithi176.stdout:7/750: mknod d4/d1f/cf6 0 2024-07-16T09:55:49.037 INFO:tasks.workunit.client.0.smithi156.stdout:9/973: chown d2/d5/d16/de2/d121/d7f/d42/l7d 10946 1 2024-07-16T09:55:49.037 INFO:tasks.workunit.client.0.smithi156.stdout:4/953: dwrite d4/d23/d30/f123 [0,4194304] 0 2024-07-16T09:55:49.037 INFO:tasks.workunit.client.1.smithi176.stdout:3/659: fdatasync d3/d17/d83/f8a 0 2024-07-16T09:55:49.037 INFO:tasks.workunit.client.0.smithi156.stdout:4/954: stat d4/db/l5a 0 2024-07-16T09:55:49.039 INFO:tasks.workunit.client.1.smithi176.stdout:7/751: truncate d4/dd/f23 1271776 0 2024-07-16T09:55:49.040 INFO:tasks.workunit.client.1.smithi176.stdout:3/660: stat d3/d17/d83/fb0 0 2024-07-16T09:55:49.040 INFO:tasks.workunit.client.1.smithi176.stdout:3/661: chown d3/d17 2 1 2024-07-16T09:55:49.041 INFO:tasks.workunit.client.0.smithi156.stdout:4/955: dread - d4/db/d8a/f96 zero size 2024-07-16T09:55:49.043 INFO:tasks.workunit.client.1.smithi176.stdout:3/662: rmdir d3/da/d33/dbd 39 2024-07-16T09:55:49.044 INFO:tasks.workunit.client.1.smithi176.stdout:3/663: chown d3/da/d56/l65 25822392 1 2024-07-16T09:55:49.045 INFO:tasks.workunit.client.1.smithi176.stdout:8/824: sync 2024-07-16T09:55:49.045 INFO:tasks.workunit.client.1.smithi176.stdout:6/927: sync 2024-07-16T09:55:49.045 INFO:tasks.workunit.client.1.smithi176.stdout:1/567: sync 2024-07-16T09:55:49.045 INFO:tasks.workunit.client.1.smithi176.stdout:5/878: sync 2024-07-16T09:55:49.046 INFO:tasks.workunit.client.1.smithi176.stdout:4/789: sync 2024-07-16T09:55:49.046 INFO:tasks.workunit.client.1.smithi176.stdout:9/927: sync 2024-07-16T09:55:49.046 INFO:tasks.workunit.client.1.smithi176.stdout:0/938: sync 2024-07-16T09:55:49.046 INFO:tasks.workunit.client.1.smithi176.stdout:2/976: sync 2024-07-16T09:55:49.046 INFO:tasks.workunit.client.0.smithi156.stdout:9/974: symlink d2/d5/d16/d57/d63/dd3/ddb/d74/dc2/dc4/d23/d111/da2/d98/dbf/d10b/l130 0 2024-07-16T09:55:49.047 INFO:tasks.workunit.client.0.smithi156.stdout:4/956: mkdir d4/d23/d52/d71/def/d109/d151 0 2024-07-16T09:55:49.048 INFO:tasks.workunit.client.0.smithi156.stdout:4/957: fdatasync d4/d23/d30/f2d 0 2024-07-16T09:55:49.048 INFO:tasks.workunit.client.1.smithi176.stdout:3/664: unlink d3/d17/d83/l9c 0 2024-07-16T09:55:49.048 INFO:tasks.workunit.client.0.smithi156.stdout:9/975: chown d2/d5/d16/d22/d36/lbb 9 1 2024-07-16T09:55:49.048 INFO:tasks.workunit.client.0.smithi156.stdout:3/858: link d8/da/d28/d43/d63/c11f d8/da/d1d/d6f/c123 0 2024-07-16T09:55:49.050 INFO:tasks.workunit.client.1.smithi176.stdout:6/928: chown d1/d5/d47/f83 534 1 2024-07-16T09:55:49.050 INFO:tasks.workunit.client.0.smithi156.stdout:3/859: dread - d8/da/d1d/d2c/de0/dad/f104 zero size 2024-07-16T09:55:49.051 INFO:tasks.workunit.client.0.smithi156.stdout:3/860: readlink d8/da/d28/d40/l6a 0 2024-07-16T09:55:49.051 INFO:tasks.workunit.client.1.smithi176.stdout:2/977: readlink dd/d2e/d35/d13f/d39/l72 0 2024-07-16T09:55:49.051 INFO:tasks.workunit.client.0.smithi156.stdout:9/976: symlink d2/d5/d16/d22/d36/deb/l131 0 2024-07-16T09:55:49.052 INFO:tasks.workunit.client.1.smithi176.stdout:9/928: write d5/d1d/d20/d78/df2/d118/d11c/f127 [744871,73903] 0 2024-07-16T09:55:49.052 INFO:tasks.workunit.client.1.smithi176.stdout:3/665: fdatasync d3/da/f13 0 2024-07-16T09:55:49.053 INFO:tasks.workunit.client.1.smithi176.stdout:4/790: chown da/dc/d4b/df0/d96/d9b/db4/d40 244737 1 2024-07-16T09:55:49.055 INFO:tasks.workunit.client.0.smithi156.stdout:3/861: rename d8/da/d21/d80/d119/d11b to d8/d14/d124 0 2024-07-16T09:55:49.056 INFO:tasks.workunit.client.0.smithi156.stdout:9/977: chown d2/d5/d16/d57/d63/dd3/ddb/d74/dc2/dc4/f2c 8288894 1 2024-07-16T09:55:49.058 INFO:tasks.workunit.client.0.smithi156.stdout:3/862: mknod d8/d14/c125 0 2024-07-16T09:55:49.058 INFO:tasks.workunit.client.1.smithi176.stdout:4/791: dread - da/d3d/d33/fcd zero size 2024-07-16T09:55:49.058 INFO:tasks.workunit.client.1.smithi176.stdout:9/929: creat d5/de/da5/dbf/f13c x:0 0 0 2024-07-16T09:55:49.059 INFO:tasks.workunit.client.0.smithi156.stdout:9/978: chown d2/d5/d16/d57/d63/dd3/ddb/d74/dc2/dc4/d23/d111/da2/d98/dbf/ccf 519531192 1 2024-07-16T09:55:49.061 INFO:tasks.workunit.client.1.smithi176.stdout:6/929: getdents d1/d5d/db9/d101/d12c 0 2024-07-16T09:55:49.063 INFO:tasks.workunit.client.0.smithi156.stdout:3/863: dwrite d8/d14/f59 [0,4194304] 0 2024-07-16T09:55:49.064 INFO:tasks.workunit.client.1.smithi176.stdout:2/978: creat dd/d2e/d35/d13f/d39/f147 x:0 0 0 2024-07-16T09:55:49.064 INFO:tasks.workunit.client.0.smithi156.stdout:9/979: creat d2/d5/d16/de2/d121/d7f/d114/f132 x:0 0 0 2024-07-16T09:55:49.065 INFO:tasks.workunit.client.0.smithi156.stdout:3/864: chown d8/da/d28/d43/d63/c95 26 1 2024-07-16T09:55:49.065 INFO:tasks.workunit.client.1.smithi176.stdout:8/825: dread d8/d23/d100/db9/dcb/d34/f84 [0,4194304] 0 2024-07-16T09:55:49.065 INFO:tasks.workunit.client.1.smithi176.stdout:6/930: stat d1/d5/d47/d62/d77/dd2/c11a 0 2024-07-16T09:55:49.067 INFO:tasks.workunit.client.0.smithi156.stdout:3/865: chown d8/da/d1d/d6f/db3/dc4/ff0 123755 1 2024-07-16T09:55:49.068 INFO:tasks.workunit.client.0.smithi156.stdout:3/866: stat d8/da/d1d/d6f/fb5 0 2024-07-16T09:55:49.068 INFO:tasks.workunit.client.0.smithi156.stdout:9/980: write d2/d5/d16/d57/d63/dd3/ddb/d74/dc2/dc4/d23/d111/da2/d98/fc6 [4374094,63584] 0 2024-07-16T09:55:49.070 INFO:tasks.workunit.client.1.smithi176.stdout:2/979: write dd/d2e/d35/d13f/f56 [938541,78535] 0 2024-07-16T09:55:49.070 INFO:tasks.workunit.client.1.smithi176.stdout:1/568: truncate d4/d8/d12/d21/d25/d30/d44/d4d/f96 1003839 0 2024-07-16T09:55:49.070 INFO:tasks.workunit.client.1.smithi176.stdout:6/931: dread - d1/d5/d47/d75/fbe zero size 2024-07-16T09:55:49.071 INFO:tasks.workunit.client.0.smithi156.stdout:3/867: dread - d8/da/d28/d43/dd8/f112 zero size 2024-07-16T09:55:49.073 INFO:tasks.workunit.client.1.smithi176.stdout:0/939: dread d5/d1c/d5a/f86 [0,4194304] 0 2024-07-16T09:55:49.073 INFO:tasks.workunit.client.1.smithi176.stdout:9/930: symlink d5/de/df9/l13d 0 2024-07-16T09:55:49.074 INFO:tasks.workunit.client.1.smithi176.stdout:8/826: rename d8/d23/d100/d31/d2f/l7b to d8/d23/d100/db9/l115 0 2024-07-16T09:55:49.075 INFO:tasks.workunit.client.0.smithi156.stdout:3/868: mkdir d8/da/d1d/db2/d126 0 2024-07-16T09:55:49.076 INFO:tasks.workunit.client.0.smithi156.stdout:5/880: dwrite d1b/d1f/d42/d6d/d9f/fda [0,4194304] 0 2024-07-16T09:55:49.076 INFO:tasks.workunit.client.0.smithi156.stdout:3/869: write d8/da/d13/fb9 [4104888,70812] 0 2024-07-16T09:55:49.080 INFO:tasks.workunit.client.1.smithi176.stdout:1/569: creat d4/d77/dad/fbd x:0 0 0 2024-07-16T09:55:49.084 INFO:tasks.workunit.client.1.smithi176.stdout:6/932: creat d1/d5/d47/d62/d80/d13d/f151 x:0 0 0 2024-07-16T09:55:49.085 INFO:tasks.workunit.client.1.smithi176.stdout:8/827: creat d8/d23/d100/d31/d46/d77/f116 x:0 0 0 2024-07-16T09:55:49.088 INFO:tasks.workunit.client.0.smithi156.stdout:9/981: symlink d2/d5/d16/d57/d63/dd3/ddb/d3d/l133 0 2024-07-16T09:55:49.088 INFO:tasks.workunit.client.0.smithi156.stdout:2/965: dwrite d0/d5/d76/fc4 [0,4194304] 0 2024-07-16T09:55:49.090 INFO:tasks.workunit.client.0.smithi156.stdout:8/813: dwrite d0/d11/f40 [0,4194304] 0 2024-07-16T09:55:49.090 INFO:tasks.workunit.client.0.smithi156.stdout:6/778: link d3/d13/d17/d4f/d94/d19/d31/d35/f91 d3/d13/d17/d4f/d1e/ffd 0 2024-07-16T09:55:49.090 INFO:tasks.workunit.client.0.smithi156.stdout:3/870: mkdir d8/da/d28/d40/d127 0 2024-07-16T09:55:49.092 INFO:tasks.workunit.client.1.smithi176.stdout:6/933: readlink d1/d5/d7/d1e/lcf 0 2024-07-16T09:55:49.092 INFO:tasks.workunit.client.1.smithi176.stdout:5/879: write d4/d14/d19/d1b/d61/de7/f120 [345303,108861] 0 2024-07-16T09:55:49.092 INFO:tasks.workunit.client.1.smithi176.stdout:1/570: chown d4/d3a/d54/d5a/d7a/c9b 83642 1 2024-07-16T09:55:49.093 INFO:tasks.workunit.client.1.smithi176.stdout:2/980: dwrite dd/d36/f94 [0,4194304] 0 2024-07-16T09:55:49.094 INFO:tasks.workunit.client.1.smithi176.stdout:4/792: write da/dc/f45 [1522873,10088] 0 2024-07-16T09:55:49.098 INFO:tasks.workunit.client.1.smithi176.stdout:6/934: write d1/dc/f9f [552075,103479] 0 2024-07-16T09:55:49.098 INFO:tasks.workunit.client.0.smithi156.stdout:9/982: write d2/f4 [4035641,2093] 0 2024-07-16T09:55:49.098 INFO:tasks.workunit.client.0.smithi156.stdout:5/881: dread d1b/f34 [0,4194304] 0 2024-07-16T09:55:49.099 INFO:tasks.workunit.client.0.smithi156.stdout:5/882: fdatasync d1b/d1f/d45/d52/d49/d4d/f62 0 2024-07-16T09:55:49.099 INFO:tasks.workunit.client.0.smithi156.stdout:8/814: rmdir d0/d11/d29/d5e 39 2024-07-16T09:55:49.099 INFO:tasks.workunit.client.0.smithi156.stdout:2/966: truncate d0/d5/d29/d2d/d5b/df3/f106 699414 0 2024-07-16T09:55:49.103 INFO:tasks.workunit.client.1.smithi176.stdout:5/880: chown d4/f6 0 1 2024-07-16T09:55:49.104 INFO:tasks.workunit.client.0.smithi156.stdout:8/815: dread d0/d11/f40 [0,4194304] 0 2024-07-16T09:55:49.106 INFO:tasks.workunit.client.0.smithi156.stdout:8/816: chown d0/d11/d20/d7f/db7 5 1 2024-07-16T09:55:49.107 INFO:tasks.workunit.client.1.smithi176.stdout:2/981: chown dd/d36/d82/d12a/d10c/f142 114216181 1 2024-07-16T09:55:49.108 INFO:tasks.workunit.client.1.smithi176.stdout:0/940: dwrite d5/d1c/db1/ff3 [0,4194304] 0 2024-07-16T09:55:49.109 INFO:tasks.workunit.client.0.smithi156.stdout:9/983: dread d2/d5/d16/d57/d63/dd3/ddb/d74/dc2/de6/ff0 [0,4194304] 0 2024-07-16T09:55:49.114 INFO:tasks.workunit.client.1.smithi176.stdout:6/935: write d1/d5d/d9e/d76/f103 [2314389,56763] 0 2024-07-16T09:55:49.114 INFO:tasks.workunit.client.1.smithi176.stdout:5/881: fdatasync d4/d6c/f7c 0 2024-07-16T09:55:49.115 INFO:tasks.workunit.client.0.smithi156.stdout:8/817: mkdir d0/d11/d20/d7f/db7/d110/d1f/d57/dc0/dff/d120 0 2024-07-16T09:55:49.115 INFO:tasks.workunit.client.1.smithi176.stdout:4/793: creat da/dc/d4b/df0/d96/d9b/db4/d40/d5d/d73/f101 x:0 0 0 2024-07-16T09:55:49.116 INFO:tasks.workunit.client.1.smithi176.stdout:3/666: dwrite d3/d17/d1d/d4f/d59/f34 [4194304,4194304] 0 2024-07-16T09:55:49.117 INFO:tasks.workunit.client.0.smithi156.stdout:9/984: creat d2/d5/d16/d57/d63/dd3/ddb/d3d/f134 x:0 0 0 2024-07-16T09:55:49.117 INFO:tasks.workunit.client.1.smithi176.stdout:0/941: chown d5/d1c/d24/d127/d84 30 1 2024-07-16T09:55:49.118 INFO:tasks.workunit.client.0.smithi156.stdout:8/818: write d0/d11/d20/d7f/db7/d110/d1f/d50/d8d/f119 [245118,127796] 0 2024-07-16T09:55:49.120 INFO:tasks.workunit.client.0.smithi156.stdout:9/985: mkdir d2/df5/d135 0 2024-07-16T09:55:49.121 INFO:tasks.workunit.client.0.smithi156.stdout:8/819: write d0/d11/d62/fa4 [4157521,15327] 0 2024-07-16T09:55:49.121 INFO:tasks.workunit.client.1.smithi176.stdout:2/982: chown dd/d2e/d35/d13f/d39/d3d/c57 1819173 1 2024-07-16T09:55:49.123 INFO:tasks.workunit.client.0.smithi156.stdout:9/986: creat d2/d5/d16/d22/d36/f136 x:0 0 0 2024-07-16T09:55:49.126 INFO:tasks.workunit.client.0.smithi156.stdout:9/987: write d2/d5/d16/d57/d63/dd3/ddb/d74/dc2/dc4/d23/d111/f9f [1471588,20386] 0 2024-07-16T09:55:49.127 INFO:tasks.workunit.client.0.smithi156.stdout:8/820: rename d0/d11/d20/d7f/db7/d110/da3 to d0/d11/d13/d1c/d24/d121 0 2024-07-16T09:55:49.127 INFO:tasks.workunit.client.1.smithi176.stdout:5/882: mkdir d4/d14/d19/d1b/d61/d141 0 2024-07-16T09:55:49.127 INFO:tasks.workunit.client.1.smithi176.stdout:3/667: creat d3/da/d33/fc9 x:0 0 0 2024-07-16T09:55:49.127 INFO:tasks.workunit.client.1.smithi176.stdout:2/983: mknod dd/d36/d45/c148 0 2024-07-16T09:55:49.130 INFO:tasks.workunit.client.1.smithi176.stdout:6/936: dwrite d1/dc/d81/fe0 [0,4194304] 0 2024-07-16T09:55:49.130 INFO:tasks.workunit.client.0.smithi156.stdout:8/821: rmdir d0/d11/d20/d3f 39 2024-07-16T09:55:49.131 INFO:tasks.workunit.client.0.smithi156.stdout:8/822: chown d0/d11/d20/d7f/db7/d110/fa7 336019 1 2024-07-16T09:55:49.132 INFO:tasks.workunit.client.0.smithi156.stdout:8/823: readlink d0/d11/d13/d1c/d24/d121/ldb 0 2024-07-16T09:55:49.133 INFO:tasks.workunit.client.1.smithi176.stdout:0/942: rename d5/d1c/d24/cad to d5/d1c/d24/d35/d50/dd5/d118/c137 0 2024-07-16T09:55:49.134 INFO:tasks.workunit.client.1.smithi176.stdout:3/668: mkdir d3/da/dca 0 2024-07-16T09:55:49.134 INFO:tasks.workunit.client.1.smithi176.stdout:2/984: mknod dd/d36/c149 0 2024-07-16T09:55:49.136 INFO:tasks.workunit.client.1.smithi176.stdout:0/943: chown d5/d1c/d24/c28 60229 1 2024-07-16T09:55:49.138 INFO:tasks.workunit.client.0.smithi156.stdout:8/824: getdents d0/d74 0 2024-07-16T09:55:49.139 INFO:tasks.workunit.client.1.smithi176.stdout:6/937: mkdir d1/d5/d7/dae/d152 0 2024-07-16T09:55:49.139 INFO:tasks.workunit.client.1.smithi176.stdout:0/944: stat d5/d1c/d24/d35/d50/d9b/de3/f125 0 2024-07-16T09:55:49.140 INFO:tasks.workunit.client.0.smithi156.stdout:8/825: write d0/d11/d4a/fc3 [4236319,49313] 0 2024-07-16T09:55:49.140 INFO:tasks.workunit.client.1.smithi176.stdout:6/938: chown d1/dc/d5b 272285 1 2024-07-16T09:55:49.140 INFO:tasks.workunit.client.1.smithi176.stdout:0/945: chown d5/c11 138 1 2024-07-16T09:55:49.140 INFO:tasks.workunit.client.1.smithi176.stdout:5/883: rename d4/d10/d17/c2b to d4/da9/dc4/deb/d122/c142 0 2024-07-16T09:55:49.141 INFO:tasks.workunit.client.1.smithi176.stdout:6/939: chown d1/d5d/db9/d101/d105 1552813 1 2024-07-16T09:55:49.141 INFO:tasks.workunit.client.1.smithi176.stdout:2/985: mkdir dd/d2e/d14a 0 2024-07-16T09:55:49.142 INFO:tasks.workunit.client.0.smithi156.stdout:9/988: dread d2/d5/d16/d57/d63/dd3/ddb/d74/dc2/dc4/d23/fbc [0,4194304] 0 2024-07-16T09:55:49.144 INFO:tasks.workunit.client.0.smithi156.stdout:9/989: write d2/d5/d16/d57/d127/f129 [290176,31607] 0 2024-07-16T09:55:49.145 INFO:tasks.workunit.client.0.smithi156.stdout:8/826: rename d0/d11/d62/cac to d0/d74/dd3/c122 0 2024-07-16T09:55:49.146 INFO:tasks.workunit.client.1.smithi176.stdout:6/940: fdatasync d1/d5/f6 0 2024-07-16T09:55:49.147 INFO:tasks.workunit.client.0.smithi156.stdout:8/827: read d0/d11/d62/dc2/dcd/fec [7082894,70492] 0 2024-07-16T09:55:49.147 INFO:tasks.workunit.client.0.smithi156.stdout:9/990: creat d2/d5/d16/d57/da3/ddd/f137 x:0 0 0 2024-07-16T09:55:49.148 INFO:tasks.workunit.client.0.smithi156.stdout:7/708: sync 2024-07-16T09:55:49.153 INFO:tasks.workunit.client.1.smithi176.stdout:2/986: dwrite dd/d2e/d35/d59/f13b [0,4194304] 0 2024-07-16T09:55:49.153 INFO:tasks.workunit.client.0.smithi156.stdout:4/958: write d4/d23/f17 [3126297,115260] 0 2024-07-16T09:55:49.155 INFO:tasks.workunit.client.1.smithi176.stdout:2/987: readlink dd/d2e/d35/d59/d66/l11f 0 2024-07-16T09:55:49.155 INFO:tasks.workunit.client.0.smithi156.stdout:7/709: mknod dc/d3e/d3f/d51/cf4 0 2024-07-16T09:55:49.156 INFO:tasks.workunit.client.1.smithi176.stdout:2/988: chown dd/d49/d52/f114 1081475119 1 2024-07-16T09:55:49.156 INFO:tasks.workunit.client.1.smithi176.stdout:6/941: dwrite d1/d5d/db9/d101/f132 [0,4194304] 0 2024-07-16T09:55:49.162 INFO:tasks.workunit.client.1.smithi176.stdout:6/942: write d1/d5/d7/d1e/f66 [209687,20534] 0 2024-07-16T09:55:49.164 INFO:tasks.workunit.client.0.smithi156.stdout:4/959: getdents d4/db/dc/d5f 0 2024-07-16T09:55:49.166 INFO:tasks.workunit.client.0.smithi156.stdout:7/710: read dc/d3e/d76/fc5 [866337,49930] 0 2024-07-16T09:55:49.166 INFO:tasks.workunit.client.1.smithi176.stdout:8/828: dread d8/d14/d1e/f60 [4194304,4194304] 0 2024-07-16T09:55:49.166 INFO:tasks.workunit.client.1.smithi176.stdout:2/989: dwrite dd/d2e/d35/d59/f13b [0,4194304] 0 2024-07-16T09:55:49.172 INFO:tasks.workunit.client.1.smithi176.stdout:6/943: mkdir d1/dc/d5b/d153 0 2024-07-16T09:55:49.174 INFO:tasks.workunit.client.0.smithi156.stdout:7/711: dwrite dc/d63/f88 [0,4194304] 0 2024-07-16T09:55:49.174 INFO:tasks.workunit.client.1.smithi176.stdout:2/990: unlink dd/d2e/d35/d4a/df9/f11b 0 2024-07-16T09:55:49.174 INFO:tasks.workunit.client.1.smithi176.stdout:8/829: getdents d8/df7 0 2024-07-16T09:55:49.175 INFO:tasks.workunit.client.1.smithi176.stdout:6/944: mknod d1/d5/d47/d75/c154 0 2024-07-16T09:55:49.176 INFO:tasks.workunit.client.1.smithi176.stdout:2/991: mknod dd/dcd/c14b 0 2024-07-16T09:55:49.177 INFO:tasks.workunit.client.0.smithi156.stdout:7/712: write dc/d13/d15/d8d/dd0/fd5 [119514,11157] 0 2024-07-16T09:55:49.180 INFO:tasks.workunit.client.1.smithi176.stdout:2/992: symlink dd/d36/ded/l14c 0 2024-07-16T09:55:49.181 INFO:tasks.workunit.client.0.smithi156.stdout:4/960: rename d4/db/dc/dcc/d58/d8f/lab to d4/db/dc/dcc/d150/l152 0 2024-07-16T09:55:49.187 INFO:tasks.workunit.client.1.smithi176.stdout:2/993: rename dd/d2e/d35/f11a to dd/d2e/d35/d4a/df9/f14d 0 2024-07-16T09:55:49.187 INFO:tasks.workunit.client.0.smithi156.stdout:3/871: truncate d8/d14/fb7 3106176 0 2024-07-16T09:55:49.187 INFO:tasks.workunit.client.1.smithi176.stdout:8/830: dwrite d8/d23/d100/fdc [0,4194304] 0 2024-07-16T09:55:49.188 INFO:tasks.workunit.client.0.smithi156.stdout:7/713: creat dc/d3e/d3f/ff5 x:0 0 0 2024-07-16T09:55:49.189 INFO:tasks.workunit.client.1.smithi176.stdout:4/794: write da/dc/d4b/f52 [1355374,54086] 0 2024-07-16T09:55:49.189 INFO:tasks.workunit.client.0.smithi156.stdout:2/967: write d0/d5/d8b/f97 [418164,57092] 0 2024-07-16T09:55:49.190 INFO:tasks.workunit.client.0.smithi156.stdout:7/714: stat dc/d13/d20/f3c 0 2024-07-16T09:55:49.195 INFO:tasks.workunit.client.1.smithi176.stdout:3/669: getdents d3/da/d33 0 2024-07-16T09:55:49.196 INFO:tasks.workunit.client.0.smithi156.stdout:4/961: dwrite d4/db/f18 [4194304,4194304] 0 2024-07-16T09:55:49.196 INFO:tasks.workunit.client.1.smithi176.stdout:1/571: dwrite d4/d8/d12/d21/d25/f86 [0,4194304] 0 2024-07-16T09:55:49.197 INFO:tasks.workunit.client.0.smithi156.stdout:8/828: getdents d0/d11/d62 0 2024-07-16T09:55:49.200 INFO:tasks.workunit.client.1.smithi176.stdout:8/831: creat d8/d14/d1e/d33/df6/f117 x:0 0 0 2024-07-16T09:55:49.201 INFO:tasks.workunit.client.1.smithi176.stdout:5/884: truncate d4/d10/fb9 1907032 0 2024-07-16T09:55:49.202 INFO:tasks.workunit.client.0.smithi156.stdout:6/779: dwrite d3/d13/d17/d4f/d94/d19/d31/d35/f91 [0,4194304] 0 2024-07-16T09:55:49.202 INFO:tasks.workunit.client.0.smithi156.stdout:4/962: truncate d4/db/d8a/d137/f14a 165326 0 2024-07-16T09:55:49.203 INFO:tasks.workunit.client.1.smithi176.stdout:3/670: stat d3/da/c3d 0 2024-07-16T09:55:49.205 INFO:tasks.workunit.client.1.smithi176.stdout:2/994: creat dd/d2e/d35/d4a/f14e x:0 0 0 2024-07-16T09:55:49.207 INFO:tasks.workunit.client.0.smithi156.stdout:8/829: creat d0/d11/d13/d113/f123 x:0 0 0 2024-07-16T09:55:49.207 INFO:tasks.workunit.client.0.smithi156.stdout:7/715: mknod dc/d3e/d3f/cf6 0 2024-07-16T09:55:49.209 INFO:tasks.workunit.client.0.smithi156.stdout:9/991: dwrite d2/d5/d16/d57/d63/dd3/ddb/d74/f9e [0,4194304] 0 2024-07-16T09:55:49.212 INFO:tasks.workunit.client.0.smithi156.stdout:4/963: write d4/d23/ff7 [1376456,12364] 0 2024-07-16T09:55:49.213 INFO:tasks.workunit.client.1.smithi176.stdout:2/995: write dd/de/fa7 [1345455,91356] 0 2024-07-16T09:55:49.213 INFO:tasks.workunit.client.1.smithi176.stdout:1/572: creat d4/d8/d12/d21/d25/d30/fbe x:0 0 0 2024-07-16T09:55:49.213 INFO:tasks.workunit.client.1.smithi176.stdout:5/885: write d4/d10/f33 [4666388,25250] 0 2024-07-16T09:55:49.214 INFO:tasks.workunit.client.1.smithi176.stdout:0/946: dwrite d5/d1c/d24/d35/d50/d58/d63/f66 [0,4194304] 0 2024-07-16T09:55:49.215 INFO:tasks.workunit.client.1.smithi176.stdout:8/832: dread d8/d14/d1e/d33/d9d/fb8 [0,4194304] 0 2024-07-16T09:55:49.216 INFO:tasks.workunit.client.0.smithi156.stdout:7/716: chown dc/d3e/d3f/d51/fb0 989 1 2024-07-16T09:55:49.216 INFO:tasks.workunit.client.0.smithi156.stdout:2/968: getdents d0/d13/d17/d1c 0 2024-07-16T09:55:49.217 INFO:tasks.workunit.client.0.smithi156.stdout:6/780: chown d3/d13/d17/d4f/d94/d19/d2e/f3e 507358627 1 2024-07-16T09:55:49.217 INFO:tasks.workunit.client.0.smithi156.stdout:9/992: read - d2/d5/d16/de2/d121/d7f/d114/f132 zero size 2024-07-16T09:55:49.218 INFO:tasks.workunit.client.1.smithi176.stdout:6/945: dread d1/d5/d7/d54/f70 [0,4194304] 0 2024-07-16T09:55:49.218 INFO:tasks.workunit.client.1.smithi176.stdout:4/795: dwrite da/dc/d4b/df0/d96/d69/fd1 [0,4194304] 0 2024-07-16T09:55:49.219 INFO:tasks.workunit.client.1.smithi176.stdout:2/996: chown dd/d49/d52/d7c 0 1 2024-07-16T09:55:49.220 INFO:tasks.workunit.client.0.smithi156.stdout:4/964: creat d4/db/dc/d7c/d146/f153 x:0 0 0 2024-07-16T09:55:49.221 INFO:tasks.workunit.client.0.smithi156.stdout:8/830: dread d0/d11/d20/d7f/db7/d110/f1e [0,4194304] 0 2024-07-16T09:55:49.222 INFO:tasks.workunit.client.0.smithi156.stdout:7/717: creat dc/d13/ff7 x:0 0 0 2024-07-16T09:55:49.223 INFO:tasks.workunit.client.1.smithi176.stdout:5/886: read d4/fe [1327797,26412] 0 2024-07-16T09:55:49.226 INFO:tasks.workunit.client.1.smithi176.stdout:4/796: symlink da/dc/d4b/df0/d96/d9b/l102 0 2024-07-16T09:55:49.227 INFO:tasks.workunit.client.0.smithi156.stdout:6/781: symlink d3/d13/d17/d4f/d30/d45/d99/lfe 0 2024-07-16T09:55:49.229 INFO:tasks.workunit.client.1.smithi176.stdout:3/671: dread d3/d17/d83/f91 [0,4194304] 0 2024-07-16T09:55:49.230 INFO:tasks.workunit.client.1.smithi176.stdout:0/947: mknod d5/d1c/d24/d35/d50/d9b/de3/d117/dbd/de6/c138 0 2024-07-16T09:55:49.232 INFO:tasks.workunit.client.1.smithi176.stdout:2/997: mkdir dd/d2e/d35/d59/d14f 0 2024-07-16T09:55:49.234 INFO:tasks.workunit.client.1.smithi176.stdout:5/887: creat d4/da9/dc4/deb/d122/f143 x:0 0 0 2024-07-16T09:55:49.234 INFO:tasks.workunit.client.1.smithi176.stdout:3/672: read d3/da/d33/f53 [406172,28416] 0 2024-07-16T09:55:49.235 INFO:tasks.workunit.client.0.smithi156.stdout:9/993: mknod d2/d5/d16/de2/d121/c138 0 2024-07-16T09:55:49.236 INFO:tasks.workunit.client.0.smithi156.stdout:7/718: creat dc/d48/ff8 x:0 0 0 2024-07-16T09:55:49.237 INFO:tasks.workunit.client.1.smithi176.stdout:3/673: fdatasync d3/da/d33/f53 0 2024-07-16T09:55:49.241 INFO:tasks.workunit.client.1.smithi176.stdout:2/998: mknod dd/d2e/d35/d59/d66/c150 0 2024-07-16T09:55:49.243 INFO:tasks.workunit.client.1.smithi176.stdout:2/999: truncate dd/d2e/d35/d4a/f14e 922025 0 2024-07-16T09:55:49.246 INFO:tasks.workunit.client.1.smithi176.stdout:1/573: getdents d4/d8 0 2024-07-16T09:55:49.246 INFO:tasks.workunit.client.1.smithi176.stdout:6/946: getdents d1/d5/d47/d62/d80/d13d 0 2024-07-16T09:55:49.247 INFO:tasks.workunit.client.1.smithi176.stdout:1/574: fdatasync d4/d77/dad/fbd 0 2024-07-16T09:55:49.249 INFO:tasks.workunit.client.1.smithi176.stdout:6/947: fdatasync d1/d5d/db9/fe3 0 2024-07-16T09:55:49.251 INFO:tasks.workunit.client.1.smithi176.stdout:1/575: creat d4/d8/d12/d4c/da4/fbf x:0 0 0 2024-07-16T09:55:49.251 INFO:tasks.workunit.client.1.smithi176.stdout:4/797: dread da/fca [0,4194304] 0 2024-07-16T09:55:49.253 INFO:tasks.workunit.client.1.smithi176.stdout:1/576: write d4/d8/d12/f4b [1066635,102064] 0 2024-07-16T09:55:49.254 INFO:tasks.workunit.client.1.smithi176.stdout:5/888: dread d4/d14/d3d/f41 [0,4194304] 0 2024-07-16T09:55:49.255 INFO:tasks.workunit.client.0.smithi156.stdout:8/831: rmdir d0/d11/d20/d3f/dbe 39 2024-07-16T09:55:49.255 INFO:tasks.workunit.client.1.smithi176.stdout:1/577: chown d4/d8/d12/d4c 1307487466 1 2024-07-16T09:55:49.256 INFO:tasks.workunit.client.0.smithi156.stdout:9/994: chown d2/d5/ld 764 1 2024-07-16T09:55:49.259 INFO:tasks.workunit.client.0.smithi156.stdout:6/782: mkdir d3/d13/d17/d4f/d94/d19/de5/dff 0 2024-07-16T09:55:49.259 INFO:tasks.workunit.client.1.smithi176.stdout:1/578: unlink d4/db/fac 0 2024-07-16T09:55:49.260 INFO:tasks.workunit.client.0.smithi156.stdout:6/783: fdatasync d3/d13/dee/ff0 0 2024-07-16T09:55:49.261 INFO:tasks.workunit.client.1.smithi176.stdout:4/798: dwrite da/dc/d4b/df0/d96/d69/f7e [0,4194304] 0 2024-07-16T09:55:49.262 INFO:tasks.workunit.client.0.smithi156.stdout:4/965: link d4/db/fc9 d4/db/dc/dcc/dec/ded/f154 0 2024-07-16T09:55:49.264 INFO:tasks.workunit.client.0.smithi156.stdout:6/784: creat d3/d13/d17/d4f/d94/dec/f100 x:0 0 0 2024-07-16T09:55:49.268 INFO:tasks.workunit.client.1.smithi176.stdout:5/889: dwrite d4/d37/d72/d38/d39/f5d [0,4194304] 0 2024-07-16T09:55:49.268 INFO:tasks.workunit.client.0.smithi156.stdout:6/785: write d3/d13/d17/d56/ff7 [261028,82444] 0 2024-07-16T09:55:49.269 INFO:tasks.workunit.client.1.smithi176.stdout:8/833: dread d8/d14/f27 [0,4194304] 0 2024-07-16T09:55:49.271 INFO:tasks.workunit.client.0.smithi156.stdout:8/832: dread d0/d11/d13/d1c/f32 [0,4194304] 0 2024-07-16T09:55:49.272 INFO:tasks.workunit.client.1.smithi176.stdout:7/752: sync 2024-07-16T09:55:49.272 INFO:tasks.workunit.client.1.smithi176.stdout:8/834: write d8/d14/d1e/d33/f105 [611976,49811] 0 2024-07-16T09:55:49.272 INFO:tasks.workunit.client.0.smithi156.stdout:4/966: truncate d4/d23/d30/dd2/feb 870995 0 2024-07-16T09:55:49.275 INFO:tasks.workunit.client.1.smithi176.stdout:8/835: readlink d8/d23/d100/l3a 0 2024-07-16T09:55:49.276 INFO:tasks.workunit.client.0.smithi156.stdout:9/995: dwrite d2/d5/d16/d57/d63/dd3/ddb/d74/dc2/dc4/d23/fbc [0,4194304] 0 2024-07-16T09:55:49.279 INFO:tasks.workunit.client.0.smithi156.stdout:8/833: chown d0/d11/d20/d3f/dd4/f11f 71581 1 2024-07-16T09:55:49.281 INFO:tasks.workunit.client.0.smithi156.stdout:9/996: fdatasync d2/d5/d16/d57/d63/dd3/ddb/d74/dc2/dc4/fc8 0 2024-07-16T09:55:49.287 INFO:tasks.workunit.client.0.smithi156.stdout:6/786: dwrite d3/d13/d17/d4f/d1e/ffd [0,4194304] 0 2024-07-16T09:55:49.287 INFO:tasks.workunit.client.0.smithi156.stdout:4/967: rename d4/db/dc/dcc/dec/ded/d87/df2/l12f to d4/db/dc/dcc/dec/ded/d87/l155 0 2024-07-16T09:55:49.291 INFO:tasks.workunit.client.0.smithi156.stdout:6/787: read d3/d13/d17/d4f/d94/d19/d31/d35/f91 [3209427,41248] 0 2024-07-16T09:55:49.292 INFO:tasks.workunit.client.1.smithi176.stdout:6/948: dread d1/d5/d7/f63 [0,4194304] 0 2024-07-16T09:55:49.294 INFO:tasks.workunit.client.0.smithi156.stdout:9/997: mkdir d2/d5/d16/d57/d63/dd3/ddb/d3d/da6/d139 0 2024-07-16T09:55:49.295 INFO:tasks.workunit.client.0.smithi156.stdout:4/968: creat d4/db/dc/d5f/f156 x:0 0 0 2024-07-16T09:55:49.297 INFO:tasks.workunit.client.0.smithi156.stdout:6/788: write d3/d13/d17/d4f/fe9 [258475,88958] 0 2024-07-16T09:55:49.301 INFO:tasks.workunit.client.0.smithi156.stdout:4/969: truncate d4/d23/d30/d33/f3a 3115860 0 2024-07-16T09:55:49.302 INFO:tasks.workunit.client.0.smithi156.stdout:9/998: truncate d2/d5/d16/d57/d63/dd3/ddb/d74/dc2/dc4/f80 979658 0 2024-07-16T09:55:49.304 INFO:tasks.workunit.client.0.smithi156.stdout:4/970: read d4/db/d47/dbe/f105 [1665708,53581] 0 2024-07-16T09:55:49.307 INFO:tasks.workunit.client.0.smithi156.stdout:9/999: creat d2/d5/d16/d22/f13a x:0 0 0 2024-07-16T09:55:49.308 INFO:tasks.workunit.client.0.smithi156.stdout:4/971: unlink d4/d23/c84 0 2024-07-16T09:55:49.309 INFO:tasks.workunit.client.0.smithi156.stdout:4/972: fsync d4/db/f3f 0 2024-07-16T09:55:49.311 INFO:tasks.workunit.client.0.smithi156.stdout:4/973: mknod d4/db/dc/d5f/c157 0 2024-07-16T09:55:49.313 INFO:tasks.workunit.client.0.smithi156.stdout:4/974: creat d4/d23/d52/d71/def/f158 x:0 0 0 2024-07-16T09:55:49.322 INFO:tasks.workunit.client.0.smithi156.stdout:8/834: dread d0/d11/d20/d7f/db7/d110/fa7 [0,4194304] 0 2024-07-16T09:55:49.328 INFO:tasks.workunit.client.1.smithi176.stdout:8/836: write d8/d14/d1e/d33/d9d/fb8 [1140329,90758] 0 2024-07-16T09:55:49.331 INFO:tasks.workunit.client.1.smithi176.stdout:9/931: sync 2024-07-16T09:55:49.332 INFO:tasks.workunit.client.1.smithi176.stdout:9/932: fdatasync d5/d1d/db3/fd1 0 2024-07-16T09:55:49.332 INFO:tasks.workunit.client.0.smithi156.stdout:8/835: dwrite d0/d11/d13/d1c/d24/fb9 [0,4194304] 0 2024-07-16T09:55:49.338 INFO:tasks.workunit.client.0.smithi156.stdout:2/969: truncate d0/fee 795134 0 2024-07-16T09:55:49.340 INFO:tasks.workunit.client.0.smithi156.stdout:2/970: chown d0/c20 417405 1 2024-07-16T09:55:49.341 INFO:tasks.workunit.client.0.smithi156.stdout:0/903: sync 2024-07-16T09:55:49.344 INFO:tasks.workunit.client.1.smithi176.stdout:1/579: truncate d4/d8/d12/d21/d25/d30/f61 191258 0 2024-07-16T09:55:49.345 INFO:tasks.workunit.client.0.smithi156.stdout:8/836: dwrite d0/d74/dd3/f116 [0,4194304] 0 2024-07-16T09:55:49.346 INFO:tasks.workunit.client.1.smithi176.stdout:5/890: fsync d4/d37/dbb/fd3 0 2024-07-16T09:55:49.347 INFO:tasks.workunit.client.0.smithi156.stdout:6/789: dread d3/d13/d17/d4f/faf [0,4194304] 0 2024-07-16T09:55:49.348 INFO:tasks.workunit.client.0.smithi156.stdout:2/971: mknod d0/d129/c138 0 2024-07-16T09:55:49.353 INFO:tasks.workunit.client.0.smithi156.stdout:6/790: truncate d3/d13/d17/d56/ff7 434300 0 2024-07-16T09:55:49.353 INFO:tasks.workunit.client.1.smithi176.stdout:8/837: symlink d8/d23/d100/db9/dcb/d34/d62/d88/l118 0 2024-07-16T09:55:49.356 INFO:tasks.workunit.client.0.smithi156.stdout:8/837: dread d0/d11/d20/d7f/db7/d110/d1f/d57/f8f [0,4194304] 0 2024-07-16T09:55:49.356 INFO:tasks.workunit.client.0.smithi156.stdout:2/972: creat d0/d129/d130/f139 x:0 0 0 2024-07-16T09:55:49.357 INFO:tasks.workunit.client.1.smithi176.stdout:0/948: dwrite d5/d1c/d24/d35/d50/d9b/de3/d117/d131/fdd [0,4194304] 0 2024-07-16T09:55:49.358 INFO:tasks.workunit.client.1.smithi176.stdout:3/674: write d3/d17/d1d/d4f/d59/da0/f3f [1300026,29444] 0 2024-07-16T09:55:49.361 INFO:tasks.workunit.client.1.smithi176.stdout:4/799: creat da/d3d/f103 x:0 0 0 2024-07-16T09:55:49.361 INFO:tasks.workunit.client.1.smithi176.stdout:3/675: truncate d3/d17/d1d/d6e/fab 977289 0 2024-07-16T09:55:49.361 INFO:tasks.workunit.client.0.smithi156.stdout:0/904: dwrite d0/d22/d41/d52/ddb/f11d [0,4194304] 0 2024-07-16T09:55:49.363 INFO:tasks.workunit.client.1.smithi176.stdout:5/891: mkdir d4/d37/d72/d38/d102/dff/dc7/d144 0 2024-07-16T09:55:49.364 INFO:tasks.workunit.client.0.smithi156.stdout:7/719: dwrite dc/d13/d20/f3c [0,4194304] 0 2024-07-16T09:55:49.364 INFO:tasks.workunit.client.0.smithi156.stdout:6/791: dread d3/d13/d17/f55 [4194304,4194304] 0 2024-07-16T09:55:49.364 INFO:tasks.workunit.client.1.smithi176.stdout:5/892: dread - d4/d10/f13c zero size 2024-07-16T09:55:49.367 INFO:tasks.workunit.client.0.smithi156.stdout:8/838: write d0/d11/d20/d7f/db7/d110/d1f/d57/db3/f10f [63863,119370] 0 2024-07-16T09:55:49.368 INFO:tasks.workunit.client.0.smithi156.stdout:8/839: fsync d0/d11/d4a/fd9 0 2024-07-16T09:55:49.370 INFO:tasks.workunit.client.1.smithi176.stdout:9/933: mkdir d5/d1d/db3/d11f/d131/d13e 0 2024-07-16T09:55:49.371 INFO:tasks.workunit.client.0.smithi156.stdout:7/720: dread - dc/d3e/d3f/d51/d8a/fdd zero size 2024-07-16T09:55:49.371 INFO:tasks.workunit.client.0.smithi156.stdout:2/973: truncate d0/d13/d17/f42 4443804 0 2024-07-16T09:55:49.371 INFO:tasks.workunit.client.1.smithi176.stdout:0/949: dread d5/d1c/d5a/f75 [0,4194304] 0 2024-07-16T09:55:49.375 INFO:tasks.workunit.client.0.smithi156.stdout:0/905: creat d0/d1a/d4d/d9/d68/f122 x:0 0 0 2024-07-16T09:55:49.381 INFO:tasks.workunit.client.0.smithi156.stdout:0/906: mknod d0/d1a/d4d/d9/d68/dcb/dd7/d100/c123 0 2024-07-16T09:55:49.381 INFO:tasks.workunit.client.0.smithi156.stdout:4/975: write d4/db/dc/f55 [4308350,11984] 0 2024-07-16T09:55:49.382 INFO:tasks.workunit.client.1.smithi176.stdout:3/676: creat d3/da/d6c/dbc/fcb x:0 0 0 2024-07-16T09:55:49.384 INFO:tasks.workunit.client.1.smithi176.stdout:9/934: write d5/d52/fa4 [3458836,85265] 0 2024-07-16T09:55:49.384 INFO:tasks.workunit.client.0.smithi156.stdout:7/721: symlink dc/d3e/d3f/d51/de3/db1/lf9 0 2024-07-16T09:55:49.387 INFO:tasks.workunit.client.1.smithi176.stdout:9/935: chown d5/d1d/db3/ca9 197 1 2024-07-16T09:55:49.388 INFO:tasks.workunit.client.1.smithi176.stdout:1/580: dread d4/f31 [0,4194304] 0 2024-07-16T09:55:49.389 INFO:tasks.workunit.client.0.smithi156.stdout:0/907: rename d0/d22/d41/da9/ff1 to d0/d1a/d4d/d61/d97/de1/de7/f124 0 2024-07-16T09:55:49.389 INFO:tasks.workunit.client.1.smithi176.stdout:9/936: truncate d5/d52/f136 118819 0 2024-07-16T09:55:49.391 INFO:tasks.workunit.client.1.smithi176.stdout:1/581: write d4/d8/d12/d21/d25/da8/f9d [1020571,93346] 0 2024-07-16T09:55:49.393 INFO:tasks.workunit.client.0.smithi156.stdout:7/722: mknod dc/d13/d15/cfa 0 2024-07-16T09:55:49.395 INFO:tasks.workunit.client.0.smithi156.stdout:0/908: mknod d0/d1a/d76/dbc/d10a/c125 0 2024-07-16T09:55:49.397 INFO:tasks.workunit.client.1.smithi176.stdout:3/677: creat d3/d17/d1d/d6e/fcc x:0 0 0 2024-07-16T09:55:49.397 INFO:tasks.workunit.client.1.smithi176.stdout:3/678: fdatasync d3/da/d7b/fb8 0 2024-07-16T09:55:49.399 INFO:tasks.workunit.client.1.smithi176.stdout:9/937: rmdir d5/d52 39 2024-07-16T09:55:49.399 INFO:tasks.workunit.client.0.smithi156.stdout:7/723: creat dc/d3e/d3f/d51/de3/d81/ffb x:0 0 0 2024-07-16T09:55:49.400 INFO:tasks.workunit.client.0.smithi156.stdout:2/974: dread d0/d5/d29/d2d/f40 [0,4194304] 0 2024-07-16T09:55:49.404 INFO:tasks.workunit.client.0.smithi156.stdout:0/909: creat d0/d1a/d4d/d61/f126 x:0 0 0 2024-07-16T09:55:49.404 INFO:tasks.workunit.client.1.smithi176.stdout:3/679: chown d3/da/d7b/l8b 607113 1 2024-07-16T09:55:49.406 INFO:tasks.workunit.client.1.smithi176.stdout:9/938: chown d5/d1d/d67/d6d/cf8 5 1 2024-07-16T09:55:49.407 INFO:tasks.workunit.client.1.smithi176.stdout:9/939: dread - d5/de/da5/dc1/f134 zero size 2024-07-16T09:55:49.408 INFO:tasks.workunit.client.0.smithi156.stdout:6/792: dwrite d3/d13/d17/d4f/d4d/f57 [0,4194304] 0 2024-07-16T09:55:49.408 INFO:tasks.workunit.client.1.smithi176.stdout:1/582: dread d4/f17 [0,4194304] 0 2024-07-16T09:55:49.409 INFO:tasks.workunit.client.1.smithi176.stdout:1/583: chown d4/db/l6e 2719479 1 2024-07-16T09:55:49.411 INFO:tasks.workunit.client.1.smithi176.stdout:9/940: write d5/d52/ff1 [606078,115631] 0 2024-07-16T09:55:49.415 INFO:tasks.workunit.client.0.smithi156.stdout:4/976: dwrite d4/db/dc/dcc/dec/ded/d87/d108/f138 [0,4194304] 0 2024-07-16T09:55:49.416 INFO:tasks.workunit.client.0.smithi156.stdout:0/910: creat d0/d1a/d4d/d61/d97/de1/de7/f127 x:0 0 0 2024-07-16T09:55:49.416 INFO:tasks.workunit.client.1.smithi176.stdout:5/893: dread d4/d37/d72/f59 [0,4194304] 0 2024-07-16T09:55:49.420 INFO:tasks.workunit.client.1.smithi176.stdout:9/941: dwrite d5/d1d/d20/f25 [0,4194304] 0 2024-07-16T09:55:49.422 INFO:tasks.workunit.client.1.smithi176.stdout:9/942: chown d5/de/da5/dc1/d112/d98/ced 50862601 1 2024-07-16T09:55:49.424 INFO:tasks.workunit.client.0.smithi156.stdout:8/840: dread d0/d11/d20/d7f/db7/d110/f1a [0,4194304] 0 2024-07-16T09:55:49.425 INFO:tasks.workunit.client.0.smithi156.stdout:2/975: mkdir d0/d9b/d10e/d13a 0 2024-07-16T09:55:49.426 INFO:tasks.workunit.client.1.smithi176.stdout:9/943: truncate d5/d1d/d20/fc5 880171 0 2024-07-16T09:55:49.428 INFO:tasks.workunit.client.1.smithi176.stdout:9/944: chown d5/d1d/db3/fd1 7 1 2024-07-16T09:55:49.428 INFO:tasks.workunit.client.0.smithi156.stdout:2/976: dread - d0/d13/d17/d1c/d51/dc6/fd4 zero size 2024-07-16T09:55:49.429 INFO:tasks.workunit.client.0.smithi156.stdout:7/724: rename dc/d3e/d3f/d51/de3 to dc/d13/d15/dfc 0 2024-07-16T09:55:49.431 INFO:tasks.workunit.client.1.smithi176.stdout:9/945: creat d5/de/da5/dc1/d112/d97/f13f x:0 0 0 2024-07-16T09:55:49.431 INFO:tasks.workunit.client.0.smithi156.stdout:4/977: symlink d4/db/dc/dcc/dec/ded/l159 0 2024-07-16T09:55:49.431 INFO:tasks.workunit.client.1.smithi176.stdout:9/946: stat d5/de/d21/f84 0 2024-07-16T09:55:49.434 INFO:tasks.workunit.client.0.smithi156.stdout:7/725: truncate dc/d13/d15/d8b/fbe 1016201 0 2024-07-16T09:55:49.435 INFO:tasks.workunit.client.1.smithi176.stdout:9/947: write d5/de/da5/dc1/d112/d97/f13f [1019893,127230] 0 2024-07-16T09:55:49.436 INFO:tasks.workunit.client.0.smithi156.stdout:1/997: sync 2024-07-16T09:55:49.436 INFO:tasks.workunit.client.0.smithi156.stdout:4/978: symlink d4/d6a/l15a 0 2024-07-16T09:55:49.436 INFO:tasks.workunit.client.0.smithi156.stdout:0/911: dread d0/d1a/d4d/d9/d27/d3d/d62/f63 [0,4194304] 0 2024-07-16T09:55:49.438 INFO:tasks.workunit.client.0.smithi156.stdout:1/998: fsync dc/d14/d3e/d55/d83/fb2 0 2024-07-16T09:55:49.439 INFO:tasks.workunit.client.0.smithi156.stdout:2/977: unlink d0/d5/d76/da1/fd6 0 2024-07-16T09:55:49.439 INFO:tasks.workunit.client.0.smithi156.stdout:8/841: rename d0/d11/d20/d7f/db7/d110/d1f/d57/da6 to d0/d11/d2e/d124 0 2024-07-16T09:55:49.440 INFO:tasks.workunit.client.1.smithi176.stdout:9/948: mkdir d5/d1d/d20/d78/df2/d118/d140 0 2024-07-16T09:55:49.445 INFO:tasks.workunit.client.0.smithi156.stdout:0/912: symlink d0/d22/d41/d66/l128 0 2024-07-16T09:55:49.445 INFO:tasks.workunit.client.1.smithi176.stdout:5/894: dread d4/d10/d17/f50 [0,4194304] 0 2024-07-16T09:55:49.446 INFO:tasks.workunit.client.0.smithi156.stdout:1/999: rmdir dc/d6b/d96/d11a 39 2024-07-16T09:55:49.447 INFO:tasks.workunit.client.0.smithi156.stdout:8/842: rmdir d0/d74 39 2024-07-16T09:55:49.448 INFO:tasks.workunit.client.0.smithi156.stdout:2/978: creat d0/d13/de8/dfc/f13b x:0 0 0 2024-07-16T09:55:49.448 INFO:tasks.workunit.client.0.smithi156.stdout:4/979: dwrite d4/d23/d30/f123 [0,4194304] 0 2024-07-16T09:55:49.449 INFO:tasks.workunit.client.1.smithi176.stdout:9/949: symlink d5/d43/l141 0 2024-07-16T09:55:49.450 INFO:tasks.workunit.client.1.smithi176.stdout:1/584: dread d4/f57 [0,4194304] 0 2024-07-16T09:55:49.454 INFO:tasks.workunit.client.0.smithi156.stdout:4/980: chown d4/db/dc/dcc/dec/ded/d87/f114 20799255 1 2024-07-16T09:55:49.454 INFO:tasks.workunit.client.0.smithi156.stdout:4/981: chown d4/db/dc/d7c/l6d 5 1 2024-07-16T09:55:49.456 INFO:tasks.workunit.client.0.smithi156.stdout:0/913: symlink d0/d1a/d4d/d9/d27/d50/d11b/d11c/l129 0 2024-07-16T09:55:49.457 INFO:tasks.workunit.client.0.smithi156.stdout:8/843: creat d0/d11/d20/d7f/db7/d110/d1f/dee/f125 x:0 0 0 2024-07-16T09:55:49.458 INFO:tasks.workunit.client.1.smithi176.stdout:9/950: creat d5/d1d/db3/d11f/f142 x:0 0 0 2024-07-16T09:55:49.458 INFO:tasks.workunit.client.0.smithi156.stdout:4/982: dread - d4/db/fc9 zero size 2024-07-16T09:55:49.460 INFO:tasks.workunit.client.0.smithi156.stdout:4/983: fsync d4/d23/ff7 0 2024-07-16T09:55:49.461 INFO:tasks.workunit.client.0.smithi156.stdout:8/844: rename d0/d11/d2e/f5c to d0/d11/d20/d3f/dd4/f126 0 2024-07-16T09:55:49.463 INFO:tasks.workunit.client.0.smithi156.stdout:8/845: read d0/d11/d20/d3f/fb1 [723340,37578] 0 2024-07-16T09:55:49.464 INFO:tasks.workunit.client.1.smithi176.stdout:9/951: mknod d5/d1d/d20/d78/df2/d118/c143 0 2024-07-16T09:55:49.465 INFO:tasks.workunit.client.0.smithi156.stdout:8/846: write d0/d11/d62/fa4 [4924508,49483] 0 2024-07-16T09:55:49.467 INFO:tasks.workunit.client.0.smithi156.stdout:4/984: readlink d4/db/dc/lae 0 2024-07-16T09:55:49.471 INFO:tasks.workunit.client.0.smithi156.stdout:4/985: rmdir d4/db/d47/dbe 39 2024-07-16T09:55:49.471 INFO:tasks.workunit.client.0.smithi156.stdout:6/793: dread d3/d13/d17/d4f/d4d/d3b/d5a/dc1/dce/fda [0,4194304] 0 2024-07-16T09:55:49.472 INFO:tasks.workunit.client.1.smithi176.stdout:9/952: dwrite d5/de/d30/f9a [4194304,4194304] 0 2024-07-16T09:55:49.473 INFO:tasks.workunit.client.0.smithi156.stdout:8/847: fdatasync d0/d11/d13/fa1 0 2024-07-16T09:55:49.475 INFO:tasks.workunit.client.0.smithi156.stdout:0/914: dread d0/d1a/d76/da7/dd3/fd5 [0,4194304] 0 2024-07-16T09:55:49.476 INFO:tasks.workunit.client.1.smithi176.stdout:7/753: write d4/dd/d26/dc2/d17/d1d/d8b/f9d [1074025,119014] 0 2024-07-16T09:55:49.477 INFO:tasks.workunit.client.1.smithi176.stdout:9/953: creat d5/d1d/d20/d78/df2/d118/d11c/f144 x:0 0 0 2024-07-16T09:55:49.477 INFO:tasks.workunit.client.0.smithi156.stdout:4/986: creat d4/d23/d52/d102/f15b x:0 0 0 2024-07-16T09:55:49.477 INFO:tasks.workunit.client.0.smithi156.stdout:7/726: write dc/d13/d36/f6a [2505579,79817] 0 2024-07-16T09:55:49.478 INFO:tasks.workunit.client.1.smithi176.stdout:6/949: write d1/d5/d47/d62/fc6 [758873,81562] 0 2024-07-16T09:55:49.479 INFO:tasks.workunit.client.0.smithi156.stdout:2/979: write d0/d5/d29/f3f [2400826,18400] 0 2024-07-16T09:55:49.479 INFO:tasks.workunit.client.1.smithi176.stdout:7/754: chown d4/dd/d26/dc2/d17/cb9 3 1 2024-07-16T09:55:49.480 INFO:tasks.workunit.client.0.smithi156.stdout:6/794: readlink d3/d13/d17/d4f/d94/l33 0 2024-07-16T09:55:49.485 INFO:tasks.workunit.client.1.smithi176.stdout:7/755: unlink d4/dd/d63/f85 0 2024-07-16T09:55:49.485 INFO:tasks.workunit.client.0.smithi156.stdout:8/848: mkdir d0/d11/d2e/d127 0 2024-07-16T09:55:49.491 INFO:tasks.workunit.client.0.smithi156.stdout:4/987: creat d4/db/dc/dcc/dec/ded/db1/f15c x:0 0 0 2024-07-16T09:55:49.491 INFO:tasks.workunit.client.0.smithi156.stdout:6/795: mkdir d3/d13/d17/d4f/d5c/d101 0 2024-07-16T09:55:49.492 INFO:tasks.workunit.client.0.smithi156.stdout:3/872: sync 2024-07-16T09:55:49.493 INFO:tasks.workunit.client.0.smithi156.stdout:5/883: sync 2024-07-16T09:55:49.494 INFO:tasks.workunit.client.0.smithi156.stdout:2/980: rename d0/f91 to d0/d13/d17/d128/f13c 0 2024-07-16T09:55:49.497 INFO:tasks.workunit.client.0.smithi156.stdout:0/915: dwrite d0/d1a/d4d/d9/d27/d50/f5a [0,4194304] 0 2024-07-16T09:55:49.497 INFO:tasks.workunit.client.0.smithi156.stdout:5/884: write d1b/d1f/d45/d52/f113 [203821,36954] 0 2024-07-16T09:55:49.498 INFO:tasks.workunit.client.0.smithi156.stdout:4/988: chown d4/db/dc/dcc/dec/ded/d87/f114 1 1 2024-07-16T09:55:49.508 INFO:tasks.workunit.client.0.smithi156.stdout:7/727: truncate dc/d13/d15/d8b/fbe 532538 0 2024-07-16T09:55:49.510 INFO:tasks.workunit.client.1.smithi176.stdout:0/950: write d5/d1c/d24/d35/fc8 [59777,118811] 0 2024-07-16T09:55:49.510 INFO:tasks.workunit.client.1.smithi176.stdout:4/800: truncate da/dc/d4b/df0/f27 491338 0 2024-07-16T09:55:49.510 INFO:tasks.workunit.client.1.smithi176.stdout:4/801: mkdir da/d3d/d33/d104 0 2024-07-16T09:55:49.510 INFO:tasks.workunit.client.1.smithi176.stdout:0/951: mknod d5/d1c/d24/d35/d50/d58/d63/db4/c139 0 2024-07-16T09:55:49.510 INFO:tasks.workunit.client.1.smithi176.stdout:4/802: truncate da/d3d/f103 283224 0 2024-07-16T09:55:49.510 INFO:tasks.workunit.client.1.smithi176.stdout:4/803: symlink da/de5/l105 0 2024-07-16T09:55:49.511 INFO:tasks.workunit.client.0.smithi156.stdout:2/981: truncate d0/d13/f26 2286273 0 2024-07-16T09:55:49.511 INFO:tasks.workunit.client.1.smithi176.stdout:0/952: fdatasync d5/d39/da3/fe2 0 2024-07-16T09:55:49.511 INFO:tasks.workunit.client.0.smithi156.stdout:6/796: symlink d3/d13/d17/d4f/d30/d45/l102 0 2024-07-16T09:55:49.512 INFO:tasks.workunit.client.1.smithi176.stdout:0/953: stat d5/d2c/d6c/l42 0 2024-07-16T09:55:49.513 INFO:tasks.workunit.client.0.smithi156.stdout:3/873: mkdir d8/da/d28/d43/d11e/d128 0 2024-07-16T09:55:49.513 INFO:tasks.workunit.client.1.smithi176.stdout:4/804: mkdir da/dc/d4b/df0/d28/d106 0 2024-07-16T09:55:49.516 INFO:tasks.workunit.client.0.smithi156.stdout:0/916: creat d0/d1a/d76/da7/f12a x:0 0 0 2024-07-16T09:55:49.516 INFO:tasks.workunit.client.1.smithi176.stdout:0/954: symlink d5/d1c/d24/d35/d50/dd5/l13a 0 2024-07-16T09:55:49.517 INFO:tasks.workunit.client.1.smithi176.stdout:0/955: chown d5/d44/ld6 0 1 2024-07-16T09:55:49.517 INFO:tasks.workunit.client.1.smithi176.stdout:6/950: dread d1/d5d/d9e/dd9/d6f/dd0/f145 [0,4194304] 0 2024-07-16T09:55:49.518 INFO:tasks.workunit.client.1.smithi176.stdout:4/805: mkdir da/d3d/d17/d54/d78/d7a/d107 0 2024-07-16T09:55:49.520 INFO:tasks.workunit.client.1.smithi176.stdout:0/956: creat d5/d44/f13b x:0 0 0 2024-07-16T09:55:49.520 INFO:tasks.workunit.client.1.smithi176.stdout:4/806: write da/d3d/d17/f31 [1102686,74953] 0 2024-07-16T09:55:49.523 INFO:tasks.workunit.client.1.smithi176.stdout:4/807: chown da/dc/d4b/df0/d28/cf2 1959463774 1 2024-07-16T09:55:49.523 INFO:tasks.workunit.client.1.smithi176.stdout:0/957: truncate d5/d1c/db1/f10b 634479 0 2024-07-16T09:55:49.523 INFO:tasks.workunit.client.0.smithi156.stdout:8/849: getdents d0/d11/d2e/d35 0 2024-07-16T09:55:49.524 INFO:tasks.workunit.client.0.smithi156.stdout:4/989: dread d4/db/dc/fcb [0,4194304] 0 2024-07-16T09:55:49.524 INFO:tasks.workunit.client.0.smithi156.stdout:0/917: symlink d0/d1a/d4d/d61/d97/l12b 0 2024-07-16T09:55:49.525 INFO:tasks.workunit.client.1.smithi176.stdout:0/958: chown d5/d1c/d24/d35/d50/d9b/de3/d117/d131/cc1 627 1 2024-07-16T09:55:49.525 INFO:tasks.workunit.client.1.smithi176.stdout:6/951: rename d1/d5d/d9e/dd9/d6f/d7c/ced to d1/d5d/d9e/dd9/d6f/dd0/c155 0 2024-07-16T09:55:49.526 INFO:tasks.workunit.client.0.smithi156.stdout:0/918: read d0/d1a/d76/d92/f10b [516992,29567] 0 2024-07-16T09:55:49.526 INFO:tasks.workunit.client.0.smithi156.stdout:5/885: truncate d1b/d1f/d54/d72/f79 364288 0 2024-07-16T09:55:49.527 INFO:tasks.workunit.client.1.smithi176.stdout:4/808: symlink da/de5/l108 0 2024-07-16T09:55:49.527 INFO:tasks.workunit.client.1.smithi176.stdout:0/959: dread - d5/d1c/d5a/d9e/fa5 zero size 2024-07-16T09:55:49.529 INFO:tasks.workunit.client.1.smithi176.stdout:8/838: unlink d8/c4e 0 2024-07-16T09:55:49.530 INFO:tasks.workunit.client.1.smithi176.stdout:0/960: symlink d5/d1c/d5a/d9e/l13c 0 2024-07-16T09:55:49.532 INFO:tasks.workunit.client.0.smithi156.stdout:8/850: read d0/d11/d29/f8e [147434,38626] 0 2024-07-16T09:55:49.533 INFO:tasks.workunit.client.0.smithi156.stdout:3/874: truncate d8/da/d28/d43/d71/da2/fd4 2355081 0 2024-07-16T09:55:49.538 INFO:tasks.workunit.client.0.smithi156.stdout:2/982: write d0/d5/d29/f111 [1610776,109754] 0 2024-07-16T09:55:49.539 INFO:tasks.workunit.client.0.smithi156.stdout:7/728: creat dc/d63/ffd x:0 0 0 2024-07-16T09:55:49.539 INFO:tasks.workunit.client.0.smithi156.stdout:2/983: chown d0/d13/d17/d1c/d51/le3 11289 1 2024-07-16T09:55:49.540 INFO:tasks.workunit.client.1.smithi176.stdout:5/895: write d4/d10/f12 [5129187,106566] 0 2024-07-16T09:55:49.541 INFO:tasks.workunit.client.1.smithi176.stdout:5/896: chown d4/d14/f25 235428896 1 2024-07-16T09:55:49.541 INFO:tasks.workunit.client.0.smithi156.stdout:2/984: stat d0/d13/d6e/f90 0 2024-07-16T09:55:49.542 INFO:tasks.workunit.client.0.smithi156.stdout:6/797: write d3/d13/d17/d4f/f34 [605564,80520] 0 2024-07-16T09:55:49.542 INFO:tasks.workunit.client.1.smithi176.stdout:1/585: truncate d4/d77/dad/d72/f74 422922 0 2024-07-16T09:55:49.544 INFO:tasks.workunit.client.0.smithi156.stdout:5/886: mkdir d1b/d1f/d42/d6d/d9f/d11c 0 2024-07-16T09:55:49.544 INFO:tasks.workunit.client.0.smithi156.stdout:6/798: dread - d3/d13/d17/d4f/d4d/d3b/fde zero size 2024-07-16T09:55:49.544 INFO:tasks.workunit.client.1.smithi176.stdout:6/952: dread d1/d5/d47/d62/fa1 [0,4194304] 0 2024-07-16T09:55:49.545 INFO:tasks.workunit.client.0.smithi156.stdout:0/919: symlink d0/d22/d41/d66/l12c 0 2024-07-16T09:55:49.546 INFO:tasks.workunit.client.1.smithi176.stdout:6/953: write d1/d5/d47/d62/de8/f10e [322488,54169] 0 2024-07-16T09:55:49.547 INFO:tasks.workunit.client.1.smithi176.stdout:9/954: write d5/d1d/dd2/f8c [5047690,121326] 0 2024-07-16T09:55:49.548 INFO:tasks.workunit.client.0.smithi156.stdout:3/875: symlink d8/da/dec/d111/l129 0 2024-07-16T09:55:49.549 INFO:tasks.workunit.client.0.smithi156.stdout:3/876: fdatasync d8/da/d1d/d2c/dee/f115 0 2024-07-16T09:55:49.550 INFO:tasks.workunit.client.0.smithi156.stdout:7/729: symlink dc/d13/d15/d8b/dcc/lfe 0 2024-07-16T09:55:49.550 INFO:tasks.workunit.client.0.smithi156.stdout:4/990: write d4/db/dc/fcb [1635583,37197] 0 2024-07-16T09:55:49.551 INFO:tasks.workunit.client.1.smithi176.stdout:6/954: creat d1/d5/d47/d75/f156 x:0 0 0 2024-07-16T09:55:49.551 INFO:tasks.workunit.client.0.smithi156.stdout:7/730: dread - dc/d13/d20/d4d/fee zero size 2024-07-16T09:55:49.552 INFO:tasks.workunit.client.0.smithi156.stdout:2/985: unlink d0/d13/d17/d1c/d86/dde/d109/l11a 0 2024-07-16T09:55:49.553 INFO:tasks.workunit.client.0.smithi156.stdout:4/991: chown d4/db/dc/dcc/dec/f12c 7016555 1 2024-07-16T09:55:49.553 INFO:tasks.workunit.client.0.smithi156.stdout:2/986: readlink d0/d5/l12c 0 2024-07-16T09:55:49.555 INFO:tasks.workunit.client.1.smithi176.stdout:9/955: dread d5/d1d/d20/d78/df2/d118/d11c/f127 [0,4194304] 0 2024-07-16T09:55:49.555 INFO:tasks.workunit.client.0.smithi156.stdout:4/992: chown d4/db/dc/dcc/dec 0 1 2024-07-16T09:55:49.556 INFO:tasks.workunit.client.0.smithi156.stdout:0/920: rmdir d0/d1a/d76/d92 39 2024-07-16T09:55:49.557 INFO:tasks.workunit.client.0.smithi156.stdout:4/993: read d4/db/d8a/fc4 [2390261,69037] 0 2024-07-16T09:55:49.559 INFO:tasks.workunit.client.1.smithi176.stdout:7/756: dwrite d4/d47/d6d/d89/fc8 [0,4194304] 0 2024-07-16T09:55:49.560 INFO:tasks.workunit.client.1.smithi176.stdout:5/897: dread d4/f9e [0,4194304] 0 2024-07-16T09:55:49.562 INFO:tasks.workunit.client.0.smithi156.stdout:6/799: write d3/d13/d17/d4f/d94/fa [1193390,56753] 0 2024-07-16T09:55:49.566 INFO:tasks.workunit.client.1.smithi176.stdout:7/757: mknod d4/d47/d6d/d89/d66/cf7 0 2024-07-16T09:55:49.567 INFO:tasks.workunit.client.1.smithi176.stdout:9/956: creat d5/d1d/db0/f145 x:0 0 0 2024-07-16T09:55:49.567 INFO:tasks.workunit.client.0.smithi156.stdout:2/987: symlink d0/d5/d29/d2d/d5b/d5f/d89/l13d 0 2024-07-16T09:55:49.567 INFO:tasks.workunit.client.0.smithi156.stdout:7/731: dwrite dc/d13/d36/f66 [0,4194304] 0 2024-07-16T09:55:49.568 INFO:tasks.workunit.client.1.smithi176.stdout:9/957: stat d5/d43 0 2024-07-16T09:55:49.575 INFO:tasks.workunit.client.0.smithi156.stdout:3/877: symlink d8/da/d21/d80/l12a 0 2024-07-16T09:55:49.575 INFO:tasks.workunit.client.0.smithi156.stdout:7/732: fsync dc/d13/d36/d56/fa2 0 2024-07-16T09:55:49.576 INFO:tasks.workunit.client.1.smithi176.stdout:7/758: dread d4/dd/d26/dc2/d17/d1d/f9b [8388608,4194304] 0 2024-07-16T09:55:49.577 INFO:tasks.workunit.client.0.smithi156.stdout:5/887: dwrite d1b/d1f/d45/d52/d43/f58 [0,4194304] 0 2024-07-16T09:55:49.578 INFO:tasks.workunit.client.0.smithi156.stdout:3/878: chown d8/da/d21/d80/d119 22507 1 2024-07-16T09:55:49.578 INFO:tasks.workunit.client.0.smithi156.stdout:4/994: link d4/db/dc/d5f/d64/l65 d4/db/dc/dcc/d58/dd9/l15d 0 2024-07-16T09:55:49.580 INFO:tasks.workunit.client.0.smithi156.stdout:6/800: link d3/d13/d17/d4f/d94/d19/d31/fa2 d3/d13/d17/d4f/d94/de4/f103 0 2024-07-16T09:55:49.584 INFO:tasks.workunit.client.0.smithi156.stdout:6/801: chown d3/d13/d17/d4f/d94/d19/d31 403399189 1 2024-07-16T09:55:49.587 INFO:tasks.workunit.client.0.smithi156.stdout:7/733: stat dc/d13/d15/dfc/ldc 0 2024-07-16T09:55:49.588 INFO:tasks.workunit.client.0.smithi156.stdout:4/995: write d4/db/dc/dcc/d58/fa6 [4664289,61514] 0 2024-07-16T09:55:49.591 INFO:tasks.workunit.client.0.smithi156.stdout:4/996: stat d4/db/d8a/cf5 0 2024-07-16T09:55:49.592 INFO:tasks.workunit.client.0.smithi156.stdout:0/921: dwrite d0/d1a/d4d/d25/fb8 [0,4194304] 0 2024-07-16T09:55:49.592 INFO:tasks.workunit.client.0.smithi156.stdout:2/988: truncate d0/d5/d29/d2d/d5b/d5f/d89/f99 1028139 0 2024-07-16T09:55:49.592 INFO:tasks.workunit.client.1.smithi176.stdout:0/961: dwrite d5/fb [4194304,4194304] 0 2024-07-16T09:55:49.592 INFO:tasks.workunit.client.0.smithi156.stdout:6/802: write d3/d13/d17/d4f/d4d/d3b/d5a/f69 [1635827,73820] 0 2024-07-16T09:55:49.593 INFO:tasks.workunit.client.0.smithi156.stdout:5/888: write d1b/d1f/d54/f74 [574479,35357] 0 2024-07-16T09:55:49.593 INFO:tasks.workunit.client.0.smithi156.stdout:3/879: getdents d8/da/d28/d43/d71/da2/d114 0 2024-07-16T09:55:49.594 INFO:tasks.workunit.client.1.smithi176.stdout:0/962: dread - d5/d1c/d24/d35/d50/d9b/de3/d117/f119 zero size 2024-07-16T09:55:49.594 INFO:tasks.workunit.client.0.smithi156.stdout:7/734: mkdir dc/d13/d15/d8d/dd0/dff 0 2024-07-16T09:55:49.595 INFO:tasks.workunit.client.1.smithi176.stdout:0/963: chown d5/d1c/d24/d35/d50/d7c 274841 1 2024-07-16T09:55:49.596 INFO:tasks.workunit.client.0.smithi156.stdout:2/989: chown d0/d5/d76/d10f 6182954 1 2024-07-16T09:55:49.598 INFO:tasks.workunit.client.0.smithi156.stdout:7/735: readlink dc/d13/d20/d4d/le1 0 2024-07-16T09:55:49.598 INFO:tasks.workunit.client.0.smithi156.stdout:4/997: rmdir d4/db/dc/dcc/d58/d8f/d11f 39 2024-07-16T09:55:49.602 INFO:tasks.workunit.client.0.smithi156.stdout:5/889: write d1b/d1f/d45/d52/d43/d85/f8f [304059,98494] 0 2024-07-16T09:55:49.605 INFO:tasks.workunit.client.0.smithi156.stdout:2/990: unlink d0/d13/d17/cf6 0 2024-07-16T09:55:49.607 INFO:tasks.workunit.client.0.smithi156.stdout:2/991: readlink d0/d5/d76/da1/la4 0 2024-07-16T09:55:49.609 INFO:tasks.workunit.client.1.smithi176.stdout:8/839: truncate d8/d23/d100/f49 2943584 0 2024-07-16T09:55:49.609 INFO:tasks.workunit.client.0.smithi156.stdout:6/803: creat d3/d13/d17/d4f/d4d/d3b/d5a/dc1/df9/f104 x:0 0 0 2024-07-16T09:55:49.610 INFO:tasks.workunit.client.1.smithi176.stdout:4/809: truncate da/dc/d4b/df0/d96/d9b/db4/f29 381957 0 2024-07-16T09:55:49.613 INFO:tasks.workunit.client.0.smithi156.stdout:0/922: link d0/d1a/d4d/d9/d68/dcb/dd7/cd0 d0/d22/d41/d52/ddb/c12d 0 2024-07-16T09:55:49.617 INFO:tasks.workunit.client.0.smithi156.stdout:3/880: write d8/d14/fb7 [3584528,119052] 0 2024-07-16T09:55:49.617 INFO:tasks.workunit.client.0.smithi156.stdout:4/998: dwrite d4/db/dc/dcc/dec/ded/d87/f114 [0,4194304] 0 2024-07-16T09:55:49.619 INFO:tasks.workunit.client.0.smithi156.stdout:7/736: dwrite dc/d13/d15/f78 [0,4194304] 0 2024-07-16T09:55:49.620 INFO:tasks.workunit.client.0.smithi156.stdout:2/992: chown d0/d13/d17/d1c/d51/dc6/fd4 3328 1 2024-07-16T09:55:49.622 INFO:tasks.workunit.client.1.smithi176.stdout:6/955: write d1/d5d/db9/f32 [2761393,21822] 0 2024-07-16T09:55:49.623 INFO:tasks.workunit.client.0.smithi156.stdout:3/881: chown d8/d14/c22 45340 1 2024-07-16T09:55:49.623 INFO:tasks.workunit.client.0.smithi156.stdout:0/923: chown d0/d1a/d4d/d60/la5 0 1 2024-07-16T09:55:49.624 INFO:tasks.workunit.client.1.smithi176.stdout:6/956: write d1/f104 [1689726,8603] 0 2024-07-16T09:55:49.624 INFO:tasks.workunit.client.0.smithi156.stdout:5/890: dwrite d1b/f34 [0,4194304] 0 2024-07-16T09:55:49.629 INFO:tasks.workunit.client.0.smithi156.stdout:5/891: chown d1b/d1f/d45/d7a/d98/cfa 65203029 1 2024-07-16T09:55:49.630 INFO:tasks.workunit.client.0.smithi156.stdout:6/804: creat d3/d13/d17/d4f/d94/d19/d2e/f105 x:0 0 0 2024-07-16T09:55:49.632 INFO:tasks.workunit.client.0.smithi156.stdout:7/737: truncate fb 3504922 0 2024-07-16T09:55:49.633 INFO:tasks.workunit.client.0.smithi156.stdout:4/999: unlink d4/db/dc/dcc/d58/dd9/f104 0 2024-07-16T09:55:49.635 INFO:tasks.workunit.client.0.smithi156.stdout:0/924: chown d0/d22/c44 195 1 2024-07-16T09:55:49.637 INFO:tasks.workunit.client.0.smithi156.stdout:2/993: dwrite d0/d13/d17/fdf [0,4194304] 0 2024-07-16T09:55:49.637 INFO:tasks.workunit.client.0.smithi156.stdout:3/882: creat d8/da/dec/df1/f12b x:0 0 0 2024-07-16T09:55:49.638 INFO:tasks.workunit.client.1.smithi176.stdout:6/957: dread d1/d5/d47/d62/de8/d114/f137 [0,4194304] 0 2024-07-16T09:55:49.640 INFO:tasks.workunit.client.0.smithi156.stdout:7/738: creat dc/d13/d24/f100 x:0 0 0 2024-07-16T09:55:49.641 INFO:tasks.workunit.client.0.smithi156.stdout:6/805: rename d3/d13/d17/d4f/d5c/l79 to d3/d13/d17/d4f/d94/d19/d2e/l106 0 2024-07-16T09:55:49.643 INFO:tasks.workunit.client.0.smithi156.stdout:2/994: chown d0/d5/d76/f108 1562 1 2024-07-16T09:55:49.644 INFO:tasks.workunit.client.1.smithi176.stdout:6/958: dwrite d1/f104 [0,4194304] 0 2024-07-16T09:55:49.644 INFO:tasks.workunit.client.0.smithi156.stdout:0/925: write d0/d1a/d4d/d9/d68/dcb/fe8 [757166,60790] 0 2024-07-16T09:55:49.645 INFO:tasks.workunit.client.0.smithi156.stdout:5/892: dread d1b/d1f/d45/d52/d43/f58 [0,4194304] 0 2024-07-16T09:55:49.645 INFO:tasks.workunit.client.0.smithi156.stdout:3/883: fdatasync d8/da/d21/d80/f82 0 2024-07-16T09:55:49.649 INFO:tasks.workunit.client.1.smithi176.stdout:7/759: creat d4/dd/db5/ff8 x:0 0 0 2024-07-16T09:55:49.651 INFO:tasks.workunit.client.1.smithi176.stdout:7/760: write d4/d47/d6d/d89/fc8 [2544324,114633] 0 2024-07-16T09:55:49.652 INFO:tasks.workunit.client.1.smithi176.stdout:8/840: write d8/d20/f5a [6439050,21118] 0 2024-07-16T09:55:49.653 INFO:tasks.workunit.client.0.smithi156.stdout:0/926: write d0/d1a/d4d/d9/f8f [456932,25670] 0 2024-07-16T09:55:49.653 INFO:tasks.workunit.client.1.smithi176.stdout:8/841: chown d8/d23/d100/db9/dcb/d34/cf8 1020 1 2024-07-16T09:55:49.655 INFO:tasks.workunit.client.0.smithi156.stdout:2/995: rename d0/d5/d29/d2d/d5b/cda to d0/d13/d6e/c13e 0 2024-07-16T09:55:49.657 INFO:tasks.workunit.client.1.smithi176.stdout:4/810: creat da/d41/d5b/dba/f109 x:0 0 0 2024-07-16T09:55:49.658 INFO:tasks.workunit.client.0.smithi156.stdout:3/884: chown d8/da/d28/d43/ca0 115285839 1 2024-07-16T09:55:49.659 INFO:tasks.workunit.client.0.smithi156.stdout:5/893: mkdir d1b/d1f/d45/d52/d43/d85/d97/d37/d88/dbc/d11d 0 2024-07-16T09:55:49.661 INFO:tasks.workunit.client.1.smithi176.stdout:6/959: mknod d1/d5d/d9e/d8f/c157 0 2024-07-16T09:55:49.663 INFO:tasks.workunit.client.0.smithi156.stdout:0/927: dwrite d0/d1a/d4d/d9/f8f [0,4194304] 0 2024-07-16T09:55:49.667 INFO:tasks.workunit.client.0.smithi156.stdout:2/996: creat d0/d9b/d10e/d13a/f13f x:0 0 0 2024-07-16T09:55:49.668 INFO:tasks.workunit.client.0.smithi156.stdout:3/885: dread d8/da/d21/d80/fea [0,4194304] 0 2024-07-16T09:55:49.669 INFO:tasks.workunit.client.1.smithi176.stdout:7/761: write d4/d47/d6d/d89/d66/f8a [3835167,97952] 0 2024-07-16T09:55:49.670 INFO:tasks.workunit.client.0.smithi156.stdout:5/894: write d1b/d1f/d45/d7f/f90 [4686672,96939] 0 2024-07-16T09:55:49.671 INFO:tasks.workunit.client.1.smithi176.stdout:6/960: creat d1/d5d/db9/d124/f158 x:0 0 0 2024-07-16T09:55:49.672 INFO:tasks.workunit.client.1.smithi176.stdout:6/961: readlink d1/d5d/db9/l119 0 2024-07-16T09:55:49.673 INFO:tasks.workunit.client.1.smithi176.stdout:5/898: creat d4/d10/f145 x:0 0 0 2024-07-16T09:55:49.674 INFO:tasks.workunit.client.0.smithi156.stdout:0/928: read d0/d1a/d4d/d60/f93 [1935789,75571] 0 2024-07-16T09:55:49.679 INFO:tasks.workunit.client.0.smithi156.stdout:6/806: dwrite d3/d13/d17/d4f/d4d/d3b/d77/fc3 [0,4194304] 0 2024-07-16T09:55:49.679 INFO:tasks.workunit.client.1.smithi176.stdout:6/962: rmdir d1/d5/d47/d62/de8/d111/d122 39 2024-07-16T09:55:49.679 INFO:tasks.workunit.client.0.smithi156.stdout:3/886: creat d8/da/d28/d43/d63/df7/f12c x:0 0 0 2024-07-16T09:55:49.680 INFO:tasks.workunit.client.1.smithi176.stdout:7/762: dwrite d4/dd/d26/dc2/fc3 [0,4194304] 0 2024-07-16T09:55:49.681 INFO:tasks.workunit.client.1.smithi176.stdout:8/842: dread f5 [0,4194304] 0 2024-07-16T09:55:49.683 INFO:tasks.workunit.client.0.smithi156.stdout:6/807: rmdir d3/d13/d17/d4f/d4d/d3b/d5a/dc1/dce 39 2024-07-16T09:55:49.685 INFO:tasks.workunit.client.0.smithi156.stdout:0/929: creat d0/d22/d41/f12e x:0 0 0 2024-07-16T09:55:49.689 INFO:tasks.workunit.client.0.smithi156.stdout:0/930: write d0/d1a/d76/f87 [2234850,55167] 0 2024-07-16T09:55:49.691 INFO:tasks.workunit.client.1.smithi176.stdout:6/963: unlink d1/d5/ccc 0 2024-07-16T09:55:49.692 INFO:tasks.workunit.client.1.smithi176.stdout:0/964: write d5/d2c/f54 [3076,53444] 0 2024-07-16T09:55:49.692 INFO:tasks.workunit.client.1.smithi176.stdout:7/763: creat d4/d47/d6d/d89/d66/ff9 x:0 0 0 2024-07-16T09:55:49.693 INFO:tasks.workunit.client.0.smithi156.stdout:7/739: truncate dc/d3e/d3f/d51/d8a/fe4 65261 0 2024-07-16T09:55:49.693 INFO:tasks.workunit.client.0.smithi156.stdout:3/887: dwrite d8/da/d13/f20 [4194304,4194304] 0 2024-07-16T09:55:49.694 INFO:tasks.workunit.client.1.smithi176.stdout:8/843: creat d8/d23/d100/d53/f119 x:0 0 0 2024-07-16T09:55:49.698 INFO:tasks.workunit.client.1.smithi176.stdout:4/811: getdents da/dc/d4b 0 2024-07-16T09:55:49.698 INFO:tasks.workunit.client.0.smithi156.stdout:0/931: creat d0/d1a/d4d/d9/d27/d2c/d6d/f12f x:0 0 0 2024-07-16T09:55:49.699 INFO:tasks.workunit.client.1.smithi176.stdout:5/899: rename d4/d37/d72/d82/c116 to d4/d14/d19/c146 0 2024-07-16T09:55:49.701 INFO:tasks.workunit.client.0.smithi156.stdout:7/740: creat dc/d13/d15/d8d/dd0/dff/f101 x:0 0 0 2024-07-16T09:55:49.702 INFO:tasks.workunit.client.0.smithi156.stdout:2/997: dread d0/d5/d29/dea/f12a [0,4194304] 0 2024-07-16T09:55:49.702 INFO:tasks.workunit.client.1.smithi176.stdout:6/964: rename d1/dc/d40/d71 to d1/dc/d40/d159 0 2024-07-16T09:55:49.702 INFO:tasks.workunit.client.1.smithi176.stdout:9/958: write d5/d1d/d20/fc5 [569066,80079] 0 2024-07-16T09:55:49.705 INFO:tasks.workunit.client.0.smithi156.stdout:7/741: write dc/d13/d15/dfc/d81/ffb [760455,51757] 0 2024-07-16T09:55:49.706 INFO:tasks.workunit.client.0.smithi156.stdout:5/895: dwrite d1b/d1f/d45/d52/d43/f5b [0,4194304] 0 2024-07-16T09:55:49.706 INFO:tasks.workunit.client.1.smithi176.stdout:7/764: dwrite d4/d5/fe5 [0,4194304] 0 2024-07-16T09:55:49.706 INFO:tasks.workunit.client.0.smithi156.stdout:2/998: unlink d0/d9b/d10e/d13a/f13f 0 2024-07-16T09:55:49.709 INFO:tasks.workunit.client.1.smithi176.stdout:0/965: rename d5/d1c/d24/d35/d130 to d5/d1c/d24/d35/d50/d9b/de3/d117/d131/d13d 0 2024-07-16T09:55:49.710 INFO:tasks.workunit.client.1.smithi176.stdout:8/844: dread d8/d23/d100/db9/dcb/d37/f57 [0,4194304] 0 2024-07-16T09:55:49.711 INFO:tasks.workunit.client.1.smithi176.stdout:5/900: truncate d4/d14/fcd 847601 0 2024-07-16T09:55:49.714 INFO:tasks.workunit.client.0.smithi156.stdout:7/742: symlink dc/d13/d15/d8b/dcc/dde/l102 0 2024-07-16T09:55:49.716 INFO:tasks.workunit.client.1.smithi176.stdout:8/845: dread - d8/d23/d100/d61/fe8 zero size 2024-07-16T09:55:49.717 INFO:tasks.workunit.client.1.smithi176.stdout:5/901: chown d4/d14/d19/l91 387913 1 2024-07-16T09:55:49.718 INFO:tasks.workunit.client.0.smithi156.stdout:3/888: write d8/da/d1d/d2c/f8e [3924053,60719] 0 2024-07-16T09:55:49.718 INFO:tasks.workunit.client.0.smithi156.stdout:5/896: symlink d1b/d1f/d54/dc6/d10e/l11e 0 2024-07-16T09:55:49.719 INFO:tasks.workunit.client.1.smithi176.stdout:4/812: dwrite da/dc/d4b/df0/d28/fb7 [0,4194304] 0 2024-07-16T09:55:49.722 INFO:tasks.workunit.client.0.smithi156.stdout:3/889: fsync d8/da/d13/f20 0 2024-07-16T09:55:49.725 INFO:tasks.workunit.client.1.smithi176.stdout:8/846: symlink d8/d14/d1e/l11a 0 2024-07-16T09:55:49.726 INFO:tasks.workunit.client.1.smithi176.stdout:5/902: creat d4/da9/dc4/f147 x:0 0 0 2024-07-16T09:55:49.727 INFO:tasks.workunit.client.0.smithi156.stdout:3/890: chown d8/da/d1d/db2/lb8 8829 1 2024-07-16T09:55:49.727 INFO:tasks.workunit.client.0.smithi156.stdout:2/999: dwrite d0/d5/d29/d2d/d5b/d5f/fef [4194304,4194304] 0 2024-07-16T09:55:49.728 INFO:tasks.workunit.client.0.smithi156.stdout:6/808: dread d3/d13/d17/d4f/d30/f7f [0,4194304] 0 2024-07-16T09:55:49.728 INFO:tasks.workunit.client.1.smithi176.stdout:7/765: dwrite d4/d47/d6d/d89/d66/f8a [4194304,4194304] 0 2024-07-16T09:55:49.730 INFO:tasks.workunit.client.1.smithi176.stdout:0/966: truncate d5/d1c/d24/d35/d50/d58/f60 3848371 0 2024-07-16T09:55:49.731 INFO:tasks.workunit.client.0.smithi156.stdout:6/809: truncate d3/d13/d17/d4f/d4d/d3b/d5a/dbe/fe6 56273 0 2024-07-16T09:55:49.733 INFO:tasks.workunit.client.1.smithi176.stdout:8/847: fdatasync d8/d23/d100/db9/dcb/d37/f57 0 2024-07-16T09:55:49.733 INFO:tasks.workunit.client.0.smithi156.stdout:3/891: mknod d8/da/d28/d43/c12d 0 2024-07-16T09:55:49.734 INFO:tasks.workunit.client.1.smithi176.stdout:4/813: symlink da/d3d/l10a 0 2024-07-16T09:55:49.736 INFO:tasks.workunit.client.0.smithi156.stdout:7/743: link dc/d3e/d3f/d45/fb3 dc/d13/d15/f103 0 2024-07-16T09:55:49.737 INFO:tasks.workunit.client.0.smithi156.stdout:3/892: dread d8/da/d1d/d2c/ffc [0,4194304] 0 2024-07-16T09:55:49.737 INFO:tasks.workunit.client.1.smithi176.stdout:7/766: mkdir d4/d47/d5d/d9a/de0/dfa 0 2024-07-16T09:55:49.737 INFO:tasks.workunit.client.1.smithi176.stdout:8/848: truncate d8/d23/d100/d53/fb7 345099 0 2024-07-16T09:55:49.738 INFO:tasks.workunit.client.0.smithi156.stdout:7/744: write dc/d3e/fc1 [10516,75992] 0 2024-07-16T09:55:49.738 INFO:tasks.workunit.client.1.smithi176.stdout:4/814: dread da/d3d/f103 [0,4194304] 0 2024-07-16T09:55:49.738 INFO:tasks.workunit.client.0.smithi156.stdout:3/893: dread - d8/da/d1d/d6f/db3/dc4/ff0 zero size 2024-07-16T09:55:49.739 INFO:tasks.workunit.client.1.smithi176.stdout:5/903: getdents d4/d9a/d10a/d10e 0 2024-07-16T09:55:49.739 INFO:tasks.workunit.client.1.smithi176.stdout:8/849: write d8/d23/d100/d53/f119 [1014005,54198] 0 2024-07-16T09:55:49.743 INFO:tasks.workunit.client.0.smithi156.stdout:5/897: rmdir d1b/d1f/d45/d52/d43/d85/d97/d115 0 2024-07-16T09:55:49.744 INFO:tasks.workunit.client.0.smithi156.stdout:0/932: dread d0/d1a/d4d/d25/f38 [0,4194304] 0 2024-07-16T09:55:49.745 INFO:tasks.workunit.client.1.smithi176.stdout:6/965: dread d1/dc/f9f [0,4194304] 0 2024-07-16T09:55:49.745 INFO:tasks.workunit.client.0.smithi156.stdout:3/894: unlink d8/da/d1d/d2c/de0/l7a 0 2024-07-16T09:55:49.745 INFO:tasks.workunit.client.0.smithi156.stdout:6/810: rmdir d3/d13/d17/d4f/d4d/d3b/d5a/dc1/dce 39 2024-07-16T09:55:49.747 INFO:tasks.workunit.client.1.smithi176.stdout:4/815: mknod da/d3d/c10b 0 2024-07-16T09:55:49.747 INFO:tasks.workunit.client.0.smithi156.stdout:7/745: rename dc/d13/d15/d8d/dd0 to dc/d3e/d3f/d45/d104 0 2024-07-16T09:55:49.748 INFO:tasks.workunit.client.1.smithi176.stdout:5/904: mkdir d4/d14/dc5/def/df5/d148 0 2024-07-16T09:55:49.749 INFO:tasks.workunit.client.1.smithi176.stdout:8/850: creat d8/d20/f11b x:0 0 0 2024-07-16T09:55:49.750 INFO:tasks.workunit.client.0.smithi156.stdout:0/933: rename d0/d1a/d4d/l24 to d0/d1a/d4d/d6e/l130 0 2024-07-16T09:55:49.751 INFO:tasks.workunit.client.1.smithi176.stdout:9/959: dwrite d5/fb [4194304,4194304] 0 2024-07-16T09:55:49.752 INFO:tasks.workunit.client.0.smithi156.stdout:3/895: unlink d8/da/cd 0 2024-07-16T09:55:49.754 INFO:tasks.workunit.client.1.smithi176.stdout:6/966: rename d1/d5/d7/d1e/f36 to d1/d5/d7/d1e/d11e/f15a 0 2024-07-16T09:55:49.756 INFO:tasks.workunit.client.0.smithi156.stdout:6/811: dwrite d3/d13/d17/d4f/d94/d19/d31/f9c [0,4194304] 0 2024-07-16T09:55:49.757 INFO:tasks.workunit.client.1.smithi176.stdout:6/967: readlink d1/d5d/le1 0 2024-07-16T09:55:49.757 INFO:tasks.workunit.client.1.smithi176.stdout:8/851: symlink d8/d14/l11c 0 2024-07-16T09:55:49.758 INFO:tasks.workunit.client.1.smithi176.stdout:5/905: dread - d4/d37/d72/d38/d102/f103 zero size 2024-07-16T09:55:49.758 INFO:tasks.workunit.client.1.smithi176.stdout:9/960: mkdir d5/d1d/db3/d11f/d131/d146 0 2024-07-16T09:55:49.762 INFO:tasks.workunit.client.0.smithi156.stdout:6/812: truncate d3/d13/d17/f78 397629 0 2024-07-16T09:55:49.763 INFO:tasks.workunit.client.1.smithi176.stdout:9/961: dread - d5/de/da5/dc1/f10c zero size 2024-07-16T09:55:49.763 INFO:tasks.workunit.client.0.smithi156.stdout:7/746: link dc/d13/d15/l2c dc/d3e/d3f/l105 0 2024-07-16T09:55:49.763 INFO:tasks.workunit.client.1.smithi176.stdout:9/962: fdatasync d5/de/f132 0 2024-07-16T09:55:49.764 INFO:tasks.workunit.client.1.smithi176.stdout:9/963: readlink d5/de/da5/l4b 0 2024-07-16T09:55:49.764 INFO:tasks.workunit.client.0.smithi156.stdout:7/747: dread - dc/d13/d15/d8b/fa7 zero size 2024-07-16T09:55:49.765 INFO:tasks.workunit.client.1.smithi176.stdout:9/964: write d5/d52/f136 [406497,68543] 0 2024-07-16T09:55:49.765 INFO:tasks.workunit.client.0.smithi156.stdout:3/896: dread d8/da/d28/d43/d71/da2/f10b [0,4194304] 0 2024-07-16T09:55:49.768 INFO:tasks.workunit.client.1.smithi176.stdout:6/968: mkdir d1/d5/d47/d62/d80/d13d/d15b 0 2024-07-16T09:55:49.771 INFO:tasks.workunit.client.0.smithi156.stdout:6/813: creat d3/d13/d17/d4f/d4d/d3b/dcc/f107 x:0 0 0 2024-07-16T09:55:49.773 INFO:tasks.workunit.client.1.smithi176.stdout:9/965: dread - d5/d52/fb2 zero size 2024-07-16T09:55:49.774 INFO:tasks.workunit.client.1.smithi176.stdout:6/969: mknod d1/d5/d47/d62/d80/c15c 0 2024-07-16T09:55:49.776 INFO:tasks.workunit.client.1.smithi176.stdout:9/966: write d5/d52/fa4 [545936,65638] 0 2024-07-16T09:55:49.776 INFO:tasks.workunit.client.1.smithi176.stdout:6/970: dread - d1/d5/d47/f142 zero size 2024-07-16T09:55:49.780 INFO:tasks.workunit.client.0.smithi156.stdout:6/814: rename d3/d13/d17/d4f/l71 to d3/d13/d17/l108 0 2024-07-16T09:55:49.780 INFO:tasks.workunit.client.1.smithi176.stdout:8/852: dread d8/d23/d100/db9/fc2 [0,4194304] 0 2024-07-16T09:55:49.781 INFO:tasks.workunit.client.0.smithi156.stdout:6/815: chown d3/d13/d17/d4f/d4d/d3b/d77/fd2 253518191 1 2024-07-16T09:55:49.783 INFO:tasks.workunit.client.1.smithi176.stdout:6/971: mknod d1/d5/d47/d62/de8/d111/d122/d12b/c15d 0 2024-07-16T09:55:49.784 INFO:tasks.workunit.client.1.smithi176.stdout:6/972: chown d1/dc/d40/fca 3 1 2024-07-16T09:55:49.786 INFO:tasks.workunit.client.0.smithi156.stdout:7/748: write dc/d13/fed [940301,91402] 0 2024-07-16T09:55:49.786 INFO:tasks.workunit.client.1.smithi176.stdout:8/853: mkdir d8/d23/d100/db9/dcb/d34/d11d 0 2024-07-16T09:55:49.787 INFO:tasks.workunit.client.1.smithi176.stdout:5/906: write f2 [328937,90431] 0 2024-07-16T09:55:49.788 INFO:tasks.workunit.client.0.smithi156.stdout:0/934: dwrite d0/d1a/d4d/d9/d27/d50/fae [0,4194304] 0 2024-07-16T09:55:49.789 INFO:tasks.workunit.client.1.smithi176.stdout:8/854: dread - d8/d23/d100/d31/d46/d77/f116 zero size 2024-07-16T09:55:49.789 INFO:tasks.workunit.client.1.smithi176.stdout:4/816: dread da/d41/dc3/f8b [0,4194304] 0 2024-07-16T09:55:49.790 INFO:tasks.workunit.client.1.smithi176.stdout:9/967: write d5/d1d/d67/fab [127811,31042] 0 2024-07-16T09:55:49.791 INFO:tasks.workunit.client.0.smithi156.stdout:8/851: sync 2024-07-16T09:55:49.791 INFO:tasks.workunit.client.1.smithi176.stdout:5/907: write d4/d6c/f6f [1277249,13237] 0 2024-07-16T09:55:49.793 INFO:tasks.workunit.client.0.smithi156.stdout:0/935: mkdir d0/d1a/d4d/d9/d27/d50/d11b/d131 0 2024-07-16T09:55:49.793 INFO:tasks.workunit.client.1.smithi176.stdout:8/855: write d8/d14/d1e/d33/df6/f10c [476210,45850] 0 2024-07-16T09:55:49.794 INFO:tasks.workunit.client.1.smithi176.stdout:4/817: creat da/d41/d5b/dba/f10c x:0 0 0 2024-07-16T09:55:49.794 INFO:tasks.workunit.client.0.smithi156.stdout:0/936: dread - d0/d1a/d4d/d9/f115 zero size 2024-07-16T09:55:49.795 INFO:tasks.workunit.client.0.smithi156.stdout:0/937: readlink d0/d1a/d4d/d25/l1f 0 2024-07-16T09:55:49.796 INFO:tasks.workunit.client.0.smithi156.stdout:7/749: link dc/d3e/d3f/d45/l71 dc/d13/d20/d4d/daa/l106 0 2024-07-16T09:55:49.797 INFO:tasks.workunit.client.1.smithi176.stdout:8/856: chown d8/d23/d100/d31/d46/d77 35016 1 2024-07-16T09:55:49.800 INFO:tasks.workunit.client.1.smithi176.stdout:4/818: rename da/dc/d4b/df0/d96/d9b/db4/fdc to da/dc/d4b/df0/d96/da4/dc2/f10d 0 2024-07-16T09:55:49.800 INFO:tasks.workunit.client.0.smithi156.stdout:7/750: rename dc/d13/d15/l9b to dc/d3e/d5d/l107 0 2024-07-16T09:55:49.803 INFO:tasks.workunit.client.0.smithi156.stdout:7/751: chown dc/d3e/d3f/d51/d8a/l93 238 1 2024-07-16T09:55:49.803 INFO:tasks.workunit.client.0.smithi156.stdout:8/852: getdents d0/d11/d13/d1c/d24/d121 0 2024-07-16T09:55:49.803 INFO:tasks.workunit.client.1.smithi176.stdout:4/819: mknod da/dc/d4b/df0/d96/d9b/db4/d40/c10e 0 2024-07-16T09:55:49.804 INFO:tasks.workunit.client.0.smithi156.stdout:8/853: write d0/d11/ded/f10c [211182,49433] 0 2024-07-16T09:55:49.805 INFO:tasks.workunit.client.1.smithi176.stdout:8/857: link d8/d23/d100/d31/d2f/ld2 d8/d10f/l11e 0 2024-07-16T09:55:49.806 INFO:tasks.workunit.client.0.smithi156.stdout:7/752: rename dc/d13/d15/dfc/f5c to dc/d13/d15/dfc/db1/f108 0 2024-07-16T09:55:49.808 INFO:tasks.workunit.client.0.smithi156.stdout:6/816: dread d3/d13/d17/d4f/d94/f2c [0,4194304] 0 2024-07-16T09:55:49.808 INFO:tasks.workunit.client.0.smithi156.stdout:8/854: rmdir d0/d11/d2e/d127 0 2024-07-16T09:55:49.810 INFO:tasks.workunit.client.1.smithi176.stdout:8/858: getdents d8/d14/d1e/d33/df6 0 2024-07-16T09:55:49.812 INFO:tasks.workunit.client.0.smithi156.stdout:8/855: chown d0/d11/d20/d7f/db7/d110/f17 31547 1 2024-07-16T09:55:49.813 INFO:tasks.workunit.client.0.smithi156.stdout:0/938: truncate d0/d1a/d4d/d9/d68/dff/fb6 3569403 0 2024-07-16T09:55:49.813 INFO:tasks.workunit.client.1.smithi176.stdout:4/820: dwrite da/dc/d4b/df0/d96/d69/fb0 [0,4194304] 0 2024-07-16T09:55:49.814 INFO:tasks.workunit.client.1.smithi176.stdout:5/908: dwrite d4/d14/d19/fd6 [0,4194304] 0 2024-07-16T09:55:49.816 INFO:tasks.workunit.client.0.smithi156.stdout:6/817: write d3/d13/d17/d4f/d4d/f61 [4595969,48884] 0 2024-07-16T09:55:49.816 INFO:tasks.workunit.client.1.smithi176.stdout:8/859: creat d8/d23/d76/d83/d103/f11f x:0 0 0 2024-07-16T09:55:49.817 INFO:tasks.workunit.client.1.smithi176.stdout:4/821: rmdir da/dc/d4b/df0/d96/da4 39 2024-07-16T09:55:49.819 INFO:tasks.workunit.client.0.smithi156.stdout:7/753: dwrite dc/d13/d20/d4d/fee [0,4194304] 0 2024-07-16T09:55:49.825 INFO:tasks.workunit.client.1.smithi176.stdout:8/860: rename d8/d14/d1e/d7c/d82/l87 to d8/d23/d100/d53/l120 0 2024-07-16T09:55:49.826 INFO:tasks.workunit.client.1.smithi176.stdout:8/861: fsync d8/d23/d76/d83/d103/f111 0 2024-07-16T09:55:49.826 INFO:tasks.workunit.client.0.smithi156.stdout:0/939: write d0/d1a/f9c [108567,92365] 0 2024-07-16T09:55:49.827 INFO:tasks.workunit.client.0.smithi156.stdout:7/754: getdents dc/d3e/d76/df1 0 2024-07-16T09:55:49.828 INFO:tasks.workunit.client.0.smithi156.stdout:0/940: chown d0/d22/d41/d66/cf2 3917141 1 2024-07-16T09:55:49.830 INFO:tasks.workunit.client.0.smithi156.stdout:7/755: write dc/d13/d36/d56/fa2 [728955,91509] 0 2024-07-16T09:55:49.832 INFO:tasks.workunit.client.0.smithi156.stdout:8/856: truncate d0/d11/d62/fa4 2306246 0 2024-07-16T09:55:49.833 INFO:tasks.workunit.client.0.smithi156.stdout:8/857: truncate d0/d11/d2e/fda 1004849 0 2024-07-16T09:55:49.835 INFO:tasks.workunit.client.0.smithi156.stdout:7/756: readlink dc/d13/d15/l19 0 2024-07-16T09:55:49.836 INFO:tasks.workunit.client.0.smithi156.stdout:0/941: write d0/d1a/d4d/d9/d27/d50/f5f [7957722,75030] 0 2024-07-16T09:55:49.837 INFO:tasks.workunit.client.1.smithi176.stdout:4/822: write da/dc/d4b/df0/d28/f7f [71857,17818] 0 2024-07-16T09:55:49.838 INFO:tasks.workunit.client.1.smithi176.stdout:4/823: read da/dc/d4b/df0/f59 [3287,9951] 0 2024-07-16T09:55:49.841 INFO:tasks.workunit.client.0.smithi156.stdout:8/858: creat d0/d11/d20/d7f/db7/d110/d1f/d57/dc0/dff/f128 x:0 0 0 2024-07-16T09:55:49.842 INFO:tasks.workunit.client.0.smithi156.stdout:7/757: truncate dc/d3e/d5d/f80 1425619 0 2024-07-16T09:55:49.843 INFO:tasks.workunit.client.0.smithi156.stdout:6/818: dread d3/d13/d17/d4f/d4d/d3b/f40 [0,4194304] 0 2024-07-16T09:55:49.843 INFO:tasks.workunit.client.0.smithi156.stdout:0/942: mkdir d0/d1a/d4d/d9/d27/d2c/d6d/d9d/ddc/d132 0 2024-07-16T09:55:49.846 INFO:tasks.workunit.client.0.smithi156.stdout:0/943: read d0/d1a/d4d/d9/d27/d2c/d6d/ff6 [131086,57461] 0 2024-07-16T09:55:49.846 INFO:tasks.workunit.client.0.smithi156.stdout:6/819: mknod d3/d13/d17/d4f/d4d/d3b/d5a/c109 0 2024-07-16T09:55:49.848 INFO:tasks.workunit.client.0.smithi156.stdout:0/944: chown d0/d1a/d4d/d9/d27/d3d/d62/l64 2 1 2024-07-16T09:55:49.849 INFO:tasks.workunit.client.0.smithi156.stdout:0/945: write d0/d1a/d4d/f95 [3898195,130230] 0 2024-07-16T09:55:49.850 INFO:tasks.workunit.client.0.smithi156.stdout:7/758: getdents dc/d63 0 2024-07-16T09:55:49.852 INFO:tasks.workunit.client.0.smithi156.stdout:0/946: write d0/d1a/d4d/d5e/d117/fd8 [1022509,13878] 0 2024-07-16T09:55:49.859 INFO:tasks.workunit.client.0.smithi156.stdout:6/820: rename d3/d13/d17/d4f/d4d/d3b/dcc to d3/d13/d17/d4f/d94/d19/d31/d10a 0 2024-07-16T09:55:49.860 INFO:tasks.workunit.client.0.smithi156.stdout:6/821: chown d3/d13/d17/d4f/d94/d19/d2e/l42 1 1 2024-07-16T09:55:49.865 INFO:tasks.workunit.client.1.smithi176.stdout:8/862: dread d8/d23/d100/db9/dcb/d37/f3c [4194304,4194304] 0 2024-07-16T09:55:49.869 INFO:tasks.workunit.client.1.smithi176.stdout:8/863: mknod d8/d23/d100/db9/dcb/d34/d62/c121 0 2024-07-16T09:55:49.870 INFO:tasks.workunit.client.1.smithi176.stdout:4/824: dread da/dc/d4b/df0/d96/d69/fae [0,4194304] 0 2024-07-16T09:55:49.870 INFO:tasks.workunit.client.0.smithi156.stdout:8/859: dwrite d0/d11/d20/d7f/db7/d110/d1f/d60/f98 [0,4194304] 0 2024-07-16T09:55:49.871 INFO:tasks.workunit.client.0.smithi156.stdout:6/822: dwrite d3/d13/d17/d4f/d4d/d3b/f40 [0,4194304] 0 2024-07-16T09:55:49.871 INFO:tasks.workunit.client.1.smithi176.stdout:4/825: stat da/dc/d4b/df0/d96/d69/fd1 0 2024-07-16T09:55:49.872 INFO:tasks.workunit.client.0.smithi156.stdout:0/947: link d0/d1a/f9c d0/d1a/d4d/d9/d27/d2c/d6d/d9d/f133 0 2024-07-16T09:55:49.872 INFO:tasks.workunit.client.0.smithi156.stdout:6/823: fsync d3/d13/d17/d4f/d94/fe 0 2024-07-16T09:55:49.873 INFO:tasks.workunit.client.1.smithi176.stdout:8/864: truncate d8/d23/d76/d83/fbd 347560 0 2024-07-16T09:55:49.874 INFO:tasks.workunit.client.0.smithi156.stdout:6/824: read - d3/d13/fab zero size 2024-07-16T09:55:49.875 INFO:tasks.workunit.client.1.smithi176.stdout:8/865: write d8/d23/d100/db9/dcb/d34/d62/fa4 [1185321,104010] 0 2024-07-16T09:55:49.878 INFO:tasks.workunit.client.1.smithi176.stdout:4/826: fdatasync da/d3d/d17/f18 0 2024-07-16T09:55:49.881 INFO:tasks.workunit.client.1.smithi176.stdout:8/866: symlink d8/d23/d100/d31/l122 0 2024-07-16T09:55:49.882 INFO:tasks.workunit.client.1.smithi176.stdout:4/827: read da/d3d/d17/d54/fce [57726,107300] 0 2024-07-16T09:55:49.883 INFO:tasks.workunit.client.1.smithi176.stdout:8/867: write d8/d23/d100/d53/d9a/dc1/fef [729889,18591] 0 2024-07-16T09:55:49.884 INFO:tasks.workunit.client.0.smithi156.stdout:6/825: chown d3/d13/d17/d4f/d4d/d3b/d5a/lc2 156180045 1 2024-07-16T09:55:49.885 INFO:tasks.workunit.client.0.smithi156.stdout:8/860: dwrite d0/d11/d20/d7f/db7/d110/d1f/d57/db3/f10f [0,4194304] 0 2024-07-16T09:55:49.885 INFO:tasks.workunit.client.0.smithi156.stdout:7/759: dread dc/d13/d36/f4c [0,4194304] 0 2024-07-16T09:55:49.887 INFO:tasks.workunit.client.0.smithi156.stdout:0/948: dread d0/d1a/d4d/d9/f8f [0,4194304] 0 2024-07-16T09:55:49.891 INFO:tasks.workunit.client.1.smithi176.stdout:8/868: dwrite d8/d14/d1e/d33/f105 [0,4194304] 0 2024-07-16T09:55:49.892 INFO:tasks.workunit.client.0.smithi156.stdout:7/760: creat dc/d3e/d3f/d45/d104/f109 x:0 0 0 2024-07-16T09:55:49.895 INFO:tasks.workunit.client.0.smithi156.stdout:6/826: fsync d3/d13/d17/d4f/d94/d19/d31/d35/fd8 0 2024-07-16T09:55:49.895 INFO:tasks.workunit.client.0.smithi156.stdout:0/949: dread - d0/d1a/d76/d92/fcc zero size 2024-07-16T09:55:49.895 INFO:tasks.workunit.client.0.smithi156.stdout:7/761: mknod dc/d3e/dac/c10a 0 2024-07-16T09:55:49.900 INFO:tasks.workunit.client.0.smithi156.stdout:7/762: mknod dc/d3e/d3f/d45/d104/dff/c10b 0 2024-07-16T09:55:49.900 INFO:tasks.workunit.client.1.smithi176.stdout:8/869: dwrite d8/d14/d1e/d55/fa3 [0,4194304] 0 2024-07-16T09:55:49.902 INFO:tasks.workunit.client.0.smithi156.stdout:8/861: dwrite d0/d11/d20/d3f/dbe/ff4 [0,4194304] 0 2024-07-16T09:55:49.903 INFO:tasks.workunit.client.0.smithi156.stdout:6/827: creat d3/d13/d17/d4f/d94/d19/d31/d35/f10b x:0 0 0 2024-07-16T09:55:49.903 INFO:tasks.workunit.client.0.smithi156.stdout:7/763: truncate dc/d3e/d3f/d45/fb3 4908218 0 2024-07-16T09:55:49.905 INFO:tasks.workunit.client.1.smithi176.stdout:1/586: sync 2024-07-16T09:55:49.906 INFO:tasks.workunit.client.1.smithi176.stdout:3/680: sync 2024-07-16T09:55:49.908 INFO:tasks.workunit.client.0.smithi156.stdout:8/862: dwrite d0/d11/fc7 [0,4194304] 0 2024-07-16T09:55:49.908 INFO:tasks.workunit.client.1.smithi176.stdout:1/587: write d4/d8/d12/d21/d25/d30/d44/d4d/f95 [142546,49310] 0 2024-07-16T09:55:49.909 INFO:tasks.workunit.client.1.smithi176.stdout:3/681: creat d3/da/d56/fcd x:0 0 0 2024-07-16T09:55:49.910 INFO:tasks.workunit.client.1.smithi176.stdout:1/588: chown c2 342 1 2024-07-16T09:55:49.912 INFO:tasks.workunit.client.1.smithi176.stdout:3/682: mknod d3/d17/d1d/cce 0 2024-07-16T09:55:49.912 INFO:tasks.workunit.client.0.smithi156.stdout:0/950: link d0/d1a/d4d/d9/d27/d2c/d6d/fec d0/d1a/d4d/d9/d27/d50/d11b/d11c/f134 0 2024-07-16T09:55:49.915 INFO:tasks.workunit.client.0.smithi156.stdout:0/951: write d0/d1a/d4d/d9/d27/d2c/d6d/f12f [329047,117878] 0 2024-07-16T09:55:49.918 INFO:tasks.workunit.client.0.smithi156.stdout:6/828: getdents d3/d13/d17/d4f/d30 0 2024-07-16T09:55:49.920 INFO:tasks.workunit.client.0.smithi156.stdout:6/829: read d3/d13/d17/d4f/d30/d45/d5f/da8/fed [258016,55571] 0 2024-07-16T09:55:49.932 INFO:tasks.workunit.client.1.smithi176.stdout:8/870: write d8/d14/f2d [432999,11059] 0 2024-07-16T09:55:49.934 INFO:tasks.workunit.client.0.smithi156.stdout:8/863: dread d0/d11/d2e/fda [0,4194304] 0 2024-07-16T09:55:49.935 INFO:tasks.workunit.client.1.smithi176.stdout:8/871: getdents d8/d23/d100/d31/db4 0 2024-07-16T09:55:49.936 INFO:tasks.workunit.client.0.smithi156.stdout:8/864: fdatasync d0/d11/d20/d7f/db7/d110/d1f/d60/f98 0 2024-07-16T09:55:49.938 INFO:tasks.workunit.client.1.smithi176.stdout:3/683: dwrite d3/d17/d1d/d4f/d59/f3a [0,4194304] 0 2024-07-16T09:55:49.945 INFO:tasks.workunit.client.0.smithi156.stdout:3/897: sync 2024-07-16T09:55:49.945 INFO:tasks.workunit.client.0.smithi156.stdout:5/898: sync 2024-07-16T09:55:49.946 INFO:tasks.workunit.client.1.smithi176.stdout:3/684: dwrite d3/da/d66/fa1 [4194304,4194304] 0 2024-07-16T09:55:49.949 INFO:tasks.workunit.client.1.smithi176.stdout:3/685: write d3/d17/d1d/d4f/d59/f93 [1908167,23451] 0 2024-07-16T09:55:49.954 INFO:tasks.workunit.client.1.smithi176.stdout:3/686: symlink d3/d17/d1d/lcf 0 2024-07-16T09:55:49.954 INFO:tasks.workunit.client.1.smithi176.stdout:8/872: dwrite d8/d23/d76/d83/fbd [0,4194304] 0 2024-07-16T09:55:49.955 INFO:tasks.workunit.client.1.smithi176.stdout:3/687: chown d3/d17/l1a 964708124 1 2024-07-16T09:55:49.958 INFO:tasks.workunit.client.1.smithi176.stdout:3/688: fsync d3/d17/d1d/d6e/f9a 0 2024-07-16T09:55:49.965 INFO:tasks.workunit.client.1.smithi176.stdout:8/873: dread d8/d23/d100/d31/d46/d6a/da8/fe4 [0,4194304] 0 2024-07-16T09:55:49.965 INFO:tasks.workunit.client.1.smithi176.stdout:8/874: dread - d8/d20/f11b zero size 2024-07-16T09:55:49.965 INFO:tasks.workunit.client.1.smithi176.stdout:3/689: creat d3/d17/d1d/d4f/d59/fd0 x:0 0 0 2024-07-16T09:55:49.969 INFO:tasks.workunit.client.1.smithi176.stdout:3/690: mkdir d3/dd1 0 2024-07-16T09:55:49.971 INFO:tasks.workunit.client.1.smithi176.stdout:3/691: chown d3/l28 6468583 1 2024-07-16T09:55:49.972 INFO:tasks.workunit.client.1.smithi176.stdout:3/692: chown d3/da/d66 29888489 1 2024-07-16T09:55:49.984 INFO:tasks.workunit.client.1.smithi176.stdout:8/875: truncate d8/d14/d1e/d55/fa3 16933 0 2024-07-16T09:55:49.986 INFO:tasks.workunit.client.1.smithi176.stdout:8/876: mkdir d8/d23/d76/d83/d123 0 2024-07-16T09:55:49.987 INFO:tasks.workunit.client.1.smithi176.stdout:8/877: read d8/d23/d100/db9/dcb/d34/fcc [266584,38723] 0 2024-07-16T09:55:49.992 INFO:tasks.workunit.client.1.smithi176.stdout:8/878: rename d8/d14/d1e/f60 to d8/d23/d100/d31/d46/d6a/da8/f124 0 2024-07-16T09:55:49.995 INFO:tasks.workunit.client.1.smithi176.stdout:3/693: dread d3/da/fe [0,4194304] 0 2024-07-16T09:55:50.002 INFO:tasks.workunit.client.1.smithi176.stdout:3/694: link d3/d17/d1d/d4f/l8f d3/da/d56/ld2 0 2024-07-16T09:55:50.005 INFO:tasks.workunit.client.1.smithi176.stdout:3/695: rmdir d3/da/d6c/dbc 39 2024-07-16T09:55:50.005 INFO:tasks.workunit.client.1.smithi176.stdout:8/879: dwrite d8/d23/d100/d53/d9a/dc1/fd0 [0,4194304] 0 2024-07-16T09:55:50.011 INFO:tasks.workunit.client.1.smithi176.stdout:8/880: creat d8/d14/d1e/d33/df6/f125 x:0 0 0 2024-07-16T09:55:50.011 INFO:tasks.workunit.client.1.smithi176.stdout:8/881: readlink d8/d23/d100/d31/d46/d77/l92 0 2024-07-16T09:55:50.015 INFO:tasks.workunit.client.1.smithi176.stdout:8/882: symlink d8/d23/d100/d31/db4/l126 0 2024-07-16T09:55:50.017 INFO:tasks.workunit.client.1.smithi176.stdout:8/883: chown d8/d23/d100/d31/d46/d77/l92 10550 1 2024-07-16T09:55:50.022 INFO:tasks.workunit.client.1.smithi176.stdout:8/884: rename d8/d23/l72 to d8/d14/d1e/d7c/d82/l127 0 2024-07-16T09:55:50.023 INFO:tasks.workunit.client.1.smithi176.stdout:3/696: dread d3/d17/d1d/d6e/fa4 [0,4194304] 0 2024-07-16T09:55:50.024 INFO:tasks.workunit.client.1.smithi176.stdout:8/885: fsync d8/d23/d100/db9/dcb/d34/d62/fb1 0 2024-07-16T09:55:50.025 INFO:tasks.workunit.client.1.smithi176.stdout:8/886: chown d8/d23/d100/db9/fc2 104878664 1 2024-07-16T09:55:50.026 INFO:tasks.workunit.client.1.smithi176.stdout:3/697: unlink d3/da/d33/c5d 0 2024-07-16T09:55:50.027 INFO:tasks.workunit.client.1.smithi176.stdout:8/887: fdatasync d8/d23/d100/db9/dcb/d34/fcc 0 2024-07-16T09:55:50.029 INFO:tasks.workunit.client.1.smithi176.stdout:3/698: symlink d3/da/ld3 0 2024-07-16T09:55:50.031 INFO:tasks.workunit.client.1.smithi176.stdout:8/888: creat d8/d14/d1e/d7c/f128 x:0 0 0 2024-07-16T09:55:50.035 INFO:tasks.workunit.client.1.smithi176.stdout:3/699: rename d3/d17/d1d/d4f/fc5 to d3/d17/d1d/d4f/fd4 0 2024-07-16T09:55:50.037 INFO:tasks.workunit.client.1.smithi176.stdout:3/700: stat d3/d17/d1d/d4f/d59/f3a 0 2024-07-16T09:55:50.039 INFO:tasks.workunit.client.0.smithi156.stdout:7/764: write dc/d3e/d3f/d51/d8a/fe4 [44380,9732] 0 2024-07-16T09:55:50.040 INFO:tasks.workunit.client.0.smithi156.stdout:5/899: rmdir d1b/d1f/d45/d52/d49 39 2024-07-16T09:55:50.041 INFO:tasks.workunit.client.1.smithi176.stdout:3/701: creat d3/d17/d1d/d6e/fd5 x:0 0 0 2024-07-16T09:55:50.041 INFO:tasks.workunit.client.1.smithi176.stdout:8/889: dwrite d8/d23/d100/d31/d46/fc7 [0,4194304] 0 2024-07-16T09:55:50.041 INFO:tasks.workunit.client.0.smithi156.stdout:3/898: truncate d8/da/dec/df1/fd6 17588 0 2024-07-16T09:55:50.042 INFO:tasks.workunit.client.0.smithi156.stdout:3/899: chown d8/da/d1d/fc0 1 1 2024-07-16T09:55:50.044 INFO:tasks.workunit.client.0.smithi156.stdout:6/830: creat d3/d13/d17/dae/dfb/f10c x:0 0 0 2024-07-16T09:55:50.045 INFO:tasks.workunit.client.1.smithi176.stdout:3/702: mkdir d3/d17/d83/dd6 0 2024-07-16T09:55:50.046 INFO:tasks.workunit.client.1.smithi176.stdout:8/890: read d8/d14/d1e/d55/fa3 [16881,1735] 0 2024-07-16T09:55:50.049 INFO:tasks.workunit.client.0.smithi156.stdout:7/765: symlink dc/d13/d15/dfc/l10c 0 2024-07-16T09:55:50.049 INFO:tasks.workunit.client.0.smithi156.stdout:5/900: rmdir d1b/d1f 39 2024-07-16T09:55:50.049 INFO:tasks.workunit.client.1.smithi176.stdout:3/703: dread d3/da/d84/fb2 [0,4194304] 0 2024-07-16T09:55:50.050 INFO:tasks.workunit.client.0.smithi156.stdout:0/952: link d0/d22/d41/d52/c75 d0/d1a/d4d/d9/d68/dcb/dd7/c135 0 2024-07-16T09:55:50.050 INFO:tasks.workunit.client.1.smithi176.stdout:3/704: write d3/d17/d1d/d4f/daa/fae [1299473,79506] 0 2024-07-16T09:55:50.051 INFO:tasks.workunit.client.0.smithi156.stdout:7/766: fdatasync dc/d3e/d3f/d45/f7d 0 2024-07-16T09:55:50.051 INFO:tasks.workunit.client.0.smithi156.stdout:3/900: symlink d8/da/dec/d111/l12e 0 2024-07-16T09:55:50.051 INFO:tasks.workunit.client.1.smithi176.stdout:3/705: fdatasync d3/da/d7b/fb8 0 2024-07-16T09:55:50.052 INFO:tasks.workunit.client.0.smithi156.stdout:6/831: unlink d3/d13/d17/d4f/d30/d45/d5f/fb4 0 2024-07-16T09:55:50.054 INFO:tasks.workunit.client.1.smithi176.stdout:8/891: symlink d8/d23/d100/d53/d9a/l129 0 2024-07-16T09:55:50.056 INFO:tasks.workunit.client.1.smithi176.stdout:3/706: unlink d3/d17/d1d/d4f/d59/da0/d7a/c96 0 2024-07-16T09:55:50.058 INFO:tasks.workunit.client.1.smithi176.stdout:8/892: rmdir d8/d23/d100/d31/d2f 39 2024-07-16T09:55:50.058 INFO:tasks.workunit.client.0.smithi156.stdout:7/767: mknod dc/d3e/d3f/d51/d6c/c10d 0 2024-07-16T09:55:50.058 INFO:tasks.workunit.client.0.smithi156.stdout:0/953: getdents d0 0 2024-07-16T09:55:50.059 INFO:tasks.workunit.client.1.smithi176.stdout:3/707: write d3/d85/fc2 [861584,74722] 0 2024-07-16T09:55:50.060 INFO:tasks.workunit.client.1.smithi176.stdout:8/893: truncate d8/d14/d1e/d7c/f8f 1872543 0 2024-07-16T09:55:50.061 INFO:tasks.workunit.client.1.smithi176.stdout:8/894: chown d8/d14 17365 1 2024-07-16T09:55:50.062 INFO:tasks.workunit.client.0.smithi156.stdout:7/768: read dc/d48/db2/fe7 [348032,125767] 0 2024-07-16T09:55:50.062 INFO:tasks.workunit.client.0.smithi156.stdout:3/901: dwrite d8/da/f3d [0,4194304] 0 2024-07-16T09:55:50.065 INFO:tasks.workunit.client.0.smithi156.stdout:6/832: rename d3/d13/d17/d4f/d94/d19/d31/d35/f91 to d3/d13/d17/d56/f10d 0 2024-07-16T09:55:50.067 INFO:tasks.workunit.client.0.smithi156.stdout:7/769: read dc/d13/d20/f99 [198161,62278] 0 2024-07-16T09:55:50.068 INFO:tasks.workunit.client.1.smithi176.stdout:3/708: dwrite d3/d17/d1d/d6e/fd5 [0,4194304] 0 2024-07-16T09:55:50.069 INFO:tasks.workunit.client.0.smithi156.stdout:3/902: creat d8/da/d1d/db2/f12f x:0 0 0 2024-07-16T09:55:50.069 INFO:tasks.workunit.client.1.smithi176.stdout:8/895: rmdir d8/d23/d100/db9/dcb/d34/d11d 0 2024-07-16T09:55:50.071 INFO:tasks.workunit.client.0.smithi156.stdout:6/833: stat d3/d13/d17/dae/dfb/f10c 0 2024-07-16T09:55:50.074 INFO:tasks.workunit.client.0.smithi156.stdout:7/770: symlink dc/d13/d15/def/l10e 0 2024-07-16T09:55:50.074 INFO:tasks.workunit.client.0.smithi156.stdout:3/903: chown d8/da/fd7 6129489 1 2024-07-16T09:55:50.077 INFO:tasks.workunit.client.1.smithi176.stdout:8/896: creat d8/d20/f12a x:0 0 0 2024-07-16T09:55:50.077 INFO:tasks.workunit.client.0.smithi156.stdout:0/954: dwrite d0/d1a/d4d/d9/f115 [0,4194304] 0 2024-07-16T09:55:50.080 INFO:tasks.workunit.client.1.smithi176.stdout:8/897: readlink d8/d23/d100/d61/la0 0 2024-07-16T09:55:50.081 INFO:tasks.workunit.client.1.smithi176.stdout:3/709: dwrite d3/da/d33/f53 [0,4194304] 0 2024-07-16T09:55:50.081 INFO:tasks.workunit.client.0.smithi156.stdout:0/955: read d0/d22/f9e [3803342,109728] 0 2024-07-16T09:55:50.084 INFO:tasks.workunit.client.0.smithi156.stdout:6/834: write d3/d13/d17/d4f/d4d/d3b/fde [602574,129772] 0 2024-07-16T09:55:50.084 INFO:tasks.workunit.client.0.smithi156.stdout:8/865: dwrite d0/d11/d2e/fa5 [0,4194304] 0 2024-07-16T09:55:50.088 INFO:tasks.workunit.client.0.smithi156.stdout:7/771: creat dc/d48/db2/f10f x:0 0 0 2024-07-16T09:55:50.089 INFO:tasks.workunit.client.0.smithi156.stdout:5/901: dread d1b/d1f/d45/d52/d43/d7c/f82 [0,4194304] 0 2024-07-16T09:55:50.089 INFO:tasks.workunit.client.0.smithi156.stdout:0/956: unlink d0/cca 0 2024-07-16T09:55:50.091 INFO:tasks.workunit.client.0.smithi156.stdout:6/835: fsync d3/d13/d17/d4f/d94/fa 0 2024-07-16T09:55:50.093 INFO:tasks.workunit.client.0.smithi156.stdout:6/836: dread - d3/d13/d17/d4f/d30/d45/f6d zero size 2024-07-16T09:55:50.093 INFO:tasks.workunit.client.0.smithi156.stdout:7/772: read fb [3003991,18192] 0 2024-07-16T09:55:50.093 INFO:tasks.workunit.client.1.smithi176.stdout:3/710: rmdir d3/d85 39 2024-07-16T09:55:50.093 INFO:tasks.workunit.client.1.smithi176.stdout:8/898: mkdir d8/d23/d76/d83/d103/d12b 0 2024-07-16T09:55:50.097 INFO:tasks.workunit.client.1.smithi176.stdout:3/711: truncate d3/da/d6c/dbc/fbf 869415 0 2024-07-16T09:55:50.098 INFO:tasks.workunit.client.1.smithi176.stdout:0/967: sync 2024-07-16T09:55:50.099 INFO:tasks.workunit.client.1.smithi176.stdout:8/899: mknod d8/d20/c12c 0 2024-07-16T09:55:50.101 INFO:tasks.workunit.client.1.smithi176.stdout:0/968: truncate d5/d1c/d24/dfe/f122 740124 0 2024-07-16T09:55:50.104 INFO:tasks.workunit.client.0.smithi156.stdout:3/904: link d8/da/d1d/d2c/d7d/f108 d8/da/d28/d43/d71/f130 0 2024-07-16T09:55:50.105 INFO:tasks.workunit.client.1.smithi176.stdout:3/712: link d3/da/f41 d3/d17/fd7 0 2024-07-16T09:55:50.106 INFO:tasks.workunit.client.1.smithi176.stdout:8/900: mkdir d8/d23/d100/d31/d46/d12d 0 2024-07-16T09:55:50.108 INFO:tasks.workunit.client.0.smithi156.stdout:3/905: read d8/da/d1d/fc0 [298911,93866] 0 2024-07-16T09:55:50.109 INFO:tasks.workunit.client.1.smithi176.stdout:0/969: truncate d5/d1c/d24/d35/d50/d58/fb6 1227441 0 2024-07-16T09:55:50.111 INFO:tasks.workunit.client.0.smithi156.stdout:6/837: mkdir d3/d13/d17/d4f/d4d/d3b/d5a/dc1/d10e 0 2024-07-16T09:55:50.112 INFO:tasks.workunit.client.1.smithi176.stdout:3/713: creat d3/d17/fd8 x:0 0 0 2024-07-16T09:55:50.113 INFO:tasks.workunit.client.0.smithi156.stdout:3/906: symlink d8/da/d28/d40/d127/l131 0 2024-07-16T09:55:50.114 INFO:tasks.workunit.client.0.smithi156.stdout:6/838: rename d3/d13/d17/d4f/d4d to d3/d13/d17/d4f/d4d/d3b/d5a/dc1/d10e/d10f 22 2024-07-16T09:55:50.114 INFO:tasks.workunit.client.1.smithi176.stdout:8/901: chown d8/d23/d100/d31/d2f/lb2 8745 1 2024-07-16T09:55:50.116 INFO:tasks.workunit.client.0.smithi156.stdout:5/902: dread d1b/f31 [0,4194304] 0 2024-07-16T09:55:50.116 INFO:tasks.workunit.client.0.smithi156.stdout:3/907: fdatasync d8/da/d13/f110 0 2024-07-16T09:55:50.116 INFO:tasks.workunit.client.1.smithi176.stdout:3/714: unlink d3/d17/d83/fb0 0 2024-07-16T09:55:50.117 INFO:tasks.workunit.client.1.smithi176.stdout:0/970: dwrite d5/d25/d82/f87 [0,4194304] 0 2024-07-16T09:55:50.117 INFO:tasks.workunit.client.0.smithi156.stdout:6/839: write d3/d13/d17/d4f/d5c/f80 [36968,54254] 0 2024-07-16T09:55:50.118 INFO:tasks.workunit.client.1.smithi176.stdout:8/902: write d8/d14/d1e/d7c/fec [916355,71835] 0 2024-07-16T09:55:50.120 INFO:tasks.workunit.client.0.smithi156.stdout:3/908: chown d8/da/d1d/f53 3 1 2024-07-16T09:55:50.121 INFO:tasks.workunit.client.0.smithi156.stdout:6/840: dread d3/d13/d17/f55 [4194304,4194304] 0 2024-07-16T09:55:50.122 INFO:tasks.workunit.client.1.smithi176.stdout:0/971: symlink d5/d1c/d5a/d9e/l13e 0 2024-07-16T09:55:50.122 INFO:tasks.workunit.client.0.smithi156.stdout:0/957: chown d0/d1a/d76/d92/fcc 43696 1 2024-07-16T09:55:50.124 INFO:tasks.workunit.client.0.smithi156.stdout:8/866: truncate d0/d11/d13/d1c/ff2 1217237 0 2024-07-16T09:55:50.124 INFO:tasks.workunit.client.1.smithi176.stdout:8/903: mknod d8/d14/d1e/d7c/d82/df5/c12e 0 2024-07-16T09:55:50.124 INFO:tasks.workunit.client.1.smithi176.stdout:3/715: unlink d3/da/d33/f67 0 2024-07-16T09:55:50.125 INFO:tasks.workunit.client.0.smithi156.stdout:5/903: truncate d1b/d1f/d45/d52/d49/ff1 4877838 0 2024-07-16T09:55:50.125 INFO:tasks.workunit.client.1.smithi176.stdout:8/904: fsync d8/d14/d1e/d7c/d82/de0/ff0 0 2024-07-16T09:55:50.127 INFO:tasks.workunit.client.1.smithi176.stdout:0/972: getdents d5/d1c/d24/d35/d50/d7c/d11e 0 2024-07-16T09:55:50.127 INFO:tasks.workunit.client.1.smithi176.stdout:8/905: write d8/d14/d1e/d33/d9d/fb8 [1987504,47327] 0 2024-07-16T09:55:50.130 INFO:tasks.workunit.client.0.smithi156.stdout:6/841: creat d3/d13/d17/d4f/d94/d19/f110 x:0 0 0 2024-07-16T09:55:50.135 INFO:tasks.workunit.client.1.smithi176.stdout:8/906: dread d8/d23/d100/db9/dcb/d34/fcc [0,4194304] 0 2024-07-16T09:55:50.136 INFO:tasks.workunit.client.0.smithi156.stdout:7/773: dread dc/f94 [0,4194304] 0 2024-07-16T09:55:50.138 INFO:tasks.workunit.client.0.smithi156.stdout:6/842: rename d3/d13/d17/d4f/d30/f7f to d3/d13/d17/d4f/d30/d45/d99/f111 0 2024-07-16T09:55:50.138 INFO:tasks.workunit.client.1.smithi176.stdout:0/973: dwrite d5/d1c/d24/d35/d50/dd5/ff6 [0,4194304] 0 2024-07-16T09:55:50.139 INFO:tasks.workunit.client.0.smithi156.stdout:8/867: getdents d0/d11/d29/d5e 0 2024-07-16T09:55:50.143 INFO:tasks.workunit.client.0.smithi156.stdout:6/843: fsync d3/d13/d17/d4f/faf 0 2024-07-16T09:55:50.148 INFO:tasks.workunit.client.0.smithi156.stdout:3/909: dread d8/da/d28/d43/f4b [0,4194304] 0 2024-07-16T09:55:50.151 INFO:tasks.workunit.client.0.smithi156.stdout:6/844: write d3/d13/d17/d4f/d4d/d3b/d77/fd2 [2365801,127678] 0 2024-07-16T09:55:50.151 INFO:tasks.workunit.client.0.smithi156.stdout:8/868: dwrite d0/d11/d20/d3f/dbe/ff4 [0,4194304] 0 2024-07-16T09:55:50.155 INFO:tasks.workunit.client.0.smithi156.stdout:0/958: write d0/d1a/d4d/d9/f56 [3313028,119215] 0 2024-07-16T09:55:50.157 INFO:tasks.workunit.client.0.smithi156.stdout:3/910: write d8/da/d1d/f53 [3541407,38965] 0 2024-07-16T09:55:50.160 INFO:tasks.workunit.client.0.smithi156.stdout:5/904: dread d1b/d1f/d45/d7f/d87/fa7 [0,4194304] 0 2024-07-16T09:55:50.162 INFO:tasks.workunit.client.0.smithi156.stdout:0/959: symlink d0/d1a/d4d/d61/da1/l136 0 2024-07-16T09:55:50.165 INFO:tasks.workunit.client.0.smithi156.stdout:5/905: creat d1b/d1f/d42/de0/f11f x:0 0 0 2024-07-16T09:55:50.165 INFO:tasks.workunit.client.1.smithi176.stdout:0/974: write d5/d25/d82/f9a [2412465,93875] 0 2024-07-16T09:55:50.168 INFO:tasks.workunit.client.1.smithi176.stdout:0/975: symlink d5/d1c/d24/dfe/l13f 0 2024-07-16T09:55:50.170 INFO:tasks.workunit.client.0.smithi156.stdout:7/774: dwrite dc/d13/d36/f7a [0,4194304] 0 2024-07-16T09:55:50.170 INFO:tasks.workunit.client.0.smithi156.stdout:3/911: creat d8/da/d13/dca/f132 x:0 0 0 2024-07-16T09:55:50.171 INFO:tasks.workunit.client.1.smithi176.stdout:0/976: read d5/d39/f7e [469236,98326] 0 2024-07-16T09:55:50.172 INFO:tasks.workunit.client.0.smithi156.stdout:5/906: write d1b/d1f/d45/d52/d43/d85/d97/d37/d88/dc7/dd9/f116 [383237,105577] 0 2024-07-16T09:55:50.173 INFO:tasks.workunit.client.1.smithi176.stdout:0/977: symlink d5/d1c/d24/d35/d50/dd5/l140 0 2024-07-16T09:55:50.173 INFO:tasks.workunit.client.0.smithi156.stdout:3/912: fsync d8/da/d1d/d2c/f84 0 2024-07-16T09:55:50.175 INFO:tasks.workunit.client.0.smithi156.stdout:0/960: rename d0/d22/d41/d66/l12c to d0/d1a/d4d/d5e/db7/l137 0 2024-07-16T09:55:50.176 INFO:tasks.workunit.client.0.smithi156.stdout:6/845: link d3/d13/d17/d4f/d30/d45/cad d3/d13/c112 0 2024-07-16T09:55:50.177 INFO:tasks.workunit.client.1.smithi176.stdout:0/978: symlink d5/d1c/d24/d35/d50/d9b/de3/d117/l141 0 2024-07-16T09:55:50.178 INFO:tasks.workunit.client.1.smithi176.stdout:0/979: chown d5/l9 18137 1 2024-07-16T09:55:50.179 INFO:tasks.workunit.client.0.smithi156.stdout:0/961: chown d0/d1a/d76/d92/fcc 29333737 1 2024-07-16T09:55:50.180 INFO:tasks.workunit.client.1.smithi176.stdout:0/980: mknod d5/d44/c142 0 2024-07-16T09:55:50.181 INFO:tasks.workunit.client.1.smithi176.stdout:0/981: stat d5/d1c/d24/d35/d59/f8c 0 2024-07-16T09:55:50.182 INFO:tasks.workunit.client.0.smithi156.stdout:8/869: dwrite d0/f2c [0,4194304] 0 2024-07-16T09:55:50.183 INFO:tasks.workunit.client.0.smithi156.stdout:3/913: symlink d8/da/d1d/d2c/de0/dad/l133 0 2024-07-16T09:55:50.184 INFO:tasks.workunit.client.0.smithi156.stdout:0/962: symlink d0/d22/d41/d66/l138 0 2024-07-16T09:55:50.184 INFO:tasks.workunit.client.1.smithi176.stdout:0/982: creat d5/d1c/d24/d127/d84/f143 x:0 0 0 2024-07-16T09:55:50.188 INFO:tasks.workunit.client.0.smithi156.stdout:5/907: rename d1b/d1f/d45/faf to d1b/d1f/d42/de0/f120 0 2024-07-16T09:55:50.188 INFO:tasks.workunit.client.1.smithi176.stdout:0/983: unlink d5/d1c/d24/d35/d50/d58/d63/fd8 0 2024-07-16T09:55:50.190 INFO:tasks.workunit.client.0.smithi156.stdout:3/914: creat d8/da/d28/d43/dd8/f134 x:0 0 0 2024-07-16T09:55:50.193 INFO:tasks.workunit.client.0.smithi156.stdout:8/870: dwrite d0/d11/d13/d1c/fbd [0,4194304] 0 2024-07-16T09:55:50.197 INFO:tasks.workunit.client.0.smithi156.stdout:7/775: write dc/d13/d15/dfc/f6d [4731690,79991] 0 2024-07-16T09:55:50.198 INFO:tasks.workunit.client.0.smithi156.stdout:5/908: fdatasync d1b/d1f/d45/d52/d43/d85/d97/d37/d88/d61/f69 0 2024-07-16T09:55:50.200 INFO:tasks.workunit.client.0.smithi156.stdout:5/909: chown d1b/d1f/d45/d52/d49/ff1 2 1 2024-07-16T09:55:50.201 INFO:tasks.workunit.client.0.smithi156.stdout:0/963: symlink d0/d1a/d4d/d5e/db7/d113/l139 0 2024-07-16T09:55:50.204 INFO:tasks.workunit.client.0.smithi156.stdout:8/871: mknod d0/d11/d20/d7f/db7/d110/d1f/dee/c129 0 2024-07-16T09:55:50.208 INFO:tasks.workunit.client.0.smithi156.stdout:5/910: symlink d1b/d1f/d45/d52/d43/l121 0 2024-07-16T09:55:50.208 INFO:tasks.workunit.client.0.smithi156.stdout:0/964: creat d0/d1a/f13a x:0 0 0 2024-07-16T09:55:50.209 INFO:tasks.workunit.client.0.smithi156.stdout:5/911: write d1b/d1f/d45/d7f/f90 [6721699,39818] 0 2024-07-16T09:55:50.211 INFO:tasks.workunit.client.0.smithi156.stdout:8/872: chown d0/d11/d13/d1c/ff2 58 1 2024-07-16T09:55:50.213 INFO:tasks.workunit.client.0.smithi156.stdout:7/776: creat dc/d48/dd7/f110 x:0 0 0 2024-07-16T09:55:50.214 INFO:tasks.workunit.client.0.smithi156.stdout:5/912: dread d1b/d1f/d45/d52/d43/d85/d97/d37/d88/dc7/f107 [0,4194304] 0 2024-07-16T09:55:50.217 INFO:tasks.workunit.client.1.smithi176.stdout:0/984: dread d5/d25/d82/dec/f10a [0,4194304] 0 2024-07-16T09:55:50.217 INFO:tasks.workunit.client.0.smithi156.stdout:0/965: creat d0/d1a/d76/da7/dd3/f13b x:0 0 0 2024-07-16T09:55:50.219 INFO:tasks.workunit.client.0.smithi156.stdout:3/915: dwrite d8/d14/f6b [0,4194304] 0 2024-07-16T09:55:50.221 INFO:tasks.workunit.client.0.smithi156.stdout:6/846: getdents d3/d13/d17/d4f/d4d/d3b/da5/dbc 0 2024-07-16T09:55:50.222 INFO:tasks.workunit.client.0.smithi156.stdout:7/777: mknod dc/d3e/dac/c111 0 2024-07-16T09:55:50.223 INFO:tasks.workunit.client.0.smithi156.stdout:5/913: rmdir d1b/d1f/d54/dc6/d10e 39 2024-07-16T09:55:50.224 INFO:tasks.workunit.client.1.smithi176.stdout:0/985: dwrite d5/d1c/d24/d35/d50/dd5/fda [0,4194304] 0 2024-07-16T09:55:50.225 INFO:tasks.workunit.client.0.smithi156.stdout:8/873: getdents d0/d11/d20/d7f/db7/d110/d1f/d57/db3/df7 0 2024-07-16T09:55:50.227 INFO:tasks.workunit.client.1.smithi176.stdout:0/986: chown d5/d1c/d24/d35/d59/dc9/ffb 3763 1 2024-07-16T09:55:50.228 INFO:tasks.workunit.client.1.smithi176.stdout:0/987: chown d5/ce 271297402 1 2024-07-16T09:55:50.231 INFO:tasks.workunit.client.1.smithi176.stdout:0/988: unlink d5/d25/d82/dec/ccf 0 2024-07-16T09:55:50.234 INFO:tasks.workunit.client.1.smithi176.stdout:0/989: dread - d5/d1c/d24/d35/d50/d9b/de3/d117/d131/ffc zero size 2024-07-16T09:55:50.239 INFO:tasks.workunit.client.1.smithi176.stdout:0/990: getdents d5/d44 0 2024-07-16T09:55:50.241 INFO:tasks.workunit.client.1.smithi176.stdout:0/991: creat d5/d39/dde/f144 x:0 0 0 2024-07-16T09:55:50.241 INFO:tasks.workunit.client.1.smithi176.stdout:0/992: fdatasync d5/d1c/d24/d127/d84/f143 0 2024-07-16T09:55:50.244 INFO:tasks.workunit.client.1.smithi176.stdout:0/993: fsync d5/d1c/d24/d35/d50/d7c/f93 0 2024-07-16T09:55:50.246 INFO:tasks.workunit.client.1.smithi176.stdout:0/994: mkdir d5/d1c/d145 0 2024-07-16T09:55:50.252 INFO:tasks.workunit.client.1.smithi176.stdout:0/995: dwrite d5/d1c/d24/d35/d50/d7c/f109 [0,4194304] 0 2024-07-16T09:55:50.256 INFO:tasks.workunit.client.1.smithi176.stdout:0/996: unlink d5/ce 0 2024-07-16T09:55:50.258 INFO:tasks.workunit.client.1.smithi176.stdout:0/997: dread d5/d1c/d5a/f75 [0,4194304] 0 2024-07-16T09:55:50.259 INFO:tasks.workunit.client.1.smithi176.stdout:0/998: mknod d5/d44/c146 0 2024-07-16T09:55:50.262 INFO:tasks.workunit.client.1.smithi176.stdout:0/999: creat d5/d1c/d24/d35/d50/d9b/de3/d117/dbd/f147 x:0 0 0 2024-07-16T09:55:50.310 INFO:tasks.workunit.client.0.smithi156.stdout:3/916: rename d8/da/d1d/d2c/de0/f49 to d8/da/d1d/db2/d126/f135 0 2024-07-16T09:55:50.310 INFO:tasks.workunit.client.0.smithi156.stdout:6/847: creat d3/d13/d17/d4f/d94/d19/de5/f113 x:0 0 0 2024-07-16T09:55:50.310 INFO:tasks.workunit.client.0.smithi156.stdout:5/914: rmdir d1b/d1f/d45/d52/d43/d85/d97/d37/d88/dc7 39 2024-07-16T09:55:50.311 INFO:tasks.workunit.client.0.smithi156.stdout:5/915: readlink l15 0 2024-07-16T09:55:50.311 INFO:tasks.workunit.client.0.smithi156.stdout:6/848: chown d3/d13/d17/d4f/d94/cb7 0 1 2024-07-16T09:55:50.313 INFO:tasks.workunit.client.0.smithi156.stdout:6/849: dread d3/d13/d17/d4f/d5c/f80 [0,4194304] 0 2024-07-16T09:55:50.316 INFO:tasks.workunit.client.0.smithi156.stdout:7/778: mkdir dc/d3e/d3f/d51/d112 0 2024-07-16T09:55:50.317 INFO:tasks.workunit.client.0.smithi156.stdout:7/779: fdatasync dc/d63/fe6 0 2024-07-16T09:55:50.317 INFO:tasks.workunit.client.0.smithi156.stdout:3/917: fdatasync d8/d14/f55 0 2024-07-16T09:55:50.318 INFO:tasks.workunit.client.0.smithi156.stdout:6/850: stat d3/d13/d17/d4f/d94/d19/d31/d35/ldf 0 2024-07-16T09:55:50.319 INFO:tasks.workunit.client.0.smithi156.stdout:7/780: truncate dc/d3e/d3f/d51/f53 9255698 0 2024-07-16T09:55:50.320 INFO:tasks.workunit.client.0.smithi156.stdout:3/918: rmdir d8/da/d1d/d6f 39 2024-07-16T09:55:50.320 INFO:tasks.workunit.client.0.smithi156.stdout:7/781: chown dc/d13/d15/l58 96174405 1 2024-07-16T09:55:50.321 INFO:tasks.workunit.client.0.smithi156.stdout:6/851: mkdir d3/d13/d17/d4f/d30/d45/d99/d114 0 2024-07-16T09:55:50.323 INFO:tasks.workunit.client.0.smithi156.stdout:3/919: truncate d8/da/d28/d40/f54 143660 0 2024-07-16T09:55:50.323 INFO:tasks.workunit.client.1.smithi176.stdout:7/767: sync 2024-07-16T09:55:50.325 INFO:tasks.workunit.client.0.smithi156.stdout:7/782: dread dc/f94 [0,4194304] 0 2024-07-16T09:55:50.325 INFO:tasks.workunit.client.0.smithi156.stdout:3/920: write d8/da/d1d/d2c/f84 [1620893,107621] 0 2024-07-16T09:55:50.325 INFO:tasks.workunit.client.1.smithi176.stdout:7/768: dread - d4/dd/d26/dc2/d17/d1d/ddc/ff4 zero size 2024-07-16T09:55:50.327 INFO:tasks.workunit.client.1.smithi176.stdout:7/769: mknod d4/d47/d5d/cfb 0 2024-07-16T09:55:50.328 INFO:tasks.workunit.client.1.smithi176.stdout:7/770: rmdir d4/d1f 39 2024-07-16T09:55:50.333 INFO:tasks.workunit.client.1.smithi176.stdout:7/771: creat d4/d1f/d22/ffc x:0 0 0 2024-07-16T09:55:50.333 INFO:tasks.workunit.client.1.smithi176.stdout:7/772: chown d4/dd/d81 281751516 1 2024-07-16T09:55:50.334 INFO:tasks.workunit.client.1.smithi176.stdout:7/773: chown d4/d47/d6d/d89/d66/ccb 1110648 1 2024-07-16T09:55:50.335 INFO:tasks.workunit.client.1.smithi176.stdout:7/774: fsync d4/d1f/d22/fec 0 2024-07-16T09:55:50.335 INFO:tasks.workunit.client.0.smithi156.stdout:7/783: dwrite dc/d13/d20/f3c [0,4194304] 0 2024-07-16T09:55:50.336 INFO:tasks.workunit.client.1.smithi176.stdout:7/775: readlink d4/dd/d49/d7a/db1/lc7 0 2024-07-16T09:55:50.341 INFO:tasks.workunit.client.0.smithi156.stdout:3/921: dwrite d8/da/d28/f66 [4194304,4194304] 0 2024-07-16T09:55:50.342 INFO:tasks.workunit.client.0.smithi156.stdout:8/874: dwrite d0/d11/d20/d7f/db7/d110/d1f/d57/dc0/fe8 [0,4194304] 0 2024-07-16T09:55:50.342 INFO:tasks.workunit.client.0.smithi156.stdout:6/852: dread d3/d13/d17/d4f/d94/d19/d31/fa2 [0,4194304] 0 2024-07-16T09:55:50.346 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:55:50 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:55:50.346 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:55:50 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:55:50.346 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:55:50 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-16T09:55:50.346 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:55:50 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-16T09:55:50.346 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:55:50 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:55:50.346 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:55:50 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-16T09:55:50.346 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:55:50 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "mgr fail", "who": "smithi156.zkvylv"}]: dispatch 2024-07-16T09:55:50.346 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:55:50 smithi176 ceph-mon[36626]: Activating manager daemon smithi176.dqbrvx 2024-07-16T09:55:50.346 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:55:50 smithi176 ceph-mon[36626]: osdmap e47: 6 total, 6 up, 6 in 2024-07-16T09:55:50.346 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:55:50 smithi176 ceph-mon[36626]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd='[{"prefix": "mgr fail", "who": "smithi156.zkvylv"}]': finished 2024-07-16T09:55:50.346 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:55:50 smithi176 ceph-mon[36626]: mgrmap e21: smithi176.dqbrvx(active, starting, since 0.0143425s) 2024-07-16T09:55:50.347 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:55:50 smithi176 ceph-mon[36626]: from='mgr.24377 172.21.15.176:0/1082993331' entity='mgr.smithi176.dqbrvx' cmd=[{"prefix": "mon metadata", "id": "smithi156"}]: dispatch 2024-07-16T09:55:50.347 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:55:50 smithi176 ceph-mon[36626]: from='mgr.24377 172.21.15.176:0/1082993331' entity='mgr.smithi176.dqbrvx' cmd=[{"prefix": "mon metadata", "id": "smithi176"}]: dispatch 2024-07-16T09:55:50.347 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:55:50 smithi176 ceph-mon[36626]: from='mgr.24377 172.21.15.176:0/1082993331' entity='mgr.smithi176.dqbrvx' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi156.rdibys"}]: dispatch 2024-07-16T09:55:50.347 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:55:50 smithi176 ceph-mon[36626]: from='mgr.24377 172.21.15.176:0/1082993331' entity='mgr.smithi176.dqbrvx' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi156.evwwaz"}]: dispatch 2024-07-16T09:55:50.347 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:55:50 smithi176 ceph-mon[36626]: from='mgr.24377 172.21.15.176:0/1082993331' entity='mgr.smithi176.dqbrvx' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi176.dvvbud"}]: dispatch 2024-07-16T09:55:50.347 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:55:50 smithi176 ceph-mon[36626]: from='mgr.24377 172.21.15.176:0/1082993331' entity='mgr.smithi176.dqbrvx' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi176.wahbsi"}]: dispatch 2024-07-16T09:55:50.347 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:55:50 smithi176 ceph-mon[36626]: from='mgr.24377 172.21.15.176:0/1082993331' entity='mgr.smithi176.dqbrvx' cmd=[{"prefix": "mgr metadata", "who": "smithi176.dqbrvx", "id": "smithi176.dqbrvx"}]: dispatch 2024-07-16T09:55:50.347 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:55:50 smithi176 ceph-mon[36626]: from='mgr.24377 172.21.15.176:0/1082993331' entity='mgr.smithi176.dqbrvx' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-07-16T09:55:50.347 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:55:50 smithi176 ceph-mon[36626]: from='mgr.24377 172.21.15.176:0/1082993331' entity='mgr.smithi176.dqbrvx' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-07-16T09:55:50.347 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:55:50 smithi176 ceph-mon[36626]: from='mgr.24377 172.21.15.176:0/1082993331' entity='mgr.smithi176.dqbrvx' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-07-16T09:55:50.347 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:55:50 smithi176 ceph-mon[36626]: from='mgr.24377 172.21.15.176:0/1082993331' entity='mgr.smithi176.dqbrvx' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-07-16T09:55:50.347 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:55:50 smithi176 ceph-mon[36626]: from='mgr.24377 172.21.15.176:0/1082993331' entity='mgr.smithi176.dqbrvx' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-07-16T09:55:50.347 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:55:50 smithi176 ceph-mon[36626]: from='mgr.24377 172.21.15.176:0/1082993331' entity='mgr.smithi176.dqbrvx' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-07-16T09:55:50.348 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:55:50 smithi176 ceph-mon[36626]: from='mgr.24377 172.21.15.176:0/1082993331' entity='mgr.smithi176.dqbrvx' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-07-16T09:55:50.348 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:55:50 smithi176 ceph-mon[36626]: from='mgr.24377 172.21.15.176:0/1082993331' entity='mgr.smithi176.dqbrvx' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-07-16T09:55:50.348 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:55:50 smithi176 ceph-mon[36626]: from='mgr.24377 172.21.15.176:0/1082993331' entity='mgr.smithi176.dqbrvx' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-07-16T09:55:50.348 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:55:50 smithi176 ceph-mon[36626]: Manager daemon smithi176.dqbrvx is now available 2024-07-16T09:55:50.348 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:55:50 smithi176 ceph-mon[36626]: from='mgr.24377 ' entity='mgr.smithi176.dqbrvx' 2024-07-16T09:55:50.348 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:55:50 smithi176 ceph-mon[36626]: from='mgr.24377 ' entity='mgr.smithi176.dqbrvx' 2024-07-16T09:55:50.348 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:55:50 smithi176 ceph-mon[36626]: from='mgr.24377 ' entity='mgr.smithi176.dqbrvx' 2024-07-16T09:55:50.348 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:55:50 smithi176 ceph-mon[36626]: from='mgr.24377 172.21.15.176:0/1082993331' entity='mgr.smithi176.dqbrvx' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-16T09:55:50.348 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:55:50 smithi176 ceph-mon[36626]: from='mgr.24377 172.21.15.176:0/1082993331' entity='mgr.smithi176.dqbrvx' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-16T09:55:50.348 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:55:50 smithi176 ceph-mon[36626]: from='mgr.24377 172.21.15.176:0/1082993331' entity='mgr.smithi176.dqbrvx' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi176.dqbrvx/mirror_snapshot_schedule"}]: dispatch 2024-07-16T09:55:50.348 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:55:50 smithi176 ceph-mon[36626]: from='mgr.24377 ' entity='mgr.smithi176.dqbrvx' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi176.dqbrvx/mirror_snapshot_schedule"}]: dispatch 2024-07-16T09:55:50.348 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:55:50 smithi176 ceph-mon[36626]: from='mgr.24377 172.21.15.176:0/1082993331' entity='mgr.smithi176.dqbrvx' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi176.dqbrvx/trash_purge_schedule"}]: dispatch 2024-07-16T09:55:50.349 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:55:50 smithi176 ceph-mon[36626]: from='mgr.24377 ' entity='mgr.smithi176.dqbrvx' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi176.dqbrvx/trash_purge_schedule"}]: dispatch 2024-07-16T09:55:50.349 INFO:tasks.workunit.client.0.smithi156.stdout:8/875: chown d0/d11/d20/d7f/db7/d110/d1f/d57/c102 5 1 2024-07-16T09:55:50.349 INFO:tasks.workunit.client.0.smithi156.stdout:8/876: chown d0/d11/ded/f10c 66304665 1 2024-07-16T09:55:50.349 INFO:tasks.workunit.client.0.smithi156.stdout:3/922: write d8/da/d1d/d6f/fd1 [4810727,35152] 0 2024-07-16T09:55:50.351 INFO:tasks.workunit.client.0.smithi156.stdout:6/853: mknod d3/d13/d17/dae/c115 0 2024-07-16T09:55:50.352 INFO:tasks.workunit.client.0.smithi156.stdout:8/877: chown d0/d11/d2e/d124/ccb 12960 1 2024-07-16T09:55:50.352 INFO:tasks.workunit.client.0.smithi156.stdout:7/784: creat dc/d3e/f113 x:0 0 0 2024-07-16T09:55:50.358 INFO:tasks.workunit.client.0.smithi156.stdout:6/854: dread d3/d13/d17/d4f/d94/d19/d31/fa2 [0,4194304] 0 2024-07-16T09:55:50.360 INFO:tasks.workunit.client.0.smithi156.stdout:6/855: fsync d3/d13/d17/d4f/d94/d19/d2e/f105 0 2024-07-16T09:55:50.362 INFO:tasks.workunit.client.0.smithi156.stdout:6/856: write d3/d13/d17/d4f/d1e/fe3 [1004842,8777] 0 2024-07-16T09:55:50.362 INFO:tasks.workunit.client.0.smithi156.stdout:7/785: rename dc/d3e/d5d to dc/d13/d36/dd4/d114 0 2024-07-16T09:55:50.362 INFO:tasks.workunit.client.1.smithi176.stdout:7/776: dread d4/d1f/f59 [0,4194304] 0 2024-07-16T09:55:50.365 INFO:tasks.workunit.client.1.smithi176.stdout:7/777: write d4/d47/d5d/fad [3557563,128320] 0 2024-07-16T09:55:50.365 INFO:tasks.workunit.client.0.smithi156.stdout:7/786: stat dc/d13/d36/d56/fa2 0 2024-07-16T09:55:50.365 INFO:tasks.workunit.client.0.smithi156.stdout:8/878: truncate d0/d11/d20/d7f/db7/d110/d1f/d57/f7e 576132 0 2024-07-16T09:55:50.367 INFO:tasks.workunit.client.1.smithi176.stdout:7/778: creat d4/d47/d5d/d9a/ffd x:0 0 0 2024-07-16T09:55:50.368 INFO:tasks.workunit.client.0.smithi156.stdout:6/857: creat d3/d13/d17/d4f/d4d/d3b/da5/f116 x:0 0 0 2024-07-16T09:55:50.370 INFO:tasks.workunit.client.1.smithi176.stdout:7/779: mknod d4/d47/cfe 0 2024-07-16T09:55:50.372 INFO:tasks.workunit.client.0.smithi156.stdout:7/787: dread dc/d63/f6b [0,4194304] 0 2024-07-16T09:55:50.373 INFO:tasks.workunit.client.1.smithi176.stdout:7/780: mkdir d4/dd/d49/d7a/db1/dff 0 2024-07-16T09:55:50.374 INFO:tasks.workunit.client.1.smithi176.stdout:7/781: read - d4/d47/d5d/d9a/ffd zero size 2024-07-16T09:55:50.375 INFO:tasks.workunit.client.0.smithi156.stdout:7/788: mkdir dc/d3e/d76/df1/d115 0 2024-07-16T09:55:50.375 INFO:tasks.workunit.client.1.smithi176.stdout:7/782: write d4/dd/d26/dc2/d17/d1d/f73 [267669,12851] 0 2024-07-16T09:55:50.377 INFO:tasks.workunit.client.1.smithi176.stdout:7/783: chown d4/dd/d26/dc2/d17/d1d/cde 74922 1 2024-07-16T09:55:50.380 INFO:tasks.workunit.client.0.smithi156.stdout:3/923: dread d8/da/d28/d43/f5a [0,4194304] 0 2024-07-16T09:55:50.381 INFO:tasks.workunit.client.0.smithi156.stdout:7/789: dread dc/d48/db2/fe7 [0,4194304] 0 2024-07-16T09:55:50.381 INFO:tasks.workunit.client.1.smithi176.stdout:7/784: creat d4/dd/d26/dc2/d17/f100 x:0 0 0 2024-07-16T09:55:50.382 INFO:tasks.workunit.client.1.smithi176.stdout:7/785: write d4/d1f/d22/f52 [3485896,67457] 0 2024-07-16T09:55:50.384 INFO:tasks.workunit.client.0.smithi156.stdout:8/879: dread d0/d11/d20/d7f/db7/d110/f10 [0,4194304] 0 2024-07-16T09:55:50.385 INFO:tasks.workunit.client.0.smithi156.stdout:6/858: dwrite d3/d13/d17/f55 [0,4194304] 0 2024-07-16T09:55:50.386 INFO:tasks.workunit.client.0.smithi156.stdout:3/924: rmdir d8/da/d28/d43/d71 39 2024-07-16T09:55:50.387 INFO:tasks.workunit.client.1.smithi176.stdout:7/786: unlink d4/dd/d26/dc2/d17/d1d/d8b/cf1 0 2024-07-16T09:55:50.388 INFO:tasks.workunit.client.1.smithi176.stdout:7/787: dread - d4/d1f/d22/ffc zero size 2024-07-16T09:55:50.388 INFO:tasks.workunit.client.0.smithi156.stdout:7/790: dread dc/d13/d15/f78 [0,4194304] 0 2024-07-16T09:55:50.388 INFO:tasks.workunit.client.0.smithi156.stdout:3/925: stat d8/da/dec/df1/f12b 0 2024-07-16T09:55:50.390 INFO:tasks.workunit.client.0.smithi156.stdout:7/791: mknod dc/d13/d24/c116 0 2024-07-16T09:55:50.393 INFO:tasks.workunit.client.0.smithi156.stdout:7/792: chown dc/d3e/d3f/ff5 152 1 2024-07-16T09:55:50.394 INFO:tasks.workunit.client.0.smithi156.stdout:3/926: mkdir d8/da/d21/d80/d99/d136 0 2024-07-16T09:55:50.395 INFO:tasks.workunit.client.0.smithi156.stdout:8/880: truncate d0/d11/d29/d80/d97/de7/fd6 3806489 0 2024-07-16T09:55:50.396 INFO:tasks.workunit.client.0.smithi156.stdout:7/793: mkdir dc/d48/dd7/d117 0 2024-07-16T09:55:50.402 INFO:tasks.workunit.client.0.smithi156.stdout:8/881: dread d0/d11/d13/d1c/ff2 [0,4194304] 0 2024-07-16T09:55:50.403 INFO:tasks.workunit.client.0.smithi156.stdout:8/882: stat d0/d11/d20/d3f/d96/fad 0 2024-07-16T09:55:50.405 INFO:tasks.workunit.client.0.smithi156.stdout:3/927: write d8/da/d1d/db2/d126/f135 [4891715,24001] 0 2024-07-16T09:55:50.407 INFO:tasks.workunit.client.0.smithi156.stdout:8/883: truncate d0/d11/d20/d3f/d96/fad 76947 0 2024-07-16T09:55:50.408 INFO:tasks.workunit.client.1.smithi176.stdout:7/788: dread d4/d47/d6d/d89/d66/f8a [0,4194304] 0 2024-07-16T09:55:50.414 INFO:tasks.workunit.client.0.smithi156.stdout:8/884: dwrite d0/d11/d29/d80/d97/de7/f7d [0,4194304] 0 2024-07-16T09:55:50.419 INFO:tasks.workunit.client.0.smithi156.stdout:3/928: dwrite d8/da/d1d/fc0 [0,4194304] 0 2024-07-16T09:55:50.421 INFO:tasks.workunit.client.0.smithi156.stdout:7/794: dread dc/d13/f35 [0,4194304] 0 2024-07-16T09:55:50.422 INFO:tasks.workunit.client.0.smithi156.stdout:8/885: write d0/d11/d62/fbc [798323,116081] 0 2024-07-16T09:55:50.423 INFO:tasks.workunit.client.1.smithi176.stdout:7/789: dread d4/dd/f1a [0,4194304] 0 2024-07-16T09:55:50.424 INFO:tasks.workunit.client.1.smithi176.stdout:7/790: dread - d4/dd/d81/fd9 zero size 2024-07-16T09:55:50.425 INFO:tasks.workunit.client.1.smithi176.stdout:7/791: chown d4/d47/d6d/d89/d66 0 1 2024-07-16T09:55:50.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:55:50 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:55:50.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:55:50 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:55:50.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:55:50 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-16T09:55:50.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:55:50 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-16T09:55:50.426 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:55:50 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' 2024-07-16T09:55:50.426 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:55:50 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-16T09:55:50.426 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:55:50 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "mgr fail", "who": "smithi156.zkvylv"}]: dispatch 2024-07-16T09:55:50.426 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:55:50 smithi156 ceph-mon[29302]: Activating manager daemon smithi176.dqbrvx 2024-07-16T09:55:50.426 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:55:50 smithi156 ceph-mon[29302]: osdmap e47: 6 total, 6 up, 6 in 2024-07-16T09:55:50.426 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:55:50 smithi156 ceph-mon[29302]: from='mgr.14208 172.21.15.156:0/3450835985' entity='mgr.smithi156.zkvylv' cmd='[{"prefix": "mgr fail", "who": "smithi156.zkvylv"}]': finished 2024-07-16T09:55:50.426 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:55:50 smithi156 ceph-mon[29302]: mgrmap e21: smithi176.dqbrvx(active, starting, since 0.0143425s) 2024-07-16T09:55:50.426 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:55:50 smithi156 ceph-mon[29302]: from='mgr.24377 172.21.15.176:0/1082993331' entity='mgr.smithi176.dqbrvx' cmd=[{"prefix": "mon metadata", "id": "smithi156"}]: dispatch 2024-07-16T09:55:50.426 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:55:50 smithi156 ceph-mon[29302]: from='mgr.24377 172.21.15.176:0/1082993331' entity='mgr.smithi176.dqbrvx' cmd=[{"prefix": "mon metadata", "id": "smithi176"}]: dispatch 2024-07-16T09:55:50.426 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:55:50 smithi156 ceph-mon[29302]: from='mgr.24377 172.21.15.176:0/1082993331' entity='mgr.smithi176.dqbrvx' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi156.rdibys"}]: dispatch 2024-07-16T09:55:50.426 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:55:50 smithi156 ceph-mon[29302]: from='mgr.24377 172.21.15.176:0/1082993331' entity='mgr.smithi176.dqbrvx' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi156.evwwaz"}]: dispatch 2024-07-16T09:55:50.426 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:55:50 smithi156 ceph-mon[29302]: from='mgr.24377 172.21.15.176:0/1082993331' entity='mgr.smithi176.dqbrvx' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi176.dvvbud"}]: dispatch 2024-07-16T09:55:50.427 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:55:50 smithi156 ceph-mon[29302]: from='mgr.24377 172.21.15.176:0/1082993331' entity='mgr.smithi176.dqbrvx' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi176.wahbsi"}]: dispatch 2024-07-16T09:55:50.427 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:55:50 smithi156 ceph-mon[29302]: from='mgr.24377 172.21.15.176:0/1082993331' entity='mgr.smithi176.dqbrvx' cmd=[{"prefix": "mgr metadata", "who": "smithi176.dqbrvx", "id": "smithi176.dqbrvx"}]: dispatch 2024-07-16T09:55:50.427 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:55:50 smithi156 ceph-mon[29302]: from='mgr.24377 172.21.15.176:0/1082993331' entity='mgr.smithi176.dqbrvx' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-07-16T09:55:50.427 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:55:50 smithi156 ceph-mon[29302]: from='mgr.24377 172.21.15.176:0/1082993331' entity='mgr.smithi176.dqbrvx' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-07-16T09:55:50.427 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:55:50 smithi156 ceph-mon[29302]: from='mgr.24377 172.21.15.176:0/1082993331' entity='mgr.smithi176.dqbrvx' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-07-16T09:55:50.427 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:55:50 smithi156 ceph-mon[29302]: from='mgr.24377 172.21.15.176:0/1082993331' entity='mgr.smithi176.dqbrvx' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-07-16T09:55:50.427 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:55:50 smithi156 ceph-mon[29302]: from='mgr.24377 172.21.15.176:0/1082993331' entity='mgr.smithi176.dqbrvx' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-07-16T09:55:50.427 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:55:50 smithi156 ceph-mon[29302]: from='mgr.24377 172.21.15.176:0/1082993331' entity='mgr.smithi176.dqbrvx' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-07-16T09:55:50.427 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:55:50 smithi156 ceph-mon[29302]: from='mgr.24377 172.21.15.176:0/1082993331' entity='mgr.smithi176.dqbrvx' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-07-16T09:55:50.427 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:55:50 smithi156 ceph-mon[29302]: from='mgr.24377 172.21.15.176:0/1082993331' entity='mgr.smithi176.dqbrvx' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-07-16T09:55:50.427 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:55:50 smithi156 ceph-mon[29302]: from='mgr.24377 172.21.15.176:0/1082993331' entity='mgr.smithi176.dqbrvx' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-07-16T09:55:50.427 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:55:50 smithi156 ceph-mon[29302]: Manager daemon smithi176.dqbrvx is now available 2024-07-16T09:55:50.428 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:55:50 smithi156 ceph-mon[29302]: from='mgr.24377 ' entity='mgr.smithi176.dqbrvx' 2024-07-16T09:55:50.428 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:55:50 smithi156 ceph-mon[29302]: from='mgr.24377 ' entity='mgr.smithi176.dqbrvx' 2024-07-16T09:55:50.428 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:55:50 smithi156 ceph-mon[29302]: from='mgr.24377 ' entity='mgr.smithi176.dqbrvx' 2024-07-16T09:55:50.428 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:55:50 smithi156 ceph-mon[29302]: from='mgr.24377 172.21.15.176:0/1082993331' entity='mgr.smithi176.dqbrvx' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-16T09:55:50.428 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:55:50 smithi156 ceph-mon[29302]: from='mgr.24377 172.21.15.176:0/1082993331' entity='mgr.smithi176.dqbrvx' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-16T09:55:50.428 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:55:50 smithi156 ceph-mon[29302]: from='mgr.24377 172.21.15.176:0/1082993331' entity='mgr.smithi176.dqbrvx' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi176.dqbrvx/mirror_snapshot_schedule"}]: dispatch 2024-07-16T09:55:50.428 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:55:50 smithi156 ceph-mon[29302]: from='mgr.24377 ' entity='mgr.smithi176.dqbrvx' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi176.dqbrvx/mirror_snapshot_schedule"}]: dispatch 2024-07-16T09:55:50.428 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:55:50 smithi156 ceph-mon[29302]: from='mgr.24377 172.21.15.176:0/1082993331' entity='mgr.smithi176.dqbrvx' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi176.dqbrvx/trash_purge_schedule"}]: dispatch 2024-07-16T09:55:50.428 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:55:50 smithi156 ceph-mon[29302]: from='mgr.24377 ' entity='mgr.smithi176.dqbrvx' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi176.dqbrvx/trash_purge_schedule"}]: dispatch 2024-07-16T09:55:50.429 INFO:tasks.workunit.client.1.smithi176.stdout:7/792: write d4/dd/d26/dc2/d17/d1d/f5b [524068,99749] 0 2024-07-16T09:55:50.429 INFO:tasks.workunit.client.0.smithi156.stdout:8/886: read d0/f1 [6916381,9036] 0 2024-07-16T09:55:50.429 INFO:tasks.workunit.client.0.smithi156.stdout:7/795: rename dc/d13/d20/c21 to dc/d13/d36/d56/c118 0 2024-07-16T09:55:50.430 INFO:tasks.workunit.client.0.smithi156.stdout:3/929: dwrite d8/da/d28/f66 [4194304,4194304] 0 2024-07-16T09:55:50.430 INFO:tasks.workunit.client.0.smithi156.stdout:8/887: read d0/d11/d20/d7f/db7/d110/f3b [251863,50029] 0 2024-07-16T09:55:50.440 INFO:tasks.workunit.client.0.smithi156.stdout:3/930: dwrite d8/da/d28/d43/fbe [0,4194304] 0 2024-07-16T09:55:50.441 INFO:tasks.workunit.client.0.smithi156.stdout:8/888: read d0/d11/d20/f7a [270957,65953] 0 2024-07-16T09:55:50.447 INFO:tasks.workunit.client.1.smithi176.stdout:7/793: dwrite d4/fb [0,4194304] 0 2024-07-16T09:55:50.456 INFO:tasks.workunit.client.1.smithi176.stdout:7/794: dwrite d4/dd/d26/dc2/d17/d1d/f9b [0,4194304] 0 2024-07-16T09:55:50.462 INFO:tasks.workunit.client.1.smithi176.stdout:7/795: dwrite d4/d5/f2f [4194304,4194304] 0 2024-07-16T09:55:50.465 INFO:tasks.workunit.client.1.smithi176.stdout:7/796: mknod d4/dd/d26/dc2/d17/d1d/ddc/c101 0 2024-07-16T09:55:50.478 INFO:tasks.workunit.client.1.smithi176.stdout:6/973: sync 2024-07-16T09:55:50.479 INFO:tasks.workunit.client.1.smithi176.stdout:6/974: read - d1/d5/fa9 zero size 2024-07-16T09:55:50.486 INFO:tasks.workunit.client.1.smithi176.stdout:6/975: dwrite d1/d5d/d9e/f110 [0,4194304] 0 2024-07-16T09:55:50.488 INFO:tasks.workunit.client.1.smithi176.stdout:6/976: chown d1/d5/d47/d62/de8/d111/d122/d12e 0 1 2024-07-16T09:55:50.488 INFO:tasks.workunit.client.0.smithi156.stdout:7/796: rename dc/d13/d36/d56 to dc/d48/d119 0 2024-07-16T09:55:50.490 INFO:tasks.workunit.client.0.smithi156.stdout:8/889: fdatasync d0/d11/d20/f7a 0 2024-07-16T09:55:50.491 INFO:tasks.workunit.client.0.smithi156.stdout:7/797: truncate dc/d3e/f113 32986 0 2024-07-16T09:55:50.493 INFO:tasks.workunit.client.1.smithi176.stdout:7/797: dread d4/d1f/d22/f52 [0,4194304] 0 2024-07-16T09:55:50.495 INFO:tasks.workunit.client.1.smithi176.stdout:6/977: link d1/d5/d7/d54/f113 d1/d5/f15e 0 2024-07-16T09:55:50.495 INFO:tasks.workunit.client.0.smithi156.stdout:3/931: link d8/da/d28/d40/f54 d8/f137 0 2024-07-16T09:55:50.496 INFO:tasks.workunit.client.1.smithi176.stdout:7/798: rmdir d4/d47/d6d/d89/d66 39 2024-07-16T09:55:50.499 INFO:tasks.workunit.client.1.smithi176.stdout:6/978: creat d1/d5/d47/d62/d77/dd2/f15f x:0 0 0 2024-07-16T09:55:50.499 INFO:tasks.workunit.client.1.smithi176.stdout:7/799: truncate d4/d5/d97/fee 485515 0 2024-07-16T09:55:50.501 INFO:tasks.workunit.client.1.smithi176.stdout:7/800: creat d4/dd/db5/f102 x:0 0 0 2024-07-16T09:55:50.506 INFO:tasks.workunit.client.1.smithi176.stdout:6/979: dwrite d1/d5d/f116 [0,4194304] 0 2024-07-16T09:55:50.510 INFO:tasks.workunit.client.0.smithi156.stdout:7/798: dwrite dc/d3e/d3f/fbf [0,4194304] 0 2024-07-16T09:55:50.511 INFO:tasks.workunit.client.1.smithi176.stdout:6/980: unlink d1/d5/d47/d62/d80/lda 0 2024-07-16T09:55:50.511 INFO:tasks.workunit.client.0.smithi156.stdout:3/932: dread d8/da/d1d/d2c/de0/dad/feb [0,4194304] 0 2024-07-16T09:55:50.515 INFO:tasks.workunit.client.0.smithi156.stdout:7/799: mkdir dc/d13/d36/dd4/d114/d11a 0 2024-07-16T09:55:50.516 INFO:tasks.workunit.client.0.smithi156.stdout:3/933: truncate d8/da/d21/d80/fa8 4136656 0 2024-07-16T09:55:50.521 INFO:tasks.workunit.client.0.smithi156.stdout:3/934: read d8/da/d1d/d2c/f8e [3760956,42686] 0 2024-07-16T09:55:50.522 INFO:tasks.workunit.client.0.smithi156.stdout:7/800: dwrite dc/d3e/d3f/d51/fb0 [0,4194304] 0 2024-07-16T09:55:50.525 INFO:tasks.workunit.client.0.smithi156.stdout:3/935: write d8/da/d13/f57 [1959305,45603] 0 2024-07-16T09:55:50.529 INFO:tasks.workunit.client.0.smithi156.stdout:7/801: link dc/d13/d15/dfc/ldc dc/d3e/d76/l11b 0 2024-07-16T09:55:50.529 INFO:tasks.workunit.client.0.smithi156.stdout:3/936: read d8/da/d28/d40/f54 [53207,80333] 0 2024-07-16T09:55:50.533 INFO:tasks.workunit.client.0.smithi156.stdout:3/937: mkdir d8/da/d1d/db2/d126/d138 0 2024-07-16T09:55:50.539 INFO:tasks.workunit.client.0.smithi156.stdout:3/938: link d8/da/d1d/l117 d8/da/d1d/d2c/dee/l139 0 2024-07-16T09:55:50.541 INFO:tasks.workunit.client.1.smithi176.stdout:6/981: dread d1/d5d/d9e/d76/f5c [0,4194304] 0 2024-07-16T09:55:50.542 INFO:tasks.workunit.client.1.smithi176.stdout:7/801: dread d4/d1f/d22/f3a [0,4194304] 0 2024-07-16T09:55:50.543 INFO:tasks.workunit.client.0.smithi156.stdout:3/939: creat d8/da/dec/df1/f13a x:0 0 0 2024-07-16T09:55:50.544 INFO:tasks.workunit.client.1.smithi176.stdout:7/802: write d4/d1f/dce/fe8 [930474,31571] 0 2024-07-16T09:55:50.544 INFO:tasks.workunit.client.0.smithi156.stdout:7/802: rmdir dc/d13 39 2024-07-16T09:55:50.547 INFO:tasks.workunit.client.0.smithi156.stdout:7/803: rmdir dc/d48/dd7 39 2024-07-16T09:55:50.547 INFO:tasks.workunit.client.1.smithi176.stdout:7/803: getdents d4/d47/d6d 0 2024-07-16T09:55:50.549 INFO:tasks.workunit.client.1.smithi176.stdout:6/982: write d1/d5/f2c [1174781,8147] 0 2024-07-16T09:55:50.551 INFO:tasks.workunit.client.1.smithi176.stdout:7/804: rename d4/dd/d26/dc2/fc3 to d4/d47/d5d/d9a/de0/f103 0 2024-07-16T09:55:50.552 INFO:tasks.workunit.client.0.smithi156.stdout:3/940: rename d8/da/d1d/d2c/de0/dad/f104 to d8/da/d28/f13b 0 2024-07-16T09:55:50.553 INFO:tasks.workunit.client.1.smithi176.stdout:6/983: mknod d1/d5d/d9e/dd9/de7/c160 0 2024-07-16T09:55:50.554 INFO:tasks.workunit.client.0.smithi156.stdout:7/804: mknod dc/d13/c11c 0 2024-07-16T09:55:50.555 INFO:tasks.workunit.client.1.smithi176.stdout:6/984: write d1/d5d/db9/fb6 [4714001,126676] 0 2024-07-16T09:55:50.555 INFO:tasks.workunit.client.0.smithi156.stdout:7/805: dread - dc/d48/dd7/f110 zero size 2024-07-16T09:55:50.557 INFO:tasks.workunit.client.0.smithi156.stdout:7/806: write dc/d63/fe6 [395088,60279] 0 2024-07-16T09:55:50.557 INFO:tasks.workunit.client.0.smithi156.stdout:3/941: symlink d8/da/d28/d43/d71/da2/d114/l13c 0 2024-07-16T09:55:50.558 INFO:tasks.workunit.client.0.smithi156.stdout:3/942: chown d8/da/d28/d43/d63 0 1 2024-07-16T09:55:50.559 INFO:tasks.workunit.client.1.smithi176.stdout:6/985: creat d1/d5d/db9/d101/f161 x:0 0 0 2024-07-16T09:55:50.561 INFO:tasks.workunit.client.1.smithi176.stdout:6/986: mknod d1/d5d/db9/c162 0 2024-07-16T09:55:50.562 INFO:tasks.workunit.client.0.smithi156.stdout:7/807: link dc/d13/c11c dc/d13/d24/c11d 0 2024-07-16T09:55:50.562 INFO:tasks.workunit.client.1.smithi176.stdout:7/805: dwrite d4/d47/d6d/d89/d66/f8a [0,4194304] 0 2024-07-16T09:55:50.564 INFO:tasks.workunit.client.0.smithi156.stdout:3/943: getdents d8/da/d13/dca 0 2024-07-16T09:55:50.565 INFO:tasks.workunit.client.1.smithi176.stdout:7/806: truncate d4/d1f/d22/fec 392660 0 2024-07-16T09:55:50.566 INFO:tasks.workunit.client.0.smithi156.stdout:3/944: fsync d8/da/d28/d43/fe2 0 2024-07-16T09:55:50.572 INFO:tasks.workunit.client.0.smithi156.stdout:3/945: chown d8/da/d21/l68 0 1 2024-07-16T09:55:50.572 INFO:tasks.workunit.client.0.smithi156.stdout:7/808: link dc/d3e/d3f/d51/cf4 dc/d48/db2/c11e 0 2024-07-16T09:55:50.572 INFO:tasks.workunit.client.0.smithi156.stdout:3/946: chown d8/da/d1d/d2c/de0 59786 1 2024-07-16T09:55:50.573 INFO:tasks.workunit.client.0.smithi156.stdout:3/947: chown d8/da/d21/d80/fea 155 1 2024-07-16T09:55:50.574 INFO:tasks.workunit.client.1.smithi176.stdout:7/807: truncate d4/d5/d97/fe6 946942 0 2024-07-16T09:55:50.575 INFO:tasks.workunit.client.0.smithi156.stdout:3/948: write d8/da/d1d/d2c/de0/fb0 [165013,105684] 0 2024-07-16T09:55:50.575 INFO:tasks.workunit.client.1.smithi176.stdout:7/808: truncate d4/dd/d26/dc2/fe7 914720 0 2024-07-16T09:55:50.579 INFO:tasks.workunit.client.0.smithi156.stdout:3/949: write d8/d14/f1c [3293975,51419] 0 2024-07-16T09:55:50.580 INFO:tasks.workunit.client.0.smithi156.stdout:7/809: rename dc/d13/d15/l1e to dc/d13/d15/d8b/dcc/dde/l11f 0 2024-07-16T09:55:50.580 INFO:tasks.workunit.client.0.smithi156.stdout:3/950: stat d8/da/d1d/d2c/de0 0 2024-07-16T09:55:50.582 INFO:tasks.workunit.client.0.smithi156.stdout:7/810: unlink dc/c26 0 2024-07-16T09:55:50.583 INFO:tasks.workunit.client.1.smithi176.stdout:6/987: dread d1/f3b [0,4194304] 0 2024-07-16T09:55:50.591 INFO:tasks.workunit.client.1.smithi176.stdout:6/988: dread d1/d5d/d9e/d8f/fd5 [0,4194304] 0 2024-07-16T09:55:50.593 INFO:tasks.workunit.client.0.smithi156.stdout:3/951: dwrite d8/d14/f55 [0,4194304] 0 2024-07-16T09:55:50.597 INFO:tasks.workunit.client.0.smithi156.stdout:3/952: mknod d8/da/d28/d43/d71/da2/c13d 0 2024-07-16T09:55:50.599 INFO:tasks.workunit.client.0.smithi156.stdout:3/953: write d8/da/d28/d43/f8b [206757,11709] 0 2024-07-16T09:55:50.602 INFO:tasks.workunit.client.0.smithi156.stdout:3/954: fdatasync d8/da/d1d/d2c/fe5 0 2024-07-16T09:55:50.604 INFO:tasks.workunit.client.0.smithi156.stdout:3/955: write d8/da/d1d/d2c/dee/f115 [914537,29241] 0 2024-07-16T09:55:50.607 INFO:tasks.workunit.client.1.smithi176.stdout:7/809: truncate d4/d1f/d22/d51/f96 312801 0 2024-07-16T09:55:50.607 INFO:tasks.workunit.client.0.smithi156.stdout:3/956: creat d8/da/d21/d80/d99/d136/f13e x:0 0 0 2024-07-16T09:55:50.608 INFO:tasks.workunit.client.1.smithi176.stdout:7/810: fdatasync d4/d1f/f6f 0 2024-07-16T09:55:50.610 INFO:tasks.workunit.client.0.smithi156.stdout:3/957: mknod d8/da/d28/d43/dd8/c13f 0 2024-07-16T09:55:50.612 INFO:tasks.workunit.client.1.smithi176.stdout:7/811: rename d4/d1f/f6f to d4/dd/d26/dc2/d17/d1d/d8b/f104 0 2024-07-16T09:55:50.615 INFO:tasks.workunit.client.0.smithi156.stdout:7/811: dread dc/d3e/d3f/d51/fa0 [0,4194304] 0 2024-07-16T09:55:50.618 INFO:tasks.workunit.client.0.smithi156.stdout:7/812: dread - dc/d13/d36/dd4/fe5 zero size 2024-07-16T09:55:50.618 INFO:tasks.workunit.client.0.smithi156.stdout:3/958: dwrite d8/da/dec/df1/f12b [0,4194304] 0 2024-07-16T09:55:50.620 INFO:tasks.workunit.client.0.smithi156.stdout:3/959: chown d8/d14/l35 23 1 2024-07-16T09:55:50.621 INFO:tasks.workunit.client.0.smithi156.stdout:7/813: mknod dc/d63/d98/c120 0 2024-07-16T09:55:50.623 INFO:tasks.workunit.client.0.smithi156.stdout:3/960: mkdir d8/da/d13/d140 0 2024-07-16T09:55:50.624 INFO:tasks.workunit.client.0.smithi156.stdout:3/961: write d8/da/d28/d43/dd8/f112 [240446,130580] 0 2024-07-16T09:55:50.625 INFO:tasks.workunit.client.0.smithi156.stdout:7/814: creat dc/d63/d98/f121 x:0 0 0 2024-07-16T09:55:50.630 INFO:tasks.workunit.client.0.smithi156.stdout:7/815: rmdir dc/d3e/d3f/d45 39 2024-07-16T09:55:50.632 INFO:tasks.workunit.client.0.smithi156.stdout:7/816: creat dc/d13/d20/d4d/f122 x:0 0 0 2024-07-16T09:55:50.633 INFO:tasks.workunit.client.0.smithi156.stdout:3/962: rename d8/da/d28/d43/d63/f73 to d8/da/d28/d43/f141 0 2024-07-16T09:55:50.635 INFO:tasks.workunit.client.0.smithi156.stdout:7/817: truncate dc/f94 1696838 0 2024-07-16T09:55:50.637 INFO:tasks.workunit.client.1.smithi176.stdout:6/989: truncate d1/d5/f98 3084546 0 2024-07-16T09:55:50.642 INFO:tasks.workunit.client.1.smithi176.stdout:6/990: getdents d1/d5d/db9/d101/d105 0 2024-07-16T09:55:50.642 INFO:tasks.workunit.client.0.smithi156.stdout:7/818: link dc/d13/d20/d4d/l77 dc/d13/d20/d4d/l123 0 2024-07-16T09:55:50.646 INFO:tasks.workunit.client.1.smithi176.stdout:6/991: getdents d1/d5/d47/d62/d77/dde 0 2024-07-16T09:55:50.648 INFO:tasks.workunit.client.1.smithi176.stdout:9/968: sync 2024-07-16T09:55:50.656 INFO:tasks.workunit.client.1.smithi176.stdout:6/992: write d1/d5d/d9e/d8f/fd5 [2718597,74138] 0 2024-07-16T09:55:50.659 INFO:tasks.workunit.client.0.smithi156.stdout:7/819: dread dc/d3e/d3f/d45/f9a [0,4194304] 0 2024-07-16T09:55:50.661 INFO:tasks.workunit.client.0.smithi156.stdout:7/820: read dc/d3e/d3f/d51/f53 [2333208,14995] 0 2024-07-16T09:55:50.661 INFO:tasks.workunit.client.0.smithi156.stdout:3/963: dread d8/da/d1d/d2c/de0/df8/f10a [0,4194304] 0 2024-07-16T09:55:50.665 INFO:tasks.workunit.client.0.smithi156.stdout:0/966: sync 2024-07-16T09:55:50.665 INFO:tasks.workunit.client.0.smithi156.stdout:3/964: creat d8/da/d28/f142 x:0 0 0 2024-07-16T09:55:50.667 INFO:tasks.workunit.client.1.smithi176.stdout:6/993: dread d1/d5/d7/d54/f70 [0,4194304] 0 2024-07-16T09:55:50.668 INFO:tasks.workunit.client.0.smithi156.stdout:3/965: creat d8/d14/d121/f143 x:0 0 0 2024-07-16T09:55:50.668 INFO:tasks.workunit.client.0.smithi156.stdout:7/821: dwrite dc/d3e/d3f/d45/f9a [0,4194304] 0 2024-07-16T09:55:50.671 INFO:tasks.workunit.client.1.smithi176.stdout:6/994: mkdir d1/d5d/db9/d101/d163 0 2024-07-16T09:55:50.672 INFO:tasks.workunit.client.0.smithi156.stdout:3/966: mknod d8/da/dec/df1/c144 0 2024-07-16T09:55:50.672 INFO:tasks.workunit.client.0.smithi156.stdout:7/822: chown dc/d3e/d3f/d51/d6c/f85 4330 1 2024-07-16T09:55:50.674 INFO:tasks.workunit.client.1.smithi176.stdout:6/995: symlink d1/d5d/db9/l164 0 2024-07-16T09:55:50.674 INFO:tasks.workunit.client.0.smithi156.stdout:0/967: dread d0/d1a/d4d/d9/d27/d50/d11b/d11c/f134 [0,4194304] 0 2024-07-16T09:55:50.675 INFO:tasks.workunit.client.0.smithi156.stdout:7/823: rmdir dc/d13/d15/dfc/db1 39 2024-07-16T09:55:50.677 INFO:tasks.workunit.client.0.smithi156.stdout:7/824: stat dc/d3e/d3f/c70 0 2024-07-16T09:55:50.680 INFO:tasks.workunit.client.0.smithi156.stdout:3/967: rename d8/da/d1d/d6f/db3 to d8/da/d1d/d2c/de0/dad/d145 0 2024-07-16T09:55:50.680 INFO:tasks.workunit.client.1.smithi176.stdout:6/996: rmdir d1/d5/d47/d62/de8/d111/d122/d12e 0 2024-07-16T09:55:50.681 INFO:tasks.workunit.client.0.smithi156.stdout:7/825: mkdir dc/d13/d124 0 2024-07-16T09:55:50.682 INFO:tasks.workunit.client.0.smithi156.stdout:0/968: dwrite d0/d1a/d76/f87 [0,4194304] 0 2024-07-16T09:55:50.685 INFO:tasks.workunit.client.1.smithi176.stdout:6/997: creat d1/d5/d47/d62/d80/f165 x:0 0 0 2024-07-16T09:55:50.688 INFO:tasks.workunit.client.0.smithi156.stdout:3/968: dwrite d8/da/d13/f57 [0,4194304] 0 2024-07-16T09:55:50.688 INFO:tasks.workunit.client.1.smithi176.stdout:6/998: rename d1/d5/d7/d1e/c22 to d1/d5/d47/d62/d80/d13d/d15b/c166 0 2024-07-16T09:55:50.689 INFO:tasks.workunit.client.0.smithi156.stdout:7/826: mknod dc/d13/d15/c125 0 2024-07-16T09:55:50.690 INFO:tasks.workunit.client.0.smithi156.stdout:3/969: fsync d8/f62 0 2024-07-16T09:55:50.691 INFO:tasks.workunit.client.0.smithi156.stdout:3/970: chown d8/d14/f30 2 1 2024-07-16T09:55:50.693 INFO:tasks.workunit.client.0.smithi156.stdout:3/971: write d8/da/dec/df1/f13a [967946,30485] 0 2024-07-16T09:55:50.697 INFO:tasks.workunit.client.0.smithi156.stdout:3/972: mkdir d8/d14/d146 0 2024-07-16T09:55:50.699 INFO:tasks.workunit.client.0.smithi156.stdout:5/916: sync 2024-07-16T09:55:50.700 INFO:tasks.workunit.client.0.smithi156.stdout:7/827: dread dc/d3e/d3f/d45/d104/fd5 [0,4194304] 0 2024-07-16T09:55:50.702 INFO:tasks.workunit.client.0.smithi156.stdout:5/917: read d1b/d1f/d45/d52/d49/d4d/f62 [778595,50834] 0 2024-07-16T09:55:50.705 INFO:tasks.workunit.client.1.smithi176.stdout:6/999: dwrite d1/db3/f95 [0,4194304] 0 2024-07-16T09:55:50.707 INFO:tasks.workunit.client.0.smithi156.stdout:5/918: rename d1b/d1f/d42/f5f to d1b/d99/f122 0 2024-07-16T09:55:50.709 INFO:tasks.workunit.client.0.smithi156.stdout:0/969: dwrite d0/d1a/d4d/d9/d27/d2c/d6d/fb3 [0,4194304] 0 2024-07-16T09:55:50.714 INFO:tasks.workunit.client.0.smithi156.stdout:7/828: dwrite dc/d13/d20/d4d/f122 [0,4194304] 0 2024-07-16T09:55:50.722 INFO:tasks.workunit.client.0.smithi156.stdout:5/919: dwrite d1b/f55 [0,4194304] 0 2024-07-16T09:55:50.723 INFO:tasks.workunit.client.0.smithi156.stdout:0/970: rename d0/d1a/d4d/d9/d27/d2c/d6d/d9d/dbb/ceb to d0/d1a/d4d/d6e/c13c 0 2024-07-16T09:55:50.728 INFO:tasks.workunit.client.0.smithi156.stdout:0/971: readlink d0/d1a/d4d/d5e/db7/le9 0 2024-07-16T09:55:50.732 INFO:tasks.workunit.client.0.smithi156.stdout:5/920: creat d1b/d1f/d45/d7f/f123 x:0 0 0 2024-07-16T09:55:50.733 INFO:tasks.workunit.client.0.smithi156.stdout:0/972: stat d0/d1a/d4d/d9/d68/dff/fb6 0 2024-07-16T09:55:50.733 INFO:tasks.workunit.client.0.smithi156.stdout:7/829: dwrite dc/d48/d119/f61 [0,4194304] 0 2024-07-16T09:55:50.735 INFO:tasks.workunit.client.0.smithi156.stdout:3/973: dwrite d8/da/d1d/f58 [0,4194304] 0 2024-07-16T09:55:50.737 INFO:tasks.workunit.client.0.smithi156.stdout:5/921: dread - d1b/d1f/d45/d52/d43/d85/d97/d37/d88/d61/f11b zero size 2024-07-16T09:55:50.740 INFO:tasks.workunit.client.0.smithi156.stdout:7/830: fdatasync dc/d3e/d3f/d51/fa0 0 2024-07-16T09:55:50.741 INFO:tasks.workunit.client.0.smithi156.stdout:3/974: creat d8/da/d1d/d2c/de0/df8/f147 x:0 0 0 2024-07-16T09:55:50.743 INFO:tasks.workunit.client.0.smithi156.stdout:0/973: creat d0/d22/d41/d66/d7d/f13d x:0 0 0 2024-07-16T09:55:50.748 INFO:tasks.workunit.client.0.smithi156.stdout:8/890: sync 2024-07-16T09:55:50.748 INFO:tasks.workunit.client.0.smithi156.stdout:6/859: sync 2024-07-16T09:55:50.750 INFO:tasks.workunit.client.0.smithi156.stdout:5/922: getdents d1b/d1f/d45/d7a/d98 0 2024-07-16T09:55:50.751 INFO:tasks.workunit.client.0.smithi156.stdout:7/831: dwrite dc/d13/d36/f66 [0,4194304] 0 2024-07-16T09:55:50.753 INFO:tasks.workunit.client.0.smithi156.stdout:8/891: fdatasync d0/d11/d62/dc2/fc9 0 2024-07-16T09:55:50.754 INFO:tasks.workunit.client.0.smithi156.stdout:7/832: fdatasync dc/d13/d15/d8b/fa7 0 2024-07-16T09:55:50.755 INFO:tasks.workunit.client.0.smithi156.stdout:5/923: fdatasync d1b/f10b 0 2024-07-16T09:55:50.758 INFO:tasks.workunit.client.0.smithi156.stdout:0/974: dwrite d0/d1a/d4d/d9/f26 [4194304,4194304] 0 2024-07-16T09:55:50.761 INFO:tasks.workunit.client.0.smithi156.stdout:3/975: dwrite d8/da/d1d/d2c/de0/dad/feb [0,4194304] 0 2024-07-16T09:55:50.763 INFO:tasks.workunit.client.0.smithi156.stdout:3/976: chown d8/da/d28/d43/f4b 76871341 1 2024-07-16T09:55:50.767 INFO:tasks.workunit.client.0.smithi156.stdout:6/860: truncate d3/d13/d17/d4f/d30/f44 451249 0 2024-07-16T09:55:50.767 INFO:tasks.workunit.client.0.smithi156.stdout:0/975: chown d0/d22/d41/d52/ddb/fe3 26613 1 2024-07-16T09:55:50.769 INFO:tasks.workunit.client.0.smithi156.stdout:5/924: unlink d1b/d1f/d45/d52/d43/d85/d97/d37/d88/dbc/fed 0 2024-07-16T09:55:50.772 INFO:tasks.workunit.client.0.smithi156.stdout:3/977: creat d8/da/d1d/d2c/d7d/f148 x:0 0 0 2024-07-16T09:55:50.772 INFO:tasks.workunit.client.0.smithi156.stdout:7/833: dwrite dc/d48/db2/fe7 [0,4194304] 0 2024-07-16T09:55:50.776 INFO:tasks.workunit.client.0.smithi156.stdout:6/861: creat d3/d13/d17/d4f/d4d/d3b/d5a/f117 x:0 0 0 2024-07-16T09:55:50.779 INFO:tasks.workunit.client.0.smithi156.stdout:7/834: rename dc/d48/d119/f61 to dc/d48/d119/f126 0 2024-07-16T09:55:50.780 INFO:tasks.workunit.client.0.smithi156.stdout:3/978: creat d8/d14/d121/f149 x:0 0 0 2024-07-16T09:55:50.784 INFO:tasks.workunit.client.0.smithi156.stdout:3/979: fdatasync d8/da/d13/f20 0 2024-07-16T09:55:50.791 INFO:tasks.workunit.client.0.smithi156.stdout:6/862: dwrite d3/d13/d17/d4f/d1e/ffd [0,4194304] 0 2024-07-16T09:55:50.796 INFO:tasks.workunit.client.0.smithi156.stdout:7/835: fsync dc/d48/f4b 0 2024-07-16T09:55:50.798 INFO:tasks.workunit.client.0.smithi156.stdout:3/980: dwrite d8/da/d28/d43/fe2 [0,4194304] 0 2024-07-16T09:55:50.801 INFO:tasks.workunit.client.0.smithi156.stdout:3/981: write d8/da/f32 [4103494,65070] 0 2024-07-16T09:55:50.803 INFO:tasks.workunit.client.0.smithi156.stdout:0/976: dread d0/d22/d41/d52/ddb/f11d [0,4194304] 0 2024-07-16T09:55:50.803 INFO:tasks.workunit.client.0.smithi156.stdout:7/836: fdatasync dc/d3e/d3f/d51/d6c/f85 0 2024-07-16T09:55:50.806 INFO:tasks.workunit.client.0.smithi156.stdout:3/982: truncate d8/da/d1d/d2c/de0/f88 4652074 0 2024-07-16T09:55:50.808 INFO:tasks.workunit.client.0.smithi156.stdout:6/863: mkdir d3/d13/d17/d4f/d94/d19/d31/d35/d118 0 2024-07-16T09:55:50.811 INFO:tasks.workunit.client.0.smithi156.stdout:0/977: creat d0/d1a/d4d/d9/d27/d2c/f13e x:0 0 0 2024-07-16T09:55:50.813 INFO:tasks.workunit.client.0.smithi156.stdout:3/983: link d8/da/d1d/d2c/de0/fa9 d8/da/dec/df1/f14a 0 2024-07-16T09:55:50.817 INFO:tasks.workunit.client.0.smithi156.stdout:6/864: dwrite d3/d13/d17/d56/f10d [4194304,4194304] 0 2024-07-16T09:55:50.821 INFO:tasks.workunit.client.0.smithi156.stdout:6/865: truncate d3/d13/d17/d4f/d4d/fa6 1023115 0 2024-07-16T09:55:50.823 INFO:tasks.workunit.client.0.smithi156.stdout:6/866: fsync d3/d13/d17/d4f/d94/dec/f100 0 2024-07-16T09:55:50.829 INFO:tasks.workunit.client.0.smithi156.stdout:6/867: rename d3/d13/d17/d4f/d94/d19/d2e to d3/d13/d17/d4f/d94/daa/d119 0 2024-07-16T09:55:50.831 INFO:tasks.workunit.client.0.smithi156.stdout:0/978: dwrite d0/d1a/d76/d92/fcc [0,4194304] 0 2024-07-16T09:55:50.832 INFO:tasks.workunit.client.0.smithi156.stdout:6/868: dread - d3/d13/d17/d4f/d30/d45/d99/fb5 zero size 2024-07-16T09:55:50.833 INFO:tasks.workunit.client.0.smithi156.stdout:7/837: dread dc/f42 [0,4194304] 0 2024-07-16T09:55:50.835 INFO:tasks.workunit.client.0.smithi156.stdout:0/979: mknod d0/d22/d41/d52/ddb/c13f 0 2024-07-16T09:55:50.836 INFO:tasks.workunit.client.0.smithi156.stdout:7/838: symlink dc/d3e/d76/df1/l127 0 2024-07-16T09:55:50.838 INFO:tasks.workunit.client.0.smithi156.stdout:6/869: truncate d3/d13/d17/d4f/d4d/d3b/d5a/f69 1438486 0 2024-07-16T09:55:50.839 INFO:tasks.workunit.client.1.smithi176.stdout:3/716: sync 2024-07-16T09:55:50.840 INFO:tasks.workunit.client.0.smithi156.stdout:0/980: symlink d0/d1a/d76/dbc/l140 0 2024-07-16T09:55:50.840 INFO:tasks.workunit.client.1.smithi176.stdout:5/909: sync 2024-07-16T09:55:50.840 INFO:tasks.workunit.client.1.smithi176.stdout:4/828: sync 2024-07-16T09:55:50.841 INFO:tasks.workunit.client.1.smithi176.stdout:1/589: sync 2024-07-16T09:55:50.841 INFO:tasks.workunit.client.1.smithi176.stdout:8/907: sync 2024-07-16T09:55:50.841 INFO:tasks.workunit.client.0.smithi156.stdout:0/981: fsync d0/d1a/d4d/d9/d27/d50/f5f 0 2024-07-16T09:55:50.844 INFO:tasks.workunit.client.1.smithi176.stdout:4/829: write da/dc/f45 [1207444,4985] 0 2024-07-16T09:55:50.846 INFO:tasks.workunit.client.1.smithi176.stdout:8/908: truncate d8/d23/d100/d31/f10b 38477 0 2024-07-16T09:55:50.849 INFO:tasks.workunit.client.1.smithi176.stdout:8/909: fdatasync d8/d23/d100/d53/f119 0 2024-07-16T09:55:50.849 INFO:tasks.workunit.client.1.smithi176.stdout:1/590: truncate d4/d3a/d54/d5a/d7a/fa6 614388 0 2024-07-16T09:55:50.850 INFO:tasks.workunit.client.0.smithi156.stdout:7/839: dread dc/d3e/d3f/d51/d6c/f85 [0,4194304] 0 2024-07-16T09:55:50.852 INFO:tasks.workunit.client.0.smithi156.stdout:7/840: write dc/d13/d15/d8b/fa7 [884689,71510] 0 2024-07-16T09:55:50.852 INFO:tasks.workunit.client.1.smithi176.stdout:3/717: dwrite d3/da/d33/f44 [0,4194304] 0 2024-07-16T09:55:50.853 INFO:tasks.workunit.client.1.smithi176.stdout:1/591: creat d4/db/fc0 x:0 0 0 2024-07-16T09:55:50.856 INFO:tasks.workunit.client.1.smithi176.stdout:5/910: write d4/d10/fb1 [1758452,1410] 0 2024-07-16T09:55:50.857 INFO:tasks.workunit.client.1.smithi176.stdout:4/830: write da/d41/dc3/f65 [2028161,11737] 0 2024-07-16T09:55:50.859 INFO:tasks.workunit.client.0.smithi156.stdout:0/982: dwrite d0/d1a/d4d/d9/d27/d2c/d6d/ff6 [0,4194304] 0 2024-07-16T09:55:50.861 INFO:tasks.workunit.client.0.smithi156.stdout:7/841: dwrite dc/d48/dd7/f110 [0,4194304] 0 2024-07-16T09:55:50.863 INFO:tasks.workunit.client.1.smithi176.stdout:4/831: readlink da/dc/d4b/df0/d96/da4/le6 0 2024-07-16T09:55:50.863 INFO:tasks.workunit.client.0.smithi156.stdout:7/842: chown dc/d13/d15/f50 5 1 2024-07-16T09:55:50.864 INFO:tasks.workunit.client.1.smithi176.stdout:5/911: creat d4/da9/dc4/deb/f149 x:0 0 0 2024-07-16T09:55:50.865 INFO:tasks.workunit.client.1.smithi176.stdout:5/912: chown d4/d37/d72/d38/f97 3058357 1 2024-07-16T09:55:50.866 INFO:tasks.workunit.client.1.smithi176.stdout:8/910: truncate d8/d23/d100/d31/d46/d6a/ffd 1582716 0 2024-07-16T09:55:50.866 INFO:tasks.workunit.client.1.smithi176.stdout:1/592: truncate d4/d8/d12/d21/d25/d30/d44/f5f 637711 0 2024-07-16T09:55:50.867 INFO:tasks.workunit.client.0.smithi156.stdout:0/983: creat d0/d1a/d4d/d9/d27/d3d/d62/f141 x:0 0 0 2024-07-16T09:55:50.867 INFO:tasks.workunit.client.0.smithi156.stdout:7/843: dread dc/d3e/d3f/d45/fb3 [4194304,4194304] 0 2024-07-16T09:55:50.869 INFO:tasks.workunit.client.0.smithi156.stdout:0/984: truncate d0/d1a/d4d/d61/f126 44229 0 2024-07-16T09:55:50.870 INFO:tasks.workunit.client.1.smithi176.stdout:5/913: creat d4/d14/d19/d1b/d61/f14a x:0 0 0 2024-07-16T09:55:50.873 INFO:tasks.workunit.client.1.smithi176.stdout:4/832: rename da/f47 to da/d3d/f10f 0 2024-07-16T09:55:50.875 INFO:tasks.workunit.client.1.smithi176.stdout:5/914: mknod d4/d37/d72/d38/d102/dbd/c14b 0 2024-07-16T09:55:50.875 INFO:tasks.workunit.client.1.smithi176.stdout:4/833: readlink da/dc/d4b/df0/d96/la8 0 2024-07-16T09:55:50.875 INFO:tasks.workunit.client.0.smithi156.stdout:0/985: creat d0/d1a/d4d/d61/d97/de1/f142 x:0 0 0 2024-07-16T09:55:50.876 INFO:tasks.workunit.client.1.smithi176.stdout:4/834: dread - da/d41/d5b/dba/f10c zero size 2024-07-16T09:55:50.877 INFO:tasks.workunit.client.1.smithi176.stdout:1/593: getdents d4/d8/d12/d21/d25/d30 0 2024-07-16T09:55:50.878 INFO:tasks.workunit.client.0.smithi156.stdout:7/844: dwrite dc/d13/d15/dfc/f6d [0,4194304] 0 2024-07-16T09:55:50.879 INFO:tasks.workunit.client.0.smithi156.stdout:0/986: creat d0/d1a/d4d/d25/f143 x:0 0 0 2024-07-16T09:55:50.880 INFO:tasks.workunit.client.1.smithi176.stdout:5/915: mknod d4/c14c 0 2024-07-16T09:55:50.881 INFO:tasks.workunit.client.1.smithi176.stdout:1/594: dread d4/d8/d12/d21/d25/d30/d44/d4d/f95 [0,4194304] 0 2024-07-16T09:55:50.882 INFO:tasks.workunit.client.0.smithi156.stdout:7/845: mkdir dc/d63/d98/d128 0 2024-07-16T09:55:50.882 INFO:tasks.workunit.client.1.smithi176.stdout:5/916: stat d4/d14/d19/d1b/c125 0 2024-07-16T09:55:50.883 INFO:tasks.workunit.client.0.smithi156.stdout:0/987: read d0/d1a/d4d/d9/f56 [518890,41081] 0 2024-07-16T09:55:50.884 INFO:tasks.workunit.client.1.smithi176.stdout:9/969: sync 2024-07-16T09:55:50.884 INFO:tasks.workunit.client.1.smithi176.stdout:7/812: sync 2024-07-16T09:55:50.885 INFO:tasks.workunit.client.0.smithi156.stdout:7/846: truncate dc/d13/d20/d4d/fd1 115539 0 2024-07-16T09:55:50.885 INFO:tasks.workunit.client.0.smithi156.stdout:7/847: fdatasync dc/d13/d15/f103 0 2024-07-16T09:55:50.886 INFO:tasks.workunit.client.0.smithi156.stdout:0/988: rmdir d0/d1a/d4d/d9/d68/dff 39 2024-07-16T09:55:50.886 INFO:tasks.workunit.client.1.smithi176.stdout:5/917: write d4/f27 [3176896,3536] 0 2024-07-16T09:55:50.887 INFO:tasks.workunit.client.0.smithi156.stdout:0/989: read - d0/d1a/d4d/d9/d27/d2c/f13e zero size 2024-07-16T09:55:50.888 INFO:tasks.workunit.client.0.smithi156.stdout:7/848: write dc/d3e/d3f/d51/d8a/dae/dd8/fe8 [506610,28130] 0 2024-07-16T09:55:50.891 INFO:tasks.workunit.client.0.smithi156.stdout:7/849: creat dc/d13/d36/dd4/f129 x:0 0 0 2024-07-16T09:55:50.894 INFO:tasks.workunit.client.1.smithi176.stdout:9/970: truncate d5/de/da5/dc1/f134 341595 0 2024-07-16T09:55:50.894 INFO:tasks.workunit.client.1.smithi176.stdout:8/911: dread d8/d14/d1e/d33/f5b [0,4194304] 0 2024-07-16T09:55:50.894 INFO:tasks.workunit.client.0.smithi156.stdout:0/990: mknod d0/d22/d41/da9/dc9/c144 0 2024-07-16T09:55:50.895 INFO:tasks.workunit.client.0.smithi156.stdout:0/991: chown d0/d1a/d76 1 1 2024-07-16T09:55:50.896 INFO:tasks.workunit.client.1.smithi176.stdout:1/595: link d4/d8/d12/d4c/l80 d4/d8/d12/d4c/lc1 0 2024-07-16T09:55:50.897 INFO:tasks.workunit.client.1.smithi176.stdout:3/718: dread d3/da/d33/f78 [0,4194304] 0 2024-07-16T09:55:50.898 INFO:tasks.workunit.client.1.smithi176.stdout:5/918: mkdir d4/d37/d72/d38/d102/dbd/d14d 0 2024-07-16T09:55:50.898 INFO:tasks.workunit.client.0.smithi156.stdout:0/992: symlink d0/d1a/da3/l145 0 2024-07-16T09:55:50.900 INFO:tasks.workunit.client.0.smithi156.stdout:7/850: getdents dc/d3e/d3f/d45/d104 0 2024-07-16T09:55:50.901 INFO:tasks.workunit.client.1.smithi176.stdout:5/919: truncate d4/d37/dbb/f134 748308 0 2024-07-16T09:55:50.903 INFO:tasks.workunit.client.0.smithi156.stdout:0/993: rename d0/d1a/d76/d92/l10d to d0/d1a/d4d/d9/d27/d50/l146 0 2024-07-16T09:55:50.904 INFO:tasks.workunit.client.1.smithi176.stdout:9/971: fsync d5/de/f61 0 2024-07-16T09:55:50.904 INFO:tasks.workunit.client.1.smithi176.stdout:4/835: truncate da/d3d/fa2 1963905 0 2024-07-16T09:55:50.905 INFO:tasks.workunit.client.1.smithi176.stdout:8/912: dwrite d8/d14/d1e/d7c/fec [0,4194304] 0 2024-07-16T09:55:50.906 INFO:tasks.workunit.client.1.smithi176.stdout:1/596: creat d4/d8/fc2 x:0 0 0 2024-07-16T09:55:50.908 INFO:tasks.workunit.client.0.smithi156.stdout:7/851: getdents dc/d13/d15/dfc/db1 0 2024-07-16T09:55:50.909 INFO:tasks.workunit.client.1.smithi176.stdout:5/920: fdatasync d4/d37/f84 0 2024-07-16T09:55:50.911 INFO:tasks.workunit.client.0.smithi156.stdout:0/994: getdents d0/d1a/d76/da7 0 2024-07-16T09:55:50.911 INFO:tasks.workunit.client.0.smithi156.stdout:0/995: stat d0/d1a/d4d/d5e/d117/lba 0 2024-07-16T09:55:50.912 INFO:tasks.workunit.client.1.smithi176.stdout:5/921: readlink d4/d37/d72/d38/d6e/dbe/ld8 0 2024-07-16T09:55:50.913 INFO:tasks.workunit.client.0.smithi156.stdout:7/852: mkdir dc/d3e/d3f/d45/d12a 0 2024-07-16T09:55:50.913 INFO:tasks.workunit.client.1.smithi176.stdout:4/836: mkdir da/d3d/d17/d110 0 2024-07-16T09:55:50.914 INFO:tasks.workunit.client.1.smithi176.stdout:3/719: dwrite d3/da/d7b/fb8 [0,4194304] 0 2024-07-16T09:55:50.916 INFO:tasks.workunit.client.1.smithi176.stdout:7/813: write d4/dd/f23 [1111450,3698] 0 2024-07-16T09:55:50.918 INFO:tasks.workunit.client.1.smithi176.stdout:4/837: chown da/dc/f58 682903 1 2024-07-16T09:55:50.919 INFO:tasks.workunit.client.1.smithi176.stdout:7/814: readlink d4/lf 0 2024-07-16T09:55:50.921 INFO:tasks.workunit.client.1.smithi176.stdout:5/922: write d4/d37/d72/d38/d39/f3a [2825982,55953] 0 2024-07-16T09:55:50.922 INFO:tasks.workunit.client.0.smithi156.stdout:0/996: dwrite d0/d1a/d76/d92/f10b [0,4194304] 0 2024-07-16T09:55:50.926 INFO:tasks.workunit.client.1.smithi176.stdout:3/720: unlink d3/d17/d83/f8a 0 2024-07-16T09:55:50.927 INFO:tasks.workunit.client.1.smithi176.stdout:1/597: mknod d4/d8/cc3 0 2024-07-16T09:55:50.928 INFO:tasks.workunit.client.0.smithi156.stdout:0/997: rmdir d0/d1a/d4d/d9/d27/d3d 39 2024-07-16T09:55:50.930 INFO:tasks.workunit.client.0.smithi156.stdout:7/853: truncate dc/d13/d15/f91 527687 0 2024-07-16T09:55:50.931 INFO:tasks.workunit.client.1.smithi176.stdout:3/721: write d3/da/d33/f44 [1299692,96208] 0 2024-07-16T09:55:50.931 INFO:tasks.workunit.client.1.smithi176.stdout:5/923: fdatasync d4/d14/dc5/fd1 0 2024-07-16T09:55:50.931 INFO:tasks.workunit.client.0.smithi156.stdout:0/998: mkdir d0/d1a/d147 0 2024-07-16T09:55:50.931 INFO:tasks.workunit.client.1.smithi176.stdout:9/972: creat d5/d1d/db3/f147 x:0 0 0 2024-07-16T09:55:50.933 INFO:tasks.workunit.client.0.smithi156.stdout:7/854: creat dc/d13/d15/def/f12b x:0 0 0 2024-07-16T09:55:50.933 INFO:tasks.workunit.client.1.smithi176.stdout:8/913: getdents d8/d14/d1e/d55 0 2024-07-16T09:55:50.935 INFO:tasks.workunit.client.1.smithi176.stdout:7/815: fdatasync d4/d5/f6c 0 2024-07-16T09:55:50.936 INFO:tasks.workunit.client.1.smithi176.stdout:9/973: dread - d5/de/f132 zero size 2024-07-16T09:55:50.936 INFO:tasks.workunit.client.0.smithi156.stdout:0/999: fsync d0/d1a/d4d/d9/d68/dff/fb6 0 2024-07-16T09:55:50.938 INFO:tasks.workunit.client.0.smithi156.stdout:7/855: creat dc/d13/d20/d4d/f12c x:0 0 0 2024-07-16T09:55:50.940 INFO:tasks.workunit.client.1.smithi176.stdout:7/816: chown d4/d1f/d22/d51/c83 139130566 1 2024-07-16T09:55:50.940 INFO:tasks.workunit.client.1.smithi176.stdout:9/974: creat d5/d1d/dd2/f148 x:0 0 0 2024-07-16T09:55:50.942 INFO:tasks.workunit.client.1.smithi176.stdout:1/598: dwrite d4/d77/dad/fbd [0,4194304] 0 2024-07-16T09:55:50.945 INFO:tasks.workunit.client.0.smithi156.stdout:7/856: dwrite dc/d63/f88 [0,4194304] 0 2024-07-16T09:55:50.946 INFO:tasks.workunit.client.0.smithi156.stdout:7/857: write dc/d13/d15/f103 [850272,84686] 0 2024-07-16T09:55:50.951 INFO:tasks.workunit.client.1.smithi176.stdout:8/914: dwrite d8/d23/d100/d53/d9a/dc1/ffb [0,4194304] 0 2024-07-16T09:55:50.953 INFO:tasks.workunit.client.1.smithi176.stdout:1/599: rmdir d4/d8/d12/d21/d25/d30/d44 39 2024-07-16T09:55:50.955 INFO:tasks.workunit.client.1.smithi176.stdout:7/817: mknod d4/dd/d26/dc2/d17/d1d/d8b/dbe/c105 0 2024-07-16T09:55:50.955 INFO:tasks.workunit.client.1.smithi176.stdout:3/722: write d3/d17/f2f [1803489,129970] 0 2024-07-16T09:55:50.957 INFO:tasks.workunit.client.1.smithi176.stdout:5/924: truncate d4/d37/d72/d38/d39/f5d 1756567 0 2024-07-16T09:55:50.958 INFO:tasks.workunit.client.0.smithi156.stdout:7/858: dwrite dc/d13/d15/f103 [0,4194304] 0 2024-07-16T09:55:50.958 INFO:tasks.workunit.client.0.smithi156.stdout:7/859: fsync dc/d63/fe6 0 2024-07-16T09:55:50.960 INFO:tasks.workunit.client.1.smithi176.stdout:9/975: write d5/de/da5/dc1/d112/d97/fa1 [593556,93379] 0 2024-07-16T09:55:50.960 INFO:tasks.workunit.client.0.smithi156.stdout:7/860: chown dc/d3e/d3f/d45/d12a 25017618 1 2024-07-16T09:55:50.961 INFO:tasks.workunit.client.1.smithi176.stdout:7/818: truncate d4/d47/d6d/f6e 1735690 0 2024-07-16T09:55:50.965 INFO:tasks.workunit.client.1.smithi176.stdout:5/925: mkdir d4/da9/dc4/deb/d14e 0 2024-07-16T09:55:50.965 INFO:tasks.workunit.client.1.smithi176.stdout:9/976: read d5/d1d/d20/f25 [2640628,68112] 0 2024-07-16T09:55:50.967 INFO:tasks.workunit.client.0.smithi156.stdout:7/861: creat dc/d13/d15/f12d x:0 0 0 2024-07-16T09:55:50.970 INFO:tasks.workunit.client.0.smithi156.stdout:7/862: unlink dc/d13/d15/def/l10e 0 2024-07-16T09:55:50.970 INFO:tasks.workunit.client.1.smithi176.stdout:3/723: mkdir d3/da/d6c/dd9 0 2024-07-16T09:55:50.970 INFO:tasks.workunit.client.1.smithi176.stdout:8/915: getdents d8/d23/d100/d53/d93 0 2024-07-16T09:55:50.971 INFO:tasks.workunit.client.1.smithi176.stdout:7/819: creat d4/d1f/f106 x:0 0 0 2024-07-16T09:55:50.971 INFO:tasks.workunit.client.1.smithi176.stdout:1/600: link d4/d3a/d54/d5a/c75 d4/d8/d12/d21/d34/daa/db8/cc4 0 2024-07-16T09:55:50.972 INFO:tasks.workunit.client.1.smithi176.stdout:9/977: unlink d5/de/da5/dc1/f12d 0 2024-07-16T09:55:50.972 INFO:tasks.workunit.client.1.smithi176.stdout:5/926: creat d4/d37/d72/d106/f14f x:0 0 0 2024-07-16T09:55:50.975 INFO:tasks.workunit.client.1.smithi176.stdout:3/724: rmdir d3/d17/d83 39 2024-07-16T09:55:50.975 INFO:tasks.workunit.client.1.smithi176.stdout:9/978: readlink d5/d1d/dc7/d89/l10a 0 2024-07-16T09:55:50.977 INFO:tasks.workunit.client.1.smithi176.stdout:5/927: rename d4/d14/f45 to d4/da9/f150 0 2024-07-16T09:55:50.977 INFO:tasks.workunit.client.1.smithi176.stdout:1/601: dread - d4/d8/d12/d4c/da4/fbf zero size 2024-07-16T09:55:50.983 INFO:tasks.workunit.client.1.smithi176.stdout:9/979: rename d5/d52/lb7 to d5/d1d/db0/dd3/l149 0 2024-07-16T09:55:50.985 INFO:tasks.workunit.client.0.smithi156.stdout:7/863: write dc/d13/f35 [1980608,124431] 0 2024-07-16T09:55:50.988 INFO:tasks.workunit.client.1.smithi176.stdout:7/820: dwrite d4/dd/db5/fe9 [0,4194304] 0 2024-07-16T09:55:50.991 INFO:tasks.workunit.client.1.smithi176.stdout:3/725: dwrite d3/da/d6c/fc1 [0,4194304] 0 2024-07-16T09:55:50.991 INFO:tasks.workunit.client.0.smithi156.stdout:7/864: dread dc/d13/d15/d8b/fa7 [0,4194304] 0 2024-07-16T09:55:50.993 INFO:tasks.workunit.client.1.smithi176.stdout:1/602: write d4/d8/d12/d21/d25/d30/fbe [766289,18554] 0 2024-07-16T09:55:50.997 INFO:tasks.workunit.client.1.smithi176.stdout:3/726: fsync d3/da/d7b/fa8 0 2024-07-16T09:55:50.997 INFO:tasks.workunit.client.0.smithi156.stdout:7/865: dread dc/d3e/d3f/d51/d8a/fe4 [0,4194304] 0 2024-07-16T09:55:50.999 INFO:tasks.workunit.client.0.smithi156.stdout:7/866: chown dc/d13/c79 284319 1 2024-07-16T09:55:51.001 INFO:tasks.workunit.client.0.smithi156.stdout:7/867: chown dc/d3e/d3f/c5b 0 1 2024-07-16T09:55:51.002 INFO:tasks.workunit.client.1.smithi176.stdout:9/980: link d5/de/d30/fe9 d5/de/da5/dc1/d112/d97/f14a 0 2024-07-16T09:55:51.003 INFO:tasks.workunit.client.0.smithi156.stdout:7/868: read dc/d3e/d3f/d51/d6c/f85 [15517,73686] 0 2024-07-16T09:55:51.008 INFO:tasks.workunit.client.1.smithi176.stdout:1/603: truncate d4/d8/d12/d21/d25/d30/d44/d4d/f95 88346 0 2024-07-16T09:55:51.008 INFO:tasks.workunit.client.0.smithi156.stdout:7/869: dread dc/d48/f4b [0,4194304] 0 2024-07-16T09:55:51.009 INFO:tasks.workunit.client.1.smithi176.stdout:8/916: truncate d8/d14/d1e/d33/f78 100116 0 2024-07-16T09:55:51.009 INFO:tasks.workunit.client.0.smithi156.stdout:7/870: chown dc/d13/l31 7633 1 2024-07-16T09:55:51.011 INFO:tasks.workunit.client.1.smithi176.stdout:1/604: read d4/d3a/f9e [301031,13494] 0 2024-07-16T09:55:51.011 INFO:tasks.workunit.client.0.smithi156.stdout:7/871: creat dc/d3e/d3f/d45/d104/f12e x:0 0 0 2024-07-16T09:55:51.012 INFO:tasks.workunit.client.1.smithi176.stdout:8/917: chown d8/d23/d100/db9/dcb/d34/l66 12874 1 2024-07-16T09:55:51.012 INFO:tasks.workunit.client.0.smithi156.stdout:7/872: stat dc/d13/f1d 0 2024-07-16T09:55:51.012 INFO:tasks.workunit.client.0.smithi156.stdout:7/873: chown dc/d13/d15 58 1 2024-07-16T09:55:51.014 INFO:tasks.workunit.client.1.smithi176.stdout:9/981: write d5/de/da5/f82 [3782198,50077] 0 2024-07-16T09:55:51.016 INFO:tasks.workunit.client.0.smithi156.stdout:7/874: rmdir dc/d3e/d3f/d45/d12a 0 2024-07-16T09:55:51.019 INFO:tasks.workunit.client.1.smithi176.stdout:1/605: rename d4/d3a/c85 to d4/d8/d12/d21/d25/d30/d44/cc5 0 2024-07-16T09:55:51.021 INFO:tasks.workunit.client.1.smithi176.stdout:7/821: dwrite d4/dd/d26/dc2/d17/f7f [0,4194304] 0 2024-07-16T09:55:51.024 INFO:tasks.workunit.client.1.smithi176.stdout:3/727: dwrite d3/f4a [0,4194304] 0 2024-07-16T09:55:51.029 INFO:tasks.workunit.client.1.smithi176.stdout:7/822: write d4/d47/d5d/f90 [2002198,41803] 0 2024-07-16T09:55:51.030 INFO:tasks.workunit.client.1.smithi176.stdout:7/823: stat d4/dd/d63/la0 0 2024-07-16T09:55:51.031 INFO:tasks.workunit.client.1.smithi176.stdout:7/824: fdatasync d4/d1f/d22/fec 0 2024-07-16T09:55:51.032 INFO:tasks.workunit.client.1.smithi176.stdout:3/728: write d3/da/fe [5502730,80620] 0 2024-07-16T09:55:51.032 INFO:tasks.workunit.client.1.smithi176.stdout:9/982: truncate d5/d1d/db3/ff6 3801631 0 2024-07-16T09:55:51.033 INFO:tasks.workunit.client.1.smithi176.stdout:8/918: link d8/d23/d100/db9/dcb/d34/l66 d8/d20/l12f 0 2024-07-16T09:55:51.034 INFO:tasks.workunit.client.1.smithi176.stdout:9/983: stat d5/de/da5/lc6 0 2024-07-16T09:55:51.034 INFO:tasks.workunit.client.1.smithi176.stdout:3/729: fdatasync d3/da/d56/fcd 0 2024-07-16T09:55:51.035 INFO:tasks.workunit.client.0.smithi156.stdout:5/925: sync 2024-07-16T09:55:51.035 INFO:tasks.workunit.client.0.smithi156.stdout:8/892: sync 2024-07-16T09:55:51.036 INFO:tasks.workunit.client.0.smithi156.stdout:3/984: sync 2024-07-16T09:55:51.036 INFO:tasks.workunit.client.0.smithi156.stdout:6/870: sync 2024-07-16T09:55:51.038 INFO:tasks.workunit.client.1.smithi176.stdout:8/919: rmdir d8/d14/d1e/d33/d9d 39 2024-07-16T09:55:51.038 INFO:tasks.workunit.client.0.smithi156.stdout:6/871: write d3/d13/d17/d56/ff7 [622678,39588] 0 2024-07-16T09:55:51.039 INFO:tasks.workunit.client.0.smithi156.stdout:8/893: dread - d0/d11/d20/d7f/db7/d110/d1f/d57/dc0/dff/f128 zero size 2024-07-16T09:55:51.040 INFO:tasks.workunit.client.0.smithi156.stdout:5/926: read d1b/d1f/d45/d52/d43/f64 [4046663,114979] 0 2024-07-16T09:55:51.040 INFO:tasks.workunit.client.0.smithi156.stdout:3/985: dread d8/f137 [0,4194304] 0 2024-07-16T09:55:51.041 INFO:tasks.workunit.client.1.smithi176.stdout:9/984: fdatasync d5/de/f34 0 2024-07-16T09:55:51.042 INFO:tasks.workunit.client.0.smithi156.stdout:8/894: read - d0/d11/d20/d7f/db7/d110/f4c zero size 2024-07-16T09:55:51.042 INFO:tasks.workunit.client.1.smithi176.stdout:1/606: write d4/d8/d12/d21/d25/d30/d44/d4d/f7e [1734518,90647] 0 2024-07-16T09:55:51.043 INFO:tasks.workunit.client.0.smithi156.stdout:6/872: creat d3/d13/d17/d4f/d94/d19/d31/d10a/f11a x:0 0 0 2024-07-16T09:55:51.043 INFO:tasks.workunit.client.0.smithi156.stdout:5/927: truncate d1b/f2f 1706651 0 2024-07-16T09:55:51.043 INFO:tasks.workunit.client.1.smithi176.stdout:8/920: creat d8/d23/d100/d31/db4/f130 x:0 0 0 2024-07-16T09:55:51.045 INFO:tasks.workunit.client.0.smithi156.stdout:3/986: creat d8/da/d28/d40/d127/f14b x:0 0 0 2024-07-16T09:55:51.045 INFO:tasks.workunit.client.1.smithi176.stdout:8/921: chown d8/d23/d76/d83/fbd 5 1 2024-07-16T09:55:51.046 INFO:tasks.workunit.client.0.smithi156.stdout:3/987: write d8/da/d1d/d6f/fb5 [1317000,70728] 0 2024-07-16T09:55:51.047 INFO:tasks.workunit.client.0.smithi156.stdout:8/895: mkdir d0/d11/d20/d7f/db7/d110/d1f/d57/d11b/d12a 0 2024-07-16T09:55:51.047 INFO:tasks.workunit.client.1.smithi176.stdout:3/730: mknod d3/d17/d1d/d4f/d59/cda 0 2024-07-16T09:55:51.047 INFO:tasks.workunit.client.1.smithi176.stdout:1/607: rmdir d4/d8/d12/d8f 39 2024-07-16T09:55:51.049 INFO:tasks.workunit.client.0.smithi156.stdout:5/928: creat d1b/d1f/d45/f124 x:0 0 0 2024-07-16T09:55:51.049 INFO:tasks.workunit.client.0.smithi156.stdout:3/988: read d8/da/d1d/db2/d126/f135 [1411011,13717] 0 2024-07-16T09:55:51.052 INFO:tasks.workunit.client.0.smithi156.stdout:3/989: write d8/da/d1d/fc6 [3222557,1774] 0 2024-07-16T09:55:51.056 INFO:tasks.workunit.client.1.smithi176.stdout:9/985: dwrite d5/de/da5/dbf/f105 [0,4194304] 0 2024-07-16T09:55:51.060 INFO:tasks.workunit.client.1.smithi176.stdout:3/731: dwrite d3/d17/d1d/d4f/d59/fd0 [0,4194304] 0 2024-07-16T09:55:51.060 INFO:tasks.workunit.client.1.smithi176.stdout:9/986: write d5/fd [5312877,90652] 0 2024-07-16T09:55:51.060 INFO:tasks.workunit.client.1.smithi176.stdout:7/825: truncate d4/d47/d6d/d89/d7e/fa1 731116 0 2024-07-16T09:55:51.061 INFO:tasks.workunit.client.0.smithi156.stdout:5/929: rename d1b/d1f/d45/d52/d43/d85/d97/d37/d88/d61 to d1b/d1f/d45/d52/d43/d85/d97/d37/d88/dbc/dc2/d125 0 2024-07-16T09:55:51.062 INFO:tasks.workunit.client.1.smithi176.stdout:1/608: creat d4/d8/fc6 x:0 0 0 2024-07-16T09:55:51.065 INFO:tasks.workunit.client.0.smithi156.stdout:5/930: rmdir d1b/d1f/d45/d52/d43/d85/d97/d37/d88 39 2024-07-16T09:55:51.067 INFO:tasks.workunit.client.1.smithi176.stdout:1/609: chown d4/d3a/d54/l79 747555 1 2024-07-16T09:55:51.067 INFO:tasks.workunit.client.1.smithi176.stdout:1/610: read - d4/d8/fc6 zero size 2024-07-16T09:55:51.069 INFO:tasks.workunit.client.0.smithi156.stdout:8/896: getdents d0/d11/d20/d7f/db7/d110/d1f/d57 0 2024-07-16T09:55:51.069 INFO:tasks.workunit.client.1.smithi176.stdout:8/922: dwrite d8/d23/d100/d61/f6e [0,4194304] 0 2024-07-16T09:55:51.069 INFO:tasks.workunit.client.1.smithi176.stdout:9/987: mkdir d5/de/da5/dc1/d14b 0 2024-07-16T09:55:51.071 INFO:tasks.workunit.client.1.smithi176.stdout:3/732: truncate d3/d17/f51 210011 0 2024-07-16T09:55:51.071 INFO:tasks.workunit.client.0.smithi156.stdout:3/990: creat d8/da/d1d/d2c/de0/f14c x:0 0 0 2024-07-16T09:55:51.074 INFO:tasks.workunit.client.0.smithi156.stdout:5/931: dread d1b/d1f/d45/d52/f102 [0,4194304] 0 2024-07-16T09:55:51.074 INFO:tasks.workunit.client.1.smithi176.stdout:3/733: write d3/d17/d1d/d4f/d6d/f9f [116219,92524] 0 2024-07-16T09:55:51.076 INFO:tasks.workunit.client.0.smithi156.stdout:3/991: symlink d8/da/d1d/d2c/de0/dad/l14d 0 2024-07-16T09:55:51.077 INFO:tasks.workunit.client.0.smithi156.stdout:3/992: stat d8/da/d28/d43/ca0 0 2024-07-16T09:55:51.079 INFO:tasks.workunit.client.1.smithi176.stdout:8/923: mkdir d8/d23/d100/d31/d46/d6a/da8/dea/d131 0 2024-07-16T09:55:51.080 INFO:tasks.workunit.client.1.smithi176.stdout:3/734: mknod d3/d17/d1d/d4f/d59/da0/d7a/cdb 0 2024-07-16T09:55:51.080 INFO:tasks.workunit.client.0.smithi156.stdout:5/932: symlink d1b/d1f/d45/d52/d43/d85/d97/d37/d88/dbc/l126 0 2024-07-16T09:55:51.082 INFO:tasks.workunit.client.1.smithi176.stdout:1/611: rename d4/d8/d12/d21/d25/d30/c48 to d4/d8/d12/d21/d34/daa/cc7 0 2024-07-16T09:55:51.082 INFO:tasks.workunit.client.0.smithi156.stdout:3/993: creat d8/da/d1d/db2/d126/f14e x:0 0 0 2024-07-16T09:55:51.082 INFO:tasks.workunit.client.0.smithi156.stdout:5/933: read d1b/d99/f122 [294653,61953] 0 2024-07-16T09:55:51.083 INFO:tasks.workunit.client.1.smithi176.stdout:8/924: chown d8/d23/d100/db9/dcb/c113 321 1 2024-07-16T09:55:51.084 INFO:tasks.workunit.client.0.smithi156.stdout:3/994: creat d8/da/d1d/d2c/de0/df8/f14f x:0 0 0 2024-07-16T09:55:51.085 INFO:tasks.workunit.client.0.smithi156.stdout:5/934: fsync d1b/d1f/d45/d52/d43/d7c/f82 0 2024-07-16T09:55:51.085 INFO:tasks.workunit.client.0.smithi156.stdout:3/995: write d8/da/f3d [3304429,8917] 0 2024-07-16T09:55:51.087 INFO:tasks.workunit.client.0.smithi156.stdout:3/996: write d8/d14/f6b [2542684,125060] 0 2024-07-16T09:55:51.089 INFO:tasks.workunit.client.1.smithi176.stdout:3/735: unlink d3/d17/d1d/d6e/f9a 0 2024-07-16T09:55:51.089 INFO:tasks.workunit.client.0.smithi156.stdout:5/935: rename d1b/d1f/d45/d52/d2d/d96/lbf to d1b/d1f/d45/d52/d2d/d96/l127 0 2024-07-16T09:55:51.093 INFO:tasks.workunit.client.1.smithi176.stdout:8/925: mkdir d8/d23/d100/d31/d2f/d132 0 2024-07-16T09:55:51.097 INFO:tasks.workunit.client.0.smithi156.stdout:5/936: dwrite d1b/d1f/d45/d52/d43/d85/d97/d37/fb7 [0,4194304] 0 2024-07-16T09:55:51.097 INFO:tasks.workunit.client.1.smithi176.stdout:8/926: chown d8/d20/c68 5 1 2024-07-16T09:55:51.098 INFO:tasks.workunit.client.0.smithi156.stdout:5/937: chown d1b/d1f/d45/d52/d49 62 1 2024-07-16T09:55:51.099 INFO:tasks.workunit.client.1.smithi176.stdout:3/736: truncate d3/f61 1487124 0 2024-07-16T09:55:51.099 INFO:tasks.workunit.client.0.smithi156.stdout:8/897: write d0/f6f [1254508,73768] 0 2024-07-16T09:55:51.101 INFO:tasks.workunit.client.1.smithi176.stdout:7/826: dread d4/d47/d6d/d89/d7e/fa1 [0,4194304] 0 2024-07-16T09:55:51.101 INFO:tasks.workunit.client.0.smithi156.stdout:5/938: mknod d1b/d1f/d45/d52/d49/d4d/c128 0 2024-07-16T09:55:51.101 INFO:tasks.workunit.client.0.smithi156.stdout:8/898: write d0/d11/d20/d7f/db7/d110/d1f/d60/f98 [2931346,60898] 0 2024-07-16T09:55:51.103 INFO:tasks.workunit.client.0.smithi156.stdout:8/899: rmdir d0/d11 39 2024-07-16T09:55:51.110 INFO:tasks.workunit.client.1.smithi176.stdout:1/612: write d4/d8/d12/d21/d34/daa/fab [401118,101951] 0 2024-07-16T09:55:51.111 INFO:tasks.workunit.client.1.smithi176.stdout:8/927: dwrite d8/d23/d76/f98 [4194304,4194304] 0 2024-07-16T09:55:51.112 INFO:tasks.workunit.client.0.smithi156.stdout:8/900: dwrite d0/d11/d29/d80/f88 [4194304,4194304] 0 2024-07-16T09:55:51.118 INFO:tasks.workunit.client.0.smithi156.stdout:8/901: symlink d0/d11/d20/d7f/db7/d110/d1f/dee/l12b 0 2024-07-16T09:55:51.118 INFO:tasks.workunit.client.1.smithi176.stdout:1/613: mkdir d4/d3a/dc8 0 2024-07-16T09:55:51.119 INFO:tasks.workunit.client.1.smithi176.stdout:1/614: fsync d4/d8/d12/d21/f9c 0 2024-07-16T09:55:51.120 INFO:tasks.workunit.client.0.smithi156.stdout:5/939: dwrite d1b/d1f/d45/d52/d43/d85/d97/d37/d88/dbc/dc2/d125/f69 [0,4194304] 0 2024-07-16T09:55:51.121 INFO:tasks.workunit.client.0.smithi156.stdout:8/902: fsync d0/d11/d20/d7f/db7/d110/d1f/d60/f98 0 2024-07-16T09:55:51.121 INFO:tasks.workunit.client.0.smithi156.stdout:5/940: chown d1b/d1f/d42/de0/f11f 2522595 1 2024-07-16T09:55:51.124 INFO:tasks.workunit.client.1.smithi176.stdout:7/827: write d4/dd/d49/d7a/f92 [850726,31183] 0 2024-07-16T09:55:51.124 INFO:tasks.workunit.client.1.smithi176.stdout:3/737: truncate d3/d17/d1d/d4f/d59/f93 1661820 0 2024-07-16T09:55:51.124 INFO:tasks.workunit.client.0.smithi156.stdout:5/941: truncate d1b/d99/f122 4677049 0 2024-07-16T09:55:51.125 INFO:tasks.workunit.client.1.smithi176.stdout:3/738: read d3/d17/d1d/d4f/d59/f3a [133216,87596] 0 2024-07-16T09:55:51.126 INFO:tasks.workunit.client.1.smithi176.stdout:1/615: symlink d4/d77/lc9 0 2024-07-16T09:55:51.126 INFO:tasks.workunit.client.1.smithi176.stdout:7/828: fsync d4/d47/d6d/d89/d66/ff9 0 2024-07-16T09:55:51.126 INFO:tasks.workunit.client.0.smithi156.stdout:8/903: symlink d0/d11/d20/d3f/dbe/l12c 0 2024-07-16T09:55:51.128 INFO:tasks.workunit.client.1.smithi176.stdout:1/616: dread - d4/d8/d12/d21/d25/da8/fb2 zero size 2024-07-16T09:55:51.129 INFO:tasks.workunit.client.1.smithi176.stdout:1/617: chown d4/d8/d12/d21/d34/c9a 1769708 1 2024-07-16T09:55:51.130 INFO:tasks.workunit.client.1.smithi176.stdout:1/618: fdatasync d4/d8/d12/d4c/da4/fbc 0 2024-07-16T09:55:51.132 INFO:tasks.workunit.client.1.smithi176.stdout:1/619: truncate d4/d8/f8c 2357902 0 2024-07-16T09:55:51.132 INFO:tasks.workunit.client.0.smithi156.stdout:8/904: creat d0/d74/d115/f12d x:0 0 0 2024-07-16T09:55:51.134 INFO:tasks.workunit.client.1.smithi176.stdout:7/829: creat d4/dd/d49/d7a/f107 x:0 0 0 2024-07-16T09:55:51.137 INFO:tasks.workunit.client.0.smithi156.stdout:5/942: dwrite d1b/d1f/d45/d52/d43/d85/d97/d37/d88/dc7/ffe [0,4194304] 0 2024-07-16T09:55:51.137 INFO:tasks.workunit.client.1.smithi176.stdout:7/830: read - d4/dd/d8e/fd0 zero size 2024-07-16T09:55:51.138 INFO:tasks.workunit.client.0.smithi156.stdout:8/905: creat d0/d11/d29/d80/d97/f12e x:0 0 0 2024-07-16T09:55:51.141 INFO:tasks.workunit.client.1.smithi176.stdout:1/620: dwrite d4/d8/d12/d21/fb9 [0,4194304] 0 2024-07-16T09:55:51.141 INFO:tasks.workunit.client.0.smithi156.stdout:5/943: creat d1b/d1f/d54/dbe/dde/f129 x:0 0 0 2024-07-16T09:55:51.144 INFO:tasks.workunit.client.1.smithi176.stdout:8/928: write d8/d23/d100/d31/d46/d6a/f7a [1035264,78042] 0 2024-07-16T09:55:51.147 INFO:tasks.workunit.client.0.smithi156.stdout:8/906: rename d0/d11/d29/d80/ceb to d0/d11/d29/c12f 0 2024-07-16T09:55:51.148 INFO:tasks.workunit.client.0.smithi156.stdout:5/944: truncate d1b/d1f/d8a/fd0 331224 0 2024-07-16T09:55:51.148 INFO:tasks.workunit.client.1.smithi176.stdout:1/621: dread d4/d3a/d54/d5a/d7a/fa6 [0,4194304] 0 2024-07-16T09:55:51.156 INFO:tasks.workunit.client.0.smithi156.stdout:5/945: unlink d1b/d1f/fa9 0 2024-07-16T09:55:51.160 INFO:tasks.workunit.client.1.smithi176.stdout:8/929: symlink d8/d23/d76/d83/d103/d12b/l133 0 2024-07-16T09:55:51.160 INFO:tasks.workunit.client.0.smithi156.stdout:5/946: mkdir d1b/d1f/d42/d6d/d9f/d11c/d12a 0 2024-07-16T09:55:51.160 INFO:tasks.workunit.client.1.smithi176.stdout:1/622: dwrite d4/d8/f8c [0,4194304] 0 2024-07-16T09:55:51.163 INFO:tasks.workunit.client.0.smithi156.stdout:5/947: mknod d1b/d1f/d45/d52/d43/d85/d97/d37/d88/dbc/dc2/d125/c12b 0 2024-07-16T09:55:51.165 INFO:tasks.workunit.client.1.smithi176.stdout:8/930: dread - d8/d23/d100/db9/dcb/d34/fde zero size 2024-07-16T09:55:51.166 INFO:tasks.workunit.client.1.smithi176.stdout:3/739: dwrite d3/da/d33/f39 [4194304,4194304] 0 2024-07-16T09:55:51.171 INFO:tasks.workunit.client.1.smithi176.stdout:8/931: fsync d8/d23/d100/d31/d46/d6a/da8/ffe 0 2024-07-16T09:55:51.171 INFO:tasks.workunit.client.1.smithi176.stdout:3/740: symlink d3/da/d33/ldc 0 2024-07-16T09:55:51.173 INFO:tasks.workunit.client.0.smithi156.stdout:5/948: getdents d1b/d1f/d45/d52/d43/d7c 0 2024-07-16T09:55:51.175 INFO:tasks.workunit.client.1.smithi176.stdout:7/831: dread d4/d5/f6c [0,4194304] 0 2024-07-16T09:55:51.176 INFO:tasks.workunit.client.1.smithi176.stdout:7/832: fdatasync d4/dd/f23 0 2024-07-16T09:55:51.176 INFO:tasks.workunit.client.1.smithi176.stdout:8/932: rename d8/d20/l12f to d8/d23/d100/d53/l134 0 2024-07-16T09:55:51.177 INFO:tasks.workunit.client.1.smithi176.stdout:7/833: readlink d4/d47/d6d/db7/le2 0 2024-07-16T09:55:51.178 INFO:tasks.workunit.client.1.smithi176.stdout:8/933: write d8/d23/d76/f98 [3830334,93965] 0 2024-07-16T09:55:51.178 INFO:tasks.workunit.client.0.smithi156.stdout:5/949: creat d1b/d1f/d45/d52/d43/d85/d97/d37/d88/dbc/dc2/d125/f12c x:0 0 0 2024-07-16T09:55:51.180 INFO:tasks.workunit.client.0.smithi156.stdout:5/950: chown d1b/d1f/c38 154845 1 2024-07-16T09:55:51.180 INFO:tasks.workunit.client.0.smithi156.stdout:8/907: dwrite d0/d11/d20/d7f/db7/d110/f3c [0,4194304] 0 2024-07-16T09:55:51.183 INFO:tasks.workunit.client.1.smithi176.stdout:8/934: symlink d8/df7/l135 0 2024-07-16T09:55:51.183 INFO:tasks.workunit.client.1.smithi176.stdout:7/834: link d4/dd/d49/f7c d4/dd/d49/dd5/f108 0 2024-07-16T09:55:51.184 INFO:tasks.workunit.client.0.smithi156.stdout:8/908: creat d0/d11/d13/d1c/d39/f130 x:0 0 0 2024-07-16T09:55:51.185 INFO:tasks.workunit.client.0.smithi156.stdout:5/951: creat d1b/d1f/d45/d52/d43/d85/d97/d37/d88/dc7/d112/f12d x:0 0 0 2024-07-16T09:55:51.186 INFO:tasks.workunit.client.0.smithi156.stdout:8/909: chown d0/d11/d20/ff3 480 1 2024-07-16T09:55:51.187 INFO:tasks.workunit.client.1.smithi176.stdout:8/935: mkdir d8/d23/d100/d31/d2f/d132/d136 0 2024-07-16T09:55:51.188 INFO:tasks.workunit.client.0.smithi156.stdout:5/952: write d1b/d1f/d45/d7f/f123 [953096,70370] 0 2024-07-16T09:55:51.188 INFO:tasks.workunit.client.0.smithi156.stdout:8/910: fdatasync d0/d11/d20/d7f/f9e 0 2024-07-16T09:55:51.191 INFO:tasks.workunit.client.1.smithi176.stdout:8/936: mknod d8/d23/d100/d31/d46/c137 0 2024-07-16T09:55:51.197 INFO:tasks.workunit.client.1.smithi176.stdout:3/741: dwrite d3/da/d7b/fa8 [0,4194304] 0 2024-07-16T09:55:51.197 INFO:tasks.workunit.client.0.smithi156.stdout:5/953: dwrite d1b/d1f/d45/d52/d43/d85/d97/d37/d88/dbc/dc2/d125/ff6 [4194304,4194304] 0 2024-07-16T09:55:51.197 INFO:tasks.workunit.client.0.smithi156.stdout:8/911: write d0/d11/d29/f79 [39330,34417] 0 2024-07-16T09:55:51.200 INFO:tasks.workunit.client.1.smithi176.stdout:8/937: dwrite d8/d14/f2d [0,4194304] 0 2024-07-16T09:55:51.202 INFO:tasks.workunit.client.0.smithi156.stdout:5/954: symlink d1b/d1f/d42/de0/de5/l12e 0 2024-07-16T09:55:51.205 INFO:tasks.workunit.client.1.smithi176.stdout:3/742: mkdir d3/d85/ddd 0 2024-07-16T09:55:51.207 INFO:tasks.workunit.client.0.smithi156.stdout:5/955: write d1b/d1f/d42/f47 [4504815,22888] 0 2024-07-16T09:55:51.208 INFO:tasks.workunit.client.0.smithi156.stdout:8/912: dwrite d0/d11/d62/dc2/fc9 [0,4194304] 0 2024-07-16T09:55:51.210 INFO:tasks.workunit.client.1.smithi176.stdout:3/743: mkdir d3/dde 0 2024-07-16T09:55:51.212 INFO:tasks.workunit.client.0.smithi156.stdout:8/913: mknod d0/d11/d29/da8/c131 0 2024-07-16T09:55:51.213 INFO:tasks.workunit.client.1.smithi176.stdout:3/744: dread - d3/da/d33/fc9 zero size 2024-07-16T09:55:51.213 INFO:tasks.workunit.client.1.smithi176.stdout:4/838: sync 2024-07-16T09:55:51.216 INFO:tasks.workunit.client.1.smithi176.stdout:8/938: write d8/d23/d100/db9/dcb/d37/fa2 [861352,116261] 0 2024-07-16T09:55:51.220 INFO:tasks.workunit.client.1.smithi176.stdout:4/839: symlink da/dc/d4b/df0/d28/l111 0 2024-07-16T09:55:51.220 INFO:tasks.workunit.client.1.smithi176.stdout:8/939: mkdir d8/d23/d76/d83/d138 0 2024-07-16T09:55:51.222 INFO:tasks.workunit.client.0.smithi156.stdout:8/914: link d0/d11/d29/d80/d97/de7/lfd d0/d11/d20/d7f/db7/d110/d1f/l132 0 2024-07-16T09:55:51.223 INFO:tasks.workunit.client.0.smithi156.stdout:8/915: truncate d0/d11/d29/d80/d97/f12e 855848 0 2024-07-16T09:55:51.224 INFO:tasks.workunit.client.1.smithi176.stdout:8/940: write d8/d23/d100/d31/db4/f130 [937941,11796] 0 2024-07-16T09:55:51.226 INFO:tasks.workunit.client.0.smithi156.stdout:8/916: mknod d0/d11/d20/d7f/db7/d110/d1f/d57/db3/df7/c133 0 2024-07-16T09:55:51.229 INFO:tasks.workunit.client.1.smithi176.stdout:3/745: dwrite d3/d17/d1d/faf [0,4194304] 0 2024-07-16T09:55:51.229 INFO:tasks.workunit.client.1.smithi176.stdout:4/840: truncate da/dc/d4b/df0/d96/d9b/db4/d4a/fe4 1023651 0 2024-07-16T09:55:51.232 INFO:tasks.workunit.client.1.smithi176.stdout:3/746: fsync d3/d17/d1d/d4f/fd4 0 2024-07-16T09:55:51.233 INFO:tasks.workunit.client.0.smithi156.stdout:8/917: dwrite d0/d11/d20/d3f/fb1 [0,4194304] 0 2024-07-16T09:55:51.235 INFO:tasks.workunit.client.0.smithi156.stdout:8/918: read - d0/d74/d115/f12d zero size 2024-07-16T09:55:51.235 INFO:tasks.workunit.client.1.smithi176.stdout:4/841: creat da/d3d/d33/f112 x:0 0 0 2024-07-16T09:55:51.235 INFO:tasks.workunit.client.1.smithi176.stdout:3/747: dread - d3/da/d33/fc9 zero size 2024-07-16T09:55:51.235 INFO:tasks.workunit.client.1.smithi176.stdout:4/842: readlink da/d3d/d17/l34 0 2024-07-16T09:55:51.236 INFO:tasks.workunit.client.1.smithi176.stdout:8/941: creat d8/d23/d100/d61/f139 x:0 0 0 2024-07-16T09:55:51.239 INFO:tasks.workunit.client.1.smithi176.stdout:8/942: creat d8/d14/d1e/d7c/d82/f13a x:0 0 0 2024-07-16T09:55:51.239 INFO:tasks.workunit.client.0.smithi156.stdout:7/875: sync 2024-07-16T09:55:51.239 INFO:tasks.workunit.client.1.smithi176.stdout:3/748: dread - d3/f74 zero size 2024-07-16T09:55:51.240 INFO:tasks.workunit.client.0.smithi156.stdout:7/876: chown dc/d13/d20 1027691 1 2024-07-16T09:55:51.242 INFO:tasks.workunit.client.1.smithi176.stdout:4/843: rename da/dc/d4b/df0/d28/f79 to da/dc/d4b/df0/d96/d9b/db4/d40/d5d/f113 0 2024-07-16T09:55:51.243 INFO:tasks.workunit.client.1.smithi176.stdout:3/749: write d3/d17/d1d/d4f/d59/da0/d7a/dac/fbb [213471,78396] 0 2024-07-16T09:55:51.244 INFO:tasks.workunit.client.0.smithi156.stdout:7/877: rename dc/d13/d36/dd4/dd9 to dc/d48/d119/d12f 0 2024-07-16T09:55:51.245 INFO:tasks.workunit.client.1.smithi176.stdout:8/943: mkdir d8/d23/d100/db9/dcb/d13b 0 2024-07-16T09:55:51.246 INFO:tasks.workunit.client.1.smithi176.stdout:8/944: chown d8/d23/d100/db9/dcb/d37/c91 29 1 2024-07-16T09:55:51.248 INFO:tasks.workunit.client.1.smithi176.stdout:3/750: fsync d3/da/d33/f78 0 2024-07-16T09:55:51.250 INFO:tasks.workunit.client.1.smithi176.stdout:3/751: rmdir d3/d17/d1d/d6e 39 2024-07-16T09:55:51.251 INFO:tasks.workunit.client.1.smithi176.stdout:8/945: dread - d8/d14/d1e/d33/df6/f125 zero size 2024-07-16T09:55:51.253 INFO:tasks.workunit.client.1.smithi176.stdout:3/752: read d3/d17/d1d/d6e/fab [135831,95625] 0 2024-07-16T09:55:51.254 INFO:tasks.workunit.client.1.smithi176.stdout:8/946: symlink d8/d23/d76/d83/d138/l13c 0 2024-07-16T09:55:51.255 INFO:tasks.workunit.client.0.smithi156.stdout:5/956: dread d1b/d1f/d45/d52/d43/d85/d97/f48 [0,4194304] 0 2024-07-16T09:55:51.257 INFO:tasks.workunit.client.1.smithi176.stdout:8/947: rmdir d8/d23/d100/db9 39 2024-07-16T09:55:51.258 INFO:tasks.workunit.client.0.smithi156.stdout:5/957: getdents d1b/d1f/d45/d7a 0 2024-07-16T09:55:51.260 INFO:tasks.workunit.client.1.smithi176.stdout:8/948: mkdir d8/d23/d100/d31/d2f/d132/d13d 0 2024-07-16T09:55:51.261 INFO:tasks.workunit.client.1.smithi176.stdout:3/753: link d3/d17/d1d/d4f/l8f d3/d17/d1d/d4f/d59/ldf 0 2024-07-16T09:55:51.261 INFO:tasks.workunit.client.1.smithi176.stdout:4/844: write da/d3d/d17/d54/d78/d7a/fd6 [909015,99504] 0 2024-07-16T09:55:51.261 INFO:tasks.workunit.client.1.smithi176.stdout:8/949: readlink d8/d14/l22 0 2024-07-16T09:55:51.263 INFO:tasks.workunit.client.0.smithi156.stdout:8/919: truncate d0/d11/d20/d7f/db7/d110/d1f/d57/dc0/fe8 2852800 0 2024-07-16T09:55:51.264 INFO:tasks.workunit.client.0.smithi156.stdout:7/878: rmdir dc 39 2024-07-16T09:55:51.265 INFO:tasks.workunit.client.1.smithi176.stdout:4/845: fsync da/dc/d4b/df0/d96/da4/dc2/f10d 0 2024-07-16T09:55:51.265 INFO:tasks.workunit.client.1.smithi176.stdout:3/754: truncate d3/d17/f88 3298483 0 2024-07-16T09:55:51.266 INFO:tasks.workunit.client.0.smithi156.stdout:7/879: write dc/d13/d15/f12d [229397,94809] 0 2024-07-16T09:55:51.266 INFO:tasks.workunit.client.0.smithi156.stdout:8/920: dread d0/d11/d20/d3f/d96/fad [0,4194304] 0 2024-07-16T09:55:51.269 INFO:tasks.workunit.client.0.smithi156.stdout:7/880: creat dc/d13/d36/dd4/d114/f130 x:0 0 0 2024-07-16T09:55:51.270 INFO:tasks.workunit.client.0.smithi156.stdout:8/921: symlink d0/d11/d2e/d117/l134 0 2024-07-16T09:55:51.271 INFO:tasks.workunit.client.0.smithi156.stdout:7/881: mkdir dc/d48/d119/d12f/d131 0 2024-07-16T09:55:51.272 INFO:tasks.workunit.client.1.smithi176.stdout:8/950: dread d8/d14/d1e/d33/f5b [0,4194304] 0 2024-07-16T09:55:51.273 INFO:tasks.workunit.client.0.smithi156.stdout:7/882: rmdir dc/d13/d15/d8b/dcc/dde 39 2024-07-16T09:55:51.273 INFO:tasks.workunit.client.1.smithi176.stdout:3/755: dwrite d3/da/d84/fc8 [0,4194304] 0 2024-07-16T09:55:51.274 INFO:tasks.workunit.client.1.smithi176.stdout:4/846: read da/dc/d4b/df0/d96/d9b/db4/f26 [7913247,68869] 0 2024-07-16T09:55:51.278 INFO:tasks.workunit.client.0.smithi156.stdout:8/922: rmdir d0/d11/d2e 39 2024-07-16T09:55:51.278 INFO:tasks.workunit.client.1.smithi176.stdout:8/951: creat d8/d14/d1e/d7c/d82/f13e x:0 0 0 2024-07-16T09:55:51.282 INFO:tasks.workunit.client.1.smithi176.stdout:4/847: creat da/d3d/d17/d54/d78/f114 x:0 0 0 2024-07-16T09:55:51.283 INFO:tasks.workunit.client.1.smithi176.stdout:4/848: chown da/d3d/d33 104 1 2024-07-16T09:55:51.283 INFO:tasks.workunit.client.1.smithi176.stdout:8/952: stat d8/d23/d100/db9/fc2 0 2024-07-16T09:55:51.285 INFO:tasks.workunit.client.1.smithi176.stdout:4/849: creat da/d41/f115 x:0 0 0 2024-07-16T09:55:51.287 INFO:tasks.workunit.client.1.smithi176.stdout:8/953: creat d8/d23/d100/d31/d2f/d132/d136/f13f x:0 0 0 2024-07-16T09:55:51.288 INFO:tasks.workunit.client.1.smithi176.stdout:3/756: dread d3/d17/d1d/d4f/d59/f3a [0,4194304] 0 2024-07-16T09:55:51.293 INFO:tasks.workunit.client.0.smithi156.stdout:7/883: write dc/d13/d15/f91 [18968,112774] 0 2024-07-16T09:55:51.293 INFO:tasks.workunit.client.0.smithi156.stdout:8/923: write d0/d11/d62/fa4 [136548,45708] 0 2024-07-16T09:55:51.294 INFO:tasks.workunit.client.1.smithi176.stdout:8/954: dread d8/d23/d100/db9/dcb/d34/d62/f7d [0,4194304] 0 2024-07-16T09:55:51.295 INFO:tasks.workunit.client.1.smithi176.stdout:4/850: dwrite da/dc/d4b/df0/d96/d9b/fe7 [0,4194304] 0 2024-07-16T09:55:51.296 INFO:tasks.workunit.client.0.smithi156.stdout:7/884: creat dc/d3e/d3f/d51/d8a/f132 x:0 0 0 2024-07-16T09:55:51.298 INFO:tasks.workunit.client.1.smithi176.stdout:3/757: write d3/d17/f51 [89860,120708] 0 2024-07-16T09:55:51.299 INFO:tasks.workunit.client.0.smithi156.stdout:8/924: read d0/d11/d29/d80/d97/de7/fd6 [2193076,40561] 0 2024-07-16T09:55:51.299 INFO:tasks.workunit.client.1.smithi176.stdout:3/758: write d3/d17/d1d/d4f/d59/f34 [81973,35144] 0 2024-07-16T09:55:51.300 INFO:tasks.workunit.client.1.smithi176.stdout:8/955: unlink d8/d23/d100/db9/dcb/d34/d62/c121 0 2024-07-16T09:55:51.300 INFO:tasks.workunit.client.0.smithi156.stdout:7/885: unlink dc/d48/d119/f126 0 2024-07-16T09:55:51.301 INFO:tasks.workunit.client.1.smithi176.stdout:4/851: creat da/dc/d4b/df0/d96/d9b/db4/d40/d5d/f116 x:0 0 0 2024-07-16T09:55:51.302 INFO:tasks.workunit.client.0.smithi156.stdout:7/886: readlink dc/d13/d15/d8b/dcc/lfe 0 2024-07-16T09:55:51.302 INFO:tasks.workunit.client.0.smithi156.stdout:8/925: write d0/d11/d20/d7f/db7/d110/d1f/d57/f8f [1510468,90855] 0 2024-07-16T09:55:51.304 INFO:tasks.workunit.client.0.smithi156.stdout:7/887: mknod dc/d63/d98/c133 0 2024-07-16T09:55:51.305 INFO:tasks.workunit.client.1.smithi176.stdout:3/759: creat d3/d17/d1d/d4f/daa/fe0 x:0 0 0 2024-07-16T09:55:51.305 INFO:tasks.workunit.client.1.smithi176.stdout:4/852: rmdir da/d3d/d17/d54/d78/d7a 39 2024-07-16T09:55:51.306 INFO:tasks.workunit.client.0.smithi156.stdout:7/888: truncate dc/d13/d36/f66 5041130 0 2024-07-16T09:55:51.307 INFO:tasks.workunit.client.1.smithi176.stdout:3/760: chown d3/d17/l62 10348962 1 2024-07-16T09:55:51.307 INFO:tasks.workunit.client.1.smithi176.stdout:4/853: chown da/dc/lfb 6 1 2024-07-16T09:55:51.307 INFO:tasks.workunit.client.0.smithi156.stdout:8/926: unlink d0/d11/f40 0 2024-07-16T09:55:51.311 INFO:tasks.workunit.client.1.smithi176.stdout:3/761: creat d3/d17/d1d/d6e/fe1 x:0 0 0 2024-07-16T09:55:51.314 INFO:tasks.workunit.client.0.smithi156.stdout:7/889: rename dc/d13/d15/d8b/dcc to dc/d3e/d76/d134 0 2024-07-16T09:55:51.317 INFO:tasks.workunit.client.1.smithi176.stdout:3/762: creat d3/d17/fe2 x:0 0 0 2024-07-16T09:55:51.320 INFO:tasks.workunit.client.0.smithi156.stdout:7/890: dwrite dc/d13/d20/d4d/fee [0,4194304] 0 2024-07-16T09:55:51.321 INFO:tasks.workunit.client.1.smithi176.stdout:8/956: dread d8/d14/d1e/d33/fd1 [0,4194304] 0 2024-07-16T09:55:51.321 INFO:tasks.workunit.client.0.smithi156.stdout:7/891: stat dc/d63/l87 0 2024-07-16T09:55:51.327 INFO:tasks.workunit.client.1.smithi176.stdout:3/763: truncate d3/da/d66/fa1 7647570 0 2024-07-16T09:55:51.330 INFO:tasks.workunit.client.1.smithi176.stdout:8/957: dwrite d8/d14/d1e/d7c/d82/de0/ff0 [0,4194304] 0 2024-07-16T09:55:51.336 INFO:tasks.workunit.client.1.smithi176.stdout:4/854: dread da/dc/d4b/df0/d96/d69/fd1 [0,4194304] 0 2024-07-16T09:55:51.340 INFO:tasks.workunit.client.1.smithi176.stdout:4/855: fdatasync da/d3d/d17/f64 0 2024-07-16T09:55:51.341 INFO:tasks.workunit.client.1.smithi176.stdout:4/856: dread - da/dc/d4b/fe0 zero size 2024-07-16T09:55:51.343 INFO:tasks.workunit.client.1.smithi176.stdout:4/857: read da/dc/d4b/df0/d96/faf [2817767,101625] 0 2024-07-16T09:55:51.343 INFO:tasks.workunit.client.1.smithi176.stdout:3/764: link d3/d17/d1d/f76 d3/d17/d1d/d4f/d59/da0/d7a/dac/fe3 0 2024-07-16T09:55:51.346 INFO:tasks.workunit.client.1.smithi176.stdout:4/858: dread - da/dc/d4b/df0/d96/d9b/db4/d40/d5d/f100 zero size 2024-07-16T09:55:51.346 INFO:tasks.workunit.client.1.smithi176.stdout:3/765: chown d3/d17/d1d/d4f/d59/da0/d7a/dac/fbb 29422 1 2024-07-16T09:55:51.346 INFO:tasks.workunit.client.1.smithi176.stdout:8/958: write d8/d23/d100/db9/dcb/d34/fde [294291,53921] 0 2024-07-16T09:55:51.349 INFO:tasks.workunit.client.0.smithi156.stdout:7/892: creat dc/d3e/d3f/d51/d112/f135 x:0 0 0 2024-07-16T09:55:51.349 INFO:tasks.workunit.client.1.smithi176.stdout:3/766: fdatasync d3/d17/f40 0 2024-07-16T09:55:51.351 INFO:tasks.workunit.client.1.smithi176.stdout:3/767: dread - d3/d17/fd8 zero size 2024-07-16T09:55:51.351 INFO:tasks.workunit.client.1.smithi176.stdout:8/959: mkdir d8/d23/d100/d31/d46/d77/d140 0 2024-07-16T09:55:51.351 INFO:tasks.workunit.client.0.smithi156.stdout:7/893: symlink dc/l136 0 2024-07-16T09:55:51.354 INFO:tasks.workunit.client.1.smithi176.stdout:8/960: creat d8/d23/d100/d31/d46/f141 x:0 0 0 2024-07-16T09:55:51.355 INFO:tasks.workunit.client.0.smithi156.stdout:7/894: unlink dc/d3e/d3f/d51/fa0 0 2024-07-16T09:55:51.356 INFO:tasks.workunit.client.1.smithi176.stdout:5/928: sync 2024-07-16T09:55:51.356 INFO:tasks.workunit.client.1.smithi176.stdout:9/988: sync 2024-07-16T09:55:51.357 INFO:tasks.workunit.client.1.smithi176.stdout:9/989: readlink d5/l7 0 2024-07-16T09:55:51.361 INFO:tasks.workunit.client.0.smithi156.stdout:7/895: dwrite dc/d3e/d3f/d51/fb0 [0,4194304] 0 2024-07-16T09:55:51.362 INFO:tasks.workunit.client.1.smithi176.stdout:3/768: fsync d3/da/d66/fa1 0 2024-07-16T09:55:51.362 INFO:tasks.workunit.client.1.smithi176.stdout:8/961: write d8/f2e [1479806,106888] 0 2024-07-16T09:55:51.365 INFO:tasks.workunit.client.1.smithi176.stdout:8/962: mknod d8/d14/d1e/d55/c142 0 2024-07-16T09:55:51.366 INFO:tasks.workunit.client.1.smithi176.stdout:8/963: write d8/d20/f12a [243184,73911] 0 2024-07-16T09:55:51.371 INFO:tasks.workunit.client.1.smithi176.stdout:4/859: dread da/d3d/d17/f35 [4194304,4194304] 0 2024-07-16T09:55:51.371 INFO:tasks.workunit.client.1.smithi176.stdout:3/769: write d3/d17/f90 [1448666,129521] 0 2024-07-16T09:55:51.373 INFO:tasks.workunit.client.0.smithi156.stdout:7/896: getdents dc/d48/dd7 0 2024-07-16T09:55:51.376 INFO:tasks.workunit.client.1.smithi176.stdout:3/770: creat d3/d17/d1d/d4f/d6d/fe4 x:0 0 0 2024-07-16T09:55:51.378 INFO:tasks.workunit.client.0.smithi156.stdout:8/927: dwrite d0/d74/f4e [0,4194304] 0 2024-07-16T09:55:51.379 INFO:tasks.workunit.client.0.smithi156.stdout:7/897: dread dc/d63/f88 [0,4194304] 0 2024-07-16T09:55:51.382 INFO:tasks.workunit.client.1.smithi176.stdout:4/860: dwrite da/d3d/f1b [0,4194304] 0 2024-07-16T09:55:51.382 INFO:tasks.workunit.client.1.smithi176.stdout:3/771: fdatasync d3/d17/d1d/d4f/d59/f93 0 2024-07-16T09:55:51.383 INFO:tasks.workunit.client.1.smithi176.stdout:8/964: dwrite d8/d14/d1e/d33/d9d/fa7 [0,4194304] 0 2024-07-16T09:55:51.386 INFO:tasks.workunit.client.0.smithi156.stdout:8/928: getdents d0/d11/d20/d7f/db7 0 2024-07-16T09:55:51.387 INFO:tasks.workunit.client.0.smithi156.stdout:8/929: dread - d0/d11/d20/d3f/dbe/ffc zero size 2024-07-16T09:55:51.388 INFO:tasks.workunit.client.0.smithi156.stdout:8/930: write d0/d11/fc7 [4863514,70939] 0 2024-07-16T09:55:51.397 INFO:tasks.workunit.client.1.smithi176.stdout:5/929: dread d4/d37/d72/f59 [0,4194304] 0 2024-07-16T09:55:51.399 INFO:tasks.workunit.client.1.smithi176.stdout:3/772: dwrite d3/da/d6c/dbc/fcb [0,4194304] 0 2024-07-16T09:55:51.400 INFO:tasks.workunit.client.1.smithi176.stdout:8/965: dwrite d8/d14/d1e/d7c/d82/de0/ff0 [0,4194304] 0 2024-07-16T09:55:51.402 INFO:tasks.workunit.client.1.smithi176.stdout:5/930: rename d4/d14/d19/d1b to d4/d14/d19/d1b/d61/de7/d151 22 2024-07-16T09:55:51.403 INFO:tasks.workunit.client.1.smithi176.stdout:3/773: chown d3/da/d33/f52 19 1 2024-07-16T09:55:51.404 INFO:tasks.workunit.client.1.smithi176.stdout:8/966: write d8/d23/d100/f67 [4410272,77155] 0 2024-07-16T09:55:51.406 INFO:tasks.workunit.client.0.smithi156.stdout:3/997: sync 2024-07-16T09:55:51.407 INFO:tasks.workunit.client.0.smithi156.stdout:6/873: sync 2024-07-16T09:55:51.407 INFO:tasks.workunit.client.1.smithi176.stdout:3/774: chown c2 12608 1 2024-07-16T09:55:51.410 INFO:tasks.workunit.client.1.smithi176.stdout:3/775: read d3/d17/d1d/d6e/fa4 [2544307,80997] 0 2024-07-16T09:55:51.412 INFO:tasks.workunit.client.1.smithi176.stdout:8/967: rename d8/d14/d1e/d33/df6/l112 to d8/d23/d100/d31/d46/d77/l143 0 2024-07-16T09:55:51.413 INFO:tasks.workunit.client.1.smithi176.stdout:4/861: write da/dc/d4b/df0/d96/d9b/fdb [418412,83243] 0 2024-07-16T09:55:51.415 INFO:tasks.workunit.client.1.smithi176.stdout:4/862: chown da/d41/dc3/ce9 28738877 1 2024-07-16T09:55:51.415 INFO:tasks.workunit.client.1.smithi176.stdout:8/968: write d8/d23/d100/d31/d2f/d132/d136/f13f [888870,100175] 0 2024-07-16T09:55:51.416 INFO:tasks.workunit.client.0.smithi156.stdout:3/998: dwrite d8/da/d28/d43/fd9 [0,4194304] 0 2024-07-16T09:55:51.418 INFO:tasks.workunit.client.1.smithi176.stdout:3/776: creat d3/d17/d1d/d4f/d59/fe5 x:0 0 0 2024-07-16T09:55:51.418 INFO:tasks.workunit.client.1.smithi176.stdout:5/931: truncate d4/f27 2027819 0 2024-07-16T09:55:51.421 INFO:tasks.workunit.client.0.smithi156.stdout:3/999: symlink d8/da/d1d/d2c/de0/l150 0 2024-07-16T09:55:51.421 INFO:tasks.workunit.client.0.smithi156.stdout:6/874: dwrite d3/d13/d17/f6c [0,4194304] 0 2024-07-16T09:55:51.426 INFO:tasks.workunit.client.1.smithi176.stdout:4/863: creat da/d3d/d33/d104/f117 x:0 0 0 2024-07-16T09:55:51.426 INFO:tasks.workunit.client.1.smithi176.stdout:8/969: symlink d8/d23/d100/db9/dcb/d13b/l144 0 2024-07-16T09:55:51.428 INFO:tasks.workunit.client.1.smithi176.stdout:5/932: mknod d4/d14/d19/d1b/d95/c152 0 2024-07-16T09:55:51.429 INFO:tasks.workunit.client.1.smithi176.stdout:4/864: read da/fcf [731050,72579] 0 2024-07-16T09:55:51.432 INFO:tasks.workunit.client.1.smithi176.stdout:8/970: fdatasync d8/d23/d100/db9/dcb/f3b 0 2024-07-16T09:55:51.432 INFO:tasks.workunit.client.1.smithi176.stdout:4/865: chown da/dc/d4b/df0/d28/c53 2664570 1 2024-07-16T09:55:51.439 INFO:tasks.workunit.client.1.smithi176.stdout:4/866: creat da/d3d/d17/d54/d78/f118 x:0 0 0 2024-07-16T09:55:51.441 INFO:tasks.workunit.client.1.smithi176.stdout:3/777: dread d3/da/f77 [0,4194304] 0 2024-07-16T09:55:51.441 INFO:tasks.workunit.client.1.smithi176.stdout:4/867: dread da/dc/d4b/df0/d96/d69/f9f [0,4194304] 0 2024-07-16T09:55:51.444 INFO:tasks.workunit.client.1.smithi176.stdout:4/868: creat da/d3d/dac/f119 x:0 0 0 2024-07-16T09:55:51.444 INFO:tasks.workunit.client.1.smithi176.stdout:4/869: chown da/d3d/d17 2126 1 2024-07-16T09:55:51.446 INFO:tasks.workunit.client.1.smithi176.stdout:4/870: truncate da/dc/d4b/fe0 1006568 0 2024-07-16T09:55:51.451 INFO:tasks.workunit.client.1.smithi176.stdout:8/971: write d8/d14/d1e/d33/f78 [241320,3167] 0 2024-07-16T09:55:51.454 INFO:tasks.workunit.client.1.smithi176.stdout:8/972: write d8/d20/f5a [7973304,97074] 0 2024-07-16T09:55:51.456 INFO:tasks.workunit.client.0.smithi156.stdout:6/875: dread d3/d13/d17/d4f/d4d/d3b/f5d [0,4194304] 0 2024-07-16T09:55:51.458 INFO:tasks.workunit.client.0.smithi156.stdout:6/876: mkdir d3/d13/d17/d56/d11b 0 2024-07-16T09:55:51.459 INFO:tasks.workunit.client.1.smithi176.stdout:8/973: creat d8/d14/d1e/f145 x:0 0 0 2024-07-16T09:55:51.459 INFO:tasks.workunit.client.1.smithi176.stdout:5/933: dwrite d4/d37/d72/d38/d39/f129 [0,4194304] 0 2024-07-16T09:55:51.459 INFO:tasks.workunit.client.0.smithi156.stdout:6/877: read - d3/d13/fab zero size 2024-07-16T09:55:51.460 INFO:tasks.workunit.client.0.smithi156.stdout:6/878: fsync d3/d13/d17/d4f/d4d/d3b/fbd 0 2024-07-16T09:55:51.461 INFO:tasks.workunit.client.1.smithi176.stdout:3/778: dwrite d3/d17/f88 [0,4194304] 0 2024-07-16T09:55:51.463 INFO:tasks.workunit.client.0.smithi156.stdout:6/879: creat d3/d13/d17/dae/f11c x:0 0 0 2024-07-16T09:55:51.466 INFO:tasks.workunit.client.1.smithi176.stdout:8/974: chown d8/d23/d100/d53/l134 39017 1 2024-07-16T09:55:51.466 INFO:tasks.workunit.client.1.smithi176.stdout:4/871: dread da/dc/d4b/df0/d96/d69/fb0 [0,4194304] 0 2024-07-16T09:55:51.470 INFO:tasks.workunit.client.0.smithi156.stdout:6/880: dwrite d3/d13/d17/f6c [4194304,4194304] 0 2024-07-16T09:55:51.471 INFO:tasks.workunit.client.1.smithi176.stdout:3/779: creat d3/d17/d1d/d4f/d59/d81/fe6 x:0 0 0 2024-07-16T09:55:51.471 INFO:tasks.workunit.client.1.smithi176.stdout:8/975: truncate d8/d14/d1e/d7c/f102 700616 0 2024-07-16T09:55:51.473 INFO:tasks.workunit.client.1.smithi176.stdout:8/976: mkdir d8/d23/d100/db9/dcb/d13b/d146 0 2024-07-16T09:55:51.474 INFO:tasks.workunit.client.0.smithi156.stdout:6/881: mknod d3/d13/d17/d4f/d4d/d3b/da5/dbc/c11d 0 2024-07-16T09:55:51.475 INFO:tasks.workunit.client.1.smithi176.stdout:4/872: dwrite da/d3d/d33/fdd [0,4194304] 0 2024-07-16T09:55:51.478 INFO:tasks.workunit.client.0.smithi156.stdout:6/882: mkdir d3/d13/d17/d4f/d4d/d3b/d77/d11e 0 2024-07-16T09:55:51.482 INFO:tasks.workunit.client.1.smithi176.stdout:8/977: symlink d8/d23/d100/db9/dcb/d34/d62/l147 0 2024-07-16T09:55:51.483 INFO:tasks.workunit.client.0.smithi156.stdout:6/883: link d3/d13/d17/f8c d3/d13/d17/d4f/d94/df3/f11f 0 2024-07-16T09:55:51.486 INFO:tasks.workunit.client.1.smithi176.stdout:8/978: unlink d8/d14/d1e/d33/f78 0 2024-07-16T09:55:51.488 INFO:tasks.workunit.client.1.smithi176.stdout:5/934: dwrite d4/d37/d72/f59 [0,4194304] 0 2024-07-16T09:55:51.488 INFO:tasks.workunit.client.0.smithi156.stdout:6/884: mkdir d3/d13/d17/d4f/d94/d19/d31/d120 0 2024-07-16T09:55:51.488 INFO:tasks.workunit.client.1.smithi176.stdout:4/873: creat da/d3d/f11a x:0 0 0 2024-07-16T09:55:51.490 INFO:tasks.workunit.client.1.smithi176.stdout:4/874: dread - da/d3d/d33/fcd zero size 2024-07-16T09:55:51.492 INFO:tasks.workunit.client.1.smithi176.stdout:4/875: chown da/d3d/d17/l5c 31 1 2024-07-16T09:55:51.493 INFO:tasks.workunit.client.0.smithi156.stdout:6/885: dread d3/d13/d17/d4f/d94/df3/f11f [0,4194304] 0 2024-07-16T09:55:51.494 INFO:tasks.workunit.client.1.smithi176.stdout:4/876: stat da/fb 0 2024-07-16T09:55:51.497 INFO:tasks.workunit.client.0.smithi156.stdout:6/886: read d3/d13/d17/d4f/d5c/f80 [32976,20535] 0 2024-07-16T09:55:51.497 INFO:tasks.workunit.client.1.smithi176.stdout:4/877: fdatasync da/d3d/d17/f18 0 2024-07-16T09:55:51.498 INFO:tasks.workunit.client.0.smithi156.stdout:6/887: write d3/d13/d17/d4f/f34 [3929050,62747] 0 2024-07-16T09:55:51.503 INFO:tasks.workunit.client.1.smithi176.stdout:4/878: link da/dc/d4b/df0/d96/d69/d86/c89 da/d3d/d17/c11b 0 2024-07-16T09:55:51.508 INFO:tasks.workunit.client.1.smithi176.stdout:8/979: dwrite d8/d14/d1e/d55/fa3 [0,4194304] 0 2024-07-16T09:55:51.508 INFO:tasks.workunit.client.1.smithi176.stdout:4/879: symlink da/dc/d4b/df0/d96/d9b/db4/d40/d5d/l11c 0 2024-07-16T09:55:51.513 INFO:tasks.workunit.client.1.smithi176.stdout:8/980: rename d8/df7 to d8/d23/d100/db9/d148 0 2024-07-16T09:55:51.514 INFO:tasks.workunit.client.1.smithi176.stdout:8/981: readlink d8/d23/d100/db9/le1 0 2024-07-16T09:55:51.517 INFO:tasks.workunit.client.1.smithi176.stdout:5/935: dread d4/d14/fb4 [0,4194304] 0 2024-07-16T09:55:51.518 INFO:tasks.workunit.client.1.smithi176.stdout:8/982: mkdir d8/d23/d100/d31/d46/d6a/da8/d149 0 2024-07-16T09:55:51.520 INFO:tasks.workunit.client.1.smithi176.stdout:7/835: sync 2024-07-16T09:55:51.521 INFO:tasks.workunit.client.1.smithi176.stdout:1/623: sync 2024-07-16T09:55:51.523 INFO:tasks.workunit.client.1.smithi176.stdout:7/836: write d4/dd/fe3 [193005,73952] 0 2024-07-16T09:55:51.525 INFO:tasks.workunit.client.1.smithi176.stdout:4/880: dwrite da/dc/d4b/df0/d96/d9b/db4/fa5 [0,4194304] 0 2024-07-16T09:55:51.528 INFO:tasks.workunit.client.1.smithi176.stdout:8/983: getdents d8/d23/d76/d83/d138 0 2024-07-16T09:55:51.528 INFO:tasks.workunit.client.1.smithi176.stdout:4/881: mknod da/dc/d4b/c11d 0 2024-07-16T09:55:51.529 INFO:tasks.workunit.client.1.smithi176.stdout:7/837: write d4/dd/d26/fc1 [790271,33684] 0 2024-07-16T09:55:51.530 INFO:tasks.workunit.client.1.smithi176.stdout:5/936: write d4/d10/f5f [4408655,51364] 0 2024-07-16T09:55:51.532 INFO:tasks.workunit.client.1.smithi176.stdout:7/838: mknod d4/dd/d8e/c109 0 2024-07-16T09:55:51.536 INFO:tasks.workunit.client.1.smithi176.stdout:7/839: mknod d4/d47/d6d/d89/d66/c10a 0 2024-07-16T09:55:51.538 INFO:tasks.workunit.client.1.smithi176.stdout:8/984: dwrite d8/d14/d1e/d33/d9d/fb8 [0,4194304] 0 2024-07-16T09:55:51.539 INFO:tasks.workunit.client.1.smithi176.stdout:7/840: symlink d4/d47/d5d/d9a/de0/l10b 0 2024-07-16T09:55:51.542 INFO:tasks.workunit.client.1.smithi176.stdout:7/841: chown d4/dd/db5/fe9 8 1 2024-07-16T09:55:51.544 INFO:tasks.workunit.client.1.smithi176.stdout:7/842: chown d4/dd/db5/fe1 248887 1 2024-07-16T09:55:51.545 INFO:tasks.workunit.client.1.smithi176.stdout:1/624: truncate d4/d8/d12/d21/d25/d30/d44/d4d/f91 809904 0 2024-07-16T09:55:51.546 INFO:tasks.workunit.client.1.smithi176.stdout:5/937: write d4/d37/d72/d38/d39/fa1 [1129672,67820] 0 2024-07-16T09:55:51.547 INFO:tasks.workunit.client.1.smithi176.stdout:8/985: dwrite d8/d14/d1e/f145 [0,4194304] 0 2024-07-16T09:55:51.548 INFO:tasks.workunit.client.1.smithi176.stdout:7/843: fdatasync d4/dd/d49/dd5/f108 0 2024-07-16T09:55:51.550 INFO:tasks.workunit.client.1.smithi176.stdout:1/625: rmdir d4/d8/d12/d21/d25/da8 39 2024-07-16T09:55:51.552 INFO:tasks.workunit.client.1.smithi176.stdout:8/986: getdents d8/d23/d100/d53/d93 0 2024-07-16T09:55:51.553 INFO:tasks.workunit.client.1.smithi176.stdout:5/938: symlink d4/d14/dc5/l153 0 2024-07-16T09:55:51.554 INFO:tasks.workunit.client.1.smithi176.stdout:1/626: symlink d4/d3a/d54/d5a/lca 0 2024-07-16T09:55:51.556 INFO:tasks.workunit.client.1.smithi176.stdout:8/987: mkdir d8/d23/d100/d31/d46/d14a 0 2024-07-16T09:55:51.558 INFO:tasks.workunit.client.1.smithi176.stdout:1/627: symlink d4/d8/d12/d21/d34/daa/db8/lcb 0 2024-07-16T09:55:51.559 INFO:tasks.workunit.client.1.smithi176.stdout:8/988: creat d8/d10f/f14b x:0 0 0 2024-07-16T09:55:51.560 INFO:tasks.workunit.client.1.smithi176.stdout:1/628: mknod d4/d3a/d54/ccc 0 2024-07-16T09:55:51.563 INFO:tasks.workunit.client.1.smithi176.stdout:8/989: rename d8/d14/d1e/f145 to d8/d23/d76/d83/d103/f14c 0 2024-07-16T09:55:51.564 INFO:tasks.workunit.client.1.smithi176.stdout:8/990: fdatasync d8/d23/d100/d31/d46/f141 0 2024-07-16T09:55:51.566 INFO:tasks.workunit.client.1.smithi176.stdout:1/629: creat d4/d8/d12/d21/d34/daa/fcd x:0 0 0 2024-07-16T09:55:51.571 INFO:tasks.workunit.client.1.smithi176.stdout:7/844: truncate d4/dd/d49/dd5/f108 1124082 0 2024-07-16T09:55:51.579 INFO:tasks.workunit.client.1.smithi176.stdout:7/845: write d4/dd/db5/ff8 [255596,128592] 0 2024-07-16T09:55:51.579 INFO:tasks.workunit.client.1.smithi176.stdout:5/939: write d4/d37/f3f [441422,35336] 0 2024-07-16T09:55:51.585 INFO:tasks.workunit.client.1.smithi176.stdout:8/991: write d8/d23/d100/d31/f5c [190987,78828] 0 2024-07-16T09:55:51.589 INFO:tasks.workunit.client.1.smithi176.stdout:1/630: dread d4/d8/d12/d21/d25/d30/fbe [0,4194304] 0 2024-07-16T09:55:51.599 INFO:tasks.workunit.client.0.smithi156.stdout:5/958: sync 2024-07-16T09:55:51.604 INFO:tasks.workunit.client.1.smithi176.stdout:7/846: creat d4/d47/d5d/d9a/de0/f10c x:0 0 0 2024-07-16T09:55:51.605 INFO:tasks.workunit.client.1.smithi176.stdout:7/847: fdatasync d4/dd/f2e 0 2024-07-16T09:55:51.611 INFO:tasks.workunit.client.1.smithi176.stdout:7/848: rmdir d4/dd/d49/d7a/db1/deb/ded 0 2024-07-16T09:55:51.611 INFO:tasks.workunit.client.1.smithi176.stdout:7/849: write d4/dd/f2e [2461555,1599] 0 2024-07-16T09:55:51.621 INFO:tasks.workunit.client.1.smithi176.stdout:7/850: dwrite d4/d47/d6d/d89/fc8 [4194304,4194304] 0 2024-07-16T09:55:51.633 INFO:tasks.workunit.client.0.smithi156.stdout:5/959: creat d1b/d1f/d45/d52/d43/d85/d97/d37/f12f x:0 0 0 2024-07-16T09:55:51.636 INFO:tasks.workunit.client.1.smithi176.stdout:8/992: dwrite d8/d23/d100/d31/f3e [0,4194304] 0 2024-07-16T09:55:51.636 INFO:tasks.workunit.client.1.smithi176.stdout:1/631: write d4/d8/d12/d21/d25/d30/d44/d4d/f91 [43067,68197] 0 2024-07-16T09:55:51.636 INFO:tasks.workunit.client.1.smithi176.stdout:5/940: dwrite d4/d10/d17/f63 [0,4194304] 0 2024-07-16T09:55:51.641 INFO:tasks.workunit.client.1.smithi176.stdout:7/851: dwrite d4/d47/d5d/d9a/ffd [0,4194304] 0 2024-07-16T09:55:51.642 INFO:tasks.workunit.client.1.smithi176.stdout:5/941: chown d4/da9/le5 1014308 1 2024-07-16T09:55:51.645 INFO:tasks.workunit.client.1.smithi176.stdout:7/852: fsync d4/d1f/dce/fe8 0 2024-07-16T09:55:51.645 INFO:tasks.workunit.client.1.smithi176.stdout:9/990: sync 2024-07-16T09:55:51.646 INFO:tasks.workunit.client.1.smithi176.stdout:1/632: write d4/d8/d12/f4b [3414752,65729] 0 2024-07-16T09:55:51.647 INFO:tasks.workunit.client.1.smithi176.stdout:8/993: write d8/d23/d100/d53/d93/ffc [3841194,87454] 0 2024-07-16T09:55:51.649 INFO:tasks.workunit.client.1.smithi176.stdout:7/853: write d4/dd/d26/dc2/fe7 [1393749,111005] 0 2024-07-16T09:55:51.653 INFO:tasks.workunit.client.1.smithi176.stdout:7/854: symlink d4/dd/d26/dc2/l10d 0 2024-07-16T09:55:51.654 INFO:tasks.workunit.client.1.smithi176.stdout:1/633: mknod d4/d8/d12/d21/d25/da8/cce 0 2024-07-16T09:55:51.655 INFO:tasks.workunit.client.1.smithi176.stdout:7/855: write d4/d47/d5d/d9a/ffd [2666101,64041] 0 2024-07-16T09:55:51.663 INFO:tasks.workunit.client.1.smithi176.stdout:1/634: mknod d4/d8/d12/d21/d25/ccf 0 2024-07-16T09:55:51.665 INFO:tasks.workunit.client.1.smithi176.stdout:1/635: creat d4/d3a/dc8/fd0 x:0 0 0 2024-07-16T09:55:51.668 INFO:tasks.workunit.client.1.smithi176.stdout:1/636: creat d4/d8/d12/d21/d25/da8/fd1 x:0 0 0 2024-07-16T09:55:51.670 INFO:tasks.workunit.client.1.smithi176.stdout:7/856: fdatasync d4/dd/d49/f7c 0 2024-07-16T09:55:51.671 INFO:tasks.workunit.client.1.smithi176.stdout:1/637: fdatasync d4/d8/d12/f4b 0 2024-07-16T09:55:51.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:55:51 smithi156 ceph-mon[29302]: Checking for cert/key for grafana.smithi156 2024-07-16T09:55:51.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:55:51 smithi156 ceph-mon[29302]: Migrating grafana.smithi156 cert to cert store 2024-07-16T09:55:51.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:55:51 smithi156 ceph-mon[29302]: Migrating grafana.smithi156 key to cert store 2024-07-16T09:55:51.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:55:51 smithi156 ceph-mon[29302]: mgrmap e22: smithi176.dqbrvx(active, since 1.01712s) 2024-07-16T09:55:51.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:55:51 smithi156 ceph-mon[29302]: pgmap v3: 65 pgs: 65 active+clean; 3.8 GiB data, 13 GiB used, 524 GiB / 536 GiB avail 2024-07-16T09:55:51.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:55:51 smithi156 ceph-mon[29302]: from='mgr.24377 ' entity='mgr.smithi176.dqbrvx' 2024-07-16T09:55:51.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:55:51 smithi156 ceph-mon[29302]: from='mgr.24377 ' entity='mgr.smithi176.dqbrvx' 2024-07-16T09:55:51.675 INFO:tasks.workunit.client.1.smithi176.stdout:9/991: write d5/d1d/d20/f81 [165193,63407] 0 2024-07-16T09:55:51.677 INFO:tasks.workunit.client.1.smithi176.stdout:4/882: sync 2024-07-16T09:55:51.677 INFO:tasks.workunit.client.1.smithi176.stdout:3/780: sync 2024-07-16T09:55:51.677 INFO:tasks.workunit.client.1.smithi176.stdout:8/994: truncate d8/d23/d100/d61/d90/f107 2462759 0 2024-07-16T09:55:51.687 INFO:tasks.workunit.client.1.smithi176.stdout:5/942: dwrite d4/d37/fd7 [0,4194304] 0 2024-07-16T09:55:51.687 INFO:tasks.workunit.client.1.smithi176.stdout:4/883: symlink da/d3d/d17/d54/d78/d7a/l11e 0 2024-07-16T09:55:51.689 INFO:tasks.workunit.client.1.smithi176.stdout:5/943: truncate d4/da9/dc4/f147 75353 0 2024-07-16T09:55:51.689 INFO:tasks.workunit.client.1.smithi176.stdout:8/995: mkdir d8/d23/d100/d31/d2f/d132/d13d/d14d 0 2024-07-16T09:55:51.697 INFO:tasks.workunit.client.1.smithi176.stdout:3/781: write d3/d17/f40 [5467473,54314] 0 2024-07-16T09:55:51.698 INFO:tasks.workunit.client.1.smithi176.stdout:3/782: fdatasync d3/d17/fe2 0 2024-07-16T09:55:51.699 INFO:tasks.workunit.client.1.smithi176.stdout:1/638: dread d4/f24 [0,4194304] 0 2024-07-16T09:55:51.705 INFO:tasks.workunit.client.1.smithi176.stdout:9/992: dread d5/fb8 [0,4194304] 0 2024-07-16T09:55:51.706 INFO:tasks.workunit.client.1.smithi176.stdout:8/996: creat d8/d23/d100/d31/d46/d6a/da8/f14e x:0 0 0 2024-07-16T09:55:51.707 INFO:tasks.workunit.client.1.smithi176.stdout:4/884: creat da/dc/d4b/d62/f11f x:0 0 0 2024-07-16T09:55:51.709 INFO:tasks.workunit.client.1.smithi176.stdout:7/857: dwrite d4/dd/d26/dc2/d17/f4d [0,4194304] 0 2024-07-16T09:55:51.716 INFO:tasks.workunit.client.1.smithi176.stdout:3/783: dread d3/da/d6c/dbc/fcb [0,4194304] 0 2024-07-16T09:55:51.716 INFO:tasks.workunit.client.1.smithi176.stdout:8/997: chown d8/d23/d100/db9/dcb/d34/d62/d88/cd8 410728153 1 2024-07-16T09:55:51.716 INFO:tasks.workunit.client.1.smithi176.stdout:4/885: read da/d3d/f103 [182931,2365] 0 2024-07-16T09:55:51.721 INFO:tasks.workunit.client.1.smithi176.stdout:9/993: dread - d5/de/d30/f10e zero size 2024-07-16T09:55:51.726 INFO:tasks.workunit.client.1.smithi176.stdout:9/994: chown d5/de/c100 6557347 1 2024-07-16T09:55:51.726 INFO:tasks.workunit.client.1.smithi176.stdout:5/944: dwrite d4/d37/d72/d38/f7e [0,4194304] 0 2024-07-16T09:55:51.731 INFO:tasks.workunit.client.1.smithi176.stdout:1/639: dwrite d4/d8/d12/d21/d25/da8/f97 [0,4194304] 0 2024-07-16T09:55:51.731 INFO:tasks.workunit.client.1.smithi176.stdout:8/998: mkdir d8/d23/d100/d31/d46/d14f 0 2024-07-16T09:55:51.734 INFO:tasks.workunit.client.1.smithi176.stdout:7/858: dwrite d4/d47/d5d/fad [0,4194304] 0 2024-07-16T09:55:51.739 INFO:tasks.workunit.client.0.smithi156.stdout:7/898: sync 2024-07-16T09:55:51.739 INFO:tasks.workunit.client.0.smithi156.stdout:8/931: sync 2024-07-16T09:55:51.740 INFO:tasks.workunit.client.1.smithi176.stdout:5/945: mknod d4/d10/c154 0 2024-07-16T09:55:51.740 INFO:tasks.workunit.client.1.smithi176.stdout:1/640: readlink d4/d3a/l70 0 2024-07-16T09:55:51.740 INFO:tasks.workunit.client.1.smithi176.stdout:5/946: readlink d4/ld2 0 2024-07-16T09:55:51.743 INFO:tasks.workunit.client.0.smithi156.stdout:7/899: write dc/d63/f6b [1319956,90835] 0 2024-07-16T09:55:51.745 INFO:tasks.workunit.client.0.smithi156.stdout:7/900: stat dc/d3e/d76/df1/l127 0 2024-07-16T09:55:51.745 INFO:tasks.workunit.client.1.smithi176.stdout:3/784: dwrite d3/da/d33/f44 [0,4194304] 0 2024-07-16T09:55:51.745 INFO:tasks.workunit.client.1.smithi176.stdout:9/995: dread d5/de/d30/fe9 [0,4194304] 0 2024-07-16T09:55:51.747 INFO:tasks.workunit.client.1.smithi176.stdout:5/947: symlink d4/d14/d19/l155 0 2024-07-16T09:55:51.748 INFO:tasks.workunit.client.0.smithi156.stdout:8/932: dwrite d0/d11/d13/d1c/d24/fb9 [0,4194304] 0 2024-07-16T09:55:51.750 INFO:tasks.workunit.client.0.smithi156.stdout:8/933: write d0/d11/ded/f10c [844925,11566] 0 2024-07-16T09:55:51.751 INFO:tasks.workunit.client.1.smithi176.stdout:9/996: write d5/d1d/d20/d78/f120 [1489525,91969] 0 2024-07-16T09:55:51.752 INFO:tasks.workunit.client.1.smithi176.stdout:8/999: truncate d8/d23/d100/d31/d2f/fe6 265891 0 2024-07-16T09:55:51.754 INFO:tasks.workunit.client.1.smithi176.stdout:5/948: chown d4/d6c/f7c 1 1 2024-07-16T09:55:51.754 INFO:tasks.workunit.client.1.smithi176.stdout:7/859: truncate d4/d47/d5d/d9a/de0/ff2 850657 0 2024-07-16T09:55:51.755 INFO:tasks.workunit.client.1.smithi176.stdout:5/949: write d4/f9e [515827,74777] 0 2024-07-16T09:55:51.756 INFO:tasks.workunit.client.0.smithi156.stdout:8/934: mkdir d0/d11/d20/d7f/db7/d110/d1f/d57/d11b/d135 0 2024-07-16T09:55:51.756 INFO:tasks.workunit.client.1.smithi176.stdout:1/641: truncate d4/d8/d12/d21/d25/d30/f61 943661 0 2024-07-16T09:55:51.759 INFO:tasks.workunit.client.1.smithi176.stdout:9/997: truncate d5/d1d/dd2/f135 645949 0 2024-07-16T09:55:51.759 INFO:tasks.workunit.client.0.smithi156.stdout:8/935: chown d0/f78 64789 1 2024-07-16T09:55:51.763 INFO:tasks.workunit.client.1.smithi176.stdout:7/860: symlink d4/dd/d49/d7a/db1/l10e 0 2024-07-16T09:55:51.763 INFO:tasks.workunit.client.0.smithi156.stdout:8/936: creat d0/d11/d62/dc2/f136 x:0 0 0 2024-07-16T09:55:51.764 INFO:tasks.workunit.client.0.smithi156.stdout:8/937: fdatasync d0/d11/ded/f10c 0 2024-07-16T09:55:51.767 INFO:tasks.workunit.client.1.smithi176.stdout:7/861: chown d4/dd/d26/l40 12234 1 2024-07-16T09:55:51.770 INFO:tasks.workunit.client.0.smithi156.stdout:8/938: link d0/d11/d20/d3f/c56 d0/d11/d62/dc2/dcd/d108/c137 0 2024-07-16T09:55:51.770 INFO:tasks.workunit.client.1.smithi176.stdout:7/862: mkdir d4/d5/d10f 0 2024-07-16T09:55:51.770 INFO:tasks.workunit.client.1.smithi176.stdout:3/785: dread d3/d17/d1d/d4f/d59/f93 [0,4194304] 0 2024-07-16T09:55:51.771 INFO:tasks.workunit.client.1.smithi176.stdout:9/998: dwrite d5/de/f132 [0,4194304] 0 2024-07-16T09:55:51.773 INFO:tasks.workunit.client.1.smithi176.stdout:3/786: unlink d3/da/ld 0 2024-07-16T09:55:51.779 INFO:tasks.workunit.client.1.smithi176.stdout:5/950: write d4/f27 [1991102,115731] 0 2024-07-16T09:55:51.780 INFO:tasks.workunit.client.1.smithi176.stdout:9/999: rename d5/de/da5/dbf/cce to d5/d1d/dd2/c14c 0 2024-07-16T09:55:51.781 INFO:tasks.workunit.client.1.smithi176.stdout:1/642: write d4/d8/d12/d21/d25/d30/d44/d4d/f96 [585793,87837] 0 2024-07-16T09:55:51.783 INFO:tasks.workunit.client.1.smithi176.stdout:1/643: truncate d4/d8/fc6 510152 0 2024-07-16T09:55:51.785 INFO:tasks.workunit.client.1.smithi176.stdout:1/644: write d4/d8/f26 [2473394,43436] 0 2024-07-16T09:55:51.787 INFO:tasks.workunit.client.1.smithi176.stdout:1/645: chown d4/d8/d12/d21/d25 940 1 2024-07-16T09:55:51.788 INFO:tasks.workunit.client.1.smithi176.stdout:7/863: write d4/dd/d26/dc2/d17/d1d/d8b/f104 [358639,88639] 0 2024-07-16T09:55:51.789 INFO:tasks.workunit.client.1.smithi176.stdout:1/646: write d4/d8/d12/d21/d25/d30/d44/d4d/f7e [614730,81956] 0 2024-07-16T09:55:51.791 INFO:tasks.workunit.client.1.smithi176.stdout:3/787: truncate d3/d17/f88 282688 0 2024-07-16T09:55:51.795 INFO:tasks.workunit.client.1.smithi176.stdout:1/647: rename d4/d8/d12/d21/d34/daa/db8 to d4/d8/d12/d21/d25/d30/d44/dd2 0 2024-07-16T09:55:51.797 INFO:tasks.workunit.client.1.smithi176.stdout:1/648: write d4/d8/d12/d21/d25/da8/fb2 [44856,57877] 0 2024-07-16T09:55:51.797 INFO:tasks.workunit.client.1.smithi176.stdout:3/788: write d3/d17/d1d/d6e/fab [392167,3412] 0 2024-07-16T09:55:51.802 INFO:tasks.workunit.client.1.smithi176.stdout:1/649: fdatasync d4/d8/d12/d21/d25/f90 0 2024-07-16T09:55:51.802 INFO:tasks.workunit.client.0.smithi156.stdout:8/939: dread d0/d11/fc7 [0,4194304] 0 2024-07-16T09:55:51.804 INFO:tasks.workunit.client.1.smithi176.stdout:5/951: dread d4/d14/f9c [0,4194304] 0 2024-07-16T09:55:51.804 INFO:tasks.workunit.client.1.smithi176.stdout:1/650: dread - d4/d8/d12/d21/f89 zero size 2024-07-16T09:55:51.805 INFO:tasks.workunit.client.1.smithi176.stdout:3/789: write d3/da/f41 [1460829,18010] 0 2024-07-16T09:55:51.806 INFO:tasks.workunit.client.0.smithi156.stdout:8/940: creat d0/d11/d20/f138 x:0 0 0 2024-07-16T09:55:51.809 INFO:tasks.workunit.client.0.smithi156.stdout:8/941: mknod d0/d11/d20/d7f/daf/c139 0 2024-07-16T09:55:51.811 INFO:tasks.workunit.client.1.smithi176.stdout:1/651: write d4/db/f5c [666809,15497] 0 2024-07-16T09:55:51.812 INFO:tasks.workunit.client.1.smithi176.stdout:3/790: dread d3/d17/d1d/faf [0,4194304] 0 2024-07-16T09:55:51.814 INFO:tasks.workunit.client.1.smithi176.stdout:5/952: dread d4/d37/d72/d38/d39/fa3 [0,4194304] 0 2024-07-16T09:55:51.814 INFO:tasks.workunit.client.1.smithi176.stdout:1/652: mknod d4/d77/dad/cd3 0 2024-07-16T09:55:51.815 INFO:tasks.workunit.client.0.smithi156.stdout:8/942: dwrite d0/d11/d20/d7f/db7/d110/d1f/d57/fbf [0,4194304] 0 2024-07-16T09:55:51.815 INFO:tasks.workunit.client.1.smithi176.stdout:1/653: stat d4/d8/d12/d4c/c50 0 2024-07-16T09:55:51.816 INFO:tasks.workunit.client.1.smithi176.stdout:3/791: symlink d3/dd1/le7 0 2024-07-16T09:55:51.817 INFO:tasks.workunit.client.1.smithi176.stdout:1/654: stat d4/d3a/d54/d5a/d7a 0 2024-07-16T09:55:51.820 INFO:tasks.workunit.client.1.smithi176.stdout:3/792: write d3/d17/d1d/d4f/d59/fd0 [3776039,16840] 0 2024-07-16T09:55:51.822 INFO:tasks.workunit.client.1.smithi176.stdout:5/953: dwrite d4/d10/fb1 [0,4194304] 0 2024-07-16T09:55:51.825 INFO:tasks.workunit.client.0.smithi156.stdout:8/943: dwrite d0/d11/d2e/d35/fe2 [0,4194304] 0 2024-07-16T09:55:51.829 INFO:tasks.workunit.client.1.smithi176.stdout:3/793: read d3/d17/f23 [736101,114794] 0 2024-07-16T09:55:51.829 INFO:tasks.workunit.client.0.smithi156.stdout:8/944: symlink d0/d11/d13/d1c/d24/l13a 0 2024-07-16T09:55:51.830 INFO:tasks.workunit.client.1.smithi176.stdout:5/954: fdatasync d4/d10/d17/f66 0 2024-07-16T09:55:51.830 INFO:tasks.workunit.client.0.smithi156.stdout:8/945: readlink d0/d11/d20/l31 0 2024-07-16T09:55:51.831 INFO:tasks.workunit.client.1.smithi176.stdout:3/794: write d3/d17/f40 [4539024,105052] 0 2024-07-16T09:55:51.833 INFO:tasks.workunit.client.0.smithi156.stdout:5/960: sync 2024-07-16T09:55:51.833 INFO:tasks.workunit.client.0.smithi156.stdout:6/888: sync 2024-07-16T09:55:51.834 INFO:tasks.workunit.client.0.smithi156.stdout:6/889: stat d3/d13/d17/dae 0 2024-07-16T09:55:51.837 INFO:tasks.workunit.client.1.smithi176.stdout:3/795: mkdir d3/da/dca/de8 0 2024-07-16T09:55:51.842 INFO:tasks.workunit.client.1.smithi176.stdout:3/796: creat d3/d17/d1d/d6e/fe9 x:0 0 0 2024-07-16T09:55:51.843 INFO:tasks.workunit.client.0.smithi156.stdout:8/946: link d0/d11/d20/d7f/db7/d110/d1f/fc6 d0/d11/d20/d7f/db7/d110/d1f/d57/dc0/dff/f13b 0 2024-07-16T09:55:51.843 INFO:tasks.workunit.client.1.smithi176.stdout:1/655: dwrite d4/d3a/d54/d5a/d7a/fa6 [0,4194304] 0 2024-07-16T09:55:51.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:55:51 smithi176 ceph-mon[36626]: Checking for cert/key for grafana.smithi156 2024-07-16T09:55:51.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:55:51 smithi176 ceph-mon[36626]: Migrating grafana.smithi156 cert to cert store 2024-07-16T09:55:51.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:55:51 smithi176 ceph-mon[36626]: Migrating grafana.smithi156 key to cert store 2024-07-16T09:55:51.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:55:51 smithi176 ceph-mon[36626]: mgrmap e22: smithi176.dqbrvx(active, since 1.01712s) 2024-07-16T09:55:51.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:55:51 smithi176 ceph-mon[36626]: pgmap v3: 65 pgs: 65 active+clean; 3.8 GiB data, 13 GiB used, 524 GiB / 536 GiB avail 2024-07-16T09:55:51.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:55:51 smithi176 ceph-mon[36626]: from='mgr.24377 ' entity='mgr.smithi176.dqbrvx' 2024-07-16T09:55:51.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:55:51 smithi176 ceph-mon[36626]: from='mgr.24377 ' entity='mgr.smithi176.dqbrvx' 2024-07-16T09:55:51.845 INFO:tasks.workunit.client.0.smithi156.stdout:6/890: unlink d3/d13/d17/d4f/d94/d19/d31/d35/l84 0 2024-07-16T09:55:51.846 INFO:tasks.workunit.client.0.smithi156.stdout:8/947: truncate d0/d11/d20/d7f/f9e 1656609 0 2024-07-16T09:55:51.848 INFO:tasks.workunit.client.0.smithi156.stdout:6/891: chown d3/d13/d17/d4f/d94/d19/d31/d35/lcf 30285247 1 2024-07-16T09:55:51.849 INFO:tasks.workunit.client.1.smithi176.stdout:3/797: fsync d3/d17/d1d/d6e/fa4 0 2024-07-16T09:55:51.850 INFO:tasks.workunit.client.1.smithi176.stdout:1/656: chown d4/d8/d12/d21/d25/lba 320571 1 2024-07-16T09:55:51.850 INFO:tasks.workunit.client.1.smithi176.stdout:5/955: dwrite d4/d14/fb4 [0,4194304] 0 2024-07-16T09:55:51.852 INFO:tasks.workunit.client.0.smithi156.stdout:5/961: dread d1b/d1f/d45/d52/d43/d85/d97/f25 [0,4194304] 0 2024-07-16T09:55:51.853 INFO:tasks.workunit.client.1.smithi176.stdout:3/798: chown d3/da/d7b/fb8 0 1 2024-07-16T09:55:51.853 INFO:tasks.workunit.client.1.smithi176.stdout:1/657: readlink d4/d77/l92 0 2024-07-16T09:55:51.854 INFO:tasks.workunit.client.0.smithi156.stdout:8/948: dwrite d0/d11/d13/d113/f11e [0,4194304] 0 2024-07-16T09:55:51.859 INFO:tasks.workunit.client.1.smithi176.stdout:3/799: chown d3/d17/d1d/d4f/d59/da0/l48 52027 1 2024-07-16T09:55:51.860 INFO:tasks.workunit.client.1.smithi176.stdout:1/658: creat d4/d8/d12/d21/d25/da8/fd4 x:0 0 0 2024-07-16T09:55:51.862 INFO:tasks.workunit.client.1.smithi176.stdout:3/800: rmdir d3/da/d7b 39 2024-07-16T09:55:51.862 INFO:tasks.workunit.client.1.smithi176.stdout:5/956: dwrite d4/d37/d72/d38/d102/dff/dc7/f139 [0,4194304] 0 2024-07-16T09:55:51.864 INFO:tasks.workunit.client.1.smithi176.stdout:5/957: symlink d4/da9/dc4/l156 0 2024-07-16T09:55:51.869 INFO:tasks.workunit.client.1.smithi176.stdout:1/659: unlink d4/d8/d12/d21/d25/da8/fb2 0 2024-07-16T09:55:51.870 INFO:tasks.workunit.client.1.smithi176.stdout:5/958: readlink d4/d9a/lc9 0 2024-07-16T09:55:51.872 INFO:tasks.workunit.client.0.smithi156.stdout:5/962: dread d1b/d1f/d42/de0/f120 [0,4194304] 0 2024-07-16T09:55:51.874 INFO:tasks.workunit.client.1.smithi176.stdout:1/660: creat d4/d77/dad/d72/fd5 x:0 0 0 2024-07-16T09:55:51.875 INFO:tasks.workunit.client.1.smithi176.stdout:5/959: mkdir d4/d9a/d10a/d10e/d157 0 2024-07-16T09:55:51.877 INFO:tasks.workunit.client.1.smithi176.stdout:1/661: mknod d4/d8/d12/d21/d34/cd6 0 2024-07-16T09:55:51.877 INFO:tasks.workunit.client.0.smithi156.stdout:5/963: rename d1b/d99/dae/c109 to d1b/d1f/d54/dc6/c130 0 2024-07-16T09:55:51.879 INFO:tasks.workunit.client.1.smithi176.stdout:5/960: truncate d4/d37/d72/d38/d102/dff/fc3 1251128 0 2024-07-16T09:55:51.880 INFO:tasks.workunit.client.1.smithi176.stdout:5/961: chown d4/f6 781 1 2024-07-16T09:55:51.881 INFO:tasks.workunit.client.1.smithi176.stdout:1/662: mkdir d4/d3a/d54/d5a/d7a/dd7 0 2024-07-16T09:55:51.882 INFO:tasks.workunit.client.1.smithi176.stdout:5/962: chown d4/da9/dc4/deb/d122/c142 14720 1 2024-07-16T09:55:51.883 INFO:tasks.workunit.client.1.smithi176.stdout:1/663: creat d4/d3a/dc8/fd8 x:0 0 0 2024-07-16T09:55:51.885 INFO:tasks.workunit.client.1.smithi176.stdout:3/801: dread d3/da/d7b/fb8 [0,4194304] 0 2024-07-16T09:55:51.886 INFO:tasks.workunit.client.1.smithi176.stdout:1/664: truncate d4/d77/dad/d72/fd5 505458 0 2024-07-16T09:55:51.886 INFO:tasks.workunit.client.1.smithi176.stdout:3/802: chown d3/d17/d1d/d6e/c79 2 1 2024-07-16T09:55:51.888 INFO:tasks.workunit.client.1.smithi176.stdout:1/665: symlink d4/db/ld9 0 2024-07-16T09:55:51.892 INFO:tasks.workunit.client.1.smithi176.stdout:3/803: dread d3/d17/d1d/d4f/d59/f3a [0,4194304] 0 2024-07-16T09:55:51.899 INFO:tasks.workunit.client.0.smithi156.stdout:6/892: dwrite d3/d13/d17/d4f/d5c/f80 [0,4194304] 0 2024-07-16T09:55:51.899 INFO:tasks.workunit.client.1.smithi176.stdout:3/804: rename d3/l4 to d3/da/d84/lea 0 2024-07-16T09:55:51.899 INFO:tasks.workunit.client.1.smithi176.stdout:5/963: write d4/d14/fcd [424518,23688] 0 2024-07-16T09:55:51.900 INFO:tasks.workunit.client.0.smithi156.stdout:5/964: dwrite d1b/d1f/d45/d52/d43/f10c [0,4194304] 0 2024-07-16T09:55:51.904 INFO:tasks.workunit.client.1.smithi176.stdout:3/805: mknod d3/d17/ceb 0 2024-07-16T09:55:51.905 INFO:tasks.workunit.client.0.smithi156.stdout:5/965: rename d1b/d1f/d54/lbd to d1b/d99/l131 0 2024-07-16T09:55:51.905 INFO:tasks.workunit.client.1.smithi176.stdout:1/666: dwrite d4/d8/f69 [0,4194304] 0 2024-07-16T09:55:51.909 INFO:tasks.workunit.client.0.smithi156.stdout:5/966: symlink d1b/d1f/d42/de0/de5/l132 0 2024-07-16T09:55:51.910 INFO:tasks.workunit.client.1.smithi176.stdout:4/886: sync 2024-07-16T09:55:51.911 INFO:tasks.workunit.client.1.smithi176.stdout:1/667: mknod d4/d3a/d54/cda 0 2024-07-16T09:55:51.914 INFO:tasks.workunit.client.1.smithi176.stdout:1/668: write d4/d8/d12/d21/d25/da8/f97 [4891042,105262] 0 2024-07-16T09:55:51.916 INFO:tasks.workunit.client.1.smithi176.stdout:4/887: dread da/d41/f99 [0,4194304] 0 2024-07-16T09:55:51.916 INFO:tasks.workunit.client.0.smithi156.stdout:5/967: rename d1b/d1f/d42/d6d/cf4 to d1b/d1f/d45/d52/d2d/d96/c133 0 2024-07-16T09:55:51.920 INFO:tasks.workunit.client.1.smithi176.stdout:4/888: mknod da/c120 0 2024-07-16T09:55:51.921 INFO:tasks.workunit.client.1.smithi176.stdout:1/669: creat d4/d3a/fdb x:0 0 0 2024-07-16T09:55:51.923 INFO:tasks.workunit.client.0.smithi156.stdout:6/893: dread d3/d13/d17/d4f/d94/d19/d31/fa2 [0,4194304] 0 2024-07-16T09:55:51.925 INFO:tasks.workunit.client.1.smithi176.stdout:1/670: rename d4/d8/d12/d21/l42 to d4/d3a/d54/d5a/ldc 0 2024-07-16T09:55:51.926 INFO:tasks.workunit.client.0.smithi156.stdout:6/894: creat d3/d13/d17/d4f/d30/d45/f121 x:0 0 0 2024-07-16T09:55:51.928 INFO:tasks.workunit.client.1.smithi176.stdout:4/889: read da/dc/d4b/df0/d96/d9b/db4/f26 [3233645,74389] 0 2024-07-16T09:55:51.931 INFO:tasks.workunit.client.1.smithi176.stdout:1/671: rename d4/d8/d12/d8f to d4/d8/d12/d21/ddd 0 2024-07-16T09:55:51.932 INFO:tasks.workunit.client.1.smithi176.stdout:1/672: write d4/d3a/d54/fb4 [795347,95919] 0 2024-07-16T09:55:51.932 INFO:tasks.workunit.client.0.smithi156.stdout:6/895: dwrite d3/d13/d17/d4f/d30/d45/f121 [0,4194304] 0 2024-07-16T09:55:51.933 INFO:tasks.workunit.client.0.smithi156.stdout:5/968: truncate d1b/d1f/d45/d52/d43/d85/d97/d37/fb7 2571229 0 2024-07-16T09:55:51.934 INFO:tasks.workunit.client.1.smithi176.stdout:4/890: getdents da/dc/d4b/df0/d96/d69/d86 0 2024-07-16T09:55:51.937 INFO:tasks.workunit.client.1.smithi176.stdout:1/673: rmdir d4/d77/dad 39 2024-07-16T09:55:51.937 INFO:tasks.workunit.client.1.smithi176.stdout:3/806: dread d3/d17/f40 [0,4194304] 0 2024-07-16T09:55:51.938 INFO:tasks.workunit.client.0.smithi156.stdout:6/896: creat d3/d13/d17/d4f/f122 x:0 0 0 2024-07-16T09:55:51.940 INFO:tasks.workunit.client.0.smithi156.stdout:5/969: creat d1b/d1f/d45/d52/d49/f134 x:0 0 0 2024-07-16T09:55:51.940 INFO:tasks.workunit.client.1.smithi176.stdout:1/674: symlink d4/d8/d12/d21/d34/lde 0 2024-07-16T09:55:51.941 INFO:tasks.workunit.client.0.smithi156.stdout:6/897: dread d3/d13/d17/d4f/fe9 [0,4194304] 0 2024-07-16T09:55:51.944 INFO:tasks.workunit.client.1.smithi176.stdout:4/891: rename da/dc/d4b/df0/d28/f7f to da/d3d/f121 0 2024-07-16T09:55:51.945 INFO:tasks.workunit.client.1.smithi176.stdout:1/675: symlink d4/d3a/d8e/ldf 0 2024-07-16T09:55:51.946 INFO:tasks.workunit.client.0.smithi156.stdout:6/898: truncate d3/d13/d17/d4f/d4d/d3b/d5a/fd9 947413 0 2024-07-16T09:55:51.946 INFO:tasks.workunit.client.1.smithi176.stdout:4/892: dread - da/dc/d4b/df0/d96/d9b/db4/d40/d5d/f116 zero size 2024-07-16T09:55:51.951 INFO:tasks.workunit.client.0.smithi156.stdout:5/970: dwrite d1b/d1f/d42/f47 [4194304,4194304] 0 2024-07-16T09:55:51.951 INFO:tasks.workunit.client.1.smithi176.stdout:4/893: stat da/dc/d4b/fa7 0 2024-07-16T09:55:51.951 INFO:tasks.workunit.client.1.smithi176.stdout:1/676: symlink d4/d3a/d54/d5a/d7a/dd7/le0 0 2024-07-16T09:55:51.953 INFO:tasks.workunit.client.0.smithi156.stdout:6/899: getdents d3/d13/d17/d4f/d94/de4 0 2024-07-16T09:55:51.958 INFO:tasks.workunit.client.0.smithi156.stdout:5/971: write d1b/d1f/d45/d52/f3c [3134202,8852] 0 2024-07-16T09:55:51.960 INFO:tasks.workunit.client.0.smithi156.stdout:6/900: write d3/f25 [8760773,17359] 0 2024-07-16T09:55:51.960 INFO:tasks.workunit.client.0.smithi156.stdout:5/972: stat d1b/d1f/d45/d52/d43/d7c/l95 0 2024-07-16T09:55:51.961 INFO:tasks.workunit.client.0.smithi156.stdout:6/901: readlink d3/d13/d17/d4f/d94/daa/lc8 0 2024-07-16T09:55:51.962 INFO:tasks.workunit.client.0.smithi156.stdout:5/973: chown d1b/d1f/d45/d52/d49/f81 981 1 2024-07-16T09:55:51.963 INFO:tasks.workunit.client.0.smithi156.stdout:6/902: dread - d3/d13/d17/d4f/d30/d45/fb0 zero size 2024-07-16T09:55:51.963 INFO:tasks.workunit.client.0.smithi156.stdout:5/974: mkdir d1b/d1f/d42/de0/de5/d135 0 2024-07-16T09:55:51.965 INFO:tasks.workunit.client.0.smithi156.stdout:5/975: chown d1b/d1f/d45/d52/d43/d85/d97/d37/l3f 2653 1 2024-07-16T09:55:51.967 INFO:tasks.workunit.client.0.smithi156.stdout:5/976: write d1b/d1f/d45/d52/d43/d85/d97/d37/d88/dbc/dc2/d125/ff6 [4304022,94769] 0 2024-07-16T09:55:51.972 INFO:tasks.workunit.client.0.smithi156.stdout:6/903: truncate d3/d13/d17/d4f/d94/d19/d31/f9c 1215503 0 2024-07-16T09:55:51.973 INFO:tasks.workunit.client.0.smithi156.stdout:6/904: write d3/d13/d17/d4f/d4d/d3b/d77/fd2 [204024,110486] 0 2024-07-16T09:55:51.977 INFO:tasks.workunit.client.0.smithi156.stdout:5/977: dwrite d1b/d1f/d42/de0/f120 [4194304,4194304] 0 2024-07-16T09:55:51.980 INFO:tasks.workunit.client.0.smithi156.stdout:6/905: rename d3/d13/d17/d56/ccd to d3/d13/d17/d4f/d4d/d3b/d5a/dc1/df9/c123 0 2024-07-16T09:55:51.982 INFO:tasks.workunit.client.0.smithi156.stdout:5/978: rename d1b/d1f/d45/d52/d43/d85/d97/d37/d88/dc7/f107 to d1b/d1f/d45/d52/d2d/d96/f136 0 2024-07-16T09:55:51.984 INFO:tasks.workunit.client.0.smithi156.stdout:5/979: chown d1b/d1f/d54/dbe/dde 1 1 2024-07-16T09:55:51.985 INFO:tasks.workunit.client.0.smithi156.stdout:6/906: unlink d3/d13/d17/d4f/d30/d45/cad 0 2024-07-16T09:55:51.990 INFO:tasks.workunit.client.0.smithi156.stdout:6/907: rename d3/d13/d17/d4f/d94/daa/d119/f105 to d3/d13/d17/d4f/d94/d19/d31/f124 0 2024-07-16T09:55:51.992 INFO:tasks.workunit.client.0.smithi156.stdout:5/980: dwrite d1b/d1f/d45/d52/d43/d85/d97/d37/d88/dbc/dc2/d125/ff6 [4194304,4194304] 0 2024-07-16T09:55:51.993 INFO:tasks.workunit.client.0.smithi156.stdout:6/908: write d3/d13/d17/d4f/d94/d19/d31/d10a/f107 [154042,66357] 0 2024-07-16T09:55:51.999 INFO:tasks.workunit.client.0.smithi156.stdout:5/981: truncate d1b/d1f/d45/d52/d43/d85/d97/f48 596989 0 2024-07-16T09:55:52.000 INFO:tasks.workunit.client.0.smithi156.stdout:5/982: readlink d1b/d1f/d45/d52/d43/d85/d97/l32 0 2024-07-16T09:55:52.004 INFO:tasks.workunit.client.0.smithi156.stdout:5/983: rename d1b/d1f/d45/d52/d43/d85/d97/d37/cbb to d1b/d1f/d45/d52/d43/d7c/c137 0 2024-07-16T09:55:52.005 INFO:tasks.workunit.client.0.smithi156.stdout:7/901: sync 2024-07-16T09:55:52.005 INFO:tasks.workunit.client.0.smithi156.stdout:5/984: fdatasync d1b/d1f/d45/d52/d43/d85/d97/d37/d88/dc7/ffe 0 2024-07-16T09:55:52.009 INFO:tasks.workunit.client.0.smithi156.stdout:7/902: unlink dc/d13/d20/d4d/l77 0 2024-07-16T09:55:52.012 INFO:tasks.workunit.client.0.smithi156.stdout:7/903: chown dc/d3e/d76/d134/dde/l102 5951 1 2024-07-16T09:55:52.012 INFO:tasks.workunit.client.0.smithi156.stdout:7/904: dread - dc/d13/fa5 zero size 2024-07-16T09:55:52.012 INFO:tasks.workunit.client.0.smithi156.stdout:7/905: chown dc/d3e/d3f/d51/d8a/l93 174 1 2024-07-16T09:55:52.013 INFO:tasks.workunit.client.0.smithi156.stdout:5/985: creat d1b/d1f/d54/dc6/f138 x:0 0 0 2024-07-16T09:55:52.016 INFO:tasks.workunit.client.1.smithi176.stdout:1/677: chown d4/d77/dad/cd3 4585 1 2024-07-16T09:55:52.018 INFO:tasks.workunit.client.1.smithi176.stdout:4/894: rename da/dc/d4b/c56 to da/dc/d4b/df0/d96/d69/d86/c122 0 2024-07-16T09:55:52.020 INFO:tasks.workunit.client.1.smithi176.stdout:1/678: getdents d4/d3a/d54 0 2024-07-16T09:55:52.023 INFO:tasks.workunit.client.1.smithi176.stdout:1/679: stat d4/d8/d12/d21/c7c 0 2024-07-16T09:55:52.024 INFO:tasks.workunit.client.0.smithi156.stdout:5/986: mknod d1b/c139 0 2024-07-16T09:55:52.024 INFO:tasks.workunit.client.1.smithi176.stdout:1/680: stat d4/d8/d12/d21/d34/daa 0 2024-07-16T09:55:52.025 INFO:tasks.workunit.client.1.smithi176.stdout:1/681: fsync d4/d8/d12/d21/fa2 0 2024-07-16T09:55:52.028 INFO:tasks.workunit.client.1.smithi176.stdout:1/682: mkdir d4/d3a/d54/d5a/d7a/dd7/de1 0 2024-07-16T09:55:52.029 INFO:tasks.workunit.client.0.smithi156.stdout:7/906: rmdir dc/d13/d124 0 2024-07-16T09:55:52.030 INFO:tasks.workunit.client.0.smithi156.stdout:7/907: fdatasync dc/d48/fbc 0 2024-07-16T09:55:52.032 INFO:tasks.workunit.client.1.smithi176.stdout:1/683: creat d4/d8/d12/d4c/da4/da9/fe2 x:0 0 0 2024-07-16T09:55:52.032 INFO:tasks.workunit.client.0.smithi156.stdout:5/987: dwrite d1b/d1f/d45/d52/d49/f81 [0,4194304] 0 2024-07-16T09:55:52.032 INFO:tasks.workunit.client.0.smithi156.stdout:7/908: fsync dc/d3e/d3f/d51/d8a/fda 0 2024-07-16T09:55:52.035 INFO:tasks.workunit.client.1.smithi176.stdout:4/895: dread da/dc/d4b/df0/d96/d9b/db4/d40/d5d/d73/f4f [0,4194304] 0 2024-07-16T09:55:52.035 INFO:tasks.workunit.client.1.smithi176.stdout:1/684: mknod d4/d8/d12/d21/ddd/ce3 0 2024-07-16T09:55:52.035 INFO:tasks.workunit.client.0.smithi156.stdout:5/988: read d1b/d1f/d45/d7f/d87/fa7 [3047056,74841] 0 2024-07-16T09:55:52.036 INFO:tasks.workunit.client.0.smithi156.stdout:5/989: write d1b/f55 [3692810,127167] 0 2024-07-16T09:55:52.038 INFO:tasks.workunit.client.0.smithi156.stdout:5/990: chown d1b/d1f/d45/d52/f113 278155034 1 2024-07-16T09:55:52.039 INFO:tasks.workunit.client.0.smithi156.stdout:5/991: readlink d1b/d1f/d45/d52/d43/d85/d97/l4f 0 2024-07-16T09:55:52.040 INFO:tasks.workunit.client.1.smithi176.stdout:1/685: fsync d4/d3a/d54/d5a/f67 0 2024-07-16T09:55:52.041 INFO:tasks.workunit.client.0.smithi156.stdout:5/992: write d1b/d1f/d45/d52/f113 [1189790,92829] 0 2024-07-16T09:55:52.043 INFO:tasks.workunit.client.1.smithi176.stdout:4/896: getdents da/d41/d5b/dba 0 2024-07-16T09:55:52.045 INFO:tasks.workunit.client.1.smithi176.stdout:1/686: creat d4/d77/dad/fe4 x:0 0 0 2024-07-16T09:55:52.046 INFO:tasks.workunit.client.1.smithi176.stdout:4/897: fdatasync da/dc/d4b/df0/d96/d9b/db4/f26 0 2024-07-16T09:55:52.046 INFO:tasks.workunit.client.1.smithi176.stdout:1/687: read - d4/d8/d12/d21/f89 zero size 2024-07-16T09:55:52.050 INFO:tasks.workunit.client.0.smithi156.stdout:5/993: creat d1b/d1f/d45/d52/d43/d85/f13a x:0 0 0 2024-07-16T09:55:52.050 INFO:tasks.workunit.client.0.smithi156.stdout:7/909: getdents dc/d48/dd7 0 2024-07-16T09:55:52.051 INFO:tasks.workunit.client.0.smithi156.stdout:7/910: readlink dc/d13/d24/l25 0 2024-07-16T09:55:52.052 INFO:tasks.workunit.client.1.smithi176.stdout:4/898: creat da/d3d/d17/d54/d78/f123 x:0 0 0 2024-07-16T09:55:52.053 INFO:tasks.workunit.client.0.smithi156.stdout:5/994: mkdir d1b/d1f/d45/d13b 0 2024-07-16T09:55:52.053 INFO:tasks.workunit.client.1.smithi176.stdout:4/899: write da/dc/d4b/df0/d96/d9b/fdb [1252398,36712] 0 2024-07-16T09:55:52.054 INFO:tasks.workunit.client.0.smithi156.stdout:7/911: mknod dc/d3e/d3f/d51/d8a/c137 0 2024-07-16T09:55:52.055 INFO:tasks.workunit.client.0.smithi156.stdout:5/995: stat d1b/d1f/d45/d52/d43/d85/d97/d37/d88/dbc/dc2/d125/f11b 0 2024-07-16T09:55:52.057 INFO:tasks.workunit.client.0.smithi156.stdout:5/996: write d1b/d1f/d45/d52/d43/f10c [509777,37057] 0 2024-07-16T09:55:52.057 INFO:tasks.workunit.client.1.smithi176.stdout:4/900: creat da/dc/d4b/df0/d96/d9b/db4/d4a/dc1/f124 x:0 0 0 2024-07-16T09:55:52.058 INFO:tasks.workunit.client.0.smithi156.stdout:5/997: chown d1b/d1f/d45/d52/d2d/cce 69457 1 2024-07-16T09:55:52.058 INFO:tasks.workunit.client.0.smithi156.stdout:7/912: creat dc/d13/d36/dd4/d114/d11a/f138 x:0 0 0 2024-07-16T09:55:52.059 INFO:tasks.workunit.client.1.smithi176.stdout:4/901: chown da/dc/d4b/df0/d28/cc5 82 1 2024-07-16T09:55:52.059 INFO:tasks.workunit.client.1.smithi176.stdout:1/688: dread d4/f31 [0,4194304] 0 2024-07-16T09:55:52.059 INFO:tasks.workunit.client.0.smithi156.stdout:5/998: fdatasync d1b/d1f/d54/dc6/f138 0 2024-07-16T09:55:52.060 INFO:tasks.workunit.client.1.smithi176.stdout:4/902: write da/f19 [1790794,30736] 0 2024-07-16T09:55:52.062 INFO:tasks.workunit.client.1.smithi176.stdout:1/689: creat d4/d8/d12/d21/d34/daa/fe5 x:0 0 0 2024-07-16T09:55:52.062 INFO:tasks.workunit.client.0.smithi156.stdout:7/913: mknod dc/d13/d15/d8d/c139 0 2024-07-16T09:55:52.063 INFO:tasks.workunit.client.1.smithi176.stdout:4/903: mkdir da/dc/d4b/d62/d125 0 2024-07-16T09:55:52.063 INFO:tasks.workunit.client.0.smithi156.stdout:5/999: symlink d1b/d1f/d45/d9d/l13c 0 2024-07-16T09:55:52.064 INFO:tasks.workunit.client.1.smithi176.stdout:1/690: mknod d4/d3a/d66/ce6 0 2024-07-16T09:55:52.064 INFO:tasks.workunit.client.1.smithi176.stdout:4/904: symlink da/de5/l126 0 2024-07-16T09:55:52.064 INFO:tasks.workunit.client.1.smithi176.stdout:1/691: chown d4/d8/fc2 257894 1 2024-07-16T09:55:52.065 INFO:tasks.workunit.client.1.smithi176.stdout:4/905: fsync da/dc/d4b/fa7 0 2024-07-16T09:55:52.068 INFO:tasks.workunit.client.1.smithi176.stdout:4/906: mkdir da/d3d/d17/d54/d127 0 2024-07-16T09:55:52.068 INFO:tasks.workunit.client.1.smithi176.stdout:4/907: write da/fb [5917206,50621] 0 2024-07-16T09:55:52.069 INFO:tasks.workunit.client.0.smithi156.stdout:7/914: dwrite dc/d3e/d76/d134/fdb [0,4194304] 0 2024-07-16T09:55:52.070 INFO:tasks.workunit.client.1.smithi176.stdout:7/864: sync 2024-07-16T09:55:52.070 INFO:tasks.workunit.client.1.smithi176.stdout:1/692: rename d4/d8/d12/d21/c7c to d4/d8/d2c/ce7 0 2024-07-16T09:55:52.071 INFO:tasks.workunit.client.1.smithi176.stdout:4/908: creat da/dc/d4b/df0/d28/f128 x:0 0 0 2024-07-16T09:55:52.072 INFO:tasks.workunit.client.1.smithi176.stdout:4/909: chown da/dc/d4b/df0/d28/c32 4260222 1 2024-07-16T09:55:52.073 INFO:tasks.workunit.client.1.smithi176.stdout:7/865: rename d4/dd/d49/d7a/db1 to d4/d5/d97/d110 0 2024-07-16T09:55:52.074 INFO:tasks.workunit.client.1.smithi176.stdout:1/693: mkdir d4/de8 0 2024-07-16T09:55:52.074 INFO:tasks.workunit.client.1.smithi176.stdout:7/866: readlink d4/dd/l78 0 2024-07-16T09:55:52.074 INFO:tasks.workunit.client.1.smithi176.stdout:4/910: chown da/dc/d4b/df0/d96/faf 59558452 1 2024-07-16T09:55:52.075 INFO:tasks.workunit.client.1.smithi176.stdout:7/867: mkdir d4/d111 0 2024-07-16T09:55:52.079 INFO:tasks.workunit.client.1.smithi176.stdout:1/694: fsync d4/d8/d12/d21/d25/d30/f61 0 2024-07-16T09:55:52.084 INFO:tasks.workunit.client.1.smithi176.stdout:1/695: getdents d4/d3a/d66 0 2024-07-16T09:55:52.084 INFO:tasks.workunit.client.1.smithi176.stdout:7/868: dwrite d4/dd/d26/dc2/d17/d1d/d8b/f104 [0,4194304] 0 2024-07-16T09:55:52.086 INFO:tasks.workunit.client.1.smithi176.stdout:7/869: readlink d4/d47/d5d/d9a/de0/l10b 0 2024-07-16T09:55:52.090 INFO:tasks.workunit.client.1.smithi176.stdout:7/870: truncate d4/d1f/f59 978893 0 2024-07-16T09:55:52.093 INFO:tasks.workunit.client.1.smithi176.stdout:1/696: dwrite d4/d8/d12/d21/d25/d30/d44/d4d/f7e [4194304,4194304] 0 2024-07-16T09:55:52.093 INFO:tasks.workunit.client.1.smithi176.stdout:7/871: fsync d4/d5/f6c 0 2024-07-16T09:55:52.094 INFO:tasks.workunit.client.1.smithi176.stdout:3/807: sync 2024-07-16T09:55:52.094 INFO:tasks.workunit.client.1.smithi176.stdout:5/964: sync 2024-07-16T09:55:52.097 INFO:tasks.workunit.client.1.smithi176.stdout:1/697: mknod d4/d3a/d54/d5a/d7a/ce9 0 2024-07-16T09:55:52.098 INFO:tasks.workunit.client.1.smithi176.stdout:5/965: read d4/d6c/f7c [2194320,49027] 0 2024-07-16T09:55:52.099 INFO:tasks.workunit.client.1.smithi176.stdout:3/808: mkdir d3/d17/d1d/dec 0 2024-07-16T09:55:52.101 INFO:tasks.workunit.client.1.smithi176.stdout:3/809: truncate d3/d17/d1d/d6e/fe1 330204 0 2024-07-16T09:55:52.102 INFO:tasks.workunit.client.1.smithi176.stdout:7/872: rename d4/dd/f23 to d4/dd/f112 0 2024-07-16T09:55:52.104 INFO:tasks.workunit.client.1.smithi176.stdout:4/911: dread da/fcc [0,4194304] 0 2024-07-16T09:55:52.107 INFO:tasks.workunit.client.1.smithi176.stdout:7/873: creat d4/dd/d81/f113 x:0 0 0 2024-07-16T09:55:52.108 INFO:tasks.workunit.client.1.smithi176.stdout:1/698: creat d4/d8/d12/fea x:0 0 0 2024-07-16T09:55:52.109 INFO:tasks.workunit.client.1.smithi176.stdout:3/810: mknod d3/da/dca/de8/ced 0 2024-07-16T09:55:52.110 INFO:tasks.workunit.client.1.smithi176.stdout:3/811: chown d3/d17/d1d/d4f/daa/fe0 2 1 2024-07-16T09:55:52.113 INFO:tasks.workunit.client.1.smithi176.stdout:7/874: mknod d4/d47/d5d/d9a/de0/c114 0 2024-07-16T09:55:52.116 INFO:tasks.workunit.client.1.smithi176.stdout:1/699: dread d4/d8/d12/d21/d25/d30/d44/d4d/f95 [0,4194304] 0 2024-07-16T09:55:52.116 INFO:tasks.workunit.client.1.smithi176.stdout:4/912: write da/dc/d4b/df0/d96/fd8 [370992,105744] 0 2024-07-16T09:55:52.117 INFO:tasks.workunit.client.1.smithi176.stdout:3/812: creat d3/d85/ddd/fee x:0 0 0 2024-07-16T09:55:52.118 INFO:tasks.workunit.client.1.smithi176.stdout:5/966: dwrite d4/d37/d72/d82/f83 [0,4194304] 0 2024-07-16T09:55:52.122 INFO:tasks.workunit.client.1.smithi176.stdout:7/875: dwrite d4/d47/d5d/fad [0,4194304] 0 2024-07-16T09:55:52.124 INFO:tasks.workunit.client.1.smithi176.stdout:1/700: dread d4/f31 [0,4194304] 0 2024-07-16T09:55:52.127 INFO:tasks.workunit.client.1.smithi176.stdout:5/967: link d4/d14/dce/f107 d4/d6c/f158 0 2024-07-16T09:55:52.128 INFO:tasks.workunit.client.1.smithi176.stdout:4/913: creat da/f129 x:0 0 0 2024-07-16T09:55:52.129 INFO:tasks.workunit.client.1.smithi176.stdout:3/813: creat d3/d17/d1d/d4f/d59/fef x:0 0 0 2024-07-16T09:55:52.129 INFO:tasks.workunit.client.1.smithi176.stdout:7/876: write d4/dd/f112 [1597922,40977] 0 2024-07-16T09:55:52.136 INFO:tasks.workunit.client.1.smithi176.stdout:4/914: write da/d41/f99 [1167078,68827] 0 2024-07-16T09:55:52.137 INFO:tasks.workunit.client.1.smithi176.stdout:5/968: rename d4/d37/d72/d38/f99 to d4/d14/dc5/def/d11f/f159 0 2024-07-16T09:55:52.137 INFO:tasks.workunit.client.1.smithi176.stdout:3/814: rmdir d3/da/d33/dbd 39 2024-07-16T09:55:52.138 INFO:tasks.workunit.client.1.smithi176.stdout:4/915: dread - da/dc/d4b/d62/f11f zero size 2024-07-16T09:55:52.139 INFO:tasks.workunit.client.1.smithi176.stdout:5/969: fdatasync d4/d10/f33 0 2024-07-16T09:55:52.141 INFO:tasks.workunit.client.1.smithi176.stdout:7/877: dwrite d4/dd/d49/d7a/f92 [0,4194304] 0 2024-07-16T09:55:52.142 INFO:tasks.workunit.client.1.smithi176.stdout:4/916: creat da/d41/d5b/df4/f12a x:0 0 0 2024-07-16T09:55:52.144 INFO:tasks.workunit.client.1.smithi176.stdout:5/970: rename d4/d37/d72/f53 to d4/d14/d19/f15a 0 2024-07-16T09:55:52.146 INFO:tasks.workunit.client.1.smithi176.stdout:7/878: stat d4/d47/d5d/l6a 0 2024-07-16T09:55:52.147 INFO:tasks.workunit.client.1.smithi176.stdout:4/917: stat da/dc/d4b/df0/d96/d69/lc8 0 2024-07-16T09:55:52.150 INFO:tasks.workunit.client.1.smithi176.stdout:7/879: chown d4/dd/l71 22477 1 2024-07-16T09:55:52.154 INFO:tasks.workunit.client.1.smithi176.stdout:4/918: write da/dc/d4b/df0/d96/faf [1798110,122163] 0 2024-07-16T09:55:52.154 INFO:tasks.workunit.client.1.smithi176.stdout:3/815: dwrite d3/d17/d1d/d4f/d59/da0/d7a/dac/fbb [0,4194304] 0 2024-07-16T09:55:52.155 INFO:tasks.workunit.client.1.smithi176.stdout:5/971: dread d4/d37/d72/d38/d39/f129 [0,4194304] 0 2024-07-16T09:55:52.155 INFO:tasks.workunit.client.1.smithi176.stdout:7/880: unlink d4/dd/d26/dc2/cbb 0 2024-07-16T09:55:52.157 INFO:tasks.workunit.client.1.smithi176.stdout:4/919: symlink da/dc/d4b/d62/d125/l12b 0 2024-07-16T09:55:52.158 INFO:tasks.workunit.client.1.smithi176.stdout:3/816: fsync d3/da/d6c/dbc/fcb 0 2024-07-16T09:55:52.159 INFO:tasks.workunit.client.1.smithi176.stdout:7/881: mkdir d4/dd/d49/d7a/d115 0 2024-07-16T09:55:52.159 INFO:tasks.workunit.client.1.smithi176.stdout:4/920: stat da/dc/d4b/df0/d28/f128 0 2024-07-16T09:55:52.161 INFO:tasks.workunit.client.1.smithi176.stdout:1/701: dread d4/d8/f8c [0,4194304] 0 2024-07-16T09:55:52.164 INFO:tasks.workunit.client.1.smithi176.stdout:5/972: dwrite d4/fd [0,4194304] 0 2024-07-16T09:55:52.170 INFO:tasks.workunit.client.1.smithi176.stdout:7/882: dread d4/d47/d5d/d9a/ffd [0,4194304] 0 2024-07-16T09:55:52.171 INFO:tasks.workunit.client.1.smithi176.stdout:4/921: dread da/dc/d4b/df0/d96/d69/fd1 [0,4194304] 0 2024-07-16T09:55:52.173 INFO:tasks.workunit.client.1.smithi176.stdout:1/702: dwrite d4/d8/d12/d21/d25/da8/f9d [0,4194304] 0 2024-07-16T09:55:52.176 INFO:tasks.workunit.client.1.smithi176.stdout:3/817: truncate d3/d17/fd7 517455 0 2024-07-16T09:55:52.183 INFO:tasks.workunit.client.1.smithi176.stdout:1/703: symlink d4/d3a/dc8/leb 0 2024-07-16T09:55:52.183 INFO:tasks.workunit.client.1.smithi176.stdout:4/922: mknod da/d3d/d17/d54/d127/c12c 0 2024-07-16T09:55:52.183 INFO:tasks.workunit.client.1.smithi176.stdout:7/883: unlink d4/d1f/d22/d51/cba 0 2024-07-16T09:55:52.183 INFO:tasks.workunit.client.1.smithi176.stdout:3/818: rename d3/d17/d1d/d4f/d6d/ca7 to d3/da/d6c/dd9/cf0 0 2024-07-16T09:55:52.186 INFO:tasks.workunit.client.1.smithi176.stdout:7/884: creat d4/d47/d6d/f116 x:0 0 0 2024-07-16T09:55:52.188 INFO:tasks.workunit.client.1.smithi176.stdout:1/704: mkdir d4/d8/d12/d21/d34/daa/dbb/dec 0 2024-07-16T09:55:52.191 INFO:tasks.workunit.client.0.smithi156.stdout:8/949: sync 2024-07-16T09:55:52.194 INFO:tasks.workunit.client.1.smithi176.stdout:4/923: dwrite da/dc/d4b/df0/d96/d69/f7e [0,4194304] 0 2024-07-16T09:55:52.195 INFO:tasks.workunit.client.0.smithi156.stdout:8/950: write d0/d11/d2e/fd1 [482441,50118] 0 2024-07-16T09:55:52.197 INFO:tasks.workunit.client.1.smithi176.stdout:5/973: dread d4/d14/d3d/f41 [0,4194304] 0 2024-07-16T09:55:52.199 INFO:tasks.workunit.client.1.smithi176.stdout:5/974: chown d4/d10/d17/f50 19962284 1 2024-07-16T09:55:52.200 INFO:tasks.workunit.client.1.smithi176.stdout:7/885: dwrite d4/dd/d26/dc2/d17/d1d/f5b [0,4194304] 0 2024-07-16T09:55:52.204 INFO:tasks.workunit.client.0.smithi156.stdout:8/951: link d0/d11/d29/d80/d97/de7/lfd d0/d11/d13/d1c/l13c 0 2024-07-16T09:55:52.205 INFO:tasks.workunit.client.0.smithi156.stdout:8/952: dread - d0/d11/d13/d1c/d39/f130 zero size 2024-07-16T09:55:52.205 INFO:tasks.workunit.client.1.smithi176.stdout:1/705: write d4/f31 [735167,98338] 0 2024-07-16T09:55:52.207 INFO:tasks.workunit.client.0.smithi156.stdout:8/953: write d0/d11/d20/d7f/db7/d110/d1f/d57/dc0/dff/f128 [963807,122493] 0 2024-07-16T09:55:52.208 INFO:tasks.workunit.client.1.smithi176.stdout:1/706: read - d4/d8/d12/fea zero size 2024-07-16T09:55:52.209 INFO:tasks.workunit.client.1.smithi176.stdout:4/924: dwrite da/dc/d4b/df0/d96/fd8 [0,4194304] 0 2024-07-16T09:55:52.209 INFO:tasks.workunit.client.1.smithi176.stdout:3/819: dread d3/da/d33/f53 [0,4194304] 0 2024-07-16T09:55:52.211 INFO:tasks.workunit.client.1.smithi176.stdout:5/975: read d4/f68 [381541,123465] 0 2024-07-16T09:55:52.213 INFO:tasks.workunit.client.1.smithi176.stdout:1/707: chown d4/d8/d12/d21/d34/c9a 103 1 2024-07-16T09:55:52.214 INFO:tasks.workunit.client.1.smithi176.stdout:1/708: write d4/f31 [701650,61234] 0 2024-07-16T09:55:52.215 INFO:tasks.workunit.client.1.smithi176.stdout:3/820: dread - d3/da/d6c/d86/f92 zero size 2024-07-16T09:55:52.215 INFO:tasks.workunit.client.1.smithi176.stdout:4/925: chown da/dc/d4b/df0/d96/d69/d86/c89 98584555 1 2024-07-16T09:55:52.217 INFO:tasks.workunit.client.1.smithi176.stdout:4/926: write da/fca [1633283,82604] 0 2024-07-16T09:55:52.219 INFO:tasks.workunit.client.0.smithi156.stdout:7/915: sync 2024-07-16T09:55:52.219 INFO:tasks.workunit.client.0.smithi156.stdout:6/909: sync 2024-07-16T09:55:52.221 INFO:tasks.workunit.client.0.smithi156.stdout:7/916: truncate dc/d63/d98/fd3 833068 0 2024-07-16T09:55:52.222 INFO:tasks.workunit.client.0.smithi156.stdout:8/954: write d0/d11/d20/d7f/db7/d110/f6a [2760375,84078] 0 2024-07-16T09:55:52.223 INFO:tasks.workunit.client.1.smithi176.stdout:3/821: rename d3/d17/d1d/d6e/fa4 to d3/d17/d1d/d4f/d59/da0/d7a/ff1 0 2024-07-16T09:55:52.223 INFO:tasks.workunit.client.0.smithi156.stdout:7/917: dread - dc/d3e/d3f/d51/d8a/fdd zero size 2024-07-16T09:55:52.224 INFO:tasks.workunit.client.1.smithi176.stdout:7/886: write d4/d1f/f59 [84968,117383] 0 2024-07-16T09:55:52.225 INFO:tasks.workunit.client.0.smithi156.stdout:7/918: write dc/d13/d20/f99 [2376552,99161] 0 2024-07-16T09:55:52.228 INFO:tasks.workunit.client.1.smithi176.stdout:3/822: symlink d3/da/d33/dbd/lf2 0 2024-07-16T09:55:52.229 INFO:tasks.workunit.client.0.smithi156.stdout:8/955: read d0/d11/d20/d7f/db7/d110/d1f/d60/f98 [3043929,113488] 0 2024-07-16T09:55:52.230 INFO:tasks.workunit.client.1.smithi176.stdout:3/823: read - d3/d17/d1d/d6e/fe9 zero size 2024-07-16T09:55:52.230 INFO:tasks.workunit.client.0.smithi156.stdout:6/910: truncate d3/d13/d17/d4f/d4d/d3b/d5a/fb9 1016406 0 2024-07-16T09:55:52.231 INFO:tasks.workunit.client.1.smithi176.stdout:7/887: getdents d4/d1f/d22/d51 0 2024-07-16T09:55:52.232 INFO:tasks.workunit.client.1.smithi176.stdout:1/709: dwrite d4/d8/d12/d21/d25/d30/d44/f78 [0,4194304] 0 2024-07-16T09:55:52.234 INFO:tasks.workunit.client.0.smithi156.stdout:7/919: dwrite dc/d48/d119/fa2 [0,4194304] 0 2024-07-16T09:55:52.234 INFO:tasks.workunit.client.1.smithi176.stdout:1/710: chown d4/d8/fc6 824 1 2024-07-16T09:55:52.234 INFO:tasks.workunit.client.1.smithi176.stdout:7/888: write d4/d47/d6d/d89/d66/f8a [8522278,29757] 0 2024-07-16T09:55:52.235 INFO:tasks.workunit.client.0.smithi156.stdout:8/956: dread d0/d11/d29/d80/d97/f12e [0,4194304] 0 2024-07-16T09:55:52.236 INFO:tasks.workunit.client.0.smithi156.stdout:6/911: truncate d3/d13/d17/d4f/d4d/d3b/da5/f116 638519 0 2024-07-16T09:55:52.236 INFO:tasks.workunit.client.1.smithi176.stdout:3/824: truncate d3/d17/d1d/d4f/d59/da0/d7a/fc6 695310 0 2024-07-16T09:55:52.240 INFO:tasks.workunit.client.1.smithi176.stdout:3/825: rmdir d3/da/d33 39 2024-07-16T09:55:52.240 INFO:tasks.workunit.client.0.smithi156.stdout:7/920: creat dc/d48/dd7/f13a x:0 0 0 2024-07-16T09:55:52.241 INFO:tasks.workunit.client.0.smithi156.stdout:6/912: mkdir d3/d13/d17/d4f/d94/d19/d31/d35/d118/d125 0 2024-07-16T09:55:52.244 INFO:tasks.workunit.client.1.smithi176.stdout:1/711: creat d4/d3a/d54/fed x:0 0 0 2024-07-16T09:55:52.244 INFO:tasks.workunit.client.0.smithi156.stdout:7/921: mkdir dc/d3e/d76/d134/d13b 0 2024-07-16T09:55:52.244 INFO:tasks.workunit.client.0.smithi156.stdout:8/957: dread d0/d11/d4a/f4f [0,4194304] 0 2024-07-16T09:55:52.245 INFO:tasks.workunit.client.0.smithi156.stdout:7/922: stat dc/d13/d36/dd4/d114 0 2024-07-16T09:55:52.246 INFO:tasks.workunit.client.0.smithi156.stdout:6/913: creat d3/d13/d17/d4f/d30/d45/d99/d114/f126 x:0 0 0 2024-07-16T09:55:52.248 INFO:tasks.workunit.client.1.smithi176.stdout:7/889: truncate d4/dd/f1a 1335602 0 2024-07-16T09:55:52.248 INFO:tasks.workunit.client.1.smithi176.stdout:3/826: mkdir d3/d17/d1d/d4f/daa/df3 0 2024-07-16T09:55:52.248 INFO:tasks.workunit.client.0.smithi156.stdout:8/958: unlink d0/d11/d29/d80/f88 0 2024-07-16T09:55:52.250 INFO:tasks.workunit.client.0.smithi156.stdout:7/923: creat dc/d48/dd7/f13c x:0 0 0 2024-07-16T09:55:52.251 INFO:tasks.workunit.client.1.smithi176.stdout:7/890: chown d4/dd/d26/dc2/d17/d1d/ddc/ff4 29 1 2024-07-16T09:55:52.251 INFO:tasks.workunit.client.0.smithi156.stdout:8/959: mkdir d0/d11/d20/d7f/d13d 0 2024-07-16T09:55:52.252 INFO:tasks.workunit.client.1.smithi176.stdout:7/891: fdatasync d4/d47/d5d/fad 0 2024-07-16T09:55:52.254 INFO:tasks.workunit.client.1.smithi176.stdout:7/892: write d4/d47/d5d/f90 [3088922,100070] 0 2024-07-16T09:55:52.254 INFO:tasks.workunit.client.0.smithi156.stdout:8/960: mknod d0/d11/d2e/d117/c13e 0 2024-07-16T09:55:52.255 INFO:tasks.workunit.client.0.smithi156.stdout:6/914: getdents d3/d13/d17/d4f/d4d/d3b/d5a/dc1 0 2024-07-16T09:55:52.256 INFO:tasks.workunit.client.0.smithi156.stdout:8/961: creat d0/d74/d115/f13f x:0 0 0 2024-07-16T09:55:52.257 INFO:tasks.workunit.client.0.smithi156.stdout:6/915: dread - d3/d13/d17/d4f/d4d/d3b/d5a/f117 zero size 2024-07-16T09:55:52.258 INFO:tasks.workunit.client.0.smithi156.stdout:8/962: rmdir d0/d11/d2e/d35 39 2024-07-16T09:55:52.262 INFO:tasks.workunit.client.1.smithi176.stdout:7/893: link d4/d47/d6d/d89/l69 d4/d47/d6d/d89/d7e/l117 0 2024-07-16T09:55:52.264 INFO:tasks.workunit.client.1.smithi176.stdout:7/894: dread - d4/d1f/d22/ffc zero size 2024-07-16T09:55:52.264 INFO:tasks.workunit.client.1.smithi176.stdout:1/712: dwrite d4/f57 [0,4194304] 0 2024-07-16T09:55:52.265 INFO:tasks.workunit.client.1.smithi176.stdout:7/895: chown d4/dd/d81/f113 27011 1 2024-07-16T09:55:52.270 INFO:tasks.workunit.client.0.smithi156.stdout:6/916: write d3/d13/d17/d4f/d4d/f3f [2158164,116028] 0 2024-07-16T09:55:52.275 INFO:tasks.workunit.client.0.smithi156.stdout:6/917: creat d3/d13/d17/d4f/d4d/dbb/f127 x:0 0 0 2024-07-16T09:55:52.277 INFO:tasks.workunit.client.0.smithi156.stdout:8/963: dwrite d0/d11/d20/d7f/db7/d110/d1f/f47 [0,4194304] 0 2024-07-16T09:55:52.279 INFO:tasks.workunit.client.1.smithi176.stdout:3/827: dread d3/da/d56/f5b [0,4194304] 0 2024-07-16T09:55:52.279 INFO:tasks.workunit.client.1.smithi176.stdout:3/828: readlink d3/d17/l1a 0 2024-07-16T09:55:52.279 INFO:tasks.workunit.client.0.smithi156.stdout:8/964: chown d0/d11/d20/d7f/db7/d110/f10 113031 1 2024-07-16T09:55:52.279 INFO:tasks.workunit.client.1.smithi176.stdout:3/829: dread - d3/d17/fd8 zero size 2024-07-16T09:55:52.282 INFO:tasks.workunit.client.0.smithi156.stdout:6/918: unlink d3/d13/d17/d4f/d94/d19/d31/d35/l63 0 2024-07-16T09:55:52.283 INFO:tasks.workunit.client.0.smithi156.stdout:8/965: creat d0/d11/d20/d7f/db7/d110/d1f/dee/f140 x:0 0 0 2024-07-16T09:55:52.284 INFO:tasks.workunit.client.0.smithi156.stdout:6/919: chown d3/d13/d17/d4f/d94/fe 945 1 2024-07-16T09:55:52.284 INFO:tasks.workunit.client.1.smithi176.stdout:1/713: rename d4/d8/d12/d4c/l80 to d4/d8/d12/d21/d25/lee 0 2024-07-16T09:55:52.285 INFO:tasks.workunit.client.1.smithi176.stdout:3/830: creat d3/d17/d1d/d4f/daa/ff4 x:0 0 0 2024-07-16T09:55:52.287 INFO:tasks.workunit.client.1.smithi176.stdout:3/831: chown d3/d17/d1d/d4f/daa/df3 150251 1 2024-07-16T09:55:52.287 INFO:tasks.workunit.client.1.smithi176.stdout:3/832: readlink d3/d17/l3e 0 2024-07-16T09:55:52.289 INFO:tasks.workunit.client.0.smithi156.stdout:8/966: mkdir d0/d11/d20/d3f/d141 0 2024-07-16T09:55:52.289 INFO:tasks.workunit.client.1.smithi176.stdout:1/714: mkdir d4/d8/d12/d21/d34/daa/dbb/def 0 2024-07-16T09:55:52.290 INFO:tasks.workunit.client.0.smithi156.stdout:6/920: mknod d3/d13/d17/d4f/d94/d19/d31/d35/d118/c128 0 2024-07-16T09:55:52.290 INFO:tasks.workunit.client.1.smithi176.stdout:1/715: fsync d4/d8/d12/d4c/da4/fbc 0 2024-07-16T09:55:52.291 INFO:tasks.workunit.client.1.smithi176.stdout:1/716: chown d4/c3b 183229 1 2024-07-16T09:55:52.293 INFO:tasks.workunit.client.0.smithi156.stdout:6/921: creat d3/d13/d17/d4f/d94/dec/f129 x:0 0 0 2024-07-16T09:55:52.293 INFO:tasks.workunit.client.1.smithi176.stdout:3/833: getdents d3/d17/d1d/d4f/d59/da0 0 2024-07-16T09:55:52.294 INFO:tasks.workunit.client.0.smithi156.stdout:8/967: dread d0/d11/d13/d1c/d24/fb9 [0,4194304] 0 2024-07-16T09:55:52.295 INFO:tasks.workunit.client.0.smithi156.stdout:8/968: fdatasync d0/d11/d62/dc2/fc9 0 2024-07-16T09:55:52.296 INFO:tasks.workunit.client.1.smithi176.stdout:3/834: symlink d3/da/d56/lf5 0 2024-07-16T09:55:52.296 INFO:tasks.workunit.client.1.smithi176.stdout:7/896: dread d4/d5/fe5 [0,4194304] 0 2024-07-16T09:55:52.297 INFO:tasks.workunit.client.0.smithi156.stdout:8/969: rmdir d0/d11/d13/d1c 39 2024-07-16T09:55:52.298 INFO:tasks.workunit.client.1.smithi176.stdout:7/897: creat d4/d47/d5d/d9a/de0/f118 x:0 0 0 2024-07-16T09:55:52.300 INFO:tasks.workunit.client.0.smithi156.stdout:6/922: dwrite d3/d13/d17/d4f/d4d/d3b/d77/fc3 [0,4194304] 0 2024-07-16T09:55:52.301 INFO:tasks.workunit.client.1.smithi176.stdout:7/898: symlink d4/dd/d81/l119 0 2024-07-16T09:55:52.304 INFO:tasks.workunit.client.0.smithi156.stdout:8/970: rename d0/d11/d20/d7f/db7/d110/d1f/d57/c102 to d0/d11/d20/d7f/c142 0 2024-07-16T09:55:52.305 INFO:tasks.workunit.client.0.smithi156.stdout:6/923: symlink d3/d13/d17/d4f/d4d/d3b/d5a/dc1/df9/l12a 0 2024-07-16T09:55:52.305 INFO:tasks.workunit.client.0.smithi156.stdout:8/971: write d0/f1 [4164686,36941] 0 2024-07-16T09:55:52.306 INFO:tasks.workunit.client.1.smithi176.stdout:7/899: creat d4/d5/d97/d110/deb/f11a x:0 0 0 2024-07-16T09:55:52.307 INFO:tasks.workunit.client.0.smithi156.stdout:8/972: chown d0/d11/d20/d7f/db7/d110/d1f/d57/db3/df7 215561181 1 2024-07-16T09:55:52.310 INFO:tasks.workunit.client.0.smithi156.stdout:6/924: unlink d3/d13/d17/d4f/d94/d19/l2f 0 2024-07-16T09:55:52.311 INFO:tasks.workunit.client.0.smithi156.stdout:6/925: chown d3/d13/d17/d4f/d30/d45/f121 8364729 1 2024-07-16T09:55:52.313 INFO:tasks.workunit.client.1.smithi176.stdout:7/900: read d4/dd/d26/fc1 [871239,93664] 0 2024-07-16T09:55:52.315 INFO:tasks.workunit.client.1.smithi176.stdout:1/717: dread d4/d8/d12/d21/d25/f86 [0,4194304] 0 2024-07-16T09:55:52.317 INFO:tasks.workunit.client.1.smithi176.stdout:3/835: dread d3/d17/f90 [0,4194304] 0 2024-07-16T09:55:52.318 INFO:tasks.workunit.client.1.smithi176.stdout:3/836: fdatasync d3/da/fe 0 2024-07-16T09:55:52.319 INFO:tasks.workunit.client.0.smithi156.stdout:8/973: fsync d0/d11/d13/d1c/d24/f42 0 2024-07-16T09:55:52.321 INFO:tasks.workunit.client.0.smithi156.stdout:8/974: write d0/d11/d20/d7f/db7/d110/d1f/d57/fbf [51240,25588] 0 2024-07-16T09:55:52.321 INFO:tasks.workunit.client.1.smithi176.stdout:7/901: dwrite d4/dd/f37 [4194304,4194304] 0 2024-07-16T09:55:52.324 INFO:tasks.workunit.client.1.smithi176.stdout:3/837: mknod d3/da/d7b/cf6 0 2024-07-16T09:55:52.324 INFO:tasks.workunit.client.0.smithi156.stdout:8/975: truncate d0/d11/d20/d7f/db7/d110/d1f/d50/d8d/f119 1404322 0 2024-07-16T09:55:52.324 INFO:tasks.workunit.client.1.smithi176.stdout:3/838: fdatasync d3/da/fe 0 2024-07-16T09:55:52.328 INFO:tasks.workunit.client.1.smithi176.stdout:3/839: write d3/d85/ddd/fee [240280,56266] 0 2024-07-16T09:55:52.333 INFO:tasks.workunit.client.1.smithi176.stdout:7/902: rename d4/d47/d6d/d89/d7e/l117 to d4/d47/d5d/d9a/l11b 0 2024-07-16T09:55:52.333 INFO:tasks.workunit.client.1.smithi176.stdout:1/718: truncate d4/fb7 315659 0 2024-07-16T09:55:52.338 INFO:tasks.workunit.client.1.smithi176.stdout:3/840: dwrite d3/d17/d1d/d6e/fab [0,4194304] 0 2024-07-16T09:55:52.339 INFO:tasks.workunit.client.1.smithi176.stdout:1/719: mknod d4/d3a/dc8/cf0 0 2024-07-16T09:55:52.343 INFO:tasks.workunit.client.1.smithi176.stdout:3/841: chown d3/d17/f88 988 1 2024-07-16T09:55:52.344 INFO:tasks.workunit.client.1.smithi176.stdout:3/842: readlink d3/l28 0 2024-07-16T09:55:52.345 INFO:tasks.workunit.client.1.smithi176.stdout:7/903: read d4/d47/d5d/d9a/de0/f103 [2047397,79652] 0 2024-07-16T09:55:52.345 INFO:tasks.workunit.client.1.smithi176.stdout:1/720: dread d4/f24 [0,4194304] 0 2024-07-16T09:55:52.352 INFO:tasks.workunit.client.1.smithi176.stdout:3/843: link d3/da/d33/f44 d3/da/d66/ff7 0 2024-07-16T09:55:52.353 INFO:tasks.workunit.client.0.smithi156.stdout:6/926: creat d3/d13/d17/d4f/d94/f12b x:0 0 0 2024-07-16T09:55:52.358 INFO:tasks.workunit.client.1.smithi176.stdout:3/844: write d3/f4a [3759370,66768] 0 2024-07-16T09:55:52.358 INFO:tasks.workunit.client.1.smithi176.stdout:7/904: rename d4/dd/d26/l3d to d4/d5/d97/d110/l11c 0 2024-07-16T09:55:52.358 INFO:tasks.workunit.client.1.smithi176.stdout:1/721: dwrite d4/d8/d12/d21/d34/f3d [0,4194304] 0 2024-07-16T09:55:52.359 INFO:tasks.workunit.client.1.smithi176.stdout:3/845: write d3/d17/d1d/f63 [1530754,39686] 0 2024-07-16T09:55:52.359 INFO:tasks.workunit.client.0.smithi156.stdout:8/976: rename d0/d11/d20/d7f/db7/d110/d1f/d50 to d0/d11/d29/d80/d97/de7/d143 0 2024-07-16T09:55:52.359 INFO:tasks.workunit.client.0.smithi156.stdout:8/977: dread - d0/d74/d115/f12d zero size 2024-07-16T09:55:52.359 INFO:tasks.workunit.client.0.smithi156.stdout:8/978: read d0/d11/d62/fbc [40142,118651] 0 2024-07-16T09:55:52.360 INFO:tasks.workunit.client.1.smithi176.stdout:3/846: write d3/da/d6c/fc1 [1981872,23694] 0 2024-07-16T09:55:52.361 INFO:tasks.workunit.client.1.smithi176.stdout:7/905: mknod d4/d47/d5d/c11d 0 2024-07-16T09:55:52.361 INFO:tasks.workunit.client.0.smithi156.stdout:6/927: link d3/d13/d17/d4f/fe9 d3/d13/d17/d4f/d94/d19/d31/d35/d118/f12c 0 2024-07-16T09:55:52.363 INFO:tasks.workunit.client.1.smithi176.stdout:1/722: mknod d4/d8/d12/d21/d25/da8/cf1 0 2024-07-16T09:55:52.366 INFO:tasks.workunit.client.1.smithi176.stdout:3/847: dread d3/da/d56/f89 [0,4194304] 0 2024-07-16T09:55:52.368 INFO:tasks.workunit.client.1.smithi176.stdout:3/848: fdatasync d3/d17/d1d/d4f/daa/fae 0 2024-07-16T09:55:52.370 INFO:tasks.workunit.client.1.smithi176.stdout:1/723: mknod d4/d8/d12/d21/ddd/cf2 0 2024-07-16T09:55:52.370 INFO:tasks.workunit.client.1.smithi176.stdout:1/724: readlink d4/d77/lc9 0 2024-07-16T09:55:52.375 INFO:tasks.workunit.client.1.smithi176.stdout:3/849: mkdir d3/da/d7b/df8 0 2024-07-16T09:55:52.377 INFO:tasks.workunit.client.1.smithi176.stdout:3/850: fdatasync d3/d17/d1d/d6e/fd5 0 2024-07-16T09:55:52.378 INFO:tasks.workunit.client.1.smithi176.stdout:1/725: dwrite d4/d77/dad/fbd [0,4194304] 0 2024-07-16T09:55:52.380 INFO:tasks.workunit.client.1.smithi176.stdout:3/851: mknod d3/da/dca/de8/cf9 0 2024-07-16T09:55:52.381 INFO:tasks.workunit.client.1.smithi176.stdout:1/726: mkdir d4/db/df3 0 2024-07-16T09:55:52.384 INFO:tasks.workunit.client.1.smithi176.stdout:1/727: fdatasync d4/d3a/d54/d5a/f5b 0 2024-07-16T09:55:52.386 INFO:tasks.workunit.client.1.smithi176.stdout:7/906: write d4/dd/d49/dd5/f108 [1265901,106908] 0 2024-07-16T09:55:52.387 INFO:tasks.workunit.client.1.smithi176.stdout:1/728: creat d4/d8/d12/d21/d34/daa/dbb/ff4 x:0 0 0 2024-07-16T09:55:52.391 INFO:tasks.workunit.client.1.smithi176.stdout:1/729: creat d4/d8/d12/d21/d25/d30/d44/d4d/ff5 x:0 0 0 2024-07-16T09:55:52.392 INFO:tasks.workunit.client.0.smithi156.stdout:6/928: dwrite d3/d13/d17/d4f/d1e/f95 [0,4194304] 0 2024-07-16T09:55:52.393 INFO:tasks.workunit.client.0.smithi156.stdout:6/929: chown d3/d13/d17/d4f/d94/d19/d31/d120 6082718 1 2024-07-16T09:55:52.393 INFO:tasks.workunit.client.1.smithi176.stdout:7/907: link d4/l9c d4/d5/d97/l11e 0 2024-07-16T09:55:52.395 INFO:tasks.workunit.client.0.smithi156.stdout:6/930: mkdir d3/d13/d17/d4f/d30/d45/d5f/da8/d12d 0 2024-07-16T09:55:52.397 INFO:tasks.workunit.client.0.smithi156.stdout:6/931: creat d3/d13/d17/d4f/d4d/d3b/d5a/dbe/f12e x:0 0 0 2024-07-16T09:55:52.427 INFO:tasks.workunit.client.0.smithi156.stdout:6/932: truncate d3/d13/d17/d4f/d4d/d3b/f93 1099416 0 2024-07-16T09:55:52.429 INFO:tasks.workunit.client.0.smithi156.stdout:6/933: rmdir d3/d13/d17/d4f/d4d/d3b/d5a/dc1/d10e 0 2024-07-16T09:55:52.431 INFO:tasks.workunit.client.0.smithi156.stdout:6/934: write d3/d13/d17/d4f/d1e/f95 [2271127,113458] 0 2024-07-16T09:55:52.432 INFO:tasks.workunit.client.0.smithi156.stdout:6/935: fdatasync d3/d13/d17/d4f/d94/dec/f100 0 2024-07-16T09:55:52.436 INFO:tasks.workunit.client.0.smithi156.stdout:6/936: rename d3/d13/d17/d4f/d94/d19/d31/d35/fd8 to d3/d13/d17/d4f/d94/f12f 0 2024-07-16T09:55:52.438 INFO:tasks.workunit.client.0.smithi156.stdout:6/937: fdatasync d3/d13/d17/d4f/d4d/d3b/f5d 0 2024-07-16T09:55:52.440 INFO:tasks.workunit.client.0.smithi156.stdout:6/938: mknod d3/d13/d17/d4f/d30/d45/d99/d114/c130 0 2024-07-16T09:55:52.441 INFO:tasks.workunit.client.0.smithi156.stdout:6/939: write d3/d13/d17/d56/ff8 [769449,6375] 0 2024-07-16T09:55:52.447 INFO:tasks.workunit.client.1.smithi176.stdout:7/908: symlink d4/d47/d5d/d9a/de0/l11f 0 2024-07-16T09:55:52.447 INFO:tasks.workunit.client.1.smithi176.stdout:1/730: creat d4/d3a/ff6 x:0 0 0 2024-07-16T09:55:52.448 INFO:tasks.workunit.client.0.smithi156.stdout:6/940: dwrite d3/d13/d17/d4f/d94/dec/f129 [0,4194304] 0 2024-07-16T09:55:52.449 INFO:tasks.workunit.client.1.smithi176.stdout:1/731: write d4/d8/d12/d21/d25/faf [4489454,39363] 0 2024-07-16T09:55:52.451 INFO:tasks.workunit.client.1.smithi176.stdout:1/732: write d4/d3a/f9e [4227614,42878] 0 2024-07-16T09:55:52.453 INFO:tasks.workunit.client.0.smithi156.stdout:6/941: write d3/d13/d17/d4f/d94/f12f [1817160,110953] 0 2024-07-16T09:55:52.453 INFO:tasks.workunit.client.1.smithi176.stdout:7/909: link d4/dd/d26/dc2/d17/d1d/d8b/f9d d4/d1f/dce/f120 0 2024-07-16T09:55:52.456 INFO:tasks.workunit.client.1.smithi176.stdout:1/733: unlink d4/db/ld9 0 2024-07-16T09:55:52.456 INFO:tasks.workunit.client.1.smithi176.stdout:7/910: creat d4/d47/d5d/d9a/f121 x:0 0 0 2024-07-16T09:55:52.457 INFO:tasks.workunit.client.0.smithi156.stdout:6/942: symlink d3/d13/d17/d4f/d30/l131 0 2024-07-16T09:55:52.459 INFO:tasks.workunit.client.1.smithi176.stdout:1/734: symlink d4/d3a/d54/d5a/d7a/dd7/lf7 0 2024-07-16T09:55:52.459 INFO:tasks.workunit.client.0.smithi156.stdout:6/943: chown d3/d13/d17/d4f/d1e/c6b 274 1 2024-07-16T09:55:52.460 INFO:tasks.workunit.client.0.smithi156.stdout:6/944: stat d3/d13 0 2024-07-16T09:55:52.461 INFO:tasks.workunit.client.0.smithi156.stdout:6/945: write d3/d13/d17/d4f/d1e/f95 [3513042,106184] 0 2024-07-16T09:55:52.465 INFO:tasks.workunit.client.0.smithi156.stdout:6/946: mknod d3/d13/d17/d4f/d94/d19/d31/d35/c132 0 2024-07-16T09:55:52.466 INFO:tasks.workunit.client.1.smithi176.stdout:1/735: dwrite d4/d8/d12/d21/d25/d30/d44/f78 [0,4194304] 0 2024-07-16T09:55:52.467 INFO:tasks.workunit.client.0.smithi156.stdout:6/947: write d3/d13/d17/d4f/f122 [61492,119334] 0 2024-07-16T09:55:52.470 INFO:tasks.workunit.client.1.smithi176.stdout:7/911: dread d4/dd/f112 [0,4194304] 0 2024-07-16T09:55:52.472 INFO:tasks.workunit.client.0.smithi156.stdout:6/948: rename d3/d13/d17/d4f/d94/d19/d31/d120 to d3/d13/d17/d4f/d4d/d3b/d77/d11e/d133 0 2024-07-16T09:55:52.474 INFO:tasks.workunit.client.1.smithi176.stdout:1/736: dwrite d4/d8/d12/fea [0,4194304] 0 2024-07-16T09:55:52.478 INFO:tasks.workunit.client.0.smithi156.stdout:6/949: link d3/d13/d17/d4f/d4d/d3b/da5/fd5 d3/d13/d17/d4f/d5c/d101/f134 0 2024-07-16T09:55:52.480 INFO:tasks.workunit.client.0.smithi156.stdout:6/950: symlink d3/d13/d17/d4f/d1e/l135 0 2024-07-16T09:55:52.481 INFO:tasks.workunit.client.1.smithi176.stdout:1/737: write d4/d8/d12/d21/d25/f90 [221819,121091] 0 2024-07-16T09:55:52.481 INFO:tasks.workunit.client.0.smithi156.stdout:6/951: stat d3/d13/d17/d4f/d1e/l4c 0 2024-07-16T09:55:52.485 INFO:tasks.workunit.client.0.smithi156.stdout:6/952: symlink d3/d13/d17/d4f/d4d/l136 0 2024-07-16T09:55:52.489 INFO:tasks.workunit.client.0.smithi156.stdout:6/953: creat d3/d13/d17/d4f/d94/d19/d31/d35/d118/d125/f137 x:0 0 0 2024-07-16T09:55:52.494 INFO:tasks.workunit.client.0.smithi156.stdout:6/954: rename d3/d13/c48 to d3/d13/d17/d4f/d4d/d3b/d77/d11e/c138 0 2024-07-16T09:55:52.496 INFO:tasks.workunit.client.1.smithi176.stdout:7/912: dread d4/d1f/d22/f52 [0,4194304] 0 2024-07-16T09:55:52.496 INFO:tasks.workunit.client.1.smithi176.stdout:7/913: fsync d4/d47/f34 0 2024-07-16T09:55:52.498 INFO:tasks.workunit.client.1.smithi176.stdout:1/738: dread d4/db/f16 [0,4194304] 0 2024-07-16T09:55:52.498 INFO:tasks.workunit.client.0.smithi156.stdout:6/955: mknod d3/d13/d17/d4f/d30/d45/d99/d114/c139 0 2024-07-16T09:55:52.500 INFO:tasks.workunit.client.0.smithi156.stdout:6/956: creat d3/d13/d17/d4f/d4d/d3b/f13a x:0 0 0 2024-07-16T09:55:52.501 INFO:tasks.workunit.client.0.smithi156.stdout:6/957: fdatasync d3/d13/d17/d4f/d1e/f3d 0 2024-07-16T09:55:52.501 INFO:tasks.workunit.client.1.smithi176.stdout:7/914: write d4/d1f/d22/d51/f67 [61140,2997] 0 2024-07-16T09:55:52.502 INFO:tasks.workunit.client.1.smithi176.stdout:1/739: write d4/d8/d12/d4c/fa7 [3188873,94965] 0 2024-07-16T09:55:52.504 INFO:tasks.workunit.client.1.smithi176.stdout:1/740: chown d4/d8/d2c/ce7 148085 1 2024-07-16T09:55:52.506 INFO:tasks.workunit.client.1.smithi176.stdout:1/741: mknod d4/d3a/d8e/cf8 0 2024-07-16T09:55:52.508 INFO:tasks.workunit.client.1.smithi176.stdout:1/742: symlink d4/d8/lf9 0 2024-07-16T09:55:52.509 INFO:tasks.workunit.client.1.smithi176.stdout:7/915: dread d4/dd/d49/dd5/f108 [0,4194304] 0 2024-07-16T09:55:52.510 INFO:tasks.workunit.client.1.smithi176.stdout:1/743: mknod d4/d8/d12/d21/d34/daa/dbb/dec/cfa 0 2024-07-16T09:55:52.511 INFO:tasks.workunit.client.1.smithi176.stdout:1/744: creat d4/d77/dad/d72/ffb x:0 0 0 2024-07-16T09:55:52.513 INFO:tasks.workunit.client.1.smithi176.stdout:7/916: write d4/d1f/dce/f120 [70417,84386] 0 2024-07-16T09:55:52.514 INFO:tasks.workunit.client.0.smithi156.stdout:7/924: sync 2024-07-16T09:55:52.514 INFO:tasks.workunit.client.0.smithi156.stdout:7/925: chown dc/d3e/d76/le2 1277783 1 2024-07-16T09:55:52.515 INFO:tasks.workunit.client.0.smithi156.stdout:7/926: chown dc/d13/d15/dfc/db1 1 1 2024-07-16T09:55:52.515 INFO:tasks.workunit.client.1.smithi176.stdout:1/745: rename d4/d8/d12/d21/ddd/cf2 to d4/d8/d12/d21/d34/daa/dbb/def/cfc 0 2024-07-16T09:55:52.518 INFO:tasks.workunit.client.0.smithi156.stdout:7/927: dread - dc/d13/d20/d4d/fb9 zero size 2024-07-16T09:55:52.518 INFO:tasks.workunit.client.1.smithi176.stdout:7/917: rename d4/d1f/d22/ccf to d4/d47/d6d/c122 0 2024-07-16T09:55:52.519 INFO:tasks.workunit.client.0.smithi156.stdout:7/928: dread - dc/d63/d98/f121 zero size 2024-07-16T09:55:52.521 INFO:tasks.workunit.client.1.smithi176.stdout:7/918: creat d4/d5/d10f/f123 x:0 0 0 2024-07-16T09:55:52.521 INFO:tasks.workunit.client.0.smithi156.stdout:7/929: mknod dc/d48/d119/d12f/d131/c13d 0 2024-07-16T09:55:52.522 INFO:tasks.workunit.client.0.smithi156.stdout:7/930: chown ca 5869722 1 2024-07-16T09:55:52.525 INFO:tasks.workunit.client.0.smithi156.stdout:7/931: write dc/d3e/d3f/d51/d8a/fe4 [765104,67794] 0 2024-07-16T09:55:52.526 INFO:tasks.workunit.client.1.smithi176.stdout:7/919: symlink d4/d5/d97/d110/l124 0 2024-07-16T09:55:52.526 INFO:tasks.workunit.client.0.smithi156.stdout:7/932: stat dc/d48/db2/fe7 0 2024-07-16T09:55:52.529 INFO:tasks.workunit.client.0.smithi156.stdout:7/933: creat dc/d3e/d3f/d51/d8a/dae/dd8/f13e x:0 0 0 2024-07-16T09:55:52.529 INFO:tasks.workunit.client.1.smithi176.stdout:1/746: dwrite d4/d8/d12/d21/f89 [0,4194304] 0 2024-07-16T09:55:52.533 INFO:tasks.workunit.client.1.smithi176.stdout:1/747: write d4/d8/d12/d21/d25/da8/fd1 [282592,70653] 0 2024-07-16T09:55:52.534 INFO:tasks.workunit.client.1.smithi176.stdout:1/748: chown d4/d3a/l6b 489965445 1 2024-07-16T09:55:52.536 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:55:52 smithi156 ceph-mon[29302]: Deploying cephadm binary to smithi156 2024-07-16T09:55:52.536 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:55:52 smithi156 ceph-mon[29302]: Deploying cephadm binary to smithi176 2024-07-16T09:55:52.536 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:55:52 smithi156 ceph-mon[29302]: pgmap v4: 65 pgs: 65 active+clean; 3.8 GiB data, 13 GiB used, 524 GiB / 536 GiB avail 2024-07-16T09:55:52.537 INFO:tasks.workunit.client.1.smithi176.stdout:7/920: dwrite d4/dd/d26/dc2/d17/d1d/d8b/f9d [4194304,4194304] 0 2024-07-16T09:55:52.539 INFO:tasks.workunit.client.1.smithi176.stdout:1/749: creat d4/d8/d12/ffd x:0 0 0 2024-07-16T09:55:52.539 INFO:tasks.workunit.client.1.smithi176.stdout:5/976: sync 2024-07-16T09:55:52.540 INFO:tasks.workunit.client.1.smithi176.stdout:4/927: sync 2024-07-16T09:55:52.542 INFO:tasks.workunit.client.1.smithi176.stdout:1/750: rmdir d4/d8/d12/d21/d34/daa 39 2024-07-16T09:55:52.546 INFO:tasks.workunit.client.1.smithi176.stdout:4/928: rename da/d3d/d17/d54/c74 to da/dc/d4b/c12d 0 2024-07-16T09:55:52.547 INFO:tasks.workunit.client.1.smithi176.stdout:7/921: dwrite d4/f11 [8388608,4194304] 0 2024-07-16T09:55:52.547 INFO:tasks.workunit.client.1.smithi176.stdout:1/751: mknod d4/d8/d12/d21/d25/d30/d44/dd2/cfe 0 2024-07-16T09:55:52.548 INFO:tasks.workunit.client.1.smithi176.stdout:4/929: read - da/d3d/d17/d54/d78/f123 zero size 2024-07-16T09:55:52.549 INFO:tasks.workunit.client.1.smithi176.stdout:1/752: fsync d4/d8/d12/d21/d25/da8/f9d 0 2024-07-16T09:55:52.555 INFO:tasks.workunit.client.1.smithi176.stdout:5/977: dread d4/d9a/ded/df4/f13d [0,4194304] 0 2024-07-16T09:55:52.556 INFO:tasks.workunit.client.0.smithi156.stdout:7/934: dread dc/d13/f35 [0,4194304] 0 2024-07-16T09:55:52.556 INFO:tasks.workunit.client.0.smithi156.stdout:7/935: stat dc/d3e/d76 0 2024-07-16T09:55:52.558 INFO:tasks.workunit.client.0.smithi156.stdout:7/936: chown dc/d3e/d3f/d51/f53 4 1 2024-07-16T09:55:52.562 INFO:tasks.workunit.client.0.smithi156.stdout:7/937: symlink dc/d48/db2/l13f 0 2024-07-16T09:55:52.562 INFO:tasks.workunit.client.0.smithi156.stdout:7/938: chown dc/d13/d15/f91 24622318 1 2024-07-16T09:55:52.562 INFO:tasks.workunit.client.1.smithi176.stdout:7/922: write d4/dd/f1a [1610748,92894] 0 2024-07-16T09:55:52.563 INFO:tasks.workunit.client.0.smithi156.stdout:7/939: symlink dc/d13/d20/l140 0 2024-07-16T09:55:52.564 INFO:tasks.workunit.client.0.smithi156.stdout:7/940: chown dc/d13/d20/f99 363 1 2024-07-16T09:55:52.565 INFO:tasks.workunit.client.1.smithi176.stdout:7/923: write d4/d5/f6c [3729934,77168] 0 2024-07-16T09:55:52.566 INFO:tasks.workunit.client.0.smithi156.stdout:7/941: creat dc/d13/d15/d8d/f141 x:0 0 0 2024-07-16T09:55:52.566 INFO:tasks.workunit.client.1.smithi176.stdout:4/930: dwrite da/dc/d4b/df0/d96/d9b/db4/d4a/f4e [0,4194304] 0 2024-07-16T09:55:52.569 INFO:tasks.workunit.client.1.smithi176.stdout:7/924: dread - d4/d5/d10f/f123 zero size 2024-07-16T09:55:52.571 INFO:tasks.workunit.client.1.smithi176.stdout:1/753: dwrite d4/d8/d2c/f2f [0,4194304] 0 2024-07-16T09:55:52.572 INFO:tasks.workunit.client.1.smithi176.stdout:7/925: write d4/d1f/f106 [1022719,38466] 0 2024-07-16T09:55:52.573 INFO:tasks.workunit.client.1.smithi176.stdout:4/931: write da/d3d/d17/f35 [3222524,130333] 0 2024-07-16T09:55:52.575 INFO:tasks.workunit.client.1.smithi176.stdout:5/978: dwrite d4/d37/d72/d82/f90 [0,4194304] 0 2024-07-16T09:55:52.576 INFO:tasks.workunit.client.1.smithi176.stdout:4/932: readlink da/d3d/l10a 0 2024-07-16T09:55:52.586 INFO:tasks.workunit.client.1.smithi176.stdout:7/926: mkdir d4/d1f/d22/d125 0 2024-07-16T09:55:52.587 INFO:tasks.workunit.client.1.smithi176.stdout:4/933: chown da/fca 293680 1 2024-07-16T09:55:52.588 INFO:tasks.workunit.client.1.smithi176.stdout:1/754: truncate d4/d8/d12/d21/d25/d30/d44/d4d/f91 1111088 0 2024-07-16T09:55:52.589 INFO:tasks.workunit.client.0.smithi156.stdout:7/942: write dc/d13/d36/dd4/d114/f96 [97677,122420] 0 2024-07-16T09:55:52.589 INFO:tasks.workunit.client.1.smithi176.stdout:7/927: fdatasync d4/dd/d26/dc2/fe7 0 2024-07-16T09:55:52.589 INFO:tasks.workunit.client.1.smithi176.stdout:4/934: write da/d3d/d17/d54/d78/f114 [250059,10389] 0 2024-07-16T09:55:52.590 INFO:tasks.workunit.client.0.smithi156.stdout:7/943: fdatasync dc/d48/dd7/f110 0 2024-07-16T09:55:52.592 INFO:tasks.workunit.client.0.smithi156.stdout:7/944: fdatasync dc/d13/d20/d4d/fd1 0 2024-07-16T09:55:52.592 INFO:tasks.workunit.client.1.smithi176.stdout:1/755: read d4/d3a/d54/fb4 [230840,37426] 0 2024-07-16T09:55:52.594 INFO:tasks.workunit.client.1.smithi176.stdout:1/756: write d4/f57 [3358005,107792] 0 2024-07-16T09:55:52.596 INFO:tasks.workunit.client.0.smithi156.stdout:7/945: creat dc/d13/d20/d4d/f142 x:0 0 0 2024-07-16T09:55:52.596 INFO:tasks.workunit.client.0.smithi156.stdout:7/946: fdatasync dc/d63/f6b 0 2024-07-16T09:55:52.598 INFO:tasks.workunit.client.0.smithi156.stdout:7/947: mknod dc/d3e/d3f/d51/d112/c143 0 2024-07-16T09:55:52.600 INFO:tasks.workunit.client.0.smithi156.stdout:7/948: creat dc/d3e/d3f/d51/d8a/dae/f144 x:0 0 0 2024-07-16T09:55:52.601 INFO:tasks.workunit.client.0.smithi156.stdout:7/949: chown dc/d13/d20/d4d/fd1 39995 1 2024-07-16T09:55:52.603 INFO:tasks.workunit.client.0.smithi156.stdout:7/950: mkdir dc/d13/d20/d4d/daa/d145 0 2024-07-16T09:55:52.605 INFO:tasks.workunit.client.0.smithi156.stdout:7/951: creat dc/d3e/d3f/d45/d104/f146 x:0 0 0 2024-07-16T09:55:52.613 INFO:tasks.workunit.client.0.smithi156.stdout:7/952: dwrite dc/d3e/f7f [0,4194304] 0 2024-07-16T09:55:52.616 INFO:tasks.workunit.client.0.smithi156.stdout:7/953: getdents dc/d48/d119 0 2024-07-16T09:55:52.618 INFO:tasks.workunit.client.1.smithi176.stdout:5/979: unlink d4/d37/d72/d38/d102/dbd/fd5 0 2024-07-16T09:55:52.620 INFO:tasks.workunit.client.1.smithi176.stdout:3/852: sync 2024-07-16T09:55:52.621 INFO:tasks.workunit.client.0.smithi156.stdout:7/954: rename dc/d13/fed to dc/d13/d20/d4d/daa/f147 0 2024-07-16T09:55:52.621 INFO:tasks.workunit.client.1.smithi176.stdout:7/928: rename d4/d1f/d22/d51/f67 to d4/dd/d26/dc2/d17/d1d/ddc/f126 0 2024-07-16T09:55:52.623 INFO:tasks.workunit.client.0.smithi156.stdout:7/955: unlink dc/d13/f1d 0 2024-07-16T09:55:52.623 INFO:tasks.workunit.client.1.smithi176.stdout:4/935: creat da/dc/d4b/df0/d96/d9b/db4/d40/d5d/d73/f12e x:0 0 0 2024-07-16T09:55:52.624 INFO:tasks.workunit.client.1.smithi176.stdout:1/757: symlink d4/d8/d12/d21/d25/d30/d44/d4d/lff 0 2024-07-16T09:55:52.626 INFO:tasks.workunit.client.1.smithi176.stdout:1/758: chown d4/d3a/l6b 528159 1 2024-07-16T09:55:52.627 INFO:tasks.workunit.client.0.smithi156.stdout:7/956: creat dc/d3e/d76/d134/f148 x:0 0 0 2024-07-16T09:55:52.628 INFO:tasks.workunit.client.1.smithi176.stdout:3/853: getdents d3 0 2024-07-16T09:55:52.628 INFO:tasks.workunit.client.1.smithi176.stdout:4/936: creat da/dc/d4b/df0/d96/d9b/db4/d40/d5d/f12f x:0 0 0 2024-07-16T09:55:52.628 INFO:tasks.workunit.client.1.smithi176.stdout:7/929: getdents d4/dd/d26/dc2/d17/d1d/d8b/dbe 0 2024-07-16T09:55:52.629 INFO:tasks.workunit.client.0.smithi156.stdout:7/957: creat dc/d3e/d3f/d51/d8a/dae/f149 x:0 0 0 2024-07-16T09:55:52.629 INFO:tasks.workunit.client.1.smithi176.stdout:1/759: mkdir d4/db/d100 0 2024-07-16T09:55:52.630 INFO:tasks.workunit.client.1.smithi176.stdout:1/760: fsync d4/d8/d2c/f2f 0 2024-07-16T09:55:52.631 INFO:tasks.workunit.client.1.smithi176.stdout:4/937: chown da/d3d/dac/fb9 77 1 2024-07-16T09:55:52.632 INFO:tasks.workunit.client.1.smithi176.stdout:7/930: write d4/dd/d26/dc2/d17/f7f [1186694,48350] 0 2024-07-16T09:55:52.636 INFO:tasks.workunit.client.1.smithi176.stdout:4/938: creat da/dc/d4b/df0/d28/d106/f130 x:0 0 0 2024-07-16T09:55:52.637 INFO:tasks.workunit.client.1.smithi176.stdout:1/761: rename d4/db/c40 to d4/d8/d12/d21/ddd/c101 0 2024-07-16T09:55:52.638 INFO:tasks.workunit.client.1.smithi176.stdout:3/854: dwrite d3/da/d84/fc8 [0,4194304] 0 2024-07-16T09:55:52.640 INFO:tasks.workunit.client.1.smithi176.stdout:1/762: dread - d4/d8/d12/ffd zero size 2024-07-16T09:55:52.642 INFO:tasks.workunit.client.1.smithi176.stdout:3/855: stat d3/d17/d1d/d4f 0 2024-07-16T09:55:52.642 INFO:tasks.workunit.client.1.smithi176.stdout:5/980: dread d4/d10/f33 [0,4194304] 0 2024-07-16T09:55:52.642 INFO:tasks.workunit.client.1.smithi176.stdout:7/931: rename d4/dd/d26/dc2/d17/d1d/d8b/dbe to d4/d47/d6d/db7/d127 0 2024-07-16T09:55:52.644 INFO:tasks.workunit.client.1.smithi176.stdout:5/981: stat d4/d37/d72/d82/f64 0 2024-07-16T09:55:52.645 INFO:tasks.workunit.client.1.smithi176.stdout:4/939: read da/dc/d4b/df0/d96/d9b/db4/f26 [4073433,96606] 0 2024-07-16T09:55:52.648 INFO:tasks.workunit.client.0.smithi156.stdout:7/958: truncate dc/d3e/d3f/d51/f53 2025849 0 2024-07-16T09:55:52.651 INFO:tasks.workunit.client.0.smithi156.stdout:8/979: sync 2024-07-16T09:55:52.651 INFO:tasks.workunit.client.1.smithi176.stdout:4/940: creat da/dc/d4b/df0/d28/d106/f131 x:0 0 0 2024-07-16T09:55:52.654 INFO:tasks.workunit.client.1.smithi176.stdout:3/856: creat d3/d17/d1d/d4f/daa/df3/ffa x:0 0 0 2024-07-16T09:55:52.660 INFO:tasks.workunit.client.1.smithi176.stdout:5/982: link d4/d14/cda d4/d37/d72/d106/c15b 0 2024-07-16T09:55:52.661 INFO:tasks.workunit.client.1.smithi176.stdout:7/932: dread d4/d1f/f62 [0,4194304] 0 2024-07-16T09:55:52.662 INFO:tasks.workunit.client.1.smithi176.stdout:5/983: symlink d4/d9a/d10a/l15c 0 2024-07-16T09:55:52.664 INFO:tasks.workunit.client.0.smithi156.stdout:7/959: rename dc/d13/d15/dfc to dc/d48/db2/dcf/d14a 0 2024-07-16T09:55:52.664 INFO:tasks.workunit.client.1.smithi176.stdout:7/933: fsync d4/f2b 0 2024-07-16T09:55:52.665 INFO:tasks.workunit.client.1.smithi176.stdout:5/984: write d4/d10/f54 [4978770,39036] 0 2024-07-16T09:55:52.668 INFO:tasks.workunit.client.1.smithi176.stdout:5/985: truncate d4/d37/dbb/f134 1132775 0 2024-07-16T09:55:52.668 INFO:tasks.workunit.client.1.smithi176.stdout:7/934: mkdir d4/d1f/d22/d125/d128 0 2024-07-16T09:55:52.668 INFO:tasks.workunit.client.0.smithi156.stdout:7/960: rmdir dc/d13/d20/d4d/daa/d145 0 2024-07-16T09:55:52.670 INFO:tasks.workunit.client.1.smithi176.stdout:7/935: symlink d4/dd/d81/l129 0 2024-07-16T09:55:52.672 INFO:tasks.workunit.client.1.smithi176.stdout:7/936: read d4/dd/f2e [558802,122564] 0 2024-07-16T09:55:52.673 INFO:tasks.workunit.client.1.smithi176.stdout:5/986: link d4/d10/f12 d4/d37/d72/d38/d102/dbd/f15d 0 2024-07-16T09:55:52.676 INFO:tasks.workunit.client.0.smithi156.stdout:8/980: dread d0/f2 [4194304,4194304] 0 2024-07-16T09:55:52.678 INFO:tasks.workunit.client.1.smithi176.stdout:3/857: dread d3/d17/d1d/d4f/d59/da0/f9e [0,4194304] 0 2024-07-16T09:55:52.678 INFO:tasks.workunit.client.0.smithi156.stdout:8/981: readlink d0/d11/d13/d1c/d24/l2f 0 2024-07-16T09:55:52.679 INFO:tasks.workunit.client.1.smithi176.stdout:5/987: rmdir d4/d6c/d12e 0 2024-07-16T09:55:52.681 INFO:tasks.workunit.client.1.smithi176.stdout:5/988: symlink d4/da9/l15e 0 2024-07-16T09:55:52.683 INFO:tasks.workunit.client.0.smithi156.stdout:7/961: dwrite dc/d13/d20/f3b [0,4194304] 0 2024-07-16T09:55:52.683 INFO:tasks.workunit.client.0.smithi156.stdout:8/982: getdents d0/d11/d20/d7f/db7/d110/d1f 0 2024-07-16T09:55:52.685 INFO:tasks.workunit.client.1.smithi176.stdout:5/989: dread d4/d10/f5f [4194304,4194304] 0 2024-07-16T09:55:52.686 INFO:tasks.workunit.client.0.smithi156.stdout:8/983: dread d0/d11/d20/d3f/d96/fad [0,4194304] 0 2024-07-16T09:55:52.686 INFO:tasks.workunit.client.0.smithi156.stdout:7/962: write dc/d3e/d3f/d51/d8a/fe4 [709689,64484] 0 2024-07-16T09:55:52.687 INFO:tasks.workunit.client.0.smithi156.stdout:7/963: chown dc/d3e/d3f/d45/l68 1 1 2024-07-16T09:55:52.687 INFO:tasks.workunit.client.1.smithi176.stdout:5/990: mkdir d4/d37/d72/d38/d102/dbd/d15f 0 2024-07-16T09:55:52.688 INFO:tasks.workunit.client.1.smithi176.stdout:5/991: stat d4/d37/d72/d106 0 2024-07-16T09:55:52.688 INFO:tasks.workunit.client.0.smithi156.stdout:8/984: truncate d0/d11/d13/d113/f123 408739 0 2024-07-16T09:55:52.688 INFO:tasks.workunit.client.0.smithi156.stdout:7/964: write dc/d3e/d3f/d45/f9a [2180533,39726] 0 2024-07-16T09:55:52.691 INFO:tasks.workunit.client.0.smithi156.stdout:7/965: mknod dc/d13/d15/def/c14b 0 2024-07-16T09:55:52.693 INFO:tasks.workunit.client.1.smithi176.stdout:5/992: creat d4/d37/d72/d38/d102/dff/f160 x:0 0 0 2024-07-16T09:55:52.693 INFO:tasks.workunit.client.0.smithi156.stdout:6/958: sync 2024-07-16T09:55:52.694 INFO:tasks.workunit.client.1.smithi176.stdout:1/763: write d4/d8/d12/d21/ddd/fa0 [669789,125778] 0 2024-07-16T09:55:52.697 INFO:tasks.workunit.client.1.smithi176.stdout:5/993: creat d4/d37/d72/d38/d39/f161 x:0 0 0 2024-07-16T09:55:52.697 INFO:tasks.workunit.client.0.smithi156.stdout:8/985: dwrite d0/d11/d62/dc2/dcd/fec [0,4194304] 0 2024-07-16T09:55:52.699 INFO:tasks.workunit.client.1.smithi176.stdout:1/764: rename d4/d3a/d54/fb4 to d4/d77/dad/d72/f102 0 2024-07-16T09:55:52.701 INFO:tasks.workunit.client.1.smithi176.stdout:4/941: dwrite da/dc/d4b/df0/d96/d9b/db4/d40/f92 [0,4194304] 0 2024-07-16T09:55:52.701 INFO:tasks.workunit.client.0.smithi156.stdout:8/986: dread - d0/d11/d20/d7f/db7/d110/d1f/dee/f125 zero size 2024-07-16T09:55:52.702 INFO:tasks.workunit.client.0.smithi156.stdout:7/966: rename dc/d13/d36/dd4/d114/d11a/f138 to dc/d48/d119/d12f/f14c 0 2024-07-16T09:55:52.702 INFO:tasks.workunit.client.1.smithi176.stdout:1/765: write d4/d8/d12/d21/d25/da8/f97 [216588,109070] 0 2024-07-16T09:55:52.706 INFO:tasks.workunit.client.0.smithi156.stdout:8/987: write d0/d11/d13/d1c/d24/fb9 [3327515,93178] 0 2024-07-16T09:55:52.709 INFO:tasks.workunit.client.0.smithi156.stdout:7/967: rename dc/d48/la4 to dc/d13/d15/d8d/l14d 0 2024-07-16T09:55:52.710 INFO:tasks.workunit.client.1.smithi176.stdout:4/942: mknod da/dc/d4b/df0/d96/d9b/db4/c132 0 2024-07-16T09:55:52.711 INFO:tasks.workunit.client.1.smithi176.stdout:4/943: chown da/dc/d4b/df0/d28/c81 26258 1 2024-07-16T09:55:52.712 INFO:tasks.workunit.client.0.smithi156.stdout:6/959: dwrite d3/d13/d17/d4f/d4d/d3b/f93 [0,4194304] 0 2024-07-16T09:55:52.714 INFO:tasks.workunit.client.0.smithi156.stdout:7/968: chown dc/d3e/d3f/fbf 198162407 1 2024-07-16T09:55:52.715 INFO:tasks.workunit.client.0.smithi156.stdout:8/988: chown d0/d11/d29/d80/d97/de7/d93/c9b 115661218 1 2024-07-16T09:55:52.716 INFO:tasks.workunit.client.1.smithi176.stdout:4/944: fdatasync da/f42 0 2024-07-16T09:55:52.720 INFO:tasks.workunit.client.0.smithi156.stdout:8/989: creat d0/d11/d20/d3f/d141/f144 x:0 0 0 2024-07-16T09:55:52.722 INFO:tasks.workunit.client.1.smithi176.stdout:3/858: dwrite d3/d17/d1d/d4f/d59/f93 [0,4194304] 0 2024-07-16T09:55:52.722 INFO:tasks.workunit.client.1.smithi176.stdout:4/945: symlink da/d3d/d17/d54/d78/l133 0 2024-07-16T09:55:52.726 INFO:tasks.workunit.client.0.smithi156.stdout:7/969: creat dc/d3e/d3f/d51/f14e x:0 0 0 2024-07-16T09:55:52.726 INFO:tasks.workunit.client.1.smithi176.stdout:5/994: write d4/d14/d3d/f79 [1127288,1089] 0 2024-07-16T09:55:52.726 INFO:tasks.workunit.client.0.smithi156.stdout:7/970: chown dc/d3e/d76/df1/d115 7139999 1 2024-07-16T09:55:52.727 INFO:tasks.workunit.client.1.smithi176.stdout:1/766: truncate d4/d8/d12/d21/d25/d30/d44/d4d/f96 275037 0 2024-07-16T09:55:52.729 INFO:tasks.workunit.client.1.smithi176.stdout:5/995: chown d4/d37/d72/d38/d39/d9f/de8/d114 1165 1 2024-07-16T09:55:52.730 INFO:tasks.workunit.client.0.smithi156.stdout:7/971: mknod dc/d13/d36/dd4/c14f 0 2024-07-16T09:55:52.731 INFO:tasks.workunit.client.1.smithi176.stdout:3/859: fdatasync d3/d17/d1d/d4f/d59/fd0 0 2024-07-16T09:55:52.732 INFO:tasks.workunit.client.1.smithi176.stdout:4/946: write da/d3d/d17/f35 [1035388,94376] 0 2024-07-16T09:55:52.732 INFO:tasks.workunit.client.1.smithi176.stdout:1/767: read d4/d8/d12/d21/d25/da8/fd1 [312124,37592] 0 2024-07-16T09:55:52.733 INFO:tasks.workunit.client.1.smithi176.stdout:5/996: readlink d4/d37/d72/d38/d6e/l8d 0 2024-07-16T09:55:52.734 INFO:tasks.workunit.client.1.smithi176.stdout:4/947: write da/d3d/f11a [984628,14999] 0 2024-07-16T09:55:52.736 INFO:tasks.workunit.client.1.smithi176.stdout:1/768: dread d4/d8/fc6 [0,4194304] 0 2024-07-16T09:55:52.737 INFO:tasks.workunit.client.1.smithi176.stdout:3/860: mkdir d3/d17/d1d/d4f/daa/df3/dfb 0 2024-07-16T09:55:52.737 INFO:tasks.workunit.client.1.smithi176.stdout:4/948: write da/dc/d4b/df0/d28/f128 [594125,4204] 0 2024-07-16T09:55:52.738 INFO:tasks.workunit.client.1.smithi176.stdout:5/997: creat d4/d37/d72/d38/d102/dbd/f162 x:0 0 0 2024-07-16T09:55:52.741 INFO:tasks.workunit.client.1.smithi176.stdout:1/769: rename d4/d77 to d4/db/df3/d103 0 2024-07-16T09:55:52.743 INFO:tasks.workunit.client.1.smithi176.stdout:5/998: mkdir d4/d37/d72/d82/d163 0 2024-07-16T09:55:52.743 INFO:tasks.workunit.client.1.smithi176.stdout:5/999: chown d4/c58 15628580 1 2024-07-16T09:55:52.745 INFO:tasks.workunit.client.1.smithi176.stdout:3/861: unlink d3/da/d33/l73 0 2024-07-16T09:55:52.748 INFO:tasks.workunit.client.1.smithi176.stdout:1/770: symlink d4/d8/d12/d21/d25/d30/l104 0 2024-07-16T09:55:52.751 INFO:tasks.workunit.client.0.smithi156.stdout:7/972: truncate dc/d13/d20/f3b 1958989 0 2024-07-16T09:55:52.751 INFO:tasks.workunit.client.1.smithi176.stdout:1/771: creat d4/d3a/d8e/f105 x:0 0 0 2024-07-16T09:55:52.753 INFO:tasks.workunit.client.1.smithi176.stdout:4/949: getdents da/dc/d4b/df0/d96/d9b/db4/d4a/dc1 0 2024-07-16T09:55:52.754 INFO:tasks.workunit.client.0.smithi156.stdout:7/973: stat dc/d13/d20/c59 0 2024-07-16T09:55:52.756 INFO:tasks.workunit.client.1.smithi176.stdout:3/862: rename d3/d17/c1c to d3/da/d6c/dd9/cfc 0 2024-07-16T09:55:52.757 INFO:tasks.workunit.client.0.smithi156.stdout:8/990: dwrite d0/d11/d20/d3f/d96/fad [0,4194304] 0 2024-07-16T09:55:52.758 INFO:tasks.workunit.client.1.smithi176.stdout:1/772: unlink d4/db/df3/d103/dad/fe4 0 2024-07-16T09:55:52.760 INFO:tasks.workunit.client.1.smithi176.stdout:4/950: write da/dc/d4b/df0/d96/d9b/db4/d40/d5d/f100 [793583,110316] 0 2024-07-16T09:55:52.761 INFO:tasks.workunit.client.0.smithi156.stdout:6/960: dread d3/f25 [4194304,4194304] 0 2024-07-16T09:55:52.766 INFO:tasks.workunit.client.1.smithi176.stdout:3/863: symlink d3/da/d33/lfd 0 2024-07-16T09:55:52.766 INFO:tasks.workunit.client.0.smithi156.stdout:6/961: mknod d3/d13/d17/d4f/d30/d45/d5f/da8/d12d/c13b 0 2024-07-16T09:55:52.767 INFO:tasks.workunit.client.1.smithi176.stdout:3/864: truncate d3/d17/d1d/d4f/d59/fe5 214198 0 2024-07-16T09:55:52.767 INFO:tasks.workunit.client.0.smithi156.stdout:8/991: write d0/d11/d62/f90 [1297891,67383] 0 2024-07-16T09:55:52.768 INFO:tasks.workunit.client.1.smithi176.stdout:1/773: creat d4/db/d100/f106 x:0 0 0 2024-07-16T09:55:52.768 INFO:tasks.workunit.client.0.smithi156.stdout:7/974: dwrite dc/d3e/d3f/d51/f5e [0,4194304] 0 2024-07-16T09:55:52.769 INFO:tasks.workunit.client.0.smithi156.stdout:6/962: fdatasync d3/d13/d17/d4f/d4d/d3b/f96 0 2024-07-16T09:55:52.772 INFO:tasks.workunit.client.0.smithi156.stdout:8/992: symlink d0/d11/d20/d7f/db7/d110/l145 0 2024-07-16T09:55:52.776 INFO:tasks.workunit.client.0.smithi156.stdout:7/975: link dc/d3e/d3f/d51/d8a/dae/f149 dc/d63/f150 0 2024-07-16T09:55:52.777 INFO:tasks.workunit.client.1.smithi176.stdout:3/865: rename d3/da/d6c/dd9/cf0 to d3/da/dca/cfe 0 2024-07-16T09:55:52.778 INFO:tasks.workunit.client.0.smithi156.stdout:8/993: creat d0/d11/d29/d80/d97/de7/d143/f146 x:0 0 0 2024-07-16T09:55:52.778 INFO:tasks.workunit.client.1.smithi176.stdout:1/774: mkdir d4/d8/d12/d21/d34/daa/dbb/d107 0 2024-07-16T09:55:52.779 INFO:tasks.workunit.client.1.smithi176.stdout:1/775: stat d4/d8 0 2024-07-16T09:55:52.779 INFO:tasks.workunit.client.0.smithi156.stdout:6/963: creat d3/d13/d17/d4f/d4d/d3b/d5a/dc1/df9/f13c x:0 0 0 2024-07-16T09:55:52.779 INFO:tasks.workunit.client.1.smithi176.stdout:4/951: getdents da/d3d/d17/d54/d78 0 2024-07-16T09:55:52.780 INFO:tasks.workunit.client.1.smithi176.stdout:3/866: symlink d3/d17/d1d/d6e/lff 0 2024-07-16T09:55:52.781 INFO:tasks.workunit.client.1.smithi176.stdout:1/776: fdatasync d4/d3a/d54/f6d 0 2024-07-16T09:55:52.781 INFO:tasks.workunit.client.0.smithi156.stdout:8/994: mknod d0/d11/d29/d5e/c147 0 2024-07-16T09:55:52.781 INFO:tasks.workunit.client.0.smithi156.stdout:6/964: readlink d3/d13/d17/d4f/d1e/l4c 0 2024-07-16T09:55:52.782 INFO:tasks.workunit.client.0.smithi156.stdout:8/995: read - d0/d11/d13/fa1 zero size 2024-07-16T09:55:52.784 INFO:tasks.workunit.client.1.smithi176.stdout:4/952: mkdir da/dc/d4b/df0/d96/d9b/db4/d40/d5d/d134 0 2024-07-16T09:55:52.784 INFO:tasks.workunit.client.0.smithi156.stdout:6/965: chown d3/d13/d17/d4f/d4d/d3b/d5a/fd9 101650 1 2024-07-16T09:55:52.784 INFO:tasks.workunit.client.0.smithi156.stdout:7/976: dwrite dc/d13/d15/f91 [0,4194304] 0 2024-07-16T09:55:52.785 INFO:tasks.workunit.client.1.smithi176.stdout:1/777: read d4/db/f1b [4964465,37577] 0 2024-07-16T09:55:52.788 INFO:tasks.workunit.client.0.smithi156.stdout:6/966: creat d3/d13/d17/d4f/d4d/d3b/d5a/f13d x:0 0 0 2024-07-16T09:55:52.789 INFO:tasks.workunit.client.0.smithi156.stdout:8/996: rename d0/d11/d20/d3f/l9f to d0/d11/d13/d1c/d9a/l148 0 2024-07-16T09:55:52.791 INFO:tasks.workunit.client.1.smithi176.stdout:3/867: rename d3/da/d84/lea to d3/d17/d1d/l100 0 2024-07-16T09:55:52.793 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:55:52 smithi176 ceph-mon[36626]: Deploying cephadm binary to smithi156 2024-07-16T09:55:52.793 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:55:52 smithi176 ceph-mon[36626]: Deploying cephadm binary to smithi176 2024-07-16T09:55:52.793 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:55:52 smithi176 ceph-mon[36626]: pgmap v4: 65 pgs: 65 active+clean; 3.8 GiB data, 13 GiB used, 524 GiB / 536 GiB avail 2024-07-16T09:55:52.793 INFO:tasks.workunit.client.0.smithi156.stdout:7/977: rename dc/d3e/d3f/d51/d8a/laf to dc/d48/db2/dcf/l151 0 2024-07-16T09:55:52.794 INFO:tasks.workunit.client.1.smithi176.stdout:1/778: creat d4/db/df3/d103/f108 x:0 0 0 2024-07-16T09:55:52.794 INFO:tasks.workunit.client.1.smithi176.stdout:3/868: mkdir d3/da/d66/d101 0 2024-07-16T09:55:52.796 INFO:tasks.workunit.client.1.smithi176.stdout:4/953: dread da/dc/fe [0,4194304] 0 2024-07-16T09:55:52.797 INFO:tasks.workunit.client.1.smithi176.stdout:1/779: mknod d4/d8/d12/d21/d34/daa/dbb/c109 0 2024-07-16T09:55:52.797 INFO:tasks.workunit.client.1.smithi176.stdout:3/869: dread d3/d17/d1d/d6e/fe1 [0,4194304] 0 2024-07-16T09:55:52.798 INFO:tasks.workunit.client.0.smithi156.stdout:6/967: rename d3/d13/d17/dae/dfb to d3/d13/d17/d4f/d30/d45/d5f/da8/d13e 0 2024-07-16T09:55:52.798 INFO:tasks.workunit.client.0.smithi156.stdout:8/997: unlink d0/d11/ded/dd7/le1 0 2024-07-16T09:55:52.800 INFO:tasks.workunit.client.1.smithi176.stdout:1/780: fdatasync d4/f17 0 2024-07-16T09:55:52.802 INFO:tasks.workunit.client.1.smithi176.stdout:4/954: unlink da/dc/d4b/df0/d96/d9b/db4/d40/d5d/f100 0 2024-07-16T09:55:52.802 INFO:tasks.workunit.client.1.smithi176.stdout:1/781: chown d4/d3a/dc8/fd0 1638335 1 2024-07-16T09:55:52.803 INFO:tasks.workunit.client.0.smithi156.stdout:7/978: dwrite dc/d3e/d3f/d51/d8a/dae/f144 [0,4194304] 0 2024-07-16T09:55:52.803 INFO:tasks.workunit.client.0.smithi156.stdout:6/968: symlink d3/d13/d17/d4f/d94/d19/d31/l13f 0 2024-07-16T09:55:52.804 INFO:tasks.workunit.client.0.smithi156.stdout:8/998: write d0/d11/d20/d7f/db7/d110/d1f/d60/f98 [3561940,51756] 0 2024-07-16T09:55:52.805 INFO:tasks.workunit.client.1.smithi176.stdout:1/782: chown d4/d8/d12/d4c/fa7 5197 1 2024-07-16T09:55:52.805 INFO:tasks.workunit.client.1.smithi176.stdout:3/870: dwrite d3/d17/f51 [0,4194304] 0 2024-07-16T09:55:52.808 INFO:tasks.workunit.client.0.smithi156.stdout:8/999: rmdir d0/d11/d20/d3f/d96 39 2024-07-16T09:55:52.810 INFO:tasks.workunit.client.1.smithi176.stdout:4/955: fsync da/d41/dc3/fd4 0 2024-07-16T09:55:52.811 INFO:tasks.workunit.client.0.smithi156.stdout:6/969: unlink d3/d13/d17/d4f/d30/l131 0 2024-07-16T09:55:52.813 INFO:tasks.workunit.client.1.smithi176.stdout:1/783: mkdir d4/d8/d12/d21/d25/d30/d44/d10a 0 2024-07-16T09:55:52.813 INFO:tasks.workunit.client.0.smithi156.stdout:6/970: chown d3/d13/d17/d4f/d4d/d3b/d5a/dc1/df9/f13c 3985998 1 2024-07-16T09:55:52.814 INFO:tasks.workunit.client.0.smithi156.stdout:7/979: getdents dc/d3e/d3f/d45 0 2024-07-16T09:55:52.815 INFO:tasks.workunit.client.1.smithi176.stdout:4/956: fdatasync da/dc/d4b/df0/d96/da4/dc2/f10d 0 2024-07-16T09:55:52.815 INFO:tasks.workunit.client.1.smithi176.stdout:3/871: dwrite d3/d17/f51 [0,4194304] 0 2024-07-16T09:55:52.816 INFO:tasks.workunit.client.0.smithi156.stdout:7/980: write dc/d13/d36/dd4/d114/f130 [1021352,32984] 0 2024-07-16T09:55:52.818 INFO:tasks.workunit.client.0.smithi156.stdout:7/981: chown dc/d13/d20/c46 1870 1 2024-07-16T09:55:52.823 INFO:tasks.workunit.client.0.smithi156.stdout:7/982: mknod dc/d48/db2/dcf/d14a/db1/c152 0 2024-07-16T09:55:52.823 INFO:tasks.workunit.client.0.smithi156.stdout:6/971: dwrite d3/d13/d17/d4f/d4d/d3b/d5a/f117 [0,4194304] 0 2024-07-16T09:55:52.824 INFO:tasks.workunit.client.1.smithi176.stdout:3/872: symlink d3/d17/d1d/d4f/d59/da0/d7a/l102 0 2024-07-16T09:55:52.826 INFO:tasks.workunit.client.0.smithi156.stdout:6/972: fdatasync d3/d13/d17/d4f/d4d/d3b/f5d 0 2024-07-16T09:55:52.826 INFO:tasks.workunit.client.1.smithi176.stdout:3/873: read d3/d17/d1d/d4f/daa/fae [1303265,71955] 0 2024-07-16T09:55:52.829 INFO:tasks.workunit.client.1.smithi176.stdout:7/937: sync 2024-07-16T09:55:52.832 INFO:tasks.workunit.client.1.smithi176.stdout:1/784: dwrite d4/d8/f8c [0,4194304] 0 2024-07-16T09:55:52.832 INFO:tasks.workunit.client.0.smithi156.stdout:6/973: fdatasync d3/d13/d17/d4f/d4d/d3b/d5a/dc1/dce/fda 0 2024-07-16T09:55:52.836 INFO:tasks.workunit.client.0.smithi156.stdout:6/974: dread d3/d13/d17/d4f/d94/df3/f11f [0,4194304] 0 2024-07-16T09:55:52.837 INFO:tasks.workunit.client.1.smithi176.stdout:7/938: unlink d4/d47/d6d/c122 0 2024-07-16T09:55:52.842 INFO:tasks.workunit.client.1.smithi176.stdout:7/939: rename d4/d1f/d22 to d4/d5/d10f/d12a 0 2024-07-16T09:55:52.843 INFO:tasks.workunit.client.0.smithi156.stdout:6/975: dwrite d3/d13/d17/d4f/faf [0,4194304] 0 2024-07-16T09:55:52.844 INFO:tasks.workunit.client.1.smithi176.stdout:7/940: chown d4/f2b 5328624 1 2024-07-16T09:55:52.845 INFO:tasks.workunit.client.1.smithi176.stdout:4/957: dwrite da/d41/fc9 [0,4194304] 0 2024-07-16T09:55:52.845 INFO:tasks.workunit.client.1.smithi176.stdout:7/941: chown d4/d111 249544 1 2024-07-16T09:55:52.847 INFO:tasks.workunit.client.0.smithi156.stdout:6/976: mkdir d3/d13/d17/d4f/d4d/d3b/da5/d140 0 2024-07-16T09:55:52.851 INFO:tasks.workunit.client.1.smithi176.stdout:4/958: rmdir da/dc/d4b/df0/d96/d9b/db4/d40/d5d/d73 39 2024-07-16T09:55:52.853 INFO:tasks.workunit.client.0.smithi156.stdout:6/977: creat d3/d13/d17/d56/d11b/f141 x:0 0 0 2024-07-16T09:55:52.856 INFO:tasks.workunit.client.0.smithi156.stdout:6/978: mknod d3/d13/d17/d4f/d30/d45/c142 0 2024-07-16T09:55:52.857 INFO:tasks.workunit.client.1.smithi176.stdout:7/942: dwrite d4/d47/d6d/d89/d66/f8a [8388608,4194304] 0 2024-07-16T09:55:52.857 INFO:tasks.workunit.client.1.smithi176.stdout:3/874: dread d3/da/d33/f39 [4194304,4194304] 0 2024-07-16T09:55:52.858 INFO:tasks.workunit.client.1.smithi176.stdout:7/943: fdatasync d4/d47/d5d/fad 0 2024-07-16T09:55:52.861 INFO:tasks.workunit.client.1.smithi176.stdout:3/875: fsync d3/da/d6c/d86/f92 0 2024-07-16T09:55:52.864 INFO:tasks.workunit.client.0.smithi156.stdout:6/979: truncate d3/d13/d17/d4f/d4d/d3b/da5/fd5 849021 0 2024-07-16T09:55:52.865 INFO:tasks.workunit.client.1.smithi176.stdout:4/959: write da/dc/d4b/df0/d96/d9b/db4/d40/d5d/d73/f4f [584972,16617] 0 2024-07-16T09:55:52.866 INFO:tasks.workunit.client.1.smithi176.stdout:7/944: link d4/dd/d63/f64 d4/dd/d26/dc2/d17/d1d/ddc/f12b 0 2024-07-16T09:55:52.869 INFO:tasks.workunit.client.1.smithi176.stdout:7/945: mkdir d4/d47/d6d/db7/d127/d12c 0 2024-07-16T09:55:52.871 INFO:tasks.workunit.client.0.smithi156.stdout:6/980: rename d3/d13/d17/d4f/d30/d45/d5f/da8/fed to d3/d13/d17/d4f/d4d/f143 0 2024-07-16T09:55:52.875 INFO:tasks.workunit.client.1.smithi176.stdout:7/946: dwrite d4/d1f/f59 [0,4194304] 0 2024-07-16T09:55:52.878 INFO:tasks.workunit.client.1.smithi176.stdout:7/947: readlink d4/d47/d6d/d89/l69 0 2024-07-16T09:55:52.878 INFO:tasks.workunit.client.0.smithi156.stdout:6/981: dwrite d3/d13/d17/d4f/d4d/d3b/d5a/dc1/dce/fda [0,4194304] 0 2024-07-16T09:55:52.878 INFO:tasks.workunit.client.1.smithi176.stdout:7/948: stat d4/d47/d6d/d89/l69 0 2024-07-16T09:55:52.881 INFO:tasks.workunit.client.1.smithi176.stdout:7/949: mknod d4/d47/d6d/db7/d127/d12c/c12d 0 2024-07-16T09:55:52.882 INFO:tasks.workunit.client.0.smithi156.stdout:6/982: mknod d3/d13/d17/d4f/d30/d45/c144 0 2024-07-16T09:55:52.885 INFO:tasks.workunit.client.1.smithi176.stdout:7/950: chown d4/d47/d6d/d89/d7e/c82 9687596 1 2024-07-16T09:55:52.886 INFO:tasks.workunit.client.1.smithi176.stdout:7/951: dread - d4/d47/d5d/d9a/da3/ff3 zero size 2024-07-16T09:55:52.889 INFO:tasks.workunit.client.1.smithi176.stdout:1/785: truncate d4/d8/d12/d21/fb9 334392 0 2024-07-16T09:55:52.890 INFO:tasks.workunit.client.0.smithi156.stdout:6/983: dread d3/d13/d17/f8a [0,4194304] 0 2024-07-16T09:55:52.891 INFO:tasks.workunit.client.1.smithi176.stdout:7/952: mkdir d4/d47/d6d/d89/d7e/d12e 0 2024-07-16T09:55:52.892 INFO:tasks.workunit.client.1.smithi176.stdout:3/876: write d3/d17/f88 [1044891,85297] 0 2024-07-16T09:55:52.893 INFO:tasks.workunit.client.1.smithi176.stdout:7/953: write d4/dd/d26/dc2/d17/f7f [4568734,54197] 0 2024-07-16T09:55:52.893 INFO:tasks.workunit.client.0.smithi156.stdout:6/984: chown d3/d13/d17/d4f/d30/f67 10285 1 2024-07-16T09:55:52.894 INFO:tasks.workunit.client.0.smithi156.stdout:6/985: fsync d3/d13/fc4 0 2024-07-16T09:55:52.894 INFO:tasks.workunit.client.1.smithi176.stdout:7/954: write d4/d5/f6c [349293,96611] 0 2024-07-16T09:55:52.894 INFO:tasks.workunit.client.1.smithi176.stdout:3/877: truncate d3/d85/ddd/fee 1030975 0 2024-07-16T09:55:52.895 INFO:tasks.workunit.client.0.smithi156.stdout:6/986: chown d3/d13/fab 174195 1 2024-07-16T09:55:52.896 INFO:tasks.workunit.client.1.smithi176.stdout:1/786: creat d4/d8/d12/d21/d25/f10b x:0 0 0 2024-07-16T09:55:52.896 INFO:tasks.workunit.client.1.smithi176.stdout:7/955: chown d4/d47/d6d/d89/d66/dc4 7416828 1 2024-07-16T09:55:52.896 INFO:tasks.workunit.client.1.smithi176.stdout:4/960: write da/d3d/dac/fb9 [639621,50269] 0 2024-07-16T09:55:52.899 INFO:tasks.workunit.client.1.smithi176.stdout:3/878: write d3/d17/d1d/d4f/d59/fd0 [317630,23896] 0 2024-07-16T09:55:52.901 INFO:tasks.workunit.client.0.smithi156.stdout:6/987: rename d3/d13/d17/d4f/d1e/c6b to d3/d13/d17/d4f/d4d/c145 0 2024-07-16T09:55:52.901 INFO:tasks.workunit.client.1.smithi176.stdout:4/961: mkdir da/dc/d4b/d62/d125/d135 0 2024-07-16T09:55:52.901 INFO:tasks.workunit.client.1.smithi176.stdout:7/956: write d4/d5/d10f/d12a/f52 [3294611,99727] 0 2024-07-16T09:55:52.901 INFO:tasks.workunit.client.1.smithi176.stdout:3/879: write d3/d17/fd8 [321799,67967] 0 2024-07-16T09:55:52.902 INFO:tasks.workunit.client.1.smithi176.stdout:4/962: stat da/fb 0 2024-07-16T09:55:52.903 INFO:tasks.workunit.client.0.smithi156.stdout:6/988: mkdir d3/d13/d17/d4f/d1e/d146 0 2024-07-16T09:55:52.903 INFO:tasks.workunit.client.1.smithi176.stdout:1/787: rename d4/d8/d12/d21/d25/c46 to d4/d8/d12/d4c/da4/da9/c10c 0 2024-07-16T09:55:52.904 INFO:tasks.workunit.client.1.smithi176.stdout:3/880: dread - d3/d17/d1d/d6e/fc0 zero size 2024-07-16T09:55:52.905 INFO:tasks.workunit.client.1.smithi176.stdout:7/957: mkdir d4/d47/d6d/d12f 0 2024-07-16T09:55:52.905 INFO:tasks.workunit.client.1.smithi176.stdout:3/881: write d3/d17/f51 [2501270,95244] 0 2024-07-16T09:55:52.905 INFO:tasks.workunit.client.0.smithi156.stdout:6/989: mknod d3/d13/d17/d4f/d4d/d3b/da5/c147 0 2024-07-16T09:55:52.906 INFO:tasks.workunit.client.1.smithi176.stdout:4/963: truncate da/f42 1664488 0 2024-07-16T09:55:52.907 INFO:tasks.workunit.client.0.smithi156.stdout:6/990: creat d3/d13/d17/d4f/d94/d19/de5/f148 x:0 0 0 2024-07-16T09:55:52.908 INFO:tasks.workunit.client.1.smithi176.stdout:3/882: unlink d3/f55 0 2024-07-16T09:55:52.912 INFO:tasks.workunit.client.0.smithi156.stdout:6/991: dread d3/d13/d17/d4f/d4d/d3b/d77/fd2 [0,4194304] 0 2024-07-16T09:55:52.913 INFO:tasks.workunit.client.1.smithi176.stdout:7/958: creat d4/dd/d26/f130 x:0 0 0 2024-07-16T09:55:52.914 INFO:tasks.workunit.client.1.smithi176.stdout:1/788: getdents d4/db/d100 0 2024-07-16T09:55:52.918 INFO:tasks.workunit.client.1.smithi176.stdout:7/959: creat d4/d47/d6d/db7/d127/f131 x:0 0 0 2024-07-16T09:55:52.926 INFO:tasks.workunit.client.1.smithi176.stdout:4/964: truncate da/dc/d4b/f52 562272 0 2024-07-16T09:55:52.927 INFO:tasks.workunit.client.1.smithi176.stdout:3/883: rmdir d3/da/d6c/dbc 39 2024-07-16T09:55:52.927 INFO:tasks.workunit.client.0.smithi156.stdout:6/992: dwrite d3/d13/d17/d4f/fe9 [0,4194304] 0 2024-07-16T09:55:52.928 INFO:tasks.workunit.client.0.smithi156.stdout:6/993: chown d3/d13/d17/d4f/d30/d45 23 1 2024-07-16T09:55:52.928 INFO:tasks.workunit.client.1.smithi176.stdout:4/965: write da/dc/d4b/df0/d96/d9b/db4/d40/d5d/f85 [3893758,40163] 0 2024-07-16T09:55:52.930 INFO:tasks.workunit.client.0.smithi156.stdout:6/994: chown d3/d13/d17/d4f/d30/d45/d99/d114/f126 5397 1 2024-07-16T09:55:52.931 INFO:tasks.workunit.client.1.smithi176.stdout:7/960: dread d4/d5/d10f/d12a/d51/f98 [0,4194304] 0 2024-07-16T09:55:52.932 INFO:tasks.workunit.client.0.smithi156.stdout:6/995: write d3/d13/d17/d4f/d1e/fe8 [808372,108126] 0 2024-07-16T09:55:52.934 INFO:tasks.workunit.client.0.smithi156.stdout:6/996: chown d3/d13/d17/d4f/d5c 1980 1 2024-07-16T09:55:52.935 INFO:tasks.workunit.client.1.smithi176.stdout:3/884: rmdir d3/d17/d1d/d4f/daa/df3 39 2024-07-16T09:55:52.935 INFO:tasks.workunit.client.1.smithi176.stdout:4/966: rmdir da/dc/d4b/df0/d96/d69 39 2024-07-16T09:55:52.936 INFO:tasks.workunit.client.0.smithi156.stdout:6/997: truncate d3/d13/d17/d4f/d94/d19/f110 563243 0 2024-07-16T09:55:52.937 INFO:tasks.workunit.client.0.smithi156.stdout:6/998: write d3/d13/d17/d4f/d30/d45/d99/d114/f126 [377630,37462] 0 2024-07-16T09:55:52.939 INFO:tasks.workunit.client.1.smithi176.stdout:7/961: mknod d4/dd/d49/d7a/d115/c132 0 2024-07-16T09:55:52.940 INFO:tasks.workunit.client.1.smithi176.stdout:7/962: readlink d4/d5/d97/d110/lc7 0 2024-07-16T09:55:52.940 INFO:tasks.workunit.client.0.smithi156.stdout:6/999: readlink d3/d13/d17/d4f/d4d/d3b/d77/ldb 0 2024-07-16T09:55:52.940 INFO:tasks.workunit.client.1.smithi176.stdout:7/963: chown d4/fb 0 1 2024-07-16T09:55:52.941 INFO:tasks.workunit.client.1.smithi176.stdout:4/967: symlink da/dc/d4b/l136 0 2024-07-16T09:55:52.943 INFO:tasks.workunit.client.1.smithi176.stdout:7/964: unlink d4/d47/d5d/d9a/da3/ff3 0 2024-07-16T09:55:52.946 INFO:tasks.workunit.client.1.smithi176.stdout:3/885: write d3/d17/d1d/d6e/fc0 [148286,96857] 0 2024-07-16T09:55:52.949 INFO:tasks.workunit.client.1.smithi176.stdout:4/968: dwrite da/dc/d4b/df0/d96/d9b/db4/d4a/dc1/f124 [0,4194304] 0 2024-07-16T09:55:52.950 INFO:tasks.workunit.client.1.smithi176.stdout:3/886: creat d3/d17/d1d/d4f/daa/f103 x:0 0 0 2024-07-16T09:55:52.951 INFO:tasks.workunit.client.1.smithi176.stdout:4/969: write da/fb [3725798,14849] 0 2024-07-16T09:55:52.954 INFO:tasks.workunit.client.1.smithi176.stdout:4/970: mkdir da/d41/d5b/dba/d137 0 2024-07-16T09:55:52.958 INFO:tasks.workunit.client.1.smithi176.stdout:4/971: fsync da/dc/d4b/df0/d96/d9b/fe7 0 2024-07-16T09:55:52.960 INFO:tasks.workunit.client.1.smithi176.stdout:3/887: dwrite d3/d17/d1d/d6e/fab [0,4194304] 0 2024-07-16T09:55:52.962 INFO:tasks.workunit.client.1.smithi176.stdout:7/965: dwrite d4/d47/d6d/d89/f50 [4194304,4194304] 0 2024-07-16T09:55:52.962 INFO:tasks.workunit.client.1.smithi176.stdout:4/972: fdatasync da/dc/d4b/df0/d96/d9b/db4/d40/d5d/f12f 0 2024-07-16T09:55:52.963 INFO:tasks.workunit.client.1.smithi176.stdout:3/888: creat d3/da/d66/f104 x:0 0 0 2024-07-16T09:55:52.965 INFO:tasks.workunit.client.1.smithi176.stdout:7/966: mkdir d4/dd/d133 0 2024-07-16T09:55:52.975 INFO:tasks.workunit.client.1.smithi176.stdout:4/973: dwrite da/dc/d4b/df0/d96/da4/dc2/fda [0,4194304] 0 2024-07-16T09:55:52.977 INFO:tasks.workunit.client.1.smithi176.stdout:7/967: link d4/dd/d81/f113 d4/d47/d6d/d12f/f134 0 2024-07-16T09:55:52.979 INFO:tasks.workunit.client.1.smithi176.stdout:7/968: rmdir d4/d47/d6d/d89 39 2024-07-16T09:55:52.980 INFO:tasks.workunit.client.1.smithi176.stdout:4/974: creat da/dc/d4b/f138 x:0 0 0 2024-07-16T09:55:52.983 INFO:tasks.workunit.client.1.smithi176.stdout:3/889: dwrite d3/da/d33/f52 [0,4194304] 0 2024-07-16T09:55:52.984 INFO:tasks.workunit.client.1.smithi176.stdout:4/975: fsync da/d3d/d17/d54/d78/f114 0 2024-07-16T09:55:52.987 INFO:tasks.workunit.client.1.smithi176.stdout:4/976: write da/d41/f99 [432665,122353] 0 2024-07-16T09:55:52.989 INFO:tasks.workunit.client.1.smithi176.stdout:3/890: chown d3/d17/d1d/d4f/d59/da0/d7a/fc6 119 1 2024-07-16T09:55:52.992 INFO:tasks.workunit.client.1.smithi176.stdout:7/969: dwrite d4/d5/d97/fee [0,4194304] 0 2024-07-16T09:55:52.995 INFO:tasks.workunit.client.1.smithi176.stdout:3/891: mknod d3/d17/d1d/d4f/d59/c105 0 2024-07-16T09:55:52.997 INFO:tasks.workunit.client.1.smithi176.stdout:3/892: truncate d3/d17/d1d/d4f/daa/ff4 951072 0 2024-07-16T09:55:52.998 INFO:tasks.workunit.client.1.smithi176.stdout:3/893: readlink d3/d17/d1d/d4f/d59/da0/l48 0 2024-07-16T09:55:53.006 INFO:tasks.workunit.client.1.smithi176.stdout:4/977: write da/d41/d5b/dba/fdf [671802,62486] 0 2024-07-16T09:55:53.007 INFO:tasks.workunit.client.1.smithi176.stdout:4/978: fdatasync da/dc/d4b/df0/d96/d9b/fe7 0 2024-07-16T09:55:53.009 INFO:tasks.workunit.client.1.smithi176.stdout:4/979: write da/dc/d4b/d62/f11f [64080,122929] 0 2024-07-16T09:55:53.010 INFO:tasks.workunit.client.1.smithi176.stdout:7/970: dread d4/d47/d6d/d89/f50 [0,4194304] 0 2024-07-16T09:55:53.012 INFO:tasks.workunit.client.1.smithi176.stdout:4/980: creat da/dc/d4b/df0/d28/d106/f139 x:0 0 0 2024-07-16T09:55:53.013 INFO:tasks.workunit.client.1.smithi176.stdout:7/971: creat d4/dd/d26/dc2/f135 x:0 0 0 2024-07-16T09:55:53.013 INFO:tasks.workunit.client.1.smithi176.stdout:4/981: mkdir da/d3d/d17/d54/d78/d13a 0 2024-07-16T09:55:53.014 INFO:tasks.workunit.client.1.smithi176.stdout:4/982: chown da/d41/dc3/f8b 5520 1 2024-07-16T09:55:53.015 INFO:tasks.workunit.client.1.smithi176.stdout:7/972: symlink d4/d47/d6d/d89/d7e/d87/l136 0 2024-07-16T09:55:53.017 INFO:tasks.workunit.client.1.smithi176.stdout:4/983: rename da/dc/d4b/lb6 to da/dc/d4b/d62/l13b 0 2024-07-16T09:55:53.018 INFO:tasks.workunit.client.1.smithi176.stdout:7/973: mknod d4/c137 0 2024-07-16T09:55:53.031 INFO:tasks.workunit.client.1.smithi176.stdout:7/974: dread d4/dd/db5/fe9 [0,4194304] 0 2024-07-16T09:55:53.031 INFO:tasks.workunit.client.1.smithi176.stdout:4/984: dread da/dc/d4b/f97 [0,4194304] 0 2024-07-16T09:55:53.032 INFO:tasks.workunit.client.1.smithi176.stdout:7/975: chown d4/d5/f2f 6407 1 2024-07-16T09:55:53.034 INFO:tasks.workunit.client.1.smithi176.stdout:7/976: creat d4/d47/d6d/d89/d7e/f138 x:0 0 0 2024-07-16T09:55:53.035 INFO:tasks.workunit.client.1.smithi176.stdout:7/977: fdatasync d4/dd/d49/f55 0 2024-07-16T09:55:53.036 INFO:tasks.workunit.client.1.smithi176.stdout:4/985: dread da/d3d/f11a [0,4194304] 0 2024-07-16T09:55:53.038 INFO:tasks.workunit.client.1.smithi176.stdout:4/986: mknod da/dc/d4b/d62/c13c 0 2024-07-16T09:55:53.040 INFO:tasks.workunit.client.1.smithi176.stdout:7/978: dread d4/dd/d26/dc2/d17/d1d/f73 [0,4194304] 0 2024-07-16T09:55:53.041 INFO:tasks.workunit.client.1.smithi176.stdout:4/987: dread da/d3d/f10f [0,4194304] 0 2024-07-16T09:55:53.042 INFO:tasks.workunit.client.1.smithi176.stdout:7/979: creat d4/d5/d10f/f139 x:0 0 0 2024-07-16T09:55:53.042 INFO:tasks.workunit.client.1.smithi176.stdout:4/988: fdatasync da/d3d/f103 0 2024-07-16T09:55:53.044 INFO:tasks.workunit.client.1.smithi176.stdout:7/980: chown d4/dd/d26/dc2/d17/d1d/f5b 1068292358 1 2024-07-16T09:55:53.046 INFO:tasks.workunit.client.1.smithi176.stdout:7/981: mkdir d4/dd/d63/d13a 0 2024-07-16T09:55:53.050 INFO:tasks.workunit.client.1.smithi176.stdout:4/989: dwrite da/dc/d4b/df0/d96/da4/dc2/ff7 [0,4194304] 0 2024-07-16T09:55:53.056 INFO:tasks.workunit.client.1.smithi176.stdout:7/982: dwrite d4/dd/f2e [4194304,4194304] 0 2024-07-16T09:55:53.061 INFO:tasks.workunit.client.1.smithi176.stdout:7/983: read d4/dd/d63/f64 [1853656,291] 0 2024-07-16T09:55:53.062 INFO:tasks.workunit.client.1.smithi176.stdout:4/990: truncate da/d41/fc9 4190727 0 2024-07-16T09:55:53.064 INFO:tasks.workunit.client.1.smithi176.stdout:7/984: creat d4/d47/d6d/d89/d66/dc4/f13b x:0 0 0 2024-07-16T09:55:53.064 INFO:tasks.workunit.client.1.smithi176.stdout:7/985: readlink d4/d5/d10f/d12a/l42 0 2024-07-16T09:55:53.066 INFO:tasks.workunit.client.1.smithi176.stdout:7/986: write d4/d47/d5d/f90 [1857327,18942] 0 2024-07-16T09:55:53.072 INFO:tasks.workunit.client.1.smithi176.stdout:7/987: dwrite d4/d47/d6d/d89/d66/ff9 [0,4194304] 0 2024-07-16T09:55:53.076 INFO:tasks.workunit.client.1.smithi176.stdout:7/988: creat d4/f13c x:0 0 0 2024-07-16T09:55:53.078 INFO:tasks.workunit.client.1.smithi176.stdout:7/989: rmdir d4/d47 39 2024-07-16T09:55:53.081 INFO:tasks.workunit.client.1.smithi176.stdout:7/990: rename d4/dd/d49/dd5/f108 to d4/d47/d6d/db7/d127/f13d 0 2024-07-16T09:55:53.082 INFO:tasks.workunit.client.1.smithi176.stdout:7/991: chown d4/dd/d26 111 1 2024-07-16T09:55:53.084 INFO:tasks.workunit.client.1.smithi176.stdout:7/992: dread d4/d47/d6d/db7/d127/f13d [0,4194304] 0 2024-07-16T09:55:53.085 INFO:tasks.workunit.client.1.smithi176.stdout:7/993: creat d4/d5/d97/d110/dff/f13e x:0 0 0 2024-07-16T09:55:53.087 INFO:tasks.workunit.client.1.smithi176.stdout:7/994: creat d4/d5/d10f/d12a/d125/f13f x:0 0 0 2024-07-16T09:55:53.114 INFO:tasks.workunit.client.0.smithi156.stdout:7/983: sync 2024-07-16T09:55:53.118 INFO:tasks.workunit.client.0.smithi156.stdout:7/984: rename dc/d3e/d76/l11b to dc/d63/d98/l153 0 2024-07-16T09:55:53.120 INFO:tasks.workunit.client.0.smithi156.stdout:7/985: write dc/d3e/d3f/d45/fb3 [2999055,61045] 0 2024-07-16T09:55:53.124 INFO:tasks.workunit.client.0.smithi156.stdout:7/986: rename dc/d48/db2/dcf/d14a/d81/ffb to dc/d3e/d3f/d45/d104/f154 0 2024-07-16T09:55:53.126 INFO:tasks.workunit.client.1.smithi176.stdout:1/789: sync 2024-07-16T09:55:53.128 INFO:tasks.workunit.client.1.smithi176.stdout:1/790: mknod d4/d3a/d8e/c10d 0 2024-07-16T09:55:53.129 INFO:tasks.workunit.client.1.smithi176.stdout:1/791: write d4/d8/d12/d21/f9c [712363,52287] 0 2024-07-16T09:55:53.130 INFO:tasks.workunit.client.0.smithi156.stdout:7/987: write dc/d13/d36/f4c [2637528,2579] 0 2024-07-16T09:55:53.130 INFO:tasks.workunit.client.1.smithi176.stdout:1/792: chown d4/d8/d12/d21/d25/d30/f61 12 1 2024-07-16T09:55:53.134 INFO:tasks.workunit.client.1.smithi176.stdout:1/793: rmdir d4/d8/d12 39 2024-07-16T09:55:53.139 INFO:tasks.workunit.client.1.smithi176.stdout:1/794: rename d4/d8/ld to d4/d3a/d54/d5a/d7a/l10e 0 2024-07-16T09:55:53.139 INFO:tasks.workunit.client.0.smithi156.stdout:7/988: dwrite dc/d13/d36/dd4/d114/f80 [0,4194304] 0 2024-07-16T09:55:53.140 INFO:tasks.workunit.client.1.smithi176.stdout:1/795: chown d4/d8 689414391 1 2024-07-16T09:55:53.140 INFO:tasks.workunit.client.0.smithi156.stdout:7/989: read - dc/d13/fa5 zero size 2024-07-16T09:55:53.144 INFO:tasks.workunit.client.1.smithi176.stdout:1/796: unlink d4/d8/d12/d21/d25/d30/d44/d4d/cb3 0 2024-07-16T09:55:53.144 INFO:tasks.workunit.client.0.smithi156.stdout:7/990: unlink dc/d3e/d3f/ff5 0 2024-07-16T09:55:53.147 INFO:tasks.workunit.client.0.smithi156.stdout:7/991: mkdir dc/d3e/d3f/d45/d104/d155 0 2024-07-16T09:55:53.152 INFO:tasks.workunit.client.1.smithi176.stdout:1/797: dwrite d4/db/d100/f106 [0,4194304] 0 2024-07-16T09:55:53.152 INFO:tasks.workunit.client.0.smithi156.stdout:7/992: getdents dc/d3e/d3f/d51/d8a/dae 0 2024-07-16T09:55:53.155 INFO:tasks.workunit.client.0.smithi156.stdout:7/993: mkdir dc/d63/d156 0 2024-07-16T09:55:53.158 INFO:tasks.workunit.client.0.smithi156.stdout:7/994: creat dc/d48/dd7/f157 x:0 0 0 2024-07-16T09:55:53.169 INFO:tasks.workunit.client.0.smithi156.stdout:7/995: write dc/d48/fa3 [5720900,80960] 0 2024-07-16T09:55:53.171 INFO:tasks.workunit.client.0.smithi156.stdout:7/996: mknod dc/d48/db2/c158 0 2024-07-16T09:55:53.173 INFO:tasks.workunit.client.0.smithi156.stdout:7/997: symlink dc/d63/l159 0 2024-07-16T09:55:53.175 INFO:tasks.workunit.client.0.smithi156.stdout:7/998: rename dc/d13/d15/f12d to dc/d48/db2/dcf/f15a 0 2024-07-16T09:55:53.176 INFO:tasks.workunit.client.0.smithi156.stdout:7/999: rmdir dc/d63/d98 39 2024-07-16T09:55:53.179 INFO:tasks.workunit.client.0.smithi156.stderr:+ rm -rf -- ./tmp.6LAWRNGYoj 2024-07-16T09:55:53.226 INFO:tasks.workunit.client.1.smithi176.stdout:1/798: mknod d4/d3a/dc8/c10f 0 2024-07-16T09:55:53.228 INFO:tasks.workunit.client.1.smithi176.stdout:1/799: rmdir d4 39 2024-07-16T09:55:53.230 INFO:tasks.workunit.client.1.smithi176.stdout:1/800: mknod d4/d8/c110 0 2024-07-16T09:55:53.244 INFO:tasks.workunit.client.1.smithi176.stdout:1/801: dread d4/d8/f69 [0,4194304] 0 2024-07-16T09:55:53.245 INFO:tasks.workunit.client.1.smithi176.stdout:1/802: write d4/d8/f8c [2517481,119496] 0 2024-07-16T09:55:53.247 INFO:tasks.workunit.client.1.smithi176.stdout:1/803: fdatasync d4/d3a/d54/d5a/f5b 0 2024-07-16T09:55:53.272 INFO:tasks.workunit.client.1.smithi176.stdout:3/894: sync 2024-07-16T09:55:53.276 INFO:tasks.workunit.client.1.smithi176.stdout:3/895: mknod d3/d17/d83/dd6/c106 0 2024-07-16T09:55:53.281 INFO:tasks.workunit.client.1.smithi176.stdout:3/896: truncate d3/d17/f40 3046464 0 2024-07-16T09:55:53.287 INFO:tasks.workunit.client.1.smithi176.stdout:3/897: dwrite d3/d85/ddd/fee [0,4194304] 0 2024-07-16T09:55:53.299 INFO:tasks.workunit.client.1.smithi176.stdout:3/898: dwrite d3/da/f77 [0,4194304] 0 2024-07-16T09:55:53.300 INFO:tasks.workunit.client.1.smithi176.stdout:3/899: stat d3/da/d33/f52 0 2024-07-16T09:55:53.306 INFO:tasks.workunit.client.1.smithi176.stdout:3/900: mkdir d3/d17/d1d/d4f/daa/df3/d107 0 2024-07-16T09:55:53.312 INFO:tasks.workunit.client.1.smithi176.stdout:3/901: dwrite d3/da/fe [0,4194304] 0 2024-07-16T09:55:53.314 INFO:tasks.workunit.client.1.smithi176.stdout:3/902: write d3/d17/d1d/d6e/fd5 [2834761,81476] 0 2024-07-16T09:55:53.338 INFO:tasks.workunit.client.1.smithi176.stdout:3/903: dwrite d3/d17/f68 [0,4194304] 0 2024-07-16T09:55:53.349 INFO:tasks.workunit.client.1.smithi176.stdout:3/904: dwrite d3/d17/d1d/d4f/d59/da0/f9e [0,4194304] 0 2024-07-16T09:55:53.352 INFO:tasks.workunit.client.1.smithi176.stdout:3/905: unlink d3/l18 0 2024-07-16T09:55:53.356 INFO:tasks.workunit.client.1.smithi176.stdout:3/906: mkdir d3/d17/d1d/d4f/daa/df3/dfb/d108 0 2024-07-16T09:55:53.360 INFO:tasks.workunit.client.1.smithi176.stdout:3/907: mkdir d3/da/d6c/dbc/d109 0 2024-07-16T09:55:53.366 INFO:tasks.workunit.client.1.smithi176.stdout:3/908: getdents d3 0 2024-07-16T09:55:53.371 INFO:tasks.workunit.client.1.smithi176.stdout:3/909: write d3/da/d6c/d86/f92 [482092,34753] 0 2024-07-16T09:55:53.372 INFO:tasks.workunit.client.1.smithi176.stdout:3/910: fdatasync d3/d17/d83/f91 0 2024-07-16T09:55:53.377 INFO:tasks.workunit.client.1.smithi176.stdout:3/911: rename d3/da/d6c/d86 to d3/da/d84/d10a 0 2024-07-16T09:55:53.379 INFO:tasks.workunit.client.1.smithi176.stdout:3/912: rename d3/d17/d83 to d3/d17/d10b 0 2024-07-16T09:55:53.381 INFO:tasks.workunit.client.1.smithi176.stdout:3/913: write d3/d17/d1d/d4f/daa/ff4 [715839,50313] 0 2024-07-16T09:55:53.382 INFO:tasks.workunit.client.1.smithi176.stdout:3/914: fsync d3/d17/f88 0 2024-07-16T09:55:53.383 INFO:tasks.workunit.client.1.smithi176.stdout:3/915: truncate d3/d17/f2f 2068516 0 2024-07-16T09:55:53.385 INFO:tasks.workunit.client.1.smithi176.stdout:3/916: read d3/d17/d1d/d4f/daa/fae [296710,111550] 0 2024-07-16T09:55:53.398 INFO:tasks.workunit.client.1.smithi176.stdout:3/917: dread d3/d17/d1d/f76 [0,4194304] 0 2024-07-16T09:55:53.400 INFO:tasks.workunit.client.1.smithi176.stdout:3/918: creat d3/f10c x:0 0 0 2024-07-16T09:55:53.402 INFO:tasks.workunit.client.1.smithi176.stdout:3/919: dread - d3/d17/d1d/d4f/d6d/fe4 zero size 2024-07-16T09:55:53.403 INFO:tasks.workunit.client.1.smithi176.stdout:3/920: dread - d3/d17/d1d/d6e/fe9 zero size 2024-07-16T09:55:53.406 INFO:tasks.workunit.client.1.smithi176.stdout:3/921: getdents d3/dd1 0 2024-07-16T09:55:53.409 INFO:tasks.workunit.client.1.smithi176.stdout:3/922: getdents d3/d17/d1d/d4f/d59/da0/d7a 0 2024-07-16T09:55:53.412 INFO:tasks.workunit.client.1.smithi176.stdout:7/995: sync 2024-07-16T09:55:53.412 INFO:tasks.workunit.client.1.smithi176.stdout:4/991: sync 2024-07-16T09:55:53.413 INFO:tasks.workunit.client.1.smithi176.stdout:3/923: mkdir d3/d17/d1d/d10d 0 2024-07-16T09:55:53.416 INFO:tasks.workunit.client.1.smithi176.stdout:7/996: creat d4/d47/f140 x:0 0 0 2024-07-16T09:55:53.416 INFO:tasks.workunit.client.1.smithi176.stdout:3/924: dread d3/d17/d1d/d6e/fc0 [0,4194304] 0 2024-07-16T09:55:53.418 INFO:tasks.workunit.client.1.smithi176.stdout:3/925: unlink d3/d17/d1d/l49 0 2024-07-16T09:55:53.419 INFO:tasks.workunit.client.1.smithi176.stdout:4/992: getdents da/d3d/d17/d54/d78/d7a 0 2024-07-16T09:55:53.421 INFO:tasks.workunit.client.1.smithi176.stdout:3/926: mknod d3/dd1/c10e 0 2024-07-16T09:55:53.421 INFO:tasks.workunit.client.1.smithi176.stdout:3/927: dread - d3/d17/fe2 zero size 2024-07-16T09:55:53.424 INFO:tasks.workunit.client.1.smithi176.stdout:3/928: creat d3/d17/d1d/d4f/f10f x:0 0 0 2024-07-16T09:55:53.430 INFO:tasks.workunit.client.1.smithi176.stdout:7/997: dread d4/d47/f46 [0,4194304] 0 2024-07-16T09:55:53.431 INFO:tasks.workunit.client.1.smithi176.stdout:3/929: dwrite d3/d17/f88 [0,4194304] 0 2024-07-16T09:55:53.432 INFO:tasks.workunit.client.1.smithi176.stdout:7/998: symlink d4/d47/d5d/l141 0 2024-07-16T09:55:53.435 INFO:tasks.workunit.client.1.smithi176.stdout:7/999: mkdir d4/dd/d49/dd5/d142 0 2024-07-16T09:55:53.440 INFO:tasks.workunit.client.1.smithi176.stdout:3/930: dwrite d3/d17/d1d/d4f/d6d/f9f [0,4194304] 0 2024-07-16T09:55:53.575 INFO:tasks.workunit.client.1.smithi176.stdout:1/804: sync 2024-07-16T09:55:53.576 INFO:tasks.workunit.client.1.smithi176.stdout:1/805: write d4/d8/d12/d21/d25/da8/fd4 [193352,105901] 0 2024-07-16T09:55:53.579 INFO:tasks.workunit.client.1.smithi176.stdout:1/806: getdents d4/d8/d12/d4c/da4/da9 0 2024-07-16T09:55:53.581 INFO:tasks.workunit.client.1.smithi176.stdout:1/807: symlink d4/d3a/d54/d5a/d7a/l111 0 2024-07-16T09:55:53.586 INFO:tasks.workunit.client.1.smithi176.stdout:1/808: rename d4/d8/f69 to d4/d8/d12/d21/d25/d30/d44/d10a/f112 0 2024-07-16T09:55:53.601 INFO:tasks.workunit.client.1.smithi176.stdout:1/809: dwrite d4/db/f1b [8388608,4194304] 0 2024-07-16T09:55:53.602 INFO:tasks.workunit.client.1.smithi176.stdout:1/810: chown d4/d8/cc3 19 1 2024-07-16T09:55:53.622 INFO:tasks.workunit.client.1.smithi176.stdout:1/811: mknod d4/d3a/c113 0 2024-07-16T09:55:53.622 INFO:tasks.workunit.client.1.smithi176.stdout:1/812: fdatasync d4/d8/d12/d4c/da4/fbc 0 2024-07-16T09:55:53.623 INFO:tasks.workunit.client.1.smithi176.stdout:1/813: chown d4/d8/d2c 528626 1 2024-07-16T09:55:53.626 INFO:tasks.workunit.client.1.smithi176.stdout:1/814: creat d4/d3a/f114 x:0 0 0 2024-07-16T09:55:53.629 INFO:tasks.workunit.client.1.smithi176.stdout:1/815: rmdir d4/d8/d12/d21/d34/daa/dbb/d107 0 2024-07-16T09:55:53.657 INFO:tasks.workunit.client.1.smithi176.stdout:1/816: dwrite d4/d8/d12/d21/d25/d30/d44/f5f [0,4194304] 0 2024-07-16T09:55:53.663 INFO:tasks.workunit.client.1.smithi176.stdout:1/817: dwrite d4/d8/d12/d21/f9c [0,4194304] 0 2024-07-16T09:55:53.665 INFO:tasks.workunit.client.1.smithi176.stdout:4/993: sync 2024-07-16T09:55:53.665 INFO:tasks.workunit.client.1.smithi176.stdout:1/818: write d4/d8/d12/d21/fa2 [7669199,37969] 0 2024-07-16T09:55:53.667 INFO:tasks.workunit.client.1.smithi176.stdout:4/994: chown da/dc/d4b/df0/d96/d9b/db4/d40/c10e 336249 1 2024-07-16T09:55:53.668 INFO:tasks.workunit.client.1.smithi176.stdout:4/995: chown da/dc/d4b/df0/d96/da4/dc2/fda 1244619 1 2024-07-16T09:55:53.673 INFO:tasks.workunit.client.1.smithi176.stdout:4/996: creat da/dc/d4b/df0/f13d x:0 0 0 2024-07-16T09:55:53.674 INFO:tasks.workunit.client.1.smithi176.stdout:4/997: dread - da/d41/d5b/dba/f109 zero size 2024-07-16T09:55:53.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:55:53 smithi156 ceph-mon[29302]: mgrmap e23: smithi176.dqbrvx(active, since 3s) 2024-07-16T09:55:53.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:55:53 smithi156 ceph-mon[29302]: [16/Jul/2024:09:55:52] ENGINE Bus STARTING 2024-07-16T09:55:53.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:55:53 smithi156 ceph-mon[29302]: [16/Jul/2024:09:55:52] ENGINE Serving on http://172.21.15.176:8765 2024-07-16T09:55:53.680 INFO:tasks.workunit.client.1.smithi176.stdout:4/998: rename da/dc/d4b/df0/d28/f128 to da/dc/d4b/df0/d96/d9b/db4/d4a/dc1/f13e 0 2024-07-16T09:55:53.683 INFO:tasks.workunit.client.1.smithi176.stdout:1/819: truncate d4/d3a/d54/d5a/d7a/fa6 3640974 0 2024-07-16T09:55:53.683 INFO:tasks.workunit.client.1.smithi176.stdout:1/820: chown d4/d8/d12/d4c 689 1 2024-07-16T09:55:53.685 INFO:tasks.workunit.client.1.smithi176.stdout:4/999: getdents da/d41 0 2024-07-16T09:55:53.685 INFO:tasks.workunit.client.1.smithi176.stdout:1/821: truncate d4/d8/d12/d21/d25/da8/fd4 1343264 0 2024-07-16T09:55:53.829 INFO:tasks.workunit.client.1.smithi176.stdout:3/931: sync 2024-07-16T09:55:53.830 INFO:tasks.workunit.client.1.smithi176.stdout:3/932: truncate d3/d17/d1d/d6e/fcc 152526 0 2024-07-16T09:55:53.832 INFO:tasks.workunit.client.1.smithi176.stdout:3/933: write d3/d17/d1d/d4f/fd4 [939178,34066] 0 2024-07-16T09:55:53.838 INFO:tasks.workunit.client.1.smithi176.stdout:3/934: link d3/da/d7b/c95 d3/d17/d1d/d4f/d59/da0/d7a/dac/c110 0 2024-07-16T09:55:53.840 INFO:tasks.workunit.client.1.smithi176.stdout:3/935: creat d3/da/d7b/df8/f111 x:0 0 0 2024-07-16T09:55:53.843 INFO:tasks.workunit.client.1.smithi176.stdout:3/936: rmdir d3/d17/d1d/d6e 39 2024-07-16T09:55:53.843 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:55:53 smithi176 ceph-mon[36626]: mgrmap e23: smithi176.dqbrvx(active, since 3s) 2024-07-16T09:55:53.843 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:55:53 smithi176 ceph-mon[36626]: [16/Jul/2024:09:55:52] ENGINE Bus STARTING 2024-07-16T09:55:53.843 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:55:53 smithi176 ceph-mon[36626]: [16/Jul/2024:09:55:52] ENGINE Serving on http://172.21.15.176:8765 2024-07-16T09:55:53.850 INFO:tasks.workunit.client.1.smithi176.stdout:3/937: truncate d3/d17/d1d/d6e/fd5 451543 0 2024-07-16T09:55:53.850 INFO:tasks.workunit.client.1.smithi176.stdout:3/938: readlink d3/da/d7b/l8b 0 2024-07-16T09:55:53.852 INFO:tasks.workunit.client.1.smithi176.stdout:3/939: mknod d3/da/d56/c112 0 2024-07-16T09:55:53.855 INFO:tasks.workunit.client.1.smithi176.stdout:3/940: creat d3/d17/d1d/d4f/daa/f113 x:0 0 0 2024-07-16T09:55:53.858 INFO:tasks.workunit.client.1.smithi176.stdout:3/941: getdents d3/d17/d1d/d4f/daa/df3 0 2024-07-16T09:55:53.871 INFO:tasks.workunit.client.1.smithi176.stdout:1/822: sync 2024-07-16T09:55:53.874 INFO:tasks.workunit.client.1.smithi176.stdout:1/823: creat d4/d3a/d54/d5a/f115 x:0 0 0 2024-07-16T09:55:53.875 INFO:tasks.workunit.client.1.smithi176.stdout:1/824: stat d4/d3a/d54/d5a/d7a/dd7/le0 0 2024-07-16T09:55:53.879 INFO:tasks.workunit.client.1.smithi176.stdout:1/825: link d4/d3a/l6b d4/db/df3/l116 0 2024-07-16T09:55:53.881 INFO:tasks.workunit.client.1.smithi176.stdout:1/826: symlink d4/d8/d12/d21/d25/d30/d44/d10a/l117 0 2024-07-16T09:55:53.882 INFO:tasks.workunit.client.1.smithi176.stdout:1/827: dread - d4/d8/d12/d21/d34/daa/fe5 zero size 2024-07-16T09:55:53.884 INFO:tasks.workunit.client.1.smithi176.stdout:1/828: creat d4/d8/d12/d4c/da4/f118 x:0 0 0 2024-07-16T09:55:53.887 INFO:tasks.workunit.client.1.smithi176.stdout:1/829: symlink d4/de8/l119 0 2024-07-16T09:55:53.891 INFO:tasks.workunit.client.1.smithi176.stdout:1/830: rename d4/db/fc0 to d4/d8/d12/d21/d34/daa/dbb/f11a 0 2024-07-16T09:55:53.894 INFO:tasks.workunit.client.1.smithi176.stdout:1/831: getdents d4/d8/d12/d21/d34/daa/dbb/dec 0 2024-07-16T09:55:53.897 INFO:tasks.workunit.client.1.smithi176.stdout:1/832: rename d4/db/c9f to d4/d3a/d8e/c11b 0 2024-07-16T09:55:53.898 INFO:tasks.workunit.client.1.smithi176.stdout:1/833: dread - d4/d8/d12/d4c/da4/da9/fe2 zero size 2024-07-16T09:55:53.904 INFO:tasks.workunit.client.1.smithi176.stdout:1/834: dwrite d4/db/f5c [0,4194304] 0 2024-07-16T09:55:53.907 INFO:tasks.workunit.client.1.smithi176.stdout:1/835: creat d4/d8/d12/d21/d25/d30/d44/d4d/f11c x:0 0 0 2024-07-16T09:55:53.907 INFO:tasks.workunit.client.1.smithi176.stdout:1/836: dread - d4/db/df3/d103/f108 zero size 2024-07-16T09:55:53.910 INFO:tasks.workunit.client.1.smithi176.stdout:1/837: unlink d4/d8/d12/d21/d34/f3d 0 2024-07-16T09:55:53.912 INFO:tasks.workunit.client.1.smithi176.stdout:1/838: creat d4/d8/d12/d4c/da4/f11d x:0 0 0 2024-07-16T09:55:53.914 INFO:tasks.workunit.client.1.smithi176.stdout:1/839: write d4/d8/d12/d21/d25/f10b [787593,109881] 0 2024-07-16T09:55:53.917 INFO:tasks.workunit.client.1.smithi176.stdout:1/840: mkdir d4/d3a/d54/d5a/d11e 0 2024-07-16T09:55:53.921 INFO:tasks.workunit.client.1.smithi176.stdout:1/841: rename d4/d3a/l6b to d4/d8/d12/d21/d25/d30/d44/dd2/l11f 0 2024-07-16T09:55:53.923 INFO:tasks.workunit.client.1.smithi176.stdout:1/842: mkdir d4/d8/d12/d4c/d120 0 2024-07-16T09:55:54.065 INFO:tasks.workunit.client.1.smithi176.stdout:3/942: sync 2024-07-16T09:55:54.066 INFO:tasks.workunit.client.1.smithi176.stdout:3/943: read d3/d17/f90 [2398545,35468] 0 2024-07-16T09:55:54.082 INFO:tasks.workunit.client.1.smithi176.stdout:3/944: write d3/d17/d1d/d6e/fd5 [337196,79897] 0 2024-07-16T09:55:54.083 INFO:tasks.workunit.client.1.smithi176.stdout:3/945: readlink d3/da/ld3 0 2024-07-16T09:55:54.087 INFO:tasks.workunit.client.1.smithi176.stdout:3/946: link d3/da/d33/cbe d3/da/d6c/dd9/c114 0 2024-07-16T09:55:54.099 INFO:tasks.workunit.client.1.smithi176.stdout:3/947: dread d3/d17/d1d/d4f/fa3 [0,4194304] 0 2024-07-16T09:55:54.102 INFO:tasks.workunit.client.1.smithi176.stdout:3/948: creat d3/d17/d1d/d4f/daa/f115 x:0 0 0 2024-07-16T09:55:54.102 INFO:tasks.workunit.client.1.smithi176.stdout:3/949: chown d3/d17/f2f 9 1 2024-07-16T09:55:54.103 INFO:tasks.workunit.client.1.smithi176.stdout:3/950: read - d3/d17/fb9 zero size 2024-07-16T09:55:54.105 INFO:tasks.workunit.client.1.smithi176.stdout:3/951: dread - d3/f74 zero size 2024-07-16T09:55:54.107 INFO:tasks.workunit.client.1.smithi176.stdout:3/952: symlink d3/d17/d1d/d4f/daa/df3/d107/l116 0 2024-07-16T09:55:54.109 INFO:tasks.workunit.client.1.smithi176.stdout:3/953: creat d3/da/d7b/df8/f117 x:0 0 0 2024-07-16T09:55:54.112 INFO:tasks.workunit.client.1.smithi176.stdout:3/954: link d3/d17/d1d/d6e/fd5 d3/da/d6c/dbc/f118 0 2024-07-16T09:55:54.117 INFO:tasks.workunit.client.1.smithi176.stdout:3/955: dwrite d3/da/d33/f39 [0,4194304] 0 2024-07-16T09:55:54.125 INFO:tasks.workunit.client.1.smithi176.stdout:1/843: sync 2024-07-16T09:55:54.131 INFO:tasks.workunit.client.1.smithi176.stdout:1/844: dread d4/f24 [0,4194304] 0 2024-07-16T09:55:54.132 INFO:tasks.workunit.client.1.smithi176.stdout:1/845: chown d4/d8/d12/d21/d25/da8/f97 29 1 2024-07-16T09:55:54.135 INFO:tasks.workunit.client.1.smithi176.stdout:1/846: mknod d4/d3a/d8e/c121 0 2024-07-16T09:55:54.136 INFO:tasks.workunit.client.1.smithi176.stdout:1/847: fdatasync d4/d3a/d54/fed 0 2024-07-16T09:55:54.138 INFO:tasks.workunit.client.1.smithi176.stdout:1/848: creat d4/db/f122 x:0 0 0 2024-07-16T09:55:54.139 INFO:tasks.workunit.client.1.smithi176.stdout:1/849: chown d4/d8/d12/d21/d34/daa/dbb/f11a 5093886 1 2024-07-16T09:55:54.144 INFO:tasks.workunit.client.1.smithi176.stdout:1/850: dwrite d4/d8/d12/d21/d25/da8/f9d [0,4194304] 0 2024-07-16T09:55:54.148 INFO:tasks.workunit.client.1.smithi176.stdout:1/851: truncate d4/d3a/dc8/fd0 133974 0 2024-07-16T09:55:54.150 INFO:tasks.workunit.client.1.smithi176.stdout:1/852: write d4/d8/d12/d21/d25/da8/fd4 [192915,108693] 0 2024-07-16T09:55:54.152 INFO:tasks.workunit.client.1.smithi176.stdout:1/853: read d4/fb7 [218645,83057] 0 2024-07-16T09:55:54.153 INFO:tasks.workunit.client.1.smithi176.stdout:1/854: chown d4/d8/d12/d21/d25/da8/fd4 940 1 2024-07-16T09:55:54.154 INFO:tasks.workunit.client.1.smithi176.stdout:1/855: readlink d4/d8/d12/d21/d25/d30/d44/dd2/lcb 0 2024-07-16T09:55:54.156 INFO:tasks.workunit.client.1.smithi176.stdout:1/856: fsync d4/d3a/d54/d5a/f67 0 2024-07-16T09:55:54.164 INFO:tasks.workunit.client.1.smithi176.stdout:1/857: write d4/d3a/f9e [5541529,108338] 0 2024-07-16T09:55:54.179 INFO:tasks.workunit.client.1.smithi176.stdout:1/858: dread d4/d8/d12/d21/d25/d30/d44/f78 [0,4194304] 0 2024-07-16T09:55:54.185 INFO:tasks.workunit.client.1.smithi176.stdout:1/859: dwrite d4/d8/d12/d21/d25/d30/d44/d4d/f91 [0,4194304] 0 2024-07-16T09:55:54.186 INFO:tasks.workunit.client.1.smithi176.stdout:1/860: write d4/db/f1b [982777,124296] 0 2024-07-16T09:55:54.189 INFO:tasks.workunit.client.1.smithi176.stdout:1/861: creat d4/d8/d12/d21/d25/da8/f123 x:0 0 0 2024-07-16T09:55:54.191 INFO:tasks.workunit.client.1.smithi176.stdout:1/862: creat d4/db/df3/f124 x:0 0 0 2024-07-16T09:55:54.193 INFO:tasks.workunit.client.1.smithi176.stdout:1/863: truncate d4/d3a/d8e/f105 660739 0 2024-07-16T09:55:54.198 INFO:tasks.workunit.client.1.smithi176.stdout:1/864: unlink d4/d8/lc 0 2024-07-16T09:55:54.199 INFO:tasks.workunit.client.1.smithi176.stdout:1/865: truncate d4/db/df3/d103/dad/d72/f74 1360291 0 2024-07-16T09:55:54.203 INFO:tasks.workunit.client.1.smithi176.stdout:1/866: unlink d4/d3a/d54/d5a/ldc 0 2024-07-16T09:55:54.204 INFO:tasks.workunit.client.1.smithi176.stdout:1/867: unlink d4/d8/d2c/ce7 0 2024-07-16T09:55:54.209 INFO:tasks.workunit.client.1.smithi176.stdout:1/868: dread d4/d8/d12/d21/d25/d30/d44/d4d/f96 [0,4194304] 0 2024-07-16T09:55:54.221 INFO:tasks.workunit.client.1.smithi176.stdout:1/869: dwrite d4/db/df3/d103/dad/d72/f74 [0,4194304] 0 2024-07-16T09:55:54.225 INFO:tasks.workunit.client.1.smithi176.stdout:1/870: rename d4/d8/d12/d21/d25/f90 to d4/d8/d12/d4c/d120/f125 0 2024-07-16T09:55:54.228 INFO:tasks.workunit.client.1.smithi176.stdout:1/871: mkdir d4/d8/d12/d21/d34/daa/dbb/dec/d126 0 2024-07-16T09:55:54.230 INFO:tasks.workunit.client.1.smithi176.stdout:1/872: creat d4/d3a/dc8/f127 x:0 0 0 2024-07-16T09:55:54.238 INFO:tasks.workunit.client.1.smithi176.stdout:1/873: dwrite d4/d8/d12/d21/d25/d30/d44/d10a/f112 [0,4194304] 0 2024-07-16T09:55:54.249 INFO:tasks.workunit.client.1.smithi176.stdout:1/874: dwrite d4/d8/d12/d21/d34/f58 [0,4194304] 0 2024-07-16T09:55:54.255 INFO:tasks.workunit.client.1.smithi176.stdout:1/875: dwrite d4/d8/d12/d4c/fa7 [0,4194304] 0 2024-07-16T09:55:54.258 INFO:tasks.workunit.client.1.smithi176.stdout:1/876: creat d4/de8/f128 x:0 0 0 2024-07-16T09:55:54.278 INFO:tasks.workunit.client.1.smithi176.stdout:3/956: sync 2024-07-16T09:55:54.331 INFO:tasks.workunit.client.1.smithi176.stdout:1/877: link d4/d3a/d54/d5a/f115 d4/d8/d12/d21/d34/daa/f129 0 2024-07-16T09:55:54.333 INFO:tasks.workunit.client.1.smithi176.stdout:3/957: mkdir d3/d17/d1d/d4f/daa/df3/d107/d119 0 2024-07-16T09:55:54.335 INFO:tasks.workunit.client.1.smithi176.stdout:1/878: dread d4/d8/d12/d21/ddd/fa0 [0,4194304] 0 2024-07-16T09:55:54.339 INFO:tasks.workunit.client.1.smithi176.stdout:3/958: truncate d3/d17/d1d/d4f/d59/f21 2211611 0 2024-07-16T09:55:54.349 INFO:tasks.workunit.client.1.smithi176.stdout:3/959: rmdir d3/da/d84 39 2024-07-16T09:55:54.353 INFO:tasks.workunit.client.1.smithi176.stdout:3/960: unlink d3/da/d6c/l7d 0 2024-07-16T09:55:54.358 INFO:tasks.workunit.client.1.smithi176.stdout:3/961: creat d3/da/d66/f11a x:0 0 0 2024-07-16T09:55:54.373 INFO:tasks.workunit.client.1.smithi176.stdout:3/962: getdents d3/d85/ddd 0 2024-07-16T09:55:54.374 INFO:tasks.workunit.client.1.smithi176.stdout:3/963: write d3/d17/d1d/d4f/fd4 [1499917,27217] 0 2024-07-16T09:55:54.401 INFO:tasks.workunit.client.1.smithi176.stdout:1/879: creat d4/d8/d12/d4c/f12a x:0 0 0 2024-07-16T09:55:54.402 INFO:tasks.workunit.client.1.smithi176.stdout:1/880: dread - d4/d3a/ff6 zero size 2024-07-16T09:55:54.407 INFO:tasks.workunit.client.1.smithi176.stdout:1/881: getdents d4/d8/d12/d21/d25 0 2024-07-16T09:55:54.408 INFO:tasks.workunit.client.1.smithi176.stdout:1/882: chown d4/d3a/d54/d5a/d7a/dd7/lf7 1125325 1 2024-07-16T09:55:54.409 INFO:tasks.workunit.client.1.smithi176.stdout:1/883: chown d4/d8/d12/d21/d34/daa/dbb/dec/d126 28477 1 2024-07-16T09:55:54.410 INFO:tasks.workunit.client.1.smithi176.stdout:1/884: stat d4/d8/d12/d21/d25/d30/d44/d4d/ff5 0 2024-07-16T09:55:54.411 INFO:tasks.workunit.client.1.smithi176.stdout:3/964: getdents d3/da/d66 0 2024-07-16T09:55:54.413 INFO:tasks.workunit.client.1.smithi176.stdout:3/965: write d3/da/d33/fc9 [104014,116074] 0 2024-07-16T09:55:54.417 INFO:tasks.workunit.client.1.smithi176.stdout:1/885: link d4/d8/d12/d21/la3 d4/d8/d12/d21/d25/d30/d44/d4d/l12b 0 2024-07-16T09:55:54.418 INFO:tasks.workunit.client.1.smithi176.stdout:3/966: link d3/d17/c6a d3/da/d33/dbd/c11b 0 2024-07-16T09:55:54.419 INFO:tasks.workunit.client.1.smithi176.stdout:1/886: mknod d4/d8/d12/d21/d34/daa/c12c 0 2024-07-16T09:55:54.422 INFO:tasks.workunit.client.1.smithi176.stdout:1/887: dread d4/db/df3/d103/dad/d72/fd5 [0,4194304] 0 2024-07-16T09:55:54.424 INFO:tasks.workunit.client.1.smithi176.stdout:1/888: dread - d4/d8/d12/d21/d25/da8/f123 zero size 2024-07-16T09:55:54.426 INFO:tasks.workunit.client.1.smithi176.stdout:1/889: creat d4/d8/d12/d21/d25/d30/d44/f12d x:0 0 0 2024-07-16T09:55:54.431 INFO:tasks.workunit.client.1.smithi176.stdout:3/967: dread d3/d17/d1d/d4f/d59/f3a [0,4194304] 0 2024-07-16T09:55:54.434 INFO:tasks.workunit.client.1.smithi176.stdout:3/968: write d3/da/d84/d10a/f92 [1557670,18830] 0 2024-07-16T09:55:54.435 INFO:tasks.workunit.client.1.smithi176.stdout:3/969: dread - d3/da/d7b/df8/f117 zero size 2024-07-16T09:55:54.438 INFO:tasks.workunit.client.1.smithi176.stdout:3/970: creat d3/d17/f11c x:0 0 0 2024-07-16T09:55:54.441 INFO:tasks.workunit.client.1.smithi176.stdout:1/890: dread d4/d8/d12/f4b [0,4194304] 0 2024-07-16T09:55:54.442 INFO:tasks.workunit.client.1.smithi176.stdout:3/971: rename d3/d17/d1d/d4f/d59/c50 to d3/d17/d1d/d4f/daa/df3/c11d 0 2024-07-16T09:55:54.443 INFO:tasks.workunit.client.1.smithi176.stdout:3/972: write d3/da/f77 [4882815,94746] 0 2024-07-16T09:55:54.451 INFO:tasks.workunit.client.1.smithi176.stdout:3/973: dwrite d3/da/d33/f78 [4194304,4194304] 0 2024-07-16T09:55:54.455 INFO:tasks.workunit.client.1.smithi176.stdout:3/974: mknod d3/d17/d1d/d4f/c11e 0 2024-07-16T09:55:54.458 INFO:tasks.workunit.client.1.smithi176.stdout:3/975: mknod d3/dd1/c11f 0 2024-07-16T09:55:54.458 INFO:tasks.workunit.client.1.smithi176.stdout:3/976: readlink l0 0 2024-07-16T09:55:54.459 INFO:tasks.workunit.client.1.smithi176.stdout:1/891: dwrite d4/d8/d12/d21/d25/d30/d44/d4d/f96 [0,4194304] 0 2024-07-16T09:55:54.466 INFO:tasks.workunit.client.1.smithi176.stdout:1/892: dread d4/f24 [0,4194304] 0 2024-07-16T09:55:54.468 INFO:tasks.workunit.client.1.smithi176.stdout:1/893: write d4/db/d100/f106 [1103706,23582] 0 2024-07-16T09:55:54.471 INFO:tasks.workunit.client.1.smithi176.stdout:3/977: truncate d3/da/d33/f53 433272 0 2024-07-16T09:55:54.475 INFO:tasks.workunit.client.1.smithi176.stdout:3/978: creat d3/da/d66/f120 x:0 0 0 2024-07-16T09:55:54.475 INFO:tasks.workunit.client.1.smithi176.stdout:1/894: getdents d4/db/df3 0 2024-07-16T09:55:54.477 INFO:tasks.workunit.client.1.smithi176.stdout:1/895: chown d4/d8/d12/d21/d25/d30/d44/d10a/l117 23143 1 2024-07-16T09:55:54.478 INFO:tasks.workunit.client.1.smithi176.stdout:3/979: unlink d3/d17/d1d/c5e 0 2024-07-16T09:55:54.481 INFO:tasks.workunit.client.1.smithi176.stdout:3/980: mknod d3/d17/d1d/c121 0 2024-07-16T09:55:54.484 INFO:tasks.workunit.client.1.smithi176.stdout:1/896: dwrite d4/d3a/d54/f6d [4194304,4194304] 0 2024-07-16T09:55:54.486 INFO:tasks.workunit.client.1.smithi176.stdout:1/897: symlink d4/d8/l12e 0 2024-07-16T09:55:54.488 INFO:tasks.workunit.client.1.smithi176.stdout:1/898: write d4/d8/d12/d21/d25/f10b [306719,49054] 0 2024-07-16T09:55:54.490 INFO:tasks.workunit.client.1.smithi176.stdout:3/981: dread d3/d17/d1d/d4f/d59/f21 [0,4194304] 0 2024-07-16T09:55:54.491 INFO:tasks.workunit.client.1.smithi176.stdout:1/899: creat d4/d8/d12/d21/d34/daa/dbb/def/f12f x:0 0 0 2024-07-16T09:55:54.499 INFO:tasks.workunit.client.1.smithi176.stdout:3/982: dwrite d3/d17/f68 [0,4194304] 0 2024-07-16T09:55:54.500 INFO:tasks.workunit.client.1.smithi176.stdout:1/900: dwrite d4/d3a/dc8/f127 [0,4194304] 0 2024-07-16T09:55:54.504 INFO:tasks.workunit.client.1.smithi176.stdout:1/901: rename d4/d3a/dc8/cf0 to d4/d8/d12/d4c/da4/da9/c130 0 2024-07-16T09:55:54.507 INFO:tasks.workunit.client.1.smithi176.stdout:3/983: link d3/d17/d1d/d6e/fd5 d3/d17/d10b/f122 0 2024-07-16T09:55:54.509 INFO:tasks.workunit.client.1.smithi176.stdout:3/984: readlink d3/d17/d1d/d4f/d59/da0/l48 0 2024-07-16T09:55:54.511 INFO:tasks.workunit.client.1.smithi176.stdout:1/902: rename d4/d3a/l70 to d4/d8/d12/d4c/da4/da9/l131 0 2024-07-16T09:55:54.511 INFO:tasks.workunit.client.1.smithi176.stdout:3/985: fsync d3/da/d7b/fb8 0 2024-07-16T09:55:54.515 INFO:tasks.workunit.client.1.smithi176.stdout:1/903: creat d4/f132 x:0 0 0 2024-07-16T09:55:54.523 INFO:tasks.workunit.client.1.smithi176.stdout:1/904: dwrite d4/db/f43 [0,4194304] 0 2024-07-16T09:55:54.524 INFO:tasks.workunit.client.1.smithi176.stdout:1/905: stat d4/db/df3 0 2024-07-16T09:55:54.526 INFO:tasks.workunit.client.1.smithi176.stdout:1/906: stat d4/db/df3/d103/dad 0 2024-07-16T09:55:54.528 INFO:tasks.workunit.client.1.smithi176.stdout:3/986: dwrite d3/d17/d1d/d4f/fa3 [0,4194304] 0 2024-07-16T09:55:54.531 INFO:tasks.workunit.client.1.smithi176.stdout:3/987: write d3/da/d7b/fb8 [3480162,852] 0 2024-07-16T09:55:54.534 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:55:54 smithi156 ceph-mon[29302]: [16/Jul/2024:09:55:52] ENGINE Serving on https://172.21.15.176:7150 2024-07-16T09:55:54.534 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:55:54 smithi156 ceph-mon[29302]: [16/Jul/2024:09:55:52] ENGINE Bus STARTED 2024-07-16T09:55:54.534 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:55:54 smithi156 ceph-mon[29302]: [16/Jul/2024:09:55:52] ENGINE Error in HTTPServer.serve 2024-07-16T09:55:54.535 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: Traceback (most recent call last): 2024-07-16T09:55:54.535 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-07-16T09:55:54.535 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: self._connections.run(self.expiration_interval) 2024-07-16T09:55:54.535 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-07-16T09:55:54.535 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: self._run(expiration_interval) 2024-07-16T09:55:54.535 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-07-16T09:55:54.535 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-07-16T09:55:54.535 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-07-16T09:55:54.535 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-07-16T09:55:54.535 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-07-16T09:55:54.535 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: s = self.context.wrap_socket( 2024-07-16T09:55:54.535 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-07-16T09:55:54.536 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: return self.sslsocket_class._create( 2024-07-16T09:55:54.536 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-07-16T09:55:54.536 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: self.do_handshake() 2024-07-16T09:55:54.536 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-07-16T09:55:54.536 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: self._sslobj.do_handshake() 2024-07-16T09:55:54.536 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-07-16T09:55:54.536 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:55:54 smithi156 ceph-mon[29302]: pgmap v5: 65 pgs: 65 active+clean; 3.8 GiB data, 13 GiB used, 524 GiB / 536 GiB avail 2024-07-16T09:55:54.536 INFO:tasks.workunit.client.1.smithi176.stdout:3/988: mkdir d3/da/d6c/d123 0 2024-07-16T09:55:54.537 INFO:tasks.workunit.client.1.smithi176.stdout:3/989: read - d3/d17/d1d/d4f/d6d/fe4 zero size 2024-07-16T09:55:54.537 INFO:tasks.workunit.client.1.smithi176.stdout:1/907: link d4/d8/d12/d21/d25/d30/cb0 d4/c133 0 2024-07-16T09:55:54.538 INFO:tasks.workunit.client.1.smithi176.stdout:3/990: chown d3/d17/d10b/cb5 238142077 1 2024-07-16T09:55:54.539 INFO:tasks.workunit.client.1.smithi176.stdout:1/908: symlink d4/d8/d12/d4c/da4/da9/l134 0 2024-07-16T09:55:54.541 INFO:tasks.workunit.client.1.smithi176.stdout:3/991: symlink d3/d17/d10b/l124 0 2024-07-16T09:55:54.542 INFO:tasks.workunit.client.1.smithi176.stdout:3/992: write d3/d17/d1d/d4f/d59/da0/f3f [2005979,130227] 0 2024-07-16T09:55:54.544 INFO:tasks.workunit.client.1.smithi176.stdout:3/993: write d3/f4a [2303672,35426] 0 2024-07-16T09:55:54.545 INFO:tasks.workunit.client.1.smithi176.stdout:3/994: write d3/da/f77 [4571173,1012] 0 2024-07-16T09:55:54.547 INFO:tasks.workunit.client.1.smithi176.stdout:3/995: mknod d3/da/dca/c125 0 2024-07-16T09:55:54.548 INFO:tasks.workunit.client.1.smithi176.stdout:3/996: rmdir d3/da/d7b 39 2024-07-16T09:55:54.550 INFO:tasks.workunit.client.1.smithi176.stdout:3/997: creat d3/da/d6c/dbc/f126 x:0 0 0 2024-07-16T09:55:54.553 INFO:tasks.workunit.client.1.smithi176.stdout:3/998: getdents d3/d17/d1d/d6e 0 2024-07-16T09:55:54.555 INFO:tasks.workunit.client.1.smithi176.stdout:3/999: dread - d3/da/d7b/df8/f111 zero size 2024-07-16T09:55:54.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:55:54 smithi176 ceph-mon[36626]: [16/Jul/2024:09:55:52] ENGINE Serving on https://172.21.15.176:7150 2024-07-16T09:55:54.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:55:54 smithi176 ceph-mon[36626]: [16/Jul/2024:09:55:52] ENGINE Bus STARTED 2024-07-16T09:55:54.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:55:54 smithi176 ceph-mon[36626]: [16/Jul/2024:09:55:52] ENGINE Error in HTTPServer.serve 2024-07-16T09:55:54.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: Traceback (most recent call last): 2024-07-16T09:55:54.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-07-16T09:55:54.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: self._connections.run(self.expiration_interval) 2024-07-16T09:55:54.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-07-16T09:55:54.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: self._run(expiration_interval) 2024-07-16T09:55:54.845 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-07-16T09:55:54.845 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-07-16T09:55:54.845 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-07-16T09:55:54.845 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-07-16T09:55:54.845 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-07-16T09:55:54.845 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: s = self.context.wrap_socket( 2024-07-16T09:55:54.845 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-07-16T09:55:54.845 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: return self.sslsocket_class._create( 2024-07-16T09:55:54.845 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-07-16T09:55:54.845 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: self.do_handshake() 2024-07-16T09:55:54.845 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-07-16T09:55:54.845 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: self._sslobj.do_handshake() 2024-07-16T09:55:54.845 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-07-16T09:55:54.846 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:55:54 smithi176 ceph-mon[36626]: pgmap v5: 65 pgs: 65 active+clean; 3.8 GiB data, 13 GiB used, 524 GiB / 536 GiB avail 2024-07-16T09:55:54.965 INFO:tasks.workunit.client.1.smithi176.stdout:1/909: sync 2024-07-16T09:55:54.969 INFO:tasks.workunit.client.1.smithi176.stdout:1/910: rename d4/d8/d12/d21/d25/d30/f61 to d4/db/df3/d103/dad/d72/f135 0 2024-07-16T09:55:54.985 INFO:tasks.workunit.client.1.smithi176.stdout:1/911: getdents d4/db/df3/d103/dad/d72 0 2024-07-16T09:55:54.990 INFO:tasks.workunit.client.1.smithi176.stdout:1/912: truncate d4/d8/d12/d21/d25/da8/f97 57966 0 2024-07-16T09:55:54.994 INFO:tasks.workunit.client.1.smithi176.stdout:1/913: link d4/d8/d12/d21/d34/daa/cc7 d4/d8/d12/d21/d34/daa/dbb/dec/d126/c136 0 2024-07-16T09:55:54.994 INFO:tasks.workunit.client.1.smithi176.stdout:1/914: readlink d4/d8/d12/d21/d25/lba 0 2024-07-16T09:55:54.997 INFO:tasks.workunit.client.1.smithi176.stdout:1/915: rename d4/c133 to d4/d3a/dc8/c137 0 2024-07-16T09:55:55.003 INFO:tasks.workunit.client.1.smithi176.stdout:1/916: write d4/f17 [1104894,119954] 0 2024-07-16T09:55:55.006 INFO:tasks.workunit.client.1.smithi176.stdout:1/917: getdents d4/d3a/d66 0 2024-07-16T09:55:55.008 INFO:tasks.workunit.client.1.smithi176.stdout:1/918: truncate d4/d8/d12/d21/fb9 324123 0 2024-07-16T09:55:55.010 INFO:tasks.workunit.client.1.smithi176.stdout:1/919: write d4/d8/d12/d21/d25/d30/fbe [997290,84712] 0 2024-07-16T09:55:55.017 INFO:tasks.workunit.client.1.smithi176.stdout:1/920: dwrite d4/d8/d12/d21/d34/daa/fab [0,4194304] 0 2024-07-16T09:55:55.019 INFO:tasks.workunit.client.1.smithi176.stdout:1/921: symlink d4/l138 0 2024-07-16T09:55:55.021 INFO:tasks.workunit.client.1.smithi176.stdout:1/922: creat d4/de8/f139 x:0 0 0 2024-07-16T09:55:55.023 INFO:tasks.workunit.client.1.smithi176.stdout:1/923: creat d4/db/df3/d103/dad/d72/f13a x:0 0 0 2024-07-16T09:55:55.026 INFO:tasks.workunit.client.1.smithi176.stdout:1/924: mkdir d4/db/df3/d103/dad/d13b 0 2024-07-16T09:55:55.029 INFO:tasks.workunit.client.1.smithi176.stdout:1/925: mkdir d4/d8/d12/d21/d34/d13c 0 2024-07-16T09:55:55.032 INFO:tasks.workunit.client.1.smithi176.stdout:1/926: link d4/db/f16 d4/d3a/dc8/f13d 0 2024-07-16T09:55:55.038 INFO:tasks.workunit.client.1.smithi176.stdout:1/927: dwrite d4/d8/d12/d21/d25/d30/fbe [0,4194304] 0 2024-07-16T09:55:55.042 INFO:tasks.workunit.client.1.smithi176.stdout:1/928: creat d4/d8/d12/f13e x:0 0 0 2024-07-16T09:55:55.075 INFO:tasks.workunit.client.1.smithi176.stdout:1/929: truncate d4/d8/d12/d4c/da4/fbc 47041 0 2024-07-16T09:55:55.075 INFO:tasks.workunit.client.1.smithi176.stdout:1/930: creat d4/d3a/d54/f13f x:0 0 0 2024-07-16T09:55:55.075 INFO:tasks.workunit.client.1.smithi176.stdout:1/931: dwrite d4/d8/d12/d21/f89 [0,4194304] 0 2024-07-16T09:55:55.075 INFO:tasks.workunit.client.1.smithi176.stdout:1/932: dread d4/f24 [0,4194304] 0 2024-07-16T09:55:55.075 INFO:tasks.workunit.client.1.smithi176.stdout:1/933: rmdir d4/d8/d12/d21/d34/daa 39 2024-07-16T09:55:55.075 INFO:tasks.workunit.client.1.smithi176.stdout:1/934: dread d4/d3a/dc8/fd0 [0,4194304] 0 2024-07-16T09:55:55.075 INFO:tasks.workunit.client.1.smithi176.stdout:1/935: write d4/d8/d12/d21/d25/d30/d44/f12d [549631,46707] 0 2024-07-16T09:55:55.076 INFO:tasks.workunit.client.1.smithi176.stdout:1/936: creat d4/db/df3/d103/dad/f140 x:0 0 0 2024-07-16T09:55:55.078 INFO:tasks.workunit.client.1.smithi176.stdout:1/937: dwrite d4/d8/d12/d21/d25/f60 [4194304,4194304] 0 2024-07-16T09:55:55.083 INFO:tasks.workunit.client.1.smithi176.stdout:1/938: creat d4/db/df3/d103/dad/f141 x:0 0 0 2024-07-16T09:55:55.086 INFO:tasks.workunit.client.1.smithi176.stdout:1/939: truncate d4/d8/d12/d21/d34/daa/fe5 723903 0 2024-07-16T09:55:55.090 INFO:tasks.workunit.client.1.smithi176.stdout:1/940: dread d4/db/df3/d103/dad/d72/f74 [0,4194304] 0 2024-07-16T09:55:55.091 INFO:tasks.workunit.client.1.smithi176.stdout:1/941: fsync d4/d8/d12/d21/d25/d30/d44/d4d/f96 0 2024-07-16T09:55:55.093 INFO:tasks.workunit.client.1.smithi176.stdout:1/942: mkdir d4/d8/d2c/d142 0 2024-07-16T09:55:55.097 INFO:tasks.workunit.client.1.smithi176.stdout:1/943: unlink d4/d8/d12/fea 0 2024-07-16T09:55:55.528 INFO:tasks.workunit.client.1.smithi176.stdout:1/944: sync 2024-07-16T09:55:55.531 INFO:tasks.workunit.client.1.smithi176.stdout:1/945: mknod d4/db/df3/d103/dad/d72/c143 0 2024-07-16T09:55:55.534 INFO:tasks.workunit.client.1.smithi176.stdout:1/946: rename d4/d3a/d54/d5a/f115 to d4/d8/d12/d21/ddd/f144 0 2024-07-16T09:55:55.540 INFO:tasks.workunit.client.1.smithi176.stdout:1/947: link d4/d8/d12/d21/d25/da8/f97 d4/d8/d12/d21/f145 0 2024-07-16T09:55:55.548 INFO:tasks.workunit.client.1.smithi176.stdout:1/948: truncate d4/f24 3568714 0 2024-07-16T09:55:55.551 INFO:tasks.workunit.client.1.smithi176.stdout:1/949: dread d4/d8/d12/d21/d25/da8/fd4 [0,4194304] 0 2024-07-16T09:55:55.553 INFO:tasks.workunit.client.1.smithi176.stdout:1/950: creat d4/db/df3/f146 x:0 0 0 2024-07-16T09:55:55.560 INFO:tasks.workunit.client.1.smithi176.stdout:1/951: dwrite d4/d8/d12/d21/d25/d30/d44/f5f [4194304,4194304] 0 2024-07-16T09:55:55.563 INFO:tasks.workunit.client.1.smithi176.stdout:1/952: creat d4/d8/d12/d21/d34/daa/dbb/def/f147 x:0 0 0 2024-07-16T09:55:55.564 INFO:tasks.workunit.client.1.smithi176.stdout:1/953: dread - d4/de8/f139 zero size 2024-07-16T09:55:55.567 INFO:tasks.workunit.client.1.smithi176.stdout:1/954: creat d4/d8/d12/d21/d34/daa/dbb/f148 x:0 0 0 2024-07-16T09:55:55.568 INFO:tasks.workunit.client.1.smithi176.stdout:1/955: mkdir d4/db/df3/d149 0 2024-07-16T09:55:55.570 INFO:tasks.workunit.client.1.smithi176.stdout:1/956: rmdir d4/d8/d2c 39 2024-07-16T09:55:55.573 INFO:tasks.workunit.client.1.smithi176.stdout:1/957: getdents d4/d8/d12/d21/d34/daa 0 2024-07-16T09:55:55.575 INFO:tasks.workunit.client.1.smithi176.stdout:1/958: creat d4/d3a/f14a x:0 0 0 2024-07-16T09:55:55.579 INFO:tasks.workunit.client.1.smithi176.stdout:1/959: rename d4/db/df3/l116 to d4/db/df3/d103/dad/d13b/l14b 0 2024-07-16T09:55:55.580 INFO:tasks.workunit.client.1.smithi176.stdout:1/960: chown d4/d8/d12/d21/d25/da8 385585 1 2024-07-16T09:55:55.581 INFO:tasks.workunit.client.1.smithi176.stdout:1/961: chown d4/db/df3/d103/dad/d72/f135 16573624 1 2024-07-16T09:55:55.582 INFO:tasks.workunit.client.1.smithi176.stdout:1/962: creat d4/db/df3/d103/dad/d72/f14c x:0 0 0 2024-07-16T09:55:55.591 INFO:tasks.workunit.client.1.smithi176.stdout:1/963: dwrite d4/db/f5c [4194304,4194304] 0 2024-07-16T09:55:55.597 INFO:tasks.workunit.client.1.smithi176.stdout:1/964: dread d4/d8/d12/f4b [0,4194304] 0 2024-07-16T09:55:55.605 INFO:tasks.workunit.client.1.smithi176.stdout:1/965: dwrite d4/d8/d12/d21/d34/daa/dbb/def/f12f [0,4194304] 0 2024-07-16T09:55:55.613 INFO:tasks.workunit.client.1.smithi176.stdout:1/966: dwrite d4/d8/d12/d21/d25/d30/d44/d10a/f112 [0,4194304] 0 2024-07-16T09:55:55.636 INFO:tasks.workunit.client.1.smithi176.stdout:1/967: creat d4/d3a/d66/f14d x:0 0 0 2024-07-16T09:55:55.636 INFO:tasks.workunit.client.1.smithi176.stdout:1/968: stat d4/d3a/f14a 0 2024-07-16T09:55:55.652 INFO:tasks.workunit.client.1.smithi176.stdout:1/969: write d4/d8/d12/d21/d25/da8/fd1 [698642,66815] 0 2024-07-16T09:55:55.654 INFO:tasks.workunit.client.1.smithi176.stdout:1/970: read d4/d8/d12/d21/d25/d30/fbe [34664,72987] 0 2024-07-16T09:55:55.660 INFO:tasks.workunit.client.1.smithi176.stdout:1/971: dwrite d4/d8/d12/d21/d25/da8/f123 [0,4194304] 0 2024-07-16T09:55:55.663 INFO:tasks.workunit.client.1.smithi176.stdout:1/972: mknod d4/d3a/d54/d5a/d7a/c14e 0 2024-07-16T09:55:55.670 INFO:tasks.workunit.client.1.smithi176.stdout:1/973: link d4/d8/d12/d4c/l7b d4/l14f 0 2024-07-16T09:55:55.672 INFO:tasks.workunit.client.1.smithi176.stdout:1/974: fdatasync d4/d8/d12/d21/ddd/fa0 0 2024-07-16T09:55:55.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:55:55 smithi156 ceph-mon[29302]: mgrmap e24: smithi176.dqbrvx(active, since 5s) 2024-07-16T09:55:55.675 INFO:tasks.workunit.client.1.smithi176.stdout:1/975: getdents d4/d8/d12/d4c/d120 0 2024-07-16T09:55:55.676 INFO:tasks.workunit.client.1.smithi176.stdout:1/976: chown d4/d8/d12/d21/d25/d30/d44/d4d/f11c 246 1 2024-07-16T09:55:55.678 INFO:tasks.workunit.client.1.smithi176.stdout:1/977: write d4/d3a/fdb [609845,23536] 0 2024-07-16T09:55:55.682 INFO:tasks.workunit.client.1.smithi176.stdout:1/978: write d4/db/f49 [3888992,51629] 0 2024-07-16T09:55:55.684 INFO:tasks.workunit.client.1.smithi176.stdout:1/979: mknod d4/db/df3/d103/dad/d13b/c150 0 2024-07-16T09:55:55.686 INFO:tasks.workunit.client.1.smithi176.stdout:1/980: mknod d4/d8/d2c/d142/c151 0 2024-07-16T09:55:55.688 INFO:tasks.workunit.client.1.smithi176.stdout:1/981: mkdir d4/d8/d12/d4c/d120/d152 0 2024-07-16T09:55:55.688 INFO:tasks.workunit.client.1.smithi176.stdout:1/982: fdatasync d4/de8/f139 0 2024-07-16T09:55:55.749 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:55:55 smithi176 ceph-mon[36626]: mgrmap e24: smithi176.dqbrvx(active, since 5s) 2024-07-16T09:55:56.194 INFO:tasks.workunit.client.1.smithi176.stdout:1/983: sync 2024-07-16T09:55:56.195 INFO:tasks.workunit.client.1.smithi176.stdout:1/984: creat d4/de8/f153 x:0 0 0 2024-07-16T09:55:56.197 INFO:tasks.workunit.client.1.smithi176.stdout:1/985: mknod d4/d8/d12/d21/ddd/c154 0 2024-07-16T09:55:56.199 INFO:tasks.workunit.client.1.smithi176.stdout:1/986: fdatasync d4/d8/fb5 0 2024-07-16T09:55:56.205 INFO:tasks.workunit.client.1.smithi176.stdout:1/987: dwrite d4/d8/d12/d21/d25/f86 [0,4194304] 0 2024-07-16T09:55:56.209 INFO:tasks.workunit.client.1.smithi176.stdout:1/988: truncate d4/d3a/d54/f6d 10232959 0 2024-07-16T09:55:56.210 INFO:tasks.workunit.client.1.smithi176.stdout:1/989: chown d4/db/f1b 20505223 1 2024-07-16T09:55:56.212 INFO:tasks.workunit.client.1.smithi176.stdout:1/990: rmdir d4/d8/d12/d21/d25/d30/d44/d10a 39 2024-07-16T09:55:56.213 INFO:tasks.workunit.client.1.smithi176.stdout:1/991: fdatasync d4/db/df3/d103/dad/f140 0 2024-07-16T09:55:56.215 INFO:tasks.workunit.client.1.smithi176.stdout:1/992: creat d4/d3a/d54/d5a/d7a/dd7/f155 x:0 0 0 2024-07-16T09:55:56.216 INFO:tasks.workunit.client.1.smithi176.stdout:1/993: stat d4/d8/d12/d21/f89 0 2024-07-16T09:55:56.222 INFO:tasks.workunit.client.1.smithi176.stdout:1/994: dwrite d4/d8/fc2 [0,4194304] 0 2024-07-16T09:55:56.224 INFO:tasks.workunit.client.1.smithi176.stdout:1/995: write d4/d8/d12/d21/d34/daa/fcd [940318,55805] 0 2024-07-16T09:55:56.227 INFO:tasks.workunit.client.1.smithi176.stdout:1/996: truncate d4/d8/d12/d21/d34/daa/f129 212277 0 2024-07-16T09:55:56.230 INFO:tasks.workunit.client.1.smithi176.stdout:1/997: creat d4/d8/d12/d21/d25/d30/d44/dd2/f156 x:0 0 0 2024-07-16T09:55:56.231 INFO:tasks.workunit.client.1.smithi176.stdout:1/998: chown d4/d8/d12/d21/d25/f55 225830901 1 2024-07-16T09:55:56.231 INFO:tasks.workunit.client.1.smithi176.stdout:1/999: stat d4/db/f122 0 2024-07-16T09:55:56.235 INFO:tasks.workunit.client.1.smithi176.stderr:+ rm -rf -- ./tmp.IB6AWFslDW 2024-07-16T09:55:56.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:55:56 smithi156 ceph-mon[29302]: pgmap v6: 65 pgs: 65 active+clean; 4.0 GiB data, 13 GiB used, 523 GiB / 536 GiB avail; 40 MiB/s rd, 101 MiB/s wr, 351 op/s 2024-07-16T09:55:56.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:55:56 smithi156 ceph-mon[29302]: Standby manager daemon smithi156.zkvylv started 2024-07-16T09:55:56.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:55:56 smithi156 ceph-mon[29302]: from='mgr.? 172.21.15.156:0/1453605671' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi156.zkvylv/crt"}]: dispatch 2024-07-16T09:55:56.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:55:56 smithi156 ceph-mon[29302]: from='mgr.? 172.21.15.156:0/1453605671' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-07-16T09:55:56.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:55:56 smithi156 ceph-mon[29302]: from='mgr.? 172.21.15.156:0/1453605671' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi156.zkvylv/key"}]: dispatch 2024-07-16T09:55:56.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:55:56 smithi156 ceph-mon[29302]: from='mgr.? 172.21.15.156:0/1453605671' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-07-16T09:55:56.843 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:55:56 smithi176 ceph-mon[36626]: pgmap v6: 65 pgs: 65 active+clean; 4.0 GiB data, 13 GiB used, 523 GiB / 536 GiB avail; 40 MiB/s rd, 101 MiB/s wr, 351 op/s 2024-07-16T09:55:56.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:55:56 smithi176 ceph-mon[36626]: Standby manager daemon smithi156.zkvylv started 2024-07-16T09:55:56.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:55:56 smithi176 ceph-mon[36626]: from='mgr.? 172.21.15.156:0/1453605671' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi156.zkvylv/crt"}]: dispatch 2024-07-16T09:55:56.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:55:56 smithi176 ceph-mon[36626]: from='mgr.? 172.21.15.156:0/1453605671' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-07-16T09:55:56.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:55:56 smithi176 ceph-mon[36626]: from='mgr.? 172.21.15.156:0/1453605671' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi156.zkvylv/key"}]: dispatch 2024-07-16T09:55:56.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:55:56 smithi176 ceph-mon[36626]: from='mgr.? 172.21.15.156:0/1453605671' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-07-16T09:55:57.541 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:55:57 smithi156 ceph-mon[29302]: mgrmap e25: smithi176.dqbrvx(active, since 7s), standbys: smithi156.zkvylv 2024-07-16T09:55:57.541 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:55:57 smithi156 ceph-mon[29302]: from='mgr.24377 172.21.15.176:0/1082993331' entity='mgr.smithi176.dqbrvx' cmd=[{"prefix": "mgr metadata", "who": "smithi156.zkvylv", "id": "smithi156.zkvylv"}]: dispatch 2024-07-16T09:55:57.843 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:55:57 smithi176 ceph-mon[36626]: mgrmap e25: smithi176.dqbrvx(active, since 7s), standbys: smithi156.zkvylv 2024-07-16T09:55:57.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:55:57 smithi176 ceph-mon[36626]: from='mgr.24377 172.21.15.176:0/1082993331' entity='mgr.smithi176.dqbrvx' cmd=[{"prefix": "mgr metadata", "who": "smithi156.zkvylv", "id": "smithi156.zkvylv"}]: dispatch 2024-07-16T09:55:58.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:55:58 smithi156 ceph-mon[29302]: pgmap v7: 65 pgs: 65 active+clean; 4.0 GiB data, 13 GiB used, 523 GiB / 536 GiB avail; 45 MiB/s rd, 121 MiB/s wr, 325 op/s 2024-07-16T09:55:58.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:55:58 smithi156 ceph-mon[29302]: from='mgr.24377 ' entity='mgr.smithi176.dqbrvx' 2024-07-16T09:55:58.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:55:58 smithi156 ceph-mon[29302]: from='mgr.24377 ' entity='mgr.smithi176.dqbrvx' 2024-07-16T09:55:58.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:55:58 smithi156 ceph-mon[29302]: from='mgr.24377 ' entity='mgr.smithi176.dqbrvx' 2024-07-16T09:55:58.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:55:58 smithi156 ceph-mon[29302]: from='mgr.24377 ' entity='mgr.smithi176.dqbrvx' 2024-07-16T09:55:58.843 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:55:58 smithi176 ceph-mon[36626]: pgmap v7: 65 pgs: 65 active+clean; 4.0 GiB data, 13 GiB used, 523 GiB / 536 GiB avail; 45 MiB/s rd, 121 MiB/s wr, 325 op/s 2024-07-16T09:55:58.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:55:58 smithi176 ceph-mon[36626]: from='mgr.24377 ' entity='mgr.smithi176.dqbrvx' 2024-07-16T09:55:58.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:55:58 smithi176 ceph-mon[36626]: from='mgr.24377 ' entity='mgr.smithi176.dqbrvx' 2024-07-16T09:55:58.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:55:58 smithi176 ceph-mon[36626]: from='mgr.24377 ' entity='mgr.smithi176.dqbrvx' 2024-07-16T09:55:58.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:55:58 smithi176 ceph-mon[36626]: from='mgr.24377 ' entity='mgr.smithi176.dqbrvx' 2024-07-16T09:56:00.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:55:59 smithi176 ceph-mon[36626]: from='mgr.24377 ' entity='mgr.smithi176.dqbrvx' 2024-07-16T09:56:00.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:55:59 smithi176 ceph-mon[36626]: from='mgr.24377 ' entity='mgr.smithi176.dqbrvx' 2024-07-16T09:56:00.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:55:59 smithi176 ceph-mon[36626]: from='mgr.24377 ' entity='mgr.smithi176.dqbrvx' 2024-07-16T09:56:00.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:55:59 smithi176 ceph-mon[36626]: from='mgr.24377 ' entity='mgr.smithi176.dqbrvx' 2024-07-16T09:56:00.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:55:59 smithi176 ceph-mon[36626]: from='mgr.24377 172.21.15.176:0/1082993331' entity='mgr.smithi176.dqbrvx' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-07-16T09:56:00.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:55:59 smithi176 ceph-mon[36626]: from='mgr.24377 ' entity='mgr.smithi176.dqbrvx' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-07-16T09:56:00.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:55:59 smithi176 ceph-mon[36626]: from='mgr.24377 172.21.15.176:0/1082993331' entity='mgr.smithi176.dqbrvx' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-07-16T09:56:00.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:55:59 smithi176 ceph-mon[36626]: from='mgr.24377 ' entity='mgr.smithi176.dqbrvx' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-07-16T09:56:00.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:55:59 smithi176 ceph-mon[36626]: from='mgr.24377 172.21.15.176:0/1082993331' entity='mgr.smithi176.dqbrvx' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-07-16T09:56:00.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:55:59 smithi176 ceph-mon[36626]: from='mgr.24377 ' entity='mgr.smithi176.dqbrvx' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-07-16T09:56:00.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:55:59 smithi176 ceph-mon[36626]: Adjusting osd_memory_target on smithi156 to 1557M 2024-07-16T09:56:00.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:55:59 smithi176 ceph-mon[36626]: from='mgr.24377 ' entity='mgr.smithi176.dqbrvx' 2024-07-16T09:56:00.095 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:55:59 smithi176 ceph-mon[36626]: pgmap v8: 65 pgs: 65 active+clean; 4.0 GiB data, 13 GiB used, 523 GiB / 536 GiB avail; 35 MiB/s rd, 94 MiB/s wr, 253 op/s 2024-07-16T09:56:00.174 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:55:59 smithi156 ceph-mon[29302]: from='mgr.24377 ' entity='mgr.smithi176.dqbrvx' 2024-07-16T09:56:00.175 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:55:59 smithi156 ceph-mon[29302]: from='mgr.24377 ' entity='mgr.smithi176.dqbrvx' 2024-07-16T09:56:00.175 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:55:59 smithi156 ceph-mon[29302]: from='mgr.24377 ' entity='mgr.smithi176.dqbrvx' 2024-07-16T09:56:00.175 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:55:59 smithi156 ceph-mon[29302]: from='mgr.24377 ' entity='mgr.smithi176.dqbrvx' 2024-07-16T09:56:00.175 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:55:59 smithi156 ceph-mon[29302]: from='mgr.24377 172.21.15.176:0/1082993331' entity='mgr.smithi176.dqbrvx' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-07-16T09:56:00.175 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:55:59 smithi156 ceph-mon[29302]: from='mgr.24377 ' entity='mgr.smithi176.dqbrvx' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-07-16T09:56:00.175 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:55:59 smithi156 ceph-mon[29302]: from='mgr.24377 172.21.15.176:0/1082993331' entity='mgr.smithi176.dqbrvx' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-07-16T09:56:00.175 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:55:59 smithi156 ceph-mon[29302]: from='mgr.24377 ' entity='mgr.smithi176.dqbrvx' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-07-16T09:56:00.175 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:55:59 smithi156 ceph-mon[29302]: from='mgr.24377 172.21.15.176:0/1082993331' entity='mgr.smithi176.dqbrvx' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-07-16T09:56:00.175 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:55:59 smithi156 ceph-mon[29302]: from='mgr.24377 ' entity='mgr.smithi176.dqbrvx' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-07-16T09:56:00.175 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:55:59 smithi156 ceph-mon[29302]: Adjusting osd_memory_target on smithi156 to 1557M 2024-07-16T09:56:00.175 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:55:59 smithi156 ceph-mon[29302]: from='mgr.24377 ' entity='mgr.smithi176.dqbrvx' 2024-07-16T09:56:00.175 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:55:59 smithi156 ceph-mon[29302]: pgmap v8: 65 pgs: 65 active+clean; 4.0 GiB data, 13 GiB used, 523 GiB / 536 GiB avail; 35 MiB/s rd, 94 MiB/s wr, 253 op/s 2024-07-16T09:56:02.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:56:02 smithi176 ceph-mon[36626]: pgmap v9: 65 pgs: 65 active+clean; 3.2 GiB data, 11 GiB used, 526 GiB / 536 GiB avail; 30 MiB/s rd, 81 MiB/s wr, 320 op/s 2024-07-16T09:56:02.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:56:02 smithi156 ceph-mon[29302]: pgmap v9: 65 pgs: 65 active+clean; 3.2 GiB data, 11 GiB used, 526 GiB / 536 GiB avail; 30 MiB/s rd, 81 MiB/s wr, 320 op/s 2024-07-16T09:56:04.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:56:04 smithi156 ceph-mon[29302]: pgmap v10: 65 pgs: 65 active+clean; 3.2 GiB data, 11 GiB used, 526 GiB / 536 GiB avail; 27 MiB/s rd, 74 MiB/s wr, 293 op/s 2024-07-16T09:56:04.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:56:04 smithi156 ceph-mon[29302]: from='mgr.24377 ' entity='mgr.smithi176.dqbrvx' 2024-07-16T09:56:04.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:56:04 smithi156 ceph-mon[29302]: from='mgr.24377 ' entity='mgr.smithi176.dqbrvx' 2024-07-16T09:56:04.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:56:04 smithi156 ceph-mon[29302]: from='mgr.24377 172.21.15.176:0/1082993331' entity='mgr.smithi176.dqbrvx' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-07-16T09:56:04.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:56:04 smithi156 ceph-mon[29302]: from='mgr.24377 ' entity='mgr.smithi176.dqbrvx' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-07-16T09:56:04.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:56:04 smithi156 ceph-mon[29302]: from='mgr.24377 172.21.15.176:0/1082993331' entity='mgr.smithi176.dqbrvx' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-07-16T09:56:04.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:56:04 smithi156 ceph-mon[29302]: from='mgr.24377 ' entity='mgr.smithi176.dqbrvx' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-07-16T09:56:04.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:56:04 smithi156 ceph-mon[29302]: from='mgr.24377 172.21.15.176:0/1082993331' entity='mgr.smithi176.dqbrvx' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-07-16T09:56:04.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:56:04 smithi156 ceph-mon[29302]: from='mgr.24377 ' entity='mgr.smithi176.dqbrvx' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-07-16T09:56:04.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:56:04 smithi156 ceph-mon[29302]: from='mgr.24377 ' entity='mgr.smithi176.dqbrvx' 2024-07-16T09:56:04.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:56:04 smithi156 ceph-mon[29302]: from='mgr.24377 172.21.15.176:0/1082993331' entity='mgr.smithi176.dqbrvx' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-16T09:56:04.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:56:04 smithi156 ceph-mon[29302]: from='mgr.24377 172.21.15.176:0/1082993331' entity='mgr.smithi176.dqbrvx' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-16T09:56:04.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:56:04 smithi176 ceph-mon[36626]: pgmap v10: 65 pgs: 65 active+clean; 3.2 GiB data, 11 GiB used, 526 GiB / 536 GiB avail; 27 MiB/s rd, 74 MiB/s wr, 293 op/s 2024-07-16T09:56:04.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:56:04 smithi176 ceph-mon[36626]: from='mgr.24377 ' entity='mgr.smithi176.dqbrvx' 2024-07-16T09:56:04.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:56:04 smithi176 ceph-mon[36626]: from='mgr.24377 ' entity='mgr.smithi176.dqbrvx' 2024-07-16T09:56:04.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:56:04 smithi176 ceph-mon[36626]: from='mgr.24377 172.21.15.176:0/1082993331' entity='mgr.smithi176.dqbrvx' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-07-16T09:56:04.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:56:04 smithi176 ceph-mon[36626]: from='mgr.24377 ' entity='mgr.smithi176.dqbrvx' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-07-16T09:56:04.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:56:04 smithi176 ceph-mon[36626]: from='mgr.24377 172.21.15.176:0/1082993331' entity='mgr.smithi176.dqbrvx' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-07-16T09:56:04.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:56:04 smithi176 ceph-mon[36626]: from='mgr.24377 ' entity='mgr.smithi176.dqbrvx' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-07-16T09:56:04.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:56:04 smithi176 ceph-mon[36626]: from='mgr.24377 172.21.15.176:0/1082993331' entity='mgr.smithi176.dqbrvx' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-07-16T09:56:04.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:56:04 smithi176 ceph-mon[36626]: from='mgr.24377 ' entity='mgr.smithi176.dqbrvx' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-07-16T09:56:04.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:56:04 smithi176 ceph-mon[36626]: from='mgr.24377 ' entity='mgr.smithi176.dqbrvx' 2024-07-16T09:56:04.845 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:56:04 smithi176 ceph-mon[36626]: from='mgr.24377 172.21.15.176:0/1082993331' entity='mgr.smithi176.dqbrvx' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-16T09:56:04.845 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:56:04 smithi176 ceph-mon[36626]: from='mgr.24377 172.21.15.176:0/1082993331' entity='mgr.smithi176.dqbrvx' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-16T09:56:05.281 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-07-16T09:56:05.281 DEBUG:teuthology.orchestra.run.smithi156:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-07-16T09:56:05.630 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:56:05 smithi156 ceph-mon[29302]: Adjusting osd_memory_target on smithi176 to 2581M 2024-07-16T09:56:05.630 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:56:05 smithi156 ceph-mon[29302]: Updating smithi156:/etc/ceph/ceph.conf 2024-07-16T09:56:05.630 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:56:05 smithi156 ceph-mon[29302]: Updating smithi176:/etc/ceph/ceph.conf 2024-07-16T09:56:05.630 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:56:05 smithi156 ceph-mon[29302]: Updating smithi156:/var/lib/ceph/26c91c26-4358-11ef-bcad-c7b262605968/config/ceph.conf 2024-07-16T09:56:05.630 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:56:05 smithi156 ceph-mon[29302]: Updating smithi176:/var/lib/ceph/26c91c26-4358-11ef-bcad-c7b262605968/config/ceph.conf 2024-07-16T09:56:05.630 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:56:05 smithi156 ceph-mon[29302]: Updating smithi156:/etc/ceph/ceph.client.admin.keyring 2024-07-16T09:56:05.630 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:56:05 smithi156 ceph-mon[29302]: Updating smithi176:/etc/ceph/ceph.client.admin.keyring 2024-07-16T09:56:05.630 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:56:05 smithi156 ceph-mon[29302]: from='mgr.24377 172.21.15.176:0/1082993331' entity='mgr.smithi176.dqbrvx' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-16T09:56:05.631 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:56:05 smithi156 ceph-mon[29302]: from='mgr.24377 ' entity='mgr.smithi176.dqbrvx' 2024-07-16T09:56:05.631 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:56:05 smithi156 ceph-mon[29302]: from='mgr.24377 ' entity='mgr.smithi176.dqbrvx' 2024-07-16T09:56:05.631 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:56:05 smithi156 ceph-mon[29302]: from='mgr.24377 ' entity='mgr.smithi176.dqbrvx' 2024-07-16T09:56:05.631 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:56:05 smithi156 ceph-mon[29302]: from='mgr.24377 ' entity='mgr.smithi176.dqbrvx' 2024-07-16T09:56:05.631 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:56:05 smithi156 ceph-mon[29302]: from='mgr.24377 ' entity='mgr.smithi176.dqbrvx' 2024-07-16T09:56:05.631 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:56:05 smithi156 ceph-mon[29302]: from='mgr.24377 ' entity='mgr.smithi176.dqbrvx' 2024-07-16T09:56:05.631 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:56:05 smithi156 ceph-mon[29302]: from='mgr.24377 ' entity='mgr.smithi176.dqbrvx' 2024-07-16T09:56:05.631 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:56:05 smithi156 ceph-mon[29302]: from='mgr.24377 ' entity='mgr.smithi176.dqbrvx' 2024-07-16T09:56:05.631 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:56:05 smithi156 ceph-mon[29302]: from='mgr.24377 ' entity='mgr.smithi176.dqbrvx' 2024-07-16T09:56:05.749 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:56:05 smithi176 ceph-mon[36626]: Adjusting osd_memory_target on smithi176 to 2581M 2024-07-16T09:56:05.749 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:56:05 smithi176 ceph-mon[36626]: Updating smithi156:/etc/ceph/ceph.conf 2024-07-16T09:56:05.749 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:56:05 smithi176 ceph-mon[36626]: Updating smithi176:/etc/ceph/ceph.conf 2024-07-16T09:56:05.749 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:56:05 smithi176 ceph-mon[36626]: Updating smithi156:/var/lib/ceph/26c91c26-4358-11ef-bcad-c7b262605968/config/ceph.conf 2024-07-16T09:56:05.749 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:56:05 smithi176 ceph-mon[36626]: Updating smithi176:/var/lib/ceph/26c91c26-4358-11ef-bcad-c7b262605968/config/ceph.conf 2024-07-16T09:56:05.750 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:56:05 smithi176 ceph-mon[36626]: Updating smithi156:/etc/ceph/ceph.client.admin.keyring 2024-07-16T09:56:05.750 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:56:05 smithi176 ceph-mon[36626]: Updating smithi176:/etc/ceph/ceph.client.admin.keyring 2024-07-16T09:56:05.750 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:56:05 smithi176 ceph-mon[36626]: from='mgr.24377 172.21.15.176:0/1082993331' entity='mgr.smithi176.dqbrvx' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-16T09:56:05.750 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:56:05 smithi176 ceph-mon[36626]: from='mgr.24377 ' entity='mgr.smithi176.dqbrvx' 2024-07-16T09:56:05.750 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:56:05 smithi176 ceph-mon[36626]: from='mgr.24377 ' entity='mgr.smithi176.dqbrvx' 2024-07-16T09:56:05.750 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:56:05 smithi176 ceph-mon[36626]: from='mgr.24377 ' entity='mgr.smithi176.dqbrvx' 2024-07-16T09:56:05.750 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:56:05 smithi176 ceph-mon[36626]: from='mgr.24377 ' entity='mgr.smithi176.dqbrvx' 2024-07-16T09:56:05.750 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:56:05 smithi176 ceph-mon[36626]: from='mgr.24377 ' entity='mgr.smithi176.dqbrvx' 2024-07-16T09:56:05.750 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:56:05 smithi176 ceph-mon[36626]: from='mgr.24377 ' entity='mgr.smithi176.dqbrvx' 2024-07-16T09:56:05.750 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:56:05 smithi176 ceph-mon[36626]: from='mgr.24377 ' entity='mgr.smithi176.dqbrvx' 2024-07-16T09:56:05.751 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:56:05 smithi176 ceph-mon[36626]: from='mgr.24377 ' entity='mgr.smithi176.dqbrvx' 2024-07-16T09:56:05.751 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:56:05 smithi176 ceph-mon[36626]: from='mgr.24377 ' entity='mgr.smithi176.dqbrvx' 2024-07-16T09:56:06.383 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:06.380+0000 7f050d6d7640 1 -- 172.21.15.156:0/2709256674 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f05080ff510 msgr2=0x7f05081018f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:56:06.383 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:06.380+0000 7f050d6d7640 1 --2- 172.21.15.156:0/2709256674 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f05080ff510 0x7f05081018f0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f04fc0048d0 tx=0x7f04fc009da0 comp rx=0 tx=0).stop 2024-07-16T09:56:06.383 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:06.380+0000 7f050d6d7640 1 -- 172.21.15.156:0/2709256674 shutdown_connections 2024-07-16T09:56:06.383 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:06.381+0000 7f050d6d7640 1 --2- 172.21.15.156:0/2709256674 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f05080ff510 0x7f05081018f0 secure :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f04fc0048d0 tx=0x7f04fc009da0 comp rx=0 tx=0).stop 2024-07-16T09:56:06.384 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:06.381+0000 7f050d6d7640 1 --2- 172.21.15.156:0/2709256674 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f05080659d0 0x7f05080fefd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:56:06.384 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:06.381+0000 7f050d6d7640 1 -- 172.21.15.156:0/2709256674 >> 172.21.15.156:0/2709256674 conn(0x7f05080f6a20 msgr2=0x7f05080f8e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:56:06.384 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:06.381+0000 7f050d6d7640 1 -- 172.21.15.156:0/2709256674 shutdown_connections 2024-07-16T09:56:06.384 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:06.381+0000 7f050d6d7640 1 -- 172.21.15.156:0/2709256674 wait complete. 2024-07-16T09:56:06.384 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:06.381+0000 7f050d6d7640 1 Processor -- start 2024-07-16T09:56:06.384 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:06.382+0000 7f050d6d7640 1 -- start start 2024-07-16T09:56:06.385 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:06.382+0000 7f050d6d7640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f05080659d0 0x7f050810d120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:56:06.385 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:06.382+0000 7f050d6d7640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f050810d660 0x7f050810e480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:56:06.385 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:06.382+0000 7f050d6d7640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f050810db20 con 0x7f050810d660 2024-07-16T09:56:06.385 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:06.382+0000 7f050d6d7640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f050810dc60 con 0x7f05080659d0 2024-07-16T09:56:06.385 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:06.382+0000 7f0506ffd640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f05080659d0 0x7f050810d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:56:06.385 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:06.382+0000 7f0506ffd640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f05080659d0 0x7f050810d120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.156:37032/0 (socket says 172.21.15.156:37032) 2024-07-16T09:56:06.386 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:06.382+0000 7f0506ffd640 1 -- 172.21.15.156:0/1669833561 learned_addr learned my addr 172.21.15.156:0/1669833561 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T09:56:06.386 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:06.382+0000 7f05067fc640 1 --2- 172.21.15.156:0/1669833561 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f050810d660 0x7f050810e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:56:06.386 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:06.382+0000 7f0506ffd640 1 -- 172.21.15.156:0/1669833561 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f050810d660 msgr2=0x7f050810e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:56:06.386 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:06.382+0000 7f0506ffd640 1 --2- 172.21.15.156:0/1669833561 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f050810d660 0x7f050810e480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:56:06.386 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:06.382+0000 7f0506ffd640 1 -- 172.21.15.156:0/1669833561 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04fc005040 con 0x7f05080659d0 2024-07-16T09:56:06.386 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:06.383+0000 7f0506ffd640 1 --2- 172.21.15.156:0/1669833561 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f05080659d0 0x7f050810d120 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f04f800eb70 tx=0x7f04f800b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:56:06.386 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:06.383+0000 7f04ef7fe640 1 -- 172.21.15.156:0/1669833561 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f04f8007c30 con 0x7f05080659d0 2024-07-16T09:56:06.386 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:06.383+0000 7f050d6d7640 1 -- 172.21.15.156:0/1669833561 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f050810ea80 con 0x7f05080659d0 2024-07-16T09:56:06.386 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:06.383+0000 7f050d6d7640 1 -- 172.21.15.156:0/1669833561 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f050810f020 con 0x7f05080659d0 2024-07-16T09:56:06.386 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:06.383+0000 7f04ef7fe640 1 -- 172.21.15.156:0/1669833561 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f04f800f040 con 0x7f05080659d0 2024-07-16T09:56:06.386 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:06.383+0000 7f04ef7fe640 1 -- 172.21.15.156:0/1669833561 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f04f800cbb0 con 0x7f05080659d0 2024-07-16T09:56:06.387 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:06.384+0000 7f050d6d7640 1 -- 172.21.15.156:0/1669833561 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04f4005240 con 0x7f05080659d0 2024-07-16T09:56:06.387 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:06.385+0000 7f04ef7fe640 1 -- 172.21.15.156:0/1669833561 <== mon.1 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 25) v1 ==== 88617+0+0 (secure 0 0 0) 0x7f04f8014440 con 0x7f05080659d0 2024-07-16T09:56:06.388 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:06.385+0000 7f04ef7fe640 1 --2- 172.21.15.156:0/1669833561 >> [v2:172.21.15.176:6828/673792884,v1:172.21.15.176:6829/673792884] conn(0x7f04f006a6d0 0x7f04f006cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:56:06.388 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:06.386+0000 7f04ef7fe640 1 -- 172.21.15.156:0/1669833561 <== mon.1 v2:172.21.15.176:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5722+0+0 (secure 0 0 0) 0x7f04f8089190 con 0x7f05080659d0 2024-07-16T09:56:06.388 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:06.386+0000 7f05067fc640 1 --2- 172.21.15.156:0/1669833561 >> [v2:172.21.15.176:6828/673792884,v1:172.21.15.176:6829/673792884] conn(0x7f04f006a6d0 0x7f04f006cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:56:06.388 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:06.386+0000 7f05067fc640 1 --2- 172.21.15.156:0/1669833561 >> [v2:172.21.15.176:6828/673792884,v1:172.21.15.176:6829/673792884] conn(0x7f04f006a6d0 0x7f04f006cb80 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f04fc009040 tx=0x7f04fc035040 comp rx=0 tx=0).ready entity=mgr.24377 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:56:06.394 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:06.392+0000 7f04ef7fe640 1 -- 172.21.15.156:0/1669833561 <== mon.1 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185936 (secure 0 0 0) 0x7f04f8052d20 con 0x7f05080659d0 2024-07-16T09:56:06.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:56:06 smithi156 ceph-mon[29302]: Updating smithi156:/var/lib/ceph/26c91c26-4358-11ef-bcad-c7b262605968/config/ceph.client.admin.keyring 2024-07-16T09:56:06.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:56:06 smithi156 ceph-mon[29302]: Updating smithi176:/var/lib/ceph/26c91c26-4358-11ef-bcad-c7b262605968/config/ceph.client.admin.keyring 2024-07-16T09:56:06.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:56:06 smithi156 ceph-mon[29302]: Reconfiguring node-exporter.smithi156 (dependencies changed)... 2024-07-16T09:56:06.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:56:06 smithi156 ceph-mon[29302]: Reconfiguring daemon node-exporter.smithi156 on smithi156 2024-07-16T09:56:06.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:56:06 smithi156 ceph-mon[29302]: pgmap v11: 65 pgs: 65 active+clean; 2.2 GiB data, 7.6 GiB used, 529 GiB / 536 GiB avail; 28 MiB/s rd, 75 MiB/s wr, 382 op/s 2024-07-16T09:56:06.678 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:06.676+0000 7f050d6d7640 1 -- 172.21.15.156:0/1669833561 --> [v2:172.21.15.176:6828/673792884,v1:172.21.15.176:6829/673792884] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f04f4000bf0 con 0x7f04f006a6d0 2024-07-16T09:56:06.679 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:06.678+0000 7f04ef7fe640 1 -- 172.21.15.156:0/1669833561 <== mgr.24377 v2:172.21.15.176:6828/673792884 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+318 (secure 0 0 0) 0x7f04f4000bf0 con 0x7f04f006a6d0 2024-07-16T09:56:06.683 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:06.682+0000 7f050d6d7640 1 -- 172.21.15.156:0/1669833561 >> [v2:172.21.15.176:6828/673792884,v1:172.21.15.176:6829/673792884] conn(0x7f04f006a6d0 msgr2=0x7f04f006cb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:56:06.683 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:06.682+0000 7f050d6d7640 1 --2- 172.21.15.156:0/1669833561 >> [v2:172.21.15.176:6828/673792884,v1:172.21.15.176:6829/673792884] conn(0x7f04f006a6d0 0x7f04f006cb80 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f04fc009040 tx=0x7f04fc035040 comp rx=0 tx=0).stop 2024-07-16T09:56:06.683 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:06.682+0000 7f050d6d7640 1 -- 172.21.15.156:0/1669833561 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f05080659d0 msgr2=0x7f050810d120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:56:06.683 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:06.682+0000 7f050d6d7640 1 --2- 172.21.15.156:0/1669833561 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f05080659d0 0x7f050810d120 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f04f800eb70 tx=0x7f04f800b040 comp rx=0 tx=0).stop 2024-07-16T09:56:06.684 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:06.682+0000 7f050d6d7640 1 -- 172.21.15.156:0/1669833561 shutdown_connections 2024-07-16T09:56:06.684 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:06.682+0000 7f050d6d7640 1 --2- 172.21.15.156:0/1669833561 >> [v2:172.21.15.176:6828/673792884,v1:172.21.15.176:6829/673792884] conn(0x7f04f006a6d0 0x7f04f006cb80 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:56:06.684 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:06.682+0000 7f050d6d7640 1 --2- 172.21.15.156:0/1669833561 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f050810d660 0x7f050810e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:56:06.684 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:06.682+0000 7f050d6d7640 1 --2- 172.21.15.156:0/1669833561 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f05080659d0 0x7f050810d120 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:56:06.684 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:06.682+0000 7f050d6d7640 1 -- 172.21.15.156:0/1669833561 >> 172.21.15.156:0/1669833561 conn(0x7f05080f6a20 msgr2=0x7f05080f6df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:56:06.684 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:06.683+0000 7f050d6d7640 1 -- 172.21.15.156:0/1669833561 shutdown_connections 2024-07-16T09:56:06.684 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:06.683+0000 7f050d6d7640 1 -- 172.21.15.156:0/1669833561 wait complete. 2024-07-16T09:56:06.698 INFO:teuthology.orchestra.run.smithi156.stdout:true 2024-07-16T09:56:06.819 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:06.817+0000 7f0ca4797640 1 -- 172.21.15.156:0/1547480700 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0c9c103c40 msgr2=0x7f0c9c1040b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:56:06.819 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:06.817+0000 7f0ca4797640 1 --2- 172.21.15.156:0/1547480700 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0c9c103c40 0x7f0c9c1040b0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f0c8c004810 tx=0x7f0c8c009da0 comp rx=0 tx=0).stop 2024-07-16T09:56:06.819 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:06.817+0000 7f0ca4797640 1 -- 172.21.15.156:0/1547480700 shutdown_connections 2024-07-16T09:56:06.819 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:06.817+0000 7f0ca4797640 1 --2- 172.21.15.156:0/1547480700 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0c9c103c40 0x7f0c9c1040b0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:56:06.819 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:06.817+0000 7f0ca4797640 1 --2- 172.21.15.156:0/1547480700 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0c9c102970 0x7f0c9c102d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:56:06.819 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:06.817+0000 7f0ca4797640 1 -- 172.21.15.156:0/1547480700 >> 172.21.15.156:0/1547480700 conn(0x7f0c9c071bd0 msgr2=0x7f0c9c074000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:56:06.819 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:06.817+0000 7f0ca4797640 1 -- 172.21.15.156:0/1547480700 shutdown_connections 2024-07-16T09:56:06.819 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:06.817+0000 7f0ca4797640 1 -- 172.21.15.156:0/1547480700 wait complete. 2024-07-16T09:56:06.820 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:06.818+0000 7f0ca4797640 1 Processor -- start 2024-07-16T09:56:06.820 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:06.819+0000 7f0ca4797640 1 -- start start 2024-07-16T09:56:06.821 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:06.819+0000 7f0ca4797640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0c9c102970 0x7f0c9c1134f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:56:06.821 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:06.819+0000 7f0ca4797640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0c9c103c40 0x7f0c9c113a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:56:06.821 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:06.819+0000 7f0ca4797640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c9c114080 con 0x7f0c9c103c40 2024-07-16T09:56:06.821 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:06.819+0000 7f0ca4797640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c9c1148a0 con 0x7f0c9c102970 2024-07-16T09:56:06.821 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:06.819+0000 7f0ca2533640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0c9c102970 0x7f0c9c1134f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:56:06.821 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:06.819+0000 7f0ca1d32640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0c9c103c40 0x7f0c9c113a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:56:06.821 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:06.819+0000 7f0ca2533640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0c9c102970 0x7f0c9c1134f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.156:37054/0 (socket says 172.21.15.156:37054) 2024-07-16T09:56:06.821 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:06.819+0000 7f0ca2533640 1 -- 172.21.15.156:0/3012672412 learned_addr learned my addr 172.21.15.156:0/3012672412 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T09:56:06.822 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:06.820+0000 7f0ca1d32640 1 -- 172.21.15.156:0/3012672412 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0c9c102970 msgr2=0x7f0c9c1134f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:56:06.822 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:06.820+0000 7f0ca1d32640 1 --2- 172.21.15.156:0/3012672412 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0c9c102970 0x7f0c9c1134f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:56:06.822 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:06.820+0000 7f0ca1d32640 1 -- 172.21.15.156:0/3012672412 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c8c005040 con 0x7f0c9c103c40 2024-07-16T09:56:06.822 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:06.820+0000 7f0ca2533640 1 --2- 172.21.15.156:0/3012672412 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0c9c102970 0x7f0c9c1134f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-16T09:56:06.822 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:06.820+0000 7f0ca1d32640 1 --2- 172.21.15.156:0/3012672412 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0c9c103c40 0x7f0c9c113a30 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f0c8c004810 tx=0x7f0c8c009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:56:06.823 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:06.821+0000 7f0c92ffd640 1 -- 172.21.15.156:0/3012672412 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0c8c038070 con 0x7f0c9c103c40 2024-07-16T09:56:06.823 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:06.821+0000 7f0c92ffd640 1 -- 172.21.15.156:0/3012672412 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0c8c032970 con 0x7f0c9c103c40 2024-07-16T09:56:06.823 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:06.821+0000 7f0c92ffd640 1 -- 172.21.15.156:0/3012672412 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0c8c02a940 con 0x7f0c9c103c40 2024-07-16T09:56:06.823 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:06.821+0000 7f0ca4797640 1 -- 172.21.15.156:0/3012672412 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c9c114ad0 con 0x7f0c9c103c40 2024-07-16T09:56:06.823 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:06.821+0000 7f0ca4797640 1 -- 172.21.15.156:0/3012672412 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c9c114ed0 con 0x7f0c9c103c40 2024-07-16T09:56:06.824 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:06.823+0000 7f0ca4797640 1 -- 172.21.15.156:0/3012672412 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c9c075800 con 0x7f0c9c103c40 2024-07-16T09:56:06.825 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:06.824+0000 7f0c92ffd640 1 -- 172.21.15.156:0/3012672412 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 25) v1 ==== 88617+0+0 (secure 0 0 0) 0x7f0c8c030020 con 0x7f0c9c103c40 2024-07-16T09:56:06.826 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:06.825+0000 7f0c92ffd640 1 --2- 172.21.15.156:0/3012672412 >> [v2:172.21.15.176:6828/673792884,v1:172.21.15.176:6829/673792884] conn(0x7f0c8806a860 0x7f0c8806cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:56:06.826 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:06.825+0000 7f0c92ffd640 1 -- 172.21.15.156:0/3012672412 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5722+0+0 (secure 0 0 0) 0x7f0c8c0a8a20 con 0x7f0c9c103c40 2024-07-16T09:56:06.827 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:06.826+0000 7f0ca2533640 1 --2- 172.21.15.156:0/3012672412 >> [v2:172.21.15.176:6828/673792884,v1:172.21.15.176:6829/673792884] conn(0x7f0c8806a860 0x7f0c8806cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:56:06.832 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:06.831+0000 7f0ca2533640 1 --2- 172.21.15.156:0/3012672412 >> [v2:172.21.15.176:6828/673792884,v1:172.21.15.176:6829/673792884] conn(0x7f0c8806a860 0x7f0c8806cd10 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f0c98009af0 tx=0x7f0c9800b040 comp rx=0 tx=0).ready entity=mgr.24377 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:56:06.836 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:06.834+0000 7f0c92ffd640 1 -- 172.21.15.156:0/3012672412 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185936 (secure 0 0 0) 0x7f0c8c072760 con 0x7f0c9c103c40 2024-07-16T09:56:06.843 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:56:06 smithi176 ceph-mon[36626]: Updating smithi156:/var/lib/ceph/26c91c26-4358-11ef-bcad-c7b262605968/config/ceph.client.admin.keyring 2024-07-16T09:56:06.843 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:56:06 smithi176 ceph-mon[36626]: Updating smithi176:/var/lib/ceph/26c91c26-4358-11ef-bcad-c7b262605968/config/ceph.client.admin.keyring 2024-07-16T09:56:06.843 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:56:06 smithi176 ceph-mon[36626]: Reconfiguring node-exporter.smithi156 (dependencies changed)... 2024-07-16T09:56:06.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:56:06 smithi176 ceph-mon[36626]: Reconfiguring daemon node-exporter.smithi156 on smithi156 2024-07-16T09:56:06.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:56:06 smithi176 ceph-mon[36626]: pgmap v11: 65 pgs: 65 active+clean; 2.2 GiB data, 7.6 GiB used, 529 GiB / 536 GiB avail; 28 MiB/s rd, 75 MiB/s wr, 382 op/s 2024-07-16T09:56:07.140 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:07.138+0000 7f0ca4797640 1 -- 172.21.15.156:0/3012672412 --> [v2:172.21.15.176:6828/673792884,v1:172.21.15.176:6829/673792884] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f0c9c1152f0 con 0x7f0c8806a860 2024-07-16T09:56:07.141 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:07.140+0000 7f0c92ffd640 1 -- 172.21.15.156:0/3012672412 <== mgr.24377 v2:172.21.15.176:6828/673792884 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+318 (secure 0 0 0) 0x7f0c9c1152f0 con 0x7f0c8806a860 2024-07-16T09:56:07.145 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:07.143+0000 7f0ca4797640 1 -- 172.21.15.156:0/3012672412 >> [v2:172.21.15.176:6828/673792884,v1:172.21.15.176:6829/673792884] conn(0x7f0c8806a860 msgr2=0x7f0c8806cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:56:07.145 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:07.143+0000 7f0ca4797640 1 --2- 172.21.15.156:0/3012672412 >> [v2:172.21.15.176:6828/673792884,v1:172.21.15.176:6829/673792884] conn(0x7f0c8806a860 0x7f0c8806cd10 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f0c98009af0 tx=0x7f0c9800b040 comp rx=0 tx=0).stop 2024-07-16T09:56:07.145 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:07.143+0000 7f0ca4797640 1 -- 172.21.15.156:0/3012672412 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0c9c103c40 msgr2=0x7f0c9c113a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:56:07.145 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:07.143+0000 7f0ca4797640 1 --2- 172.21.15.156:0/3012672412 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0c9c103c40 0x7f0c9c113a30 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f0c8c004810 tx=0x7f0c8c009da0 comp rx=0 tx=0).stop 2024-07-16T09:56:07.145 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:07.144+0000 7f0ca4797640 1 -- 172.21.15.156:0/3012672412 shutdown_connections 2024-07-16T09:56:07.145 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:07.144+0000 7f0ca4797640 1 --2- 172.21.15.156:0/3012672412 >> [v2:172.21.15.176:6828/673792884,v1:172.21.15.176:6829/673792884] conn(0x7f0c8806a860 0x7f0c8806cd10 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:56:07.145 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:07.144+0000 7f0ca4797640 1 --2- 172.21.15.156:0/3012672412 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0c9c103c40 0x7f0c9c113a30 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:56:07.145 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:07.144+0000 7f0ca4797640 1 --2- 172.21.15.156:0/3012672412 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0c9c102970 0x7f0c9c1134f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:56:07.145 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:07.144+0000 7f0ca4797640 1 -- 172.21.15.156:0/3012672412 >> 172.21.15.156:0/3012672412 conn(0x7f0c9c071bd0 msgr2=0x7f0c9c0731f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:56:07.145 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:07.144+0000 7f0ca4797640 1 -- 172.21.15.156:0/3012672412 shutdown_connections 2024-07-16T09:56:07.146 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:07.144+0000 7f0ca4797640 1 -- 172.21.15.156:0/3012672412 wait complete. 2024-07-16T09:56:07.251 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:07.250+0000 7f444aecb640 1 -- 172.21.15.156:0/1489348660 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f44440fc510 msgr2=0x7f44440fc980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:56:07.251 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:07.250+0000 7f444aecb640 1 --2- 172.21.15.156:0/1489348660 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f44440fc510 0x7f44440fc980 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f44300048d0 tx=0x7f4430009da0 comp rx=0 tx=0).stop 2024-07-16T09:56:07.251 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:07.250+0000 7f444aecb640 1 -- 172.21.15.156:0/1489348660 shutdown_connections 2024-07-16T09:56:07.251 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:07.250+0000 7f444aecb640 1 --2- 172.21.15.156:0/1489348660 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f44440fc510 0x7f44440fc980 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:56:07.252 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:07.250+0000 7f444aecb640 1 --2- 172.21.15.156:0/1489348660 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f44440fb240 0x7f44440fb650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:56:07.252 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:07.250+0000 7f444aecb640 1 -- 172.21.15.156:0/1489348660 >> 172.21.15.156:0/1489348660 conn(0x7f44440f6a20 msgr2=0x7f44440f8e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:56:07.252 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:07.250+0000 7f444aecb640 1 -- 172.21.15.156:0/1489348660 shutdown_connections 2024-07-16T09:56:07.252 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:07.250+0000 7f444aecb640 1 -- 172.21.15.156:0/1489348660 wait complete. 2024-07-16T09:56:07.254 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:07.251+0000 7f444aecb640 1 Processor -- start 2024-07-16T09:56:07.254 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:07.251+0000 7f444aecb640 1 -- start start 2024-07-16T09:56:07.254 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:07.251+0000 7f444aecb640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f44440fb240 0x7f444410cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:56:07.254 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:07.251+0000 7f444aecb640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f44440fc510 0x7f444410d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:56:07.254 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:07.251+0000 7f444aecb640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f444410db30 con 0x7f44440fb240 2024-07-16T09:56:07.254 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:07.251+0000 7f444aecb640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f444410e350 con 0x7f44440fc510 2024-07-16T09:56:07.255 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:07.251+0000 7f4448c67640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f44440fb240 0x7f444410cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:56:07.255 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:07.251+0000 7f4443fff640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f44440fc510 0x7f444410d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:56:07.255 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:07.251+0000 7f4443fff640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f44440fc510 0x7f444410d4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.156:37070/0 (socket says 172.21.15.156:37070) 2024-07-16T09:56:07.255 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:07.251+0000 7f4443fff640 1 -- 172.21.15.156:0/691520626 learned_addr learned my addr 172.21.15.156:0/691520626 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T09:56:07.255 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:07.252+0000 7f4443fff640 1 -- 172.21.15.156:0/691520626 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f44440fb240 msgr2=0x7f444410cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:56:07.255 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:07.252+0000 7f4443fff640 1 --2- 172.21.15.156:0/691520626 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f44440fb240 0x7f444410cfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:56:07.255 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:07.252+0000 7f4443fff640 1 -- 172.21.15.156:0/691520626 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4430005040 con 0x7f44440fc510 2024-07-16T09:56:07.255 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:07.252+0000 7f4443fff640 1 --2- 172.21.15.156:0/691520626 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f44440fc510 0x7f444410d4e0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f44300041d0 tx=0x7f4430009da0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:56:07.255 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:07.252+0000 7f44417fa640 1 -- 172.21.15.156:0/691520626 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4430038070 con 0x7f44440fc510 2024-07-16T09:56:07.255 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:07.252+0000 7f44417fa640 1 -- 172.21.15.156:0/691520626 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f44300092b0 con 0x7f44440fc510 2024-07-16T09:56:07.255 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:07.252+0000 7f44417fa640 1 -- 172.21.15.156:0/691520626 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4430033710 con 0x7f44440fc510 2024-07-16T09:56:07.255 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:07.252+0000 7f444aecb640 1 -- 172.21.15.156:0/691520626 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f444410e580 con 0x7f44440fc510 2024-07-16T09:56:07.256 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:07.253+0000 7f444aecb640 1 -- 172.21.15.156:0/691520626 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f444410eac0 con 0x7f44440fc510 2024-07-16T09:56:07.256 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:07.254+0000 7f444aecb640 1 -- 172.21.15.156:0/691520626 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44440fa6a0 con 0x7f44440fc510 2024-07-16T09:56:07.256 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:07.255+0000 7f44417fa640 1 -- 172.21.15.156:0/691520626 <== mon.1 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 25) v1 ==== 88617+0+0 (secure 0 0 0) 0x7f4430030020 con 0x7f44440fc510 2024-07-16T09:56:07.257 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:07.255+0000 7f44417fa640 1 --2- 172.21.15.156:0/691520626 >> [v2:172.21.15.176:6828/673792884,v1:172.21.15.176:6829/673792884] conn(0x7f443406a6d0 0x7f443406cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:56:07.257 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:07.256+0000 7f4448c67640 1 --2- 172.21.15.156:0/691520626 >> [v2:172.21.15.176:6828/673792884,v1:172.21.15.176:6829/673792884] conn(0x7f443406a6d0 0x7f443406cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:56:07.257 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:07.256+0000 7f44417fa640 1 -- 172.21.15.156:0/691520626 <== mon.1 v2:172.21.15.176:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5722+0+0 (secure 0 0 0) 0x7f44300a8c70 con 0x7f44440fc510 2024-07-16T09:56:07.257 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:07.256+0000 7f4448c67640 1 --2- 172.21.15.156:0/691520626 >> [v2:172.21.15.176:6828/673792884,v1:172.21.15.176:6829/673792884] conn(0x7f443406a6d0 0x7f443406cb80 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f4438009af0 tx=0x7f443800b040 comp rx=0 tx=0).ready entity=mgr.24377 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:56:07.263 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:07.261+0000 7f44417fa640 1 -- 172.21.15.156:0/691520626 <== mon.1 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185936 (secure 0 0 0) 0x7f44300764f0 con 0x7f44440fc510 2024-07-16T09:56:07.383 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-07-16T09:56:07.383 DEBUG:teuthology.orchestra.run.smithi176:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.1/client.1/tmp 2024-07-16T09:56:07.488 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:07.486+0000 7f444aecb640 1 -- 172.21.15.156:0/691520626 --> [v2:172.21.15.176:6828/673792884,v1:172.21.15.176:6829/673792884] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f444410ee40 con 0x7f443406a6d0 2024-07-16T09:56:07.498 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:07.497+0000 7f44417fa640 1 -- 172.21.15.156:0/691520626 <== mgr.24377 v2:172.21.15.176:6828/673792884 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3388 (secure 0 0 0) 0x7f444410ee40 con 0x7f443406a6d0 2024-07-16T09:56:07.499 INFO:teuthology.orchestra.run.smithi156.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-07-16T09:56:07.501 INFO:teuthology.orchestra.run.smithi156.stdout:alertmanager.smithi156 smithi156 *:9093,9094 running (6m) 9s ago 8m 24.4M - 0.25.0 c8568f914cd2 83419a2a37f4 2024-07-16T09:56:07.501 INFO:teuthology.orchestra.run.smithi156.stdout:crash.smithi156 smithi156 running (8m) 9s ago 8m 6899k - 17.2.7-1104-ge6231b04 d5de5f953b95 bf6d49ea06b4 2024-07-16T09:56:07.501 INFO:teuthology.orchestra.run.smithi156.stdout:crash.smithi176 smithi176 running (7m) 9s ago 7m 6903k - 17.2.7-1104-ge6231b04 d5de5f953b95 907c9fa1e890 2024-07-16T09:56:07.501 INFO:teuthology.orchestra.run.smithi156.stdout:grafana.smithi156 smithi156 *:3000 running (6m) 9s ago 7m 85.9M - 9.4.7 954c08fa6188 184256cf055d 2024-07-16T09:56:07.501 INFO:teuthology.orchestra.run.smithi156.stdout:mds.cephfs.smithi156.evwwaz smithi156 running (3m) 9s ago 3m 82.5M - 17.2.7-1104-ge6231b04 d5de5f953b95 7c0ff9ae43f8 2024-07-16T09:56:07.501 INFO:teuthology.orchestra.run.smithi156.stdout:mds.cephfs.smithi156.rdibys smithi156 running (3m) 9s ago 3m 144M - 17.2.7-1104-ge6231b04 d5de5f953b95 b985cd1e6924 2024-07-16T09:56:07.501 INFO:teuthology.orchestra.run.smithi156.stdout:mds.cephfs.smithi176.dvvbud smithi176 running (3m) 9s ago 3m 135M - 17.2.7-1104-ge6231b04 d5de5f953b95 1b00147950f9 2024-07-16T09:56:07.501 INFO:teuthology.orchestra.run.smithi156.stdout:mds.cephfs.smithi176.wahbsi smithi176 running (3m) 9s ago 3m 82.5M - 17.2.7-1104-ge6231b04 d5de5f953b95 866664af2311 2024-07-16T09:56:07.502 INFO:teuthology.orchestra.run.smithi156.stdout:mgr.smithi156.zkvylv smithi156 *:9283 running (9m) 9s ago 9m 271M - 17.2.7-1104-ge6231b04 d5de5f953b95 a6937c84750a 2024-07-16T09:56:07.502 INFO:teuthology.orchestra.run.smithi156.stdout:mgr.smithi176.dqbrvx smithi176 *:8443,9283 running (70s) 9s ago 7m 482M - 19.0.0-3495-ga6ca461b 16f7feb04342 a7c332f2d676 2024-07-16T09:56:07.502 INFO:teuthology.orchestra.run.smithi156.stdout:mon.smithi156 smithi156 running (9m) 9s ago 9m 51.1M 2048M 17.2.7-1104-ge6231b04 d5de5f953b95 6471c8acff40 2024-07-16T09:56:07.502 INFO:teuthology.orchestra.run.smithi156.stdout:mon.smithi176 smithi176 running (7m) 9s ago 7m 40.4M 2048M 17.2.7-1104-ge6231b04 d5de5f953b95 3a293ac6ef01 2024-07-16T09:56:07.502 INFO:teuthology.orchestra.run.smithi156.stdout:node-exporter.smithi156 smithi156 *:9100 running (8m) 9s ago 8m 15.5M - 1.5.0 0da6a335fe13 2c136840a7a9 2024-07-16T09:56:07.502 INFO:teuthology.orchestra.run.smithi156.stdout:node-exporter.smithi176 smithi176 *:9100 running (7m) 9s ago 7m 16.3M - 1.5.0 0da6a335fe13 536716db6afe 2024-07-16T09:56:07.502 INFO:teuthology.orchestra.run.smithi156.stdout:osd.0 smithi156 running (6m) 9s ago 6m 239M 1557M 17.2.7-1104-ge6231b04 d5de5f953b95 0471758ec349 2024-07-16T09:56:07.502 INFO:teuthology.orchestra.run.smithi156.stdout:osd.1 smithi156 running (6m) 9s ago 6m 235M 1557M 17.2.7-1104-ge6231b04 d5de5f953b95 149a0bda2f61 2024-07-16T09:56:07.503 INFO:teuthology.orchestra.run.smithi156.stdout:osd.2 smithi156 running (5m) 9s ago 5m 212M 1557M 17.2.7-1104-ge6231b04 d5de5f953b95 f28acec5cb79 2024-07-16T09:56:07.503 INFO:teuthology.orchestra.run.smithi156.stdout:osd.3 smithi176 running (5m) 9s ago 5m 256M 2581M 17.2.7-1104-ge6231b04 d5de5f953b95 733861ba84dc 2024-07-16T09:56:07.503 INFO:teuthology.orchestra.run.smithi156.stdout:osd.4 smithi176 running (5m) 9s ago 5m 233M 2581M 17.2.7-1104-ge6231b04 d5de5f953b95 ef0c67cec086 2024-07-16T09:56:07.503 INFO:teuthology.orchestra.run.smithi156.stdout:osd.5 smithi176 running (4m) 9s ago 4m 268M 2581M 17.2.7-1104-ge6231b04 d5de5f953b95 996e15448a52 2024-07-16T09:56:07.503 INFO:teuthology.orchestra.run.smithi156.stdout:prometheus.smithi156 smithi156 *:9095 running (6m) 9s ago 7m 57.9M - 2.43.0 a07b618ecd1d 1655b4227cd0 2024-07-16T09:56:07.503 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:07.499+0000 7f444aecb640 1 -- 172.21.15.156:0/691520626 >> [v2:172.21.15.176:6828/673792884,v1:172.21.15.176:6829/673792884] conn(0x7f443406a6d0 msgr2=0x7f443406cb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:56:07.504 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:07.499+0000 7f444aecb640 1 --2- 172.21.15.156:0/691520626 >> [v2:172.21.15.176:6828/673792884,v1:172.21.15.176:6829/673792884] conn(0x7f443406a6d0 0x7f443406cb80 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f4438009af0 tx=0x7f443800b040 comp rx=0 tx=0).stop 2024-07-16T09:56:07.504 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:07.499+0000 7f444aecb640 1 -- 172.21.15.156:0/691520626 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f44440fc510 msgr2=0x7f444410d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:56:07.504 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:07.499+0000 7f444aecb640 1 --2- 172.21.15.156:0/691520626 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f44440fc510 0x7f444410d4e0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f44300041d0 tx=0x7f4430009da0 comp rx=0 tx=0).stop 2024-07-16T09:56:07.504 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:07.500+0000 7f444aecb640 1 -- 172.21.15.156:0/691520626 shutdown_connections 2024-07-16T09:56:07.504 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:07.500+0000 7f444aecb640 1 --2- 172.21.15.156:0/691520626 >> [v2:172.21.15.176:6828/673792884,v1:172.21.15.176:6829/673792884] conn(0x7f443406a6d0 0x7f443406cb80 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:56:07.504 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:07.500+0000 7f444aecb640 1 --2- 172.21.15.156:0/691520626 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f44440fc510 0x7f444410d4e0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:56:07.504 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:07.500+0000 7f444aecb640 1 --2- 172.21.15.156:0/691520626 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f44440fb240 0x7f444410cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:56:07.504 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:07.500+0000 7f444aecb640 1 -- 172.21.15.156:0/691520626 >> 172.21.15.156:0/691520626 conn(0x7f44440f6a20 msgr2=0x7f44440f8140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:56:07.504 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:07.500+0000 7f444aecb640 1 -- 172.21.15.156:0/691520626 shutdown_connections 2024-07-16T09:56:07.505 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:07.500+0000 7f444aecb640 1 -- 172.21.15.156:0/691520626 wait complete. 2024-07-16T09:56:07.599 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:07.597+0000 7fc2075c5640 1 -- 172.21.15.156:0/211661854 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc200104750 msgr2=0x7fc200104b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:56:07.599 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:07.597+0000 7fc2075c5640 1 --2- 172.21.15.156:0/211661854 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc200104750 0x7fc200104b60 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fc1fc0048d0 tx=0x7fc1fc009da0 comp rx=0 tx=0).stop 2024-07-16T09:56:07.599 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:07.597+0000 7fc2075c5640 1 -- 172.21.15.156:0/211661854 shutdown_connections 2024-07-16T09:56:07.599 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:07.597+0000 7fc2075c5640 1 --2- 172.21.15.156:0/211661854 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc200105150 0x7fc20006e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:56:07.599 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:07.597+0000 7fc2075c5640 1 --2- 172.21.15.156:0/211661854 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc200104750 0x7fc200104b60 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:56:07.599 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:07.597+0000 7fc2075c5640 1 -- 172.21.15.156:0/211661854 >> 172.21.15.156:0/211661854 conn(0x7fc200069370 msgr2=0x7fc20006b7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:56:07.599 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:07.597+0000 7fc2075c5640 1 -- 172.21.15.156:0/211661854 shutdown_connections 2024-07-16T09:56:07.599 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:07.597+0000 7fc2075c5640 1 -- 172.21.15.156:0/211661854 wait complete. 2024-07-16T09:56:07.601 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:07.598+0000 7fc2075c5640 1 Processor -- start 2024-07-16T09:56:07.601 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:07.598+0000 7fc2075c5640 1 -- start start 2024-07-16T09:56:07.601 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:07.598+0000 7fc2075c5640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc200104750 0x7fc2001179a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:56:07.601 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:07.598+0000 7fc2075c5640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc200117ee0 0x7fc200118d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:56:07.601 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:07.598+0000 7fc2075c5640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2001183a0 con 0x7fc200117ee0 2024-07-16T09:56:07.601 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:07.598+0000 7fc2075c5640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2001184e0 con 0x7fc200104750 2024-07-16T09:56:07.601 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:07.598+0000 7fc204b60640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc200117ee0 0x7fc200118d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:56:07.601 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:07.598+0000 7fc204b60640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc200117ee0 0x7fc200118d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:57028/0 (socket says 172.21.15.156:57028) 2024-07-16T09:56:07.601 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:07.598+0000 7fc204b60640 1 -- 172.21.15.156:0/1834887665 learned_addr learned my addr 172.21.15.156:0/1834887665 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T09:56:07.601 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:07.598+0000 7fc205361640 1 --2- 172.21.15.156:0/1834887665 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc200104750 0x7fc2001179a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:56:07.601 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:07.599+0000 7fc204b60640 1 -- 172.21.15.156:0/1834887665 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc200104750 msgr2=0x7fc2001179a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:56:07.602 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:07.599+0000 7fc204b60640 1 --2- 172.21.15.156:0/1834887665 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc200104750 0x7fc2001179a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:56:07.602 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:07.599+0000 7fc204b60640 1 -- 172.21.15.156:0/1834887665 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1fc005040 con 0x7fc200117ee0 2024-07-16T09:56:07.602 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:07.599+0000 7fc204b60640 1 --2- 172.21.15.156:0/1834887665 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc200117ee0 0x7fc200118d50 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fc1f0002a80 tx=0x7fc1f0004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:56:07.602 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:07.599+0000 7fc1f5ffb640 1 -- 172.21.15.156:0/1834887665 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc1f0007c30 con 0x7fc200117ee0 2024-07-16T09:56:07.602 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:07.599+0000 7fc1f5ffb640 1 -- 172.21.15.156:0/1834887665 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc1f000f040 con 0x7fc200117ee0 2024-07-16T09:56:07.602 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:07.599+0000 7fc2075c5640 1 -- 172.21.15.156:0/1834887665 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc200119350 con 0x7fc200117ee0 2024-07-16T09:56:07.602 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:07.599+0000 7fc1f5ffb640 1 -- 172.21.15.156:0/1834887665 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc1f00095f0 con 0x7fc200117ee0 2024-07-16T09:56:07.602 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:07.600+0000 7fc2075c5640 1 -- 172.21.15.156:0/1834887665 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc200119870 con 0x7fc200117ee0 2024-07-16T09:56:07.603 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:07.602+0000 7fc2075c5640 1 -- 172.21.15.156:0/1834887665 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc20006cfc0 con 0x7fc200117ee0 2024-07-16T09:56:07.603 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:07.602+0000 7fc1f5ffb640 1 -- 172.21.15.156:0/1834887665 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 25) v1 ==== 88617+0+0 (secure 0 0 0) 0x7fc1f0009750 con 0x7fc200117ee0 2024-07-16T09:56:07.604 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:07.602+0000 7fc1f5ffb640 1 --2- 172.21.15.156:0/1834887665 >> [v2:172.21.15.176:6828/673792884,v1:172.21.15.176:6829/673792884] conn(0x7fc1ec06a6d0 0x7fc1ec06cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:56:07.604 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:07.603+0000 7fc1f5ffb640 1 -- 172.21.15.156:0/1834887665 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5722+0+0 (secure 0 0 0) 0x7fc1f0088900 con 0x7fc200117ee0 2024-07-16T09:56:07.604 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:07.603+0000 7fc205361640 1 --2- 172.21.15.156:0/1834887665 >> [v2:172.21.15.176:6828/673792884,v1:172.21.15.176:6829/673792884] conn(0x7fc1ec06a6d0 0x7fc1ec06cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:56:07.605 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:07.604+0000 7fc205361640 1 --2- 172.21.15.156:0/1834887665 >> [v2:172.21.15.176:6828/673792884,v1:172.21.15.176:6829/673792884] conn(0x7fc1ec06a6d0 0x7fc1ec06cb80 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fc1fc00b020 tx=0x7fc1fc035040 comp rx=0 tx=0).ready entity=mgr.24377 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:56:07.611 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:07.609+0000 7fc1f5ffb640 1 -- 172.21.15.156:0/1834887665 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185936 (secure 0 0 0) 0x7fc1f004f660 con 0x7fc200117ee0 2024-07-16T09:56:08.025 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:08.024+0000 7fc2075c5640 1 -- 172.21.15.156:0/1834887665 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fc20010c5b0 con 0x7fc200117ee0 2024-07-16T09:56:08.026 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:08.025+0000 7fc1f5ffb640 1 -- 172.21.15.156:0/1834887665 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+849 (secure 0 0 0) 0x7fc1f004f660 con 0x7fc200117ee0 2024-07-16T09:56:08.026 INFO:teuthology.orchestra.run.smithi156.stdout:{ 2024-07-16T09:56:08.026 INFO:teuthology.orchestra.run.smithi156.stdout: "mon": { 2024-07-16T09:56:08.026 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph version 17.2.7-1104-ge6231b04 (e6231b04b52ebf3b9b31803282dd3074439e51ac) quincy (stable)": 2 2024-07-16T09:56:08.026 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-07-16T09:56:08.026 INFO:teuthology.orchestra.run.smithi156.stdout: "mgr": { 2024-07-16T09:56:08.026 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph version 17.2.7-1104-ge6231b04 (e6231b04b52ebf3b9b31803282dd3074439e51ac) quincy (stable)": 1, 2024-07-16T09:56:08.026 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph version 19.0.0-3495-ga6ca461b (a6ca461be7cc1aff3658c2c6df4cff975a24449a) squid (rc)": 1 2024-07-16T09:56:08.026 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-07-16T09:56:08.027 INFO:teuthology.orchestra.run.smithi156.stdout: "osd": { 2024-07-16T09:56:08.027 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph version 17.2.7-1104-ge6231b04 (e6231b04b52ebf3b9b31803282dd3074439e51ac) quincy (stable)": 6 2024-07-16T09:56:08.027 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-07-16T09:56:08.027 INFO:teuthology.orchestra.run.smithi156.stdout: "mds": { 2024-07-16T09:56:08.027 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph version 17.2.7-1104-ge6231b04 (e6231b04b52ebf3b9b31803282dd3074439e51ac) quincy (stable)": 4 2024-07-16T09:56:08.027 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-07-16T09:56:08.027 INFO:teuthology.orchestra.run.smithi156.stdout: "overall": { 2024-07-16T09:56:08.027 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph version 17.2.7-1104-ge6231b04 (e6231b04b52ebf3b9b31803282dd3074439e51ac) quincy (stable)": 13, 2024-07-16T09:56:08.027 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph version 19.0.0-3495-ga6ca461b (a6ca461be7cc1aff3658c2c6df4cff975a24449a) squid (rc)": 1 2024-07-16T09:56:08.027 INFO:teuthology.orchestra.run.smithi156.stdout: } 2024-07-16T09:56:08.027 INFO:teuthology.orchestra.run.smithi156.stdout:} 2024-07-16T09:56:08.029 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:08.027+0000 7fc2075c5640 1 -- 172.21.15.156:0/1834887665 >> [v2:172.21.15.176:6828/673792884,v1:172.21.15.176:6829/673792884] conn(0x7fc1ec06a6d0 msgr2=0x7fc1ec06cb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:56:08.029 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:08.027+0000 7fc2075c5640 1 --2- 172.21.15.156:0/1834887665 >> [v2:172.21.15.176:6828/673792884,v1:172.21.15.176:6829/673792884] conn(0x7fc1ec06a6d0 0x7fc1ec06cb80 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fc1fc00b020 tx=0x7fc1fc035040 comp rx=0 tx=0).stop 2024-07-16T09:56:08.029 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:08.027+0000 7fc2075c5640 1 -- 172.21.15.156:0/1834887665 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc200117ee0 msgr2=0x7fc200118d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:56:08.029 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:08.027+0000 7fc2075c5640 1 --2- 172.21.15.156:0/1834887665 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc200117ee0 0x7fc200118d50 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fc1f0002a80 tx=0x7fc1f0004600 comp rx=0 tx=0).stop 2024-07-16T09:56:08.029 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:08.027+0000 7fc2075c5640 1 -- 172.21.15.156:0/1834887665 shutdown_connections 2024-07-16T09:56:08.029 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:08.027+0000 7fc2075c5640 1 --2- 172.21.15.156:0/1834887665 >> [v2:172.21.15.176:6828/673792884,v1:172.21.15.176:6829/673792884] conn(0x7fc1ec06a6d0 0x7fc1ec06cb80 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:56:08.029 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:08.027+0000 7fc2075c5640 1 --2- 172.21.15.156:0/1834887665 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc200117ee0 0x7fc200118d50 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:56:08.029 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:08.027+0000 7fc2075c5640 1 --2- 172.21.15.156:0/1834887665 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc200104750 0x7fc2001179a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:56:08.029 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:08.027+0000 7fc2075c5640 1 -- 172.21.15.156:0/1834887665 >> 172.21.15.156:0/1834887665 conn(0x7fc200069370 msgr2=0x7fc20006a990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:56:08.029 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:08.028+0000 7fc2075c5640 1 -- 172.21.15.156:0/1834887665 shutdown_connections 2024-07-16T09:56:08.030 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:08.028+0000 7fc2075c5640 1 -- 172.21.15.156:0/1834887665 wait complete. 2024-07-16T09:56:08.131 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:08.129+0000 7f3c0a738640 1 -- 172.21.15.156:0/642512797 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3c040fa9c0 msgr2=0x7f3c040fadd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:56:08.131 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:08.129+0000 7f3c0a738640 1 --2- 172.21.15.156:0/642512797 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3c040fa9c0 0x7f3c040fadd0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f3bf40049c0 tx=0x7f3bf400aeb0 comp rx=0 tx=0).stop 2024-07-16T09:56:08.131 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:08.129+0000 7f3c0a738640 1 -- 172.21.15.156:0/642512797 shutdown_connections 2024-07-16T09:56:08.131 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:08.129+0000 7f3c0a738640 1 --2- 172.21.15.156:0/642512797 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3c040fbc90 0x7f3c040fc100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:56:08.131 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:08.129+0000 7f3c0a738640 1 --2- 172.21.15.156:0/642512797 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3c040fa9c0 0x7f3c040fadd0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:56:08.131 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:08.129+0000 7f3c0a738640 1 -- 172.21.15.156:0/642512797 >> 172.21.15.156:0/642512797 conn(0x7f3c040f61a0 msgr2=0x7f3c040f85b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:56:08.131 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:08.129+0000 7f3c0a738640 1 -- 172.21.15.156:0/642512797 shutdown_connections 2024-07-16T09:56:08.131 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:08.129+0000 7f3c0a738640 1 -- 172.21.15.156:0/642512797 wait complete. 2024-07-16T09:56:08.131 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:08.129+0000 7f3c0a738640 1 Processor -- start 2024-07-16T09:56:08.131 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:08.130+0000 7f3c0a738640 1 -- start start 2024-07-16T09:56:08.133 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:08.130+0000 7f3c0a738640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3c040fa9c0 0x7f3c040feae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:56:08.133 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:08.130+0000 7f3c0a738640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3c040fbc90 0x7f3c040fcfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:56:08.133 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:08.130+0000 7f3c0a738640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c040fd560 con 0x7f3c040fa9c0 2024-07-16T09:56:08.133 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:08.130+0000 7f3c0a738640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c040fd6a0 con 0x7f3c040fbc90 2024-07-16T09:56:08.133 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:08.130+0000 7f3c03fff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3c040fa9c0 0x7f3c040feae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:56:08.133 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:08.130+0000 7f3c037fe640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3c040fbc90 0x7f3c040fcfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:56:08.133 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:08.130+0000 7f3c037fe640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3c040fbc90 0x7f3c040fcfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.156:37116/0 (socket says 172.21.15.156:37116) 2024-07-16T09:56:08.133 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:08.130+0000 7f3c037fe640 1 -- 172.21.15.156:0/3179148400 learned_addr learned my addr 172.21.15.156:0/3179148400 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T09:56:08.133 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:08.130+0000 7f3c03fff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3c040fa9c0 0x7f3c040feae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:57040/0 (socket says 172.21.15.156:57040) 2024-07-16T09:56:08.133 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:08.130+0000 7f3c03fff640 1 -- 172.21.15.156:0/3179148400 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3c040fbc90 msgr2=0x7f3c040fcfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:56:08.134 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:08.130+0000 7f3c03fff640 1 --2- 172.21.15.156:0/3179148400 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3c040fbc90 0x7f3c040fcfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:56:08.134 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:08.130+0000 7f3c03fff640 1 -- 172.21.15.156:0/3179148400 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bf4005040 con 0x7f3c040fa9c0 2024-07-16T09:56:08.134 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:08.130+0000 7f3c037fe640 1 --2- 172.21.15.156:0/3179148400 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3c040fbc90 0x7f3c040fcfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-16T09:56:08.134 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:08.131+0000 7f3c03fff640 1 --2- 172.21.15.156:0/3179148400 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3c040fa9c0 0x7f3c040feae0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f3bf4000c00 tx=0x7f3bf400aeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:56:08.134 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:08.131+0000 7f3c00ff9640 1 -- 172.21.15.156:0/3179148400 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3bf4033780 con 0x7f3c040fa9c0 2024-07-16T09:56:08.134 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:08.131+0000 7f3c0a738640 1 -- 172.21.15.156:0/3179148400 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c040fd920 con 0x7f3c040fa9c0 2024-07-16T09:56:08.134 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:08.131+0000 7f3c00ff9640 1 -- 172.21.15.156:0/3179148400 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3bf4033db0 con 0x7f3c040fa9c0 2024-07-16T09:56:08.134 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:08.131+0000 7f3c0a738640 1 -- 172.21.15.156:0/3179148400 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c04115070 con 0x7f3c040fa9c0 2024-07-16T09:56:08.134 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:08.132+0000 7f3c00ff9640 1 -- 172.21.15.156:0/3179148400 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3bf4034cc0 con 0x7f3c040fa9c0 2024-07-16T09:56:08.135 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:08.132+0000 7f3c0a738640 1 -- 172.21.15.156:0/3179148400 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c040f9e20 con 0x7f3c040fa9c0 2024-07-16T09:56:08.135 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:08.133+0000 7f3c00ff9640 1 -- 172.21.15.156:0/3179148400 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 25) v1 ==== 88617+0+0 (secure 0 0 0) 0x7f3bf4031020 con 0x7f3c040fa9c0 2024-07-16T09:56:08.135 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:08.134+0000 7f3c00ff9640 1 --2- 172.21.15.156:0/3179148400 >> [v2:172.21.15.176:6828/673792884,v1:172.21.15.176:6829/673792884] conn(0x7f3bec06a6d0 0x7f3bec06cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:56:08.135 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:08.134+0000 7f3c00ff9640 1 -- 172.21.15.156:0/3179148400 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5722+0+0 (secure 0 0 0) 0x7f3bf40a8e10 con 0x7f3c040fa9c0 2024-07-16T09:56:08.135 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:08.134+0000 7f3c037fe640 1 --2- 172.21.15.156:0/3179148400 >> [v2:172.21.15.176:6828/673792884,v1:172.21.15.176:6829/673792884] conn(0x7f3bec06a6d0 0x7f3bec06cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:56:08.136 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:08.134+0000 7f3c037fe640 1 --2- 172.21.15.156:0/3179148400 >> [v2:172.21.15.176:6828/673792884,v1:172.21.15.176:6829/673792884] conn(0x7f3bec06a6d0 0x7f3bec06cb80 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f3bf8007e60 tx=0x7f3bf800b040 comp rx=0 tx=0).ready entity=mgr.24377 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:56:08.140 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:08.139+0000 7f3c00ff9640 1 -- 172.21.15.156:0/3179148400 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185936 (secure 0 0 0) 0x7f3bf4072ab0 con 0x7f3c040fa9c0 2024-07-16T09:56:08.440 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:08.439+0000 7f3c0a738640 1 -- 172.21.15.156:0/3179148400 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f3c04115260 con 0x7f3c040fa9c0 2024-07-16T09:56:08.441 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:08.439+0000 7f3c00ff9640 1 -- 172.21.15.156:0/3179148400 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 18 v18) v1 ==== 76+0+1869 (secure 0 0 0) 0x7f3bf4042760 con 0x7f3c040fa9c0 2024-07-16T09:56:08.441 INFO:teuthology.orchestra.run.smithi156.stdout:e18 2024-07-16T09:56:08.441 INFO:teuthology.orchestra.run.smithi156.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-07-16T09:56:08.441 INFO:teuthology.orchestra.run.smithi156.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-07-16T09:56:08.442 INFO:teuthology.orchestra.run.smithi156.stdout:legacy client fscid: 1 2024-07-16T09:56:08.442 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-07-16T09:56:08.442 INFO:teuthology.orchestra.run.smithi156.stdout:Filesystem 'cephfs' (1) 2024-07-16T09:56:08.442 INFO:teuthology.orchestra.run.smithi156.stdout:fs_name cephfs 2024-07-16T09:56:08.442 INFO:teuthology.orchestra.run.smithi156.stdout:epoch 18 2024-07-16T09:56:08.442 INFO:teuthology.orchestra.run.smithi156.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-07-16T09:56:08.442 INFO:teuthology.orchestra.run.smithi156.stdout:created 2024-07-16T09:52:55.214975+0000 2024-07-16T09:56:08.442 INFO:teuthology.orchestra.run.smithi156.stdout:modified 2024-07-16T09:55:41.914320+0000 2024-07-16T09:56:08.442 INFO:teuthology.orchestra.run.smithi156.stdout:tableserver 0 2024-07-16T09:56:08.442 INFO:teuthology.orchestra.run.smithi156.stdout:root 0 2024-07-16T09:56:08.442 INFO:teuthology.orchestra.run.smithi156.stdout:session_timeout 60 2024-07-16T09:56:08.442 INFO:teuthology.orchestra.run.smithi156.stdout:session_autoclose 300 2024-07-16T09:56:08.442 INFO:teuthology.orchestra.run.smithi156.stdout:max_file_size 1099511627776 2024-07-16T09:56:08.443 INFO:teuthology.orchestra.run.smithi156.stdout:required_client_features {} 2024-07-16T09:56:08.443 INFO:teuthology.orchestra.run.smithi156.stdout:last_failure 0 2024-07-16T09:56:08.443 INFO:teuthology.orchestra.run.smithi156.stdout:last_failure_osd_epoch 0 2024-07-16T09:56:08.443 INFO:teuthology.orchestra.run.smithi156.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-07-16T09:56:08.443 INFO:teuthology.orchestra.run.smithi156.stdout:max_mds 2 2024-07-16T09:56:08.443 INFO:teuthology.orchestra.run.smithi156.stdout:in 0,1 2024-07-16T09:56:08.443 INFO:teuthology.orchestra.run.smithi156.stdout:up {0=14458,1=24261} 2024-07-16T09:56:08.443 INFO:teuthology.orchestra.run.smithi156.stdout:failed 2024-07-16T09:56:08.443 INFO:teuthology.orchestra.run.smithi156.stdout:damaged 2024-07-16T09:56:08.443 INFO:teuthology.orchestra.run.smithi156.stdout:stopped 2024-07-16T09:56:08.443 INFO:teuthology.orchestra.run.smithi156.stdout:data_pools [3] 2024-07-16T09:56:08.443 INFO:teuthology.orchestra.run.smithi156.stdout:metadata_pool 2 2024-07-16T09:56:08.443 INFO:teuthology.orchestra.run.smithi156.stdout:inline_data disabled 2024-07-16T09:56:08.444 INFO:teuthology.orchestra.run.smithi156.stdout:balancer 2024-07-16T09:56:08.444 INFO:teuthology.orchestra.run.smithi156.stdout:standby_count_wanted 1 2024-07-16T09:56:08.444 INFO:teuthology.orchestra.run.smithi156.stdout:[mds.cephfs.smithi156.rdibys{0:14458} state up:active seq 4 export targets 1 join_fscid=1 addr [v2:172.21.15.156:6826/1080176013,v1:172.21.15.156:6827/1080176013] compat {c=[1],r=[1],i=[7ff]}] 2024-07-16T09:56:08.444 INFO:teuthology.orchestra.run.smithi156.stdout:[mds.cephfs.smithi156.evwwaz{0:14468} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.156:6828/560329502,v1:172.21.15.156:6829/560329502] compat {c=[1],r=[1],i=[7ff]}] 2024-07-16T09:56:08.444 INFO:teuthology.orchestra.run.smithi156.stdout:[mds.cephfs.smithi176.dvvbud{1:24261} state up:active seq 4 export targets 0 join_fscid=1 addr [v2:172.21.15.176:6824/1577019287,v1:172.21.15.176:6825/1577019287] compat {c=[1],r=[1],i=[7ff]}] 2024-07-16T09:56:08.444 INFO:teuthology.orchestra.run.smithi156.stdout:[mds.cephfs.smithi176.wahbsi{1:24273} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.176:6826/2196775412,v1:172.21.15.176:6827/2196775412] compat {c=[1],r=[1],i=[7ff]}] 2024-07-16T09:56:08.444 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-07-16T09:56:08.444 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-07-16T09:56:08.445 INFO:teuthology.orchestra.run.smithi156.stderr:dumped fsmap epoch 18 2024-07-16T09:56:08.445 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:08.442+0000 7f3c0a738640 1 -- 172.21.15.156:0/3179148400 >> [v2:172.21.15.176:6828/673792884,v1:172.21.15.176:6829/673792884] conn(0x7f3bec06a6d0 msgr2=0x7f3bec06cb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:56:08.445 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:08.442+0000 7f3c0a738640 1 --2- 172.21.15.156:0/3179148400 >> [v2:172.21.15.176:6828/673792884,v1:172.21.15.176:6829/673792884] conn(0x7f3bec06a6d0 0x7f3bec06cb80 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f3bf8007e60 tx=0x7f3bf800b040 comp rx=0 tx=0).stop 2024-07-16T09:56:08.445 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:08.442+0000 7f3c0a738640 1 -- 172.21.15.156:0/3179148400 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3c040fa9c0 msgr2=0x7f3c040feae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:56:08.445 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:08.442+0000 7f3c0a738640 1 --2- 172.21.15.156:0/3179148400 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3c040fa9c0 0x7f3c040feae0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f3bf4000c00 tx=0x7f3bf400aeb0 comp rx=0 tx=0).stop 2024-07-16T09:56:08.445 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:08.442+0000 7f3c0a738640 1 -- 172.21.15.156:0/3179148400 shutdown_connections 2024-07-16T09:56:08.445 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:08.442+0000 7f3c0a738640 1 --2- 172.21.15.156:0/3179148400 >> [v2:172.21.15.176:6828/673792884,v1:172.21.15.176:6829/673792884] conn(0x7f3bec06a6d0 0x7f3bec06cb80 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:56:08.445 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:08.442+0000 7f3c0a738640 1 --2- 172.21.15.156:0/3179148400 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3c040fbc90 0x7f3c040fcfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:56:08.445 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:08.442+0000 7f3c0a738640 1 --2- 172.21.15.156:0/3179148400 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3c040fa9c0 0x7f3c040feae0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:56:08.445 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:08.442+0000 7f3c0a738640 1 -- 172.21.15.156:0/3179148400 >> 172.21.15.156:0/3179148400 conn(0x7f3c040f61a0 msgr2=0x7f3c040f7840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:56:08.446 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:08.442+0000 7f3c0a738640 1 -- 172.21.15.156:0/3179148400 shutdown_connections 2024-07-16T09:56:08.446 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:08.442+0000 7f3c0a738640 1 -- 172.21.15.156:0/3179148400 wait complete. 2024-07-16T09:56:08.542 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:08.540+0000 7fd0b5083640 1 -- 172.21.15.156:0/2750446123 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd0b00fe390 msgr2=0x7fd0b00fa4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:56:08.542 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:08.540+0000 7fd0b5083640 1 --2- 172.21.15.156:0/2750446123 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd0b00fe390 0x7fd0b00fa4f0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fd0a40048d0 tx=0x7fd0a4009da0 comp rx=0 tx=0).stop 2024-07-16T09:56:08.542 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:08.540+0000 7fd0b5083640 1 -- 172.21.15.156:0/2750446123 shutdown_connections 2024-07-16T09:56:08.542 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:08.540+0000 7fd0b5083640 1 --2- 172.21.15.156:0/2750446123 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd0b00fe390 0x7fd0b00fa4f0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:56:08.542 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:08.540+0000 7fd0b5083640 1 --2- 172.21.15.156:0/2750446123 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd0b00fda40 0x7fd0b00fde50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:56:08.542 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:08.540+0000 7fd0b5083640 1 -- 172.21.15.156:0/2750446123 >> 172.21.15.156:0/2750446123 conn(0x7fd0b00f61a0 msgr2=0x7fd0b00f85b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:56:08.543 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:08.540+0000 7fd0b5083640 1 -- 172.21.15.156:0/2750446123 shutdown_connections 2024-07-16T09:56:08.543 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:08.540+0000 7fd0b5083640 1 -- 172.21.15.156:0/2750446123 wait complete. 2024-07-16T09:56:08.544 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:08.541+0000 7fd0b5083640 1 Processor -- start 2024-07-16T09:56:08.544 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:08.541+0000 7fd0b5083640 1 -- start start 2024-07-16T09:56:08.544 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:08.541+0000 7fd0b5083640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd0b00fda40 0x7fd0b00fd230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:56:08.544 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:08.541+0000 7fd0b5083640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd0b00fe390 0x7fd0b00fb720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:56:08.544 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:08.541+0000 7fd0b5083640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0b00fbcb0 con 0x7fd0b00fe390 2024-07-16T09:56:08.545 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:08.541+0000 7fd0b5083640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0b00fbdf0 con 0x7fd0b00fda40 2024-07-16T09:56:08.545 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:08.542+0000 7fd0ae59c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd0b00fe390 0x7fd0b00fb720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:56:08.545 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:08.542+0000 7fd0aed9d640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd0b00fda40 0x7fd0b00fd230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:56:08.545 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:08.542+0000 7fd0ae59c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd0b00fe390 0x7fd0b00fb720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:57060/0 (socket says 172.21.15.156:57060) 2024-07-16T09:56:08.545 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:08.542+0000 7fd0ae59c640 1 -- 172.21.15.156:0/1369743345 learned_addr learned my addr 172.21.15.156:0/1369743345 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T09:56:08.545 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:08.542+0000 7fd0ae59c640 1 -- 172.21.15.156:0/1369743345 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd0b00fda40 msgr2=0x7fd0b00fd230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:56:08.545 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:08.542+0000 7fd0ae59c640 1 --2- 172.21.15.156:0/1369743345 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd0b00fda40 0x7fd0b00fd230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:56:08.545 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:08.542+0000 7fd0ae59c640 1 -- 172.21.15.156:0/1369743345 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0a4005040 con 0x7fd0b00fe390 2024-07-16T09:56:08.545 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:08.542+0000 7fd0aed9d640 1 --2- 172.21.15.156:0/1369743345 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd0b00fda40 0x7fd0b00fd230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-16T09:56:08.545 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:08.542+0000 7fd0ae59c640 1 --2- 172.21.15.156:0/1369743345 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd0b00fe390 0x7fd0b00fb720 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fd0a4000c00 tx=0x7fd0a4009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:56:08.545 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:08.542+0000 7fd0977fe640 1 -- 172.21.15.156:0/1369743345 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd0a4038070 con 0x7fd0b00fe390 2024-07-16T09:56:08.545 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:08.543+0000 7fd0977fe640 1 -- 172.21.15.156:0/1369743345 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd0a4032970 con 0x7fd0b00fe390 2024-07-16T09:56:08.546 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:08.543+0000 7fd0977fe640 1 -- 172.21.15.156:0/1369743345 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd0a40338e0 con 0x7fd0b00fe390 2024-07-16T09:56:08.546 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:08.543+0000 7fd0b5083640 1 -- 172.21.15.156:0/1369743345 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0b00fc070 con 0x7fd0b00fe390 2024-07-16T09:56:08.546 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:08.543+0000 7fd0b5083640 1 -- 172.21.15.156:0/1369743345 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0b0115020 con 0x7fd0b00fe390 2024-07-16T09:56:08.546 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:08.544+0000 7fd0b5083640 1 -- 172.21.15.156:0/1369743345 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0b00f9e20 con 0x7fd0b00fe390 2024-07-16T09:56:08.547 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:08.545+0000 7fd0977fe640 1 -- 172.21.15.156:0/1369743345 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 25) v1 ==== 88617+0+0 (secure 0 0 0) 0x7fd0a4030020 con 0x7fd0b00fe390 2024-07-16T09:56:08.547 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:08.545+0000 7fd0977fe640 1 --2- 172.21.15.156:0/1369743345 >> [v2:172.21.15.176:6828/673792884,v1:172.21.15.176:6829/673792884] conn(0x7fd09806a6d0 0x7fd09806cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:56:08.547 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:08.545+0000 7fd0977fe640 1 -- 172.21.15.156:0/1369743345 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5722+0+0 (secure 0 0 0) 0x7fd0a40a8d00 con 0x7fd0b00fe390 2024-07-16T09:56:08.547 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:08.545+0000 7fd0aed9d640 1 --2- 172.21.15.156:0/1369743345 >> [v2:172.21.15.176:6828/673792884,v1:172.21.15.176:6829/673792884] conn(0x7fd09806a6d0 0x7fd09806cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:56:08.547 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:08.546+0000 7fd0aed9d640 1 --2- 172.21.15.156:0/1369743345 >> [v2:172.21.15.176:6828/673792884,v1:172.21.15.176:6829/673792884] conn(0x7fd09806a6d0 0x7fd09806cb80 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fd0b00fcfe0 tx=0x7fd0a000b040 comp rx=0 tx=0).ready entity=mgr.24377 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:56:08.553 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:08.552+0000 7fd0977fe640 1 -- 172.21.15.156:0/1369743345 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185936 (secure 0 0 0) 0x7fd0a40ac030 con 0x7fd0b00fe390 2024-07-16T09:56:08.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:56:08 smithi156 ceph-mon[29302]: from='client.24449 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-16T09:56:08.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:56:08 smithi156 ceph-mon[29302]: from='client.14672 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-16T09:56:08.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:56:08 smithi156 ceph-mon[29302]: pgmap v12: 65 pgs: 65 active+clean; 1.5 GiB data, 6.3 GiB used, 530 GiB / 536 GiB avail; 12 MiB/s rd, 33 MiB/s wr, 260 op/s 2024-07-16T09:56:08.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:56:08 smithi156 ceph-mon[29302]: from='client.24457 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-07-16T09:56:08.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:56:08 smithi156 ceph-mon[29302]: from='mgr.24377 ' entity='mgr.smithi176.dqbrvx' 2024-07-16T09:56:08.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:56:08 smithi156 ceph-mon[29302]: from='mgr.24377 ' entity='mgr.smithi176.dqbrvx' 2024-07-16T09:56:08.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:56:08 smithi156 ceph-mon[29302]: from='client.? 172.21.15.156:0/1834887665' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T09:56:08.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:56:08 smithi176 ceph-mon[36626]: from='client.24449 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-16T09:56:08.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:56:08 smithi176 ceph-mon[36626]: from='client.14672 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-16T09:56:08.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:56:08 smithi176 ceph-mon[36626]: pgmap v12: 65 pgs: 65 active+clean; 1.5 GiB data, 6.3 GiB used, 530 GiB / 536 GiB avail; 12 MiB/s rd, 33 MiB/s wr, 260 op/s 2024-07-16T09:56:08.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:56:08 smithi176 ceph-mon[36626]: from='client.24457 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-07-16T09:56:08.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:56:08 smithi176 ceph-mon[36626]: from='mgr.24377 ' entity='mgr.smithi176.dqbrvx' 2024-07-16T09:56:08.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:56:08 smithi176 ceph-mon[36626]: from='mgr.24377 ' entity='mgr.smithi176.dqbrvx' 2024-07-16T09:56:08.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:56:08 smithi176 ceph-mon[36626]: from='client.? 172.21.15.156:0/1834887665' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T09:56:08.889 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:08.887+0000 7fd0b5083640 1 -- 172.21.15.156:0/1369743345 --> [v2:172.21.15.176:6828/673792884,v1:172.21.15.176:6829/673792884] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd0b0101b70 con 0x7fd09806a6d0 2024-07-16T09:56:08.889 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:08.888+0000 7fd0977fe640 1 -- 172.21.15.156:0/1369743345 <== mgr.24377 v2:172.21.15.176:6828/673792884 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+318 (secure 0 0 0) 0x7fd0b0101b70 con 0x7fd09806a6d0 2024-07-16T09:56:08.890 INFO:teuthology.orchestra.run.smithi156.stdout:{ 2024-07-16T09:56:08.890 INFO:teuthology.orchestra.run.smithi156.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:c15ec3aa8e6c0a9c12ee24ff79f3e7dd4d01b3d1a702dfeb7d7fff9b0811a393", 2024-07-16T09:56:08.890 INFO:teuthology.orchestra.run.smithi156.stdout: "in_progress": true, 2024-07-16T09:56:08.890 INFO:teuthology.orchestra.run.smithi156.stdout: "which": "Upgrading all daemon types on all hosts", 2024-07-16T09:56:08.890 INFO:teuthology.orchestra.run.smithi156.stdout: "services_complete": [], 2024-07-16T09:56:08.890 INFO:teuthology.orchestra.run.smithi156.stdout: "progress": "1/21 daemons upgraded", 2024-07-16T09:56:08.890 INFO:teuthology.orchestra.run.smithi156.stdout: "message": "", 2024-07-16T09:56:08.890 INFO:teuthology.orchestra.run.smithi156.stdout: "is_paused": false 2024-07-16T09:56:08.890 INFO:teuthology.orchestra.run.smithi156.stdout:} 2024-07-16T09:56:08.892 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:08.891+0000 7fd0b5083640 1 -- 172.21.15.156:0/1369743345 >> [v2:172.21.15.176:6828/673792884,v1:172.21.15.176:6829/673792884] conn(0x7fd09806a6d0 msgr2=0x7fd09806cb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:56:08.893 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:08.891+0000 7fd0b5083640 1 --2- 172.21.15.156:0/1369743345 >> [v2:172.21.15.176:6828/673792884,v1:172.21.15.176:6829/673792884] conn(0x7fd09806a6d0 0x7fd09806cb80 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fd0b00fcfe0 tx=0x7fd0a000b040 comp rx=0 tx=0).stop 2024-07-16T09:56:08.893 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:08.891+0000 7fd0b5083640 1 -- 172.21.15.156:0/1369743345 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd0b00fe390 msgr2=0x7fd0b00fb720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:56:08.893 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:08.891+0000 7fd0b5083640 1 --2- 172.21.15.156:0/1369743345 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd0b00fe390 0x7fd0b00fb720 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fd0a4000c00 tx=0x7fd0a4009da0 comp rx=0 tx=0).stop 2024-07-16T09:56:08.893 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:08.891+0000 7fd0b5083640 1 -- 172.21.15.156:0/1369743345 shutdown_connections 2024-07-16T09:56:08.893 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:08.891+0000 7fd0b5083640 1 --2- 172.21.15.156:0/1369743345 >> [v2:172.21.15.176:6828/673792884,v1:172.21.15.176:6829/673792884] conn(0x7fd09806a6d0 0x7fd09806cb80 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:56:08.893 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:08.891+0000 7fd0b5083640 1 --2- 172.21.15.156:0/1369743345 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd0b00fe390 0x7fd0b00fb720 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:56:08.893 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:08.891+0000 7fd0b5083640 1 --2- 172.21.15.156:0/1369743345 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd0b00fda40 0x7fd0b00fd230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:56:08.893 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:08.891+0000 7fd0b5083640 1 -- 172.21.15.156:0/1369743345 >> 172.21.15.156:0/1369743345 conn(0x7fd0b00f61a0 msgr2=0x7fd0b00f7890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:56:08.893 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:08.891+0000 7fd0b5083640 1 -- 172.21.15.156:0/1369743345 shutdown_connections 2024-07-16T09:56:08.893 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:08.891+0000 7fd0b5083640 1 -- 172.21.15.156:0/1369743345 wait complete. 2024-07-16T09:56:08.982 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:08.980+0000 7f6ac4782640 1 -- 172.21.15.156:0/3221250086 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6abc0fc510 msgr2=0x7f6abc0fc980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:56:08.982 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:08.980+0000 7f6ac4782640 1 --2- 172.21.15.156:0/3221250086 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6abc0fc510 0x7f6abc0fc980 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f6aac0048d0 tx=0x7f6aac009da0 comp rx=0 tx=0).stop 2024-07-16T09:56:08.982 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:08.980+0000 7f6ac4782640 1 -- 172.21.15.156:0/3221250086 shutdown_connections 2024-07-16T09:56:08.982 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:08.980+0000 7f6ac4782640 1 --2- 172.21.15.156:0/3221250086 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6abc0fc510 0x7f6abc0fc980 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:56:08.982 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:08.980+0000 7f6ac4782640 1 --2- 172.21.15.156:0/3221250086 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6abc0fb240 0x7f6abc0fb650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:56:08.982 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:08.980+0000 7f6ac4782640 1 -- 172.21.15.156:0/3221250086 >> 172.21.15.156:0/3221250086 conn(0x7f6abc0f6a20 msgr2=0x7f6abc0f8e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:56:08.982 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:08.980+0000 7f6ac4782640 1 -- 172.21.15.156:0/3221250086 shutdown_connections 2024-07-16T09:56:08.982 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:08.980+0000 7f6ac4782640 1 -- 172.21.15.156:0/3221250086 wait complete. 2024-07-16T09:56:08.983 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:08.981+0000 7f6ac4782640 1 Processor -- start 2024-07-16T09:56:08.983 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:08.981+0000 7f6ac4782640 1 -- start start 2024-07-16T09:56:08.983 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:08.981+0000 7f6ac4782640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6abc0fb240 0x7f6abc10bea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:56:08.984 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:08.981+0000 7f6ac4782640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6abc0fc510 0x7f6abc10c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:56:08.984 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:08.981+0000 7f6ac4782640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6abc10ca30 con 0x7f6abc0fc510 2024-07-16T09:56:08.984 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:08.981+0000 7f6ac4782640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6abc10d250 con 0x7f6abc0fb240 2024-07-16T09:56:08.984 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:08.981+0000 7f6ac1d1d640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6abc0fc510 0x7f6abc10c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:56:08.984 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:08.981+0000 7f6ac1d1d640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6abc0fc510 0x7f6abc10c3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:57084/0 (socket says 172.21.15.156:57084) 2024-07-16T09:56:08.984 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:08.981+0000 7f6ac1d1d640 1 -- 172.21.15.156:0/3942763935 learned_addr learned my addr 172.21.15.156:0/3942763935 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T09:56:08.984 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:08.981+0000 7f6ac251e640 1 --2- 172.21.15.156:0/3942763935 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6abc0fb240 0x7f6abc10bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:56:08.984 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:08.982+0000 7f6ac1d1d640 1 -- 172.21.15.156:0/3942763935 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6abc0fb240 msgr2=0x7f6abc10bea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:56:08.984 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:08.982+0000 7f6ac1d1d640 1 --2- 172.21.15.156:0/3942763935 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6abc0fb240 0x7f6abc10bea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:56:08.984 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:08.982+0000 7f6ac1d1d640 1 -- 172.21.15.156:0/3942763935 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6aac005040 con 0x7f6abc0fc510 2024-07-16T09:56:08.984 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:08.982+0000 7f6ac251e640 1 --2- 172.21.15.156:0/3942763935 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6abc0fb240 0x7f6abc10bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-16T09:56:08.984 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:08.982+0000 7f6ac1d1d640 1 --2- 172.21.15.156:0/3942763935 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6abc0fc510 0x7f6abc10c3e0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f6aac0041d0 tx=0x7f6aac009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:56:08.985 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:08.982+0000 7f6ab2ffd640 1 -- 172.21.15.156:0/3942763935 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6aac03a070 con 0x7f6abc0fc510 2024-07-16T09:56:08.985 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:08.982+0000 7f6ac4782640 1 -- 172.21.15.156:0/3942763935 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6abc10d480 con 0x7f6abc0fc510 2024-07-16T09:56:08.985 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:08.982+0000 7f6ab2ffd640 1 -- 172.21.15.156:0/3942763935 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6aac0092b0 con 0x7f6abc0fc510 2024-07-16T09:56:08.985 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:08.983+0000 7f6ab2ffd640 1 -- 172.21.15.156:0/3942763935 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6aac032940 con 0x7f6abc0fc510 2024-07-16T09:56:08.985 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:08.983+0000 7f6ac4782640 1 -- 172.21.15.156:0/3942763935 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6abc10d9c0 con 0x7f6abc0fc510 2024-07-16T09:56:08.985 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:08.984+0000 7f6ac4782640 1 -- 172.21.15.156:0/3942763935 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6abc0fa6a0 con 0x7f6abc0fc510 2024-07-16T09:56:08.986 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:08.984+0000 7f6ab2ffd640 1 -- 172.21.15.156:0/3942763935 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 25) v1 ==== 88617+0+0 (secure 0 0 0) 0x7f6aac030020 con 0x7f6abc0fc510 2024-07-16T09:56:08.986 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:08.985+0000 7f6ab2ffd640 1 --2- 172.21.15.156:0/3942763935 >> [v2:172.21.15.176:6828/673792884,v1:172.21.15.176:6829/673792884] conn(0x7f6aa806a4b0 0x7f6aa806c960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:56:08.986 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:08.985+0000 7f6ab2ffd640 1 -- 172.21.15.156:0/3942763935 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5722+0+0 (secure 0 0 0) 0x7f6aac0aae30 con 0x7f6abc0fc510 2024-07-16T09:56:08.986 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:08.985+0000 7f6ac251e640 1 --2- 172.21.15.156:0/3942763935 >> [v2:172.21.15.176:6828/673792884,v1:172.21.15.176:6829/673792884] conn(0x7f6aa806a4b0 0x7f6aa806c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:56:08.987 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:08.986+0000 7f6ac251e640 1 --2- 172.21.15.156:0/3942763935 >> [v2:172.21.15.176:6828/673792884,v1:172.21.15.176:6829/673792884] conn(0x7f6aa806a4b0 0x7f6aa806c960 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f6ab8004fd0 tx=0x7f6ab800b040 comp rx=0 tx=0).ready entity=mgr.24377 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:56:08.991 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:08.990+0000 7f6ab2ffd640 1 -- 172.21.15.156:0/3942763935 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185936 (secure 0 0 0) 0x7f6aac074b00 con 0x7f6abc0fc510 2024-07-16T09:56:09.339 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:09.337+0000 7f6ac4782640 1 -- 172.21.15.156:0/3942763935 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f6abc103db0 con 0x7f6abc0fc510 2024-07-16T09:56:09.340 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:09.338+0000 7f6ab2ffd640 1 -- 172.21.15.156:0/3942763935 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7f6aac0781c0 con 0x7f6abc0fc510 2024-07-16T09:56:09.340 INFO:teuthology.orchestra.run.smithi156.stdout:HEALTH_OK 2024-07-16T09:56:09.342 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:09.340+0000 7f6ac4782640 1 -- 172.21.15.156:0/3942763935 >> [v2:172.21.15.176:6828/673792884,v1:172.21.15.176:6829/673792884] conn(0x7f6aa806a4b0 msgr2=0x7f6aa806c960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:56:09.342 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:09.340+0000 7f6ac4782640 1 --2- 172.21.15.156:0/3942763935 >> [v2:172.21.15.176:6828/673792884,v1:172.21.15.176:6829/673792884] conn(0x7f6aa806a4b0 0x7f6aa806c960 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f6ab8004fd0 tx=0x7f6ab800b040 comp rx=0 tx=0).stop 2024-07-16T09:56:09.342 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:09.340+0000 7f6ac4782640 1 -- 172.21.15.156:0/3942763935 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6abc0fc510 msgr2=0x7f6abc10c3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:56:09.342 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:09.340+0000 7f6ac4782640 1 --2- 172.21.15.156:0/3942763935 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6abc0fc510 0x7f6abc10c3e0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f6aac0041d0 tx=0x7f6aac009da0 comp rx=0 tx=0).stop 2024-07-16T09:56:09.342 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:09.341+0000 7f6ac4782640 1 -- 172.21.15.156:0/3942763935 shutdown_connections 2024-07-16T09:56:09.342 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:09.341+0000 7f6ac4782640 1 --2- 172.21.15.156:0/3942763935 >> [v2:172.21.15.176:6828/673792884,v1:172.21.15.176:6829/673792884] conn(0x7f6aa806a4b0 0x7f6aa806c960 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:56:09.342 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:09.341+0000 7f6ac4782640 1 --2- 172.21.15.156:0/3942763935 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6abc0fc510 0x7f6abc10c3e0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:56:09.343 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:09.341+0000 7f6ac4782640 1 --2- 172.21.15.156:0/3942763935 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6abc0fb240 0x7f6abc10bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:56:09.343 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:09.341+0000 7f6ac4782640 1 -- 172.21.15.156:0/3942763935 >> 172.21.15.156:0/3942763935 conn(0x7f6abc0f6a20 msgr2=0x7f6abc0f8140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:56:09.343 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:09.341+0000 7f6ac4782640 1 -- 172.21.15.156:0/3942763935 shutdown_connections 2024-07-16T09:56:09.343 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:09.341+0000 7f6ac4782640 1 -- 172.21.15.156:0/3942763935 wait complete. 2024-07-16T09:56:09.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:56:09 smithi156 ceph-mon[29302]: Reconfiguring alertmanager.smithi156 (dependencies changed)... 2024-07-16T09:56:09.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:56:09 smithi156 ceph-mon[29302]: Reconfiguring daemon alertmanager.smithi156 on smithi156 2024-07-16T09:56:09.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:56:09 smithi156 ceph-mon[29302]: from='client.? 172.21.15.156:0/3179148400' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-07-16T09:56:09.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:56:09 smithi156 ceph-mon[29302]: from='client.? 172.21.15.156:0/3942763935' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-07-16T09:56:09.843 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:56:09 smithi176 ceph-mon[36626]: Reconfiguring alertmanager.smithi156 (dependencies changed)... 2024-07-16T09:56:09.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:56:09 smithi176 ceph-mon[36626]: Reconfiguring daemon alertmanager.smithi156 on smithi156 2024-07-16T09:56:09.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:56:09 smithi176 ceph-mon[36626]: from='client.? 172.21.15.156:0/3179148400' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-07-16T09:56:09.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:56:09 smithi176 ceph-mon[36626]: from='client.? 172.21.15.156:0/3942763935' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-07-16T09:56:10.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:56:10 smithi156 ceph-mon[29302]: from='client.14688 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-16T09:56:10.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:56:10 smithi156 ceph-mon[29302]: pgmap v13: 65 pgs: 65 active+clean; 1.5 GiB data, 6.3 GiB used, 530 GiB / 536 GiB avail; 1.7 MiB/s rd, 4.1 MiB/s wr, 216 op/s 2024-07-16T09:56:10.843 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:56:10 smithi176 ceph-mon[36626]: from='client.14688 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-16T09:56:10.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:56:10 smithi176 ceph-mon[36626]: pgmap v13: 65 pgs: 65 active+clean; 1.5 GiB data, 6.3 GiB used, 530 GiB / 536 GiB avail; 1.7 MiB/s rd, 4.1 MiB/s wr, 216 op/s 2024-07-16T09:56:12.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:56:12 smithi156 ceph-mon[29302]: pgmap v14: 65 pgs: 65 active+clean; 280 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 2.6 MiB/s rd, 4.9 MiB/s wr, 377 op/s 2024-07-16T09:56:12.843 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:56:12 smithi176 ceph-mon[36626]: pgmap v14: 65 pgs: 65 active+clean; 280 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 2.6 MiB/s rd, 4.9 MiB/s wr, 377 op/s 2024-07-16T09:56:14.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:56:14 smithi176 ceph-mon[36626]: pgmap v15: 65 pgs: 65 active+clean; 280 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.8 MiB/s rd, 1.7 MiB/s wr, 273 op/s 2024-07-16T09:56:14.869 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:56:14 smithi156 ceph-mon[29302]: pgmap v15: 65 pgs: 65 active+clean; 280 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.8 MiB/s rd, 1.7 MiB/s wr, 273 op/s 2024-07-16T09:56:16.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:56:16 smithi176 ceph-mon[36626]: pgmap v16: 65 pgs: 65 active+clean; 286 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 2.6 MiB/s rd, 2.7 MiB/s wr, 505 op/s 2024-07-16T09:56:16.924 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:56:16 smithi156 ceph-mon[29302]: pgmap v16: 65 pgs: 65 active+clean; 286 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 2.6 MiB/s rd, 2.7 MiB/s wr, 505 op/s 2024-07-16T09:56:18.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:56:18 smithi156 ceph-mon[29302]: pgmap v17: 65 pgs: 65 active+clean; 285 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.7 MiB/s rd, 1.9 MiB/s wr, 459 op/s 2024-07-16T09:56:18.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:56:18 smithi156 ceph-mon[29302]: from='mgr.24377 ' entity='mgr.smithi176.dqbrvx' 2024-07-16T09:56:18.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:56:18 smithi156 ceph-mon[29302]: from='mgr.24377 ' entity='mgr.smithi176.dqbrvx' 2024-07-16T09:56:18.843 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:56:18 smithi176 ceph-mon[36626]: pgmap v17: 65 pgs: 65 active+clean; 285 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.7 MiB/s rd, 1.9 MiB/s wr, 459 op/s 2024-07-16T09:56:18.843 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:56:18 smithi176 ceph-mon[36626]: from='mgr.24377 ' entity='mgr.smithi176.dqbrvx' 2024-07-16T09:56:18.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:56:18 smithi176 ceph-mon[36626]: from='mgr.24377 ' entity='mgr.smithi176.dqbrvx' 2024-07-16T09:56:19.843 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:56:19 smithi176 ceph-mon[36626]: Reconfiguring grafana.smithi156 (dependencies changed)... 2024-07-16T09:56:19.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:56:19 smithi176 ceph-mon[36626]: Reconfiguring daemon grafana.smithi156 on smithi156 2024-07-16T09:56:19.924 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:56:19 smithi156 ceph-mon[29302]: Reconfiguring grafana.smithi156 (dependencies changed)... 2024-07-16T09:56:19.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:56:19 smithi156 ceph-mon[29302]: Reconfiguring daemon grafana.smithi156 on smithi156 2024-07-16T09:56:20.843 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:56:20 smithi176 ceph-mon[36626]: pgmap v18: 65 pgs: 65 active+clean; 285 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.7 MiB/s rd, 1.9 MiB/s wr, 435 op/s 2024-07-16T09:56:20.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:56:20 smithi176 ceph-mon[36626]: from='mgr.24377 172.21.15.176:0/1082993331' entity='mgr.smithi176.dqbrvx' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-16T09:56:20.924 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:56:20 smithi156 ceph-mon[29302]: pgmap v18: 65 pgs: 65 active+clean; 285 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.7 MiB/s rd, 1.9 MiB/s wr, 435 op/s 2024-07-16T09:56:20.924 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:56:20 smithi156 ceph-mon[29302]: from='mgr.24377 172.21.15.176:0/1082993331' entity='mgr.smithi176.dqbrvx' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-16T09:56:22.843 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:56:22 smithi176 ceph-mon[36626]: pgmap v19: 65 pgs: 65 active+clean; 290 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 2.6 MiB/s rd, 2.7 MiB/s wr, 670 op/s 2024-07-16T09:56:22.924 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:56:22 smithi156 ceph-mon[29302]: pgmap v19: 65 pgs: 65 active+clean; 290 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 2.6 MiB/s rd, 2.7 MiB/s wr, 670 op/s 2024-07-16T09:56:24.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:56:24 smithi156 ceph-mon[29302]: pgmap v20: 65 pgs: 65 active+clean; 290 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 509 op/s 2024-07-16T09:56:24.843 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:56:24 smithi176 ceph-mon[36626]: pgmap v20: 65 pgs: 65 active+clean; 290 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 509 op/s 2024-07-16T09:56:26.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:56:26 smithi156 ceph-mon[29302]: pgmap v21: 65 pgs: 65 active+clean; 279 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 2.2 MiB/s rd, 2.4 MiB/s wr, 713 op/s 2024-07-16T09:56:26.843 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:56:26 smithi176 ceph-mon[36626]: pgmap v21: 65 pgs: 65 active+clean; 279 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 2.2 MiB/s rd, 2.4 MiB/s wr, 713 op/s 2024-07-16T09:56:28.843 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:56:28 smithi176 ceph-mon[36626]: pgmap v22: 65 pgs: 65 active+clean; 276 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.7 MiB/s rd, 1.6 MiB/s wr, 532 op/s 2024-07-16T09:56:28.924 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:56:28 smithi156 ceph-mon[29302]: pgmap v22: 65 pgs: 65 active+clean; 276 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.7 MiB/s rd, 1.6 MiB/s wr, 532 op/s 2024-07-16T09:56:30.843 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:56:30 smithi176 ceph-mon[36626]: pgmap v23: 65 pgs: 65 active+clean; 276 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.7 MiB/s rd, 1.6 MiB/s wr, 488 op/s 2024-07-16T09:56:30.924 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:56:30 smithi156 ceph-mon[29302]: pgmap v23: 65 pgs: 65 active+clean; 276 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.7 MiB/s rd, 1.6 MiB/s wr, 488 op/s 2024-07-16T09:56:32.843 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:56:32 smithi176 ceph-mon[36626]: pgmap v24: 65 pgs: 65 active+clean; 270 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 2.6 MiB/s rd, 2.6 MiB/s wr, 710 op/s 2024-07-16T09:56:32.922 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:56:32 smithi156 ceph-mon[29302]: pgmap v24: 65 pgs: 65 active+clean; 270 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 2.6 MiB/s rd, 2.6 MiB/s wr, 710 op/s 2024-07-16T09:56:34.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:56:34 smithi176 ceph-mon[36626]: pgmap v25: 65 pgs: 65 active+clean; 270 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 476 op/s 2024-07-16T09:56:34.782 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:56:34 smithi156 ceph-mon[29302]: pgmap v25: 65 pgs: 65 active+clean; 270 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 476 op/s 2024-07-16T09:56:35.748 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:56:35 smithi176 ceph-mon[36626]: from='mgr.24377 172.21.15.176:0/1082993331' entity='mgr.smithi176.dqbrvx' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-16T09:56:35.748 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:56:35 smithi176 ceph-mon[36626]: from='mgr.24377 ' entity='mgr.smithi176.dqbrvx' 2024-07-16T09:56:35.748 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:56:35 smithi176 ceph-mon[36626]: from='mgr.24377 ' entity='mgr.smithi176.dqbrvx' 2024-07-16T09:56:35.784 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:56:35 smithi156 ceph-mon[29302]: from='mgr.24377 172.21.15.176:0/1082993331' entity='mgr.smithi176.dqbrvx' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-16T09:56:35.785 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:56:35 smithi156 ceph-mon[29302]: from='mgr.24377 ' entity='mgr.smithi176.dqbrvx' 2024-07-16T09:56:35.785 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:56:35 smithi156 ceph-mon[29302]: from='mgr.24377 ' entity='mgr.smithi176.dqbrvx' 2024-07-16T09:56:36.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:56:36 smithi156 ceph-mon[29302]: Reconfiguring prometheus.smithi156 (dependencies changed)... 2024-07-16T09:56:36.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:56:36 smithi156 ceph-mon[29302]: Reconfiguring daemon prometheus.smithi156 on smithi156 2024-07-16T09:56:36.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:56:36 smithi156 ceph-mon[29302]: pgmap v26: 65 pgs: 65 active+clean; 269 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 2.4 MiB/s rd, 2.5 MiB/s wr, 748 op/s 2024-07-16T09:56:36.843 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:56:36 smithi176 ceph-mon[36626]: Reconfiguring prometheus.smithi156 (dependencies changed)... 2024-07-16T09:56:36.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:56:36 smithi176 ceph-mon[36626]: Reconfiguring daemon prometheus.smithi156 on smithi156 2024-07-16T09:56:36.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:56:36 smithi176 ceph-mon[36626]: pgmap v26: 65 pgs: 65 active+clean; 269 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 2.4 MiB/s rd, 2.5 MiB/s wr, 748 op/s 2024-07-16T09:56:38.825 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:56:38 smithi156 ceph-mon[29302]: pgmap v27: 65 pgs: 65 active+clean; 269 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 2.0 MiB/s rd, 2.0 MiB/s wr, 544 op/s 2024-07-16T09:56:38.843 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:56:38 smithi176 ceph-mon[36626]: pgmap v27: 65 pgs: 65 active+clean; 269 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 2.0 MiB/s rd, 2.0 MiB/s wr, 544 op/s 2024-07-16T09:56:39.479 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:39.477+0000 7fda2effd640 1 -- 172.21.15.156:0/2948694941 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fda280329b0 con 0x7fda30102990 2024-07-16T09:56:39.480 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:39.478+0000 7fda3698e640 1 -- 172.21.15.156:0/2948694941 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fda30102990 msgr2=0x7fda30102da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:56:39.480 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:39.478+0000 7fda3698e640 1 --2- 172.21.15.156:0/2948694941 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fda30102990 0x7fda30102da0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fda28009c50 tx=0x7fda28004d10 comp rx=0 tx=0).stop 2024-07-16T09:56:39.480 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:39.478+0000 7fda3698e640 1 -- 172.21.15.156:0/2948694941 shutdown_connections 2024-07-16T09:56:39.480 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:39.478+0000 7fda3698e640 1 --2- 172.21.15.156:0/2948694941 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fda30103c60 0x7fda301040d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:56:39.480 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:39.478+0000 7fda3698e640 1 --2- 172.21.15.156:0/2948694941 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fda30102990 0x7fda30102da0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:56:39.480 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:39.478+0000 7fda3698e640 1 -- 172.21.15.156:0/2948694941 >> 172.21.15.156:0/2948694941 conn(0x7fda30071ba0 msgr2=0x7fda30073fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:56:39.481 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:39.478+0000 7fda3698e640 1 -- 172.21.15.156:0/2948694941 shutdown_connections 2024-07-16T09:56:39.481 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:39.478+0000 7fda3698e640 1 -- 172.21.15.156:0/2948694941 wait complete. 2024-07-16T09:56:39.481 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:39.479+0000 7fda3698e640 1 Processor -- start 2024-07-16T09:56:39.481 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:39.479+0000 7fda3698e640 1 -- start start 2024-07-16T09:56:39.481 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:39.480+0000 7fda3698e640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fda30102990 0x7fda30113560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:56:39.481 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:39.480+0000 7fda3698e640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fda30103c60 0x7fda30113aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:56:39.481 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:39.480+0000 7fda3698e640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda301140f0 con 0x7fda30103c60 2024-07-16T09:56:39.481 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:39.480+0000 7fda3698e640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda30114910 con 0x7fda30102990 2024-07-16T09:56:39.482 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:39.480+0000 7fda2f7fe640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fda30103c60 0x7fda30113aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:56:39.482 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:39.480+0000 7fda2f7fe640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fda30103c60 0x7fda30113aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:36158/0 (socket says 172.21.15.156:36158) 2024-07-16T09:56:39.482 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:39.480+0000 7fda2f7fe640 1 -- 172.21.15.156:0/914122401 learned_addr learned my addr 172.21.15.156:0/914122401 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T09:56:39.482 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:39.480+0000 7fda2ffff640 1 --2- 172.21.15.156:0/914122401 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fda30102990 0x7fda30113560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:56:39.483 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:39.481+0000 7fda2ffff640 1 -- 172.21.15.156:0/914122401 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fda30103c60 msgr2=0x7fda30113aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:56:39.483 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:39.481+0000 7fda2ffff640 1 --2- 172.21.15.156:0/914122401 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fda30103c60 0x7fda30113aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:56:39.483 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:39.481+0000 7fda2ffff640 1 -- 172.21.15.156:0/914122401 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda24005040 con 0x7fda30102990 2024-07-16T09:56:39.483 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:39.481+0000 7fda2ffff640 1 --2- 172.21.15.156:0/914122401 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fda30102990 0x7fda30113560 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fda28009c20 tx=0x7fda28004d10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:56:39.483 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:39.481+0000 7fda2cff9640 1 -- 172.21.15.156:0/914122401 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fda28008de0 con 0x7fda30102990 2024-07-16T09:56:39.483 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:39.481+0000 7fda2cff9640 1 -- 172.21.15.156:0/914122401 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fda28032c00 con 0x7fda30102990 2024-07-16T09:56:39.483 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:39.481+0000 7fda3698e640 1 -- 172.21.15.156:0/914122401 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda280098b0 con 0x7fda30102990 2024-07-16T09:56:39.483 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:39.482+0000 7fda3698e640 1 -- 172.21.15.156:0/914122401 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda30114ed0 con 0x7fda30102990 2024-07-16T09:56:39.483 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:39.482+0000 7fda2cff9640 1 -- 172.21.15.156:0/914122401 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fda2803db80 con 0x7fda30102990 2024-07-16T09:56:39.484 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:39.483+0000 7fda3698e640 1 -- 172.21.15.156:0/914122401 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda1c005240 con 0x7fda30102990 2024-07-16T09:56:39.485 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:39.484+0000 7fda2cff9640 1 -- 172.21.15.156:0/914122401 <== mon.1 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 25) v1 ==== 88617+0+0 (secure 0 0 0) 0x7fda2802f030 con 0x7fda30102990 2024-07-16T09:56:39.486 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:39.485+0000 7fda2cff9640 1 --2- 172.21.15.156:0/914122401 >> [v2:172.21.15.176:6828/673792884,v1:172.21.15.176:6829/673792884] conn(0x7fda1806a7a0 0x7fda1806cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:56:39.486 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:39.485+0000 7fda2cff9640 1 -- 172.21.15.156:0/914122401 <== mon.1 v2:172.21.15.176:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5722+0+0 (secure 0 0 0) 0x7fda280aa010 con 0x7fda30102990 2024-07-16T09:56:39.486 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:39.485+0000 7fda2f7fe640 1 --2- 172.21.15.156:0/914122401 >> [v2:172.21.15.176:6828/673792884,v1:172.21.15.176:6829/673792884] conn(0x7fda1806a7a0 0x7fda1806cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:56:39.486 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:39.485+0000 7fda2f7fe640 1 --2- 172.21.15.156:0/914122401 >> [v2:172.21.15.176:6828/673792884,v1:172.21.15.176:6829/673792884] conn(0x7fda1806a7a0 0x7fda1806cc50 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fda30113310 tx=0x7fda2400b040 comp rx=0 tx=0).ready entity=mgr.24377 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:56:39.490 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:39.489+0000 7fda2cff9640 1 -- 172.21.15.156:0/914122401 <== mon.1 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185936 (secure 0 0 0) 0x7fda28073b00 con 0x7fda30102990 2024-07-16T09:56:39.787 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:39.786+0000 7fda3698e640 1 -- 172.21.15.156:0/914122401 --> [v2:172.21.15.176:6828/673792884,v1:172.21.15.176:6829/673792884] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fda1c000bf0 con 0x7fda1806a7a0 2024-07-16T09:56:39.788 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:39.787+0000 7fda2cff9640 1 -- 172.21.15.156:0/914122401 <== mgr.24377 v2:172.21.15.176:6828/673792884 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+318 (secure 0 0 0) 0x7fda1c000bf0 con 0x7fda1806a7a0 2024-07-16T09:56:39.792 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:39.790+0000 7fda3698e640 1 -- 172.21.15.156:0/914122401 >> [v2:172.21.15.176:6828/673792884,v1:172.21.15.176:6829/673792884] conn(0x7fda1806a7a0 msgr2=0x7fda1806cc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:56:39.792 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:39.790+0000 7fda3698e640 1 --2- 172.21.15.156:0/914122401 >> [v2:172.21.15.176:6828/673792884,v1:172.21.15.176:6829/673792884] conn(0x7fda1806a7a0 0x7fda1806cc50 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fda30113310 tx=0x7fda2400b040 comp rx=0 tx=0).stop 2024-07-16T09:56:39.792 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:39.791+0000 7fda3698e640 1 -- 172.21.15.156:0/914122401 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fda30102990 msgr2=0x7fda30113560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:56:39.792 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:39.791+0000 7fda3698e640 1 --2- 172.21.15.156:0/914122401 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fda30102990 0x7fda30113560 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fda28009c20 tx=0x7fda28004d10 comp rx=0 tx=0).stop 2024-07-16T09:56:39.792 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:39.791+0000 7fda3698e640 1 -- 172.21.15.156:0/914122401 shutdown_connections 2024-07-16T09:56:39.792 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:39.791+0000 7fda3698e640 1 --2- 172.21.15.156:0/914122401 >> [v2:172.21.15.176:6828/673792884,v1:172.21.15.176:6829/673792884] conn(0x7fda1806a7a0 0x7fda1806cc50 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:56:39.792 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:39.791+0000 7fda3698e640 1 --2- 172.21.15.156:0/914122401 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fda30103c60 0x7fda30113aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:56:39.793 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:39.791+0000 7fda3698e640 1 --2- 172.21.15.156:0/914122401 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fda30102990 0x7fda30113560 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:56:39.793 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:39.791+0000 7fda3698e640 1 -- 172.21.15.156:0/914122401 >> 172.21.15.156:0/914122401 conn(0x7fda30071ba0 msgr2=0x7fda30073270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:56:39.793 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:39.791+0000 7fda3698e640 1 -- 172.21.15.156:0/914122401 shutdown_connections 2024-07-16T09:56:39.793 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:39.791+0000 7fda3698e640 1 -- 172.21.15.156:0/914122401 wait complete. 2024-07-16T09:56:39.802 INFO:teuthology.orchestra.run.smithi156.stdout:true 2024-07-16T09:56:39.905 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:39.903+0000 7fa39e16f640 1 -- 172.21.15.156:0/1357755981 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa3980ff4b0 msgr2=0x7fa398101890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:56:39.905 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:39.903+0000 7fa39e16f640 1 --2- 172.21.15.156:0/1357755981 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa3980ff4b0 0x7fa398101890 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fa38c0048d0 tx=0x7fa38c009da0 comp rx=0 tx=0).stop 2024-07-16T09:56:39.905 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:39.903+0000 7fa39e16f640 1 -- 172.21.15.156:0/1357755981 shutdown_connections 2024-07-16T09:56:39.905 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:39.903+0000 7fa39e16f640 1 --2- 172.21.15.156:0/1357755981 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa3980ff4b0 0x7fa398101890 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:56:39.905 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:39.903+0000 7fa39e16f640 1 --2- 172.21.15.156:0/1357755981 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa398066520 0x7fa3980fef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:56:39.905 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:39.903+0000 7fa39e16f640 1 -- 172.21.15.156:0/1357755981 >> 172.21.15.156:0/1357755981 conn(0x7fa3980f6a20 msgr2=0x7fa3980f8e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:56:39.905 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:39.903+0000 7fa39e16f640 1 -- 172.21.15.156:0/1357755981 shutdown_connections 2024-07-16T09:56:39.906 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:39.903+0000 7fa39e16f640 1 -- 172.21.15.156:0/1357755981 wait complete. 2024-07-16T09:56:39.906 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:39.904+0000 7fa39e16f640 1 Processor -- start 2024-07-16T09:56:39.907 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:39.904+0000 7fa39e16f640 1 -- start start 2024-07-16T09:56:39.907 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:39.904+0000 7fa39e16f640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa398066520 0x7fa39810cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:56:39.907 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:39.904+0000 7fa39e16f640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa3980ff4b0 0x7fa39810d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:56:39.908 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:39.904+0000 7fa39e16f640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa39810db40 con 0x7fa398066520 2024-07-16T09:56:39.908 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:39.904+0000 7fa39e16f640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa39810e360 con 0x7fa3980ff4b0 2024-07-16T09:56:39.908 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:39.904+0000 7fa3977fe640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa398066520 0x7fa39810cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:56:39.908 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:39.904+0000 7fa3977fe640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa398066520 0x7fa39810cfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:36174/0 (socket says 172.21.15.156:36174) 2024-07-16T09:56:39.908 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:39.905+0000 7fa3977fe640 1 -- 172.21.15.156:0/1948334395 learned_addr learned my addr 172.21.15.156:0/1948334395 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T09:56:39.908 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:39.904+0000 7fa396ffd640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa3980ff4b0 0x7fa39810d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:56:39.908 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:39.905+0000 7fa396ffd640 1 -- 172.21.15.156:0/1948334395 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa398066520 msgr2=0x7fa39810cfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:56:39.908 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:39.905+0000 7fa396ffd640 1 --2- 172.21.15.156:0/1948334395 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa398066520 0x7fa39810cfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:56:39.908 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:39.905+0000 7fa396ffd640 1 -- 172.21.15.156:0/1948334395 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa38c005040 con 0x7fa3980ff4b0 2024-07-16T09:56:39.908 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:39.905+0000 7fa3977fe640 1 --2- 172.21.15.156:0/1948334395 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa398066520 0x7fa39810cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-16T09:56:39.908 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:39.905+0000 7fa396ffd640 1 --2- 172.21.15.156:0/1948334395 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa3980ff4b0 0x7fa39810d4f0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fa38c0048a0 tx=0x7fa38c009da0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:56:39.909 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:39.905+0000 7fa37ffff640 1 -- 172.21.15.156:0/1948334395 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa38c038070 con 0x7fa3980ff4b0 2024-07-16T09:56:39.909 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:39.905+0000 7fa39e16f640 1 -- 172.21.15.156:0/1948334395 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa39810e590 con 0x7fa3980ff4b0 2024-07-16T09:56:39.909 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:39.906+0000 7fa39e16f640 1 -- 172.21.15.156:0/1948334395 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa39810eaa0 con 0x7fa3980ff4b0 2024-07-16T09:56:39.909 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:39.906+0000 7fa37ffff640 1 -- 172.21.15.156:0/1948334395 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa38c0092b0 con 0x7fa3980ff4b0 2024-07-16T09:56:39.909 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:39.906+0000 7fa37ffff640 1 -- 172.21.15.156:0/1948334395 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa38c033680 con 0x7fa3980ff4b0 2024-07-16T09:56:39.909 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:39.907+0000 7fa39e16f640 1 -- 172.21.15.156:0/1948334395 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa384005240 con 0x7fa3980ff4b0 2024-07-16T09:56:39.910 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:39.908+0000 7fa37ffff640 1 -- 172.21.15.156:0/1948334395 <== mon.1 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 25) v1 ==== 88617+0+0 (secure 0 0 0) 0x7fa38c030020 con 0x7fa3980ff4b0 2024-07-16T09:56:39.910 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:39.908+0000 7fa37ffff640 1 --2- 172.21.15.156:0/1948334395 >> [v2:172.21.15.176:6828/673792884,v1:172.21.15.176:6829/673792884] conn(0x7fa38006ebb0 0x7fa380071060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:56:39.910 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:39.909+0000 7fa3977fe640 1 --2- 172.21.15.156:0/1948334395 >> [v2:172.21.15.176:6828/673792884,v1:172.21.15.176:6829/673792884] conn(0x7fa38006ebb0 0x7fa380071060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:56:39.910 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:39.909+0000 7fa37ffff640 1 -- 172.21.15.156:0/1948334395 <== mon.1 v2:172.21.15.176:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5722+0+0 (secure 0 0 0) 0x7fa38c0a8c40 con 0x7fa3980ff4b0 2024-07-16T09:56:39.911 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:39.909+0000 7fa3977fe640 1 --2- 172.21.15.156:0/1948334395 >> [v2:172.21.15.176:6828/673792884,v1:172.21.15.176:6829/673792884] conn(0x7fa38006ebb0 0x7fa380071060 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fa38800f690 tx=0x7fa38800b040 comp rx=0 tx=0).ready entity=mgr.24377 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:56:39.914 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:39.913+0000 7fa37ffff640 1 -- 172.21.15.156:0/1948334395 <== mon.1 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185936 (secure 0 0 0) 0x7fa38c0728f0 con 0x7fa3980ff4b0 2024-07-16T09:56:40.228 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:40.226+0000 7fa39e16f640 1 -- 172.21.15.156:0/1948334395 --> [v2:172.21.15.176:6828/673792884,v1:172.21.15.176:6829/673792884] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa384000bf0 con 0x7fa38006ebb0 2024-07-16T09:56:40.229 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:40.228+0000 7fa37ffff640 1 -- 172.21.15.156:0/1948334395 <== mgr.24377 v2:172.21.15.176:6828/673792884 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+318 (secure 0 0 0) 0x7fa384000bf0 con 0x7fa38006ebb0 2024-07-16T09:56:40.233 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:40.231+0000 7fa39e16f640 1 -- 172.21.15.156:0/1948334395 >> [v2:172.21.15.176:6828/673792884,v1:172.21.15.176:6829/673792884] conn(0x7fa38006ebb0 msgr2=0x7fa380071060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:56:40.233 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:40.231+0000 7fa39e16f640 1 --2- 172.21.15.156:0/1948334395 >> [v2:172.21.15.176:6828/673792884,v1:172.21.15.176:6829/673792884] conn(0x7fa38006ebb0 0x7fa380071060 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fa38800f690 tx=0x7fa38800b040 comp rx=0 tx=0).stop 2024-07-16T09:56:40.233 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:40.232+0000 7fa39e16f640 1 -- 172.21.15.156:0/1948334395 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa3980ff4b0 msgr2=0x7fa39810d4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:56:40.233 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:40.232+0000 7fa39e16f640 1 --2- 172.21.15.156:0/1948334395 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa3980ff4b0 0x7fa39810d4f0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fa38c0048a0 tx=0x7fa38c009da0 comp rx=0 tx=0).stop 2024-07-16T09:56:40.234 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:40.232+0000 7fa39e16f640 1 -- 172.21.15.156:0/1948334395 shutdown_connections 2024-07-16T09:56:40.234 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:40.232+0000 7fa39e16f640 1 --2- 172.21.15.156:0/1948334395 >> [v2:172.21.15.176:6828/673792884,v1:172.21.15.176:6829/673792884] conn(0x7fa38006ebb0 0x7fa380071060 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:56:40.234 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:40.232+0000 7fa39e16f640 1 --2- 172.21.15.156:0/1948334395 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa3980ff4b0 0x7fa39810d4f0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:56:40.235 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:40.232+0000 7fa39e16f640 1 --2- 172.21.15.156:0/1948334395 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa398066520 0x7fa39810cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:56:40.235 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:40.232+0000 7fa39e16f640 1 -- 172.21.15.156:0/1948334395 >> 172.21.15.156:0/1948334395 conn(0x7fa3980f6a20 msgr2=0x7fa3980f6e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:56:40.235 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:40.232+0000 7fa39e16f640 1 -- 172.21.15.156:0/1948334395 shutdown_connections 2024-07-16T09:56:40.235 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:40.232+0000 7fa39e16f640 1 -- 172.21.15.156:0/1948334395 wait complete. 2024-07-16T09:56:40.374 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:40.372+0000 7f01ca7fc640 1 -- 172.21.15.156:0/3811393625 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f01b4032610 con 0x7f01cc0ff480 2024-07-16T09:56:40.374 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:40.372+0000 7f01d19e9640 1 -- 172.21.15.156:0/3811393625 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f01cc0ff480 msgr2=0x7f01cc101860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:56:40.374 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:40.372+0000 7f01d19e9640 1 --2- 172.21.15.156:0/3811393625 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f01cc0ff480 0x7f01cc101860 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f01b4004810 tx=0x7f01b4009da0 comp rx=0 tx=0).stop 2024-07-16T09:56:40.374 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:40.372+0000 7f01d19e9640 1 -- 172.21.15.156:0/3811393625 shutdown_connections 2024-07-16T09:56:40.374 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:40.372+0000 7f01d19e9640 1 --2- 172.21.15.156:0/3811393625 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f01cc0ff480 0x7f01cc101860 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:56:40.374 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:40.372+0000 7f01d19e9640 1 --2- 172.21.15.156:0/3811393625 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f01cc0662e0 0x7f01cc0fef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:56:40.374 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:40.372+0000 7f01d19e9640 1 -- 172.21.15.156:0/3811393625 >> 172.21.15.156:0/3811393625 conn(0x7f01cc0f6a10 msgr2=0x7f01cc0f8e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:56:40.374 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:40.373+0000 7f01d19e9640 1 -- 172.21.15.156:0/3811393625 shutdown_connections 2024-07-16T09:56:40.375 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:40.373+0000 7f01d19e9640 1 -- 172.21.15.156:0/3811393625 wait complete. 2024-07-16T09:56:40.375 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:40.374+0000 7f01d19e9640 1 Processor -- start 2024-07-16T09:56:40.375 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:40.374+0000 7f01d19e9640 1 -- start start 2024-07-16T09:56:40.376 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:40.374+0000 7f01d19e9640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f01cc0662e0 0x7f01cc10f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:56:40.376 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:40.374+0000 7f01d19e9640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f01cc0ff480 0x7f01cc10f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:56:40.376 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:40.374+0000 7f01d19e9640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01cc10fd10 con 0x7f01cc0ff480 2024-07-16T09:56:40.376 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:40.374+0000 7f01d19e9640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01cc1104e0 con 0x7f01cc0662e0 2024-07-16T09:56:40.377 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:40.375+0000 7f01c3fff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f01cc0ff480 0x7f01cc10f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:56:40.377 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:40.375+0000 7f01c3fff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f01cc0ff480 0x7f01cc10f6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:36196/0 (socket says 172.21.15.156:36196) 2024-07-16T09:56:40.377 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:40.375+0000 7f01c3fff640 1 -- 172.21.15.156:0/1467378923 learned_addr learned my addr 172.21.15.156:0/1467378923 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T09:56:40.377 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:40.375+0000 7f01caffd640 1 --2- 172.21.15.156:0/1467378923 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f01cc0662e0 0x7f01cc10f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:56:40.378 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:40.375+0000 7f01c3fff640 1 -- 172.21.15.156:0/1467378923 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f01cc0662e0 msgr2=0x7f01cc10f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:56:40.378 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:40.375+0000 7f01c3fff640 1 --2- 172.21.15.156:0/1467378923 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f01cc0662e0 0x7f01cc10f180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:56:40.378 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:40.375+0000 7f01c3fff640 1 -- 172.21.15.156:0/1467378923 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01b4005040 con 0x7f01cc0ff480 2024-07-16T09:56:40.378 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:40.376+0000 7f01c3fff640 1 --2- 172.21.15.156:0/1467378923 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f01cc0ff480 0x7f01cc10f6c0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f01b4004870 tx=0x7f01b4009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:56:40.379 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:40.376+0000 7f01c37fe640 1 -- 172.21.15.156:0/1467378923 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f01b4038070 con 0x7f01cc0ff480 2024-07-16T09:56:40.379 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:40.376+0000 7f01c37fe640 1 -- 172.21.15.156:0/1467378923 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f01b4032e70 con 0x7f01cc0ff480 2024-07-16T09:56:40.379 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:40.376+0000 7f01d19e9640 1 -- 172.21.15.156:0/1467378923 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01cc110710 con 0x7f01cc0ff480 2024-07-16T09:56:40.381 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:40.376+0000 7f01d19e9640 1 -- 172.21.15.156:0/1467378923 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01cc110bf0 con 0x7f01cc0ff480 2024-07-16T09:56:40.381 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:40.376+0000 7f01c37fe640 1 -- 172.21.15.156:0/1467378923 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f01b4032610 con 0x7f01cc0ff480 2024-07-16T09:56:40.381 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:40.379+0000 7f01d19e9640 1 -- 172.21.15.156:0/1467378923 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01ac005240 con 0x7f01cc0ff480 2024-07-16T09:56:40.381 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:40.379+0000 7f01c37fe640 1 -- 172.21.15.156:0/1467378923 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 25) v1 ==== 88617+0+0 (secure 0 0 0) 0x7f01b4030020 con 0x7f01cc0ff480 2024-07-16T09:56:40.382 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:40.380+0000 7f01c37fe640 1 --2- 172.21.15.156:0/1467378923 >> [v2:172.21.15.176:6828/673792884,v1:172.21.15.176:6829/673792884] conn(0x7f01b806a6d0 0x7f01b806cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:56:40.382 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:40.380+0000 7f01c37fe640 1 -- 172.21.15.156:0/1467378923 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5722+0+0 (secure 0 0 0) 0x7f01b40a95d0 con 0x7f01cc0ff480 2024-07-16T09:56:40.382 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:40.381+0000 7f01caffd640 1 --2- 172.21.15.156:0/1467378923 >> [v2:172.21.15.176:6828/673792884,v1:172.21.15.176:6829/673792884] conn(0x7f01b806a6d0 0x7f01b806cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:56:40.384 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:40.383+0000 7f01caffd640 1 --2- 172.21.15.156:0/1467378923 >> [v2:172.21.15.176:6828/673792884,v1:172.21.15.176:6829/673792884] conn(0x7f01b806a6d0 0x7f01b806cb80 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f01cc066040 tx=0x7f01bc009040 comp rx=0 tx=0).ready entity=mgr.24377 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:56:40.389 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:40.388+0000 7f01c37fe640 1 -- 172.21.15.156:0/1467378923 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185936 (secure 0 0 0) 0x7f01b4073280 con 0x7f01cc0ff480 2024-07-16T09:56:40.587 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:56:40 smithi156 ceph-mon[29302]: pgmap v28: 65 pgs: 65 active+clean; 269 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 494 op/s 2024-07-16T09:56:40.680 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:40.679+0000 7f01d19e9640 1 -- 172.21.15.156:0/1467378923 --> [v2:172.21.15.176:6828/673792884,v1:172.21.15.176:6829/673792884] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f01ac000bf0 con 0x7f01b806a6d0 2024-07-16T09:56:40.690 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:40.689+0000 7f01c37fe640 1 -- 172.21.15.156:0/1467378923 <== mgr.24377 v2:172.21.15.176:6828/673792884 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3410 (secure 0 0 0) 0x7f01ac000bf0 con 0x7f01b806a6d0 2024-07-16T09:56:40.690 INFO:teuthology.orchestra.run.smithi156.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-07-16T09:56:40.691 INFO:teuthology.orchestra.run.smithi156.stdout:alertmanager.smithi156 smithi156 *:9093,9094 starting - - - - 2024-07-16T09:56:40.691 INFO:teuthology.orchestra.run.smithi156.stdout:crash.smithi156 smithi156 running (9m) 42s ago 9m 6899k - 17.2.7-1104-ge6231b04 d5de5f953b95 bf6d49ea06b4 2024-07-16T09:56:40.691 INFO:teuthology.orchestra.run.smithi156.stdout:crash.smithi176 smithi176 running (8m) 42s ago 8m 6903k - 17.2.7-1104-ge6231b04 d5de5f953b95 907c9fa1e890 2024-07-16T09:56:40.691 INFO:teuthology.orchestra.run.smithi156.stdout:grafana.smithi156 smithi156 *:3000 starting - - - - 2024-07-16T09:56:40.691 INFO:teuthology.orchestra.run.smithi156.stdout:mds.cephfs.smithi156.evwwaz smithi156 running (3m) 42s ago 3m 82.5M - 17.2.7-1104-ge6231b04 d5de5f953b95 7c0ff9ae43f8 2024-07-16T09:56:40.691 INFO:teuthology.orchestra.run.smithi156.stdout:mds.cephfs.smithi156.rdibys smithi156 running (3m) 42s ago 3m 144M - 17.2.7-1104-ge6231b04 d5de5f953b95 b985cd1e6924 2024-07-16T09:56:40.691 INFO:teuthology.orchestra.run.smithi156.stdout:mds.cephfs.smithi176.dvvbud smithi176 running (3m) 42s ago 3m 135M - 17.2.7-1104-ge6231b04 d5de5f953b95 1b00147950f9 2024-07-16T09:56:40.691 INFO:teuthology.orchestra.run.smithi156.stdout:mds.cephfs.smithi176.wahbsi smithi176 running (3m) 42s ago 3m 82.5M - 17.2.7-1104-ge6231b04 d5de5f953b95 866664af2311 2024-07-16T09:56:40.691 INFO:teuthology.orchestra.run.smithi156.stdout:mgr.smithi156.zkvylv smithi156 *:9283 running (10m) 42s ago 10m 271M - 17.2.7-1104-ge6231b04 d5de5f953b95 a6937c84750a 2024-07-16T09:56:40.691 INFO:teuthology.orchestra.run.smithi156.stdout:mgr.smithi176.dqbrvx smithi176 *:8443,9283 running (103s) 42s ago 7m 482M - 19.0.0-3495-ga6ca461b 16f7feb04342 a7c332f2d676 2024-07-16T09:56:40.691 INFO:teuthology.orchestra.run.smithi156.stdout:mon.smithi156 smithi156 running (10m) 42s ago 10m 51.1M 2048M 17.2.7-1104-ge6231b04 d5de5f953b95 6471c8acff40 2024-07-16T09:56:40.692 INFO:teuthology.orchestra.run.smithi156.stdout:mon.smithi176 smithi176 running (7m) 42s ago 7m 40.4M 2048M 17.2.7-1104-ge6231b04 d5de5f953b95 3a293ac6ef01 2024-07-16T09:56:40.692 INFO:teuthology.orchestra.run.smithi156.stdout:node-exporter.smithi156 smithi156 *:9100 starting - - - - 2024-07-16T09:56:40.692 INFO:teuthology.orchestra.run.smithi156.stdout:node-exporter.smithi176 smithi176 *:9100 running (8m) 42s ago 8m 16.3M - 1.5.0 0da6a335fe13 536716db6afe 2024-07-16T09:56:40.692 INFO:teuthology.orchestra.run.smithi156.stdout:osd.0 smithi156 running (7m) 42s ago 7m 239M 1557M 17.2.7-1104-ge6231b04 d5de5f953b95 0471758ec349 2024-07-16T09:56:40.692 INFO:teuthology.orchestra.run.smithi156.stdout:osd.1 smithi156 running (6m) 42s ago 6m 235M 1557M 17.2.7-1104-ge6231b04 d5de5f953b95 149a0bda2f61 2024-07-16T09:56:40.692 INFO:teuthology.orchestra.run.smithi156.stdout:osd.2 smithi156 running (6m) 42s ago 6m 212M 1557M 17.2.7-1104-ge6231b04 d5de5f953b95 f28acec5cb79 2024-07-16T09:56:40.692 INFO:teuthology.orchestra.run.smithi156.stdout:osd.3 smithi176 running (5m) 42s ago 5m 256M 2581M 17.2.7-1104-ge6231b04 d5de5f953b95 733861ba84dc 2024-07-16T09:56:40.692 INFO:teuthology.orchestra.run.smithi156.stdout:osd.4 smithi176 running (5m) 42s ago 5m 233M 2581M 17.2.7-1104-ge6231b04 d5de5f953b95 ef0c67cec086 2024-07-16T09:56:40.692 INFO:teuthology.orchestra.run.smithi156.stdout:osd.5 smithi176 running (5m) 42s ago 5m 268M 2581M 17.2.7-1104-ge6231b04 d5de5f953b95 996e15448a52 2024-07-16T09:56:40.692 INFO:teuthology.orchestra.run.smithi156.stdout:prometheus.smithi156 smithi156 *:9095 running (7m) 42s ago 8m 57.9M - 2.43.0 a07b618ecd1d 1655b4227cd0 2024-07-16T09:56:40.693 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:40.691+0000 7f01d19e9640 1 -- 172.21.15.156:0/1467378923 >> [v2:172.21.15.176:6828/673792884,v1:172.21.15.176:6829/673792884] conn(0x7f01b806a6d0 msgr2=0x7f01b806cb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:56:40.693 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:40.691+0000 7f01d19e9640 1 --2- 172.21.15.156:0/1467378923 >> [v2:172.21.15.176:6828/673792884,v1:172.21.15.176:6829/673792884] conn(0x7f01b806a6d0 0x7f01b806cb80 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f01cc066040 tx=0x7f01bc009040 comp rx=0 tx=0).stop 2024-07-16T09:56:40.693 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:40.691+0000 7f01d19e9640 1 -- 172.21.15.156:0/1467378923 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f01cc0ff480 msgr2=0x7f01cc10f6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:56:40.694 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:40.691+0000 7f01d19e9640 1 --2- 172.21.15.156:0/1467378923 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f01cc0ff480 0x7f01cc10f6c0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f01b4004870 tx=0x7f01b4009da0 comp rx=0 tx=0).stop 2024-07-16T09:56:40.694 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:40.692+0000 7f01d19e9640 1 -- 172.21.15.156:0/1467378923 shutdown_connections 2024-07-16T09:56:40.694 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:40.692+0000 7f01d19e9640 1 --2- 172.21.15.156:0/1467378923 >> [v2:172.21.15.176:6828/673792884,v1:172.21.15.176:6829/673792884] conn(0x7f01b806a6d0 0x7f01b806cb80 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:56:40.694 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:40.692+0000 7f01d19e9640 1 --2- 172.21.15.156:0/1467378923 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f01cc0ff480 0x7f01cc10f6c0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:56:40.694 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:40.692+0000 7f01d19e9640 1 --2- 172.21.15.156:0/1467378923 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f01cc0662e0 0x7f01cc10f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:56:40.694 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:40.692+0000 7f01d19e9640 1 -- 172.21.15.156:0/1467378923 >> 172.21.15.156:0/1467378923 conn(0x7f01cc0f6a10 msgr2=0x7f01cc0f6de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:56:40.694 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:40.692+0000 7f01d19e9640 1 -- 172.21.15.156:0/1467378923 shutdown_connections 2024-07-16T09:56:40.694 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:40.692+0000 7f01d19e9640 1 -- 172.21.15.156:0/1467378923 wait complete. 2024-07-16T09:56:40.798 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:40.797+0000 7f6683ece640 1 -- 172.21.15.156:0/1204487926 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f667c0fb240 msgr2=0x7f667c0fb650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:56:40.799 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:40.797+0000 7f6683ece640 1 --2- 172.21.15.156:0/1204487926 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f667c0fb240 0x7f667c0fb650 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f66780048d0 tx=0x7f6678009da0 comp rx=0 tx=0).stop 2024-07-16T09:56:40.799 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:40.797+0000 7f6683ece640 1 -- 172.21.15.156:0/1204487926 shutdown_connections 2024-07-16T09:56:40.799 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:40.797+0000 7f6683ece640 1 --2- 172.21.15.156:0/1204487926 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f667c0fc510 0x7f667c0fc980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:56:40.799 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:40.797+0000 7f6683ece640 1 --2- 172.21.15.156:0/1204487926 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f667c0fb240 0x7f667c0fb650 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:56:40.799 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:40.797+0000 7f6683ece640 1 -- 172.21.15.156:0/1204487926 >> 172.21.15.156:0/1204487926 conn(0x7f667c0f6a20 msgr2=0x7f667c0f8e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:56:40.799 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:40.797+0000 7f6683ece640 1 -- 172.21.15.156:0/1204487926 shutdown_connections 2024-07-16T09:56:40.799 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:40.797+0000 7f6683ece640 1 -- 172.21.15.156:0/1204487926 wait complete. 2024-07-16T09:56:40.800 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:40.798+0000 7f6683ece640 1 Processor -- start 2024-07-16T09:56:40.801 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:40.798+0000 7f6683ece640 1 -- start start 2024-07-16T09:56:40.801 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:40.799+0000 7f6683ece640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f667c0fb240 0x7f667c10f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:56:40.801 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:40.799+0000 7f6683ece640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f667c0fc510 0x7f667c10f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:56:40.801 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:40.799+0000 7f6683ece640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f667c10fd50 con 0x7f667c0fc510 2024-07-16T09:56:40.801 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:40.799+0000 7f6681c6a640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f667c0fb240 0x7f667c10f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:56:40.801 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:40.799+0000 7f6683ece640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f667c110520 con 0x7f667c0fb240 2024-07-16T09:56:40.801 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:40.799+0000 7f6681469640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f667c0fc510 0x7f667c10f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:56:40.801 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:40.799+0000 7f6681469640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f667c0fc510 0x7f667c10f700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:36220/0 (socket says 172.21.15.156:36220) 2024-07-16T09:56:40.801 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:40.799+0000 7f6681469640 1 -- 172.21.15.156:0/1919950663 learned_addr learned my addr 172.21.15.156:0/1919950663 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T09:56:40.801 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:40.799+0000 7f6681469640 1 -- 172.21.15.156:0/1919950663 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f667c0fb240 msgr2=0x7f667c10f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:56:40.802 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:40.799+0000 7f6681469640 1 --2- 172.21.15.156:0/1919950663 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f667c0fb240 0x7f667c10f1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:56:40.802 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:40.799+0000 7f6681469640 1 -- 172.21.15.156:0/1919950663 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6678005040 con 0x7f667c0fc510 2024-07-16T09:56:40.803 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:40.800+0000 7f6681469640 1 --2- 172.21.15.156:0/1919950663 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f667c0fc510 0x7f667c10f700 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f666c002a80 tx=0x7f666c004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:56:40.803 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:40.800+0000 7f66727fc640 1 -- 172.21.15.156:0/1919950663 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f666c007c30 con 0x7f667c0fc510 2024-07-16T09:56:40.803 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:40.800+0000 7f66727fc640 1 -- 172.21.15.156:0/1919950663 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f666c00f040 con 0x7f667c0fc510 2024-07-16T09:56:40.803 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:40.800+0000 7f6683ece640 1 -- 172.21.15.156:0/1919950663 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f667c1107b0 con 0x7f667c0fc510 2024-07-16T09:56:40.803 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:40.800+0000 7f66727fc640 1 -- 172.21.15.156:0/1919950663 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f666c0095f0 con 0x7f667c0fc510 2024-07-16T09:56:40.803 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:40.800+0000 7f6683ece640 1 -- 172.21.15.156:0/1919950663 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f667c110cd0 con 0x7f667c0fc510 2024-07-16T09:56:40.803 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:40.801+0000 7f6683ece640 1 -- 172.21.15.156:0/1919950663 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f667c0fa6a0 con 0x7f667c0fc510 2024-07-16T09:56:40.803 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:40.802+0000 7f66727fc640 1 -- 172.21.15.156:0/1919950663 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 25) v1 ==== 88617+0+0 (secure 0 0 0) 0x7f666c009750 con 0x7f667c0fc510 2024-07-16T09:56:40.804 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:40.802+0000 7f66727fc640 1 --2- 172.21.15.156:0/1919950663 >> [v2:172.21.15.176:6828/673792884,v1:172.21.15.176:6829/673792884] conn(0x7f666806a6d0 0x7f666806cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:56:40.804 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:40.802+0000 7f66727fc640 1 -- 172.21.15.156:0/1919950663 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5722+0+0 (secure 0 0 0) 0x7f666c088900 con 0x7f667c0fc510 2024-07-16T09:56:40.804 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:40.803+0000 7f6681c6a640 1 --2- 172.21.15.156:0/1919950663 >> [v2:172.21.15.176:6828/673792884,v1:172.21.15.176:6829/673792884] conn(0x7f666806a6d0 0x7f666806cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:56:40.805 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:40.803+0000 7f6681c6a640 1 --2- 172.21.15.156:0/1919950663 >> [v2:172.21.15.176:6828/673792884,v1:172.21.15.176:6829/673792884] conn(0x7f666806a6d0 0x7f666806cb80 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f66780043d0 tx=0x7f6678035040 comp rx=0 tx=0).ready entity=mgr.24377 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:56:40.812 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:40.810+0000 7f66727fc640 1 -- 172.21.15.156:0/1919950663 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185936 (secure 0 0 0) 0x7f666c04f660 con 0x7f667c0fc510 2024-07-16T09:56:40.843 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:56:40 smithi176 ceph-mon[36626]: pgmap v28: 65 pgs: 65 active+clean; 269 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 494 op/s 2024-07-16T09:56:41.206 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:41.205+0000 7f6683ece640 1 -- 172.21.15.156:0/1919950663 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f667c1070a0 con 0x7f667c0fc510 2024-07-16T09:56:41.207 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:41.206+0000 7f66727fc640 1 -- 172.21.15.156:0/1919950663 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+849 (secure 0 0 0) 0x7f666c052540 con 0x7f667c0fc510 2024-07-16T09:56:41.207 INFO:teuthology.orchestra.run.smithi156.stdout:{ 2024-07-16T09:56:41.208 INFO:teuthology.orchestra.run.smithi156.stdout: "mon": { 2024-07-16T09:56:41.208 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph version 17.2.7-1104-ge6231b04 (e6231b04b52ebf3b9b31803282dd3074439e51ac) quincy (stable)": 2 2024-07-16T09:56:41.208 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-07-16T09:56:41.208 INFO:teuthology.orchestra.run.smithi156.stdout: "mgr": { 2024-07-16T09:56:41.208 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph version 17.2.7-1104-ge6231b04 (e6231b04b52ebf3b9b31803282dd3074439e51ac) quincy (stable)": 1, 2024-07-16T09:56:41.208 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph version 19.0.0-3495-ga6ca461b (a6ca461be7cc1aff3658c2c6df4cff975a24449a) squid (rc)": 1 2024-07-16T09:56:41.208 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-07-16T09:56:41.208 INFO:teuthology.orchestra.run.smithi156.stdout: "osd": { 2024-07-16T09:56:41.208 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph version 17.2.7-1104-ge6231b04 (e6231b04b52ebf3b9b31803282dd3074439e51ac) quincy (stable)": 6 2024-07-16T09:56:41.208 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-07-16T09:56:41.208 INFO:teuthology.orchestra.run.smithi156.stdout: "mds": { 2024-07-16T09:56:41.208 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph version 17.2.7-1104-ge6231b04 (e6231b04b52ebf3b9b31803282dd3074439e51ac) quincy (stable)": 4 2024-07-16T09:56:41.209 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-07-16T09:56:41.209 INFO:teuthology.orchestra.run.smithi156.stdout: "overall": { 2024-07-16T09:56:41.209 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph version 17.2.7-1104-ge6231b04 (e6231b04b52ebf3b9b31803282dd3074439e51ac) quincy (stable)": 13, 2024-07-16T09:56:41.209 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph version 19.0.0-3495-ga6ca461b (a6ca461be7cc1aff3658c2c6df4cff975a24449a) squid (rc)": 1 2024-07-16T09:56:41.209 INFO:teuthology.orchestra.run.smithi156.stdout: } 2024-07-16T09:56:41.209 INFO:teuthology.orchestra.run.smithi156.stdout:} 2024-07-16T09:56:41.211 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:41.210+0000 7f6683ece640 1 -- 172.21.15.156:0/1919950663 >> [v2:172.21.15.176:6828/673792884,v1:172.21.15.176:6829/673792884] conn(0x7f666806a6d0 msgr2=0x7f666806cb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:56:41.211 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:41.210+0000 7f6683ece640 1 --2- 172.21.15.156:0/1919950663 >> [v2:172.21.15.176:6828/673792884,v1:172.21.15.176:6829/673792884] conn(0x7f666806a6d0 0x7f666806cb80 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f66780043d0 tx=0x7f6678035040 comp rx=0 tx=0).stop 2024-07-16T09:56:41.211 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:41.210+0000 7f6683ece640 1 -- 172.21.15.156:0/1919950663 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f667c0fc510 msgr2=0x7f667c10f700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:56:41.211 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:41.210+0000 7f6683ece640 1 --2- 172.21.15.156:0/1919950663 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f667c0fc510 0x7f667c10f700 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f666c002a80 tx=0x7f666c004600 comp rx=0 tx=0).stop 2024-07-16T09:56:41.212 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:41.210+0000 7f6683ece640 1 -- 172.21.15.156:0/1919950663 shutdown_connections 2024-07-16T09:56:41.212 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:41.210+0000 7f6683ece640 1 --2- 172.21.15.156:0/1919950663 >> [v2:172.21.15.176:6828/673792884,v1:172.21.15.176:6829/673792884] conn(0x7f666806a6d0 0x7f666806cb80 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:56:41.212 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:41.210+0000 7f6683ece640 1 --2- 172.21.15.156:0/1919950663 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f667c0fc510 0x7f667c10f700 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:56:41.212 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:41.210+0000 7f6683ece640 1 --2- 172.21.15.156:0/1919950663 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f667c0fb240 0x7f667c10f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:56:41.212 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:41.210+0000 7f6683ece640 1 -- 172.21.15.156:0/1919950663 >> 172.21.15.156:0/1919950663 conn(0x7f667c0f6a20 msgr2=0x7f667c0f8100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:56:41.212 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:41.211+0000 7f6683ece640 1 -- 172.21.15.156:0/1919950663 shutdown_connections 2024-07-16T09:56:41.212 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:41.211+0000 7f6683ece640 1 -- 172.21.15.156:0/1919950663 wait complete. 2024-07-16T09:56:41.338 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:41.336+0000 7efdc40b7640 1 -- 172.21.15.156:0/2181374088 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efdbc0fb6e0 msgr2=0x7efdbc0fbaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:56:41.338 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:41.336+0000 7efdc40b7640 1 --2- 172.21.15.156:0/2181374088 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efdbc0fb6e0 0x7efdbc0fbaf0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7efdb4009c50 tx=0x7efdb4004d10 comp rx=0 tx=0).stop 2024-07-16T09:56:41.338 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:41.336+0000 7efdc40b7640 1 -- 172.21.15.156:0/2181374088 shutdown_connections 2024-07-16T09:56:41.338 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:41.336+0000 7efdc40b7640 1 --2- 172.21.15.156:0/2181374088 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7efdbc0fc030 0x7efdbc0fa460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:56:41.338 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:41.336+0000 7efdc40b7640 1 --2- 172.21.15.156:0/2181374088 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efdbc0fb6e0 0x7efdbc0fbaf0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:56:41.338 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:41.336+0000 7efdc40b7640 1 -- 172.21.15.156:0/2181374088 >> 172.21.15.156:0/2181374088 conn(0x7efdbc0f61b0 msgr2=0x7efdbc0f85e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:56:41.338 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:41.336+0000 7efdc40b7640 1 -- 172.21.15.156:0/2181374088 shutdown_connections 2024-07-16T09:56:41.339 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:41.337+0000 7efdc40b7640 1 -- 172.21.15.156:0/2181374088 wait complete. 2024-07-16T09:56:41.339 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:41.337+0000 7efdc40b7640 1 Processor -- start 2024-07-16T09:56:41.339 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:41.337+0000 7efdc40b7640 1 -- start start 2024-07-16T09:56:41.340 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:41.338+0000 7efdc40b7640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efdbc0fb6e0 0x7efdbc10ada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:56:41.340 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:41.338+0000 7efdc40b7640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7efdbc0fc030 0x7efdbc10b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:56:41.340 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:41.338+0000 7efdc40b7640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdbc10b930 con 0x7efdbc0fb6e0 2024-07-16T09:56:41.340 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:41.338+0000 7efdc40b7640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdbc10c070 con 0x7efdbc0fc030 2024-07-16T09:56:41.340 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:41.338+0000 7efdc1e53640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efdbc0fb6e0 0x7efdbc10ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:56:41.340 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:41.338+0000 7efdc1e53640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efdbc0fb6e0 0x7efdbc10ada0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:36234/0 (socket says 172.21.15.156:36234) 2024-07-16T09:56:41.340 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:41.338+0000 7efdc1e53640 1 -- 172.21.15.156:0/2319123684 learned_addr learned my addr 172.21.15.156:0/2319123684 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T09:56:41.340 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:41.338+0000 7efdc1e53640 1 -- 172.21.15.156:0/2319123684 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7efdbc0fc030 msgr2=0x7efdbc10b2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:56:41.340 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:41.338+0000 7efdc1e53640 1 --2- 172.21.15.156:0/2319123684 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7efdbc0fc030 0x7efdbc10b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:56:41.340 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:41.338+0000 7efdc1e53640 1 -- 172.21.15.156:0/2319123684 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdb40098b0 con 0x7efdbc0fb6e0 2024-07-16T09:56:41.341 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:41.339+0000 7efdc1e53640 1 --2- 172.21.15.156:0/2319123684 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efdbc0fb6e0 0x7efdbc10ada0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7efdb4009c20 tx=0x7efdb4004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:56:41.341 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:41.339+0000 7efdb27fc640 1 -- 172.21.15.156:0/2319123684 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efdb4032b20 con 0x7efdbc0fb6e0 2024-07-16T09:56:41.341 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:41.339+0000 7efdc40b7640 1 -- 172.21.15.156:0/2319123684 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efdbc10c2a0 con 0x7efdbc0fb6e0 2024-07-16T09:56:41.341 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:41.339+0000 7efdb27fc640 1 -- 172.21.15.156:0/2319123684 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7efdb4032c80 con 0x7efdbc0fb6e0 2024-07-16T09:56:41.341 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:41.339+0000 7efdc40b7640 1 -- 172.21.15.156:0/2319123684 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efdbc10c7c0 con 0x7efdbc0fb6e0 2024-07-16T09:56:41.341 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:41.339+0000 7efdb27fc640 1 -- 172.21.15.156:0/2319123684 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efdb40313d0 con 0x7efdbc0fb6e0 2024-07-16T09:56:41.342 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:41.340+0000 7efdc40b7640 1 -- 172.21.15.156:0/2319123684 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efdbc0f9e50 con 0x7efdbc0fb6e0 2024-07-16T09:56:41.343 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:41.341+0000 7efdb27fc640 1 -- 172.21.15.156:0/2319123684 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 25) v1 ==== 88617+0+0 (secure 0 0 0) 0x7efdb402f030 con 0x7efdbc0fb6e0 2024-07-16T09:56:41.344 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:41.342+0000 7efdb27fc640 1 --2- 172.21.15.156:0/2319123684 >> [v2:172.21.15.176:6828/673792884,v1:172.21.15.176:6829/673792884] conn(0x7efda806a7a0 0x7efda806cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:56:41.344 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:41.342+0000 7efdb27fc640 1 -- 172.21.15.156:0/2319123684 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5722+0+0 (secure 0 0 0) 0x7efdb40aab40 con 0x7efdbc0fb6e0 2024-07-16T09:56:41.344 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:41.343+0000 7efdc1652640 1 --2- 172.21.15.156:0/2319123684 >> [v2:172.21.15.176:6828/673792884,v1:172.21.15.176:6829/673792884] conn(0x7efda806a7a0 0x7efda806cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:56:41.347 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:41.346+0000 7efdc1652640 1 --2- 172.21.15.156:0/2319123684 >> [v2:172.21.15.176:6828/673792884,v1:172.21.15.176:6829/673792884] conn(0x7efda806a7a0 0x7efda806cc50 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7efdac004c80 tx=0x7efdac01ac60 comp rx=0 tx=0).ready entity=mgr.24377 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:56:41.351 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:41.350+0000 7efdb27fc640 1 -- 172.21.15.156:0/2319123684 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185936 (secure 0 0 0) 0x7efdb4074630 con 0x7efdbc0fb6e0 2024-07-16T09:56:41.635 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:41.634+0000 7efdc40b7640 1 -- 172.21.15.156:0/2319123684 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7efdbc102ca0 con 0x7efdbc0fb6e0 2024-07-16T09:56:41.635 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:41.634+0000 7efdb27fc640 1 -- 172.21.15.156:0/2319123684 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 18 v18) v1 ==== 76+0+1869 (secure 0 0 0) 0x7efdb4041760 con 0x7efdbc0fb6e0 2024-07-16T09:56:41.636 INFO:teuthology.orchestra.run.smithi156.stdout:e18 2024-07-16T09:56:41.636 INFO:teuthology.orchestra.run.smithi156.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-07-16T09:56:41.636 INFO:teuthology.orchestra.run.smithi156.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-07-16T09:56:41.636 INFO:teuthology.orchestra.run.smithi156.stdout:legacy client fscid: 1 2024-07-16T09:56:41.636 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-07-16T09:56:41.636 INFO:teuthology.orchestra.run.smithi156.stdout:Filesystem 'cephfs' (1) 2024-07-16T09:56:41.636 INFO:teuthology.orchestra.run.smithi156.stdout:fs_name cephfs 2024-07-16T09:56:41.636 INFO:teuthology.orchestra.run.smithi156.stdout:epoch 18 2024-07-16T09:56:41.636 INFO:teuthology.orchestra.run.smithi156.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-07-16T09:56:41.636 INFO:teuthology.orchestra.run.smithi156.stdout:created 2024-07-16T09:52:55.214975+0000 2024-07-16T09:56:41.637 INFO:teuthology.orchestra.run.smithi156.stdout:modified 2024-07-16T09:55:41.914320+0000 2024-07-16T09:56:41.637 INFO:teuthology.orchestra.run.smithi156.stdout:tableserver 0 2024-07-16T09:56:41.637 INFO:teuthology.orchestra.run.smithi156.stdout:root 0 2024-07-16T09:56:41.637 INFO:teuthology.orchestra.run.smithi156.stdout:session_timeout 60 2024-07-16T09:56:41.637 INFO:teuthology.orchestra.run.smithi156.stdout:session_autoclose 300 2024-07-16T09:56:41.637 INFO:teuthology.orchestra.run.smithi156.stdout:max_file_size 1099511627776 2024-07-16T09:56:41.637 INFO:teuthology.orchestra.run.smithi156.stdout:required_client_features {} 2024-07-16T09:56:41.637 INFO:teuthology.orchestra.run.smithi156.stdout:last_failure 0 2024-07-16T09:56:41.637 INFO:teuthology.orchestra.run.smithi156.stdout:last_failure_osd_epoch 0 2024-07-16T09:56:41.637 INFO:teuthology.orchestra.run.smithi156.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-07-16T09:56:41.637 INFO:teuthology.orchestra.run.smithi156.stdout:max_mds 2 2024-07-16T09:56:41.637 INFO:teuthology.orchestra.run.smithi156.stdout:in 0,1 2024-07-16T09:56:41.637 INFO:teuthology.orchestra.run.smithi156.stdout:up {0=14458,1=24261} 2024-07-16T09:56:41.637 INFO:teuthology.orchestra.run.smithi156.stdout:failed 2024-07-16T09:56:41.638 INFO:teuthology.orchestra.run.smithi156.stdout:damaged 2024-07-16T09:56:41.638 INFO:teuthology.orchestra.run.smithi156.stdout:stopped 2024-07-16T09:56:41.638 INFO:teuthology.orchestra.run.smithi156.stdout:data_pools [3] 2024-07-16T09:56:41.638 INFO:teuthology.orchestra.run.smithi156.stdout:metadata_pool 2 2024-07-16T09:56:41.638 INFO:teuthology.orchestra.run.smithi156.stdout:inline_data disabled 2024-07-16T09:56:41.638 INFO:teuthology.orchestra.run.smithi156.stdout:balancer 2024-07-16T09:56:41.638 INFO:teuthology.orchestra.run.smithi156.stdout:standby_count_wanted 1 2024-07-16T09:56:41.638 INFO:teuthology.orchestra.run.smithi156.stdout:[mds.cephfs.smithi156.rdibys{0:14458} state up:active seq 4 export targets 1 join_fscid=1 addr [v2:172.21.15.156:6826/1080176013,v1:172.21.15.156:6827/1080176013] compat {c=[1],r=[1],i=[7ff]}] 2024-07-16T09:56:41.638 INFO:teuthology.orchestra.run.smithi156.stdout:[mds.cephfs.smithi156.evwwaz{0:14468} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.156:6828/560329502,v1:172.21.15.156:6829/560329502] compat {c=[1],r=[1],i=[7ff]}] 2024-07-16T09:56:41.638 INFO:teuthology.orchestra.run.smithi156.stdout:[mds.cephfs.smithi176.dvvbud{1:24261} state up:active seq 4 export targets 0 join_fscid=1 addr [v2:172.21.15.176:6824/1577019287,v1:172.21.15.176:6825/1577019287] compat {c=[1],r=[1],i=[7ff]}] 2024-07-16T09:56:41.638 INFO:teuthology.orchestra.run.smithi156.stdout:[mds.cephfs.smithi176.wahbsi{1:24273} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.176:6826/2196775412,v1:172.21.15.176:6827/2196775412] compat {c=[1],r=[1],i=[7ff]}] 2024-07-16T09:56:41.638 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-07-16T09:56:41.639 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-07-16T09:56:41.639 INFO:teuthology.orchestra.run.smithi156.stderr:dumped fsmap epoch 18 2024-07-16T09:56:41.639 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:41.637+0000 7efdc40b7640 1 -- 172.21.15.156:0/2319123684 >> [v2:172.21.15.176:6828/673792884,v1:172.21.15.176:6829/673792884] conn(0x7efda806a7a0 msgr2=0x7efda806cc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:56:41.639 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:41.637+0000 7efdc40b7640 1 --2- 172.21.15.156:0/2319123684 >> [v2:172.21.15.176:6828/673792884,v1:172.21.15.176:6829/673792884] conn(0x7efda806a7a0 0x7efda806cc50 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7efdac004c80 tx=0x7efdac01ac60 comp rx=0 tx=0).stop 2024-07-16T09:56:41.640 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:41.637+0000 7efdc40b7640 1 -- 172.21.15.156:0/2319123684 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efdbc0fb6e0 msgr2=0x7efdbc10ada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:56:41.640 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:41.637+0000 7efdc40b7640 1 --2- 172.21.15.156:0/2319123684 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efdbc0fb6e0 0x7efdbc10ada0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7efdb4009c20 tx=0x7efdb4004d10 comp rx=0 tx=0).stop 2024-07-16T09:56:41.640 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:41.637+0000 7efdc40b7640 1 -- 172.21.15.156:0/2319123684 shutdown_connections 2024-07-16T09:56:41.640 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:41.637+0000 7efdc40b7640 1 --2- 172.21.15.156:0/2319123684 >> [v2:172.21.15.176:6828/673792884,v1:172.21.15.176:6829/673792884] conn(0x7efda806a7a0 0x7efda806cc50 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:56:41.640 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:41.638+0000 7efdc40b7640 1 --2- 172.21.15.156:0/2319123684 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7efdbc0fc030 0x7efdbc10b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:56:41.640 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:41.638+0000 7efdc40b7640 1 --2- 172.21.15.156:0/2319123684 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efdbc0fb6e0 0x7efdbc10ada0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:56:41.640 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:41.638+0000 7efdc40b7640 1 -- 172.21.15.156:0/2319123684 >> 172.21.15.156:0/2319123684 conn(0x7efdbc0f61b0 msgr2=0x7efdbc0f7870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:56:41.640 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:41.638+0000 7efdc40b7640 1 -- 172.21.15.156:0/2319123684 shutdown_connections 2024-07-16T09:56:41.640 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:41.638+0000 7efdc40b7640 1 -- 172.21.15.156:0/2319123684 wait complete. 2024-07-16T09:56:41.741 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:41.740+0000 7f925e321640 1 -- 172.21.15.156:0/628667691 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f92580fb240 msgr2=0x7f92580fb650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:56:41.742 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:41.740+0000 7f925e321640 1 --2- 172.21.15.156:0/628667691 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f92580fb240 0x7f92580fb650 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f92480048d0 tx=0x7f9248009da0 comp rx=0 tx=0).stop 2024-07-16T09:56:41.742 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:41.740+0000 7f925e321640 1 -- 172.21.15.156:0/628667691 shutdown_connections 2024-07-16T09:56:41.742 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:41.740+0000 7f925e321640 1 --2- 172.21.15.156:0/628667691 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f92580fc510 0x7f92580fc980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:56:41.742 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:41.740+0000 7f925e321640 1 --2- 172.21.15.156:0/628667691 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f92580fb240 0x7f92580fb650 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:56:41.742 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:41.740+0000 7f925e321640 1 -- 172.21.15.156:0/628667691 >> 172.21.15.156:0/628667691 conn(0x7f92580f6a20 msgr2=0x7f92580f8e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:56:41.742 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:41.740+0000 7f925e321640 1 -- 172.21.15.156:0/628667691 shutdown_connections 2024-07-16T09:56:41.742 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:56:41 smithi156 ceph-mon[29302]: from='client.24477 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-16T09:56:41.742 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:56:41 smithi156 ceph-mon[29302]: from='client.24481 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-16T09:56:41.743 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:56:41 smithi156 ceph-mon[29302]: from='mgr.24377 ' entity='mgr.smithi176.dqbrvx' 2024-07-16T09:56:41.743 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:56:41 smithi156 ceph-mon[29302]: from='mgr.24377 ' entity='mgr.smithi176.dqbrvx' 2024-07-16T09:56:41.743 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:56:41 smithi156 ceph-mon[29302]: from='client.? 172.21.15.156:0/1919950663' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T09:56:41.744 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:41.740+0000 7f925e321640 1 -- 172.21.15.156:0/628667691 wait complete. 2024-07-16T09:56:41.744 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:41.741+0000 7f925e321640 1 Processor -- start 2024-07-16T09:56:41.744 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:41.741+0000 7f925e321640 1 -- start start 2024-07-16T09:56:41.744 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:41.741+0000 7f925e321640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f92580fb240 0x7f925810ad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:56:41.744 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:41.741+0000 7f925e321640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f92580fc510 0x7f925810b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:56:41.744 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:41.741+0000 7f925e321640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f925810b8d0 con 0x7f92580fc510 2024-07-16T09:56:41.744 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:41.741+0000 7f925e321640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f925810c140 con 0x7f92580fb240 2024-07-16T09:56:41.744 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:41.742+0000 7f9257fff640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f92580fb240 0x7f925810ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:56:41.744 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:41.742+0000 7f9257fff640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f92580fb240 0x7f925810ad90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.156:50010/0 (socket says 172.21.15.156:50010) 2024-07-16T09:56:41.744 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:41.742+0000 7f9257fff640 1 -- 172.21.15.156:0/3117265798 learned_addr learned my addr 172.21.15.156:0/3117265798 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T09:56:41.744 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:41.742+0000 7f9257fff640 1 -- 172.21.15.156:0/3117265798 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f92580fc510 msgr2=0x7f925810b2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:56:41.744 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:41.742+0000 7f9257fff640 1 --2- 172.21.15.156:0/3117265798 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f92580fc510 0x7f925810b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:56:41.744 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:41.742+0000 7f9257fff640 1 -- 172.21.15.156:0/3117265798 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9248005040 con 0x7f92580fb240 2024-07-16T09:56:41.745 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:41.742+0000 7f9257fff640 1 --2- 172.21.15.156:0/3117265798 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f92580fb240 0x7f925810ad90 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f92480041d0 tx=0x7f9248009da0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:56:41.745 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:41.743+0000 7f9254ff9640 1 -- 172.21.15.156:0/3117265798 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9248038070 con 0x7f92580fb240 2024-07-16T09:56:41.745 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:41.743+0000 7f9254ff9640 1 -- 172.21.15.156:0/3117265798 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f92480092b0 con 0x7f92580fb240 2024-07-16T09:56:41.745 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:41.743+0000 7f9254ff9640 1 -- 172.21.15.156:0/3117265798 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9248033710 con 0x7f92580fb240 2024-07-16T09:56:41.746 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:41.743+0000 7f925e321640 1 -- 172.21.15.156:0/3117265798 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f925810c3c0 con 0x7f92580fb240 2024-07-16T09:56:41.746 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:41.743+0000 7f925e321640 1 -- 172.21.15.156:0/3117265798 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f925810c840 con 0x7f92580fb240 2024-07-16T09:56:41.746 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:41.744+0000 7f925e321640 1 -- 172.21.15.156:0/3117265798 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92580fa6a0 con 0x7f92580fb240 2024-07-16T09:56:41.746 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:41.744+0000 7f9254ff9640 1 -- 172.21.15.156:0/3117265798 <== mon.1 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 25) v1 ==== 88617+0+0 (secure 0 0 0) 0x7f9248030020 con 0x7f92580fb240 2024-07-16T09:56:41.746 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:41.745+0000 7f9254ff9640 1 --2- 172.21.15.156:0/3117265798 >> [v2:172.21.15.176:6828/673792884,v1:172.21.15.176:6829/673792884] conn(0x7f924006a780 0x7f924006cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:56:41.746 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:41.745+0000 7f9254ff9640 1 -- 172.21.15.156:0/3117265798 <== mon.1 v2:172.21.15.176:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5722+0+0 (secure 0 0 0) 0x7f92480a8d00 con 0x7f92580fb240 2024-07-16T09:56:41.746 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:41.745+0000 7f92577fe640 1 --2- 172.21.15.156:0/3117265798 >> [v2:172.21.15.176:6828/673792884,v1:172.21.15.176:6829/673792884] conn(0x7f924006a780 0x7f924006cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:56:41.747 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:41.746+0000 7f92577fe640 1 --2- 172.21.15.156:0/3117265798 >> [v2:172.21.15.176:6828/673792884,v1:172.21.15.176:6829/673792884] conn(0x7f924006a780 0x7f924006cc30 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f924c004d40 tx=0x7f924c00b040 comp rx=0 tx=0).ready entity=mgr.24377 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:56:41.752 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:41.751+0000 7f9254ff9640 1 -- 172.21.15.156:0/3117265798 <== mon.1 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185936 (secure 0 0 0) 0x7f92480ac030 con 0x7f92580fb240 2024-07-16T09:56:41.787 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:56:41 smithi176 ceph-mon[36626]: from='client.24477 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-16T09:56:41.787 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:56:41 smithi176 ceph-mon[36626]: from='client.24481 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-16T09:56:41.787 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:56:41 smithi176 ceph-mon[36626]: from='mgr.24377 ' entity='mgr.smithi176.dqbrvx' 2024-07-16T09:56:41.787 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:56:41 smithi176 ceph-mon[36626]: from='mgr.24377 ' entity='mgr.smithi176.dqbrvx' 2024-07-16T09:56:41.787 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:56:41 smithi176 ceph-mon[36626]: from='client.? 172.21.15.156:0/1919950663' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T09:56:42.008 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:42.006+0000 7f925e321640 1 -- 172.21.15.156:0/3117265798 --> [v2:172.21.15.176:6828/673792884,v1:172.21.15.176:6829/673792884] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f9258105010 con 0x7f924006a780 2024-07-16T09:56:42.009 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:42.008+0000 7f9254ff9640 1 -- 172.21.15.156:0/3117265798 <== mgr.24377 v2:172.21.15.176:6828/673792884 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+318 (secure 0 0 0) 0x7f9258105010 con 0x7f924006a780 2024-07-16T09:56:42.009 INFO:teuthology.orchestra.run.smithi156.stdout:{ 2024-07-16T09:56:42.009 INFO:teuthology.orchestra.run.smithi156.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:c15ec3aa8e6c0a9c12ee24ff79f3e7dd4d01b3d1a702dfeb7d7fff9b0811a393", 2024-07-16T09:56:42.009 INFO:teuthology.orchestra.run.smithi156.stdout: "in_progress": true, 2024-07-16T09:56:42.009 INFO:teuthology.orchestra.run.smithi156.stdout: "which": "Upgrading all daemon types on all hosts", 2024-07-16T09:56:42.009 INFO:teuthology.orchestra.run.smithi156.stdout: "services_complete": [], 2024-07-16T09:56:42.009 INFO:teuthology.orchestra.run.smithi156.stdout: "progress": "1/21 daemons upgraded", 2024-07-16T09:56:42.010 INFO:teuthology.orchestra.run.smithi156.stdout: "message": "", 2024-07-16T09:56:42.010 INFO:teuthology.orchestra.run.smithi156.stdout: "is_paused": false 2024-07-16T09:56:42.010 INFO:teuthology.orchestra.run.smithi156.stdout:} 2024-07-16T09:56:42.012 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:42.010+0000 7f925e321640 1 -- 172.21.15.156:0/3117265798 >> [v2:172.21.15.176:6828/673792884,v1:172.21.15.176:6829/673792884] conn(0x7f924006a780 msgr2=0x7f924006cc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:56:42.012 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:42.010+0000 7f925e321640 1 --2- 172.21.15.156:0/3117265798 >> [v2:172.21.15.176:6828/673792884,v1:172.21.15.176:6829/673792884] conn(0x7f924006a780 0x7f924006cc30 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f924c004d40 tx=0x7f924c00b040 comp rx=0 tx=0).stop 2024-07-16T09:56:42.012 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:42.010+0000 7f925e321640 1 -- 172.21.15.156:0/3117265798 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f92580fb240 msgr2=0x7f925810ad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:56:42.012 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:42.010+0000 7f925e321640 1 --2- 172.21.15.156:0/3117265798 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f92580fb240 0x7f925810ad90 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f92480041d0 tx=0x7f9248009da0 comp rx=0 tx=0).stop 2024-07-16T09:56:42.012 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:42.010+0000 7f925e321640 1 -- 172.21.15.156:0/3117265798 shutdown_connections 2024-07-16T09:56:42.012 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:42.010+0000 7f925e321640 1 --2- 172.21.15.156:0/3117265798 >> [v2:172.21.15.176:6828/673792884,v1:172.21.15.176:6829/673792884] conn(0x7f924006a780 0x7f924006cc30 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:56:42.012 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:42.010+0000 7f925e321640 1 --2- 172.21.15.156:0/3117265798 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f92580fc510 0x7f925810b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:56:42.012 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:42.010+0000 7f925e321640 1 --2- 172.21.15.156:0/3117265798 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f92580fb240 0x7f925810ad90 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:56:42.012 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:42.010+0000 7f925e321640 1 -- 172.21.15.156:0/3117265798 >> 172.21.15.156:0/3117265798 conn(0x7f92580f6a20 msgr2=0x7f92580f8050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:56:42.013 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:42.011+0000 7f925e321640 1 -- 172.21.15.156:0/3117265798 shutdown_connections 2024-07-16T09:56:42.013 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:42.011+0000 7f925e321640 1 -- 172.21.15.156:0/3117265798 wait complete. 2024-07-16T09:56:42.110 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:42.108+0000 7fd2cc4e3640 1 -- 172.21.15.156:0/1801444963 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd2c40fb240 msgr2=0x7fd2c40fb650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:56:42.110 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:42.108+0000 7fd2cc4e3640 1 --2- 172.21.15.156:0/1801444963 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd2c40fb240 0x7fd2c40fb650 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fd2c00048d0 tx=0x7fd2c0009da0 comp rx=0 tx=0).stop 2024-07-16T09:56:42.110 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:42.109+0000 7fd2cc4e3640 1 -- 172.21.15.156:0/1801444963 shutdown_connections 2024-07-16T09:56:42.110 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:42.109+0000 7fd2cc4e3640 1 --2- 172.21.15.156:0/1801444963 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd2c40fc510 0x7fd2c40fc980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:56:42.111 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:42.109+0000 7fd2cc4e3640 1 --2- 172.21.15.156:0/1801444963 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd2c40fb240 0x7fd2c40fb650 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:56:42.111 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:42.109+0000 7fd2cc4e3640 1 -- 172.21.15.156:0/1801444963 >> 172.21.15.156:0/1801444963 conn(0x7fd2c40f6a20 msgr2=0x7fd2c40f8e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:56:42.111 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:42.109+0000 7fd2cc4e3640 1 -- 172.21.15.156:0/1801444963 shutdown_connections 2024-07-16T09:56:42.111 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:42.109+0000 7fd2cc4e3640 1 -- 172.21.15.156:0/1801444963 wait complete. 2024-07-16T09:56:42.112 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:42.109+0000 7fd2cc4e3640 1 Processor -- start 2024-07-16T09:56:42.112 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:42.110+0000 7fd2cc4e3640 1 -- start start 2024-07-16T09:56:42.112 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:42.110+0000 7fd2cc4e3640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd2c40fb240 0x7fd2c410fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:56:42.113 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:42.110+0000 7fd2cc4e3640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd2c40fc510 0x7fd2c41121d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:56:42.113 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:42.110+0000 7fd2cc4e3640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2c4065a70 con 0x7fd2c40fc510 2024-07-16T09:56:42.113 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:42.110+0000 7fd2cc4e3640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2c4065be0 con 0x7fd2c40fb240 2024-07-16T09:56:42.113 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:42.110+0000 7fd2c9a7e640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd2c40fc510 0x7fd2c41121d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:56:42.113 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:42.110+0000 7fd2c9a7e640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd2c40fc510 0x7fd2c41121d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:36272/0 (socket says 172.21.15.156:36272) 2024-07-16T09:56:42.113 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:42.110+0000 7fd2c9a7e640 1 -- 172.21.15.156:0/2422751304 learned_addr learned my addr 172.21.15.156:0/2422751304 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T09:56:42.113 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:42.110+0000 7fd2ca27f640 1 --2- 172.21.15.156:0/2422751304 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd2c40fb240 0x7fd2c410fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:56:42.113 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:42.110+0000 7fd2c9a7e640 1 -- 172.21.15.156:0/2422751304 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd2c40fb240 msgr2=0x7fd2c410fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:56:42.113 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:42.110+0000 7fd2c9a7e640 1 --2- 172.21.15.156:0/2422751304 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd2c40fb240 0x7fd2c410fc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:56:42.113 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:42.110+0000 7fd2c9a7e640 1 -- 172.21.15.156:0/2422751304 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2c0005040 con 0x7fd2c40fc510 2024-07-16T09:56:42.113 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:42.111+0000 7fd2c9a7e640 1 --2- 172.21.15.156:0/2422751304 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd2c40fc510 0x7fd2c41121d0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fd2b400eb70 tx=0x7fd2b400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:56:42.113 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:42.111+0000 7fd2baffd640 1 -- 172.21.15.156:0/2422751304 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd2b4007c30 con 0x7fd2c40fc510 2024-07-16T09:56:42.113 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:42.111+0000 7fd2baffd640 1 -- 172.21.15.156:0/2422751304 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd2b400f040 con 0x7fd2c40fc510 2024-07-16T09:56:42.114 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:42.111+0000 7fd2cc4e3640 1 -- 172.21.15.156:0/2422751304 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2c4112710 con 0x7fd2c40fc510 2024-07-16T09:56:42.114 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:42.111+0000 7fd2cc4e3640 1 -- 172.21.15.156:0/2422751304 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2c4112c80 con 0x7fd2c40fc510 2024-07-16T09:56:42.114 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:42.111+0000 7fd2baffd640 1 -- 172.21.15.156:0/2422751304 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd2b40095f0 con 0x7fd2c40fc510 2024-07-16T09:56:42.115 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:42.112+0000 7fd2cc4e3640 1 -- 172.21.15.156:0/2422751304 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2a8005240 con 0x7fd2c40fc510 2024-07-16T09:56:42.115 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:42.113+0000 7fd2baffd640 1 -- 172.21.15.156:0/2422751304 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 25) v1 ==== 88617+0+0 (secure 0 0 0) 0x7fd2b400c6f0 con 0x7fd2c40fc510 2024-07-16T09:56:42.115 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:42.113+0000 7fd2baffd640 1 --2- 172.21.15.156:0/2422751304 >> [v2:172.21.15.176:6828/673792884,v1:172.21.15.176:6829/673792884] conn(0x7fd2b006a6d0 0x7fd2b006cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:56:42.115 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:42.113+0000 7fd2baffd640 1 -- 172.21.15.156:0/2422751304 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5722+0+0 (secure 0 0 0) 0x7fd2b4088c00 con 0x7fd2c40fc510 2024-07-16T09:56:42.115 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:42.114+0000 7fd2ca27f640 1 --2- 172.21.15.156:0/2422751304 >> [v2:172.21.15.176:6828/673792884,v1:172.21.15.176:6829/673792884] conn(0x7fd2b006a6d0 0x7fd2b006cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:56:42.115 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:42.114+0000 7fd2ca27f640 1 --2- 172.21.15.156:0/2422751304 >> [v2:172.21.15.176:6828/673792884,v1:172.21.15.176:6829/673792884] conn(0x7fd2b006a6d0 0x7fd2b006cb80 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fd2c0009f80 tx=0x7fd2c0035040 comp rx=0 tx=0).ready entity=mgr.24377 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:56:42.119 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:42.118+0000 7fd2baffd640 1 -- 172.21.15.156:0/2422751304 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185936 (secure 0 0 0) 0x7fd2b4050660 con 0x7fd2c40fc510 2024-07-16T09:56:42.537 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:42.535+0000 7fd2cc4e3640 1 -- 172.21.15.156:0/2422751304 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fd2a80057c0 con 0x7fd2c40fc510 2024-07-16T09:56:42.537 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:42.536+0000 7fd2baffd640 1 -- 172.21.15.156:0/2422751304 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7fd2b408e020 con 0x7fd2c40fc510 2024-07-16T09:56:42.537 INFO:teuthology.orchestra.run.smithi156.stdout:HEALTH_OK 2024-07-16T09:56:42.540 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:42.538+0000 7fd2cc4e3640 1 -- 172.21.15.156:0/2422751304 >> [v2:172.21.15.176:6828/673792884,v1:172.21.15.176:6829/673792884] conn(0x7fd2b006a6d0 msgr2=0x7fd2b006cb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:56:42.540 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:42.538+0000 7fd2cc4e3640 1 --2- 172.21.15.156:0/2422751304 >> [v2:172.21.15.176:6828/673792884,v1:172.21.15.176:6829/673792884] conn(0x7fd2b006a6d0 0x7fd2b006cb80 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fd2c0009f80 tx=0x7fd2c0035040 comp rx=0 tx=0).stop 2024-07-16T09:56:42.540 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:42.538+0000 7fd2cc4e3640 1 -- 172.21.15.156:0/2422751304 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd2c40fc510 msgr2=0x7fd2c41121d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:56:42.540 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:42.538+0000 7fd2cc4e3640 1 --2- 172.21.15.156:0/2422751304 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd2c40fc510 0x7fd2c41121d0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fd2b400eb70 tx=0x7fd2b400b040 comp rx=0 tx=0).stop 2024-07-16T09:56:42.540 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:42.538+0000 7fd2cc4e3640 1 -- 172.21.15.156:0/2422751304 shutdown_connections 2024-07-16T09:56:42.540 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:42.538+0000 7fd2cc4e3640 1 --2- 172.21.15.156:0/2422751304 >> [v2:172.21.15.176:6828/673792884,v1:172.21.15.176:6829/673792884] conn(0x7fd2b006a6d0 0x7fd2b006cb80 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:56:42.541 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:42.538+0000 7fd2cc4e3640 1 --2- 172.21.15.156:0/2422751304 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd2c40fc510 0x7fd2c41121d0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:56:42.541 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:42.538+0000 7fd2cc4e3640 1 --2- 172.21.15.156:0/2422751304 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd2c40fb240 0x7fd2c410fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:56:42.541 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:42.539+0000 7fd2cc4e3640 1 -- 172.21.15.156:0/2422751304 >> 172.21.15.156:0/2422751304 conn(0x7fd2c40f6a20 msgr2=0x7fd2c40f8110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:56:42.541 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:42.539+0000 7fd2cc4e3640 1 -- 172.21.15.156:0/2422751304 shutdown_connections 2024-07-16T09:56:42.541 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:56:42.539+0000 7fd2cc4e3640 1 -- 172.21.15.156:0/2422751304 wait complete. 2024-07-16T09:56:42.843 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:56:42 smithi176 ceph-mon[36626]: from='client.14704 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-07-16T09:56:42.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:56:42 smithi176 ceph-mon[36626]: Reconfiguring node-exporter.smithi176 (dependencies changed)... 2024-07-16T09:56:42.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:56:42 smithi176 ceph-mon[36626]: Reconfiguring daemon node-exporter.smithi176 on smithi176 2024-07-16T09:56:42.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:56:42 smithi176 ceph-mon[36626]: pgmap v29: 65 pgs: 65 active+clean; 271 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 2.5 MiB/s rd, 2.5 MiB/s wr, 718 op/s 2024-07-16T09:56:42.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:56:42 smithi176 ceph-mon[36626]: from='client.? 172.21.15.156:0/2319123684' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-07-16T09:56:42.924 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:56:42 smithi156 ceph-mon[29302]: from='client.14704 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-07-16T09:56:42.924 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:56:42 smithi156 ceph-mon[29302]: Reconfiguring node-exporter.smithi176 (dependencies changed)... 2024-07-16T09:56:42.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:56:42 smithi156 ceph-mon[29302]: Reconfiguring daemon node-exporter.smithi176 on smithi176 2024-07-16T09:56:42.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:56:42 smithi156 ceph-mon[29302]: pgmap v29: 65 pgs: 65 active+clean; 271 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 2.5 MiB/s rd, 2.5 MiB/s wr, 718 op/s 2024-07-16T09:56:42.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:56:42 smithi156 ceph-mon[29302]: from='client.? 172.21.15.156:0/2319123684' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-07-16T09:56:43.843 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:56:43 smithi176 ceph-mon[36626]: from='client.24495 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-16T09:56:43.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:56:43 smithi176 ceph-mon[36626]: from='client.? 172.21.15.156:0/2422751304' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-07-16T09:56:43.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:56:43 smithi176 ceph-mon[36626]: from='mgr.24377 ' entity='mgr.smithi176.dqbrvx' 2024-07-16T09:56:43.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:56:43 smithi176 ceph-mon[36626]: from='mgr.24377 ' entity='mgr.smithi176.dqbrvx' 2024-07-16T09:56:43.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:56:43 smithi176 ceph-mon[36626]: from='mgr.24377 172.21.15.176:0/1082993331' entity='mgr.smithi176.dqbrvx' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-07-16T09:56:43.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:56:43 smithi176 ceph-mon[36626]: from='mgr.24377 172.21.15.176:0/1082993331' entity='mgr.smithi176.dqbrvx' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-07-16T09:56:43.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:56:43 smithi176 ceph-mon[36626]: from='mgr.24377 172.21.15.176:0/1082993331' entity='mgr.smithi176.dqbrvx' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-07-16T09:56:43.924 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:56:43 smithi156 ceph-mon[29302]: from='client.24495 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-16T09:56:43.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:56:43 smithi156 ceph-mon[29302]: from='client.? 172.21.15.156:0/2422751304' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-07-16T09:56:43.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:56:43 smithi156 ceph-mon[29302]: from='mgr.24377 ' entity='mgr.smithi176.dqbrvx' 2024-07-16T09:56:43.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:56:43 smithi156 ceph-mon[29302]: from='mgr.24377 ' entity='mgr.smithi176.dqbrvx' 2024-07-16T09:56:43.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:56:43 smithi156 ceph-mon[29302]: from='mgr.24377 172.21.15.176:0/1082993331' entity='mgr.smithi176.dqbrvx' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-07-16T09:56:43.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:56:43 smithi156 ceph-mon[29302]: from='mgr.24377 172.21.15.176:0/1082993331' entity='mgr.smithi176.dqbrvx' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-07-16T09:56:43.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:56:43 smithi156 ceph-mon[29302]: from='mgr.24377 172.21.15.176:0/1082993331' entity='mgr.smithi176.dqbrvx' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-07-16T09:56:44.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:56:44 smithi176 ceph-mon[36626]: pgmap v30: 65 pgs: 65 active+clean; 271 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 496 op/s 2024-07-16T09:56:44.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:56:44 smithi176 ceph-mon[36626]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-07-16T09:56:44.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:56:44 smithi176 ceph-mon[36626]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-07-16T09:56:44.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:56:44 smithi176 ceph-mon[36626]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-07-16T09:56:44.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:56:44 smithi176 ceph-mon[36626]: from='mgr.24377 172.21.15.176:0/1082993331' entity='mgr.smithi176.dqbrvx' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-16T09:56:44.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:56:44 smithi176 ceph-mon[36626]: Upgrade: Need to upgrade myself (mgr.smithi176.dqbrvx) 2024-07-16T09:56:44.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:56:44 smithi176 ceph-mon[36626]: Upgrade: Need to upgrade myself (mgr.smithi176.dqbrvx) 2024-07-16T09:56:44.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:56:44 smithi156 ceph-mon[29302]: pgmap v30: 65 pgs: 65 active+clean; 271 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 496 op/s 2024-07-16T09:56:44.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:56:44 smithi156 ceph-mon[29302]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-07-16T09:56:44.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:56:44 smithi156 ceph-mon[29302]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-07-16T09:56:44.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:56:44 smithi156 ceph-mon[29302]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-07-16T09:56:44.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:56:44 smithi156 ceph-mon[29302]: from='mgr.24377 172.21.15.176:0/1082993331' entity='mgr.smithi176.dqbrvx' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-16T09:56:44.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:56:44 smithi156 ceph-mon[29302]: Upgrade: Need to upgrade myself (mgr.smithi176.dqbrvx) 2024-07-16T09:56:44.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:56:44 smithi156 ceph-mon[29302]: Upgrade: Need to upgrade myself (mgr.smithi176.dqbrvx) 2024-07-16T09:56:46.334 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:56:46 smithi156 ceph-mon[29302]: Upgrade: Updating mgr.smithi156.zkvylv 2024-07-16T09:56:46.334 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:56:46 smithi156 ceph-mon[29302]: from='mgr.24377 ' entity='mgr.smithi176.dqbrvx' 2024-07-16T09:56:46.334 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:56:46 smithi156 ceph-mon[29302]: from='mgr.24377 172.21.15.176:0/1082993331' entity='mgr.smithi176.dqbrvx' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi156.zkvylv", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-07-16T09:56:46.334 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:56:46 smithi156 ceph-mon[29302]: from='mgr.24377 ' entity='mgr.smithi176.dqbrvx' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi156.zkvylv", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-07-16T09:56:46.334 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:56:46 smithi156 ceph-mon[29302]: from='mgr.24377 172.21.15.176:0/1082993331' entity='mgr.smithi176.dqbrvx' cmd=[{"prefix": "mgr services"}]: dispatch 2024-07-16T09:56:46.334 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:56:46 smithi156 ceph-mon[29302]: from='mgr.24377 172.21.15.176:0/1082993331' entity='mgr.smithi176.dqbrvx' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-16T09:56:46.334 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:56:46 smithi156 ceph-mon[29302]: Deploying daemon mgr.smithi156.zkvylv on smithi156 2024-07-16T09:56:46.334 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:56:46 smithi156 ceph-mon[29302]: pgmap v31: 65 pgs: 65 active+clean; 268 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 2.3 MiB/s rd, 2.4 MiB/s wr, 738 op/s 2024-07-16T09:56:46.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:56:46 smithi176 ceph-mon[36626]: Upgrade: Updating mgr.smithi156.zkvylv 2024-07-16T09:56:46.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:56:46 smithi176 ceph-mon[36626]: from='mgr.24377 ' entity='mgr.smithi176.dqbrvx' 2024-07-16T09:56:46.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:56:46 smithi176 ceph-mon[36626]: from='mgr.24377 172.21.15.176:0/1082993331' entity='mgr.smithi176.dqbrvx' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi156.zkvylv", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-07-16T09:56:46.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:56:46 smithi176 ceph-mon[36626]: from='mgr.24377 ' entity='mgr.smithi176.dqbrvx' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi156.zkvylv", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-07-16T09:56:46.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:56:46 smithi176 ceph-mon[36626]: from='mgr.24377 172.21.15.176:0/1082993331' entity='mgr.smithi176.dqbrvx' cmd=[{"prefix": "mgr services"}]: dispatch 2024-07-16T09:56:46.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:56:46 smithi176 ceph-mon[36626]: from='mgr.24377 172.21.15.176:0/1082993331' entity='mgr.smithi176.dqbrvx' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-16T09:56:46.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:56:46 smithi176 ceph-mon[36626]: Deploying daemon mgr.smithi156.zkvylv on smithi156 2024-07-16T09:56:46.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:56:46 smithi176 ceph-mon[36626]: pgmap v31: 65 pgs: 65 active+clean; 268 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 2.3 MiB/s rd, 2.4 MiB/s wr, 738 op/s 2024-07-16T09:56:48.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:56:48 smithi176 ceph-mon[36626]: pgmap v32: 65 pgs: 65 active+clean; 268 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.5 MiB/s rd, 1.7 MiB/s wr, 465 op/s 2024-07-16T09:56:48.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:56:48 smithi156 ceph-mon[29302]: pgmap v32: 65 pgs: 65 active+clean; 268 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.5 MiB/s rd, 1.7 MiB/s wr, 465 op/s 2024-07-16T09:56:50.780 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:56:50 smithi156 ceph-mon[29302]: pgmap v33: 65 pgs: 65 active+clean; 268 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.5 MiB/s rd, 1.7 MiB/s wr, 465 op/s 2024-07-16T09:56:50.781 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:56:50 smithi156 ceph-mon[29302]: from='mgr.24377 172.21.15.176:0/1082993331' entity='mgr.smithi176.dqbrvx' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-16T09:56:50.843 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:56:50 smithi176 ceph-mon[36626]: pgmap v33: 65 pgs: 65 active+clean; 268 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.5 MiB/s rd, 1.7 MiB/s wr, 465 op/s 2024-07-16T09:56:50.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:56:50 smithi176 ceph-mon[36626]: from='mgr.24377 172.21.15.176:0/1082993331' entity='mgr.smithi176.dqbrvx' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-16T09:56:52.113 INFO:tasks.workunit:Stopping ['suites/fsstress.sh'] on client.1... 2024-07-16T09:56:52.114 DEBUG:teuthology.orchestra.run.smithi176:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.1 /home/ubuntu/cephtest/clone.client.1 2024-07-16T09:56:52.520 DEBUG:teuthology.parallel:result is None 2024-07-16T09:56:52.701 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:56:52 smithi156 ceph-mon[29302]: pgmap v34: 65 pgs: 65 active+clean; 264 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.3 MiB/s rd, 2.5 MiB/s wr, 705 op/s 2024-07-16T09:56:52.701 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:56:52 smithi156 ceph-mon[29302]: from='mgr.24377 ' entity='mgr.smithi176.dqbrvx' 2024-07-16T09:56:52.701 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:56:52 smithi156 ceph-mon[29302]: from='mgr.24377 ' entity='mgr.smithi176.dqbrvx' 2024-07-16T09:56:52.701 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:56:52 smithi156 ceph-mon[29302]: from='mgr.24377 172.21.15.176:0/1082993331' entity='mgr.smithi176.dqbrvx' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-16T09:56:52.703 INFO:tasks.workunit:Stopping ['suites/fsstress.sh'] on client.0... 2024-07-16T09:56:52.703 DEBUG:teuthology.orchestra.run.smithi156:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-07-16T09:56:52.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:56:52 smithi176 ceph-mon[36626]: pgmap v34: 65 pgs: 65 active+clean; 264 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.3 MiB/s rd, 2.5 MiB/s wr, 705 op/s 2024-07-16T09:56:52.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:56:52 smithi176 ceph-mon[36626]: from='mgr.24377 ' entity='mgr.smithi176.dqbrvx' 2024-07-16T09:56:52.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:56:52 smithi176 ceph-mon[36626]: from='mgr.24377 ' entity='mgr.smithi176.dqbrvx' 2024-07-16T09:56:52.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:56:52 smithi176 ceph-mon[36626]: from='mgr.24377 172.21.15.176:0/1082993331' entity='mgr.smithi176.dqbrvx' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-16T09:56:52.992 DEBUG:teuthology.parallel:result is None 2024-07-16T09:56:52.993 DEBUG:teuthology.orchestra.run.smithi156:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-07-16T09:56:53.057 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-07-16T09:56:53.057 DEBUG:teuthology.orchestra.run.smithi176:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.1/client.1 2024-07-16T09:56:53.094 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.1/client.1 2024-07-16T09:56:53.094 DEBUG:teuthology.parallel:result is None 2024-07-16T09:56:54.780 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:56:54 smithi176 ceph-mon[36626]: pgmap v35: 65 pgs: 65 active+clean; 264 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 481 op/s 2024-07-16T09:56:54.873 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:56:54 smithi156 ceph-mon[29302]: pgmap v35: 65 pgs: 65 active+clean; 264 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 481 op/s 2024-07-16T09:56:56.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:56:56 smithi156 ceph-mon[29302]: pgmap v36: 65 pgs: 65 active+clean; 253 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.0 MiB/s rd, 2.0 MiB/s wr, 667 op/s 2024-07-16T09:56:56.843 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:56:56 smithi176 ceph-mon[36626]: pgmap v36: 65 pgs: 65 active+clean; 253 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.0 MiB/s rd, 2.0 MiB/s wr, 667 op/s 2024-07-16T09:56:57.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:56:57 smithi156 ceph-mon[29302]: Standby manager daemon smithi156.zkvylv restarted 2024-07-16T09:56:57.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:56:57 smithi156 ceph-mon[29302]: Standby manager daemon smithi156.zkvylv started 2024-07-16T09:56:57.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:56:57 smithi156 ceph-mon[29302]: from='mgr.? 172.21.15.156:0/647865130' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi156.zkvylv/crt"}]: dispatch 2024-07-16T09:56:57.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:56:57 smithi156 ceph-mon[29302]: from='mgr.? 172.21.15.156:0/647865130' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-07-16T09:56:57.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:56:57 smithi156 ceph-mon[29302]: from='mgr.? 172.21.15.156:0/647865130' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi156.zkvylv/key"}]: dispatch 2024-07-16T09:56:57.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:56:57 smithi156 ceph-mon[29302]: from='mgr.? 172.21.15.156:0/647865130' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-07-16T09:56:57.818 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:56:57 smithi176 ceph-mon[36626]: Standby manager daemon smithi156.zkvylv restarted 2024-07-16T09:56:57.818 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:56:57 smithi176 ceph-mon[36626]: Standby manager daemon smithi156.zkvylv started 2024-07-16T09:56:57.818 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:56:57 smithi176 ceph-mon[36626]: from='mgr.? 172.21.15.156:0/647865130' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi156.zkvylv/crt"}]: dispatch 2024-07-16T09:56:57.818 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:56:57 smithi176 ceph-mon[36626]: from='mgr.? 172.21.15.156:0/647865130' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-07-16T09:56:57.819 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:56:57 smithi176 ceph-mon[36626]: from='mgr.? 172.21.15.156:0/647865130' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi156.zkvylv/key"}]: dispatch 2024-07-16T09:56:57.819 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:56:57 smithi176 ceph-mon[36626]: from='mgr.? 172.21.15.156:0/647865130' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-07-16T09:56:58.843 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:56:58 smithi176 ceph-mon[36626]: pgmap v37: 65 pgs: 65 active+clean; 253 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 425 op/s 2024-07-16T09:56:58.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:56:58 smithi176 ceph-mon[36626]: mgrmap e26: smithi176.dqbrvx(active, since 68s), standbys: smithi156.zkvylv 2024-07-16T09:56:58.924 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:56:58 smithi156 ceph-mon[29302]: pgmap v37: 65 pgs: 65 active+clean; 253 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 425 op/s 2024-07-16T09:56:58.924 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:56:58 smithi156 ceph-mon[29302]: mgrmap e26: smithi176.dqbrvx(active, since 68s), standbys: smithi156.zkvylv 2024-07-16T09:56:59.843 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:56:59 smithi176 ceph-mon[36626]: from='mgr.24377 ' entity='mgr.smithi176.dqbrvx' 2024-07-16T09:56:59.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:56:59 smithi176 ceph-mon[36626]: from='mgr.24377 ' entity='mgr.smithi176.dqbrvx' 2024-07-16T09:56:59.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:56:59 smithi156 ceph-mon[29302]: from='mgr.24377 ' entity='mgr.smithi176.dqbrvx' 2024-07-16T09:56:59.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:56:59 smithi156 ceph-mon[29302]: from='mgr.24377 ' entity='mgr.smithi176.dqbrvx' 2024-07-16T09:57:00.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:00 smithi176 ceph-mon[36626]: pgmap v38: 65 pgs: 65 active+clean; 253 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 425 op/s 2024-07-16T09:57:00.924 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:00 smithi156 ceph-mon[29302]: pgmap v38: 65 pgs: 65 active+clean; 253 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 425 op/s 2024-07-16T09:57:02.646 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:02 smithi176 ceph-mon[36626]: pgmap v39: 65 pgs: 65 active+clean; 253 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 430 op/s 2024-07-16T09:57:02.886 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:02 smithi156 ceph-mon[29302]: pgmap v39: 65 pgs: 65 active+clean; 253 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 430 op/s 2024-07-16T09:57:04.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:04 smithi176 ceph-mon[36626]: pgmap v40: 65 pgs: 65 active+clean; 253 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 499 KiB/s rd, 453 KiB/s wr, 190 op/s 2024-07-16T09:57:04.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:04 smithi176 ceph-mon[36626]: from='mgr.24377 ' entity='mgr.smithi176.dqbrvx' 2024-07-16T09:57:04.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:04 smithi176 ceph-mon[36626]: from='mgr.24377 ' entity='mgr.smithi176.dqbrvx' 2024-07-16T09:57:04.924 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:04 smithi156 ceph-mon[29302]: pgmap v40: 65 pgs: 65 active+clean; 253 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 499 KiB/s rd, 453 KiB/s wr, 190 op/s 2024-07-16T09:57:04.924 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:04 smithi156 ceph-mon[29302]: from='mgr.24377 ' entity='mgr.smithi176.dqbrvx' 2024-07-16T09:57:04.924 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:04 smithi156 ceph-mon[29302]: from='mgr.24377 ' entity='mgr.smithi176.dqbrvx' 2024-07-16T09:57:05.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:05 smithi156 ceph-mon[29302]: from='mgr.24377 ' entity='mgr.smithi176.dqbrvx' 2024-07-16T09:57:05.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:05 smithi156 ceph-mon[29302]: from='mgr.24377 ' entity='mgr.smithi176.dqbrvx' 2024-07-16T09:57:05.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:05 smithi156 ceph-mon[29302]: from='mgr.24377 172.21.15.176:0/1082993331' entity='mgr.smithi176.dqbrvx' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-16T09:57:05.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:05 smithi156 ceph-mon[29302]: pgmap v41: 65 pgs: 65 active+clean; 253 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 500 KiB/s rd, 453 KiB/s wr, 193 op/s 2024-07-16T09:57:06.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:05 smithi176 ceph-mon[36626]: from='mgr.24377 ' entity='mgr.smithi176.dqbrvx' 2024-07-16T09:57:06.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:05 smithi176 ceph-mon[36626]: from='mgr.24377 ' entity='mgr.smithi176.dqbrvx' 2024-07-16T09:57:06.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:05 smithi176 ceph-mon[36626]: from='mgr.24377 172.21.15.176:0/1082993331' entity='mgr.smithi176.dqbrvx' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-16T09:57:06.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:05 smithi176 ceph-mon[36626]: pgmap v41: 65 pgs: 65 active+clean; 253 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 500 KiB/s rd, 453 KiB/s wr, 193 op/s 2024-07-16T09:57:08.538 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:08 smithi176 ceph-mon[36626]: pgmap v42: 65 pgs: 65 active+clean; 253 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 19 KiB/s rd, 17 KiB/s wr, 7 op/s 2024-07-16T09:57:08.538 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:08 smithi176 ceph-mon[36626]: from='mgr.24377 ' entity='mgr.smithi176.dqbrvx' 2024-07-16T09:57:08.538 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:08 smithi176 ceph-mon[36626]: from='mgr.24377 ' entity='mgr.smithi176.dqbrvx' 2024-07-16T09:57:08.538 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:08 smithi176 ceph-mon[36626]: from='mgr.24377 172.21.15.176:0/1082993331' entity='mgr.smithi176.dqbrvx' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-16T09:57:08.538 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:08 smithi176 ceph-mon[36626]: from='mgr.24377 172.21.15.176:0/1082993331' entity='mgr.smithi176.dqbrvx' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-16T09:57:08.538 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:08 smithi176 ceph-mon[36626]: from='mgr.24377 ' entity='mgr.smithi176.dqbrvx' 2024-07-16T09:57:08.538 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:08 smithi176 ceph-mon[36626]: from='mgr.24377 172.21.15.176:0/1082993331' entity='mgr.smithi176.dqbrvx' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-16T09:57:08.539 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:08 smithi176 ceph-mon[36626]: from='mgr.24377 172.21.15.176:0/1082993331' entity='mgr.smithi176.dqbrvx' cmd=[{"prefix": "mgr fail", "who": "smithi176.dqbrvx"}]: dispatch 2024-07-16T09:57:08.539 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:08 smithi176 ceph-mon[36626]: from='mgr.24377 ' entity='mgr.smithi176.dqbrvx' cmd=[{"prefix": "mgr fail", "who": "smithi176.dqbrvx"}]: dispatch 2024-07-16T09:57:08.539 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:08 smithi176 ceph-mon[36626]: Activating manager daemon smithi156.zkvylv 2024-07-16T09:57:08.539 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:08 smithi176 ceph-mon[36626]: osdmap e48: 6 total, 6 up, 6 in 2024-07-16T09:57:08.539 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:08 smithi176 ceph-mon[36626]: from='mgr.24377 ' entity='mgr.smithi176.dqbrvx' cmd='[{"prefix": "mgr fail", "who": "smithi176.dqbrvx"}]': finished 2024-07-16T09:57:08.539 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:08 smithi176 ceph-mon[36626]: mgrmap e27: smithi156.zkvylv(active, starting, since 0.00641853s) 2024-07-16T09:57:08.539 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:08 smithi176 ceph-mon[36626]: from='mgr.14722 172.21.15.156:0/647865130' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "mon metadata", "id": "smithi156"}]: dispatch 2024-07-16T09:57:08.539 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:08 smithi176 ceph-mon[36626]: from='mgr.14722 172.21.15.156:0/647865130' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "mon metadata", "id": "smithi176"}]: dispatch 2024-07-16T09:57:08.539 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:08 smithi176 ceph-mon[36626]: from='mgr.14722 172.21.15.156:0/647865130' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi156.rdibys"}]: dispatch 2024-07-16T09:57:08.539 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:08 smithi176 ceph-mon[36626]: from='mgr.14722 172.21.15.156:0/647865130' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi156.evwwaz"}]: dispatch 2024-07-16T09:57:08.539 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:08 smithi176 ceph-mon[36626]: from='mgr.14722 172.21.15.156:0/647865130' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi176.dvvbud"}]: dispatch 2024-07-16T09:57:08.539 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:08 smithi176 ceph-mon[36626]: from='mgr.14722 172.21.15.156:0/647865130' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi176.wahbsi"}]: dispatch 2024-07-16T09:57:08.540 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:08 smithi176 ceph-mon[36626]: from='mgr.14722 172.21.15.156:0/647865130' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "mgr metadata", "who": "smithi156.zkvylv", "id": "smithi156.zkvylv"}]: dispatch 2024-07-16T09:57:08.540 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:08 smithi176 ceph-mon[36626]: from='mgr.14722 172.21.15.156:0/647865130' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-07-16T09:57:08.540 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:08 smithi176 ceph-mon[36626]: from='mgr.14722 172.21.15.156:0/647865130' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-07-16T09:57:08.540 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:08 smithi176 ceph-mon[36626]: from='mgr.14722 172.21.15.156:0/647865130' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-07-16T09:57:08.540 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:08 smithi176 ceph-mon[36626]: from='mgr.14722 172.21.15.156:0/647865130' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-07-16T09:57:08.540 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:08 smithi176 ceph-mon[36626]: from='mgr.14722 172.21.15.156:0/647865130' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-07-16T09:57:08.540 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:08 smithi176 ceph-mon[36626]: from='mgr.14722 172.21.15.156:0/647865130' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-07-16T09:57:08.540 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:08 smithi176 ceph-mon[36626]: from='mgr.14722 172.21.15.156:0/647865130' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-07-16T09:57:08.540 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:08 smithi176 ceph-mon[36626]: from='mgr.14722 172.21.15.156:0/647865130' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-07-16T09:57:08.540 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:08 smithi176 ceph-mon[36626]: from='mgr.14722 172.21.15.156:0/647865130' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-07-16T09:57:08.745 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:08 smithi156 ceph-mon[29302]: pgmap v42: 65 pgs: 65 active+clean; 253 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 19 KiB/s rd, 17 KiB/s wr, 7 op/s 2024-07-16T09:57:08.745 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:08 smithi156 ceph-mon[29302]: from='mgr.24377 ' entity='mgr.smithi176.dqbrvx' 2024-07-16T09:57:08.745 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:08 smithi156 ceph-mon[29302]: from='mgr.24377 ' entity='mgr.smithi176.dqbrvx' 2024-07-16T09:57:08.745 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:08 smithi156 ceph-mon[29302]: from='mgr.24377 172.21.15.176:0/1082993331' entity='mgr.smithi176.dqbrvx' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-16T09:57:08.745 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:08 smithi156 ceph-mon[29302]: from='mgr.24377 172.21.15.176:0/1082993331' entity='mgr.smithi176.dqbrvx' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-16T09:57:08.745 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:08 smithi156 ceph-mon[29302]: from='mgr.24377 ' entity='mgr.smithi176.dqbrvx' 2024-07-16T09:57:08.745 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:08 smithi156 ceph-mon[29302]: from='mgr.24377 172.21.15.176:0/1082993331' entity='mgr.smithi176.dqbrvx' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-16T09:57:08.745 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:08 smithi156 ceph-mon[29302]: from='mgr.24377 172.21.15.176:0/1082993331' entity='mgr.smithi176.dqbrvx' cmd=[{"prefix": "mgr fail", "who": "smithi176.dqbrvx"}]: dispatch 2024-07-16T09:57:08.746 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:08 smithi156 ceph-mon[29302]: from='mgr.24377 ' entity='mgr.smithi176.dqbrvx' cmd=[{"prefix": "mgr fail", "who": "smithi176.dqbrvx"}]: dispatch 2024-07-16T09:57:08.746 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:08 smithi156 ceph-mon[29302]: Activating manager daemon smithi156.zkvylv 2024-07-16T09:57:08.746 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:08 smithi156 ceph-mon[29302]: osdmap e48: 6 total, 6 up, 6 in 2024-07-16T09:57:08.746 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:08 smithi156 ceph-mon[29302]: from='mgr.24377 ' entity='mgr.smithi176.dqbrvx' cmd='[{"prefix": "mgr fail", "who": "smithi176.dqbrvx"}]': finished 2024-07-16T09:57:08.746 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:08 smithi156 ceph-mon[29302]: mgrmap e27: smithi156.zkvylv(active, starting, since 0.00641853s) 2024-07-16T09:57:08.746 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:08 smithi156 ceph-mon[29302]: from='mgr.14722 172.21.15.156:0/647865130' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "mon metadata", "id": "smithi156"}]: dispatch 2024-07-16T09:57:08.746 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:08 smithi156 ceph-mon[29302]: from='mgr.14722 172.21.15.156:0/647865130' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "mon metadata", "id": "smithi176"}]: dispatch 2024-07-16T09:57:08.746 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:08 smithi156 ceph-mon[29302]: from='mgr.14722 172.21.15.156:0/647865130' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi156.rdibys"}]: dispatch 2024-07-16T09:57:08.746 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:08 smithi156 ceph-mon[29302]: from='mgr.14722 172.21.15.156:0/647865130' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi156.evwwaz"}]: dispatch 2024-07-16T09:57:08.746 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:08 smithi156 ceph-mon[29302]: from='mgr.14722 172.21.15.156:0/647865130' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi176.dvvbud"}]: dispatch 2024-07-16T09:57:08.746 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:08 smithi156 ceph-mon[29302]: from='mgr.14722 172.21.15.156:0/647865130' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi176.wahbsi"}]: dispatch 2024-07-16T09:57:08.746 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:08 smithi156 ceph-mon[29302]: from='mgr.14722 172.21.15.156:0/647865130' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "mgr metadata", "who": "smithi156.zkvylv", "id": "smithi156.zkvylv"}]: dispatch 2024-07-16T09:57:08.747 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:08 smithi156 ceph-mon[29302]: from='mgr.14722 172.21.15.156:0/647865130' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-07-16T09:57:08.747 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:08 smithi156 ceph-mon[29302]: from='mgr.14722 172.21.15.156:0/647865130' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-07-16T09:57:08.747 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:08 smithi156 ceph-mon[29302]: from='mgr.14722 172.21.15.156:0/647865130' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-07-16T09:57:08.747 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:08 smithi156 ceph-mon[29302]: from='mgr.14722 172.21.15.156:0/647865130' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-07-16T09:57:08.747 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:08 smithi156 ceph-mon[29302]: from='mgr.14722 172.21.15.156:0/647865130' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-07-16T09:57:08.747 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:08 smithi156 ceph-mon[29302]: from='mgr.14722 172.21.15.156:0/647865130' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-07-16T09:57:08.747 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:08 smithi156 ceph-mon[29302]: from='mgr.14722 172.21.15.156:0/647865130' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-07-16T09:57:08.747 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:08 smithi156 ceph-mon[29302]: from='mgr.14722 172.21.15.156:0/647865130' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-07-16T09:57:08.747 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:08 smithi156 ceph-mon[29302]: from='mgr.14722 172.21.15.156:0/647865130' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-07-16T09:57:09.776 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:09 smithi176 ceph-mon[36626]: Manager daemon smithi156.zkvylv is now available 2024-07-16T09:57:09.776 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:09 smithi176 ceph-mon[36626]: from='mgr.14722 172.21.15.156:0/647865130' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-16T09:57:09.776 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:09 smithi176 ceph-mon[36626]: from='mgr.14722 172.21.15.156:0/647865130' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-16T09:57:09.776 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:09 smithi176 ceph-mon[36626]: from='mgr.14722 172.21.15.156:0/647865130' entity='mgr.smithi156.zkvylv' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi156.zkvylv/mirror_snapshot_schedule"}]: dispatch 2024-07-16T09:57:09.776 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:09 smithi176 ceph-mon[36626]: from='mgr.14722 172.21.15.156:0/647865130' entity='mgr.smithi156.zkvylv' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi156.zkvylv/trash_purge_schedule"}]: dispatch 2024-07-16T09:57:09.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:09 smithi156 ceph-mon[29302]: Manager daemon smithi156.zkvylv is now available 2024-07-16T09:57:09.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:09 smithi156 ceph-mon[29302]: from='mgr.14722 172.21.15.156:0/647865130' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-16T09:57:09.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:09 smithi156 ceph-mon[29302]: from='mgr.14722 172.21.15.156:0/647865130' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-16T09:57:09.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:09 smithi156 ceph-mon[29302]: from='mgr.14722 172.21.15.156:0/647865130' entity='mgr.smithi156.zkvylv' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi156.zkvylv/mirror_snapshot_schedule"}]: dispatch 2024-07-16T09:57:09.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:09 smithi156 ceph-mon[29302]: from='mgr.14722 172.21.15.156:0/647865130' entity='mgr.smithi156.zkvylv' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi156.zkvylv/trash_purge_schedule"}]: dispatch 2024-07-16T09:57:10.639 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:10 smithi176 ceph-mon[36626]: mgrmap e28: smithi156.zkvylv(active, since 1.20695s) 2024-07-16T09:57:10.639 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:10 smithi176 ceph-mon[36626]: pgmap v3: 65 pgs: 65 active+clean; 253 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-07-16T09:57:10.639 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:10 smithi176 ceph-mon[36626]: from='mgr.14722 172.21.15.156:0/647865130' entity='mgr.smithi156.zkvylv' 2024-07-16T09:57:10.639 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:10 smithi176 ceph-mon[36626]: from='mgr.14722 172.21.15.156:0/647865130' entity='mgr.smithi156.zkvylv' 2024-07-16T09:57:10.695 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:10 smithi156 ceph-mon[29302]: mgrmap e28: smithi156.zkvylv(active, since 1.20695s) 2024-07-16T09:57:10.695 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:10 smithi156 ceph-mon[29302]: pgmap v3: 65 pgs: 65 active+clean; 253 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-07-16T09:57:10.696 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:10 smithi156 ceph-mon[29302]: from='mgr.14722 172.21.15.156:0/647865130' entity='mgr.smithi156.zkvylv' 2024-07-16T09:57:10.696 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:10 smithi156 ceph-mon[29302]: from='mgr.14722 172.21.15.156:0/647865130' entity='mgr.smithi156.zkvylv' 2024-07-16T09:57:11.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:11 smithi176 ceph-mon[36626]: [16/Jul/2024:09:57:10] ENGINE Bus STARTING 2024-07-16T09:57:11.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:11 smithi176 ceph-mon[36626]: [16/Jul/2024:09:57:10] ENGINE Serving on https://172.21.15.156:7150 2024-07-16T09:57:11.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:11 smithi176 ceph-mon[36626]: [16/Jul/2024:09:57:10] ENGINE Error in HTTPServer.serve 2024-07-16T09:57:11.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: Traceback (most recent call last): 2024-07-16T09:57:11.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-07-16T09:57:11.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: self._connections.run(self.expiration_interval) 2024-07-16T09:57:11.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-07-16T09:57:11.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: self._run(expiration_interval) 2024-07-16T09:57:11.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-07-16T09:57:11.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-07-16T09:57:11.845 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-07-16T09:57:11.845 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-07-16T09:57:11.845 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-07-16T09:57:11.845 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: s = self.context.wrap_socket( 2024-07-16T09:57:11.845 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-07-16T09:57:11.845 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: return self.sslsocket_class._create( 2024-07-16T09:57:11.845 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-07-16T09:57:11.845 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: self.do_handshake() 2024-07-16T09:57:11.845 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-07-16T09:57:11.845 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: self._sslobj.do_handshake() 2024-07-16T09:57:11.845 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-07-16T09:57:11.845 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:11 smithi176 ceph-mon[36626]: pgmap v4: 65 pgs: 65 active+clean; 253 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-07-16T09:57:11.845 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:11 smithi176 ceph-mon[36626]: [16/Jul/2024:09:57:10] ENGINE Serving on http://172.21.15.156:8765 2024-07-16T09:57:11.845 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:11 smithi176 ceph-mon[36626]: [16/Jul/2024:09:57:10] ENGINE Bus STARTED 2024-07-16T09:57:11.924 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:11 smithi156 ceph-mon[29302]: [16/Jul/2024:09:57:10] ENGINE Bus STARTING 2024-07-16T09:57:11.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:11 smithi156 ceph-mon[29302]: [16/Jul/2024:09:57:10] ENGINE Serving on https://172.21.15.156:7150 2024-07-16T09:57:11.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:11 smithi156 ceph-mon[29302]: [16/Jul/2024:09:57:10] ENGINE Error in HTTPServer.serve 2024-07-16T09:57:11.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: Traceback (most recent call last): 2024-07-16T09:57:11.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-07-16T09:57:11.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: self._connections.run(self.expiration_interval) 2024-07-16T09:57:11.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-07-16T09:57:11.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: self._run(expiration_interval) 2024-07-16T09:57:11.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-07-16T09:57:11.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-07-16T09:57:11.926 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-07-16T09:57:11.926 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-07-16T09:57:11.926 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-07-16T09:57:11.926 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: s = self.context.wrap_socket( 2024-07-16T09:57:11.926 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-07-16T09:57:11.926 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: return self.sslsocket_class._create( 2024-07-16T09:57:11.926 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-07-16T09:57:11.926 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: self.do_handshake() 2024-07-16T09:57:11.926 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-07-16T09:57:11.926 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: self._sslobj.do_handshake() 2024-07-16T09:57:11.926 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-07-16T09:57:11.926 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:11 smithi156 ceph-mon[29302]: pgmap v4: 65 pgs: 65 active+clean; 253 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-07-16T09:57:11.926 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:11 smithi156 ceph-mon[29302]: [16/Jul/2024:09:57:10] ENGINE Serving on http://172.21.15.156:8765 2024-07-16T09:57:11.927 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:11 smithi156 ceph-mon[29302]: [16/Jul/2024:09:57:10] ENGINE Bus STARTED 2024-07-16T09:57:12.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:12 smithi176 ceph-mon[36626]: mgrmap e29: smithi156.zkvylv(active, since 3s) 2024-07-16T09:57:12.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:12 smithi176 ceph-mon[36626]: from='mgr.14722 172.21.15.156:0/647865130' entity='mgr.smithi156.zkvylv' 2024-07-16T09:57:12.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:12 smithi176 ceph-mon[36626]: from='mgr.14722 172.21.15.156:0/647865130' entity='mgr.smithi156.zkvylv' 2024-07-16T09:57:12.638 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:12.634+0000 7fdeaad62640 1 -- 172.21.15.156:0/2773875462 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdea40ff490 msgr2=0x7fdea4101870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:57:12.638 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:12.634+0000 7fdeaad62640 1 --2- 172.21.15.156:0/2773875462 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdea40ff490 0x7fdea4101870 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fde900048d0 tx=0x7fde90009da0 comp rx=0 tx=0).stop 2024-07-16T09:57:12.638 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:12.634+0000 7fdeaad62640 1 -- 172.21.15.156:0/2773875462 shutdown_connections 2024-07-16T09:57:12.638 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:12.634+0000 7fdeaad62640 1 --2- 172.21.15.156:0/2773875462 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdea40ff490 0x7fdea4101870 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:57:12.638 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:12.634+0000 7fdeaad62640 1 --2- 172.21.15.156:0/2773875462 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fdea4065a50 0x7fdea40fef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:57:12.638 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:12.634+0000 7fdeaad62640 1 -- 172.21.15.156:0/2773875462 >> 172.21.15.156:0/2773875462 conn(0x7fdea40f6a20 msgr2=0x7fdea40f8e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:57:12.638 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:12.634+0000 7fdeaad62640 1 -- 172.21.15.156:0/2773875462 shutdown_connections 2024-07-16T09:57:12.638 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:12.634+0000 7fdeaad62640 1 -- 172.21.15.156:0/2773875462 wait complete. 2024-07-16T09:57:12.638 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:12.635+0000 7fdeaad62640 1 Processor -- start 2024-07-16T09:57:12.638 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:12.635+0000 7fdeaad62640 1 -- start start 2024-07-16T09:57:12.639 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:12.635+0000 7fdeaad62640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdea4065a50 0x7fdea40fc910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:57:12.639 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:12.635+0000 7fdeaad62640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fdea40ff490 0x7fdea40fae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:57:12.639 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:12.635+0000 7fdeaad62640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdea40fb390 con 0x7fdea4065a50 2024-07-16T09:57:12.639 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:12.635+0000 7fdeaad62640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdea40fb4d0 con 0x7fdea40ff490 2024-07-16T09:57:12.639 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:12.635+0000 7fdea8afe640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdea4065a50 0x7fdea40fc910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:57:12.639 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:12.635+0000 7fdea8afe640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdea4065a50 0x7fdea40fc910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:59840/0 (socket says 172.21.15.156:59840) 2024-07-16T09:57:12.639 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:12.635+0000 7fdea8afe640 1 -- 172.21.15.156:0/3757259104 learned_addr learned my addr 172.21.15.156:0/3757259104 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T09:57:12.639 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:12.635+0000 7fdea3fff640 1 --2- 172.21.15.156:0/3757259104 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fdea40ff490 0x7fdea40fae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:57:12.639 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:12.636+0000 7fdea3fff640 1 -- 172.21.15.156:0/3757259104 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdea4065a50 msgr2=0x7fdea40fc910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:57:12.639 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:12.636+0000 7fdea3fff640 1 --2- 172.21.15.156:0/3757259104 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdea4065a50 0x7fdea40fc910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:57:12.639 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:12.636+0000 7fdea3fff640 1 -- 172.21.15.156:0/3757259104 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde90005040 con 0x7fdea40ff490 2024-07-16T09:57:12.639 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:12.636+0000 7fdea8afe640 1 --2- 172.21.15.156:0/3757259104 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdea4065a50 0x7fdea40fc910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-16T09:57:12.641 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:12.636+0000 7fdea3fff640 1 --2- 172.21.15.156:0/3757259104 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fdea40ff490 0x7fdea40fae50 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fde900048a0 tx=0x7fde90009da0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:57:12.641 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:12.636+0000 7fdea17fa640 1 -- 172.21.15.156:0/3757259104 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fde90038070 con 0x7fdea40ff490 2024-07-16T09:57:12.641 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:12.636+0000 7fdea17fa640 1 -- 172.21.15.156:0/3757259104 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fde900092b0 con 0x7fdea40ff490 2024-07-16T09:57:12.641 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:12.636+0000 7fdeaad62640 1 -- 172.21.15.156:0/3757259104 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdea40fb750 con 0x7fdea40ff490 2024-07-16T09:57:12.641 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:12.636+0000 7fdea17fa640 1 -- 172.21.15.156:0/3757259104 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fde90033660 con 0x7fdea40ff490 2024-07-16T09:57:12.641 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:12.636+0000 7fdeaad62640 1 -- 172.21.15.156:0/3757259104 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdea40fbba0 con 0x7fdea40ff490 2024-07-16T09:57:12.641 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:12.637+0000 7fdea17fa640 1 -- 172.21.15.156:0/3757259104 <== mon.1 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 29) v1 ==== 46297+0+0 (secure 0 0 0) 0x7fde90030020 con 0x7fdea40ff490 2024-07-16T09:57:12.641 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:12.637+0000 7fdeaad62640 1 -- 172.21.15.156:0/3757259104 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdea40fa6a0 con 0x7fdea40ff490 2024-07-16T09:57:12.641 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:12.638+0000 7fdea17fa640 1 --2- 172.21.15.156:0/3757259104 >> [v2:172.21.15.156:6800/55311371,v1:172.21.15.156:6801/55311371] conn(0x7fde94038e90 0x7fde9403b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:57:12.641 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:12.638+0000 7fdea17fa640 1 -- 172.21.15.156:0/3757259104 <== mon.1 v2:172.21.15.176:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7fde9006c990 con 0x7fdea40ff490 2024-07-16T09:57:12.641 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:12.638+0000 7fdea8afe640 1 --2- 172.21.15.156:0/3757259104 >> [v2:172.21.15.156:6800/55311371,v1:172.21.15.156:6801/55311371] conn(0x7fde94038e90 0x7fde9403b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:57:12.641 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:12.638+0000 7fdea8afe640 1 --2- 172.21.15.156:0/3757259104 >> [v2:172.21.15.156:6800/55311371,v1:172.21.15.156:6801/55311371] conn(0x7fde94038e90 0x7fde9403b340 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fdea40fc6c0 tx=0x7fde9800b040 comp rx=0 tx=0).ready entity=mgr.14722 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:57:12.645 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:12.643+0000 7fdea17fa640 1 -- 172.21.15.156:0/3757259104 <== mon.1 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185936 (secure 0 0 0) 0x7fde9003a900 con 0x7fdea40ff490 2024-07-16T09:57:12.805 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:12 smithi156 ceph-mon[29302]: mgrmap e29: smithi156.zkvylv(active, since 3s) 2024-07-16T09:57:12.806 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:12 smithi156 ceph-mon[29302]: from='mgr.14722 172.21.15.156:0/647865130' entity='mgr.smithi156.zkvylv' 2024-07-16T09:57:12.806 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:12 smithi156 ceph-mon[29302]: from='mgr.14722 172.21.15.156:0/647865130' entity='mgr.smithi156.zkvylv' 2024-07-16T09:57:12.842 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:12.840+0000 7fdeaad62640 1 -- 172.21.15.156:0/3757259104 --> [v2:172.21.15.156:6800/55311371,v1:172.21.15.156:6801/55311371] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fdea40f8510 con 0x7fde94038e90 2024-07-16T09:57:12.843 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:12.841+0000 7fdea17fa640 1 -- 172.21.15.156:0/3757259104 <== mgr.14722 v2:172.21.15.156:6800/55311371 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+337 (secure 0 0 0) 0x7fdea40f8510 con 0x7fde94038e90 2024-07-16T09:57:12.846 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:12.843+0000 7fdeaad62640 1 -- 172.21.15.156:0/3757259104 >> [v2:172.21.15.156:6800/55311371,v1:172.21.15.156:6801/55311371] conn(0x7fde94038e90 msgr2=0x7fde9403b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:57:12.846 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:12.843+0000 7fdeaad62640 1 --2- 172.21.15.156:0/3757259104 >> [v2:172.21.15.156:6800/55311371,v1:172.21.15.156:6801/55311371] conn(0x7fde94038e90 0x7fde9403b340 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fdea40fc6c0 tx=0x7fde9800b040 comp rx=0 tx=0).stop 2024-07-16T09:57:12.846 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:12.843+0000 7fdeaad62640 1 -- 172.21.15.156:0/3757259104 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fdea40ff490 msgr2=0x7fdea40fae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:57:12.846 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:12.843+0000 7fdeaad62640 1 --2- 172.21.15.156:0/3757259104 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fdea40ff490 0x7fdea40fae50 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fde900048a0 tx=0x7fde90009da0 comp rx=0 tx=0).stop 2024-07-16T09:57:12.846 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:12.843+0000 7fdeaad62640 1 -- 172.21.15.156:0/3757259104 shutdown_connections 2024-07-16T09:57:12.846 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:12.843+0000 7fdeaad62640 1 --2- 172.21.15.156:0/3757259104 >> [v2:172.21.15.156:6800/55311371,v1:172.21.15.156:6801/55311371] conn(0x7fde94038e90 0x7fde9403b340 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:57:12.846 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:12.843+0000 7fdeaad62640 1 --2- 172.21.15.156:0/3757259104 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fdea40ff490 0x7fdea40fae50 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:57:12.846 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:12.843+0000 7fdeaad62640 1 --2- 172.21.15.156:0/3757259104 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdea4065a50 0x7fdea40fc910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:57:12.846 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:12.843+0000 7fdeaad62640 1 -- 172.21.15.156:0/3757259104 >> 172.21.15.156:0/3757259104 conn(0x7fdea40f6a20 msgr2=0x7fdea40f6df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:57:12.847 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:12.843+0000 7fdeaad62640 1 -- 172.21.15.156:0/3757259104 shutdown_connections 2024-07-16T09:57:12.847 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:12.843+0000 7fdeaad62640 1 -- 172.21.15.156:0/3757259104 wait complete. 2024-07-16T09:57:12.854 INFO:teuthology.orchestra.run.smithi156.stdout:true 2024-07-16T09:57:12.931 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:12.928+0000 7f32ca1aa640 1 -- 172.21.15.156:0/2259390490 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f32c40ff490 msgr2=0x7f32c4101870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:57:12.931 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:12.928+0000 7f32ca1aa640 1 --2- 172.21.15.156:0/2259390490 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f32c40ff490 0x7f32c4101870 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f32b80048d0 tx=0x7f32b8009da0 comp rx=0 tx=0).stop 2024-07-16T09:57:12.931 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:12.929+0000 7f32ca1aa640 1 -- 172.21.15.156:0/2259390490 shutdown_connections 2024-07-16T09:57:12.932 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:12.929+0000 7f32ca1aa640 1 --2- 172.21.15.156:0/2259390490 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f32c40ff490 0x7f32c4101870 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:57:12.932 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:12.929+0000 7f32ca1aa640 1 --2- 172.21.15.156:0/2259390490 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f32c4065a50 0x7f32c40fef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:57:12.932 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:12.929+0000 7f32ca1aa640 1 -- 172.21.15.156:0/2259390490 >> 172.21.15.156:0/2259390490 conn(0x7f32c40f6a20 msgr2=0x7f32c40f8e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:57:12.932 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:12.929+0000 7f32ca1aa640 1 -- 172.21.15.156:0/2259390490 shutdown_connections 2024-07-16T09:57:12.932 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:12.929+0000 7f32ca1aa640 1 -- 172.21.15.156:0/2259390490 wait complete. 2024-07-16T09:57:12.932 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:12.929+0000 7f32ca1aa640 1 Processor -- start 2024-07-16T09:57:12.933 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:12.930+0000 7f32ca1aa640 1 -- start start 2024-07-16T09:57:12.933 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:12.930+0000 7f32ca1aa640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f32c4065a50 0x7f32c410cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:57:12.934 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:12.930+0000 7f32ca1aa640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f32c40ff490 0x7f32c410d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:57:12.934 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:12.930+0000 7f32ca1aa640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32c410db20 con 0x7f32c40ff490 2024-07-16T09:57:12.934 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:12.930+0000 7f32ca1aa640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32c410e2f0 con 0x7f32c4065a50 2024-07-16T09:57:12.934 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:12.930+0000 7f32c2ffd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f32c40ff490 0x7f32c410d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:57:12.934 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:12.930+0000 7f32c37fe640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f32c4065a50 0x7f32c410cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:57:12.934 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:12.930+0000 7f32c2ffd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f32c40ff490 0x7f32c410d4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:58842/0 (socket says 172.21.15.156:58842) 2024-07-16T09:57:12.934 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:12.930+0000 7f32c37fe640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f32c4065a50 0x7f32c410cf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.156:46670/0 (socket says 172.21.15.156:46670) 2024-07-16T09:57:12.934 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:12.930+0000 7f32c2ffd640 1 -- 172.21.15.156:0/1572361199 learned_addr learned my addr 172.21.15.156:0/1572361199 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T09:57:12.934 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:12.930+0000 7f32c2ffd640 1 -- 172.21.15.156:0/1572361199 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f32c4065a50 msgr2=0x7f32c410cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:57:12.934 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:12.930+0000 7f32c2ffd640 1 --2- 172.21.15.156:0/1572361199 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f32c4065a50 0x7f32c410cf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:57:12.934 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:12.930+0000 7f32c2ffd640 1 -- 172.21.15.156:0/1572361199 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32b8005040 con 0x7f32c40ff490 2024-07-16T09:57:12.935 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:12.931+0000 7f32c2ffd640 1 --2- 172.21.15.156:0/1572361199 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f32c40ff490 0x7f32c410d4d0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f32b8009f80 tx=0x7f32b8009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:57:12.935 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:12.931+0000 7f32abfff640 1 -- 172.21.15.156:0/1572361199 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f32b8038070 con 0x7f32c40ff490 2024-07-16T09:57:12.935 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:12.931+0000 7f32abfff640 1 -- 172.21.15.156:0/1572361199 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f32b8032970 con 0x7f32c40ff490 2024-07-16T09:57:12.935 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:12.931+0000 7f32ca1aa640 1 -- 172.21.15.156:0/1572361199 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32c410e520 con 0x7f32c40ff490 2024-07-16T09:57:12.935 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:12.931+0000 7f32abfff640 1 -- 172.21.15.156:0/1572361199 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f32b8033860 con 0x7f32c40ff490 2024-07-16T09:57:12.935 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:12.931+0000 7f32ca1aa640 1 -- 172.21.15.156:0/1572361199 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32c410e9e0 con 0x7f32c40ff490 2024-07-16T09:57:12.936 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:12.932+0000 7f32abfff640 1 -- 172.21.15.156:0/1572361199 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 29) v1 ==== 46297+0+0 (secure 0 0 0) 0x7f32b8030020 con 0x7f32c40ff490 2024-07-16T09:57:12.936 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:12.932+0000 7f32abfff640 1 --2- 172.21.15.156:0/1572361199 >> [v2:172.21.15.156:6800/55311371,v1:172.21.15.156:6801/55311371] conn(0x7f32ac038e90 0x7f32ac03b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:57:12.936 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:12.932+0000 7f32abfff640 1 -- 172.21.15.156:0/1572361199 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f32b806cb50 con 0x7f32c40ff490 2024-07-16T09:57:12.936 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:12.932+0000 7f32c37fe640 1 --2- 172.21.15.156:0/1572361199 >> [v2:172.21.15.156:6800/55311371,v1:172.21.15.156:6801/55311371] conn(0x7f32ac038e90 0x7f32ac03b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:57:12.936 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:12.932+0000 7f32ca1aa640 1 -- 172.21.15.156:0/1572361199 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32c40fa6a0 con 0x7f32c40ff490 2024-07-16T09:57:12.936 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:12.933+0000 7f32c37fe640 1 --2- 172.21.15.156:0/1572361199 >> [v2:172.21.15.156:6800/55311371,v1:172.21.15.156:6801/55311371] conn(0x7f32ac038e90 0x7f32ac03b340 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f32c40657b0 tx=0x7f32b4009040 comp rx=0 tx=0).ready entity=mgr.14722 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:57:12.940 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:12.938+0000 7f32abfff640 1 -- 172.21.15.156:0/1572361199 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185936 (secure 0 0 0) 0x7f32b8046440 con 0x7f32c40ff490 2024-07-16T09:57:13.158 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:13.155+0000 7f32ca1aa640 1 -- 172.21.15.156:0/1572361199 --> [v2:172.21.15.156:6800/55311371,v1:172.21.15.156:6801/55311371] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f32c410ed30 con 0x7f32ac038e90 2024-07-16T09:57:13.159 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:13.157+0000 7f32abfff640 1 -- 172.21.15.156:0/1572361199 <== mgr.14722 v2:172.21.15.156:6800/55311371 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+337 (secure 0 0 0) 0x7f32c410ed30 con 0x7f32ac038e90 2024-07-16T09:57:13.161 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:13.159+0000 7f32ca1aa640 1 -- 172.21.15.156:0/1572361199 >> [v2:172.21.15.156:6800/55311371,v1:172.21.15.156:6801/55311371] conn(0x7f32ac038e90 msgr2=0x7f32ac03b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:57:13.161 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:13.159+0000 7f32ca1aa640 1 --2- 172.21.15.156:0/1572361199 >> [v2:172.21.15.156:6800/55311371,v1:172.21.15.156:6801/55311371] conn(0x7f32ac038e90 0x7f32ac03b340 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f32c40657b0 tx=0x7f32b4009040 comp rx=0 tx=0).stop 2024-07-16T09:57:13.161 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:13.159+0000 7f32ca1aa640 1 -- 172.21.15.156:0/1572361199 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f32c40ff490 msgr2=0x7f32c410d4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:57:13.161 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:13.159+0000 7f32ca1aa640 1 --2- 172.21.15.156:0/1572361199 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f32c40ff490 0x7f32c410d4d0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f32b8009f80 tx=0x7f32b8009da0 comp rx=0 tx=0).stop 2024-07-16T09:57:13.162 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:13.159+0000 7f32ca1aa640 1 -- 172.21.15.156:0/1572361199 shutdown_connections 2024-07-16T09:57:13.162 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:13.159+0000 7f32ca1aa640 1 --2- 172.21.15.156:0/1572361199 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f32c40ff490 0x7f32c410d4d0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:57:13.162 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:13.159+0000 7f32ca1aa640 1 --2- 172.21.15.156:0/1572361199 >> [v2:172.21.15.156:6800/55311371,v1:172.21.15.156:6801/55311371] conn(0x7f32ac038e90 0x7f32ac03b340 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:57:13.162 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:13.159+0000 7f32ca1aa640 1 --2- 172.21.15.156:0/1572361199 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f32c4065a50 0x7f32c410cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:57:13.162 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:13.159+0000 7f32ca1aa640 1 -- 172.21.15.156:0/1572361199 >> 172.21.15.156:0/1572361199 conn(0x7f32c40f6a20 msgr2=0x7f32c40f6df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:57:13.162 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:13.159+0000 7f32ca1aa640 1 -- 172.21.15.156:0/1572361199 shutdown_connections 2024-07-16T09:57:13.162 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:13.159+0000 7f32ca1aa640 1 -- 172.21.15.156:0/1572361199 wait complete. 2024-07-16T09:57:13.244 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:13.240+0000 7fa86e4cf640 1 -- 172.21.15.156:0/2999058202 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa8680fc360 msgr2=0x7fa8680fc770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:57:13.244 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:13.240+0000 7fa86e4cf640 1 --2- 172.21.15.156:0/2999058202 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa8680fc360 0x7fa8680fc770 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fa8580048d0 tx=0x7fa858009da0 comp rx=0 tx=0).stop 2024-07-16T09:57:13.244 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:13.240+0000 7fa86e4cf640 1 -- 172.21.15.156:0/2999058202 shutdown_connections 2024-07-16T09:57:13.244 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:13.240+0000 7fa86e4cf640 1 --2- 172.21.15.156:0/2999058202 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa8680fd630 0x7fa8680fdaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:57:13.244 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:13.240+0000 7fa86e4cf640 1 --2- 172.21.15.156:0/2999058202 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa8680fc360 0x7fa8680fc770 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:57:13.244 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:13.240+0000 7fa86e4cf640 1 -- 172.21.15.156:0/2999058202 >> 172.21.15.156:0/2999058202 conn(0x7fa8680f7b40 msgr2=0x7fa8680f9f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:57:13.245 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:13.240+0000 7fa86e4cf640 1 -- 172.21.15.156:0/2999058202 shutdown_connections 2024-07-16T09:57:13.245 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:13.240+0000 7fa86e4cf640 1 -- 172.21.15.156:0/2999058202 wait complete. 2024-07-16T09:57:13.245 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:13.241+0000 7fa86e4cf640 1 Processor -- start 2024-07-16T09:57:13.245 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:13.241+0000 7fa86e4cf640 1 -- start start 2024-07-16T09:57:13.245 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:13.241+0000 7fa86e4cf640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa8680fc360 0x7fa868071330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:57:13.245 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:13.241+0000 7fa86e4cf640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa8680fd630 0x7fa86806f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:57:13.245 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:13.241+0000 7fa86e4cf640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa86806fdb0 con 0x7fa8680fd630 2024-07-16T09:57:13.245 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:13.241+0000 7fa86e4cf640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa86806fef0 con 0x7fa8680fc360 2024-07-16T09:57:13.245 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:13.241+0000 7fa8677fe640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa8680fd630 0x7fa86806f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:57:13.245 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:13.241+0000 7fa8677fe640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa8680fd630 0x7fa86806f870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:58864/0 (socket says 172.21.15.156:58864) 2024-07-16T09:57:13.245 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:13.241+0000 7fa8677fe640 1 -- 172.21.15.156:0/961672873 learned_addr learned my addr 172.21.15.156:0/961672873 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T09:57:13.246 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:13.241+0000 7fa867fff640 1 --2- 172.21.15.156:0/961672873 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa8680fc360 0x7fa868071330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:57:13.246 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:13.241+0000 7fa8677fe640 1 -- 172.21.15.156:0/961672873 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa8680fc360 msgr2=0x7fa868071330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:57:13.246 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:13.241+0000 7fa8677fe640 1 --2- 172.21.15.156:0/961672873 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa8680fc360 0x7fa868071330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:57:13.246 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:13.241+0000 7fa8677fe640 1 -- 172.21.15.156:0/961672873 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa858005040 con 0x7fa8680fd630 2024-07-16T09:57:13.246 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:13.242+0000 7fa8677fe640 1 --2- 172.21.15.156:0/961672873 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa8680fd630 0x7fa86806f870 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fa85c002a50 tx=0x7fa85c004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:57:13.246 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:13.242+0000 7fa864ff9640 1 -- 172.21.15.156:0/961672873 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa85c007c30 con 0x7fa8680fd630 2024-07-16T09:57:13.246 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:13.242+0000 7fa864ff9640 1 -- 172.21.15.156:0/961672873 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa85c00f040 con 0x7fa8680fd630 2024-07-16T09:57:13.246 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:13.242+0000 7fa864ff9640 1 -- 172.21.15.156:0/961672873 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa85c00dad0 con 0x7fa8680fd630 2024-07-16T09:57:13.246 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:13.242+0000 7fa86e4cf640 1 -- 172.21.15.156:0/961672873 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8680701d0 con 0x7fa8680fd630 2024-07-16T09:57:13.246 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:13.242+0000 7fa86e4cf640 1 -- 172.21.15.156:0/961672873 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa868115040 con 0x7fa8680fd630 2024-07-16T09:57:13.247 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:13.243+0000 7fa864ff9640 1 -- 172.21.15.156:0/961672873 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 29) v1 ==== 46297+0+0 (secure 0 0 0) 0x7fa85c00dd40 con 0x7fa8680fd630 2024-07-16T09:57:13.247 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:13.243+0000 7fa86e4cf640 1 -- 172.21.15.156:0/961672873 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8680fb7c0 con 0x7fa8680fd630 2024-07-16T09:57:13.248 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:13.243+0000 7fa864ff9640 1 --2- 172.21.15.156:0/961672873 >> [v2:172.21.15.156:6800/55311371,v1:172.21.15.156:6801/55311371] conn(0x7fa850038e90 0x7fa85003b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:57:13.248 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:13.243+0000 7fa864ff9640 1 -- 172.21.15.156:0/961672873 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7fa85c04c4e0 con 0x7fa8680fd630 2024-07-16T09:57:13.248 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:13.243+0000 7fa867fff640 1 --2- 172.21.15.156:0/961672873 >> [v2:172.21.15.156:6800/55311371,v1:172.21.15.156:6801/55311371] conn(0x7fa850038e90 0x7fa85003b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:57:13.248 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:13.244+0000 7fa867fff640 1 --2- 172.21.15.156:0/961672873 >> [v2:172.21.15.156:6800/55311371,v1:172.21.15.156:6801/55311371] conn(0x7fa850038e90 0x7fa85003b340 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fa8580043d0 tx=0x7fa858035040 comp rx=0 tx=0).ready entity=mgr.14722 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:57:13.250 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:13.248+0000 7fa864ff9640 1 -- 172.21.15.156:0/961672873 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185936 (secure 0 0 0) 0x7fa85c019490 con 0x7fa8680fd630 2024-07-16T09:57:13.458 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:13.456+0000 7fa86e4cf640 1 -- 172.21.15.156:0/961672873 --> [v2:172.21.15.156:6800/55311371,v1:172.21.15.156:6801/55311371] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fa8681019a0 con 0x7fa850038e90 2024-07-16T09:57:13.467 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:13.465+0000 7fa864ff9640 1 -- 172.21.15.156:0/961672873 <== mgr.14722 v2:172.21.15.156:6800/55311371 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7fa8681019a0 con 0x7fa850038e90 2024-07-16T09:57:13.468 INFO:teuthology.orchestra.run.smithi156.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-07-16T09:57:13.468 INFO:teuthology.orchestra.run.smithi156.stdout:alertmanager.smithi156 smithi156 *:9093,9094 running (60s) 9s ago 9m 16.7M - 0.25.0 c8568f914cd2 d524a2c3ff15 2024-07-16T09:57:13.468 INFO:teuthology.orchestra.run.smithi156.stdout:crash.smithi156 smithi156 running (9m) 9s ago 9m 6899k - 17.2.7-1104-ge6231b04 d5de5f953b95 bf6d49ea06b4 2024-07-16T09:57:13.468 INFO:teuthology.orchestra.run.smithi156.stdout:crash.smithi176 smithi176 running (8m) 1s ago 8m 6903k - 17.2.7-1104-ge6231b04 d5de5f953b95 907c9fa1e890 2024-07-16T09:57:13.468 INFO:teuthology.orchestra.run.smithi156.stdout:grafana.smithi156 smithi156 *:3000 running (39s) 9s ago 9m 75.1M - 9.4.7 954c08fa6188 deb9e1bfeb66 2024-07-16T09:57:13.468 INFO:teuthology.orchestra.run.smithi156.stdout:mds.cephfs.smithi156.evwwaz smithi156 running (4m) 9s ago 4m 70.2M - 17.2.7-1104-ge6231b04 d5de5f953b95 7c0ff9ae43f8 2024-07-16T09:57:13.468 INFO:teuthology.orchestra.run.smithi156.stdout:mds.cephfs.smithi156.rdibys smithi156 running (4m) 9s ago 4m 132M - 17.2.7-1104-ge6231b04 d5de5f953b95 b985cd1e6924 2024-07-16T09:57:13.468 INFO:teuthology.orchestra.run.smithi156.stdout:mds.cephfs.smithi176.dvvbud smithi176 running (4m) 1s ago 4m 109M - 17.2.7-1104-ge6231b04 d5de5f953b95 1b00147950f9 2024-07-16T09:57:13.468 INFO:teuthology.orchestra.run.smithi156.stdout:mds.cephfs.smithi176.wahbsi smithi176 running (4m) 1s ago 4m 62.0M - 17.2.7-1104-ge6231b04 d5de5f953b95 866664af2311 2024-07-16T09:57:13.468 INFO:teuthology.orchestra.run.smithi156.stdout:mgr.smithi156.zkvylv smithi156 *:8443,9283,8765 running (22s) 9s ago 10m 52.0M - 19.0.0-3495-ga6ca461b 16f7feb04342 ee53bd604b7c 2024-07-16T09:57:13.468 INFO:teuthology.orchestra.run.smithi156.stdout:mgr.smithi176.dqbrvx smithi176 *:8443,9283 running (2m) 1s ago 8m 131M - 19.0.0-3495-ga6ca461b 16f7feb04342 a7c332f2d676 2024-07-16T09:57:13.468 INFO:teuthology.orchestra.run.smithi156.stdout:mon.smithi156 smithi156 running (10m) 9s ago 10m 52.5M 2048M 17.2.7-1104-ge6231b04 d5de5f953b95 6471c8acff40 2024-07-16T09:57:13.468 INFO:teuthology.orchestra.run.smithi156.stdout:mon.smithi176 smithi176 running (8m) 1s ago 8m 46.3M 2048M 17.2.7-1104-ge6231b04 d5de5f953b95 3a293ac6ef01 2024-07-16T09:57:13.468 INFO:teuthology.orchestra.run.smithi156.stdout:node-exporter.smithi156 smithi156 *:9100 running (66s) 9s ago 9m 7956k - 1.5.0 0da6a335fe13 e5d86f23a14c 2024-07-16T09:57:13.469 INFO:teuthology.orchestra.run.smithi156.stdout:node-exporter.smithi176 smithi176 *:9100 running (30s) 1s ago 8m 8644k - 1.5.0 0da6a335fe13 ea842728144a 2024-07-16T09:57:13.469 INFO:teuthology.orchestra.run.smithi156.stdout:osd.0 smithi156 running (7m) 9s ago 7m 221M 1557M 17.2.7-1104-ge6231b04 d5de5f953b95 0471758ec349 2024-07-16T09:57:13.469 INFO:teuthology.orchestra.run.smithi156.stdout:osd.1 smithi156 running (7m) 9s ago 7m 244M 1557M 17.2.7-1104-ge6231b04 d5de5f953b95 149a0bda2f61 2024-07-16T09:57:13.469 INFO:teuthology.orchestra.run.smithi156.stdout:osd.2 smithi156 running (6m) 9s ago 6m 198M 1557M 17.2.7-1104-ge6231b04 d5de5f953b95 f28acec5cb79 2024-07-16T09:57:13.469 INFO:teuthology.orchestra.run.smithi156.stdout:osd.3 smithi176 running (6m) 1s ago 6m 259M 2581M 17.2.7-1104-ge6231b04 d5de5f953b95 733861ba84dc 2024-07-16T09:57:13.469 INFO:teuthology.orchestra.run.smithi156.stdout:osd.4 smithi176 running (6m) 1s ago 6m 229M 2581M 17.2.7-1104-ge6231b04 d5de5f953b95 ef0c67cec086 2024-07-16T09:57:13.469 INFO:teuthology.orchestra.run.smithi156.stdout:osd.5 smithi176 running (5m) 1s ago 5m 222M 2581M 17.2.7-1104-ge6231b04 d5de5f953b95 996e15448a52 2024-07-16T09:57:13.469 INFO:teuthology.orchestra.run.smithi156.stdout:prometheus.smithi156 smithi156 *:9095 running (32s) 9s ago 8m 44.7M - 2.43.0 a07b618ecd1d d9e88bf38225 2024-07-16T09:57:13.470 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:13.468+0000 7fa86e4cf640 1 -- 172.21.15.156:0/961672873 >> [v2:172.21.15.156:6800/55311371,v1:172.21.15.156:6801/55311371] conn(0x7fa850038e90 msgr2=0x7fa85003b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:57:13.470 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:13.468+0000 7fa86e4cf640 1 --2- 172.21.15.156:0/961672873 >> [v2:172.21.15.156:6800/55311371,v1:172.21.15.156:6801/55311371] conn(0x7fa850038e90 0x7fa85003b340 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fa8580043d0 tx=0x7fa858035040 comp rx=0 tx=0).stop 2024-07-16T09:57:13.470 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:13.468+0000 7fa86e4cf640 1 -- 172.21.15.156:0/961672873 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa8680fd630 msgr2=0x7fa86806f870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:57:13.470 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:13.468+0000 7fa86e4cf640 1 --2- 172.21.15.156:0/961672873 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa8680fd630 0x7fa86806f870 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fa85c002a50 tx=0x7fa85c004600 comp rx=0 tx=0).stop 2024-07-16T09:57:13.470 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:13.468+0000 7fa86e4cf640 1 -- 172.21.15.156:0/961672873 shutdown_connections 2024-07-16T09:57:13.470 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:13.468+0000 7fa86e4cf640 1 --2- 172.21.15.156:0/961672873 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa8680fd630 0x7fa86806f870 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:57:13.470 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:13.468+0000 7fa86e4cf640 1 --2- 172.21.15.156:0/961672873 >> [v2:172.21.15.156:6800/55311371,v1:172.21.15.156:6801/55311371] conn(0x7fa850038e90 0x7fa85003b340 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:57:13.470 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:13.468+0000 7fa86e4cf640 1 --2- 172.21.15.156:0/961672873 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa8680fc360 0x7fa868071330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:57:13.470 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:13.468+0000 7fa86e4cf640 1 -- 172.21.15.156:0/961672873 >> 172.21.15.156:0/961672873 conn(0x7fa8680f7b40 msgr2=0x7fa8680f9220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:57:13.471 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:13.469+0000 7fa86e4cf640 1 -- 172.21.15.156:0/961672873 shutdown_connections 2024-07-16T09:57:13.471 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:13.469+0000 7fa86e4cf640 1 -- 172.21.15.156:0/961672873 wait complete. 2024-07-16T09:57:13.554 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:13.550+0000 7f090eb75640 1 -- 172.21.15.156:0/3199281786 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f09080fc360 msgr2=0x7f09080fc770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:57:13.554 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:13.550+0000 7f090eb75640 1 --2- 172.21.15.156:0/3199281786 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f09080fc360 0x7f09080fc770 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f08fc0048d0 tx=0x7f08fc009da0 comp rx=0 tx=0).stop 2024-07-16T09:57:13.554 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:13.551+0000 7f090eb75640 1 -- 172.21.15.156:0/3199281786 shutdown_connections 2024-07-16T09:57:13.554 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:13.551+0000 7f090eb75640 1 --2- 172.21.15.156:0/3199281786 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f09080fd630 0x7f09080fdaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:57:13.554 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:13.551+0000 7f090eb75640 1 --2- 172.21.15.156:0/3199281786 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f09080fc360 0x7f09080fc770 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:57:13.554 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:13.551+0000 7f090eb75640 1 -- 172.21.15.156:0/3199281786 >> 172.21.15.156:0/3199281786 conn(0x7f09080f7b40 msgr2=0x7f09080f9f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:57:13.554 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:13.551+0000 7f090eb75640 1 -- 172.21.15.156:0/3199281786 shutdown_connections 2024-07-16T09:57:13.554 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:13.551+0000 7f090eb75640 1 -- 172.21.15.156:0/3199281786 wait complete. 2024-07-16T09:57:13.554 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:13.551+0000 7f090eb75640 1 Processor -- start 2024-07-16T09:57:13.555 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:13.551+0000 7f090eb75640 1 -- start start 2024-07-16T09:57:13.555 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:13.551+0000 7f090eb75640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f09080fc360 0x7f090810cff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:57:13.555 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:13.551+0000 7f090eb75640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f09080fd630 0x7f090810d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:57:13.555 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:13.551+0000 7f090eb75640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f090810db80 con 0x7f09080fc360 2024-07-16T09:57:13.555 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:13.551+0000 7f090eb75640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f090810e350 con 0x7f09080fd630 2024-07-16T09:57:13.555 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:13.552+0000 7f090c911640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f09080fc360 0x7f090810cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:57:13.555 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:13.552+0000 7f090c911640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f09080fc360 0x7f090810cff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:58872/0 (socket says 172.21.15.156:58872) 2024-07-16T09:57:13.555 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:13.552+0000 7f090c911640 1 -- 172.21.15.156:0/267913264 learned_addr learned my addr 172.21.15.156:0/267913264 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T09:57:13.555 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:13.552+0000 7f0907fff640 1 --2- 172.21.15.156:0/267913264 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f09080fd630 0x7f090810d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:57:13.555 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:13.552+0000 7f090c911640 1 -- 172.21.15.156:0/267913264 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f09080fd630 msgr2=0x7f090810d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:57:13.556 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:13.552+0000 7f090c911640 1 --2- 172.21.15.156:0/267913264 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f09080fd630 0x7f090810d530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:57:13.556 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:13.552+0000 7f090c911640 1 -- 172.21.15.156:0/267913264 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08fc005040 con 0x7f09080fc360 2024-07-16T09:57:13.556 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:13.552+0000 7f090c911640 1 --2- 172.21.15.156:0/267913264 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f09080fc360 0x7f090810cff0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f08fc009f80 tx=0x7f08fc009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:57:13.556 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:13.552+0000 7f09057fa640 1 -- 172.21.15.156:0/267913264 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f08fc038070 con 0x7f09080fc360 2024-07-16T09:57:13.556 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:13.552+0000 7f09057fa640 1 -- 172.21.15.156:0/267913264 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f08fc032970 con 0x7f09080fc360 2024-07-16T09:57:13.556 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:13.552+0000 7f090eb75640 1 -- 172.21.15.156:0/267913264 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f090810e580 con 0x7f09080fc360 2024-07-16T09:57:13.556 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:13.552+0000 7f09057fa640 1 -- 172.21.15.156:0/267913264 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f08fc033860 con 0x7f09080fc360 2024-07-16T09:57:13.556 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:13.552+0000 7f090eb75640 1 -- 172.21.15.156:0/267913264 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f090810eac0 con 0x7f09080fc360 2024-07-16T09:57:13.557 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:13.553+0000 7f09057fa640 1 -- 172.21.15.156:0/267913264 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 29) v1 ==== 46297+0+0 (secure 0 0 0) 0x7f08fc030020 con 0x7f09080fc360 2024-07-16T09:57:13.557 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:13.553+0000 7f090eb75640 1 -- 172.21.15.156:0/267913264 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09080fb7c0 con 0x7f09080fc360 2024-07-16T09:57:13.557 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:13.553+0000 7f09057fa640 1 --2- 172.21.15.156:0/267913264 >> [v2:172.21.15.156:6800/55311371,v1:172.21.15.156:6801/55311371] conn(0x7f08f8038e90 0x7f08f803b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:57:13.557 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:13.553+0000 7f09057fa640 1 -- 172.21.15.156:0/267913264 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f08fc06cad0 con 0x7f09080fc360 2024-07-16T09:57:13.557 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:13.554+0000 7f0907fff640 1 --2- 172.21.15.156:0/267913264 >> [v2:172.21.15.156:6800/55311371,v1:172.21.15.156:6801/55311371] conn(0x7f08f8038e90 0x7f08f803b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:57:13.557 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:13.554+0000 7f0907fff640 1 --2- 172.21.15.156:0/267913264 >> [v2:172.21.15.156:6800/55311371,v1:172.21.15.156:6801/55311371] conn(0x7f08f8038e90 0x7f08f803b340 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f090810cda0 tx=0x7f08f400b040 comp rx=0 tx=0).ready entity=mgr.14722 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:57:13.559 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:13.558+0000 7f09057fa640 1 -- 172.21.15.156:0/267913264 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185936 (secure 0 0 0) 0x7f08fc046440 con 0x7f09080fc360 2024-07-16T09:57:13.747 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:13 smithi156 ceph-mon[29302]: pgmap v5: 65 pgs: 65 active+clean; 253 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-07-16T09:57:13.747 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:13 smithi156 ceph-mon[29302]: from='mgr.14722 172.21.15.156:0/647865130' entity='mgr.smithi156.zkvylv' 2024-07-16T09:57:13.748 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:13 smithi156 ceph-mon[29302]: from='mgr.14722 172.21.15.156:0/647865130' entity='mgr.smithi156.zkvylv' 2024-07-16T09:57:13.748 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:13 smithi156 ceph-mon[29302]: Standby manager daemon smithi176.dqbrvx started 2024-07-16T09:57:13.748 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:13 smithi156 ceph-mon[29302]: from='mgr.? 172.21.15.176:0/4272015421' entity='mgr.smithi176.dqbrvx' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi176.dqbrvx/crt"}]: dispatch 2024-07-16T09:57:13.748 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:13 smithi156 ceph-mon[29302]: from='mgr.? 172.21.15.176:0/4272015421' entity='mgr.smithi176.dqbrvx' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-07-16T09:57:13.748 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:13 smithi156 ceph-mon[29302]: from='mgr.? 172.21.15.176:0/4272015421' entity='mgr.smithi176.dqbrvx' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi176.dqbrvx/key"}]: dispatch 2024-07-16T09:57:13.748 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:13 smithi156 ceph-mon[29302]: from='mgr.? 172.21.15.176:0/4272015421' entity='mgr.smithi176.dqbrvx' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-07-16T09:57:13.818 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:13.816+0000 7f090eb75640 1 -- 172.21.15.156:0/267913264 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f090810ed80 con 0x7f09080fc360 2024-07-16T09:57:13.818 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:13.817+0000 7f09057fa640 1 -- 172.21.15.156:0/267913264 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+741 (secure 0 0 0) 0x7f08fc041070 con 0x7f09080fc360 2024-07-16T09:57:13.819 INFO:teuthology.orchestra.run.smithi156.stdout:{ 2024-07-16T09:57:13.819 INFO:teuthology.orchestra.run.smithi156.stdout: "mon": { 2024-07-16T09:57:13.819 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph version 17.2.7-1104-ge6231b04 (e6231b04b52ebf3b9b31803282dd3074439e51ac) quincy (stable)": 2 2024-07-16T09:57:13.819 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-07-16T09:57:13.819 INFO:teuthology.orchestra.run.smithi156.stdout: "mgr": { 2024-07-16T09:57:13.819 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph version 19.0.0-3495-ga6ca461b (a6ca461be7cc1aff3658c2c6df4cff975a24449a) squid (rc)": 1 2024-07-16T09:57:13.819 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-07-16T09:57:13.819 INFO:teuthology.orchestra.run.smithi156.stdout: "osd": { 2024-07-16T09:57:13.819 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph version 17.2.7-1104-ge6231b04 (e6231b04b52ebf3b9b31803282dd3074439e51ac) quincy (stable)": 6 2024-07-16T09:57:13.819 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-07-16T09:57:13.819 INFO:teuthology.orchestra.run.smithi156.stdout: "mds": { 2024-07-16T09:57:13.820 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph version 17.2.7-1104-ge6231b04 (e6231b04b52ebf3b9b31803282dd3074439e51ac) quincy (stable)": 4 2024-07-16T09:57:13.820 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-07-16T09:57:13.820 INFO:teuthology.orchestra.run.smithi156.stdout: "overall": { 2024-07-16T09:57:13.820 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph version 17.2.7-1104-ge6231b04 (e6231b04b52ebf3b9b31803282dd3074439e51ac) quincy (stable)": 12, 2024-07-16T09:57:13.820 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph version 19.0.0-3495-ga6ca461b (a6ca461be7cc1aff3658c2c6df4cff975a24449a) squid (rc)": 1 2024-07-16T09:57:13.820 INFO:teuthology.orchestra.run.smithi156.stdout: } 2024-07-16T09:57:13.820 INFO:teuthology.orchestra.run.smithi156.stdout:} 2024-07-16T09:57:13.821 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:13.819+0000 7f090eb75640 1 -- 172.21.15.156:0/267913264 >> [v2:172.21.15.156:6800/55311371,v1:172.21.15.156:6801/55311371] conn(0x7f08f8038e90 msgr2=0x7f08f803b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:57:13.821 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:13.819+0000 7f090eb75640 1 --2- 172.21.15.156:0/267913264 >> [v2:172.21.15.156:6800/55311371,v1:172.21.15.156:6801/55311371] conn(0x7f08f8038e90 0x7f08f803b340 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f090810cda0 tx=0x7f08f400b040 comp rx=0 tx=0).stop 2024-07-16T09:57:13.821 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:13.819+0000 7f090eb75640 1 -- 172.21.15.156:0/267913264 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f09080fc360 msgr2=0x7f090810cff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:57:13.821 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:13.819+0000 7f090eb75640 1 --2- 172.21.15.156:0/267913264 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f09080fc360 0x7f090810cff0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f08fc009f80 tx=0x7f08fc009da0 comp rx=0 tx=0).stop 2024-07-16T09:57:13.821 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:13.819+0000 7f090eb75640 1 -- 172.21.15.156:0/267913264 shutdown_connections 2024-07-16T09:57:13.821 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:13.819+0000 7f090eb75640 1 --2- 172.21.15.156:0/267913264 >> [v2:172.21.15.156:6800/55311371,v1:172.21.15.156:6801/55311371] conn(0x7f08f8038e90 0x7f08f803b340 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:57:13.821 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:13.819+0000 7f090eb75640 1 --2- 172.21.15.156:0/267913264 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f09080fd630 0x7f090810d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:57:13.821 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:13.819+0000 7f090eb75640 1 --2- 172.21.15.156:0/267913264 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f09080fc360 0x7f090810cff0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:57:13.821 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:13.819+0000 7f090eb75640 1 -- 172.21.15.156:0/267913264 >> 172.21.15.156:0/267913264 conn(0x7f09080f7b40 msgr2=0x7f09080f9220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:57:13.821 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:13.819+0000 7f090eb75640 1 -- 172.21.15.156:0/267913264 shutdown_connections 2024-07-16T09:57:13.822 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:13.819+0000 7f090eb75640 1 -- 172.21.15.156:0/267913264 wait complete. 2024-07-16T09:57:13.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:13 smithi176 ceph-mon[36626]: pgmap v5: 65 pgs: 65 active+clean; 253 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-07-16T09:57:13.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:13 smithi176 ceph-mon[36626]: from='mgr.14722 172.21.15.156:0/647865130' entity='mgr.smithi156.zkvylv' 2024-07-16T09:57:13.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:13 smithi176 ceph-mon[36626]: from='mgr.14722 172.21.15.156:0/647865130' entity='mgr.smithi156.zkvylv' 2024-07-16T09:57:13.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:13 smithi176 ceph-mon[36626]: Standby manager daemon smithi176.dqbrvx started 2024-07-16T09:57:13.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:13 smithi176 ceph-mon[36626]: from='mgr.? 172.21.15.176:0/4272015421' entity='mgr.smithi176.dqbrvx' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi176.dqbrvx/crt"}]: dispatch 2024-07-16T09:57:13.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:13 smithi176 ceph-mon[36626]: from='mgr.? 172.21.15.176:0/4272015421' entity='mgr.smithi176.dqbrvx' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-07-16T09:57:13.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:13 smithi176 ceph-mon[36626]: from='mgr.? 172.21.15.176:0/4272015421' entity='mgr.smithi176.dqbrvx' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi176.dqbrvx/key"}]: dispatch 2024-07-16T09:57:13.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:13 smithi176 ceph-mon[36626]: from='mgr.? 172.21.15.176:0/4272015421' entity='mgr.smithi176.dqbrvx' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-07-16T09:57:13.902 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:13.899+0000 7f6b8c8cd640 1 -- 172.21.15.156:0/1063782230 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6b880fb240 msgr2=0x7f6b880fb650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:57:13.902 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:13.899+0000 7f6b8c8cd640 1 --2- 172.21.15.156:0/1063782230 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6b880fb240 0x7f6b880fb650 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f6b800048d0 tx=0x7f6b80009da0 comp rx=0 tx=0).stop 2024-07-16T09:57:13.902 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:13.899+0000 7f6b8c8cd640 1 -- 172.21.15.156:0/1063782230 shutdown_connections 2024-07-16T09:57:13.902 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:13.899+0000 7f6b8c8cd640 1 --2- 172.21.15.156:0/1063782230 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6b880fc510 0x7f6b880fc980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:57:13.902 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:13.899+0000 7f6b8c8cd640 1 --2- 172.21.15.156:0/1063782230 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6b880fb240 0x7f6b880fb650 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:57:13.902 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:13.899+0000 7f6b8c8cd640 1 -- 172.21.15.156:0/1063782230 >> 172.21.15.156:0/1063782230 conn(0x7f6b880f6a20 msgr2=0x7f6b880f8e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:57:13.903 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:13.899+0000 7f6b8c8cd640 1 -- 172.21.15.156:0/1063782230 shutdown_connections 2024-07-16T09:57:13.903 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:13.899+0000 7f6b8c8cd640 1 -- 172.21.15.156:0/1063782230 wait complete. 2024-07-16T09:57:13.903 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:13.899+0000 7f6b8c8cd640 1 Processor -- start 2024-07-16T09:57:13.903 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:13.899+0000 7f6b8c8cd640 1 -- start start 2024-07-16T09:57:13.903 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:13.900+0000 7f6b8c8cd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6b880fb240 0x7f6b8810cf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:57:13.903 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:13.900+0000 7f6b8c8cd640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6b880fc510 0x7f6b8810d490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:57:13.903 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:13.900+0000 7f6b8c8cd640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b8810dae0 con 0x7f6b880fb240 2024-07-16T09:57:13.903 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:13.900+0000 7f6b8c8cd640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b8810e300 con 0x7f6b880fc510 2024-07-16T09:57:13.903 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:13.900+0000 7f6b8659c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6b880fb240 0x7f6b8810cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:57:13.903 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:13.900+0000 7f6b8659c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6b880fb240 0x7f6b8810cf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:58890/0 (socket says 172.21.15.156:58890) 2024-07-16T09:57:13.904 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:13.900+0000 7f6b8659c640 1 -- 172.21.15.156:0/1908888821 learned_addr learned my addr 172.21.15.156:0/1908888821 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T09:57:13.904 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:13.900+0000 7f6b85d9b640 1 --2- 172.21.15.156:0/1908888821 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6b880fc510 0x7f6b8810d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:57:13.904 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:13.900+0000 7f6b8659c640 1 -- 172.21.15.156:0/1908888821 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6b880fc510 msgr2=0x7f6b8810d490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:57:13.904 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:13.900+0000 7f6b8659c640 1 --2- 172.21.15.156:0/1908888821 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6b880fc510 0x7f6b8810d490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:57:13.904 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:13.900+0000 7f6b8659c640 1 -- 172.21.15.156:0/1908888821 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b80005040 con 0x7f6b880fb240 2024-07-16T09:57:13.904 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:13.900+0000 7f6b8659c640 1 --2- 172.21.15.156:0/1908888821 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6b880fb240 0x7f6b8810cf50 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f6b800048a0 tx=0x7f6b80009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:57:13.904 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:13.900+0000 7f6b7effd640 1 -- 172.21.15.156:0/1908888821 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6b80038070 con 0x7f6b880fb240 2024-07-16T09:57:13.904 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:13.901+0000 7f6b7effd640 1 -- 172.21.15.156:0/1908888821 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6b80032970 con 0x7f6b880fb240 2024-07-16T09:57:13.904 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:13.901+0000 7f6b8c8cd640 1 -- 172.21.15.156:0/1908888821 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b8810e530 con 0x7f6b880fb240 2024-07-16T09:57:13.904 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:13.901+0000 7f6b7effd640 1 -- 172.21.15.156:0/1908888821 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6b8003d9a0 con 0x7f6b880fb240 2024-07-16T09:57:13.904 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:13.901+0000 7f6b8c8cd640 1 -- 172.21.15.156:0/1908888821 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b8810ea70 con 0x7f6b880fb240 2024-07-16T09:57:13.905 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:13.902+0000 7f6b7effd640 1 -- 172.21.15.156:0/1908888821 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 29) v1 ==== 46297+0+0 (secure 0 0 0) 0x7f6b80030020 con 0x7f6b880fb240 2024-07-16T09:57:13.905 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:13.902+0000 7f6b8c8cd640 1 -- 172.21.15.156:0/1908888821 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b880fa6a0 con 0x7f6b880fb240 2024-07-16T09:57:13.905 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:13.902+0000 7f6b7effd640 1 --2- 172.21.15.156:0/1908888821 >> [v2:172.21.15.156:6800/55311371,v1:172.21.15.156:6801/55311371] conn(0x7f6b74038e80 0x7f6b7403b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:57:13.906 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:13.902+0000 7f6b7effd640 1 -- 172.21.15.156:0/1908888821 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f6b8006d370 con 0x7f6b880fb240 2024-07-16T09:57:13.906 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:13.902+0000 7f6b85d9b640 1 --2- 172.21.15.156:0/1908888821 >> [v2:172.21.15.156:6800/55311371,v1:172.21.15.156:6801/55311371] conn(0x7f6b74038e80 0x7f6b7403b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:57:13.906 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:13.902+0000 7f6b85d9b640 1 --2- 172.21.15.156:0/1908888821 >> [v2:172.21.15.156:6800/55311371,v1:172.21.15.156:6801/55311371] conn(0x7f6b74038e80 0x7f6b7403b330 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f6b8810cd00 tx=0x7f6b70009040 comp rx=0 tx=0).ready entity=mgr.14722 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:57:13.908 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:13.907+0000 7f6b7effd640 1 -- 172.21.15.156:0/1908888821 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185936 (secure 0 0 0) 0x7f6b80045960 con 0x7f6b880fb240 2024-07-16T09:57:13.972 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:13.971+0000 7f6b7effd640 1 -- 172.21.15.156:0/1908888821 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mgrmap(e 30) v1 ==== 92029+0+0 (secure 0 0 0) 0x7f6b8003ccc0 con 0x7f6b880fb240 2024-07-16T09:57:14.135 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:14.134+0000 7f6b8c8cd640 1 -- 172.21.15.156:0/1908888821 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f6b88103e70 con 0x7f6b880fb240 2024-07-16T09:57:14.136 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:14.134+0000 7f6b7effd640 1 -- 172.21.15.156:0/1908888821 <== mon.0 v2:172.21.15.156:3300/0 8 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 18 v18) v1 ==== 76+0+1869 (secure 0 0 0) 0x7f6b8004f3b0 con 0x7f6b880fb240 2024-07-16T09:57:14.136 INFO:teuthology.orchestra.run.smithi156.stdout:e18 2024-07-16T09:57:14.136 INFO:teuthology.orchestra.run.smithi156.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-07-16T09:57:14.136 INFO:teuthology.orchestra.run.smithi156.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-07-16T09:57:14.136 INFO:teuthology.orchestra.run.smithi156.stdout:legacy client fscid: 1 2024-07-16T09:57:14.136 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-07-16T09:57:14.136 INFO:teuthology.orchestra.run.smithi156.stdout:Filesystem 'cephfs' (1) 2024-07-16T09:57:14.136 INFO:teuthology.orchestra.run.smithi156.stdout:fs_name cephfs 2024-07-16T09:57:14.136 INFO:teuthology.orchestra.run.smithi156.stdout:epoch 18 2024-07-16T09:57:14.137 INFO:teuthology.orchestra.run.smithi156.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-07-16T09:57:14.137 INFO:teuthology.orchestra.run.smithi156.stdout:created 2024-07-16T09:52:55.214975+0000 2024-07-16T09:57:14.137 INFO:teuthology.orchestra.run.smithi156.stdout:modified 2024-07-16T09:55:41.914320+0000 2024-07-16T09:57:14.137 INFO:teuthology.orchestra.run.smithi156.stdout:tableserver 0 2024-07-16T09:57:14.137 INFO:teuthology.orchestra.run.smithi156.stdout:root 0 2024-07-16T09:57:14.137 INFO:teuthology.orchestra.run.smithi156.stdout:session_timeout 60 2024-07-16T09:57:14.137 INFO:teuthology.orchestra.run.smithi156.stdout:session_autoclose 300 2024-07-16T09:57:14.137 INFO:teuthology.orchestra.run.smithi156.stdout:max_file_size 1099511627776 2024-07-16T09:57:14.137 INFO:teuthology.orchestra.run.smithi156.stdout:required_client_features {} 2024-07-16T09:57:14.137 INFO:teuthology.orchestra.run.smithi156.stdout:last_failure 0 2024-07-16T09:57:14.137 INFO:teuthology.orchestra.run.smithi156.stdout:last_failure_osd_epoch 0 2024-07-16T09:57:14.137 INFO:teuthology.orchestra.run.smithi156.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-07-16T09:57:14.138 INFO:teuthology.orchestra.run.smithi156.stdout:max_mds 2 2024-07-16T09:57:14.138 INFO:teuthology.orchestra.run.smithi156.stdout:in 0,1 2024-07-16T09:57:14.138 INFO:teuthology.orchestra.run.smithi156.stdout:up {0=14458,1=24261} 2024-07-16T09:57:14.138 INFO:teuthology.orchestra.run.smithi156.stdout:failed 2024-07-16T09:57:14.138 INFO:teuthology.orchestra.run.smithi156.stdout:damaged 2024-07-16T09:57:14.138 INFO:teuthology.orchestra.run.smithi156.stdout:stopped 2024-07-16T09:57:14.138 INFO:teuthology.orchestra.run.smithi156.stdout:data_pools [3] 2024-07-16T09:57:14.138 INFO:teuthology.orchestra.run.smithi156.stdout:metadata_pool 2 2024-07-16T09:57:14.138 INFO:teuthology.orchestra.run.smithi156.stdout:inline_data disabled 2024-07-16T09:57:14.138 INFO:teuthology.orchestra.run.smithi156.stdout:balancer 2024-07-16T09:57:14.138 INFO:teuthology.orchestra.run.smithi156.stdout:standby_count_wanted 1 2024-07-16T09:57:14.138 INFO:teuthology.orchestra.run.smithi156.stdout:[mds.cephfs.smithi156.rdibys{0:14458} state up:active seq 4 export targets 1 join_fscid=1 addr [v2:172.21.15.156:6826/1080176013,v1:172.21.15.156:6827/1080176013] compat {c=[1],r=[1],i=[7ff]}] 2024-07-16T09:57:14.139 INFO:teuthology.orchestra.run.smithi156.stdout:[mds.cephfs.smithi156.evwwaz{0:14468} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.156:6828/560329502,v1:172.21.15.156:6829/560329502] compat {c=[1],r=[1],i=[7ff]}] 2024-07-16T09:57:14.139 INFO:teuthology.orchestra.run.smithi156.stdout:[mds.cephfs.smithi176.dvvbud{1:24261} state up:active seq 4 export targets 0 join_fscid=1 addr [v2:172.21.15.176:6824/1577019287,v1:172.21.15.176:6825/1577019287] compat {c=[1],r=[1],i=[7ff]}] 2024-07-16T09:57:14.139 INFO:teuthology.orchestra.run.smithi156.stdout:[mds.cephfs.smithi176.wahbsi{1:24273} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.176:6826/2196775412,v1:172.21.15.176:6827/2196775412] compat {c=[1],r=[1],i=[7ff]}] 2024-07-16T09:57:14.139 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-07-16T09:57:14.139 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-07-16T09:57:14.139 INFO:teuthology.orchestra.run.smithi156.stderr:dumped fsmap epoch 18 2024-07-16T09:57:14.140 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:14.136+0000 7f6b8c8cd640 1 -- 172.21.15.156:0/1908888821 >> [v2:172.21.15.156:6800/55311371,v1:172.21.15.156:6801/55311371] conn(0x7f6b74038e80 msgr2=0x7f6b7403b330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:57:14.140 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:14.136+0000 7f6b8c8cd640 1 --2- 172.21.15.156:0/1908888821 >> [v2:172.21.15.156:6800/55311371,v1:172.21.15.156:6801/55311371] conn(0x7f6b74038e80 0x7f6b7403b330 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f6b8810cd00 tx=0x7f6b70009040 comp rx=0 tx=0).stop 2024-07-16T09:57:14.140 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:14.136+0000 7f6b8c8cd640 1 -- 172.21.15.156:0/1908888821 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6b880fb240 msgr2=0x7f6b8810cf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:57:14.140 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:14.136+0000 7f6b8c8cd640 1 --2- 172.21.15.156:0/1908888821 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6b880fb240 0x7f6b8810cf50 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f6b800048a0 tx=0x7f6b80009da0 comp rx=0 tx=0).stop 2024-07-16T09:57:14.140 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:14.136+0000 7f6b8c8cd640 1 -- 172.21.15.156:0/1908888821 shutdown_connections 2024-07-16T09:57:14.140 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:14.136+0000 7f6b8c8cd640 1 --2- 172.21.15.156:0/1908888821 >> [v2:172.21.15.156:6800/55311371,v1:172.21.15.156:6801/55311371] conn(0x7f6b74038e80 0x7f6b7403b330 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:57:14.140 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:14.136+0000 7f6b8c8cd640 1 --2- 172.21.15.156:0/1908888821 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6b880fc510 0x7f6b8810d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:57:14.140 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:14.136+0000 7f6b8c8cd640 1 --2- 172.21.15.156:0/1908888821 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6b880fb240 0x7f6b8810cf50 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:57:14.140 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:14.136+0000 7f6b8c8cd640 1 -- 172.21.15.156:0/1908888821 >> 172.21.15.156:0/1908888821 conn(0x7f6b880f6a20 msgr2=0x7f6b88065300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:57:14.140 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:14.136+0000 7f6b8c8cd640 1 -- 172.21.15.156:0/1908888821 shutdown_connections 2024-07-16T09:57:14.140 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:14.136+0000 7f6b8c8cd640 1 -- 172.21.15.156:0/1908888821 wait complete. 2024-07-16T09:57:14.218 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:14.216+0000 7f453e247640 1 -- 172.21.15.156:0/3975248048 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f45380fb230 msgr2=0x7f45380fb640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:57:14.218 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:14.216+0000 7f453e247640 1 --2- 172.21.15.156:0/3975248048 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f45380fb230 0x7f45380fb640 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f45280048d0 tx=0x7f4528009da0 comp rx=0 tx=0).stop 2024-07-16T09:57:14.218 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:14.217+0000 7f453e247640 1 -- 172.21.15.156:0/3975248048 shutdown_connections 2024-07-16T09:57:14.218 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:14.217+0000 7f453e247640 1 --2- 172.21.15.156:0/3975248048 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f45380fc500 0x7f45380fc970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:57:14.218 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:14.217+0000 7f453e247640 1 --2- 172.21.15.156:0/3975248048 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f45380fb230 0x7f45380fb640 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:57:14.219 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:14.217+0000 7f453e247640 1 -- 172.21.15.156:0/3975248048 >> 172.21.15.156:0/3975248048 conn(0x7f45380f6a10 msgr2=0x7f45380f8e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:57:14.219 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:14.217+0000 7f453e247640 1 -- 172.21.15.156:0/3975248048 shutdown_connections 2024-07-16T09:57:14.219 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:14.217+0000 7f453e247640 1 -- 172.21.15.156:0/3975248048 wait complete. 2024-07-16T09:57:14.219 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:14.217+0000 7f453e247640 1 Processor -- start 2024-07-16T09:57:14.220 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:14.217+0000 7f453e247640 1 -- start start 2024-07-16T09:57:14.220 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:14.217+0000 7f453e247640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f45380fb230 0x7f453810be80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:57:14.220 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:14.217+0000 7f453e247640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f45380fc500 0x7f453810c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:57:14.220 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:14.217+0000 7f453e247640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f453810ca10 con 0x7f45380fc500 2024-07-16T09:57:14.221 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:14.217+0000 7f453e247640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f453810d150 con 0x7f45380fb230 2024-07-16T09:57:14.221 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:14.218+0000 7f4536ffd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f45380fc500 0x7f453810c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:57:14.221 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:14.218+0000 7f4536ffd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f45380fc500 0x7f453810c3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:58902/0 (socket says 172.21.15.156:58902) 2024-07-16T09:57:14.221 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:14.218+0000 7f4536ffd640 1 -- 172.21.15.156:0/3652093710 learned_addr learned my addr 172.21.15.156:0/3652093710 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T09:57:14.221 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:14.218+0000 7f45377fe640 1 --2- 172.21.15.156:0/3652093710 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f45380fb230 0x7f453810be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:57:14.221 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:14.218+0000 7f4536ffd640 1 -- 172.21.15.156:0/3652093710 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f45380fb230 msgr2=0x7f453810be80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:57:14.221 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:14.218+0000 7f4536ffd640 1 --2- 172.21.15.156:0/3652093710 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f45380fb230 0x7f453810be80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:57:14.221 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:14.218+0000 7f4536ffd640 1 -- 172.21.15.156:0/3652093710 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4528005040 con 0x7f45380fc500 2024-07-16T09:57:14.221 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:14.218+0000 7f4536ffd640 1 --2- 172.21.15.156:0/3652093710 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f45380fc500 0x7f453810c3c0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f452c00c830 tx=0x7f452c004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:57:14.221 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:14.218+0000 7f451ffff640 1 -- 172.21.15.156:0/3652093710 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f452c00a850 con 0x7f45380fc500 2024-07-16T09:57:14.221 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:14.218+0000 7f451ffff640 1 -- 172.21.15.156:0/3652093710 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f452c00ae80 con 0x7f45380fc500 2024-07-16T09:57:14.221 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:14.218+0000 7f453e247640 1 -- 172.21.15.156:0/3652093710 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f453810d3e0 con 0x7f45380fc500 2024-07-16T09:57:14.221 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:14.218+0000 7f451ffff640 1 -- 172.21.15.156:0/3652093710 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f452c012ea0 con 0x7f45380fc500 2024-07-16T09:57:14.222 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:14.218+0000 7f453e247640 1 -- 172.21.15.156:0/3652093710 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f453810d980 con 0x7f45380fc500 2024-07-16T09:57:14.222 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:14.219+0000 7f453e247640 1 -- 172.21.15.156:0/3652093710 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45380fa690 con 0x7f45380fc500 2024-07-16T09:57:14.222 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:14.220+0000 7f451ffff640 1 -- 172.21.15.156:0/3652093710 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 30) v1 ==== 92029+0+0 (secure 0 0 0) 0x7f452c012400 con 0x7f45380fc500 2024-07-16T09:57:14.222 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:14.220+0000 7f451ffff640 1 --2- 172.21.15.156:0/3652093710 >> [v2:172.21.15.156:6800/55311371,v1:172.21.15.156:6801/55311371] conn(0x7f452006dae0 0x7f452006ff90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:57:14.223 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:14.220+0000 7f451ffff640 1 -- 172.21.15.156:0/3652093710 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f452c08d600 con 0x7f45380fc500 2024-07-16T09:57:14.223 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:14.220+0000 7f45377fe640 1 --2- 172.21.15.156:0/3652093710 >> [v2:172.21.15.156:6800/55311371,v1:172.21.15.156:6801/55311371] conn(0x7f452006dae0 0x7f452006ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:57:14.223 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:14.221+0000 7f45377fe640 1 --2- 172.21.15.156:0/3652093710 >> [v2:172.21.15.156:6800/55311371,v1:172.21.15.156:6801/55311371] conn(0x7f452006dae0 0x7f452006ff90 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f45280047f0 tx=0x7f4528009da0 comp rx=0 tx=0).ready entity=mgr.14722 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:57:14.226 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:14.225+0000 7f451ffff640 1 -- 172.21.15.156:0/3652093710 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185936 (secure 0 0 0) 0x7f452c0570e0 con 0x7f45380fc500 2024-07-16T09:57:14.443 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:14.442+0000 7f453e247640 1 -- 172.21.15.156:0/3652093710 --> [v2:172.21.15.156:6800/55311371,v1:172.21.15.156:6801/55311371] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f453810de40 con 0x7f452006dae0 2024-07-16T09:57:14.444 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:14.443+0000 7f451ffff640 1 -- 172.21.15.156:0/3652093710 <== mgr.14722 v2:172.21.15.156:6800/55311371 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+337 (secure 0 0 0) 0x7f453810de40 con 0x7f452006dae0 2024-07-16T09:57:14.445 INFO:teuthology.orchestra.run.smithi156.stdout:{ 2024-07-16T09:57:14.445 INFO:teuthology.orchestra.run.smithi156.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:c15ec3aa8e6c0a9c12ee24ff79f3e7dd4d01b3d1a702dfeb7d7fff9b0811a393", 2024-07-16T09:57:14.445 INFO:teuthology.orchestra.run.smithi156.stdout: "in_progress": true, 2024-07-16T09:57:14.445 INFO:teuthology.orchestra.run.smithi156.stdout: "which": "Upgrading all daemon types on all hosts", 2024-07-16T09:57:14.445 INFO:teuthology.orchestra.run.smithi156.stdout: "services_complete": [ 2024-07-16T09:57:14.445 INFO:teuthology.orchestra.run.smithi156.stdout: "mgr" 2024-07-16T09:57:14.445 INFO:teuthology.orchestra.run.smithi156.stdout: ], 2024-07-16T09:57:14.445 INFO:teuthology.orchestra.run.smithi156.stdout: "progress": "2/21 daemons upgraded", 2024-07-16T09:57:14.445 INFO:teuthology.orchestra.run.smithi156.stdout: "message": "", 2024-07-16T09:57:14.445 INFO:teuthology.orchestra.run.smithi156.stdout: "is_paused": false 2024-07-16T09:57:14.445 INFO:teuthology.orchestra.run.smithi156.stdout:} 2024-07-16T09:57:14.447 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:14.445+0000 7f453e247640 1 -- 172.21.15.156:0/3652093710 >> [v2:172.21.15.156:6800/55311371,v1:172.21.15.156:6801/55311371] conn(0x7f452006dae0 msgr2=0x7f452006ff90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:57:14.447 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:14.445+0000 7f453e247640 1 --2- 172.21.15.156:0/3652093710 >> [v2:172.21.15.156:6800/55311371,v1:172.21.15.156:6801/55311371] conn(0x7f452006dae0 0x7f452006ff90 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f45280047f0 tx=0x7f4528009da0 comp rx=0 tx=0).stop 2024-07-16T09:57:14.447 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:14.445+0000 7f453e247640 1 -- 172.21.15.156:0/3652093710 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f45380fc500 msgr2=0x7f453810c3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:57:14.447 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:14.445+0000 7f453e247640 1 --2- 172.21.15.156:0/3652093710 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f45380fc500 0x7f453810c3c0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f452c00c830 tx=0x7f452c004430 comp rx=0 tx=0).stop 2024-07-16T09:57:14.447 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:14.445+0000 7f453e247640 1 -- 172.21.15.156:0/3652093710 shutdown_connections 2024-07-16T09:57:14.447 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:14.445+0000 7f453e247640 1 --2- 172.21.15.156:0/3652093710 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f45380fc500 0x7f453810c3c0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:57:14.447 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:14.445+0000 7f453e247640 1 --2- 172.21.15.156:0/3652093710 >> [v2:172.21.15.156:6800/55311371,v1:172.21.15.156:6801/55311371] conn(0x7f452006dae0 0x7f452006ff90 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:57:14.447 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:14.445+0000 7f453e247640 1 --2- 172.21.15.156:0/3652093710 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f45380fb230 0x7f453810be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:57:14.447 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:14.446+0000 7f453e247640 1 -- 172.21.15.156:0/3652093710 >> 172.21.15.156:0/3652093710 conn(0x7f45380f6a10 msgr2=0x7f45380f8040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:57:14.448 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:14.446+0000 7f453e247640 1 -- 172.21.15.156:0/3652093710 shutdown_connections 2024-07-16T09:57:14.448 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:14.446+0000 7f453e247640 1 -- 172.21.15.156:0/3652093710 wait complete. 2024-07-16T09:57:14.536 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:14.535+0000 7fc72dbcb640 1 -- 172.21.15.156:0/2851921273 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc7280fd980 msgr2=0x7fc7280fdd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:57:14.536 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:14.535+0000 7fc72dbcb640 1 --2- 172.21.15.156:0/2851921273 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc7280fd980 0x7fc7280fdd90 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fc7180048d0 tx=0x7fc718009da0 comp rx=0 tx=0).stop 2024-07-16T09:57:14.536 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:14.535+0000 7fc72dbcb640 1 -- 172.21.15.156:0/2851921273 shutdown_connections 2024-07-16T09:57:14.536 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:14.535+0000 7fc72dbcb640 1 --2- 172.21.15.156:0/2851921273 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc7280fe2d0 0x7fc7280fa430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:57:14.536 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:14.535+0000 7fc72dbcb640 1 --2- 172.21.15.156:0/2851921273 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc7280fd980 0x7fc7280fdd90 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:57:14.537 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:14.535+0000 7fc72dbcb640 1 -- 172.21.15.156:0/2851921273 >> 172.21.15.156:0/2851921273 conn(0x7fc7280f61a0 msgr2=0x7fc7280f85b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:57:14.537 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:14.536+0000 7fc72dbcb640 1 -- 172.21.15.156:0/2851921273 shutdown_connections 2024-07-16T09:57:14.537 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:14.536+0000 7fc72dbcb640 1 -- 172.21.15.156:0/2851921273 wait complete. 2024-07-16T09:57:14.537 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:14.536+0000 7fc72dbcb640 1 Processor -- start 2024-07-16T09:57:14.539 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:14.536+0000 7fc72dbcb640 1 -- start start 2024-07-16T09:57:14.539 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:14.536+0000 7fc72dbcb640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc7280fd980 0x7fc7280fd170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:57:14.539 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:14.536+0000 7fc72dbcb640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc7280fe2d0 0x7fc7280fb660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:57:14.539 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:14.536+0000 7fc72dbcb640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7280fbbf0 con 0x7fc7280fe2d0 2024-07-16T09:57:14.539 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:14.536+0000 7fc72dbcb640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7280fbd30 con 0x7fc7280fd980 2024-07-16T09:57:14.539 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:14.537+0000 7fc71effd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc7280fe2d0 0x7fc7280fb660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:57:14.539 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:14.537+0000 7fc71effd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc7280fe2d0 0x7fc7280fb660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:58912/0 (socket says 172.21.15.156:58912) 2024-07-16T09:57:14.540 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:14.537+0000 7fc71effd640 1 -- 172.21.15.156:0/2299519186 learned_addr learned my addr 172.21.15.156:0/2299519186 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T09:57:14.540 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:14.537+0000 7fc7277fe640 1 --2- 172.21.15.156:0/2299519186 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc7280fd980 0x7fc7280fd170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:57:14.540 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:14.537+0000 7fc71effd640 1 -- 172.21.15.156:0/2299519186 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc7280fd980 msgr2=0x7fc7280fd170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:57:14.540 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:14.537+0000 7fc71effd640 1 --2- 172.21.15.156:0/2299519186 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc7280fd980 0x7fc7280fd170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:57:14.540 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:14.537+0000 7fc71effd640 1 -- 172.21.15.156:0/2299519186 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc718005040 con 0x7fc7280fe2d0 2024-07-16T09:57:14.540 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:14.537+0000 7fc71effd640 1 --2- 172.21.15.156:0/2299519186 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc7280fe2d0 0x7fc7280fb660 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fc710002870 tx=0x7fc710004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:57:14.540 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:14.538+0000 7fc724ff9640 1 -- 172.21.15.156:0/2299519186 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc710007c30 con 0x7fc7280fe2d0 2024-07-16T09:57:14.540 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:14.538+0000 7fc724ff9640 1 -- 172.21.15.156:0/2299519186 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc71000f040 con 0x7fc7280fe2d0 2024-07-16T09:57:14.540 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:14.538+0000 7fc72dbcb640 1 -- 172.21.15.156:0/2299519186 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7280fc010 con 0x7fc7280fe2d0 2024-07-16T09:57:14.541 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:14.538+0000 7fc724ff9640 1 -- 172.21.15.156:0/2299519186 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc71000da90 con 0x7fc7280fe2d0 2024-07-16T09:57:14.541 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:14.538+0000 7fc72dbcb640 1 -- 172.21.15.156:0/2299519186 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc728114fe0 con 0x7fc7280fe2d0 2024-07-16T09:57:14.541 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:14.539+0000 7fc724ff9640 1 -- 172.21.15.156:0/2299519186 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 30) v1 ==== 92029+0+0 (secure 0 0 0) 0x7fc71000dd50 con 0x7fc7280fe2d0 2024-07-16T09:57:14.542 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:14.539+0000 7fc72dbcb640 1 -- 172.21.15.156:0/2299519186 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7280f9e20 con 0x7fc7280fe2d0 2024-07-16T09:57:14.542 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:14.540+0000 7fc724ff9640 1 --2- 172.21.15.156:0/2299519186 >> [v2:172.21.15.156:6800/55311371,v1:172.21.15.156:6801/55311371] conn(0x7fc71406dae0 0x7fc71406ff90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:57:14.542 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:14.540+0000 7fc724ff9640 1 -- 172.21.15.156:0/2299519186 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7fc71008cb90 con 0x7fc7280fe2d0 2024-07-16T09:57:14.542 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:14.540+0000 7fc7277fe640 1 --2- 172.21.15.156:0/2299519186 >> [v2:172.21.15.156:6800/55311371,v1:172.21.15.156:6801/55311371] conn(0x7fc71406dae0 0x7fc71406ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:57:14.542 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:14.540+0000 7fc7277fe640 1 --2- 172.21.15.156:0/2299519186 >> [v2:172.21.15.156:6800/55311371,v1:172.21.15.156:6801/55311371] conn(0x7fc71406dae0 0x7fc71406ff90 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fc7180043d0 tx=0x7fc718035040 comp rx=0 tx=0).ready entity=mgr.14722 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:57:14.545 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:14.544+0000 7fc724ff9640 1 -- 172.21.15.156:0/2299519186 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185936 (secure 0 0 0) 0x7fc710056660 con 0x7fc7280fe2d0 2024-07-16T09:57:14.795 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:14 smithi156 ceph-mon[29302]: from='client.24535 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-16T09:57:14.795 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:14 smithi156 ceph-mon[29302]: from='client.14756 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-16T09:57:14.795 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:14 smithi156 ceph-mon[29302]: from='client.14760 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-07-16T09:57:14.795 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:14 smithi156 ceph-mon[29302]: from='client.? 172.21.15.156:0/267913264' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T09:57:14.795 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:14 smithi156 ceph-mon[29302]: mgrmap e30: smithi156.zkvylv(active, since 5s), standbys: smithi176.dqbrvx 2024-07-16T09:57:14.795 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:14 smithi156 ceph-mon[29302]: from='mgr.14722 172.21.15.156:0/647865130' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "mgr metadata", "who": "smithi176.dqbrvx", "id": "smithi176.dqbrvx"}]: dispatch 2024-07-16T09:57:14.795 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:14 smithi156 ceph-mon[29302]: from='client.? 172.21.15.156:0/1908888821' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-07-16T09:57:14.795 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:14 smithi156 ceph-mon[29302]: from='mgr.14722 172.21.15.156:0/647865130' entity='mgr.smithi156.zkvylv' 2024-07-16T09:57:14.796 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:14 smithi156 ceph-mon[29302]: from='mgr.14722 172.21.15.156:0/647865130' entity='mgr.smithi156.zkvylv' 2024-07-16T09:57:14.819 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:14.818+0000 7fc72dbcb640 1 -- 172.21.15.156:0/2299519186 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fc72804b530 con 0x7fc7280fe2d0 2024-07-16T09:57:14.819 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:14.818+0000 7fc724ff9640 1 -- 172.21.15.156:0/2299519186 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7fc710059d20 con 0x7fc7280fe2d0 2024-07-16T09:57:14.819 INFO:teuthology.orchestra.run.smithi156.stdout:HEALTH_OK 2024-07-16T09:57:14.822 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:14.820+0000 7fc72dbcb640 1 -- 172.21.15.156:0/2299519186 >> [v2:172.21.15.156:6800/55311371,v1:172.21.15.156:6801/55311371] conn(0x7fc71406dae0 msgr2=0x7fc71406ff90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:57:14.822 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:14.820+0000 7fc72dbcb640 1 --2- 172.21.15.156:0/2299519186 >> [v2:172.21.15.156:6800/55311371,v1:172.21.15.156:6801/55311371] conn(0x7fc71406dae0 0x7fc71406ff90 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fc7180043d0 tx=0x7fc718035040 comp rx=0 tx=0).stop 2024-07-16T09:57:14.822 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:14.820+0000 7fc72dbcb640 1 -- 172.21.15.156:0/2299519186 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc7280fe2d0 msgr2=0x7fc7280fb660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:57:14.822 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:14.820+0000 7fc72dbcb640 1 --2- 172.21.15.156:0/2299519186 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc7280fe2d0 0x7fc7280fb660 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fc710002870 tx=0x7fc710004600 comp rx=0 tx=0).stop 2024-07-16T09:57:14.822 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:14.821+0000 7fc72dbcb640 1 -- 172.21.15.156:0/2299519186 shutdown_connections 2024-07-16T09:57:14.823 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:14.821+0000 7fc72dbcb640 1 --2- 172.21.15.156:0/2299519186 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc7280fe2d0 0x7fc7280fb660 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:57:14.823 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:14.821+0000 7fc72dbcb640 1 --2- 172.21.15.156:0/2299519186 >> [v2:172.21.15.156:6800/55311371,v1:172.21.15.156:6801/55311371] conn(0x7fc71406dae0 0x7fc71406ff90 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:57:14.823 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:14.821+0000 7fc72dbcb640 1 --2- 172.21.15.156:0/2299519186 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc7280fd980 0x7fc7280fd170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:57:14.823 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:14.821+0000 7fc72dbcb640 1 -- 172.21.15.156:0/2299519186 >> 172.21.15.156:0/2299519186 conn(0x7fc7280f61a0 msgr2=0x7fc7280f7850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:57:14.823 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:14.821+0000 7fc72dbcb640 1 -- 172.21.15.156:0/2299519186 shutdown_connections 2024-07-16T09:57:14.823 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:14.821+0000 7fc72dbcb640 1 -- 172.21.15.156:0/2299519186 wait complete. 2024-07-16T09:57:15.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:14 smithi176 ceph-mon[36626]: from='client.24535 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-16T09:57:15.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:14 smithi176 ceph-mon[36626]: from='client.14756 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-16T09:57:15.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:14 smithi176 ceph-mon[36626]: from='client.14760 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-07-16T09:57:15.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:14 smithi176 ceph-mon[36626]: from='client.? 172.21.15.156:0/267913264' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T09:57:15.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:14 smithi176 ceph-mon[36626]: mgrmap e30: smithi156.zkvylv(active, since 5s), standbys: smithi176.dqbrvx 2024-07-16T09:57:15.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:14 smithi176 ceph-mon[36626]: from='mgr.14722 172.21.15.156:0/647865130' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "mgr metadata", "who": "smithi176.dqbrvx", "id": "smithi176.dqbrvx"}]: dispatch 2024-07-16T09:57:15.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:14 smithi176 ceph-mon[36626]: from='client.? 172.21.15.156:0/1908888821' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-07-16T09:57:15.095 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:14 smithi176 ceph-mon[36626]: from='mgr.14722 172.21.15.156:0/647865130' entity='mgr.smithi156.zkvylv' 2024-07-16T09:57:15.095 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:14 smithi176 ceph-mon[36626]: from='mgr.14722 172.21.15.156:0/647865130' entity='mgr.smithi156.zkvylv' 2024-07-16T09:57:15.892 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:15 smithi156 ceph-mon[29302]: pgmap v6: 65 pgs: 65 active+clean; 253 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-07-16T09:57:15.893 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:15 smithi156 ceph-mon[29302]: from='client.14772 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-16T09:57:15.893 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:15 smithi156 ceph-mon[29302]: from='client.? 172.21.15.156:0/2299519186' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-07-16T09:57:15.893 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:15 smithi156 ceph-mon[29302]: from='mgr.14722 172.21.15.156:0/647865130' entity='mgr.smithi156.zkvylv' 2024-07-16T09:57:15.893 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:15 smithi156 ceph-mon[29302]: from='mgr.14722 172.21.15.156:0/647865130' entity='mgr.smithi156.zkvylv' 2024-07-16T09:57:16.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:15 smithi176 ceph-mon[36626]: pgmap v6: 65 pgs: 65 active+clean; 253 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-07-16T09:57:16.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:15 smithi176 ceph-mon[36626]: from='client.14772 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-16T09:57:16.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:15 smithi176 ceph-mon[36626]: from='client.? 172.21.15.156:0/2299519186' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-07-16T09:57:16.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:15 smithi176 ceph-mon[36626]: from='mgr.14722 172.21.15.156:0/647865130' entity='mgr.smithi156.zkvylv' 2024-07-16T09:57:16.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:15 smithi176 ceph-mon[36626]: from='mgr.14722 172.21.15.156:0/647865130' entity='mgr.smithi156.zkvylv' 2024-07-16T09:57:17.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:17 smithi156 ceph-mon[29302]: pgmap v7: 65 pgs: 65 active+clean; 253 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 32 KiB/s rd, 0 B/s wr, 20 op/s 2024-07-16T09:57:18.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:17 smithi176 ceph-mon[36626]: pgmap v7: 65 pgs: 65 active+clean; 253 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 32 KiB/s rd, 0 B/s wr, 20 op/s 2024-07-16T09:57:19.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:19 smithi176 ceph-mon[36626]: pgmap v8: 65 pgs: 65 active+clean; 253 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 25 KiB/s rd, 0 B/s wr, 15 op/s 2024-07-16T09:57:19.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:19 smithi176 ceph-mon[36626]: from='mgr.14722 172.21.15.156:0/647865130' entity='mgr.smithi156.zkvylv' 2024-07-16T09:57:19.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:19 smithi176 ceph-mon[36626]: from='mgr.14722 172.21.15.156:0/647865130' entity='mgr.smithi156.zkvylv' 2024-07-16T09:57:19.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:19 smithi176 ceph-mon[36626]: from='mgr.14722 172.21.15.156:0/647865130' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-16T09:57:19.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:19 smithi176 ceph-mon[36626]: from='mgr.14722 172.21.15.156:0/647865130' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-16T09:57:19.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:19 smithi176 ceph-mon[36626]: Updating smithi156:/etc/ceph/ceph.conf 2024-07-16T09:57:19.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:19 smithi176 ceph-mon[36626]: Updating smithi176:/etc/ceph/ceph.conf 2024-07-16T09:57:19.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:19 smithi176 ceph-mon[36626]: from='mgr.14722 172.21.15.156:0/647865130' entity='mgr.smithi156.zkvylv' 2024-07-16T09:57:19.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:19 smithi176 ceph-mon[36626]: from='mgr.14722 172.21.15.156:0/647865130' entity='mgr.smithi156.zkvylv' 2024-07-16T09:57:19.845 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:19 smithi176 ceph-mon[36626]: from='mgr.14722 172.21.15.156:0/647865130' entity='mgr.smithi156.zkvylv' 2024-07-16T09:57:19.845 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:19 smithi176 ceph-mon[36626]: from='mgr.14722 172.21.15.156:0/647865130' entity='mgr.smithi156.zkvylv' 2024-07-16T09:57:19.845 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:19 smithi176 ceph-mon[36626]: from='mgr.14722 172.21.15.156:0/647865130' entity='mgr.smithi156.zkvylv' 2024-07-16T09:57:19.924 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:19 smithi156 ceph-mon[29302]: pgmap v8: 65 pgs: 65 active+clean; 253 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 25 KiB/s rd, 0 B/s wr, 15 op/s 2024-07-16T09:57:19.924 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:19 smithi156 ceph-mon[29302]: from='mgr.14722 172.21.15.156:0/647865130' entity='mgr.smithi156.zkvylv' 2024-07-16T09:57:19.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:19 smithi156 ceph-mon[29302]: from='mgr.14722 172.21.15.156:0/647865130' entity='mgr.smithi156.zkvylv' 2024-07-16T09:57:19.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:19 smithi156 ceph-mon[29302]: from='mgr.14722 172.21.15.156:0/647865130' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-16T09:57:19.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:19 smithi156 ceph-mon[29302]: from='mgr.14722 172.21.15.156:0/647865130' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-16T09:57:19.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:19 smithi156 ceph-mon[29302]: Updating smithi156:/etc/ceph/ceph.conf 2024-07-16T09:57:19.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:19 smithi156 ceph-mon[29302]: Updating smithi176:/etc/ceph/ceph.conf 2024-07-16T09:57:19.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:19 smithi156 ceph-mon[29302]: from='mgr.14722 172.21.15.156:0/647865130' entity='mgr.smithi156.zkvylv' 2024-07-16T09:57:19.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:19 smithi156 ceph-mon[29302]: from='mgr.14722 172.21.15.156:0/647865130' entity='mgr.smithi156.zkvylv' 2024-07-16T09:57:19.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:19 smithi156 ceph-mon[29302]: from='mgr.14722 172.21.15.156:0/647865130' entity='mgr.smithi156.zkvylv' 2024-07-16T09:57:19.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:19 smithi156 ceph-mon[29302]: from='mgr.14722 172.21.15.156:0/647865130' entity='mgr.smithi156.zkvylv' 2024-07-16T09:57:19.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:19 smithi156 ceph-mon[29302]: from='mgr.14722 172.21.15.156:0/647865130' entity='mgr.smithi156.zkvylv' 2024-07-16T09:57:20.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:20 smithi156 ceph-mon[29302]: Updating smithi156:/var/lib/ceph/26c91c26-4358-11ef-bcad-c7b262605968/config/ceph.conf 2024-07-16T09:57:20.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:20 smithi156 ceph-mon[29302]: Updating smithi176:/var/lib/ceph/26c91c26-4358-11ef-bcad-c7b262605968/config/ceph.conf 2024-07-16T09:57:20.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:20 smithi156 ceph-mon[29302]: Updating smithi156:/etc/ceph/ceph.client.admin.keyring 2024-07-16T09:57:20.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:20 smithi156 ceph-mon[29302]: Updating smithi176:/etc/ceph/ceph.client.admin.keyring 2024-07-16T09:57:20.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:20 smithi156 ceph-mon[29302]: Updating smithi156:/var/lib/ceph/26c91c26-4358-11ef-bcad-c7b262605968/config/ceph.client.admin.keyring 2024-07-16T09:57:20.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:20 smithi156 ceph-mon[29302]: Updating smithi176:/var/lib/ceph/26c91c26-4358-11ef-bcad-c7b262605968/config/ceph.client.admin.keyring 2024-07-16T09:57:20.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:20 smithi156 ceph-mon[29302]: Reconfiguring prometheus.smithi156 (dependencies changed)... 2024-07-16T09:57:20.843 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:20 smithi176 ceph-mon[36626]: Updating smithi156:/var/lib/ceph/26c91c26-4358-11ef-bcad-c7b262605968/config/ceph.conf 2024-07-16T09:57:20.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:20 smithi176 ceph-mon[36626]: Updating smithi176:/var/lib/ceph/26c91c26-4358-11ef-bcad-c7b262605968/config/ceph.conf 2024-07-16T09:57:20.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:20 smithi176 ceph-mon[36626]: Updating smithi156:/etc/ceph/ceph.client.admin.keyring 2024-07-16T09:57:20.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:20 smithi176 ceph-mon[36626]: Updating smithi176:/etc/ceph/ceph.client.admin.keyring 2024-07-16T09:57:20.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:20 smithi176 ceph-mon[36626]: Updating smithi156:/var/lib/ceph/26c91c26-4358-11ef-bcad-c7b262605968/config/ceph.client.admin.keyring 2024-07-16T09:57:20.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:20 smithi176 ceph-mon[36626]: Updating smithi176:/var/lib/ceph/26c91c26-4358-11ef-bcad-c7b262605968/config/ceph.client.admin.keyring 2024-07-16T09:57:20.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:20 smithi176 ceph-mon[36626]: Reconfiguring prometheus.smithi156 (dependencies changed)... 2024-07-16T09:57:21.782 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:21 smithi156 ceph-mon[29302]: Reconfiguring daemon prometheus.smithi156 on smithi156 2024-07-16T09:57:21.782 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:21 smithi156 ceph-mon[29302]: pgmap v9: 65 pgs: 65 active+clean; 253 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 21 KiB/s rd, 0 B/s wr, 14 op/s 2024-07-16T09:57:21.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:21 smithi176 ceph-mon[36626]: Reconfiguring daemon prometheus.smithi156 on smithi156 2024-07-16T09:57:21.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:21 smithi176 ceph-mon[36626]: pgmap v9: 65 pgs: 65 active+clean; 253 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 21 KiB/s rd, 0 B/s wr, 14 op/s 2024-07-16T09:57:23.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:23 smithi176 ceph-mon[36626]: pgmap v10: 65 pgs: 65 active+clean; 253 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 19 KiB/s rd, 0 B/s wr, 12 op/s 2024-07-16T09:57:23.848 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:23 smithi156 ceph-mon[29302]: pgmap v10: 65 pgs: 65 active+clean; 253 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 19 KiB/s rd, 0 B/s wr, 12 op/s 2024-07-16T09:57:24.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:24 smithi176 ceph-mon[36626]: from='mgr.14722 172.21.15.156:0/647865130' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-16T09:57:24.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:24 smithi176 ceph-mon[36626]: from='mgr.14722 172.21.15.156:0/647865130' entity='mgr.smithi156.zkvylv' 2024-07-16T09:57:24.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:24 smithi176 ceph-mon[36626]: from='mgr.14722 172.21.15.156:0/647865130' entity='mgr.smithi156.zkvylv' 2024-07-16T09:57:24.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:24 smithi176 ceph-mon[36626]: from='mgr.14722 172.21.15.156:0/647865130' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-07-16T09:57:24.924 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:24 smithi156 ceph-mon[29302]: from='mgr.14722 172.21.15.156:0/647865130' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-16T09:57:24.924 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:24 smithi156 ceph-mon[29302]: from='mgr.14722 172.21.15.156:0/647865130' entity='mgr.smithi156.zkvylv' 2024-07-16T09:57:24.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:24 smithi156 ceph-mon[29302]: from='mgr.14722 172.21.15.156:0/647865130' entity='mgr.smithi156.zkvylv' 2024-07-16T09:57:24.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:24 smithi156 ceph-mon[29302]: from='mgr.14722 172.21.15.156:0/647865130' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-07-16T09:57:25.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:25 smithi176 ceph-mon[36626]: pgmap v11: 65 pgs: 65 active+clean; 253 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 19 KiB/s rd, 0 B/s wr, 12 op/s 2024-07-16T09:57:25.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:25 smithi176 ceph-mon[36626]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-07-16T09:57:25.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:25 smithi176 ceph-mon[36626]: from='mgr.14722 172.21.15.156:0/647865130' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-16T09:57:25.924 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:25 smithi156 ceph-mon[29302]: pgmap v11: 65 pgs: 65 active+clean; 253 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 19 KiB/s rd, 0 B/s wr, 12 op/s 2024-07-16T09:57:25.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:25 smithi156 ceph-mon[29302]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-07-16T09:57:25.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:25 smithi156 ceph-mon[29302]: from='mgr.14722 172.21.15.156:0/647865130' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-16T09:57:26.843 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:26 smithi176 ceph-mon[36626]: from='mgr.14722 172.21.15.156:0/647865130' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi176.dqbrvx", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-07-16T09:57:26.843 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:26 smithi176 ceph-mon[36626]: from='mgr.14722 172.21.15.156:0/647865130' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "mgr services"}]: dispatch 2024-07-16T09:57:26.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:26 smithi176 ceph-mon[36626]: from='mgr.14722 172.21.15.156:0/647865130' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-16T09:57:26.924 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:26 smithi156 ceph-mon[29302]: from='mgr.14722 172.21.15.156:0/647865130' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi176.dqbrvx", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-07-16T09:57:26.924 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:26 smithi156 ceph-mon[29302]: from='mgr.14722 172.21.15.156:0/647865130' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "mgr services"}]: dispatch 2024-07-16T09:57:26.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:26 smithi156 ceph-mon[29302]: from='mgr.14722 172.21.15.156:0/647865130' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-16T09:57:27.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:27 smithi176 ceph-mon[36626]: Upgrade: Updating mgr.smithi176.dqbrvx 2024-07-16T09:57:27.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:27 smithi176 ceph-mon[36626]: Deploying daemon mgr.smithi176.dqbrvx on smithi176 2024-07-16T09:57:27.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:27 smithi176 ceph-mon[36626]: pgmap v12: 65 pgs: 65 active+clean; 253 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 20 KiB/s rd, 0 B/s wr, 14 op/s 2024-07-16T09:57:27.924 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:27 smithi156 ceph-mon[29302]: Upgrade: Updating mgr.smithi176.dqbrvx 2024-07-16T09:57:27.924 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:27 smithi156 ceph-mon[29302]: Deploying daemon mgr.smithi176.dqbrvx on smithi176 2024-07-16T09:57:27.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:27 smithi156 ceph-mon[29302]: pgmap v12: 65 pgs: 65 active+clean; 253 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 20 KiB/s rd, 0 B/s wr, 14 op/s 2024-07-16T09:57:29.842 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:29 smithi176 ceph-mon[36626]: pgmap v13: 65 pgs: 65 active+clean; 253 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-07-16T09:57:29.842 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:29 smithi176 ceph-mon[36626]: from='mgr.14722 172.21.15.156:0/647865130' entity='mgr.smithi156.zkvylv' 2024-07-16T09:57:29.842 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:29 smithi176 ceph-mon[36626]: from='mgr.14722 172.21.15.156:0/647865130' entity='mgr.smithi156.zkvylv' 2024-07-16T09:57:29.843 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:29 smithi176 ceph-mon[36626]: from='mgr.14722 172.21.15.156:0/647865130' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-16T09:57:29.924 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:29 smithi156 ceph-mon[29302]: pgmap v13: 65 pgs: 65 active+clean; 253 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-07-16T09:57:29.924 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:29 smithi156 ceph-mon[29302]: from='mgr.14722 172.21.15.156:0/647865130' entity='mgr.smithi156.zkvylv' 2024-07-16T09:57:29.924 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:29 smithi156 ceph-mon[29302]: from='mgr.14722 172.21.15.156:0/647865130' entity='mgr.smithi156.zkvylv' 2024-07-16T09:57:29.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:29 smithi156 ceph-mon[29302]: from='mgr.14722 172.21.15.156:0/647865130' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-16T09:57:31.630 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:31 smithi176 ceph-mon[36626]: pgmap v14: 65 pgs: 65 active+clean; 253 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-07-16T09:57:31.924 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:31 smithi156 ceph-mon[29302]: pgmap v14: 65 pgs: 65 active+clean; 253 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-07-16T09:57:33.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:33 smithi176 ceph-mon[36626]: pgmap v15: 65 pgs: 65 active+clean; 253 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-07-16T09:57:33.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:33 smithi176 ceph-mon[36626]: from='mgr.14722 172.21.15.156:0/647865130' entity='mgr.smithi156.zkvylv' 2024-07-16T09:57:33.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:33 smithi176 ceph-mon[36626]: from='mgr.14722 172.21.15.156:0/647865130' entity='mgr.smithi156.zkvylv' 2024-07-16T09:57:33.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:33 smithi176 ceph-mon[36626]: from='mgr.14722 172.21.15.156:0/647865130' entity='mgr.smithi156.zkvylv' 2024-07-16T09:57:33.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:33 smithi176 ceph-mon[36626]: from='mgr.14722 172.21.15.156:0/647865130' entity='mgr.smithi156.zkvylv' 2024-07-16T09:57:33.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:33 smithi156 ceph-mon[29302]: pgmap v15: 65 pgs: 65 active+clean; 253 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-07-16T09:57:33.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:33 smithi156 ceph-mon[29302]: from='mgr.14722 172.21.15.156:0/647865130' entity='mgr.smithi156.zkvylv' 2024-07-16T09:57:33.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:33 smithi156 ceph-mon[29302]: from='mgr.14722 172.21.15.156:0/647865130' entity='mgr.smithi156.zkvylv' 2024-07-16T09:57:33.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:33 smithi156 ceph-mon[29302]: from='mgr.14722 172.21.15.156:0/647865130' entity='mgr.smithi156.zkvylv' 2024-07-16T09:57:33.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:33 smithi156 ceph-mon[29302]: from='mgr.14722 172.21.15.156:0/647865130' entity='mgr.smithi156.zkvylv' 2024-07-16T09:57:35.537 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:35 smithi156 ceph-mon[29302]: from='mgr.14722 172.21.15.156:0/647865130' entity='mgr.smithi156.zkvylv' 2024-07-16T09:57:35.537 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:35 smithi156 ceph-mon[29302]: from='mgr.14722 172.21.15.156:0/647865130' entity='mgr.smithi156.zkvylv' 2024-07-16T09:57:35.537 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:35 smithi156 ceph-mon[29302]: pgmap v16: 65 pgs: 65 active+clean; 253 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-07-16T09:57:35.537 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:35 smithi156 ceph-mon[29302]: Standby manager daemon smithi176.dqbrvx restarted 2024-07-16T09:57:35.537 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:35 smithi156 ceph-mon[29302]: Standby manager daemon smithi176.dqbrvx started 2024-07-16T09:57:35.537 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:35 smithi156 ceph-mon[29302]: from='mgr.? 172.21.15.176:0/2043786867' entity='mgr.smithi176.dqbrvx' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi176.dqbrvx/crt"}]: dispatch 2024-07-16T09:57:35.537 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:35 smithi156 ceph-mon[29302]: from='mgr.? 172.21.15.176:0/2043786867' entity='mgr.smithi176.dqbrvx' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-07-16T09:57:35.537 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:35 smithi156 ceph-mon[29302]: from='mgr.? 172.21.15.176:0/2043786867' entity='mgr.smithi176.dqbrvx' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi176.dqbrvx/key"}]: dispatch 2024-07-16T09:57:35.538 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:35 smithi156 ceph-mon[29302]: from='mgr.? 172.21.15.176:0/2043786867' entity='mgr.smithi176.dqbrvx' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-07-16T09:57:35.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:35 smithi176 ceph-mon[36626]: from='mgr.14722 172.21.15.156:0/647865130' entity='mgr.smithi156.zkvylv' 2024-07-16T09:57:35.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:35 smithi176 ceph-mon[36626]: from='mgr.14722 172.21.15.156:0/647865130' entity='mgr.smithi156.zkvylv' 2024-07-16T09:57:35.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:35 smithi176 ceph-mon[36626]: pgmap v16: 65 pgs: 65 active+clean; 253 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-07-16T09:57:35.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:35 smithi176 ceph-mon[36626]: Standby manager daemon smithi176.dqbrvx restarted 2024-07-16T09:57:35.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:35 smithi176 ceph-mon[36626]: Standby manager daemon smithi176.dqbrvx started 2024-07-16T09:57:35.595 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:35 smithi176 ceph-mon[36626]: from='mgr.? 172.21.15.176:0/2043786867' entity='mgr.smithi176.dqbrvx' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi176.dqbrvx/crt"}]: dispatch 2024-07-16T09:57:35.595 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:35 smithi176 ceph-mon[36626]: from='mgr.? 172.21.15.176:0/2043786867' entity='mgr.smithi176.dqbrvx' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-07-16T09:57:35.595 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:35 smithi176 ceph-mon[36626]: from='mgr.? 172.21.15.176:0/2043786867' entity='mgr.smithi176.dqbrvx' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi176.dqbrvx/key"}]: dispatch 2024-07-16T09:57:35.595 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:35 smithi176 ceph-mon[36626]: from='mgr.? 172.21.15.176:0/2043786867' entity='mgr.smithi176.dqbrvx' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-07-16T09:57:36.343 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:36 smithi176 ceph-mon[36626]: mgrmap e31: smithi156.zkvylv(active, since 27s), standbys: smithi176.dqbrvx 2024-07-16T09:57:36.424 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:36 smithi156 ceph-mon[29302]: mgrmap e31: smithi156.zkvylv(active, since 27s), standbys: smithi176.dqbrvx 2024-07-16T09:57:37.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:37 smithi156 ceph-mon[29302]: pgmap v17: 65 pgs: 65 active+clean; 253 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-07-16T09:57:37.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:37 smithi156 ceph-mon[29302]: from='mgr.14722 172.21.15.156:0/647865130' entity='mgr.smithi156.zkvylv' 2024-07-16T09:57:37.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:37 smithi156 ceph-mon[29302]: from='mgr.14722 172.21.15.156:0/647865130' entity='mgr.smithi156.zkvylv' 2024-07-16T09:57:37.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:37 smithi156 ceph-mon[29302]: from='mgr.14722 172.21.15.156:0/647865130' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-16T09:57:37.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:37 smithi156 ceph-mon[29302]: from='mgr.14722 172.21.15.156:0/647865130' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-16T09:57:37.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:37 smithi156 ceph-mon[29302]: from='mgr.14722 172.21.15.156:0/647865130' entity='mgr.smithi156.zkvylv' 2024-07-16T09:57:37.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:37 smithi156 ceph-mon[29302]: from='mgr.14722 172.21.15.156:0/647865130' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-16T09:57:37.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:37 smithi156 ceph-mon[29302]: from='mgr.14722 172.21.15.156:0/647865130' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T09:57:37.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:37 smithi156 ceph-mon[29302]: Upgrade: Setting container_image for all mgr 2024-07-16T09:57:37.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:37 smithi156 ceph-mon[29302]: from='mgr.14722 172.21.15.156:0/647865130' entity='mgr.smithi156.zkvylv' 2024-07-16T09:57:37.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:37 smithi156 ceph-mon[29302]: from='mgr.14722 172.21.15.156:0/647865130' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi156.zkvylv"}]: dispatch 2024-07-16T09:57:37.426 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:37 smithi156 ceph-mon[29302]: from='mgr.14722 172.21.15.156:0/647865130' entity='mgr.smithi156.zkvylv' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi156.zkvylv"}]': finished 2024-07-16T09:57:37.426 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:37 smithi156 ceph-mon[29302]: from='mgr.14722 172.21.15.156:0/647865130' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi176.dqbrvx"}]: dispatch 2024-07-16T09:57:37.426 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:37 smithi156 ceph-mon[29302]: from='mgr.14722 172.21.15.156:0/647865130' entity='mgr.smithi156.zkvylv' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi176.dqbrvx"}]': finished 2024-07-16T09:57:37.426 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:37 smithi156 ceph-mon[29302]: from='mgr.14722 172.21.15.156:0/647865130' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-07-16T09:57:37.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:37 smithi176 ceph-mon[36626]: pgmap v17: 65 pgs: 65 active+clean; 253 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-07-16T09:57:37.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:37 smithi176 ceph-mon[36626]: from='mgr.14722 172.21.15.156:0/647865130' entity='mgr.smithi156.zkvylv' 2024-07-16T09:57:37.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:37 smithi176 ceph-mon[36626]: from='mgr.14722 172.21.15.156:0/647865130' entity='mgr.smithi156.zkvylv' 2024-07-16T09:57:37.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:37 smithi176 ceph-mon[36626]: from='mgr.14722 172.21.15.156:0/647865130' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-16T09:57:37.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:37 smithi176 ceph-mon[36626]: from='mgr.14722 172.21.15.156:0/647865130' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-16T09:57:37.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:37 smithi176 ceph-mon[36626]: from='mgr.14722 172.21.15.156:0/647865130' entity='mgr.smithi156.zkvylv' 2024-07-16T09:57:37.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:37 smithi176 ceph-mon[36626]: from='mgr.14722 172.21.15.156:0/647865130' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-16T09:57:37.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:37 smithi176 ceph-mon[36626]: from='mgr.14722 172.21.15.156:0/647865130' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T09:57:37.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:37 smithi176 ceph-mon[36626]: Upgrade: Setting container_image for all mgr 2024-07-16T09:57:37.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:37 smithi176 ceph-mon[36626]: from='mgr.14722 172.21.15.156:0/647865130' entity='mgr.smithi156.zkvylv' 2024-07-16T09:57:37.845 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:37 smithi176 ceph-mon[36626]: from='mgr.14722 172.21.15.156:0/647865130' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi156.zkvylv"}]: dispatch 2024-07-16T09:57:37.845 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:37 smithi176 ceph-mon[36626]: from='mgr.14722 172.21.15.156:0/647865130' entity='mgr.smithi156.zkvylv' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi156.zkvylv"}]': finished 2024-07-16T09:57:37.845 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:37 smithi176 ceph-mon[36626]: from='mgr.14722 172.21.15.156:0/647865130' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi176.dqbrvx"}]: dispatch 2024-07-16T09:57:37.845 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:37 smithi176 ceph-mon[36626]: from='mgr.14722 172.21.15.156:0/647865130' entity='mgr.smithi156.zkvylv' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi176.dqbrvx"}]': finished 2024-07-16T09:57:37.845 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:37 smithi176 ceph-mon[36626]: from='mgr.14722 172.21.15.156:0/647865130' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-07-16T09:57:39.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:38 smithi176 ceph-mon[36626]: Upgrade: Updating mon.smithi156 2024-07-16T09:57:39.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:38 smithi176 ceph-mon[36626]: from='mgr.14722 172.21.15.156:0/647865130' entity='mgr.smithi156.zkvylv' 2024-07-16T09:57:39.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:38 smithi176 ceph-mon[36626]: from='mgr.14722 172.21.15.156:0/647865130' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-07-16T09:57:39.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:38 smithi176 ceph-mon[36626]: from='mgr.14722 172.21.15.156:0/647865130' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-07-16T09:57:39.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:38 smithi176 ceph-mon[36626]: from='mgr.14722 172.21.15.156:0/647865130' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-16T09:57:39.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:38 smithi176 ceph-mon[36626]: Deploying daemon mon.smithi156 on smithi156 2024-07-16T09:57:39.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:38 smithi176 ceph-mon[36626]: pgmap v18: 65 pgs: 65 active+clean; 253 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-07-16T09:57:39.345 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:38 smithi176 ceph-mon[36626]: from='mgr.14722 172.21.15.156:0/647865130' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-16T09:57:39.367 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:38 smithi156 ceph-mon[29302]: Upgrade: Updating mon.smithi156 2024-07-16T09:57:39.367 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:38 smithi156 ceph-mon[29302]: from='mgr.14722 172.21.15.156:0/647865130' entity='mgr.smithi156.zkvylv' 2024-07-16T09:57:39.367 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:38 smithi156 ceph-mon[29302]: from='mgr.14722 172.21.15.156:0/647865130' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-07-16T09:57:39.367 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:38 smithi156 ceph-mon[29302]: from='mgr.14722 172.21.15.156:0/647865130' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-07-16T09:57:39.367 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:38 smithi156 ceph-mon[29302]: from='mgr.14722 172.21.15.156:0/647865130' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-16T09:57:39.367 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:38 smithi156 ceph-mon[29302]: Deploying daemon mon.smithi156 on smithi156 2024-07-16T09:57:39.367 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:38 smithi156 ceph-mon[29302]: pgmap v18: 65 pgs: 65 active+clean; 253 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-07-16T09:57:39.367 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:38 smithi156 ceph-mon[29302]: from='mgr.14722 172.21.15.156:0/647865130' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-16T09:57:40.125 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:39 smithi156 systemd[1]: Stopping Ceph mon.smithi156 for 26c91c26-4358-11ef-bcad-c7b262605968... 2024-07-16T09:57:40.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:40 smithi156 ceph-26c91c26-4358-11ef-bcad-c7b262605968-mon-smithi156[29298]: 2024-07-16T09:57:40.123+0000 7fdb887ef640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi156 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-07-16T09:57:40.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:40 smithi156 ceph-26c91c26-4358-11ef-bcad-c7b262605968-mon-smithi156[29298]: 2024-07-16T09:57:40.123+0000 7fdb887ef640 -1 mon.smithi156@0(leader) e2 *** Got Signal Terminated *** 2024-07-16T09:57:40.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:40 smithi156 podman[92892]: 2024-07-16 09:57:40.215821102 +0000 UTC m=+0.187149024 container died 6471c8acff4002d36c1d1a5a5b009315b53bee9c4642cd88cbfec151eed1ac2b (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-26c91c26-4358-11ef-bcad-c7b262605968-mon-smithi156, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=quincy-e6231b0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240709, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-07-16T09:57:40.924 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:40 smithi156 podman[92892]: 2024-07-16 09:57:40.690999651 +0000 UTC m=+0.662327617 container cleanup 6471c8acff4002d36c1d1a5a5b009315b53bee9c4642cd88cbfec151eed1ac2b (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-26c91c26-4358-11ef-bcad-c7b262605968-mon-smithi156, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240709, io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=quincy-e6231b0, maintainer=Guillaume Abrioux ) 2024-07-16T09:57:40.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:40 smithi156 bash[92892]: ceph-26c91c26-4358-11ef-bcad-c7b262605968-mon-smithi156 2024-07-16T09:57:41.318 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:40 smithi156 podman[92904]: 2024-07-16 09:57:40.948136729 +0000 UTC m=+0.734185989 container remove 6471c8acff4002d36c1d1a5a5b009315b53bee9c4642cd88cbfec151eed1ac2b (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-26c91c26-4358-11ef-bcad-c7b262605968-mon-smithi156, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=quincy-e6231b0, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240709, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image) 2024-07-16T09:57:41.319 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 systemd[1]: ceph-26c91c26-4358-11ef-bcad-c7b262605968@mon.smithi156.service: Deactivated successfully. 2024-07-16T09:57:41.319 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 systemd[1]: Stopped Ceph mon.smithi156 for 26c91c26-4358-11ef-bcad-c7b262605968. 2024-07-16T09:57:41.319 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 systemd[1]: ceph-26c91c26-4358-11ef-bcad-c7b262605968@mon.smithi156.service: Consumed 10.100s CPU time. 2024-07-16T09:57:41.670 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 systemd[1]: Starting Ceph mon.smithi156 for 26c91c26-4358-11ef-bcad-c7b262605968... 2024-07-16T09:57:41.670 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 podman[93048]: 2024-07-16 09:57:41.437607921 +0000 UTC m=+0.017696460 image pull 16f7feb043422ce37a2471403d4bc52f550d49cef4b677482112fcc0db3f67ab quay.ceph.io/ceph-ci/ceph@sha256:c15ec3aa8e6c0a9c12ee24ff79f3e7dd4d01b3d1a702dfeb7d7fff9b0811a393 2024-07-16T09:57:41.671 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 podman[93048]: 2024-07-16 09:57:41.560703006 +0000 UTC m=+0.140791556 container create 77b6ec8c50eb5165a042b8c22a28eb8ecd167c807708bd75457f3a70b576c408 (image=quay.ceph.io/ceph-ci/ceph@sha256:c15ec3aa8e6c0a9c12ee24ff79f3e7dd4d01b3d1a702dfeb7d7fff9b0811a393, name=ceph-26c91c26-4358-11ef-bcad-c7b262605968-mon-smithi156, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-adk4-testing-2024-07-15-1832-squid-a6ca461, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240709, GIT_CLEAN=True) 2024-07-16T09:57:42.175 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 podman[93048]: 2024-07-16 09:57:41.788145979 +0000 UTC m=+0.368234521 container init 77b6ec8c50eb5165a042b8c22a28eb8ecd167c807708bd75457f3a70b576c408 (image=quay.ceph.io/ceph-ci/ceph@sha256:c15ec3aa8e6c0a9c12ee24ff79f3e7dd4d01b3d1a702dfeb7d7fff9b0811a393, name=ceph-26c91c26-4358-11ef-bcad-c7b262605968-mon-smithi156, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-adk4-testing-2024-07-15-1832-squid-a6ca461, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.36.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240709) 2024-07-16T09:57:42.175 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 podman[93048]: 2024-07-16 09:57:41.791643078 +0000 UTC m=+0.371731609 container start 77b6ec8c50eb5165a042b8c22a28eb8ecd167c807708bd75457f3a70b576c408 (image=quay.ceph.io/ceph-ci/ceph@sha256:c15ec3aa8e6c0a9c12ee24ff79f3e7dd4d01b3d1a702dfeb7d7fff9b0811a393, name=ceph-26c91c26-4358-11ef-bcad-c7b262605968-mon-smithi156, RELEASE=wip-adk4-testing-2024-07-15-1832-squid-a6ca461, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.36.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240709, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0) 2024-07-16T09:57:42.175 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 ceph-mon[93066]: set uid:gid to 167:167 (ceph:ceph) 2024-07-16T09:57:42.175 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 ceph-mon[93066]: ceph version 19.0.0-3495-ga6ca461b (a6ca461be7cc1aff3658c2c6df4cff975a24449a) squid (rc), process ceph-mon, pid 2 2024-07-16T09:57:42.175 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 ceph-mon[93066]: pidfile_write: ignore empty --pid-file 2024-07-16T09:57:42.175 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 ceph-mon[93066]: load: jerasure load: lrc 2024-07-16T09:57:42.176 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 ceph-mon[93066]: rocksdb: RocksDB version: 7.9.2 2024-07-16T09:57:42.176 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 ceph-mon[93066]: rocksdb: Git sha 0 2024-07-16T09:57:42.176 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 ceph-mon[93066]: rocksdb: Compile date 2024-07-15 23:29:57 2024-07-16T09:57:42.176 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 ceph-mon[93066]: rocksdb: DB SUMMARY 2024-07-16T09:57:42.176 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 ceph-mon[93066]: rocksdb: DB Session ID: 9GULJKWWIEUEMILYGNVJ 2024-07-16T09:57:42.176 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 ceph-mon[93066]: rocksdb: CURRENT file: CURRENT 2024-07-16T09:57:42.176 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 ceph-mon[93066]: rocksdb: IDENTITY file: IDENTITY 2024-07-16T09:57:42.176 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 ceph-mon[93066]: rocksdb: MANIFEST file: MANIFEST-000015 size: 972 Bytes 2024-07-16T09:57:42.176 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 ceph-mon[93066]: rocksdb: SST files in /var/lib/ceph/mon/ceph-smithi156/store.db dir, Total Num: 1, files: 000030.sst 2024-07-16T09:57:42.176 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 ceph-mon[93066]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-smithi156/store.db: 000028.log size: 1768543 ; 2024-07-16T09:57:42.176 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 ceph-mon[93066]: rocksdb: Options.error_if_exists: 0 2024-07-16T09:57:42.176 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 ceph-mon[93066]: rocksdb: Options.create_if_missing: 0 2024-07-16T09:57:42.177 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 ceph-mon[93066]: rocksdb: Options.paranoid_checks: 1 2024-07-16T09:57:42.177 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 ceph-mon[93066]: rocksdb: Options.flush_verify_memtable_count: 1 2024-07-16T09:57:42.177 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 ceph-mon[93066]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-07-16T09:57:42.177 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 ceph-mon[93066]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-07-16T09:57:42.177 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 ceph-mon[93066]: rocksdb: Options.env: 0x558c8630d740 2024-07-16T09:57:42.177 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 ceph-mon[93066]: rocksdb: Options.fs: PosixFileSystem 2024-07-16T09:57:42.177 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 ceph-mon[93066]: rocksdb: Options.info_log: 0x558c877296a0 2024-07-16T09:57:42.177 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 ceph-mon[93066]: rocksdb: Options.max_file_opening_threads: 16 2024-07-16T09:57:42.179 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 ceph-mon[93066]: rocksdb: Options.statistics: (nil) 2024-07-16T09:57:42.179 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 ceph-mon[93066]: rocksdb: Options.use_fsync: 0 2024-07-16T09:57:42.179 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 ceph-mon[93066]: rocksdb: Options.max_log_file_size: 0 2024-07-16T09:57:42.179 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 ceph-mon[93066]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-07-16T09:57:42.179 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 ceph-mon[93066]: rocksdb: Options.log_file_time_to_roll: 0 2024-07-16T09:57:42.179 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 ceph-mon[93066]: rocksdb: Options.keep_log_file_num: 1000 2024-07-16T09:57:42.180 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 ceph-mon[93066]: rocksdb: Options.recycle_log_file_num: 0 2024-07-16T09:57:42.180 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 ceph-mon[93066]: rocksdb: Options.allow_fallocate: 1 2024-07-16T09:57:42.180 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 ceph-mon[93066]: rocksdb: Options.allow_mmap_reads: 0 2024-07-16T09:57:42.180 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 ceph-mon[93066]: rocksdb: Options.allow_mmap_writes: 0 2024-07-16T09:57:42.180 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 ceph-mon[93066]: rocksdb: Options.use_direct_reads: 0 2024-07-16T09:57:42.180 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 ceph-mon[93066]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-07-16T09:57:42.180 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 ceph-mon[93066]: rocksdb: Options.create_missing_column_families: 0 2024-07-16T09:57:42.180 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 ceph-mon[93066]: rocksdb: Options.db_log_dir: 2024-07-16T09:57:42.180 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 ceph-mon[93066]: rocksdb: Options.wal_dir: 2024-07-16T09:57:42.180 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 ceph-mon[93066]: rocksdb: Options.table_cache_numshardbits: 6 2024-07-16T09:57:42.180 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 ceph-mon[93066]: rocksdb: Options.WAL_ttl_seconds: 0 2024-07-16T09:57:42.180 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 ceph-mon[93066]: rocksdb: Options.WAL_size_limit_MB: 0 2024-07-16T09:57:42.181 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 ceph-mon[93066]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-07-16T09:57:42.181 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 ceph-mon[93066]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-07-16T09:57:42.181 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 ceph-mon[93066]: rocksdb: Options.is_fd_close_on_exec: 1 2024-07-16T09:57:42.181 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 ceph-mon[93066]: rocksdb: Options.advise_random_on_open: 1 2024-07-16T09:57:42.181 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 ceph-mon[93066]: rocksdb: Options.db_write_buffer_size: 0 2024-07-16T09:57:42.181 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 ceph-mon[93066]: rocksdb: Options.write_buffer_manager: 0x558c87738dc0 2024-07-16T09:57:42.181 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 ceph-mon[93066]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-07-16T09:57:42.181 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 ceph-mon[93066]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-07-16T09:57:42.181 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 ceph-mon[93066]: rocksdb: Options.use_adaptive_mutex: 0 2024-07-16T09:57:42.181 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 ceph-mon[93066]: rocksdb: Options.rate_limiter: (nil) 2024-07-16T09:57:42.181 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 ceph-mon[93066]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-07-16T09:57:42.181 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 ceph-mon[93066]: rocksdb: Options.wal_recovery_mode: 2 2024-07-16T09:57:42.181 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 ceph-mon[93066]: rocksdb: Options.enable_thread_tracking: 0 2024-07-16T09:57:42.182 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 ceph-mon[93066]: rocksdb: Options.enable_pipelined_write: 0 2024-07-16T09:57:42.182 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 ceph-mon[93066]: rocksdb: Options.unordered_write: 0 2024-07-16T09:57:42.182 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 ceph-mon[93066]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-07-16T09:57:42.182 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 ceph-mon[93066]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-07-16T09:57:42.182 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 ceph-mon[93066]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-07-16T09:57:42.182 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 ceph-mon[93066]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-07-16T09:57:42.182 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 ceph-mon[93066]: rocksdb: Options.row_cache: None 2024-07-16T09:57:42.182 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 ceph-mon[93066]: rocksdb: Options.wal_filter: None 2024-07-16T09:57:42.182 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 ceph-mon[93066]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-07-16T09:57:42.182 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 ceph-mon[93066]: rocksdb: Options.allow_ingest_behind: 0 2024-07-16T09:57:42.182 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 ceph-mon[93066]: rocksdb: Options.two_write_queues: 0 2024-07-16T09:57:42.182 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 ceph-mon[93066]: rocksdb: Options.manual_wal_flush: 0 2024-07-16T09:57:42.182 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 ceph-mon[93066]: rocksdb: Options.wal_compression: 0 2024-07-16T09:57:42.183 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 ceph-mon[93066]: rocksdb: Options.atomic_flush: 0 2024-07-16T09:57:42.183 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 ceph-mon[93066]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-07-16T09:57:42.183 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 ceph-mon[93066]: rocksdb: Options.persist_stats_to_disk: 0 2024-07-16T09:57:42.183 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 ceph-mon[93066]: rocksdb: Options.write_dbid_to_manifest: 0 2024-07-16T09:57:42.183 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 ceph-mon[93066]: rocksdb: Options.log_readahead_size: 0 2024-07-16T09:57:42.183 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 ceph-mon[93066]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-07-16T09:57:42.183 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 ceph-mon[93066]: rocksdb: Options.best_efforts_recovery: 0 2024-07-16T09:57:42.183 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 ceph-mon[93066]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-07-16T09:57:42.183 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 ceph-mon[93066]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-07-16T09:57:42.183 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 ceph-mon[93066]: rocksdb: Options.allow_data_in_errors: 0 2024-07-16T09:57:42.183 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 ceph-mon[93066]: rocksdb: Options.db_host_id: __hostname__ 2024-07-16T09:57:42.183 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 ceph-mon[93066]: rocksdb: Options.enforce_single_del_contracts: true 2024-07-16T09:57:42.184 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 ceph-mon[93066]: rocksdb: Options.max_background_jobs: 2 2024-07-16T09:57:42.184 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 ceph-mon[93066]: rocksdb: Options.max_background_compactions: -1 2024-07-16T09:57:42.184 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 ceph-mon[93066]: rocksdb: Options.max_subcompactions: 1 2024-07-16T09:57:42.184 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 ceph-mon[93066]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-07-16T09:57:42.184 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 ceph-mon[93066]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-07-16T09:57:42.184 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 ceph-mon[93066]: rocksdb: Options.delayed_write_rate : 16777216 2024-07-16T09:57:42.184 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 ceph-mon[93066]: rocksdb: Options.max_total_wal_size: 0 2024-07-16T09:57:42.184 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 ceph-mon[93066]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-07-16T09:57:42.184 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 ceph-mon[93066]: rocksdb: Options.stats_dump_period_sec: 600 2024-07-16T09:57:42.184 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 ceph-mon[93066]: rocksdb: Options.stats_persist_period_sec: 600 2024-07-16T09:57:42.185 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 ceph-mon[93066]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-07-16T09:57:42.185 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 ceph-mon[93066]: rocksdb: Options.max_open_files: -1 2024-07-16T09:57:42.185 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 ceph-mon[93066]: rocksdb: Options.bytes_per_sync: 0 2024-07-16T09:57:42.185 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 ceph-mon[93066]: rocksdb: Options.wal_bytes_per_sync: 0 2024-07-16T09:57:42.185 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 ceph-mon[93066]: rocksdb: Options.strict_bytes_per_sync: 0 2024-07-16T09:57:42.185 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 ceph-mon[93066]: rocksdb: Options.compaction_readahead_size: 0 2024-07-16T09:57:42.185 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 ceph-mon[93066]: rocksdb: Options.max_background_flushes: -1 2024-07-16T09:57:42.185 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 ceph-mon[93066]: rocksdb: Compression algorithms supported: 2024-07-16T09:57:42.185 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 ceph-mon[93066]: rocksdb: kZSTD supported: 0 2024-07-16T09:57:42.185 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 ceph-mon[93066]: rocksdb: kXpressCompression supported: 0 2024-07-16T09:57:42.185 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 ceph-mon[93066]: rocksdb: kBZip2Compression supported: 0 2024-07-16T09:57:42.185 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 ceph-mon[93066]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-07-16T09:57:42.185 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 ceph-mon[93066]: rocksdb: kLZ4Compression supported: 1 2024-07-16T09:57:42.186 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 ceph-mon[93066]: rocksdb: kZlibCompression supported: 1 2024-07-16T09:57:42.186 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 ceph-mon[93066]: rocksdb: kLZ4HCCompression supported: 1 2024-07-16T09:57:42.186 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 ceph-mon[93066]: rocksdb: kSnappyCompression supported: 1 2024-07-16T09:57:42.186 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 ceph-mon[93066]: rocksdb: Fast CRC32 supported: Supported on x86 2024-07-16T09:57:42.186 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 ceph-mon[93066]: rocksdb: DMutex implementation: pthread_mutex_t 2024-07-16T09:57:42.186 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 ceph-mon[93066]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-smithi156/store.db/MANIFEST-000015 2024-07-16T09:57:42.186 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 ceph-mon[93066]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-07-16T09:57:42.186 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 ceph-mon[93066]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-07-16T09:57:42.186 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 ceph-mon[93066]: rocksdb: Options.merge_operator: 2024-07-16T09:57:42.186 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 ceph-mon[93066]: rocksdb: Options.compaction_filter: None 2024-07-16T09:57:42.186 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 ceph-mon[93066]: rocksdb: Options.compaction_filter_factory: None 2024-07-16T09:57:42.186 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 ceph-mon[93066]: rocksdb: Options.sst_partitioner_factory: None 2024-07-16T09:57:42.186 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 ceph-mon[93066]: rocksdb: Options.memtable_factory: SkipListFactory 2024-07-16T09:57:42.187 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 ceph-mon[93066]: rocksdb: Options.table_factory: BlockBasedTable 2024-07-16T09:57:42.187 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 ceph-mon[93066]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x558c877293a0) 2024-07-16T09:57:42.187 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: cache_index_and_filter_blocks: 1 2024-07-16T09:57:42.187 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-07-16T09:57:42.187 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-07-16T09:57:42.187 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: pin_top_level_index_and_filter: 1 2024-07-16T09:57:42.187 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: index_type: 0 2024-07-16T09:57:42.187 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: data_block_index_type: 0 2024-07-16T09:57:42.187 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: index_shortening: 1 2024-07-16T09:57:42.187 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: data_block_hash_table_util_ratio: 0.750000 2024-07-16T09:57:42.187 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: checksum: 4 2024-07-16T09:57:42.187 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: no_block_cache: 0 2024-07-16T09:57:42.187 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: block_cache: 0x558c8771d090 2024-07-16T09:57:42.188 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: block_cache_name: BinnedLRUCache 2024-07-16T09:57:42.188 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: block_cache_options: 2024-07-16T09:57:42.188 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: capacity : 536870912 2024-07-16T09:57:42.188 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: num_shard_bits : 4 2024-07-16T09:57:42.188 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: strict_capacity_limit : 0 2024-07-16T09:57:42.188 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: high_pri_pool_ratio: 0.000 2024-07-16T09:57:42.188 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: block_cache_compressed: (nil) 2024-07-16T09:57:42.188 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: persistent_cache: (nil) 2024-07-16T09:57:42.188 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: block_size: 4096 2024-07-16T09:57:42.189 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: block_size_deviation: 10 2024-07-16T09:57:42.189 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: block_restart_interval: 16 2024-07-16T09:57:42.189 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: index_block_restart_interval: 1 2024-07-16T09:57:42.189 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: metadata_block_size: 4096 2024-07-16T09:57:42.189 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: partition_filters: 0 2024-07-16T09:57:42.189 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: use_delta_encoding: 1 2024-07-16T09:57:42.190 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: filter_policy: bloomfilter 2024-07-16T09:57:42.190 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: whole_key_filtering: 1 2024-07-16T09:57:42.190 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: verify_compression: 0 2024-07-16T09:57:42.190 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: read_amp_bytes_per_bit: 0 2024-07-16T09:57:42.190 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: format_version: 5 2024-07-16T09:57:42.190 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: enable_index_compression: 1 2024-07-16T09:57:42.190 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: block_align: 0 2024-07-16T09:57:42.190 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: max_auto_readahead_size: 262144 2024-07-16T09:57:42.190 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: prepopulate_block_cache: 0 2024-07-16T09:57:42.190 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: initial_auto_readahead_size: 8192 2024-07-16T09:57:42.190 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: num_file_reads_for_auto_readahead: 2 2024-07-16T09:57:42.190 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 ceph-mon[93066]: rocksdb: Options.write_buffer_size: 33554432 2024-07-16T09:57:42.191 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 ceph-mon[93066]: rocksdb: Options.max_write_buffer_number: 2 2024-07-16T09:57:42.191 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 ceph-mon[93066]: rocksdb: Options.compression: NoCompression 2024-07-16T09:57:42.191 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 ceph-mon[93066]: rocksdb: Options.bottommost_compression: Disabled 2024-07-16T09:57:42.191 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 ceph-mon[93066]: rocksdb: Options.prefix_extractor: nullptr 2024-07-16T09:57:42.191 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 ceph-mon[93066]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-07-16T09:57:42.191 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 ceph-mon[93066]: rocksdb: Options.num_levels: 7 2024-07-16T09:57:42.191 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 ceph-mon[93066]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-07-16T09:57:42.191 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 ceph-mon[93066]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-07-16T09:57:42.191 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 ceph-mon[93066]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-07-16T09:57:42.191 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 ceph-mon[93066]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-07-16T09:57:42.191 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 ceph-mon[93066]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-07-16T09:57:42.191 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 ceph-mon[93066]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-07-16T09:57:42.191 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 ceph-mon[93066]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-07-16T09:57:42.191 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 ceph-mon[93066]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-07-16T09:57:42.192 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 ceph-mon[93066]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-07-16T09:57:42.192 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 ceph-mon[93066]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-07-16T09:57:42.192 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 ceph-mon[93066]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-07-16T09:57:42.192 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 ceph-mon[93066]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-07-16T09:57:42.192 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 ceph-mon[93066]: rocksdb: Options.compression_opts.window_bits: -14 2024-07-16T09:57:42.192 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 ceph-mon[93066]: rocksdb: Options.compression_opts.level: 32767 2024-07-16T09:57:42.192 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 ceph-mon[93066]: rocksdb: Options.compression_opts.strategy: 0 2024-07-16T09:57:42.192 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 ceph-mon[93066]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-07-16T09:57:42.192 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 ceph-mon[93066]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-07-16T09:57:42.192 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 ceph-mon[93066]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-07-16T09:57:42.192 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 ceph-mon[93066]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-07-16T09:57:42.192 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 ceph-mon[93066]: rocksdb: Options.compression_opts.enabled: false 2024-07-16T09:57:42.192 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 ceph-mon[93066]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-07-16T09:57:42.193 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 ceph-mon[93066]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-07-16T09:57:42.193 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 ceph-mon[93066]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-07-16T09:57:42.193 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 ceph-mon[93066]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-07-16T09:57:42.193 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 ceph-mon[93066]: rocksdb: Options.target_file_size_base: 67108864 2024-07-16T09:57:42.193 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 ceph-mon[93066]: rocksdb: Options.target_file_size_multiplier: 1 2024-07-16T09:57:42.193 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 ceph-mon[93066]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-07-16T09:57:42.193 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 ceph-mon[93066]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-07-16T09:57:42.193 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 ceph-mon[93066]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-07-16T09:57:42.193 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 ceph-mon[93066]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-07-16T09:57:42.193 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 ceph-mon[93066]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-07-16T09:57:42.193 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 ceph-mon[93066]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-07-16T09:57:42.193 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 ceph-mon[93066]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-07-16T09:57:42.193 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 ceph-mon[93066]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-07-16T09:57:42.193 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 ceph-mon[93066]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-07-16T09:57:42.194 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 ceph-mon[93066]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-07-16T09:57:42.194 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 ceph-mon[93066]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-07-16T09:57:42.194 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 ceph-mon[93066]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-07-16T09:57:42.194 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 ceph-mon[93066]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-07-16T09:57:42.194 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 ceph-mon[93066]: rocksdb: Options.arena_block_size: 1048576 2024-07-16T09:57:42.194 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 ceph-mon[93066]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-07-16T09:57:42.194 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 ceph-mon[93066]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-07-16T09:57:42.194 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 ceph-mon[93066]: rocksdb: Options.disable_auto_compactions: 0 2024-07-16T09:57:42.194 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 ceph-mon[93066]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-07-16T09:57:42.194 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 ceph-mon[93066]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-07-16T09:57:42.194 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 ceph-mon[93066]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-07-16T09:57:42.194 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 ceph-mon[93066]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-07-16T09:57:42.195 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 ceph-mon[93066]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-07-16T09:57:42.195 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 ceph-mon[93066]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-07-16T09:57:42.195 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 ceph-mon[93066]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-07-16T09:57:42.195 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 ceph-mon[93066]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-07-16T09:57:42.195 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 ceph-mon[93066]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-07-16T09:57:42.195 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 ceph-mon[93066]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-07-16T09:57:42.195 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 ceph-mon[93066]: rocksdb: Options.table_properties_collectors: 2024-07-16T09:57:42.195 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 ceph-mon[93066]: rocksdb: Options.inplace_update_support: 0 2024-07-16T09:57:42.195 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 ceph-mon[93066]: rocksdb: Options.inplace_update_num_locks: 10000 2024-07-16T09:57:42.195 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 ceph-mon[93066]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-07-16T09:57:42.196 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 ceph-mon[93066]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-07-16T09:57:42.196 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 ceph-mon[93066]: rocksdb: Options.memtable_huge_page_size: 0 2024-07-16T09:57:42.196 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 ceph-mon[93066]: rocksdb: Options.bloom_locality: 0 2024-07-16T09:57:42.196 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 ceph-mon[93066]: rocksdb: Options.max_successive_merges: 0 2024-07-16T09:57:42.196 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 ceph-mon[93066]: rocksdb: Options.optimize_filters_for_hits: 0 2024-07-16T09:57:42.196 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 ceph-mon[93066]: rocksdb: Options.paranoid_file_checks: 0 2024-07-16T09:57:42.196 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 ceph-mon[93066]: rocksdb: Options.force_consistency_checks: 1 2024-07-16T09:57:42.196 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 ceph-mon[93066]: rocksdb: Options.report_bg_io_stats: 0 2024-07-16T09:57:42.196 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 ceph-mon[93066]: rocksdb: Options.ttl: 2592000 2024-07-16T09:57:42.196 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 ceph-mon[93066]: rocksdb: Options.periodic_compaction_seconds: 0 2024-07-16T09:57:42.196 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 ceph-mon[93066]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-07-16T09:57:42.196 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 ceph-mon[93066]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-07-16T09:57:42.196 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 ceph-mon[93066]: rocksdb: Options.enable_blob_files: false 2024-07-16T09:57:42.197 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 ceph-mon[93066]: rocksdb: Options.min_blob_size: 0 2024-07-16T09:57:42.197 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 ceph-mon[93066]: rocksdb: Options.blob_file_size: 268435456 2024-07-16T09:57:42.197 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 ceph-mon[93066]: rocksdb: Options.blob_compression_type: NoCompression 2024-07-16T09:57:42.197 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 ceph-mon[93066]: rocksdb: Options.enable_blob_garbage_collection: false 2024-07-16T09:57:42.197 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 ceph-mon[93066]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-07-16T09:57:42.197 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 ceph-mon[93066]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-07-16T09:57:42.197 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 ceph-mon[93066]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-07-16T09:57:42.197 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 ceph-mon[93066]: rocksdb: Options.blob_file_starting_level: 0 2024-07-16T09:57:42.197 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 ceph-mon[93066]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-07-16T09:57:42.197 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 ceph-mon[93066]: rocksdb: [table/block_based/block_based_table_reader.cc:721] At least one SST file opened without unique ID to verify: 30.sst 2024-07-16T09:57:42.197 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 ceph-mon[93066]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-07-16T09:57:42.197 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 ceph-mon[93066]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-smithi156/store.db/MANIFEST-000015 succeeded,manifest_file_number is 15, next_file_number is 32, last_sequence is 10463, log_number is 28,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-07-16T09:57:42.198 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 ceph-mon[93066]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 28 2024-07-16T09:57:42.198 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 ceph-mon[93066]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 6257f666-ec9f-4718-960d-4b6be1c33f87 2024-07-16T09:57:42.198 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 ceph-mon[93066]: rocksdb: EVENT_LOG_v1 {"time_micros": 1721123861818030, "job": 1, "event": "recovery_started", "wal_files": [28]} 2024-07-16T09:57:42.198 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 ceph-mon[93066]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #28 mode 2 2024-07-16T09:57:42.198 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 ceph-mon[93066]: rocksdb: EVENT_LOG_v1 {"time_micros": 1721123861823632, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 33, "file_size": 1605691, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 10464, "largest_seqno": 11113, "table_properties": {"data_size": 1601729, "index_size": 2180, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 773, "raw_key_size": 7194, "raw_average_key_size": 23, "raw_value_size": 1594918, "raw_average_value_size": 5281, "num_data_blocks": 100, "num_entries": 302, "num_filter_entries": 302, "num_deletions": 2, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1721123861, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "6257f666-ec9f-4718-960d-4b6be1c33f87", "db_session_id": "9GULJKWWIEUEMILYGNVJ", "orig_file_number": 33, "seqno_to_time_mapping": "N/A"}} 2024-07-16T09:57:42.198 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 ceph-mon[93066]: rocksdb: EVENT_LOG_v1 {"time_micros": 1721123861823743, "job": 1, "event": "recovery_finished"} 2024-07-16T09:57:42.198 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 ceph-mon[93066]: rocksdb: [db/version_set.cc:5047] Creating manifest 35 2024-07-16T09:57:42.198 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 ceph-mon[93066]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-07-16T09:57:42.198 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 ceph-mon[93066]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-smithi156/store.db/000028.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-07-16T09:57:42.198 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 ceph-mon[93066]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x558c8774ae00 2024-07-16T09:57:42.199 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 ceph-mon[93066]: rocksdb: DB pointer 0x558c87844000 2024-07-16T09:57:42.199 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 ceph-mon[93066]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-07-16T09:57:42.199 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 ceph-mon[93066]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-07-16T09:57:42.199 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: ** DB Stats ** 2024-07-16T09:57:42.199 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-07-16T09:57:42.199 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-07-16T09:57:42.199 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-07-16T09:57:42.199 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-07-16T09:57:42.199 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-07-16T09:57:42.199 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-07-16T09:57:42.199 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-07-16T09:57:42.199 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: 2024-07-16T09:57:42.199 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: ** Compaction Stats [default] ** 2024-07-16T09:57:42.200 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-07-16T09:57:42.200 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-07-16T09:57:42.200 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: L0 1/0 1.53 MB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 497.0 0.00 0.00 1 0.003 0 0 0.0 0.0 2024-07-16T09:57:42.200 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: L6 1/0 5.56 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 2024-07-16T09:57:42.200 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: Sum 2/0 7.09 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 497.0 0.00 0.00 1 0.003 0 0 0.0 0.0 2024-07-16T09:57:42.200 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 497.0 0.00 0.00 1 0.003 0 0 0.0 0.0 2024-07-16T09:57:42.201 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: 2024-07-16T09:57:42.201 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: ** Compaction Stats [default] ** 2024-07-16T09:57:42.201 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-07-16T09:57:42.201 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-07-16T09:57:42.201 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 497.0 0.00 0.00 1 0.003 0 0 0.0 0.0 2024-07-16T09:57:42.201 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: 2024-07-16T09:57:42.202 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-07-16T09:57:42.202 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: 2024-07-16T09:57:42.202 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-07-16T09:57:42.202 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: Flush(GB): cumulative 0.001, interval 0.001 2024-07-16T09:57:42.202 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-07-16T09:57:42.202 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-07-16T09:57:42.203 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-07-16T09:57:42.203 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: AddFile(Keys): cumulative 0, interval 0 2024-07-16T09:57:42.203 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: Cumulative compaction: 0.00 GB write, 190.65 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-07-16T09:57:42.203 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: Interval compaction: 0.00 GB write, 190.65 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-07-16T09:57:42.203 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-07-16T09:57:42.203 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: Block cache BinnedLRUCache@0x558c8771d090#2 capacity: 512.00 MB usage: 3.23 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.1e-05 secs_since: 0 2024-07-16T09:57:42.204 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.91 KB,0.000172853%) IndexBlock(1,2.33 KB,0.000444055%) Misc(1,0.00 KB,0%) 2024-07-16T09:57:42.204 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: 2024-07-16T09:57:42.204 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: ** File Read Latency Histogram By Level [default] ** 2024-07-16T09:57:42.204 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 ceph-mon[93066]: starting mon.smithi156 rank 0 at public addrs [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] at bind addrs [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] mon_data /var/lib/ceph/mon/ceph-smithi156 fsid 26c91c26-4358-11ef-bcad-c7b262605968 2024-07-16T09:57:42.204 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 ceph-mon[93066]: mon.smithi156@-1(???) e2 preinit fsid 26c91c26-4358-11ef-bcad-c7b262605968 2024-07-16T09:57:42.205 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 ceph-mon[93066]: mon.smithi156@-1(???).mds e18 new map 2024-07-16T09:57:42.205 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 ceph-mon[93066]: mon.smithi156@-1(???).mds e18 print_map 2024-07-16T09:57:42.205 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: e18 2024-07-16T09:57:42.205 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-07-16T09:57:42.205 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-07-16T09:57:42.205 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: legacy client fscid: 1 2024-07-16T09:57:42.206 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: 2024-07-16T09:57:42.206 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: Filesystem 'cephfs' (1) 2024-07-16T09:57:42.206 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: fs_name cephfs 2024-07-16T09:57:42.206 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: epoch 18 2024-07-16T09:57:42.206 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-07-16T09:57:42.206 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: created 2024-07-16T09:52:55.214975+0000 2024-07-16T09:57:42.207 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: modified 2024-07-16T09:55:41.914320+0000 2024-07-16T09:57:42.207 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: tableserver 0 2024-07-16T09:57:42.207 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: root 0 2024-07-16T09:57:42.207 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: session_timeout 60 2024-07-16T09:57:42.207 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: session_autoclose 300 2024-07-16T09:57:42.208 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: max_file_size 1099511627776 2024-07-16T09:57:42.208 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: max_xattr_size 65536 2024-07-16T09:57:42.208 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: required_client_features {} 2024-07-16T09:57:42.208 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: last_failure 0 2024-07-16T09:57:42.208 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: last_failure_osd_epoch 0 2024-07-16T09:57:42.208 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-07-16T09:57:42.209 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: max_mds 2 2024-07-16T09:57:42.209 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: in 0,1 2024-07-16T09:57:42.209 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: up {0=14458,1=24261} 2024-07-16T09:57:42.209 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: failed 2024-07-16T09:57:42.209 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: damaged 2024-07-16T09:57:42.210 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: stopped 2024-07-16T09:57:42.210 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: data_pools [3] 2024-07-16T09:57:42.210 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: metadata_pool 2 2024-07-16T09:57:42.210 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: inline_data disabled 2024-07-16T09:57:42.210 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: balancer 2024-07-16T09:57:42.211 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: bal_rank_mask -1 2024-07-16T09:57:42.211 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: standby_count_wanted 1 2024-07-16T09:57:42.211 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: qdb_cluster leader: 0 members: 2024-07-16T09:57:42.211 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: [mds.cephfs.smithi156.rdibys{0:14458} state up:active seq 4 export targets 1 join_fscid=1 addr [v2:172.21.15.156:6826/1080176013,v1:172.21.15.156:6827/1080176013] compat {c=[1],r=[1],i=[7ff]}] 2024-07-16T09:57:42.211 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: [mds.cephfs.smithi156.evwwaz{0:14468} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.156:6828/560329502,v1:172.21.15.156:6829/560329502] compat {c=[1],r=[1],i=[7ff]}] 2024-07-16T09:57:42.211 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: [mds.cephfs.smithi176.dvvbud{1:24261} state up:active seq 4 export targets 0 join_fscid=1 addr [v2:172.21.15.176:6824/1577019287,v1:172.21.15.176:6825/1577019287] compat {c=[1],r=[1],i=[7ff]}] 2024-07-16T09:57:42.212 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: [mds.cephfs.smithi176.wahbsi{1:24273} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.176:6826/2196775412,v1:172.21.15.176:6827/2196775412] compat {c=[1],r=[1],i=[7ff]}] 2024-07-16T09:57:42.212 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: 2024-07-16T09:57:42.212 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: 2024-07-16T09:57:42.212 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 ceph-mon[93066]: mon.smithi156@-1(???).osd e48 crush map has features 3314933000852226048, adjusting msgr requires 2024-07-16T09:57:42.212 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 ceph-mon[93066]: mon.smithi156@-1(???).osd e48 crush map has features 288514051259236352, adjusting msgr requires 2024-07-16T09:57:42.212 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 ceph-mon[93066]: mon.smithi156@-1(???).osd e48 crush map has features 288514051259236352, adjusting msgr requires 2024-07-16T09:57:42.213 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 ceph-mon[93066]: mon.smithi156@-1(???).osd e48 crush map has features 288514051259236352, adjusting msgr requires 2024-07-16T09:57:42.213 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 ceph-mon[93066]: mon.smithi156@-1(???).paxosservice(auth 1..20) refresh upgraded, format 0 -> 3 2024-07-16T09:57:42.213 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 bash[93048]: 77b6ec8c50eb5165a042b8c22a28eb8ecd167c807708bd75457f3a70b576c408 2024-07-16T09:57:42.213 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:41 smithi156 systemd[1]: Started Ceph mon.smithi156 for 26c91c26-4358-11ef-bcad-c7b262605968. 2024-07-16T09:57:43.566 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:43 smithi176 ceph-mon[36626]: from='mgr.14722 172.21.15.156:0/647865130' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "mon metadata", "id": "smithi156"}]: dispatch 2024-07-16T09:57:43.566 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:43 smithi176 ceph-mon[36626]: from='mgr.14722 172.21.15.156:0/647865130' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "mon metadata", "id": "smithi176"}]: dispatch 2024-07-16T09:57:43.567 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:43 smithi176 ceph-mon[36626]: pgmap v19: 65 pgs: 65 active+clean; 253 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.4 KiB/s rd, 4 op/s 2024-07-16T09:57:43.567 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:43 smithi176 ceph-mon[36626]: pgmap v20: 65 pgs: 65 active+clean; 253 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.6 KiB/s rd, 3 op/s 2024-07-16T09:57:43.567 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:43 smithi176 ceph-mon[36626]: mon.smithi156 calling monitor election 2024-07-16T09:57:43.567 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:43 smithi176 ceph-mon[36626]: mon.smithi156 is new leader, mons smithi156,smithi176 in quorum (ranks 0,1) 2024-07-16T09:57:43.567 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:43 smithi176 ceph-mon[36626]: monmap epoch 2 2024-07-16T09:57:43.567 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:43 smithi176 ceph-mon[36626]: fsid 26c91c26-4358-11ef-bcad-c7b262605968 2024-07-16T09:57:43.568 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:43 smithi176 ceph-mon[36626]: last_changed 2024-07-16T09:48:46.842263+0000 2024-07-16T09:57:43.568 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:43 smithi176 ceph-mon[36626]: created 2024-07-16T09:46:20.852914+0000 2024-07-16T09:57:43.568 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:43 smithi176 ceph-mon[36626]: min_mon_release 17 (quincy) 2024-07-16T09:57:43.568 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:43 smithi176 ceph-mon[36626]: election_strategy: 1 2024-07-16T09:57:43.568 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:43 smithi176 ceph-mon[36626]: 0: [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] mon.smithi156 2024-07-16T09:57:43.568 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:43 smithi176 ceph-mon[36626]: 1: [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] mon.smithi176 2024-07-16T09:57:43.568 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:43 smithi176 ceph-mon[36626]: fsmap cephfs:2 {0=cephfs.smithi156.rdibys=up:active,1=cephfs.smithi176.dvvbud=up:active} 2 up:standby-replay 2024-07-16T09:57:43.568 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:43 smithi176 ceph-mon[36626]: osdmap e48: 6 total, 6 up, 6 in 2024-07-16T09:57:43.568 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:43 smithi176 ceph-mon[36626]: mgrmap e31: smithi156.zkvylv(active, since 35s), standbys: smithi176.dqbrvx 2024-07-16T09:57:43.568 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:43 smithi176 ceph-mon[36626]: overall HEALTH_OK 2024-07-16T09:57:43.568 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:43 smithi176 ceph-mon[36626]: from='mgr.14722 ' entity='' 2024-07-16T09:57:43.569 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:43 smithi176 ceph-mon[36626]: mgrmap e32: smithi156.zkvylv(active, since 35s), standbys: smithi176.dqbrvx 2024-07-16T09:57:43.574 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:43 smithi156 ceph-mon[93066]: from='mgr.14722 172.21.15.156:0/647865130' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "mon metadata", "id": "smithi156"}]: dispatch 2024-07-16T09:57:43.575 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:43 smithi156 ceph-mon[93066]: from='mgr.14722 172.21.15.156:0/647865130' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "mon metadata", "id": "smithi176"}]: dispatch 2024-07-16T09:57:43.575 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:43 smithi156 ceph-mon[93066]: pgmap v19: 65 pgs: 65 active+clean; 253 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.4 KiB/s rd, 4 op/s 2024-07-16T09:57:43.575 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:43 smithi156 ceph-mon[93066]: pgmap v20: 65 pgs: 65 active+clean; 253 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.6 KiB/s rd, 3 op/s 2024-07-16T09:57:43.575 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:43 smithi156 ceph-mon[93066]: mon.smithi156 calling monitor election 2024-07-16T09:57:43.575 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:43 smithi156 ceph-mon[93066]: mon.smithi156 is new leader, mons smithi156,smithi176 in quorum (ranks 0,1) 2024-07-16T09:57:43.575 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:43 smithi156 ceph-mon[93066]: monmap epoch 2 2024-07-16T09:57:43.575 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:43 smithi156 ceph-mon[93066]: fsid 26c91c26-4358-11ef-bcad-c7b262605968 2024-07-16T09:57:43.575 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:43 smithi156 ceph-mon[93066]: last_changed 2024-07-16T09:48:46.842263+0000 2024-07-16T09:57:43.575 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:43 smithi156 ceph-mon[93066]: created 2024-07-16T09:46:20.852914+0000 2024-07-16T09:57:43.575 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:43 smithi156 ceph-mon[93066]: min_mon_release 17 (quincy) 2024-07-16T09:57:43.575 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:43 smithi156 ceph-mon[93066]: election_strategy: 1 2024-07-16T09:57:43.575 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:43 smithi156 ceph-mon[93066]: 0: [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] mon.smithi156 2024-07-16T09:57:43.576 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:43 smithi156 ceph-mon[93066]: 1: [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] mon.smithi176 2024-07-16T09:57:43.576 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:43 smithi156 ceph-mon[93066]: fsmap cephfs:2 {0=cephfs.smithi156.rdibys=up:active,1=cephfs.smithi176.dvvbud=up:active} 2 up:standby-replay 2024-07-16T09:57:43.576 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:43 smithi156 ceph-mon[93066]: osdmap e48: 6 total, 6 up, 6 in 2024-07-16T09:57:43.576 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:43 smithi156 ceph-mon[93066]: mgrmap e31: smithi156.zkvylv(active, since 35s), standbys: smithi176.dqbrvx 2024-07-16T09:57:43.576 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:43 smithi156 ceph-mon[93066]: overall HEALTH_OK 2024-07-16T09:57:43.576 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:43 smithi156 ceph-mon[93066]: from='mgr.14722 ' entity='' 2024-07-16T09:57:43.576 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:43 smithi156 ceph-mon[93066]: mgrmap e32: smithi156.zkvylv(active, since 35s), standbys: smithi176.dqbrvx 2024-07-16T09:57:44.931 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:44.927+0000 7fb253496640 1 -- 172.21.15.156:0/1365058902 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb24c0ff490 msgr2=0x7fb24c101870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:57:44.931 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:44.927+0000 7fb253496640 1 --2- 172.21.15.156:0/1365058902 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb24c0ff490 0x7fb24c101870 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fb23c0048d0 tx=0x7fb23c009da0 comp rx=0 tx=0).stop 2024-07-16T09:57:44.931 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:44.928+0000 7fb253496640 1 -- 172.21.15.156:0/1365058902 shutdown_connections 2024-07-16T09:57:44.931 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:44.928+0000 7fb253496640 1 --2- 172.21.15.156:0/1365058902 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb24c0ff490 0x7fb24c101870 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:57:44.932 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:44.928+0000 7fb253496640 1 --2- 172.21.15.156:0/1365058902 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb24c065a50 0x7fb24c0fef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:57:44.932 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:44.928+0000 7fb253496640 1 -- 172.21.15.156:0/1365058902 >> 172.21.15.156:0/1365058902 conn(0x7fb24c0f6a20 msgr2=0x7fb24c0f8e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:57:44.932 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:44.928+0000 7fb253496640 1 -- 172.21.15.156:0/1365058902 shutdown_connections 2024-07-16T09:57:44.932 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:44.928+0000 7fb253496640 1 -- 172.21.15.156:0/1365058902 wait complete. 2024-07-16T09:57:44.932 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:44.928+0000 7fb253496640 1 Processor -- start 2024-07-16T09:57:44.932 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:44.928+0000 7fb253496640 1 -- start start 2024-07-16T09:57:44.932 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:44.929+0000 7fb253496640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb24c065a50 0x7fb24c0fc910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:57:44.932 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:44.929+0000 7fb253496640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb24c0ff490 0x7fb24c0fae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:57:44.932 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:44.929+0000 7fb253496640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb24c0fb390 con 0x7fb24c065a50 2024-07-16T09:57:44.932 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:44.929+0000 7fb253496640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb24c0fb4d0 con 0x7fb24c0ff490 2024-07-16T09:57:44.932 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:44.929+0000 7fb250a31640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb24c0ff490 0x7fb24c0fae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:57:44.932 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:44.929+0000 7fb250a31640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb24c0ff490 0x7fb24c0fae50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.156:49580/0 (socket says 172.21.15.156:49580) 2024-07-16T09:57:44.932 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:44.929+0000 7fb251232640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb24c065a50 0x7fb24c0fc910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:57:44.932 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:44.929+0000 7fb250a31640 1 -- 172.21.15.156:0/1895012115 learned_addr learned my addr 172.21.15.156:0/1895012115 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T09:57:44.933 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:44.929+0000 7fb250a31640 1 -- 172.21.15.156:0/1895012115 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb24c065a50 msgr2=0x7fb24c0fc910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:57:44.933 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:44.929+0000 7fb250a31640 1 --2- 172.21.15.156:0/1895012115 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb24c065a50 0x7fb24c0fc910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:57:44.933 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:44.929+0000 7fb250a31640 1 -- 172.21.15.156:0/1895012115 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb23c005040 con 0x7fb24c0ff490 2024-07-16T09:57:44.933 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:44.929+0000 7fb251232640 1 --2- 172.21.15.156:0/1895012115 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb24c065a50 0x7fb24c0fc910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-16T09:57:44.933 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:44.929+0000 7fb250a31640 1 --2- 172.21.15.156:0/1895012115 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb24c0ff490 0x7fb24c0fae50 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fb23c0048a0 tx=0x7fb23c009da0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:57:44.933 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:44.930+0000 7fb241ffb640 1 -- 172.21.15.156:0/1895012115 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb23c038070 con 0x7fb24c0ff490 2024-07-16T09:57:44.933 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:44.930+0000 7fb253496640 1 -- 172.21.15.156:0/1895012115 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb24c0fb750 con 0x7fb24c0ff490 2024-07-16T09:57:44.933 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:44.930+0000 7fb241ffb640 1 -- 172.21.15.156:0/1895012115 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb23c0092b0 con 0x7fb24c0ff490 2024-07-16T09:57:44.933 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:44.930+0000 7fb241ffb640 1 -- 172.21.15.156:0/1895012115 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb23c0338a0 con 0x7fb24c0ff490 2024-07-16T09:57:44.933 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:44.930+0000 7fb253496640 1 -- 172.21.15.156:0/1895012115 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb24c0fbc90 con 0x7fb24c0ff490 2024-07-16T09:57:44.934 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:44.931+0000 7fb253496640 1 -- 172.21.15.156:0/1895012115 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb24c0fa6a0 con 0x7fb24c0ff490 2024-07-16T09:57:44.934 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:44.932+0000 7fb241ffb640 1 -- 172.21.15.156:0/1895012115 <== mon.1 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 32) v1 ==== 92307+0+0 (secure 0 0 0) 0x7fb23c030020 con 0x7fb24c0ff490 2024-07-16T09:57:44.934 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:44.932+0000 7fb241ffb640 1 --2- 172.21.15.156:0/1895012115 >> [v2:172.21.15.156:6800/55311371,v1:172.21.15.156:6801/55311371] conn(0x7fb23806e1e0 0x7fb238070690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:57:44.934 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:44.932+0000 7fb251232640 1 -- 172.21.15.156:0/1895012115 >> [v2:172.21.15.156:6800/55311371,v1:172.21.15.156:6801/55311371] conn(0x7fb23806e1e0 msgr2=0x7fb238070690 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.156:6800/55311371 2024-07-16T09:57:44.934 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:44.933+0000 7fb241ffb640 1 -- 172.21.15.156:0/1895012115 <== mon.1 v2:172.21.15.176:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7fb23c0ad8e0 con 0x7fb24c0ff490 2024-07-16T09:57:44.934 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:44.933+0000 7fb251232640 1 --2- 172.21.15.156:0/1895012115 >> [v2:172.21.15.156:6800/55311371,v1:172.21.15.156:6801/55311371] conn(0x7fb23806e1e0 0x7fb238070690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-16T09:57:44.938 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:44.937+0000 7fb241ffb640 1 -- 172.21.15.156:0/1895012115 <== mon.1 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187716 (secure 0 0 0) 0x7fb23c076dd0 con 0x7fb24c0ff490 2024-07-16T09:57:45.135 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:45.134+0000 7fb251232640 1 -- 172.21.15.156:0/1895012115 >> [v2:172.21.15.156:6800/55311371,v1:172.21.15.156:6801/55311371] conn(0x7fb23806e1e0 msgr2=0x7fb238070690 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.156:6800/55311371 2024-07-16T09:57:45.135 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:45.134+0000 7fb251232640 1 --2- 172.21.15.156:0/1895012115 >> [v2:172.21.15.156:6800/55311371,v1:172.21.15.156:6801/55311371] conn(0x7fb23806e1e0 0x7fb238070690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-16T09:57:45.143 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:45.142+0000 7fb253496640 1 -- 172.21.15.156:0/1895012115 --> [v2:172.21.15.156:6800/55311371,v1:172.21.15.156:6801/55311371] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb24c0f8510 con 0x7fb23806e1e0 2024-07-16T09:57:45.536 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:45.535+0000 7fb251232640 1 -- 172.21.15.156:0/1895012115 >> [v2:172.21.15.156:6800/55311371,v1:172.21.15.156:6801/55311371] conn(0x7fb23806e1e0 msgr2=0x7fb238070690 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.156:6800/55311371 2024-07-16T09:57:45.536 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:45.535+0000 7fb251232640 1 --2- 172.21.15.156:0/1895012115 >> [v2:172.21.15.156:6800/55311371,v1:172.21.15.156:6801/55311371] conn(0x7fb23806e1e0 0x7fb238070690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-07-16T09:57:46.337 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:46.336+0000 7fb251232640 1 -- 172.21.15.156:0/1895012115 >> [v2:172.21.15.156:6800/55311371,v1:172.21.15.156:6801/55311371] conn(0x7fb23806e1e0 msgr2=0x7fb238070690 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.156:6800/55311371 2024-07-16T09:57:46.337 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:46.336+0000 7fb251232640 1 --2- 172.21.15.156:0/1895012115 >> [v2:172.21.15.156:6800/55311371,v1:172.21.15.156:6801/55311371] conn(0x7fb23806e1e0 0x7fb238070690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-07-16T09:57:47.939 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:47.937+0000 7fb251232640 1 -- 172.21.15.156:0/1895012115 >> [v2:172.21.15.156:6800/55311371,v1:172.21.15.156:6801/55311371] conn(0x7fb23806e1e0 msgr2=0x7fb238070690 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.156:6800/55311371 2024-07-16T09:57:47.939 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:47.937+0000 7fb251232640 1 --2- 172.21.15.156:0/1895012115 >> [v2:172.21.15.156:6800/55311371,v1:172.21.15.156:6801/55311371] conn(0x7fb23806e1e0 0x7fb238070690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-07-16T09:57:48.244 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:48.242+0000 7fb241ffb640 1 -- 172.21.15.156:0/1895012115 <== mon.1 v2:172.21.15.176:3300/0 7 ==== mgrmap(e 33) v1 ==== 92307+0+0 (secure 0 0 0) 0x7fb23c0afd50 con 0x7fb24c0ff490 2024-07-16T09:57:48.291 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:48 smithi176 ceph-mon[36626]: Standby manager daemon smithi176.dqbrvx restarted 2024-07-16T09:57:48.291 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:48 smithi176 ceph-mon[36626]: Standby manager daemon smithi176.dqbrvx started 2024-07-16T09:57:48.291 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:48 smithi176 ceph-mon[36626]: from='mgr.? 172.21.15.176:0/2455540524' entity='mgr.smithi176.dqbrvx' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi176.dqbrvx/crt"}]: dispatch 2024-07-16T09:57:48.291 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:48 smithi176 ceph-mon[36626]: from='mgr.? 172.21.15.176:0/2455540524' entity='mgr.smithi176.dqbrvx' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-07-16T09:57:48.291 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:48 smithi176 ceph-mon[36626]: from='mgr.? 172.21.15.176:0/2455540524' entity='mgr.smithi176.dqbrvx' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi176.dqbrvx/key"}]: dispatch 2024-07-16T09:57:48.291 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:48 smithi176 ceph-mon[36626]: from='mgr.? 172.21.15.176:0/2455540524' entity='mgr.smithi176.dqbrvx' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-07-16T09:57:48.311 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:48.310+0000 7fb241ffb640 1 -- 172.21.15.156:0/1895012115 <== mon.1 v2:172.21.15.176:3300/0 8 ==== mgrmap(e 34) v1 ==== 91828+0+0 (secure 0 0 0) 0x7fb23c032c00 con 0x7fb24c0ff490 2024-07-16T09:57:48.311 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:48.310+0000 7fb241ffb640 1 -- 172.21.15.156:0/1895012115 >> [v2:172.21.15.156:6800/55311371,v1:172.21.15.156:6801/55311371] conn(0x7fb23806e1e0 msgr2=0x7fb238070690 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-16T09:57:48.312 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:48.310+0000 7fb241ffb640 1 --2- 172.21.15.156:0/1895012115 >> [v2:172.21.15.156:6800/55311371,v1:172.21.15.156:6801/55311371] conn(0x7fb23806e1e0 0x7fb238070690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:57:48.519 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:48 smithi156 ceph-mon[93066]: Standby manager daemon smithi176.dqbrvx restarted 2024-07-16T09:57:48.519 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:48 smithi156 ceph-mon[93066]: Standby manager daemon smithi176.dqbrvx started 2024-07-16T09:57:48.519 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:48 smithi156 ceph-mon[93066]: from='mgr.? 172.21.15.176:0/2455540524' entity='mgr.smithi176.dqbrvx' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi176.dqbrvx/crt"}]: dispatch 2024-07-16T09:57:48.519 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:48 smithi156 ceph-mon[93066]: from='mgr.? 172.21.15.176:0/2455540524' entity='mgr.smithi176.dqbrvx' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-07-16T09:57:48.519 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:48 smithi156 ceph-mon[93066]: from='mgr.? 172.21.15.176:0/2455540524' entity='mgr.smithi176.dqbrvx' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi176.dqbrvx/key"}]: dispatch 2024-07-16T09:57:48.519 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:48 smithi156 ceph-mon[93066]: from='mgr.? 172.21.15.176:0/2455540524' entity='mgr.smithi176.dqbrvx' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-07-16T09:57:49.315 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:49.313+0000 7fb241ffb640 1 -- 172.21.15.156:0/1895012115 <== mon.1 v2:172.21.15.176:3300/0 9 ==== mgrmap(e 35) v1 ==== 91955+0+0 (secure 0 0 0) 0x7fb23c0abc80 con 0x7fb24c0ff490 2024-07-16T09:57:49.316 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:49.314+0000 7fb241ffb640 1 --2- 172.21.15.156:0/1895012115 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7fb238028c90 0x7fb238029060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:57:49.316 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:49.314+0000 7fb241ffb640 1 -- 172.21.15.156:0/1895012115 --> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb24c0f8510 con 0x7fb238028c90 2024-07-16T09:57:49.316 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:49.314+0000 7fb251232640 1 --2- 172.21.15.156:0/1895012115 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7fb238028c90 0x7fb238029060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:57:49.316 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:49.315+0000 7fb251232640 1 --2- 172.21.15.156:0/1895012115 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7fb238028c90 0x7fb238029060 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fb24c0fc6c0 tx=0x7fb248009f50 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:57:49.318 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:49.317+0000 7fb241ffb640 1 -- 172.21.15.156:0/1895012115 <== mgr.34104 v2:172.21.15.156:6800/3367765292 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+337 (secure 0 0 0) 0x7fb24c0f8510 con 0x7fb238028c90 2024-07-16T09:57:49.321 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:49.320+0000 7fb253496640 1 -- 172.21.15.156:0/1895012115 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7fb238028c90 msgr2=0x7fb238029060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:57:49.322 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:49.320+0000 7fb253496640 1 --2- 172.21.15.156:0/1895012115 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7fb238028c90 0x7fb238029060 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fb24c0fc6c0 tx=0x7fb248009f50 comp rx=0 tx=0).stop 2024-07-16T09:57:49.322 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:49.320+0000 7fb253496640 1 -- 172.21.15.156:0/1895012115 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb24c0ff490 msgr2=0x7fb24c0fae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:57:49.322 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:49.320+0000 7fb253496640 1 --2- 172.21.15.156:0/1895012115 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb24c0ff490 0x7fb24c0fae50 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fb23c0048a0 tx=0x7fb23c009da0 comp rx=0 tx=0).stop 2024-07-16T09:57:49.322 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:49.320+0000 7fb253496640 1 -- 172.21.15.156:0/1895012115 shutdown_connections 2024-07-16T09:57:49.322 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:49.320+0000 7fb253496640 1 --2- 172.21.15.156:0/1895012115 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7fb238028c90 0x7fb238029060 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:57:49.322 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:49.320+0000 7fb253496640 1 --2- 172.21.15.156:0/1895012115 >> [v2:172.21.15.156:6800/55311371,v1:172.21.15.156:6801/55311371] conn(0x7fb23806e1e0 0x7fb238070690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:57:49.322 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:49.320+0000 7fb253496640 1 --2- 172.21.15.156:0/1895012115 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb24c0ff490 0x7fb24c0fae50 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:57:49.322 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:49.320+0000 7fb253496640 1 --2- 172.21.15.156:0/1895012115 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb24c065a50 0x7fb24c0fc910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:57:49.322 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:49.320+0000 7fb253496640 1 -- 172.21.15.156:0/1895012115 >> 172.21.15.156:0/1895012115 conn(0x7fb24c0f6a20 msgr2=0x7fb24c0f6df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:57:49.322 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:49.320+0000 7fb253496640 1 -- 172.21.15.156:0/1895012115 shutdown_connections 2024-07-16T09:57:49.322 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:49.320+0000 7fb253496640 1 -- 172.21.15.156:0/1895012115 wait complete. 2024-07-16T09:57:49.330 INFO:teuthology.orchestra.run.smithi156.stdout:true 2024-07-16T09:57:49.408 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:49.407+0000 7f997269e640 1 -- 172.21.15.156:0/16568782 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f996c0ff4f0 msgr2=0x7f996c1018d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:57:49.409 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:49.407+0000 7f997269e640 1 --2- 172.21.15.156:0/16568782 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f996c0ff4f0 0x7f996c1018d0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f99600048d0 tx=0x7f9960009da0 comp rx=0 tx=0).stop 2024-07-16T09:57:49.409 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:49.407+0000 7f997269e640 1 -- 172.21.15.156:0/16568782 shutdown_connections 2024-07-16T09:57:49.409 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:49.407+0000 7f997269e640 1 --2- 172.21.15.156:0/16568782 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f996c0ff4f0 0x7f996c1018d0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:57:49.409 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:49.407+0000 7f997269e640 1 --2- 172.21.15.156:0/16568782 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f996c0659f0 0x7f996c0fefb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:57:49.409 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:49.407+0000 7f997269e640 1 -- 172.21.15.156:0/16568782 >> 172.21.15.156:0/16568782 conn(0x7f996c0f6a20 msgr2=0x7f996c0f8e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:57:49.409 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:49.407+0000 7f997269e640 1 -- 172.21.15.156:0/16568782 shutdown_connections 2024-07-16T09:57:49.409 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:49.407+0000 7f997269e640 1 -- 172.21.15.156:0/16568782 wait complete. 2024-07-16T09:57:49.409 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:49.407+0000 7f997269e640 1 Processor -- start 2024-07-16T09:57:49.411 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:49.408+0000 7f997269e640 1 -- start start 2024-07-16T09:57:49.411 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:49.408+0000 7f997269e640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f996c0659f0 0x7f996c10cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:57:49.411 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:49.408+0000 7f997269e640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f996c0ff4f0 0x7f996c10d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:57:49.411 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:49.408+0000 7f997269e640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f996c10db20 con 0x7f996c0ff4f0 2024-07-16T09:57:49.411 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:49.408+0000 7f997269e640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f996c10e2f0 con 0x7f996c0659f0 2024-07-16T09:57:49.411 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:49.408+0000 7f996bfff640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f996c0659f0 0x7f996c10cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:57:49.411 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:49.408+0000 7f996b7fe640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f996c0ff4f0 0x7f996c10d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:57:49.411 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:49.408+0000 7f996bfff640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f996c0659f0 0x7f996c10cf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.156:49680/0 (socket says 172.21.15.156:49680) 2024-07-16T09:57:49.411 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:49.408+0000 7f996bfff640 1 -- 172.21.15.156:0/2326718343 learned_addr learned my addr 172.21.15.156:0/2326718343 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T09:57:49.411 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:49.408+0000 7f996bfff640 1 -- 172.21.15.156:0/2326718343 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f996c0ff4f0 msgr2=0x7f996c10d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:57:49.412 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:49.408+0000 7f996bfff640 1 --2- 172.21.15.156:0/2326718343 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f996c0ff4f0 0x7f996c10d4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:57:49.412 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:49.408+0000 7f996bfff640 1 -- 172.21.15.156:0/2326718343 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9960005040 con 0x7f996c0659f0 2024-07-16T09:57:49.412 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:49.408+0000 7f996b7fe640 1 --2- 172.21.15.156:0/2326718343 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f996c0ff4f0 0x7f996c10d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-16T09:57:49.412 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:49.409+0000 7f996bfff640 1 --2- 172.21.15.156:0/2326718343 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f996c0659f0 0x7f996c10cf90 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f995c00eb70 tx=0x7f995c00b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:57:49.412 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:49 smithi156 ceph-mon[93066]: mgrmap e33: smithi156.zkvylv(active, since 40s), standbys: smithi176.dqbrvx 2024-07-16T09:57:49.412 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:49 smithi156 ceph-mon[93066]: Active manager daemon smithi156.zkvylv restarted 2024-07-16T09:57:49.412 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:49 smithi156 ceph-mon[93066]: Activating manager daemon smithi156.zkvylv 2024-07-16T09:57:49.412 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:49 smithi156 ceph-mon[93066]: osdmap e49: 6 total, 6 up, 6 in 2024-07-16T09:57:49.412 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:49 smithi156 ceph-mon[93066]: mgrmap e34: smithi156.zkvylv(active, starting, since 0.00545627s), standbys: smithi176.dqbrvx 2024-07-16T09:57:49.413 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:49 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "mon metadata", "id": "smithi156"}]: dispatch 2024-07-16T09:57:49.413 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:49 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "mon metadata", "id": "smithi176"}]: dispatch 2024-07-16T09:57:49.413 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:49 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi156.rdibys"}]: dispatch 2024-07-16T09:57:49.413 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:49 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi156.evwwaz"}]: dispatch 2024-07-16T09:57:49.413 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:49 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi176.dvvbud"}]: dispatch 2024-07-16T09:57:49.413 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:49 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi176.wahbsi"}]: dispatch 2024-07-16T09:57:49.413 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:49 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "mgr metadata", "who": "smithi156.zkvylv", "id": "smithi156.zkvylv"}]: dispatch 2024-07-16T09:57:49.413 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:49 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "mgr metadata", "who": "smithi176.dqbrvx", "id": "smithi176.dqbrvx"}]: dispatch 2024-07-16T09:57:49.413 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:49 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-07-16T09:57:49.413 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:49 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-07-16T09:57:49.413 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:49 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-07-16T09:57:49.413 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:49 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-07-16T09:57:49.414 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:49 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-07-16T09:57:49.414 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:49 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-07-16T09:57:49.414 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:49 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-07-16T09:57:49.414 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:49 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-07-16T09:57:49.414 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:49 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-07-16T09:57:49.414 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:49 smithi156 ceph-mon[93066]: Manager daemon smithi156.zkvylv is now available 2024-07-16T09:57:49.414 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:49 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-16T09:57:49.414 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:49 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-16T09:57:49.414 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:49.409+0000 7f9968ff9640 1 -- 172.21.15.156:0/2326718343 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f995c007c30 con 0x7f996c0659f0 2024-07-16T09:57:49.415 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:49 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi156.zkvylv/mirror_snapshot_schedule"}]: dispatch 2024-07-16T09:57:49.415 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:49 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi156.zkvylv/trash_purge_schedule"}]: dispatch 2024-07-16T09:57:49.416 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:49.409+0000 7f997269e640 1 -- 172.21.15.156:0/2326718343 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f996c10e580 con 0x7f996c0659f0 2024-07-16T09:57:49.416 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:49.409+0000 7f9968ff9640 1 -- 172.21.15.156:0/2326718343 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f995c00f040 con 0x7f996c0659f0 2024-07-16T09:57:49.416 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:49.410+0000 7f9968ff9640 1 -- 172.21.15.156:0/2326718343 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f995c00cc10 con 0x7f996c0659f0 2024-07-16T09:57:49.416 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:49.410+0000 7f997269e640 1 -- 172.21.15.156:0/2326718343 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f996c10eb20 con 0x7f996c0659f0 2024-07-16T09:57:49.416 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:49.411+0000 7f9968ff9640 1 -- 172.21.15.156:0/2326718343 <== mon.1 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 35) v1 ==== 91955+0+0 (secure 0 0 0) 0x7f995c009110 con 0x7f996c0659f0 2024-07-16T09:57:49.416 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:49.411+0000 7f9968ff9640 1 --2- 172.21.15.156:0/2326718343 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f995406ddc0 0x7f9954070270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:57:49.416 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:49.411+0000 7f997269e640 1 -- 172.21.15.156:0/2326718343 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f996c0fa6a0 con 0x7f996c0659f0 2024-07-16T09:57:49.416 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:49.411+0000 7f9968ff9640 1 -- 172.21.15.156:0/2326718343 <== mon.1 v2:172.21.15.176:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f995c08d0e0 con 0x7f996c0659f0 2024-07-16T09:57:49.416 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:49.411+0000 7f996b7fe640 1 --2- 172.21.15.156:0/2326718343 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f995406ddc0 0x7f9954070270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:57:49.416 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:49.412+0000 7f996b7fe640 1 --2- 172.21.15.156:0/2326718343 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f995406ddc0 0x7f9954070270 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f996c10cd40 tx=0x7f9960035040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:57:49.417 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:49.416+0000 7f9968ff9640 1 -- 172.21.15.156:0/2326718343 <== mon.1 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187716 (secure 0 0 0) 0x7f995c05a320 con 0x7f996c0659f0 2024-07-16T09:57:49.565 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:49 smithi176 ceph-mon[36626]: mgrmap e33: smithi156.zkvylv(active, since 40s), standbys: smithi176.dqbrvx 2024-07-16T09:57:49.565 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:49 smithi176 ceph-mon[36626]: Active manager daemon smithi156.zkvylv restarted 2024-07-16T09:57:49.565 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:49 smithi176 ceph-mon[36626]: Activating manager daemon smithi156.zkvylv 2024-07-16T09:57:49.565 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:49 smithi176 ceph-mon[36626]: osdmap e49: 6 total, 6 up, 6 in 2024-07-16T09:57:49.565 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:49 smithi176 ceph-mon[36626]: mgrmap e34: smithi156.zkvylv(active, starting, since 0.00545627s), standbys: smithi176.dqbrvx 2024-07-16T09:57:49.566 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:49 smithi176 ceph-mon[36626]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "mon metadata", "id": "smithi156"}]: dispatch 2024-07-16T09:57:49.566 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:49 smithi176 ceph-mon[36626]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "mon metadata", "id": "smithi176"}]: dispatch 2024-07-16T09:57:49.566 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:49 smithi176 ceph-mon[36626]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi156.rdibys"}]: dispatch 2024-07-16T09:57:49.566 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:49 smithi176 ceph-mon[36626]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi156.evwwaz"}]: dispatch 2024-07-16T09:57:49.566 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:49 smithi176 ceph-mon[36626]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi176.dvvbud"}]: dispatch 2024-07-16T09:57:49.566 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:49 smithi176 ceph-mon[36626]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi176.wahbsi"}]: dispatch 2024-07-16T09:57:49.566 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:49 smithi176 ceph-mon[36626]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "mgr metadata", "who": "smithi156.zkvylv", "id": "smithi156.zkvylv"}]: dispatch 2024-07-16T09:57:49.566 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:49 smithi176 ceph-mon[36626]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "mgr metadata", "who": "smithi176.dqbrvx", "id": "smithi176.dqbrvx"}]: dispatch 2024-07-16T09:57:49.566 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:49 smithi176 ceph-mon[36626]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-07-16T09:57:49.566 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:49 smithi176 ceph-mon[36626]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-07-16T09:57:49.566 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:49 smithi176 ceph-mon[36626]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-07-16T09:57:49.567 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:49 smithi176 ceph-mon[36626]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-07-16T09:57:49.567 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:49 smithi176 ceph-mon[36626]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-07-16T09:57:49.567 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:49 smithi176 ceph-mon[36626]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-07-16T09:57:49.567 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:49 smithi176 ceph-mon[36626]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-07-16T09:57:49.567 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:49 smithi176 ceph-mon[36626]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-07-16T09:57:49.567 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:49 smithi176 ceph-mon[36626]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-07-16T09:57:49.567 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:49 smithi176 ceph-mon[36626]: Manager daemon smithi156.zkvylv is now available 2024-07-16T09:57:49.567 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:49 smithi176 ceph-mon[36626]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-16T09:57:49.567 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:49 smithi176 ceph-mon[36626]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-16T09:57:49.567 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:49 smithi176 ceph-mon[36626]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi156.zkvylv/mirror_snapshot_schedule"}]: dispatch 2024-07-16T09:57:49.567 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:49 smithi176 ceph-mon[36626]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi156.zkvylv/trash_purge_schedule"}]: dispatch 2024-07-16T09:57:49.623 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:49.622+0000 7f997269e640 1 -- 172.21.15.156:0/2326718343 --> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f996c10ef30 con 0x7f995406ddc0 2024-07-16T09:57:49.624 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:49.623+0000 7f9968ff9640 1 -- 172.21.15.156:0/2326718343 <== mgr.34104 v2:172.21.15.156:6800/3367765292 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+337 (secure 0 0 0) 0x7f996c10ef30 con 0x7f995406ddc0 2024-07-16T09:57:49.627 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:49.625+0000 7f997269e640 1 -- 172.21.15.156:0/2326718343 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f995406ddc0 msgr2=0x7f9954070270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:57:49.627 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:49.625+0000 7f997269e640 1 --2- 172.21.15.156:0/2326718343 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f995406ddc0 0x7f9954070270 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f996c10cd40 tx=0x7f9960035040 comp rx=0 tx=0).stop 2024-07-16T09:57:49.627 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:49.625+0000 7f997269e640 1 -- 172.21.15.156:0/2326718343 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f996c0659f0 msgr2=0x7f996c10cf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:57:49.627 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:49.625+0000 7f997269e640 1 --2- 172.21.15.156:0/2326718343 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f996c0659f0 0x7f996c10cf90 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f995c00eb70 tx=0x7f995c00b040 comp rx=0 tx=0).stop 2024-07-16T09:57:49.627 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:49.625+0000 7f997269e640 1 -- 172.21.15.156:0/2326718343 shutdown_connections 2024-07-16T09:57:49.627 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:49.625+0000 7f997269e640 1 --2- 172.21.15.156:0/2326718343 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f995406ddc0 0x7f9954070270 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:57:49.627 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:49.625+0000 7f997269e640 1 --2- 172.21.15.156:0/2326718343 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f996c0ff4f0 0x7f996c10d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:57:49.627 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:49.625+0000 7f997269e640 1 --2- 172.21.15.156:0/2326718343 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f996c0659f0 0x7f996c10cf90 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:57:49.628 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:49.625+0000 7f997269e640 1 -- 172.21.15.156:0/2326718343 >> 172.21.15.156:0/2326718343 conn(0x7f996c0f6a20 msgr2=0x7f996c0f6df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:57:49.628 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:49.625+0000 7f997269e640 1 -- 172.21.15.156:0/2326718343 shutdown_connections 2024-07-16T09:57:49.628 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:49.625+0000 7f997269e640 1 -- 172.21.15.156:0/2326718343 wait complete. 2024-07-16T09:57:49.710 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:49.707+0000 7f589c739640 1 -- 172.21.15.156:0/3509031217 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f58940fe310 msgr2=0x7f58940fa470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:57:49.710 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:49.707+0000 7f589c739640 1 --2- 172.21.15.156:0/3509031217 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f58940fe310 0x7f58940fa470 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f58840048d0 tx=0x7f5884009da0 comp rx=0 tx=0).stop 2024-07-16T09:57:49.710 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:49.707+0000 7f589c739640 1 -- 172.21.15.156:0/3509031217 shutdown_connections 2024-07-16T09:57:49.711 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:49.707+0000 7f589c739640 1 --2- 172.21.15.156:0/3509031217 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f58940fe310 0x7f58940fa470 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:57:49.711 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:49.707+0000 7f589c739640 1 --2- 172.21.15.156:0/3509031217 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f58940fd9c0 0x7f58940fddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:57:49.711 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:49.707+0000 7f589c739640 1 -- 172.21.15.156:0/3509031217 >> 172.21.15.156:0/3509031217 conn(0x7f58940f61c0 msgr2=0x7f58940f85f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:57:49.711 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:49.707+0000 7f589c739640 1 -- 172.21.15.156:0/3509031217 shutdown_connections 2024-07-16T09:57:49.711 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:49.707+0000 7f589c739640 1 -- 172.21.15.156:0/3509031217 wait complete. 2024-07-16T09:57:49.711 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:49.708+0000 7f589c739640 1 Processor -- start 2024-07-16T09:57:49.711 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:49.708+0000 7f589c739640 1 -- start start 2024-07-16T09:57:49.711 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:49.708+0000 7f589c739640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f58940fd9c0 0x7f589410cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:57:49.711 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:49.708+0000 7f589c739640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f58940fe310 0x7f589410d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:57:49.711 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:49.708+0000 7f589c739640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f589410daf0 con 0x7f58940fe310 2024-07-16T09:57:49.711 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:49.708+0000 7f589c739640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f589410e310 con 0x7f58940fd9c0 2024-07-16T09:57:49.711 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:49.708+0000 7f5899cd4640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f58940fe310 0x7f589410d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:57:49.712 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:49.708+0000 7f5899cd4640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f58940fe310 0x7f589410d4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:39284/0 (socket says 172.21.15.156:39284) 2024-07-16T09:57:49.712 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:49.708+0000 7f5899cd4640 1 -- 172.21.15.156:0/2676732476 learned_addr learned my addr 172.21.15.156:0/2676732476 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T09:57:49.712 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:49.708+0000 7f589a4d5640 1 --2- 172.21.15.156:0/2676732476 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f58940fd9c0 0x7f589410cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:57:49.712 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:49.708+0000 7f5899cd4640 1 -- 172.21.15.156:0/2676732476 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f58940fd9c0 msgr2=0x7f589410cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:57:49.712 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:49.708+0000 7f5899cd4640 1 --2- 172.21.15.156:0/2676732476 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f58940fd9c0 0x7f589410cf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:57:49.712 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:49.708+0000 7f5899cd4640 1 -- 172.21.15.156:0/2676732476 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5884005040 con 0x7f58940fe310 2024-07-16T09:57:49.712 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:49.708+0000 7f5899cd4640 1 --2- 172.21.15.156:0/2676732476 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f58940fe310 0x7f589410d4a0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f5884000c00 tx=0x7f5884009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:57:49.712 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:49.709+0000 7f588affd640 1 -- 172.21.15.156:0/2676732476 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5884038070 con 0x7f58940fe310 2024-07-16T09:57:49.712 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:49.709+0000 7f588affd640 1 -- 172.21.15.156:0/2676732476 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5884032970 con 0x7f58940fe310 2024-07-16T09:57:49.712 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:49.709+0000 7f589c739640 1 -- 172.21.15.156:0/2676732476 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f589410e540 con 0x7f58940fe310 2024-07-16T09:57:49.712 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:49.709+0000 7f588affd640 1 -- 172.21.15.156:0/2676732476 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5884033950 con 0x7f58940fe310 2024-07-16T09:57:49.712 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:49.709+0000 7f589c739640 1 -- 172.21.15.156:0/2676732476 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f589410ea80 con 0x7f58940fe310 2024-07-16T09:57:49.713 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:49.710+0000 7f589c739640 1 -- 172.21.15.156:0/2676732476 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58940f9e60 con 0x7f58940fe310 2024-07-16T09:57:49.713 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:49.710+0000 7f588affd640 1 -- 172.21.15.156:0/2676732476 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 35) v1 ==== 91955+0+0 (secure 0 0 0) 0x7f5884030020 con 0x7f58940fe310 2024-07-16T09:57:49.713 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:49.711+0000 7f588affd640 1 --2- 172.21.15.156:0/2676732476 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f588006ddc0 0x7f5880070270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:57:49.714 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:49.711+0000 7f588affd640 1 -- 172.21.15.156:0/2676732476 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f58840ad660 con 0x7f58940fe310 2024-07-16T09:57:49.714 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:49.711+0000 7f589a4d5640 1 --2- 172.21.15.156:0/2676732476 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f588006ddc0 0x7f5880070270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:57:49.714 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:49.711+0000 7f589a4d5640 1 --2- 172.21.15.156:0/2676732476 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f588006ddc0 0x7f5880070270 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f5890007fd0 tx=0x7f5890009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:57:49.716 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:49.715+0000 7f588affd640 1 -- 172.21.15.156:0/2676732476 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187716 (secure 0 0 0) 0x7f5884076cf0 con 0x7f58940fe310 2024-07-16T09:57:49.908 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:49.907+0000 7f589c739640 1 -- 172.21.15.156:0/2676732476 --> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f589410ed60 con 0x7f588006ddc0 2024-07-16T09:57:49.918 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:49.917+0000 7f588affd640 1 -- 172.21.15.156:0/2676732476 <== mgr.34104 v2:172.21.15.156:6800/3367765292 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7f589410ed60 con 0x7f588006ddc0 2024-07-16T09:57:49.919 INFO:teuthology.orchestra.run.smithi156.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-07-16T09:57:49.919 INFO:teuthology.orchestra.run.smithi156.stdout:alertmanager.smithi156 smithi156 *:9093,9094 running (96s) 15s ago 9m 16.7M - 0.25.0 c8568f914cd2 d524a2c3ff15 2024-07-16T09:57:49.919 INFO:teuthology.orchestra.run.smithi156.stdout:crash.smithi156 smithi156 running (10m) 15s ago 10m 6899k - 17.2.7-1104-ge6231b04 d5de5f953b95 bf6d49ea06b4 2024-07-16T09:57:49.919 INFO:teuthology.orchestra.run.smithi156.stdout:crash.smithi176 smithi176 running (9m) 17s ago 9m 6903k - 17.2.7-1104-ge6231b04 d5de5f953b95 907c9fa1e890 2024-07-16T09:57:49.919 INFO:teuthology.orchestra.run.smithi156.stdout:grafana.smithi156 smithi156 *:3000 running (75s) 15s ago 9m 78.0M - 9.4.7 954c08fa6188 deb9e1bfeb66 2024-07-16T09:57:49.919 INFO:teuthology.orchestra.run.smithi156.stdout:mds.cephfs.smithi156.evwwaz smithi156 running (4m) 15s ago 4m 67.1M - 17.2.7-1104-ge6231b04 d5de5f953b95 7c0ff9ae43f8 2024-07-16T09:57:49.919 INFO:teuthology.orchestra.run.smithi156.stdout:mds.cephfs.smithi156.rdibys smithi156 running (4m) 15s ago 4m 121M - 17.2.7-1104-ge6231b04 d5de5f953b95 b985cd1e6924 2024-07-16T09:57:49.919 INFO:teuthology.orchestra.run.smithi156.stdout:mds.cephfs.smithi176.dvvbud smithi176 running (4m) 17s ago 4m 108M - 17.2.7-1104-ge6231b04 d5de5f953b95 1b00147950f9 2024-07-16T09:57:49.919 INFO:teuthology.orchestra.run.smithi156.stdout:mds.cephfs.smithi176.wahbsi smithi176 running (4m) 17s ago 4m 61.8M - 17.2.7-1104-ge6231b04 d5de5f953b95 866664af2311 2024-07-16T09:57:49.920 INFO:teuthology.orchestra.run.smithi156.stdout:mgr.smithi156.zkvylv smithi156 *:8443,9283,8765 running (58s) 15s ago 11m 491M - 19.0.0-3495-ga6ca461b 16f7feb04342 ee53bd604b7c 2024-07-16T09:57:49.920 INFO:teuthology.orchestra.run.smithi156.stdout:mgr.smithi176.dqbrvx smithi176 *:8443,9283,8765 running (21s) 17s ago 9m 54.1M - 19.0.0-3495-ga6ca461b 16f7feb04342 ce9eaae4c2d0 2024-07-16T09:57:49.920 INFO:teuthology.orchestra.run.smithi156.stdout:mon.smithi156 smithi156 running (11m) 15s ago 11m 56.7M 2048M 17.2.7-1104-ge6231b04 d5de5f953b95 6471c8acff40 2024-07-16T09:57:49.920 INFO:teuthology.orchestra.run.smithi156.stdout:mon.smithi176 smithi176 running (9m) 17s ago 9m 44.2M 2048M 17.2.7-1104-ge6231b04 d5de5f953b95 3a293ac6ef01 2024-07-16T09:57:49.920 INFO:teuthology.orchestra.run.smithi156.stdout:node-exporter.smithi156 smithi156 *:9100 running (102s) 15s ago 10m 8711k - 1.5.0 0da6a335fe13 e5d86f23a14c 2024-07-16T09:57:49.920 INFO:teuthology.orchestra.run.smithi156.stdout:node-exporter.smithi176 smithi176 *:9100 running (67s) 17s ago 9m 8136k - 1.5.0 0da6a335fe13 ea842728144a 2024-07-16T09:57:49.920 INFO:teuthology.orchestra.run.smithi156.stdout:osd.0 smithi156 running (8m) 15s ago 8m 221M 1557M 17.2.7-1104-ge6231b04 d5de5f953b95 0471758ec349 2024-07-16T09:57:49.920 INFO:teuthology.orchestra.run.smithi156.stdout:osd.1 smithi156 running (7m) 15s ago 7m 245M 1557M 17.2.7-1104-ge6231b04 d5de5f953b95 149a0bda2f61 2024-07-16T09:57:49.920 INFO:teuthology.orchestra.run.smithi156.stdout:osd.2 smithi156 running (7m) 15s ago 7m 197M 1557M 17.2.7-1104-ge6231b04 d5de5f953b95 f28acec5cb79 2024-07-16T09:57:49.920 INFO:teuthology.orchestra.run.smithi156.stdout:osd.3 smithi176 running (7m) 17s ago 7m 259M 2581M 17.2.7-1104-ge6231b04 d5de5f953b95 733861ba84dc 2024-07-16T09:57:49.920 INFO:teuthology.orchestra.run.smithi156.stdout:osd.4 smithi176 running (6m) 17s ago 6m 229M 2581M 17.2.7-1104-ge6231b04 d5de5f953b95 ef0c67cec086 2024-07-16T09:57:49.920 INFO:teuthology.orchestra.run.smithi156.stdout:osd.5 smithi176 running (6m) 17s ago 6m 223M 2581M 17.2.7-1104-ge6231b04 d5de5f953b95 996e15448a52 2024-07-16T09:57:49.920 INFO:teuthology.orchestra.run.smithi156.stdout:prometheus.smithi156 smithi156 *:9095 running (25s) 15s ago 9m 44.0M - 2.43.0 a07b618ecd1d 20c195ecd665 2024-07-16T09:57:49.921 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:49.920+0000 7f589c739640 1 -- 172.21.15.156:0/2676732476 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f588006ddc0 msgr2=0x7f5880070270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:57:49.921 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:49.920+0000 7f589c739640 1 --2- 172.21.15.156:0/2676732476 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f588006ddc0 0x7f5880070270 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f5890007fd0 tx=0x7f5890009040 comp rx=0 tx=0).stop 2024-07-16T09:57:49.921 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:49.920+0000 7f589c739640 1 -- 172.21.15.156:0/2676732476 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f58940fe310 msgr2=0x7f589410d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:57:49.922 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:49.920+0000 7f589c739640 1 --2- 172.21.15.156:0/2676732476 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f58940fe310 0x7f589410d4a0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f5884000c00 tx=0x7f5884009da0 comp rx=0 tx=0).stop 2024-07-16T09:57:49.922 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:49.920+0000 7f589c739640 1 -- 172.21.15.156:0/2676732476 shutdown_connections 2024-07-16T09:57:49.922 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:49.920+0000 7f589c739640 1 --2- 172.21.15.156:0/2676732476 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f588006ddc0 0x7f5880070270 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:57:49.922 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:49.920+0000 7f589c739640 1 --2- 172.21.15.156:0/2676732476 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f58940fe310 0x7f589410d4a0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:57:49.922 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:49.920+0000 7f589c739640 1 --2- 172.21.15.156:0/2676732476 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f58940fd9c0 0x7f589410cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:57:49.922 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:49.920+0000 7f589c739640 1 -- 172.21.15.156:0/2676732476 >> 172.21.15.156:0/2676732476 conn(0x7f58940f61c0 msgr2=0x7f58940f7860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:57:49.922 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:49.920+0000 7f589c739640 1 -- 172.21.15.156:0/2676732476 shutdown_connections 2024-07-16T09:57:49.922 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:49.920+0000 7f589c739640 1 -- 172.21.15.156:0/2676732476 wait complete. 2024-07-16T09:57:50.006 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:50.005+0000 7f1bea9f1640 1 -- 172.21.15.156:0/2739728191 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1be40fb0c0 msgr2=0x7f1be40fd940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:57:50.006 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:50.005+0000 7f1bea9f1640 1 --2- 172.21.15.156:0/2739728191 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1be40fb0c0 0x7f1be40fd940 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f1bd80048d0 tx=0x7f1bd8009da0 comp rx=0 tx=0).stop 2024-07-16T09:57:50.007 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:50.005+0000 7f1bea9f1640 1 -- 172.21.15.156:0/2739728191 shutdown_connections 2024-07-16T09:57:50.007 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:50.005+0000 7f1bea9f1640 1 --2- 172.21.15.156:0/2739728191 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1be40fb0c0 0x7f1be40fd940 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:57:50.007 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:50.005+0000 7f1bea9f1640 1 --2- 172.21.15.156:0/2739728191 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1be40fa770 0x7f1be40fab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:57:50.007 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:50.005+0000 7f1bea9f1640 1 -- 172.21.15.156:0/2739728191 >> 172.21.15.156:0/2739728191 conn(0x7f1be40f61b0 msgr2=0x7f1be40f85e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:57:50.007 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:50.005+0000 7f1bea9f1640 1 -- 172.21.15.156:0/2739728191 shutdown_connections 2024-07-16T09:57:50.007 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:50.005+0000 7f1bea9f1640 1 -- 172.21.15.156:0/2739728191 wait complete. 2024-07-16T09:57:50.008 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:50.006+0000 7f1bea9f1640 1 Processor -- start 2024-07-16T09:57:50.009 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:50.006+0000 7f1bea9f1640 1 -- start start 2024-07-16T09:57:50.009 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:50.006+0000 7f1bea9f1640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1be40fa770 0x7f1be410d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:57:50.009 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:50.006+0000 7f1bea9f1640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1be40fb0c0 0x7f1be410d750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:57:50.009 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:50.006+0000 7f1bea9f1640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1be410dda0 con 0x7f1be40fa770 2024-07-16T09:57:50.009 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:50.006+0000 7f1bea9f1640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1be410e570 con 0x7f1be40fb0c0 2024-07-16T09:57:50.009 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:50.006+0000 7f1be3fff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1be40fa770 0x7f1be410d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:57:50.009 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:50.006+0000 7f1be3fff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1be40fa770 0x7f1be410d210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:39300/0 (socket says 172.21.15.156:39300) 2024-07-16T09:57:50.009 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:50.006+0000 7f1be3fff640 1 -- 172.21.15.156:0/2969339561 learned_addr learned my addr 172.21.15.156:0/2969339561 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T09:57:50.009 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:50.006+0000 7f1be37fe640 1 --2- 172.21.15.156:0/2969339561 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1be40fb0c0 0x7f1be410d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:57:50.009 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:50.007+0000 7f1be3fff640 1 -- 172.21.15.156:0/2969339561 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1be40fb0c0 msgr2=0x7f1be410d750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:57:50.009 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:50.007+0000 7f1be3fff640 1 --2- 172.21.15.156:0/2969339561 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1be40fb0c0 0x7f1be410d750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:57:50.009 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:50.007+0000 7f1be3fff640 1 -- 172.21.15.156:0/2969339561 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bd8005040 con 0x7f1be40fa770 2024-07-16T09:57:50.009 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:50.007+0000 7f1be3fff640 1 --2- 172.21.15.156:0/2969339561 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1be40fa770 0x7f1be410d210 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f1bd400eb70 tx=0x7f1bd400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:57:50.010 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:50.007+0000 7f1be0ff9640 1 -- 172.21.15.156:0/2969339561 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1bd4007c30 con 0x7f1be40fa770 2024-07-16T09:57:50.010 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:50.007+0000 7f1be0ff9640 1 -- 172.21.15.156:0/2969339561 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1bd400f040 con 0x7f1be40fa770 2024-07-16T09:57:50.010 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:50.007+0000 7f1bea9f1640 1 -- 172.21.15.156:0/2969339561 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1be410e7a0 con 0x7f1be40fa770 2024-07-16T09:57:50.010 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:50.007+0000 7f1be0ff9640 1 -- 172.21.15.156:0/2969339561 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1bd40095f0 con 0x7f1be40fa770 2024-07-16T09:57:50.010 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:50.007+0000 7f1bea9f1640 1 -- 172.21.15.156:0/2969339561 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1be410ed10 con 0x7f1be40fa770 2024-07-16T09:57:50.011 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:50.008+0000 7f1bea9f1640 1 -- 172.21.15.156:0/2969339561 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1be4105920 con 0x7f1be40fa770 2024-07-16T09:57:50.011 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:50.009+0000 7f1be0ff9640 1 -- 172.21.15.156:0/2969339561 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 35) v1 ==== 91955+0+0 (secure 0 0 0) 0x7f1bd4009790 con 0x7f1be40fa770 2024-07-16T09:57:50.011 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:50.009+0000 7f1be0ff9640 1 --2- 172.21.15.156:0/2969339561 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f1bcc06ddc0 0x7f1bcc070270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:57:50.011 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:50.010+0000 7f1be0ff9640 1 -- 172.21.15.156:0/2969339561 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f1bd408d300 con 0x7f1be40fa770 2024-07-16T09:57:50.011 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:50.010+0000 7f1be37fe640 1 --2- 172.21.15.156:0/2969339561 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f1bcc06ddc0 0x7f1bcc070270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:57:50.011 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:50.010+0000 7f1be37fe640 1 --2- 172.21.15.156:0/2969339561 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f1bcc06ddc0 0x7f1bcc070270 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f1bd8000c00 tx=0x7f1bd8035040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:57:50.015 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:50.014+0000 7f1be0ff9640 1 -- 172.21.15.156:0/2969339561 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187716 (secure 0 0 0) 0x7f1bd4056990 con 0x7f1be40fa770 2024-07-16T09:57:50.273 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:50.272+0000 7f1bea9f1640 1 -- 172.21.15.156:0/2969339561 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f1be41052a0 con 0x7f1be40fa770 2024-07-16T09:57:50.273 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:50.272+0000 7f1be0ff9640 1 -- 172.21.15.156:0/2969339561 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+844 (secure 0 0 0) 0x7f1be41052a0 con 0x7f1be40fa770 2024-07-16T09:57:50.274 INFO:teuthology.orchestra.run.smithi156.stdout:{ 2024-07-16T09:57:50.274 INFO:teuthology.orchestra.run.smithi156.stdout: "mon": { 2024-07-16T09:57:50.274 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph version 17.2.7-1104-ge6231b04 (e6231b04b52ebf3b9b31803282dd3074439e51ac) quincy (stable)": 1, 2024-07-16T09:57:50.274 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph version 19.0.0-3495-ga6ca461b (a6ca461be7cc1aff3658c2c6df4cff975a24449a) squid (rc)": 1 2024-07-16T09:57:50.274 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-07-16T09:57:50.274 INFO:teuthology.orchestra.run.smithi156.stdout: "mgr": { 2024-07-16T09:57:50.274 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph version 19.0.0-3495-ga6ca461b (a6ca461be7cc1aff3658c2c6df4cff975a24449a) squid (rc)": 2 2024-07-16T09:57:50.274 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-07-16T09:57:50.274 INFO:teuthology.orchestra.run.smithi156.stdout: "osd": { 2024-07-16T09:57:50.274 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph version 17.2.7-1104-ge6231b04 (e6231b04b52ebf3b9b31803282dd3074439e51ac) quincy (stable)": 6 2024-07-16T09:57:50.274 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-07-16T09:57:50.274 INFO:teuthology.orchestra.run.smithi156.stdout: "mds": { 2024-07-16T09:57:50.275 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph version 17.2.7-1104-ge6231b04 (e6231b04b52ebf3b9b31803282dd3074439e51ac) quincy (stable)": 4 2024-07-16T09:57:50.275 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-07-16T09:57:50.275 INFO:teuthology.orchestra.run.smithi156.stdout: "overall": { 2024-07-16T09:57:50.275 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph version 17.2.7-1104-ge6231b04 (e6231b04b52ebf3b9b31803282dd3074439e51ac) quincy (stable)": 11, 2024-07-16T09:57:50.275 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph version 19.0.0-3495-ga6ca461b (a6ca461be7cc1aff3658c2c6df4cff975a24449a) squid (rc)": 3 2024-07-16T09:57:50.275 INFO:teuthology.orchestra.run.smithi156.stdout: } 2024-07-16T09:57:50.275 INFO:teuthology.orchestra.run.smithi156.stdout:} 2024-07-16T09:57:50.276 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:50.274+0000 7f1bea9f1640 1 -- 172.21.15.156:0/2969339561 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f1bcc06ddc0 msgr2=0x7f1bcc070270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:57:50.276 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:50.274+0000 7f1bea9f1640 1 --2- 172.21.15.156:0/2969339561 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f1bcc06ddc0 0x7f1bcc070270 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f1bd8000c00 tx=0x7f1bd8035040 comp rx=0 tx=0).stop 2024-07-16T09:57:50.276 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:50.274+0000 7f1bea9f1640 1 -- 172.21.15.156:0/2969339561 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1be40fa770 msgr2=0x7f1be410d210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:57:50.276 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:50.274+0000 7f1bea9f1640 1 --2- 172.21.15.156:0/2969339561 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1be40fa770 0x7f1be410d210 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f1bd400eb70 tx=0x7f1bd400b040 comp rx=0 tx=0).stop 2024-07-16T09:57:50.276 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:50.274+0000 7f1bea9f1640 1 -- 172.21.15.156:0/2969339561 shutdown_connections 2024-07-16T09:57:50.276 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:50.275+0000 7f1bea9f1640 1 --2- 172.21.15.156:0/2969339561 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f1bcc06ddc0 0x7f1bcc070270 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:57:50.276 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:50.275+0000 7f1bea9f1640 1 --2- 172.21.15.156:0/2969339561 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1be40fb0c0 0x7f1be410d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:57:50.276 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:50.275+0000 7f1bea9f1640 1 --2- 172.21.15.156:0/2969339561 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1be40fa770 0x7f1be410d210 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:57:50.277 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:50.275+0000 7f1bea9f1640 1 -- 172.21.15.156:0/2969339561 >> 172.21.15.156:0/2969339561 conn(0x7f1be40f61b0 msgr2=0x7f1be40f7bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:57:50.277 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:50.275+0000 7f1bea9f1640 1 -- 172.21.15.156:0/2969339561 shutdown_connections 2024-07-16T09:57:50.277 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:50.275+0000 7f1bea9f1640 1 -- 172.21.15.156:0/2969339561 wait complete. 2024-07-16T09:57:50.363 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:50.361+0000 7f21c99d6640 1 -- 172.21.15.156:0/3635213172 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f21c40fc510 msgr2=0x7f21c40fc980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:57:50.364 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:50.361+0000 7f21c99d6640 1 --2- 172.21.15.156:0/3635213172 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f21c40fc510 0x7f21c40fc980 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f21b40048d0 tx=0x7f21b4009da0 comp rx=0 tx=0).stop 2024-07-16T09:57:50.364 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:50.361+0000 7f21c99d6640 1 -- 172.21.15.156:0/3635213172 shutdown_connections 2024-07-16T09:57:50.364 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:50.361+0000 7f21c99d6640 1 --2- 172.21.15.156:0/3635213172 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f21c40fc510 0x7f21c40fc980 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:57:50.364 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:50.361+0000 7f21c99d6640 1 --2- 172.21.15.156:0/3635213172 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f21c40fb240 0x7f21c40fb650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:57:50.364 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:50.361+0000 7f21c99d6640 1 -- 172.21.15.156:0/3635213172 >> 172.21.15.156:0/3635213172 conn(0x7f21c40f6a20 msgr2=0x7f21c40f8e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:57:50.364 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:50.362+0000 7f21c99d6640 1 -- 172.21.15.156:0/3635213172 shutdown_connections 2024-07-16T09:57:50.364 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:50.362+0000 7f21c99d6640 1 -- 172.21.15.156:0/3635213172 wait complete. 2024-07-16T09:57:50.364 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:50.362+0000 7f21c99d6640 1 Processor -- start 2024-07-16T09:57:50.364 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:50.362+0000 7f21c99d6640 1 -- start start 2024-07-16T09:57:50.364 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:50.362+0000 7f21c99d6640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f21c40fb240 0x7f21c410cf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:57:50.364 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:50.362+0000 7f21c99d6640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f21c40fc510 0x7f21c410d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:57:50.364 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:50.362+0000 7f21c99d6640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21c410db10 con 0x7f21c40fb240 2024-07-16T09:57:50.365 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:50.362+0000 7f21c99d6640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21c410e330 con 0x7f21c40fc510 2024-07-16T09:57:50.366 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:50.363+0000 7f21c2ffd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f21c40fb240 0x7f21c410cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:57:50.366 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:50.363+0000 7f21c2ffd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f21c40fb240 0x7f21c410cf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:39324/0 (socket says 172.21.15.156:39324) 2024-07-16T09:57:50.366 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:50.363+0000 7f21c27fc640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f21c40fc510 0x7f21c410d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:57:50.366 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:50.363+0000 7f21c2ffd640 1 -- 172.21.15.156:0/1691039189 learned_addr learned my addr 172.21.15.156:0/1691039189 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T09:57:50.366 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:50.363+0000 7f21c27fc640 1 -- 172.21.15.156:0/1691039189 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f21c40fb240 msgr2=0x7f21c410cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:57:50.366 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:50.363+0000 7f21c27fc640 1 --2- 172.21.15.156:0/1691039189 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f21c40fb240 0x7f21c410cf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:57:50.366 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:50.363+0000 7f21c27fc640 1 -- 172.21.15.156:0/1691039189 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21b4005040 con 0x7f21c40fc510 2024-07-16T09:57:50.366 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:50.363+0000 7f21c27fc640 1 --2- 172.21.15.156:0/1691039189 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f21c40fc510 0x7f21c410d4c0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f21b402a530 tx=0x7f21b4009da0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:57:50.367 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:50.363+0000 7f21bb7fe640 1 -- 172.21.15.156:0/1691039189 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f21b4038070 con 0x7f21c40fc510 2024-07-16T09:57:50.367 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:50.363+0000 7f21bb7fe640 1 -- 172.21.15.156:0/1691039189 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f21b4032970 con 0x7f21c40fc510 2024-07-16T09:57:50.367 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:50.363+0000 7f21c99d6640 1 -- 172.21.15.156:0/1691039189 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21c410e560 con 0x7f21c40fc510 2024-07-16T09:57:50.367 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:50.363+0000 7f21bb7fe640 1 -- 172.21.15.156:0/1691039189 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f21b403d9d0 con 0x7f21c40fc510 2024-07-16T09:57:50.367 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:50.363+0000 7f21c99d6640 1 -- 172.21.15.156:0/1691039189 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21c410eaa0 con 0x7f21c40fc510 2024-07-16T09:57:50.367 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:50.365+0000 7f21c99d6640 1 -- 172.21.15.156:0/1691039189 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21c40fa6a0 con 0x7f21c40fc510 2024-07-16T09:57:50.367 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:50.365+0000 7f21bb7fe640 1 -- 172.21.15.156:0/1691039189 <== mon.1 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 36) v1 ==== 92251+0+0 (secure 0 0 0) 0x7f21b4030020 con 0x7f21c40fc510 2024-07-16T09:57:50.368 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:50.365+0000 7f21bb7fe640 1 --2- 172.21.15.156:0/1691039189 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f21b006e140 0x7f21b00705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:57:50.368 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:50.366+0000 7f21bb7fe640 1 -- 172.21.15.156:0/1691039189 <== mon.1 v2:172.21.15.176:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f21b40ae360 con 0x7f21c40fc510 2024-07-16T09:57:50.368 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:50.366+0000 7f21c2ffd640 1 --2- 172.21.15.156:0/1691039189 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f21b006e140 0x7f21b00705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:57:50.368 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:50.366+0000 7f21c2ffd640 1 --2- 172.21.15.156:0/1691039189 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f21b006e140 0x7f21b00705f0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f21bc007bb0 tx=0x7f21bc00b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:57:50.371 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:50.370+0000 7f21bb7fe640 1 -- 172.21.15.156:0/1691039189 <== mon.1 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187716 (secure 0 0 0) 0x7f21b40775f0 con 0x7f21c40fc510 2024-07-16T09:57:50.590 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:50.589+0000 7f21c99d6640 1 -- 172.21.15.156:0/1691039189 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f21c4101010 con 0x7f21c40fc510 2024-07-16T09:57:50.591 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:50 smithi156 ceph-mon[93066]: mgrmap e35: smithi156.zkvylv(active, since 1.00889s), standbys: smithi176.dqbrvx 2024-07-16T09:57:50.591 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:50 smithi156 ceph-mon[93066]: from='client.24569 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-16T09:57:50.591 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:50 smithi156 ceph-mon[93066]: pgmap v3: 65 pgs: 65 active+clean; 253 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-07-16T09:57:50.591 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:50 smithi156 ceph-mon[93066]: from='client.? 172.21.15.156:0/2969339561' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T09:57:50.591 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:50.590+0000 7f21bb7fe640 1 -- 172.21.15.156:0/1691039189 <== mon.1 v2:172.21.15.176:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 18 v18) v1 ==== 76+0+1869 (secure 0 0 0) 0x7f21b407acb0 con 0x7f21c40fc510 2024-07-16T09:57:50.592 INFO:teuthology.orchestra.run.smithi156.stdout:e18 2024-07-16T09:57:50.592 INFO:teuthology.orchestra.run.smithi156.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-07-16T09:57:50.592 INFO:teuthology.orchestra.run.smithi156.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-07-16T09:57:50.592 INFO:teuthology.orchestra.run.smithi156.stdout:legacy client fscid: 1 2024-07-16T09:57:50.592 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-07-16T09:57:50.592 INFO:teuthology.orchestra.run.smithi156.stdout:Filesystem 'cephfs' (1) 2024-07-16T09:57:50.592 INFO:teuthology.orchestra.run.smithi156.stdout:fs_name cephfs 2024-07-16T09:57:50.592 INFO:teuthology.orchestra.run.smithi156.stdout:epoch 18 2024-07-16T09:57:50.592 INFO:teuthology.orchestra.run.smithi156.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-07-16T09:57:50.592 INFO:teuthology.orchestra.run.smithi156.stdout:created 2024-07-16T09:52:55.214975+0000 2024-07-16T09:57:50.593 INFO:teuthology.orchestra.run.smithi156.stdout:modified 2024-07-16T09:55:41.914320+0000 2024-07-16T09:57:50.593 INFO:teuthology.orchestra.run.smithi156.stdout:tableserver 0 2024-07-16T09:57:50.593 INFO:teuthology.orchestra.run.smithi156.stdout:root 0 2024-07-16T09:57:50.593 INFO:teuthology.orchestra.run.smithi156.stdout:session_timeout 60 2024-07-16T09:57:50.593 INFO:teuthology.orchestra.run.smithi156.stdout:session_autoclose 300 2024-07-16T09:57:50.593 INFO:teuthology.orchestra.run.smithi156.stdout:max_file_size 1099511627776 2024-07-16T09:57:50.593 INFO:teuthology.orchestra.run.smithi156.stdout:required_client_features {} 2024-07-16T09:57:50.593 INFO:teuthology.orchestra.run.smithi156.stdout:last_failure 0 2024-07-16T09:57:50.593 INFO:teuthology.orchestra.run.smithi156.stdout:last_failure_osd_epoch 0 2024-07-16T09:57:50.593 INFO:teuthology.orchestra.run.smithi156.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-07-16T09:57:50.593 INFO:teuthology.orchestra.run.smithi156.stdout:max_mds 2 2024-07-16T09:57:50.593 INFO:teuthology.orchestra.run.smithi156.stdout:in 0,1 2024-07-16T09:57:50.593 INFO:teuthology.orchestra.run.smithi156.stdout:up {0=14458,1=24261} 2024-07-16T09:57:50.594 INFO:teuthology.orchestra.run.smithi156.stdout:failed 2024-07-16T09:57:50.594 INFO:teuthology.orchestra.run.smithi156.stdout:damaged 2024-07-16T09:57:50.594 INFO:teuthology.orchestra.run.smithi156.stdout:stopped 2024-07-16T09:57:50.594 INFO:teuthology.orchestra.run.smithi156.stdout:data_pools [3] 2024-07-16T09:57:50.594 INFO:teuthology.orchestra.run.smithi156.stdout:metadata_pool 2 2024-07-16T09:57:50.594 INFO:teuthology.orchestra.run.smithi156.stdout:inline_data disabled 2024-07-16T09:57:50.594 INFO:teuthology.orchestra.run.smithi156.stdout:balancer 2024-07-16T09:57:50.594 INFO:teuthology.orchestra.run.smithi156.stdout:standby_count_wanted 1 2024-07-16T09:57:50.594 INFO:teuthology.orchestra.run.smithi156.stdout:[mds.cephfs.smithi156.rdibys{0:14458} state up:active seq 4 export targets 1 join_fscid=1 addr [v2:172.21.15.156:6826/1080176013,v1:172.21.15.156:6827/1080176013] compat {c=[1],r=[1],i=[7ff]}] 2024-07-16T09:57:50.594 INFO:teuthology.orchestra.run.smithi156.stdout:[mds.cephfs.smithi156.evwwaz{0:14468} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.156:6828/560329502,v1:172.21.15.156:6829/560329502] compat {c=[1],r=[1],i=[7ff]}] 2024-07-16T09:57:50.594 INFO:teuthology.orchestra.run.smithi156.stdout:[mds.cephfs.smithi176.dvvbud{1:24261} state up:active seq 4 export targets 0 join_fscid=1 addr [v2:172.21.15.176:6824/1577019287,v1:172.21.15.176:6825/1577019287] compat {c=[1],r=[1],i=[7ff]}] 2024-07-16T09:57:50.594 INFO:teuthology.orchestra.run.smithi156.stdout:[mds.cephfs.smithi176.wahbsi{1:24273} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.176:6826/2196775412,v1:172.21.15.176:6827/2196775412] compat {c=[1],r=[1],i=[7ff]}] 2024-07-16T09:57:50.595 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-07-16T09:57:50.595 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-07-16T09:57:50.595 INFO:teuthology.orchestra.run.smithi156.stderr:dumped fsmap epoch 18 2024-07-16T09:57:50.595 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:50.592+0000 7f21c99d6640 1 -- 172.21.15.156:0/1691039189 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f21b006e140 msgr2=0x7f21b00705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:57:50.595 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:50.592+0000 7f21c99d6640 1 --2- 172.21.15.156:0/1691039189 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f21b006e140 0x7f21b00705f0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f21bc007bb0 tx=0x7f21bc00b040 comp rx=0 tx=0).stop 2024-07-16T09:57:50.595 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:50.592+0000 7f21c99d6640 1 -- 172.21.15.156:0/1691039189 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f21c40fc510 msgr2=0x7f21c410d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:57:50.596 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:50.592+0000 7f21c99d6640 1 --2- 172.21.15.156:0/1691039189 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f21c40fc510 0x7f21c410d4c0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f21b402a530 tx=0x7f21b4009da0 comp rx=0 tx=0).stop 2024-07-16T09:57:50.596 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:50.592+0000 7f21c99d6640 1 -- 172.21.15.156:0/1691039189 shutdown_connections 2024-07-16T09:57:50.596 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:50.592+0000 7f21c99d6640 1 --2- 172.21.15.156:0/1691039189 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f21b006e140 0x7f21b00705f0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:57:50.596 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:50.592+0000 7f21c99d6640 1 --2- 172.21.15.156:0/1691039189 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f21c40fc510 0x7f21c410d4c0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:57:50.596 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:50.592+0000 7f21c99d6640 1 --2- 172.21.15.156:0/1691039189 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f21c40fb240 0x7f21c410cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:57:50.596 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:50.592+0000 7f21c99d6640 1 -- 172.21.15.156:0/1691039189 >> 172.21.15.156:0/1691039189 conn(0x7f21c40f6a20 msgr2=0x7f21c4065340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:57:50.596 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:50.592+0000 7f21c99d6640 1 -- 172.21.15.156:0/1691039189 shutdown_connections 2024-07-16T09:57:50.596 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:50.592+0000 7f21c99d6640 1 -- 172.21.15.156:0/1691039189 wait complete. 2024-07-16T09:57:50.596 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:50 smithi176 ceph-mon[36626]: mgrmap e35: smithi156.zkvylv(active, since 1.00889s), standbys: smithi176.dqbrvx 2024-07-16T09:57:50.596 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:50 smithi176 ceph-mon[36626]: from='client.24569 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-16T09:57:50.597 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:50 smithi176 ceph-mon[36626]: pgmap v3: 65 pgs: 65 active+clean; 253 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-07-16T09:57:50.597 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:50 smithi176 ceph-mon[36626]: from='client.? 172.21.15.156:0/2969339561' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T09:57:50.677 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:50.674+0000 7fce10468640 1 -- 172.21.15.156:0/2806902284 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fce080fc510 msgr2=0x7fce080fc980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:57:50.677 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:50.674+0000 7fce10468640 1 --2- 172.21.15.156:0/2806902284 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fce080fc510 0x7fce080fc980 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fcdf80048d0 tx=0x7fcdf8009da0 comp rx=0 tx=0).stop 2024-07-16T09:57:50.677 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:50.674+0000 7fce10468640 1 -- 172.21.15.156:0/2806902284 shutdown_connections 2024-07-16T09:57:50.677 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:50.674+0000 7fce10468640 1 --2- 172.21.15.156:0/2806902284 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fce080fc510 0x7fce080fc980 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:57:50.677 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:50.674+0000 7fce10468640 1 --2- 172.21.15.156:0/2806902284 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fce080fb240 0x7fce080fb650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:57:50.678 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:50.674+0000 7fce10468640 1 -- 172.21.15.156:0/2806902284 >> 172.21.15.156:0/2806902284 conn(0x7fce080f6a20 msgr2=0x7fce080f8e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:57:50.678 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:50.674+0000 7fce10468640 1 -- 172.21.15.156:0/2806902284 shutdown_connections 2024-07-16T09:57:50.678 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:50.674+0000 7fce10468640 1 -- 172.21.15.156:0/2806902284 wait complete. 2024-07-16T09:57:50.678 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:50.675+0000 7fce10468640 1 Processor -- start 2024-07-16T09:57:50.678 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:50.675+0000 7fce10468640 1 -- start start 2024-07-16T09:57:50.678 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:50.675+0000 7fce10468640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fce080fb240 0x7fce0810cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:57:50.678 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:50.675+0000 7fce10468640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fce080fc510 0x7fce0810d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:57:50.678 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:50.675+0000 7fce10468640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce0810db30 con 0x7fce080fc510 2024-07-16T09:57:50.678 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:50.675+0000 7fce10468640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce0810e350 con 0x7fce080fb240 2024-07-16T09:57:50.678 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:50.675+0000 7fce0da03640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fce080fc510 0x7fce0810d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:57:50.678 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:50.675+0000 7fce0da03640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fce080fc510 0x7fce0810d4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:39366/0 (socket says 172.21.15.156:39366) 2024-07-16T09:57:50.678 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:50.675+0000 7fce0da03640 1 -- 172.21.15.156:0/3017168780 learned_addr learned my addr 172.21.15.156:0/3017168780 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T09:57:50.679 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:50.675+0000 7fce0e204640 1 --2- 172.21.15.156:0/3017168780 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fce080fb240 0x7fce0810cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:57:50.679 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:50.675+0000 7fce0da03640 1 -- 172.21.15.156:0/3017168780 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fce080fb240 msgr2=0x7fce0810cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:57:50.679 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:50.675+0000 7fce0da03640 1 --2- 172.21.15.156:0/3017168780 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fce080fb240 0x7fce0810cfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:57:50.679 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:50.675+0000 7fce0da03640 1 -- 172.21.15.156:0/3017168780 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdf8005040 con 0x7fce080fc510 2024-07-16T09:57:50.679 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:50.676+0000 7fce0da03640 1 --2- 172.21.15.156:0/3017168780 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fce080fc510 0x7fce0810d4e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fcdf80048a0 tx=0x7fcdf8009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:57:50.679 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:50.676+0000 7fcdfeffd640 1 -- 172.21.15.156:0/3017168780 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcdf8038070 con 0x7fce080fc510 2024-07-16T09:57:50.679 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:50.676+0000 7fcdfeffd640 1 -- 172.21.15.156:0/3017168780 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcdf80092b0 con 0x7fce080fc510 2024-07-16T09:57:50.679 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:50.676+0000 7fce10468640 1 -- 172.21.15.156:0/3017168780 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce0810e580 con 0x7fce080fc510 2024-07-16T09:57:50.679 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:50.676+0000 7fcdfeffd640 1 -- 172.21.15.156:0/3017168780 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcdf8033780 con 0x7fce080fc510 2024-07-16T09:57:50.679 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:50.676+0000 7fce10468640 1 -- 172.21.15.156:0/3017168780 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce0810eac0 con 0x7fce080fc510 2024-07-16T09:57:50.680 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:50.677+0000 7fce10468640 1 -- 172.21.15.156:0/3017168780 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce080fa6a0 con 0x7fce080fc510 2024-07-16T09:57:50.680 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:50.677+0000 7fcdfeffd640 1 -- 172.21.15.156:0/3017168780 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 36) v1 ==== 92251+0+0 (secure 0 0 0) 0x7fcdf8030020 con 0x7fce080fc510 2024-07-16T09:57:50.680 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:50.678+0000 7fcdfeffd640 1 --2- 172.21.15.156:0/3017168780 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7fcdf406e140 0x7fcdf40705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:57:50.680 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:50.678+0000 7fcdfeffd640 1 -- 172.21.15.156:0/3017168780 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fcdf80ad8f0 con 0x7fce080fc510 2024-07-16T09:57:50.680 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:50.678+0000 7fce0e204640 1 --2- 172.21.15.156:0/3017168780 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7fcdf406e140 0x7fcdf40705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:57:50.681 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:50.678+0000 7fce0e204640 1 --2- 172.21.15.156:0/3017168780 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7fcdf406e140 0x7fcdf40705f0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fce04009af0 tx=0x7fce0400b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:57:50.683 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:50.682+0000 7fcdfeffd640 1 -- 172.21.15.156:0/3017168780 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187716 (secure 0 0 0) 0x7fcdf8076cf0 con 0x7fce080fc510 2024-07-16T09:57:50.892 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:50.890+0000 7fce10468640 1 -- 172.21.15.156:0/3017168780 --> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fce0810ee10 con 0x7fcdf406e140 2024-07-16T09:57:50.892 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:50.891+0000 7fcdfeffd640 1 -- 172.21.15.156:0/3017168780 <== mgr.34104 v2:172.21.15.156:6800/3367765292 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+337 (secure 0 0 0) 0x7fce0810ee10 con 0x7fcdf406e140 2024-07-16T09:57:50.893 INFO:teuthology.orchestra.run.smithi156.stdout:{ 2024-07-16T09:57:50.893 INFO:teuthology.orchestra.run.smithi156.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:c15ec3aa8e6c0a9c12ee24ff79f3e7dd4d01b3d1a702dfeb7d7fff9b0811a393", 2024-07-16T09:57:50.893 INFO:teuthology.orchestra.run.smithi156.stdout: "in_progress": true, 2024-07-16T09:57:50.893 INFO:teuthology.orchestra.run.smithi156.stdout: "which": "Upgrading all daemon types on all hosts", 2024-07-16T09:57:50.893 INFO:teuthology.orchestra.run.smithi156.stdout: "services_complete": [ 2024-07-16T09:57:50.893 INFO:teuthology.orchestra.run.smithi156.stdout: "mgr" 2024-07-16T09:57:50.893 INFO:teuthology.orchestra.run.smithi156.stdout: ], 2024-07-16T09:57:50.893 INFO:teuthology.orchestra.run.smithi156.stdout: "progress": "2/21 daemons upgraded", 2024-07-16T09:57:50.893 INFO:teuthology.orchestra.run.smithi156.stdout: "message": "", 2024-07-16T09:57:50.893 INFO:teuthology.orchestra.run.smithi156.stdout: "is_paused": false 2024-07-16T09:57:50.893 INFO:teuthology.orchestra.run.smithi156.stdout:} 2024-07-16T09:57:50.895 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:50.893+0000 7fce10468640 1 -- 172.21.15.156:0/3017168780 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7fcdf406e140 msgr2=0x7fcdf40705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:57:50.895 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:50.893+0000 7fce10468640 1 --2- 172.21.15.156:0/3017168780 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7fcdf406e140 0x7fcdf40705f0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fce04009af0 tx=0x7fce0400b040 comp rx=0 tx=0).stop 2024-07-16T09:57:50.895 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:50.893+0000 7fce10468640 1 -- 172.21.15.156:0/3017168780 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fce080fc510 msgr2=0x7fce0810d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:57:50.895 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:50.893+0000 7fce10468640 1 --2- 172.21.15.156:0/3017168780 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fce080fc510 0x7fce0810d4e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fcdf80048a0 tx=0x7fcdf8009da0 comp rx=0 tx=0).stop 2024-07-16T09:57:50.895 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:50.893+0000 7fce10468640 1 -- 172.21.15.156:0/3017168780 shutdown_connections 2024-07-16T09:57:50.895 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:50.893+0000 7fce10468640 1 --2- 172.21.15.156:0/3017168780 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7fcdf406e140 0x7fcdf40705f0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:57:50.895 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:50.893+0000 7fce10468640 1 --2- 172.21.15.156:0/3017168780 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fce080fc510 0x7fce0810d4e0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:57:50.895 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:50.893+0000 7fce10468640 1 --2- 172.21.15.156:0/3017168780 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fce080fb240 0x7fce0810cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:57:50.895 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:50.894+0000 7fce10468640 1 -- 172.21.15.156:0/3017168780 >> 172.21.15.156:0/3017168780 conn(0x7fce080f6a20 msgr2=0x7fce080f8140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:57:50.896 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:50.894+0000 7fce10468640 1 -- 172.21.15.156:0/3017168780 shutdown_connections 2024-07-16T09:57:50.896 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:50.894+0000 7fce10468640 1 -- 172.21.15.156:0/3017168780 wait complete. 2024-07-16T09:57:50.974 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:50.972+0000 7f0685ffb640 1 -- 172.21.15.156:0/4175124632 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0678032720 con 0x7f0688066520 2024-07-16T09:57:50.974 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:50.973+0000 7f068da0b640 1 -- 172.21.15.156:0/4175124632 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0688066520 msgr2=0x7f06880feff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:57:50.974 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:50.973+0000 7f068da0b640 1 --2- 172.21.15.156:0/4175124632 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0688066520 0x7f06880feff0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f06780048d0 tx=0x7f0678009da0 comp rx=0 tx=0).stop 2024-07-16T09:57:50.975 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:50.973+0000 7f068da0b640 1 -- 172.21.15.156:0/4175124632 shutdown_connections 2024-07-16T09:57:50.975 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:50.973+0000 7f068da0b640 1 --2- 172.21.15.156:0/4175124632 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f06880ff530 0x7f0688101910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:57:50.975 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:50.973+0000 7f068da0b640 1 --2- 172.21.15.156:0/4175124632 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0688066520 0x7f06880feff0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:57:50.975 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:50.973+0000 7f068da0b640 1 -- 172.21.15.156:0/4175124632 >> 172.21.15.156:0/4175124632 conn(0x7f06880f6a20 msgr2=0x7f06880f8e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:57:50.975 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:50.973+0000 7f068da0b640 1 -- 172.21.15.156:0/4175124632 shutdown_connections 2024-07-16T09:57:50.975 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:50.973+0000 7f068da0b640 1 -- 172.21.15.156:0/4175124632 wait complete. 2024-07-16T09:57:50.976 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:50.973+0000 7f068da0b640 1 Processor -- start 2024-07-16T09:57:50.977 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:50.973+0000 7f068da0b640 1 -- start start 2024-07-16T09:57:50.977 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:50.974+0000 7f068da0b640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0688066520 0x7f068810d060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:57:50.977 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:50.974+0000 7f068da0b640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f06880ff530 0x7f068810d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:57:50.977 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:50.974+0000 7f068da0b640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f068810dbf0 con 0x7f0688066520 2024-07-16T09:57:50.977 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:50.974+0000 7f068da0b640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f068810e410 con 0x7f06880ff530 2024-07-16T09:57:50.977 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:50.974+0000 7f0686ffd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0688066520 0x7f068810d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:57:50.977 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:50.974+0000 7f0686ffd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0688066520 0x7f068810d060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:39374/0 (socket says 172.21.15.156:39374) 2024-07-16T09:57:50.977 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:50.974+0000 7f0686ffd640 1 -- 172.21.15.156:0/1025514250 learned_addr learned my addr 172.21.15.156:0/1025514250 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T09:57:50.977 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:50.974+0000 7f06867fc640 1 --2- 172.21.15.156:0/1025514250 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f06880ff530 0x7f068810d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:57:50.977 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:50.974+0000 7f0686ffd640 1 -- 172.21.15.156:0/1025514250 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f06880ff530 msgr2=0x7f068810d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:57:50.977 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:50.974+0000 7f0686ffd640 1 --2- 172.21.15.156:0/1025514250 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f06880ff530 0x7f068810d5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:57:50.977 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:50.974+0000 7f0686ffd640 1 -- 172.21.15.156:0/1025514250 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0678005040 con 0x7f0688066520 2024-07-16T09:57:50.978 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:50.974+0000 7f0686ffd640 1 --2- 172.21.15.156:0/1025514250 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0688066520 0x7f068810d060 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f0678000c00 tx=0x7f0678009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:57:50.978 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:50.974+0000 7f067f7fe640 1 -- 172.21.15.156:0/1025514250 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0678038070 con 0x7f0688066520 2024-07-16T09:57:50.978 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:50.974+0000 7f067f7fe640 1 -- 172.21.15.156:0/1025514250 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0678008ee0 con 0x7f0688066520 2024-07-16T09:57:50.978 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:50.974+0000 7f068da0b640 1 -- 172.21.15.156:0/1025514250 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f068810e640 con 0x7f0688066520 2024-07-16T09:57:50.978 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:50.975+0000 7f067f7fe640 1 -- 172.21.15.156:0/1025514250 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0678033780 con 0x7f0688066520 2024-07-16T09:57:50.978 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:50.975+0000 7f068da0b640 1 -- 172.21.15.156:0/1025514250 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f068810eb80 con 0x7f0688066520 2024-07-16T09:57:50.978 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:50.975+0000 7f068da0b640 1 -- 172.21.15.156:0/1025514250 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06880fa6c0 con 0x7f0688066520 2024-07-16T09:57:50.979 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:50.976+0000 7f067f7fe640 1 -- 172.21.15.156:0/1025514250 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 36) v1 ==== 92251+0+0 (secure 0 0 0) 0x7f0678030020 con 0x7f0688066520 2024-07-16T09:57:50.979 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:50.976+0000 7f067f7fe640 1 --2- 172.21.15.156:0/1025514250 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f067406e140 0x7f06740705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:57:50.979 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:50.976+0000 7f067f7fe640 1 -- 172.21.15.156:0/1025514250 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f06780ad8b0 con 0x7f0688066520 2024-07-16T09:57:50.979 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:50.976+0000 7f06867fc640 1 --2- 172.21.15.156:0/1025514250 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f067406e140 0x7f06740705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:57:50.979 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:50.977+0000 7f06867fc640 1 --2- 172.21.15.156:0/1025514250 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f067406e140 0x7f06740705f0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f068810ce10 tx=0x7f067000b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:57:50.982 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:50.981+0000 7f067f7fe640 1 -- 172.21.15.156:0/1025514250 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187716 (secure 0 0 0) 0x7f0678076d30 con 0x7f0688066520 2024-07-16T09:57:51.236 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:51.234+0000 7f068da0b640 1 -- 172.21.15.156:0/1025514250 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f068804b530 con 0x7f0688066520 2024-07-16T09:57:51.236 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:51.235+0000 7f067f7fe640 1 -- 172.21.15.156:0/1025514250 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7f067807a3f0 con 0x7f0688066520 2024-07-16T09:57:51.236 INFO:teuthology.orchestra.run.smithi156.stdout:HEALTH_OK 2024-07-16T09:57:51.239 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:51.237+0000 7f068da0b640 1 -- 172.21.15.156:0/1025514250 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f067406e140 msgr2=0x7f06740705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:57:51.239 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:51.237+0000 7f068da0b640 1 --2- 172.21.15.156:0/1025514250 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f067406e140 0x7f06740705f0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f068810ce10 tx=0x7f067000b040 comp rx=0 tx=0).stop 2024-07-16T09:57:51.239 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:51.237+0000 7f068da0b640 1 -- 172.21.15.156:0/1025514250 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0688066520 msgr2=0x7f068810d060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:57:51.239 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:51.237+0000 7f068da0b640 1 --2- 172.21.15.156:0/1025514250 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0688066520 0x7f068810d060 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f0678000c00 tx=0x7f0678009da0 comp rx=0 tx=0).stop 2024-07-16T09:57:51.239 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:51.237+0000 7f068da0b640 1 -- 172.21.15.156:0/1025514250 shutdown_connections 2024-07-16T09:57:51.239 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:51.237+0000 7f068da0b640 1 --2- 172.21.15.156:0/1025514250 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f067406e140 0x7f06740705f0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:57:51.239 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:51.237+0000 7f068da0b640 1 --2- 172.21.15.156:0/1025514250 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f06880ff530 0x7f068810d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:57:51.240 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:51.237+0000 7f068da0b640 1 --2- 172.21.15.156:0/1025514250 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0688066520 0x7f068810d060 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:57:51.240 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:51.237+0000 7f068da0b640 1 -- 172.21.15.156:0/1025514250 >> 172.21.15.156:0/1025514250 conn(0x7f06880f6a20 msgr2=0x7f06880f6e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:57:51.240 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:51.238+0000 7f068da0b640 1 -- 172.21.15.156:0/1025514250 shutdown_connections 2024-07-16T09:57:51.240 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:57:51.238+0000 7f068da0b640 1 -- 172.21.15.156:0/1025514250 wait complete. 2024-07-16T09:57:51.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:51 smithi176 ceph-mon[36626]: [16/Jul/2024:09:57:49] ENGINE Bus STARTING 2024-07-16T09:57:51.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:51 smithi176 ceph-mon[36626]: from='client.24587 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-16T09:57:51.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:51 smithi176 ceph-mon[36626]: [16/Jul/2024:09:57:49] ENGINE Serving on http://172.21.15.156:8765 2024-07-16T09:57:51.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:51 smithi176 ceph-mon[36626]: [16/Jul/2024:09:57:49] ENGINE Serving on https://172.21.15.156:7150 2024-07-16T09:57:51.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:51 smithi176 ceph-mon[36626]: [16/Jul/2024:09:57:49] ENGINE Bus STARTED 2024-07-16T09:57:51.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:51 smithi176 ceph-mon[36626]: [16/Jul/2024:09:57:49] ENGINE Error in HTTPServer.serve 2024-07-16T09:57:51.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: Traceback (most recent call last): 2024-07-16T09:57:51.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-07-16T09:57:51.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: self._connections.run(self.expiration_interval) 2024-07-16T09:57:51.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-07-16T09:57:51.595 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: self._run(expiration_interval) 2024-07-16T09:57:51.595 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-07-16T09:57:51.595 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-07-16T09:57:51.595 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-07-16T09:57:51.595 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-07-16T09:57:51.595 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-07-16T09:57:51.595 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: s = self.context.wrap_socket( 2024-07-16T09:57:51.595 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-07-16T09:57:51.595 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: return self.sslsocket_class._create( 2024-07-16T09:57:51.595 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-07-16T09:57:51.595 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: self.do_handshake() 2024-07-16T09:57:51.595 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-07-16T09:57:51.595 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: self._sslobj.do_handshake() 2024-07-16T09:57:51.596 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-07-16T09:57:51.596 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:51 smithi176 ceph-mon[36626]: from='client.34136 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-07-16T09:57:51.596 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:51 smithi176 ceph-mon[36626]: pgmap v4: 65 pgs: 65 active+clean; 253 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-07-16T09:57:51.596 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:51 smithi176 ceph-mon[36626]: mgrmap e36: smithi156.zkvylv(active, since 2s), standbys: smithi176.dqbrvx 2024-07-16T09:57:51.596 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:51 smithi176 ceph-mon[36626]: from='client.? 172.21.15.156:0/1691039189' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-07-16T09:57:51.596 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:51 smithi176 ceph-mon[36626]: from='client.? 172.21.15.156:0/1025514250' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-07-16T09:57:51.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:51 smithi156 ceph-mon[93066]: [16/Jul/2024:09:57:49] ENGINE Bus STARTING 2024-07-16T09:57:51.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:51 smithi156 ceph-mon[93066]: from='client.24587 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-16T09:57:51.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:51 smithi156 ceph-mon[93066]: [16/Jul/2024:09:57:49] ENGINE Serving on http://172.21.15.156:8765 2024-07-16T09:57:51.676 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:51 smithi156 ceph-mon[93066]: [16/Jul/2024:09:57:49] ENGINE Serving on https://172.21.15.156:7150 2024-07-16T09:57:51.676 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:51 smithi156 ceph-mon[93066]: [16/Jul/2024:09:57:49] ENGINE Bus STARTED 2024-07-16T09:57:51.676 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:51 smithi156 ceph-mon[93066]: [16/Jul/2024:09:57:49] ENGINE Error in HTTPServer.serve 2024-07-16T09:57:51.676 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: Traceback (most recent call last): 2024-07-16T09:57:51.676 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-07-16T09:57:51.676 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: self._connections.run(self.expiration_interval) 2024-07-16T09:57:51.677 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-07-16T09:57:51.677 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: self._run(expiration_interval) 2024-07-16T09:57:51.677 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-07-16T09:57:51.677 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-07-16T09:57:51.677 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-07-16T09:57:51.677 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-07-16T09:57:51.677 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-07-16T09:57:51.677 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: s = self.context.wrap_socket( 2024-07-16T09:57:51.677 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-07-16T09:57:51.677 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: return self.sslsocket_class._create( 2024-07-16T09:57:51.678 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-07-16T09:57:51.678 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: self.do_handshake() 2024-07-16T09:57:51.678 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-07-16T09:57:51.678 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: self._sslobj.do_handshake() 2024-07-16T09:57:51.678 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-07-16T09:57:51.678 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:51 smithi156 ceph-mon[93066]: from='client.34136 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-07-16T09:57:51.679 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:51 smithi156 ceph-mon[93066]: pgmap v4: 65 pgs: 65 active+clean; 253 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-07-16T09:57:51.679 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:51 smithi156 ceph-mon[93066]: mgrmap e36: smithi156.zkvylv(active, since 2s), standbys: smithi176.dqbrvx 2024-07-16T09:57:51.679 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:51 smithi156 ceph-mon[93066]: from='client.? 172.21.15.156:0/1691039189' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-07-16T09:57:51.679 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:51 smithi156 ceph-mon[93066]: from='client.? 172.21.15.156:0/1025514250' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-07-16T09:57:52.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:52 smithi176 ceph-mon[36626]: from='client.34152 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-16T09:57:52.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:52 smithi176 ceph-mon[36626]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T09:57:52.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:52 smithi176 ceph-mon[36626]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T09:57:52.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:52 smithi176 ceph-mon[36626]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T09:57:52.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:52 smithi176 ceph-mon[36626]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T09:57:52.620 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:52 smithi156 ceph-mon[93066]: from='client.34152 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-16T09:57:52.620 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:52 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T09:57:52.620 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:52 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T09:57:52.621 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:52 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T09:57:52.621 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:52 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T09:57:53.424 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:53 smithi156 ceph-mon[93066]: pgmap v5: 65 pgs: 65 active+clean; 253 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-07-16T09:57:53.424 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:53 smithi156 ceph-mon[93066]: mgrmap e37: smithi156.zkvylv(active, since 4s), standbys: smithi176.dqbrvx 2024-07-16T09:57:53.573 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:53 smithi176 ceph-mon[36626]: pgmap v5: 65 pgs: 65 active+clean; 253 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-07-16T09:57:53.573 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:53 smithi176 ceph-mon[36626]: mgrmap e37: smithi156.zkvylv(active, since 4s), standbys: smithi176.dqbrvx 2024-07-16T09:57:55.030 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:54 smithi176 ceph-mon[36626]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T09:57:55.030 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:54 smithi176 ceph-mon[36626]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T09:57:55.031 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:54 smithi176 ceph-mon[36626]: pgmap v6: 65 pgs: 65 active+clean; 253 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-07-16T09:57:55.031 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:54 smithi176 ceph-mon[36626]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T09:57:55.031 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:54 smithi176 ceph-mon[36626]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T09:57:55.174 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:54 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T09:57:55.174 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:54 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T09:57:55.175 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:54 smithi156 ceph-mon[93066]: pgmap v6: 65 pgs: 65 active+clean; 253 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-07-16T09:57:55.175 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:54 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T09:57:55.175 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:54 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T09:57:56.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:56 smithi176 ceph-mon[36626]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T09:57:56.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:56 smithi176 ceph-mon[36626]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T09:57:56.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:56 smithi176 ceph-mon[36626]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-16T09:57:56.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:56 smithi176 ceph-mon[36626]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-16T09:57:56.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:56 smithi176 ceph-mon[36626]: Updating smithi156:/etc/ceph/ceph.conf 2024-07-16T09:57:56.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:56 smithi176 ceph-mon[36626]: Updating smithi176:/etc/ceph/ceph.conf 2024-07-16T09:57:56.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:56 smithi176 ceph-mon[36626]: Updating smithi156:/var/lib/ceph/26c91c26-4358-11ef-bcad-c7b262605968/config/ceph.conf 2024-07-16T09:57:56.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:56 smithi176 ceph-mon[36626]: Updating smithi176:/var/lib/ceph/26c91c26-4358-11ef-bcad-c7b262605968/config/ceph.conf 2024-07-16T09:57:56.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:56 smithi176 ceph-mon[36626]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T09:57:56.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:56 smithi176 ceph-mon[36626]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T09:57:56.345 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:56 smithi176 ceph-mon[36626]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T09:57:56.345 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:56 smithi176 ceph-mon[36626]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T09:57:56.345 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:56 smithi176 ceph-mon[36626]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T09:57:56.345 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:56 smithi176 ceph-mon[36626]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-16T09:57:56.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:56 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T09:57:56.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:56 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T09:57:56.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:56 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-16T09:57:56.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:56 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-16T09:57:56.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:56 smithi156 ceph-mon[93066]: Updating smithi156:/etc/ceph/ceph.conf 2024-07-16T09:57:56.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:56 smithi156 ceph-mon[93066]: Updating smithi176:/etc/ceph/ceph.conf 2024-07-16T09:57:56.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:56 smithi156 ceph-mon[93066]: Updating smithi156:/var/lib/ceph/26c91c26-4358-11ef-bcad-c7b262605968/config/ceph.conf 2024-07-16T09:57:56.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:56 smithi156 ceph-mon[93066]: Updating smithi176:/var/lib/ceph/26c91c26-4358-11ef-bcad-c7b262605968/config/ceph.conf 2024-07-16T09:57:56.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:56 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T09:57:56.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:56 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T09:57:56.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:56 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T09:57:56.426 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:56 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T09:57:56.426 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:56 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T09:57:56.426 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:56 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-16T09:57:57.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:57 smithi176 ceph-mon[36626]: Updating smithi156:/etc/ceph/ceph.client.admin.keyring 2024-07-16T09:57:57.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:57 smithi176 ceph-mon[36626]: Updating smithi176:/etc/ceph/ceph.client.admin.keyring 2024-07-16T09:57:57.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:57 smithi176 ceph-mon[36626]: Updating smithi156:/var/lib/ceph/26c91c26-4358-11ef-bcad-c7b262605968/config/ceph.client.admin.keyring 2024-07-16T09:57:57.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:57 smithi176 ceph-mon[36626]: Updating smithi176:/var/lib/ceph/26c91c26-4358-11ef-bcad-c7b262605968/config/ceph.client.admin.keyring 2024-07-16T09:57:57.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:57 smithi176 ceph-mon[36626]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T09:57:57.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:57 smithi176 ceph-mon[36626]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-07-16T09:57:57.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:57 smithi176 ceph-mon[36626]: pgmap v7: 65 pgs: 65 active+clean; 253 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 31 KiB/s rd, 0 B/s wr, 19 op/s 2024-07-16T09:57:57.424 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:57 smithi156 ceph-mon[93066]: Updating smithi156:/etc/ceph/ceph.client.admin.keyring 2024-07-16T09:57:57.424 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:57 smithi156 ceph-mon[93066]: Updating smithi176:/etc/ceph/ceph.client.admin.keyring 2024-07-16T09:57:57.424 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:57 smithi156 ceph-mon[93066]: Updating smithi156:/var/lib/ceph/26c91c26-4358-11ef-bcad-c7b262605968/config/ceph.client.admin.keyring 2024-07-16T09:57:57.424 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:57 smithi156 ceph-mon[93066]: Updating smithi176:/var/lib/ceph/26c91c26-4358-11ef-bcad-c7b262605968/config/ceph.client.admin.keyring 2024-07-16T09:57:57.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:57 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T09:57:57.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:57 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-07-16T09:57:57.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:57 smithi156 ceph-mon[93066]: pgmap v7: 65 pgs: 65 active+clean; 253 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 31 KiB/s rd, 0 B/s wr, 19 op/s 2024-07-16T09:57:58.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:58 smithi156 ceph-mon[93066]: Upgrade: Updating mon.smithi176 2024-07-16T09:57:58.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:58 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T09:57:58.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:58 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-07-16T09:57:58.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:58 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-07-16T09:57:58.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:58 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-16T09:57:58.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:58 smithi156 ceph-mon[93066]: Deploying daemon mon.smithi176 on smithi176 2024-07-16T09:57:58.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:57:58 smithi156 ceph-mon[93066]: pgmap v8: 65 pgs: 65 active+clean; 253 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 24 KiB/s rd, 0 B/s wr, 15 op/s 2024-07-16T09:57:58.686 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:58 smithi176 ceph-mon[36626]: Upgrade: Updating mon.smithi176 2024-07-16T09:57:58.686 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:58 smithi176 ceph-mon[36626]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T09:57:58.687 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:58 smithi176 ceph-mon[36626]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-07-16T09:57:58.687 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:58 smithi176 ceph-mon[36626]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-07-16T09:57:58.687 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:58 smithi176 ceph-mon[36626]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-16T09:57:58.687 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:58 smithi176 ceph-mon[36626]: Deploying daemon mon.smithi176 on smithi176 2024-07-16T09:57:58.687 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:58 smithi176 ceph-mon[36626]: pgmap v8: 65 pgs: 65 active+clean; 253 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 24 KiB/s rd, 0 B/s wr, 15 op/s 2024-07-16T09:57:58.963 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:58 smithi176 systemd[1]: Stopping Ceph mon.smithi176 for 26c91c26-4358-11ef-bcad-c7b262605968... 2024-07-16T09:57:59.290 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:59 smithi176 ceph-26c91c26-4358-11ef-bcad-c7b262605968-mon-smithi176[36609]: 2024-07-16T09:57:59.206+0000 7fbfe811d640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi176 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-07-16T09:57:59.290 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:59 smithi176 ceph-26c91c26-4358-11ef-bcad-c7b262605968-mon-smithi176[36609]: 2024-07-16T09:57:59.206+0000 7fbfe811d640 -1 mon.smithi176@1(peon) e2 *** Got Signal Terminated *** 2024-07-16T09:57:59.573 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:59 smithi176 podman[81530]: 2024-07-16 09:57:59.289006404 +0000 UTC m=+0.180122517 container died 3a293ac6ef01ce4d72aa929b26a2eca18f94ebeee9ffeca9867f45c5e5ac172e (image=quay.ceph.io/ceph-ci/ceph@sha256:7b86ef6bffe0efc5f1aefc9fef8c854a67a511c50993acee3ac684f23bde83df, name=ceph-26c91c26-4358-11ef-bcad-c7b262605968-mon-smithi176, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=quincy-e6231b0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240709) 2024-07-16T09:58:00.109 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:59 smithi176 podman[81530]: 2024-07-16 09:57:59.849214132 +0000 UTC m=+0.740330236 container cleanup 3a293ac6ef01ce4d72aa929b26a2eca18f94ebeee9ffeca9867f45c5e5ac172e (image=quay.ceph.io/ceph-ci/ceph@sha256:7b86ef6bffe0efc5f1aefc9fef8c854a67a511c50993acee3ac684f23bde83df, name=ceph-26c91c26-4358-11ef-bcad-c7b262605968-mon-smithi176, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=quincy-e6231b0, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240709, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=) 2024-07-16T09:58:00.109 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:57:59 smithi176 bash[81530]: ceph-26c91c26-4358-11ef-bcad-c7b262605968-mon-smithi176 2024-07-16T09:58:00.481 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 podman[81542]: 2024-07-16 09:58:00.108771698 +0000 UTC m=+0.821121697 container remove 3a293ac6ef01ce4d72aa929b26a2eca18f94ebeee9ffeca9867f45c5e5ac172e (image=quay.ceph.io/ceph-ci/ceph@sha256:7b86ef6bffe0efc5f1aefc9fef8c854a67a511c50993acee3ac684f23bde83df, name=ceph-26c91c26-4358-11ef-bcad-c7b262605968-mon-smithi176, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240709, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=quincy-e6231b0, io.buildah.version=1.36.0, org.label-schema.license=GPLv2) 2024-07-16T09:58:00.482 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 systemd[1]: ceph-26c91c26-4358-11ef-bcad-c7b262605968@mon.smithi176.service: Deactivated successfully. 2024-07-16T09:58:00.482 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 systemd[1]: Stopped Ceph mon.smithi176 for 26c91c26-4358-11ef-bcad-c7b262605968. 2024-07-16T09:58:00.482 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 systemd[1]: ceph-26c91c26-4358-11ef-bcad-c7b262605968@mon.smithi176.service: Consumed 5.740s CPU time. 2024-07-16T09:58:00.768 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 systemd[1]: Starting Ceph mon.smithi176 for 26c91c26-4358-11ef-bcad-c7b262605968... 2024-07-16T09:58:00.768 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 podman[81670]: 2024-07-16 09:58:00.693086503 +0000 UTC m=+0.110284184 container create ef92814cec2bc58325935136a26d0a5a2db0964b14e066585628006ac725759b (image=quay.ceph.io/ceph-ci/ceph@sha256:c15ec3aa8e6c0a9c12ee24ff79f3e7dd4d01b3d1a702dfeb7d7fff9b0811a393, name=ceph-26c91c26-4358-11ef-bcad-c7b262605968-mon-smithi176, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.36.0, RELEASE=wip-adk4-testing-2024-07-15-1832-squid-a6ca461, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240709, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-07-16T09:58:00.768 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 podman[81670]: 2024-07-16 09:58:00.600126571 +0000 UTC m=+0.017324215 image pull 16f7feb043422ce37a2471403d4bc52f550d49cef4b677482112fcc0db3f67ab quay.ceph.io/ceph-ci/ceph@sha256:c15ec3aa8e6c0a9c12ee24ff79f3e7dd4d01b3d1a702dfeb7d7fff9b0811a393 2024-07-16T09:58:01.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 podman[81670]: 2024-07-16 09:58:00.86226921 +0000 UTC m=+0.279466842 container init ef92814cec2bc58325935136a26d0a5a2db0964b14e066585628006ac725759b (image=quay.ceph.io/ceph-ci/ceph@sha256:c15ec3aa8e6c0a9c12ee24ff79f3e7dd4d01b3d1a702dfeb7d7fff9b0811a393, name=ceph-26c91c26-4358-11ef-bcad-c7b262605968-mon-smithi176, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux , RELEASE=wip-adk4-testing-2024-07-15-1832-squid-a6ca461, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240709, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True) 2024-07-16T09:58:01.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 podman[81670]: 2024-07-16 09:58:00.86546016 +0000 UTC m=+0.282657795 container start ef92814cec2bc58325935136a26d0a5a2db0964b14e066585628006ac725759b (image=quay.ceph.io/ceph-ci/ceph@sha256:c15ec3aa8e6c0a9c12ee24ff79f3e7dd4d01b3d1a702dfeb7d7fff9b0811a393, name=ceph-26c91c26-4358-11ef-bcad-c7b262605968-mon-smithi176, io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240709, GIT_CLEAN=True, ceph=True, RELEASE=wip-adk4-testing-2024-07-15-1832-squid-a6ca461, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-07-16T09:58:01.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 ceph-mon[81689]: set uid:gid to 167:167 (ceph:ceph) 2024-07-16T09:58:01.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 ceph-mon[81689]: ceph version 19.0.0-3495-ga6ca461b (a6ca461be7cc1aff3658c2c6df4cff975a24449a) squid (rc), process ceph-mon, pid 2 2024-07-16T09:58:01.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 ceph-mon[81689]: pidfile_write: ignore empty --pid-file 2024-07-16T09:58:01.095 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 ceph-mon[81689]: load: jerasure load: lrc 2024-07-16T09:58:01.095 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 ceph-mon[81689]: rocksdb: RocksDB version: 7.9.2 2024-07-16T09:58:01.095 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 ceph-mon[81689]: rocksdb: Git sha 0 2024-07-16T09:58:01.095 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 ceph-mon[81689]: rocksdb: Compile date 2024-07-15 23:29:57 2024-07-16T09:58:01.095 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 ceph-mon[81689]: rocksdb: DB SUMMARY 2024-07-16T09:58:01.095 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 ceph-mon[81689]: rocksdb: DB Session ID: V4RLS32CHM6TZ0TWCQU4 2024-07-16T09:58:01.095 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 ceph-mon[81689]: rocksdb: CURRENT file: CURRENT 2024-07-16T09:58:01.095 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 ceph-mon[81689]: rocksdb: IDENTITY file: IDENTITY 2024-07-16T09:58:01.095 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 ceph-mon[81689]: rocksdb: MANIFEST file: MANIFEST-000009 size: 897 Bytes 2024-07-16T09:58:01.095 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 ceph-mon[81689]: rocksdb: SST files in /var/lib/ceph/mon/ceph-smithi176/store.db dir, Total Num: 1, files: 000024.sst 2024-07-16T09:58:01.095 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 ceph-mon[81689]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-smithi176/store.db: 000022.log size: 5480738 ; 2024-07-16T09:58:01.095 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 ceph-mon[81689]: rocksdb: Options.error_if_exists: 0 2024-07-16T09:58:01.095 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 ceph-mon[81689]: rocksdb: Options.create_if_missing: 0 2024-07-16T09:58:01.096 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 ceph-mon[81689]: rocksdb: Options.paranoid_checks: 1 2024-07-16T09:58:01.096 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 ceph-mon[81689]: rocksdb: Options.flush_verify_memtable_count: 1 2024-07-16T09:58:01.096 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 ceph-mon[81689]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-07-16T09:58:01.096 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 ceph-mon[81689]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-07-16T09:58:01.096 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 ceph-mon[81689]: rocksdb: Options.env: 0x563503d58740 2024-07-16T09:58:01.096 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 ceph-mon[81689]: rocksdb: Options.fs: PosixFileSystem 2024-07-16T09:58:01.096 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 ceph-mon[81689]: rocksdb: Options.info_log: 0x563504cf96a0 2024-07-16T09:58:01.096 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 ceph-mon[81689]: rocksdb: Options.max_file_opening_threads: 16 2024-07-16T09:58:01.097 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 ceph-mon[81689]: rocksdb: Options.statistics: (nil) 2024-07-16T09:58:01.097 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 ceph-mon[81689]: rocksdb: Options.use_fsync: 0 2024-07-16T09:58:01.098 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 ceph-mon[81689]: rocksdb: Options.max_log_file_size: 0 2024-07-16T09:58:01.098 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 ceph-mon[81689]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-07-16T09:58:01.099 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 ceph-mon[81689]: rocksdb: Options.log_file_time_to_roll: 0 2024-07-16T09:58:01.099 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 ceph-mon[81689]: rocksdb: Options.keep_log_file_num: 1000 2024-07-16T09:58:01.099 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 ceph-mon[81689]: rocksdb: Options.recycle_log_file_num: 0 2024-07-16T09:58:01.099 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 ceph-mon[81689]: rocksdb: Options.allow_fallocate: 1 2024-07-16T09:58:01.099 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 ceph-mon[81689]: rocksdb: Options.allow_mmap_reads: 0 2024-07-16T09:58:01.099 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 ceph-mon[81689]: rocksdb: Options.allow_mmap_writes: 0 2024-07-16T09:58:01.099 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 ceph-mon[81689]: rocksdb: Options.use_direct_reads: 0 2024-07-16T09:58:01.099 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 ceph-mon[81689]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-07-16T09:58:01.099 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 ceph-mon[81689]: rocksdb: Options.create_missing_column_families: 0 2024-07-16T09:58:01.099 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 ceph-mon[81689]: rocksdb: Options.db_log_dir: 2024-07-16T09:58:01.099 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 ceph-mon[81689]: rocksdb: Options.wal_dir: 2024-07-16T09:58:01.099 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 ceph-mon[81689]: rocksdb: Options.table_cache_numshardbits: 6 2024-07-16T09:58:01.100 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 ceph-mon[81689]: rocksdb: Options.WAL_ttl_seconds: 0 2024-07-16T09:58:01.100 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 ceph-mon[81689]: rocksdb: Options.WAL_size_limit_MB: 0 2024-07-16T09:58:01.100 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 ceph-mon[81689]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-07-16T09:58:01.100 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 ceph-mon[81689]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-07-16T09:58:01.100 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 ceph-mon[81689]: rocksdb: Options.is_fd_close_on_exec: 1 2024-07-16T09:58:01.100 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 ceph-mon[81689]: rocksdb: Options.advise_random_on_open: 1 2024-07-16T09:58:01.100 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 ceph-mon[81689]: rocksdb: Options.db_write_buffer_size: 0 2024-07-16T09:58:01.100 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 ceph-mon[81689]: rocksdb: Options.write_buffer_manager: 0x563504d08dc0 2024-07-16T09:58:01.100 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 ceph-mon[81689]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-07-16T09:58:01.100 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 ceph-mon[81689]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-07-16T09:58:01.100 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 ceph-mon[81689]: rocksdb: Options.use_adaptive_mutex: 0 2024-07-16T09:58:01.100 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 ceph-mon[81689]: rocksdb: Options.rate_limiter: (nil) 2024-07-16T09:58:01.101 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 ceph-mon[81689]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-07-16T09:58:01.101 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 ceph-mon[81689]: rocksdb: Options.wal_recovery_mode: 2 2024-07-16T09:58:01.101 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 ceph-mon[81689]: rocksdb: Options.enable_thread_tracking: 0 2024-07-16T09:58:01.101 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 ceph-mon[81689]: rocksdb: Options.enable_pipelined_write: 0 2024-07-16T09:58:01.101 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 ceph-mon[81689]: rocksdb: Options.unordered_write: 0 2024-07-16T09:58:01.101 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 ceph-mon[81689]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-07-16T09:58:01.101 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 ceph-mon[81689]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-07-16T09:58:01.101 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 ceph-mon[81689]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-07-16T09:58:01.101 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 ceph-mon[81689]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-07-16T09:58:01.101 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 ceph-mon[81689]: rocksdb: Options.row_cache: None 2024-07-16T09:58:01.101 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 ceph-mon[81689]: rocksdb: Options.wal_filter: None 2024-07-16T09:58:01.101 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 ceph-mon[81689]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-07-16T09:58:01.101 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 ceph-mon[81689]: rocksdb: Options.allow_ingest_behind: 0 2024-07-16T09:58:01.102 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 ceph-mon[81689]: rocksdb: Options.two_write_queues: 0 2024-07-16T09:58:01.102 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 ceph-mon[81689]: rocksdb: Options.manual_wal_flush: 0 2024-07-16T09:58:01.102 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 ceph-mon[81689]: rocksdb: Options.wal_compression: 0 2024-07-16T09:58:01.102 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 ceph-mon[81689]: rocksdb: Options.atomic_flush: 0 2024-07-16T09:58:01.102 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 ceph-mon[81689]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-07-16T09:58:01.102 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 ceph-mon[81689]: rocksdb: Options.persist_stats_to_disk: 0 2024-07-16T09:58:01.102 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 ceph-mon[81689]: rocksdb: Options.write_dbid_to_manifest: 0 2024-07-16T09:58:01.102 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 ceph-mon[81689]: rocksdb: Options.log_readahead_size: 0 2024-07-16T09:58:01.102 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 ceph-mon[81689]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-07-16T09:58:01.103 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 ceph-mon[81689]: rocksdb: Options.best_efforts_recovery: 0 2024-07-16T09:58:01.103 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 ceph-mon[81689]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-07-16T09:58:01.103 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 ceph-mon[81689]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-07-16T09:58:01.103 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 ceph-mon[81689]: rocksdb: Options.allow_data_in_errors: 0 2024-07-16T09:58:01.103 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 ceph-mon[81689]: rocksdb: Options.db_host_id: __hostname__ 2024-07-16T09:58:01.103 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 ceph-mon[81689]: rocksdb: Options.enforce_single_del_contracts: true 2024-07-16T09:58:01.103 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 ceph-mon[81689]: rocksdb: Options.max_background_jobs: 2 2024-07-16T09:58:01.103 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 ceph-mon[81689]: rocksdb: Options.max_background_compactions: -1 2024-07-16T09:58:01.103 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 ceph-mon[81689]: rocksdb: Options.max_subcompactions: 1 2024-07-16T09:58:01.103 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 ceph-mon[81689]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-07-16T09:58:01.103 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 ceph-mon[81689]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-07-16T09:58:01.103 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 ceph-mon[81689]: rocksdb: Options.delayed_write_rate : 16777216 2024-07-16T09:58:01.103 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 ceph-mon[81689]: rocksdb: Options.max_total_wal_size: 0 2024-07-16T09:58:01.104 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 ceph-mon[81689]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-07-16T09:58:01.104 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 ceph-mon[81689]: rocksdb: Options.stats_dump_period_sec: 600 2024-07-16T09:58:01.104 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 ceph-mon[81689]: rocksdb: Options.stats_persist_period_sec: 600 2024-07-16T09:58:01.104 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 ceph-mon[81689]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-07-16T09:58:01.104 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 ceph-mon[81689]: rocksdb: Options.max_open_files: -1 2024-07-16T09:58:01.104 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 ceph-mon[81689]: rocksdb: Options.bytes_per_sync: 0 2024-07-16T09:58:01.104 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 ceph-mon[81689]: rocksdb: Options.wal_bytes_per_sync: 0 2024-07-16T09:58:01.104 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 ceph-mon[81689]: rocksdb: Options.strict_bytes_per_sync: 0 2024-07-16T09:58:01.104 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 ceph-mon[81689]: rocksdb: Options.compaction_readahead_size: 0 2024-07-16T09:58:01.104 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 ceph-mon[81689]: rocksdb: Options.max_background_flushes: -1 2024-07-16T09:58:01.104 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 ceph-mon[81689]: rocksdb: Compression algorithms supported: 2024-07-16T09:58:01.104 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 ceph-mon[81689]: rocksdb: kZSTD supported: 0 2024-07-16T09:58:01.104 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 ceph-mon[81689]: rocksdb: kXpressCompression supported: 0 2024-07-16T09:58:01.104 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 ceph-mon[81689]: rocksdb: kBZip2Compression supported: 0 2024-07-16T09:58:01.105 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 ceph-mon[81689]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-07-16T09:58:01.105 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 ceph-mon[81689]: rocksdb: kLZ4Compression supported: 1 2024-07-16T09:58:01.105 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 ceph-mon[81689]: rocksdb: kZlibCompression supported: 1 2024-07-16T09:58:01.105 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 ceph-mon[81689]: rocksdb: kLZ4HCCompression supported: 1 2024-07-16T09:58:01.105 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 ceph-mon[81689]: rocksdb: kSnappyCompression supported: 1 2024-07-16T09:58:01.105 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 ceph-mon[81689]: rocksdb: Fast CRC32 supported: Supported on x86 2024-07-16T09:58:01.105 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 ceph-mon[81689]: rocksdb: DMutex implementation: pthread_mutex_t 2024-07-16T09:58:01.105 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 ceph-mon[81689]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-smithi176/store.db/MANIFEST-000009 2024-07-16T09:58:01.105 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 ceph-mon[81689]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-07-16T09:58:01.105 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 ceph-mon[81689]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-07-16T09:58:01.105 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 ceph-mon[81689]: rocksdb: Options.merge_operator: 2024-07-16T09:58:01.105 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 ceph-mon[81689]: rocksdb: Options.compaction_filter: None 2024-07-16T09:58:01.105 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 ceph-mon[81689]: rocksdb: Options.compaction_filter_factory: None 2024-07-16T09:58:01.106 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 ceph-mon[81689]: rocksdb: Options.sst_partitioner_factory: None 2024-07-16T09:58:01.106 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 ceph-mon[81689]: rocksdb: Options.memtable_factory: SkipListFactory 2024-07-16T09:58:01.106 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 ceph-mon[81689]: rocksdb: Options.table_factory: BlockBasedTable 2024-07-16T09:58:01.106 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 ceph-mon[81689]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x563504cf93a0) 2024-07-16T09:58:01.106 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: cache_index_and_filter_blocks: 1 2024-07-16T09:58:01.106 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-07-16T09:58:01.106 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-07-16T09:58:01.106 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: pin_top_level_index_and_filter: 1 2024-07-16T09:58:01.106 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: index_type: 0 2024-07-16T09:58:01.106 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: data_block_index_type: 0 2024-07-16T09:58:01.106 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: index_shortening: 1 2024-07-16T09:58:01.106 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: data_block_hash_table_util_ratio: 0.750000 2024-07-16T09:58:01.107 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: checksum: 4 2024-07-16T09:58:01.107 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: no_block_cache: 0 2024-07-16T09:58:01.107 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: block_cache: 0x563504ced090 2024-07-16T09:58:01.107 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: block_cache_name: BinnedLRUCache 2024-07-16T09:58:01.107 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: block_cache_options: 2024-07-16T09:58:01.107 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: capacity : 536870912 2024-07-16T09:58:01.107 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: num_shard_bits : 4 2024-07-16T09:58:01.107 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: strict_capacity_limit : 0 2024-07-16T09:58:01.107 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: high_pri_pool_ratio: 0.000 2024-07-16T09:58:01.107 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: block_cache_compressed: (nil) 2024-07-16T09:58:01.107 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: persistent_cache: (nil) 2024-07-16T09:58:01.107 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: block_size: 4096 2024-07-16T09:58:01.107 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: block_size_deviation: 10 2024-07-16T09:58:01.107 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: block_restart_interval: 16 2024-07-16T09:58:01.108 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: index_block_restart_interval: 1 2024-07-16T09:58:01.108 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: metadata_block_size: 4096 2024-07-16T09:58:01.108 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: partition_filters: 0 2024-07-16T09:58:01.108 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: use_delta_encoding: 1 2024-07-16T09:58:01.108 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: filter_policy: bloomfilter 2024-07-16T09:58:01.108 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: whole_key_filtering: 1 2024-07-16T09:58:01.108 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: verify_compression: 0 2024-07-16T09:58:01.108 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: read_amp_bytes_per_bit: 0 2024-07-16T09:58:01.108 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: format_version: 5 2024-07-16T09:58:01.108 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: enable_index_compression: 1 2024-07-16T09:58:01.108 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: block_align: 0 2024-07-16T09:58:01.108 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: max_auto_readahead_size: 262144 2024-07-16T09:58:01.108 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: prepopulate_block_cache: 0 2024-07-16T09:58:01.109 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: initial_auto_readahead_size: 8192 2024-07-16T09:58:01.109 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: num_file_reads_for_auto_readahead: 2 2024-07-16T09:58:01.109 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 ceph-mon[81689]: rocksdb: Options.write_buffer_size: 33554432 2024-07-16T09:58:01.109 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 ceph-mon[81689]: rocksdb: Options.max_write_buffer_number: 2 2024-07-16T09:58:01.109 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 ceph-mon[81689]: rocksdb: Options.compression: NoCompression 2024-07-16T09:58:01.109 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 ceph-mon[81689]: rocksdb: Options.bottommost_compression: Disabled 2024-07-16T09:58:01.109 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 ceph-mon[81689]: rocksdb: Options.prefix_extractor: nullptr 2024-07-16T09:58:01.109 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 ceph-mon[81689]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-07-16T09:58:01.109 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 ceph-mon[81689]: rocksdb: Options.num_levels: 7 2024-07-16T09:58:01.109 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 ceph-mon[81689]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-07-16T09:58:01.109 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 ceph-mon[81689]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-07-16T09:58:01.109 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 ceph-mon[81689]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-07-16T09:58:01.109 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 ceph-mon[81689]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-07-16T09:58:01.109 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 ceph-mon[81689]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-07-16T09:58:01.110 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 ceph-mon[81689]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-07-16T09:58:01.110 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 ceph-mon[81689]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-07-16T09:58:01.110 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 ceph-mon[81689]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-07-16T09:58:01.110 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 ceph-mon[81689]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-07-16T09:58:01.110 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 ceph-mon[81689]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-07-16T09:58:01.110 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 ceph-mon[81689]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-07-16T09:58:01.110 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 ceph-mon[81689]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-07-16T09:58:01.110 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 ceph-mon[81689]: rocksdb: Options.compression_opts.window_bits: -14 2024-07-16T09:58:01.110 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 ceph-mon[81689]: rocksdb: Options.compression_opts.level: 32767 2024-07-16T09:58:01.110 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 ceph-mon[81689]: rocksdb: Options.compression_opts.strategy: 0 2024-07-16T09:58:01.110 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 ceph-mon[81689]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-07-16T09:58:01.110 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 ceph-mon[81689]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-07-16T09:58:01.110 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 ceph-mon[81689]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-07-16T09:58:01.111 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 ceph-mon[81689]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-07-16T09:58:01.111 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 ceph-mon[81689]: rocksdb: Options.compression_opts.enabled: false 2024-07-16T09:58:01.111 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 ceph-mon[81689]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-07-16T09:58:01.111 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 ceph-mon[81689]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-07-16T09:58:01.111 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 ceph-mon[81689]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-07-16T09:58:01.111 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 ceph-mon[81689]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-07-16T09:58:01.111 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 ceph-mon[81689]: rocksdb: Options.target_file_size_base: 67108864 2024-07-16T09:58:01.111 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 ceph-mon[81689]: rocksdb: Options.target_file_size_multiplier: 1 2024-07-16T09:58:01.111 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 ceph-mon[81689]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-07-16T09:58:01.111 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 ceph-mon[81689]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-07-16T09:58:01.111 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 ceph-mon[81689]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-07-16T09:58:01.111 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 ceph-mon[81689]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-07-16T09:58:01.112 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 ceph-mon[81689]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-07-16T09:58:01.112 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 ceph-mon[81689]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-07-16T09:58:01.112 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 ceph-mon[81689]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-07-16T09:58:01.113 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 ceph-mon[81689]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-07-16T09:58:01.113 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 ceph-mon[81689]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-07-16T09:58:01.113 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 ceph-mon[81689]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-07-16T09:58:01.113 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 ceph-mon[81689]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-07-16T09:58:01.113 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 ceph-mon[81689]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-07-16T09:58:01.113 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 ceph-mon[81689]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-07-16T09:58:01.113 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 ceph-mon[81689]: rocksdb: Options.arena_block_size: 1048576 2024-07-16T09:58:01.113 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 ceph-mon[81689]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-07-16T09:58:01.113 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 ceph-mon[81689]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-07-16T09:58:01.113 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 ceph-mon[81689]: rocksdb: Options.disable_auto_compactions: 0 2024-07-16T09:58:01.113 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 ceph-mon[81689]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-07-16T09:58:01.113 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 ceph-mon[81689]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-07-16T09:58:01.113 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 ceph-mon[81689]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-07-16T09:58:01.114 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 ceph-mon[81689]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-07-16T09:58:01.114 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 ceph-mon[81689]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-07-16T09:58:01.114 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 ceph-mon[81689]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-07-16T09:58:01.114 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 ceph-mon[81689]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-07-16T09:58:01.114 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 ceph-mon[81689]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-07-16T09:58:01.114 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 ceph-mon[81689]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-07-16T09:58:01.114 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 ceph-mon[81689]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-07-16T09:58:01.114 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 ceph-mon[81689]: rocksdb: Options.table_properties_collectors: 2024-07-16T09:58:01.114 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 ceph-mon[81689]: rocksdb: Options.inplace_update_support: 0 2024-07-16T09:58:01.114 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 ceph-mon[81689]: rocksdb: Options.inplace_update_num_locks: 10000 2024-07-16T09:58:01.114 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 ceph-mon[81689]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-07-16T09:58:01.114 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 ceph-mon[81689]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-07-16T09:58:01.114 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 ceph-mon[81689]: rocksdb: Options.memtable_huge_page_size: 0 2024-07-16T09:58:01.115 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 ceph-mon[81689]: rocksdb: Options.bloom_locality: 0 2024-07-16T09:58:01.115 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 ceph-mon[81689]: rocksdb: Options.max_successive_merges: 0 2024-07-16T09:58:01.115 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 ceph-mon[81689]: rocksdb: Options.optimize_filters_for_hits: 0 2024-07-16T09:58:01.115 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 ceph-mon[81689]: rocksdb: Options.paranoid_file_checks: 0 2024-07-16T09:58:01.115 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 ceph-mon[81689]: rocksdb: Options.force_consistency_checks: 1 2024-07-16T09:58:01.115 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 ceph-mon[81689]: rocksdb: Options.report_bg_io_stats: 0 2024-07-16T09:58:01.115 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 ceph-mon[81689]: rocksdb: Options.ttl: 2592000 2024-07-16T09:58:01.115 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 ceph-mon[81689]: rocksdb: Options.periodic_compaction_seconds: 0 2024-07-16T09:58:01.115 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 ceph-mon[81689]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-07-16T09:58:01.115 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 ceph-mon[81689]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-07-16T09:58:01.115 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 ceph-mon[81689]: rocksdb: Options.enable_blob_files: false 2024-07-16T09:58:01.115 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 ceph-mon[81689]: rocksdb: Options.min_blob_size: 0 2024-07-16T09:58:01.115 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 ceph-mon[81689]: rocksdb: Options.blob_file_size: 268435456 2024-07-16T09:58:01.115 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 ceph-mon[81689]: rocksdb: Options.blob_compression_type: NoCompression 2024-07-16T09:58:01.116 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 ceph-mon[81689]: rocksdb: Options.enable_blob_garbage_collection: false 2024-07-16T09:58:01.116 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 ceph-mon[81689]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-07-16T09:58:01.116 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 ceph-mon[81689]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-07-16T09:58:01.116 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 ceph-mon[81689]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-07-16T09:58:01.116 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 ceph-mon[81689]: rocksdb: Options.blob_file_starting_level: 0 2024-07-16T09:58:01.116 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 ceph-mon[81689]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-07-16T09:58:01.116 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 ceph-mon[81689]: rocksdb: [table/block_based/block_based_table_reader.cc:721] At least one SST file opened without unique ID to verify: 24.sst 2024-07-16T09:58:01.116 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 ceph-mon[81689]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-07-16T09:58:01.116 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 ceph-mon[81689]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-smithi176/store.db/MANIFEST-000009 succeeded,manifest_file_number is 9, next_file_number is 26, last_sequence is 10579, log_number is 22,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-07-16T09:58:01.116 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 ceph-mon[81689]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 22 2024-07-16T09:58:01.116 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 ceph-mon[81689]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: f4d403a2-86c0-456f-af47-a37e48e39236 2024-07-16T09:58:01.116 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 ceph-mon[81689]: rocksdb: EVENT_LOG_v1 {"time_micros": 1721123880892338, "job": 1, "event": "recovery_started", "wal_files": [22]} 2024-07-16T09:58:01.116 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 ceph-mon[81689]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #22 mode 2 2024-07-16T09:58:01.117 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 ceph-mon[81689]: rocksdb: EVENT_LOG_v1 {"time_micros": 1721123880904216, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 27, "file_size": 3293335, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 10584, "largest_seqno": 11757, "table_properties": {"data_size": 3287548, "index_size": 3488, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 1285, "raw_key_size": 12137, "raw_average_key_size": 23, "raw_value_size": 3276467, "raw_average_value_size": 6424, "num_data_blocks": 159, "num_entries": 510, "num_filter_entries": 510, "num_deletions": 2, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1721123880, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "f4d403a2-86c0-456f-af47-a37e48e39236", "db_session_id": "V4RLS32CHM6TZ0TWCQU4", "orig_file_number": 27, "seqno_to_time_mapping": "N/A"}} 2024-07-16T09:58:01.117 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 ceph-mon[81689]: rocksdb: EVENT_LOG_v1 {"time_micros": 1721123880904346, "job": 1, "event": "recovery_finished"} 2024-07-16T09:58:01.117 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 ceph-mon[81689]: rocksdb: [db/version_set.cc:5047] Creating manifest 29 2024-07-16T09:58:01.117 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 ceph-mon[81689]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-07-16T09:58:01.117 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 ceph-mon[81689]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-smithi176/store.db/000022.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-07-16T09:58:01.117 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 ceph-mon[81689]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x563504d1ae00 2024-07-16T09:58:01.117 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 ceph-mon[81689]: rocksdb: DB pointer 0x563504e14000 2024-07-16T09:58:01.117 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 ceph-mon[81689]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-07-16T09:58:01.117 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 ceph-mon[81689]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-07-16T09:58:01.117 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: ** DB Stats ** 2024-07-16T09:58:01.117 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-07-16T09:58:01.117 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-07-16T09:58:01.117 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-07-16T09:58:01.118 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-07-16T09:58:01.118 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-07-16T09:58:01.118 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-07-16T09:58:01.118 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-07-16T09:58:01.118 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: 2024-07-16T09:58:01.118 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: ** Compaction Stats [default] ** 2024-07-16T09:58:01.118 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-07-16T09:58:01.118 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-07-16T09:58:01.118 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: L0 1/0 3.14 MB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 641.8 0.00 0.00 1 0.005 0 0 0.0 0.0 2024-07-16T09:58:01.118 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: L6 1/0 5.56 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 2024-07-16T09:58:01.118 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: Sum 2/0 8.70 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 641.8 0.00 0.00 1 0.005 0 0 0.0 0.0 2024-07-16T09:58:01.118 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 641.8 0.00 0.00 1 0.005 0 0 0.0 0.0 2024-07-16T09:58:01.119 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: 2024-07-16T09:58:01.119 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: ** Compaction Stats [default] ** 2024-07-16T09:58:01.119 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-07-16T09:58:01.119 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-07-16T09:58:01.120 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 641.8 0.00 0.00 1 0.005 0 0 0.0 0.0 2024-07-16T09:58:01.120 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: 2024-07-16T09:58:01.120 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-07-16T09:58:01.120 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: 2024-07-16T09:58:01.120 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-07-16T09:58:01.120 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: Flush(GB): cumulative 0.003, interval 0.003 2024-07-16T09:58:01.121 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-07-16T09:58:01.121 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-07-16T09:58:01.121 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-07-16T09:58:01.121 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: AddFile(Keys): cumulative 0, interval 0 2024-07-16T09:58:01.121 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: Cumulative compaction: 0.00 GB write, 192.80 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-07-16T09:58:01.121 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: Interval compaction: 0.00 GB write, 192.80 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-07-16T09:58:01.121 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-07-16T09:58:01.121 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: Block cache BinnedLRUCache@0x563504ced090#2 capacity: 512.00 MB usage: 5.48 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.2e-05 secs_since: 0 2024-07-16T09:58:01.121 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,1.41 KB,0.000268221%) IndexBlock(1,4.08 KB,0.000777841%) Misc(1,0.00 KB,0%) 2024-07-16T09:58:01.121 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: 2024-07-16T09:58:01.121 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: ** File Read Latency Histogram By Level [default] ** 2024-07-16T09:58:01.121 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 ceph-mon[81689]: starting mon.smithi176 rank 1 at public addrs [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] at bind addrs [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] mon_data /var/lib/ceph/mon/ceph-smithi176 fsid 26c91c26-4358-11ef-bcad-c7b262605968 2024-07-16T09:58:01.121 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 ceph-mon[81689]: mon.smithi176@-1(???) e2 preinit fsid 26c91c26-4358-11ef-bcad-c7b262605968 2024-07-16T09:58:01.122 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 ceph-mon[81689]: mon.smithi176@-1(???).mds e18 new map 2024-07-16T09:58:01.122 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 ceph-mon[81689]: mon.smithi176@-1(???).mds e18 print_map 2024-07-16T09:58:01.122 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: e18 2024-07-16T09:58:01.122 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-07-16T09:58:01.122 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-07-16T09:58:01.122 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: legacy client fscid: 1 2024-07-16T09:58:01.122 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: 2024-07-16T09:58:01.122 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: Filesystem 'cephfs' (1) 2024-07-16T09:58:01.122 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: fs_name cephfs 2024-07-16T09:58:01.122 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: epoch 18 2024-07-16T09:58:01.122 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-07-16T09:58:01.122 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: created 2024-07-16T09:52:55.214975+0000 2024-07-16T09:58:01.122 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: modified 2024-07-16T09:55:41.914320+0000 2024-07-16T09:58:01.123 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: tableserver 0 2024-07-16T09:58:01.123 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: root 0 2024-07-16T09:58:01.123 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: session_timeout 60 2024-07-16T09:58:01.123 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: session_autoclose 300 2024-07-16T09:58:01.123 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: max_file_size 1099511627776 2024-07-16T09:58:01.123 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: max_xattr_size 65536 2024-07-16T09:58:01.123 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: required_client_features {} 2024-07-16T09:58:01.123 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: last_failure 0 2024-07-16T09:58:01.123 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: last_failure_osd_epoch 0 2024-07-16T09:58:01.123 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-07-16T09:58:01.123 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: max_mds 2 2024-07-16T09:58:01.123 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: in 0,1 2024-07-16T09:58:01.123 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: up {0=14458,1=24261} 2024-07-16T09:58:01.124 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: failed 2024-07-16T09:58:01.124 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: damaged 2024-07-16T09:58:01.124 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: stopped 2024-07-16T09:58:01.124 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: data_pools [3] 2024-07-16T09:58:01.124 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: metadata_pool 2 2024-07-16T09:58:01.124 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: inline_data disabled 2024-07-16T09:58:01.124 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: balancer 2024-07-16T09:58:01.124 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: bal_rank_mask -1 2024-07-16T09:58:01.124 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: standby_count_wanted 1 2024-07-16T09:58:01.124 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: qdb_cluster leader: 0 members: 2024-07-16T09:58:01.124 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: [mds.cephfs.smithi156.rdibys{0:14458} state up:active seq 4 export targets 1 join_fscid=1 addr [v2:172.21.15.156:6826/1080176013,v1:172.21.15.156:6827/1080176013] compat {c=[1],r=[1],i=[7ff]}] 2024-07-16T09:58:01.124 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: [mds.cephfs.smithi156.evwwaz{0:14468} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.156:6828/560329502,v1:172.21.15.156:6829/560329502] compat {c=[1],r=[1],i=[7ff]}] 2024-07-16T09:58:01.124 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: [mds.cephfs.smithi176.dvvbud{1:24261} state up:active seq 4 export targets 0 join_fscid=1 addr [v2:172.21.15.176:6824/1577019287,v1:172.21.15.176:6825/1577019287] compat {c=[1],r=[1],i=[7ff]}] 2024-07-16T09:58:01.124 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: [mds.cephfs.smithi176.wahbsi{1:24273} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.176:6826/2196775412,v1:172.21.15.176:6827/2196775412] compat {c=[1],r=[1],i=[7ff]}] 2024-07-16T09:58:01.125 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: 2024-07-16T09:58:01.125 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: 2024-07-16T09:58:01.125 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 ceph-mon[81689]: mon.smithi176@-1(???).osd e49 crush map has features 3314933000852226048, adjusting msgr requires 2024-07-16T09:58:01.125 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 ceph-mon[81689]: mon.smithi176@-1(???).osd e49 crush map has features 288514051259236352, adjusting msgr requires 2024-07-16T09:58:01.125 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 ceph-mon[81689]: mon.smithi176@-1(???).osd e49 crush map has features 288514051259236352, adjusting msgr requires 2024-07-16T09:58:01.125 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 ceph-mon[81689]: mon.smithi176@-1(???).osd e49 crush map has features 288514051259236352, adjusting msgr requires 2024-07-16T09:58:01.125 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 ceph-mon[81689]: mon.smithi176@-1(???).paxosservice(auth 1..21) refresh upgraded, format 0 -> 3 2024-07-16T09:58:01.125 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 bash[81670]: ef92814cec2bc58325935136a26d0a5a2db0964b14e066585628006ac725759b 2024-07-16T09:58:01.125 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:00 smithi176 systemd[1]: Started Ceph mon.smithi176 for 26c91c26-4358-11ef-bcad-c7b262605968. 2024-07-16T09:58:03.586 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:03 smithi176 ceph-mon[81689]: mgrmap e37: smithi156.zkvylv(active, since 13s), standbys: smithi176.dqbrvx 2024-07-16T09:58:03.586 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:03 smithi176 ceph-mon[81689]: overall HEALTH_OK 2024-07-16T09:58:03.586 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:03 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "mon metadata", "id": "smithi156"}]: dispatch 2024-07-16T09:58:03.586 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:03 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "mon metadata", "id": "smithi176"}]: dispatch 2024-07-16T09:58:03.586 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:03 smithi176 ceph-mon[81689]: mon.smithi176 calling monitor election 2024-07-16T09:58:03.587 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:03 smithi176 ceph-mon[81689]: mon.smithi156 calling monitor election 2024-07-16T09:58:03.587 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:03 smithi176 ceph-mon[81689]: mon.smithi156 is new leader, mons smithi156,smithi176 in quorum (ranks 0,1) 2024-07-16T09:58:03.587 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:03 smithi176 ceph-mon[81689]: monmap epoch 3 2024-07-16T09:58:03.587 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:03 smithi176 ceph-mon[81689]: fsid 26c91c26-4358-11ef-bcad-c7b262605968 2024-07-16T09:58:03.587 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:03 smithi176 ceph-mon[81689]: last_changed 2024-07-16T09:58:02.254416+0000 2024-07-16T09:58:03.587 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:03 smithi176 ceph-mon[81689]: created 2024-07-16T09:46:20.852914+0000 2024-07-16T09:58:03.587 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:03 smithi176 ceph-mon[81689]: min_mon_release 19 (squid) 2024-07-16T09:58:03.587 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:03 smithi176 ceph-mon[81689]: election_strategy: 1 2024-07-16T09:58:03.587 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:03 smithi176 ceph-mon[81689]: 0: [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] mon.smithi156 2024-07-16T09:58:03.587 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:03 smithi176 ceph-mon[81689]: 1: [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] mon.smithi176 2024-07-16T09:58:03.587 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:03 smithi176 ceph-mon[81689]: fsmap cephfs:2 {0=cephfs.smithi156.rdibys=up:active,1=cephfs.smithi176.dvvbud=up:active} 2 up:standby-replay 2024-07-16T09:58:03.587 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:03 smithi176 ceph-mon[81689]: osdmap e49: 6 total, 6 up, 6 in 2024-07-16T09:58:03.587 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:03 smithi176 ceph-mon[81689]: mgrmap e37: smithi156.zkvylv(active, since 13s), standbys: smithi176.dqbrvx 2024-07-16T09:58:03.588 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:03 smithi176 ceph-mon[81689]: overall HEALTH_OK 2024-07-16T09:58:03.588 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:03 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T09:58:03.588 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:03 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T09:58:03.588 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:03 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-16T09:58:03.588 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:03 smithi176 ceph-mon[81689]: pgmap v10: 65 pgs: 65 active+clean; 253 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 19 KiB/s rd, 0 B/s wr, 12 op/s 2024-07-16T09:58:03.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:03 smithi156 ceph-mon[93066]: mgrmap e37: smithi156.zkvylv(active, since 13s), standbys: smithi176.dqbrvx 2024-07-16T09:58:03.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:03 smithi156 ceph-mon[93066]: overall HEALTH_OK 2024-07-16T09:58:03.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:03 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "mon metadata", "id": "smithi156"}]: dispatch 2024-07-16T09:58:03.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:03 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "mon metadata", "id": "smithi176"}]: dispatch 2024-07-16T09:58:03.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:03 smithi156 ceph-mon[93066]: mon.smithi176 calling monitor election 2024-07-16T09:58:03.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:03 smithi156 ceph-mon[93066]: mon.smithi156 calling monitor election 2024-07-16T09:58:03.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:03 smithi156 ceph-mon[93066]: mon.smithi156 is new leader, mons smithi156,smithi176 in quorum (ranks 0,1) 2024-07-16T09:58:03.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:03 smithi156 ceph-mon[93066]: monmap epoch 3 2024-07-16T09:58:03.676 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:03 smithi156 ceph-mon[93066]: fsid 26c91c26-4358-11ef-bcad-c7b262605968 2024-07-16T09:58:03.676 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:03 smithi156 ceph-mon[93066]: last_changed 2024-07-16T09:58:02.254416+0000 2024-07-16T09:58:03.676 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:03 smithi156 ceph-mon[93066]: created 2024-07-16T09:46:20.852914+0000 2024-07-16T09:58:03.676 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:03 smithi156 ceph-mon[93066]: min_mon_release 19 (squid) 2024-07-16T09:58:03.676 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:03 smithi156 ceph-mon[93066]: election_strategy: 1 2024-07-16T09:58:03.676 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:03 smithi156 ceph-mon[93066]: 0: [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] mon.smithi156 2024-07-16T09:58:03.676 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:03 smithi156 ceph-mon[93066]: 1: [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] mon.smithi176 2024-07-16T09:58:03.676 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:03 smithi156 ceph-mon[93066]: fsmap cephfs:2 {0=cephfs.smithi156.rdibys=up:active,1=cephfs.smithi176.dvvbud=up:active} 2 up:standby-replay 2024-07-16T09:58:03.676 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:03 smithi156 ceph-mon[93066]: osdmap e49: 6 total, 6 up, 6 in 2024-07-16T09:58:03.676 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:03 smithi156 ceph-mon[93066]: mgrmap e37: smithi156.zkvylv(active, since 13s), standbys: smithi176.dqbrvx 2024-07-16T09:58:03.676 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:03 smithi156 ceph-mon[93066]: overall HEALTH_OK 2024-07-16T09:58:03.676 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:03 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T09:58:03.676 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:03 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T09:58:03.677 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:03 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-16T09:58:03.677 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:03 smithi156 ceph-mon[93066]: pgmap v10: 65 pgs: 65 active+clean; 253 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 19 KiB/s rd, 0 B/s wr, 12 op/s 2024-07-16T09:58:04.539 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:04 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-16T09:58:04.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:04 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-16T09:58:05.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:05 smithi176 ceph-mon[81689]: pgmap v11: 65 pgs: 65 active+clean; 253 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 19 KiB/s rd, 0 B/s wr, 13 op/s 2024-07-16T09:58:05.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:05 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T09:58:05.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:05 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T09:58:05.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:05 smithi156 ceph-mon[93066]: pgmap v11: 65 pgs: 65 active+clean; 253 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 19 KiB/s rd, 0 B/s wr, 13 op/s 2024-07-16T09:58:05.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:05 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T09:58:05.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:05 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T09:58:06.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:06 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T09:58:06.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:06 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T09:58:06.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:06 smithi176 ceph-mon[81689]: pgmap v12: 65 pgs: 65 active+clean; 253 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 20 KiB/s rd, 0 B/s wr, 14 op/s 2024-07-16T09:58:07.174 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:06 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T09:58:07.175 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:06 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T09:58:07.175 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:06 smithi156 ceph-mon[93066]: pgmap v12: 65 pgs: 65 active+clean; 253 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 20 KiB/s rd, 0 B/s wr, 14 op/s 2024-07-16T09:58:08.481 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:08 smithi176 ceph-mon[81689]: pgmap v13: 65 pgs: 65 active+clean; 253 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-07-16T09:58:08.660 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:08 smithi156 ceph-mon[93066]: pgmap v13: 65 pgs: 65 active+clean; 253 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-07-16T09:58:10.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:09 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T09:58:10.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:09 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T09:58:10.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:09 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-16T09:58:10.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:09 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-16T09:58:10.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:09 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T09:58:10.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:09 smithi176 ceph-mon[81689]: Reconfiguring mon.smithi156 (monmap changed)... 2024-07-16T09:58:10.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:09 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-07-16T09:58:10.095 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:09 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-07-16T09:58:10.095 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:09 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-16T09:58:10.095 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:09 smithi176 ceph-mon[81689]: Reconfiguring daemon mon.smithi156 on smithi156 2024-07-16T09:58:10.174 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:09 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T09:58:10.174 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:09 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T09:58:10.175 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:09 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-16T09:58:10.175 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:09 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-16T09:58:10.175 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:09 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T09:58:10.175 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:09 smithi156 ceph-mon[93066]: Reconfiguring mon.smithi156 (monmap changed)... 2024-07-16T09:58:10.175 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:09 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-07-16T09:58:10.175 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:09 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-07-16T09:58:10.175 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:09 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-16T09:58:10.175 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:09 smithi156 ceph-mon[93066]: Reconfiguring daemon mon.smithi156 on smithi156 2024-07-16T09:58:11.093 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:10 smithi176 ceph-mon[81689]: pgmap v14: 65 pgs: 65 active+clean; 253 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-07-16T09:58:11.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:10 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T09:58:11.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:10 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T09:58:11.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:10 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi156.zkvylv", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-07-16T09:58:11.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:10 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "mgr services"}]: dispatch 2024-07-16T09:58:11.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:10 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-16T09:58:11.126 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:10 smithi156 ceph-mon[93066]: pgmap v14: 65 pgs: 65 active+clean; 253 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-07-16T09:58:11.126 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:10 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T09:58:11.126 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:10 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T09:58:11.126 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:10 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi156.zkvylv", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-07-16T09:58:11.126 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:10 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "mgr services"}]: dispatch 2024-07-16T09:58:11.126 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:10 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-16T09:58:12.073 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:11 smithi156 ceph-mon[93066]: Reconfiguring mgr.smithi156.zkvylv (monmap changed)... 2024-07-16T09:58:12.074 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:11 smithi156 ceph-mon[93066]: Reconfiguring daemon mgr.smithi156.zkvylv on smithi156 2024-07-16T09:58:12.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:11 smithi176 ceph-mon[81689]: Reconfiguring mgr.smithi156.zkvylv (monmap changed)... 2024-07-16T09:58:12.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:11 smithi176 ceph-mon[81689]: Reconfiguring daemon mgr.smithi156.zkvylv on smithi156 2024-07-16T09:58:13.424 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:13 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T09:58:13.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:13 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T09:58:13.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:13 smithi156 ceph-mon[93066]: Reconfiguring crash.smithi156 (monmap changed)... 2024-07-16T09:58:13.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:13 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi156", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-07-16T09:58:13.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:13 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-16T09:58:13.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:13 smithi156 ceph-mon[93066]: Reconfiguring daemon crash.smithi156 on smithi156 2024-07-16T09:58:13.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:13 smithi156 ceph-mon[93066]: pgmap v15: 65 pgs: 65 active+clean; 253 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-07-16T09:58:13.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:13 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T09:58:13.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:13 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T09:58:13.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:13 smithi176 ceph-mon[81689]: Reconfiguring crash.smithi156 (monmap changed)... 2024-07-16T09:58:13.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:13 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi156", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-07-16T09:58:13.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:13 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-16T09:58:13.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:13 smithi176 ceph-mon[81689]: Reconfiguring daemon crash.smithi156 on smithi156 2024-07-16T09:58:13.595 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:13 smithi176 ceph-mon[81689]: pgmap v15: 65 pgs: 65 active+clean; 253 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-07-16T09:58:14.830 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:14 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T09:58:14.830 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:14 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T09:58:14.831 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:14 smithi156 ceph-mon[93066]: Reconfiguring osd.0 (monmap changed)... 2024-07-16T09:58:14.831 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:14 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-07-16T09:58:14.831 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:14 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-16T09:58:14.831 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:14 smithi156 ceph-mon[93066]: Reconfiguring daemon osd.0 on smithi156 2024-07-16T09:58:14.831 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:14 smithi156 ceph-mon[93066]: pgmap v16: 65 pgs: 65 active+clean; 253 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-07-16T09:58:14.843 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:14 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T09:58:14.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:14 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T09:58:14.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:14 smithi176 ceph-mon[81689]: Reconfiguring osd.0 (monmap changed)... 2024-07-16T09:58:14.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:14 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-07-16T09:58:14.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:14 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-16T09:58:14.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:14 smithi176 ceph-mon[81689]: Reconfiguring daemon osd.0 on smithi156 2024-07-16T09:58:14.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:14 smithi176 ceph-mon[81689]: pgmap v16: 65 pgs: 65 active+clean; 253 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-07-16T09:58:15.843 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:15 smithi176 ceph-mon[81689]: fsmap cephfs:2 {0=cephfs.smithi156.rdibys=up:active,1=cephfs.smithi176.dvvbud=up:active} 2 up:standby-replay 2024-07-16T09:58:15.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:15 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T09:58:15.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:15 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T09:58:15.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:15 smithi176 ceph-mon[81689]: Reconfiguring osd.1 (monmap changed)... 2024-07-16T09:58:15.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:15 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-07-16T09:58:15.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:15 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-16T09:58:15.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:15 smithi176 ceph-mon[81689]: Reconfiguring daemon osd.1 on smithi156 2024-07-16T09:58:15.924 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:15 smithi156 ceph-mon[93066]: fsmap cephfs:2 {0=cephfs.smithi156.rdibys=up:active,1=cephfs.smithi176.dvvbud=up:active} 2 up:standby-replay 2024-07-16T09:58:15.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:15 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T09:58:15.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:15 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T09:58:15.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:15 smithi156 ceph-mon[93066]: Reconfiguring osd.1 (monmap changed)... 2024-07-16T09:58:15.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:15 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-07-16T09:58:15.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:15 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-16T09:58:15.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:15 smithi156 ceph-mon[93066]: Reconfiguring daemon osd.1 on smithi156 2024-07-16T09:58:16.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:16 smithi176 ceph-mon[81689]: pgmap v17: 65 pgs: 65 active+clean; 253 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T09:58:16.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:16 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T09:58:16.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:16 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T09:58:16.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:16 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-07-16T09:58:16.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:16 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-16T09:58:16.854 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:16 smithi156 ceph-mon[93066]: pgmap v17: 65 pgs: 65 active+clean; 253 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T09:58:16.854 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:16 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T09:58:16.854 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:16 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T09:58:16.854 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:16 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-07-16T09:58:16.854 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:16 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-16T09:58:17.728 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:17 smithi156 ceph-mon[93066]: Reconfiguring osd.2 (monmap changed)... 2024-07-16T09:58:17.728 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:17 smithi156 ceph-mon[93066]: Reconfiguring daemon osd.2 on smithi156 2024-07-16T09:58:17.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:17 smithi176 ceph-mon[81689]: Reconfiguring osd.2 (monmap changed)... 2024-07-16T09:58:17.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:17 smithi176 ceph-mon[81689]: Reconfiguring daemon osd.2 on smithi156 2024-07-16T09:58:19.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:18 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T09:58:19.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:18 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T09:58:19.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:18 smithi176 ceph-mon[81689]: Reconfiguring mds.cephfs.smithi156.rdibys (monmap changed)... 2024-07-16T09:58:19.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:18 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi156.rdibys", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-16T09:58:19.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:18 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-16T09:58:19.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:18 smithi176 ceph-mon[81689]: Reconfiguring daemon mds.cephfs.smithi156.rdibys on smithi156 2024-07-16T09:58:19.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:18 smithi176 ceph-mon[81689]: pgmap v18: 65 pgs: 65 active+clean; 253 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-07-16T09:58:19.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:18 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-16T09:58:19.175 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:18 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T09:58:19.175 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:18 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T09:58:19.175 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:18 smithi156 ceph-mon[93066]: Reconfiguring mds.cephfs.smithi156.rdibys (monmap changed)... 2024-07-16T09:58:19.175 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:18 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi156.rdibys", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-16T09:58:19.175 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:18 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-16T09:58:19.175 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:18 smithi156 ceph-mon[93066]: Reconfiguring daemon mds.cephfs.smithi156.rdibys on smithi156 2024-07-16T09:58:19.175 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:18 smithi156 ceph-mon[93066]: pgmap v18: 65 pgs: 65 active+clean; 253 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-07-16T09:58:19.175 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:18 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-16T09:58:20.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:20 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T09:58:20.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:20 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T09:58:20.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:20 smithi176 ceph-mon[81689]: Reconfiguring mds.cephfs.smithi156.evwwaz (monmap changed)... 2024-07-16T09:58:20.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:20 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi156.evwwaz", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-16T09:58:20.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:20 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-16T09:58:20.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:20 smithi176 ceph-mon[81689]: Reconfiguring daemon mds.cephfs.smithi156.evwwaz on smithi156 2024-07-16T09:58:20.610 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:20 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T09:58:20.611 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:20 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T09:58:20.611 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:20 smithi156 ceph-mon[93066]: Reconfiguring mds.cephfs.smithi156.evwwaz (monmap changed)... 2024-07-16T09:58:20.611 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:20 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi156.evwwaz", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-16T09:58:20.611 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:20 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-16T09:58:20.611 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:20 smithi156 ceph-mon[93066]: Reconfiguring daemon mds.cephfs.smithi156.evwwaz on smithi156 2024-07-16T09:58:21.325 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:21.321+0000 7fc68ab09640 1 -- 172.21.15.156:0/3989150078 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc6840fb230 msgr2=0x7fc6840fb640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:58:21.325 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:21.321+0000 7fc68ab09640 1 --2- 172.21.15.156:0/3989150078 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc6840fb230 0x7fc6840fb640 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fc6780048d0 tx=0x7fc678009da0 comp rx=0 tx=0).stop 2024-07-16T09:58:21.325 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:21.321+0000 7fc68ab09640 1 -- 172.21.15.156:0/3989150078 shutdown_connections 2024-07-16T09:58:21.325 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:21.321+0000 7fc68ab09640 1 --2- 172.21.15.156:0/3989150078 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc6840fc500 0x7fc6840fc970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:58:21.325 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:21.321+0000 7fc68ab09640 1 --2- 172.21.15.156:0/3989150078 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc6840fb230 0x7fc6840fb640 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:58:21.326 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:21.321+0000 7fc68ab09640 1 -- 172.21.15.156:0/3989150078 >> 172.21.15.156:0/3989150078 conn(0x7fc6840f6a10 msgr2=0x7fc6840f8e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:58:21.326 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:21.321+0000 7fc68ab09640 1 -- 172.21.15.156:0/3989150078 shutdown_connections 2024-07-16T09:58:21.326 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:21.321+0000 7fc68ab09640 1 -- 172.21.15.156:0/3989150078 wait complete. 2024-07-16T09:58:21.326 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:21.321+0000 7fc68ab09640 1 Processor -- start 2024-07-16T09:58:21.326 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:21.321+0000 7fc68ab09640 1 -- start start 2024-07-16T09:58:21.326 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:21.322+0000 7fc68ab09640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc6840fb230 0x7fc6840ff300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:58:21.326 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:21.322+0000 7fc68ab09640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc6840fc500 0x7fc6840fd840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:58:21.326 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:21.322+0000 7fc68ab09640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6840fdd80 con 0x7fc6840fc500 2024-07-16T09:58:21.326 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:21.322+0000 7fc68ab09640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6840fdec0 con 0x7fc6840fb230 2024-07-16T09:58:21.327 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:21.322+0000 7fc683fff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc6840fc500 0x7fc6840fd840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:58:21.327 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:21.322+0000 7fc683fff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc6840fc500 0x7fc6840fd840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:50660/0 (socket says 172.21.15.156:50660) 2024-07-16T09:58:21.327 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:21.322+0000 7fc683fff640 1 -- 172.21.15.156:0/1941900893 learned_addr learned my addr 172.21.15.156:0/1941900893 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T09:58:21.327 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:21.322+0000 7fc6888a5640 1 --2- 172.21.15.156:0/1941900893 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc6840fb230 0x7fc6840ff300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:58:21.327 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:21.322+0000 7fc683fff640 1 -- 172.21.15.156:0/1941900893 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc6840fb230 msgr2=0x7fc6840ff300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:58:21.327 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:21.322+0000 7fc683fff640 1 --2- 172.21.15.156:0/1941900893 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc6840fb230 0x7fc6840ff300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:58:21.327 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:21.322+0000 7fc683fff640 1 -- 172.21.15.156:0/1941900893 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc678005040 con 0x7fc6840fc500 2024-07-16T09:58:21.327 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:21.322+0000 7fc683fff640 1 --2- 172.21.15.156:0/1941900893 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc6840fc500 0x7fc6840fd840 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fc67000eb70 tx=0x7fc67000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:58:21.327 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:21.323+0000 7fc6817fa640 1 -- 172.21.15.156:0/1941900893 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc670007c30 con 0x7fc6840fc500 2024-07-16T09:58:21.327 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:21.323+0000 7fc6817fa640 1 -- 172.21.15.156:0/1941900893 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc67000f040 con 0x7fc6840fc500 2024-07-16T09:58:21.327 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:21.323+0000 7fc68ab09640 1 -- 172.21.15.156:0/1941900893 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6840fe140 con 0x7fc6840fc500 2024-07-16T09:58:21.328 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:21.323+0000 7fc6817fa640 1 -- 172.21.15.156:0/1941900893 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc67000cad0 con 0x7fc6840fc500 2024-07-16T09:58:21.328 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:21.323+0000 7fc68ab09640 1 -- 172.21.15.156:0/1941900893 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6840fe6b0 con 0x7fc6840fc500 2024-07-16T09:58:21.329 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:21.324+0000 7fc68ab09640 1 -- 172.21.15.156:0/1941900893 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6840fa690 con 0x7fc6840fc500 2024-07-16T09:58:21.329 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:21.324+0000 7fc6817fa640 1 -- 172.21.15.156:0/1941900893 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 37) v1 ==== 92307+0+0 (secure 0 0 0) 0x7fc67000cc50 con 0x7fc6840fc500 2024-07-16T09:58:21.329 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:21.325+0000 7fc6817fa640 1 --2- 172.21.15.156:0/1941900893 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7fc67406e1e0 0x7fc674070690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:58:21.329 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:21.325+0000 7fc6817fa640 1 -- 172.21.15.156:0/1941900893 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fc67008d8a0 con 0x7fc6840fc500 2024-07-16T09:58:21.329 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:21.325+0000 7fc6888a5640 1 --2- 172.21.15.156:0/1941900893 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7fc67406e1e0 0x7fc674070690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:58:21.329 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:21.325+0000 7fc6888a5640 1 --2- 172.21.15.156:0/1941900893 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7fc67406e1e0 0x7fc674070690 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fc678009f80 tx=0x7fc678035040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:58:21.331 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:21.329+0000 7fc6817fa640 1 -- 172.21.15.156:0/1941900893 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187716 (secure 0 0 0) 0x7fc670056ab0 con 0x7fc6840fc500 2024-07-16T09:58:21.525 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:21.522+0000 7fc68ab09640 1 -- 172.21.15.156:0/1941900893 --> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc6840fe6b0 con 0x7fc67406e1e0 2024-07-16T09:58:21.526 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:21.524+0000 7fc6817fa640 1 -- 172.21.15.156:0/1941900893 <== mgr.34104 v2:172.21.15.156:6800/3367765292 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+383 (secure 0 0 0) 0x7fc6840fe6b0 con 0x7fc67406e1e0 2024-07-16T09:58:21.528 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:21.526+0000 7fc68ab09640 1 -- 172.21.15.156:0/1941900893 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7fc67406e1e0 msgr2=0x7fc674070690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:58:21.528 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:21.526+0000 7fc68ab09640 1 --2- 172.21.15.156:0/1941900893 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7fc67406e1e0 0x7fc674070690 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fc678009f80 tx=0x7fc678035040 comp rx=0 tx=0).stop 2024-07-16T09:58:21.528 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:21.526+0000 7fc68ab09640 1 -- 172.21.15.156:0/1941900893 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc6840fc500 msgr2=0x7fc6840fd840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:58:21.529 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:21.526+0000 7fc68ab09640 1 --2- 172.21.15.156:0/1941900893 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc6840fc500 0x7fc6840fd840 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fc67000eb70 tx=0x7fc67000b040 comp rx=0 tx=0).stop 2024-07-16T09:58:21.529 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:21.526+0000 7fc68ab09640 1 -- 172.21.15.156:0/1941900893 shutdown_connections 2024-07-16T09:58:21.529 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:21.526+0000 7fc68ab09640 1 --2- 172.21.15.156:0/1941900893 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7fc67406e1e0 0x7fc674070690 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:58:21.529 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:21.526+0000 7fc68ab09640 1 --2- 172.21.15.156:0/1941900893 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc6840fc500 0x7fc6840fd840 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:58:21.529 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:21.526+0000 7fc68ab09640 1 --2- 172.21.15.156:0/1941900893 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc6840fb230 0x7fc6840ff300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:58:21.529 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:21.526+0000 7fc68ab09640 1 -- 172.21.15.156:0/1941900893 >> 172.21.15.156:0/1941900893 conn(0x7fc6840f6a10 msgr2=0x7fc684065c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:58:21.529 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:21.526+0000 7fc68ab09640 1 -- 172.21.15.156:0/1941900893 shutdown_connections 2024-07-16T09:58:21.529 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:21.526+0000 7fc68ab09640 1 -- 172.21.15.156:0/1941900893 wait complete. 2024-07-16T09:58:21.537 INFO:teuthology.orchestra.run.smithi156.stdout:true 2024-07-16T09:58:21.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:21 smithi176 ceph-mon[81689]: pgmap v19: 65 pgs: 65 active+clean; 253 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-07-16T09:58:21.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:21 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T09:58:21.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:21 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T09:58:21.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:21 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi176", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-07-16T09:58:21.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:21 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-16T09:58:21.608 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:21.606+0000 7f57158a5640 1 -- 172.21.15.156:0/258919229 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f57100fd9d0 msgr2=0x7f57100fdde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:58:21.608 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:21.606+0000 7f57158a5640 1 --2- 172.21.15.156:0/258919229 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f57100fd9d0 0x7f57100fdde0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f5700009c50 tx=0x7f5700004d10 comp rx=0 tx=0).stop 2024-07-16T09:58:21.608 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:21.606+0000 7f57158a5640 1 -- 172.21.15.156:0/258919229 shutdown_connections 2024-07-16T09:58:21.609 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:21.606+0000 7f57158a5640 1 --2- 172.21.15.156:0/258919229 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f57100fe320 0x7f57100fa480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:58:21.609 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:21.606+0000 7f57158a5640 1 --2- 172.21.15.156:0/258919229 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f57100fd9d0 0x7f57100fdde0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:58:21.609 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:21.606+0000 7f57158a5640 1 -- 172.21.15.156:0/258919229 >> 172.21.15.156:0/258919229 conn(0x7f57100f61d0 msgr2=0x7f57100f8600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:58:21.609 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:21.606+0000 7f57158a5640 1 -- 172.21.15.156:0/258919229 shutdown_connections 2024-07-16T09:58:21.609 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:21 smithi156 ceph-mon[93066]: pgmap v19: 65 pgs: 65 active+clean; 253 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-07-16T09:58:21.609 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:21 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T09:58:21.609 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:21 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T09:58:21.609 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:21 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi176", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-07-16T09:58:21.609 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:21 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-16T09:58:21.611 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:21.606+0000 7f57158a5640 1 -- 172.21.15.156:0/258919229 wait complete. 2024-07-16T09:58:21.611 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:21.607+0000 7f57158a5640 1 Processor -- start 2024-07-16T09:58:21.611 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:21.607+0000 7f57158a5640 1 -- start start 2024-07-16T09:58:21.611 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:21.607+0000 7f57158a5640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f57100fd9d0 0x7f571010f250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:58:21.611 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:21.607+0000 7f57158a5640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f57100fe320 0x7f571010f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:58:21.611 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:21.607+0000 7f57158a5640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f571010fde0 con 0x7f57100fd9d0 2024-07-16T09:58:21.612 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:21.607+0000 7f57158a5640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5710110600 con 0x7f57100fe320 2024-07-16T09:58:21.612 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:21.607+0000 7f570effd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f57100fd9d0 0x7f571010f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:58:21.612 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:21.607+0000 7f570effd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f57100fd9d0 0x7f571010f250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:50688/0 (socket says 172.21.15.156:50688) 2024-07-16T09:58:21.612 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:21.607+0000 7f570effd640 1 -- 172.21.15.156:0/3592757293 learned_addr learned my addr 172.21.15.156:0/3592757293 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T09:58:21.612 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:21.607+0000 7f570e7fc640 1 --2- 172.21.15.156:0/3592757293 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f57100fe320 0x7f571010f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:58:21.612 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:21.607+0000 7f570effd640 1 -- 172.21.15.156:0/3592757293 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f57100fe320 msgr2=0x7f571010f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:58:21.612 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:21.607+0000 7f570effd640 1 --2- 172.21.15.156:0/3592757293 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f57100fe320 0x7f571010f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:58:21.612 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:21.607+0000 7f570effd640 1 -- 172.21.15.156:0/3592757293 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57000098b0 con 0x7f57100fd9d0 2024-07-16T09:58:21.612 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:21.608+0000 7f570effd640 1 --2- 172.21.15.156:0/3592757293 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f57100fd9d0 0x7f571010f250 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f5700009c20 tx=0x7f5700004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:58:21.613 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:21.608+0000 7f57077fe640 1 -- 172.21.15.156:0/3592757293 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5700008e60 con 0x7f57100fd9d0 2024-07-16T09:58:21.613 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:21.608+0000 7f57077fe640 1 -- 172.21.15.156:0/3592757293 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5700032c00 con 0x7f57100fd9d0 2024-07-16T09:58:21.613 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:21.608+0000 7f57158a5640 1 -- 172.21.15.156:0/3592757293 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5710110830 con 0x7f57100fd9d0 2024-07-16T09:58:21.613 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:21.608+0000 7f57077fe640 1 -- 172.21.15.156:0/3592757293 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f570003dbf0 con 0x7f57100fd9d0 2024-07-16T09:58:21.613 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:21.608+0000 7f57158a5640 1 -- 172.21.15.156:0/3592757293 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5710110d50 con 0x7f57100fd9d0 2024-07-16T09:58:21.613 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:21.609+0000 7f57158a5640 1 -- 172.21.15.156:0/3592757293 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57100f9e70 con 0x7f57100fd9d0 2024-07-16T09:58:21.614 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:21.610+0000 7f57077fe640 1 -- 172.21.15.156:0/3592757293 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 37) v1 ==== 92307+0+0 (secure 0 0 0) 0x7f570002f030 con 0x7f57100fd9d0 2024-07-16T09:58:21.614 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:21.610+0000 7f57077fe640 1 --2- 172.21.15.156:0/3592757293 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f56fc06e1e0 0x7f56fc070690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:58:21.614 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:21.610+0000 7f570e7fc640 1 --2- 172.21.15.156:0/3592757293 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f56fc06e1e0 0x7f56fc070690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:58:21.614 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:21.610+0000 7f57077fe640 1 -- 172.21.15.156:0/3592757293 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f57000af350 con 0x7f57100fd9d0 2024-07-16T09:58:21.614 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:21.611+0000 7f570e7fc640 1 --2- 172.21.15.156:0/3592757293 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f56fc06e1e0 0x7f56fc070690 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f571010f000 tx=0x7f56f800b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:58:21.617 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:21.615+0000 7f57077fe640 1 -- 172.21.15.156:0/3592757293 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187716 (secure 0 0 0) 0x7f57000785a0 con 0x7f57100fd9d0 2024-07-16T09:58:21.809 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:21.807+0000 7f57158a5640 1 -- 172.21.15.156:0/3592757293 --> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f5710110f30 con 0x7f56fc06e1e0 2024-07-16T09:58:21.810 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:21.808+0000 7f57077fe640 1 -- 172.21.15.156:0/3592757293 <== mgr.34104 v2:172.21.15.156:6800/3367765292 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+383 (secure 0 0 0) 0x7f5710110f30 con 0x7f56fc06e1e0 2024-07-16T09:58:21.813 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:21.810+0000 7f57158a5640 1 -- 172.21.15.156:0/3592757293 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f56fc06e1e0 msgr2=0x7f56fc070690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:58:21.813 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:21.810+0000 7f57158a5640 1 --2- 172.21.15.156:0/3592757293 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f56fc06e1e0 0x7f56fc070690 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f571010f000 tx=0x7f56f800b040 comp rx=0 tx=0).stop 2024-07-16T09:58:21.813 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:21.810+0000 7f57158a5640 1 -- 172.21.15.156:0/3592757293 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f57100fd9d0 msgr2=0x7f571010f250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:58:21.813 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:21.810+0000 7f57158a5640 1 --2- 172.21.15.156:0/3592757293 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f57100fd9d0 0x7f571010f250 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f5700009c20 tx=0x7f5700004d10 comp rx=0 tx=0).stop 2024-07-16T09:58:21.814 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:21.810+0000 7f57158a5640 1 -- 172.21.15.156:0/3592757293 shutdown_connections 2024-07-16T09:58:21.814 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:21.810+0000 7f57158a5640 1 --2- 172.21.15.156:0/3592757293 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f56fc06e1e0 0x7f56fc070690 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:58:21.814 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:21.810+0000 7f57158a5640 1 --2- 172.21.15.156:0/3592757293 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f57100fe320 0x7f571010f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:58:21.814 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:21.810+0000 7f57158a5640 1 --2- 172.21.15.156:0/3592757293 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f57100fd9d0 0x7f571010f250 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:58:21.814 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:21.810+0000 7f57158a5640 1 -- 172.21.15.156:0/3592757293 >> 172.21.15.156:0/3592757293 conn(0x7f57100f61d0 msgr2=0x7f57100f78f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:58:21.814 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:21.810+0000 7f57158a5640 1 -- 172.21.15.156:0/3592757293 shutdown_connections 2024-07-16T09:58:21.815 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:21.811+0000 7f57158a5640 1 -- 172.21.15.156:0/3592757293 wait complete. 2024-07-16T09:58:21.894 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:21.889+0000 7f929ad33640 1 -- 172.21.15.156:0/3688311545 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f92940fb250 msgr2=0x7f92940fb660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:58:21.895 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:21.889+0000 7f929ad33640 1 --2- 172.21.15.156:0/3688311545 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f92940fb250 0x7f92940fb660 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f92880048d0 tx=0x7f9288009da0 comp rx=0 tx=0).stop 2024-07-16T09:58:21.895 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:21.890+0000 7f929ad33640 1 -- 172.21.15.156:0/3688311545 shutdown_connections 2024-07-16T09:58:21.895 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:21.890+0000 7f929ad33640 1 --2- 172.21.15.156:0/3688311545 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f92940fc520 0x7f92940fc990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:58:21.895 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:21.890+0000 7f929ad33640 1 --2- 172.21.15.156:0/3688311545 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f92940fb250 0x7f92940fb660 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:58:21.895 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:21.890+0000 7f929ad33640 1 -- 172.21.15.156:0/3688311545 >> 172.21.15.156:0/3688311545 conn(0x7f92940f6a30 msgr2=0x7f92940f8e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:58:21.896 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:21.890+0000 7f929ad33640 1 -- 172.21.15.156:0/3688311545 shutdown_connections 2024-07-16T09:58:21.896 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:21.890+0000 7f929ad33640 1 -- 172.21.15.156:0/3688311545 wait complete. 2024-07-16T09:58:21.896 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:21.890+0000 7f929ad33640 1 Processor -- start 2024-07-16T09:58:21.896 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:21.890+0000 7f929ad33640 1 -- start start 2024-07-16T09:58:21.897 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:21.891+0000 7f929ad33640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f92940fb250 0x7f929410cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:58:21.897 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:21.891+0000 7f929ad33640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f92940fc520 0x7f929410d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:58:21.897 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:21.891+0000 7f929ad33640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f929410db40 con 0x7f92940fb250 2024-07-16T09:58:21.897 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:21.891+0000 7f929ad33640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f929410e360 con 0x7f92940fc520 2024-07-16T09:58:21.897 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:21.891+0000 7f9298acf640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f92940fb250 0x7f929410cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:58:21.897 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:21.891+0000 7f9298acf640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f92940fb250 0x7f929410cfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:50708/0 (socket says 172.21.15.156:50708) 2024-07-16T09:58:21.897 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:21.891+0000 7f9298acf640 1 -- 172.21.15.156:0/580987134 learned_addr learned my addr 172.21.15.156:0/580987134 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T09:58:21.898 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:21.891+0000 7f9293fff640 1 --2- 172.21.15.156:0/580987134 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f92940fc520 0x7f929410d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:58:21.898 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:21.891+0000 7f9298acf640 1 -- 172.21.15.156:0/580987134 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f92940fc520 msgr2=0x7f929410d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:58:21.898 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:21.891+0000 7f9298acf640 1 --2- 172.21.15.156:0/580987134 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f92940fc520 0x7f929410d4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:58:21.898 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:21.891+0000 7f9298acf640 1 -- 172.21.15.156:0/580987134 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9288005040 con 0x7f92940fb250 2024-07-16T09:58:21.898 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:21.891+0000 7f9298acf640 1 --2- 172.21.15.156:0/580987134 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f92940fb250 0x7f929410cfb0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f92880048a0 tx=0x7f9288009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:58:21.898 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:21.891+0000 7f92917fa640 1 -- 172.21.15.156:0/580987134 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9288038070 con 0x7f92940fb250 2024-07-16T09:58:21.898 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:21.892+0000 7f929ad33640 1 -- 172.21.15.156:0/580987134 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f929410e590 con 0x7f92940fb250 2024-07-16T09:58:21.898 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:21.892+0000 7f92917fa640 1 -- 172.21.15.156:0/580987134 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f92880092b0 con 0x7f92940fb250 2024-07-16T09:58:21.898 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:21.892+0000 7f92917fa640 1 -- 172.21.15.156:0/580987134 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f92880337a0 con 0x7f92940fb250 2024-07-16T09:58:21.898 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:21.892+0000 7f929ad33640 1 -- 172.21.15.156:0/580987134 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f929410ead0 con 0x7f92940fb250 2024-07-16T09:58:21.899 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:21.893+0000 7f929ad33640 1 -- 172.21.15.156:0/580987134 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92940fa6b0 con 0x7f92940fb250 2024-07-16T09:58:21.899 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:21.893+0000 7f92917fa640 1 -- 172.21.15.156:0/580987134 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 37) v1 ==== 92307+0+0 (secure 0 0 0) 0x7f9288030020 con 0x7f92940fb250 2024-07-16T09:58:21.900 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:21.894+0000 7f92917fa640 1 --2- 172.21.15.156:0/580987134 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f928406e1e0 0x7f9284070690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:58:21.900 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:21.894+0000 7f92917fa640 1 -- 172.21.15.156:0/580987134 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f92880adb00 con 0x7f92940fb250 2024-07-16T09:58:21.900 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:21.894+0000 7f9293fff640 1 --2- 172.21.15.156:0/580987134 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f928406e1e0 0x7f9284070690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:58:21.900 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:21.894+0000 7f9293fff640 1 --2- 172.21.15.156:0/580987134 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f928406e1e0 0x7f9284070690 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f929410cd60 tx=0x7f928000b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:58:21.900 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:21.898+0000 7f92917fa640 1 -- 172.21.15.156:0/580987134 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187716 (secure 0 0 0) 0x7f9288076e20 con 0x7f92940fb250 2024-07-16T09:58:22.088 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:22.086+0000 7f929ad33640 1 -- 172.21.15.156:0/580987134 --> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f9294103f20 con 0x7f928406e1e0 2024-07-16T09:58:22.097 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:22.095+0000 7f92917fa640 1 -- 172.21.15.156:0/580987134 <== mgr.34104 v2:172.21.15.156:6800/3367765292 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7f9294103f20 con 0x7f928406e1e0 2024-07-16T09:58:22.097 INFO:teuthology.orchestra.run.smithi156.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-07-16T09:58:22.097 INFO:teuthology.orchestra.run.smithi156.stdout:alertmanager.smithi156 smithi156 *:9093,9094 running (2m) 28s ago 10m 16.7M - 0.25.0 c8568f914cd2 d524a2c3ff15 2024-07-16T09:58:22.097 INFO:teuthology.orchestra.run.smithi156.stdout:crash.smithi156 smithi156 running (10m) 28s ago 10m 6899k - 17.2.7-1104-ge6231b04 d5de5f953b95 bf6d49ea06b4 2024-07-16T09:58:22.097 INFO:teuthology.orchestra.run.smithi156.stdout:crash.smithi176 smithi176 running (9m) 17s ago 9m 6903k - 17.2.7-1104-ge6231b04 d5de5f953b95 907c9fa1e890 2024-07-16T09:58:22.098 INFO:teuthology.orchestra.run.smithi156.stdout:grafana.smithi156 smithi156 *:3000 running (107s) 28s ago 10m 78.0M - 9.4.7 954c08fa6188 deb9e1bfeb66 2024-07-16T09:58:22.098 INFO:teuthology.orchestra.run.smithi156.stdout:mds.cephfs.smithi156.evwwaz smithi156 running (5m) 28s ago 5m 67.4M - 17.2.7-1104-ge6231b04 d5de5f953b95 7c0ff9ae43f8 2024-07-16T09:58:22.098 INFO:teuthology.orchestra.run.smithi156.stdout:mds.cephfs.smithi156.rdibys smithi156 running (5m) 28s ago 5m 121M - 17.2.7-1104-ge6231b04 d5de5f953b95 b985cd1e6924 2024-07-16T09:58:22.098 INFO:teuthology.orchestra.run.smithi156.stdout:mds.cephfs.smithi176.dvvbud smithi176 running (5m) 17s ago 5m 108M - 17.2.7-1104-ge6231b04 d5de5f953b95 1b00147950f9 2024-07-16T09:58:22.098 INFO:teuthology.orchestra.run.smithi156.stdout:mds.cephfs.smithi176.wahbsi smithi176 running (5m) 17s ago 5m 62.2M - 17.2.7-1104-ge6231b04 d5de5f953b95 866664af2311 2024-07-16T09:58:22.098 INFO:teuthology.orchestra.run.smithi156.stdout:mgr.smithi156.zkvylv smithi156 *:8443,9283,8765 running (90s) 28s ago 11m 472M - 19.0.0-3495-ga6ca461b 16f7feb04342 ee53bd604b7c 2024-07-16T09:58:22.098 INFO:teuthology.orchestra.run.smithi156.stdout:mgr.smithi176.dqbrvx smithi176 *:8443,9283,8765 running (53s) 17s ago 9m 437M - 19.0.0-3495-ga6ca461b 16f7feb04342 ce9eaae4c2d0 2024-07-16T09:58:22.098 INFO:teuthology.orchestra.run.smithi156.stdout:mon.smithi156 smithi156 running (40s) 28s ago 11m 26.9M 2048M 19.0.0-3495-ga6ca461b 16f7feb04342 77b6ec8c50eb 2024-07-16T09:58:22.098 INFO:teuthology.orchestra.run.smithi156.stdout:mon.smithi176 smithi176 running (21s) 17s ago 9m 23.3M 2048M 19.0.0-3495-ga6ca461b 16f7feb04342 ef92814cec2b 2024-07-16T09:58:22.098 INFO:teuthology.orchestra.run.smithi156.stdout:node-exporter.smithi156 smithi156 *:9100 running (2m) 28s ago 10m 8242k - 1.5.0 0da6a335fe13 e5d86f23a14c 2024-07-16T09:58:22.098 INFO:teuthology.orchestra.run.smithi156.stdout:node-exporter.smithi176 smithi176 *:9100 running (99s) 17s ago 9m 15.7M - 1.5.0 0da6a335fe13 ea842728144a 2024-07-16T09:58:22.098 INFO:teuthology.orchestra.run.smithi156.stdout:osd.0 smithi156 running (8m) 28s ago 8m 222M 1557M 17.2.7-1104-ge6231b04 d5de5f953b95 0471758ec349 2024-07-16T09:58:22.099 INFO:teuthology.orchestra.run.smithi156.stdout:osd.1 smithi156 running (8m) 28s ago 8m 246M 1557M 17.2.7-1104-ge6231b04 d5de5f953b95 149a0bda2f61 2024-07-16T09:58:22.099 INFO:teuthology.orchestra.run.smithi156.stdout:osd.2 smithi156 running (8m) 28s ago 8m 198M 1557M 17.2.7-1104-ge6231b04 d5de5f953b95 f28acec5cb79 2024-07-16T09:58:22.099 INFO:teuthology.orchestra.run.smithi156.stdout:osd.3 smithi176 running (7m) 17s ago 7m 259M 2581M 17.2.7-1104-ge6231b04 d5de5f953b95 733861ba84dc 2024-07-16T09:58:22.099 INFO:teuthology.orchestra.run.smithi156.stdout:osd.4 smithi176 running (7m) 17s ago 7m 229M 2581M 17.2.7-1104-ge6231b04 d5de5f953b95 ef0c67cec086 2024-07-16T09:58:22.099 INFO:teuthology.orchestra.run.smithi156.stdout:osd.5 smithi176 running (6m) 17s ago 6m 224M 2581M 17.2.7-1104-ge6231b04 d5de5f953b95 996e15448a52 2024-07-16T09:58:22.099 INFO:teuthology.orchestra.run.smithi156.stdout:prometheus.smithi156 smithi156 *:9095 running (58s) 28s ago 10m 48.8M - 2.43.0 a07b618ecd1d 20c195ecd665 2024-07-16T09:58:22.100 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:22.097+0000 7f929ad33640 1 -- 172.21.15.156:0/580987134 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f928406e1e0 msgr2=0x7f9284070690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:58:22.100 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:22.097+0000 7f929ad33640 1 --2- 172.21.15.156:0/580987134 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f928406e1e0 0x7f9284070690 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f929410cd60 tx=0x7f928000b040 comp rx=0 tx=0).stop 2024-07-16T09:58:22.100 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:22.097+0000 7f929ad33640 1 -- 172.21.15.156:0/580987134 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f92940fb250 msgr2=0x7f929410cfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:58:22.100 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:22.097+0000 7f929ad33640 1 --2- 172.21.15.156:0/580987134 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f92940fb250 0x7f929410cfb0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f92880048a0 tx=0x7f9288009da0 comp rx=0 tx=0).stop 2024-07-16T09:58:22.100 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:22.097+0000 7f929ad33640 1 -- 172.21.15.156:0/580987134 shutdown_connections 2024-07-16T09:58:22.100 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:22.097+0000 7f929ad33640 1 --2- 172.21.15.156:0/580987134 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f928406e1e0 0x7f9284070690 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:58:22.100 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:22.097+0000 7f929ad33640 1 --2- 172.21.15.156:0/580987134 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f92940fc520 0x7f929410d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:58:22.100 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:22.097+0000 7f929ad33640 1 --2- 172.21.15.156:0/580987134 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f92940fb250 0x7f929410cfb0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:58:22.100 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:22.097+0000 7f929ad33640 1 -- 172.21.15.156:0/580987134 >> 172.21.15.156:0/580987134 conn(0x7f92940f6a30 msgr2=0x7f9294065db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:58:22.100 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:22.097+0000 7f929ad33640 1 -- 172.21.15.156:0/580987134 shutdown_connections 2024-07-16T09:58:22.100 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:22.097+0000 7f929ad33640 1 -- 172.21.15.156:0/580987134 wait complete. 2024-07-16T09:58:22.180 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:22.176+0000 7fe7a1775640 1 -- 172.21.15.156:0/3247691041 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe79c0fb240 msgr2=0x7fe79c0fb650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:58:22.180 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:22.176+0000 7fe7a1775640 1 --2- 172.21.15.156:0/3247691041 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe79c0fb240 0x7fe79c0fb650 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fe78c0048d0 tx=0x7fe78c009da0 comp rx=0 tx=0).stop 2024-07-16T09:58:22.180 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:22.176+0000 7fe7a1775640 1 -- 172.21.15.156:0/3247691041 shutdown_connections 2024-07-16T09:58:22.180 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:22.176+0000 7fe7a1775640 1 --2- 172.21.15.156:0/3247691041 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe79c0fc510 0x7fe79c0fc980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:58:22.180 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:22.176+0000 7fe7a1775640 1 --2- 172.21.15.156:0/3247691041 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe79c0fb240 0x7fe79c0fb650 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:58:22.180 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:22.176+0000 7fe7a1775640 1 -- 172.21.15.156:0/3247691041 >> 172.21.15.156:0/3247691041 conn(0x7fe79c0f6a20 msgr2=0x7fe79c0f8e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:58:22.180 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:22.176+0000 7fe7a1775640 1 -- 172.21.15.156:0/3247691041 shutdown_connections 2024-07-16T09:58:22.181 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:22.176+0000 7fe7a1775640 1 -- 172.21.15.156:0/3247691041 wait complete. 2024-07-16T09:58:22.181 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:22.177+0000 7fe7a1775640 1 Processor -- start 2024-07-16T09:58:22.181 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:22.177+0000 7fe7a1775640 1 -- start start 2024-07-16T09:58:22.181 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:22.177+0000 7fe7a1775640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe79c0fb240 0x7fe79c10cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:58:22.181 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:22.177+0000 7fe7a1775640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe79c0fc510 0x7fe79c10d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:58:22.181 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:22.177+0000 7fe7a1775640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe79c10db20 con 0x7fe79c0fb240 2024-07-16T09:58:22.181 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:22.177+0000 7fe7a1775640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe79c10e2f0 con 0x7fe79c0fc510 2024-07-16T09:58:22.181 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:22.177+0000 7fe79affd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe79c0fb240 0x7fe79c10cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:58:22.181 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:22.177+0000 7fe79affd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe79c0fb240 0x7fe79c10cf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:50716/0 (socket says 172.21.15.156:50716) 2024-07-16T09:58:22.181 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:22.177+0000 7fe79affd640 1 -- 172.21.15.156:0/2963281108 learned_addr learned my addr 172.21.15.156:0/2963281108 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T09:58:22.181 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:22.177+0000 7fe793fff640 1 --2- 172.21.15.156:0/2963281108 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe79c0fc510 0x7fe79c10d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:58:22.181 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:22.177+0000 7fe79affd640 1 -- 172.21.15.156:0/2963281108 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe79c0fc510 msgr2=0x7fe79c10d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:58:22.182 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:22.177+0000 7fe79affd640 1 --2- 172.21.15.156:0/2963281108 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe79c0fc510 0x7fe79c10d4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:58:22.182 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:22.177+0000 7fe79affd640 1 -- 172.21.15.156:0/2963281108 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe78c005040 con 0x7fe79c0fb240 2024-07-16T09:58:22.182 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:22.177+0000 7fe79affd640 1 --2- 172.21.15.156:0/2963281108 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe79c0fb240 0x7fe79c10cf90 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fe78c009f80 tx=0x7fe78c009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:58:22.182 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:22.178+0000 7fe7937fe640 1 -- 172.21.15.156:0/2963281108 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe78c038070 con 0x7fe79c0fb240 2024-07-16T09:58:22.182 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:22.178+0000 7fe7937fe640 1 -- 172.21.15.156:0/2963281108 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe78c032970 con 0x7fe79c0fb240 2024-07-16T09:58:22.182 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:22.178+0000 7fe7937fe640 1 -- 172.21.15.156:0/2963281108 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe78c033860 con 0x7fe79c0fb240 2024-07-16T09:58:22.182 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:22.178+0000 7fe7a1775640 1 -- 172.21.15.156:0/2963281108 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe79c10e520 con 0x7fe79c0fb240 2024-07-16T09:58:22.182 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:22.178+0000 7fe7a1775640 1 -- 172.21.15.156:0/2963281108 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe79c10ea10 con 0x7fe79c0fb240 2024-07-16T09:58:22.183 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:22.179+0000 7fe7a1775640 1 -- 172.21.15.156:0/2963281108 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe79c0fa6a0 con 0x7fe79c0fb240 2024-07-16T09:58:22.183 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:22.179+0000 7fe7937fe640 1 -- 172.21.15.156:0/2963281108 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 37) v1 ==== 92307+0+0 (secure 0 0 0) 0x7fe78c030020 con 0x7fe79c0fb240 2024-07-16T09:58:22.183 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:22.180+0000 7fe7937fe640 1 --2- 172.21.15.156:0/2963281108 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7fe78806e1e0 0x7fe788070690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:58:22.183 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:22.180+0000 7fe793fff640 1 --2- 172.21.15.156:0/2963281108 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7fe78806e1e0 0x7fe788070690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:58:22.183 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:22.180+0000 7fe7937fe640 1 -- 172.21.15.156:0/2963281108 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fe78c0adb70 con 0x7fe79c0fb240 2024-07-16T09:58:22.183 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:22.180+0000 7fe793fff640 1 --2- 172.21.15.156:0/2963281108 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7fe78806e1e0 0x7fe788070690 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fe79c10cd40 tx=0x7fe78400b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:58:22.186 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:22.184+0000 7fe7937fe640 1 -- 172.21.15.156:0/2963281108 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187716 (secure 0 0 0) 0x7fe78c076e90 con 0x7fe79c0fb240 2024-07-16T09:58:22.434 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:22.431+0000 7fe7a1775640 1 -- 172.21.15.156:0/2963281108 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fe79c103eb0 con 0x7fe79c0fb240 2024-07-16T09:58:22.434 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:22 smithi156 ceph-mon[93066]: Reconfiguring crash.smithi176 (monmap changed)... 2024-07-16T09:58:22.434 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:22 smithi156 ceph-mon[93066]: Reconfiguring daemon crash.smithi176 on smithi176 2024-07-16T09:58:22.434 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:22 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T09:58:22.434 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:22 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T09:58:22.434 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:22 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi176.dqbrvx", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-07-16T09:58:22.434 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:22 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "mgr services"}]: dispatch 2024-07-16T09:58:22.435 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:22 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-16T09:58:22.435 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:22.432+0000 7fe7937fe640 1 -- 172.21.15.156:0/2963281108 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+736 (secure 0 0 0) 0x7fe78c07a550 con 0x7fe79c0fb240 2024-07-16T09:58:22.435 INFO:teuthology.orchestra.run.smithi156.stdout:{ 2024-07-16T09:58:22.435 INFO:teuthology.orchestra.run.smithi156.stdout: "mon": { 2024-07-16T09:58:22.435 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph version 19.0.0-3495-ga6ca461b (a6ca461be7cc1aff3658c2c6df4cff975a24449a) squid (rc)": 2 2024-07-16T09:58:22.435 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-07-16T09:58:22.435 INFO:teuthology.orchestra.run.smithi156.stdout: "mgr": { 2024-07-16T09:58:22.435 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph version 19.0.0-3495-ga6ca461b (a6ca461be7cc1aff3658c2c6df4cff975a24449a) squid (rc)": 2 2024-07-16T09:58:22.435 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-07-16T09:58:22.436 INFO:teuthology.orchestra.run.smithi156.stdout: "osd": { 2024-07-16T09:58:22.436 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph version 17.2.7-1104-ge6231b04 (e6231b04b52ebf3b9b31803282dd3074439e51ac) quincy (stable)": 6 2024-07-16T09:58:22.436 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-07-16T09:58:22.436 INFO:teuthology.orchestra.run.smithi156.stdout: "mds": { 2024-07-16T09:58:22.436 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph version 17.2.7-1104-ge6231b04 (e6231b04b52ebf3b9b31803282dd3074439e51ac) quincy (stable)": 4 2024-07-16T09:58:22.436 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-07-16T09:58:22.436 INFO:teuthology.orchestra.run.smithi156.stdout: "overall": { 2024-07-16T09:58:22.436 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph version 17.2.7-1104-ge6231b04 (e6231b04b52ebf3b9b31803282dd3074439e51ac) quincy (stable)": 10, 2024-07-16T09:58:22.436 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph version 19.0.0-3495-ga6ca461b (a6ca461be7cc1aff3658c2c6df4cff975a24449a) squid (rc)": 4 2024-07-16T09:58:22.436 INFO:teuthology.orchestra.run.smithi156.stdout: } 2024-07-16T09:58:22.436 INFO:teuthology.orchestra.run.smithi156.stdout:} 2024-07-16T09:58:22.437 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:22.434+0000 7fe7a1775640 1 -- 172.21.15.156:0/2963281108 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7fe78806e1e0 msgr2=0x7fe788070690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:58:22.437 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:22.434+0000 7fe7a1775640 1 --2- 172.21.15.156:0/2963281108 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7fe78806e1e0 0x7fe788070690 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fe79c10cd40 tx=0x7fe78400b040 comp rx=0 tx=0).stop 2024-07-16T09:58:22.437 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:22.434+0000 7fe7a1775640 1 -- 172.21.15.156:0/2963281108 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe79c0fb240 msgr2=0x7fe79c10cf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:58:22.437 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:22.434+0000 7fe7a1775640 1 --2- 172.21.15.156:0/2963281108 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe79c0fb240 0x7fe79c10cf90 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fe78c009f80 tx=0x7fe78c009da0 comp rx=0 tx=0).stop 2024-07-16T09:58:22.437 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:22.434+0000 7fe7a1775640 1 -- 172.21.15.156:0/2963281108 shutdown_connections 2024-07-16T09:58:22.437 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:22.434+0000 7fe7a1775640 1 --2- 172.21.15.156:0/2963281108 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7fe78806e1e0 0x7fe788070690 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:58:22.437 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:22.434+0000 7fe7a1775640 1 --2- 172.21.15.156:0/2963281108 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe79c0fc510 0x7fe79c10d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:58:22.438 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:22.434+0000 7fe7a1775640 1 --2- 172.21.15.156:0/2963281108 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe79c0fb240 0x7fe79c10cf90 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:58:22.438 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:22.434+0000 7fe7a1775640 1 -- 172.21.15.156:0/2963281108 >> 172.21.15.156:0/2963281108 conn(0x7fe79c0f6a20 msgr2=0x7fe79c065330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:58:22.438 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:22.434+0000 7fe7a1775640 1 -- 172.21.15.156:0/2963281108 shutdown_connections 2024-07-16T09:58:22.438 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:22.434+0000 7fe7a1775640 1 -- 172.21.15.156:0/2963281108 wait complete. 2024-07-16T09:58:22.517 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:22.514+0000 7f8646d7f640 1 -- 172.21.15.156:0/1563876363 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f86400fc360 msgr2=0x7f86400fc770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:58:22.517 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:22.514+0000 7f8646d7f640 1 --2- 172.21.15.156:0/1563876363 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f86400fc360 0x7f86400fc770 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f86340048d0 tx=0x7f8634009da0 comp rx=0 tx=0).stop 2024-07-16T09:58:22.518 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:22.514+0000 7f8646d7f640 1 -- 172.21.15.156:0/1563876363 shutdown_connections 2024-07-16T09:58:22.518 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:22.514+0000 7f8646d7f640 1 --2- 172.21.15.156:0/1563876363 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f86400fd630 0x7f86400fdaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:58:22.518 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:22.514+0000 7f8646d7f640 1 --2- 172.21.15.156:0/1563876363 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f86400fc360 0x7f86400fc770 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:58:22.518 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:22.514+0000 7f8646d7f640 1 -- 172.21.15.156:0/1563876363 >> 172.21.15.156:0/1563876363 conn(0x7f86400f7b40 msgr2=0x7f86400f9f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:58:22.518 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:22.514+0000 7f8646d7f640 1 -- 172.21.15.156:0/1563876363 shutdown_connections 2024-07-16T09:58:22.518 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:22.514+0000 7f8646d7f640 1 -- 172.21.15.156:0/1563876363 wait complete. 2024-07-16T09:58:22.518 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:22.514+0000 7f8646d7f640 1 Processor -- start 2024-07-16T09:58:22.518 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:22.514+0000 7f8646d7f640 1 -- start start 2024-07-16T09:58:22.518 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:22.515+0000 7f8646d7f640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f86400fc360 0x7f864010f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:58:22.518 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:22.515+0000 7f8646d7f640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f86400fd630 0x7f864010f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:58:22.518 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:22.515+0000 7f8646d7f640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f864010fd20 con 0x7f86400fd630 2024-07-16T09:58:22.518 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:22.515+0000 7f8646d7f640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8640110540 con 0x7f86400fc360 2024-07-16T09:58:22.519 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:22.515+0000 7f863ffff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f86400fd630 0x7f864010f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:58:22.519 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:22.515+0000 7f863ffff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f86400fd630 0x7f864010f6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:50734/0 (socket says 172.21.15.156:50734) 2024-07-16T09:58:22.519 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:22.515+0000 7f863ffff640 1 -- 172.21.15.156:0/1201167564 learned_addr learned my addr 172.21.15.156:0/1201167564 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T09:58:22.519 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:22.515+0000 7f8644b1b640 1 --2- 172.21.15.156:0/1201167564 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f86400fc360 0x7f864010f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:58:22.519 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:22.515+0000 7f863ffff640 1 -- 172.21.15.156:0/1201167564 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f86400fc360 msgr2=0x7f864010f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:58:22.519 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:22.515+0000 7f863ffff640 1 --2- 172.21.15.156:0/1201167564 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f86400fc360 0x7f864010f190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:58:22.519 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:22.515+0000 7f863ffff640 1 -- 172.21.15.156:0/1201167564 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8634005040 con 0x7f86400fd630 2024-07-16T09:58:22.519 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:22.515+0000 7f863ffff640 1 --2- 172.21.15.156:0/1201167564 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f86400fd630 0x7f864010f6d0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f862c00eb70 tx=0x7f862c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:58:22.520 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:22.516+0000 7f863d7fa640 1 -- 172.21.15.156:0/1201167564 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f862c007c30 con 0x7f86400fd630 2024-07-16T09:58:22.520 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:22.516+0000 7f863d7fa640 1 -- 172.21.15.156:0/1201167564 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f862c00f040 con 0x7f86400fd630 2024-07-16T09:58:22.520 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:22.516+0000 7f8646d7f640 1 -- 172.21.15.156:0/1201167564 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8640110770 con 0x7f86400fd630 2024-07-16T09:58:22.521 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:22.516+0000 7f863d7fa640 1 -- 172.21.15.156:0/1201167564 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f862c00cad0 con 0x7f86400fd630 2024-07-16T09:58:22.521 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:22.516+0000 7f8646d7f640 1 -- 172.21.15.156:0/1201167564 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8640110d10 con 0x7f86400fd630 2024-07-16T09:58:22.521 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:22.517+0000 7f8646d7f640 1 -- 172.21.15.156:0/1201167564 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86400fb7c0 con 0x7f86400fd630 2024-07-16T09:58:22.521 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:22.517+0000 7f863d7fa640 1 -- 172.21.15.156:0/1201167564 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 37) v1 ==== 92307+0+0 (secure 0 0 0) 0x7f862c00cc50 con 0x7f86400fd630 2024-07-16T09:58:22.521 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:22.518+0000 7f863d7fa640 1 --2- 172.21.15.156:0/1201167564 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f863006e1e0 0x7f8630070690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:58:22.521 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:22.518+0000 7f863d7fa640 1 -- 172.21.15.156:0/1201167564 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f862c08da10 con 0x7f86400fd630 2024-07-16T09:58:22.521 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:22.518+0000 7f8644b1b640 1 --2- 172.21.15.156:0/1201167564 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f863006e1e0 0x7f8630070690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:58:22.521 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:22.518+0000 7f8644b1b640 1 --2- 172.21.15.156:0/1201167564 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f863006e1e0 0x7f8630070690 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f8634009f80 tx=0x7f8634035040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:58:22.524 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:22.522+0000 7f863d7fa640 1 -- 172.21.15.156:0/1201167564 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187716 (secure 0 0 0) 0x7f862c056c20 con 0x7f86400fd630 2024-07-16T09:58:22.582 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:22 smithi176 ceph-mon[81689]: Reconfiguring crash.smithi176 (monmap changed)... 2024-07-16T09:58:22.582 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:22 smithi176 ceph-mon[81689]: Reconfiguring daemon crash.smithi176 on smithi176 2024-07-16T09:58:22.582 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:22 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T09:58:22.583 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:22 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T09:58:22.583 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:22 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi176.dqbrvx", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-07-16T09:58:22.583 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:22 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "mgr services"}]: dispatch 2024-07-16T09:58:22.583 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:22 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-16T09:58:22.737 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:22.734+0000 7f8646d7f640 1 -- 172.21.15.156:0/1201167564 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f864004b530 con 0x7f86400fd630 2024-07-16T09:58:22.737 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:22.735+0000 7f863d7fa640 1 -- 172.21.15.156:0/1201167564 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 19 v19) v1 ==== 76+0+1937 (secure 0 0 0) 0x7f862c05a2e0 con 0x7f86400fd630 2024-07-16T09:58:22.737 INFO:teuthology.orchestra.run.smithi156.stdout:e19 2024-07-16T09:58:22.737 INFO:teuthology.orchestra.run.smithi156.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-07-16T09:58:22.738 INFO:teuthology.orchestra.run.smithi156.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-07-16T09:58:22.738 INFO:teuthology.orchestra.run.smithi156.stdout:legacy client fscid: 1 2024-07-16T09:58:22.738 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-07-16T09:58:22.738 INFO:teuthology.orchestra.run.smithi156.stdout:Filesystem 'cephfs' (1) 2024-07-16T09:58:22.738 INFO:teuthology.orchestra.run.smithi156.stdout:fs_name cephfs 2024-07-16T09:58:22.738 INFO:teuthology.orchestra.run.smithi156.stdout:epoch 19 2024-07-16T09:58:22.738 INFO:teuthology.orchestra.run.smithi156.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-07-16T09:58:22.738 INFO:teuthology.orchestra.run.smithi156.stdout:created 2024-07-16T09:52:55.214975+0000 2024-07-16T09:58:22.738 INFO:teuthology.orchestra.run.smithi156.stdout:modified 2024-07-16T09:58:14.542481+0000 2024-07-16T09:58:22.738 INFO:teuthology.orchestra.run.smithi156.stdout:tableserver 0 2024-07-16T09:58:22.738 INFO:teuthology.orchestra.run.smithi156.stdout:root 0 2024-07-16T09:58:22.738 INFO:teuthology.orchestra.run.smithi156.stdout:session_timeout 60 2024-07-16T09:58:22.738 INFO:teuthology.orchestra.run.smithi156.stdout:session_autoclose 300 2024-07-16T09:58:22.739 INFO:teuthology.orchestra.run.smithi156.stdout:max_file_size 1099511627776 2024-07-16T09:58:22.739 INFO:teuthology.orchestra.run.smithi156.stdout:max_xattr_size 65536 2024-07-16T09:58:22.739 INFO:teuthology.orchestra.run.smithi156.stdout:required_client_features {} 2024-07-16T09:58:22.739 INFO:teuthology.orchestra.run.smithi156.stdout:last_failure 0 2024-07-16T09:58:22.739 INFO:teuthology.orchestra.run.smithi156.stdout:last_failure_osd_epoch 0 2024-07-16T09:58:22.739 INFO:teuthology.orchestra.run.smithi156.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-07-16T09:58:22.739 INFO:teuthology.orchestra.run.smithi156.stdout:max_mds 2 2024-07-16T09:58:22.739 INFO:teuthology.orchestra.run.smithi156.stdout:in 0,1 2024-07-16T09:58:22.739 INFO:teuthology.orchestra.run.smithi156.stdout:up {0=14458,1=24261} 2024-07-16T09:58:22.739 INFO:teuthology.orchestra.run.smithi156.stdout:failed 2024-07-16T09:58:22.739 INFO:teuthology.orchestra.run.smithi156.stdout:damaged 2024-07-16T09:58:22.739 INFO:teuthology.orchestra.run.smithi156.stdout:stopped 2024-07-16T09:58:22.739 INFO:teuthology.orchestra.run.smithi156.stdout:data_pools [3] 2024-07-16T09:58:22.740 INFO:teuthology.orchestra.run.smithi156.stdout:metadata_pool 2 2024-07-16T09:58:22.740 INFO:teuthology.orchestra.run.smithi156.stdout:inline_data disabled 2024-07-16T09:58:22.740 INFO:teuthology.orchestra.run.smithi156.stdout:balancer 2024-07-16T09:58:22.740 INFO:teuthology.orchestra.run.smithi156.stdout:bal_rank_mask -1 2024-07-16T09:58:22.740 INFO:teuthology.orchestra.run.smithi156.stdout:standby_count_wanted 1 2024-07-16T09:58:22.740 INFO:teuthology.orchestra.run.smithi156.stdout:qdb_cluster leader: 14458 members: 14458,24261 2024-07-16T09:58:22.740 INFO:teuthology.orchestra.run.smithi156.stdout:[mds.cephfs.smithi156.rdibys{0:14458} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.156:6826/1080176013,v1:172.21.15.156:6827/1080176013] compat {c=[1],r=[1],i=[7ff]}] 2024-07-16T09:58:22.740 INFO:teuthology.orchestra.run.smithi156.stdout:[mds.cephfs.smithi156.evwwaz{0:14468} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.156:6828/560329502,v1:172.21.15.156:6829/560329502] compat {c=[1],r=[1],i=[7ff]}] 2024-07-16T09:58:22.740 INFO:teuthology.orchestra.run.smithi156.stdout:[mds.cephfs.smithi176.dvvbud{1:24261} state up:active seq 4 export targets 0 join_fscid=1 addr [v2:172.21.15.176:6824/1577019287,v1:172.21.15.176:6825/1577019287] compat {c=[1],r=[1],i=[7ff]}] 2024-07-16T09:58:22.740 INFO:teuthology.orchestra.run.smithi156.stdout:[mds.cephfs.smithi176.wahbsi{1:24273} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.176:6826/2196775412,v1:172.21.15.176:6827/2196775412] compat {c=[1],r=[1],i=[7ff]}] 2024-07-16T09:58:22.740 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-07-16T09:58:22.740 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-07-16T09:58:22.741 INFO:teuthology.orchestra.run.smithi156.stderr:dumped fsmap epoch 19 2024-07-16T09:58:22.741 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:22.737+0000 7f8646d7f640 1 -- 172.21.15.156:0/1201167564 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f863006e1e0 msgr2=0x7f8630070690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:58:22.741 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:22.737+0000 7f8646d7f640 1 --2- 172.21.15.156:0/1201167564 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f863006e1e0 0x7f8630070690 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f8634009f80 tx=0x7f8634035040 comp rx=0 tx=0).stop 2024-07-16T09:58:22.741 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:22.737+0000 7f8646d7f640 1 -- 172.21.15.156:0/1201167564 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f86400fd630 msgr2=0x7f864010f6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:58:22.741 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:22.737+0000 7f8646d7f640 1 --2- 172.21.15.156:0/1201167564 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f86400fd630 0x7f864010f6d0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f862c00eb70 tx=0x7f862c00b040 comp rx=0 tx=0).stop 2024-07-16T09:58:22.742 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:22.737+0000 7f8646d7f640 1 -- 172.21.15.156:0/1201167564 shutdown_connections 2024-07-16T09:58:22.742 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:22.737+0000 7f8646d7f640 1 --2- 172.21.15.156:0/1201167564 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f863006e1e0 0x7f8630070690 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:58:22.742 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:22.737+0000 7f8646d7f640 1 --2- 172.21.15.156:0/1201167564 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f86400fd630 0x7f864010f6d0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:58:22.742 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:22.737+0000 7f8646d7f640 1 --2- 172.21.15.156:0/1201167564 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f86400fc360 0x7f864010f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:58:22.742 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:22.737+0000 7f8646d7f640 1 -- 172.21.15.156:0/1201167564 >> 172.21.15.156:0/1201167564 conn(0x7f86400f7b40 msgr2=0x7f86400f9170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:58:22.742 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:22.737+0000 7f8646d7f640 1 -- 172.21.15.156:0/1201167564 shutdown_connections 2024-07-16T09:58:22.742 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:22.737+0000 7f8646d7f640 1 -- 172.21.15.156:0/1201167564 wait complete. 2024-07-16T09:58:22.822 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:22.818+0000 7f14cbb78640 1 -- 172.21.15.156:0/1426138176 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f14c40fd2a0 msgr2=0x7f14c40ff720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:58:22.822 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:22.818+0000 7f14cbb78640 1 --2- 172.21.15.156:0/1426138176 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f14c40fd2a0 0x7f14c40ff720 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f14b40048d0 tx=0x7f14b4009da0 comp rx=0 tx=0).stop 2024-07-16T09:58:22.822 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:22.818+0000 7f14cbb78640 1 -- 172.21.15.156:0/1426138176 shutdown_connections 2024-07-16T09:58:22.822 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:22.818+0000 7f14cbb78640 1 --2- 172.21.15.156:0/1426138176 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f14c40fd2a0 0x7f14c40ff720 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:58:22.822 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:22.818+0000 7f14cbb78640 1 --2- 172.21.15.156:0/1426138176 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f14c40665a0 0x7f14c40fcd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:58:22.822 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:22.818+0000 7f14cbb78640 1 -- 172.21.15.156:0/1426138176 >> 172.21.15.156:0/1426138176 conn(0x7f14c40f6a20 msgr2=0x7f14c40f8e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:58:22.822 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:22.818+0000 7f14cbb78640 1 -- 172.21.15.156:0/1426138176 shutdown_connections 2024-07-16T09:58:22.823 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:22.818+0000 7f14cbb78640 1 -- 172.21.15.156:0/1426138176 wait complete. 2024-07-16T09:58:22.823 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:22.819+0000 7f14cbb78640 1 Processor -- start 2024-07-16T09:58:22.823 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:22.819+0000 7f14cbb78640 1 -- start start 2024-07-16T09:58:22.823 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:22.819+0000 7f14cbb78640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f14c40665a0 0x7f14c41135b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:58:22.823 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:22.819+0000 7f14cbb78640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f14c40fd2a0 0x7f14c4113af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:58:22.823 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:22.819+0000 7f14cbb78640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14c4114140 con 0x7f14c40665a0 2024-07-16T09:58:22.823 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:22.819+0000 7f14cbb78640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14c4114960 con 0x7f14c40fd2a0 2024-07-16T09:58:22.823 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:22.819+0000 7f14c9914640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f14c40665a0 0x7f14c41135b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:58:22.823 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:22.819+0000 7f14c9914640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f14c40665a0 0x7f14c41135b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:39568/0 (socket says 172.21.15.156:39568) 2024-07-16T09:58:22.823 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:22.819+0000 7f14c9914640 1 -- 172.21.15.156:0/891342695 learned_addr learned my addr 172.21.15.156:0/891342695 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T09:58:22.823 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:22.819+0000 7f14c9113640 1 --2- 172.21.15.156:0/891342695 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f14c40fd2a0 0x7f14c4113af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:58:22.823 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:22.819+0000 7f14c9914640 1 -- 172.21.15.156:0/891342695 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f14c40fd2a0 msgr2=0x7f14c4113af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:58:22.823 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:22.819+0000 7f14c9914640 1 --2- 172.21.15.156:0/891342695 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f14c40fd2a0 0x7f14c4113af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:58:22.824 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:22.819+0000 7f14c9914640 1 -- 172.21.15.156:0/891342695 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14b4005040 con 0x7f14c40665a0 2024-07-16T09:58:22.824 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:22.820+0000 7f14c9914640 1 --2- 172.21.15.156:0/891342695 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f14c40665a0 0x7f14c41135b0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f14c0002a80 tx=0x7f14c0004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:58:22.824 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:22.820+0000 7f14ba7fc640 1 -- 172.21.15.156:0/891342695 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f14c0007c30 con 0x7f14c40665a0 2024-07-16T09:58:22.824 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:22.820+0000 7f14ba7fc640 1 -- 172.21.15.156:0/891342695 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f14c000f040 con 0x7f14c40665a0 2024-07-16T09:58:22.824 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:22.820+0000 7f14cbb78640 1 -- 172.21.15.156:0/891342695 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14c4114bf0 con 0x7f14c40665a0 2024-07-16T09:58:22.824 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:22.820+0000 7f14ba7fc640 1 -- 172.21.15.156:0/891342695 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f14c000dbe0 con 0x7f14c40665a0 2024-07-16T09:58:22.824 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:22.820+0000 7f14cbb78640 1 -- 172.21.15.156:0/891342695 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14c41150e0 con 0x7f14c40665a0 2024-07-16T09:58:22.825 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:22.821+0000 7f14cbb78640 1 -- 172.21.15.156:0/891342695 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14c410bcc0 con 0x7f14c40665a0 2024-07-16T09:58:22.825 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:22.821+0000 7f14ba7fc640 1 -- 172.21.15.156:0/891342695 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 37) v1 ==== 92307+0+0 (secure 0 0 0) 0x7f14c000dd40 con 0x7f14c40665a0 2024-07-16T09:58:22.825 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:22.822+0000 7f14ba7fc640 1 --2- 172.21.15.156:0/891342695 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f14b006e1e0 0x7f14b0070690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:58:22.825 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:22.822+0000 7f14ba7fc640 1 -- 172.21.15.156:0/891342695 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f14c008dad0 con 0x7f14c40665a0 2024-07-16T09:58:22.825 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:22.822+0000 7f14c9113640 1 --2- 172.21.15.156:0/891342695 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f14b006e1e0 0x7f14b0070690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:58:22.825 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:22.822+0000 7f14c9113640 1 --2- 172.21.15.156:0/891342695 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f14b006e1e0 0x7f14b0070690 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f14b40043d0 tx=0x7f14b4035040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:58:22.828 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:22.827+0000 7f14ba7fc640 1 -- 172.21.15.156:0/891342695 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187716 (secure 0 0 0) 0x7f14c0056c30 con 0x7f14c40665a0 2024-07-16T09:58:23.019 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:23.016+0000 7f14cbb78640 1 -- 172.21.15.156:0/891342695 --> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f14c4115440 con 0x7f14b006e1e0 2024-07-16T09:58:23.020 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:23.018+0000 7f14ba7fc640 1 -- 172.21.15.156:0/891342695 <== mgr.34104 v2:172.21.15.156:6800/3367765292 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+383 (secure 0 0 0) 0x7f14c4115440 con 0x7f14b006e1e0 2024-07-16T09:58:23.020 INFO:teuthology.orchestra.run.smithi156.stdout:{ 2024-07-16T09:58:23.020 INFO:teuthology.orchestra.run.smithi156.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:c15ec3aa8e6c0a9c12ee24ff79f3e7dd4d01b3d1a702dfeb7d7fff9b0811a393", 2024-07-16T09:58:23.020 INFO:teuthology.orchestra.run.smithi156.stdout: "in_progress": true, 2024-07-16T09:58:23.020 INFO:teuthology.orchestra.run.smithi156.stdout: "which": "Upgrading all daemon types on all hosts", 2024-07-16T09:58:23.020 INFO:teuthology.orchestra.run.smithi156.stdout: "services_complete": [ 2024-07-16T09:58:23.020 INFO:teuthology.orchestra.run.smithi156.stdout: "mon", 2024-07-16T09:58:23.020 INFO:teuthology.orchestra.run.smithi156.stdout: "mgr" 2024-07-16T09:58:23.020 INFO:teuthology.orchestra.run.smithi156.stdout: ], 2024-07-16T09:58:23.021 INFO:teuthology.orchestra.run.smithi156.stdout: "progress": "4/21 daemons upgraded", 2024-07-16T09:58:23.021 INFO:teuthology.orchestra.run.smithi156.stdout: "message": "Currently upgrading mon daemons", 2024-07-16T09:58:23.021 INFO:teuthology.orchestra.run.smithi156.stdout: "is_paused": false 2024-07-16T09:58:23.021 INFO:teuthology.orchestra.run.smithi156.stdout:} 2024-07-16T09:58:23.022 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:23.020+0000 7f14cbb78640 1 -- 172.21.15.156:0/891342695 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f14b006e1e0 msgr2=0x7f14b0070690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:58:23.022 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:23.020+0000 7f14cbb78640 1 --2- 172.21.15.156:0/891342695 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f14b006e1e0 0x7f14b0070690 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f14b40043d0 tx=0x7f14b4035040 comp rx=0 tx=0).stop 2024-07-16T09:58:23.022 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:23.020+0000 7f14cbb78640 1 -- 172.21.15.156:0/891342695 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f14c40665a0 msgr2=0x7f14c41135b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:58:23.022 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:23.020+0000 7f14cbb78640 1 --2- 172.21.15.156:0/891342695 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f14c40665a0 0x7f14c41135b0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f14c0002a80 tx=0x7f14c0004600 comp rx=0 tx=0).stop 2024-07-16T09:58:23.022 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:23.020+0000 7f14cbb78640 1 -- 172.21.15.156:0/891342695 shutdown_connections 2024-07-16T09:58:23.022 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:23.020+0000 7f14cbb78640 1 --2- 172.21.15.156:0/891342695 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f14b006e1e0 0x7f14b0070690 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:58:23.023 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:23.020+0000 7f14cbb78640 1 --2- 172.21.15.156:0/891342695 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f14c40fd2a0 0x7f14c4113af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:58:23.023 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:23.020+0000 7f14cbb78640 1 --2- 172.21.15.156:0/891342695 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f14c40665a0 0x7f14c41135b0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:58:23.023 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:23.020+0000 7f14cbb78640 1 -- 172.21.15.156:0/891342695 >> 172.21.15.156:0/891342695 conn(0x7f14c40f6a20 msgr2=0x7f14c40f6e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:58:23.023 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:23.020+0000 7f14cbb78640 1 -- 172.21.15.156:0/891342695 shutdown_connections 2024-07-16T09:58:23.023 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:23.020+0000 7f14cbb78640 1 -- 172.21.15.156:0/891342695 wait complete. 2024-07-16T09:58:23.103 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:23.100+0000 7f6c57727640 1 -- 172.21.15.156:0/3510200515 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6c500fc510 msgr2=0x7f6c500fc980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:58:23.104 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:23.100+0000 7f6c57727640 1 --2- 172.21.15.156:0/3510200515 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6c500fc510 0x7f6c500fc980 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f6c4c0048d0 tx=0x7f6c4c009da0 comp rx=0 tx=0).stop 2024-07-16T09:58:23.104 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:23.100+0000 7f6c57727640 1 -- 172.21.15.156:0/3510200515 shutdown_connections 2024-07-16T09:58:23.104 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:23.100+0000 7f6c57727640 1 --2- 172.21.15.156:0/3510200515 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6c500fc510 0x7f6c500fc980 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:58:23.104 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:23.100+0000 7f6c57727640 1 --2- 172.21.15.156:0/3510200515 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6c500fb240 0x7f6c500fb650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:58:23.104 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:23.100+0000 7f6c57727640 1 -- 172.21.15.156:0/3510200515 >> 172.21.15.156:0/3510200515 conn(0x7f6c500f6a20 msgr2=0x7f6c500f8e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:58:23.104 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:23.100+0000 7f6c57727640 1 -- 172.21.15.156:0/3510200515 shutdown_connections 2024-07-16T09:58:23.104 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:23.100+0000 7f6c57727640 1 -- 172.21.15.156:0/3510200515 wait complete. 2024-07-16T09:58:23.104 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:23.100+0000 7f6c57727640 1 Processor -- start 2024-07-16T09:58:23.104 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:23.100+0000 7f6c57727640 1 -- start start 2024-07-16T09:58:23.104 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:23.100+0000 7f6c57727640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6c500fb240 0x7f6c5010fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:58:23.104 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:23.100+0000 7f6c57727640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6c500fc510 0x7f6c50112210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:58:23.104 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:23.100+0000 7f6c57727640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c50065a90 con 0x7f6c500fb240 2024-07-16T09:58:23.104 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:23.100+0000 7f6c57727640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c50065c00 con 0x7f6c500fc510 2024-07-16T09:58:23.105 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:23.101+0000 7f6c554c3640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6c500fb240 0x7f6c5010fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:58:23.105 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:23.101+0000 7f6c554c3640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6c500fb240 0x7f6c5010fcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:39584/0 (socket says 172.21.15.156:39584) 2024-07-16T09:58:23.105 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:23.101+0000 7f6c554c3640 1 -- 172.21.15.156:0/1753440403 learned_addr learned my addr 172.21.15.156:0/1753440403 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T09:58:23.105 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:23.101+0000 7f6c54cc2640 1 --2- 172.21.15.156:0/1753440403 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6c500fc510 0x7f6c50112210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:58:23.105 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:23.101+0000 7f6c554c3640 1 -- 172.21.15.156:0/1753440403 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6c500fc510 msgr2=0x7f6c50112210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:58:23.105 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:23.101+0000 7f6c554c3640 1 --2- 172.21.15.156:0/1753440403 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6c500fc510 0x7f6c50112210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:58:23.105 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:23.101+0000 7f6c554c3640 1 -- 172.21.15.156:0/1753440403 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c4c005040 con 0x7f6c500fb240 2024-07-16T09:58:23.105 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:23.101+0000 7f6c554c3640 1 --2- 172.21.15.156:0/1753440403 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6c500fb240 0x7f6c5010fcc0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f6c40002a50 tx=0x7f6c40004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:58:23.105 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:23.101+0000 7f6c45ffb640 1 -- 172.21.15.156:0/1753440403 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6c40007c30 con 0x7f6c500fb240 2024-07-16T09:58:23.105 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:23.101+0000 7f6c45ffb640 1 -- 172.21.15.156:0/1753440403 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6c4000f040 con 0x7f6c500fb240 2024-07-16T09:58:23.105 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:23.101+0000 7f6c45ffb640 1 -- 172.21.15.156:0/1753440403 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6c4000db50 con 0x7f6c500fb240 2024-07-16T09:58:23.105 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:23.101+0000 7f6c57727640 1 -- 172.21.15.156:0/1753440403 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c501127b0 con 0x7f6c500fb240 2024-07-16T09:58:23.105 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:23.101+0000 7f6c57727640 1 -- 172.21.15.156:0/1753440403 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c50112d50 con 0x7f6c500fb240 2024-07-16T09:58:23.106 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:23.102+0000 7f6c57727640 1 -- 172.21.15.156:0/1753440403 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c500fa6a0 con 0x7f6c500fb240 2024-07-16T09:58:23.106 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:23.103+0000 7f6c45ffb640 1 -- 172.21.15.156:0/1753440403 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 37) v1 ==== 92307+0+0 (secure 0 0 0) 0x7f6c4000dcf0 con 0x7f6c500fb240 2024-07-16T09:58:23.106 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:23.103+0000 7f6c45ffb640 1 --2- 172.21.15.156:0/1753440403 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f6c3c06e1e0 0x7f6c3c070690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:58:23.107 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:23.103+0000 7f6c45ffb640 1 -- 172.21.15.156:0/1753440403 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f6c4008d610 con 0x7f6c500fb240 2024-07-16T09:58:23.107 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:23.103+0000 7f6c54cc2640 1 --2- 172.21.15.156:0/1753440403 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f6c3c06e1e0 0x7f6c3c070690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:58:23.107 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:23.104+0000 7f6c54cc2640 1 --2- 172.21.15.156:0/1753440403 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f6c3c06e1e0 0x7f6c3c070690 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f6c4c000c00 tx=0x7f6c4c035040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:58:23.109 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:23.107+0000 7f6c45ffb640 1 -- 172.21.15.156:0/1753440403 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187716 (secure 0 0 0) 0x7f6c400567f0 con 0x7f6c500fb240 2024-07-16T09:58:23.363 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:23.360+0000 7f6c57727640 1 -- 172.21.15.156:0/1753440403 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f6c5004b530 con 0x7f6c500fb240 2024-07-16T09:58:23.363 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:23.360+0000 7f6c45ffb640 1 -- 172.21.15.156:0/1753440403 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7f6c40059eb0 con 0x7f6c500fb240 2024-07-16T09:58:23.363 INFO:teuthology.orchestra.run.smithi156.stdout:HEALTH_OK 2024-07-16T09:58:23.365 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:23.363+0000 7f6c57727640 1 -- 172.21.15.156:0/1753440403 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f6c3c06e1e0 msgr2=0x7f6c3c070690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:58:23.365 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:23.363+0000 7f6c57727640 1 --2- 172.21.15.156:0/1753440403 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f6c3c06e1e0 0x7f6c3c070690 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f6c4c000c00 tx=0x7f6c4c035040 comp rx=0 tx=0).stop 2024-07-16T09:58:23.365 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:23.363+0000 7f6c57727640 1 -- 172.21.15.156:0/1753440403 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6c500fb240 msgr2=0x7f6c5010fcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:58:23.365 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:23.363+0000 7f6c57727640 1 --2- 172.21.15.156:0/1753440403 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6c500fb240 0x7f6c5010fcc0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f6c40002a50 tx=0x7f6c40004600 comp rx=0 tx=0).stop 2024-07-16T09:58:23.365 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:23.363+0000 7f6c57727640 1 -- 172.21.15.156:0/1753440403 shutdown_connections 2024-07-16T09:58:23.365 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:23.363+0000 7f6c57727640 1 --2- 172.21.15.156:0/1753440403 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f6c3c06e1e0 0x7f6c3c070690 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:58:23.366 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:23.363+0000 7f6c57727640 1 --2- 172.21.15.156:0/1753440403 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6c500fc510 0x7f6c50112210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:58:23.366 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:23.363+0000 7f6c57727640 1 --2- 172.21.15.156:0/1753440403 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6c500fb240 0x7f6c5010fcc0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:58:23.366 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:23.363+0000 7f6c57727640 1 -- 172.21.15.156:0/1753440403 >> 172.21.15.156:0/1753440403 conn(0x7f6c500f6a20 msgr2=0x7f6c500f8140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:58:23.366 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:23.363+0000 7f6c57727640 1 -- 172.21.15.156:0/1753440403 shutdown_connections 2024-07-16T09:58:23.366 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:23.363+0000 7f6c57727640 1 -- 172.21.15.156:0/1753440403 wait complete. 2024-07-16T09:58:23.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:23 smithi176 ceph-mon[81689]: from='client.34160 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-16T09:58:23.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:23 smithi176 ceph-mon[81689]: from='client.34164 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-16T09:58:23.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:23 smithi176 ceph-mon[81689]: Reconfiguring mgr.smithi176.dqbrvx (monmap changed)... 2024-07-16T09:58:23.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:23 smithi176 ceph-mon[81689]: Reconfiguring daemon mgr.smithi176.dqbrvx on smithi176 2024-07-16T09:58:23.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:23 smithi176 ceph-mon[81689]: from='client.34168 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-07-16T09:58:23.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:23 smithi176 ceph-mon[81689]: pgmap v20: 65 pgs: 65 active+clean; 253 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-07-16T09:58:23.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:23 smithi176 ceph-mon[81689]: from='client.? 172.21.15.156:0/2963281108' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T09:58:23.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:23 smithi176 ceph-mon[81689]: from='client.? 172.21.15.156:0/1201167564' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-07-16T09:58:23.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:23 smithi156 ceph-mon[93066]: from='client.34160 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-16T09:58:23.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:23 smithi156 ceph-mon[93066]: from='client.34164 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-16T09:58:23.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:23 smithi156 ceph-mon[93066]: Reconfiguring mgr.smithi176.dqbrvx (monmap changed)... 2024-07-16T09:58:23.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:23 smithi156 ceph-mon[93066]: Reconfiguring daemon mgr.smithi176.dqbrvx on smithi176 2024-07-16T09:58:23.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:23 smithi156 ceph-mon[93066]: from='client.34168 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-07-16T09:58:23.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:23 smithi156 ceph-mon[93066]: pgmap v20: 65 pgs: 65 active+clean; 253 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-07-16T09:58:23.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:23 smithi156 ceph-mon[93066]: from='client.? 172.21.15.156:0/2963281108' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T09:58:23.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:23 smithi156 ceph-mon[93066]: from='client.? 172.21.15.156:0/1201167564' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-07-16T09:58:24.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:24 smithi176 ceph-mon[81689]: from='client.34180 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-16T09:58:24.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:24 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T09:58:24.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:24 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T09:58:24.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:24 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-07-16T09:58:24.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:24 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-07-16T09:58:24.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:24 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-16T09:58:24.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:24 smithi176 ceph-mon[81689]: from='client.? 172.21.15.156:0/1753440403' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-07-16T09:58:24.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:24 smithi156 ceph-mon[93066]: from='client.34180 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-16T09:58:24.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:24 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T09:58:24.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:24 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T09:58:24.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:24 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-07-16T09:58:24.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:24 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-07-16T09:58:24.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:24 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-16T09:58:24.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:24 smithi156 ceph-mon[93066]: from='client.? 172.21.15.156:0/1753440403' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-07-16T09:58:25.405 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:25 smithi176 ceph-mon[81689]: Reconfiguring mon.smithi176 (monmap changed)... 2024-07-16T09:58:25.405 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:25 smithi176 ceph-mon[81689]: Reconfiguring daemon mon.smithi176 on smithi176 2024-07-16T09:58:25.405 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:25 smithi176 ceph-mon[81689]: pgmap v21: 65 pgs: 65 active+clean; 253 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-07-16T09:58:25.405 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:25 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T09:58:25.405 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:25 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T09:58:25.405 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:25 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-07-16T09:58:25.405 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:25 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-16T09:58:25.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:25 smithi156 ceph-mon[93066]: Reconfiguring mon.smithi176 (monmap changed)... 2024-07-16T09:58:25.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:25 smithi156 ceph-mon[93066]: Reconfiguring daemon mon.smithi176 on smithi176 2024-07-16T09:58:25.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:25 smithi156 ceph-mon[93066]: pgmap v21: 65 pgs: 65 active+clean; 253 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-07-16T09:58:25.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:25 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T09:58:25.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:25 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T09:58:25.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:25 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-07-16T09:58:25.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:25 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-16T09:58:26.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:26 smithi176 ceph-mon[81689]: Reconfiguring osd.3 (monmap changed)... 2024-07-16T09:58:26.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:26 smithi176 ceph-mon[81689]: Reconfiguring daemon osd.3 on smithi176 2024-07-16T09:58:26.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:26 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T09:58:26.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:26 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T09:58:26.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:26 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-07-16T09:58:26.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:26 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-16T09:58:26.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:26 smithi156 ceph-mon[93066]: Reconfiguring osd.3 (monmap changed)... 2024-07-16T09:58:26.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:26 smithi156 ceph-mon[93066]: Reconfiguring daemon osd.3 on smithi176 2024-07-16T09:58:26.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:26 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T09:58:26.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:26 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T09:58:26.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:26 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-07-16T09:58:26.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:26 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-16T09:58:27.424 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:27 smithi156 ceph-mon[93066]: Reconfiguring osd.4 (monmap changed)... 2024-07-16T09:58:27.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:27 smithi156 ceph-mon[93066]: Reconfiguring daemon osd.4 on smithi176 2024-07-16T09:58:27.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:27 smithi156 ceph-mon[93066]: pgmap v22: 65 pgs: 65 active+clean; 253 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-07-16T09:58:27.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:27 smithi176 ceph-mon[81689]: Reconfiguring osd.4 (monmap changed)... 2024-07-16T09:58:27.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:27 smithi176 ceph-mon[81689]: Reconfiguring daemon osd.4 on smithi176 2024-07-16T09:58:27.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:27 smithi176 ceph-mon[81689]: pgmap v22: 65 pgs: 65 active+clean; 253 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-07-16T09:58:28.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:28 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T09:58:28.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:28 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T09:58:28.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:28 smithi156 ceph-mon[93066]: Reconfiguring osd.5 (monmap changed)... 2024-07-16T09:58:28.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:28 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-07-16T09:58:28.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:28 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-16T09:58:28.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:28 smithi156 ceph-mon[93066]: Reconfiguring daemon osd.5 on smithi176 2024-07-16T09:58:28.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:28 smithi156 ceph-mon[93066]: pgmap v23: 65 pgs: 65 active+clean; 253 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-07-16T09:58:29.005 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:28 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T09:58:29.005 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:28 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T09:58:29.005 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:28 smithi176 ceph-mon[81689]: Reconfiguring osd.5 (monmap changed)... 2024-07-16T09:58:29.006 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:28 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-07-16T09:58:29.006 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:28 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-16T09:58:29.006 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:28 smithi176 ceph-mon[81689]: Reconfiguring daemon osd.5 on smithi176 2024-07-16T09:58:29.006 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:28 smithi176 ceph-mon[81689]: pgmap v23: 65 pgs: 65 active+clean; 253 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-07-16T09:58:30.370 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:30 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T09:58:30.371 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:30 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T09:58:30.371 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:30 smithi176 ceph-mon[81689]: Reconfiguring mds.cephfs.smithi176.dvvbud (monmap changed)... 2024-07-16T09:58:30.371 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:30 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi176.dvvbud", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-16T09:58:30.371 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:30 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-16T09:58:30.371 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:30 smithi176 ceph-mon[81689]: Reconfiguring daemon mds.cephfs.smithi176.dvvbud on smithi176 2024-07-16T09:58:30.424 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:30 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T09:58:30.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:30 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T09:58:30.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:30 smithi156 ceph-mon[93066]: Reconfiguring mds.cephfs.smithi176.dvvbud (monmap changed)... 2024-07-16T09:58:30.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:30 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi176.dvvbud", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-16T09:58:30.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:30 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-16T09:58:30.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:30 smithi156 ceph-mon[93066]: Reconfiguring daemon mds.cephfs.smithi176.dvvbud on smithi176 2024-07-16T09:58:31.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:31 smithi156 ceph-mon[93066]: pgmap v24: 65 pgs: 65 active+clean; 253 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-07-16T09:58:31.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:31 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T09:58:31.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:31 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T09:58:31.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:31 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi176.wahbsi", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-16T09:58:31.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:31 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-16T09:58:31.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:31 smithi176 ceph-mon[81689]: pgmap v24: 65 pgs: 65 active+clean; 253 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-07-16T09:58:31.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:31 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T09:58:31.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:31 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T09:58:31.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:31 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi176.wahbsi", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-16T09:58:31.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:31 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-16T09:58:32.368 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:32 smithi156 ceph-mon[93066]: Reconfiguring mds.cephfs.smithi176.wahbsi (monmap changed)... 2024-07-16T09:58:32.368 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:32 smithi156 ceph-mon[93066]: Reconfiguring daemon mds.cephfs.smithi176.wahbsi on smithi176 2024-07-16T09:58:32.368 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:32 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T09:58:32.368 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:32 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T09:58:32.368 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:32 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-16T09:58:32.369 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:32 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T09:58:32.369 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:32 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T09:58:32.369 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:32 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T09:58:32.369 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:32 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi156"}]: dispatch 2024-07-16T09:58:32.369 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:32 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi156"}]': finished 2024-07-16T09:58:32.369 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:32 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi176"}]: dispatch 2024-07-16T09:58:32.369 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:32 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi176"}]': finished 2024-07-16T09:58:32.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:32 smithi176 ceph-mon[81689]: Reconfiguring mds.cephfs.smithi176.wahbsi (monmap changed)... 2024-07-16T09:58:32.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:32 smithi176 ceph-mon[81689]: Reconfiguring daemon mds.cephfs.smithi176.wahbsi on smithi176 2024-07-16T09:58:32.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:32 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T09:58:32.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:32 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T09:58:32.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:32 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-16T09:58:32.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:32 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T09:58:32.595 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:32 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T09:58:32.595 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:32 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T09:58:32.595 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:32 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi156"}]: dispatch 2024-07-16T09:58:32.595 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:32 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi156"}]': finished 2024-07-16T09:58:32.595 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:32 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi176"}]: dispatch 2024-07-16T09:58:32.595 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:32 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi176"}]': finished 2024-07-16T09:58:33.366 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:33 smithi176 ceph-mon[81689]: Upgrade: Setting container_image for all mon 2024-07-16T09:58:33.366 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:33 smithi176 ceph-mon[81689]: pgmap v25: 65 pgs: 65 active+clean; 253 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-07-16T09:58:33.424 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:33 smithi156 ceph-mon[93066]: Upgrade: Setting container_image for all mon 2024-07-16T09:58:33.424 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:33 smithi156 ceph-mon[93066]: pgmap v25: 65 pgs: 65 active+clean; 253 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-07-16T09:58:34.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:34 smithi176 ceph-mon[81689]: Upgrade: Updating crash.smithi156 (1/2) 2024-07-16T09:58:34.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:34 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T09:58:34.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:34 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi156", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-07-16T09:58:34.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:34 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-16T09:58:34.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:34 smithi176 ceph-mon[81689]: Deploying daemon crash.smithi156 on smithi156 2024-07-16T09:58:34.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:34 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-16T09:58:34.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:34 smithi156 ceph-mon[93066]: Upgrade: Updating crash.smithi156 (1/2) 2024-07-16T09:58:34.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:34 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T09:58:34.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:34 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi156", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-07-16T09:58:34.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:34 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-16T09:58:34.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:34 smithi156 ceph-mon[93066]: Deploying daemon crash.smithi156 on smithi156 2024-07-16T09:58:34.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:34 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-16T09:58:35.539 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:35 smithi156 ceph-mon[93066]: pgmap v26: 65 pgs: 65 active+clean; 253 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-07-16T09:58:35.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:35 smithi176 ceph-mon[81689]: pgmap v26: 65 pgs: 65 active+clean; 253 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-07-16T09:58:37.500 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:37 smithi176 ceph-mon[81689]: pgmap v27: 65 pgs: 65 active+clean; 253 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-07-16T09:58:37.500 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:37 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T09:58:37.500 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:37 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T09:58:37.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:37 smithi156 ceph-mon[93066]: pgmap v27: 65 pgs: 65 active+clean; 253 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-07-16T09:58:37.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:37 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T09:58:37.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:37 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T09:58:39.424 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:39 smithi156 ceph-mon[93066]: Upgrade: Updating crash.smithi176 (2/2) 2024-07-16T09:58:39.424 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:39 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T09:58:39.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:39 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi176", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-07-16T09:58:39.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:39 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-16T09:58:39.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:39 smithi156 ceph-mon[93066]: Deploying daemon crash.smithi176 on smithi176 2024-07-16T09:58:39.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:39 smithi156 ceph-mon[93066]: pgmap v28: 65 pgs: 65 active+clean; 253 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-07-16T09:58:39.541 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:39 smithi176 ceph-mon[81689]: Upgrade: Updating crash.smithi176 (2/2) 2024-07-16T09:58:39.541 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:39 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T09:58:39.541 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:39 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi176", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-07-16T09:58:39.542 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:39 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-16T09:58:39.542 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:39 smithi176 ceph-mon[81689]: Deploying daemon crash.smithi176 on smithi176 2024-07-16T09:58:39.542 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:39 smithi176 ceph-mon[81689]: pgmap v28: 65 pgs: 65 active+clean; 253 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-07-16T09:58:40.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:40 smithi156 ceph-mon[93066]: pgmap v29: 65 pgs: 65 active+clean; 253 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-07-16T09:58:40.721 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:40 smithi176 ceph-mon[81689]: pgmap v29: 65 pgs: 65 active+clean; 253 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-07-16T09:58:42.664 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:42 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T09:58:42.664 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:42 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T09:58:42.664 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:42 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-16T09:58:42.664 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:42 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T09:58:42.664 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:42 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T09:58:42.664 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:42 smithi176 ceph-mon[81689]: pgmap v30: 65 pgs: 65 active+clean; 253 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-07-16T09:58:42.924 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:42 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T09:58:42.924 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:42 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T09:58:42.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:42 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-16T09:58:42.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:42 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T09:58:42.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:42 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T09:58:42.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:42 smithi156 ceph-mon[93066]: pgmap v30: 65 pgs: 65 active+clean; 253 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-07-16T09:58:44.455 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:44 smithi156 ceph-mon[93066]: pgmap v31: 65 pgs: 65 active+clean; 253 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-07-16T09:58:44.843 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:44 smithi176 ceph-mon[81689]: pgmap v31: 65 pgs: 65 active+clean; 253 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-07-16T09:58:45.955 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:45 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T09:58:45.955 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:45 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T09:58:45.955 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:45 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T09:58:45.955 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:45 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T09:58:45.998 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:45 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T09:58:45.999 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:45 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T09:58:45.999 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:45 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T09:58:45.999 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:45 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T09:58:47.027 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:46 smithi176 ceph-mon[81689]: pgmap v32: 65 pgs: 65 active+clean; 253 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-07-16T09:58:47.027 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:46 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T09:58:47.027 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:46 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T09:58:47.174 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:46 smithi156 ceph-mon[93066]: pgmap v32: 65 pgs: 65 active+clean; 253 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-07-16T09:58:47.174 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:46 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T09:58:47.175 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:46 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T09:58:48.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:48 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T09:58:48.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:48 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T09:58:48.660 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:48 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T09:58:48.660 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:48 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T09:58:49.573 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:49 smithi156 ceph-mon[93066]: pgmap v33: 65 pgs: 65 active+clean; 253 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-07-16T09:58:49.573 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:49 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-16T09:58:49.573 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:49 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T09:58:49.573 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:49 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T09:58:49.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:49 smithi176 ceph-mon[81689]: pgmap v33: 65 pgs: 65 active+clean; 253 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-07-16T09:58:49.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:49 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-16T09:58:49.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:49 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T09:58:49.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:49 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T09:58:50.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:50 smithi156 ceph-mon[93066]: pgmap v34: 65 pgs: 65 active+clean; 253 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-07-16T09:58:50.843 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:50 smithi176 ceph-mon[81689]: pgmap v34: 65 pgs: 65 active+clean; 253 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-07-16T09:58:51.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:51 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T09:58:51.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:51 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T09:58:51.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:51 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-16T09:58:51.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:51 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-16T09:58:51.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:51 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T09:58:51.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:51 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-16T09:58:51.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:51 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T09:58:51.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:51 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T09:58:51.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:51 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T09:58:51.676 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:51 smithi156 ceph-mon[93066]: Upgrade: Setting container_image for all crash 2024-07-16T09:58:51.676 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:51 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T09:58:51.676 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:51 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi156"}]: dispatch 2024-07-16T09:58:51.676 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:51 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi156"}]': finished 2024-07-16T09:58:51.676 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:51 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi176"}]: dispatch 2024-07-16T09:58:51.676 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:51 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi176"}]': finished 2024-07-16T09:58:51.676 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:51 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-07-16T09:58:51.676 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:51 smithi156 ceph-mon[93066]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-07-16T09:58:51.676 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:51 smithi156 ceph-mon[93066]: Upgrade: osd.0 is safe to restart 2024-07-16T09:58:51.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:51 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T09:58:51.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:51 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T09:58:51.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:51 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-16T09:58:51.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:51 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-16T09:58:51.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:51 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T09:58:51.845 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:51 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-16T09:58:51.845 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:51 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T09:58:51.845 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:51 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T09:58:51.845 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:51 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T09:58:51.845 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:51 smithi176 ceph-mon[81689]: Upgrade: Setting container_image for all crash 2024-07-16T09:58:51.845 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:51 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T09:58:51.845 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:51 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi156"}]: dispatch 2024-07-16T09:58:51.845 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:51 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi156"}]': finished 2024-07-16T09:58:51.845 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:51 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi176"}]: dispatch 2024-07-16T09:58:51.845 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:51 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi176"}]': finished 2024-07-16T09:58:51.845 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:51 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-07-16T09:58:51.845 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:51 smithi176 ceph-mon[81689]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-07-16T09:58:51.845 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:51 smithi176 ceph-mon[81689]: Upgrade: osd.0 is safe to restart 2024-07-16T09:58:53.174 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:52 smithi156 ceph-mon[93066]: Upgrade: Updating osd.0 2024-07-16T09:58:53.174 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:52 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T09:58:53.175 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:52 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-07-16T09:58:53.175 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:52 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-16T09:58:53.175 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:52 smithi156 ceph-mon[93066]: Deploying daemon osd.0 on smithi156 2024-07-16T09:58:53.175 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:52 smithi156 ceph-mon[93066]: pgmap v35: 65 pgs: 65 active+clean; 253 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-07-16T09:58:53.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:52 smithi176 ceph-mon[81689]: Upgrade: Updating osd.0 2024-07-16T09:58:53.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:52 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T09:58:53.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:52 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-07-16T09:58:53.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:52 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-16T09:58:53.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:52 smithi176 ceph-mon[81689]: Deploying daemon osd.0 on smithi156 2024-07-16T09:58:53.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:52 smithi176 ceph-mon[81689]: pgmap v35: 65 pgs: 65 active+clean; 253 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-07-16T09:58:53.456 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:53.453+0000 7fac3b45e640 1 -- 172.21.15.156:0/3088442237 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fac340fc360 msgr2=0x7fac340fc770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:58:53.456 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:53.453+0000 7fac3b45e640 1 --2- 172.21.15.156:0/3088442237 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fac340fc360 0x7fac340fc770 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fac300048d0 tx=0x7fac30009da0 comp rx=0 tx=0).stop 2024-07-16T09:58:53.456 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:53.453+0000 7fac3b45e640 1 -- 172.21.15.156:0/3088442237 shutdown_connections 2024-07-16T09:58:53.456 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:53.453+0000 7fac3b45e640 1 --2- 172.21.15.156:0/3088442237 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fac340fd630 0x7fac340fdaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:58:53.456 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:53.453+0000 7fac3b45e640 1 --2- 172.21.15.156:0/3088442237 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fac340fc360 0x7fac340fc770 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:58:53.456 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:53.453+0000 7fac3b45e640 1 -- 172.21.15.156:0/3088442237 >> 172.21.15.156:0/3088442237 conn(0x7fac340f7b40 msgr2=0x7fac340f9f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:58:53.456 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:53.453+0000 7fac3b45e640 1 -- 172.21.15.156:0/3088442237 shutdown_connections 2024-07-16T09:58:53.456 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:53.453+0000 7fac3b45e640 1 -- 172.21.15.156:0/3088442237 wait complete. 2024-07-16T09:58:53.456 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:53.454+0000 7fac3b45e640 1 Processor -- start 2024-07-16T09:58:53.456 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:53.454+0000 7fac3b45e640 1 -- start start 2024-07-16T09:58:53.456 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:53.454+0000 7fac3b45e640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fac340fc360 0x7fac3410cf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:58:53.457 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:53.454+0000 7fac3b45e640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fac340fd630 0x7fac3410d490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:58:53.457 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:53.454+0000 7fac3b45e640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac3410dae0 con 0x7fac340fd630 2024-07-16T09:58:53.457 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:53.454+0000 7fac3b45e640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac3410e300 con 0x7fac340fc360 2024-07-16T09:58:53.457 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:53.454+0000 7fac389f9640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fac340fd630 0x7fac3410d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:58:53.457 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:53.454+0000 7fac389f9640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fac340fd630 0x7fac3410d490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:46636/0 (socket says 172.21.15.156:46636) 2024-07-16T09:58:53.457 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:53.454+0000 7fac389f9640 1 -- 172.21.15.156:0/1125907637 learned_addr learned my addr 172.21.15.156:0/1125907637 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T09:58:53.457 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:53.454+0000 7fac391fa640 1 --2- 172.21.15.156:0/1125907637 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fac340fc360 0x7fac3410cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:58:53.457 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:53.455+0000 7fac389f9640 1 -- 172.21.15.156:0/1125907637 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fac340fc360 msgr2=0x7fac3410cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:58:53.457 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:53.455+0000 7fac389f9640 1 --2- 172.21.15.156:0/1125907637 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fac340fc360 0x7fac3410cf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:58:53.457 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:53.455+0000 7fac389f9640 1 -- 172.21.15.156:0/1125907637 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac30005040 con 0x7fac340fd630 2024-07-16T09:58:53.457 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:53.455+0000 7fac389f9640 1 --2- 172.21.15.156:0/1125907637 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fac340fd630 0x7fac3410d490 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fac24002a50 tx=0x7fac24004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:58:53.459 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:53.455+0000 7fac29ffb640 1 -- 172.21.15.156:0/1125907637 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fac24007c30 con 0x7fac340fd630 2024-07-16T09:58:53.459 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:53.455+0000 7fac3b45e640 1 -- 172.21.15.156:0/1125907637 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac3410e590 con 0x7fac340fd630 2024-07-16T09:58:53.459 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:53.455+0000 7fac29ffb640 1 -- 172.21.15.156:0/1125907637 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fac2400f040 con 0x7fac340fd630 2024-07-16T09:58:53.459 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:53.455+0000 7fac29ffb640 1 -- 172.21.15.156:0/1125907637 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fac2400dad0 con 0x7fac340fd630 2024-07-16T09:58:53.459 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:53.455+0000 7fac3b45e640 1 -- 172.21.15.156:0/1125907637 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac3410eb30 con 0x7fac340fd630 2024-07-16T09:58:53.459 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:53.456+0000 7fac3b45e640 1 -- 172.21.15.156:0/1125907637 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac340fb7c0 con 0x7fac340fd630 2024-07-16T09:58:53.459 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:53.457+0000 7fac29ffb640 1 -- 172.21.15.156:0/1125907637 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 37) v1 ==== 92307+0+0 (secure 0 0 0) 0x7fac2401c400 con 0x7fac340fd630 2024-07-16T09:58:53.459 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:53.457+0000 7fac29ffb640 1 --2- 172.21.15.156:0/1125907637 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7fac2006e1e0 0x7fac20070690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:58:53.459 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:53.457+0000 7fac29ffb640 1 -- 172.21.15.156:0/1125907637 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fac2408d5a0 con 0x7fac340fd630 2024-07-16T09:58:53.459 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:53.457+0000 7fac391fa640 1 --2- 172.21.15.156:0/1125907637 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7fac2006e1e0 0x7fac20070690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:58:53.460 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:53.458+0000 7fac391fa640 1 --2- 172.21.15.156:0/1125907637 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7fac2006e1e0 0x7fac20070690 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fac300043d0 tx=0x7fac30035040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:58:53.463 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:53.462+0000 7fac29ffb640 1 -- 172.21.15.156:0/1125907637 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187716 (secure 0 0 0) 0x7fac240567b0 con 0x7fac340fd630 2024-07-16T09:58:53.666 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:53.665+0000 7fac3b45e640 1 -- 172.21.15.156:0/1125907637 --> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fac3410ee60 con 0x7fac2006e1e0 2024-07-16T09:58:53.668 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:53.666+0000 7fac29ffb640 1 -- 172.21.15.156:0/1125907637 <== mgr.34104 v2:172.21.15.156:6800/3367765292 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7fac3410ee60 con 0x7fac2006e1e0 2024-07-16T09:58:53.670 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:53.668+0000 7fac3b45e640 1 -- 172.21.15.156:0/1125907637 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7fac2006e1e0 msgr2=0x7fac20070690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:58:53.670 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:53.668+0000 7fac3b45e640 1 --2- 172.21.15.156:0/1125907637 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7fac2006e1e0 0x7fac20070690 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fac300043d0 tx=0x7fac30035040 comp rx=0 tx=0).stop 2024-07-16T09:58:53.670 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:53.669+0000 7fac3b45e640 1 -- 172.21.15.156:0/1125907637 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fac340fd630 msgr2=0x7fac3410d490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:58:53.670 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:53.669+0000 7fac3b45e640 1 --2- 172.21.15.156:0/1125907637 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fac340fd630 0x7fac3410d490 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fac24002a50 tx=0x7fac24004600 comp rx=0 tx=0).stop 2024-07-16T09:58:53.670 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:53.669+0000 7fac3b45e640 1 -- 172.21.15.156:0/1125907637 shutdown_connections 2024-07-16T09:58:53.670 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:53.669+0000 7fac3b45e640 1 --2- 172.21.15.156:0/1125907637 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7fac2006e1e0 0x7fac20070690 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:58:53.670 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:53.669+0000 7fac3b45e640 1 --2- 172.21.15.156:0/1125907637 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fac340fd630 0x7fac3410d490 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:58:53.671 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:53.669+0000 7fac3b45e640 1 --2- 172.21.15.156:0/1125907637 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fac340fc360 0x7fac3410cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:58:53.671 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:53.669+0000 7fac3b45e640 1 -- 172.21.15.156:0/1125907637 >> 172.21.15.156:0/1125907637 conn(0x7fac340f7b40 msgr2=0x7fac340f91b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:58:53.671 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:53.669+0000 7fac3b45e640 1 -- 172.21.15.156:0/1125907637 shutdown_connections 2024-07-16T09:58:53.671 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:53.669+0000 7fac3b45e640 1 -- 172.21.15.156:0/1125907637 wait complete. 2024-07-16T09:58:53.679 INFO:teuthology.orchestra.run.smithi156.stdout:true 2024-07-16T09:58:53.754 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:53.751+0000 7ff36e48b640 1 -- 172.21.15.156:0/1427088915 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff3680fc550 msgr2=0x7ff3680fc9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:58:53.754 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:53.751+0000 7ff36e48b640 1 --2- 172.21.15.156:0/1427088915 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff3680fc550 0x7ff3680fc9c0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7ff3500048d0 tx=0x7ff350009da0 comp rx=0 tx=0).stop 2024-07-16T09:58:53.754 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:53.751+0000 7ff36e48b640 1 -- 172.21.15.156:0/1427088915 shutdown_connections 2024-07-16T09:58:53.755 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:53.751+0000 7ff36e48b640 1 --2- 172.21.15.156:0/1427088915 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff3680fc550 0x7ff3680fc9c0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:58:53.755 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:53.751+0000 7ff36e48b640 1 --2- 172.21.15.156:0/1427088915 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff3680fb280 0x7ff3680fb690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:58:53.755 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:53.751+0000 7ff36e48b640 1 -- 172.21.15.156:0/1427088915 >> 172.21.15.156:0/1427088915 conn(0x7ff3680f6a40 msgr2=0x7ff3680f8e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:58:53.755 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:53.751+0000 7ff36e48b640 1 -- 172.21.15.156:0/1427088915 shutdown_connections 2024-07-16T09:58:53.755 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:53.751+0000 7ff36e48b640 1 -- 172.21.15.156:0/1427088915 wait complete. 2024-07-16T09:58:53.755 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:53.751+0000 7ff36e48b640 1 Processor -- start 2024-07-16T09:58:53.755 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:53.751+0000 7ff36e48b640 1 -- start start 2024-07-16T09:58:53.756 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:53.751+0000 7ff36e48b640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff3680fb280 0x7ff36810d020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:58:53.756 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:53.751+0000 7ff36e48b640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff3680fc550 0x7ff36810d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:58:53.757 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:53.751+0000 7ff36e48b640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff36810dbb0 con 0x7ff3680fb280 2024-07-16T09:58:53.757 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:53.751+0000 7ff36e48b640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff36810e3d0 con 0x7ff3680fc550 2024-07-16T09:58:53.757 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:53.751+0000 7ff367fff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff3680fb280 0x7ff36810d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:58:53.757 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:53.751+0000 7ff367fff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff3680fb280 0x7ff36810d020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:46652/0 (socket says 172.21.15.156:46652) 2024-07-16T09:58:53.757 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:53.751+0000 7ff367fff640 1 -- 172.21.15.156:0/1281330916 learned_addr learned my addr 172.21.15.156:0/1281330916 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T09:58:53.757 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:53.751+0000 7ff3677fe640 1 --2- 172.21.15.156:0/1281330916 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff3680fc550 0x7ff36810d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:58:53.757 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:53.751+0000 7ff367fff640 1 -- 172.21.15.156:0/1281330916 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff3680fc550 msgr2=0x7ff36810d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:58:53.757 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:53.751+0000 7ff367fff640 1 --2- 172.21.15.156:0/1281330916 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff3680fc550 0x7ff36810d560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:58:53.758 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:53.751+0000 7ff367fff640 1 -- 172.21.15.156:0/1281330916 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff350005040 con 0x7ff3680fb280 2024-07-16T09:58:53.758 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:53.751+0000 7ff3677fe640 1 --2- 172.21.15.156:0/1281330916 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff3680fc550 0x7ff36810d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-16T09:58:53.758 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:53.751+0000 7ff367fff640 1 --2- 172.21.15.156:0/1281330916 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff3680fb280 0x7ff36810d020 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7ff358002a50 tx=0x7ff358004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:58:53.758 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:53.751+0000 7ff364ff9640 1 -- 172.21.15.156:0/1281330916 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff358007c30 con 0x7ff3680fb280 2024-07-16T09:58:53.758 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:53.751+0000 7ff36e48b640 1 -- 172.21.15.156:0/1281330916 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff36810e660 con 0x7ff3680fb280 2024-07-16T09:58:53.758 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:53.751+0000 7ff364ff9640 1 -- 172.21.15.156:0/1281330916 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff35800f040 con 0x7ff3680fb280 2024-07-16T09:58:53.758 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:53.751+0000 7ff364ff9640 1 -- 172.21.15.156:0/1281330916 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff35800db50 con 0x7ff3680fb280 2024-07-16T09:58:53.758 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:53.751+0000 7ff36e48b640 1 -- 172.21.15.156:0/1281330916 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff36810ec00 con 0x7ff3680fb280 2024-07-16T09:58:53.758 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:53.751+0000 7ff36e48b640 1 -- 172.21.15.156:0/1281330916 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3680fa6e0 con 0x7ff3680fb280 2024-07-16T09:58:53.759 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:53.756+0000 7ff364ff9640 1 -- 172.21.15.156:0/1281330916 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 37) v1 ==== 92307+0+0 (secure 0 0 0) 0x7ff35801c400 con 0x7ff3680fb280 2024-07-16T09:58:53.759 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:53.756+0000 7ff364ff9640 1 --2- 172.21.15.156:0/1281330916 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7ff35406e1e0 0x7ff354070690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:58:53.759 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:53.756+0000 7ff364ff9640 1 -- 172.21.15.156:0/1281330916 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6410+0+0 (secure 0 0 0) 0x7ff35808d5e0 con 0x7ff3680fb280 2024-07-16T09:58:53.759 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:53.756+0000 7ff3677fe640 1 --2- 172.21.15.156:0/1281330916 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7ff35406e1e0 0x7ff354070690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:58:53.759 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:53.756+0000 7ff3677fe640 1 --2- 172.21.15.156:0/1281330916 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7ff35406e1e0 0x7ff354070690 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7ff350000c00 tx=0x7ff350035040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:58:53.762 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:53.761+0000 7ff364ff9640 1 -- 172.21.15.156:0/1281330916 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187716 (secure 0 0 0) 0x7ff3580567c0 con 0x7ff3680fb280 2024-07-16T09:58:53.962 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:53.961+0000 7ff36e48b640 1 -- 172.21.15.156:0/1281330916 --> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff368103fa0 con 0x7ff35406e1e0 2024-07-16T09:58:53.963 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:53.962+0000 7ff364ff9640 1 -- 172.21.15.156:0/1281330916 <== mgr.34104 v2:172.21.15.156:6800/3367765292 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7ff350008da0 con 0x7ff35406e1e0 2024-07-16T09:58:53.966 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:53.965+0000 7ff36e48b640 1 -- 172.21.15.156:0/1281330916 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7ff35406e1e0 msgr2=0x7ff354070690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:58:53.966 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:53.965+0000 7ff36e48b640 1 --2- 172.21.15.156:0/1281330916 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7ff35406e1e0 0x7ff354070690 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7ff350000c00 tx=0x7ff350035040 comp rx=0 tx=0).stop 2024-07-16T09:58:53.966 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:53.965+0000 7ff36e48b640 1 -- 172.21.15.156:0/1281330916 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff3680fb280 msgr2=0x7ff36810d020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:58:53.966 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:53.965+0000 7ff36e48b640 1 --2- 172.21.15.156:0/1281330916 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff3680fb280 0x7ff36810d020 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7ff358002a50 tx=0x7ff358004600 comp rx=0 tx=0).stop 2024-07-16T09:58:53.966 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:53.965+0000 7ff36e48b640 1 -- 172.21.15.156:0/1281330916 shutdown_connections 2024-07-16T09:58:53.966 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:53.965+0000 7ff36e48b640 1 --2- 172.21.15.156:0/1281330916 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7ff35406e1e0 0x7ff354070690 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:58:53.967 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:53.965+0000 7ff36e48b640 1 --2- 172.21.15.156:0/1281330916 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff3680fc550 0x7ff36810d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:58:53.967 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:53.965+0000 7ff36e48b640 1 --2- 172.21.15.156:0/1281330916 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff3680fb280 0x7ff36810d020 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:58:53.967 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:53.965+0000 7ff36e48b640 1 -- 172.21.15.156:0/1281330916 >> 172.21.15.156:0/1281330916 conn(0x7ff3680f6a40 msgr2=0x7ff368065f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:58:53.967 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:53.965+0000 7ff36e48b640 1 -- 172.21.15.156:0/1281330916 shutdown_connections 2024-07-16T09:58:53.967 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:53.965+0000 7ff36e48b640 1 -- 172.21.15.156:0/1281330916 wait complete. 2024-07-16T09:58:54.051 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:54.048+0000 7f2897194640 1 -- 172.21.15.156:0/3355241159 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f28900fb270 msgr2=0x7f28900fb680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:58:54.051 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:54.048+0000 7f2897194640 1 --2- 172.21.15.156:0/3355241159 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f28900fb270 0x7f28900fb680 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f28840048d0 tx=0x7f2884009da0 comp rx=0 tx=0).stop 2024-07-16T09:58:54.051 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:54.048+0000 7f2897194640 1 -- 172.21.15.156:0/3355241159 shutdown_connections 2024-07-16T09:58:54.051 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:54.048+0000 7f2897194640 1 --2- 172.21.15.156:0/3355241159 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f28900fc540 0x7f28900fc9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:58:54.052 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:54.048+0000 7f2897194640 1 --2- 172.21.15.156:0/3355241159 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f28900fb270 0x7f28900fb680 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:58:54.052 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:54.048+0000 7f2897194640 1 -- 172.21.15.156:0/3355241159 >> 172.21.15.156:0/3355241159 conn(0x7f28900f6a50 msgr2=0x7f28900f8e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:58:54.052 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:54.048+0000 7f2897194640 1 -- 172.21.15.156:0/3355241159 shutdown_connections 2024-07-16T09:58:54.052 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:54.048+0000 7f2897194640 1 -- 172.21.15.156:0/3355241159 wait complete. 2024-07-16T09:58:54.052 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:54.049+0000 7f2897194640 1 Processor -- start 2024-07-16T09:58:54.052 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:54.049+0000 7f2897194640 1 -- start start 2024-07-16T09:58:54.052 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:54.049+0000 7f2897194640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f28900fb270 0x7f289010cf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:58:54.052 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:54.049+0000 7f2897194640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f28900fc540 0x7f289010d490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:58:54.052 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:54.049+0000 7f2897194640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f289010dae0 con 0x7f28900fb270 2024-07-16T09:58:54.052 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:54.049+0000 7f2897194640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f289010e300 con 0x7f28900fc540 2024-07-16T09:58:54.053 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:54.050+0000 7f2894f30640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f28900fb270 0x7f289010cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:58:54.053 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:54.050+0000 7f2894f30640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f28900fb270 0x7f289010cf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:46660/0 (socket says 172.21.15.156:46660) 2024-07-16T09:58:54.053 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:54.050+0000 7f2894f30640 1 -- 172.21.15.156:0/976245586 learned_addr learned my addr 172.21.15.156:0/976245586 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T09:58:54.053 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:54.050+0000 7f288ffff640 1 --2- 172.21.15.156:0/976245586 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f28900fc540 0x7f289010d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:58:54.053 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:54.050+0000 7f2894f30640 1 -- 172.21.15.156:0/976245586 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f28900fc540 msgr2=0x7f289010d490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:58:54.053 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:54.050+0000 7f2894f30640 1 --2- 172.21.15.156:0/976245586 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f28900fc540 0x7f289010d490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:58:54.053 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:54.050+0000 7f2894f30640 1 -- 172.21.15.156:0/976245586 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2884005040 con 0x7f28900fb270 2024-07-16T09:58:54.053 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:54.050+0000 7f288ffff640 1 --2- 172.21.15.156:0/976245586 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f28900fc540 0x7f289010d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-16T09:58:54.053 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:54.050+0000 7f2894f30640 1 --2- 172.21.15.156:0/976245586 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f28900fb270 0x7f289010cf50 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f2884009f80 tx=0x7f2884009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:58:54.055 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:54.050+0000 7f288d7fa640 1 -- 172.21.15.156:0/976245586 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2884038070 con 0x7f28900fb270 2024-07-16T09:58:54.055 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:54.050+0000 7f2897194640 1 -- 172.21.15.156:0/976245586 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f289010e530 con 0x7f28900fb270 2024-07-16T09:58:54.055 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:54.050+0000 7f2897194640 1 -- 172.21.15.156:0/976245586 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f289010ea70 con 0x7f28900fb270 2024-07-16T09:58:54.055 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:54.051+0000 7f288d7fa640 1 -- 172.21.15.156:0/976245586 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2884032970 con 0x7f28900fb270 2024-07-16T09:58:54.055 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:54.051+0000 7f288d7fa640 1 -- 172.21.15.156:0/976245586 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2884033860 con 0x7f28900fb270 2024-07-16T09:58:54.055 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:54.052+0000 7f2897194640 1 -- 172.21.15.156:0/976245586 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2874005240 con 0x7f28900fb270 2024-07-16T09:58:54.055 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:54.052+0000 7f288d7fa640 1 -- 172.21.15.156:0/976245586 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 37) v1 ==== 92307+0+0 (secure 0 0 0) 0x7f2884030020 con 0x7f28900fb270 2024-07-16T09:58:54.055 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:54.053+0000 7f288d7fa640 1 --2- 172.21.15.156:0/976245586 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f288006e1e0 0x7f2880070690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:58:54.055 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:54.053+0000 7f288d7fa640 1 -- 172.21.15.156:0/976245586 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f28840adb90 con 0x7f28900fb270 2024-07-16T09:58:54.056 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:54.053+0000 7f288ffff640 1 --2- 172.21.15.156:0/976245586 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f288006e1e0 0x7f2880070690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:58:54.056 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:54.053+0000 7f288ffff640 1 --2- 172.21.15.156:0/976245586 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f288006e1e0 0x7f2880070690 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f287c009fd0 tx=0x7f287c009e50 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:58:54.062 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:54.061+0000 7f288d7fa640 1 -- 172.21.15.156:0/976245586 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187716 (secure 0 0 0) 0x7f2884076e00 con 0x7f28900fb270 2024-07-16T09:58:54.252 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:54.251+0000 7f2897194640 1 -- 172.21.15.156:0/976245586 --> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f2874000bf0 con 0x7f288006e1e0 2024-07-16T09:58:54.262 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:54.261+0000 7f288d7fa640 1 -- 172.21.15.156:0/976245586 <== mgr.34104 v2:172.21.15.156:6800/3367765292 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f2874000bf0 con 0x7f288006e1e0 2024-07-16T09:58:54.262 INFO:teuthology.orchestra.run.smithi156.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-07-16T09:58:54.263 INFO:teuthology.orchestra.run.smithi156.stdout:alertmanager.smithi156 smithi156 *:9093,9094 running (2m) 7s ago 10m 16.2M - 0.25.0 c8568f914cd2 d524a2c3ff15 2024-07-16T09:58:54.263 INFO:teuthology.orchestra.run.smithi156.stdout:crash.smithi156 smithi156 running (17s) 7s ago 11m 6899k - 19.0.0-3495-ga6ca461b 16f7feb04342 11648cf29af4 2024-07-16T09:58:54.263 INFO:teuthology.orchestra.run.smithi156.stdout:crash.smithi176 smithi176 running (13s) 9s ago 10m 6899k - 19.0.0-3495-ga6ca461b 16f7feb04342 67f3ccc034f8 2024-07-16T09:58:54.263 INFO:teuthology.orchestra.run.smithi156.stdout:grafana.smithi156 smithi156 *:3000 running (2m) 7s ago 10m 78.1M - 9.4.7 954c08fa6188 deb9e1bfeb66 2024-07-16T09:58:54.263 INFO:teuthology.orchestra.run.smithi156.stdout:mds.cephfs.smithi156.evwwaz smithi156 running (5m) 7s ago 5m 67.3M - 17.2.7-1104-ge6231b04 d5de5f953b95 7c0ff9ae43f8 2024-07-16T09:58:54.263 INFO:teuthology.orchestra.run.smithi156.stdout:mds.cephfs.smithi156.rdibys smithi156 running (5m) 7s ago 5m 120M - 17.2.7-1104-ge6231b04 d5de5f953b95 b985cd1e6924 2024-07-16T09:58:54.263 INFO:teuthology.orchestra.run.smithi156.stdout:mds.cephfs.smithi176.dvvbud smithi176 running (5m) 9s ago 5m 108M - 17.2.7-1104-ge6231b04 d5de5f953b95 1b00147950f9 2024-07-16T09:58:54.263 INFO:teuthology.orchestra.run.smithi156.stdout:mds.cephfs.smithi176.wahbsi smithi176 running (5m) 9s ago 5m 62.2M - 17.2.7-1104-ge6231b04 d5de5f953b95 866664af2311 2024-07-16T09:58:54.263 INFO:teuthology.orchestra.run.smithi156.stdout:mgr.smithi156.zkvylv smithi156 *:8443,9283,8765 running (2m) 7s ago 12m 491M - 19.0.0-3495-ga6ca461b 16f7feb04342 ee53bd604b7c 2024-07-16T09:58:54.263 INFO:teuthology.orchestra.run.smithi156.stdout:mgr.smithi176.dqbrvx smithi176 *:8443,9283,8765 running (85s) 9s ago 10m 438M - 19.0.0-3495-ga6ca461b 16f7feb04342 ce9eaae4c2d0 2024-07-16T09:58:54.263 INFO:teuthology.orchestra.run.smithi156.stdout:mon.smithi156 smithi156 running (72s) 7s ago 12m 40.6M 2048M 19.0.0-3495-ga6ca461b 16f7feb04342 77b6ec8c50eb 2024-07-16T09:58:54.263 INFO:teuthology.orchestra.run.smithi156.stdout:mon.smithi176 smithi176 running (53s) 9s ago 10m 33.4M 2048M 19.0.0-3495-ga6ca461b 16f7feb04342 ef92814cec2b 2024-07-16T09:58:54.264 INFO:teuthology.orchestra.run.smithi156.stdout:node-exporter.smithi156 smithi156 *:9100 running (2m) 7s ago 11m 14.4M - 1.5.0 0da6a335fe13 e5d86f23a14c 2024-07-16T09:58:54.264 INFO:teuthology.orchestra.run.smithi156.stdout:node-exporter.smithi176 smithi176 *:9100 running (2m) 9s ago 10m 16.1M - 1.5.0 0da6a335fe13 ea842728144a 2024-07-16T09:58:54.264 INFO:teuthology.orchestra.run.smithi156.stdout:osd.0 smithi156 running (9m) 7s ago 9m 223M 1557M 17.2.7-1104-ge6231b04 d5de5f953b95 0471758ec349 2024-07-16T09:58:54.264 INFO:teuthology.orchestra.run.smithi156.stdout:osd.1 smithi156 running (8m) 7s ago 8m 246M 1557M 17.2.7-1104-ge6231b04 d5de5f953b95 149a0bda2f61 2024-07-16T09:58:54.264 INFO:teuthology.orchestra.run.smithi156.stdout:osd.2 smithi156 running (8m) 7s ago 8m 198M 1557M 17.2.7-1104-ge6231b04 d5de5f953b95 f28acec5cb79 2024-07-16T09:58:54.264 INFO:teuthology.orchestra.run.smithi156.stdout:osd.3 smithi176 running (8m) 9s ago 8m 260M 2581M 17.2.7-1104-ge6231b04 d5de5f953b95 733861ba84dc 2024-07-16T09:58:54.264 INFO:teuthology.orchestra.run.smithi156.stdout:osd.4 smithi176 running (7m) 9s ago 7m 229M 2581M 17.2.7-1104-ge6231b04 d5de5f953b95 ef0c67cec086 2024-07-16T09:58:54.264 INFO:teuthology.orchestra.run.smithi156.stdout:osd.5 smithi176 running (7m) 9s ago 7m 224M 2581M 17.2.7-1104-ge6231b04 d5de5f953b95 996e15448a52 2024-07-16T09:58:54.264 INFO:teuthology.orchestra.run.smithi156.stdout:prometheus.smithi156 smithi156 *:9095 running (90s) 7s ago 10m 53.6M - 2.43.0 a07b618ecd1d 20c195ecd665 2024-07-16T09:58:54.265 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:54.263+0000 7f2897194640 1 -- 172.21.15.156:0/976245586 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f288006e1e0 msgr2=0x7f2880070690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:58:54.265 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:54.263+0000 7f2897194640 1 --2- 172.21.15.156:0/976245586 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f288006e1e0 0x7f2880070690 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f287c009fd0 tx=0x7f287c009e50 comp rx=0 tx=0).stop 2024-07-16T09:58:54.265 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:54.263+0000 7f2897194640 1 -- 172.21.15.156:0/976245586 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f28900fb270 msgr2=0x7f289010cf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:58:54.265 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:54.263+0000 7f2897194640 1 --2- 172.21.15.156:0/976245586 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f28900fb270 0x7f289010cf50 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f2884009f80 tx=0x7f2884009da0 comp rx=0 tx=0).stop 2024-07-16T09:58:54.265 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:54.263+0000 7f2897194640 1 -- 172.21.15.156:0/976245586 shutdown_connections 2024-07-16T09:58:54.265 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:54.263+0000 7f2897194640 1 --2- 172.21.15.156:0/976245586 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f288006e1e0 0x7f2880070690 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:58:54.265 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:54.263+0000 7f2897194640 1 --2- 172.21.15.156:0/976245586 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f28900fc540 0x7f289010d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:58:54.265 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:54.263+0000 7f2897194640 1 --2- 172.21.15.156:0/976245586 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f28900fb270 0x7f289010cf50 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:58:54.266 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:54.263+0000 7f2897194640 1 -- 172.21.15.156:0/976245586 >> 172.21.15.156:0/976245586 conn(0x7f28900f6a50 msgr2=0x7f2890065d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:58:54.266 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:54.263+0000 7f2897194640 1 -- 172.21.15.156:0/976245586 shutdown_connections 2024-07-16T09:58:54.266 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:54.263+0000 7f2897194640 1 -- 172.21.15.156:0/976245586 wait complete. 2024-07-16T09:58:54.347 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:54.345+0000 7fe262fbd640 1 -- 172.21.15.156:0/1957002632 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe25c0664c0 msgr2=0x7fe25c0ff010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:58:54.347 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:54.345+0000 7fe262fbd640 1 --2- 172.21.15.156:0/1957002632 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe25c0664c0 0x7fe25c0ff010 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fe2540048d0 tx=0x7fe254009da0 comp rx=0 tx=0).stop 2024-07-16T09:58:54.347 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:54.345+0000 7fe262fbd640 1 -- 172.21.15.156:0/1957002632 shutdown_connections 2024-07-16T09:58:54.347 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:54.345+0000 7fe262fbd640 1 --2- 172.21.15.156:0/1957002632 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe25c0ff550 0x7fe25c101930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:58:54.347 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:54.345+0000 7fe262fbd640 1 --2- 172.21.15.156:0/1957002632 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe25c0664c0 0x7fe25c0ff010 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:58:54.348 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:54.345+0000 7fe262fbd640 1 -- 172.21.15.156:0/1957002632 >> 172.21.15.156:0/1957002632 conn(0x7fe25c0f6a40 msgr2=0x7fe25c0f8e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:58:54.348 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:54.345+0000 7fe262fbd640 1 -- 172.21.15.156:0/1957002632 shutdown_connections 2024-07-16T09:58:54.348 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:54.345+0000 7fe262fbd640 1 -- 172.21.15.156:0/1957002632 wait complete. 2024-07-16T09:58:54.348 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:54.345+0000 7fe262fbd640 1 Processor -- start 2024-07-16T09:58:54.348 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:54.345+0000 7fe262fbd640 1 -- start start 2024-07-16T09:58:54.348 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:54.346+0000 7fe262fbd640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe25c0664c0 0x7fe25c10d020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:58:54.348 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:54.346+0000 7fe262fbd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe25c0ff550 0x7fe25c10d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:58:54.348 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:54.346+0000 7fe262fbd640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe25c10dbb0 con 0x7fe25c0ff550 2024-07-16T09:58:54.348 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:54.346+0000 7fe262fbd640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe25c10e3d0 con 0x7fe25c0664c0 2024-07-16T09:58:54.348 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:54.346+0000 7fe25bfff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe25c0ff550 0x7fe25c10d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:58:54.348 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:54.346+0000 7fe25bfff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe25c0ff550 0x7fe25c10d560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:46688/0 (socket says 172.21.15.156:46688) 2024-07-16T09:58:54.348 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:54.346+0000 7fe25bfff640 1 -- 172.21.15.156:0/3560806479 learned_addr learned my addr 172.21.15.156:0/3560806479 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T09:58:54.349 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:54.346+0000 7fe260d59640 1 --2- 172.21.15.156:0/3560806479 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe25c0664c0 0x7fe25c10d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:58:54.349 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:54.346+0000 7fe25bfff640 1 -- 172.21.15.156:0/3560806479 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe25c0664c0 msgr2=0x7fe25c10d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:58:54.349 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:54.346+0000 7fe25bfff640 1 --2- 172.21.15.156:0/3560806479 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe25c0664c0 0x7fe25c10d020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:58:54.349 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:54.346+0000 7fe25bfff640 1 -- 172.21.15.156:0/3560806479 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe254005040 con 0x7fe25c0ff550 2024-07-16T09:58:54.350 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:54.346+0000 7fe25bfff640 1 --2- 172.21.15.156:0/3560806479 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe25c0ff550 0x7fe25c10d560 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fe24c00cb10 tx=0x7fe24c007e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:58:54.350 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:54.346+0000 7fe2597fa640 1 -- 172.21.15.156:0/3560806479 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe24c00aa50 con 0x7fe25c0ff550 2024-07-16T09:58:54.350 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:54.346+0000 7fe262fbd640 1 -- 172.21.15.156:0/3560806479 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe25c10e660 con 0x7fe25c0ff550 2024-07-16T09:58:54.350 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:54.346+0000 7fe2597fa640 1 -- 172.21.15.156:0/3560806479 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe24c00abb0 con 0x7fe25c0ff550 2024-07-16T09:58:54.350 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:54.347+0000 7fe2597fa640 1 -- 172.21.15.156:0/3560806479 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe24c0115e0 con 0x7fe25c0ff550 2024-07-16T09:58:54.351 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:54.347+0000 7fe262fbd640 1 -- 172.21.15.156:0/3560806479 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe25c10ec00 con 0x7fe25c0ff550 2024-07-16T09:58:54.351 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:54.347+0000 7fe262fbd640 1 -- 172.21.15.156:0/3560806479 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe25c0fa6e0 con 0x7fe25c0ff550 2024-07-16T09:58:54.351 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:54.348+0000 7fe2597fa640 1 -- 172.21.15.156:0/3560806479 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 37) v1 ==== 92307+0+0 (secure 0 0 0) 0x7fe24c012a10 con 0x7fe25c0ff550 2024-07-16T09:58:54.351 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:54.348+0000 7fe2597fa640 1 --2- 172.21.15.156:0/3560806479 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7fe24806e1e0 0x7fe248070690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:58:54.351 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:54.349+0000 7fe2597fa640 1 -- 172.21.15.156:0/3560806479 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fe24c08e4c0 con 0x7fe25c0ff550 2024-07-16T09:58:54.351 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:54.349+0000 7fe260d59640 1 --2- 172.21.15.156:0/3560806479 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7fe24806e1e0 0x7fe248070690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:58:54.351 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:54.349+0000 7fe260d59640 1 --2- 172.21.15.156:0/3560806479 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7fe24806e1e0 0x7fe248070690 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fe254000c00 tx=0x7fe254035040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:58:54.354 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:54.353+0000 7fe2597fa640 1 -- 172.21.15.156:0/3560806479 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187716 (secure 0 0 0) 0x7fe24c057730 con 0x7fe25c0ff550 2024-07-16T09:58:54.610 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:54.609+0000 7fe262fbd640 1 -- 172.21.15.156:0/3560806479 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fe25c10ee70 con 0x7fe25c0ff550 2024-07-16T09:58:54.611 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:54 smithi156 ceph-mon[93066]: from='client.34188 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-16T09:58:54.611 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:54 smithi156 ceph-mon[93066]: from='client.34192 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-16T09:58:54.611 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:54 smithi156 ceph-mon[93066]: from='client.34196 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-07-16T09:58:54.611 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:54 smithi156 ceph-mon[93066]: pgmap v36: 65 pgs: 65 active+clean; 253 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-07-16T09:58:54.611 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:54.609+0000 7fe2597fa640 1 -- 172.21.15.156:0/3560806479 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+736 (secure 0 0 0) 0x7fe24c05adf0 con 0x7fe25c0ff550 2024-07-16T09:58:54.611 INFO:teuthology.orchestra.run.smithi156.stdout:{ 2024-07-16T09:58:54.611 INFO:teuthology.orchestra.run.smithi156.stdout: "mon": { 2024-07-16T09:58:54.612 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph version 19.0.0-3495-ga6ca461b (a6ca461be7cc1aff3658c2c6df4cff975a24449a) squid (rc)": 2 2024-07-16T09:58:54.612 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-07-16T09:58:54.612 INFO:teuthology.orchestra.run.smithi156.stdout: "mgr": { 2024-07-16T09:58:54.612 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph version 19.0.0-3495-ga6ca461b (a6ca461be7cc1aff3658c2c6df4cff975a24449a) squid (rc)": 2 2024-07-16T09:58:54.612 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-07-16T09:58:54.612 INFO:teuthology.orchestra.run.smithi156.stdout: "osd": { 2024-07-16T09:58:54.612 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph version 17.2.7-1104-ge6231b04 (e6231b04b52ebf3b9b31803282dd3074439e51ac) quincy (stable)": 6 2024-07-16T09:58:54.612 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-07-16T09:58:54.612 INFO:teuthology.orchestra.run.smithi156.stdout: "mds": { 2024-07-16T09:58:54.612 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph version 17.2.7-1104-ge6231b04 (e6231b04b52ebf3b9b31803282dd3074439e51ac) quincy (stable)": 4 2024-07-16T09:58:54.612 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-07-16T09:58:54.612 INFO:teuthology.orchestra.run.smithi156.stdout: "overall": { 2024-07-16T09:58:54.612 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph version 17.2.7-1104-ge6231b04 (e6231b04b52ebf3b9b31803282dd3074439e51ac) quincy (stable)": 10, 2024-07-16T09:58:54.613 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph version 19.0.0-3495-ga6ca461b (a6ca461be7cc1aff3658c2c6df4cff975a24449a) squid (rc)": 4 2024-07-16T09:58:54.613 INFO:teuthology.orchestra.run.smithi156.stdout: } 2024-07-16T09:58:54.613 INFO:teuthology.orchestra.run.smithi156.stdout:} 2024-07-16T09:58:54.613 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:54.612+0000 7fe262fbd640 1 -- 172.21.15.156:0/3560806479 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7fe24806e1e0 msgr2=0x7fe248070690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:58:54.613 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:54.612+0000 7fe262fbd640 1 --2- 172.21.15.156:0/3560806479 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7fe24806e1e0 0x7fe248070690 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fe254000c00 tx=0x7fe254035040 comp rx=0 tx=0).stop 2024-07-16T09:58:54.614 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:54.612+0000 7fe262fbd640 1 -- 172.21.15.156:0/3560806479 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe25c0ff550 msgr2=0x7fe25c10d560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:58:54.614 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:54.612+0000 7fe262fbd640 1 --2- 172.21.15.156:0/3560806479 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe25c0ff550 0x7fe25c10d560 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fe24c00cb10 tx=0x7fe24c007e20 comp rx=0 tx=0).stop 2024-07-16T09:58:54.614 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:54.612+0000 7fe262fbd640 1 -- 172.21.15.156:0/3560806479 shutdown_connections 2024-07-16T09:58:54.614 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:54.612+0000 7fe262fbd640 1 --2- 172.21.15.156:0/3560806479 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7fe24806e1e0 0x7fe248070690 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:58:54.614 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:54.612+0000 7fe262fbd640 1 --2- 172.21.15.156:0/3560806479 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe25c0ff550 0x7fe25c10d560 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:58:54.614 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:54.612+0000 7fe262fbd640 1 --2- 172.21.15.156:0/3560806479 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe25c0664c0 0x7fe25c10d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:58:54.614 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:54.612+0000 7fe262fbd640 1 -- 172.21.15.156:0/3560806479 >> 172.21.15.156:0/3560806479 conn(0x7fe25c0f6a40 msgr2=0x7fe25c0f6e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:58:54.614 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:54.612+0000 7fe262fbd640 1 -- 172.21.15.156:0/3560806479 shutdown_connections 2024-07-16T09:58:54.614 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:54.612+0000 7fe262fbd640 1 -- 172.21.15.156:0/3560806479 wait complete. 2024-07-16T09:58:54.695 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:54.694+0000 7f73a8fa1640 1 -- 172.21.15.156:0/2020539532 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f73a4065900 msgr2=0x7f73a40feda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:58:54.695 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:54.694+0000 7f73a8fa1640 1 --2- 172.21.15.156:0/2020539532 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f73a4065900 0x7f73a40feda0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f73940048d0 tx=0x7f7394009da0 comp rx=0 tx=0).stop 2024-07-16T09:58:54.695 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:54.694+0000 7f73a8fa1640 1 -- 172.21.15.156:0/2020539532 shutdown_connections 2024-07-16T09:58:54.696 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:54.694+0000 7f73a8fa1640 1 --2- 172.21.15.156:0/2020539532 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f73a40ff2e0 0x7f73a41016c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:58:54.696 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:54.694+0000 7f73a8fa1640 1 --2- 172.21.15.156:0/2020539532 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f73a4065900 0x7f73a40feda0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:58:54.696 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:54.694+0000 7f73a8fa1640 1 -- 172.21.15.156:0/2020539532 >> 172.21.15.156:0/2020539532 conn(0x7f73a40f6850 msgr2=0x7f73a40f8c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:58:54.696 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:54.694+0000 7f73a8fa1640 1 -- 172.21.15.156:0/2020539532 shutdown_connections 2024-07-16T09:58:54.696 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:54.694+0000 7f73a8fa1640 1 -- 172.21.15.156:0/2020539532 wait complete. 2024-07-16T09:58:54.699 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:54.694+0000 7f73a8fa1640 1 Processor -- start 2024-07-16T09:58:54.699 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:54.695+0000 7f73a8fa1640 1 -- start start 2024-07-16T09:58:54.699 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:54.695+0000 7f73a8fa1640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f73a4065900 0x7f73a40fc7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:58:54.699 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:54.695+0000 7f73a8fa1640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f73a40ff2e0 0x7f73a40faca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:58:54.699 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:54.695+0000 7f73a8fa1640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73a40fb230 con 0x7f73a4065900 2024-07-16T09:58:54.699 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:54.695+0000 7f73a8fa1640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73a40fb370 con 0x7f73a40ff2e0 2024-07-16T09:58:54.700 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:54.695+0000 7f73a37fe640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f73a4065900 0x7f73a40fc7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:58:54.700 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:54.695+0000 7f73a37fe640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f73a4065900 0x7f73a40fc7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:46702/0 (socket says 172.21.15.156:46702) 2024-07-16T09:58:54.700 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:54.695+0000 7f73a37fe640 1 -- 172.21.15.156:0/1756310233 learned_addr learned my addr 172.21.15.156:0/1756310233 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T09:58:54.700 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:54.695+0000 7f73a2ffd640 1 --2- 172.21.15.156:0/1756310233 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f73a40ff2e0 0x7f73a40faca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:58:54.700 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:54.695+0000 7f73a37fe640 1 -- 172.21.15.156:0/1756310233 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f73a40ff2e0 msgr2=0x7f73a40faca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:58:54.700 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:54.695+0000 7f73a37fe640 1 --2- 172.21.15.156:0/1756310233 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f73a40ff2e0 0x7f73a40faca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:58:54.700 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:54.695+0000 7f73a37fe640 1 -- 172.21.15.156:0/1756310233 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7394005040 con 0x7f73a4065900 2024-07-16T09:58:54.701 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:54.696+0000 7f73a37fe640 1 --2- 172.21.15.156:0/1756310233 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f73a4065900 0x7f73a40fc7b0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f7394000c00 tx=0x7f7394009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:58:54.701 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:54.696+0000 7f738bfff640 1 -- 172.21.15.156:0/1756310233 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7394038070 con 0x7f73a4065900 2024-07-16T09:58:54.701 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:54.696+0000 7f738bfff640 1 -- 172.21.15.156:0/1756310233 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7394032970 con 0x7f73a4065900 2024-07-16T09:58:54.701 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:54.696+0000 7f738bfff640 1 -- 172.21.15.156:0/1756310233 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7394033950 con 0x7f73a4065900 2024-07-16T09:58:54.701 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:54.696+0000 7f73a8fa1640 1 -- 172.21.15.156:0/1756310233 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73a40fb5f0 con 0x7f73a4065900 2024-07-16T09:58:54.701 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:54.696+0000 7f73a8fa1640 1 -- 172.21.15.156:0/1756310233 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73a4114e30 con 0x7f73a4065900 2024-07-16T09:58:54.701 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:54.697+0000 7f73a8fa1640 1 -- 172.21.15.156:0/1756310233 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73a405eb80 con 0x7f73a4065900 2024-07-16T09:58:54.701 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:54.697+0000 7f738bfff640 1 -- 172.21.15.156:0/1756310233 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 37) v1 ==== 92307+0+0 (secure 0 0 0) 0x7f7394030020 con 0x7f73a4065900 2024-07-16T09:58:54.702 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:54.698+0000 7f738bfff640 1 --2- 172.21.15.156:0/1756310233 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f738c06e1d0 0x7f738c070680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:58:54.702 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:54.698+0000 7f738bfff640 1 -- 172.21.15.156:0/1756310233 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f73940adb90 con 0x7f73a4065900 2024-07-16T09:58:54.702 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:54.698+0000 7f73a2ffd640 1 --2- 172.21.15.156:0/1756310233 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f738c06e1d0 0x7f738c070680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:58:54.702 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:54.699+0000 7f73a2ffd640 1 --2- 172.21.15.156:0/1756310233 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f738c06e1d0 0x7f738c070680 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f7398007fd0 tx=0x7f7398009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:58:54.707 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:54.706+0000 7f738bfff640 1 -- 172.21.15.156:0/1756310233 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187716 (secure 0 0 0) 0x7f7394076f30 con 0x7f73a4065900 2024-07-16T09:58:54.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:54 smithi176 ceph-mon[81689]: from='client.34188 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-16T09:58:54.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:54 smithi176 ceph-mon[81689]: from='client.34192 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-16T09:58:54.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:54 smithi176 ceph-mon[81689]: from='client.34196 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-07-16T09:58:54.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:54 smithi176 ceph-mon[81689]: pgmap v36: 65 pgs: 65 active+clean; 253 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-07-16T09:58:54.935 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:54.933+0000 7f73a8fa1640 1 -- 172.21.15.156:0/1756310233 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f73a4104ce0 con 0x7f73a4065900 2024-07-16T09:58:54.935 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:54.934+0000 7f738bfff640 1 -- 172.21.15.156:0/1756310233 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 19 v19) v1 ==== 76+0+1937 (secure 0 0 0) 0x7f7394041070 con 0x7f73a4065900 2024-07-16T09:58:54.936 INFO:teuthology.orchestra.run.smithi156.stdout:e19 2024-07-16T09:58:54.936 INFO:teuthology.orchestra.run.smithi156.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-07-16T09:58:54.936 INFO:teuthology.orchestra.run.smithi156.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-07-16T09:58:54.936 INFO:teuthology.orchestra.run.smithi156.stdout:legacy client fscid: 1 2024-07-16T09:58:54.936 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-07-16T09:58:54.936 INFO:teuthology.orchestra.run.smithi156.stdout:Filesystem 'cephfs' (1) 2024-07-16T09:58:54.936 INFO:teuthology.orchestra.run.smithi156.stdout:fs_name cephfs 2024-07-16T09:58:54.937 INFO:teuthology.orchestra.run.smithi156.stdout:epoch 19 2024-07-16T09:58:54.937 INFO:teuthology.orchestra.run.smithi156.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-07-16T09:58:54.937 INFO:teuthology.orchestra.run.smithi156.stdout:created 2024-07-16T09:52:55.214975+0000 2024-07-16T09:58:54.937 INFO:teuthology.orchestra.run.smithi156.stdout:modified 2024-07-16T09:58:14.542481+0000 2024-07-16T09:58:54.937 INFO:teuthology.orchestra.run.smithi156.stdout:tableserver 0 2024-07-16T09:58:54.937 INFO:teuthology.orchestra.run.smithi156.stdout:root 0 2024-07-16T09:58:54.937 INFO:teuthology.orchestra.run.smithi156.stdout:session_timeout 60 2024-07-16T09:58:54.937 INFO:teuthology.orchestra.run.smithi156.stdout:session_autoclose 300 2024-07-16T09:58:54.937 INFO:teuthology.orchestra.run.smithi156.stdout:max_file_size 1099511627776 2024-07-16T09:58:54.937 INFO:teuthology.orchestra.run.smithi156.stdout:max_xattr_size 65536 2024-07-16T09:58:54.937 INFO:teuthology.orchestra.run.smithi156.stdout:required_client_features {} 2024-07-16T09:58:54.937 INFO:teuthology.orchestra.run.smithi156.stdout:last_failure 0 2024-07-16T09:58:54.937 INFO:teuthology.orchestra.run.smithi156.stdout:last_failure_osd_epoch 0 2024-07-16T09:58:54.938 INFO:teuthology.orchestra.run.smithi156.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-07-16T09:58:54.938 INFO:teuthology.orchestra.run.smithi156.stdout:max_mds 2 2024-07-16T09:58:54.938 INFO:teuthology.orchestra.run.smithi156.stdout:in 0,1 2024-07-16T09:58:54.938 INFO:teuthology.orchestra.run.smithi156.stdout:up {0=14458,1=24261} 2024-07-16T09:58:54.938 INFO:teuthology.orchestra.run.smithi156.stdout:failed 2024-07-16T09:58:54.938 INFO:teuthology.orchestra.run.smithi156.stdout:damaged 2024-07-16T09:58:54.938 INFO:teuthology.orchestra.run.smithi156.stdout:stopped 2024-07-16T09:58:54.938 INFO:teuthology.orchestra.run.smithi156.stdout:data_pools [3] 2024-07-16T09:58:54.938 INFO:teuthology.orchestra.run.smithi156.stdout:metadata_pool 2 2024-07-16T09:58:54.938 INFO:teuthology.orchestra.run.smithi156.stdout:inline_data disabled 2024-07-16T09:58:54.938 INFO:teuthology.orchestra.run.smithi156.stdout:balancer 2024-07-16T09:58:54.938 INFO:teuthology.orchestra.run.smithi156.stdout:bal_rank_mask -1 2024-07-16T09:58:54.939 INFO:teuthology.orchestra.run.smithi156.stdout:standby_count_wanted 1 2024-07-16T09:58:54.939 INFO:teuthology.orchestra.run.smithi156.stdout:qdb_cluster leader: 14458 members: 14458,24261 2024-07-16T09:58:54.939 INFO:teuthology.orchestra.run.smithi156.stdout:[mds.cephfs.smithi156.rdibys{0:14458} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.156:6826/1080176013,v1:172.21.15.156:6827/1080176013] compat {c=[1],r=[1],i=[7ff]}] 2024-07-16T09:58:54.939 INFO:teuthology.orchestra.run.smithi156.stdout:[mds.cephfs.smithi156.evwwaz{0:14468} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.156:6828/560329502,v1:172.21.15.156:6829/560329502] compat {c=[1],r=[1],i=[7ff]}] 2024-07-16T09:58:54.939 INFO:teuthology.orchestra.run.smithi156.stdout:[mds.cephfs.smithi176.dvvbud{1:24261} state up:active seq 4 export targets 0 join_fscid=1 addr [v2:172.21.15.176:6824/1577019287,v1:172.21.15.176:6825/1577019287] compat {c=[1],r=[1],i=[7ff]}] 2024-07-16T09:58:54.939 INFO:teuthology.orchestra.run.smithi156.stdout:[mds.cephfs.smithi176.wahbsi{1:24273} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.176:6826/2196775412,v1:172.21.15.176:6827/2196775412] compat {c=[1],r=[1],i=[7ff]}] 2024-07-16T09:58:54.939 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-07-16T09:58:54.939 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-07-16T09:58:54.940 INFO:teuthology.orchestra.run.smithi156.stderr:dumped fsmap epoch 19 2024-07-16T09:58:54.940 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:54.936+0000 7f73a8fa1640 1 -- 172.21.15.156:0/1756310233 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f738c06e1d0 msgr2=0x7f738c070680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:58:54.940 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:54.936+0000 7f73a8fa1640 1 --2- 172.21.15.156:0/1756310233 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f738c06e1d0 0x7f738c070680 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f7398007fd0 tx=0x7f7398009040 comp rx=0 tx=0).stop 2024-07-16T09:58:54.940 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:54.936+0000 7f73a8fa1640 1 -- 172.21.15.156:0/1756310233 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f73a4065900 msgr2=0x7f73a40fc7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:58:54.940 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:54.936+0000 7f73a8fa1640 1 --2- 172.21.15.156:0/1756310233 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f73a4065900 0x7f73a40fc7b0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f7394000c00 tx=0x7f7394009da0 comp rx=0 tx=0).stop 2024-07-16T09:58:54.940 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:54.936+0000 7f73a8fa1640 1 -- 172.21.15.156:0/1756310233 shutdown_connections 2024-07-16T09:58:54.940 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:54.936+0000 7f73a8fa1640 1 --2- 172.21.15.156:0/1756310233 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f738c06e1d0 0x7f738c070680 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:58:54.940 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:54.936+0000 7f73a8fa1640 1 --2- 172.21.15.156:0/1756310233 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f73a40ff2e0 0x7f73a40faca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:58:54.940 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:54.936+0000 7f73a8fa1640 1 --2- 172.21.15.156:0/1756310233 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f73a4065900 0x7f73a40fc7b0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:58:54.940 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:54.936+0000 7f73a8fa1640 1 -- 172.21.15.156:0/1756310233 >> 172.21.15.156:0/1756310233 conn(0x7f73a40f6850 msgr2=0x7f73a40f6c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:58:54.940 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:54.936+0000 7f73a8fa1640 1 -- 172.21.15.156:0/1756310233 shutdown_connections 2024-07-16T09:58:54.941 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:54.936+0000 7f73a8fa1640 1 -- 172.21.15.156:0/1756310233 wait complete. 2024-07-16T09:58:55.024 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:55.020+0000 7f31d8b53640 1 -- 172.21.15.156:0/1644112370 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f31d4065a50 msgr2=0x7f31d40fef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:58:55.024 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:55.020+0000 7f31d8b53640 1 --2- 172.21.15.156:0/1644112370 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f31d4065a50 0x7f31d40fef50 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f31c40048d0 tx=0x7f31c4009da0 comp rx=0 tx=0).stop 2024-07-16T09:58:55.024 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:55.020+0000 7f31d8b53640 1 -- 172.21.15.156:0/1644112370 shutdown_connections 2024-07-16T09:58:55.024 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:55.020+0000 7f31d8b53640 1 --2- 172.21.15.156:0/1644112370 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f31d40ff490 0x7f31d4101870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:58:55.024 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:55.020+0000 7f31d8b53640 1 --2- 172.21.15.156:0/1644112370 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f31d4065a50 0x7f31d40fef50 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:58:55.024 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:55.020+0000 7f31d8b53640 1 -- 172.21.15.156:0/1644112370 >> 172.21.15.156:0/1644112370 conn(0x7f31d40f6a20 msgr2=0x7f31d40f8e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:58:55.024 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:55.020+0000 7f31d8b53640 1 -- 172.21.15.156:0/1644112370 shutdown_connections 2024-07-16T09:58:55.024 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:55.021+0000 7f31d8b53640 1 -- 172.21.15.156:0/1644112370 wait complete. 2024-07-16T09:58:55.024 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:55.021+0000 7f31d8b53640 1 Processor -- start 2024-07-16T09:58:55.025 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:55.021+0000 7f31d8b53640 1 -- start start 2024-07-16T09:58:55.025 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:55.021+0000 7f31d8b53640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f31d4065a50 0x7f31d410cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:58:55.025 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:55.021+0000 7f31d8b53640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f31d40ff490 0x7f31d410d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:58:55.025 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:55.021+0000 7f31d8b53640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31d410db20 con 0x7f31d40ff490 2024-07-16T09:58:55.025 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:55.021+0000 7f31d8b53640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31d410e340 con 0x7f31d4065a50 2024-07-16T09:58:55.025 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:55.021+0000 7f31cbfff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f31d40ff490 0x7f31d410d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:58:55.025 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:55.021+0000 7f31cbfff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f31d40ff490 0x7f31d410d4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:46716/0 (socket says 172.21.15.156:46716) 2024-07-16T09:58:55.025 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:55.021+0000 7f31cbfff640 1 -- 172.21.15.156:0/1652386513 learned_addr learned my addr 172.21.15.156:0/1652386513 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T09:58:55.025 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:55.022+0000 7f31d259c640 1 --2- 172.21.15.156:0/1652386513 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f31d4065a50 0x7f31d410cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:58:55.025 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:55.022+0000 7f31cbfff640 1 -- 172.21.15.156:0/1652386513 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f31d4065a50 msgr2=0x7f31d410cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:58:55.025 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:55.022+0000 7f31cbfff640 1 --2- 172.21.15.156:0/1652386513 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f31d4065a50 0x7f31d410cf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:58:55.025 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:55.022+0000 7f31cbfff640 1 -- 172.21.15.156:0/1652386513 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31c4005040 con 0x7f31d40ff490 2024-07-16T09:58:55.026 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:55.022+0000 7f31d259c640 1 --2- 172.21.15.156:0/1652386513 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f31d4065a50 0x7f31d410cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-16T09:58:55.026 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:55.022+0000 7f31cbfff640 1 --2- 172.21.15.156:0/1652386513 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f31d40ff490 0x7f31d410d4d0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f31bc002a50 tx=0x7f31bc004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:58:55.026 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:55.022+0000 7f31caffd640 1 -- 172.21.15.156:0/1652386513 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f31bc007c30 con 0x7f31d40ff490 2024-07-16T09:58:55.026 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:55.022+0000 7f31caffd640 1 -- 172.21.15.156:0/1652386513 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f31bc00f040 con 0x7f31d40ff490 2024-07-16T09:58:55.026 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:55.022+0000 7f31d8b53640 1 -- 172.21.15.156:0/1652386513 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31d410e5d0 con 0x7f31d40ff490 2024-07-16T09:58:55.026 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:55.022+0000 7f31caffd640 1 -- 172.21.15.156:0/1652386513 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f31bc00dbe0 con 0x7f31d40ff490 2024-07-16T09:58:55.026 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:55.022+0000 7f31d8b53640 1 -- 172.21.15.156:0/1652386513 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31d410eb70 con 0x7f31d40ff490 2024-07-16T09:58:55.027 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:55.024+0000 7f31d8b53640 1 -- 172.21.15.156:0/1652386513 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31d40fa6a0 con 0x7f31d40ff490 2024-07-16T09:58:55.027 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:55.024+0000 7f31caffd640 1 -- 172.21.15.156:0/1652386513 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 37) v1 ==== 92307+0+0 (secure 0 0 0) 0x7f31bc00dd40 con 0x7f31d40ff490 2024-07-16T09:58:55.027 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:55.024+0000 7f31caffd640 1 --2- 172.21.15.156:0/1652386513 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f31c006e1e0 0x7f31c0070690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:58:55.027 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:55.024+0000 7f31caffd640 1 -- 172.21.15.156:0/1652386513 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f31bc08ddb0 con 0x7f31d40ff490 2024-07-16T09:58:55.027 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:55.025+0000 7f31d259c640 1 --2- 172.21.15.156:0/1652386513 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f31c006e1e0 0x7f31c0070690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:58:55.027 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:55.025+0000 7f31d259c640 1 --2- 172.21.15.156:0/1652386513 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f31c006e1e0 0x7f31c0070690 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f31c40047f0 tx=0x7f31c4009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:58:55.030 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:55.029+0000 7f31caffd640 1 -- 172.21.15.156:0/1652386513 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187716 (secure 0 0 0) 0x7f31bc056f10 con 0x7f31d40ff490 2024-07-16T09:58:55.254 INFO:journalctl@ceph.osd.0.smithi156.stdout:Jul 16 09:58:55 smithi156 systemd[1]: Stopping Ceph osd.0 for 26c91c26-4358-11ef-bcad-c7b262605968... 2024-07-16T09:58:55.255 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:55.252+0000 7f31d8b53640 1 -- 172.21.15.156:0/1652386513 --> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f31d410ef00 con 0x7f31c006e1e0 2024-07-16T09:58:55.255 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:55.253+0000 7f31caffd640 1 -- 172.21.15.156:0/1652386513 <== mgr.34104 v2:172.21.15.156:6800/3367765292 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f31d410ef00 con 0x7f31c006e1e0 2024-07-16T09:58:55.255 INFO:teuthology.orchestra.run.smithi156.stdout:{ 2024-07-16T09:58:55.255 INFO:teuthology.orchestra.run.smithi156.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:c15ec3aa8e6c0a9c12ee24ff79f3e7dd4d01b3d1a702dfeb7d7fff9b0811a393", 2024-07-16T09:58:55.256 INFO:teuthology.orchestra.run.smithi156.stdout: "in_progress": true, 2024-07-16T09:58:55.256 INFO:teuthology.orchestra.run.smithi156.stdout: "which": "Upgrading all daemon types on all hosts", 2024-07-16T09:58:55.256 INFO:teuthology.orchestra.run.smithi156.stdout: "services_complete": [ 2024-07-16T09:58:55.256 INFO:teuthology.orchestra.run.smithi156.stdout: "mon", 2024-07-16T09:58:55.256 INFO:teuthology.orchestra.run.smithi156.stdout: "mgr", 2024-07-16T09:58:55.256 INFO:teuthology.orchestra.run.smithi156.stdout: "crash" 2024-07-16T09:58:55.256 INFO:teuthology.orchestra.run.smithi156.stdout: ], 2024-07-16T09:58:55.256 INFO:teuthology.orchestra.run.smithi156.stdout: "progress": "6/21 daemons upgraded", 2024-07-16T09:58:55.256 INFO:teuthology.orchestra.run.smithi156.stdout: "message": "Currently upgrading osd daemons", 2024-07-16T09:58:55.256 INFO:teuthology.orchestra.run.smithi156.stdout: "is_paused": false 2024-07-16T09:58:55.256 INFO:teuthology.orchestra.run.smithi156.stdout:} 2024-07-16T09:58:55.257 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:55.255+0000 7f31d8b53640 1 -- 172.21.15.156:0/1652386513 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f31c006e1e0 msgr2=0x7f31c0070690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:58:55.257 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:55.255+0000 7f31d8b53640 1 --2- 172.21.15.156:0/1652386513 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f31c006e1e0 0x7f31c0070690 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f31c40047f0 tx=0x7f31c4009da0 comp rx=0 tx=0).stop 2024-07-16T09:58:55.257 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:55.256+0000 7f31d8b53640 1 -- 172.21.15.156:0/1652386513 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f31d40ff490 msgr2=0x7f31d410d4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:58:55.258 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:55.256+0000 7f31d8b53640 1 --2- 172.21.15.156:0/1652386513 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f31d40ff490 0x7f31d410d4d0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f31bc002a50 tx=0x7f31bc004600 comp rx=0 tx=0).stop 2024-07-16T09:58:55.258 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:55.256+0000 7f31d8b53640 1 -- 172.21.15.156:0/1652386513 shutdown_connections 2024-07-16T09:58:55.258 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:55.256+0000 7f31d8b53640 1 --2- 172.21.15.156:0/1652386513 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f31c006e1e0 0x7f31c0070690 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:58:55.258 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:55.256+0000 7f31d8b53640 1 --2- 172.21.15.156:0/1652386513 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f31d40ff490 0x7f31d410d4d0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:58:55.258 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:55.256+0000 7f31d8b53640 1 --2- 172.21.15.156:0/1652386513 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f31d4065a50 0x7f31d410cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:58:55.258 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:55.256+0000 7f31d8b53640 1 -- 172.21.15.156:0/1652386513 >> 172.21.15.156:0/1652386513 conn(0x7f31d40f6a20 msgr2=0x7f31d40f6df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:58:55.258 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:55.256+0000 7f31d8b53640 1 -- 172.21.15.156:0/1652386513 shutdown_connections 2024-07-16T09:58:55.259 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:55.256+0000 7f31d8b53640 1 -- 172.21.15.156:0/1652386513 wait complete. 2024-07-16T09:58:55.338 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:55.336+0000 7f81f2008640 1 -- 172.21.15.156:0/1134241458 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f81ec0664e0 msgr2=0x7f81ec0fefd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:58:55.338 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:55.336+0000 7f81f2008640 1 --2- 172.21.15.156:0/1134241458 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f81ec0664e0 0x7f81ec0fefd0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f81dc0048d0 tx=0x7f81dc009da0 comp rx=0 tx=0).stop 2024-07-16T09:58:55.338 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:55.336+0000 7f81f2008640 1 -- 172.21.15.156:0/1134241458 shutdown_connections 2024-07-16T09:58:55.339 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:55.336+0000 7f81f2008640 1 --2- 172.21.15.156:0/1134241458 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f81ec0ff510 0x7f81ec1018f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:58:55.339 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:55.336+0000 7f81f2008640 1 --2- 172.21.15.156:0/1134241458 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f81ec0664e0 0x7f81ec0fefd0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:58:55.339 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:55.336+0000 7f81f2008640 1 -- 172.21.15.156:0/1134241458 >> 172.21.15.156:0/1134241458 conn(0x7f81ec0f6a20 msgr2=0x7f81ec0f8e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:58:55.339 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:55.337+0000 7f81f2008640 1 -- 172.21.15.156:0/1134241458 shutdown_connections 2024-07-16T09:58:55.339 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:55.337+0000 7f81f2008640 1 -- 172.21.15.156:0/1134241458 wait complete. 2024-07-16T09:58:55.339 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:55.337+0000 7f81f2008640 1 Processor -- start 2024-07-16T09:58:55.339 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:55.337+0000 7f81f2008640 1 -- start start 2024-07-16T09:58:55.340 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:55.337+0000 7f81f2008640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f81ec0664e0 0x7f81ec10d020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:58:55.340 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:55.337+0000 7f81f2008640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f81ec0ff510 0x7f81ec10d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:58:55.341 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:55.337+0000 7f81f2008640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81ec10dbb0 con 0x7f81ec0ff510 2024-07-16T09:58:55.341 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:55.337+0000 7f81f2008640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81ec10e3d0 con 0x7f81ec0664e0 2024-07-16T09:58:55.341 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:55.337+0000 7f81eaffd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f81ec0ff510 0x7f81ec10d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:58:55.341 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:55.338+0000 7f81eaffd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f81ec0ff510 0x7f81ec10d560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:46742/0 (socket says 172.21.15.156:46742) 2024-07-16T09:58:55.341 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:55.338+0000 7f81eaffd640 1 -- 172.21.15.156:0/1161848014 learned_addr learned my addr 172.21.15.156:0/1161848014 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T09:58:55.341 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:55.338+0000 7f81eb7fe640 1 --2- 172.21.15.156:0/1161848014 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f81ec0664e0 0x7f81ec10d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:58:55.341 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:55.338+0000 7f81eaffd640 1 -- 172.21.15.156:0/1161848014 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f81ec0664e0 msgr2=0x7f81ec10d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:58:55.341 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:55.338+0000 7f81eaffd640 1 --2- 172.21.15.156:0/1161848014 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f81ec0664e0 0x7f81ec10d020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:58:55.341 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:55.338+0000 7f81eaffd640 1 -- 172.21.15.156:0/1161848014 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81dc005040 con 0x7f81ec0ff510 2024-07-16T09:58:55.341 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:55.338+0000 7f81eaffd640 1 --2- 172.21.15.156:0/1161848014 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f81ec0ff510 0x7f81ec10d560 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f81e0002a50 tx=0x7f81e0004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:58:55.341 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:55.338+0000 7f81d3fff640 1 -- 172.21.15.156:0/1161848014 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f81e0007c30 con 0x7f81ec0ff510 2024-07-16T09:58:55.341 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:55.338+0000 7f81f2008640 1 -- 172.21.15.156:0/1161848014 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81ec10e660 con 0x7f81ec0ff510 2024-07-16T09:58:55.342 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:55.338+0000 7f81d3fff640 1 -- 172.21.15.156:0/1161848014 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f81e000f040 con 0x7f81ec0ff510 2024-07-16T09:58:55.342 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:55.338+0000 7f81d3fff640 1 -- 172.21.15.156:0/1161848014 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f81e000dad0 con 0x7f81ec0ff510 2024-07-16T09:58:55.342 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:55.338+0000 7f81f2008640 1 -- 172.21.15.156:0/1161848014 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81ec10ec00 con 0x7f81ec0ff510 2024-07-16T09:58:55.342 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:55.339+0000 7f81f2008640 1 -- 172.21.15.156:0/1161848014 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81ec0fa6c0 con 0x7f81ec0ff510 2024-07-16T09:58:55.343 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:55.340+0000 7f81d3fff640 1 -- 172.21.15.156:0/1161848014 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 37) v1 ==== 92307+0+0 (secure 0 0 0) 0x7f81e000dd40 con 0x7f81ec0ff510 2024-07-16T09:58:55.343 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:55.340+0000 7f81d3fff640 1 --2- 172.21.15.156:0/1161848014 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f81d406e1e0 0x7f81d4070690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:58:55.343 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:55.340+0000 7f81d3fff640 1 -- 172.21.15.156:0/1161848014 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f81e008d5c0 con 0x7f81ec0ff510 2024-07-16T09:58:55.343 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:55.340+0000 7f81eb7fe640 1 --2- 172.21.15.156:0/1161848014 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f81d406e1e0 0x7f81d4070690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:58:55.343 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:55.341+0000 7f81eb7fe640 1 --2- 172.21.15.156:0/1161848014 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f81d406e1e0 0x7f81d4070690 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f81dc000c00 tx=0x7f81dc035040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:58:55.346 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:55.344+0000 7f81d3fff640 1 -- 172.21.15.156:0/1161848014 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187716 (secure 0 0 0) 0x7f81e00567d0 con 0x7f81ec0ff510 2024-07-16T09:58:55.598 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:55 smithi156 ceph-mon[93066]: from='client.? 172.21.15.156:0/3560806479' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T09:58:55.599 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:55 smithi156 ceph-mon[93066]: from='client.? 172.21.15.156:0/1756310233' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-07-16T09:58:55.599 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:55 smithi156 ceph-mon[93066]: from='client.34208 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-16T09:58:55.599 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:55 smithi156 ceph-mon[93066]: osd.0 marked itself down and dead 2024-07-16T09:58:55.599 INFO:journalctl@ceph.osd.0.smithi156.stdout:Jul 16 09:58:55 smithi156 ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-0[47153]: 2024-07-16T09:58:55.349+0000 7f63848de640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-07-16T09:58:55.599 INFO:journalctl@ceph.osd.0.smithi156.stdout:Jul 16 09:58:55 smithi156 ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-0[47153]: 2024-07-16T09:58:55.349+0000 7f63848de640 -1 osd.0 49 *** Got signal Terminated *** 2024-07-16T09:58:55.599 INFO:journalctl@ceph.osd.0.smithi156.stdout:Jul 16 09:58:55 smithi156 ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-0[47153]: 2024-07-16T09:58:55.349+0000 7f63848de640 -1 osd.0 49 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-07-16T09:58:55.600 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:55.597+0000 7f81f2008640 1 -- 172.21.15.156:0/1161848014 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f81ec04b530 con 0x7f81ec0ff510 2024-07-16T09:58:55.600 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:55.597+0000 7f81d3fff640 1 -- 172.21.15.156:0/1161848014 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+100 (secure 0 0 0) 0x7f81e0059e90 con 0x7f81ec0ff510 2024-07-16T09:58:55.600 INFO:teuthology.orchestra.run.smithi156.stdout:HEALTH_WARN 1 osds down 2024-07-16T09:58:55.600 INFO:teuthology.orchestra.run.smithi156.stdout:[WRN] OSD_DOWN: 1 osds down 2024-07-16T09:58:55.600 INFO:teuthology.orchestra.run.smithi156.stdout: osd.0 (root=default,host=smithi156) is down 2024-07-16T09:58:55.601 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:55.599+0000 7f81f2008640 1 -- 172.21.15.156:0/1161848014 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f81d406e1e0 msgr2=0x7f81d4070690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:58:55.601 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:55.599+0000 7f81f2008640 1 --2- 172.21.15.156:0/1161848014 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f81d406e1e0 0x7f81d4070690 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f81dc000c00 tx=0x7f81dc035040 comp rx=0 tx=0).stop 2024-07-16T09:58:55.601 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:55.599+0000 7f81f2008640 1 -- 172.21.15.156:0/1161848014 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f81ec0ff510 msgr2=0x7f81ec10d560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:58:55.601 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:55.599+0000 7f81f2008640 1 --2- 172.21.15.156:0/1161848014 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f81ec0ff510 0x7f81ec10d560 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f81e0002a50 tx=0x7f81e0004600 comp rx=0 tx=0).stop 2024-07-16T09:58:55.601 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:55.599+0000 7f81f2008640 1 -- 172.21.15.156:0/1161848014 shutdown_connections 2024-07-16T09:58:55.601 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:55.599+0000 7f81f2008640 1 --2- 172.21.15.156:0/1161848014 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f81d406e1e0 0x7f81d4070690 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:58:55.601 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:55.599+0000 7f81f2008640 1 --2- 172.21.15.156:0/1161848014 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f81ec0ff510 0x7f81ec10d560 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:58:55.601 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:55.599+0000 7f81f2008640 1 --2- 172.21.15.156:0/1161848014 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f81ec0664e0 0x7f81ec10d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:58:55.601 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:55.599+0000 7f81f2008640 1 -- 172.21.15.156:0/1161848014 >> 172.21.15.156:0/1161848014 conn(0x7f81ec0f6a20 msgr2=0x7f81ec0f6e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:58:55.602 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:55.599+0000 7f81f2008640 1 -- 172.21.15.156:0/1161848014 shutdown_connections 2024-07-16T09:58:55.602 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:58:55.599+0000 7f81f2008640 1 -- 172.21.15.156:0/1161848014 wait complete. 2024-07-16T09:58:55.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:55 smithi176 ceph-mon[81689]: from='client.? 172.21.15.156:0/3560806479' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T09:58:55.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:55 smithi176 ceph-mon[81689]: from='client.? 172.21.15.156:0/1756310233' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-07-16T09:58:55.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:55 smithi176 ceph-mon[81689]: from='client.34208 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-16T09:58:55.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:55 smithi176 ceph-mon[81689]: osd.0 marked itself down and dead 2024-07-16T09:58:56.131 INFO:journalctl@ceph.osd.0.smithi156.stdout:Jul 16 09:58:55 smithi156 podman[101137]: 2024-07-16 09:58:55.869565385 +0000 UTC m=+0.668299113 container died 0471758ec3499584612d45f357a72419e76b6739419c2fccba0c728530c00eff (image=quay.ceph.io/ceph-ci/ceph@sha256:7b86ef6bffe0efc5f1aefc9fef8c854a67a511c50993acee3ac684f23bde83df, name=ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=quincy-e6231b0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240709, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image) 2024-07-16T09:58:56.397 INFO:journalctl@ceph.osd.0.smithi156.stdout:Jul 16 09:58:56 smithi156 podman[101137]: 2024-07-16 09:58:56.220997345 +0000 UTC m=+1.019731062 container cleanup 0471758ec3499584612d45f357a72419e76b6739419c2fccba0c728530c00eff (image=quay.ceph.io/ceph-ci/ceph@sha256:7b86ef6bffe0efc5f1aefc9fef8c854a67a511c50993acee3ac684f23bde83df, name=ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=quincy-e6231b0, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240709) 2024-07-16T09:58:56.397 INFO:journalctl@ceph.osd.0.smithi156.stdout:Jul 16 09:58:56 smithi156 bash[101137]: ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-0 2024-07-16T09:58:56.675 INFO:journalctl@ceph.osd.0.smithi156.stdout:Jul 16 09:58:56 smithi156 podman[101177]: 2024-07-16 09:58:56.531323213 +0000 UTC m=+0.660806621 container remove 0471758ec3499584612d45f357a72419e76b6739419c2fccba0c728530c00eff (image=quay.ceph.io/ceph-ci/ceph@sha256:7b86ef6bffe0efc5f1aefc9fef8c854a67a511c50993acee3ac684f23bde83df, name=ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240709, RELEASE=quincy-e6231b0, io.buildah.version=1.36.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-07-16T09:58:56.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:56 smithi156 ceph-mon[93066]: Health check failed: 1 osds down (OSD_DOWN) 2024-07-16T09:58:56.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:56 smithi156 ceph-mon[93066]: osdmap e50: 6 total, 5 up, 6 in 2024-07-16T09:58:56.676 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:56 smithi156 ceph-mon[93066]: from='client.? 172.21.15.156:0/1161848014' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-07-16T09:58:56.676 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:56 smithi156 ceph-mon[93066]: pgmap v38: 65 pgs: 9 stale+active+clean, 56 active+clean; 253 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s 2024-07-16T09:58:56.676 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:56 smithi156 ceph-mon[93066]: osdmap e51: 6 total, 5 up, 6 in 2024-07-16T09:58:56.843 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:56 smithi176 ceph-mon[81689]: Health check failed: 1 osds down (OSD_DOWN) 2024-07-16T09:58:56.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:56 smithi176 ceph-mon[81689]: osdmap e50: 6 total, 5 up, 6 in 2024-07-16T09:58:56.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:56 smithi176 ceph-mon[81689]: from='client.? 172.21.15.156:0/1161848014' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-07-16T09:58:56.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:56 smithi176 ceph-mon[81689]: pgmap v38: 65 pgs: 9 stale+active+clean, 56 active+clean; 253 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s 2024-07-16T09:58:56.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:56 smithi176 ceph-mon[81689]: osdmap e51: 6 total, 5 up, 6 in 2024-07-16T09:58:57.175 INFO:journalctl@ceph.osd.0.smithi156.stdout:Jul 16 09:58:56 smithi156 podman[101267]: 2024-07-16 09:58:56.802248394 +0000 UTC m=+0.017981446 image pull 16f7feb043422ce37a2471403d4bc52f550d49cef4b677482112fcc0db3f67ab quay.ceph.io/ceph-ci/ceph@sha256:c15ec3aa8e6c0a9c12ee24ff79f3e7dd4d01b3d1a702dfeb7d7fff9b0811a393 2024-07-16T09:58:57.175 INFO:journalctl@ceph.osd.0.smithi156.stdout:Jul 16 09:58:56 smithi156 podman[101267]: 2024-07-16 09:58:56.940618797 +0000 UTC m=+0.156351866 container create 1e956a6972c0dab88b30672e7b6ea092b5feda425e3b777dab88bb4fcb50320c (image=quay.ceph.io/ceph-ci/ceph@sha256:c15ec3aa8e6c0a9c12ee24ff79f3e7dd4d01b3d1a702dfeb7d7fff9b0811a393, name=ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-0-deactivate, GIT_BRANCH=HEAD, org.label-schema.build-date=20240709, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.36.0, org.label-schema.schema-version=1.0, RELEASE=wip-adk4-testing-2024-07-15-1832-squid-a6ca461, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True) 2024-07-16T09:58:57.674 INFO:journalctl@ceph.osd.0.smithi156.stdout:Jul 16 09:58:57 smithi156 podman[101267]: 2024-07-16 09:58:57.176501063 +0000 UTC m=+0.392234109 container init 1e956a6972c0dab88b30672e7b6ea092b5feda425e3b777dab88bb4fcb50320c (image=quay.ceph.io/ceph-ci/ceph@sha256:c15ec3aa8e6c0a9c12ee24ff79f3e7dd4d01b3d1a702dfeb7d7fff9b0811a393, name=ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-0-deactivate, io.buildah.version=1.36.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240709, RELEASE=wip-adk4-testing-2024-07-15-1832-squid-a6ca461, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True) 2024-07-16T09:58:57.675 INFO:journalctl@ceph.osd.0.smithi156.stdout:Jul 16 09:58:57 smithi156 podman[101267]: 2024-07-16 09:58:57.179797426 +0000 UTC m=+0.395530475 container start 1e956a6972c0dab88b30672e7b6ea092b5feda425e3b777dab88bb4fcb50320c (image=quay.ceph.io/ceph-ci/ceph@sha256:c15ec3aa8e6c0a9c12ee24ff79f3e7dd4d01b3d1a702dfeb7d7fff9b0811a393, name=ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-0-deactivate, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240709, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-adk4-testing-2024-07-15-1832-squid-a6ca461, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True) 2024-07-16T09:58:57.675 INFO:journalctl@ceph.osd.0.smithi156.stdout:Jul 16 09:58:57 smithi156 podman[101267]: 2024-07-16 09:58:57.276318742 +0000 UTC m=+0.492051816 container attach 1e956a6972c0dab88b30672e7b6ea092b5feda425e3b777dab88bb4fcb50320c (image=quay.ceph.io/ceph-ci/ceph@sha256:c15ec3aa8e6c0a9c12ee24ff79f3e7dd4d01b3d1a702dfeb7d7fff9b0811a393, name=ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-0-deactivate, GIT_CLEAN=True, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240709, RELEASE=wip-adk4-testing-2024-07-15-1832-squid-a6ca461, io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-07-16T09:58:57.675 INFO:journalctl@ceph.osd.0.smithi156.stdout:Jul 16 09:58:57 smithi156 podman[101267]: 2024-07-16 09:58:57.373556878 +0000 UTC m=+0.589289927 container died 1e956a6972c0dab88b30672e7b6ea092b5feda425e3b777dab88bb4fcb50320c (image=quay.ceph.io/ceph-ci/ceph@sha256:c15ec3aa8e6c0a9c12ee24ff79f3e7dd4d01b3d1a702dfeb7d7fff9b0811a393, name=ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-0-deactivate, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.36.0, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240709, RELEASE=wip-adk4-testing-2024-07-15-1832-squid-a6ca461) 2024-07-16T09:58:58.394 INFO:journalctl@ceph.osd.0.smithi156.stdout:Jul 16 09:58:57 smithi156 podman[101290]: 2024-07-16 09:58:57.947084046 +0000 UTC m=+0.567801905 container remove 1e956a6972c0dab88b30672e7b6ea092b5feda425e3b777dab88bb4fcb50320c (image=quay.ceph.io/ceph-ci/ceph@sha256:c15ec3aa8e6c0a9c12ee24ff79f3e7dd4d01b3d1a702dfeb7d7fff9b0811a393, name=ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-0-deactivate, GIT_CLEAN=True, org.label-schema.build-date=20240709, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.36.0, org.label-schema.vendor=CentOS, RELEASE=wip-adk4-testing-2024-07-15-1832-squid-a6ca461, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-07-16T09:58:58.394 INFO:journalctl@ceph.osd.0.smithi156.stdout:Jul 16 09:58:57 smithi156 systemd[1]: ceph-26c91c26-4358-11ef-bcad-c7b262605968@osd.0.service: Deactivated successfully. 2024-07-16T09:58:58.394 INFO:journalctl@ceph.osd.0.smithi156.stdout:Jul 16 09:58:57 smithi156 systemd[1]: Stopped Ceph osd.0 for 26c91c26-4358-11ef-bcad-c7b262605968. 2024-07-16T09:58:58.394 INFO:journalctl@ceph.osd.0.smithi156.stdout:Jul 16 09:58:57 smithi156 systemd[1]: ceph-26c91c26-4358-11ef-bcad-c7b262605968@osd.0.service: Consumed 57.567s CPU time. 2024-07-16T09:58:58.394 INFO:journalctl@ceph.osd.0.smithi156.stdout:Jul 16 09:58:58 smithi156 systemd[1]: Starting Ceph osd.0 for 26c91c26-4358-11ef-bcad-c7b262605968... 2024-07-16T09:58:58.660 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:58 smithi156 ceph-mon[93066]: fsmap cephfs:2 {0=cephfs.smithi156.rdibys=up:active,1=cephfs.smithi176.dvvbud=up:active} 2 up:standby-replay 2024-07-16T09:58:58.660 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:58:58 smithi156 ceph-mon[93066]: pgmap v40: 65 pgs: 9 stale+active+clean, 56 active+clean; 253 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-07-16T09:58:58.660 INFO:journalctl@ceph.osd.0.smithi156.stdout:Jul 16 09:58:58 smithi156 podman[101387]: 2024-07-16 09:58:58.295639788 +0000 UTC m=+0.017635958 image pull 16f7feb043422ce37a2471403d4bc52f550d49cef4b677482112fcc0db3f67ab quay.ceph.io/ceph-ci/ceph@sha256:c15ec3aa8e6c0a9c12ee24ff79f3e7dd4d01b3d1a702dfeb7d7fff9b0811a393 2024-07-16T09:58:58.660 INFO:journalctl@ceph.osd.0.smithi156.stdout:Jul 16 09:58:58 smithi156 podman[101387]: 2024-07-16 09:58:58.423320616 +0000 UTC m=+0.145316769 container create 6889b734d2bb3a84bf2be143c0643bee2bc20f1d5dce85b1f95f247fa76f69ce (image=quay.ceph.io/ceph-ci/ceph@sha256:c15ec3aa8e6c0a9c12ee24ff79f3e7dd4d01b3d1a702dfeb7d7fff9b0811a393, name=ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-0-activate, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.36.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.build-date=20240709, RELEASE=wip-adk4-testing-2024-07-15-1832-squid-a6ca461) 2024-07-16T09:58:58.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:58 smithi176 ceph-mon[81689]: fsmap cephfs:2 {0=cephfs.smithi156.rdibys=up:active,1=cephfs.smithi176.dvvbud=up:active} 2 up:standby-replay 2024-07-16T09:58:58.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:58:58 smithi176 ceph-mon[81689]: pgmap v40: 65 pgs: 9 stale+active+clean, 56 active+clean; 253 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-07-16T09:58:58.925 INFO:journalctl@ceph.osd.0.smithi156.stdout:Jul 16 09:58:58 smithi156 podman[101387]: 2024-07-16 09:58:58.718535889 +0000 UTC m=+0.440532058 container init 6889b734d2bb3a84bf2be143c0643bee2bc20f1d5dce85b1f95f247fa76f69ce (image=quay.ceph.io/ceph-ci/ceph@sha256:c15ec3aa8e6c0a9c12ee24ff79f3e7dd4d01b3d1a702dfeb7d7fff9b0811a393, name=ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-0-activate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240709, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.36.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , ceph=True, RELEASE=wip-adk4-testing-2024-07-15-1832-squid-a6ca461, org.label-schema.name=CentOS Stream 9 Base Image) 2024-07-16T09:58:58.925 INFO:journalctl@ceph.osd.0.smithi156.stdout:Jul 16 09:58:58 smithi156 podman[101387]: 2024-07-16 09:58:58.721856416 +0000 UTC m=+0.443852569 container start 6889b734d2bb3a84bf2be143c0643bee2bc20f1d5dce85b1f95f247fa76f69ce (image=quay.ceph.io/ceph-ci/ceph@sha256:c15ec3aa8e6c0a9c12ee24ff79f3e7dd4d01b3d1a702dfeb7d7fff9b0811a393, name=ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-0-activate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240709, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, RELEASE=wip-adk4-testing-2024-07-15-1832-squid-a6ca461, org.label-schema.schema-version=1.0, io.buildah.version=1.36.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-07-16T09:58:58.925 INFO:journalctl@ceph.osd.0.smithi156.stdout:Jul 16 09:58:58 smithi156 podman[101387]: 2024-07-16 09:58:58.828389646 +0000 UTC m=+0.550385802 container attach 6889b734d2bb3a84bf2be143c0643bee2bc20f1d5dce85b1f95f247fa76f69ce (image=quay.ceph.io/ceph-ci/ceph@sha256:c15ec3aa8e6c0a9c12ee24ff79f3e7dd4d01b3d1a702dfeb7d7fff9b0811a393, name=ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-0-activate, io.buildah.version=1.36.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240709, org.label-schema.license=GPLv2, RELEASE=wip-adk4-testing-2024-07-15-1832-squid-a6ca461, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0) 2024-07-16T09:58:58.925 INFO:journalctl@ceph.osd.0.smithi156.stdout:Jul 16 09:58:58 smithi156 ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-0-activate[101401]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-07-16T09:58:58.925 INFO:journalctl@ceph.osd.0.smithi156.stdout:Jul 16 09:58:58 smithi156 bash[101387]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-07-16T09:58:58.925 INFO:journalctl@ceph.osd.0.smithi156.stdout:Jul 16 09:58:58 smithi156 ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-0-activate[101401]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-07-16T09:58:58.925 INFO:journalctl@ceph.osd.0.smithi156.stdout:Jul 16 09:58:58 smithi156 bash[101387]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-07-16T09:58:59.423 INFO:journalctl@ceph.osd.0.smithi156.stdout:Jul 16 09:58:59 smithi156 ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-0-activate[101401]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-07-16T09:58:59.423 INFO:journalctl@ceph.osd.0.smithi156.stdout:Jul 16 09:58:59 smithi156 bash[101387]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-07-16T09:58:59.423 INFO:journalctl@ceph.osd.0.smithi156.stdout:Jul 16 09:58:59 smithi156 ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-0-activate[101401]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-0 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-07-16T09:58:59.423 INFO:journalctl@ceph.osd.0.smithi156.stdout:Jul 16 09:58:59 smithi156 bash[101387]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-0 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-07-16T09:58:59.423 INFO:journalctl@ceph.osd.0.smithi156.stdout:Jul 16 09:58:59 smithi156 ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-0-activate[101401]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-07-16T09:58:59.423 INFO:journalctl@ceph.osd.0.smithi156.stdout:Jul 16 09:58:59 smithi156 bash[101387]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-07-16T09:58:59.423 INFO:journalctl@ceph.osd.0.smithi156.stdout:Jul 16 09:58:59 smithi156 ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-0-activate[101401]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-07-16T09:58:59.424 INFO:journalctl@ceph.osd.0.smithi156.stdout:Jul 16 09:58:59 smithi156 bash[101387]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-07-16T09:58:59.424 INFO:journalctl@ceph.osd.0.smithi156.stdout:Jul 16 09:58:59 smithi156 ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-0-activate[101401]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-0/block 2024-07-16T09:58:59.424 INFO:journalctl@ceph.osd.0.smithi156.stdout:Jul 16 09:58:59 smithi156 bash[101387]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-0/block 2024-07-16T09:58:59.424 INFO:journalctl@ceph.osd.0.smithi156.stdout:Jul 16 09:58:59 smithi156 ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-0-activate[101401]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-07-16T09:58:59.424 INFO:journalctl@ceph.osd.0.smithi156.stdout:Jul 16 09:58:59 smithi156 bash[101387]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-07-16T09:58:59.424 INFO:journalctl@ceph.osd.0.smithi156.stdout:Jul 16 09:58:59 smithi156 ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-0-activate[101401]: --> ceph-volume raw activate successful for osd ID: 0 2024-07-16T09:58:59.424 INFO:journalctl@ceph.osd.0.smithi156.stdout:Jul 16 09:58:59 smithi156 bash[101387]: --> ceph-volume raw activate successful for osd ID: 0 2024-07-16T09:58:59.424 INFO:journalctl@ceph.osd.0.smithi156.stdout:Jul 16 09:58:59 smithi156 podman[101387]: 2024-07-16 09:58:59.101391396 +0000 UTC m=+0.823387569 container died 6889b734d2bb3a84bf2be143c0643bee2bc20f1d5dce85b1f95f247fa76f69ce (image=quay.ceph.io/ceph-ci/ceph@sha256:c15ec3aa8e6c0a9c12ee24ff79f3e7dd4d01b3d1a702dfeb7d7fff9b0811a393, name=ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-0-activate, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-adk4-testing-2024-07-15-1832-squid-a6ca461, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.36.0, org.label-schema.build-date=20240709, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-07-16T09:59:00.047 INFO:journalctl@ceph.osd.0.smithi156.stdout:Jul 16 09:58:59 smithi156 podman[101527]: 2024-07-16 09:58:59.714503055 +0000 UTC m=+0.606854961 container remove 6889b734d2bb3a84bf2be143c0643bee2bc20f1d5dce85b1f95f247fa76f69ce (image=quay.ceph.io/ceph-ci/ceph@sha256:c15ec3aa8e6c0a9c12ee24ff79f3e7dd4d01b3d1a702dfeb7d7fff9b0811a393, name=ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-0-activate, GIT_CLEAN=True, RELEASE=wip-adk4-testing-2024-07-15-1832-squid-a6ca461, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240709, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.36.0) 2024-07-16T09:59:00.047 INFO:journalctl@ceph.osd.0.smithi156.stdout:Jul 16 09:58:59 smithi156 podman[101585]: 2024-07-16 09:58:59.829598443 +0000 UTC m=+0.017387886 image pull 16f7feb043422ce37a2471403d4bc52f550d49cef4b677482112fcc0db3f67ab quay.ceph.io/ceph-ci/ceph@sha256:c15ec3aa8e6c0a9c12ee24ff79f3e7dd4d01b3d1a702dfeb7d7fff9b0811a393 2024-07-16T09:59:00.048 INFO:journalctl@ceph.osd.0.smithi156.stdout:Jul 16 09:58:59 smithi156 podman[101585]: 2024-07-16 09:58:59.948939435 +0000 UTC m=+0.136728910 container create 8633df1689cccb177b417e4827d55d249d75ad77e20260e3883d2368b89e44d9 (image=quay.ceph.io/ceph-ci/ceph@sha256:c15ec3aa8e6c0a9c12ee24ff79f3e7dd4d01b3d1a702dfeb7d7fff9b0811a393, name=ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-0, RELEASE=wip-adk4-testing-2024-07-15-1832-squid-a6ca461, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240709, io.buildah.version=1.36.0, org.label-schema.license=GPLv2) 2024-07-16T09:59:00.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:59:00 smithi156 ceph-mon[93066]: pgmap v41: 65 pgs: 15 active+undersized, 19 active+undersized+degraded, 31 active+clean; 253 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 2 op/s; 63/318 objects degraded (19.811%) 2024-07-16T09:59:00.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:59:00 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T09:59:00.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:59:00 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T09:59:00.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:59:00 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-16T09:59:00.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:59:00 smithi156 ceph-mon[93066]: Health check failed: Degraded data redundancy: 63/318 objects degraded (19.811%), 19 pgs degraded (PG_DEGRADED) 2024-07-16T09:59:00.675 INFO:journalctl@ceph.osd.0.smithi156.stdout:Jul 16 09:59:00 smithi156 podman[101585]: 2024-07-16 09:59:00.184429906 +0000 UTC m=+0.372219326 container init 8633df1689cccb177b417e4827d55d249d75ad77e20260e3883d2368b89e44d9 (image=quay.ceph.io/ceph-ci/ceph@sha256:c15ec3aa8e6c0a9c12ee24ff79f3e7dd4d01b3d1a702dfeb7d7fff9b0811a393, name=ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240709, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, io.buildah.version=1.36.0, RELEASE=wip-adk4-testing-2024-07-15-1832-squid-a6ca461, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-07-16T09:59:00.675 INFO:journalctl@ceph.osd.0.smithi156.stdout:Jul 16 09:59:00 smithi156 podman[101585]: 2024-07-16 09:59:00.187641234 +0000 UTC m=+0.375430667 container start 8633df1689cccb177b417e4827d55d249d75ad77e20260e3883d2368b89e44d9 (image=quay.ceph.io/ceph-ci/ceph@sha256:c15ec3aa8e6c0a9c12ee24ff79f3e7dd4d01b3d1a702dfeb7d7fff9b0811a393, name=ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-0, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, RELEASE=wip-adk4-testing-2024-07-15-1832-squid-a6ca461, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240709, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.36.0) 2024-07-16T09:59:00.675 INFO:journalctl@ceph.osd.0.smithi156.stdout:Jul 16 09:59:00 smithi156 ceph-osd[101604]: -- 172.21.15.156:0/2178991925 <== mon.0 v2:172.21.15.156:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5562ba8301e0 con 0x5562ba81a000 2024-07-16T09:59:00.675 INFO:journalctl@ceph.osd.0.smithi156.stdout:Jul 16 09:59:00 smithi156 bash[101585]: 8633df1689cccb177b417e4827d55d249d75ad77e20260e3883d2368b89e44d9 2024-07-16T09:59:00.675 INFO:journalctl@ceph.osd.0.smithi156.stdout:Jul 16 09:59:00 smithi156 systemd[1]: Started Ceph osd.0 for 26c91c26-4358-11ef-bcad-c7b262605968. 2024-07-16T09:59:00.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:59:00 smithi176 ceph-mon[81689]: pgmap v41: 65 pgs: 15 active+undersized, 19 active+undersized+degraded, 31 active+clean; 253 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 2 op/s; 63/318 objects degraded (19.811%) 2024-07-16T09:59:00.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:59:00 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T09:59:00.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:59:00 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T09:59:00.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:59:00 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-16T09:59:00.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:59:00 smithi176 ceph-mon[81689]: Health check failed: Degraded data redundancy: 63/318 objects degraded (19.811%), 19 pgs degraded (PG_DEGRADED) 2024-07-16T09:59:01.064 INFO:journalctl@ceph.osd.0.smithi156.stdout:Jul 16 09:59:00 smithi156 ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-0[101600]: 2024-07-16T09:59:00.731+0000 7f7f1642b640 -1 Falling back to public interface 2024-07-16T09:59:02.629 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:59:02 smithi156 ceph-mon[93066]: pgmap v42: 65 pgs: 15 active+undersized, 19 active+undersized+degraded, 31 active+clean; 253 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 0 op/s; 63/318 objects degraded (19.811%) 2024-07-16T09:59:02.646 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:59:02 smithi176 ceph-mon[81689]: pgmap v42: 65 pgs: 15 active+undersized, 19 active+undersized+degraded, 31 active+clean; 253 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 0 op/s; 63/318 objects degraded (19.811%) 2024-07-16T09:59:04.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:59:04 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T09:59:04.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:59:04 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-16T09:59:04.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:59:04 smithi156 ceph-mon[93066]: pgmap v43: 65 pgs: 15 active+undersized, 19 active+undersized+degraded, 31 active+clean; 253 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 2 op/s; 63/318 objects degraded (19.811%) 2024-07-16T09:59:04.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:59:04 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T09:59:04.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:59:04 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-16T09:59:04.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:59:04 smithi176 ceph-mon[81689]: pgmap v43: 65 pgs: 15 active+undersized, 19 active+undersized+degraded, 31 active+clean; 253 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 2 op/s; 63/318 objects degraded (19.811%) 2024-07-16T09:59:05.924 INFO:journalctl@ceph.osd.0.smithi156.stdout:Jul 16 09:59:05 smithi156 ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-0[101600]: 2024-07-16T09:59:05.596+0000 7f7f1642b640 -1 osd.0 49 log_to_monitors true 2024-07-16T09:59:06.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:59:06 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T09:59:06.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:59:06 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T09:59:06.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:59:06 smithi156 ceph-mon[93066]: from='osd.0 [v2:172.21.15.156:6802/2707468841,v1:172.21.15.156:6803/2707468841]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-07-16T09:59:06.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:59:06 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T09:59:06.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:59:06 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T09:59:06.676 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:59:06 smithi156 ceph-mon[93066]: pgmap v44: 65 pgs: 15 active+undersized, 19 active+undersized+degraded, 31 active+clean; 253 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 2 op/s; 63/318 objects degraded (19.811%) 2024-07-16T09:59:06.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:59:06 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T09:59:06.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:59:06 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T09:59:06.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:59:06 smithi176 ceph-mon[81689]: from='osd.0 [v2:172.21.15.156:6802/2707468841,v1:172.21.15.156:6803/2707468841]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-07-16T09:59:06.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:59:06 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T09:59:06.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:59:06 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T09:59:06.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:59:06 smithi176 ceph-mon[81689]: pgmap v44: 65 pgs: 15 active+undersized, 19 active+undersized+degraded, 31 active+clean; 253 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 2 op/s; 63/318 objects degraded (19.811%) 2024-07-16T09:59:07.405 INFO:journalctl@ceph.osd.0.smithi156.stdout:Jul 16 09:59:07 smithi156 ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-0[101600]: 2024-07-16T09:59:07.124+0000 7f7f0a1bd640 -1 osd.0 49 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-07-16T09:59:07.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:59:07 smithi156 ceph-mon[93066]: from='osd.0 [v2:172.21.15.156:6802/2707468841,v1:172.21.15.156:6803/2707468841]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-07-16T09:59:07.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:59:07 smithi156 ceph-mon[93066]: osdmap e52: 6 total, 5 up, 6 in 2024-07-16T09:59:07.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:59:07 smithi156 ceph-mon[93066]: from='osd.0 [v2:172.21.15.156:6802/2707468841,v1:172.21.15.156:6803/2707468841]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2024-07-16T09:59:07.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:59:07 smithi156 ceph-mon[93066]: from='osd.0 [v2:172.21.15.156:6802/2707468841,v1:172.21.15.156:6803/2707468841]' entity='osd.0' 2024-07-16T09:59:07.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:59:07 smithi176 ceph-mon[81689]: from='osd.0 [v2:172.21.15.156:6802/2707468841,v1:172.21.15.156:6803/2707468841]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-07-16T09:59:07.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:59:07 smithi176 ceph-mon[81689]: osdmap e52: 6 total, 5 up, 6 in 2024-07-16T09:59:07.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:59:07 smithi176 ceph-mon[81689]: from='osd.0 [v2:172.21.15.156:6802/2707468841,v1:172.21.15.156:6803/2707468841]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2024-07-16T09:59:07.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:59:07 smithi176 ceph-mon[81689]: from='osd.0 [v2:172.21.15.156:6802/2707468841,v1:172.21.15.156:6803/2707468841]' entity='osd.0' 2024-07-16T09:59:08.456 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:59:08 smithi156 ceph-mon[93066]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-07-16T09:59:08.457 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:59:08 smithi156 ceph-mon[93066]: osd.0 [v2:172.21.15.156:6802/2707468841,v1:172.21.15.156:6803/2707468841] boot 2024-07-16T09:59:08.457 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:59:08 smithi156 ceph-mon[93066]: osdmap e53: 6 total, 6 up, 6 in 2024-07-16T09:59:08.457 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:59:08 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-07-16T09:59:08.457 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:59:08 smithi156 ceph-mon[93066]: pgmap v47: 65 pgs: 15 active+undersized, 19 active+undersized+degraded, 31 active+clean; 253 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s; 63/318 objects degraded (19.811%) 2024-07-16T09:59:08.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:59:08 smithi176 ceph-mon[81689]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-07-16T09:59:08.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:59:08 smithi176 ceph-mon[81689]: osd.0 [v2:172.21.15.156:6802/2707468841,v1:172.21.15.156:6803/2707468841] boot 2024-07-16T09:59:08.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:59:08 smithi176 ceph-mon[81689]: osdmap e53: 6 total, 6 up, 6 in 2024-07-16T09:59:08.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:59:08 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-07-16T09:59:08.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:59:08 smithi176 ceph-mon[81689]: pgmap v47: 65 pgs: 15 active+undersized, 19 active+undersized+degraded, 31 active+clean; 253 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s; 63/318 objects degraded (19.811%) 2024-07-16T09:59:10.424 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:59:10 smithi156 ceph-mon[93066]: osdmap e54: 6 total, 6 up, 6 in 2024-07-16T09:59:10.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:59:10 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T09:59:10.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:59:10 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T09:59:10.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:59:10 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-16T09:59:10.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:59:10 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-16T09:59:10.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:59:10 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T09:59:10.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:59:10 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-16T09:59:10.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:59:10 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T09:59:10.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:59:10 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T09:59:10.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:59:10 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T09:59:10.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:59:10 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-07-16T09:59:10.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:59:10 smithi176 ceph-mon[81689]: osdmap e54: 6 total, 6 up, 6 in 2024-07-16T09:59:10.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:59:10 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T09:59:10.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:59:10 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T09:59:10.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:59:10 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-16T09:59:10.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:59:10 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-16T09:59:10.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:59:10 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T09:59:10.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:59:10 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-16T09:59:10.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:59:10 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T09:59:10.595 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:59:10 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T09:59:10.595 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:59:10 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T09:59:10.595 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:59:10 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-07-16T09:59:11.424 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:59:11 smithi156 ceph-mon[93066]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-07-16T09:59:11.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:59:11 smithi156 ceph-mon[93066]: Upgrade: unsafe to stop osd(s) at this time (15 PGs are or would become offline) 2024-07-16T09:59:11.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:59:11 smithi156 ceph-mon[93066]: pgmap v49: 65 pgs: 6 active+undersized, 3 active+undersized+degraded, 56 active+clean; 253 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 2 op/s; 11/318 objects degraded (3.459%) 2024-07-16T09:59:11.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:59:11 smithi176 ceph-mon[81689]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-07-16T09:59:11.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:59:11 smithi176 ceph-mon[81689]: Upgrade: unsafe to stop osd(s) at this time (15 PGs are or would become offline) 2024-07-16T09:59:11.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:59:11 smithi176 ceph-mon[81689]: pgmap v49: 65 pgs: 6 active+undersized, 3 active+undersized+degraded, 56 active+clean; 253 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 2 op/s; 11/318 objects degraded (3.459%) 2024-07-16T09:59:12.424 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:59:12 smithi156 ceph-mon[93066]: Health check update: Degraded data redundancy: 11/318 objects degraded (3.459%), 3 pgs degraded (PG_DEGRADED) 2024-07-16T09:59:12.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:59:12 smithi176 ceph-mon[81689]: Health check update: Degraded data redundancy: 11/318 objects degraded (3.459%), 3 pgs degraded (PG_DEGRADED) 2024-07-16T09:59:13.424 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:59:13 smithi156 ceph-mon[93066]: pgmap v50: 65 pgs: 65 active+clean; 253 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 2 op/s 2024-07-16T09:59:13.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:59:13 smithi176 ceph-mon[81689]: pgmap v50: 65 pgs: 65 active+clean; 253 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 2 op/s 2024-07-16T09:59:14.343 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:59:14 smithi176 ceph-mon[81689]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 11/318 objects degraded (3.459%), 3 pgs degraded) 2024-07-16T09:59:14.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:59:14 smithi176 ceph-mon[81689]: Cluster is now healthy 2024-07-16T09:59:14.424 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:59:14 smithi156 ceph-mon[93066]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 11/318 objects degraded (3.459%), 3 pgs degraded) 2024-07-16T09:59:14.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:59:14 smithi156 ceph-mon[93066]: Cluster is now healthy 2024-07-16T09:59:15.424 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:59:15 smithi156 ceph-mon[93066]: pgmap v51: 65 pgs: 65 active+clean; 253 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-07-16T09:59:15.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:59:15 smithi176 ceph-mon[81689]: pgmap v51: 65 pgs: 65 active+clean; 253 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-07-16T09:59:16.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:59:16 smithi156 ceph-mon[93066]: pgmap v52: 65 pgs: 65 active+clean; 253 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 2.4 KiB/s rd, 4 op/s 2024-07-16T09:59:16.843 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:59:16 smithi176 ceph-mon[81689]: pgmap v52: 65 pgs: 65 active+clean; 253 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 2.4 KiB/s rd, 4 op/s 2024-07-16T09:59:18.659 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:59:18 smithi156 ceph-mon[93066]: pgmap v53: 65 pgs: 65 active+clean; 253 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s 2024-07-16T09:59:18.659 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:59:18 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T09:59:18.659 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:59:18 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-16T09:59:18.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:59:18 smithi176 ceph-mon[81689]: pgmap v53: 65 pgs: 65 active+clean; 253 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s 2024-07-16T09:59:18.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:59:18 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T09:59:18.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:59:18 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-16T09:59:19.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:59:19 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T09:59:19.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:59:19 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T09:59:20.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:59:20 smithi156 ceph-mon[93066]: pgmap v54: 65 pgs: 65 active+clean; 253 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 2.9 KiB/s rd, 547 B/s wr, 5 op/s 2024-07-16T09:59:20.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:59:20 smithi176 ceph-mon[81689]: pgmap v54: 65 pgs: 65 active+clean; 253 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 2.9 KiB/s rd, 547 B/s wr, 5 op/s 2024-07-16T09:59:21.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:59:21 smithi156 ceph-mon[93066]: mgrmap e38: smithi156.zkvylv(active, since 92s), standbys: smithi176.dqbrvx 2024-07-16T09:59:21.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:59:21 smithi176 ceph-mon[81689]: mgrmap e38: smithi156.zkvylv(active, since 92s), standbys: smithi176.dqbrvx 2024-07-16T09:59:22.678 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:59:22 smithi156 ceph-mon[93066]: pgmap v55: 65 pgs: 65 active+clean; 253 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 1.8 KiB/s rd, 511 B/s wr, 3 op/s 2024-07-16T09:59:22.843 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:59:22 smithi176 ceph-mon[81689]: pgmap v55: 65 pgs: 65 active+clean; 253 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 1.8 KiB/s rd, 511 B/s wr, 3 op/s 2024-07-16T09:59:24.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:59:24 smithi156 ceph-mon[93066]: pgmap v56: 65 pgs: 65 active+clean; 253 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 2.7 KiB/s rd, 511 B/s wr, 5 op/s 2024-07-16T09:59:24.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:59:24 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-07-16T09:59:24.843 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:59:24 smithi176 ceph-mon[81689]: pgmap v56: 65 pgs: 65 active+clean; 253 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 2.7 KiB/s rd, 511 B/s wr, 5 op/s 2024-07-16T09:59:24.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:59:24 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-07-16T09:59:25.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:59:25 smithi156 ceph-mon[93066]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-07-16T09:59:25.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:59:25 smithi156 ceph-mon[93066]: Upgrade: osd.1 is safe to restart 2024-07-16T09:59:25.686 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:25.684+0000 7f5e267db640 1 -- 172.21.15.156:0/1414458560 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5e200fb240 msgr2=0x7f5e200fb650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:59:25.686 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:25.684+0000 7f5e267db640 1 --2- 172.21.15.156:0/1414458560 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5e200fb240 0x7f5e200fb650 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f5e100048d0 tx=0x7f5e10009da0 comp rx=0 tx=0).stop 2024-07-16T09:59:25.686 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:25.684+0000 7f5e267db640 1 -- 172.21.15.156:0/1414458560 shutdown_connections 2024-07-16T09:59:25.686 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:25.684+0000 7f5e267db640 1 --2- 172.21.15.156:0/1414458560 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5e200fc510 0x7f5e200fc980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:59:25.686 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:25.684+0000 7f5e267db640 1 --2- 172.21.15.156:0/1414458560 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5e200fb240 0x7f5e200fb650 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:59:25.686 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:25.684+0000 7f5e267db640 1 -- 172.21.15.156:0/1414458560 >> 172.21.15.156:0/1414458560 conn(0x7f5e200f6a20 msgr2=0x7f5e200f8e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:59:25.686 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:25.684+0000 7f5e267db640 1 -- 172.21.15.156:0/1414458560 shutdown_connections 2024-07-16T09:59:25.687 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:25.684+0000 7f5e267db640 1 -- 172.21.15.156:0/1414458560 wait complete. 2024-07-16T09:59:25.687 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:25.685+0000 7f5e267db640 1 Processor -- start 2024-07-16T09:59:25.687 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:25.685+0000 7f5e267db640 1 -- start start 2024-07-16T09:59:25.688 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:25.685+0000 7f5e267db640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5e200fb240 0x7f5e2010d000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:59:25.688 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:25.685+0000 7f5e267db640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5e200fc510 0x7f5e2010d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:59:25.688 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:25.685+0000 7f5e267db640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e2010e3b0 con 0x7f5e200fb240 2024-07-16T09:59:25.689 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:25.685+0000 7f5e267db640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e2010e520 con 0x7f5e200fc510 2024-07-16T09:59:25.689 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:25.685+0000 7f5e1ffff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5e200fb240 0x7f5e2010d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:59:25.689 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:25.685+0000 7f5e1ffff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5e200fb240 0x7f5e2010d000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:42068/0 (socket says 172.21.15.156:42068) 2024-07-16T09:59:25.689 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:25.685+0000 7f5e1ffff640 1 -- 172.21.15.156:0/1392995757 learned_addr learned my addr 172.21.15.156:0/1392995757 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T09:59:25.689 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:25.686+0000 7f5e1f7fe640 1 --2- 172.21.15.156:0/1392995757 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5e200fc510 0x7f5e2010d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:59:25.689 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:25.686+0000 7f5e1ffff640 1 -- 172.21.15.156:0/1392995757 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5e200fc510 msgr2=0x7f5e2010d540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:59:25.689 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:25.686+0000 7f5e1ffff640 1 --2- 172.21.15.156:0/1392995757 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5e200fc510 0x7f5e2010d540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:59:25.689 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:25.686+0000 7f5e1ffff640 1 -- 172.21.15.156:0/1392995757 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e10005040 con 0x7f5e200fb240 2024-07-16T09:59:25.689 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:25.686+0000 7f5e1ffff640 1 --2- 172.21.15.156:0/1392995757 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5e200fb240 0x7f5e2010d000 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f5e10003080 tx=0x7f5e10009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:59:25.689 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:25.686+0000 7f5e1cff9640 1 -- 172.21.15.156:0/1392995757 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5e10038070 con 0x7f5e200fb240 2024-07-16T09:59:25.689 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:25.686+0000 7f5e267db640 1 -- 172.21.15.156:0/1392995757 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e2010e7a0 con 0x7f5e200fb240 2024-07-16T09:59:25.690 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:25.686+0000 7f5e1cff9640 1 -- 172.21.15.156:0/1392995757 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5e10032970 con 0x7f5e200fb240 2024-07-16T09:59:25.690 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:25.686+0000 7f5e1cff9640 1 -- 172.21.15.156:0/1392995757 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5e10033900 con 0x7f5e200fb240 2024-07-16T09:59:25.690 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:25.686+0000 7f5e267db640 1 -- 172.21.15.156:0/1392995757 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e2010ec60 con 0x7f5e200fb240 2024-07-16T09:59:25.690 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:25.687+0000 7f5e267db640 1 -- 172.21.15.156:0/1392995757 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e20104620 con 0x7f5e200fb240 2024-07-16T09:59:25.690 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:25.688+0000 7f5e1cff9640 1 -- 172.21.15.156:0/1392995757 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 38) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f5e10030020 con 0x7f5e200fb240 2024-07-16T09:59:25.691 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:25.688+0000 7f5e1cff9640 1 --2- 172.21.15.156:0/1392995757 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f5e0806e0b0 0x7f5e08070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:59:25.691 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:25.688+0000 7f5e1cff9640 1 -- 172.21.15.156:0/1392995757 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f5e100adbd0 con 0x7f5e200fb240 2024-07-16T09:59:25.691 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:25.688+0000 7f5e1f7fe640 1 --2- 172.21.15.156:0/1392995757 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f5e0806e0b0 0x7f5e08070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:59:25.691 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:25.689+0000 7f5e1f7fe640 1 --2- 172.21.15.156:0/1392995757 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f5e0806e0b0 0x7f5e08070560 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f5e2010cdb0 tx=0x7f5e14008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:59:25.694 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:25.692+0000 7f5e1cff9640 1 -- 172.21.15.156:0/1392995757 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187716 (secure 0 0 0) 0x7f5e10076f50 con 0x7f5e200fb240 2024-07-16T09:59:25.843 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:59:25 smithi176 ceph-mon[81689]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-07-16T09:59:25.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:59:25 smithi176 ceph-mon[81689]: Upgrade: osd.1 is safe to restart 2024-07-16T09:59:25.900 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:25.898+0000 7f5e267db640 1 -- 172.21.15.156:0/1392995757 --> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f5e20103e10 con 0x7f5e0806e0b0 2024-07-16T09:59:25.900 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:25.899+0000 7f5e1cff9640 1 -- 172.21.15.156:0/1392995757 <== mgr.34104 v2:172.21.15.156:6800/3367765292 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f5e20103e10 con 0x7f5e0806e0b0 2024-07-16T09:59:25.903 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:25.901+0000 7f5e267db640 1 -- 172.21.15.156:0/1392995757 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f5e0806e0b0 msgr2=0x7f5e08070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:59:25.903 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:25.902+0000 7f5e267db640 1 --2- 172.21.15.156:0/1392995757 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f5e0806e0b0 0x7f5e08070560 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f5e2010cdb0 tx=0x7f5e14008040 comp rx=0 tx=0).stop 2024-07-16T09:59:25.903 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:25.902+0000 7f5e267db640 1 -- 172.21.15.156:0/1392995757 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5e200fb240 msgr2=0x7f5e2010d000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:59:25.903 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:25.902+0000 7f5e267db640 1 --2- 172.21.15.156:0/1392995757 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5e200fb240 0x7f5e2010d000 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f5e10003080 tx=0x7f5e10009da0 comp rx=0 tx=0).stop 2024-07-16T09:59:25.903 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:25.902+0000 7f5e267db640 1 -- 172.21.15.156:0/1392995757 shutdown_connections 2024-07-16T09:59:25.903 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:25.902+0000 7f5e267db640 1 --2- 172.21.15.156:0/1392995757 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f5e0806e0b0 0x7f5e08070560 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:59:25.904 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:25.902+0000 7f5e267db640 1 --2- 172.21.15.156:0/1392995757 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5e200fc510 0x7f5e2010d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:59:25.904 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:25.902+0000 7f5e267db640 1 --2- 172.21.15.156:0/1392995757 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5e200fb240 0x7f5e2010d000 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:59:25.904 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:25.902+0000 7f5e267db640 1 -- 172.21.15.156:0/1392995757 >> 172.21.15.156:0/1392995757 conn(0x7f5e200f6a20 msgr2=0x7f5e20065400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:59:25.904 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:25.902+0000 7f5e267db640 1 -- 172.21.15.156:0/1392995757 shutdown_connections 2024-07-16T09:59:25.904 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:25.902+0000 7f5e267db640 1 -- 172.21.15.156:0/1392995757 wait complete. 2024-07-16T09:59:25.912 INFO:teuthology.orchestra.run.smithi156.stdout:true 2024-07-16T09:59:25.987 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:25.985+0000 7f5bf274c640 1 -- 172.21.15.156:0/3444345975 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5bec0fc510 msgr2=0x7f5bec0fc980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:59:25.987 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:25.985+0000 7f5bf274c640 1 --2- 172.21.15.156:0/3444345975 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5bec0fc510 0x7f5bec0fc980 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f5be00048d0 tx=0x7f5be0009da0 comp rx=0 tx=0).stop 2024-07-16T09:59:25.987 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:25.985+0000 7f5bf274c640 1 -- 172.21.15.156:0/3444345975 shutdown_connections 2024-07-16T09:59:25.987 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:25.985+0000 7f5bf274c640 1 --2- 172.21.15.156:0/3444345975 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5bec0fc510 0x7f5bec0fc980 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:59:25.987 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:25.985+0000 7f5bf274c640 1 --2- 172.21.15.156:0/3444345975 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5bec0fb240 0x7f5bec0fb650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:59:25.988 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:25.985+0000 7f5bf274c640 1 -- 172.21.15.156:0/3444345975 >> 172.21.15.156:0/3444345975 conn(0x7f5bec0f6a20 msgr2=0x7f5bec0f8e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:59:25.988 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:25.985+0000 7f5bf274c640 1 -- 172.21.15.156:0/3444345975 shutdown_connections 2024-07-16T09:59:25.988 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:25.985+0000 7f5bf274c640 1 -- 172.21.15.156:0/3444345975 wait complete. 2024-07-16T09:59:25.988 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:25.986+0000 7f5bf274c640 1 Processor -- start 2024-07-16T09:59:25.988 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:25.986+0000 7f5bf274c640 1 -- start start 2024-07-16T09:59:25.989 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:25.986+0000 7f5bf274c640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5bec0fb240 0x7f5bec10cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:59:25.990 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:25.986+0000 7f5bf274c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5bec0fc510 0x7f5bec10d520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:59:25.990 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:25.986+0000 7f5bf274c640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bec10db70 con 0x7f5bec0fc510 2024-07-16T09:59:25.990 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:25.986+0000 7f5bf274c640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bec10e340 con 0x7f5bec0fb240 2024-07-16T09:59:25.990 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:25.986+0000 7f5beb7fe640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5bec0fc510 0x7f5bec10d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:59:25.990 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:25.986+0000 7f5bebfff640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5bec0fb240 0x7f5bec10cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:59:25.990 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:25.986+0000 7f5bebfff640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5bec0fb240 0x7f5bec10cfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.156:36480/0 (socket says 172.21.15.156:36480) 2024-07-16T09:59:25.990 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:25.986+0000 7f5beb7fe640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5bec0fc510 0x7f5bec10d520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:42094/0 (socket says 172.21.15.156:42094) 2024-07-16T09:59:25.990 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:25.987+0000 7f5bebfff640 1 -- 172.21.15.156:0/4149013270 learned_addr learned my addr 172.21.15.156:0/4149013270 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T09:59:25.990 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:25.987+0000 7f5bebfff640 1 -- 172.21.15.156:0/4149013270 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5bec0fc510 msgr2=0x7f5bec10d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:59:25.990 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:25.987+0000 7f5bebfff640 1 --2- 172.21.15.156:0/4149013270 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5bec0fc510 0x7f5bec10d520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:59:25.990 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:25.987+0000 7f5bebfff640 1 -- 172.21.15.156:0/4149013270 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5be0005040 con 0x7f5bec0fb240 2024-07-16T09:59:25.991 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:25.987+0000 7f5bebfff640 1 --2- 172.21.15.156:0/4149013270 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5bec0fb240 0x7f5bec10cfe0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f5bdc00eb70 tx=0x7f5bdc00b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:59:25.991 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:25.987+0000 7f5be8ff9640 1 -- 172.21.15.156:0/4149013270 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5bdc007c30 con 0x7f5bec0fb240 2024-07-16T09:59:25.991 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:25.987+0000 7f5be8ff9640 1 -- 172.21.15.156:0/4149013270 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5bdc00f040 con 0x7f5bec0fb240 2024-07-16T09:59:25.991 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:25.987+0000 7f5bf274c640 1 -- 172.21.15.156:0/4149013270 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5bec10e570 con 0x7f5bec0fb240 2024-07-16T09:59:25.991 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:25.987+0000 7f5be8ff9640 1 -- 172.21.15.156:0/4149013270 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5bdc00cad0 con 0x7f5bec0fb240 2024-07-16T09:59:25.991 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:25.987+0000 7f5bf274c640 1 -- 172.21.15.156:0/4149013270 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5bec10eae0 con 0x7f5bec0fb240 2024-07-16T09:59:25.991 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:25.988+0000 7f5bf274c640 1 -- 172.21.15.156:0/4149013270 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5bec1045c0 con 0x7f5bec0fb240 2024-07-16T09:59:25.992 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:25.989+0000 7f5be8ff9640 1 -- 172.21.15.156:0/4149013270 <== mon.1 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 38) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f5bdc00cd10 con 0x7f5bec0fb240 2024-07-16T09:59:25.992 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:25.989+0000 7f5be8ff9640 1 --2- 172.21.15.156:0/4149013270 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f5bd406e0b0 0x7f5bd4070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:59:25.992 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:25.990+0000 7f5be8ff9640 1 -- 172.21.15.156:0/4149013270 <== mon.1 v2:172.21.15.176:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f5bdc08d7d0 con 0x7f5bec0fb240 2024-07-16T09:59:25.992 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:25.990+0000 7f5beb7fe640 1 --2- 172.21.15.156:0/4149013270 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f5bd406e0b0 0x7f5bd4070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:59:25.992 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:25.990+0000 7f5beb7fe640 1 --2- 172.21.15.156:0/4149013270 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f5bd406e0b0 0x7f5bd4070560 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f5be0009f80 tx=0x7f5be0035040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:59:25.995 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:25.994+0000 7f5be8ff9640 1 -- 172.21.15.156:0/4149013270 <== mon.1 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187716 (secure 0 0 0) 0x7f5bdc056c00 con 0x7f5bec0fb240 2024-07-16T09:59:26.199 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:26.198+0000 7f5bf274c640 1 -- 172.21.15.156:0/4149013270 --> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f5bec103f20 con 0x7f5bd406e0b0 2024-07-16T09:59:26.200 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:26.199+0000 7f5be8ff9640 1 -- 172.21.15.156:0/4149013270 <== mgr.34104 v2:172.21.15.156:6800/3367765292 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f5bec103f20 con 0x7f5bd406e0b0 2024-07-16T09:59:26.203 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:26.202+0000 7f5bf274c640 1 -- 172.21.15.156:0/4149013270 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f5bd406e0b0 msgr2=0x7f5bd4070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:59:26.203 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:26.202+0000 7f5bf274c640 1 --2- 172.21.15.156:0/4149013270 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f5bd406e0b0 0x7f5bd4070560 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f5be0009f80 tx=0x7f5be0035040 comp rx=0 tx=0).stop 2024-07-16T09:59:26.203 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:26.202+0000 7f5bf274c640 1 -- 172.21.15.156:0/4149013270 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5bec0fb240 msgr2=0x7f5bec10cfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:59:26.203 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:26.202+0000 7f5bf274c640 1 --2- 172.21.15.156:0/4149013270 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5bec0fb240 0x7f5bec10cfe0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f5bdc00eb70 tx=0x7f5bdc00b040 comp rx=0 tx=0).stop 2024-07-16T09:59:26.203 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:26.202+0000 7f5bf274c640 1 -- 172.21.15.156:0/4149013270 shutdown_connections 2024-07-16T09:59:26.203 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:26.202+0000 7f5bf274c640 1 --2- 172.21.15.156:0/4149013270 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f5bd406e0b0 0x7f5bd4070560 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:59:26.203 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:26.202+0000 7f5bf274c640 1 --2- 172.21.15.156:0/4149013270 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5bec0fc510 0x7f5bec10d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:59:26.204 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:26.202+0000 7f5bf274c640 1 --2- 172.21.15.156:0/4149013270 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5bec0fb240 0x7f5bec10cfe0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:59:26.204 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:26.202+0000 7f5bf274c640 1 -- 172.21.15.156:0/4149013270 >> 172.21.15.156:0/4149013270 conn(0x7f5bec0f6a20 msgr2=0x7f5bec065380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:59:26.204 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:26.202+0000 7f5bf274c640 1 -- 172.21.15.156:0/4149013270 shutdown_connections 2024-07-16T09:59:26.204 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:26.202+0000 7f5bf274c640 1 -- 172.21.15.156:0/4149013270 wait complete. 2024-07-16T09:59:26.284 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:26.283+0000 7fb82e024640 1 -- 172.21.15.156:0/1765573812 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb8280fe300 msgr2=0x7fb8280fa460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:59:26.284 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:26.283+0000 7fb82e024640 1 --2- 172.21.15.156:0/1765573812 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb8280fe300 0x7fb8280fa460 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fb81c0048d0 tx=0x7fb81c009da0 comp rx=0 tx=0).stop 2024-07-16T09:59:26.285 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:26.283+0000 7fb82e024640 1 -- 172.21.15.156:0/1765573812 shutdown_connections 2024-07-16T09:59:26.285 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:26.283+0000 7fb82e024640 1 --2- 172.21.15.156:0/1765573812 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb8280fe300 0x7fb8280fa460 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:59:26.285 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:26.283+0000 7fb82e024640 1 --2- 172.21.15.156:0/1765573812 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb8280fd9b0 0x7fb8280fddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:59:26.285 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:26.283+0000 7fb82e024640 1 -- 172.21.15.156:0/1765573812 >> 172.21.15.156:0/1765573812 conn(0x7fb8280f61b0 msgr2=0x7fb8280f85e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:59:26.285 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:26.283+0000 7fb82e024640 1 -- 172.21.15.156:0/1765573812 shutdown_connections 2024-07-16T09:59:26.285 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:26.283+0000 7fb82e024640 1 -- 172.21.15.156:0/1765573812 wait complete. 2024-07-16T09:59:26.287 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:26.284+0000 7fb82e024640 1 Processor -- start 2024-07-16T09:59:26.287 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:26.284+0000 7fb82e024640 1 -- start start 2024-07-16T09:59:26.287 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:26.284+0000 7fb82e024640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb8280fd9b0 0x7fb82810cf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:59:26.287 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:26.284+0000 7fb82e024640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb8280fe300 0x7fb82810d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:59:26.287 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:26.284+0000 7fb82e024640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb82810dad0 con 0x7fb8280fd9b0 2024-07-16T09:59:26.287 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:26.284+0000 7fb82e024640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb82810e2f0 con 0x7fb8280fe300 2024-07-16T09:59:26.287 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:26.284+0000 7fb8277fe640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb8280fd9b0 0x7fb82810cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:59:26.287 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:26.284+0000 7fb8277fe640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb8280fd9b0 0x7fb82810cf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:42114/0 (socket says 172.21.15.156:42114) 2024-07-16T09:59:26.287 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:26.284+0000 7fb8277fe640 1 -- 172.21.15.156:0/3262714433 learned_addr learned my addr 172.21.15.156:0/3262714433 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T09:59:26.288 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:26.284+0000 7fb826ffd640 1 --2- 172.21.15.156:0/3262714433 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb8280fe300 0x7fb82810d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:59:26.288 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:26.285+0000 7fb8277fe640 1 -- 172.21.15.156:0/3262714433 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb8280fe300 msgr2=0x7fb82810d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:59:26.288 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:26.285+0000 7fb8277fe640 1 --2- 172.21.15.156:0/3262714433 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb8280fe300 0x7fb82810d480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:59:26.288 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:26.285+0000 7fb8277fe640 1 -- 172.21.15.156:0/3262714433 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb81c005040 con 0x7fb8280fd9b0 2024-07-16T09:59:26.288 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:26.285+0000 7fb8277fe640 1 --2- 172.21.15.156:0/3262714433 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb8280fd9b0 0x7fb82810cf40 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fb81800c810 tx=0x7fb8180043c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:59:26.288 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:26.285+0000 7fb80ffff640 1 -- 172.21.15.156:0/3262714433 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb81800a7a0 con 0x7fb8280fd9b0 2024-07-16T09:59:26.288 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:26.285+0000 7fb82e024640 1 -- 172.21.15.156:0/3262714433 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb82810e580 con 0x7fb8280fd9b0 2024-07-16T09:59:26.288 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:26.285+0000 7fb80ffff640 1 -- 172.21.15.156:0/3262714433 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb81800add0 con 0x7fb8280fd9b0 2024-07-16T09:59:26.288 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:26.285+0000 7fb80ffff640 1 -- 172.21.15.156:0/3262714433 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb8180115c0 con 0x7fb8280fd9b0 2024-07-16T09:59:26.288 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:26.285+0000 7fb82e024640 1 -- 172.21.15.156:0/3262714433 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb82810eaf0 con 0x7fb8280fd9b0 2024-07-16T09:59:26.289 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:26.286+0000 7fb82e024640 1 -- 172.21.15.156:0/3262714433 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8281056e0 con 0x7fb8280fd9b0 2024-07-16T09:59:26.289 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:26.287+0000 7fb80ffff640 1 -- 172.21.15.156:0/3262714433 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 38) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fb818012420 con 0x7fb8280fd9b0 2024-07-16T09:59:26.289 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:26.287+0000 7fb80ffff640 1 --2- 172.21.15.156:0/3262714433 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7fb81006e0b0 0x7fb810070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:59:26.289 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:26.287+0000 7fb80ffff640 1 -- 172.21.15.156:0/3262714433 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fb81808dff0 con 0x7fb8280fd9b0 2024-07-16T09:59:26.289 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:26.287+0000 7fb826ffd640 1 --2- 172.21.15.156:0/3262714433 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7fb81006e0b0 0x7fb810070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:59:26.290 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:26.288+0000 7fb826ffd640 1 --2- 172.21.15.156:0/3262714433 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7fb81006e0b0 0x7fb810070560 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fb81c003080 tx=0x7fb81c035040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:59:26.292 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:26.291+0000 7fb80ffff640 1 -- 172.21.15.156:0/3262714433 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187716 (secure 0 0 0) 0x7fb818057370 con 0x7fb8280fd9b0 2024-07-16T09:59:26.486 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:26.484+0000 7fb82e024640 1 -- 172.21.15.156:0/3262714433 --> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fb82810ee80 con 0x7fb81006e0b0 2024-07-16T09:59:26.498 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:26.495+0000 7fb80ffff640 1 -- 172.21.15.156:0/3262714433 <== mgr.34104 v2:172.21.15.156:6800/3367765292 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7fb82810ee80 con 0x7fb81006e0b0 2024-07-16T09:59:26.498 INFO:teuthology.orchestra.run.smithi156.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-07-16T09:59:26.498 INFO:teuthology.orchestra.run.smithi156.stdout:alertmanager.smithi156 smithi156 *:9093,9094 running (3m) 21s ago 11m 16.2M - 0.25.0 c8568f914cd2 d524a2c3ff15 2024-07-16T09:59:26.499 INFO:teuthology.orchestra.run.smithi156.stdout:crash.smithi156 smithi156 running (50s) 21s ago 11m 6899k - 19.0.0-3495-ga6ca461b 16f7feb04342 11648cf29af4 2024-07-16T09:59:26.499 INFO:teuthology.orchestra.run.smithi156.stdout:crash.smithi176 smithi176 running (45s) 41s ago 10m 6899k - 19.0.0-3495-ga6ca461b 16f7feb04342 67f3ccc034f8 2024-07-16T09:59:26.499 INFO:teuthology.orchestra.run.smithi156.stdout:grafana.smithi156 smithi156 *:3000 running (2m) 21s ago 11m 78.1M - 9.4.7 954c08fa6188 deb9e1bfeb66 2024-07-16T09:59:26.499 INFO:teuthology.orchestra.run.smithi156.stdout:mds.cephfs.smithi156.evwwaz smithi156 running (6m) 21s ago 6m 67.3M - 17.2.7-1104-ge6231b04 d5de5f953b95 7c0ff9ae43f8 2024-07-16T09:59:26.499 INFO:teuthology.orchestra.run.smithi156.stdout:mds.cephfs.smithi156.rdibys smithi156 running (6m) 21s ago 6m 120M - 17.2.7-1104-ge6231b04 d5de5f953b95 b985cd1e6924 2024-07-16T09:59:26.499 INFO:teuthology.orchestra.run.smithi156.stdout:mds.cephfs.smithi176.dvvbud smithi176 running (6m) 41s ago 6m 108M - 17.2.7-1104-ge6231b04 d5de5f953b95 1b00147950f9 2024-07-16T09:59:26.499 INFO:teuthology.orchestra.run.smithi156.stdout:mds.cephfs.smithi176.wahbsi smithi176 running (6m) 41s ago 6m 62.2M - 17.2.7-1104-ge6231b04 d5de5f953b95 866664af2311 2024-07-16T09:59:26.499 INFO:teuthology.orchestra.run.smithi156.stdout:mgr.smithi156.zkvylv smithi156 *:8443,9283,8765 running (2m) 21s ago 12m 494M - 19.0.0-3495-ga6ca461b 16f7feb04342 ee53bd604b7c 2024-07-16T09:59:26.499 INFO:teuthology.orchestra.run.smithi156.stdout:mgr.smithi176.dqbrvx smithi176 *:8443,9283,8765 running (117s) 41s ago 10m 438M - 19.0.0-3495-ga6ca461b 16f7feb04342 ce9eaae4c2d0 2024-07-16T09:59:26.499 INFO:teuthology.orchestra.run.smithi156.stdout:mon.smithi156 smithi156 running (105s) 21s ago 13m 41.9M 2048M 19.0.0-3495-ga6ca461b 16f7feb04342 77b6ec8c50eb 2024-07-16T09:59:26.499 INFO:teuthology.orchestra.run.smithi156.stdout:mon.smithi176 smithi176 running (85s) 41s ago 10m 33.4M 2048M 19.0.0-3495-ga6ca461b 16f7feb04342 ef92814cec2b 2024-07-16T09:59:26.499 INFO:teuthology.orchestra.run.smithi156.stdout:node-exporter.smithi156 smithi156 *:9100 running (3m) 21s ago 11m 16.6M - 1.5.0 0da6a335fe13 e5d86f23a14c 2024-07-16T09:59:26.499 INFO:teuthology.orchestra.run.smithi156.stdout:node-exporter.smithi176 smithi176 *:9100 running (2m) 41s ago 10m 16.1M - 1.5.0 0da6a335fe13 ea842728144a 2024-07-16T09:59:26.499 INFO:teuthology.orchestra.run.smithi156.stdout:osd.0 smithi156 running (26s) 21s ago 9m 11.9M 1557M 19.0.0-3495-ga6ca461b 16f7feb04342 8633df1689cc 2024-07-16T09:59:26.500 INFO:teuthology.orchestra.run.smithi156.stdout:osd.1 smithi156 running (9m) 21s ago 9m 245M 1557M 17.2.7-1104-ge6231b04 d5de5f953b95 149a0bda2f61 2024-07-16T09:59:26.500 INFO:teuthology.orchestra.run.smithi156.stdout:osd.2 smithi156 running (9m) 21s ago 9m 198M 1557M 17.2.7-1104-ge6231b04 d5de5f953b95 f28acec5cb79 2024-07-16T09:59:26.500 INFO:teuthology.orchestra.run.smithi156.stdout:osd.3 smithi176 running (8m) 41s ago 8m 260M 2581M 17.2.7-1104-ge6231b04 d5de5f953b95 733861ba84dc 2024-07-16T09:59:26.500 INFO:teuthology.orchestra.run.smithi156.stdout:osd.4 smithi176 running (8m) 41s ago 8m 229M 2581M 17.2.7-1104-ge6231b04 d5de5f953b95 ef0c67cec086 2024-07-16T09:59:26.500 INFO:teuthology.orchestra.run.smithi156.stdout:osd.5 smithi176 running (7m) 41s ago 7m 224M 2581M 17.2.7-1104-ge6231b04 d5de5f953b95 996e15448a52 2024-07-16T09:59:26.500 INFO:teuthology.orchestra.run.smithi156.stdout:prometheus.smithi156 smithi156 *:9095 running (2m) 21s ago 11m 53.8M - 2.43.0 a07b618ecd1d 20c195ecd665 2024-07-16T09:59:26.500 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:26.497+0000 7fb82e024640 1 -- 172.21.15.156:0/3262714433 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7fb81006e0b0 msgr2=0x7fb810070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:59:26.500 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:26.497+0000 7fb82e024640 1 --2- 172.21.15.156:0/3262714433 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7fb81006e0b0 0x7fb810070560 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fb81c003080 tx=0x7fb81c035040 comp rx=0 tx=0).stop 2024-07-16T09:59:26.501 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:26.497+0000 7fb82e024640 1 -- 172.21.15.156:0/3262714433 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb8280fd9b0 msgr2=0x7fb82810cf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:59:26.501 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:26.497+0000 7fb82e024640 1 --2- 172.21.15.156:0/3262714433 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb8280fd9b0 0x7fb82810cf40 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fb81800c810 tx=0x7fb8180043c0 comp rx=0 tx=0).stop 2024-07-16T09:59:26.501 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:26.497+0000 7fb82e024640 1 -- 172.21.15.156:0/3262714433 shutdown_connections 2024-07-16T09:59:26.501 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:26.497+0000 7fb82e024640 1 --2- 172.21.15.156:0/3262714433 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7fb81006e0b0 0x7fb810070560 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:59:26.501 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:26.497+0000 7fb82e024640 1 --2- 172.21.15.156:0/3262714433 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb8280fe300 0x7fb82810d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:59:26.501 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:26.497+0000 7fb82e024640 1 --2- 172.21.15.156:0/3262714433 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb8280fd9b0 0x7fb82810cf40 secure :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fb81800c810 tx=0x7fb8180043c0 comp rx=0 tx=0).stop 2024-07-16T09:59:26.501 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:26.497+0000 7fb82e024640 1 -- 172.21.15.156:0/3262714433 >> 172.21.15.156:0/3262714433 conn(0x7fb8280f61b0 msgr2=0x7fb8280f7830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:59:26.501 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:26.498+0000 7fb82e024640 1 -- 172.21.15.156:0/3262714433 shutdown_connections 2024-07-16T09:59:26.501 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:26.498+0000 7fb82e024640 1 -- 172.21.15.156:0/3262714433 wait complete. 2024-07-16T09:59:26.580 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:26.577+0000 7f8e0dcee640 1 -- 172.21.15.156:0/2879270571 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8e080ff4d0 msgr2=0x7f8e081018b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:59:26.580 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:26.577+0000 7f8e0dcee640 1 --2- 172.21.15.156:0/2879270571 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8e080ff4d0 0x7f8e081018b0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f8dfc0048d0 tx=0x7f8dfc009da0 comp rx=0 tx=0).stop 2024-07-16T09:59:26.580 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:26.577+0000 7f8e0dcee640 1 -- 172.21.15.156:0/2879270571 shutdown_connections 2024-07-16T09:59:26.581 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:26.577+0000 7f8e0dcee640 1 --2- 172.21.15.156:0/2879270571 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8e080ff4d0 0x7f8e081018b0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:59:26.581 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:26.577+0000 7f8e0dcee640 1 --2- 172.21.15.156:0/2879270571 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8e08066520 0x7f8e080fef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:59:26.581 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:26.577+0000 7f8e0dcee640 1 -- 172.21.15.156:0/2879270571 >> 172.21.15.156:0/2879270571 conn(0x7f8e080f6a20 msgr2=0x7f8e080f8e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:59:26.581 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:26.577+0000 7f8e0dcee640 1 -- 172.21.15.156:0/2879270571 shutdown_connections 2024-07-16T09:59:26.581 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:26.577+0000 7f8e0dcee640 1 -- 172.21.15.156:0/2879270571 wait complete. 2024-07-16T09:59:26.581 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:26.578+0000 7f8e0dcee640 1 Processor -- start 2024-07-16T09:59:26.581 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:26.578+0000 7f8e0dcee640 1 -- start start 2024-07-16T09:59:26.581 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:26.578+0000 7f8e0dcee640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8e08066520 0x7f8e0810cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:59:26.581 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:26.578+0000 7f8e0dcee640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8e080ff4d0 0x7f8e0810d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:59:26.581 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:26.578+0000 7f8e0dcee640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e0810db20 con 0x7f8e080ff4d0 2024-07-16T09:59:26.581 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:26.578+0000 7f8e0dcee640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e0810e2f0 con 0x7f8e08066520 2024-07-16T09:59:26.582 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:26.578+0000 7f8e06ffd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8e080ff4d0 0x7f8e0810d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:59:26.582 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:26.578+0000 7f8e06ffd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8e080ff4d0 0x7f8e0810d4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:42140/0 (socket says 172.21.15.156:42140) 2024-07-16T09:59:26.582 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:26.578+0000 7f8e06ffd640 1 -- 172.21.15.156:0/3145442940 learned_addr learned my addr 172.21.15.156:0/3145442940 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T09:59:26.582 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:26.578+0000 7f8e077fe640 1 --2- 172.21.15.156:0/3145442940 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8e08066520 0x7f8e0810cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:59:26.582 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:26.579+0000 7f8e06ffd640 1 -- 172.21.15.156:0/3145442940 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8e08066520 msgr2=0x7f8e0810cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:59:26.582 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:26.579+0000 7f8e06ffd640 1 --2- 172.21.15.156:0/3145442940 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8e08066520 0x7f8e0810cf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:59:26.582 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:26.579+0000 7f8e06ffd640 1 -- 172.21.15.156:0/3145442940 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8dfc005040 con 0x7f8e080ff4d0 2024-07-16T09:59:26.582 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:26.579+0000 7f8e06ffd640 1 --2- 172.21.15.156:0/3145442940 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8e080ff4d0 0x7f8e0810d4d0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f8dfc009f80 tx=0x7f8dfc009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:59:26.582 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:26.579+0000 7f8deffff640 1 -- 172.21.15.156:0/3145442940 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8dfc038070 con 0x7f8e080ff4d0 2024-07-16T09:59:26.583 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:26.579+0000 7f8deffff640 1 -- 172.21.15.156:0/3145442940 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8dfc032970 con 0x7f8e080ff4d0 2024-07-16T09:59:26.583 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:26.579+0000 7f8e0dcee640 1 -- 172.21.15.156:0/3145442940 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e0810e520 con 0x7f8e080ff4d0 2024-07-16T09:59:26.583 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:26.579+0000 7f8deffff640 1 -- 172.21.15.156:0/3145442940 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8dfc033860 con 0x7f8e080ff4d0 2024-07-16T09:59:26.583 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:26.579+0000 7f8e0dcee640 1 -- 172.21.15.156:0/3145442940 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e0810ea60 con 0x7f8e080ff4d0 2024-07-16T09:59:26.584 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:26.580+0000 7f8e0dcee640 1 -- 172.21.15.156:0/3145442940 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e080fa6e0 con 0x7f8e080ff4d0 2024-07-16T09:59:26.584 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:26.580+0000 7f8deffff640 1 -- 172.21.15.156:0/3145442940 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 38) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f8dfc030020 con 0x7f8e080ff4d0 2024-07-16T09:59:26.584 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:26.581+0000 7f8deffff640 1 --2- 172.21.15.156:0/3145442940 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f8df006e0b0 0x7f8df0070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:59:26.584 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:26.581+0000 7f8deffff640 1 -- 172.21.15.156:0/3145442940 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f8dfc0adab0 con 0x7f8e080ff4d0 2024-07-16T09:59:26.584 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:26.581+0000 7f8e077fe640 1 --2- 172.21.15.156:0/3145442940 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f8df006e0b0 0x7f8df0070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:59:26.584 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:26.581+0000 7f8e077fe640 1 --2- 172.21.15.156:0/3145442940 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f8df006e0b0 0x7f8df0070560 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f8df8009fd0 tx=0x7f8df8009e50 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:59:26.586 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:26.585+0000 7f8deffff640 1 -- 172.21.15.156:0/3145442940 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187716 (secure 0 0 0) 0x7f8dfc076e30 con 0x7f8e080ff4d0 2024-07-16T09:59:26.840 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:26.839+0000 7f8e0dcee640 1 -- 172.21.15.156:0/3145442940 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f8e0804b530 con 0x7f8e080ff4d0 2024-07-16T09:59:26.841 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:59:26 smithi156 ceph-mon[93066]: Upgrade: Updating osd.1 2024-07-16T09:59:26.841 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:59:26 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T09:59:26.841 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:59:26 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-07-16T09:59:26.841 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:59:26 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-16T09:59:26.841 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:59:26 smithi156 ceph-mon[93066]: Deploying daemon osd.1 on smithi156 2024-07-16T09:59:26.842 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:59:26 smithi156 ceph-mon[93066]: from='client.34220 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-16T09:59:26.842 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:59:26 smithi156 ceph-mon[93066]: from='client.44153 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-16T09:59:26.842 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:59:26 smithi156 ceph-mon[93066]: pgmap v57: 65 pgs: 65 active+clean; 253 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 1.8 KiB/s rd, 511 B/s wr, 3 op/s 2024-07-16T09:59:26.842 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:26.840+0000 7f8deffff640 1 -- 172.21.15.156:0/3145442940 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+838 (secure 0 0 0) 0x7f8dfc07a4f0 con 0x7f8e080ff4d0 2024-07-16T09:59:26.842 INFO:teuthology.orchestra.run.smithi156.stdout:{ 2024-07-16T09:59:26.842 INFO:teuthology.orchestra.run.smithi156.stdout: "mon": { 2024-07-16T09:59:26.843 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph version 19.0.0-3495-ga6ca461b (a6ca461be7cc1aff3658c2c6df4cff975a24449a) squid (rc)": 2 2024-07-16T09:59:26.843 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-07-16T09:59:26.843 INFO:teuthology.orchestra.run.smithi156.stdout: "mgr": { 2024-07-16T09:59:26.843 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph version 19.0.0-3495-ga6ca461b (a6ca461be7cc1aff3658c2c6df4cff975a24449a) squid (rc)": 2 2024-07-16T09:59:26.843 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-07-16T09:59:26.843 INFO:teuthology.orchestra.run.smithi156.stdout: "osd": { 2024-07-16T09:59:26.843 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph version 17.2.7-1104-ge6231b04 (e6231b04b52ebf3b9b31803282dd3074439e51ac) quincy (stable)": 5, 2024-07-16T09:59:26.843 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph version 19.0.0-3495-ga6ca461b (a6ca461be7cc1aff3658c2c6df4cff975a24449a) squid (rc)": 1 2024-07-16T09:59:26.843 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-07-16T09:59:26.843 INFO:teuthology.orchestra.run.smithi156.stdout: "mds": { 2024-07-16T09:59:26.843 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph version 17.2.7-1104-ge6231b04 (e6231b04b52ebf3b9b31803282dd3074439e51ac) quincy (stable)": 4 2024-07-16T09:59:26.843 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-07-16T09:59:26.843 INFO:teuthology.orchestra.run.smithi156.stdout: "overall": { 2024-07-16T09:59:26.844 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph version 17.2.7-1104-ge6231b04 (e6231b04b52ebf3b9b31803282dd3074439e51ac) quincy (stable)": 9, 2024-07-16T09:59:26.844 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph version 19.0.0-3495-ga6ca461b (a6ca461be7cc1aff3658c2c6df4cff975a24449a) squid (rc)": 5 2024-07-16T09:59:26.844 INFO:teuthology.orchestra.run.smithi156.stdout: } 2024-07-16T09:59:26.844 INFO:teuthology.orchestra.run.smithi156.stdout:} 2024-07-16T09:59:26.844 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:26.842+0000 7f8e0dcee640 1 -- 172.21.15.156:0/3145442940 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f8df006e0b0 msgr2=0x7f8df0070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:59:26.845 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:26.842+0000 7f8e0dcee640 1 --2- 172.21.15.156:0/3145442940 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f8df006e0b0 0x7f8df0070560 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f8df8009fd0 tx=0x7f8df8009e50 comp rx=0 tx=0).stop 2024-07-16T09:59:26.845 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:26.842+0000 7f8e0dcee640 1 -- 172.21.15.156:0/3145442940 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8e080ff4d0 msgr2=0x7f8e0810d4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:59:26.845 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:26.842+0000 7f8e0dcee640 1 --2- 172.21.15.156:0/3145442940 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8e080ff4d0 0x7f8e0810d4d0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f8dfc009f80 tx=0x7f8dfc009da0 comp rx=0 tx=0).stop 2024-07-16T09:59:26.845 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:26.842+0000 7f8e0dcee640 1 -- 172.21.15.156:0/3145442940 shutdown_connections 2024-07-16T09:59:26.845 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:26.842+0000 7f8e0dcee640 1 --2- 172.21.15.156:0/3145442940 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f8df006e0b0 0x7f8df0070560 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:59:26.845 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:26.842+0000 7f8e0dcee640 1 --2- 172.21.15.156:0/3145442940 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8e080ff4d0 0x7f8e0810d4d0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:59:26.845 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:26.842+0000 7f8e0dcee640 1 --2- 172.21.15.156:0/3145442940 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8e08066520 0x7f8e0810cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:59:26.845 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:26.842+0000 7f8e0dcee640 1 -- 172.21.15.156:0/3145442940 >> 172.21.15.156:0/3145442940 conn(0x7f8e080f6a20 msgr2=0x7f8e080f6e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:59:26.845 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:26.842+0000 7f8e0dcee640 1 -- 172.21.15.156:0/3145442940 shutdown_connections 2024-07-16T09:59:26.845 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:26.842+0000 7f8e0dcee640 1 -- 172.21.15.156:0/3145442940 wait complete. 2024-07-16T09:59:26.925 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:26.922+0000 7fc2e514e640 1 -- 172.21.15.156:0/2808992030 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc2e00fc530 msgr2=0x7fc2e00fc9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:59:26.925 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:26.922+0000 7fc2e514e640 1 --2- 172.21.15.156:0/2808992030 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc2e00fc530 0x7fc2e00fc9a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fc2c80048d0 tx=0x7fc2c8009da0 comp rx=0 tx=0).stop 2024-07-16T09:59:26.925 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:26.922+0000 7fc2e514e640 1 -- 172.21.15.156:0/2808992030 shutdown_connections 2024-07-16T09:59:26.925 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:26.922+0000 7fc2e514e640 1 --2- 172.21.15.156:0/2808992030 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc2e00fc530 0x7fc2e00fc9a0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:59:26.925 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:26.922+0000 7fc2e514e640 1 --2- 172.21.15.156:0/2808992030 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc2e00fb260 0x7fc2e00fb670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:59:26.925 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:26.922+0000 7fc2e514e640 1 -- 172.21.15.156:0/2808992030 >> 172.21.15.156:0/2808992030 conn(0x7fc2e00f6a20 msgr2=0x7fc2e00f8e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:59:26.926 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:26.922+0000 7fc2e514e640 1 -- 172.21.15.156:0/2808992030 shutdown_connections 2024-07-16T09:59:26.926 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:26.922+0000 7fc2e514e640 1 -- 172.21.15.156:0/2808992030 wait complete. 2024-07-16T09:59:26.926 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:26.923+0000 7fc2e514e640 1 Processor -- start 2024-07-16T09:59:26.926 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:26.923+0000 7fc2e514e640 1 -- start start 2024-07-16T09:59:26.926 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:26.923+0000 7fc2e514e640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc2e00fb260 0x7fc2e010bea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:59:26.926 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:26.923+0000 7fc2e514e640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc2e00fc530 0x7fc2e010c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:59:26.926 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:26.923+0000 7fc2e514e640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2e010ca30 con 0x7fc2e00fc530 2024-07-16T09:59:26.926 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:26.923+0000 7fc2e514e640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2e010d250 con 0x7fc2e00fb260 2024-07-16T09:59:26.926 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:26.923+0000 7fc2d7fff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc2e00fc530 0x7fc2e010c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:59:26.926 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:26.923+0000 7fc2d7fff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc2e00fc530 0x7fc2e010c3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:42152/0 (socket says 172.21.15.156:42152) 2024-07-16T09:59:26.926 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:26.923+0000 7fc2d7fff640 1 -- 172.21.15.156:0/3301435296 learned_addr learned my addr 172.21.15.156:0/3301435296 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T09:59:26.927 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:26.923+0000 7fc2ded9d640 1 --2- 172.21.15.156:0/3301435296 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc2e00fb260 0x7fc2e010bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:59:26.927 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:26.923+0000 7fc2d7fff640 1 -- 172.21.15.156:0/3301435296 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc2e00fb260 msgr2=0x7fc2e010bea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:59:26.927 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:26.923+0000 7fc2d7fff640 1 --2- 172.21.15.156:0/3301435296 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc2e00fb260 0x7fc2e010bea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:59:26.927 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:26.923+0000 7fc2d7fff640 1 -- 172.21.15.156:0/3301435296 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2c8005040 con 0x7fc2e00fc530 2024-07-16T09:59:26.927 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:26.923+0000 7fc2ded9d640 1 --2- 172.21.15.156:0/3301435296 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc2e00fb260 0x7fc2e010bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-16T09:59:26.927 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:26.924+0000 7fc2d7fff640 1 --2- 172.21.15.156:0/3301435296 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc2e00fc530 0x7fc2e010c3e0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fc2c80048a0 tx=0x7fc2c8009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:59:26.927 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:26.924+0000 7fc2d77fe640 1 -- 172.21.15.156:0/3301435296 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc2c8038070 con 0x7fc2e00fc530 2024-07-16T09:59:26.927 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:26.924+0000 7fc2e514e640 1 -- 172.21.15.156:0/3301435296 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2e010d480 con 0x7fc2e00fc530 2024-07-16T09:59:26.927 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:26.924+0000 7fc2d77fe640 1 -- 172.21.15.156:0/3301435296 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc2c80092b0 con 0x7fc2e00fc530 2024-07-16T09:59:26.927 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:26.924+0000 7fc2d77fe640 1 -- 172.21.15.156:0/3301435296 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc2c80337a0 con 0x7fc2e00fc530 2024-07-16T09:59:26.927 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:26.924+0000 7fc2e514e640 1 -- 172.21.15.156:0/3301435296 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2e010d9c0 con 0x7fc2e00fc530 2024-07-16T09:59:26.928 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:26.925+0000 7fc2e514e640 1 -- 172.21.15.156:0/3301435296 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2e00fa6c0 con 0x7fc2e00fc530 2024-07-16T09:59:26.928 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:26.925+0000 7fc2d77fe640 1 -- 172.21.15.156:0/3301435296 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 38) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fc2c8030020 con 0x7fc2e00fc530 2024-07-16T09:59:26.929 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:26.926+0000 7fc2d77fe640 1 --2- 172.21.15.156:0/3301435296 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7fc2cc06e0b0 0x7fc2cc070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:59:26.929 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:26.926+0000 7fc2d77fe640 1 -- 172.21.15.156:0/3301435296 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fc2c80adaa0 con 0x7fc2e00fc530 2024-07-16T09:59:26.929 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:26.926+0000 7fc2ded9d640 1 --2- 172.21.15.156:0/3301435296 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7fc2cc06e0b0 0x7fc2cc070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:59:26.929 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:26.926+0000 7fc2ded9d640 1 --2- 172.21.15.156:0/3301435296 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7fc2cc06e0b0 0x7fc2cc070560 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fc2d0009fd0 tx=0x7fc2d0009e50 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:59:26.931 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:26.930+0000 7fc2d77fe640 1 -- 172.21.15.156:0/3301435296 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187716 (secure 0 0 0) 0x7fc2c8076e20 con 0x7fc2e00fc530 2024-07-16T09:59:27.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:59:26 smithi176 ceph-mon[81689]: Upgrade: Updating osd.1 2024-07-16T09:59:27.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:59:26 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T09:59:27.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:59:26 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-07-16T09:59:27.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:59:26 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-16T09:59:27.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:59:26 smithi176 ceph-mon[81689]: Deploying daemon osd.1 on smithi156 2024-07-16T09:59:27.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:59:26 smithi176 ceph-mon[81689]: from='client.34220 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-16T09:59:27.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:59:26 smithi176 ceph-mon[81689]: from='client.44153 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-16T09:59:27.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:59:26 smithi176 ceph-mon[81689]: pgmap v57: 65 pgs: 65 active+clean; 253 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 1.8 KiB/s rd, 511 B/s wr, 3 op/s 2024-07-16T09:59:27.151 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:27.150+0000 7fc2e514e640 1 -- 172.21.15.156:0/3301435296 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fc2e004b530 con 0x7fc2e00fc530 2024-07-16T09:59:27.151 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:27.150+0000 7fc2d77fe640 1 -- 172.21.15.156:0/3301435296 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 20 v20) v1 ==== 76+0+1920 (secure 0 0 0) 0x7fc2c8041790 con 0x7fc2e00fc530 2024-07-16T09:59:27.152 INFO:teuthology.orchestra.run.smithi156.stdout:e20 2024-07-16T09:59:27.152 INFO:teuthology.orchestra.run.smithi156.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-07-16T09:59:27.152 INFO:teuthology.orchestra.run.smithi156.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-07-16T09:59:27.152 INFO:teuthology.orchestra.run.smithi156.stdout:legacy client fscid: 1 2024-07-16T09:59:27.152 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-07-16T09:59:27.153 INFO:teuthology.orchestra.run.smithi156.stdout:Filesystem 'cephfs' (1) 2024-07-16T09:59:27.153 INFO:teuthology.orchestra.run.smithi156.stdout:fs_name cephfs 2024-07-16T09:59:27.153 INFO:teuthology.orchestra.run.smithi156.stdout:epoch 20 2024-07-16T09:59:27.153 INFO:teuthology.orchestra.run.smithi156.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-07-16T09:59:27.153 INFO:teuthology.orchestra.run.smithi156.stdout:created 2024-07-16T09:52:55.214975+0000 2024-07-16T09:59:27.153 INFO:teuthology.orchestra.run.smithi156.stdout:modified 2024-07-16T09:58:56.574272+0000 2024-07-16T09:59:27.153 INFO:teuthology.orchestra.run.smithi156.stdout:tableserver 0 2024-07-16T09:59:27.153 INFO:teuthology.orchestra.run.smithi156.stdout:root 0 2024-07-16T09:59:27.153 INFO:teuthology.orchestra.run.smithi156.stdout:session_timeout 60 2024-07-16T09:59:27.153 INFO:teuthology.orchestra.run.smithi156.stdout:session_autoclose 300 2024-07-16T09:59:27.154 INFO:teuthology.orchestra.run.smithi156.stdout:max_file_size 1099511627776 2024-07-16T09:59:27.154 INFO:teuthology.orchestra.run.smithi156.stdout:max_xattr_size 65536 2024-07-16T09:59:27.154 INFO:teuthology.orchestra.run.smithi156.stdout:required_client_features {} 2024-07-16T09:59:27.154 INFO:teuthology.orchestra.run.smithi156.stdout:last_failure 0 2024-07-16T09:59:27.154 INFO:teuthology.orchestra.run.smithi156.stdout:last_failure_osd_epoch 0 2024-07-16T09:59:27.154 INFO:teuthology.orchestra.run.smithi156.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-07-16T09:59:27.154 INFO:teuthology.orchestra.run.smithi156.stdout:max_mds 2 2024-07-16T09:59:27.154 INFO:teuthology.orchestra.run.smithi156.stdout:in 0,1 2024-07-16T09:59:27.154 INFO:teuthology.orchestra.run.smithi156.stdout:up {0=14458,1=24261} 2024-07-16T09:59:27.154 INFO:teuthology.orchestra.run.smithi156.stdout:failed 2024-07-16T09:59:27.154 INFO:teuthology.orchestra.run.smithi156.stdout:damaged 2024-07-16T09:59:27.154 INFO:teuthology.orchestra.run.smithi156.stdout:stopped 2024-07-16T09:59:27.154 INFO:teuthology.orchestra.run.smithi156.stdout:data_pools [3] 2024-07-16T09:59:27.155 INFO:teuthology.orchestra.run.smithi156.stdout:metadata_pool 2 2024-07-16T09:59:27.155 INFO:teuthology.orchestra.run.smithi156.stdout:inline_data disabled 2024-07-16T09:59:27.155 INFO:teuthology.orchestra.run.smithi156.stdout:balancer 2024-07-16T09:59:27.155 INFO:teuthology.orchestra.run.smithi156.stdout:bal_rank_mask -1 2024-07-16T09:59:27.155 INFO:teuthology.orchestra.run.smithi156.stdout:standby_count_wanted 1 2024-07-16T09:59:27.155 INFO:teuthology.orchestra.run.smithi156.stdout:qdb_cluster leader: 14458 members: 24261,14458 2024-07-16T09:59:27.155 INFO:teuthology.orchestra.run.smithi156.stdout:[mds.cephfs.smithi156.rdibys{0:14458} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.156:6826/1080176013,v1:172.21.15.156:6827/1080176013] compat {c=[1],r=[1],i=[7ff]}] 2024-07-16T09:59:27.155 INFO:teuthology.orchestra.run.smithi156.stdout:[mds.cephfs.smithi156.evwwaz{0:14468} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.156:6828/560329502,v1:172.21.15.156:6829/560329502] compat {c=[1],r=[1],i=[7ff]}] 2024-07-16T09:59:27.155 INFO:teuthology.orchestra.run.smithi156.stdout:[mds.cephfs.smithi176.dvvbud{1:24261} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.176:6824/1577019287,v1:172.21.15.176:6825/1577019287] compat {c=[1],r=[1],i=[7ff]}] 2024-07-16T09:59:27.155 INFO:teuthology.orchestra.run.smithi156.stdout:[mds.cephfs.smithi176.wahbsi{1:24273} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.176:6826/2196775412,v1:172.21.15.176:6827/2196775412] compat {c=[1],r=[1],i=[7ff]}] 2024-07-16T09:59:27.155 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-07-16T09:59:27.155 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-07-16T09:59:27.156 INFO:teuthology.orchestra.run.smithi156.stderr:dumped fsmap epoch 20 2024-07-16T09:59:27.156 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:27.152+0000 7fc2e514e640 1 -- 172.21.15.156:0/3301435296 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7fc2cc06e0b0 msgr2=0x7fc2cc070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:59:27.156 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:27.152+0000 7fc2e514e640 1 --2- 172.21.15.156:0/3301435296 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7fc2cc06e0b0 0x7fc2cc070560 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fc2d0009fd0 tx=0x7fc2d0009e50 comp rx=0 tx=0).stop 2024-07-16T09:59:27.156 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:27.152+0000 7fc2e514e640 1 -- 172.21.15.156:0/3301435296 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc2e00fc530 msgr2=0x7fc2e010c3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:59:27.156 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:27.152+0000 7fc2e514e640 1 --2- 172.21.15.156:0/3301435296 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc2e00fc530 0x7fc2e010c3e0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fc2c80048a0 tx=0x7fc2c8009da0 comp rx=0 tx=0).stop 2024-07-16T09:59:27.156 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:27.152+0000 7fc2e514e640 1 -- 172.21.15.156:0/3301435296 shutdown_connections 2024-07-16T09:59:27.157 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:27.152+0000 7fc2e514e640 1 --2- 172.21.15.156:0/3301435296 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7fc2cc06e0b0 0x7fc2cc070560 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:59:27.157 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:27.152+0000 7fc2e514e640 1 --2- 172.21.15.156:0/3301435296 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc2e00fc530 0x7fc2e010c3e0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:59:27.157 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:27.152+0000 7fc2e514e640 1 --2- 172.21.15.156:0/3301435296 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc2e00fb260 0x7fc2e010bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:59:27.157 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:27.152+0000 7fc2e514e640 1 -- 172.21.15.156:0/3301435296 >> 172.21.15.156:0/3301435296 conn(0x7fc2e00f6a20 msgr2=0x7fc2e00f8170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:59:27.157 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:27.152+0000 7fc2e514e640 1 -- 172.21.15.156:0/3301435296 shutdown_connections 2024-07-16T09:59:27.157 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:27.152+0000 7fc2e514e640 1 -- 172.21.15.156:0/3301435296 wait complete. 2024-07-16T09:59:27.238 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:27.236+0000 7fa502135640 1 -- 172.21.15.156:0/1785566917 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa4fc0fd600 msgr2=0x7fa4fc0fda70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:59:27.238 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:27.236+0000 7fa502135640 1 --2- 172.21.15.156:0/1785566917 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa4fc0fd600 0x7fa4fc0fda70 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fa4f00048d0 tx=0x7fa4f0009da0 comp rx=0 tx=0).stop 2024-07-16T09:59:27.239 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:27.236+0000 7fa502135640 1 -- 172.21.15.156:0/1785566917 shutdown_connections 2024-07-16T09:59:27.239 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:27.236+0000 7fa502135640 1 --2- 172.21.15.156:0/1785566917 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa4fc0fd600 0x7fa4fc0fda70 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:59:27.239 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:27.236+0000 7fa502135640 1 --2- 172.21.15.156:0/1785566917 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa4fc0fc330 0x7fa4fc0fc740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:59:27.239 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:27.236+0000 7fa502135640 1 -- 172.21.15.156:0/1785566917 >> 172.21.15.156:0/1785566917 conn(0x7fa4fc0f7af0 msgr2=0x7fa4fc0f9f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:59:27.239 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:27.236+0000 7fa502135640 1 -- 172.21.15.156:0/1785566917 shutdown_connections 2024-07-16T09:59:27.239 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:27.236+0000 7fa502135640 1 -- 172.21.15.156:0/1785566917 wait complete. 2024-07-16T09:59:27.239 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:27.236+0000 7fa502135640 1 Processor -- start 2024-07-16T09:59:27.239 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:27.237+0000 7fa502135640 1 -- start start 2024-07-16T09:59:27.239 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:27.237+0000 7fa502135640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa4fc0fc330 0x7fa4fc10cf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:59:27.239 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:27.237+0000 7fa502135640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa4fc0fd600 0x7fa4fc10d490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:59:27.239 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:27.237+0000 7fa502135640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4fc10dae0 con 0x7fa4fc0fc330 2024-07-16T09:59:27.239 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:27.237+0000 7fa502135640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4fc10e300 con 0x7fa4fc0fd600 2024-07-16T09:59:27.240 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:27.237+0000 7fa4fb7fe640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa4fc0fc330 0x7fa4fc10cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:59:27.240 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:27.237+0000 7fa4fb7fe640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa4fc0fc330 0x7fa4fc10cf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:42156/0 (socket says 172.21.15.156:42156) 2024-07-16T09:59:27.240 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:27.237+0000 7fa4fb7fe640 1 -- 172.21.15.156:0/2810881309 learned_addr learned my addr 172.21.15.156:0/2810881309 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T09:59:27.240 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:27.237+0000 7fa4faffd640 1 --2- 172.21.15.156:0/2810881309 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa4fc0fd600 0x7fa4fc10d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:59:27.240 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:27.237+0000 7fa4fb7fe640 1 -- 172.21.15.156:0/2810881309 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa4fc0fd600 msgr2=0x7fa4fc10d490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:59:27.240 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:27.237+0000 7fa4fb7fe640 1 --2- 172.21.15.156:0/2810881309 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa4fc0fd600 0x7fa4fc10d490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:59:27.240 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:27.237+0000 7fa4fb7fe640 1 -- 172.21.15.156:0/2810881309 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4f0005040 con 0x7fa4fc0fc330 2024-07-16T09:59:27.242 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:27.238+0000 7fa4fb7fe640 1 --2- 172.21.15.156:0/2810881309 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa4fc0fc330 0x7fa4fc10cf50 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fa4ec00eb70 tx=0x7fa4ec00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:59:27.242 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:27.238+0000 7fa4e3fff640 1 -- 172.21.15.156:0/2810881309 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa4ec007c30 con 0x7fa4fc0fc330 2024-07-16T09:59:27.242 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:27.238+0000 7fa4e3fff640 1 -- 172.21.15.156:0/2810881309 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa4ec00f040 con 0x7fa4fc0fc330 2024-07-16T09:59:27.242 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:27.238+0000 7fa502135640 1 -- 172.21.15.156:0/2810881309 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4fc10e530 con 0x7fa4fc0fc330 2024-07-16T09:59:27.242 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:27.238+0000 7fa4e3fff640 1 -- 172.21.15.156:0/2810881309 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa4ec00cad0 con 0x7fa4fc0fc330 2024-07-16T09:59:27.242 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:27.238+0000 7fa502135640 1 -- 172.21.15.156:0/2810881309 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4fc10ea50 con 0x7fa4fc0fc330 2024-07-16T09:59:27.242 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:27.239+0000 7fa502135640 1 -- 172.21.15.156:0/2810881309 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4fc1056b0 con 0x7fa4fc0fc330 2024-07-16T09:59:27.242 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:27.239+0000 7fa4e3fff640 1 -- 172.21.15.156:0/2810881309 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 38) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fa4ec00cc50 con 0x7fa4fc0fc330 2024-07-16T09:59:27.242 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:27.240+0000 7fa4e3fff640 1 --2- 172.21.15.156:0/2810881309 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7fa4e406e0b0 0x7fa4e4070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:59:27.243 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:27.240+0000 7fa4e3fff640 1 -- 172.21.15.156:0/2810881309 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fa4ec08d7b0 con 0x7fa4fc0fc330 2024-07-16T09:59:27.243 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:27.240+0000 7fa4faffd640 1 --2- 172.21.15.156:0/2810881309 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7fa4e406e0b0 0x7fa4e4070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:59:27.243 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:27.241+0000 7fa4faffd640 1 --2- 172.21.15.156:0/2810881309 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7fa4e406e0b0 0x7fa4e4070560 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fa4f0009f80 tx=0x7fa4f0035040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:59:27.246 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:27.245+0000 7fa4e3fff640 1 -- 172.21.15.156:0/2810881309 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187716 (secure 0 0 0) 0x7fa4ec056be0 con 0x7fa4fc0fc330 2024-07-16T09:59:27.480 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:27.478+0000 7fa502135640 1 -- 172.21.15.156:0/2810881309 --> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa4fc10edd0 con 0x7fa4e406e0b0 2024-07-16T09:59:27.481 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:27.480+0000 7fa4e3fff640 1 -- 172.21.15.156:0/2810881309 <== mgr.34104 v2:172.21.15.156:6800/3367765292 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7fa4fc10edd0 con 0x7fa4e406e0b0 2024-07-16T09:59:27.481 INFO:teuthology.orchestra.run.smithi156.stdout:{ 2024-07-16T09:59:27.481 INFO:teuthology.orchestra.run.smithi156.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:c15ec3aa8e6c0a9c12ee24ff79f3e7dd4d01b3d1a702dfeb7d7fff9b0811a393", 2024-07-16T09:59:27.481 INFO:teuthology.orchestra.run.smithi156.stdout: "in_progress": true, 2024-07-16T09:59:27.481 INFO:teuthology.orchestra.run.smithi156.stdout: "which": "Upgrading all daemon types on all hosts", 2024-07-16T09:59:27.481 INFO:teuthology.orchestra.run.smithi156.stdout: "services_complete": [ 2024-07-16T09:59:27.482 INFO:teuthology.orchestra.run.smithi156.stdout: "mon", 2024-07-16T09:59:27.482 INFO:teuthology.orchestra.run.smithi156.stdout: "mgr", 2024-07-16T09:59:27.482 INFO:teuthology.orchestra.run.smithi156.stdout: "crash" 2024-07-16T09:59:27.482 INFO:teuthology.orchestra.run.smithi156.stdout: ], 2024-07-16T09:59:27.482 INFO:teuthology.orchestra.run.smithi156.stdout: "progress": "7/21 daemons upgraded", 2024-07-16T09:59:27.482 INFO:teuthology.orchestra.run.smithi156.stdout: "message": "Currently upgrading osd daemons", 2024-07-16T09:59:27.482 INFO:teuthology.orchestra.run.smithi156.stdout: "is_paused": false 2024-07-16T09:59:27.482 INFO:teuthology.orchestra.run.smithi156.stdout:} 2024-07-16T09:59:27.483 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:27.482+0000 7fa502135640 1 -- 172.21.15.156:0/2810881309 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7fa4e406e0b0 msgr2=0x7fa4e4070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:59:27.483 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:27.482+0000 7fa502135640 1 --2- 172.21.15.156:0/2810881309 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7fa4e406e0b0 0x7fa4e4070560 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fa4f0009f80 tx=0x7fa4f0035040 comp rx=0 tx=0).stop 2024-07-16T09:59:27.483 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:27.482+0000 7fa502135640 1 -- 172.21.15.156:0/2810881309 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa4fc0fc330 msgr2=0x7fa4fc10cf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:59:27.483 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:27.482+0000 7fa502135640 1 --2- 172.21.15.156:0/2810881309 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa4fc0fc330 0x7fa4fc10cf50 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fa4ec00eb70 tx=0x7fa4ec00b040 comp rx=0 tx=0).stop 2024-07-16T09:59:27.484 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:27.482+0000 7fa502135640 1 -- 172.21.15.156:0/2810881309 shutdown_connections 2024-07-16T09:59:27.484 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:27.482+0000 7fa502135640 1 --2- 172.21.15.156:0/2810881309 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7fa4e406e0b0 0x7fa4e4070560 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:59:27.484 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:27.482+0000 7fa502135640 1 --2- 172.21.15.156:0/2810881309 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa4fc0fd600 0x7fa4fc10d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:59:27.484 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:27.482+0000 7fa502135640 1 --2- 172.21.15.156:0/2810881309 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa4fc0fc330 0x7fa4fc10cf50 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:59:27.484 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:27.482+0000 7fa502135640 1 -- 172.21.15.156:0/2810881309 >> 172.21.15.156:0/2810881309 conn(0x7fa4fc0f7af0 msgr2=0x7fa4fc0f91c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:59:27.484 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:27.482+0000 7fa502135640 1 -- 172.21.15.156:0/2810881309 shutdown_connections 2024-07-16T09:59:27.484 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:27.482+0000 7fa502135640 1 -- 172.21.15.156:0/2810881309 wait complete. 2024-07-16T09:59:27.565 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:27.562+0000 7fc3a80a7640 1 -- 172.21.15.156:0/2520726244 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc3a00fb260 msgr2=0x7fc3a00fb670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:59:27.565 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:27.562+0000 7fc3a80a7640 1 --2- 172.21.15.156:0/2520726244 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc3a00fb260 0x7fc3a00fb670 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fc39c0048d0 tx=0x7fc39c009da0 comp rx=0 tx=0).stop 2024-07-16T09:59:27.565 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:27.562+0000 7fc3a80a7640 1 -- 172.21.15.156:0/2520726244 shutdown_connections 2024-07-16T09:59:27.565 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:27.562+0000 7fc3a80a7640 1 --2- 172.21.15.156:0/2520726244 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc3a00fc530 0x7fc3a00fc9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:59:27.565 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:27.562+0000 7fc3a80a7640 1 --2- 172.21.15.156:0/2520726244 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc3a00fb260 0x7fc3a00fb670 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:59:27.565 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:27.562+0000 7fc3a80a7640 1 -- 172.21.15.156:0/2520726244 >> 172.21.15.156:0/2520726244 conn(0x7fc3a00f6a20 msgr2=0x7fc3a00f8e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:59:27.565 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:27.562+0000 7fc3a80a7640 1 -- 172.21.15.156:0/2520726244 shutdown_connections 2024-07-16T09:59:27.566 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:27.562+0000 7fc3a80a7640 1 -- 172.21.15.156:0/2520726244 wait complete. 2024-07-16T09:59:27.566 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:27.563+0000 7fc3a80a7640 1 Processor -- start 2024-07-16T09:59:27.566 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:27.563+0000 7fc3a80a7640 1 -- start start 2024-07-16T09:59:27.566 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:27.563+0000 7fc3a80a7640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc3a00fb260 0x7fc3a00ff380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:59:27.566 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:27.563+0000 7fc3a80a7640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc3a00fc530 0x7fc3a00fd870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:59:27.566 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:27.563+0000 7fc3a80a7640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3a00fde90 con 0x7fc3a00fc530 2024-07-16T09:59:27.566 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:27.563+0000 7fc3a80a7640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3a00fdfd0 con 0x7fc3a00fb260 2024-07-16T09:59:27.566 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:27.563+0000 7fc3a5642640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc3a00fc530 0x7fc3a00fd870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:59:27.566 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:27.563+0000 7fc3a5642640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc3a00fc530 0x7fc3a00fd870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:42170/0 (socket says 172.21.15.156:42170) 2024-07-16T09:59:27.566 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:27.563+0000 7fc3a5642640 1 -- 172.21.15.156:0/444620857 learned_addr learned my addr 172.21.15.156:0/444620857 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T09:59:27.566 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:27.563+0000 7fc3a5e43640 1 --2- 172.21.15.156:0/444620857 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc3a00fb260 0x7fc3a00ff380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:59:27.567 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:27.563+0000 7fc3a5642640 1 -- 172.21.15.156:0/444620857 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc3a00fb260 msgr2=0x7fc3a00ff380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:59:27.567 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:27.563+0000 7fc3a5642640 1 --2- 172.21.15.156:0/444620857 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc3a00fb260 0x7fc3a00ff380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:59:27.567 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:27.563+0000 7fc3a5642640 1 -- 172.21.15.156:0/444620857 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc39c005040 con 0x7fc3a00fc530 2024-07-16T09:59:27.567 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:27.563+0000 7fc3a5642640 1 --2- 172.21.15.156:0/444620857 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc3a00fc530 0x7fc3a00fd870 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fc39000b790 tx=0x7fc3900091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:59:27.567 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:27.563+0000 7fc3967fc640 1 -- 172.21.15.156:0/444620857 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc39000c700 con 0x7fc3a00fc530 2024-07-16T09:59:27.567 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:27.564+0000 7fc3967fc640 1 -- 172.21.15.156:0/444620857 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc39000cd30 con 0x7fc3a00fc530 2024-07-16T09:59:27.567 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:27.564+0000 7fc3a80a7640 1 -- 172.21.15.156:0/444620857 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3a00fe2b0 con 0x7fc3a00fc530 2024-07-16T09:59:27.567 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:27.564+0000 7fc3967fc640 1 -- 172.21.15.156:0/444620857 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc390013370 con 0x7fc3a00fc530 2024-07-16T09:59:27.567 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:27.564+0000 7fc3a80a7640 1 -- 172.21.15.156:0/444620857 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3a01150d0 con 0x7fc3a00fc530 2024-07-16T09:59:27.568 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:27.565+0000 7fc3a80a7640 1 -- 172.21.15.156:0/444620857 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3a00fa6c0 con 0x7fc3a00fc530 2024-07-16T09:59:27.568 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:27.565+0000 7fc3967fc640 1 -- 172.21.15.156:0/444620857 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 38) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fc390012070 con 0x7fc3a00fc530 2024-07-16T09:59:27.568 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:27.566+0000 7fc3967fc640 1 --2- 172.21.15.156:0/444620857 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7fc38c06e0b0 0x7fc38c070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:59:27.568 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:27.566+0000 7fc3967fc640 1 -- 172.21.15.156:0/444620857 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fc39008d690 con 0x7fc3a00fc530 2024-07-16T09:59:27.569 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:27.566+0000 7fc3a5e43640 1 --2- 172.21.15.156:0/444620857 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7fc38c06e0b0 0x7fc38c070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:59:27.569 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:27.566+0000 7fc3a5e43640 1 --2- 172.21.15.156:0/444620857 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7fc38c06e0b0 0x7fc38c070560 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fc39c0047f0 tx=0x7fc39c009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:59:27.571 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:27.570+0000 7fc3967fc640 1 -- 172.21.15.156:0/444620857 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187716 (secure 0 0 0) 0x7fc390056a90 con 0x7fc3a00fc530 2024-07-16T09:59:27.827 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:27.825+0000 7fc3a80a7640 1 -- 172.21.15.156:0/444620857 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fc3a0103f30 con 0x7fc3a00fc530 2024-07-16T09:59:27.827 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:27.826+0000 7fc3967fc640 1 -- 172.21.15.156:0/444620857 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7fc39005a150 con 0x7fc3a00fc530 2024-07-16T09:59:27.827 INFO:teuthology.orchestra.run.smithi156.stdout:HEALTH_OK 2024-07-16T09:59:27.829 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:27.828+0000 7fc3a80a7640 1 -- 172.21.15.156:0/444620857 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7fc38c06e0b0 msgr2=0x7fc38c070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:59:27.829 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:27.828+0000 7fc3a80a7640 1 --2- 172.21.15.156:0/444620857 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7fc38c06e0b0 0x7fc38c070560 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fc39c0047f0 tx=0x7fc39c009da0 comp rx=0 tx=0).stop 2024-07-16T09:59:27.829 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:27.828+0000 7fc3a80a7640 1 -- 172.21.15.156:0/444620857 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc3a00fc530 msgr2=0x7fc3a00fd870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:59:27.829 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:27.828+0000 7fc3a80a7640 1 --2- 172.21.15.156:0/444620857 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc3a00fc530 0x7fc3a00fd870 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fc39000b790 tx=0x7fc3900091d0 comp rx=0 tx=0).stop 2024-07-16T09:59:27.830 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:27.828+0000 7fc3a80a7640 1 -- 172.21.15.156:0/444620857 shutdown_connections 2024-07-16T09:59:27.830 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:27.828+0000 7fc3a80a7640 1 --2- 172.21.15.156:0/444620857 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7fc38c06e0b0 0x7fc38c070560 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:59:27.830 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:27.828+0000 7fc3a80a7640 1 --2- 172.21.15.156:0/444620857 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc3a00fc530 0x7fc3a00fd870 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:59:27.830 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:27.828+0000 7fc3a80a7640 1 --2- 172.21.15.156:0/444620857 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc3a00fb260 0x7fc3a00ff380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:59:27.830 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:27.828+0000 7fc3a80a7640 1 -- 172.21.15.156:0/444620857 >> 172.21.15.156:0/444620857 conn(0x7fc3a00f6a20 msgr2=0x7fc3a0065ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:59:27.830 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:27.828+0000 7fc3a80a7640 1 -- 172.21.15.156:0/444620857 shutdown_connections 2024-07-16T09:59:27.830 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:27.828+0000 7fc3a80a7640 1 -- 172.21.15.156:0/444620857 wait complete. 2024-07-16T09:59:28.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:59:27 smithi176 ceph-mon[81689]: from='client.34228 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-07-16T09:59:28.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:59:27 smithi176 ceph-mon[81689]: from='client.? 172.21.15.156:0/3145442940' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T09:59:28.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:59:27 smithi176 ceph-mon[81689]: from='client.? 172.21.15.156:0/3301435296' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-07-16T09:59:28.174 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:59:27 smithi156 ceph-mon[93066]: from='client.34228 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-07-16T09:59:28.175 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:59:27 smithi156 ceph-mon[93066]: from='client.? 172.21.15.156:0/3145442940' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T09:59:28.175 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:59:27 smithi156 ceph-mon[93066]: from='client.? 172.21.15.156:0/3301435296' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-07-16T09:59:29.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:59:28 smithi176 ceph-mon[81689]: from='client.34240 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-16T09:59:29.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:59:28 smithi176 ceph-mon[81689]: from='client.? 172.21.15.156:0/444620857' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-07-16T09:59:29.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:59:28 smithi176 ceph-mon[81689]: pgmap v58: 65 pgs: 65 active+clean; 253 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 1.8 KiB/s rd, 511 B/s wr, 3 op/s 2024-07-16T09:59:29.113 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:59:28 smithi156 ceph-mon[93066]: from='client.34240 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-16T09:59:29.113 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:59:28 smithi156 ceph-mon[93066]: from='client.? 172.21.15.156:0/444620857' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-07-16T09:59:29.113 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:59:28 smithi156 ceph-mon[93066]: pgmap v58: 65 pgs: 65 active+clean; 253 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 1.8 KiB/s rd, 511 B/s wr, 3 op/s 2024-07-16T09:59:29.113 INFO:journalctl@ceph.osd.1.smithi156.stdout:Jul 16 09:59:28 smithi156 systemd[1]: Stopping Ceph osd.1 for 26c91c26-4358-11ef-bcad-c7b262605968... 2024-07-16T09:59:29.424 INFO:journalctl@ceph.osd.1.smithi156.stdout:Jul 16 09:59:29 smithi156 ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-1[54640]: 2024-07-16T09:59:29.110+0000 7f0f29aa1640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-07-16T09:59:29.424 INFO:journalctl@ceph.osd.1.smithi156.stdout:Jul 16 09:59:29 smithi156 ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-1[54640]: 2024-07-16T09:59:29.110+0000 7f0f29aa1640 -1 osd.1 54 *** Got signal Terminated *** 2024-07-16T09:59:29.425 INFO:journalctl@ceph.osd.1.smithi156.stdout:Jul 16 09:59:29 smithi156 ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-1[54640]: 2024-07-16T09:59:29.110+0000 7f0f29aa1640 -1 osd.1 54 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-07-16T09:59:30.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:59:29 smithi176 ceph-mon[81689]: osd.1 marked itself down and dead 2024-07-16T09:59:30.174 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:59:29 smithi156 ceph-mon[93066]: osd.1 marked itself down and dead 2024-07-16T09:59:30.463 INFO:journalctl@ceph.osd.1.smithi156.stdout:Jul 16 09:59:30 smithi156 podman[106490]: 2024-07-16 09:59:30.193781035 +0000 UTC m=+1.181988269 container died 149a0bda2f61c403b89b249b65762e0eb1426e72e8abef4cf14babacf2d1b508 (image=quay.ceph.io/ceph-ci/ceph@sha256:7b86ef6bffe0efc5f1aefc9fef8c854a67a511c50993acee3ac684f23bde83df, name=ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-1, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240709, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, RELEASE=quincy-e6231b0, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-07-16T09:59:30.826 INFO:journalctl@ceph.osd.1.smithi156.stdout:Jul 16 09:59:30 smithi156 podman[106490]: 2024-07-16 09:59:30.552863399 +0000 UTC m=+1.541070631 container cleanup 149a0bda2f61c403b89b249b65762e0eb1426e72e8abef4cf14babacf2d1b508 (image=quay.ceph.io/ceph-ci/ceph@sha256:7b86ef6bffe0efc5f1aefc9fef8c854a67a511c50993acee3ac684f23bde83df, name=ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-1, RELEASE=quincy-e6231b0, io.buildah.version=1.36.0, org.label-schema.build-date=20240709, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-07-16T09:59:30.826 INFO:journalctl@ceph.osd.1.smithi156.stdout:Jul 16 09:59:30 smithi156 bash[106490]: ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-1 2024-07-16T09:59:31.093 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:59:30 smithi176 ceph-mon[81689]: Health check failed: 1 osds down (OSD_DOWN) 2024-07-16T09:59:31.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:59:30 smithi176 ceph-mon[81689]: osdmap e55: 6 total, 5 up, 6 in 2024-07-16T09:59:31.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:59:30 smithi176 ceph-mon[81689]: pgmap v60: 65 pgs: 12 stale+active+clean, 53 active+clean; 253 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s 2024-07-16T09:59:31.174 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:59:30 smithi156 ceph-mon[93066]: Health check failed: 1 osds down (OSD_DOWN) 2024-07-16T09:59:31.175 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:59:30 smithi156 ceph-mon[93066]: osdmap e55: 6 total, 5 up, 6 in 2024-07-16T09:59:31.175 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:59:30 smithi156 ceph-mon[93066]: pgmap v60: 65 pgs: 12 stale+active+clean, 53 active+clean; 253 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s 2024-07-16T09:59:31.175 INFO:journalctl@ceph.osd.1.smithi156.stdout:Jul 16 09:59:30 smithi156 podman[106508]: 2024-07-16 09:59:30.854228595 +0000 UTC m=+0.660763668 container remove 149a0bda2f61c403b89b249b65762e0eb1426e72e8abef4cf14babacf2d1b508 (image=quay.ceph.io/ceph-ci/ceph@sha256:7b86ef6bffe0efc5f1aefc9fef8c854a67a511c50993acee3ac684f23bde83df, name=ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-1, RELEASE=quincy-e6231b0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240709, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.36.0, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-07-16T09:59:31.526 INFO:journalctl@ceph.osd.1.smithi156.stdout:Jul 16 09:59:31 smithi156 podman[106595]: 2024-07-16 09:59:31.12158536 +0000 UTC m=+0.017684615 image pull 16f7feb043422ce37a2471403d4bc52f550d49cef4b677482112fcc0db3f67ab quay.ceph.io/ceph-ci/ceph@sha256:c15ec3aa8e6c0a9c12ee24ff79f3e7dd4d01b3d1a702dfeb7d7fff9b0811a393 2024-07-16T09:59:31.526 INFO:journalctl@ceph.osd.1.smithi156.stdout:Jul 16 09:59:31 smithi156 podman[106595]: 2024-07-16 09:59:31.255494915 +0000 UTC m=+0.151594154 container create c792a5df56a4fafa0603ba789569ce84ac9a45abe3fdff46bd7daae6507a2ba1 (image=quay.ceph.io/ceph-ci/ceph@sha256:c15ec3aa8e6c0a9c12ee24ff79f3e7dd4d01b3d1a702dfeb7d7fff9b0811a393, name=ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-1-deactivate, GIT_CLEAN=True, io.buildah.version=1.36.0, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240709, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-adk4-testing-2024-07-15-1832-squid-a6ca461, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-07-16T09:59:31.835 INFO:journalctl@ceph.osd.1.smithi156.stdout:Jul 16 09:59:31 smithi156 podman[106595]: 2024-07-16 09:59:31.525536284 +0000 UTC m=+0.421635556 container init c792a5df56a4fafa0603ba789569ce84ac9a45abe3fdff46bd7daae6507a2ba1 (image=quay.ceph.io/ceph-ci/ceph@sha256:c15ec3aa8e6c0a9c12ee24ff79f3e7dd4d01b3d1a702dfeb7d7fff9b0811a393, name=ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-1-deactivate, org.label-schema.build-date=20240709, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.36.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, RELEASE=wip-adk4-testing-2024-07-15-1832-squid-a6ca461, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True) 2024-07-16T09:59:31.835 INFO:journalctl@ceph.osd.1.smithi156.stdout:Jul 16 09:59:31 smithi156 podman[106595]: 2024-07-16 09:59:31.529232398 +0000 UTC m=+0.425331648 container start c792a5df56a4fafa0603ba789569ce84ac9a45abe3fdff46bd7daae6507a2ba1 (image=quay.ceph.io/ceph-ci/ceph@sha256:c15ec3aa8e6c0a9c12ee24ff79f3e7dd4d01b3d1a702dfeb7d7fff9b0811a393, name=ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-1-deactivate, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.36.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240709, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-adk4-testing-2024-07-15-1832-squid-a6ca461, org.label-schema.vendor=CentOS) 2024-07-16T09:59:31.835 INFO:journalctl@ceph.osd.1.smithi156.stdout:Jul 16 09:59:31 smithi156 podman[106595]: 2024-07-16 09:59:31.660351649 +0000 UTC m=+0.556450903 container attach c792a5df56a4fafa0603ba789569ce84ac9a45abe3fdff46bd7daae6507a2ba1 (image=quay.ceph.io/ceph-ci/ceph@sha256:c15ec3aa8e6c0a9c12ee24ff79f3e7dd4d01b3d1a702dfeb7d7fff9b0811a393, name=ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-1-deactivate, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240709, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-adk4-testing-2024-07-15-1832-squid-a6ca461, GIT_BRANCH=HEAD, io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-07-16T09:59:31.835 INFO:journalctl@ceph.osd.1.smithi156.stdout:Jul 16 09:59:31 smithi156 podman[106618]: 2024-07-16 09:59:31.757208673 +0000 UTC m=+0.019039231 container died c792a5df56a4fafa0603ba789569ce84ac9a45abe3fdff46bd7daae6507a2ba1 (image=quay.ceph.io/ceph-ci/ceph@sha256:c15ec3aa8e6c0a9c12ee24ff79f3e7dd4d01b3d1a702dfeb7d7fff9b0811a393, name=ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-1-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-adk4-testing-2024-07-15-1832-squid-a6ca461, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240709, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.36.0, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-07-16T09:59:32.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:59:31 smithi176 ceph-mon[81689]: osdmap e56: 6 total, 5 up, 6 in 2024-07-16T09:59:32.174 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:59:31 smithi156 ceph-mon[93066]: osdmap e56: 6 total, 5 up, 6 in 2024-07-16T09:59:32.675 INFO:journalctl@ceph.osd.1.smithi156.stdout:Jul 16 09:59:32 smithi156 podman[106618]: 2024-07-16 09:59:32.31412109 +0000 UTC m=+0.575951649 container remove c792a5df56a4fafa0603ba789569ce84ac9a45abe3fdff46bd7daae6507a2ba1 (image=quay.ceph.io/ceph-ci/ceph@sha256:c15ec3aa8e6c0a9c12ee24ff79f3e7dd4d01b3d1a702dfeb7d7fff9b0811a393, name=ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-1-deactivate, org.label-schema.build-date=20240709, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-adk4-testing-2024-07-15-1832-squid-a6ca461, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.36.0, ceph=True) 2024-07-16T09:59:32.675 INFO:journalctl@ceph.osd.1.smithi156.stdout:Jul 16 09:59:32 smithi156 systemd[1]: ceph-26c91c26-4358-11ef-bcad-c7b262605968@osd.1.service: Deactivated successfully. 2024-07-16T09:59:32.675 INFO:journalctl@ceph.osd.1.smithi156.stdout:Jul 16 09:59:32 smithi156 systemd[1]: Stopped Ceph osd.1 for 26c91c26-4358-11ef-bcad-c7b262605968. 2024-07-16T09:59:32.675 INFO:journalctl@ceph.osd.1.smithi156.stdout:Jul 16 09:59:32 smithi156 systemd[1]: ceph-26c91c26-4358-11ef-bcad-c7b262605968@osd.1.service: Consumed 1min 7.720s CPU time. 2024-07-16T09:59:32.675 INFO:journalctl@ceph.osd.1.smithi156.stdout:Jul 16 09:59:32 smithi156 systemd[1]: Starting Ceph osd.1 for 26c91c26-4358-11ef-bcad-c7b262605968... 2024-07-16T09:59:33.019 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:59:32 smithi156 ceph-mon[93066]: pgmap v62: 65 pgs: 3 active+undersized, 9 stale+active+clean, 2 active+undersized+degraded, 51 active+clean; 253 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 6/318 objects degraded (1.887%) 2024-07-16T09:59:33.019 INFO:journalctl@ceph.osd.1.smithi156.stdout:Jul 16 09:59:32 smithi156 podman[106718]: 2024-07-16 09:59:32.669512736 +0000 UTC m=+0.017257397 image pull 16f7feb043422ce37a2471403d4bc52f550d49cef4b677482112fcc0db3f67ab quay.ceph.io/ceph-ci/ceph@sha256:c15ec3aa8e6c0a9c12ee24ff79f3e7dd4d01b3d1a702dfeb7d7fff9b0811a393 2024-07-16T09:59:33.019 INFO:journalctl@ceph.osd.1.smithi156.stdout:Jul 16 09:59:32 smithi156 podman[106718]: 2024-07-16 09:59:32.77363431 +0000 UTC m=+0.121379010 container create 1b8c2945855ed9bee5ee3240f0f27387df2faa2bac8a8797c1f366a46f598c58 (image=quay.ceph.io/ceph-ci/ceph@sha256:c15ec3aa8e6c0a9c12ee24ff79f3e7dd4d01b3d1a702dfeb7d7fff9b0811a393, name=ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-1-activate, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.36.0, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-adk4-testing-2024-07-15-1832-squid-a6ca461, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240709, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-07-16T09:59:33.019 INFO:journalctl@ceph.osd.1.smithi156.stdout:Jul 16 09:59:33 smithi156 podman[106718]: 2024-07-16 09:59:33.018408104 +0000 UTC m=+0.366152768 container init 1b8c2945855ed9bee5ee3240f0f27387df2faa2bac8a8797c1f366a46f598c58 (image=quay.ceph.io/ceph-ci/ceph@sha256:c15ec3aa8e6c0a9c12ee24ff79f3e7dd4d01b3d1a702dfeb7d7fff9b0811a393, name=ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-1-activate, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240709, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-adk4-testing-2024-07-15-1832-squid-a6ca461, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.36.0) 2024-07-16T09:59:33.093 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:59:32 smithi176 ceph-mon[81689]: pgmap v62: 65 pgs: 3 active+undersized, 9 stale+active+clean, 2 active+undersized+degraded, 51 active+clean; 253 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 6/318 objects degraded (1.887%) 2024-07-16T09:59:33.359 INFO:journalctl@ceph.osd.1.smithi156.stdout:Jul 16 09:59:33 smithi156 podman[106718]: 2024-07-16 09:59:33.022157267 +0000 UTC m=+0.369901924 container start 1b8c2945855ed9bee5ee3240f0f27387df2faa2bac8a8797c1f366a46f598c58 (image=quay.ceph.io/ceph-ci/ceph@sha256:c15ec3aa8e6c0a9c12ee24ff79f3e7dd4d01b3d1a702dfeb7d7fff9b0811a393, name=ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-1-activate, RELEASE=wip-adk4-testing-2024-07-15-1832-squid-a6ca461, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.36.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240709, ceph=True) 2024-07-16T09:59:33.359 INFO:journalctl@ceph.osd.1.smithi156.stdout:Jul 16 09:59:33 smithi156 podman[106718]: 2024-07-16 09:59:33.109473614 +0000 UTC m=+0.457218282 container attach 1b8c2945855ed9bee5ee3240f0f27387df2faa2bac8a8797c1f366a46f598c58 (image=quay.ceph.io/ceph-ci/ceph@sha256:c15ec3aa8e6c0a9c12ee24ff79f3e7dd4d01b3d1a702dfeb7d7fff9b0811a393, name=ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-1-activate, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240709, RELEASE=wip-adk4-testing-2024-07-15-1832-squid-a6ca461) 2024-07-16T09:59:33.359 INFO:journalctl@ceph.osd.1.smithi156.stdout:Jul 16 09:59:33 smithi156 ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-1-activate[106734]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-07-16T09:59:33.359 INFO:journalctl@ceph.osd.1.smithi156.stdout:Jul 16 09:59:33 smithi156 bash[106718]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-07-16T09:59:33.360 INFO:journalctl@ceph.osd.1.smithi156.stdout:Jul 16 09:59:33 smithi156 ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-1-activate[106734]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-07-16T09:59:33.360 INFO:journalctl@ceph.osd.1.smithi156.stdout:Jul 16 09:59:33 smithi156 bash[106718]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-07-16T09:59:33.360 INFO:journalctl@ceph.osd.1.smithi156.stdout:Jul 16 09:59:33 smithi156 ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-1-activate[106734]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-07-16T09:59:33.360 INFO:journalctl@ceph.osd.1.smithi156.stdout:Jul 16 09:59:33 smithi156 bash[106718]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-07-16T09:59:33.675 INFO:journalctl@ceph.osd.1.smithi156.stdout:Jul 16 09:59:33 smithi156 ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-1-activate[106734]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-1 --no-mon-config --dev /dev/mapper/vg_nvme-lv_3 2024-07-16T09:59:33.675 INFO:journalctl@ceph.osd.1.smithi156.stdout:Jul 16 09:59:33 smithi156 bash[106718]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-1 --no-mon-config --dev /dev/mapper/vg_nvme-lv_3 2024-07-16T09:59:33.675 INFO:journalctl@ceph.osd.1.smithi156.stdout:Jul 16 09:59:33 smithi156 ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-1-activate[106734]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_3 2024-07-16T09:59:33.675 INFO:journalctl@ceph.osd.1.smithi156.stdout:Jul 16 09:59:33 smithi156 bash[106718]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_3 2024-07-16T09:59:33.675 INFO:journalctl@ceph.osd.1.smithi156.stdout:Jul 16 09:59:33 smithi156 ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-1-activate[106734]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-07-16T09:59:33.676 INFO:journalctl@ceph.osd.1.smithi156.stdout:Jul 16 09:59:33 smithi156 bash[106718]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-07-16T09:59:33.676 INFO:journalctl@ceph.osd.1.smithi156.stdout:Jul 16 09:59:33 smithi156 ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-1-activate[106734]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-1/block 2024-07-16T09:59:33.676 INFO:journalctl@ceph.osd.1.smithi156.stdout:Jul 16 09:59:33 smithi156 bash[106718]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-1/block 2024-07-16T09:59:33.676 INFO:journalctl@ceph.osd.1.smithi156.stdout:Jul 16 09:59:33 smithi156 ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-1-activate[106734]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-07-16T09:59:33.676 INFO:journalctl@ceph.osd.1.smithi156.stdout:Jul 16 09:59:33 smithi156 bash[106718]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-07-16T09:59:33.676 INFO:journalctl@ceph.osd.1.smithi156.stdout:Jul 16 09:59:33 smithi156 ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-1-activate[106734]: --> ceph-volume raw activate successful for osd ID: 1 2024-07-16T09:59:33.677 INFO:journalctl@ceph.osd.1.smithi156.stdout:Jul 16 09:59:33 smithi156 bash[106718]: --> ceph-volume raw activate successful for osd ID: 1 2024-07-16T09:59:33.677 INFO:journalctl@ceph.osd.1.smithi156.stdout:Jul 16 09:59:33 smithi156 podman[106718]: 2024-07-16 09:59:33.407574378 +0000 UTC m=+0.755319046 container died 1b8c2945855ed9bee5ee3240f0f27387df2faa2bac8a8797c1f366a46f598c58 (image=quay.ceph.io/ceph-ci/ceph@sha256:c15ec3aa8e6c0a9c12ee24ff79f3e7dd4d01b3d1a702dfeb7d7fff9b0811a393, name=ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-1-activate, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240709, RELEASE=wip-adk4-testing-2024-07-15-1832-squid-a6ca461, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.36.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True) 2024-07-16T09:59:34.039 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:59:33 smithi156 ceph-mon[93066]: Health check failed: Degraded data redundancy: 6/318 objects degraded (1.887%), 2 pgs degraded (PG_DEGRADED) 2024-07-16T09:59:34.039 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:59:33 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T09:59:34.039 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:59:33 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-16T09:59:34.093 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:59:33 smithi176 ceph-mon[81689]: Health check failed: Degraded data redundancy: 6/318 objects degraded (1.887%), 2 pgs degraded (PG_DEGRADED) 2024-07-16T09:59:34.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:59:33 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T09:59:34.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:59:33 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-16T09:59:34.355 INFO:journalctl@ceph.osd.1.smithi156.stdout:Jul 16 09:59:34 smithi156 podman[106870]: 2024-07-16 09:59:34.038737164 +0000 UTC m=+0.625414355 container remove 1b8c2945855ed9bee5ee3240f0f27387df2faa2bac8a8797c1f366a46f598c58 (image=quay.ceph.io/ceph-ci/ceph@sha256:c15ec3aa8e6c0a9c12ee24ff79f3e7dd4d01b3d1a702dfeb7d7fff9b0811a393, name=ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-1-activate, io.buildah.version=1.36.0, RELEASE=wip-adk4-testing-2024-07-15-1832-squid-a6ca461, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240709, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True) 2024-07-16T09:59:34.356 INFO:journalctl@ceph.osd.1.smithi156.stdout:Jul 16 09:59:34 smithi156 podman[106929]: 2024-07-16 09:59:34.23981106 +0000 UTC m=+0.100183077 container create e6fae9653b460a3931256034fdf7318a813be739a41d87ec058129ace21cd2e4 (image=quay.ceph.io/ceph-ci/ceph@sha256:c15ec3aa8e6c0a9c12ee24ff79f3e7dd4d01b3d1a702dfeb7d7fff9b0811a393, name=ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-1, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-adk4-testing-2024-07-15-1832-squid-a6ca461, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240709) 2024-07-16T09:59:34.356 INFO:journalctl@ceph.osd.1.smithi156.stdout:Jul 16 09:59:34 smithi156 podman[106929]: 2024-07-16 09:59:34.156830208 +0000 UTC m=+0.017202242 image pull 16f7feb043422ce37a2471403d4bc52f550d49cef4b677482112fcc0db3f67ab quay.ceph.io/ceph-ci/ceph@sha256:c15ec3aa8e6c0a9c12ee24ff79f3e7dd4d01b3d1a702dfeb7d7fff9b0811a393 2024-07-16T09:59:34.845 INFO:journalctl@ceph.osd.1.smithi156.stdout:Jul 16 09:59:34 smithi156 podman[106929]: 2024-07-16 09:59:34.475518591 +0000 UTC m=+0.335890604 container init e6fae9653b460a3931256034fdf7318a813be739a41d87ec058129ace21cd2e4 (image=quay.ceph.io/ceph-ci/ceph@sha256:c15ec3aa8e6c0a9c12ee24ff79f3e7dd4d01b3d1a702dfeb7d7fff9b0811a393, name=ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-1, GIT_BRANCH=HEAD, org.label-schema.build-date=20240709, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.36.0, GIT_CLEAN=True, RELEASE=wip-adk4-testing-2024-07-15-1832-squid-a6ca461, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-07-16T09:59:34.845 INFO:journalctl@ceph.osd.1.smithi156.stdout:Jul 16 09:59:34 smithi156 podman[106929]: 2024-07-16 09:59:34.478928961 +0000 UTC m=+0.339300988 container start e6fae9653b460a3931256034fdf7318a813be739a41d87ec058129ace21cd2e4 (image=quay.ceph.io/ceph-ci/ceph@sha256:c15ec3aa8e6c0a9c12ee24ff79f3e7dd4d01b3d1a702dfeb7d7fff9b0811a393, name=ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-1, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, RELEASE=wip-adk4-testing-2024-07-15-1832-squid-a6ca461, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240709, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-07-16T09:59:34.845 INFO:journalctl@ceph.osd.1.smithi156.stdout:Jul 16 09:59:34 smithi156 ceph-osd[106947]: -- 172.21.15.156:0/127109112 <== mon.0 v2:172.21.15.156:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x564b0f8501e0 con 0x564b0f7e8000 2024-07-16T09:59:34.845 INFO:journalctl@ceph.osd.1.smithi156.stdout:Jul 16 09:59:34 smithi156 bash[106929]: e6fae9653b460a3931256034fdf7318a813be739a41d87ec058129ace21cd2e4 2024-07-16T09:59:34.845 INFO:journalctl@ceph.osd.1.smithi156.stdout:Jul 16 09:59:34 smithi156 systemd[1]: Started Ceph osd.1 for 26c91c26-4358-11ef-bcad-c7b262605968. 2024-07-16T09:59:35.174 INFO:journalctl@ceph.osd.1.smithi156.stdout:Jul 16 09:59:35 smithi156 ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-1[106943]: 2024-07-16T09:59:35.016+0000 7f64051b3640 -1 Falling back to public interface 2024-07-16T09:59:35.175 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:59:34 smithi156 ceph-mon[93066]: pgmap v63: 65 pgs: 18 active+undersized, 16 active+undersized+degraded, 31 active+clean; 253 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 51/318 objects degraded (16.038%) 2024-07-16T09:59:35.175 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:59:34 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T09:59:35.175 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:59:34 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T09:59:35.175 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:59:34 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-16T09:59:35.343 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:59:34 smithi176 ceph-mon[81689]: pgmap v63: 65 pgs: 18 active+undersized, 16 active+undersized+degraded, 31 active+clean; 253 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 51/318 objects degraded (16.038%) 2024-07-16T09:59:35.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:59:34 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T09:59:35.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:59:34 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T09:59:35.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:59:34 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-16T09:59:36.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:59:36 smithi156 ceph-mon[93066]: pgmap v64: 65 pgs: 18 active+undersized, 16 active+undersized+degraded, 31 active+clean; 253 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 51/318 objects degraded (16.038%) 2024-07-16T09:59:36.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:59:36 smithi176 ceph-mon[81689]: pgmap v64: 65 pgs: 18 active+undersized, 16 active+undersized+degraded, 31 active+clean; 253 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 51/318 objects degraded (16.038%) 2024-07-16T09:59:38.470 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:59:38 smithi156 ceph-mon[93066]: pgmap v65: 65 pgs: 18 active+undersized, 16 active+undersized+degraded, 31 active+clean; 253 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 51/318 objects degraded (16.038%) 2024-07-16T09:59:38.843 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:59:38 smithi176 ceph-mon[81689]: pgmap v65: 65 pgs: 18 active+undersized, 16 active+undersized+degraded, 31 active+clean; 253 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 51/318 objects degraded (16.038%) 2024-07-16T09:59:40.425 INFO:journalctl@ceph.osd.1.smithi156.stdout:Jul 16 09:59:40 smithi156 ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-1[106943]: 2024-07-16T09:59:40.062+0000 7f64051b3640 -1 osd.1 54 log_to_monitors true 2024-07-16T09:59:40.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:59:40 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T09:59:40.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:59:40 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T09:59:40.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:59:40 smithi176 ceph-mon[81689]: from='osd.1 [v2:172.21.15.156:6810/2180713647,v1:172.21.15.156:6811/2180713647]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-07-16T09:59:40.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:59:40 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T09:59:40.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:59:40 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T09:59:40.924 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:59:40 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T09:59:40.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:59:40 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T09:59:40.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:59:40 smithi156 ceph-mon[93066]: from='osd.1 [v2:172.21.15.156:6810/2180713647,v1:172.21.15.156:6811/2180713647]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-07-16T09:59:40.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:59:40 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T09:59:40.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:59:40 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T09:59:41.424 INFO:journalctl@ceph.osd.1.smithi156.stdout:Jul 16 09:59:41 smithi156 ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-1[106943]: 2024-07-16T09:59:41.244+0000 7f63f8f45640 -1 osd.1 54 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-07-16T09:59:41.843 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:59:41 smithi176 ceph-mon[81689]: pgmap v66: 65 pgs: 18 active+undersized, 16 active+undersized+degraded, 31 active+clean; 253 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 51/318 objects degraded (16.038%) 2024-07-16T09:59:41.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:59:41 smithi176 ceph-mon[81689]: from='osd.1 [v2:172.21.15.156:6810/2180713647,v1:172.21.15.156:6811/2180713647]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-07-16T09:59:41.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:59:41 smithi176 ceph-mon[81689]: osdmap e57: 6 total, 5 up, 6 in 2024-07-16T09:59:41.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:59:41 smithi176 ceph-mon[81689]: from='osd.1 [v2:172.21.15.156:6810/2180713647,v1:172.21.15.156:6811/2180713647]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2024-07-16T09:59:41.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:59:41 smithi176 ceph-mon[81689]: from='osd.1 [v2:172.21.15.156:6810/2180713647,v1:172.21.15.156:6811/2180713647]' entity='osd.1' 2024-07-16T09:59:41.904 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:59:41 smithi156 ceph-mon[93066]: pgmap v66: 65 pgs: 18 active+undersized, 16 active+undersized+degraded, 31 active+clean; 253 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 51/318 objects degraded (16.038%) 2024-07-16T09:59:41.904 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:59:41 smithi156 ceph-mon[93066]: from='osd.1 [v2:172.21.15.156:6810/2180713647,v1:172.21.15.156:6811/2180713647]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-07-16T09:59:41.904 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:59:41 smithi156 ceph-mon[93066]: osdmap e57: 6 total, 5 up, 6 in 2024-07-16T09:59:41.904 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:59:41 smithi156 ceph-mon[93066]: from='osd.1 [v2:172.21.15.156:6810/2180713647,v1:172.21.15.156:6811/2180713647]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2024-07-16T09:59:41.905 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:59:41 smithi156 ceph-mon[93066]: from='osd.1 [v2:172.21.15.156:6810/2180713647,v1:172.21.15.156:6811/2180713647]' entity='osd.1' 2024-07-16T09:59:42.843 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:59:42 smithi176 ceph-mon[81689]: Health check update: Degraded data redundancy: 51/318 objects degraded (16.038%), 16 pgs degraded (PG_DEGRADED) 2024-07-16T09:59:42.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:59:42 smithi176 ceph-mon[81689]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-07-16T09:59:42.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:59:42 smithi176 ceph-mon[81689]: osd.1 [v2:172.21.15.156:6810/2180713647,v1:172.21.15.156:6811/2180713647] boot 2024-07-16T09:59:42.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:59:42 smithi176 ceph-mon[81689]: osdmap e58: 6 total, 6 up, 6 in 2024-07-16T09:59:42.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:59:42 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-07-16T09:59:42.924 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:59:42 smithi156 ceph-mon[93066]: Health check update: Degraded data redundancy: 51/318 objects degraded (16.038%), 16 pgs degraded (PG_DEGRADED) 2024-07-16T09:59:42.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:59:42 smithi156 ceph-mon[93066]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-07-16T09:59:42.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:59:42 smithi156 ceph-mon[93066]: osd.1 [v2:172.21.15.156:6810/2180713647,v1:172.21.15.156:6811/2180713647] boot 2024-07-16T09:59:42.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:59:42 smithi156 ceph-mon[93066]: osdmap e58: 6 total, 6 up, 6 in 2024-07-16T09:59:42.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:59:42 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-07-16T09:59:44.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:59:44 smithi176 ceph-mon[81689]: pgmap v69: 65 pgs: 18 active+undersized, 16 active+undersized+degraded, 31 active+clean; 253 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s; 51/318 objects degraded (16.038%) 2024-07-16T09:59:44.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:59:44 smithi176 ceph-mon[81689]: osdmap e59: 6 total, 6 up, 6 in 2024-07-16T09:59:44.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:59:44 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T09:59:44.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:59:44 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T09:59:44.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:59:44 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-16T09:59:44.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:59:44 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-16T09:59:44.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:59:44 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T09:59:44.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:59:44 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-16T09:59:44.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:59:44 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T09:59:44.595 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:59:44 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T09:59:44.595 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:59:44 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T09:59:44.595 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:59:44 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-07-16T09:59:44.614 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:59:44 smithi156 ceph-mon[93066]: pgmap v69: 65 pgs: 18 active+undersized, 16 active+undersized+degraded, 31 active+clean; 253 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s; 51/318 objects degraded (16.038%) 2024-07-16T09:59:44.614 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:59:44 smithi156 ceph-mon[93066]: osdmap e59: 6 total, 6 up, 6 in 2024-07-16T09:59:44.614 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:59:44 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T09:59:44.615 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:59:44 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T09:59:44.615 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:59:44 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-16T09:59:44.615 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:59:44 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-16T09:59:44.615 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:59:44 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T09:59:44.615 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:59:44 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-16T09:59:44.615 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:59:44 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T09:59:44.615 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:59:44 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T09:59:44.615 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:59:44 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T09:59:44.615 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:59:44 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-07-16T09:59:45.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:59:45 smithi176 ceph-mon[81689]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-07-16T09:59:45.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:59:45 smithi176 ceph-mon[81689]: Upgrade: unsafe to stop osd(s) at this time (12 PGs are or would become offline) 2024-07-16T09:59:45.625 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:59:45 smithi156 ceph-mon[93066]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-07-16T09:59:45.625 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:59:45 smithi156 ceph-mon[93066]: Upgrade: unsafe to stop osd(s) at this time (12 PGs are or would become offline) 2024-07-16T09:59:46.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:59:46 smithi176 ceph-mon[81689]: pgmap v71: 65 pgs: 7 active+undersized, 7 active+undersized+degraded, 51 active+clean; 253 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 3.0 KiB/s rd, 5 op/s; 22/318 objects degraded (6.918%) 2024-07-16T09:59:46.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:59:46 smithi156 ceph-mon[93066]: pgmap v71: 65 pgs: 7 active+undersized, 7 active+undersized+degraded, 51 active+clean; 253 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 3.0 KiB/s rd, 5 op/s; 22/318 objects degraded (6.918%) 2024-07-16T09:59:47.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:59:47 smithi176 ceph-mon[81689]: Health check update: Degraded data redundancy: 22/318 objects degraded (6.918%), 7 pgs degraded (PG_DEGRADED) 2024-07-16T09:59:47.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:59:47 smithi156 ceph-mon[93066]: Health check update: Degraded data redundancy: 22/318 objects degraded (6.918%), 7 pgs degraded (PG_DEGRADED) 2024-07-16T09:59:48.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:59:48 smithi176 ceph-mon[81689]: pgmap v72: 65 pgs: 2 active+undersized, 63 active+clean; 253 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-07-16T09:59:48.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:59:48 smithi176 ceph-mon[81689]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 22/318 objects degraded (6.918%), 7 pgs degraded) 2024-07-16T09:59:48.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:59:48 smithi176 ceph-mon[81689]: Cluster is now healthy 2024-07-16T09:59:48.659 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:59:48 smithi156 ceph-mon[93066]: pgmap v72: 65 pgs: 2 active+undersized, 63 active+clean; 253 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-07-16T09:59:48.660 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:59:48 smithi156 ceph-mon[93066]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 22/318 objects degraded (6.918%), 7 pgs degraded) 2024-07-16T09:59:48.660 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:59:48 smithi156 ceph-mon[93066]: Cluster is now healthy 2024-07-16T09:59:49.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:59:49 smithi156 ceph-mon[93066]: pgmap v73: 65 pgs: 65 active+clean; 253 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 1.0 KiB/s rd, 0 op/s 2024-07-16T09:59:49.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:59:49 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T09:59:49.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:59:49 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-16T09:59:49.843 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:59:49 smithi176 ceph-mon[81689]: pgmap v73: 65 pgs: 65 active+clean; 253 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 1.0 KiB/s rd, 0 op/s 2024-07-16T09:59:49.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:59:49 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T09:59:49.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:59:49 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-16T09:59:51.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:59:51 smithi156 ceph-mon[93066]: pgmap v74: 65 pgs: 65 active+clean; 253 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 2.2 KiB/s rd, 2 op/s 2024-07-16T09:59:51.843 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:59:51 smithi176 ceph-mon[81689]: pgmap v74: 65 pgs: 65 active+clean; 253 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 2.2 KiB/s rd, 2 op/s 2024-07-16T09:59:53.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:59:53 smithi156 ceph-mon[93066]: pgmap v75: 65 pgs: 65 active+clean; 253 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 1.8 KiB/s rd, 2 op/s 2024-07-16T09:59:53.843 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:59:53 smithi176 ceph-mon[81689]: pgmap v75: 65 pgs: 65 active+clean; 253 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 1.8 KiB/s rd, 2 op/s 2024-07-16T09:59:54.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:59:54 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T09:59:54.843 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:59:54 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T09:59:55.673 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:59:55 smithi156 ceph-mon[93066]: pgmap v76: 65 pgs: 65 active+clean; 253 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 3 op/s 2024-07-16T09:59:55.843 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:59:55 smithi176 ceph-mon[81689]: pgmap v76: 65 pgs: 65 active+clean; 253 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 3 op/s 2024-07-16T09:59:57.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:59:57 smithi156 ceph-mon[93066]: pgmap v77: 65 pgs: 65 active+clean; 253 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 2 op/s 2024-07-16T09:59:57.843 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:59:57 smithi176 ceph-mon[81689]: pgmap v77: 65 pgs: 65 active+clean; 253 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 2 op/s 2024-07-16T09:59:57.924 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:57.920+0000 7fa3deb48640 1 -- 172.21.15.156:0/188466900 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa3d80fd630 msgr2=0x7fa3d80fdaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:59:57.924 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:57.920+0000 7fa3deb48640 1 --2- 172.21.15.156:0/188466900 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa3d80fd630 0x7fa3d80fdaa0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fa3c00048d0 tx=0x7fa3c0009da0 comp rx=0 tx=0).stop 2024-07-16T09:59:57.924 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:57.920+0000 7fa3deb48640 1 -- 172.21.15.156:0/188466900 shutdown_connections 2024-07-16T09:59:57.924 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:57.920+0000 7fa3deb48640 1 --2- 172.21.15.156:0/188466900 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa3d80fd630 0x7fa3d80fdaa0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:59:57.924 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:57.920+0000 7fa3deb48640 1 --2- 172.21.15.156:0/188466900 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa3d80fc360 0x7fa3d80fc770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:59:57.924 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:57.920+0000 7fa3deb48640 1 -- 172.21.15.156:0/188466900 >> 172.21.15.156:0/188466900 conn(0x7fa3d80f7b40 msgr2=0x7fa3d80f9f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:59:57.924 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:57.920+0000 7fa3deb48640 1 -- 172.21.15.156:0/188466900 shutdown_connections 2024-07-16T09:59:57.924 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:57.920+0000 7fa3deb48640 1 -- 172.21.15.156:0/188466900 wait complete. 2024-07-16T09:59:57.924 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:57.921+0000 7fa3deb48640 1 Processor -- start 2024-07-16T09:59:57.924 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:57.921+0000 7fa3deb48640 1 -- start start 2024-07-16T09:59:57.925 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:57.921+0000 7fa3deb48640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa3d80fc360 0x7fa3d810cf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:59:57.925 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:57.921+0000 7fa3deb48640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa3d80fd630 0x7fa3d810d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:59:57.925 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:57.921+0000 7fa3deb48640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3d810db10 con 0x7fa3d80fd630 2024-07-16T09:59:57.925 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:57.921+0000 7fa3deb48640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3d810e330 con 0x7fa3d80fc360 2024-07-16T09:59:57.925 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:57.921+0000 7fa3d7fff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa3d80fd630 0x7fa3d810d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:59:57.925 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:57.921+0000 7fa3d7fff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa3d80fd630 0x7fa3d810d4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:34416/0 (socket says 172.21.15.156:34416) 2024-07-16T09:59:57.925 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:57.921+0000 7fa3d7fff640 1 -- 172.21.15.156:0/4170399909 learned_addr learned my addr 172.21.15.156:0/4170399909 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T09:59:57.925 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:57.921+0000 7fa3dc8e4640 1 --2- 172.21.15.156:0/4170399909 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa3d80fc360 0x7fa3d810cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:59:57.925 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:57.922+0000 7fa3d7fff640 1 -- 172.21.15.156:0/4170399909 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa3d80fc360 msgr2=0x7fa3d810cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:59:57.925 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:57.922+0000 7fa3d7fff640 1 --2- 172.21.15.156:0/4170399909 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa3d80fc360 0x7fa3d810cf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:59:57.925 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:57.922+0000 7fa3d7fff640 1 -- 172.21.15.156:0/4170399909 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3c0005040 con 0x7fa3d80fd630 2024-07-16T09:59:57.926 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:57.922+0000 7fa3d7fff640 1 --2- 172.21.15.156:0/4170399909 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa3d80fd630 0x7fa3d810d4c0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fa3c00048a0 tx=0x7fa3c0009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:59:57.926 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:57.922+0000 7fa3d57fa640 1 -- 172.21.15.156:0/4170399909 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa3c0038070 con 0x7fa3d80fd630 2024-07-16T09:59:57.926 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:57.922+0000 7fa3deb48640 1 -- 172.21.15.156:0/4170399909 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3d810e560 con 0x7fa3d80fd630 2024-07-16T09:59:57.926 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:57.922+0000 7fa3d57fa640 1 -- 172.21.15.156:0/4170399909 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa3c00092b0 con 0x7fa3d80fd630 2024-07-16T09:59:57.926 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:57.922+0000 7fa3d57fa640 1 -- 172.21.15.156:0/4170399909 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa3c0033780 con 0x7fa3d80fd630 2024-07-16T09:59:57.926 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:57.922+0000 7fa3deb48640 1 -- 172.21.15.156:0/4170399909 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3d810eaa0 con 0x7fa3d80fd630 2024-07-16T09:59:57.927 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:57.923+0000 7fa3deb48640 1 -- 172.21.15.156:0/4170399909 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3d80fb7c0 con 0x7fa3d80fd630 2024-07-16T09:59:57.927 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:57.924+0000 7fa3d57fa640 1 -- 172.21.15.156:0/4170399909 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 38) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fa3c0030020 con 0x7fa3d80fd630 2024-07-16T09:59:57.927 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:57.924+0000 7fa3d57fa640 1 --2- 172.21.15.156:0/4170399909 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7fa3c406e0b0 0x7fa3c4070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:59:57.927 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:57.924+0000 7fa3d57fa640 1 -- 172.21.15.156:0/4170399909 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fa3c00ad990 con 0x7fa3d80fd630 2024-07-16T09:59:57.927 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:57.924+0000 7fa3dc8e4640 1 --2- 172.21.15.156:0/4170399909 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7fa3c406e0b0 0x7fa3c4070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:59:57.927 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:57.925+0000 7fa3dc8e4640 1 --2- 172.21.15.156:0/4170399909 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7fa3c406e0b0 0x7fa3c4070560 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fa3c8004a70 tx=0x7fa3c800b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:59:57.930 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:57.929+0000 7fa3d57fa640 1 -- 172.21.15.156:0/4170399909 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187716 (secure 0 0 0) 0x7fa3c0076d80 con 0x7fa3d80fd630 2024-07-16T09:59:58.123 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:58.121+0000 7fa3deb48640 1 -- 172.21.15.156:0/4170399909 --> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa3d810edd0 con 0x7fa3c406e0b0 2024-07-16T09:59:58.124 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:58.123+0000 7fa3d57fa640 1 -- 172.21.15.156:0/4170399909 <== mgr.34104 v2:172.21.15.156:6800/3367765292 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7fa3d810edd0 con 0x7fa3c406e0b0 2024-07-16T09:59:58.126 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:58.125+0000 7fa3deb48640 1 -- 172.21.15.156:0/4170399909 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7fa3c406e0b0 msgr2=0x7fa3c4070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:59:58.126 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:58.125+0000 7fa3deb48640 1 --2- 172.21.15.156:0/4170399909 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7fa3c406e0b0 0x7fa3c4070560 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fa3c8004a70 tx=0x7fa3c800b040 comp rx=0 tx=0).stop 2024-07-16T09:59:58.126 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:58.125+0000 7fa3deb48640 1 -- 172.21.15.156:0/4170399909 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa3d80fd630 msgr2=0x7fa3d810d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:59:58.126 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:58.125+0000 7fa3deb48640 1 --2- 172.21.15.156:0/4170399909 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa3d80fd630 0x7fa3d810d4c0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fa3c00048a0 tx=0x7fa3c0009da0 comp rx=0 tx=0).stop 2024-07-16T09:59:58.127 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:58.125+0000 7fa3deb48640 1 -- 172.21.15.156:0/4170399909 shutdown_connections 2024-07-16T09:59:58.127 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:58.125+0000 7fa3deb48640 1 --2- 172.21.15.156:0/4170399909 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7fa3c406e0b0 0x7fa3c4070560 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:59:58.127 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:58.125+0000 7fa3deb48640 1 --2- 172.21.15.156:0/4170399909 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa3d80fd630 0x7fa3d810d4c0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:59:58.127 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:58.125+0000 7fa3deb48640 1 --2- 172.21.15.156:0/4170399909 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa3d80fc360 0x7fa3d810cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:59:58.127 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:58.125+0000 7fa3deb48640 1 -- 172.21.15.156:0/4170399909 >> 172.21.15.156:0/4170399909 conn(0x7fa3d80f7b40 msgr2=0x7fa3d80f9230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:59:58.127 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:58.125+0000 7fa3deb48640 1 -- 172.21.15.156:0/4170399909 shutdown_connections 2024-07-16T09:59:58.127 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:58.125+0000 7fa3deb48640 1 -- 172.21.15.156:0/4170399909 wait complete. 2024-07-16T09:59:58.135 INFO:teuthology.orchestra.run.smithi156.stdout:true 2024-07-16T09:59:58.208 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:58.207+0000 7f158142f640 1 -- 172.21.15.156:0/214043404 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f157c0659d0 msgr2=0x7f157c0fef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:59:58.208 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:58.207+0000 7f158142f640 1 --2- 172.21.15.156:0/214043404 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f157c0659d0 0x7f157c0fef80 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f15640048d0 tx=0x7f1564009da0 comp rx=0 tx=0).stop 2024-07-16T09:59:58.208 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:58.207+0000 7f158142f640 1 -- 172.21.15.156:0/214043404 shutdown_connections 2024-07-16T09:59:58.209 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:58.207+0000 7f158142f640 1 --2- 172.21.15.156:0/214043404 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f157c0ff4c0 0x7f157c1018a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:59:58.209 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:58.207+0000 7f158142f640 1 --2- 172.21.15.156:0/214043404 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f157c0659d0 0x7f157c0fef80 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:59:58.209 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:58.207+0000 7f158142f640 1 -- 172.21.15.156:0/214043404 >> 172.21.15.156:0/214043404 conn(0x7f157c0f6990 msgr2=0x7f157c0f8de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:59:58.209 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:58.207+0000 7f158142f640 1 -- 172.21.15.156:0/214043404 shutdown_connections 2024-07-16T09:59:58.209 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:58.207+0000 7f158142f640 1 -- 172.21.15.156:0/214043404 wait complete. 2024-07-16T09:59:58.211 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:58.207+0000 7f158142f640 1 Processor -- start 2024-07-16T09:59:58.211 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:58.208+0000 7f158142f640 1 -- start start 2024-07-16T09:59:58.211 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:58.208+0000 7f158142f640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f157c0659d0 0x7f157c10cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:59:58.211 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:58.208+0000 7f158142f640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f157c0ff4c0 0x7f157c10d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:59:58.211 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:58.208+0000 7f158142f640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f157c10db60 con 0x7f157c0ff4c0 2024-07-16T09:59:58.211 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:58.208+0000 7f158142f640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f157c10e330 con 0x7f157c0659d0 2024-07-16T09:59:58.211 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:58.208+0000 7f157a7fc640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f157c0ff4c0 0x7f157c10d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:59:58.212 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:58.208+0000 7f157a7fc640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f157c0ff4c0 0x7f157c10d510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:34424/0 (socket says 172.21.15.156:34424) 2024-07-16T09:59:58.212 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:58.208+0000 7f157a7fc640 1 -- 172.21.15.156:0/3719293548 learned_addr learned my addr 172.21.15.156:0/3719293548 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T09:59:58.212 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:58.208+0000 7f157affd640 1 --2- 172.21.15.156:0/3719293548 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f157c0659d0 0x7f157c10cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:59:58.212 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:58.208+0000 7f157a7fc640 1 -- 172.21.15.156:0/3719293548 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f157c0659d0 msgr2=0x7f157c10cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:59:58.212 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:58.208+0000 7f157a7fc640 1 --2- 172.21.15.156:0/3719293548 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f157c0659d0 0x7f157c10cfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:59:58.212 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:58.208+0000 7f157a7fc640 1 -- 172.21.15.156:0/3719293548 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1564005040 con 0x7f157c0ff4c0 2024-07-16T09:59:58.212 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:58.208+0000 7f157a7fc640 1 --2- 172.21.15.156:0/3719293548 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f157c0ff4c0 0x7f157c10d510 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f156c002f40 tx=0x7f156c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:59:58.212 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:58.209+0000 7f15737fe640 1 -- 172.21.15.156:0/3719293548 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f156c007c30 con 0x7f157c0ff4c0 2024-07-16T09:59:58.212 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:58.209+0000 7f15737fe640 1 -- 172.21.15.156:0/3719293548 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f156c00f040 con 0x7f157c0ff4c0 2024-07-16T09:59:58.212 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:58.209+0000 7f158142f640 1 -- 172.21.15.156:0/3719293548 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f157c10e560 con 0x7f157c0ff4c0 2024-07-16T09:59:58.213 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:58.209+0000 7f15737fe640 1 -- 172.21.15.156:0/3719293548 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f156c00cc30 con 0x7f157c0ff4c0 2024-07-16T09:59:58.213 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:58.209+0000 7f158142f640 1 -- 172.21.15.156:0/3719293548 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f157c10eb00 con 0x7f157c0ff4c0 2024-07-16T09:59:58.213 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:58.210+0000 7f158142f640 1 -- 172.21.15.156:0/3719293548 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f157c05eb80 con 0x7f157c0ff4c0 2024-07-16T09:59:58.213 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:58.210+0000 7f15737fe640 1 -- 172.21.15.156:0/3719293548 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 38) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f156c0090d0 con 0x7f157c0ff4c0 2024-07-16T09:59:58.214 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:58.211+0000 7f15737fe640 1 --2- 172.21.15.156:0/3719293548 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f156806e0b0 0x7f1568070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:59:58.214 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:58.211+0000 7f15737fe640 1 -- 172.21.15.156:0/3719293548 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f156c08d750 con 0x7f157c0ff4c0 2024-07-16T09:59:58.214 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:58.211+0000 7f157affd640 1 --2- 172.21.15.156:0/3719293548 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f156806e0b0 0x7f1568070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:59:58.214 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:58.211+0000 7f157affd640 1 --2- 172.21.15.156:0/3719293548 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f156806e0b0 0x7f1568070560 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f1564000c00 tx=0x7f1564035040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:59:58.216 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:58.215+0000 7f15737fe640 1 -- 172.21.15.156:0/3719293548 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187716 (secure 0 0 0) 0x7f156c056b80 con 0x7f157c0ff4c0 2024-07-16T09:59:58.411 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:58.410+0000 7f158142f640 1 -- 172.21.15.156:0/3719293548 --> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f157c10ee30 con 0x7f156806e0b0 2024-07-16T09:59:58.412 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:58.411+0000 7f15737fe640 1 -- 172.21.15.156:0/3719293548 <== mgr.34104 v2:172.21.15.156:6800/3367765292 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f157c10ee30 con 0x7f156806e0b0 2024-07-16T09:59:58.414 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:58.413+0000 7f158142f640 1 -- 172.21.15.156:0/3719293548 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f156806e0b0 msgr2=0x7f1568070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:59:58.415 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:58.413+0000 7f158142f640 1 --2- 172.21.15.156:0/3719293548 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f156806e0b0 0x7f1568070560 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f1564000c00 tx=0x7f1564035040 comp rx=0 tx=0).stop 2024-07-16T09:59:58.415 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:58.413+0000 7f158142f640 1 -- 172.21.15.156:0/3719293548 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f157c0ff4c0 msgr2=0x7f157c10d510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:59:58.415 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:58.413+0000 7f158142f640 1 --2- 172.21.15.156:0/3719293548 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f157c0ff4c0 0x7f157c10d510 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f156c002f40 tx=0x7f156c00b040 comp rx=0 tx=0).stop 2024-07-16T09:59:58.415 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:58.413+0000 7f158142f640 1 -- 172.21.15.156:0/3719293548 shutdown_connections 2024-07-16T09:59:58.415 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:58.413+0000 7f158142f640 1 --2- 172.21.15.156:0/3719293548 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f156806e0b0 0x7f1568070560 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:59:58.415 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:58.413+0000 7f158142f640 1 --2- 172.21.15.156:0/3719293548 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f157c0ff4c0 0x7f157c10d510 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:59:58.415 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:58.413+0000 7f158142f640 1 --2- 172.21.15.156:0/3719293548 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f157c0659d0 0x7f157c10cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:59:58.415 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:58.413+0000 7f158142f640 1 -- 172.21.15.156:0/3719293548 >> 172.21.15.156:0/3719293548 conn(0x7f157c0f6990 msgr2=0x7f157c0f6dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:59:58.415 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:58.413+0000 7f158142f640 1 -- 172.21.15.156:0/3719293548 shutdown_connections 2024-07-16T09:59:58.415 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:58.413+0000 7f158142f640 1 -- 172.21.15.156:0/3719293548 wait complete. 2024-07-16T09:59:58.494 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:58.492+0000 7f9cf9d3c640 1 -- 172.21.15.156:0/1691166738 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9cf40fb260 msgr2=0x7f9cf40fb670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:59:58.494 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:58.492+0000 7f9cf9d3c640 1 --2- 172.21.15.156:0/1691166738 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9cf40fb260 0x7f9cf40fb670 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f9ce40048d0 tx=0x7f9ce4009da0 comp rx=0 tx=0).stop 2024-07-16T09:59:58.494 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:58.492+0000 7f9cf9d3c640 1 -- 172.21.15.156:0/1691166738 shutdown_connections 2024-07-16T09:59:58.494 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:58.492+0000 7f9cf9d3c640 1 --2- 172.21.15.156:0/1691166738 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9cf40fc530 0x7f9cf40fc9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:59:58.494 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:58.492+0000 7f9cf9d3c640 1 --2- 172.21.15.156:0/1691166738 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9cf40fb260 0x7f9cf40fb670 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:59:58.494 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:58.492+0000 7f9cf9d3c640 1 -- 172.21.15.156:0/1691166738 >> 172.21.15.156:0/1691166738 conn(0x7f9cf40f6a20 msgr2=0x7f9cf40f8e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:59:58.495 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:59:58 smithi156 ceph-mon[93066]: from='client.34252 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-16T09:59:58.496 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:58.492+0000 7f9cf9d3c640 1 -- 172.21.15.156:0/1691166738 shutdown_connections 2024-07-16T09:59:58.497 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:58.492+0000 7f9cf9d3c640 1 -- 172.21.15.156:0/1691166738 wait complete. 2024-07-16T09:59:58.497 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:58.493+0000 7f9cf9d3c640 1 Processor -- start 2024-07-16T09:59:58.497 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:58.493+0000 7f9cf9d3c640 1 -- start start 2024-07-16T09:59:58.497 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:58.493+0000 7f9cf9d3c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9cf40fb260 0x7f9cf410fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:59:58.497 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:58.493+0000 7f9cf9d3c640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9cf40fc530 0x7f9cf4112300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:59:58.497 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:58.493+0000 7f9cf9d3c640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cf40665d0 con 0x7f9cf40fb260 2024-07-16T09:59:58.497 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:58.493+0000 7f9cf9d3c640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cf4066740 con 0x7f9cf40fc530 2024-07-16T09:59:58.497 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:58.494+0000 7f9cf37fe640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9cf40fb260 0x7f9cf410fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:59:58.497 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:58.494+0000 7f9cf37fe640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9cf40fb260 0x7f9cf410fdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:34434/0 (socket says 172.21.15.156:34434) 2024-07-16T09:59:58.497 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:58.494+0000 7f9cf37fe640 1 -- 172.21.15.156:0/3522635117 learned_addr learned my addr 172.21.15.156:0/3522635117 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T09:59:58.497 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:58.494+0000 7f9cf37fe640 1 -- 172.21.15.156:0/3522635117 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9cf40fc530 msgr2=0x7f9cf4112300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:59:58.498 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:58.494+0000 7f9ceaffd640 1 --2- 172.21.15.156:0/3522635117 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9cf40fc530 0x7f9cf4112300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:59:58.498 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:58.494+0000 7f9cf37fe640 1 --2- 172.21.15.156:0/3522635117 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9cf40fc530 0x7f9cf4112300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:59:58.498 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:58.494+0000 7f9cf37fe640 1 -- 172.21.15.156:0/3522635117 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ce4005040 con 0x7f9cf40fb260 2024-07-16T09:59:58.498 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:58.494+0000 7f9cf37fe640 1 --2- 172.21.15.156:0/3522635117 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9cf40fb260 0x7f9cf410fdb0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f9ce40048a0 tx=0x7f9ce4009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:59:58.498 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:58.494+0000 7f9cf0ff9640 1 -- 172.21.15.156:0/3522635117 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9ce4038070 con 0x7f9cf40fb260 2024-07-16T09:59:58.498 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:58.494+0000 7f9cf0ff9640 1 -- 172.21.15.156:0/3522635117 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9ce4032970 con 0x7f9cf40fb260 2024-07-16T09:59:58.498 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:58.494+0000 7f9cf9d3c640 1 -- 172.21.15.156:0/3522635117 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9cf41128a0 con 0x7f9cf40fb260 2024-07-16T09:59:58.498 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:58.494+0000 7f9cf0ff9640 1 -- 172.21.15.156:0/3522635117 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9ce403d8c0 con 0x7f9cf40fb260 2024-07-16T09:59:58.498 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:58.494+0000 7f9cf9d3c640 1 -- 172.21.15.156:0/3522635117 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9cf4112dc0 con 0x7f9cf40fb260 2024-07-16T09:59:58.499 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:58.496+0000 7f9cf9d3c640 1 -- 172.21.15.156:0/3522635117 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9cf40fa6c0 con 0x7f9cf40fb260 2024-07-16T09:59:58.499 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:58.496+0000 7f9cf0ff9640 1 -- 172.21.15.156:0/3522635117 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 38) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f9ce4030020 con 0x7f9cf40fb260 2024-07-16T09:59:58.499 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:58.497+0000 7f9cf0ff9640 1 --2- 172.21.15.156:0/3522635117 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f9ce006e0b0 0x7f9ce0070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:59:58.499 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:58.497+0000 7f9cf0ff9640 1 -- 172.21.15.156:0/3522635117 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f9ce40ae230 con 0x7f9cf40fb260 2024-07-16T09:59:58.500 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:58.497+0000 7f9ceaffd640 1 --2- 172.21.15.156:0/3522635117 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f9ce006e0b0 0x7f9ce0070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:59:58.500 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:58.497+0000 7f9ceaffd640 1 --2- 172.21.15.156:0/3522635117 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f9ce006e0b0 0x7f9ce0070560 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f9cdc00a900 tx=0x7f9cdc008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:59:58.502 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:58.501+0000 7f9cf0ff9640 1 -- 172.21.15.156:0/3522635117 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187716 (secure 0 0 0) 0x7f9ce4077670 con 0x7f9cf40fb260 2024-07-16T09:59:58.697 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:58.695+0000 7f9cf9d3c640 1 -- 172.21.15.156:0/3522635117 --> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f9cf41008e0 con 0x7f9ce006e0b0 2024-07-16T09:59:58.706 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:58.705+0000 7f9cf0ff9640 1 -- 172.21.15.156:0/3522635117 <== mgr.34104 v2:172.21.15.156:6800/3367765292 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7f9cf41008e0 con 0x7f9ce006e0b0 2024-07-16T09:59:58.706 INFO:teuthology.orchestra.run.smithi156.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-07-16T09:59:58.706 INFO:teuthology.orchestra.run.smithi156.stdout:alertmanager.smithi156 smithi156 *:9093,9094 running (3m) 19s ago 12m 25.0M - 0.25.0 c8568f914cd2 d524a2c3ff15 2024-07-16T09:59:58.706 INFO:teuthology.orchestra.run.smithi156.stdout:crash.smithi156 smithi156 running (82s) 19s ago 12m 6899k - 19.0.0-3495-ga6ca461b 16f7feb04342 11648cf29af4 2024-07-16T09:59:58.707 INFO:teuthology.orchestra.run.smithi156.stdout:crash.smithi176 smithi176 running (77s) 74s ago 11m 6899k - 19.0.0-3495-ga6ca461b 16f7feb04342 67f3ccc034f8 2024-07-16T09:59:58.707 INFO:teuthology.orchestra.run.smithi156.stdout:grafana.smithi156 smithi156 *:3000 running (3m) 19s ago 11m 75.5M - 9.4.7 954c08fa6188 deb9e1bfeb66 2024-07-16T09:59:58.707 INFO:teuthology.orchestra.run.smithi156.stdout:mds.cephfs.smithi156.evwwaz smithi156 running (6m) 19s ago 6m 67.2M - 17.2.7-1104-ge6231b04 d5de5f953b95 7c0ff9ae43f8 2024-07-16T09:59:58.707 INFO:teuthology.orchestra.run.smithi156.stdout:mds.cephfs.smithi156.rdibys smithi156 running (6m) 19s ago 6m 120M - 17.2.7-1104-ge6231b04 d5de5f953b95 b985cd1e6924 2024-07-16T09:59:58.707 INFO:teuthology.orchestra.run.smithi156.stdout:mds.cephfs.smithi176.dvvbud smithi176 running (6m) 74s ago 6m 108M - 17.2.7-1104-ge6231b04 d5de5f953b95 1b00147950f9 2024-07-16T09:59:58.707 INFO:teuthology.orchestra.run.smithi156.stdout:mds.cephfs.smithi176.wahbsi smithi176 running (6m) 74s ago 6m 62.2M - 17.2.7-1104-ge6231b04 d5de5f953b95 866664af2311 2024-07-16T09:59:58.707 INFO:teuthology.orchestra.run.smithi156.stdout:mgr.smithi156.zkvylv smithi156 *:8443,9283,8765 running (3m) 19s ago 13m 496M - 19.0.0-3495-ga6ca461b 16f7feb04342 ee53bd604b7c 2024-07-16T09:59:58.707 INFO:teuthology.orchestra.run.smithi156.stdout:mgr.smithi176.dqbrvx smithi176 *:8443,9283,8765 running (2m) 74s ago 11m 438M - 19.0.0-3495-ga6ca461b 16f7feb04342 ce9eaae4c2d0 2024-07-16T09:59:58.707 INFO:teuthology.orchestra.run.smithi156.stdout:mon.smithi156 smithi156 running (2m) 19s ago 13m 44.2M 2048M 19.0.0-3495-ga6ca461b 16f7feb04342 77b6ec8c50eb 2024-07-16T09:59:58.707 INFO:teuthology.orchestra.run.smithi156.stdout:mon.smithi176 smithi176 running (118s) 74s ago 11m 33.4M 2048M 19.0.0-3495-ga6ca461b 16f7feb04342 ef92814cec2b 2024-07-16T09:59:58.708 INFO:teuthology.orchestra.run.smithi156.stdout:node-exporter.smithi156 smithi156 *:9100 running (3m) 19s ago 12m 16.5M - 1.5.0 0da6a335fe13 e5d86f23a14c 2024-07-16T09:59:58.708 INFO:teuthology.orchestra.run.smithi156.stdout:node-exporter.smithi176 smithi176 *:9100 running (3m) 74s ago 11m 16.1M - 1.5.0 0da6a335fe13 ea842728144a 2024-07-16T09:59:58.708 INFO:teuthology.orchestra.run.smithi156.stdout:osd.0 smithi156 running (58s) 19s ago 10m 92.0M 1557M 19.0.0-3495-ga6ca461b 16f7feb04342 8633df1689cc 2024-07-16T09:59:58.708 INFO:teuthology.orchestra.run.smithi156.stdout:osd.1 smithi156 running (24s) 19s ago 10m 14.5M 1557M 19.0.0-3495-ga6ca461b 16f7feb04342 e6fae9653b46 2024-07-16T09:59:58.708 INFO:teuthology.orchestra.run.smithi156.stdout:osd.2 smithi156 running (9m) 19s ago 9m 199M 1557M 17.2.7-1104-ge6231b04 d5de5f953b95 f28acec5cb79 2024-07-16T09:59:58.708 INFO:teuthology.orchestra.run.smithi156.stdout:osd.3 smithi176 running (9m) 74s ago 9m 260M 2581M 17.2.7-1104-ge6231b04 d5de5f953b95 733861ba84dc 2024-07-16T09:59:58.708 INFO:teuthology.orchestra.run.smithi156.stdout:osd.4 smithi176 running (8m) 74s ago 8m 229M 2581M 17.2.7-1104-ge6231b04 d5de5f953b95 ef0c67cec086 2024-07-16T09:59:58.708 INFO:teuthology.orchestra.run.smithi156.stdout:osd.5 smithi176 running (8m) 74s ago 8m 224M 2581M 17.2.7-1104-ge6231b04 d5de5f953b95 996e15448a52 2024-07-16T09:59:58.708 INFO:teuthology.orchestra.run.smithi156.stdout:prometheus.smithi156 smithi156 *:9095 running (2m) 19s ago 11m 53.9M - 2.43.0 a07b618ecd1d 20c195ecd665 2024-07-16T09:59:58.709 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:58.707+0000 7f9cf9d3c640 1 -- 172.21.15.156:0/3522635117 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f9ce006e0b0 msgr2=0x7f9ce0070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:59:58.709 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:58.707+0000 7f9cf9d3c640 1 --2- 172.21.15.156:0/3522635117 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f9ce006e0b0 0x7f9ce0070560 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f9cdc00a900 tx=0x7f9cdc008040 comp rx=0 tx=0).stop 2024-07-16T09:59:58.709 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:58.707+0000 7f9cf9d3c640 1 -- 172.21.15.156:0/3522635117 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9cf40fb260 msgr2=0x7f9cf410fdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:59:58.709 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:58.707+0000 7f9cf9d3c640 1 --2- 172.21.15.156:0/3522635117 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9cf40fb260 0x7f9cf410fdb0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f9ce40048a0 tx=0x7f9ce4009da0 comp rx=0 tx=0).stop 2024-07-16T09:59:58.709 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:58.707+0000 7f9cf9d3c640 1 -- 172.21.15.156:0/3522635117 shutdown_connections 2024-07-16T09:59:58.709 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:58.707+0000 7f9cf9d3c640 1 --2- 172.21.15.156:0/3522635117 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f9ce006e0b0 0x7f9ce0070560 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:59:58.709 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:58.707+0000 7f9cf9d3c640 1 --2- 172.21.15.156:0/3522635117 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9cf40fc530 0x7f9cf4112300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:59:58.709 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:58.707+0000 7f9cf9d3c640 1 --2- 172.21.15.156:0/3522635117 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9cf40fb260 0x7f9cf410fdb0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:59:58.709 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:58.707+0000 7f9cf9d3c640 1 -- 172.21.15.156:0/3522635117 >> 172.21.15.156:0/3522635117 conn(0x7f9cf40f6a20 msgr2=0x7f9cf40f8150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:59:58.709 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:58.707+0000 7f9cf9d3c640 1 -- 172.21.15.156:0/3522635117 shutdown_connections 2024-07-16T09:59:58.710 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:58.707+0000 7f9cf9d3c640 1 -- 172.21.15.156:0/3522635117 wait complete. 2024-07-16T09:59:58.791 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:58.789+0000 7f8a4b15f640 1 -- 172.21.15.156:0/2008247053 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8a440659f0 msgr2=0x7f8a441013b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:59:58.791 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:58.789+0000 7f8a4b15f640 1 --2- 172.21.15.156:0/2008247053 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8a440659f0 0x7f8a441013b0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f8a380048d0 tx=0x7f8a38009da0 comp rx=0 tx=0).stop 2024-07-16T09:59:58.791 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:58.789+0000 7f8a4b15f640 1 -- 172.21.15.156:0/2008247053 shutdown_connections 2024-07-16T09:59:58.791 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:58.789+0000 7f8a4b15f640 1 --2- 172.21.15.156:0/2008247053 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8a441018f0 0x7f8a44103cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:59:58.792 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:58.789+0000 7f8a4b15f640 1 --2- 172.21.15.156:0/2008247053 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8a440659f0 0x7f8a441013b0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:59:58.792 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:58.789+0000 7f8a4b15f640 1 -- 172.21.15.156:0/2008247053 >> 172.21.15.156:0/2008247053 conn(0x7f8a440f6a20 msgr2=0x7f8a440f8e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:59:58.792 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:58.789+0000 7f8a4b15f640 1 -- 172.21.15.156:0/2008247053 shutdown_connections 2024-07-16T09:59:58.792 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:58.789+0000 7f8a4b15f640 1 -- 172.21.15.156:0/2008247053 wait complete. 2024-07-16T09:59:58.792 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:58.790+0000 7f8a4b15f640 1 Processor -- start 2024-07-16T09:59:58.792 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:58.790+0000 7f8a4b15f640 1 -- start start 2024-07-16T09:59:58.793 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:58.790+0000 7f8a4b15f640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8a440659f0 0x7f8a4410f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:59:58.793 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:58.790+0000 7f8a4b15f640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8a441018f0 0x7f8a4410f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:59:58.793 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:58.790+0000 7f8a4b15f640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a4410fd10 con 0x7f8a441018f0 2024-07-16T09:59:58.794 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:58.790+0000 7f8a4b15f640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a44110530 con 0x7f8a440659f0 2024-07-16T09:59:58.794 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:58.790+0000 7f8a43fff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8a441018f0 0x7f8a4410f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:59:58.794 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:58.790+0000 7f8a43fff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8a441018f0 0x7f8a4410f6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:34458/0 (socket says 172.21.15.156:34458) 2024-07-16T09:59:58.794 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:58.790+0000 7f8a43fff640 1 -- 172.21.15.156:0/4285645158 learned_addr learned my addr 172.21.15.156:0/4285645158 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T09:59:58.794 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:58.790+0000 7f8a48efb640 1 --2- 172.21.15.156:0/4285645158 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8a440659f0 0x7f8a4410f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:59:58.794 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:58.791+0000 7f8a43fff640 1 -- 172.21.15.156:0/4285645158 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8a440659f0 msgr2=0x7f8a4410f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:59:58.794 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:58.791+0000 7f8a43fff640 1 --2- 172.21.15.156:0/4285645158 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8a440659f0 0x7f8a4410f180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:59:58.794 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:58.791+0000 7f8a43fff640 1 -- 172.21.15.156:0/4285645158 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a38005040 con 0x7f8a441018f0 2024-07-16T09:59:58.794 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:58.791+0000 7f8a43fff640 1 --2- 172.21.15.156:0/4285645158 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8a441018f0 0x7f8a4410f6c0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f8a30002a80 tx=0x7f8a30004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:59:58.794 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:58.791+0000 7f8a417fa640 1 -- 172.21.15.156:0/4285645158 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8a30007c30 con 0x7f8a441018f0 2024-07-16T09:59:58.794 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:58.791+0000 7f8a4b15f640 1 -- 172.21.15.156:0/4285645158 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a441107c0 con 0x7f8a441018f0 2024-07-16T09:59:58.795 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:58.791+0000 7f8a4b15f640 1 -- 172.21.15.156:0/4285645158 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a44110ce0 con 0x7f8a441018f0 2024-07-16T09:59:58.795 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:58.791+0000 7f8a417fa640 1 -- 172.21.15.156:0/4285645158 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8a3000f040 con 0x7f8a441018f0 2024-07-16T09:59:58.795 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:58.791+0000 7f8a417fa640 1 -- 172.21.15.156:0/4285645158 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8a3000db70 con 0x7f8a441018f0 2024-07-16T09:59:58.795 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:58.793+0000 7f8a4b15f640 1 -- 172.21.15.156:0/4285645158 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a28005240 con 0x7f8a441018f0 2024-07-16T09:59:58.796 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:58.793+0000 7f8a417fa640 1 -- 172.21.15.156:0/4285645158 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 38) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f8a3001c400 con 0x7f8a441018f0 2024-07-16T09:59:58.796 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:58.794+0000 7f8a417fa640 1 --2- 172.21.15.156:0/4285645158 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f8a3406e0b0 0x7f8a34070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:59:58.796 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:58.794+0000 7f8a417fa640 1 -- 172.21.15.156:0/4285645158 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f8a3008d270 con 0x7f8a441018f0 2024-07-16T09:59:58.796 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:58.794+0000 7f8a48efb640 1 --2- 172.21.15.156:0/4285645158 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f8a3406e0b0 0x7f8a34070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:59:58.796 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:58.794+0000 7f8a48efb640 1 --2- 172.21.15.156:0/4285645158 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f8a3406e0b0 0x7f8a34070560 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f8a38000c00 tx=0x7f8a38035040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:59:58.799 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:58.798+0000 7f8a417fa640 1 -- 172.21.15.156:0/4285645158 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187716 (secure 0 0 0) 0x7f8a30056720 con 0x7f8a441018f0 2024-07-16T09:59:58.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:59:58 smithi176 ceph-mon[81689]: from='client.34252 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-16T09:59:59.064 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:59.063+0000 7f8a4b15f640 1 -- 172.21.15.156:0/4285645158 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f8a28005d00 con 0x7f8a441018f0 2024-07-16T09:59:59.065 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:59.064+0000 7f8a417fa640 1 -- 172.21.15.156:0/4285645158 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+838 (secure 0 0 0) 0x7f8a30059de0 con 0x7f8a441018f0 2024-07-16T09:59:59.065 INFO:teuthology.orchestra.run.smithi156.stdout:{ 2024-07-16T09:59:59.065 INFO:teuthology.orchestra.run.smithi156.stdout: "mon": { 2024-07-16T09:59:59.065 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph version 19.0.0-3495-ga6ca461b (a6ca461be7cc1aff3658c2c6df4cff975a24449a) squid (rc)": 2 2024-07-16T09:59:59.065 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-07-16T09:59:59.065 INFO:teuthology.orchestra.run.smithi156.stdout: "mgr": { 2024-07-16T09:59:59.065 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph version 19.0.0-3495-ga6ca461b (a6ca461be7cc1aff3658c2c6df4cff975a24449a) squid (rc)": 2 2024-07-16T09:59:59.066 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-07-16T09:59:59.066 INFO:teuthology.orchestra.run.smithi156.stdout: "osd": { 2024-07-16T09:59:59.066 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph version 17.2.7-1104-ge6231b04 (e6231b04b52ebf3b9b31803282dd3074439e51ac) quincy (stable)": 4, 2024-07-16T09:59:59.066 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph version 19.0.0-3495-ga6ca461b (a6ca461be7cc1aff3658c2c6df4cff975a24449a) squid (rc)": 2 2024-07-16T09:59:59.066 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-07-16T09:59:59.066 INFO:teuthology.orchestra.run.smithi156.stdout: "mds": { 2024-07-16T09:59:59.066 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph version 17.2.7-1104-ge6231b04 (e6231b04b52ebf3b9b31803282dd3074439e51ac) quincy (stable)": 4 2024-07-16T09:59:59.066 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-07-16T09:59:59.066 INFO:teuthology.orchestra.run.smithi156.stdout: "overall": { 2024-07-16T09:59:59.066 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph version 17.2.7-1104-ge6231b04 (e6231b04b52ebf3b9b31803282dd3074439e51ac) quincy (stable)": 8, 2024-07-16T09:59:59.066 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph version 19.0.0-3495-ga6ca461b (a6ca461be7cc1aff3658c2c6df4cff975a24449a) squid (rc)": 6 2024-07-16T09:59:59.066 INFO:teuthology.orchestra.run.smithi156.stdout: } 2024-07-16T09:59:59.067 INFO:teuthology.orchestra.run.smithi156.stdout:} 2024-07-16T09:59:59.067 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:59.066+0000 7f8a4b15f640 1 -- 172.21.15.156:0/4285645158 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f8a3406e0b0 msgr2=0x7f8a34070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:59:59.068 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:59.066+0000 7f8a4b15f640 1 --2- 172.21.15.156:0/4285645158 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f8a3406e0b0 0x7f8a34070560 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f8a38000c00 tx=0x7f8a38035040 comp rx=0 tx=0).stop 2024-07-16T09:59:59.068 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:59.066+0000 7f8a4b15f640 1 -- 172.21.15.156:0/4285645158 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8a441018f0 msgr2=0x7f8a4410f6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:59:59.068 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:59.066+0000 7f8a4b15f640 1 --2- 172.21.15.156:0/4285645158 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8a441018f0 0x7f8a4410f6c0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f8a30002a80 tx=0x7f8a30004600 comp rx=0 tx=0).stop 2024-07-16T09:59:59.068 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:59.066+0000 7f8a4b15f640 1 -- 172.21.15.156:0/4285645158 shutdown_connections 2024-07-16T09:59:59.068 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:59.066+0000 7f8a4b15f640 1 --2- 172.21.15.156:0/4285645158 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f8a3406e0b0 0x7f8a34070560 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:59:59.068 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:59.066+0000 7f8a4b15f640 1 --2- 172.21.15.156:0/4285645158 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8a441018f0 0x7f8a4410f6c0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:59:59.068 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:59.066+0000 7f8a4b15f640 1 --2- 172.21.15.156:0/4285645158 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8a440659f0 0x7f8a4410f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:59:59.068 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:59.066+0000 7f8a4b15f640 1 -- 172.21.15.156:0/4285645158 >> 172.21.15.156:0/4285645158 conn(0x7f8a440f6a20 msgr2=0x7f8a440f6df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:59:59.068 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:59.066+0000 7f8a4b15f640 1 -- 172.21.15.156:0/4285645158 shutdown_connections 2024-07-16T09:59:59.069 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:59.066+0000 7f8a4b15f640 1 -- 172.21.15.156:0/4285645158 wait complete. 2024-07-16T09:59:59.152 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:59.149+0000 7ff325584640 1 -- 172.21.15.156:0/271902847 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff3200fd630 msgr2=0x7ff3200fdaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:59:59.152 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:59.149+0000 7ff325584640 1 --2- 172.21.15.156:0/271902847 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff3200fd630 0x7ff3200fdaa0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7ff3140048d0 tx=0x7ff314009da0 comp rx=0 tx=0).stop 2024-07-16T09:59:59.152 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:59.149+0000 7ff325584640 1 -- 172.21.15.156:0/271902847 shutdown_connections 2024-07-16T09:59:59.152 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:59.149+0000 7ff325584640 1 --2- 172.21.15.156:0/271902847 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff3200fd630 0x7ff3200fdaa0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:59:59.153 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:59.149+0000 7ff325584640 1 --2- 172.21.15.156:0/271902847 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff3200fc360 0x7ff3200fc770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:59:59.153 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:59.149+0000 7ff325584640 1 -- 172.21.15.156:0/271902847 >> 172.21.15.156:0/271902847 conn(0x7ff3200f7b40 msgr2=0x7ff3200f9f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:59:59.153 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:59.149+0000 7ff325584640 1 -- 172.21.15.156:0/271902847 shutdown_connections 2024-07-16T09:59:59.153 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:59.149+0000 7ff325584640 1 -- 172.21.15.156:0/271902847 wait complete. 2024-07-16T09:59:59.153 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:59.150+0000 7ff325584640 1 Processor -- start 2024-07-16T09:59:59.153 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:59.150+0000 7ff325584640 1 -- start start 2024-07-16T09:59:59.153 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:59.150+0000 7ff325584640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff3200fc360 0x7ff32010cf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:59:59.153 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:59.150+0000 7ff325584640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff3200fd630 0x7ff32010d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:59:59.153 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:59.150+0000 7ff325584640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff32010db10 con 0x7ff3200fc360 2024-07-16T09:59:59.153 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:59.150+0000 7ff325584640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff32010e330 con 0x7ff3200fd630 2024-07-16T09:59:59.153 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:59.150+0000 7ff31effd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff3200fc360 0x7ff32010cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:59:59.153 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:59.150+0000 7ff31effd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff3200fc360 0x7ff32010cf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:34478/0 (socket says 172.21.15.156:34478) 2024-07-16T09:59:59.154 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:59.150+0000 7ff31effd640 1 -- 172.21.15.156:0/1052938376 learned_addr learned my addr 172.21.15.156:0/1052938376 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T09:59:59.154 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:59.150+0000 7ff31e7fc640 1 --2- 172.21.15.156:0/1052938376 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff3200fd630 0x7ff32010d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:59:59.154 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:59.150+0000 7ff31effd640 1 -- 172.21.15.156:0/1052938376 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff3200fd630 msgr2=0x7ff32010d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:59:59.154 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:59.150+0000 7ff31effd640 1 --2- 172.21.15.156:0/1052938376 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff3200fd630 0x7ff32010d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:59:59.154 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:59.151+0000 7ff31effd640 1 -- 172.21.15.156:0/1052938376 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff314005040 con 0x7ff3200fc360 2024-07-16T09:59:59.154 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:59.151+0000 7ff31e7fc640 1 --2- 172.21.15.156:0/1052938376 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff3200fd630 0x7ff32010d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-16T09:59:59.154 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:59.151+0000 7ff31effd640 1 --2- 172.21.15.156:0/1052938376 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff3200fc360 0x7ff32010cf80 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7ff31000c790 tx=0x7ff3100043c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:59:59.154 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:59.151+0000 7ff3077fe640 1 -- 172.21.15.156:0/1052938376 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff31000a820 con 0x7ff3200fc360 2024-07-16T09:59:59.154 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:59.151+0000 7ff3077fe640 1 -- 172.21.15.156:0/1052938376 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff31000ae50 con 0x7ff3200fc360 2024-07-16T09:59:59.154 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:59.151+0000 7ff325584640 1 -- 172.21.15.156:0/1052938376 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff32010e5c0 con 0x7ff3200fc360 2024-07-16T09:59:59.154 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:59.151+0000 7ff3077fe640 1 -- 172.21.15.156:0/1052938376 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff3100115e0 con 0x7ff3200fc360 2024-07-16T09:59:59.154 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:59.151+0000 7ff325584640 1 -- 172.21.15.156:0/1052938376 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff32010eb60 con 0x7ff3200fc360 2024-07-16T09:59:59.155 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:59.152+0000 7ff325584640 1 -- 172.21.15.156:0/1052938376 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3200fb7c0 con 0x7ff3200fc360 2024-07-16T09:59:59.155 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:59.153+0000 7ff3077fe640 1 -- 172.21.15.156:0/1052938376 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 38) v1 ==== 92205+0+0 (secure 0 0 0) 0x7ff310012400 con 0x7ff3200fc360 2024-07-16T09:59:59.156 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:59.153+0000 7ff3077fe640 1 --2- 172.21.15.156:0/1052938376 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7ff30806e0b0 0x7ff308070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:59:59.156 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:59.153+0000 7ff3077fe640 1 -- 172.21.15.156:0/1052938376 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6410+0+0 (secure 0 0 0) 0x7ff31008dce0 con 0x7ff3200fc360 2024-07-16T09:59:59.156 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:59.153+0000 7ff31e7fc640 1 --2- 172.21.15.156:0/1052938376 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7ff30806e0b0 0x7ff308070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:59:59.156 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:59.154+0000 7ff31e7fc640 1 --2- 172.21.15.156:0/1052938376 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7ff30806e0b0 0x7ff308070560 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7ff3140047f0 tx=0x7ff314009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:59:59.159 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:59.158+0000 7ff3077fe640 1 -- 172.21.15.156:0/1052938376 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187716 (secure 0 0 0) 0x7ff3100570e0 con 0x7ff3200fc360 2024-07-16T09:59:59.385 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:59.384+0000 7ff325584640 1 -- 172.21.15.156:0/1052938376 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7ff320104ed0 con 0x7ff3200fc360 2024-07-16T09:59:59.386 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:59.384+0000 7ff3077fe640 1 -- 172.21.15.156:0/1052938376 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 20 v20) v1 ==== 76+0+1920 (secure 0 0 0) 0x7ff31005a7a0 con 0x7ff3200fc360 2024-07-16T09:59:59.386 INFO:teuthology.orchestra.run.smithi156.stdout:e20 2024-07-16T09:59:59.386 INFO:teuthology.orchestra.run.smithi156.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-07-16T09:59:59.386 INFO:teuthology.orchestra.run.smithi156.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-07-16T09:59:59.386 INFO:teuthology.orchestra.run.smithi156.stdout:legacy client fscid: 1 2024-07-16T09:59:59.386 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-07-16T09:59:59.386 INFO:teuthology.orchestra.run.smithi156.stdout:Filesystem 'cephfs' (1) 2024-07-16T09:59:59.386 INFO:teuthology.orchestra.run.smithi156.stdout:fs_name cephfs 2024-07-16T09:59:59.387 INFO:teuthology.orchestra.run.smithi156.stdout:epoch 20 2024-07-16T09:59:59.387 INFO:teuthology.orchestra.run.smithi156.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-07-16T09:59:59.387 INFO:teuthology.orchestra.run.smithi156.stdout:created 2024-07-16T09:52:55.214975+0000 2024-07-16T09:59:59.387 INFO:teuthology.orchestra.run.smithi156.stdout:modified 2024-07-16T09:58:56.574272+0000 2024-07-16T09:59:59.387 INFO:teuthology.orchestra.run.smithi156.stdout:tableserver 0 2024-07-16T09:59:59.387 INFO:teuthology.orchestra.run.smithi156.stdout:root 0 2024-07-16T09:59:59.387 INFO:teuthology.orchestra.run.smithi156.stdout:session_timeout 60 2024-07-16T09:59:59.387 INFO:teuthology.orchestra.run.smithi156.stdout:session_autoclose 300 2024-07-16T09:59:59.387 INFO:teuthology.orchestra.run.smithi156.stdout:max_file_size 1099511627776 2024-07-16T09:59:59.387 INFO:teuthology.orchestra.run.smithi156.stdout:max_xattr_size 65536 2024-07-16T09:59:59.387 INFO:teuthology.orchestra.run.smithi156.stdout:required_client_features {} 2024-07-16T09:59:59.387 INFO:teuthology.orchestra.run.smithi156.stdout:last_failure 0 2024-07-16T09:59:59.388 INFO:teuthology.orchestra.run.smithi156.stdout:last_failure_osd_epoch 0 2024-07-16T09:59:59.388 INFO:teuthology.orchestra.run.smithi156.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-07-16T09:59:59.388 INFO:teuthology.orchestra.run.smithi156.stdout:max_mds 2 2024-07-16T09:59:59.388 INFO:teuthology.orchestra.run.smithi156.stdout:in 0,1 2024-07-16T09:59:59.388 INFO:teuthology.orchestra.run.smithi156.stdout:up {0=14458,1=24261} 2024-07-16T09:59:59.388 INFO:teuthology.orchestra.run.smithi156.stdout:failed 2024-07-16T09:59:59.388 INFO:teuthology.orchestra.run.smithi156.stdout:damaged 2024-07-16T09:59:59.388 INFO:teuthology.orchestra.run.smithi156.stdout:stopped 2024-07-16T09:59:59.388 INFO:teuthology.orchestra.run.smithi156.stdout:data_pools [3] 2024-07-16T09:59:59.388 INFO:teuthology.orchestra.run.smithi156.stdout:metadata_pool 2 2024-07-16T09:59:59.388 INFO:teuthology.orchestra.run.smithi156.stdout:inline_data disabled 2024-07-16T09:59:59.388 INFO:teuthology.orchestra.run.smithi156.stdout:balancer 2024-07-16T09:59:59.388 INFO:teuthology.orchestra.run.smithi156.stdout:bal_rank_mask -1 2024-07-16T09:59:59.389 INFO:teuthology.orchestra.run.smithi156.stdout:standby_count_wanted 1 2024-07-16T09:59:59.389 INFO:teuthology.orchestra.run.smithi156.stdout:qdb_cluster leader: 14458 members: 24261,14458 2024-07-16T09:59:59.389 INFO:teuthology.orchestra.run.smithi156.stdout:[mds.cephfs.smithi156.rdibys{0:14458} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.156:6826/1080176013,v1:172.21.15.156:6827/1080176013] compat {c=[1],r=[1],i=[7ff]}] 2024-07-16T09:59:59.389 INFO:teuthology.orchestra.run.smithi156.stdout:[mds.cephfs.smithi156.evwwaz{0:14468} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.156:6828/560329502,v1:172.21.15.156:6829/560329502] compat {c=[1],r=[1],i=[7ff]}] 2024-07-16T09:59:59.389 INFO:teuthology.orchestra.run.smithi156.stdout:[mds.cephfs.smithi176.dvvbud{1:24261} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.176:6824/1577019287,v1:172.21.15.176:6825/1577019287] compat {c=[1],r=[1],i=[7ff]}] 2024-07-16T09:59:59.389 INFO:teuthology.orchestra.run.smithi156.stdout:[mds.cephfs.smithi176.wahbsi{1:24273} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.176:6826/2196775412,v1:172.21.15.176:6827/2196775412] compat {c=[1],r=[1],i=[7ff]}] 2024-07-16T09:59:59.389 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-07-16T09:59:59.389 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-07-16T09:59:59.390 INFO:teuthology.orchestra.run.smithi156.stderr:dumped fsmap epoch 20 2024-07-16T09:59:59.390 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:59.386+0000 7ff325584640 1 -- 172.21.15.156:0/1052938376 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7ff30806e0b0 msgr2=0x7ff308070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:59:59.390 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:59.386+0000 7ff325584640 1 --2- 172.21.15.156:0/1052938376 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7ff30806e0b0 0x7ff308070560 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7ff3140047f0 tx=0x7ff314009da0 comp rx=0 tx=0).stop 2024-07-16T09:59:59.390 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:59.386+0000 7ff325584640 1 -- 172.21.15.156:0/1052938376 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff3200fc360 msgr2=0x7ff32010cf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:59:59.390 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:59.386+0000 7ff325584640 1 --2- 172.21.15.156:0/1052938376 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff3200fc360 0x7ff32010cf80 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7ff31000c790 tx=0x7ff3100043c0 comp rx=0 tx=0).stop 2024-07-16T09:59:59.390 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:59.386+0000 7ff325584640 1 -- 172.21.15.156:0/1052938376 shutdown_connections 2024-07-16T09:59:59.390 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:59.386+0000 7ff325584640 1 --2- 172.21.15.156:0/1052938376 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7ff30806e0b0 0x7ff308070560 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:59:59.390 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:59.386+0000 7ff325584640 1 --2- 172.21.15.156:0/1052938376 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff3200fd630 0x7ff32010d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:59:59.391 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:59.386+0000 7ff325584640 1 --2- 172.21.15.156:0/1052938376 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff3200fc360 0x7ff32010cf80 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:59:59.391 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:59.386+0000 7ff325584640 1 -- 172.21.15.156:0/1052938376 >> 172.21.15.156:0/1052938376 conn(0x7ff3200f7b40 msgr2=0x7ff3200f9230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:59:59.391 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:59.387+0000 7ff325584640 1 -- 172.21.15.156:0/1052938376 shutdown_connections 2024-07-16T09:59:59.391 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:59.387+0000 7ff325584640 1 -- 172.21.15.156:0/1052938376 wait complete. 2024-07-16T09:59:59.471 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:59.469+0000 7fb85f88f640 1 -- 172.21.15.156:0/3884700890 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb8580fc510 msgr2=0x7fb8580fc980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:59:59.471 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:59.469+0000 7fb85f88f640 1 --2- 172.21.15.156:0/3884700890 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb8580fc510 0x7fb8580fc980 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fb84c009c50 tx=0x7fb84c004d10 comp rx=0 tx=0).stop 2024-07-16T09:59:59.472 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:59.469+0000 7fb85f88f640 1 -- 172.21.15.156:0/3884700890 shutdown_connections 2024-07-16T09:59:59.472 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:59.469+0000 7fb85f88f640 1 --2- 172.21.15.156:0/3884700890 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb8580fc510 0x7fb8580fc980 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:59:59.472 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:59.469+0000 7fb85f88f640 1 --2- 172.21.15.156:0/3884700890 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb8580fb240 0x7fb8580fb650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:59:59.472 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:59.469+0000 7fb85f88f640 1 -- 172.21.15.156:0/3884700890 >> 172.21.15.156:0/3884700890 conn(0x7fb8580f6a20 msgr2=0x7fb8580f8e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:59:59.472 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:59.469+0000 7fb85f88f640 1 -- 172.21.15.156:0/3884700890 shutdown_connections 2024-07-16T09:59:59.472 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:59.469+0000 7fb85f88f640 1 -- 172.21.15.156:0/3884700890 wait complete. 2024-07-16T09:59:59.472 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:59.469+0000 7fb85f88f640 1 Processor -- start 2024-07-16T09:59:59.472 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:59.470+0000 7fb85f88f640 1 -- start start 2024-07-16T09:59:59.472 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:59.470+0000 7fb85f88f640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb8580fb240 0x7fb8580ff310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:59:59.472 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:59.470+0000 7fb85f88f640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb8580fc510 0x7fb8580fd850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:59:59.472 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:59.470+0000 7fb85f88f640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8580fdd90 con 0x7fb8580fb240 2024-07-16T09:59:59.472 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:59.470+0000 7fb85f88f640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8580fded0 con 0x7fb8580fc510 2024-07-16T09:59:59.472 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:59.470+0000 7fb85d62b640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb8580fb240 0x7fb8580ff310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:59:59.473 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:59.470+0000 7fb85d62b640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb8580fb240 0x7fb8580ff310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:34484/0 (socket says 172.21.15.156:34484) 2024-07-16T09:59:59.473 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:59.470+0000 7fb85d62b640 1 -- 172.21.15.156:0/3334008025 learned_addr learned my addr 172.21.15.156:0/3334008025 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T09:59:59.473 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:59.470+0000 7fb85ce2a640 1 --2- 172.21.15.156:0/3334008025 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb8580fc510 0x7fb8580fd850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:59:59.473 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:59.470+0000 7fb85d62b640 1 -- 172.21.15.156:0/3334008025 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb8580fc510 msgr2=0x7fb8580fd850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:59:59.473 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:59.470+0000 7fb85d62b640 1 --2- 172.21.15.156:0/3334008025 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb8580fc510 0x7fb8580fd850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:59:59.473 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:59.470+0000 7fb85d62b640 1 -- 172.21.15.156:0/3334008025 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb84c0098b0 con 0x7fb8580fb240 2024-07-16T09:59:59.473 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:59.470+0000 7fb85d62b640 1 --2- 172.21.15.156:0/3334008025 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb8580fb240 0x7fb8580ff310 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fb85000c790 tx=0x7fb8500043c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:59:59.474 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:59.470+0000 7fb855ffb640 1 -- 172.21.15.156:0/3334008025 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb85000a820 con 0x7fb8580fb240 2024-07-16T09:59:59.474 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:59.471+0000 7fb855ffb640 1 -- 172.21.15.156:0/3334008025 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb85000ae50 con 0x7fb8580fb240 2024-07-16T09:59:59.474 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:59.471+0000 7fb85f88f640 1 -- 172.21.15.156:0/3334008025 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8580fe1b0 con 0x7fb8580fb240 2024-07-16T09:59:59.475 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:59.471+0000 7fb855ffb640 1 -- 172.21.15.156:0/3334008025 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb8500115e0 con 0x7fb8580fb240 2024-07-16T09:59:59.475 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:59.471+0000 7fb85f88f640 1 -- 172.21.15.156:0/3334008025 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8580fe6d0 con 0x7fb8580fb240 2024-07-16T09:59:59.475 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:59.472+0000 7fb85f88f640 1 -- 172.21.15.156:0/3334008025 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8580fa6a0 con 0x7fb8580fb240 2024-07-16T09:59:59.475 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:59.472+0000 7fb855ffb640 1 -- 172.21.15.156:0/3334008025 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 38) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fb850012400 con 0x7fb8580fb240 2024-07-16T09:59:59.475 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:59.473+0000 7fb855ffb640 1 --2- 172.21.15.156:0/3334008025 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7fb84006e0b0 0x7fb840070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:59:59.475 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:59.473+0000 7fb855ffb640 1 -- 172.21.15.156:0/3334008025 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fb85008dc30 con 0x7fb8580fb240 2024-07-16T09:59:59.475 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:59.473+0000 7fb85ce2a640 1 --2- 172.21.15.156:0/3334008025 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7fb84006e0b0 0x7fb840070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:59:59.475 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:59.473+0000 7fb85ce2a640 1 --2- 172.21.15.156:0/3334008025 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7fb84006e0b0 0x7fb840070560 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fb84c000f80 tx=0x7fb84c004d10 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:59:59.478 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:59.477+0000 7fb855ffb640 1 -- 172.21.15.156:0/3334008025 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187716 (secure 0 0 0) 0x7fb850057030 con 0x7fb8580fb240 2024-07-16T09:59:59.641 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:59:59 smithi156 ceph-mon[93066]: pgmap v78: 65 pgs: 65 active+clean; 253 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-07-16T09:59:59.641 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:59:59 smithi156 ceph-mon[93066]: from='client.34256 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-16T09:59:59.641 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:59:59 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-07-16T09:59:59.641 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:59:59 smithi156 ceph-mon[93066]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-07-16T09:59:59.641 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:59:59 smithi156 ceph-mon[93066]: Upgrade: osd.2 is safe to restart 2024-07-16T09:59:59.641 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:59:59 smithi156 ceph-mon[93066]: from='client.34260 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-07-16T09:59:59.641 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:59:59 smithi156 ceph-mon[93066]: from='client.? 172.21.15.156:0/4285645158' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T09:59:59.641 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 09:59:59 smithi156 ceph-mon[93066]: from='client.? 172.21.15.156:0/1052938376' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-07-16T09:59:59.679 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:59.678+0000 7fb85f88f640 1 -- 172.21.15.156:0/3334008025 --> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb85805d960 con 0x7fb84006e0b0 2024-07-16T09:59:59.680 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:59.679+0000 7fb855ffb640 1 -- 172.21.15.156:0/3334008025 <== mgr.34104 v2:172.21.15.156:6800/3367765292 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7fb85805d960 con 0x7fb84006e0b0 2024-07-16T09:59:59.681 INFO:teuthology.orchestra.run.smithi156.stdout:{ 2024-07-16T09:59:59.681 INFO:teuthology.orchestra.run.smithi156.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:c15ec3aa8e6c0a9c12ee24ff79f3e7dd4d01b3d1a702dfeb7d7fff9b0811a393", 2024-07-16T09:59:59.681 INFO:teuthology.orchestra.run.smithi156.stdout: "in_progress": true, 2024-07-16T09:59:59.681 INFO:teuthology.orchestra.run.smithi156.stdout: "which": "Upgrading all daemon types on all hosts", 2024-07-16T09:59:59.681 INFO:teuthology.orchestra.run.smithi156.stdout: "services_complete": [ 2024-07-16T09:59:59.681 INFO:teuthology.orchestra.run.smithi156.stdout: "mon", 2024-07-16T09:59:59.681 INFO:teuthology.orchestra.run.smithi156.stdout: "mgr", 2024-07-16T09:59:59.681 INFO:teuthology.orchestra.run.smithi156.stdout: "crash" 2024-07-16T09:59:59.681 INFO:teuthology.orchestra.run.smithi156.stdout: ], 2024-07-16T09:59:59.681 INFO:teuthology.orchestra.run.smithi156.stdout: "progress": "8/21 daemons upgraded", 2024-07-16T09:59:59.681 INFO:teuthology.orchestra.run.smithi156.stdout: "message": "Currently upgrading osd daemons", 2024-07-16T09:59:59.682 INFO:teuthology.orchestra.run.smithi156.stdout: "is_paused": false 2024-07-16T09:59:59.682 INFO:teuthology.orchestra.run.smithi156.stdout:} 2024-07-16T09:59:59.683 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:59.681+0000 7fb85f88f640 1 -- 172.21.15.156:0/3334008025 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7fb84006e0b0 msgr2=0x7fb840070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:59:59.683 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:59.681+0000 7fb85f88f640 1 --2- 172.21.15.156:0/3334008025 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7fb84006e0b0 0x7fb840070560 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fb84c000f80 tx=0x7fb84c004d10 comp rx=0 tx=0).stop 2024-07-16T09:59:59.683 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:59.681+0000 7fb85f88f640 1 -- 172.21.15.156:0/3334008025 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb8580fb240 msgr2=0x7fb8580ff310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:59:59.683 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:59.681+0000 7fb85f88f640 1 --2- 172.21.15.156:0/3334008025 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb8580fb240 0x7fb8580ff310 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fb85000c790 tx=0x7fb8500043c0 comp rx=0 tx=0).stop 2024-07-16T09:59:59.683 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:59.682+0000 7fb85f88f640 1 -- 172.21.15.156:0/3334008025 shutdown_connections 2024-07-16T09:59:59.683 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:59.682+0000 7fb85f88f640 1 --2- 172.21.15.156:0/3334008025 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7fb84006e0b0 0x7fb840070560 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:59:59.684 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:59.682+0000 7fb85f88f640 1 --2- 172.21.15.156:0/3334008025 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb8580fc510 0x7fb8580fd850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:59:59.684 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:59.682+0000 7fb85f88f640 1 --2- 172.21.15.156:0/3334008025 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb8580fb240 0x7fb8580ff310 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:59:59.684 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:59.682+0000 7fb85f88f640 1 -- 172.21.15.156:0/3334008025 >> 172.21.15.156:0/3334008025 conn(0x7fb8580f6a20 msgr2=0x7fb8580653a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:59:59.684 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:59.682+0000 7fb85f88f640 1 -- 172.21.15.156:0/3334008025 shutdown_connections 2024-07-16T09:59:59.684 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:59.682+0000 7fb85f88f640 1 -- 172.21.15.156:0/3334008025 wait complete. 2024-07-16T09:59:59.763 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:59.761+0000 7fabe261c640 1 -- 172.21.15.156:0/2117756965 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fabdc0fd600 msgr2=0x7fabdc0fda70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:59:59.763 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:59.761+0000 7fabe261c640 1 --2- 172.21.15.156:0/2117756965 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fabdc0fd600 0x7fabdc0fda70 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fabd00048d0 tx=0x7fabd0009da0 comp rx=0 tx=0).stop 2024-07-16T09:59:59.763 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:59.761+0000 7fabe261c640 1 -- 172.21.15.156:0/2117756965 shutdown_connections 2024-07-16T09:59:59.763 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:59.761+0000 7fabe261c640 1 --2- 172.21.15.156:0/2117756965 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fabdc0fd600 0x7fabdc0fda70 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:59:59.763 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:59.761+0000 7fabe261c640 1 --2- 172.21.15.156:0/2117756965 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fabdc0fc330 0x7fabdc0fc740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:59:59.763 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:59.761+0000 7fabe261c640 1 -- 172.21.15.156:0/2117756965 >> 172.21.15.156:0/2117756965 conn(0x7fabdc0f7af0 msgr2=0x7fabdc0f9f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T09:59:59.764 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:59.761+0000 7fabe261c640 1 -- 172.21.15.156:0/2117756965 shutdown_connections 2024-07-16T09:59:59.764 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:59.761+0000 7fabe261c640 1 -- 172.21.15.156:0/2117756965 wait complete. 2024-07-16T09:59:59.764 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:59.762+0000 7fabe261c640 1 Processor -- start 2024-07-16T09:59:59.765 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:59.762+0000 7fabe261c640 1 -- start start 2024-07-16T09:59:59.765 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:59.762+0000 7fabe261c640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fabdc0fc330 0x7fabdc10cf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:59:59.765 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:59.762+0000 7fabe261c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fabdc0fd600 0x7fabdc10d490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:59:59.766 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:59.762+0000 7fabe261c640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabdc10dae0 con 0x7fabdc0fd600 2024-07-16T09:59:59.766 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:59.762+0000 7fabe261c640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabdc10e300 con 0x7fabdc0fc330 2024-07-16T09:59:59.766 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:59.763+0000 7fabdb7fe640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fabdc0fd600 0x7fabdc10d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:59:59.766 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:59.763+0000 7fabdb7fe640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fabdc0fd600 0x7fabdc10d490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:34508/0 (socket says 172.21.15.156:34508) 2024-07-16T09:59:59.766 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:59.763+0000 7fabdb7fe640 1 -- 172.21.15.156:0/333135351 learned_addr learned my addr 172.21.15.156:0/333135351 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T09:59:59.766 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:59.763+0000 7fabdbfff640 1 --2- 172.21.15.156:0/333135351 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fabdc0fc330 0x7fabdc10cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:59:59.766 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:59.763+0000 7fabdb7fe640 1 -- 172.21.15.156:0/333135351 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fabdc0fc330 msgr2=0x7fabdc10cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T09:59:59.766 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:59.763+0000 7fabdb7fe640 1 --2- 172.21.15.156:0/333135351 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fabdc0fc330 0x7fabdc10cf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T09:59:59.766 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:59.763+0000 7fabdb7fe640 1 -- 172.21.15.156:0/333135351 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabd0005040 con 0x7fabdc0fd600 2024-07-16T09:59:59.766 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:59.763+0000 7fabdb7fe640 1 --2- 172.21.15.156:0/333135351 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fabdc0fd600 0x7fabdc10d490 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fabd0009f80 tx=0x7fabd0009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:59:59.766 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:59.763+0000 7fabd8ff9640 1 -- 172.21.15.156:0/333135351 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fabd0038070 con 0x7fabdc0fd600 2024-07-16T09:59:59.767 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:59.763+0000 7fabd8ff9640 1 -- 172.21.15.156:0/333135351 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fabd0032970 con 0x7fabdc0fd600 2024-07-16T09:59:59.767 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:59.763+0000 7fabe261c640 1 -- 172.21.15.156:0/333135351 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabdc10e530 con 0x7fabdc0fd600 2024-07-16T09:59:59.767 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:59.763+0000 7fabd8ff9640 1 -- 172.21.15.156:0/333135351 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fabd0033860 con 0x7fabdc0fd600 2024-07-16T09:59:59.767 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:59.763+0000 7fabe261c640 1 -- 172.21.15.156:0/333135351 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabdc10ea70 con 0x7fabdc0fd600 2024-07-16T09:59:59.767 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:59.764+0000 7fabe261c640 1 -- 172.21.15.156:0/333135351 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fabdc0fb790 con 0x7fabdc0fd600 2024-07-16T09:59:59.767 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:59.765+0000 7fabd8ff9640 1 -- 172.21.15.156:0/333135351 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 38) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fabd0030020 con 0x7fabdc0fd600 2024-07-16T09:59:59.768 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:59.765+0000 7fabd8ff9640 1 --2- 172.21.15.156:0/333135351 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7fabc406e0b0 0x7fabc4070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T09:59:59.768 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:59.766+0000 7fabd8ff9640 1 -- 172.21.15.156:0/333135351 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fabd00ada00 con 0x7fabdc0fd600 2024-07-16T09:59:59.768 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:59.766+0000 7fabdbfff640 1 --2- 172.21.15.156:0/333135351 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7fabc406e0b0 0x7fabc4070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T09:59:59.768 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:59.766+0000 7fabdbfff640 1 --2- 172.21.15.156:0/333135351 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7fabc406e0b0 0x7fabc4070560 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fabcc004ed0 tx=0x7fabcc00b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T09:59:59.771 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T09:59:59.770+0000 7fabd8ff9640 1 -- 172.21.15.156:0/333135351 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187716 (secure 0 0 0) 0x7fabd0076e30 con 0x7fabdc0fd600 2024-07-16T09:59:59.843 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:59:59 smithi176 ceph-mon[81689]: pgmap v78: 65 pgs: 65 active+clean; 253 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-07-16T09:59:59.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:59:59 smithi176 ceph-mon[81689]: from='client.34256 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-16T09:59:59.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:59:59 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-07-16T09:59:59.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:59:59 smithi176 ceph-mon[81689]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-07-16T09:59:59.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:59:59 smithi176 ceph-mon[81689]: Upgrade: osd.2 is safe to restart 2024-07-16T09:59:59.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:59:59 smithi176 ceph-mon[81689]: from='client.34260 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-07-16T09:59:59.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:59:59 smithi176 ceph-mon[81689]: from='client.? 172.21.15.156:0/4285645158' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T09:59:59.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 09:59:59 smithi176 ceph-mon[81689]: from='client.? 172.21.15.156:0/1052938376' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-07-16T10:00:00.046 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:00.045+0000 7fabe261c640 1 -- 172.21.15.156:0/333135351 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fabdc10ed50 con 0x7fabdc0fd600 2024-07-16T10:00:00.046 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:00.045+0000 7fabd8ff9640 1 -- 172.21.15.156:0/333135351 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7fabd007a4f0 con 0x7fabdc0fd600 2024-07-16T10:00:00.047 INFO:teuthology.orchestra.run.smithi156.stdout:HEALTH_OK 2024-07-16T10:00:00.050 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:00.048+0000 7fabe261c640 1 -- 172.21.15.156:0/333135351 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7fabc406e0b0 msgr2=0x7fabc4070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:00:00.050 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:00.048+0000 7fabe261c640 1 --2- 172.21.15.156:0/333135351 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7fabc406e0b0 0x7fabc4070560 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fabcc004ed0 tx=0x7fabcc00b040 comp rx=0 tx=0).stop 2024-07-16T10:00:00.050 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:00.049+0000 7fabe261c640 1 -- 172.21.15.156:0/333135351 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fabdc0fd600 msgr2=0x7fabdc10d490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:00:00.050 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:00.049+0000 7fabe261c640 1 --2- 172.21.15.156:0/333135351 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fabdc0fd600 0x7fabdc10d490 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fabd0009f80 tx=0x7fabd0009da0 comp rx=0 tx=0).stop 2024-07-16T10:00:00.051 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:00.049+0000 7fabe261c640 1 -- 172.21.15.156:0/333135351 shutdown_connections 2024-07-16T10:00:00.051 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:00.049+0000 7fabe261c640 1 --2- 172.21.15.156:0/333135351 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7fabc406e0b0 0x7fabc4070560 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:00:00.051 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:00.049+0000 7fabe261c640 1 --2- 172.21.15.156:0/333135351 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fabdc0fd600 0x7fabdc10d490 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:00:00.051 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:00.049+0000 7fabe261c640 1 --2- 172.21.15.156:0/333135351 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fabdc0fc330 0x7fabdc10cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:00:00.051 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:00.049+0000 7fabe261c640 1 -- 172.21.15.156:0/333135351 >> 172.21.15.156:0/333135351 conn(0x7fabdc0f7af0 msgr2=0x7fabdc0f91c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T10:00:00.051 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:00.049+0000 7fabe261c640 1 -- 172.21.15.156:0/333135351 shutdown_connections 2024-07-16T10:00:00.051 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:00.049+0000 7fabe261c640 1 -- 172.21.15.156:0/333135351 wait complete. 2024-07-16T10:00:01.306 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:00:00 smithi156 ceph-mon[93066]: from='client.34272 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-16T10:00:01.306 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:00:00 smithi156 ceph-mon[93066]: Upgrade: Updating osd.2 2024-07-16T10:00:01.306 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:00:00 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:00:01.306 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:00:00 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-07-16T10:00:01.307 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:00:00 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-16T10:00:01.307 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:00:00 smithi156 ceph-mon[93066]: Deploying daemon osd.2 on smithi156 2024-07-16T10:00:01.307 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:00:00 smithi156 ceph-mon[93066]: overall HEALTH_OK 2024-07-16T10:00:01.307 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:00:00 smithi156 ceph-mon[93066]: from='client.? 172.21.15.156:0/333135351' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-07-16T10:00:01.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:00:00 smithi176 ceph-mon[81689]: from='client.34272 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-16T10:00:01.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:00:00 smithi176 ceph-mon[81689]: Upgrade: Updating osd.2 2024-07-16T10:00:01.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:00:00 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:00:01.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:00:00 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-07-16T10:00:01.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:00:00 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-16T10:00:01.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:00:00 smithi176 ceph-mon[81689]: Deploying daemon osd.2 on smithi156 2024-07-16T10:00:01.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:00:00 smithi176 ceph-mon[81689]: overall HEALTH_OK 2024-07-16T10:00:01.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:00:00 smithi176 ceph-mon[81689]: from='client.? 172.21.15.156:0/333135351' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-07-16T10:00:02.174 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:00:01 smithi156 ceph-mon[93066]: pgmap v79: 65 pgs: 65 active+clean; 253 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-07-16T10:00:02.245 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:00:01 smithi176 ceph-mon[81689]: pgmap v79: 65 pgs: 65 active+clean; 253 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-07-16T10:00:03.169 INFO:journalctl@ceph.osd.2.smithi156.stdout:Jul 16 10:00:02 smithi156 systemd[1]: Stopping Ceph osd.2 for 26c91c26-4358-11ef-bcad-c7b262605968... 2024-07-16T10:00:03.424 INFO:journalctl@ceph.osd.2.smithi156.stdout:Jul 16 10:00:03 smithi156 ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-2[62327]: 2024-07-16T10:00:03.167+0000 7fbb8ce39640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-07-16T10:00:03.424 INFO:journalctl@ceph.osd.2.smithi156.stdout:Jul 16 10:00:03 smithi156 ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-2[62327]: 2024-07-16T10:00:03.167+0000 7fbb8ce39640 -1 osd.2 59 *** Got signal Terminated *** 2024-07-16T10:00:03.425 INFO:journalctl@ceph.osd.2.smithi156.stdout:Jul 16 10:00:03 smithi156 ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-2[62327]: 2024-07-16T10:00:03.167+0000 7fbb8ce39640 -1 osd.2 59 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-07-16T10:00:04.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:00:03 smithi176 ceph-mon[81689]: pgmap v80: 65 pgs: 65 active+clean; 253 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-07-16T10:00:04.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:00:03 smithi176 ceph-mon[81689]: osd.2 marked itself down and dead 2024-07-16T10:00:04.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:00:03 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-16T10:00:04.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:00:03 smithi156 ceph-mon[93066]: pgmap v80: 65 pgs: 65 active+clean; 253 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-07-16T10:00:04.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:00:03 smithi156 ceph-mon[93066]: osd.2 marked itself down and dead 2024-07-16T10:00:04.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:00:03 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-16T10:00:04.811 INFO:journalctl@ceph.osd.2.smithi156.stdout:Jul 16 10:00:04 smithi156 podman[111972]: 2024-07-16 10:00:04.516996693 +0000 UTC m=+1.447643417 container died f28acec5cb79beea69ec8a8e84537d1d0806ac3428d11a947c9ec365f1a30a22 (image=quay.ceph.io/ceph-ci/ceph@sha256:7b86ef6bffe0efc5f1aefc9fef8c854a67a511c50993acee3ac684f23bde83df, name=ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-2, io.buildah.version=1.36.0, RELEASE=quincy-e6231b0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240709, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-07-16T10:00:05.095 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:00:04 smithi156 ceph-mon[93066]: Health check failed: 1 osds down (OSD_DOWN) 2024-07-16T10:00:05.095 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:00:04 smithi156 ceph-mon[93066]: osdmap e60: 6 total, 5 up, 6 in 2024-07-16T10:00:05.095 INFO:journalctl@ceph.osd.2.smithi156.stdout:Jul 16 10:00:04 smithi156 podman[111972]: 2024-07-16 10:00:04.909276604 +0000 UTC m=+1.839923320 container cleanup f28acec5cb79beea69ec8a8e84537d1d0806ac3428d11a947c9ec365f1a30a22 (image=quay.ceph.io/ceph-ci/ceph@sha256:7b86ef6bffe0efc5f1aefc9fef8c854a67a511c50993acee3ac684f23bde83df, name=ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-2, maintainer=Guillaume Abrioux , io.buildah.version=1.36.0, GIT_CLEAN=True, org.label-schema.build-date=20240709, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, RELEASE=quincy-e6231b0, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image) 2024-07-16T10:00:05.095 INFO:journalctl@ceph.osd.2.smithi156.stdout:Jul 16 10:00:04 smithi156 bash[111972]: ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-2 2024-07-16T10:00:05.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:00:04 smithi176 ceph-mon[81689]: Health check failed: 1 osds down (OSD_DOWN) 2024-07-16T10:00:05.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:00:04 smithi176 ceph-mon[81689]: osdmap e60: 6 total, 5 up, 6 in 2024-07-16T10:00:05.424 INFO:journalctl@ceph.osd.2.smithi156.stdout:Jul 16 10:00:05 smithi156 podman[111989]: 2024-07-16 10:00:05.094108351 +0000 UTC m=+0.570417201 container remove f28acec5cb79beea69ec8a8e84537d1d0806ac3428d11a947c9ec365f1a30a22 (image=quay.ceph.io/ceph-ci/ceph@sha256:7b86ef6bffe0efc5f1aefc9fef8c854a67a511c50993acee3ac684f23bde83df, name=ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240709, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, RELEASE=quincy-e6231b0, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux ) 2024-07-16T10:00:05.765 INFO:journalctl@ceph.osd.2.smithi156.stdout:Jul 16 10:00:05 smithi156 podman[112075]: 2024-07-16 10:00:05.406981207 +0000 UTC m=+0.017661613 image pull 16f7feb043422ce37a2471403d4bc52f550d49cef4b677482112fcc0db3f67ab quay.ceph.io/ceph-ci/ceph@sha256:c15ec3aa8e6c0a9c12ee24ff79f3e7dd4d01b3d1a702dfeb7d7fff9b0811a393 2024-07-16T10:00:05.765 INFO:journalctl@ceph.osd.2.smithi156.stdout:Jul 16 10:00:05 smithi156 podman[112075]: 2024-07-16 10:00:05.520231092 +0000 UTC m=+0.130911497 container create 1d99384f9108f5d164619df3e601d18b8a9a4588afaec9da4d6a84f7edaa2cfa (image=quay.ceph.io/ceph-ci/ceph@sha256:c15ec3aa8e6c0a9c12ee24ff79f3e7dd4d01b3d1a702dfeb7d7fff9b0811a393, name=ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-2-deactivate, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240709, org.label-schema.vendor=CentOS, RELEASE=wip-adk4-testing-2024-07-15-1832-squid-a6ca461, GIT_CLEAN=True, ceph=True, io.buildah.version=1.36.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0) 2024-07-16T10:00:05.765 INFO:journalctl@ceph.osd.2.smithi156.stdout:Jul 16 10:00:05 smithi156 podman[112075]: 2024-07-16 10:00:05.76444191 +0000 UTC m=+0.375122326 container init 1d99384f9108f5d164619df3e601d18b8a9a4588afaec9da4d6a84f7edaa2cfa (image=quay.ceph.io/ceph-ci/ceph@sha256:c15ec3aa8e6c0a9c12ee24ff79f3e7dd4d01b3d1a702dfeb7d7fff9b0811a393, name=ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-2-deactivate, org.label-schema.build-date=20240709, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , RELEASE=wip-adk4-testing-2024-07-15-1832-squid-a6ca461, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.36.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-07-16T10:00:06.175 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:00:05 smithi156 ceph-mon[93066]: pgmap v82: 65 pgs: 12 peering, 7 stale+active+clean, 46 active+clean; 253 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 1.4 KiB/s rd, 2 op/s 2024-07-16T10:00:06.175 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:00:05 smithi156 ceph-mon[93066]: Health check failed: Reduced data availability: 1 pg inactive, 2 pgs peering (PG_AVAILABILITY) 2024-07-16T10:00:06.175 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:00:05 smithi156 ceph-mon[93066]: osdmap e61: 6 total, 5 up, 6 in 2024-07-16T10:00:06.175 INFO:journalctl@ceph.osd.2.smithi156.stdout:Jul 16 10:00:05 smithi156 podman[112075]: 2024-07-16 10:00:05.76772791 +0000 UTC m=+0.378408321 container start 1d99384f9108f5d164619df3e601d18b8a9a4588afaec9da4d6a84f7edaa2cfa (image=quay.ceph.io/ceph-ci/ceph@sha256:c15ec3aa8e6c0a9c12ee24ff79f3e7dd4d01b3d1a702dfeb7d7fff9b0811a393, name=ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-2-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240709, io.buildah.version=1.36.0, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-adk4-testing-2024-07-15-1832-squid-a6ca461) 2024-07-16T10:00:06.175 INFO:journalctl@ceph.osd.2.smithi156.stdout:Jul 16 10:00:05 smithi156 podman[112075]: 2024-07-16 10:00:05.847701705 +0000 UTC m=+0.458382127 container attach 1d99384f9108f5d164619df3e601d18b8a9a4588afaec9da4d6a84f7edaa2cfa (image=quay.ceph.io/ceph-ci/ceph@sha256:c15ec3aa8e6c0a9c12ee24ff79f3e7dd4d01b3d1a702dfeb7d7fff9b0811a393, name=ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-2-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, RELEASE=wip-adk4-testing-2024-07-15-1832-squid-a6ca461, org.label-schema.build-date=20240709, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-07-16T10:00:06.175 INFO:journalctl@ceph.osd.2.smithi156.stdout:Jul 16 10:00:05 smithi156 podman[112075]: 2024-07-16 10:00:05.956308672 +0000 UTC m=+0.566989078 container died 1d99384f9108f5d164619df3e601d18b8a9a4588afaec9da4d6a84f7edaa2cfa (image=quay.ceph.io/ceph-ci/ceph@sha256:c15ec3aa8e6c0a9c12ee24ff79f3e7dd4d01b3d1a702dfeb7d7fff9b0811a393, name=ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-2-deactivate, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240709, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.36.0, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-adk4-testing-2024-07-15-1832-squid-a6ca461, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.license=GPLv2) 2024-07-16T10:00:06.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:00:05 smithi176 ceph-mon[81689]: pgmap v82: 65 pgs: 12 peering, 7 stale+active+clean, 46 active+clean; 253 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 1.4 KiB/s rd, 2 op/s 2024-07-16T10:00:06.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:00:05 smithi176 ceph-mon[81689]: Health check failed: Reduced data availability: 1 pg inactive, 2 pgs peering (PG_AVAILABILITY) 2024-07-16T10:00:06.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:00:05 smithi176 ceph-mon[81689]: osdmap e61: 6 total, 5 up, 6 in 2024-07-16T10:00:06.925 INFO:journalctl@ceph.osd.2.smithi156.stdout:Jul 16 10:00:06 smithi156 podman[112098]: 2024-07-16 10:00:06.534950829 +0000 UTC m=+0.568763745 container remove 1d99384f9108f5d164619df3e601d18b8a9a4588afaec9da4d6a84f7edaa2cfa (image=quay.ceph.io/ceph-ci/ceph@sha256:c15ec3aa8e6c0a9c12ee24ff79f3e7dd4d01b3d1a702dfeb7d7fff9b0811a393, name=ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-2-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0, RELEASE=wip-adk4-testing-2024-07-15-1832-squid-a6ca461, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240709, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-07-16T10:00:06.925 INFO:journalctl@ceph.osd.2.smithi156.stdout:Jul 16 10:00:06 smithi156 systemd[1]: ceph-26c91c26-4358-11ef-bcad-c7b262605968@osd.2.service: Deactivated successfully. 2024-07-16T10:00:06.925 INFO:journalctl@ceph.osd.2.smithi156.stdout:Jul 16 10:00:06 smithi156 systemd[1]: Stopped Ceph osd.2 for 26c91c26-4358-11ef-bcad-c7b262605968. 2024-07-16T10:00:06.925 INFO:journalctl@ceph.osd.2.smithi156.stdout:Jul 16 10:00:06 smithi156 systemd[1]: ceph-26c91c26-4358-11ef-bcad-c7b262605968@osd.2.service: Consumed 54.041s CPU time. 2024-07-16T10:00:06.925 INFO:journalctl@ceph.osd.2.smithi156.stdout:Jul 16 10:00:06 smithi156 systemd[1]: Starting Ceph osd.2 for 26c91c26-4358-11ef-bcad-c7b262605968... 2024-07-16T10:00:07.255 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:00:06 smithi156 ceph-mon[93066]: Health check failed: Degraded data redundancy: 13/318 objects degraded (4.088%), 3 pgs degraded (PG_DEGRADED) 2024-07-16T10:00:07.256 INFO:journalctl@ceph.osd.2.smithi156.stdout:Jul 16 10:00:06 smithi156 podman[112199]: 2024-07-16 10:00:06.886840779 +0000 UTC m=+0.017658060 image pull 16f7feb043422ce37a2471403d4bc52f550d49cef4b677482112fcc0db3f67ab quay.ceph.io/ceph-ci/ceph@sha256:c15ec3aa8e6c0a9c12ee24ff79f3e7dd4d01b3d1a702dfeb7d7fff9b0811a393 2024-07-16T10:00:07.256 INFO:journalctl@ceph.osd.2.smithi156.stdout:Jul 16 10:00:06 smithi156 podman[112199]: 2024-07-16 10:00:06.994529809 +0000 UTC m=+0.125347076 container create db4b0c401def3342f346992c0ef1e0dba33dc5da69346cfb3757db427315b5b4 (image=quay.ceph.io/ceph-ci/ceph@sha256:c15ec3aa8e6c0a9c12ee24ff79f3e7dd4d01b3d1a702dfeb7d7fff9b0811a393, name=ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-2-activate, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-adk4-testing-2024-07-15-1832-squid-a6ca461, org.label-schema.schema-version=1.0, io.buildah.version=1.36.0, GIT_CLEAN=True, org.label-schema.build-date=20240709, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-07-16T10:00:07.256 INFO:journalctl@ceph.osd.2.smithi156.stdout:Jul 16 10:00:07 smithi156 podman[112199]: 2024-07-16 10:00:07.255481967 +0000 UTC m=+0.386299235 container init db4b0c401def3342f346992c0ef1e0dba33dc5da69346cfb3757db427315b5b4 (image=quay.ceph.io/ceph-ci/ceph@sha256:c15ec3aa8e6c0a9c12ee24ff79f3e7dd4d01b3d1a702dfeb7d7fff9b0811a393, name=ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-2-activate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.36.0, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240709, RELEASE=wip-adk4-testing-2024-07-15-1832-squid-a6ca461) 2024-07-16T10:00:07.343 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:00:06 smithi176 ceph-mon[81689]: Health check failed: Degraded data redundancy: 13/318 objects degraded (4.088%), 3 pgs degraded (PG_DEGRADED) 2024-07-16T10:00:07.601 INFO:journalctl@ceph.osd.2.smithi156.stdout:Jul 16 10:00:07 smithi156 podman[112199]: 2024-07-16 10:00:07.260196509 +0000 UTC m=+0.391013794 container start db4b0c401def3342f346992c0ef1e0dba33dc5da69346cfb3757db427315b5b4 (image=quay.ceph.io/ceph-ci/ceph@sha256:c15ec3aa8e6c0a9c12ee24ff79f3e7dd4d01b3d1a702dfeb7d7fff9b0811a393, name=ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-2-activate, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=wip-adk4-testing-2024-07-15-1832-squid-a6ca461, org.label-schema.build-date=20240709, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.36.0, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD) 2024-07-16T10:00:07.601 INFO:journalctl@ceph.osd.2.smithi156.stdout:Jul 16 10:00:07 smithi156 podman[112199]: 2024-07-16 10:00:07.363928985 +0000 UTC m=+0.494746253 container attach db4b0c401def3342f346992c0ef1e0dba33dc5da69346cfb3757db427315b5b4 (image=quay.ceph.io/ceph-ci/ceph@sha256:c15ec3aa8e6c0a9c12ee24ff79f3e7dd4d01b3d1a702dfeb7d7fff9b0811a393, name=ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-2-activate, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-adk4-testing-2024-07-15-1832-squid-a6ca461, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240709, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , io.buildah.version=1.36.0, GIT_CLEAN=True) 2024-07-16T10:00:07.601 INFO:journalctl@ceph.osd.2.smithi156.stdout:Jul 16 10:00:07 smithi156 ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-2-activate[112213]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-07-16T10:00:07.602 INFO:journalctl@ceph.osd.2.smithi156.stdout:Jul 16 10:00:07 smithi156 bash[112199]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-07-16T10:00:07.602 INFO:journalctl@ceph.osd.2.smithi156.stdout:Jul 16 10:00:07 smithi156 ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-2-activate[112213]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-07-16T10:00:07.602 INFO:journalctl@ceph.osd.2.smithi156.stdout:Jul 16 10:00:07 smithi156 bash[112199]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-07-16T10:00:07.602 INFO:journalctl@ceph.osd.2.smithi156.stdout:Jul 16 10:00:07 smithi156 ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-2-activate[112213]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-07-16T10:00:07.602 INFO:journalctl@ceph.osd.2.smithi156.stdout:Jul 16 10:00:07 smithi156 bash[112199]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-07-16T10:00:07.925 INFO:journalctl@ceph.osd.2.smithi156.stdout:Jul 16 10:00:07 smithi156 ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-2-activate[112213]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-2 --no-mon-config --dev /dev/mapper/vg_nvme-lv_2 2024-07-16T10:00:07.925 INFO:journalctl@ceph.osd.2.smithi156.stdout:Jul 16 10:00:07 smithi156 bash[112199]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-2 --no-mon-config --dev /dev/mapper/vg_nvme-lv_2 2024-07-16T10:00:07.925 INFO:journalctl@ceph.osd.2.smithi156.stdout:Jul 16 10:00:07 smithi156 ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-2-activate[112213]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2024-07-16T10:00:07.925 INFO:journalctl@ceph.osd.2.smithi156.stdout:Jul 16 10:00:07 smithi156 bash[112199]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2024-07-16T10:00:07.925 INFO:journalctl@ceph.osd.2.smithi156.stdout:Jul 16 10:00:07 smithi156 ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-2-activate[112213]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-07-16T10:00:07.925 INFO:journalctl@ceph.osd.2.smithi156.stdout:Jul 16 10:00:07 smithi156 bash[112199]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-07-16T10:00:07.925 INFO:journalctl@ceph.osd.2.smithi156.stdout:Jul 16 10:00:07 smithi156 ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-2-activate[112213]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-2/block 2024-07-16T10:00:07.925 INFO:journalctl@ceph.osd.2.smithi156.stdout:Jul 16 10:00:07 smithi156 bash[112199]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-2/block 2024-07-16T10:00:07.926 INFO:journalctl@ceph.osd.2.smithi156.stdout:Jul 16 10:00:07 smithi156 ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-2-activate[112213]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-07-16T10:00:07.926 INFO:journalctl@ceph.osd.2.smithi156.stdout:Jul 16 10:00:07 smithi156 bash[112199]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-07-16T10:00:07.926 INFO:journalctl@ceph.osd.2.smithi156.stdout:Jul 16 10:00:07 smithi156 ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-2-activate[112213]: --> ceph-volume raw activate successful for osd ID: 2 2024-07-16T10:00:07.926 INFO:journalctl@ceph.osd.2.smithi156.stdout:Jul 16 10:00:07 smithi156 bash[112199]: --> ceph-volume raw activate successful for osd ID: 2 2024-07-16T10:00:07.926 INFO:journalctl@ceph.osd.2.smithi156.stdout:Jul 16 10:00:07 smithi156 podman[112199]: 2024-07-16 10:00:07.649146846 +0000 UTC m=+0.779964125 container died db4b0c401def3342f346992c0ef1e0dba33dc5da69346cfb3757db427315b5b4 (image=quay.ceph.io/ceph-ci/ceph@sha256:c15ec3aa8e6c0a9c12ee24ff79f3e7dd4d01b3d1a702dfeb7d7fff9b0811a393, name=ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-2-activate, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240709, RELEASE=wip-adk4-testing-2024-07-15-1832-squid-a6ca461, io.buildah.version=1.36.0, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux ) 2024-07-16T10:00:08.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:00:07 smithi176 ceph-mon[81689]: pgmap v84: 65 pgs: 4 active+undersized, 12 peering, 3 stale+active+clean, 3 active+undersized+degraded, 43 active+clean; 253 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 13/318 objects degraded (4.088%) 2024-07-16T10:00:08.424 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:00:07 smithi156 ceph-mon[93066]: pgmap v84: 65 pgs: 4 active+undersized, 12 peering, 3 stale+active+clean, 3 active+undersized+degraded, 43 active+clean; 253 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 13/318 objects degraded (4.088%) 2024-07-16T10:00:08.425 INFO:journalctl@ceph.osd.2.smithi156.stdout:Jul 16 10:00:08 smithi156 podman[112355]: 2024-07-16 10:00:08.22651807 +0000 UTC m=+0.571535322 container remove db4b0c401def3342f346992c0ef1e0dba33dc5da69346cfb3757db427315b5b4 (image=quay.ceph.io/ceph-ci/ceph@sha256:c15ec3aa8e6c0a9c12ee24ff79f3e7dd4d01b3d1a702dfeb7d7fff9b0811a393, name=ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-2-activate, RELEASE=wip-adk4-testing-2024-07-15-1832-squid-a6ca461, io.buildah.version=1.36.0, org.label-schema.build-date=20240709, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image) 2024-07-16T10:00:08.697 INFO:journalctl@ceph.osd.2.smithi156.stdout:Jul 16 10:00:08 smithi156 podman[112413]: 2024-07-16 10:00:08.343237075 +0000 UTC m=+0.017603080 image pull 16f7feb043422ce37a2471403d4bc52f550d49cef4b677482112fcc0db3f67ab quay.ceph.io/ceph-ci/ceph@sha256:c15ec3aa8e6c0a9c12ee24ff79f3e7dd4d01b3d1a702dfeb7d7fff9b0811a393 2024-07-16T10:00:08.697 INFO:journalctl@ceph.osd.2.smithi156.stdout:Jul 16 10:00:08 smithi156 podman[112413]: 2024-07-16 10:00:08.456384238 +0000 UTC m=+0.130750225 container create c6ecf18de9bfff8ebce934b708f7a97c99b90a23559675f76cf95efaf7ce61ba (image=quay.ceph.io/ceph-ci/ceph@sha256:c15ec3aa8e6c0a9c12ee24ff79f3e7dd4d01b3d1a702dfeb7d7fff9b0811a393, name=ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-adk4-testing-2024-07-15-1832-squid-a6ca461, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, io.buildah.version=1.36.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240709, CEPH_POINT_RELEASE=) 2024-07-16T10:00:08.697 INFO:journalctl@ceph.osd.2.smithi156.stdout:Jul 16 10:00:08 smithi156 podman[112413]: 2024-07-16 10:00:08.67180427 +0000 UTC m=+0.346170269 container init c6ecf18de9bfff8ebce934b708f7a97c99b90a23559675f76cf95efaf7ce61ba (image=quay.ceph.io/ceph-ci/ceph@sha256:c15ec3aa8e6c0a9c12ee24ff79f3e7dd4d01b3d1a702dfeb7d7fff9b0811a393, name=ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.36.0, org.label-schema.build-date=20240709, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-adk4-testing-2024-07-15-1832-squid-a6ca461) 2024-07-16T10:00:08.697 INFO:journalctl@ceph.osd.2.smithi156.stdout:Jul 16 10:00:08 smithi156 podman[112413]: 2024-07-16 10:00:08.675376716 +0000 UTC m=+0.349742705 container start c6ecf18de9bfff8ebce934b708f7a97c99b90a23559675f76cf95efaf7ce61ba (image=quay.ceph.io/ceph-ci/ceph@sha256:c15ec3aa8e6c0a9c12ee24ff79f3e7dd4d01b3d1a702dfeb7d7fff9b0811a393, name=ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240709, GIT_BRANCH=HEAD, RELEASE=wip-adk4-testing-2024-07-15-1832-squid-a6ca461, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-07-16T10:00:09.174 INFO:journalctl@ceph.osd.2.smithi156.stdout:Jul 16 10:00:08 smithi156 ceph-osd[112431]: -- 172.21.15.156:0/2761455713 <== mon.0 v2:172.21.15.156:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d832c981e0 con 0x55d831e4bc00 2024-07-16T10:00:09.175 INFO:journalctl@ceph.osd.2.smithi156.stdout:Jul 16 10:00:08 smithi156 bash[112413]: c6ecf18de9bfff8ebce934b708f7a97c99b90a23559675f76cf95efaf7ce61ba 2024-07-16T10:00:09.175 INFO:journalctl@ceph.osd.2.smithi156.stdout:Jul 16 10:00:08 smithi156 systemd[1]: Started Ceph osd.2 for 26c91c26-4358-11ef-bcad-c7b262605968. 2024-07-16T10:00:09.489 INFO:journalctl@ceph.osd.2.smithi156.stdout:Jul 16 10:00:09 smithi156 ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-2[112427]: 2024-07-16T10:00:09.221+0000 7fe54909d640 -1 Falling back to public interface 2024-07-16T10:00:10.093 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:00:09 smithi176 ceph-mon[81689]: pgmap v85: 65 pgs: 8 active+undersized, 12 peering, 7 active+undersized+degraded, 38 active+clean; 253 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 24/318 objects degraded (7.547%) 2024-07-16T10:00:10.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:00:09 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:00:10.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:00:09 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:00:10.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:00:09 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-16T10:00:10.175 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:00:09 smithi156 ceph-mon[93066]: pgmap v85: 65 pgs: 8 active+undersized, 12 peering, 7 active+undersized+degraded, 38 active+clean; 253 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 24/318 objects degraded (7.547%) 2024-07-16T10:00:10.175 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:00:09 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:00:10.175 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:00:09 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:00:10.175 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:00:09 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-16T10:00:11.174 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:00:10 smithi156 ceph-mon[93066]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 2 pgs peering) 2024-07-16T10:00:11.343 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:00:10 smithi176 ceph-mon[81689]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 2 pgs peering) 2024-07-16T10:00:12.174 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:00:11 smithi156 ceph-mon[93066]: pgmap v86: 65 pgs: 15 active+undersized, 12 active+undersized+degraded, 38 active+clean; 253 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 37/318 objects degraded (11.635%) 2024-07-16T10:00:12.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:00:11 smithi176 ceph-mon[81689]: pgmap v86: 65 pgs: 15 active+undersized, 12 active+undersized+degraded, 38 active+clean; 253 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 37/318 objects degraded (11.635%) 2024-07-16T10:00:14.174 INFO:journalctl@ceph.osd.2.smithi156.stdout:Jul 16 10:00:13 smithi156 ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-2[112427]: 2024-07-16T10:00:13.775+0000 7fe54909d640 -1 osd.2 59 log_to_monitors true 2024-07-16T10:00:14.175 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:00:13 smithi156 ceph-mon[93066]: pgmap v87: 65 pgs: 15 active+undersized, 12 active+undersized+degraded, 38 active+clean; 253 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 37/318 objects degraded (11.635%) 2024-07-16T10:00:14.175 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:00:13 smithi156 ceph-mon[93066]: from='osd.2 [v2:172.21.15.156:6818/1034795579,v1:172.21.15.156:6819/1034795579]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-07-16T10:00:14.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:00:13 smithi176 ceph-mon[81689]: pgmap v87: 65 pgs: 15 active+undersized, 12 active+undersized+degraded, 38 active+clean; 253 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 37/318 objects degraded (11.635%) 2024-07-16T10:00:14.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:00:13 smithi176 ceph-mon[81689]: from='osd.2 [v2:172.21.15.156:6818/1034795579,v1:172.21.15.156:6819/1034795579]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-07-16T10:00:14.754 INFO:journalctl@ceph.osd.2.smithi156.stdout:Jul 16 10:00:14 smithi156 ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-2[112427]: 2024-07-16T10:00:14.646+0000 7fe53ce2f640 -1 osd.2 59 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-07-16T10:00:15.174 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:00:14 smithi156 ceph-mon[93066]: from='osd.2 [v2:172.21.15.156:6818/1034795579,v1:172.21.15.156:6819/1034795579]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-07-16T10:00:15.175 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:00:14 smithi156 ceph-mon[93066]: osdmap e62: 6 total, 5 up, 6 in 2024-07-16T10:00:15.175 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:00:14 smithi156 ceph-mon[93066]: from='osd.2 [v2:172.21.15.156:6818/1034795579,v1:172.21.15.156:6819/1034795579]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2024-07-16T10:00:15.175 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:00:14 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:00:15.175 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:00:14 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:00:15.175 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:00:14 smithi156 ceph-mon[93066]: from='osd.2 [v2:172.21.15.156:6818/1034795579,v1:172.21.15.156:6819/1034795579]' entity='osd.2' 2024-07-16T10:00:15.175 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:00:14 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:00:15.175 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:00:14 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:00:15.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:00:14 smithi176 ceph-mon[81689]: from='osd.2 [v2:172.21.15.156:6818/1034795579,v1:172.21.15.156:6819/1034795579]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-07-16T10:00:15.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:00:14 smithi176 ceph-mon[81689]: osdmap e62: 6 total, 5 up, 6 in 2024-07-16T10:00:15.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:00:14 smithi176 ceph-mon[81689]: from='osd.2 [v2:172.21.15.156:6818/1034795579,v1:172.21.15.156:6819/1034795579]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2024-07-16T10:00:15.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:00:14 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:00:15.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:00:14 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:00:15.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:00:14 smithi176 ceph-mon[81689]: from='osd.2 [v2:172.21.15.156:6818/1034795579,v1:172.21.15.156:6819/1034795579]' entity='osd.2' 2024-07-16T10:00:15.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:00:14 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:00:15.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:00:14 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:00:16.311 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:00:15 smithi156 ceph-mon[93066]: pgmap v89: 65 pgs: 15 active+undersized, 12 active+undersized+degraded, 38 active+clean; 253 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 37/318 objects degraded (11.635%) 2024-07-16T10:00:16.312 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:00:15 smithi156 ceph-mon[93066]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-07-16T10:00:16.312 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:00:15 smithi156 ceph-mon[93066]: osd.2 [v2:172.21.15.156:6818/1034795579,v1:172.21.15.156:6819/1034795579] boot 2024-07-16T10:00:16.312 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:00:15 smithi156 ceph-mon[93066]: osdmap e63: 6 total, 6 up, 6 in 2024-07-16T10:00:16.312 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:00:15 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-07-16T10:00:16.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:00:15 smithi176 ceph-mon[81689]: pgmap v89: 65 pgs: 15 active+undersized, 12 active+undersized+degraded, 38 active+clean; 253 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 37/318 objects degraded (11.635%) 2024-07-16T10:00:16.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:00:15 smithi176 ceph-mon[81689]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-07-16T10:00:16.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:00:15 smithi176 ceph-mon[81689]: osd.2 [v2:172.21.15.156:6818/1034795579,v1:172.21.15.156:6819/1034795579] boot 2024-07-16T10:00:16.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:00:15 smithi176 ceph-mon[81689]: osdmap e63: 6 total, 6 up, 6 in 2024-07-16T10:00:16.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:00:15 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-07-16T10:00:17.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:00:17 smithi156 ceph-mon[93066]: pgmap v91: 65 pgs: 15 active+undersized, 12 active+undersized+degraded, 38 active+clean; 253 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.4 KiB/s rd, 4 op/s; 37/318 objects degraded (11.635%) 2024-07-16T10:00:17.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:00:17 smithi156 ceph-mon[93066]: osdmap e64: 6 total, 6 up, 6 in 2024-07-16T10:00:17.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:00:17 smithi156 ceph-mon[93066]: Health check update: Degraded data redundancy: 37/318 objects degraded (11.635%), 12 pgs degraded (PG_DEGRADED) 2024-07-16T10:00:18.093 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:00:17 smithi176 ceph-mon[81689]: pgmap v91: 65 pgs: 15 active+undersized, 12 active+undersized+degraded, 38 active+clean; 253 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.4 KiB/s rd, 4 op/s; 37/318 objects degraded (11.635%) 2024-07-16T10:00:18.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:00:17 smithi176 ceph-mon[81689]: osdmap e64: 6 total, 6 up, 6 in 2024-07-16T10:00:18.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:00:17 smithi176 ceph-mon[81689]: Health check update: Degraded data redundancy: 37/318 objects degraded (11.635%), 12 pgs degraded (PG_DEGRADED) 2024-07-16T10:00:19.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:00:19 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:00:19.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:00:19 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:00:19.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:00:19 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-16T10:00:19.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:00:19 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-16T10:00:19.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:00:19 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:00:19.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:00:19 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-16T10:00:19.345 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:00:19 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:00:19.345 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:00:19 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:00:19.345 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:00:19 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:00:19.345 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:00:19 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-07-16T10:00:19.345 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:00:19 smithi176 ceph-mon[81689]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-07-16T10:00:19.345 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:00:19 smithi176 ceph-mon[81689]: Upgrade: unsafe to stop osd(s) at this time (11 PGs are or would become offline) 2024-07-16T10:00:19.345 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:00:19 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:00:19.345 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:00:19 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-16T10:00:19.426 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:00:19 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:00:19.426 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:00:19 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:00:19.426 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:00:19 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-16T10:00:19.426 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:00:19 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-16T10:00:19.426 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:00:19 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:00:19.426 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:00:19 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-16T10:00:19.426 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:00:19 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:00:19.426 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:00:19 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:00:19.426 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:00:19 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:00:19.427 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:00:19 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-07-16T10:00:19.427 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:00:19 smithi156 ceph-mon[93066]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-07-16T10:00:19.427 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:00:19 smithi156 ceph-mon[93066]: Upgrade: unsafe to stop osd(s) at this time (11 PGs are or would become offline) 2024-07-16T10:00:19.427 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:00:19 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:00:19.427 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:00:19 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-16T10:00:20.343 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:00:20 smithi176 ceph-mon[81689]: pgmap v93: 65 pgs: 5 peering, 12 active+undersized, 10 active+undersized+degraded, 38 active+clean; 253 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s; 29/318 objects degraded (9.119%) 2024-07-16T10:00:20.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:00:20 smithi156 ceph-mon[93066]: pgmap v93: 65 pgs: 5 peering, 12 active+undersized, 10 active+undersized+degraded, 38 active+clean; 253 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s; 29/318 objects degraded (9.119%) 2024-07-16T10:00:22.343 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:00:22 smithi176 ceph-mon[81689]: pgmap v94: 65 pgs: 5 peering, 2 active+undersized, 1 active+undersized+degraded, 57 active+clean; 253 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s; 3/318 objects degraded (0.943%) 2024-07-16T10:00:22.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:00:22 smithi176 ceph-mon[81689]: Health check update: Degraded data redundancy: 3/318 objects degraded (0.943%), 1 pg degraded (PG_DEGRADED) 2024-07-16T10:00:22.424 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:00:22 smithi156 ceph-mon[93066]: pgmap v94: 65 pgs: 5 peering, 2 active+undersized, 1 active+undersized+degraded, 57 active+clean; 253 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s; 3/318 objects degraded (0.943%) 2024-07-16T10:00:22.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:00:22 smithi156 ceph-mon[93066]: Health check update: Degraded data redundancy: 3/318 objects degraded (0.943%), 1 pg degraded (PG_DEGRADED) 2024-07-16T10:00:23.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:00:23 smithi156 ceph-mon[93066]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 3/318 objects degraded (0.943%), 1 pg degraded) 2024-07-16T10:00:23.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:00:23 smithi156 ceph-mon[93066]: Cluster is now healthy 2024-07-16T10:00:23.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:00:23 smithi176 ceph-mon[81689]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 3/318 objects degraded (0.943%), 1 pg degraded) 2024-07-16T10:00:23.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:00:23 smithi176 ceph-mon[81689]: Cluster is now healthy 2024-07-16T10:00:24.424 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:00:24 smithi156 ceph-mon[93066]: pgmap v95: 65 pgs: 65 active+clean; 253 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 767 B/s rd, 2 op/s 2024-07-16T10:00:24.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:00:24 smithi176 ceph-mon[81689]: pgmap v95: 65 pgs: 65 active+clean; 253 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 767 B/s rd, 2 op/s 2024-07-16T10:00:26.424 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:00:26 smithi156 ceph-mon[93066]: pgmap v96: 65 pgs: 65 active+clean; 253 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.8 KiB/s rd, 3 op/s 2024-07-16T10:00:26.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:00:26 smithi176 ceph-mon[81689]: pgmap v96: 65 pgs: 65 active+clean; 253 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.8 KiB/s rd, 3 op/s 2024-07-16T10:00:28.424 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:00:28 smithi156 ceph-mon[93066]: pgmap v97: 65 pgs: 65 active+clean; 253 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.6 KiB/s rd, 3 op/s 2024-07-16T10:00:28.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:00:28 smithi176 ceph-mon[81689]: pgmap v97: 65 pgs: 65 active+clean; 253 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.6 KiB/s rd, 3 op/s 2024-07-16T10:00:29.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:00:29 smithi156 ceph-mon[93066]: pgmap v98: 65 pgs: 65 active+clean; 253 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.4 KiB/s rd, 2 op/s 2024-07-16T10:00:29.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:00:29 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:00:29.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:00:29 smithi176 ceph-mon[81689]: pgmap v98: 65 pgs: 65 active+clean; 253 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.4 KiB/s rd, 2 op/s 2024-07-16T10:00:29.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:00:29 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:00:30.147 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:30.145+0000 7f4818878640 1 -- 172.21.15.156:0/708629560 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f48140fc550 msgr2=0x7f48140fc9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:00:30.147 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:30.145+0000 7f4818878640 1 --2- 172.21.15.156:0/708629560 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f48140fc550 0x7f48140fc9c0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f48080048d0 tx=0x7f4808009da0 comp rx=0 tx=0).stop 2024-07-16T10:00:30.148 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:30.146+0000 7f4818878640 1 -- 172.21.15.156:0/708629560 shutdown_connections 2024-07-16T10:00:30.148 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:30.146+0000 7f4818878640 1 --2- 172.21.15.156:0/708629560 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f48140fc550 0x7f48140fc9c0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:00:30.148 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:30.146+0000 7f4818878640 1 --2- 172.21.15.156:0/708629560 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f48140fb280 0x7f48140fb690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:00:30.148 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:30.146+0000 7f4818878640 1 -- 172.21.15.156:0/708629560 >> 172.21.15.156:0/708629560 conn(0x7f48140f6a40 msgr2=0x7f48140f8e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T10:00:30.148 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:30.146+0000 7f4818878640 1 -- 172.21.15.156:0/708629560 shutdown_connections 2024-07-16T10:00:30.148 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:30.146+0000 7f4818878640 1 -- 172.21.15.156:0/708629560 wait complete. 2024-07-16T10:00:30.148 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:30.146+0000 7f4818878640 1 Processor -- start 2024-07-16T10:00:30.148 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:30.146+0000 7f4818878640 1 -- start start 2024-07-16T10:00:30.150 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:30.147+0000 7f4818878640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f48140fb280 0x7f481410cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:00:30.150 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:30.147+0000 7f4818878640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f48140fc550 0x7f481410d500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:00:30.150 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:30.147+0000 7f4818878640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f481410db50 con 0x7f48140fb280 2024-07-16T10:00:30.150 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:30.147+0000 7f4818878640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f481410e370 con 0x7f48140fc550 2024-07-16T10:00:30.150 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:30.147+0000 7f481259c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f48140fb280 0x7f481410cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:00:30.150 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:30.147+0000 7f481259c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f48140fb280 0x7f481410cfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:46700/0 (socket says 172.21.15.156:46700) 2024-07-16T10:00:30.150 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:30.147+0000 7f481259c640 1 -- 172.21.15.156:0/3198706940 learned_addr learned my addr 172.21.15.156:0/3198706940 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T10:00:30.150 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:30.147+0000 7f4811d9b640 1 --2- 172.21.15.156:0/3198706940 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f48140fc550 0x7f481410d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:00:30.150 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:30.147+0000 7f481259c640 1 -- 172.21.15.156:0/3198706940 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f48140fc550 msgr2=0x7f481410d500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:00:30.150 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:30.147+0000 7f481259c640 1 --2- 172.21.15.156:0/3198706940 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f48140fc550 0x7f481410d500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:00:30.150 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:30.147+0000 7f481259c640 1 -- 172.21.15.156:0/3198706940 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4808005040 con 0x7f48140fb280 2024-07-16T10:00:30.151 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:30.147+0000 7f4811d9b640 1 --2- 172.21.15.156:0/3198706940 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f48140fc550 0x7f481410d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-16T10:00:30.151 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:30.147+0000 7f481259c640 1 --2- 172.21.15.156:0/3198706940 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f48140fb280 0x7f481410cfc0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f4804002a50 tx=0x7f4804004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T10:00:30.151 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:30.147+0000 7f4802ffd640 1 -- 172.21.15.156:0/3198706940 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4804007c30 con 0x7f48140fb280 2024-07-16T10:00:30.151 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:30.147+0000 7f4802ffd640 1 -- 172.21.15.156:0/3198706940 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f480400f040 con 0x7f48140fb280 2024-07-16T10:00:30.151 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:30.147+0000 7f4802ffd640 1 -- 172.21.15.156:0/3198706940 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f480400dbe0 con 0x7f48140fb280 2024-07-16T10:00:30.151 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:30.147+0000 7f4818878640 1 -- 172.21.15.156:0/3198706940 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f481410e600 con 0x7f48140fb280 2024-07-16T10:00:30.151 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:30.148+0000 7f4818878640 1 -- 172.21.15.156:0/3198706940 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f481410eb50 con 0x7f48140fb280 2024-07-16T10:00:30.152 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:30.149+0000 7f4818878640 1 -- 172.21.15.156:0/3198706940 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48140fa6e0 con 0x7f48140fb280 2024-07-16T10:00:30.152 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:30.149+0000 7f4802ffd640 1 -- 172.21.15.156:0/3198706940 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 38) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f480400dd40 con 0x7f48140fb280 2024-07-16T10:00:30.152 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:30.150+0000 7f4802ffd640 1 --2- 172.21.15.156:0/3198706940 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f47fc06e0b0 0x7f47fc070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:00:30.152 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:30.150+0000 7f4811d9b640 1 --2- 172.21.15.156:0/3198706940 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f47fc06e0b0 0x7f47fc070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:00:30.152 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:30.150+0000 7f4802ffd640 1 -- 172.21.15.156:0/3198706940 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f480408db20 con 0x7f48140fb280 2024-07-16T10:00:30.152 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:30.150+0000 7f4811d9b640 1 --2- 172.21.15.156:0/3198706940 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f47fc06e0b0 0x7f47fc070560 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f4808000c00 tx=0x7f4808035040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T10:00:30.156 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:30.155+0000 7f4802ffd640 1 -- 172.21.15.156:0/3198706940 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187716 (secure 0 0 0) 0x7f4804056ea0 con 0x7f48140fb280 2024-07-16T10:00:30.351 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:30.349+0000 7f4818878640 1 -- 172.21.15.156:0/3198706940 --> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f481410edc0 con 0x7f47fc06e0b0 2024-07-16T10:00:30.352 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:30.351+0000 7f4802ffd640 1 -- 172.21.15.156:0/3198706940 <== mgr.34104 v2:172.21.15.156:6800/3367765292 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f481410edc0 con 0x7f47fc06e0b0 2024-07-16T10:00:30.354 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:30.353+0000 7f4818878640 1 -- 172.21.15.156:0/3198706940 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f47fc06e0b0 msgr2=0x7f47fc070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:00:30.354 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:30.353+0000 7f4818878640 1 --2- 172.21.15.156:0/3198706940 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f47fc06e0b0 0x7f47fc070560 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f4808000c00 tx=0x7f4808035040 comp rx=0 tx=0).stop 2024-07-16T10:00:30.354 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:30.353+0000 7f4818878640 1 -- 172.21.15.156:0/3198706940 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f48140fb280 msgr2=0x7f481410cfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:00:30.354 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:30.353+0000 7f4818878640 1 --2- 172.21.15.156:0/3198706940 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f48140fb280 0x7f481410cfc0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f4804002a50 tx=0x7f4804004600 comp rx=0 tx=0).stop 2024-07-16T10:00:30.355 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:30.353+0000 7f4818878640 1 -- 172.21.15.156:0/3198706940 shutdown_connections 2024-07-16T10:00:30.355 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:30.353+0000 7f4818878640 1 --2- 172.21.15.156:0/3198706940 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f47fc06e0b0 0x7f47fc070560 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:00:30.355 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:30.353+0000 7f4818878640 1 --2- 172.21.15.156:0/3198706940 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f48140fc550 0x7f481410d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:00:30.355 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:30.353+0000 7f4818878640 1 --2- 172.21.15.156:0/3198706940 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f48140fb280 0x7f481410cfc0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:00:30.355 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:30.353+0000 7f4818878640 1 -- 172.21.15.156:0/3198706940 >> 172.21.15.156:0/3198706940 conn(0x7f48140f6a40 msgr2=0x7f4814065e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T10:00:30.355 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:30.353+0000 7f4818878640 1 -- 172.21.15.156:0/3198706940 shutdown_connections 2024-07-16T10:00:30.355 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:30.353+0000 7f4818878640 1 -- 172.21.15.156:0/3198706940 wait complete. 2024-07-16T10:00:30.363 INFO:teuthology.orchestra.run.smithi156.stdout:true 2024-07-16T10:00:30.443 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:30.440+0000 7f2ae8749640 1 -- 172.21.15.156:0/2509982396 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2ae00fd600 msgr2=0x7f2ae00fda70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:00:30.443 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:30.440+0000 7f2ae8749640 1 --2- 172.21.15.156:0/2509982396 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2ae00fd600 0x7f2ae00fda70 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f2ad40048d0 tx=0x7f2ad4009da0 comp rx=0 tx=0).stop 2024-07-16T10:00:30.443 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:30.440+0000 7f2ae8749640 1 -- 172.21.15.156:0/2509982396 shutdown_connections 2024-07-16T10:00:30.443 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:30.440+0000 7f2ae8749640 1 --2- 172.21.15.156:0/2509982396 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2ae00fd600 0x7f2ae00fda70 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:00:30.443 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:30.440+0000 7f2ae8749640 1 --2- 172.21.15.156:0/2509982396 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2ae00fc330 0x7f2ae00fc740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:00:30.443 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:30.440+0000 7f2ae8749640 1 -- 172.21.15.156:0/2509982396 >> 172.21.15.156:0/2509982396 conn(0x7f2ae00f7af0 msgr2=0x7f2ae00f9f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T10:00:30.443 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:30.440+0000 7f2ae8749640 1 -- 172.21.15.156:0/2509982396 shutdown_connections 2024-07-16T10:00:30.443 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:30.440+0000 7f2ae8749640 1 -- 172.21.15.156:0/2509982396 wait complete. 2024-07-16T10:00:30.443 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:30.440+0000 7f2ae8749640 1 Processor -- start 2024-07-16T10:00:30.443 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:30.440+0000 7f2ae8749640 1 -- start start 2024-07-16T10:00:30.444 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:30.440+0000 7f2ae8749640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2ae00fc330 0x7f2ae010f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:00:30.444 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:30.440+0000 7f2ae8749640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2ae00fd600 0x7f2ae010f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:00:30.444 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:30.440+0000 7f2ae8749640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ae010fd10 con 0x7f2ae00fd600 2024-07-16T10:00:30.444 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:30.440+0000 7f2ae8749640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ae0110530 con 0x7f2ae00fc330 2024-07-16T10:00:30.444 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:30.440+0000 7f2ae5ce4640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2ae00fd600 0x7f2ae010f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:00:30.444 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:30.441+0000 7f2ae5ce4640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2ae00fd600 0x7f2ae010f6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:46724/0 (socket says 172.21.15.156:46724) 2024-07-16T10:00:30.444 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:30.441+0000 7f2ae5ce4640 1 -- 172.21.15.156:0/2597720205 learned_addr learned my addr 172.21.15.156:0/2597720205 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T10:00:30.444 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:30.441+0000 7f2ae5ce4640 1 -- 172.21.15.156:0/2597720205 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2ae00fc330 msgr2=0x7f2ae010f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:00:30.444 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:30.441+0000 7f2ae64e5640 1 --2- 172.21.15.156:0/2597720205 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2ae00fc330 0x7f2ae010f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:00:30.444 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:30.441+0000 7f2ae5ce4640 1 --2- 172.21.15.156:0/2597720205 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2ae00fc330 0x7f2ae010f180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:00:30.444 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:30.441+0000 7f2ae5ce4640 1 -- 172.21.15.156:0/2597720205 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ad4005040 con 0x7f2ae00fd600 2024-07-16T10:00:30.444 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:30.441+0000 7f2ae64e5640 1 --2- 172.21.15.156:0/2597720205 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2ae00fc330 0x7f2ae010f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-16T10:00:30.444 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:30.441+0000 7f2ae5ce4640 1 --2- 172.21.15.156:0/2597720205 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2ae00fd600 0x7f2ae010f6c0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f2ad4003080 tx=0x7f2ad4009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T10:00:30.445 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:30.441+0000 7f2adeffd640 1 -- 172.21.15.156:0/2597720205 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2ad4038070 con 0x7f2ae00fd600 2024-07-16T10:00:30.445 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:30.441+0000 7f2adeffd640 1 -- 172.21.15.156:0/2597720205 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2ad4032970 con 0x7f2ae00fd600 2024-07-16T10:00:30.445 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:30.441+0000 7f2ae8749640 1 -- 172.21.15.156:0/2597720205 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ae0110760 con 0x7f2ae00fd600 2024-07-16T10:00:30.445 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:30.441+0000 7f2adeffd640 1 -- 172.21.15.156:0/2597720205 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2ad4033900 con 0x7f2ae00fd600 2024-07-16T10:00:30.445 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:30.441+0000 7f2ae8749640 1 -- 172.21.15.156:0/2597720205 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ae0110ca0 con 0x7f2ae00fd600 2024-07-16T10:00:30.446 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:30.442+0000 7f2ae8749640 1 -- 172.21.15.156:0/2597720205 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ae00fb790 con 0x7f2ae00fd600 2024-07-16T10:00:30.446 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:30.443+0000 7f2adeffd640 1 -- 172.21.15.156:0/2597720205 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 38) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f2ad4030020 con 0x7f2ae00fd600 2024-07-16T10:00:30.446 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:30.443+0000 7f2adeffd640 1 --2- 172.21.15.156:0/2597720205 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f2ac806e0b0 0x7f2ac8070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:00:30.446 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:30.443+0000 7f2adeffd640 1 -- 172.21.15.156:0/2597720205 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f2ad40adbd0 con 0x7f2ae00fd600 2024-07-16T10:00:30.446 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:30.443+0000 7f2ae64e5640 1 --2- 172.21.15.156:0/2597720205 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f2ac806e0b0 0x7f2ac8070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:00:30.446 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:30.444+0000 7f2ae64e5640 1 --2- 172.21.15.156:0/2597720205 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f2ac806e0b0 0x7f2ac8070560 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f2ad80041d0 tx=0x7f2ad8009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T10:00:30.449 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:30.448+0000 7f2adeffd640 1 -- 172.21.15.156:0/2597720205 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187716 (secure 0 0 0) 0x7f2ad4076f50 con 0x7f2ae00fd600 2024-07-16T10:00:30.642 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:30.641+0000 7f2ae8749640 1 -- 172.21.15.156:0/2597720205 --> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f2ae0104fb0 con 0x7f2ac806e0b0 2024-07-16T10:00:30.643 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:30.642+0000 7f2adeffd640 1 -- 172.21.15.156:0/2597720205 <== mgr.34104 v2:172.21.15.156:6800/3367765292 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f2ae0104fb0 con 0x7f2ac806e0b0 2024-07-16T10:00:30.645 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:30.644+0000 7f2ae8749640 1 -- 172.21.15.156:0/2597720205 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f2ac806e0b0 msgr2=0x7f2ac8070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:00:30.646 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:30.644+0000 7f2ae8749640 1 --2- 172.21.15.156:0/2597720205 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f2ac806e0b0 0x7f2ac8070560 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f2ad80041d0 tx=0x7f2ad8009040 comp rx=0 tx=0).stop 2024-07-16T10:00:30.646 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:30.644+0000 7f2ae8749640 1 -- 172.21.15.156:0/2597720205 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2ae00fd600 msgr2=0x7f2ae010f6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:00:30.646 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:30.644+0000 7f2ae8749640 1 --2- 172.21.15.156:0/2597720205 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2ae00fd600 0x7f2ae010f6c0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f2ad4003080 tx=0x7f2ad4009da0 comp rx=0 tx=0).stop 2024-07-16T10:00:30.646 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:30.644+0000 7f2ae8749640 1 -- 172.21.15.156:0/2597720205 shutdown_connections 2024-07-16T10:00:30.646 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:30.644+0000 7f2ae8749640 1 --2- 172.21.15.156:0/2597720205 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f2ac806e0b0 0x7f2ac8070560 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:00:30.646 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:30.644+0000 7f2ae8749640 1 --2- 172.21.15.156:0/2597720205 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2ae00fd600 0x7f2ae010f6c0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:00:30.646 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:30.644+0000 7f2ae8749640 1 --2- 172.21.15.156:0/2597720205 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2ae00fc330 0x7f2ae010f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:00:30.646 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:30.644+0000 7f2ae8749640 1 -- 172.21.15.156:0/2597720205 >> 172.21.15.156:0/2597720205 conn(0x7f2ae00f7af0 msgr2=0x7f2ae00f90e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T10:00:30.646 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:30.644+0000 7f2ae8749640 1 -- 172.21.15.156:0/2597720205 shutdown_connections 2024-07-16T10:00:30.646 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:30.644+0000 7f2ae8749640 1 -- 172.21.15.156:0/2597720205 wait complete. 2024-07-16T10:00:30.727 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:30.725+0000 7f498ba82640 1 -- 172.21.15.156:0/1941822912 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f49840fd630 msgr2=0x7f49840fdaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:00:30.727 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:30.725+0000 7f498ba82640 1 --2- 172.21.15.156:0/1941822912 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f49840fd630 0x7f49840fdaa0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f49800048d0 tx=0x7f4980009da0 comp rx=0 tx=0).stop 2024-07-16T10:00:30.728 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:30.725+0000 7f498ba82640 1 -- 172.21.15.156:0/1941822912 shutdown_connections 2024-07-16T10:00:30.728 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:30.725+0000 7f498ba82640 1 --2- 172.21.15.156:0/1941822912 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f49840fd630 0x7f49840fdaa0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:00:30.728 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:30.725+0000 7f498ba82640 1 --2- 172.21.15.156:0/1941822912 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f49840fc360 0x7f49840fc770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:00:30.728 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:30.725+0000 7f498ba82640 1 -- 172.21.15.156:0/1941822912 >> 172.21.15.156:0/1941822912 conn(0x7f49840f7b40 msgr2=0x7f49840f9f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T10:00:30.728 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:30.725+0000 7f498ba82640 1 -- 172.21.15.156:0/1941822912 shutdown_connections 2024-07-16T10:00:30.728 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:30.725+0000 7f498ba82640 1 -- 172.21.15.156:0/1941822912 wait complete. 2024-07-16T10:00:30.728 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:30.726+0000 7f498ba82640 1 Processor -- start 2024-07-16T10:00:30.728 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:30.726+0000 7f498ba82640 1 -- start start 2024-07-16T10:00:30.728 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:30.726+0000 7f498ba82640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f49840fc360 0x7f498410cf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:00:30.728 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:30.726+0000 7f498ba82640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f49840fd630 0x7f498410d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:00:30.728 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:30.726+0000 7f498ba82640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f498410db10 con 0x7f49840fc360 2024-07-16T10:00:30.728 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:30.726+0000 7f498ba82640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f498410e330 con 0x7f49840fd630 2024-07-16T10:00:30.728 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:30.726+0000 7f498981e640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f49840fc360 0x7f498410cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:00:30.729 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:30.726+0000 7f498981e640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f49840fc360 0x7f498410cf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:46736/0 (socket says 172.21.15.156:46736) 2024-07-16T10:00:30.729 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:30.726+0000 7f498981e640 1 -- 172.21.15.156:0/1098232663 learned_addr learned my addr 172.21.15.156:0/1098232663 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T10:00:30.729 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:30.726+0000 7f498901d640 1 --2- 172.21.15.156:0/1098232663 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f49840fd630 0x7f498410d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:00:30.729 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:30.726+0000 7f498981e640 1 -- 172.21.15.156:0/1098232663 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f49840fd630 msgr2=0x7f498410d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:00:30.729 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:30.726+0000 7f498981e640 1 --2- 172.21.15.156:0/1098232663 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f49840fd630 0x7f498410d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:00:30.729 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:30.726+0000 7f498981e640 1 -- 172.21.15.156:0/1098232663 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4980005040 con 0x7f49840fc360 2024-07-16T10:00:30.731 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:30.727+0000 7f498981e640 1 --2- 172.21.15.156:0/1098232663 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f49840fc360 0x7f498410cf80 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f4974002a50 tx=0x7f4974004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T10:00:30.731 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:30.727+0000 7f497a7fc640 1 -- 172.21.15.156:0/1098232663 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4974007c30 con 0x7f49840fc360 2024-07-16T10:00:30.731 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:30.727+0000 7f497a7fc640 1 -- 172.21.15.156:0/1098232663 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f497400f040 con 0x7f49840fc360 2024-07-16T10:00:30.731 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:30.727+0000 7f498ba82640 1 -- 172.21.15.156:0/1098232663 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f498410e5c0 con 0x7f49840fc360 2024-07-16T10:00:30.731 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:30.727+0000 7f497a7fc640 1 -- 172.21.15.156:0/1098232663 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f497400dad0 con 0x7f49840fc360 2024-07-16T10:00:30.731 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:30.727+0000 7f498ba82640 1 -- 172.21.15.156:0/1098232663 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f498410eb30 con 0x7f49840fc360 2024-07-16T10:00:30.731 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:30.728+0000 7f498ba82640 1 -- 172.21.15.156:0/1098232663 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49841056e0 con 0x7f49840fc360 2024-07-16T10:00:30.732 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:30.728+0000 7f497a7fc640 1 -- 172.21.15.156:0/1098232663 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 38) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f497401c400 con 0x7f49840fc360 2024-07-16T10:00:30.732 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:30.729+0000 7f497a7fc640 1 --2- 172.21.15.156:0/1098232663 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f497006e0b0 0x7f4970070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:00:30.732 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:30.729+0000 7f497a7fc640 1 -- 172.21.15.156:0/1098232663 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f497408d310 con 0x7f49840fc360 2024-07-16T10:00:30.732 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:30.729+0000 7f498901d640 1 --2- 172.21.15.156:0/1098232663 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f497006e0b0 0x7f4970070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:00:30.732 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:30.729+0000 7f498901d640 1 --2- 172.21.15.156:0/1098232663 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f497006e0b0 0x7f4970070560 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f49800043d0 tx=0x7f4980035040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T10:00:30.734 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:30.733+0000 7f497a7fc640 1 -- 172.21.15.156:0/1098232663 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187716 (secure 0 0 0) 0x7f4974056740 con 0x7f49840fc360 2024-07-16T10:00:30.918 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:30.917+0000 7f498ba82640 1 -- 172.21.15.156:0/1098232663 --> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f498410eeb0 con 0x7f497006e0b0 2024-07-16T10:00:30.927 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:30.926+0000 7f497a7fc640 1 -- 172.21.15.156:0/1098232663 <== mgr.34104 v2:172.21.15.156:6800/3367765292 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7f498410eeb0 con 0x7f497006e0b0 2024-07-16T10:00:30.927 INFO:teuthology.orchestra.run.smithi156.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-07-16T10:00:30.928 INFO:teuthology.orchestra.run.smithi156.stdout:alertmanager.smithi156 smithi156 *:9093,9094 running (4m) 17s ago 12m 25.0M - 0.25.0 c8568f914cd2 d524a2c3ff15 2024-07-16T10:00:30.928 INFO:teuthology.orchestra.run.smithi156.stdout:crash.smithi156 smithi156 running (114s) 17s ago 12m 6899k - 19.0.0-3495-ga6ca461b 16f7feb04342 11648cf29af4 2024-07-16T10:00:30.928 INFO:teuthology.orchestra.run.smithi156.stdout:crash.smithi176 smithi176 running (109s) 106s ago 11m 6899k - 19.0.0-3495-ga6ca461b 16f7feb04342 67f3ccc034f8 2024-07-16T10:00:30.928 INFO:teuthology.orchestra.run.smithi156.stdout:grafana.smithi156 smithi156 *:3000 running (3m) 17s ago 12m 77.8M - 9.4.7 954c08fa6188 deb9e1bfeb66 2024-07-16T10:00:30.928 INFO:teuthology.orchestra.run.smithi156.stdout:mds.cephfs.smithi156.evwwaz smithi156 running (7m) 17s ago 7m 67.2M - 17.2.7-1104-ge6231b04 d5de5f953b95 7c0ff9ae43f8 2024-07-16T10:00:30.928 INFO:teuthology.orchestra.run.smithi156.stdout:mds.cephfs.smithi156.rdibys smithi156 running (7m) 17s ago 7m 120M - 17.2.7-1104-ge6231b04 d5de5f953b95 b985cd1e6924 2024-07-16T10:00:30.928 INFO:teuthology.orchestra.run.smithi156.stdout:mds.cephfs.smithi176.dvvbud smithi176 running (7m) 106s ago 7m 108M - 17.2.7-1104-ge6231b04 d5de5f953b95 1b00147950f9 2024-07-16T10:00:30.928 INFO:teuthology.orchestra.run.smithi156.stdout:mds.cephfs.smithi176.wahbsi smithi176 running (7m) 106s ago 7m 62.2M - 17.2.7-1104-ge6231b04 d5de5f953b95 866664af2311 2024-07-16T10:00:30.928 INFO:teuthology.orchestra.run.smithi156.stdout:mgr.smithi156.zkvylv smithi156 *:8443,9283,8765 running (3m) 17s ago 14m 497M - 19.0.0-3495-ga6ca461b 16f7feb04342 ee53bd604b7c 2024-07-16T10:00:30.928 INFO:teuthology.orchestra.run.smithi156.stdout:mgr.smithi176.dqbrvx smithi176 *:8443,9283,8765 running (3m) 106s ago 11m 438M - 19.0.0-3495-ga6ca461b 16f7feb04342 ce9eaae4c2d0 2024-07-16T10:00:30.928 INFO:teuthology.orchestra.run.smithi156.stdout:mon.smithi156 smithi156 running (2m) 17s ago 14m 45.4M 2048M 19.0.0-3495-ga6ca461b 16f7feb04342 77b6ec8c50eb 2024-07-16T10:00:30.928 INFO:teuthology.orchestra.run.smithi156.stdout:mon.smithi176 smithi176 running (2m) 106s ago 11m 33.4M 2048M 19.0.0-3495-ga6ca461b 16f7feb04342 ef92814cec2b 2024-07-16T10:00:30.929 INFO:teuthology.orchestra.run.smithi156.stdout:node-exporter.smithi156 smithi156 *:9100 running (4m) 17s ago 12m 17.2M - 1.5.0 0da6a335fe13 e5d86f23a14c 2024-07-16T10:00:30.929 INFO:teuthology.orchestra.run.smithi156.stdout:node-exporter.smithi176 smithi176 *:9100 running (3m) 106s ago 11m 16.1M - 1.5.0 0da6a335fe13 ea842728144a 2024-07-16T10:00:30.929 INFO:teuthology.orchestra.run.smithi156.stdout:osd.0 smithi156 running (91s) 17s ago 11m 93.7M 1557M 19.0.0-3495-ga6ca461b 16f7feb04342 8633df1689cc 2024-07-16T10:00:30.929 INFO:teuthology.orchestra.run.smithi156.stdout:osd.1 smithi156 running (56s) 17s ago 10m 101M 1557M 19.0.0-3495-ga6ca461b 16f7feb04342 e6fae9653b46 2024-07-16T10:00:30.929 INFO:teuthology.orchestra.run.smithi156.stdout:osd.2 smithi156 running (22s) 17s ago 10m 11.8M 1557M 19.0.0-3495-ga6ca461b 16f7feb04342 c6ecf18de9bf 2024-07-16T10:00:30.929 INFO:teuthology.orchestra.run.smithi156.stdout:osd.3 smithi176 running (9m) 106s ago 9m 260M 2581M 17.2.7-1104-ge6231b04 d5de5f953b95 733861ba84dc 2024-07-16T10:00:30.929 INFO:teuthology.orchestra.run.smithi156.stdout:osd.4 smithi176 running (9m) 106s ago 9m 229M 2581M 17.2.7-1104-ge6231b04 d5de5f953b95 ef0c67cec086 2024-07-16T10:00:30.929 INFO:teuthology.orchestra.run.smithi156.stdout:osd.5 smithi176 running (8m) 106s ago 8m 224M 2581M 17.2.7-1104-ge6231b04 d5de5f953b95 996e15448a52 2024-07-16T10:00:30.929 INFO:teuthology.orchestra.run.smithi156.stdout:prometheus.smithi156 smithi156 *:9095 running (3m) 17s ago 12m 64.9M - 2.43.0 a07b618ecd1d 20c195ecd665 2024-07-16T10:00:30.930 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:30.929+0000 7f498ba82640 1 -- 172.21.15.156:0/1098232663 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f497006e0b0 msgr2=0x7f4970070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:00:30.930 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:30.929+0000 7f498ba82640 1 --2- 172.21.15.156:0/1098232663 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f497006e0b0 0x7f4970070560 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f49800043d0 tx=0x7f4980035040 comp rx=0 tx=0).stop 2024-07-16T10:00:30.930 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:30.929+0000 7f498ba82640 1 -- 172.21.15.156:0/1098232663 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f49840fc360 msgr2=0x7f498410cf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:00:30.930 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:30.929+0000 7f498ba82640 1 --2- 172.21.15.156:0/1098232663 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f49840fc360 0x7f498410cf80 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f4974002a50 tx=0x7f4974004600 comp rx=0 tx=0).stop 2024-07-16T10:00:30.930 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:30.929+0000 7f498ba82640 1 -- 172.21.15.156:0/1098232663 shutdown_connections 2024-07-16T10:00:30.930 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:30.929+0000 7f498ba82640 1 --2- 172.21.15.156:0/1098232663 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f497006e0b0 0x7f4970070560 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:00:30.931 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:30.929+0000 7f498ba82640 1 --2- 172.21.15.156:0/1098232663 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f49840fd630 0x7f498410d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:00:30.931 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:30.929+0000 7f498ba82640 1 --2- 172.21.15.156:0/1098232663 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f49840fc360 0x7f498410cf80 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:00:30.931 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:30.929+0000 7f498ba82640 1 -- 172.21.15.156:0/1098232663 >> 172.21.15.156:0/1098232663 conn(0x7f49840f7b40 msgr2=0x7f49840f9230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T10:00:30.931 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:30.929+0000 7f498ba82640 1 -- 172.21.15.156:0/1098232663 shutdown_connections 2024-07-16T10:00:30.931 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:30.929+0000 7f498ba82640 1 -- 172.21.15.156:0/1098232663 wait complete. 2024-07-16T10:00:31.010 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:31.007+0000 7f03200e5640 1 -- 172.21.15.156:0/1650642704 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f03180fb260 msgr2=0x7f03180fb670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:00:31.010 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:31.007+0000 7f03200e5640 1 --2- 172.21.15.156:0/1650642704 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f03180fb260 0x7f03180fb670 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f03140048d0 tx=0x7f0314009da0 comp rx=0 tx=0).stop 2024-07-16T10:00:31.010 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:31.007+0000 7f03200e5640 1 -- 172.21.15.156:0/1650642704 shutdown_connections 2024-07-16T10:00:31.011 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:31.007+0000 7f03200e5640 1 --2- 172.21.15.156:0/1650642704 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f03180fc530 0x7f03180fc9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:00:31.011 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:31.007+0000 7f03200e5640 1 --2- 172.21.15.156:0/1650642704 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f03180fb260 0x7f03180fb670 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:00:31.011 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:31.007+0000 7f03200e5640 1 -- 172.21.15.156:0/1650642704 >> 172.21.15.156:0/1650642704 conn(0x7f03180f6a20 msgr2=0x7f03180f8e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T10:00:31.011 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:31.007+0000 7f03200e5640 1 -- 172.21.15.156:0/1650642704 shutdown_connections 2024-07-16T10:00:31.011 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:31.007+0000 7f03200e5640 1 -- 172.21.15.156:0/1650642704 wait complete. 2024-07-16T10:00:31.011 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:31.007+0000 7f03200e5640 1 Processor -- start 2024-07-16T10:00:31.011 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:31.008+0000 7f03200e5640 1 -- start start 2024-07-16T10:00:31.011 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:31.008+0000 7f03200e5640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f03180fb260 0x7f0318066450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:00:31.011 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:31.008+0000 7f03200e5640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f03180fc530 0x7f0318111d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:00:31.011 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:31.008+0000 7f03200e5640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0318112350 con 0x7f03180fb260 2024-07-16T10:00:31.011 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:31.008+0000 7f03200e5640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0318112490 con 0x7f03180fc530 2024-07-16T10:00:31.011 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:31.008+0000 7f031de81640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f03180fb260 0x7f0318066450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:00:31.011 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:31.008+0000 7f031de81640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f03180fb260 0x7f0318066450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:46760/0 (socket says 172.21.15.156:46760) 2024-07-16T10:00:31.012 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:31.008+0000 7f031de81640 1 -- 172.21.15.156:0/3497708925 learned_addr learned my addr 172.21.15.156:0/3497708925 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T10:00:31.012 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:31.008+0000 7f031d680640 1 --2- 172.21.15.156:0/3497708925 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f03180fc530 0x7f0318111d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:00:31.012 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:31.008+0000 7f031de81640 1 -- 172.21.15.156:0/3497708925 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f03180fc530 msgr2=0x7f0318111d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:00:31.012 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:31.008+0000 7f031de81640 1 --2- 172.21.15.156:0/3497708925 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f03180fc530 0x7f0318111d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:00:31.012 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:31.008+0000 7f031de81640 1 -- 172.21.15.156:0/3497708925 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0314005040 con 0x7f03180fb260 2024-07-16T10:00:31.012 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:31.008+0000 7f031de81640 1 --2- 172.21.15.156:0/3497708925 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f03180fb260 0x7f0318066450 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f0314009f80 tx=0x7f0314009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T10:00:31.012 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:31.009+0000 7f030e7fc640 1 -- 172.21.15.156:0/3497708925 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0314038070 con 0x7f03180fb260 2024-07-16T10:00:31.012 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:31.009+0000 7f030e7fc640 1 -- 172.21.15.156:0/3497708925 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0314008ee0 con 0x7f03180fb260 2024-07-16T10:00:31.012 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:31.009+0000 7f03200e5640 1 -- 172.21.15.156:0/3497708925 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0318112710 con 0x7f03180fb260 2024-07-16T10:00:31.012 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:31.009+0000 7f030e7fc640 1 -- 172.21.15.156:0/3497708925 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0314033890 con 0x7f03180fb260 2024-07-16T10:00:31.012 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:31.009+0000 7f03200e5640 1 -- 172.21.15.156:0/3497708925 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0318112bd0 con 0x7f03180fb260 2024-07-16T10:00:31.013 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:31.010+0000 7f03200e5640 1 -- 172.21.15.156:0/3497708925 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03180fa6c0 con 0x7f03180fb260 2024-07-16T10:00:31.013 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:31.010+0000 7f030e7fc640 1 -- 172.21.15.156:0/3497708925 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 38) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f0314030020 con 0x7f03180fb260 2024-07-16T10:00:31.013 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:31.011+0000 7f030e7fc640 1 --2- 172.21.15.156:0/3497708925 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f030406e0b0 0x7f0304070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:00:31.013 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:31.011+0000 7f030e7fc640 1 -- 172.21.15.156:0/3497708925 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f03140ad8d0 con 0x7f03180fb260 2024-07-16T10:00:31.013 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:31.011+0000 7f031d680640 1 --2- 172.21.15.156:0/3497708925 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f030406e0b0 0x7f0304070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:00:31.014 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:31.011+0000 7f031d680640 1 --2- 172.21.15.156:0/3497708925 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f030406e0b0 0x7f0304070560 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f0308009470 tx=0x7f0308009ed0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T10:00:31.017 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:31.016+0000 7f030e7fc640 1 -- 172.21.15.156:0/3497708925 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187716 (secure 0 0 0) 0x7f0314076cd0 con 0x7f03180fb260 2024-07-16T10:00:31.261 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:31.260+0000 7f03200e5640 1 -- 172.21.15.156:0/3497708925 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f0318112ff0 con 0x7f03180fb260 2024-07-16T10:00:31.262 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:31.261+0000 7f030e7fc640 1 -- 172.21.15.156:0/3497708925 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+838 (secure 0 0 0) 0x7f031407a390 con 0x7f03180fb260 2024-07-16T10:00:31.262 INFO:teuthology.orchestra.run.smithi156.stdout:{ 2024-07-16T10:00:31.262 INFO:teuthology.orchestra.run.smithi156.stdout: "mon": { 2024-07-16T10:00:31.262 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph version 19.0.0-3495-ga6ca461b (a6ca461be7cc1aff3658c2c6df4cff975a24449a) squid (rc)": 2 2024-07-16T10:00:31.263 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-07-16T10:00:31.263 INFO:teuthology.orchestra.run.smithi156.stdout: "mgr": { 2024-07-16T10:00:31.263 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph version 19.0.0-3495-ga6ca461b (a6ca461be7cc1aff3658c2c6df4cff975a24449a) squid (rc)": 2 2024-07-16T10:00:31.263 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-07-16T10:00:31.263 INFO:teuthology.orchestra.run.smithi156.stdout: "osd": { 2024-07-16T10:00:31.263 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph version 17.2.7-1104-ge6231b04 (e6231b04b52ebf3b9b31803282dd3074439e51ac) quincy (stable)": 3, 2024-07-16T10:00:31.263 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph version 19.0.0-3495-ga6ca461b (a6ca461be7cc1aff3658c2c6df4cff975a24449a) squid (rc)": 3 2024-07-16T10:00:31.263 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-07-16T10:00:31.263 INFO:teuthology.orchestra.run.smithi156.stdout: "mds": { 2024-07-16T10:00:31.263 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph version 17.2.7-1104-ge6231b04 (e6231b04b52ebf3b9b31803282dd3074439e51ac) quincy (stable)": 4 2024-07-16T10:00:31.263 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-07-16T10:00:31.263 INFO:teuthology.orchestra.run.smithi156.stdout: "overall": { 2024-07-16T10:00:31.264 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph version 17.2.7-1104-ge6231b04 (e6231b04b52ebf3b9b31803282dd3074439e51ac) quincy (stable)": 7, 2024-07-16T10:00:31.264 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph version 19.0.0-3495-ga6ca461b (a6ca461be7cc1aff3658c2c6df4cff975a24449a) squid (rc)": 7 2024-07-16T10:00:31.264 INFO:teuthology.orchestra.run.smithi156.stdout: } 2024-07-16T10:00:31.264 INFO:teuthology.orchestra.run.smithi156.stdout:} 2024-07-16T10:00:31.264 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:31.263+0000 7f03200e5640 1 -- 172.21.15.156:0/3497708925 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f030406e0b0 msgr2=0x7f0304070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:00:31.265 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:31.263+0000 7f03200e5640 1 --2- 172.21.15.156:0/3497708925 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f030406e0b0 0x7f0304070560 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f0308009470 tx=0x7f0308009ed0 comp rx=0 tx=0).stop 2024-07-16T10:00:31.265 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:31.263+0000 7f03200e5640 1 -- 172.21.15.156:0/3497708925 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f03180fb260 msgr2=0x7f0318066450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:00:31.265 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:31.263+0000 7f03200e5640 1 --2- 172.21.15.156:0/3497708925 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f03180fb260 0x7f0318066450 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f0314009f80 tx=0x7f0314009da0 comp rx=0 tx=0).stop 2024-07-16T10:00:31.265 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:31.263+0000 7f03200e5640 1 -- 172.21.15.156:0/3497708925 shutdown_connections 2024-07-16T10:00:31.265 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:31.263+0000 7f03200e5640 1 --2- 172.21.15.156:0/3497708925 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f030406e0b0 0x7f0304070560 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:00:31.265 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:31.263+0000 7f03200e5640 1 --2- 172.21.15.156:0/3497708925 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f03180fc530 0x7f0318111d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:00:31.265 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:31.263+0000 7f03200e5640 1 --2- 172.21.15.156:0/3497708925 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f03180fb260 0x7f0318066450 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:00:31.265 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:31.263+0000 7f03200e5640 1 -- 172.21.15.156:0/3497708925 >> 172.21.15.156:0/3497708925 conn(0x7f03180f6a20 msgr2=0x7f03180f8150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T10:00:31.265 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:31.263+0000 7f03200e5640 1 -- 172.21.15.156:0/3497708925 shutdown_connections 2024-07-16T10:00:31.265 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:31.263+0000 7f03200e5640 1 -- 172.21.15.156:0/3497708925 wait complete. 2024-07-16T10:00:31.346 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:31.344+0000 7f26b69b5640 1 -- 172.21.15.156:0/976339286 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f26b00fd600 msgr2=0x7f26b00fda70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:00:31.346 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:31.344+0000 7f26b69b5640 1 --2- 172.21.15.156:0/976339286 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f26b00fd600 0x7f26b00fda70 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f26980048d0 tx=0x7f2698009da0 comp rx=0 tx=0).stop 2024-07-16T10:00:31.346 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:31.344+0000 7f26b69b5640 1 -- 172.21.15.156:0/976339286 shutdown_connections 2024-07-16T10:00:31.346 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:31.344+0000 7f26b69b5640 1 --2- 172.21.15.156:0/976339286 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f26b00fd600 0x7f26b00fda70 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:00:31.346 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:31.344+0000 7f26b69b5640 1 --2- 172.21.15.156:0/976339286 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f26b00fc330 0x7f26b00fc740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:00:31.346 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:31.344+0000 7f26b69b5640 1 -- 172.21.15.156:0/976339286 >> 172.21.15.156:0/976339286 conn(0x7f26b00f7af0 msgr2=0x7f26b00f9f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T10:00:31.346 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:31.344+0000 7f26b69b5640 1 -- 172.21.15.156:0/976339286 shutdown_connections 2024-07-16T10:00:31.347 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:31.344+0000 7f26b69b5640 1 -- 172.21.15.156:0/976339286 wait complete. 2024-07-16T10:00:31.347 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:31.344+0000 7f26b69b5640 1 Processor -- start 2024-07-16T10:00:31.347 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:31.344+0000 7f26b69b5640 1 -- start start 2024-07-16T10:00:31.347 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:31.345+0000 7f26b69b5640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f26b00fc330 0x7f26b010cf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:00:31.347 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:31.345+0000 7f26b69b5640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f26b00fd600 0x7f26b010d490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:00:31.347 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:31.345+0000 7f26b69b5640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26b010dae0 con 0x7f26b00fd600 2024-07-16T10:00:31.347 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:31.345+0000 7f26b69b5640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26b010e300 con 0x7f26b00fc330 2024-07-16T10:00:31.347 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:31.345+0000 7f26af7fe640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f26b00fd600 0x7f26b010d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:00:31.347 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:31.345+0000 7f26af7fe640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f26b00fd600 0x7f26b010d490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:46780/0 (socket says 172.21.15.156:46780) 2024-07-16T10:00:31.347 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:31.345+0000 7f26af7fe640 1 -- 172.21.15.156:0/983633744 learned_addr learned my addr 172.21.15.156:0/983633744 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T10:00:31.349 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:31.345+0000 7f26affff640 1 --2- 172.21.15.156:0/983633744 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f26b00fc330 0x7f26b010cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:00:31.349 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:31.345+0000 7f26af7fe640 1 -- 172.21.15.156:0/983633744 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f26b00fc330 msgr2=0x7f26b010cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:00:31.349 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:31.345+0000 7f26af7fe640 1 --2- 172.21.15.156:0/983633744 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f26b00fc330 0x7f26b010cf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:00:31.349 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:31.345+0000 7f26af7fe640 1 -- 172.21.15.156:0/983633744 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2698005040 con 0x7f26b00fd600 2024-07-16T10:00:31.349 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:31.345+0000 7f26af7fe640 1 --2- 172.21.15.156:0/983633744 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f26b00fd600 0x7f26b010d490 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f26980048a0 tx=0x7f2698009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T10:00:31.349 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:31.345+0000 7f26acff9640 1 -- 172.21.15.156:0/983633744 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2698038070 con 0x7f26b00fd600 2024-07-16T10:00:31.349 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:31.345+0000 7f26acff9640 1 -- 172.21.15.156:0/983633744 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f26980092b0 con 0x7f26b00fd600 2024-07-16T10:00:31.349 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:31.345+0000 7f26acff9640 1 -- 172.21.15.156:0/983633744 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2698033780 con 0x7f26b00fd600 2024-07-16T10:00:31.349 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:31.345+0000 7f26b69b5640 1 -- 172.21.15.156:0/983633744 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26b010e530 con 0x7f26b00fd600 2024-07-16T10:00:31.349 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:31.346+0000 7f26b69b5640 1 -- 172.21.15.156:0/983633744 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26b010ea70 con 0x7f26b00fd600 2024-07-16T10:00:31.349 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:31.347+0000 7f26b69b5640 1 -- 172.21.15.156:0/983633744 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26b00fb790 con 0x7f26b00fd600 2024-07-16T10:00:31.350 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:31.347+0000 7f26acff9640 1 -- 172.21.15.156:0/983633744 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 38) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f2698030020 con 0x7f26b00fd600 2024-07-16T10:00:31.350 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:31.347+0000 7f26acff9640 1 --2- 172.21.15.156:0/983633744 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f269c06e0b0 0x7f269c070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:00:31.350 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:31.348+0000 7f26acff9640 1 -- 172.21.15.156:0/983633744 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f26980ad910 con 0x7f26b00fd600 2024-07-16T10:00:31.350 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:31.348+0000 7f26affff640 1 --2- 172.21.15.156:0/983633744 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f269c06e0b0 0x7f269c070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:00:31.350 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:31.348+0000 7f26affff640 1 --2- 172.21.15.156:0/983633744 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f269c06e0b0 0x7f269c070560 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f26a0004a70 tx=0x7f26a000b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T10:00:31.353 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:31.352+0000 7f26acff9640 1 -- 172.21.15.156:0/983633744 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187716 (secure 0 0 0) 0x7f2698076d10 con 0x7f26b00fd600 2024-07-16T10:00:31.567 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:31.566+0000 7f26b69b5640 1 -- 172.21.15.156:0/983633744 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f26b010ed80 con 0x7f26b00fd600 2024-07-16T10:00:31.567 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:00:31 smithi156 ceph-mon[93066]: pgmap v99: 65 pgs: 65 active+clean; 253 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.2 KiB/s rd, 3 op/s 2024-07-16T10:00:31.567 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:00:31 smithi156 ceph-mon[93066]: from='client.34284 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-16T10:00:31.567 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:00:31 smithi156 ceph-mon[93066]: from='client.34288 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-16T10:00:31.568 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:00:31 smithi156 ceph-mon[93066]: from='client.34292 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-07-16T10:00:31.568 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:00:31 smithi156 ceph-mon[93066]: from='client.? 172.21.15.156:0/3497708925' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:00:31.568 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:31.566+0000 7f26acff9640 1 -- 172.21.15.156:0/983633744 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 20 v20) v1 ==== 76+0+1920 (secure 0 0 0) 0x7f269807a3d0 con 0x7f26b00fd600 2024-07-16T10:00:31.568 INFO:teuthology.orchestra.run.smithi156.stdout:e20 2024-07-16T10:00:31.568 INFO:teuthology.orchestra.run.smithi156.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-07-16T10:00:31.568 INFO:teuthology.orchestra.run.smithi156.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-07-16T10:00:31.569 INFO:teuthology.orchestra.run.smithi156.stdout:legacy client fscid: 1 2024-07-16T10:00:31.569 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-07-16T10:00:31.569 INFO:teuthology.orchestra.run.smithi156.stdout:Filesystem 'cephfs' (1) 2024-07-16T10:00:31.569 INFO:teuthology.orchestra.run.smithi156.stdout:fs_name cephfs 2024-07-16T10:00:31.569 INFO:teuthology.orchestra.run.smithi156.stdout:epoch 20 2024-07-16T10:00:31.569 INFO:teuthology.orchestra.run.smithi156.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-07-16T10:00:31.569 INFO:teuthology.orchestra.run.smithi156.stdout:created 2024-07-16T09:52:55.214975+0000 2024-07-16T10:00:31.569 INFO:teuthology.orchestra.run.smithi156.stdout:modified 2024-07-16T09:58:56.574272+0000 2024-07-16T10:00:31.569 INFO:teuthology.orchestra.run.smithi156.stdout:tableserver 0 2024-07-16T10:00:31.569 INFO:teuthology.orchestra.run.smithi156.stdout:root 0 2024-07-16T10:00:31.569 INFO:teuthology.orchestra.run.smithi156.stdout:session_timeout 60 2024-07-16T10:00:31.569 INFO:teuthology.orchestra.run.smithi156.stdout:session_autoclose 300 2024-07-16T10:00:31.569 INFO:teuthology.orchestra.run.smithi156.stdout:max_file_size 1099511627776 2024-07-16T10:00:31.570 INFO:teuthology.orchestra.run.smithi156.stdout:max_xattr_size 65536 2024-07-16T10:00:31.570 INFO:teuthology.orchestra.run.smithi156.stdout:required_client_features {} 2024-07-16T10:00:31.570 INFO:teuthology.orchestra.run.smithi156.stdout:last_failure 0 2024-07-16T10:00:31.570 INFO:teuthology.orchestra.run.smithi156.stdout:last_failure_osd_epoch 0 2024-07-16T10:00:31.570 INFO:teuthology.orchestra.run.smithi156.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-07-16T10:00:31.570 INFO:teuthology.orchestra.run.smithi156.stdout:max_mds 2 2024-07-16T10:00:31.570 INFO:teuthology.orchestra.run.smithi156.stdout:in 0,1 2024-07-16T10:00:31.570 INFO:teuthology.orchestra.run.smithi156.stdout:up {0=14458,1=24261} 2024-07-16T10:00:31.570 INFO:teuthology.orchestra.run.smithi156.stdout:failed 2024-07-16T10:00:31.570 INFO:teuthology.orchestra.run.smithi156.stdout:damaged 2024-07-16T10:00:31.570 INFO:teuthology.orchestra.run.smithi156.stdout:stopped 2024-07-16T10:00:31.571 INFO:teuthology.orchestra.run.smithi156.stdout:data_pools [3] 2024-07-16T10:00:31.571 INFO:teuthology.orchestra.run.smithi156.stdout:metadata_pool 2 2024-07-16T10:00:31.571 INFO:teuthology.orchestra.run.smithi156.stdout:inline_data disabled 2024-07-16T10:00:31.571 INFO:teuthology.orchestra.run.smithi156.stdout:balancer 2024-07-16T10:00:31.571 INFO:teuthology.orchestra.run.smithi156.stdout:bal_rank_mask -1 2024-07-16T10:00:31.571 INFO:teuthology.orchestra.run.smithi156.stdout:standby_count_wanted 1 2024-07-16T10:00:31.571 INFO:teuthology.orchestra.run.smithi156.stdout:qdb_cluster leader: 14458 members: 24261,14458 2024-07-16T10:00:31.571 INFO:teuthology.orchestra.run.smithi156.stdout:[mds.cephfs.smithi156.rdibys{0:14458} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.156:6826/1080176013,v1:172.21.15.156:6827/1080176013] compat {c=[1],r=[1],i=[7ff]}] 2024-07-16T10:00:31.571 INFO:teuthology.orchestra.run.smithi156.stdout:[mds.cephfs.smithi156.evwwaz{0:14468} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.156:6828/560329502,v1:172.21.15.156:6829/560329502] compat {c=[1],r=[1],i=[7ff]}] 2024-07-16T10:00:31.571 INFO:teuthology.orchestra.run.smithi156.stdout:[mds.cephfs.smithi176.dvvbud{1:24261} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.176:6824/1577019287,v1:172.21.15.176:6825/1577019287] compat {c=[1],r=[1],i=[7ff]}] 2024-07-16T10:00:31.571 INFO:teuthology.orchestra.run.smithi156.stdout:[mds.cephfs.smithi176.wahbsi{1:24273} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.176:6826/2196775412,v1:172.21.15.176:6827/2196775412] compat {c=[1],r=[1],i=[7ff]}] 2024-07-16T10:00:31.571 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-07-16T10:00:31.571 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-07-16T10:00:31.572 INFO:teuthology.orchestra.run.smithi156.stderr:dumped fsmap epoch 20 2024-07-16T10:00:31.572 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:31.568+0000 7f26b69b5640 1 -- 172.21.15.156:0/983633744 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f269c06e0b0 msgr2=0x7f269c070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:00:31.572 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:31.568+0000 7f26b69b5640 1 --2- 172.21.15.156:0/983633744 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f269c06e0b0 0x7f269c070560 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f26a0004a70 tx=0x7f26a000b040 comp rx=0 tx=0).stop 2024-07-16T10:00:31.572 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:31.568+0000 7f26b69b5640 1 -- 172.21.15.156:0/983633744 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f26b00fd600 msgr2=0x7f26b010d490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:00:31.572 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:31.568+0000 7f26b69b5640 1 --2- 172.21.15.156:0/983633744 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f26b00fd600 0x7f26b010d490 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f26980048a0 tx=0x7f2698009da0 comp rx=0 tx=0).stop 2024-07-16T10:00:31.573 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:31.568+0000 7f26b69b5640 1 -- 172.21.15.156:0/983633744 shutdown_connections 2024-07-16T10:00:31.573 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:31.568+0000 7f26b69b5640 1 --2- 172.21.15.156:0/983633744 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f269c06e0b0 0x7f269c070560 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:00:31.573 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:31.568+0000 7f26b69b5640 1 --2- 172.21.15.156:0/983633744 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f26b00fd600 0x7f26b010d490 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:00:31.573 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:31.568+0000 7f26b69b5640 1 --2- 172.21.15.156:0/983633744 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f26b00fc330 0x7f26b010cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:00:31.573 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:31.568+0000 7f26b69b5640 1 -- 172.21.15.156:0/983633744 >> 172.21.15.156:0/983633744 conn(0x7f26b00f7af0 msgr2=0x7f26b00f91c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T10:00:31.573 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:31.568+0000 7f26b69b5640 1 -- 172.21.15.156:0/983633744 shutdown_connections 2024-07-16T10:00:31.573 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:31.568+0000 7f26b69b5640 1 -- 172.21.15.156:0/983633744 wait complete. 2024-07-16T10:00:31.652 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:31.650+0000 7f3a809ea640 1 -- 172.21.15.156:0/1172997353 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3a7c0fc530 msgr2=0x7f3a7c0fc9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:00:31.653 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:31.650+0000 7f3a809ea640 1 --2- 172.21.15.156:0/1172997353 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3a7c0fc530 0x7f3a7c0fc9a0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f3a640048d0 tx=0x7f3a64009da0 comp rx=0 tx=0).stop 2024-07-16T10:00:31.653 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:31.650+0000 7f3a809ea640 1 -- 172.21.15.156:0/1172997353 shutdown_connections 2024-07-16T10:00:31.653 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:31.650+0000 7f3a809ea640 1 --2- 172.21.15.156:0/1172997353 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3a7c0fc530 0x7f3a7c0fc9a0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:00:31.653 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:31.650+0000 7f3a809ea640 1 --2- 172.21.15.156:0/1172997353 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3a7c0fb260 0x7f3a7c0fb670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:00:31.653 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:31.650+0000 7f3a809ea640 1 -- 172.21.15.156:0/1172997353 >> 172.21.15.156:0/1172997353 conn(0x7f3a7c0f6a20 msgr2=0x7f3a7c0f8e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T10:00:31.653 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:31.650+0000 7f3a809ea640 1 -- 172.21.15.156:0/1172997353 shutdown_connections 2024-07-16T10:00:31.653 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:31.650+0000 7f3a809ea640 1 -- 172.21.15.156:0/1172997353 wait complete. 2024-07-16T10:00:31.653 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:31.650+0000 7f3a809ea640 1 Processor -- start 2024-07-16T10:00:31.653 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:31.650+0000 7f3a809ea640 1 -- start start 2024-07-16T10:00:31.653 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:31.650+0000 7f3a809ea640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3a7c0fb260 0x7f3a7c1120a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:00:31.653 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:31.650+0000 7f3a809ea640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3a7c0fc530 0x7f3a7c1145f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:00:31.653 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:31.650+0000 7f3a809ea640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a7c10db50 con 0x7f3a7c0fb260 2024-07-16T10:00:31.654 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:31.650+0000 7f3a809ea640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a7c066530 con 0x7f3a7c0fc530 2024-07-16T10:00:31.654 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:31.651+0000 7f3a7a59c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3a7c0fb260 0x7f3a7c1120a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:00:31.654 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:31.651+0000 7f3a7a59c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3a7c0fb260 0x7f3a7c1120a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:46800/0 (socket says 172.21.15.156:46800) 2024-07-16T10:00:31.654 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:31.651+0000 7f3a7a59c640 1 -- 172.21.15.156:0/197312523 learned_addr learned my addr 172.21.15.156:0/197312523 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T10:00:31.654 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:31.651+0000 7f3a73fff640 1 --2- 172.21.15.156:0/197312523 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3a7c0fc530 0x7f3a7c1145f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:00:31.654 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:31.651+0000 7f3a7a59c640 1 -- 172.21.15.156:0/197312523 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3a7c0fc530 msgr2=0x7f3a7c1145f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:00:31.654 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:31.651+0000 7f3a7a59c640 1 --2- 172.21.15.156:0/197312523 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3a7c0fc530 0x7f3a7c1145f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:00:31.654 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:31.651+0000 7f3a7a59c640 1 -- 172.21.15.156:0/197312523 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a64005040 con 0x7f3a7c0fb260 2024-07-16T10:00:31.654 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:31.651+0000 7f3a7a59c640 1 --2- 172.21.15.156:0/197312523 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3a7c0fb260 0x7f3a7c1120a0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f3a6c002a50 tx=0x7f3a6c004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T10:00:31.654 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:31.651+0000 7f3a72ffd640 1 -- 172.21.15.156:0/197312523 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3a6c007c30 con 0x7f3a7c0fb260 2024-07-16T10:00:31.654 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:31.651+0000 7f3a72ffd640 1 -- 172.21.15.156:0/197312523 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3a6c00f040 con 0x7f3a7c0fb260 2024-07-16T10:00:31.654 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:31.651+0000 7f3a809ea640 1 -- 172.21.15.156:0/197312523 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a7c066810 con 0x7f3a7c0fb260 2024-07-16T10:00:31.655 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:31.651+0000 7f3a72ffd640 1 -- 172.21.15.156:0/197312523 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3a6c00dbe0 con 0x7f3a7c0fb260 2024-07-16T10:00:31.655 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:31.651+0000 7f3a809ea640 1 -- 172.21.15.156:0/197312523 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a7c114ef0 con 0x7f3a7c0fb260 2024-07-16T10:00:31.656 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:31.652+0000 7f3a809ea640 1 -- 172.21.15.156:0/197312523 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a7c0fa6c0 con 0x7f3a7c0fb260 2024-07-16T10:00:31.656 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:31.653+0000 7f3a72ffd640 1 -- 172.21.15.156:0/197312523 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 38) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f3a6c00dd40 con 0x7f3a7c0fb260 2024-07-16T10:00:31.656 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:31.653+0000 7f3a72ffd640 1 --2- 172.21.15.156:0/197312523 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f3a6806e0b0 0x7f3a68070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:00:31.656 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:31.654+0000 7f3a72ffd640 1 -- 172.21.15.156:0/197312523 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f3a6c08db10 con 0x7f3a7c0fb260 2024-07-16T10:00:31.656 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:31.654+0000 7f3a73fff640 1 --2- 172.21.15.156:0/197312523 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f3a6806e0b0 0x7f3a68070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:00:31.656 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:31.654+0000 7f3a73fff640 1 --2- 172.21.15.156:0/197312523 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f3a6806e0b0 0x7f3a68070560 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f3a6402a530 tx=0x7f3a64009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T10:00:31.659 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:31.658+0000 7f3a72ffd640 1 -- 172.21.15.156:0/197312523 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187716 (secure 0 0 0) 0x7f3a6c056e90 con 0x7f3a7c0fb260 2024-07-16T10:00:31.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:00:31 smithi176 ceph-mon[81689]: pgmap v99: 65 pgs: 65 active+clean; 253 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.2 KiB/s rd, 3 op/s 2024-07-16T10:00:31.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:00:31 smithi176 ceph-mon[81689]: from='client.34284 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-16T10:00:31.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:00:31 smithi176 ceph-mon[81689]: from='client.34288 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-16T10:00:31.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:00:31 smithi176 ceph-mon[81689]: from='client.34292 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-07-16T10:00:31.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:00:31 smithi176 ceph-mon[81689]: from='client.? 172.21.15.156:0/3497708925' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:00:31.849 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:31.848+0000 7f3a809ea640 1 -- 172.21.15.156:0/197312523 --> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f3a7c100850 con 0x7f3a6806e0b0 2024-07-16T10:00:31.851 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:31.850+0000 7f3a72ffd640 1 -- 172.21.15.156:0/197312523 <== mgr.34104 v2:172.21.15.156:6800/3367765292 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f3a7c100850 con 0x7f3a6806e0b0 2024-07-16T10:00:31.851 INFO:teuthology.orchestra.run.smithi156.stdout:{ 2024-07-16T10:00:31.851 INFO:teuthology.orchestra.run.smithi156.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:c15ec3aa8e6c0a9c12ee24ff79f3e7dd4d01b3d1a702dfeb7d7fff9b0811a393", 2024-07-16T10:00:31.851 INFO:teuthology.orchestra.run.smithi156.stdout: "in_progress": true, 2024-07-16T10:00:31.851 INFO:teuthology.orchestra.run.smithi156.stdout: "which": "Upgrading all daemon types on all hosts", 2024-07-16T10:00:31.851 INFO:teuthology.orchestra.run.smithi156.stdout: "services_complete": [ 2024-07-16T10:00:31.851 INFO:teuthology.orchestra.run.smithi156.stdout: "mon", 2024-07-16T10:00:31.851 INFO:teuthology.orchestra.run.smithi156.stdout: "mgr", 2024-07-16T10:00:31.852 INFO:teuthology.orchestra.run.smithi156.stdout: "crash" 2024-07-16T10:00:31.852 INFO:teuthology.orchestra.run.smithi156.stdout: ], 2024-07-16T10:00:31.852 INFO:teuthology.orchestra.run.smithi156.stdout: "progress": "9/21 daemons upgraded", 2024-07-16T10:00:31.852 INFO:teuthology.orchestra.run.smithi156.stdout: "message": "Currently upgrading osd daemons", 2024-07-16T10:00:31.852 INFO:teuthology.orchestra.run.smithi156.stdout: "is_paused": false 2024-07-16T10:00:31.852 INFO:teuthology.orchestra.run.smithi156.stdout:} 2024-07-16T10:00:31.853 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:31.852+0000 7f3a809ea640 1 -- 172.21.15.156:0/197312523 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f3a6806e0b0 msgr2=0x7f3a68070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:00:31.853 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:31.852+0000 7f3a809ea640 1 --2- 172.21.15.156:0/197312523 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f3a6806e0b0 0x7f3a68070560 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f3a6402a530 tx=0x7f3a64009da0 comp rx=0 tx=0).stop 2024-07-16T10:00:31.853 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:31.852+0000 7f3a809ea640 1 -- 172.21.15.156:0/197312523 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3a7c0fb260 msgr2=0x7f3a7c1120a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:00:31.853 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:31.852+0000 7f3a809ea640 1 --2- 172.21.15.156:0/197312523 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3a7c0fb260 0x7f3a7c1120a0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f3a6c002a50 tx=0x7f3a6c004600 comp rx=0 tx=0).stop 2024-07-16T10:00:31.853 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:31.852+0000 7f3a809ea640 1 -- 172.21.15.156:0/197312523 shutdown_connections 2024-07-16T10:00:31.853 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:31.852+0000 7f3a809ea640 1 --2- 172.21.15.156:0/197312523 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f3a6806e0b0 0x7f3a68070560 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:00:31.853 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:31.852+0000 7f3a809ea640 1 --2- 172.21.15.156:0/197312523 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3a7c0fc530 0x7f3a7c1145f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:00:31.853 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:31.852+0000 7f3a809ea640 1 --2- 172.21.15.156:0/197312523 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3a7c0fb260 0x7f3a7c1120a0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:00:31.854 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:31.852+0000 7f3a809ea640 1 -- 172.21.15.156:0/197312523 >> 172.21.15.156:0/197312523 conn(0x7f3a7c0f6a20 msgr2=0x7f3a7c0f8120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T10:00:31.854 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:31.852+0000 7f3a809ea640 1 -- 172.21.15.156:0/197312523 shutdown_connections 2024-07-16T10:00:31.854 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:31.852+0000 7f3a809ea640 1 -- 172.21.15.156:0/197312523 wait complete. 2024-07-16T10:00:31.934 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:31.931+0000 7f6df905f640 1 -- 172.21.15.156:0/3725190475 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6df40fc360 msgr2=0x7f6df40fc770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:00:31.934 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:31.931+0000 7f6df905f640 1 --2- 172.21.15.156:0/3725190475 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6df40fc360 0x7f6df40fc770 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f6de40048d0 tx=0x7f6de4009da0 comp rx=0 tx=0).stop 2024-07-16T10:00:31.934 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:31.931+0000 7f6df905f640 1 -- 172.21.15.156:0/3725190475 shutdown_connections 2024-07-16T10:00:31.934 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:31.931+0000 7f6df905f640 1 --2- 172.21.15.156:0/3725190475 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6df40fd630 0x7f6df40fdaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:00:31.934 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:31.931+0000 7f6df905f640 1 --2- 172.21.15.156:0/3725190475 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6df40fc360 0x7f6df40fc770 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:00:31.934 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:31.931+0000 7f6df905f640 1 -- 172.21.15.156:0/3725190475 >> 172.21.15.156:0/3725190475 conn(0x7f6df40f7b40 msgr2=0x7f6df40f9f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T10:00:31.934 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:31.931+0000 7f6df905f640 1 -- 172.21.15.156:0/3725190475 shutdown_connections 2024-07-16T10:00:31.934 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:31.931+0000 7f6df905f640 1 -- 172.21.15.156:0/3725190475 wait complete. 2024-07-16T10:00:31.934 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:31.932+0000 7f6df905f640 1 Processor -- start 2024-07-16T10:00:31.934 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:31.932+0000 7f6df905f640 1 -- start start 2024-07-16T10:00:31.935 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:31.932+0000 7f6df905f640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6df40fc360 0x7f6df410cff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:00:31.935 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:31.932+0000 7f6df905f640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6df40fd630 0x7f6df410d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:00:31.935 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:31.932+0000 7f6df905f640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6df410db80 con 0x7f6df40fc360 2024-07-16T10:00:31.935 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:31.932+0000 7f6df905f640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6df410e350 con 0x7f6df40fd630 2024-07-16T10:00:31.935 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:31.932+0000 7f6df2d9d640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6df40fc360 0x7f6df410cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:00:31.935 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:31.932+0000 7f6df2d9d640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6df40fc360 0x7f6df410cff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:46808/0 (socket says 172.21.15.156:46808) 2024-07-16T10:00:31.935 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:31.932+0000 7f6df2d9d640 1 -- 172.21.15.156:0/73134298 learned_addr learned my addr 172.21.15.156:0/73134298 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T10:00:31.935 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:31.932+0000 7f6debfff640 1 --2- 172.21.15.156:0/73134298 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6df40fd630 0x7f6df410d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:00:31.935 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:31.932+0000 7f6df2d9d640 1 -- 172.21.15.156:0/73134298 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6df40fd630 msgr2=0x7f6df410d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:00:31.935 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:31.932+0000 7f6df2d9d640 1 --2- 172.21.15.156:0/73134298 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6df40fd630 0x7f6df410d530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:00:31.935 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:31.932+0000 7f6df2d9d640 1 -- 172.21.15.156:0/73134298 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6de4005040 con 0x7f6df40fc360 2024-07-16T10:00:31.935 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:31.933+0000 7f6df2d9d640 1 --2- 172.21.15.156:0/73134298 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6df40fc360 0x7f6df410cff0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f6de4009f80 tx=0x7f6de4009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T10:00:31.937 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:31.933+0000 7f6deb7fe640 1 -- 172.21.15.156:0/73134298 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6de4038070 con 0x7f6df40fc360 2024-07-16T10:00:31.937 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:31.933+0000 7f6deb7fe640 1 -- 172.21.15.156:0/73134298 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6de4032970 con 0x7f6df40fc360 2024-07-16T10:00:31.937 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:31.933+0000 7f6df905f640 1 -- 172.21.15.156:0/73134298 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6df410e580 con 0x7f6df40fc360 2024-07-16T10:00:31.937 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:31.933+0000 7f6deb7fe640 1 -- 172.21.15.156:0/73134298 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6de4033860 con 0x7f6df40fc360 2024-07-16T10:00:31.937 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:31.933+0000 7f6df905f640 1 -- 172.21.15.156:0/73134298 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6df410eac0 con 0x7f6df40fc360 2024-07-16T10:00:31.937 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:31.934+0000 7f6df905f640 1 -- 172.21.15.156:0/73134298 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6df40fb7c0 con 0x7f6df40fc360 2024-07-16T10:00:31.937 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:31.934+0000 7f6deb7fe640 1 -- 172.21.15.156:0/73134298 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 38) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f6de4030020 con 0x7f6df40fc360 2024-07-16T10:00:31.937 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:31.935+0000 7f6deb7fe640 1 --2- 172.21.15.156:0/73134298 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f6de006e0b0 0x7f6de0070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:00:31.937 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:31.935+0000 7f6deb7fe640 1 -- 172.21.15.156:0/73134298 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f6de40adaa0 con 0x7f6df40fc360 2024-07-16T10:00:31.937 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:31.935+0000 7f6debfff640 1 --2- 172.21.15.156:0/73134298 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f6de006e0b0 0x7f6de0070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:00:31.937 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:31.935+0000 7f6debfff640 1 --2- 172.21.15.156:0/73134298 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f6de006e0b0 0x7f6de0070560 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f6ddc007fd0 tx=0x7f6ddc009e90 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T10:00:31.940 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:31.939+0000 7f6deb7fe640 1 -- 172.21.15.156:0/73134298 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187716 (secure 0 0 0) 0x7f6de4076e20 con 0x7f6df40fc360 2024-07-16T10:00:32.187 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:32.185+0000 7f6df905f640 1 -- 172.21.15.156:0/73134298 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f6df404b530 con 0x7f6df40fc360 2024-07-16T10:00:32.187 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:32.186+0000 7f6deb7fe640 1 -- 172.21.15.156:0/73134298 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7f6de407a4e0 con 0x7f6df40fc360 2024-07-16T10:00:32.187 INFO:teuthology.orchestra.run.smithi156.stdout:HEALTH_OK 2024-07-16T10:00:32.190 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:32.188+0000 7f6df905f640 1 -- 172.21.15.156:0/73134298 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f6de006e0b0 msgr2=0x7f6de0070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:00:32.190 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:32.188+0000 7f6df905f640 1 --2- 172.21.15.156:0/73134298 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f6de006e0b0 0x7f6de0070560 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f6ddc007fd0 tx=0x7f6ddc009e90 comp rx=0 tx=0).stop 2024-07-16T10:00:32.190 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:32.188+0000 7f6df905f640 1 -- 172.21.15.156:0/73134298 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6df40fc360 msgr2=0x7f6df410cff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:00:32.190 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:32.188+0000 7f6df905f640 1 --2- 172.21.15.156:0/73134298 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6df40fc360 0x7f6df410cff0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f6de4009f80 tx=0x7f6de4009da0 comp rx=0 tx=0).stop 2024-07-16T10:00:32.190 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:32.188+0000 7f6df905f640 1 -- 172.21.15.156:0/73134298 shutdown_connections 2024-07-16T10:00:32.190 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:32.188+0000 7f6df905f640 1 --2- 172.21.15.156:0/73134298 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f6de006e0b0 0x7f6de0070560 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:00:32.190 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:32.188+0000 7f6df905f640 1 --2- 172.21.15.156:0/73134298 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6df40fd630 0x7f6df410d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:00:32.190 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:32.188+0000 7f6df905f640 1 --2- 172.21.15.156:0/73134298 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6df40fc360 0x7f6df410cff0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:00:32.190 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:32.188+0000 7f6df905f640 1 -- 172.21.15.156:0/73134298 >> 172.21.15.156:0/73134298 conn(0x7f6df40f7b40 msgr2=0x7f6df40f9220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T10:00:32.190 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:32.188+0000 7f6df905f640 1 -- 172.21.15.156:0/73134298 shutdown_connections 2024-07-16T10:00:32.190 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:00:32.188+0000 7f6df905f640 1 -- 172.21.15.156:0/73134298 wait complete. 2024-07-16T10:00:32.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:00:32 smithi156 ceph-mon[93066]: from='client.? 172.21.15.156:0/983633744' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-07-16T10:00:32.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:00:32 smithi156 ceph-mon[93066]: from='client.34304 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-16T10:00:32.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:00:32 smithi156 ceph-mon[93066]: from='client.? 172.21.15.156:0/73134298' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-07-16T10:00:32.745 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:00:32 smithi176 ceph-mon[81689]: from='client.? 172.21.15.156:0/983633744' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-07-16T10:00:32.745 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:00:32 smithi176 ceph-mon[81689]: from='client.34304 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-16T10:00:32.745 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:00:32 smithi176 ceph-mon[81689]: from='client.? 172.21.15.156:0/73134298' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-07-16T10:00:33.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:00:33 smithi156 ceph-mon[93066]: pgmap v100: 65 pgs: 65 active+clean; 253 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-07-16T10:00:33.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:00:33 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-07-16T10:00:33.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:00:33 smithi156 ceph-mon[93066]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-07-16T10:00:33.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:00:33 smithi156 ceph-mon[93066]: Upgrade: osd.3 is safe to restart 2024-07-16T10:00:33.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:00:33 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:00:33.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:00:33 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-16T10:00:33.750 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:00:33 smithi176 ceph-mon[81689]: pgmap v100: 65 pgs: 65 active+clean; 253 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-07-16T10:00:33.751 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:00:33 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-07-16T10:00:33.751 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:00:33 smithi176 ceph-mon[81689]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-07-16T10:00:33.751 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:00:33 smithi176 ceph-mon[81689]: Upgrade: osd.3 is safe to restart 2024-07-16T10:00:33.751 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:00:33 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:00:33.751 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:00:33 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-16T10:00:35.924 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:00:35 smithi156 ceph-mon[93066]: pgmap v101: 65 pgs: 65 active+clean; 253 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-07-16T10:00:35.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:00:35 smithi156 ceph-mon[93066]: Upgrade: Updating osd.3 2024-07-16T10:00:35.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:00:35 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:00:35.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:00:35 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-07-16T10:00:35.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:00:35 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-16T10:00:35.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:00:35 smithi156 ceph-mon[93066]: Deploying daemon osd.3 on smithi176 2024-07-16T10:00:35.960 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:00:35 smithi176 ceph-mon[81689]: pgmap v101: 65 pgs: 65 active+clean; 253 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-07-16T10:00:35.960 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:00:35 smithi176 ceph-mon[81689]: Upgrade: Updating osd.3 2024-07-16T10:00:35.960 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:00:35 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:00:35.961 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:00:35 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-07-16T10:00:35.961 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:00:35 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-16T10:00:35.961 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:00:35 smithi176 ceph-mon[81689]: Deploying daemon osd.3 on smithi176 2024-07-16T10:00:37.806 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:00:37 smithi176 ceph-mon[81689]: pgmap v102: 65 pgs: 65 active+clean; 253 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-07-16T10:00:37.807 INFO:journalctl@ceph.osd.3.smithi176.stdout:Jul 16 10:00:37 smithi176 systemd[1]: Stopping Ceph osd.3 for 26c91c26-4358-11ef-bcad-c7b262605968... 2024-07-16T10:00:37.924 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:00:37 smithi156 ceph-mon[93066]: pgmap v102: 65 pgs: 65 active+clean; 253 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-07-16T10:00:38.094 INFO:journalctl@ceph.osd.3.smithi176.stdout:Jul 16 10:00:37 smithi176 ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-3[42094]: 2024-07-16T10:00:37.887+0000 7fc83887c640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-07-16T10:00:38.094 INFO:journalctl@ceph.osd.3.smithi176.stdout:Jul 16 10:00:37 smithi176 ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-3[42094]: 2024-07-16T10:00:37.887+0000 7fc83887c640 -1 osd.3 64 *** Got signal Terminated *** 2024-07-16T10:00:38.094 INFO:journalctl@ceph.osd.3.smithi176.stdout:Jul 16 10:00:37 smithi176 ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-3[42094]: 2024-07-16T10:00:37.887+0000 7fc83887c640 -1 osd.3 64 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-07-16T10:00:38.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:00:38 smithi156 ceph-mon[93066]: osd.3 marked itself down and dead 2024-07-16T10:00:39.093 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:00:38 smithi176 ceph-mon[81689]: osd.3 marked itself down and dead 2024-07-16T10:00:39.430 INFO:journalctl@ceph.osd.3.smithi176.stdout:Jul 16 10:00:39 smithi176 podman[90604]: 2024-07-16 10:00:39.097265164 +0000 UTC m=+1.393589900 container died 733861ba84dce5d85830356bf4005bc813a6e4b15109eceef1fa258226b4f92a (image=quay.ceph.io/ceph-ci/ceph@sha256:7b86ef6bffe0efc5f1aefc9fef8c854a67a511c50993acee3ac684f23bde83df, name=ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-3, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=quincy-e6231b0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240709, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.36.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-07-16T10:00:39.430 INFO:journalctl@ceph.osd.3.smithi176.stdout:Jul 16 10:00:39 smithi176 podman[90604]: 2024-07-16 10:00:39.429421614 +0000 UTC m=+1.725746334 container cleanup 733861ba84dce5d85830356bf4005bc813a6e4b15109eceef1fa258226b4f92a (image=quay.ceph.io/ceph-ci/ceph@sha256:7b86ef6bffe0efc5f1aefc9fef8c854a67a511c50993acee3ac684f23bde83df, name=ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-3, ceph=True, org.label-schema.build-date=20240709, maintainer=Guillaume Abrioux , io.buildah.version=1.36.0, CEPH_POINT_RELEASE=, RELEASE=quincy-e6231b0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image) 2024-07-16T10:00:39.843 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:00:39 smithi176 ceph-mon[81689]: pgmap v103: 65 pgs: 65 active+clean; 253 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-07-16T10:00:39.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:00:39 smithi176 ceph-mon[81689]: Health check failed: 1 osds down (OSD_DOWN) 2024-07-16T10:00:39.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:00:39 smithi176 ceph-mon[81689]: osdmap e65: 6 total, 5 up, 6 in 2024-07-16T10:00:39.844 INFO:journalctl@ceph.osd.3.smithi176.stdout:Jul 16 10:00:39 smithi176 bash[90604]: ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-3 2024-07-16T10:00:39.844 INFO:journalctl@ceph.osd.3.smithi176.stdout:Jul 16 10:00:39 smithi176 podman[90633]: 2024-07-16 10:00:39.630752282 +0000 UTC m=+0.529221746 container remove 733861ba84dce5d85830356bf4005bc813a6e4b15109eceef1fa258226b4f92a (image=quay.ceph.io/ceph-ci/ceph@sha256:7b86ef6bffe0efc5f1aefc9fef8c854a67a511c50993acee3ac684f23bde83df, name=ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-3, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.36.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240709, RELEASE=quincy-e6231b0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-07-16T10:00:39.924 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:00:39 smithi156 ceph-mon[93066]: pgmap v103: 65 pgs: 65 active+clean; 253 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-07-16T10:00:39.924 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:00:39 smithi156 ceph-mon[93066]: Health check failed: 1 osds down (OSD_DOWN) 2024-07-16T10:00:39.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:00:39 smithi156 ceph-mon[93066]: osdmap e65: 6 total, 5 up, 6 in 2024-07-16T10:00:40.344 INFO:journalctl@ceph.osd.3.smithi176.stdout:Jul 16 10:00:40 smithi176 podman[90719]: 2024-07-16 10:00:39.967284627 +0000 UTC m=+0.017078865 image pull 16f7feb043422ce37a2471403d4bc52f550d49cef4b677482112fcc0db3f67ab quay.ceph.io/ceph-ci/ceph@sha256:c15ec3aa8e6c0a9c12ee24ff79f3e7dd4d01b3d1a702dfeb7d7fff9b0811a393 2024-07-16T10:00:40.344 INFO:journalctl@ceph.osd.3.smithi176.stdout:Jul 16 10:00:40 smithi176 podman[90719]: 2024-07-16 10:00:40.081746406 +0000 UTC m=+0.131540641 container create 6b80f4b818e2876627fd2fae8c7d79c0fe8b5976df12c9ea0ac3c8a084761015 (image=quay.ceph.io/ceph-ci/ceph@sha256:c15ec3aa8e6c0a9c12ee24ff79f3e7dd4d01b3d1a702dfeb7d7fff9b0811a393, name=ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-3-deactivate, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-adk4-testing-2024-07-15-1832-squid-a6ca461, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240709, io.buildah.version=1.36.0, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-07-16T10:00:40.344 INFO:journalctl@ceph.osd.3.smithi176.stdout:Jul 16 10:00:40 smithi176 podman[90719]: 2024-07-16 10:00:40.267328975 +0000 UTC m=+0.317123216 container init 6b80f4b818e2876627fd2fae8c7d79c0fe8b5976df12c9ea0ac3c8a084761015 (image=quay.ceph.io/ceph-ci/ceph@sha256:c15ec3aa8e6c0a9c12ee24ff79f3e7dd4d01b3d1a702dfeb7d7fff9b0811a393, name=ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-3-deactivate, io.buildah.version=1.36.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240709, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-adk4-testing-2024-07-15-1832-squid-a6ca461, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-07-16T10:00:40.344 INFO:journalctl@ceph.osd.3.smithi176.stdout:Jul 16 10:00:40 smithi176 podman[90719]: 2024-07-16 10:00:40.270764776 +0000 UTC m=+0.320559013 container start 6b80f4b818e2876627fd2fae8c7d79c0fe8b5976df12c9ea0ac3c8a084761015 (image=quay.ceph.io/ceph-ci/ceph@sha256:c15ec3aa8e6c0a9c12ee24ff79f3e7dd4d01b3d1a702dfeb7d7fff9b0811a393, name=ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-3-deactivate, RELEASE=wip-adk4-testing-2024-07-15-1832-squid-a6ca461, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.36.0, GIT_CLEAN=True, org.label-schema.build-date=20240709, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-07-16T10:00:40.661 INFO:journalctl@ceph.osd.3.smithi176.stdout:Jul 16 10:00:40 smithi176 podman[90719]: 2024-07-16 10:00:40.385448294 +0000 UTC m=+0.435242539 container attach 6b80f4b818e2876627fd2fae8c7d79c0fe8b5976df12c9ea0ac3c8a084761015 (image=quay.ceph.io/ceph-ci/ceph@sha256:c15ec3aa8e6c0a9c12ee24ff79f3e7dd4d01b3d1a702dfeb7d7fff9b0811a393, name=ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-3-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , ceph=True, RELEASE=wip-adk4-testing-2024-07-15-1832-squid-a6ca461, io.buildah.version=1.36.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240709, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-07-16T10:00:40.661 INFO:journalctl@ceph.osd.3.smithi176.stdout:Jul 16 10:00:40 smithi176 podman[90719]: 2024-07-16 10:00:40.464435134 +0000 UTC m=+0.514229377 container died 6b80f4b818e2876627fd2fae8c7d79c0fe8b5976df12c9ea0ac3c8a084761015 (image=quay.ceph.io/ceph-ci/ceph@sha256:c15ec3aa8e6c0a9c12ee24ff79f3e7dd4d01b3d1a702dfeb7d7fff9b0811a393, name=ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-3-deactivate, org.label-schema.license=GPLv2, org.label-schema.build-date=20240709, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.36.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-adk4-testing-2024-07-15-1832-squid-a6ca461) 2024-07-16T10:00:40.924 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:00:40 smithi156 ceph-mon[93066]: osdmap e66: 6 total, 5 up, 6 in 2024-07-16T10:00:41.046 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:00:40 smithi176 ceph-mon[81689]: osdmap e66: 6 total, 5 up, 6 in 2024-07-16T10:00:41.344 INFO:journalctl@ceph.osd.3.smithi176.stdout:Jul 16 10:00:41 smithi176 podman[90743]: 2024-07-16 10:00:41.046096228 +0000 UTC m=+0.570739805 container remove 6b80f4b818e2876627fd2fae8c7d79c0fe8b5976df12c9ea0ac3c8a084761015 (image=quay.ceph.io/ceph-ci/ceph@sha256:c15ec3aa8e6c0a9c12ee24ff79f3e7dd4d01b3d1a702dfeb7d7fff9b0811a393, name=ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-3-deactivate, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-adk4-testing-2024-07-15-1832-squid-a6ca461, org.label-schema.build-date=20240709, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0) 2024-07-16T10:00:41.344 INFO:journalctl@ceph.osd.3.smithi176.stdout:Jul 16 10:00:41 smithi176 systemd[1]: ceph-26c91c26-4358-11ef-bcad-c7b262605968@osd.3.service: Deactivated successfully. 2024-07-16T10:00:41.344 INFO:journalctl@ceph.osd.3.smithi176.stdout:Jul 16 10:00:41 smithi176 systemd[1]: Stopped Ceph osd.3 for 26c91c26-4358-11ef-bcad-c7b262605968. 2024-07-16T10:00:41.344 INFO:journalctl@ceph.osd.3.smithi176.stdout:Jul 16 10:00:41 smithi176 systemd[1]: ceph-26c91c26-4358-11ef-bcad-c7b262605968@osd.3.service: Consumed 1min 11.662s CPU time. 2024-07-16T10:00:41.344 INFO:journalctl@ceph.osd.3.smithi176.stdout:Jul 16 10:00:41 smithi176 systemd[1]: Starting Ceph osd.3 for 26c91c26-4358-11ef-bcad-c7b262605968... 2024-07-16T10:00:41.783 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:00:41 smithi176 ceph-mon[81689]: pgmap v106: 65 pgs: 22 peering, 5 stale+active+clean, 38 active+clean; 253 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-07-16T10:00:41.783 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:00:41 smithi176 ceph-mon[81689]: Health check failed: Reduced data availability: 11 pgs peering (PG_AVAILABILITY) 2024-07-16T10:00:41.783 INFO:journalctl@ceph.osd.3.smithi176.stdout:Jul 16 10:00:41 smithi176 podman[90843]: 2024-07-16 10:00:41.380762705 +0000 UTC m=+0.017117034 image pull 16f7feb043422ce37a2471403d4bc52f550d49cef4b677482112fcc0db3f67ab quay.ceph.io/ceph-ci/ceph@sha256:c15ec3aa8e6c0a9c12ee24ff79f3e7dd4d01b3d1a702dfeb7d7fff9b0811a393 2024-07-16T10:00:41.783 INFO:journalctl@ceph.osd.3.smithi176.stdout:Jul 16 10:00:41 smithi176 podman[90843]: 2024-07-16 10:00:41.488656574 +0000 UTC m=+0.125010897 container create e3cad9bcbb60339f03862e6a143d8184279c3bd371104fdf84a912df0fc01820 (image=quay.ceph.io/ceph-ci/ceph@sha256:c15ec3aa8e6c0a9c12ee24ff79f3e7dd4d01b3d1a702dfeb7d7fff9b0811a393, name=ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-3-activate, GIT_CLEAN=True, RELEASE=wip-adk4-testing-2024-07-15-1832-squid-a6ca461, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240709, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-07-16T10:00:41.784 INFO:journalctl@ceph.osd.3.smithi176.stdout:Jul 16 10:00:41 smithi176 podman[90843]: 2024-07-16 10:00:41.691403121 +0000 UTC m=+0.327757460 container init e3cad9bcbb60339f03862e6a143d8184279c3bd371104fdf84a912df0fc01820 (image=quay.ceph.io/ceph-ci/ceph@sha256:c15ec3aa8e6c0a9c12ee24ff79f3e7dd4d01b3d1a702dfeb7d7fff9b0811a393, name=ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-3-activate, RELEASE=wip-adk4-testing-2024-07-15-1832-squid-a6ca461, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240709, org.label-schema.vendor=CentOS, io.buildah.version=1.36.0, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD) 2024-07-16T10:00:41.784 INFO:journalctl@ceph.osd.3.smithi176.stdout:Jul 16 10:00:41 smithi176 podman[90843]: 2024-07-16 10:00:41.695000749 +0000 UTC m=+0.331355072 container start e3cad9bcbb60339f03862e6a143d8184279c3bd371104fdf84a912df0fc01820 (image=quay.ceph.io/ceph-ci/ceph@sha256:c15ec3aa8e6c0a9c12ee24ff79f3e7dd4d01b3d1a702dfeb7d7fff9b0811a393, name=ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-3-activate, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240709, org.label-schema.vendor=CentOS, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-adk4-testing-2024-07-15-1832-squid-a6ca461, maintainer=Guillaume Abrioux ) 2024-07-16T10:00:41.784 INFO:journalctl@ceph.osd.3.smithi176.stdout:Jul 16 10:00:41 smithi176 podman[90843]: 2024-07-16 10:00:41.782626982 +0000 UTC m=+0.418981317 container attach e3cad9bcbb60339f03862e6a143d8184279c3bd371104fdf84a912df0fc01820 (image=quay.ceph.io/ceph-ci/ceph@sha256:c15ec3aa8e6c0a9c12ee24ff79f3e7dd4d01b3d1a702dfeb7d7fff9b0811a393, name=ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-3-activate, RELEASE=wip-adk4-testing-2024-07-15-1832-squid-a6ca461, io.buildah.version=1.36.0, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240709) 2024-07-16T10:00:41.924 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:00:41 smithi156 ceph-mon[93066]: pgmap v106: 65 pgs: 22 peering, 5 stale+active+clean, 38 active+clean; 253 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-07-16T10:00:41.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:00:41 smithi156 ceph-mon[93066]: Health check failed: Reduced data availability: 11 pgs peering (PG_AVAILABILITY) 2024-07-16T10:00:42.034 INFO:journalctl@ceph.osd.3.smithi176.stdout:Jul 16 10:00:41 smithi176 ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-3-activate[90858]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-07-16T10:00:42.034 INFO:journalctl@ceph.osd.3.smithi176.stdout:Jul 16 10:00:41 smithi176 bash[90843]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-07-16T10:00:42.034 INFO:journalctl@ceph.osd.3.smithi176.stdout:Jul 16 10:00:41 smithi176 ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-3-activate[90858]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-07-16T10:00:42.034 INFO:journalctl@ceph.osd.3.smithi176.stdout:Jul 16 10:00:41 smithi176 bash[90843]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-07-16T10:00:42.034 INFO:journalctl@ceph.osd.3.smithi176.stdout:Jul 16 10:00:42 smithi176 ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-3-activate[90858]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-07-16T10:00:42.034 INFO:journalctl@ceph.osd.3.smithi176.stdout:Jul 16 10:00:42 smithi176 bash[90843]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-07-16T10:00:42.034 INFO:journalctl@ceph.osd.3.smithi176.stdout:Jul 16 10:00:42 smithi176 ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-3-activate[90858]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-3 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-07-16T10:00:42.034 INFO:journalctl@ceph.osd.3.smithi176.stdout:Jul 16 10:00:42 smithi176 bash[90843]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-3 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-07-16T10:00:42.344 INFO:journalctl@ceph.osd.3.smithi176.stdout:Jul 16 10:00:42 smithi176 ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-3-activate[90858]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-07-16T10:00:42.344 INFO:journalctl@ceph.osd.3.smithi176.stdout:Jul 16 10:00:42 smithi176 bash[90843]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-07-16T10:00:42.344 INFO:journalctl@ceph.osd.3.smithi176.stdout:Jul 16 10:00:42 smithi176 ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-3-activate[90858]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-07-16T10:00:42.344 INFO:journalctl@ceph.osd.3.smithi176.stdout:Jul 16 10:00:42 smithi176 bash[90843]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-07-16T10:00:42.344 INFO:journalctl@ceph.osd.3.smithi176.stdout:Jul 16 10:00:42 smithi176 ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-3-activate[90858]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3/block 2024-07-16T10:00:42.344 INFO:journalctl@ceph.osd.3.smithi176.stdout:Jul 16 10:00:42 smithi176 bash[90843]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3/block 2024-07-16T10:00:42.344 INFO:journalctl@ceph.osd.3.smithi176.stdout:Jul 16 10:00:42 smithi176 ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-3-activate[90858]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-07-16T10:00:42.345 INFO:journalctl@ceph.osd.3.smithi176.stdout:Jul 16 10:00:42 smithi176 bash[90843]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-07-16T10:00:42.345 INFO:journalctl@ceph.osd.3.smithi176.stdout:Jul 16 10:00:42 smithi176 ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-3-activate[90858]: --> ceph-volume raw activate successful for osd ID: 3 2024-07-16T10:00:42.345 INFO:journalctl@ceph.osd.3.smithi176.stdout:Jul 16 10:00:42 smithi176 bash[90843]: --> ceph-volume raw activate successful for osd ID: 3 2024-07-16T10:00:42.345 INFO:journalctl@ceph.osd.3.smithi176.stdout:Jul 16 10:00:42 smithi176 podman[90843]: 2024-07-16 10:00:42.078623075 +0000 UTC m=+0.714977403 container died e3cad9bcbb60339f03862e6a143d8184279c3bd371104fdf84a912df0fc01820 (image=quay.ceph.io/ceph-ci/ceph@sha256:c15ec3aa8e6c0a9c12ee24ff79f3e7dd4d01b3d1a702dfeb7d7fff9b0811a393, name=ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-3-activate, maintainer=Guillaume Abrioux , RELEASE=wip-adk4-testing-2024-07-15-1832-squid-a6ca461, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240709, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0) 2024-07-16T10:00:42.821 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:00:42 smithi176 ceph-mon[81689]: Health check failed: Degraded data redundancy: 26/318 objects degraded (8.176%), 9 pgs degraded (PG_DEGRADED) 2024-07-16T10:00:42.822 INFO:journalctl@ceph.osd.3.smithi176.stdout:Jul 16 10:00:42 smithi176 podman[90995]: 2024-07-16 10:00:42.603065326 +0000 UTC m=+0.518817983 container remove e3cad9bcbb60339f03862e6a143d8184279c3bd371104fdf84a912df0fc01820 (image=quay.ceph.io/ceph-ci/ceph@sha256:c15ec3aa8e6c0a9c12ee24ff79f3e7dd4d01b3d1a702dfeb7d7fff9b0811a393, name=ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-3-activate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240709, org.label-schema.schema-version=1.0, RELEASE=wip-adk4-testing-2024-07-15-1832-squid-a6ca461, org.label-schema.vendor=CentOS) 2024-07-16T10:00:43.073 INFO:journalctl@ceph.osd.3.smithi176.stdout:Jul 16 10:00:42 smithi176 podman[91053]: 2024-07-16 10:00:42.723136657 +0000 UTC m=+0.017775200 image pull 16f7feb043422ce37a2471403d4bc52f550d49cef4b677482112fcc0db3f67ab quay.ceph.io/ceph-ci/ceph@sha256:c15ec3aa8e6c0a9c12ee24ff79f3e7dd4d01b3d1a702dfeb7d7fff9b0811a393 2024-07-16T10:00:43.074 INFO:journalctl@ceph.osd.3.smithi176.stdout:Jul 16 10:00:42 smithi176 podman[91053]: 2024-07-16 10:00:42.8373117 +0000 UTC m=+0.131950232 container create d216534d60278450eeca86761d90f8e64ee51d272a186899f32cb2836e07576a (image=quay.ceph.io/ceph-ci/ceph@sha256:c15ec3aa8e6c0a9c12ee24ff79f3e7dd4d01b3d1a702dfeb7d7fff9b0811a393, name=ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-3, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240709, org.label-schema.license=GPLv2, io.buildah.version=1.36.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-adk4-testing-2024-07-15-1832-squid-a6ca461, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-07-16T10:00:43.174 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:00:42 smithi156 ceph-mon[93066]: Health check failed: Degraded data redundancy: 26/318 objects degraded (8.176%), 9 pgs degraded (PG_DEGRADED) 2024-07-16T10:00:43.344 INFO:journalctl@ceph.osd.3.smithi176.stdout:Jul 16 10:00:43 smithi176 podman[91053]: 2024-07-16 10:00:43.072884787 +0000 UTC m=+0.367523339 container init d216534d60278450eeca86761d90f8e64ee51d272a186899f32cb2836e07576a (image=quay.ceph.io/ceph-ci/ceph@sha256:c15ec3aa8e6c0a9c12ee24ff79f3e7dd4d01b3d1a702dfeb7d7fff9b0811a393, name=ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-3, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-adk4-testing-2024-07-15-1832-squid-a6ca461, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.36.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240709, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-07-16T10:00:43.344 INFO:journalctl@ceph.osd.3.smithi176.stdout:Jul 16 10:00:43 smithi176 podman[91053]: 2024-07-16 10:00:43.076727244 +0000 UTC m=+0.371365789 container start d216534d60278450eeca86761d90f8e64ee51d272a186899f32cb2836e07576a (image=quay.ceph.io/ceph-ci/ceph@sha256:c15ec3aa8e6c0a9c12ee24ff79f3e7dd4d01b3d1a702dfeb7d7fff9b0811a393, name=ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-3, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240709, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-adk4-testing-2024-07-15-1832-squid-a6ca461, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.36.0) 2024-07-16T10:00:43.344 INFO:journalctl@ceph.osd.3.smithi176.stdout:Jul 16 10:00:43 smithi176 ceph-osd[91072]: -- 172.21.15.176:0/4179189339 <== mon.1 v2:172.21.15.176:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x564a541b21e0 con 0x564a53365c00 2024-07-16T10:00:43.344 INFO:journalctl@ceph.osd.3.smithi176.stdout:Jul 16 10:00:43 smithi176 bash[91053]: d216534d60278450eeca86761d90f8e64ee51d272a186899f32cb2836e07576a 2024-07-16T10:00:43.344 INFO:journalctl@ceph.osd.3.smithi176.stdout:Jul 16 10:00:43 smithi176 systemd[1]: Started Ceph osd.3 for 26c91c26-4358-11ef-bcad-c7b262605968. 2024-07-16T10:00:43.944 INFO:journalctl@ceph.osd.3.smithi176.stdout:Jul 16 10:00:43 smithi176 ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-3[91068]: 2024-07-16T10:00:43.624+0000 7f9d164a0640 -1 Falling back to public interface 2024-07-16T10:00:44.480 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:00:44 smithi176 ceph-mon[81689]: pgmap v107: 65 pgs: 7 active+undersized, 22 peering, 9 active+undersized+degraded, 27 active+clean; 253 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 26/318 objects degraded (8.176%) 2024-07-16T10:00:44.480 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:00:44 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:00:44.480 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:00:44 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:00:44.480 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:00:44 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-16T10:00:44.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:00:44 smithi156 ceph-mon[93066]: pgmap v107: 65 pgs: 7 active+undersized, 22 peering, 9 active+undersized+degraded, 27 active+clean; 253 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 26/318 objects degraded (8.176%) 2024-07-16T10:00:44.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:00:44 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:00:44.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:00:44 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:00:44.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:00:44 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-16T10:00:45.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:00:45 smithi176 ceph-mon[81689]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 11 pgs peering) 2024-07-16T10:00:45.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:00:45 smithi156 ceph-mon[93066]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 11 pgs peering) 2024-07-16T10:00:46.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:00:46 smithi176 ceph-mon[81689]: pgmap v108: 65 pgs: 19 active+undersized, 19 active+undersized+degraded, 27 active+clean; 253 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 65/318 objects degraded (20.440%) 2024-07-16T10:00:46.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:00:46 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:00:46.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:00:46 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:00:46.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:00:46 smithi156 ceph-mon[93066]: pgmap v108: 65 pgs: 19 active+undersized, 19 active+undersized+degraded, 27 active+clean; 253 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 65/318 objects degraded (20.440%) 2024-07-16T10:00:46.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:00:46 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:00:46.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:00:46 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:00:47.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:00:47 smithi176 ceph-mon[81689]: pgmap v109: 65 pgs: 19 active+undersized, 19 active+undersized+degraded, 27 active+clean; 253 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 65/318 objects degraded (20.440%) 2024-07-16T10:00:47.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:00:47 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:00:47.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:00:47 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:00:47.924 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:00:47 smithi156 ceph-mon[93066]: pgmap v109: 65 pgs: 19 active+undersized, 19 active+undersized+degraded, 27 active+clean; 253 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 65/318 objects degraded (20.440%) 2024-07-16T10:00:47.924 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:00:47 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:00:47.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:00:47 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:00:48.931 INFO:journalctl@ceph.osd.3.smithi176.stdout:Jul 16 10:00:48 smithi176 ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-3[91068]: 2024-07-16T10:00:48.671+0000 7f9d164a0640 -1 osd.3 64 log_to_monitors true 2024-07-16T10:00:49.468 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:00:49 smithi176 ceph-mon[81689]: pgmap v110: 65 pgs: 19 active+undersized, 19 active+undersized+degraded, 27 active+clean; 253 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 65/318 objects degraded (20.440%) 2024-07-16T10:00:49.468 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:00:49 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:00:49.468 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:00:49 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-16T10:00:49.468 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:00:49 smithi176 ceph-mon[81689]: from='osd.3 [v2:172.21.15.176:6800/1497836353,v1:172.21.15.176:6801/1497836353]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-07-16T10:00:49.468 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:00:49 smithi176 ceph-mon[81689]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-07-16T10:00:49.635 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:00:49 smithi156 ceph-mon[93066]: pgmap v110: 65 pgs: 19 active+undersized, 19 active+undersized+degraded, 27 active+clean; 253 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 65/318 objects degraded (20.440%) 2024-07-16T10:00:49.636 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:00:49 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:00:49.636 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:00:49 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-16T10:00:49.636 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:00:49 smithi156 ceph-mon[93066]: from='osd.3 [v2:172.21.15.176:6800/1497836353,v1:172.21.15.176:6801/1497836353]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-07-16T10:00:49.636 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:00:49 smithi156 ceph-mon[93066]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-07-16T10:00:49.844 INFO:journalctl@ceph.osd.3.smithi176.stdout:Jul 16 10:00:49 smithi176 ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-3[91068]: 2024-07-16T10:00:49.466+0000 7f9d0e23a640 -1 osd.3 64 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-07-16T10:00:50.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:00:50 smithi176 ceph-mon[81689]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-07-16T10:00:50.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:00:50 smithi176 ceph-mon[81689]: from='osd.3 [v2:172.21.15.176:6800/1497836353,v1:172.21.15.176:6801/1497836353]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi176", "root=default"]}]: dispatch 2024-07-16T10:00:50.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:00:50 smithi176 ceph-mon[81689]: osdmap e67: 6 total, 5 up, 6 in 2024-07-16T10:00:50.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:00:50 smithi176 ceph-mon[81689]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi176", "root=default"]}]: dispatch 2024-07-16T10:00:50.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:00:50 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:00:50.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:00:50 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:00:50.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:00:50 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-16T10:00:50.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:00:50 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-16T10:00:50.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:00:50 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:00:50.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:00:50 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-16T10:00:50.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:00:50 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:00:50.845 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:00:50 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:00:50.845 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:00:50 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:00:50.845 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:00:50 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-07-16T10:00:50.845 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:00:50 smithi176 ceph-mon[81689]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-07-16T10:00:50.845 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:00:50 smithi176 ceph-mon[81689]: Upgrade: unsafe to stop osd(s) at this time (18 PGs are or would become offline) 2024-07-16T10:00:50.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:00:50 smithi156 ceph-mon[93066]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-07-16T10:00:50.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:00:50 smithi156 ceph-mon[93066]: from='osd.3 [v2:172.21.15.176:6800/1497836353,v1:172.21.15.176:6801/1497836353]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi176", "root=default"]}]: dispatch 2024-07-16T10:00:50.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:00:50 smithi156 ceph-mon[93066]: osdmap e67: 6 total, 5 up, 6 in 2024-07-16T10:00:50.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:00:50 smithi156 ceph-mon[93066]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi176", "root=default"]}]: dispatch 2024-07-16T10:00:50.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:00:50 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:00:50.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:00:50 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:00:50.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:00:50 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-16T10:00:50.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:00:50 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-16T10:00:50.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:00:50 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:00:50.926 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:00:50 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-16T10:00:50.926 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:00:50 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:00:50.926 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:00:50 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:00:50.926 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:00:50 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:00:50.926 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:00:50 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-07-16T10:00:50.926 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:00:50 smithi156 ceph-mon[93066]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-07-16T10:00:50.926 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:00:50 smithi156 ceph-mon[93066]: Upgrade: unsafe to stop osd(s) at this time (18 PGs are or would become offline) 2024-07-16T10:00:51.843 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:00:51 smithi176 ceph-mon[81689]: pgmap v112: 65 pgs: 19 active+undersized, 19 active+undersized+degraded, 27 active+clean; 253 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 0 op/s; 65/318 objects degraded (20.440%) 2024-07-16T10:00:51.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:00:51 smithi176 ceph-mon[81689]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-07-16T10:00:51.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:00:51 smithi176 ceph-mon[81689]: osd.3 [v2:172.21.15.176:6800/1497836353,v1:172.21.15.176:6801/1497836353] boot 2024-07-16T10:00:51.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:00:51 smithi176 ceph-mon[81689]: osdmap e68: 6 total, 6 up, 6 in 2024-07-16T10:00:51.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:00:51 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-07-16T10:00:51.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:00:51 smithi176 ceph-mon[81689]: osdmap e69: 6 total, 6 up, 6 in 2024-07-16T10:00:51.924 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:00:51 smithi156 ceph-mon[93066]: pgmap v112: 65 pgs: 19 active+undersized, 19 active+undersized+degraded, 27 active+clean; 253 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 0 op/s; 65/318 objects degraded (20.440%) 2024-07-16T10:00:51.924 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:00:51 smithi156 ceph-mon[93066]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-07-16T10:00:51.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:00:51 smithi156 ceph-mon[93066]: osd.3 [v2:172.21.15.176:6800/1497836353,v1:172.21.15.176:6801/1497836353] boot 2024-07-16T10:00:51.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:00:51 smithi156 ceph-mon[93066]: osdmap e68: 6 total, 6 up, 6 in 2024-07-16T10:00:51.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:00:51 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-07-16T10:00:51.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:00:51 smithi156 ceph-mon[93066]: osdmap e69: 6 total, 6 up, 6 in 2024-07-16T10:00:52.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:00:52 smithi156 ceph-mon[93066]: Health check update: Degraded data redundancy: 65/318 objects degraded (20.440%), 19 pgs degraded (PG_DEGRADED) 2024-07-16T10:00:52.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:00:52 smithi176 ceph-mon[81689]: Health check update: Degraded data redundancy: 65/318 objects degraded (20.440%), 19 pgs degraded (PG_DEGRADED) 2024-07-16T10:00:53.843 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:00:53 smithi176 ceph-mon[81689]: pgmap v115: 65 pgs: 11 peering, 13 active+undersized, 14 active+undersized+degraded, 27 active+clean; 253 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s; 50/318 objects degraded (15.723%) 2024-07-16T10:00:53.924 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:00:53 smithi156 ceph-mon[93066]: pgmap v115: 65 pgs: 11 peering, 13 active+undersized, 14 active+undersized+degraded, 27 active+clean; 253 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s; 50/318 objects degraded (15.723%) 2024-07-16T10:00:55.843 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:00:55 smithi176 ceph-mon[81689]: pgmap v116: 65 pgs: 11 peering, 6 active+undersized, 10 active+undersized+degraded, 38 active+clean; 253 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.3 KiB/s rd, 3 op/s; 40/318 objects degraded (12.579%) 2024-07-16T10:00:55.924 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:00:55 smithi156 ceph-mon[93066]: pgmap v116: 65 pgs: 11 peering, 6 active+undersized, 10 active+undersized+degraded, 38 active+clean; 253 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.3 KiB/s rd, 3 op/s; 40/318 objects degraded (12.579%) 2024-07-16T10:00:56.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:00:56 smithi156 ceph-mon[93066]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 40/318 objects degraded (12.579%), 10 pgs degraded) 2024-07-16T10:00:56.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:00:56 smithi156 ceph-mon[93066]: Cluster is now healthy 2024-07-16T10:00:56.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:00:56 smithi176 ceph-mon[81689]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 40/318 objects degraded (12.579%), 10 pgs degraded) 2024-07-16T10:00:56.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:00:56 smithi176 ceph-mon[81689]: Cluster is now healthy 2024-07-16T10:00:57.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:00:57 smithi176 ceph-mon[81689]: pgmap v117: 65 pgs: 4 peering, 61 active+clean; 253 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-07-16T10:00:57.924 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:00:57 smithi156 ceph-mon[93066]: pgmap v117: 65 pgs: 4 peering, 61 active+clean; 253 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-07-16T10:00:59.843 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:00:59 smithi176 ceph-mon[81689]: pgmap v118: 65 pgs: 65 active+clean; 253 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 895 B/s rd, 1 op/s 2024-07-16T10:00:59.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:00:59 smithi156 ceph-mon[93066]: pgmap v118: 65 pgs: 65 active+clean; 253 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 895 B/s rd, 1 op/s 2024-07-16T10:01:01.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:01:01 smithi176 ceph-mon[81689]: pgmap v119: 65 pgs: 65 active+clean; 253 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 2 op/s 2024-07-16T10:01:01.924 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:01:01 smithi156 ceph-mon[93066]: pgmap v119: 65 pgs: 65 active+clean; 253 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 2 op/s 2024-07-16T10:01:02.284 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:02.280+0000 7fee8a0c3640 1 -- 172.21.15.156:0/726403422 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fee840fb260 msgr2=0x7fee840fb670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:01:02.284 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:02.280+0000 7fee8a0c3640 1 --2- 172.21.15.156:0/726403422 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fee840fb260 0x7fee840fb670 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fee780048d0 tx=0x7fee78009da0 comp rx=0 tx=0).stop 2024-07-16T10:01:02.284 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:02.280+0000 7fee8a0c3640 1 -- 172.21.15.156:0/726403422 shutdown_connections 2024-07-16T10:01:02.284 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:02.280+0000 7fee8a0c3640 1 --2- 172.21.15.156:0/726403422 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fee840fc530 0x7fee840fc9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:01:02.284 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:02.280+0000 7fee8a0c3640 1 --2- 172.21.15.156:0/726403422 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fee840fb260 0x7fee840fb670 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:01:02.284 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:02.280+0000 7fee8a0c3640 1 -- 172.21.15.156:0/726403422 >> 172.21.15.156:0/726403422 conn(0x7fee840f6a20 msgr2=0x7fee840f8e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T10:01:02.284 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:02.280+0000 7fee8a0c3640 1 -- 172.21.15.156:0/726403422 shutdown_connections 2024-07-16T10:01:02.285 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:02.280+0000 7fee8a0c3640 1 -- 172.21.15.156:0/726403422 wait complete. 2024-07-16T10:01:02.285 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:02.281+0000 7fee8a0c3640 1 Processor -- start 2024-07-16T10:01:02.285 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:02.281+0000 7fee8a0c3640 1 -- start start 2024-07-16T10:01:02.285 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:02.281+0000 7fee8a0c3640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fee840fb260 0x7fee8410fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:01:02.285 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:02.281+0000 7fee8a0c3640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fee840fc530 0x7fee84112250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:01:02.285 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:02.281+0000 7fee8a0c3640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee84066540 con 0x7fee840fc530 2024-07-16T10:01:02.285 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:02.281+0000 7fee8a0c3640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee840666b0 con 0x7fee840fb260 2024-07-16T10:01:02.285 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:02.282+0000 7fee82ffd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fee840fc530 0x7fee84112250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:01:02.285 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:02.282+0000 7fee82ffd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fee840fc530 0x7fee84112250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:42542/0 (socket says 172.21.15.156:42542) 2024-07-16T10:01:02.285 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:02.282+0000 7fee82ffd640 1 -- 172.21.15.156:0/4130549051 learned_addr learned my addr 172.21.15.156:0/4130549051 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T10:01:02.286 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:02.282+0000 7fee837fe640 1 --2- 172.21.15.156:0/4130549051 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fee840fb260 0x7fee8410fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:01:02.286 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:02.282+0000 7fee82ffd640 1 -- 172.21.15.156:0/4130549051 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fee840fb260 msgr2=0x7fee8410fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:01:02.286 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:02.282+0000 7fee82ffd640 1 --2- 172.21.15.156:0/4130549051 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fee840fb260 0x7fee8410fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:01:02.286 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:02.282+0000 7fee82ffd640 1 -- 172.21.15.156:0/4130549051 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee78005040 con 0x7fee840fc530 2024-07-16T10:01:02.286 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:02.282+0000 7fee82ffd640 1 --2- 172.21.15.156:0/4130549051 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fee840fc530 0x7fee84112250 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fee7400c790 tx=0x7fee740043c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T10:01:02.286 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:02.282+0000 7fee6bfff640 1 -- 172.21.15.156:0/4130549051 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fee7400a820 con 0x7fee840fc530 2024-07-16T10:01:02.286 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:02.282+0000 7fee8a0c3640 1 -- 172.21.15.156:0/4130549051 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee841127f0 con 0x7fee840fc530 2024-07-16T10:01:02.286 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:02.282+0000 7fee6bfff640 1 -- 172.21.15.156:0/4130549051 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fee7400ae50 con 0x7fee840fc530 2024-07-16T10:01:02.286 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:02.282+0000 7fee6bfff640 1 -- 172.21.15.156:0/4130549051 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fee740115e0 con 0x7fee840fc530 2024-07-16T10:01:02.286 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:02.282+0000 7fee8a0c3640 1 -- 172.21.15.156:0/4130549051 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee84112d60 con 0x7fee840fc530 2024-07-16T10:01:02.287 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:02.283+0000 7fee8a0c3640 1 -- 172.21.15.156:0/4130549051 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee840fa6c0 con 0x7fee840fc530 2024-07-16T10:01:02.287 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:02.284+0000 7fee6bfff640 1 -- 172.21.15.156:0/4130549051 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 38) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fee74012400 con 0x7fee840fc530 2024-07-16T10:01:02.287 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:02.284+0000 7fee6bfff640 1 --2- 172.21.15.156:0/4130549051 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7fee6c06e0b0 0x7fee6c070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:01:02.287 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:02.285+0000 7fee6bfff640 1 -- 172.21.15.156:0/4130549051 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fee7408dce0 con 0x7fee840fc530 2024-07-16T10:01:02.288 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:02.285+0000 7fee837fe640 1 --2- 172.21.15.156:0/4130549051 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7fee6c06e0b0 0x7fee6c070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:01:02.288 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:02.285+0000 7fee837fe640 1 --2- 172.21.15.156:0/4130549051 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7fee6c06e0b0 0x7fee6c070560 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fee78000c00 tx=0x7fee78035040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T10:01:02.290 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:02.289+0000 7fee6bfff640 1 -- 172.21.15.156:0/4130549051 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187716 (secure 0 0 0) 0x7fee740570e0 con 0x7fee840fc530 2024-07-16T10:01:02.484 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:02.483+0000 7fee8a0c3640 1 -- 172.21.15.156:0/4130549051 --> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fee84100840 con 0x7fee6c06e0b0 2024-07-16T10:01:02.485 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:02.484+0000 7fee6bfff640 1 -- 172.21.15.156:0/4130549051 <== mgr.34104 v2:172.21.15.156:6800/3367765292 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7fee84100840 con 0x7fee6c06e0b0 2024-07-16T10:01:02.487 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:02.486+0000 7fee8a0c3640 1 -- 172.21.15.156:0/4130549051 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7fee6c06e0b0 msgr2=0x7fee6c070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:01:02.487 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:02.486+0000 7fee8a0c3640 1 --2- 172.21.15.156:0/4130549051 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7fee6c06e0b0 0x7fee6c070560 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fee78000c00 tx=0x7fee78035040 comp rx=0 tx=0).stop 2024-07-16T10:01:02.488 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:02.486+0000 7fee8a0c3640 1 -- 172.21.15.156:0/4130549051 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fee840fc530 msgr2=0x7fee84112250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:01:02.488 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:02.486+0000 7fee8a0c3640 1 --2- 172.21.15.156:0/4130549051 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fee840fc530 0x7fee84112250 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fee7400c790 tx=0x7fee740043c0 comp rx=0 tx=0).stop 2024-07-16T10:01:02.488 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:02.486+0000 7fee8a0c3640 1 -- 172.21.15.156:0/4130549051 shutdown_connections 2024-07-16T10:01:02.488 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:02.486+0000 7fee8a0c3640 1 --2- 172.21.15.156:0/4130549051 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7fee6c06e0b0 0x7fee6c070560 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:01:02.488 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:02.486+0000 7fee8a0c3640 1 --2- 172.21.15.156:0/4130549051 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fee840fc530 0x7fee84112250 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:01:02.488 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:02.486+0000 7fee8a0c3640 1 --2- 172.21.15.156:0/4130549051 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fee840fb260 0x7fee8410fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:01:02.488 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:02.486+0000 7fee8a0c3640 1 -- 172.21.15.156:0/4130549051 >> 172.21.15.156:0/4130549051 conn(0x7fee840f6a20 msgr2=0x7fee840f8170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T10:01:02.488 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:02.486+0000 7fee8a0c3640 1 -- 172.21.15.156:0/4130549051 shutdown_connections 2024-07-16T10:01:02.488 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:02.486+0000 7fee8a0c3640 1 -- 172.21.15.156:0/4130549051 wait complete. 2024-07-16T10:01:02.496 INFO:teuthology.orchestra.run.smithi156.stdout:true 2024-07-16T10:01:02.569 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:02.568+0000 7f4ceef32640 1 -- 172.21.15.156:0/4244025549 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4ce80ff4f0 msgr2=0x7f4ce81018d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:01:02.570 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:02.568+0000 7f4ceef32640 1 --2- 172.21.15.156:0/4244025549 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4ce80ff4f0 0x7f4ce81018d0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f4cd40048d0 tx=0x7f4cd4009da0 comp rx=0 tx=0).stop 2024-07-16T10:01:02.570 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:02.568+0000 7f4ceef32640 1 -- 172.21.15.156:0/4244025549 shutdown_connections 2024-07-16T10:01:02.570 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:02.568+0000 7f4ceef32640 1 --2- 172.21.15.156:0/4244025549 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4ce80ff4f0 0x7f4ce81018d0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:01:02.570 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:02.568+0000 7f4ceef32640 1 --2- 172.21.15.156:0/4244025549 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4ce80659f0 0x7f4ce80fefb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:01:02.570 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:02.568+0000 7f4ceef32640 1 -- 172.21.15.156:0/4244025549 >> 172.21.15.156:0/4244025549 conn(0x7f4ce80f6a20 msgr2=0x7f4ce80f8e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T10:01:02.570 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:02.568+0000 7f4ceef32640 1 -- 172.21.15.156:0/4244025549 shutdown_connections 2024-07-16T10:01:02.570 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:02.568+0000 7f4ceef32640 1 -- 172.21.15.156:0/4244025549 wait complete. 2024-07-16T10:01:02.572 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:02.569+0000 7f4ceef32640 1 Processor -- start 2024-07-16T10:01:02.572 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:02.569+0000 7f4ceef32640 1 -- start start 2024-07-16T10:01:02.572 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:02.569+0000 7f4ceef32640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4ce80659f0 0x7f4ce810cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:01:02.572 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:02.569+0000 7f4ceef32640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4ce80ff4f0 0x7f4ce810d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:01:02.572 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:02.569+0000 7f4ceef32640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ce810db30 con 0x7f4ce80ff4f0 2024-07-16T10:01:02.572 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:02.569+0000 7f4ceef32640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ce810e300 con 0x7f4ce80659f0 2024-07-16T10:01:02.572 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:02.569+0000 7f4ce7fff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4ce80ff4f0 0x7f4ce810d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:01:02.572 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:02.569+0000 7f4ce7fff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4ce80ff4f0 0x7f4ce810d4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:42552/0 (socket says 172.21.15.156:42552) 2024-07-16T10:01:02.573 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:02.569+0000 7f4ce7fff640 1 -- 172.21.15.156:0/716564808 learned_addr learned my addr 172.21.15.156:0/716564808 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T10:01:02.573 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:02.570+0000 7f4ceccce640 1 --2- 172.21.15.156:0/716564808 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4ce80659f0 0x7f4ce810cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:01:02.573 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:02.570+0000 7f4ce7fff640 1 -- 172.21.15.156:0/716564808 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4ce80659f0 msgr2=0x7f4ce810cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:01:02.573 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:02.570+0000 7f4ce7fff640 1 --2- 172.21.15.156:0/716564808 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4ce80659f0 0x7f4ce810cfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:01:02.573 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:02.570+0000 7f4ce7fff640 1 -- 172.21.15.156:0/716564808 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cd4005040 con 0x7f4ce80ff4f0 2024-07-16T10:01:02.573 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:02.570+0000 7f4ce7fff640 1 --2- 172.21.15.156:0/716564808 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4ce80ff4f0 0x7f4ce810d4e0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f4cd4003080 tx=0x7f4cd4009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T10:01:02.573 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:02.570+0000 7f4ce57fa640 1 -- 172.21.15.156:0/716564808 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4cd4038070 con 0x7f4ce80ff4f0 2024-07-16T10:01:02.573 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:02.570+0000 7f4ceef32640 1 -- 172.21.15.156:0/716564808 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ce810e530 con 0x7f4ce80ff4f0 2024-07-16T10:01:02.573 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:02.570+0000 7f4ce57fa640 1 -- 172.21.15.156:0/716564808 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4cd4009760 con 0x7f4ce80ff4f0 2024-07-16T10:01:02.573 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:02.570+0000 7f4ce57fa640 1 -- 172.21.15.156:0/716564808 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4cd403dcc0 con 0x7f4ce80ff4f0 2024-07-16T10:01:02.573 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:02.570+0000 7f4ceef32640 1 -- 172.21.15.156:0/716564808 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ce810ea70 con 0x7f4ce80ff4f0 2024-07-16T10:01:02.574 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:02.571+0000 7f4ceef32640 1 -- 172.21.15.156:0/716564808 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ce80fa6a0 con 0x7f4ce80ff4f0 2024-07-16T10:01:02.574 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:02.572+0000 7f4ce57fa640 1 -- 172.21.15.156:0/716564808 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 38) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f4cd4030020 con 0x7f4ce80ff4f0 2024-07-16T10:01:02.574 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:02.572+0000 7f4ce57fa640 1 --2- 172.21.15.156:0/716564808 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f4cd806e0b0 0x7f4cd8070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:01:02.574 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:02.573+0000 7f4ceccce640 1 --2- 172.21.15.156:0/716564808 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f4cd806e0b0 0x7f4cd8070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:01:02.575 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:02.573+0000 7f4ce57fa640 1 -- 172.21.15.156:0/716564808 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f4cd40ae9c0 con 0x7f4ce80ff4f0 2024-07-16T10:01:02.575 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:02.573+0000 7f4ceccce640 1 --2- 172.21.15.156:0/716564808 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f4cd806e0b0 0x7f4cd8070560 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f4ce8065750 tx=0x7f4cdc009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T10:01:02.578 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:02.577+0000 7f4ce57fa640 1 -- 172.21.15.156:0/716564808 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187716 (secure 0 0 0) 0x7f4cd4077d40 con 0x7f4ce80ff4f0 2024-07-16T10:01:02.771 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:02.770+0000 7f4ceef32640 1 -- 172.21.15.156:0/716564808 --> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f4ce810edc0 con 0x7f4cd806e0b0 2024-07-16T10:01:02.772 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:02.771+0000 7f4ce57fa640 1 -- 172.21.15.156:0/716564808 <== mgr.34104 v2:172.21.15.156:6800/3367765292 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7f4ce810edc0 con 0x7f4cd806e0b0 2024-07-16T10:01:02.775 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:02.773+0000 7f4ceef32640 1 -- 172.21.15.156:0/716564808 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f4cd806e0b0 msgr2=0x7f4cd8070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:01:02.775 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:02.773+0000 7f4ceef32640 1 --2- 172.21.15.156:0/716564808 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f4cd806e0b0 0x7f4cd8070560 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f4ce8065750 tx=0x7f4cdc009040 comp rx=0 tx=0).stop 2024-07-16T10:01:02.775 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:02.773+0000 7f4ceef32640 1 -- 172.21.15.156:0/716564808 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4ce80ff4f0 msgr2=0x7f4ce810d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:01:02.775 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:02.773+0000 7f4ceef32640 1 --2- 172.21.15.156:0/716564808 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4ce80ff4f0 0x7f4ce810d4e0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f4cd4003080 tx=0x7f4cd4009da0 comp rx=0 tx=0).stop 2024-07-16T10:01:02.775 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:02.773+0000 7f4ceef32640 1 -- 172.21.15.156:0/716564808 shutdown_connections 2024-07-16T10:01:02.775 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:02.773+0000 7f4ceef32640 1 --2- 172.21.15.156:0/716564808 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f4cd806e0b0 0x7f4cd8070560 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:01:02.775 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:02.773+0000 7f4ceef32640 1 --2- 172.21.15.156:0/716564808 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4ce80ff4f0 0x7f4ce810d4e0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:01:02.775 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:02.773+0000 7f4ceef32640 1 --2- 172.21.15.156:0/716564808 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4ce80659f0 0x7f4ce810cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:01:02.775 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:02.773+0000 7f4ceef32640 1 -- 172.21.15.156:0/716564808 >> 172.21.15.156:0/716564808 conn(0x7f4ce80f6a20 msgr2=0x7f4ce80f6df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T10:01:02.775 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:02.773+0000 7f4ceef32640 1 -- 172.21.15.156:0/716564808 shutdown_connections 2024-07-16T10:01:02.776 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:02.773+0000 7f4ceef32640 1 -- 172.21.15.156:0/716564808 wait complete. 2024-07-16T10:01:02.856 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:02.854+0000 7fde04ca3640 1 -- 172.21.15.156:0/72410090 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fde000fd600 msgr2=0x7fde000fda70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:01:02.856 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:02.854+0000 7fde04ca3640 1 --2- 172.21.15.156:0/72410090 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fde000fd600 0x7fde000fda70 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fddf40048d0 tx=0x7fddf4009da0 comp rx=0 tx=0).stop 2024-07-16T10:01:02.856 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:02.855+0000 7fde04ca3640 1 -- 172.21.15.156:0/72410090 shutdown_connections 2024-07-16T10:01:02.856 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:02.855+0000 7fde04ca3640 1 --2- 172.21.15.156:0/72410090 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fde000fd600 0x7fde000fda70 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:01:02.856 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:02.855+0000 7fde04ca3640 1 --2- 172.21.15.156:0/72410090 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fde000fc330 0x7fde000fc740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:01:02.857 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:02.855+0000 7fde04ca3640 1 -- 172.21.15.156:0/72410090 >> 172.21.15.156:0/72410090 conn(0x7fde000f7af0 msgr2=0x7fde000f9f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T10:01:02.857 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:02.855+0000 7fde04ca3640 1 -- 172.21.15.156:0/72410090 shutdown_connections 2024-07-16T10:01:02.857 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:02.855+0000 7fde04ca3640 1 -- 172.21.15.156:0/72410090 wait complete. 2024-07-16T10:01:02.858 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:02.855+0000 7fde04ca3640 1 Processor -- start 2024-07-16T10:01:02.858 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:02.855+0000 7fde04ca3640 1 -- start start 2024-07-16T10:01:02.859 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:02.856+0000 7fde04ca3640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fde000fc330 0x7fde0010cf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:01:02.859 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:02.856+0000 7fde04ca3640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fde000fd600 0x7fde0010d490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:01:02.859 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:02.856+0000 7fde04ca3640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde0010dae0 con 0x7fde000fc330 2024-07-16T10:01:02.859 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:02.856+0000 7fde04ca3640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde0010e300 con 0x7fde000fd600 2024-07-16T10:01:02.859 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:02.856+0000 7fddfe59c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fde000fc330 0x7fde0010cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:01:02.859 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:02.856+0000 7fddfe59c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fde000fc330 0x7fde0010cf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:51942/0 (socket says 172.21.15.156:51942) 2024-07-16T10:01:02.859 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:02.856+0000 7fddfe59c640 1 -- 172.21.15.156:0/3498285290 learned_addr learned my addr 172.21.15.156:0/3498285290 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T10:01:02.859 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:02.856+0000 7fddfdd9b640 1 --2- 172.21.15.156:0/3498285290 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fde000fd600 0x7fde0010d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:01:02.859 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:02.856+0000 7fddfe59c640 1 -- 172.21.15.156:0/3498285290 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fde000fd600 msgr2=0x7fde0010d490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:01:02.859 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:02.856+0000 7fddfe59c640 1 --2- 172.21.15.156:0/3498285290 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fde000fd600 0x7fde0010d490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:01:02.859 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:02.856+0000 7fddfe59c640 1 -- 172.21.15.156:0/3498285290 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddf4005040 con 0x7fde000fc330 2024-07-16T10:01:02.859 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:02.856+0000 7fddfe59c640 1 --2- 172.21.15.156:0/3498285290 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fde000fc330 0x7fde0010cf50 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fddf000db00 tx=0x7fddf0004e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T10:01:02.860 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:02.856+0000 7fddeeffd640 1 -- 172.21.15.156:0/3498285290 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fddf0017070 con 0x7fde000fc330 2024-07-16T10:01:02.860 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:02.857+0000 7fddeeffd640 1 -- 172.21.15.156:0/3498285290 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fddf000f040 con 0x7fde000fc330 2024-07-16T10:01:02.860 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:02.857+0000 7fde04ca3640 1 -- 172.21.15.156:0/3498285290 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde0010e590 con 0x7fde000fc330 2024-07-16T10:01:02.860 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:02.857+0000 7fddeeffd640 1 -- 172.21.15.156:0/3498285290 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fddf0003a70 con 0x7fde000fc330 2024-07-16T10:01:02.860 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:02.857+0000 7fde04ca3640 1 -- 172.21.15.156:0/3498285290 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde0010eb00 con 0x7fde000fc330 2024-07-16T10:01:02.861 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:02.858+0000 7fddeeffd640 1 -- 172.21.15.156:0/3498285290 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 38) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fddf0003bd0 con 0x7fde000fc330 2024-07-16T10:01:02.861 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:02.858+0000 7fde04ca3640 1 -- 172.21.15.156:0/3498285290 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde001056b0 con 0x7fde000fc330 2024-07-16T10:01:02.861 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:02.858+0000 7fddeeffd640 1 --2- 172.21.15.156:0/3498285290 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7fdde806e0b0 0x7fdde8070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:01:02.861 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:02.859+0000 7fddeeffd640 1 -- 172.21.15.156:0/3498285290 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fddf008da90 con 0x7fde000fc330 2024-07-16T10:01:02.861 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:02.859+0000 7fddfdd9b640 1 --2- 172.21.15.156:0/3498285290 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7fdde806e0b0 0x7fdde8070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:01:02.861 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:02.859+0000 7fddfdd9b640 1 --2- 172.21.15.156:0/3498285290 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7fdde806e0b0 0x7fdde8070560 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fddf40043d0 tx=0x7fddf4035040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T10:01:02.864 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:02.863+0000 7fddeeffd640 1 -- 172.21.15.156:0/3498285290 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187716 (secure 0 0 0) 0x7fddf0056e90 con 0x7fde000fc330 2024-07-16T10:01:03.047 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:03.045+0000 7fde04ca3640 1 -- 172.21.15.156:0/3498285290 --> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fde0010ee80 con 0x7fdde806e0b0 2024-07-16T10:01:03.055 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:03.054+0000 7fddeeffd640 1 -- 172.21.15.156:0/3498285290 <== mgr.34104 v2:172.21.15.156:6800/3367765292 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7fde0010ee80 con 0x7fdde806e0b0 2024-07-16T10:01:03.056 INFO:teuthology.orchestra.run.smithi156.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-07-16T10:01:03.056 INFO:teuthology.orchestra.run.smithi156.stdout:alertmanager.smithi156 smithi156 *:9093,9094 running (4m) 49s ago 13m 25.0M - 0.25.0 c8568f914cd2 d524a2c3ff15 2024-07-16T10:01:03.056 INFO:teuthology.orchestra.run.smithi156.stdout:crash.smithi156 smithi156 running (2m) 49s ago 13m 6899k - 19.0.0-3495-ga6ca461b 16f7feb04342 11648cf29af4 2024-07-16T10:01:03.056 INFO:teuthology.orchestra.run.smithi156.stdout:crash.smithi176 smithi176 running (2m) 17s ago 12m 6899k - 19.0.0-3495-ga6ca461b 16f7feb04342 67f3ccc034f8 2024-07-16T10:01:03.056 INFO:teuthology.orchestra.run.smithi156.stdout:grafana.smithi156 smithi156 *:3000 running (4m) 49s ago 12m 77.8M - 9.4.7 954c08fa6188 deb9e1bfeb66 2024-07-16T10:01:03.056 INFO:teuthology.orchestra.run.smithi156.stdout:mds.cephfs.smithi156.evwwaz smithi156 running (7m) 49s ago 7m 67.2M - 17.2.7-1104-ge6231b04 d5de5f953b95 7c0ff9ae43f8 2024-07-16T10:01:03.056 INFO:teuthology.orchestra.run.smithi156.stdout:mds.cephfs.smithi156.rdibys smithi156 running (8m) 49s ago 8m 120M - 17.2.7-1104-ge6231b04 d5de5f953b95 b985cd1e6924 2024-07-16T10:01:03.056 INFO:teuthology.orchestra.run.smithi156.stdout:mds.cephfs.smithi176.dvvbud smithi176 running (8m) 17s ago 8m 108M - 17.2.7-1104-ge6231b04 d5de5f953b95 1b00147950f9 2024-07-16T10:01:03.056 INFO:teuthology.orchestra.run.smithi156.stdout:mds.cephfs.smithi176.wahbsi smithi176 running (7m) 17s ago 7m 62.2M - 17.2.7-1104-ge6231b04 d5de5f953b95 866664af2311 2024-07-16T10:01:03.057 INFO:teuthology.orchestra.run.smithi156.stdout:mgr.smithi156.zkvylv smithi156 *:8443,9283,8765 running (4m) 49s ago 14m 497M - 19.0.0-3495-ga6ca461b 16f7feb04342 ee53bd604b7c 2024-07-16T10:01:03.057 INFO:teuthology.orchestra.run.smithi156.stdout:mgr.smithi176.dqbrvx smithi176 *:8443,9283,8765 running (3m) 17s ago 12m 440M - 19.0.0-3495-ga6ca461b 16f7feb04342 ce9eaae4c2d0 2024-07-16T10:01:03.057 INFO:teuthology.orchestra.run.smithi156.stdout:mon.smithi156 smithi156 running (3m) 49s ago 14m 45.4M 2048M 19.0.0-3495-ga6ca461b 16f7feb04342 77b6ec8c50eb 2024-07-16T10:01:03.057 INFO:teuthology.orchestra.run.smithi156.stdout:mon.smithi176 smithi176 running (3m) 17s ago 12m 38.7M 2048M 19.0.0-3495-ga6ca461b 16f7feb04342 ef92814cec2b 2024-07-16T10:01:03.057 INFO:teuthology.orchestra.run.smithi156.stdout:node-exporter.smithi156 smithi156 *:9100 running (4m) 49s ago 13m 17.2M - 1.5.0 0da6a335fe13 e5d86f23a14c 2024-07-16T10:01:03.057 INFO:teuthology.orchestra.run.smithi156.stdout:node-exporter.smithi176 smithi176 *:9100 running (4m) 17s ago 12m 15.3M - 1.5.0 0da6a335fe13 ea842728144a 2024-07-16T10:01:03.057 INFO:teuthology.orchestra.run.smithi156.stdout:osd.0 smithi156 running (2m) 49s ago 11m 93.7M 1557M 19.0.0-3495-ga6ca461b 16f7feb04342 8633df1689cc 2024-07-16T10:01:03.057 INFO:teuthology.orchestra.run.smithi156.stdout:osd.1 smithi156 running (88s) 49s ago 11m 101M 1557M 19.0.0-3495-ga6ca461b 16f7feb04342 e6fae9653b46 2024-07-16T10:01:03.057 INFO:teuthology.orchestra.run.smithi156.stdout:osd.2 smithi156 running (54s) 49s ago 10m 11.8M 1557M 19.0.0-3495-ga6ca461b 16f7feb04342 c6ecf18de9bf 2024-07-16T10:01:03.057 INFO:teuthology.orchestra.run.smithi156.stdout:osd.3 smithi176 running (20s) 17s ago 10m 12.1M 2581M 19.0.0-3495-ga6ca461b 16f7feb04342 d216534d6027 2024-07-16T10:01:03.057 INFO:teuthology.orchestra.run.smithi156.stdout:osd.4 smithi176 running (9m) 17s ago 9m 229M 2581M 17.2.7-1104-ge6231b04 d5de5f953b95 ef0c67cec086 2024-07-16T10:01:03.057 INFO:teuthology.orchestra.run.smithi156.stdout:osd.5 smithi176 running (9m) 17s ago 9m 224M 2581M 17.2.7-1104-ge6231b04 d5de5f953b95 996e15448a52 2024-07-16T10:01:03.057 INFO:teuthology.orchestra.run.smithi156.stdout:prometheus.smithi156 smithi156 *:9095 running (3m) 49s ago 12m 64.9M - 2.43.0 a07b618ecd1d 20c195ecd665 2024-07-16T10:01:03.058 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:03.056+0000 7fde04ca3640 1 -- 172.21.15.156:0/3498285290 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7fdde806e0b0 msgr2=0x7fdde8070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:01:03.058 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:03.056+0000 7fde04ca3640 1 --2- 172.21.15.156:0/3498285290 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7fdde806e0b0 0x7fdde8070560 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fddf40043d0 tx=0x7fddf4035040 comp rx=0 tx=0).stop 2024-07-16T10:01:03.058 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:03.056+0000 7fde04ca3640 1 -- 172.21.15.156:0/3498285290 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fde000fc330 msgr2=0x7fde0010cf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:01:03.059 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:03.056+0000 7fde04ca3640 1 --2- 172.21.15.156:0/3498285290 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fde000fc330 0x7fde0010cf50 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fddf000db00 tx=0x7fddf0004e60 comp rx=0 tx=0).stop 2024-07-16T10:01:03.059 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:03.056+0000 7fde04ca3640 1 -- 172.21.15.156:0/3498285290 shutdown_connections 2024-07-16T10:01:03.059 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:03.056+0000 7fde04ca3640 1 --2- 172.21.15.156:0/3498285290 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7fdde806e0b0 0x7fdde8070560 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:01:03.059 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:03.056+0000 7fde04ca3640 1 --2- 172.21.15.156:0/3498285290 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fde000fd600 0x7fde0010d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:01:03.059 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:03.056+0000 7fde04ca3640 1 --2- 172.21.15.156:0/3498285290 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fde000fc330 0x7fde0010cf50 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:01:03.059 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:03.056+0000 7fde04ca3640 1 -- 172.21.15.156:0/3498285290 >> 172.21.15.156:0/3498285290 conn(0x7fde000f7af0 msgr2=0x7fde000f91c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T10:01:03.059 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:03.057+0000 7fde04ca3640 1 -- 172.21.15.156:0/3498285290 shutdown_connections 2024-07-16T10:01:03.059 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:03.057+0000 7fde04ca3640 1 -- 172.21.15.156:0/3498285290 wait complete. 2024-07-16T10:01:03.140 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:03.136+0000 7f27e360b640 1 -- 172.21.15.156:0/2962681593 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f27dc0fc530 msgr2=0x7f27dc0fc9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:01:03.140 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:03.136+0000 7f27e360b640 1 --2- 172.21.15.156:0/2962681593 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f27dc0fc530 0x7f27dc0fc9a0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f27cc0048d0 tx=0x7f27cc009da0 comp rx=0 tx=0).stop 2024-07-16T10:01:03.140 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:03.136+0000 7f27e360b640 1 -- 172.21.15.156:0/2962681593 shutdown_connections 2024-07-16T10:01:03.140 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:03.136+0000 7f27e360b640 1 --2- 172.21.15.156:0/2962681593 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f27dc0fc530 0x7f27dc0fc9a0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:01:03.140 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:03.136+0000 7f27e360b640 1 --2- 172.21.15.156:0/2962681593 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f27dc0fb260 0x7f27dc0fb670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:01:03.141 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:03.136+0000 7f27e360b640 1 -- 172.21.15.156:0/2962681593 >> 172.21.15.156:0/2962681593 conn(0x7f27dc0f6a20 msgr2=0x7f27dc0f8e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T10:01:03.141 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:03.136+0000 7f27e360b640 1 -- 172.21.15.156:0/2962681593 shutdown_connections 2024-07-16T10:01:03.141 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:03.136+0000 7f27e360b640 1 -- 172.21.15.156:0/2962681593 wait complete. 2024-07-16T10:01:03.141 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:03.137+0000 7f27e360b640 1 Processor -- start 2024-07-16T10:01:03.141 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:03.137+0000 7f27e360b640 1 -- start start 2024-07-16T10:01:03.141 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:03.137+0000 7f27e360b640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f27dc0fb260 0x7f27dc10cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:01:03.142 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:03.137+0000 7f27e360b640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f27dc0fc530 0x7f27dc10d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:01:03.142 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:03.137+0000 7f27e360b640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27dc10db30 con 0x7f27dc0fb260 2024-07-16T10:01:03.142 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:03.137+0000 7f27e360b640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27dc10e350 con 0x7f27dc0fc530 2024-07-16T10:01:03.142 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:03.137+0000 7f27e13a7640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f27dc0fb260 0x7f27dc10cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:01:03.142 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:03.137+0000 7f27e13a7640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f27dc0fb260 0x7f27dc10cfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:51970/0 (socket says 172.21.15.156:51970) 2024-07-16T10:01:03.143 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:03.137+0000 7f27e13a7640 1 -- 172.21.15.156:0/2376034688 learned_addr learned my addr 172.21.15.156:0/2376034688 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T10:01:03.143 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:03.137+0000 7f27e0ba6640 1 --2- 172.21.15.156:0/2376034688 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f27dc0fc530 0x7f27dc10d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:01:03.143 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:03.137+0000 7f27e13a7640 1 -- 172.21.15.156:0/2376034688 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f27dc0fc530 msgr2=0x7f27dc10d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:01:03.143 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:03.137+0000 7f27e13a7640 1 --2- 172.21.15.156:0/2376034688 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f27dc0fc530 0x7f27dc10d4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:01:03.143 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:03.137+0000 7f27e13a7640 1 -- 172.21.15.156:0/2376034688 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27cc005040 con 0x7f27dc0fb260 2024-07-16T10:01:03.143 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:03.137+0000 7f27e13a7640 1 --2- 172.21.15.156:0/2376034688 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f27dc0fb260 0x7f27dc10cfa0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f27d800c810 tx=0x7f27d80043c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T10:01:03.144 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:03.138+0000 7f27d1ffb640 1 -- 172.21.15.156:0/2376034688 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f27d800a820 con 0x7f27dc0fb260 2024-07-16T10:01:03.144 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:03.138+0000 7f27d1ffb640 1 -- 172.21.15.156:0/2376034688 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f27d800ae50 con 0x7f27dc0fb260 2024-07-16T10:01:03.144 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:03.138+0000 7f27d1ffb640 1 -- 172.21.15.156:0/2376034688 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f27d80115c0 con 0x7f27dc0fb260 2024-07-16T10:01:03.144 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:03.138+0000 7f27e360b640 1 -- 172.21.15.156:0/2376034688 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27dc10e5e0 con 0x7f27dc0fb260 2024-07-16T10:01:03.144 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:03.138+0000 7f27e360b640 1 -- 172.21.15.156:0/2376034688 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27dc10eb80 con 0x7f27dc0fb260 2024-07-16T10:01:03.146 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:03.139+0000 7f27e360b640 1 -- 172.21.15.156:0/2376034688 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27dc0fa6c0 con 0x7f27dc0fb260 2024-07-16T10:01:03.146 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:03.139+0000 7f27d1ffb640 1 -- 172.21.15.156:0/2376034688 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 38) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f27d800a980 con 0x7f27dc0fb260 2024-07-16T10:01:03.146 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:03.140+0000 7f27d1ffb640 1 --2- 172.21.15.156:0/2376034688 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f27c806e0b0 0x7f27c8070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:01:03.146 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:03.140+0000 7f27d1ffb640 1 -- 172.21.15.156:0/2376034688 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f27d808e050 con 0x7f27dc0fb260 2024-07-16T10:01:03.146 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:03.140+0000 7f27e0ba6640 1 --2- 172.21.15.156:0/2376034688 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f27c806e0b0 0x7f27c8070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:01:03.146 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:03.140+0000 7f27e0ba6640 1 --2- 172.21.15.156:0/2376034688 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f27c806e0b0 0x7f27c8070560 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f27cc009f80 tx=0x7f27cc035040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T10:01:03.146 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:03.144+0000 7f27d1ffb640 1 -- 172.21.15.156:0/2376034688 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187716 (secure 0 0 0) 0x7f27d805af80 con 0x7f27dc0fb260 2024-07-16T10:01:03.394 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:03.393+0000 7f27e360b640 1 -- 172.21.15.156:0/2376034688 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f27dc04b530 con 0x7f27dc0fb260 2024-07-16T10:01:03.395 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:03.393+0000 7f27d1ffb640 1 -- 172.21.15.156:0/2376034688 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+838 (secure 0 0 0) 0x7f27d805ab10 con 0x7f27dc0fb260 2024-07-16T10:01:03.395 INFO:teuthology.orchestra.run.smithi156.stdout:{ 2024-07-16T10:01:03.395 INFO:teuthology.orchestra.run.smithi156.stdout: "mon": { 2024-07-16T10:01:03.395 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph version 19.0.0-3495-ga6ca461b (a6ca461be7cc1aff3658c2c6df4cff975a24449a) squid (rc)": 2 2024-07-16T10:01:03.396 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-07-16T10:01:03.396 INFO:teuthology.orchestra.run.smithi156.stdout: "mgr": { 2024-07-16T10:01:03.396 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph version 19.0.0-3495-ga6ca461b (a6ca461be7cc1aff3658c2c6df4cff975a24449a) squid (rc)": 2 2024-07-16T10:01:03.396 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-07-16T10:01:03.396 INFO:teuthology.orchestra.run.smithi156.stdout: "osd": { 2024-07-16T10:01:03.396 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph version 17.2.7-1104-ge6231b04 (e6231b04b52ebf3b9b31803282dd3074439e51ac) quincy (stable)": 2, 2024-07-16T10:01:03.396 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph version 19.0.0-3495-ga6ca461b (a6ca461be7cc1aff3658c2c6df4cff975a24449a) squid (rc)": 4 2024-07-16T10:01:03.396 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-07-16T10:01:03.396 INFO:teuthology.orchestra.run.smithi156.stdout: "mds": { 2024-07-16T10:01:03.396 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph version 17.2.7-1104-ge6231b04 (e6231b04b52ebf3b9b31803282dd3074439e51ac) quincy (stable)": 4 2024-07-16T10:01:03.396 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-07-16T10:01:03.397 INFO:teuthology.orchestra.run.smithi156.stdout: "overall": { 2024-07-16T10:01:03.397 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph version 17.2.7-1104-ge6231b04 (e6231b04b52ebf3b9b31803282dd3074439e51ac) quincy (stable)": 6, 2024-07-16T10:01:03.397 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph version 19.0.0-3495-ga6ca461b (a6ca461be7cc1aff3658c2c6df4cff975a24449a) squid (rc)": 8 2024-07-16T10:01:03.397 INFO:teuthology.orchestra.run.smithi156.stdout: } 2024-07-16T10:01:03.397 INFO:teuthology.orchestra.run.smithi156.stdout:} 2024-07-16T10:01:03.398 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:03.396+0000 7f27e360b640 1 -- 172.21.15.156:0/2376034688 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f27c806e0b0 msgr2=0x7f27c8070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:01:03.398 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:03.396+0000 7f27e360b640 1 --2- 172.21.15.156:0/2376034688 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f27c806e0b0 0x7f27c8070560 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f27cc009f80 tx=0x7f27cc035040 comp rx=0 tx=0).stop 2024-07-16T10:01:03.398 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:03.396+0000 7f27e360b640 1 -- 172.21.15.156:0/2376034688 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f27dc0fb260 msgr2=0x7f27dc10cfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:01:03.398 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:03.396+0000 7f27e360b640 1 --2- 172.21.15.156:0/2376034688 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f27dc0fb260 0x7f27dc10cfa0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f27d800c810 tx=0x7f27d80043c0 comp rx=0 tx=0).stop 2024-07-16T10:01:03.398 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:03.396+0000 7f27e360b640 1 -- 172.21.15.156:0/2376034688 shutdown_connections 2024-07-16T10:01:03.398 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:03.396+0000 7f27e360b640 1 --2- 172.21.15.156:0/2376034688 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f27c806e0b0 0x7f27c8070560 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:01:03.398 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:03.396+0000 7f27e360b640 1 --2- 172.21.15.156:0/2376034688 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f27dc0fc530 0x7f27dc10d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:01:03.398 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:03.396+0000 7f27e360b640 1 --2- 172.21.15.156:0/2376034688 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f27dc0fb260 0x7f27dc10cfa0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:01:03.398 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:03.396+0000 7f27e360b640 1 -- 172.21.15.156:0/2376034688 >> 172.21.15.156:0/2376034688 conn(0x7f27dc0f6a20 msgr2=0x7f27dc065e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T10:01:03.398 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:03.396+0000 7f27e360b640 1 -- 172.21.15.156:0/2376034688 shutdown_connections 2024-07-16T10:01:03.398 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:03.396+0000 7f27e360b640 1 -- 172.21.15.156:0/2376034688 wait complete. 2024-07-16T10:01:03.479 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:03.476+0000 7f3abbc0b640 1 -- 172.21.15.156:0/3484536996 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3ab40ff4b0 msgr2=0x7f3ab4101890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:01:03.479 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:03.476+0000 7f3abbc0b640 1 --2- 172.21.15.156:0/3484536996 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3ab40ff4b0 0x7f3ab4101890 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f3ab00048d0 tx=0x7f3ab0009da0 comp rx=0 tx=0).stop 2024-07-16T10:01:03.480 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:03.476+0000 7f3abbc0b640 1 -- 172.21.15.156:0/3484536996 shutdown_connections 2024-07-16T10:01:03.480 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:03.476+0000 7f3abbc0b640 1 --2- 172.21.15.156:0/3484536996 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3ab40ff4b0 0x7f3ab4101890 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:01:03.480 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:03.476+0000 7f3abbc0b640 1 --2- 172.21.15.156:0/3484536996 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3ab4066520 0x7f3ab40fef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:01:03.480 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:03.476+0000 7f3abbc0b640 1 -- 172.21.15.156:0/3484536996 >> 172.21.15.156:0/3484536996 conn(0x7f3ab40f6a20 msgr2=0x7f3ab40f8e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T10:01:03.480 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:03.476+0000 7f3abbc0b640 1 -- 172.21.15.156:0/3484536996 shutdown_connections 2024-07-16T10:01:03.480 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:03.476+0000 7f3abbc0b640 1 -- 172.21.15.156:0/3484536996 wait complete. 2024-07-16T10:01:03.480 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:03.477+0000 7f3abbc0b640 1 Processor -- start 2024-07-16T10:01:03.480 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:03.477+0000 7f3abbc0b640 1 -- start start 2024-07-16T10:01:03.480 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:03.477+0000 7f3abbc0b640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3ab4066520 0x7f3ab410d010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:01:03.480 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:03.477+0000 7f3abbc0b640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3ab40ff4b0 0x7f3ab410d550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:01:03.480 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:03.477+0000 7f3abbc0b640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ab410dba0 con 0x7f3ab40ff4b0 2024-07-16T10:01:03.480 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:03.477+0000 7f3abbc0b640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ab410e3c0 con 0x7f3ab4066520 2024-07-16T10:01:03.481 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:03.477+0000 7f3ab91a6640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3ab40ff4b0 0x7f3ab410d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:01:03.481 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:03.477+0000 7f3ab91a6640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3ab40ff4b0 0x7f3ab410d550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:51986/0 (socket says 172.21.15.156:51986) 2024-07-16T10:01:03.481 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:03.477+0000 7f3ab91a6640 1 -- 172.21.15.156:0/111416957 learned_addr learned my addr 172.21.15.156:0/111416957 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T10:01:03.481 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:03.477+0000 7f3ab91a6640 1 -- 172.21.15.156:0/111416957 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3ab4066520 msgr2=0x7f3ab410d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:01:03.481 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:03.477+0000 7f3ab91a6640 1 --2- 172.21.15.156:0/111416957 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3ab4066520 0x7f3ab410d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:01:03.481 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:03.477+0000 7f3ab91a6640 1 -- 172.21.15.156:0/111416957 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ab0005040 con 0x7f3ab40ff4b0 2024-07-16T10:01:03.481 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:03.478+0000 7f3ab91a6640 1 --2- 172.21.15.156:0/111416957 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3ab40ff4b0 0x7f3ab410d550 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f3ab0009f80 tx=0x7f3ab0009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T10:01:03.481 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:03.478+0000 7f3aaa7fc640 1 -- 172.21.15.156:0/111416957 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3ab0038070 con 0x7f3ab40ff4b0 2024-07-16T10:01:03.481 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:03.478+0000 7f3abbc0b640 1 -- 172.21.15.156:0/111416957 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ab410e5f0 con 0x7f3ab40ff4b0 2024-07-16T10:01:03.481 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:03.478+0000 7f3aaa7fc640 1 -- 172.21.15.156:0/111416957 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3ab0008ee0 con 0x7f3ab40ff4b0 2024-07-16T10:01:03.481 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:03.478+0000 7f3aaa7fc640 1 -- 172.21.15.156:0/111416957 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3ab0033890 con 0x7f3ab40ff4b0 2024-07-16T10:01:03.481 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:03.478+0000 7f3abbc0b640 1 -- 172.21.15.156:0/111416957 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ab410eb30 con 0x7f3ab40ff4b0 2024-07-16T10:01:03.482 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:03.479+0000 7f3abbc0b640 1 -- 172.21.15.156:0/111416957 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ab40fa6c0 con 0x7f3ab40ff4b0 2024-07-16T10:01:03.482 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:03.479+0000 7f3aaa7fc640 1 -- 172.21.15.156:0/111416957 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 38) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f3ab0030020 con 0x7f3ab40ff4b0 2024-07-16T10:01:03.482 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:03.480+0000 7f3aaa7fc640 1 --2- 172.21.15.156:0/111416957 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f3aa006e0b0 0x7f3aa0070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:01:03.483 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:03.480+0000 7f3aaa7fc640 1 -- 172.21.15.156:0/111416957 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f3ab00ad9a0 con 0x7f3ab40ff4b0 2024-07-16T10:01:03.483 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:03.480+0000 7f3ab99a7640 1 --2- 172.21.15.156:0/111416957 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f3aa006e0b0 0x7f3aa0070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:01:03.483 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:03.480+0000 7f3ab99a7640 1 --2- 172.21.15.156:0/111416957 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f3aa006e0b0 0x7f3aa0070560 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f3aa4009fd0 tx=0x7f3aa4009c50 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T10:01:03.485 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:03.484+0000 7f3aaa7fc640 1 -- 172.21.15.156:0/111416957 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187716 (secure 0 0 0) 0x7f3ab0076d20 con 0x7f3ab40ff4b0 2024-07-16T10:01:03.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:01:03 smithi156 ceph-mon[93066]: pgmap v120: 65 pgs: 65 active+clean; 253 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.6 KiB/s rd, 2 op/s 2024-07-16T10:01:03.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:01:03 smithi156 ceph-mon[93066]: from='client.34314 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-16T10:01:03.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:01:03 smithi156 ceph-mon[93066]: from='client.34318 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-16T10:01:03.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:01:03 smithi156 ceph-mon[93066]: from='client.34322 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-07-16T10:01:03.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:01:03 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:01:03.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:01:03 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-16T10:01:03.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:01:03 smithi156 ceph-mon[93066]: from='client.? 172.21.15.156:0/2376034688' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:01:03.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:01:03 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:01:03.704 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:03.703+0000 7f3abbc0b640 1 -- 172.21.15.156:0/111416957 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f3ab410eeb0 con 0x7f3ab40ff4b0 2024-07-16T10:01:03.704 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:03.703+0000 7f3aaa7fc640 1 -- 172.21.15.156:0/111416957 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 20 v20) v1 ==== 76+0+1920 (secure 0 0 0) 0x7f3ab007a3e0 con 0x7f3ab40ff4b0 2024-07-16T10:01:03.705 INFO:teuthology.orchestra.run.smithi156.stdout:e20 2024-07-16T10:01:03.705 INFO:teuthology.orchestra.run.smithi156.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-07-16T10:01:03.705 INFO:teuthology.orchestra.run.smithi156.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-07-16T10:01:03.705 INFO:teuthology.orchestra.run.smithi156.stdout:legacy client fscid: 1 2024-07-16T10:01:03.705 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-07-16T10:01:03.705 INFO:teuthology.orchestra.run.smithi156.stdout:Filesystem 'cephfs' (1) 2024-07-16T10:01:03.705 INFO:teuthology.orchestra.run.smithi156.stdout:fs_name cephfs 2024-07-16T10:01:03.705 INFO:teuthology.orchestra.run.smithi156.stdout:epoch 20 2024-07-16T10:01:03.705 INFO:teuthology.orchestra.run.smithi156.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-07-16T10:01:03.705 INFO:teuthology.orchestra.run.smithi156.stdout:created 2024-07-16T09:52:55.214975+0000 2024-07-16T10:01:03.705 INFO:teuthology.orchestra.run.smithi156.stdout:modified 2024-07-16T09:58:56.574272+0000 2024-07-16T10:01:03.706 INFO:teuthology.orchestra.run.smithi156.stdout:tableserver 0 2024-07-16T10:01:03.706 INFO:teuthology.orchestra.run.smithi156.stdout:root 0 2024-07-16T10:01:03.706 INFO:teuthology.orchestra.run.smithi156.stdout:session_timeout 60 2024-07-16T10:01:03.706 INFO:teuthology.orchestra.run.smithi156.stdout:session_autoclose 300 2024-07-16T10:01:03.706 INFO:teuthology.orchestra.run.smithi156.stdout:max_file_size 1099511627776 2024-07-16T10:01:03.706 INFO:teuthology.orchestra.run.smithi156.stdout:max_xattr_size 65536 2024-07-16T10:01:03.706 INFO:teuthology.orchestra.run.smithi156.stdout:required_client_features {} 2024-07-16T10:01:03.706 INFO:teuthology.orchestra.run.smithi156.stdout:last_failure 0 2024-07-16T10:01:03.706 INFO:teuthology.orchestra.run.smithi156.stdout:last_failure_osd_epoch 0 2024-07-16T10:01:03.706 INFO:teuthology.orchestra.run.smithi156.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-07-16T10:01:03.706 INFO:teuthology.orchestra.run.smithi156.stdout:max_mds 2 2024-07-16T10:01:03.707 INFO:teuthology.orchestra.run.smithi156.stdout:in 0,1 2024-07-16T10:01:03.707 INFO:teuthology.orchestra.run.smithi156.stdout:up {0=14458,1=24261} 2024-07-16T10:01:03.707 INFO:teuthology.orchestra.run.smithi156.stdout:failed 2024-07-16T10:01:03.707 INFO:teuthology.orchestra.run.smithi156.stdout:damaged 2024-07-16T10:01:03.707 INFO:teuthology.orchestra.run.smithi156.stdout:stopped 2024-07-16T10:01:03.707 INFO:teuthology.orchestra.run.smithi156.stdout:data_pools [3] 2024-07-16T10:01:03.707 INFO:teuthology.orchestra.run.smithi156.stdout:metadata_pool 2 2024-07-16T10:01:03.707 INFO:teuthology.orchestra.run.smithi156.stdout:inline_data disabled 2024-07-16T10:01:03.707 INFO:teuthology.orchestra.run.smithi156.stdout:balancer 2024-07-16T10:01:03.707 INFO:teuthology.orchestra.run.smithi156.stdout:bal_rank_mask -1 2024-07-16T10:01:03.707 INFO:teuthology.orchestra.run.smithi156.stdout:standby_count_wanted 1 2024-07-16T10:01:03.707 INFO:teuthology.orchestra.run.smithi156.stdout:qdb_cluster leader: 14458 members: 24261,14458 2024-07-16T10:01:03.707 INFO:teuthology.orchestra.run.smithi156.stdout:[mds.cephfs.smithi156.rdibys{0:14458} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.156:6826/1080176013,v1:172.21.15.156:6827/1080176013] compat {c=[1],r=[1],i=[7ff]}] 2024-07-16T10:01:03.708 INFO:teuthology.orchestra.run.smithi156.stdout:[mds.cephfs.smithi156.evwwaz{0:14468} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.156:6828/560329502,v1:172.21.15.156:6829/560329502] compat {c=[1],r=[1],i=[7ff]}] 2024-07-16T10:01:03.708 INFO:teuthology.orchestra.run.smithi156.stdout:[mds.cephfs.smithi176.dvvbud{1:24261} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.176:6824/1577019287,v1:172.21.15.176:6825/1577019287] compat {c=[1],r=[1],i=[7ff]}] 2024-07-16T10:01:03.708 INFO:teuthology.orchestra.run.smithi156.stdout:[mds.cephfs.smithi176.wahbsi{1:24273} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.176:6826/2196775412,v1:172.21.15.176:6827/2196775412] compat {c=[1],r=[1],i=[7ff]}] 2024-07-16T10:01:03.708 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-07-16T10:01:03.708 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-07-16T10:01:03.709 INFO:teuthology.orchestra.run.smithi156.stderr:dumped fsmap epoch 20 2024-07-16T10:01:03.709 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:03.705+0000 7f3abbc0b640 1 -- 172.21.15.156:0/111416957 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f3aa006e0b0 msgr2=0x7f3aa0070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:01:03.709 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:03.705+0000 7f3abbc0b640 1 --2- 172.21.15.156:0/111416957 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f3aa006e0b0 0x7f3aa0070560 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f3aa4009fd0 tx=0x7f3aa4009c50 comp rx=0 tx=0).stop 2024-07-16T10:01:03.709 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:03.705+0000 7f3abbc0b640 1 -- 172.21.15.156:0/111416957 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3ab40ff4b0 msgr2=0x7f3ab410d550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:01:03.709 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:03.705+0000 7f3abbc0b640 1 --2- 172.21.15.156:0/111416957 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3ab40ff4b0 0x7f3ab410d550 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f3ab0009f80 tx=0x7f3ab0009da0 comp rx=0 tx=0).stop 2024-07-16T10:01:03.709 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:03.705+0000 7f3abbc0b640 1 -- 172.21.15.156:0/111416957 shutdown_connections 2024-07-16T10:01:03.709 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:03.705+0000 7f3abbc0b640 1 --2- 172.21.15.156:0/111416957 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f3aa006e0b0 0x7f3aa0070560 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:01:03.709 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:03.705+0000 7f3abbc0b640 1 --2- 172.21.15.156:0/111416957 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3ab40ff4b0 0x7f3ab410d550 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:01:03.709 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:03.705+0000 7f3abbc0b640 1 --2- 172.21.15.156:0/111416957 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3ab4066520 0x7f3ab410d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:01:03.709 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:03.705+0000 7f3abbc0b640 1 -- 172.21.15.156:0/111416957 >> 172.21.15.156:0/111416957 conn(0x7f3ab40f6a20 msgr2=0x7f3ab40f6e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T10:01:03.709 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:03.705+0000 7f3abbc0b640 1 -- 172.21.15.156:0/111416957 shutdown_connections 2024-07-16T10:01:03.710 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:03.705+0000 7f3abbc0b640 1 -- 172.21.15.156:0/111416957 wait complete. 2024-07-16T10:01:03.788 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:03.786+0000 7f8c2285d640 1 -- 172.21.15.156:0/2857390447 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8c1c0fc360 msgr2=0x7f8c1c0fc770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:01:03.788 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:03.786+0000 7f8c2285d640 1 --2- 172.21.15.156:0/2857390447 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8c1c0fc360 0x7f8c1c0fc770 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f8c0c0048d0 tx=0x7f8c0c009da0 comp rx=0 tx=0).stop 2024-07-16T10:01:03.788 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:03.786+0000 7f8c2285d640 1 -- 172.21.15.156:0/2857390447 shutdown_connections 2024-07-16T10:01:03.788 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:03.786+0000 7f8c2285d640 1 --2- 172.21.15.156:0/2857390447 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8c1c0fd630 0x7f8c1c0fdaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:01:03.788 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:03.786+0000 7f8c2285d640 1 --2- 172.21.15.156:0/2857390447 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8c1c0fc360 0x7f8c1c0fc770 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:01:03.788 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:03.786+0000 7f8c2285d640 1 -- 172.21.15.156:0/2857390447 >> 172.21.15.156:0/2857390447 conn(0x7f8c1c0f7b40 msgr2=0x7f8c1c0f9f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T10:01:03.788 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:03.786+0000 7f8c2285d640 1 -- 172.21.15.156:0/2857390447 shutdown_connections 2024-07-16T10:01:03.788 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:03.786+0000 7f8c2285d640 1 -- 172.21.15.156:0/2857390447 wait complete. 2024-07-16T10:01:03.788 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:03.786+0000 7f8c2285d640 1 Processor -- start 2024-07-16T10:01:03.789 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:03.787+0000 7f8c2285d640 1 -- start start 2024-07-16T10:01:03.790 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:03.787+0000 7f8c2285d640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8c1c0fc360 0x7f8c1c10cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:01:03.790 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:03.787+0000 7f8c2285d640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8c1c0fd630 0x7f8c1c10d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:01:03.790 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:03.787+0000 7f8c2285d640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c1c10e360 con 0x7f8c1c0fd630 2024-07-16T10:01:03.790 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:03.787+0000 7f8c2285d640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c1c10e4d0 con 0x7f8c1c0fc360 2024-07-16T10:01:03.790 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:03.787+0000 7f8c1b7fe640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8c1c0fd630 0x7f8c1c10d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:01:03.790 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:03.787+0000 7f8c1b7fe640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8c1c0fd630 0x7f8c1c10d4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:52000/0 (socket says 172.21.15.156:52000) 2024-07-16T10:01:03.790 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:03.787+0000 7f8c1b7fe640 1 -- 172.21.15.156:0/3932545875 learned_addr learned my addr 172.21.15.156:0/3932545875 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T10:01:03.791 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:03.787+0000 7f8c1bfff640 1 --2- 172.21.15.156:0/3932545875 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8c1c0fc360 0x7f8c1c10cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:01:03.791 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:03.787+0000 7f8c1b7fe640 1 -- 172.21.15.156:0/3932545875 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8c1c0fc360 msgr2=0x7f8c1c10cfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:01:03.791 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:03.787+0000 7f8c1b7fe640 1 --2- 172.21.15.156:0/3932545875 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8c1c0fc360 0x7f8c1c10cfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:01:03.791 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:03.787+0000 7f8c1b7fe640 1 -- 172.21.15.156:0/3932545875 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c0c005040 con 0x7f8c1c0fd630 2024-07-16T10:01:03.791 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:03.788+0000 7f8c1b7fe640 1 --2- 172.21.15.156:0/3932545875 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8c1c0fd630 0x7f8c1c10d4f0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f8c1000b790 tx=0x7f8c100091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T10:01:03.791 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:03.788+0000 7f8c18ff9640 1 -- 172.21.15.156:0/3932545875 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8c1000c700 con 0x7f8c1c0fd630 2024-07-16T10:01:03.791 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:03.788+0000 7f8c2285d640 1 -- 172.21.15.156:0/3932545875 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c1c10e7b0 con 0x7f8c1c0fd630 2024-07-16T10:01:03.791 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:03.788+0000 7f8c18ff9640 1 -- 172.21.15.156:0/3932545875 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8c1000cd30 con 0x7f8c1c0fd630 2024-07-16T10:01:03.791 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:03.788+0000 7f8c18ff9640 1 -- 172.21.15.156:0/3932545875 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8c10013370 con 0x7f8c1c0fd630 2024-07-16T10:01:03.791 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:03.788+0000 7f8c2285d640 1 -- 172.21.15.156:0/3932545875 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c1c10ecd0 con 0x7f8c1c0fd630 2024-07-16T10:01:03.791 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:03.789+0000 7f8c2285d640 1 -- 172.21.15.156:0/3932545875 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c1c0fb7c0 con 0x7f8c1c0fd630 2024-07-16T10:01:03.792 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:03.789+0000 7f8c18ff9640 1 -- 172.21.15.156:0/3932545875 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 38) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f8c10012070 con 0x7f8c1c0fd630 2024-07-16T10:01:03.792 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:03.790+0000 7f8c18ff9640 1 --2- 172.21.15.156:0/3932545875 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f8c0406e0b0 0x7f8c04070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:01:03.792 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:03.790+0000 7f8c18ff9640 1 -- 172.21.15.156:0/3932545875 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f8c1008d480 con 0x7f8c1c0fd630 2024-07-16T10:01:03.792 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:03.790+0000 7f8c1bfff640 1 --2- 172.21.15.156:0/3932545875 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f8c0406e0b0 0x7f8c04070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:01:03.792 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:03.790+0000 7f8c1bfff640 1 --2- 172.21.15.156:0/3932545875 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f8c0406e0b0 0x7f8c04070560 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f8c0c000c00 tx=0x7f8c0c035040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T10:01:03.795 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:03.794+0000 7f8c18ff9640 1 -- 172.21.15.156:0/3932545875 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187716 (secure 0 0 0) 0x7f8c10056800 con 0x7f8c1c0fd630 2024-07-16T10:01:03.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:01:03 smithi176 ceph-mon[81689]: pgmap v120: 65 pgs: 65 active+clean; 253 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.6 KiB/s rd, 2 op/s 2024-07-16T10:01:03.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:01:03 smithi176 ceph-mon[81689]: from='client.34314 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-16T10:01:03.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:01:03 smithi176 ceph-mon[81689]: from='client.34318 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-16T10:01:03.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:01:03 smithi176 ceph-mon[81689]: from='client.34322 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-07-16T10:01:03.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:01:03 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:01:03.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:01:03 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-16T10:01:03.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:01:03 smithi176 ceph-mon[81689]: from='client.? 172.21.15.156:0/2376034688' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:01:03.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:01:03 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:01:03.989 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:03.987+0000 7f8c2285d640 1 -- 172.21.15.156:0/3932545875 --> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f8c1c10ef50 con 0x7f8c0406e0b0 2024-07-16T10:01:03.990 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:03.989+0000 7f8c18ff9640 1 -- 172.21.15.156:0/3932545875 <== mgr.34104 v2:172.21.15.156:6800/3367765292 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7f8c1c10ef50 con 0x7f8c0406e0b0 2024-07-16T10:01:03.990 INFO:teuthology.orchestra.run.smithi156.stdout:{ 2024-07-16T10:01:03.990 INFO:teuthology.orchestra.run.smithi156.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:c15ec3aa8e6c0a9c12ee24ff79f3e7dd4d01b3d1a702dfeb7d7fff9b0811a393", 2024-07-16T10:01:03.990 INFO:teuthology.orchestra.run.smithi156.stdout: "in_progress": true, 2024-07-16T10:01:03.990 INFO:teuthology.orchestra.run.smithi156.stdout: "which": "Upgrading all daemon types on all hosts", 2024-07-16T10:01:03.990 INFO:teuthology.orchestra.run.smithi156.stdout: "services_complete": [ 2024-07-16T10:01:03.990 INFO:teuthology.orchestra.run.smithi156.stdout: "mon", 2024-07-16T10:01:03.991 INFO:teuthology.orchestra.run.smithi156.stdout: "mgr", 2024-07-16T10:01:03.991 INFO:teuthology.orchestra.run.smithi156.stdout: "crash" 2024-07-16T10:01:03.991 INFO:teuthology.orchestra.run.smithi156.stdout: ], 2024-07-16T10:01:03.991 INFO:teuthology.orchestra.run.smithi156.stdout: "progress": "10/21 daemons upgraded", 2024-07-16T10:01:03.991 INFO:teuthology.orchestra.run.smithi156.stdout: "message": "Currently upgrading osd daemons", 2024-07-16T10:01:03.991 INFO:teuthology.orchestra.run.smithi156.stdout: "is_paused": false 2024-07-16T10:01:03.991 INFO:teuthology.orchestra.run.smithi156.stdout:} 2024-07-16T10:01:03.992 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:03.991+0000 7f8c2285d640 1 -- 172.21.15.156:0/3932545875 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f8c0406e0b0 msgr2=0x7f8c04070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:01:03.993 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:03.991+0000 7f8c2285d640 1 --2- 172.21.15.156:0/3932545875 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f8c0406e0b0 0x7f8c04070560 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f8c0c000c00 tx=0x7f8c0c035040 comp rx=0 tx=0).stop 2024-07-16T10:01:03.993 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:03.991+0000 7f8c2285d640 1 -- 172.21.15.156:0/3932545875 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8c1c0fd630 msgr2=0x7f8c1c10d4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:01:03.993 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:03.991+0000 7f8c2285d640 1 --2- 172.21.15.156:0/3932545875 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8c1c0fd630 0x7f8c1c10d4f0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f8c1000b790 tx=0x7f8c100091d0 comp rx=0 tx=0).stop 2024-07-16T10:01:03.993 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:03.991+0000 7f8c2285d640 1 -- 172.21.15.156:0/3932545875 shutdown_connections 2024-07-16T10:01:03.993 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:03.991+0000 7f8c2285d640 1 --2- 172.21.15.156:0/3932545875 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f8c0406e0b0 0x7f8c04070560 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:01:03.993 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:03.991+0000 7f8c2285d640 1 --2- 172.21.15.156:0/3932545875 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8c1c0fd630 0x7f8c1c10d4f0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:01:03.993 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:03.991+0000 7f8c2285d640 1 --2- 172.21.15.156:0/3932545875 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8c1c0fc360 0x7f8c1c10cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:01:03.993 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:03.991+0000 7f8c2285d640 1 -- 172.21.15.156:0/3932545875 >> 172.21.15.156:0/3932545875 conn(0x7f8c1c0f7b40 msgr2=0x7f8c1c0f91b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T10:01:03.993 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:03.991+0000 7f8c2285d640 1 -- 172.21.15.156:0/3932545875 shutdown_connections 2024-07-16T10:01:03.993 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:03.991+0000 7f8c2285d640 1 -- 172.21.15.156:0/3932545875 wait complete. 2024-07-16T10:01:04.074 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:04.071+0000 7fe89ff65640 1 -- 172.21.15.156:0/20886538 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe8980fc540 msgr2=0x7fe8980fc9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:01:04.074 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:04.071+0000 7fe89ff65640 1 --2- 172.21.15.156:0/20886538 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe8980fc540 0x7fe8980fc9b0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fe8940048d0 tx=0x7fe894009da0 comp rx=0 tx=0).stop 2024-07-16T10:01:04.074 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:04.071+0000 7fe89ff65640 1 -- 172.21.15.156:0/20886538 shutdown_connections 2024-07-16T10:01:04.074 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:04.071+0000 7fe89ff65640 1 --2- 172.21.15.156:0/20886538 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe8980fc540 0x7fe8980fc9b0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:01:04.074 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:04.071+0000 7fe89ff65640 1 --2- 172.21.15.156:0/20886538 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe8980fb270 0x7fe8980fb680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:01:04.074 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:04.071+0000 7fe89ff65640 1 -- 172.21.15.156:0/20886538 >> 172.21.15.156:0/20886538 conn(0x7fe8980f6a50 msgr2=0x7fe8980f8e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T10:01:04.075 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:04.071+0000 7fe89ff65640 1 -- 172.21.15.156:0/20886538 shutdown_connections 2024-07-16T10:01:04.075 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:04.071+0000 7fe89ff65640 1 -- 172.21.15.156:0/20886538 wait complete. 2024-07-16T10:01:04.075 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:04.072+0000 7fe89ff65640 1 Processor -- start 2024-07-16T10:01:04.075 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:04.072+0000 7fe89ff65640 1 -- start start 2024-07-16T10:01:04.075 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:04.072+0000 7fe89ff65640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe8980fb270 0x7fe89810cf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:01:04.075 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:04.072+0000 7fe89ff65640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe8980fc540 0x7fe89810d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:01:04.075 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:04.072+0000 7fe89ff65640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe89810db00 con 0x7fe8980fb270 2024-07-16T10:01:04.075 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:04.072+0000 7fe89ff65640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe89810e320 con 0x7fe8980fc540 2024-07-16T10:01:04.075 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:04.072+0000 7fe89dd01640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe8980fb270 0x7fe89810cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:01:04.075 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:04.072+0000 7fe89dd01640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe8980fb270 0x7fe89810cf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:52014/0 (socket says 172.21.15.156:52014) 2024-07-16T10:01:04.075 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:04.072+0000 7fe89dd01640 1 -- 172.21.15.156:0/1640287020 learned_addr learned my addr 172.21.15.156:0/1640287020 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T10:01:04.075 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:04.072+0000 7fe89d500640 1 --2- 172.21.15.156:0/1640287020 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe8980fc540 0x7fe89810d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:01:04.075 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:04.072+0000 7fe89dd01640 1 -- 172.21.15.156:0/1640287020 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe8980fc540 msgr2=0x7fe89810d4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:01:04.076 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:04.072+0000 7fe89dd01640 1 --2- 172.21.15.156:0/1640287020 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe8980fc540 0x7fe89810d4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:01:04.076 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:04.072+0000 7fe89dd01640 1 -- 172.21.15.156:0/1640287020 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe894005040 con 0x7fe8980fb270 2024-07-16T10:01:04.076 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:04.072+0000 7fe89d500640 1 --2- 172.21.15.156:0/1640287020 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe8980fc540 0x7fe89810d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-16T10:01:04.076 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:04.072+0000 7fe89dd01640 1 --2- 172.21.15.156:0/1640287020 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe8980fb270 0x7fe89810cf70 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fe888002a50 tx=0x7fe888004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T10:01:04.076 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:04.073+0000 7fe88e7fc640 1 -- 172.21.15.156:0/1640287020 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe888007c30 con 0x7fe8980fb270 2024-07-16T10:01:04.076 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:04.073+0000 7fe88e7fc640 1 -- 172.21.15.156:0/1640287020 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe88800f040 con 0x7fe8980fb270 2024-07-16T10:01:04.076 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:04.073+0000 7fe89ff65640 1 -- 172.21.15.156:0/1640287020 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe89810e5b0 con 0x7fe8980fb270 2024-07-16T10:01:04.076 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:04.073+0000 7fe88e7fc640 1 -- 172.21.15.156:0/1640287020 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe88800db50 con 0x7fe8980fb270 2024-07-16T10:01:04.076 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:04.073+0000 7fe89ff65640 1 -- 172.21.15.156:0/1640287020 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe89810eb50 con 0x7fe8980fb270 2024-07-16T10:01:04.077 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:04.074+0000 7fe89ff65640 1 -- 172.21.15.156:0/1640287020 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8980fa6d0 con 0x7fe8980fb270 2024-07-16T10:01:04.077 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:04.074+0000 7fe88e7fc640 1 -- 172.21.15.156:0/1640287020 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 38) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fe88801c400 con 0x7fe8980fb270 2024-07-16T10:01:04.077 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:04.075+0000 7fe88e7fc640 1 --2- 172.21.15.156:0/1640287020 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7fe88406e0b0 0x7fe884070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:01:04.077 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:04.075+0000 7fe88e7fc640 1 -- 172.21.15.156:0/1640287020 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fe88808d420 con 0x7fe8980fb270 2024-07-16T10:01:04.078 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:04.075+0000 7fe89d500640 1 --2- 172.21.15.156:0/1640287020 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7fe88406e0b0 0x7fe884070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:01:04.078 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:04.075+0000 7fe89d500640 1 --2- 172.21.15.156:0/1640287020 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7fe88406e0b0 0x7fe884070560 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fe894000c00 tx=0x7fe894035040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T10:01:04.080 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:04.079+0000 7fe88e7fc640 1 -- 172.21.15.156:0/1640287020 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187716 (secure 0 0 0) 0x7fe8880567a0 con 0x7fe8980fb270 2024-07-16T10:01:04.322 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:04.320+0000 7fe89ff65640 1 -- 172.21.15.156:0/1640287020 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fe89804b530 con 0x7fe8980fb270 2024-07-16T10:01:04.322 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:04.321+0000 7fe88e7fc640 1 -- 172.21.15.156:0/1640287020 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7fe888059e60 con 0x7fe8980fb270 2024-07-16T10:01:04.322 INFO:teuthology.orchestra.run.smithi156.stdout:HEALTH_OK 2024-07-16T10:01:04.324 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:04.323+0000 7fe89ff65640 1 -- 172.21.15.156:0/1640287020 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7fe88406e0b0 msgr2=0x7fe884070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:01:04.324 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:04.323+0000 7fe89ff65640 1 --2- 172.21.15.156:0/1640287020 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7fe88406e0b0 0x7fe884070560 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fe894000c00 tx=0x7fe894035040 comp rx=0 tx=0).stop 2024-07-16T10:01:04.325 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:04.323+0000 7fe89ff65640 1 -- 172.21.15.156:0/1640287020 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe8980fb270 msgr2=0x7fe89810cf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:01:04.325 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:04.323+0000 7fe89ff65640 1 --2- 172.21.15.156:0/1640287020 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe8980fb270 0x7fe89810cf70 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fe888002a50 tx=0x7fe888004600 comp rx=0 tx=0).stop 2024-07-16T10:01:04.325 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:04.323+0000 7fe89ff65640 1 -- 172.21.15.156:0/1640287020 shutdown_connections 2024-07-16T10:01:04.325 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:04.323+0000 7fe89ff65640 1 --2- 172.21.15.156:0/1640287020 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7fe88406e0b0 0x7fe884070560 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:01:04.325 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:04.323+0000 7fe89ff65640 1 --2- 172.21.15.156:0/1640287020 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe8980fc540 0x7fe89810d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:01:04.325 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:04.323+0000 7fe89ff65640 1 --2- 172.21.15.156:0/1640287020 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe8980fb270 0x7fe89810cf70 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:01:04.325 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:04.323+0000 7fe89ff65640 1 -- 172.21.15.156:0/1640287020 >> 172.21.15.156:0/1640287020 conn(0x7fe8980f6a50 msgr2=0x7fe8980f80c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T10:01:04.325 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:04.323+0000 7fe89ff65640 1 -- 172.21.15.156:0/1640287020 shutdown_connections 2024-07-16T10:01:04.325 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:04.323+0000 7fe89ff65640 1 -- 172.21.15.156:0/1640287020 wait complete. 2024-07-16T10:01:04.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:01:04 smithi156 ceph-mon[93066]: from='client.? 172.21.15.156:0/111416957' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-07-16T10:01:04.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:01:04 smithi156 ceph-mon[93066]: from='client.34334 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-16T10:01:04.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:01:04 smithi156 ceph-mon[93066]: from='client.? 172.21.15.156:0/1640287020' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-07-16T10:01:04.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:01:04 smithi176 ceph-mon[81689]: from='client.? 172.21.15.156:0/111416957' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-07-16T10:01:04.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:01:04 smithi176 ceph-mon[81689]: from='client.34334 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-16T10:01:04.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:01:04 smithi176 ceph-mon[81689]: from='client.? 172.21.15.156:0/1640287020' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-07-16T10:01:05.843 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:01:05 smithi176 ceph-mon[81689]: pgmap v121: 65 pgs: 65 active+clean; 253 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.8 KiB/s rd, 2 op/s 2024-07-16T10:01:05.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:01:05 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-07-16T10:01:05.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:01:05 smithi176 ceph-mon[81689]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-07-16T10:01:05.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:01:05 smithi176 ceph-mon[81689]: Upgrade: osd.4 is safe to restart 2024-07-16T10:01:05.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:01:05 smithi156 ceph-mon[93066]: pgmap v121: 65 pgs: 65 active+clean; 253 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.8 KiB/s rd, 2 op/s 2024-07-16T10:01:05.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:01:05 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-07-16T10:01:05.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:01:05 smithi156 ceph-mon[93066]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-07-16T10:01:05.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:01:05 smithi156 ceph-mon[93066]: Upgrade: osd.4 is safe to restart 2024-07-16T10:01:07.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:01:07 smithi176 ceph-mon[81689]: Upgrade: Updating osd.4 2024-07-16T10:01:07.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:01:07 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:01:07.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:01:07 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-07-16T10:01:07.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:01:07 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-16T10:01:07.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:01:07 smithi176 ceph-mon[81689]: Deploying daemon osd.4 on smithi176 2024-07-16T10:01:07.424 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:01:07 smithi156 ceph-mon[93066]: Upgrade: Updating osd.4 2024-07-16T10:01:07.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:01:07 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:01:07.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:01:07 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-07-16T10:01:07.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:01:07 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-16T10:01:07.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:01:07 smithi156 ceph-mon[93066]: Deploying daemon osd.4 on smithi176 2024-07-16T10:01:08.343 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:01:08 smithi176 ceph-mon[81689]: pgmap v122: 65 pgs: 65 active+clean; 253 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.9 KiB/s rd, 3 op/s 2024-07-16T10:01:08.424 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:01:08 smithi156 ceph-mon[93066]: pgmap v122: 65 pgs: 65 active+clean; 253 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.9 KiB/s rd, 3 op/s 2024-07-16T10:01:09.094 INFO:journalctl@ceph.osd.4.smithi176.stdout:Jul 16 10:01:08 smithi176 systemd[1]: Stopping Ceph osd.4 for 26c91c26-4358-11ef-bcad-c7b262605968... 2024-07-16T10:01:09.594 INFO:journalctl@ceph.osd.4.smithi176.stdout:Jul 16 10:01:09 smithi176 ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-4[48461]: 2024-07-16T10:01:09.171+0000 7fb82e5b0640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-07-16T10:01:09.594 INFO:journalctl@ceph.osd.4.smithi176.stdout:Jul 16 10:01:09 smithi176 ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-4[48461]: 2024-07-16T10:01:09.171+0000 7fb82e5b0640 -1 osd.4 69 *** Got signal Terminated *** 2024-07-16T10:01:09.594 INFO:journalctl@ceph.osd.4.smithi176.stdout:Jul 16 10:01:09 smithi176 ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-4[48461]: 2024-07-16T10:01:09.171+0000 7fb82e5b0640 -1 osd.4 69 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-07-16T10:01:10.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:01:10 smithi176 ceph-mon[81689]: pgmap v123: 65 pgs: 65 active+clean; 253 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-07-16T10:01:10.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:01:10 smithi176 ceph-mon[81689]: osd.4 marked itself down and dead 2024-07-16T10:01:10.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:01:10 smithi156 ceph-mon[93066]: pgmap v123: 65 pgs: 65 active+clean; 253 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-07-16T10:01:10.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:01:10 smithi156 ceph-mon[93066]: osd.4 marked itself down and dead 2024-07-16T10:01:10.722 INFO:journalctl@ceph.osd.4.smithi176.stdout:Jul 16 10:01:10 smithi176 podman[95450]: 2024-07-16 10:01:10.449859262 +0000 UTC m=+1.372126615 container died ef0c67cec086b4d96fca178291c98c4d725e0c0a895b707d84245f11b0af0f1b (image=quay.ceph.io/ceph-ci/ceph@sha256:7b86ef6bffe0efc5f1aefc9fef8c854a67a511c50993acee3ac684f23bde83df, name=ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-4, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.36.0, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240709, RELEASE=quincy-e6231b0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-07-16T10:01:11.064 INFO:journalctl@ceph.osd.4.smithi176.stdout:Jul 16 10:01:10 smithi176 podman[95450]: 2024-07-16 10:01:10.81271044 +0000 UTC m=+1.734977793 container cleanup ef0c67cec086b4d96fca178291c98c4d725e0c0a895b707d84245f11b0af0f1b (image=quay.ceph.io/ceph-ci/ceph@sha256:7b86ef6bffe0efc5f1aefc9fef8c854a67a511c50993acee3ac684f23bde83df, name=ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-4, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=quincy-e6231b0, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.36.0, org.label-schema.build-date=20240709, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0) 2024-07-16T10:01:11.064 INFO:journalctl@ceph.osd.4.smithi176.stdout:Jul 16 10:01:10 smithi176 bash[95450]: ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-4 2024-07-16T10:01:11.343 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:01:11 smithi176 ceph-mon[81689]: Health check failed: 1 osds down (OSD_DOWN) 2024-07-16T10:01:11.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:01:11 smithi176 ceph-mon[81689]: osdmap e70: 6 total, 5 up, 6 in 2024-07-16T10:01:11.344 INFO:journalctl@ceph.osd.4.smithi176.stdout:Jul 16 10:01:11 smithi176 podman[95465]: 2024-07-16 10:01:11.103724774 +0000 UTC m=+0.647701711 container remove ef0c67cec086b4d96fca178291c98c4d725e0c0a895b707d84245f11b0af0f1b (image=quay.ceph.io/ceph-ci/ceph@sha256:7b86ef6bffe0efc5f1aefc9fef8c854a67a511c50993acee3ac684f23bde83df, name=ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-4, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240709, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, RELEASE=quincy-e6231b0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.36.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0) 2024-07-16T10:01:11.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:01:11 smithi156 ceph-mon[93066]: Health check failed: 1 osds down (OSD_DOWN) 2024-07-16T10:01:11.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:01:11 smithi156 ceph-mon[93066]: osdmap e70: 6 total, 5 up, 6 in 2024-07-16T10:01:11.697 INFO:journalctl@ceph.osd.4.smithi176.stdout:Jul 16 10:01:11 smithi176 podman[95554]: 2024-07-16 10:01:11.347326944 +0000 UTC m=+0.017227668 image pull 16f7feb043422ce37a2471403d4bc52f550d49cef4b677482112fcc0db3f67ab quay.ceph.io/ceph-ci/ceph@sha256:c15ec3aa8e6c0a9c12ee24ff79f3e7dd4d01b3d1a702dfeb7d7fff9b0811a393 2024-07-16T10:01:11.697 INFO:journalctl@ceph.osd.4.smithi176.stdout:Jul 16 10:01:11 smithi176 podman[95554]: 2024-07-16 10:01:11.448439065 +0000 UTC m=+0.118339770 container create 8bf73cc33d709f923d3de4c5ef09c0d37fe88fe01e1e28027a63ae5fe7dd8782 (image=quay.ceph.io/ceph-ci/ceph@sha256:c15ec3aa8e6c0a9c12ee24ff79f3e7dd4d01b3d1a702dfeb7d7fff9b0811a393, name=ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-4-deactivate, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240709, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, io.buildah.version=1.36.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-adk4-testing-2024-07-15-1832-squid-a6ca461, maintainer=Guillaume Abrioux ) 2024-07-16T10:01:11.697 INFO:journalctl@ceph.osd.4.smithi176.stdout:Jul 16 10:01:11 smithi176 podman[95554]: 2024-07-16 10:01:11.692538155 +0000 UTC m=+0.362438866 container init 8bf73cc33d709f923d3de4c5ef09c0d37fe88fe01e1e28027a63ae5fe7dd8782 (image=quay.ceph.io/ceph-ci/ceph@sha256:c15ec3aa8e6c0a9c12ee24ff79f3e7dd4d01b3d1a702dfeb7d7fff9b0811a393, name=ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-4-deactivate, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.36.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240709, org.label-schema.schema-version=1.0, RELEASE=wip-adk4-testing-2024-07-15-1832-squid-a6ca461, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-07-16T10:01:11.697 INFO:journalctl@ceph.osd.4.smithi176.stdout:Jul 16 10:01:11 smithi176 podman[95554]: 2024-07-16 10:01:11.696139431 +0000 UTC m=+0.366040152 container start 8bf73cc33d709f923d3de4c5ef09c0d37fe88fe01e1e28027a63ae5fe7dd8782 (image=quay.ceph.io/ceph-ci/ceph@sha256:c15ec3aa8e6c0a9c12ee24ff79f3e7dd4d01b3d1a702dfeb7d7fff9b0811a393, name=ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-4-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, RELEASE=wip-adk4-testing-2024-07-15-1832-squid-a6ca461, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240709) 2024-07-16T10:01:12.073 INFO:journalctl@ceph.osd.4.smithi176.stdout:Jul 16 10:01:11 smithi176 podman[95554]: 2024-07-16 10:01:11.767425323 +0000 UTC m=+0.437326060 container attach 8bf73cc33d709f923d3de4c5ef09c0d37fe88fe01e1e28027a63ae5fe7dd8782 (image=quay.ceph.io/ceph-ci/ceph@sha256:c15ec3aa8e6c0a9c12ee24ff79f3e7dd4d01b3d1a702dfeb7d7fff9b0811a393, name=ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-4-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240709, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-adk4-testing-2024-07-15-1832-squid-a6ca461, io.buildah.version=1.36.0, ceph=True) 2024-07-16T10:01:12.073 INFO:journalctl@ceph.osd.4.smithi176.stdout:Jul 16 10:01:11 smithi176 podman[95554]: 2024-07-16 10:01:11.893329175 +0000 UTC m=+0.563229891 container died 8bf73cc33d709f923d3de4c5ef09c0d37fe88fe01e1e28027a63ae5fe7dd8782 (image=quay.ceph.io/ceph-ci/ceph@sha256:c15ec3aa8e6c0a9c12ee24ff79f3e7dd4d01b3d1a702dfeb7d7fff9b0811a393, name=ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-4-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-adk4-testing-2024-07-15-1832-squid-a6ca461, io.buildah.version=1.36.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240709, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-07-16T10:01:12.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:01:12 smithi176 ceph-mon[81689]: pgmap v125: 65 pgs: 6 stale+active+clean, 59 active+clean; 253 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s 2024-07-16T10:01:12.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:01:12 smithi176 ceph-mon[81689]: osdmap e71: 6 total, 5 up, 6 in 2024-07-16T10:01:12.424 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:01:12 smithi156 ceph-mon[93066]: pgmap v125: 65 pgs: 6 stale+active+clean, 59 active+clean; 253 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s 2024-07-16T10:01:12.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:01:12 smithi156 ceph-mon[93066]: osdmap e71: 6 total, 5 up, 6 in 2024-07-16T10:01:12.844 INFO:journalctl@ceph.osd.4.smithi176.stdout:Jul 16 10:01:12 smithi176 podman[95577]: 2024-07-16 10:01:12.454066591 +0000 UTC m=+0.551193272 container remove 8bf73cc33d709f923d3de4c5ef09c0d37fe88fe01e1e28027a63ae5fe7dd8782 (image=quay.ceph.io/ceph-ci/ceph@sha256:c15ec3aa8e6c0a9c12ee24ff79f3e7dd4d01b3d1a702dfeb7d7fff9b0811a393, name=ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-4-deactivate, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-adk4-testing-2024-07-15-1832-squid-a6ca461, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240709, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-07-16T10:01:12.844 INFO:journalctl@ceph.osd.4.smithi176.stdout:Jul 16 10:01:12 smithi176 systemd[1]: ceph-26c91c26-4358-11ef-bcad-c7b262605968@osd.4.service: Deactivated successfully. 2024-07-16T10:01:12.845 INFO:journalctl@ceph.osd.4.smithi176.stdout:Jul 16 10:01:12 smithi176 systemd[1]: Stopped Ceph osd.4 for 26c91c26-4358-11ef-bcad-c7b262605968. 2024-07-16T10:01:12.845 INFO:journalctl@ceph.osd.4.smithi176.stdout:Jul 16 10:01:12 smithi176 systemd[1]: ceph-26c91c26-4358-11ef-bcad-c7b262605968@osd.4.service: Consumed 1min 1.106s CPU time. 2024-07-16T10:01:12.845 INFO:journalctl@ceph.osd.4.smithi176.stdout:Jul 16 10:01:12 smithi176 systemd[1]: Starting Ceph osd.4 for 26c91c26-4358-11ef-bcad-c7b262605968... 2024-07-16T10:01:13.175 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:01:13 smithi176 ceph-mon[81689]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-07-16T10:01:13.176 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:01:13 smithi176 ceph-mon[81689]: Health check failed: Degraded data redundancy: 13/318 objects degraded (4.088%), 4 pgs degraded (PG_DEGRADED) 2024-07-16T10:01:13.176 INFO:journalctl@ceph.osd.4.smithi176.stdout:Jul 16 10:01:12 smithi176 podman[95677]: 2024-07-16 10:01:12.79075902 +0000 UTC m=+0.017276136 image pull 16f7feb043422ce37a2471403d4bc52f550d49cef4b677482112fcc0db3f67ab quay.ceph.io/ceph-ci/ceph@sha256:c15ec3aa8e6c0a9c12ee24ff79f3e7dd4d01b3d1a702dfeb7d7fff9b0811a393 2024-07-16T10:01:13.176 INFO:journalctl@ceph.osd.4.smithi176.stdout:Jul 16 10:01:12 smithi176 podman[95677]: 2024-07-16 10:01:12.888431975 +0000 UTC m=+0.114949104 container create 4935999556e45dc68a26b00d0382112e6cf2832c86e45289a0323100c7cd8031 (image=quay.ceph.io/ceph-ci/ceph@sha256:c15ec3aa8e6c0a9c12ee24ff79f3e7dd4d01b3d1a702dfeb7d7fff9b0811a393, name=ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-4-activate, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.36.0, RELEASE=wip-adk4-testing-2024-07-15-1832-squid-a6ca461, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240709, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-07-16T10:01:13.176 INFO:journalctl@ceph.osd.4.smithi176.stdout:Jul 16 10:01:13 smithi176 podman[95677]: 2024-07-16 10:01:13.174902606 +0000 UTC m=+0.401419726 container init 4935999556e45dc68a26b00d0382112e6cf2832c86e45289a0323100c7cd8031 (image=quay.ceph.io/ceph-ci/ceph@sha256:c15ec3aa8e6c0a9c12ee24ff79f3e7dd4d01b3d1a702dfeb7d7fff9b0811a393, name=ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-4-activate, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.36.0, RELEASE=wip-adk4-testing-2024-07-15-1832-squid-a6ca461, org.label-schema.build-date=20240709, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-07-16T10:01:13.424 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:01:13 smithi156 ceph-mon[93066]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-07-16T10:01:13.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:01:13 smithi156 ceph-mon[93066]: Health check failed: Degraded data redundancy: 13/318 objects degraded (4.088%), 4 pgs degraded (PG_DEGRADED) 2024-07-16T10:01:13.515 INFO:journalctl@ceph.osd.4.smithi176.stdout:Jul 16 10:01:13 smithi176 podman[95677]: 2024-07-16 10:01:13.178671531 +0000 UTC m=+0.405188638 container start 4935999556e45dc68a26b00d0382112e6cf2832c86e45289a0323100c7cd8031 (image=quay.ceph.io/ceph-ci/ceph@sha256:c15ec3aa8e6c0a9c12ee24ff79f3e7dd4d01b3d1a702dfeb7d7fff9b0811a393, name=ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-4-activate, org.label-schema.build-date=20240709, io.buildah.version=1.36.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-adk4-testing-2024-07-15-1832-squid-a6ca461, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-07-16T10:01:13.515 INFO:journalctl@ceph.osd.4.smithi176.stdout:Jul 16 10:01:13 smithi176 podman[95677]: 2024-07-16 10:01:13.249385064 +0000 UTC m=+0.475902180 container attach 4935999556e45dc68a26b00d0382112e6cf2832c86e45289a0323100c7cd8031 (image=quay.ceph.io/ceph-ci/ceph@sha256:c15ec3aa8e6c0a9c12ee24ff79f3e7dd4d01b3d1a702dfeb7d7fff9b0811a393, name=ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-4-activate, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=wip-adk4-testing-2024-07-15-1832-squid-a6ca461, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.build-date=20240709, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0, GIT_BRANCH=HEAD) 2024-07-16T10:01:13.515 INFO:journalctl@ceph.osd.4.smithi176.stdout:Jul 16 10:01:13 smithi176 ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-4-activate[95691]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-07-16T10:01:13.515 INFO:journalctl@ceph.osd.4.smithi176.stdout:Jul 16 10:01:13 smithi176 bash[95677]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-07-16T10:01:13.515 INFO:journalctl@ceph.osd.4.smithi176.stdout:Jul 16 10:01:13 smithi176 ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-4-activate[95691]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-07-16T10:01:13.515 INFO:journalctl@ceph.osd.4.smithi176.stdout:Jul 16 10:01:13 smithi176 bash[95677]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-07-16T10:01:13.516 INFO:journalctl@ceph.osd.4.smithi176.stdout:Jul 16 10:01:13 smithi176 ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-4-activate[95691]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-07-16T10:01:13.516 INFO:journalctl@ceph.osd.4.smithi176.stdout:Jul 16 10:01:13 smithi176 bash[95677]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-07-16T10:01:13.844 INFO:journalctl@ceph.osd.4.smithi176.stdout:Jul 16 10:01:13 smithi176 ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-4-activate[95691]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-4 --no-mon-config --dev /dev/mapper/vg_nvme-lv_3 2024-07-16T10:01:13.844 INFO:journalctl@ceph.osd.4.smithi176.stdout:Jul 16 10:01:13 smithi176 bash[95677]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-4 --no-mon-config --dev /dev/mapper/vg_nvme-lv_3 2024-07-16T10:01:13.844 INFO:journalctl@ceph.osd.4.smithi176.stdout:Jul 16 10:01:13 smithi176 ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-4-activate[95691]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_3 2024-07-16T10:01:13.844 INFO:journalctl@ceph.osd.4.smithi176.stdout:Jul 16 10:01:13 smithi176 bash[95677]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_3 2024-07-16T10:01:13.844 INFO:journalctl@ceph.osd.4.smithi176.stdout:Jul 16 10:01:13 smithi176 ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-4-activate[95691]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-07-16T10:01:13.844 INFO:journalctl@ceph.osd.4.smithi176.stdout:Jul 16 10:01:13 smithi176 bash[95677]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-07-16T10:01:13.844 INFO:journalctl@ceph.osd.4.smithi176.stdout:Jul 16 10:01:13 smithi176 ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-4-activate[95691]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-4/block 2024-07-16T10:01:13.845 INFO:journalctl@ceph.osd.4.smithi176.stdout:Jul 16 10:01:13 smithi176 bash[95677]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-4/block 2024-07-16T10:01:13.845 INFO:journalctl@ceph.osd.4.smithi176.stdout:Jul 16 10:01:13 smithi176 ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-4-activate[95691]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-07-16T10:01:13.845 INFO:journalctl@ceph.osd.4.smithi176.stdout:Jul 16 10:01:13 smithi176 bash[95677]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-07-16T10:01:13.845 INFO:journalctl@ceph.osd.4.smithi176.stdout:Jul 16 10:01:13 smithi176 ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-4-activate[95691]: --> ceph-volume raw activate successful for osd ID: 4 2024-07-16T10:01:13.845 INFO:journalctl@ceph.osd.4.smithi176.stdout:Jul 16 10:01:13 smithi176 bash[95677]: --> ceph-volume raw activate successful for osd ID: 4 2024-07-16T10:01:13.845 INFO:journalctl@ceph.osd.4.smithi176.stdout:Jul 16 10:01:13 smithi176 podman[95677]: 2024-07-16 10:01:13.562480883 +0000 UTC m=+0.788997989 container died 4935999556e45dc68a26b00d0382112e6cf2832c86e45289a0323100c7cd8031 (image=quay.ceph.io/ceph-ci/ceph@sha256:c15ec3aa8e6c0a9c12ee24ff79f3e7dd4d01b3d1a702dfeb7d7fff9b0811a393, name=ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-4-activate, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240709, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-adk4-testing-2024-07-15-1832-squid-a6ca461) 2024-07-16T10:01:14.279 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:01:14 smithi176 ceph-mon[81689]: pgmap v127: 65 pgs: 4 active+undersized, 8 peering, 3 stale+active+clean, 4 active+undersized+degraded, 46 active+clean; 253 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 2 op/s; 13/318 objects degraded (4.088%) 2024-07-16T10:01:14.279 INFO:journalctl@ceph.osd.4.smithi176.stdout:Jul 16 10:01:14 smithi176 podman[95818]: 2024-07-16 10:01:14.069395555 +0000 UTC m=+0.501319325 container remove 4935999556e45dc68a26b00d0382112e6cf2832c86e45289a0323100c7cd8031 (image=quay.ceph.io/ceph-ci/ceph@sha256:c15ec3aa8e6c0a9c12ee24ff79f3e7dd4d01b3d1a702dfeb7d7fff9b0811a393, name=ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-4-activate, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , RELEASE=wip-adk4-testing-2024-07-15-1832-squid-a6ca461, io.buildah.version=1.36.0, org.label-schema.build-date=20240709, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD) 2024-07-16T10:01:14.424 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:01:14 smithi156 ceph-mon[93066]: pgmap v127: 65 pgs: 4 active+undersized, 8 peering, 3 stale+active+clean, 4 active+undersized+degraded, 46 active+clean; 253 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 2 op/s; 13/318 objects degraded (4.088%) 2024-07-16T10:01:14.548 INFO:journalctl@ceph.osd.4.smithi176.stdout:Jul 16 10:01:14 smithi176 podman[95876]: 2024-07-16 10:01:14.278654633 +0000 UTC m=+0.110110060 container create 1bd3a25317b011abe1e64a630e6fb3932de6f017001ef671784eaf4a40b15409 (image=quay.ceph.io/ceph-ci/ceph@sha256:c15ec3aa8e6c0a9c12ee24ff79f3e7dd4d01b3d1a702dfeb7d7fff9b0811a393, name=ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-4, io.buildah.version=1.36.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, RELEASE=wip-adk4-testing-2024-07-15-1832-squid-a6ca461, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240709, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-07-16T10:01:14.548 INFO:journalctl@ceph.osd.4.smithi176.stdout:Jul 16 10:01:14 smithi176 podman[95876]: 2024-07-16 10:01:14.185670033 +0000 UTC m=+0.017125415 image pull 16f7feb043422ce37a2471403d4bc52f550d49cef4b677482112fcc0db3f67ab quay.ceph.io/ceph-ci/ceph@sha256:c15ec3aa8e6c0a9c12ee24ff79f3e7dd4d01b3d1a702dfeb7d7fff9b0811a393 2024-07-16T10:01:14.548 INFO:journalctl@ceph.osd.4.smithi176.stdout:Jul 16 10:01:14 smithi176 podman[95876]: 2024-07-16 10:01:14.547711907 +0000 UTC m=+0.379167273 container init 1bd3a25317b011abe1e64a630e6fb3932de6f017001ef671784eaf4a40b15409 (image=quay.ceph.io/ceph-ci/ceph@sha256:c15ec3aa8e6c0a9c12ee24ff79f3e7dd4d01b3d1a702dfeb7d7fff9b0811a393, name=ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-4, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-adk4-testing-2024-07-15-1832-squid-a6ca461, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240709, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True) 2024-07-16T10:01:14.844 INFO:journalctl@ceph.osd.4.smithi176.stdout:Jul 16 10:01:14 smithi176 podman[95876]: 2024-07-16 10:01:14.551074772 +0000 UTC m=+0.382530145 container start 1bd3a25317b011abe1e64a630e6fb3932de6f017001ef671784eaf4a40b15409 (image=quay.ceph.io/ceph-ci/ceph@sha256:c15ec3aa8e6c0a9c12ee24ff79f3e7dd4d01b3d1a702dfeb7d7fff9b0811a393, name=ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-4, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240709, RELEASE=wip-adk4-testing-2024-07-15-1832-squid-a6ca461) 2024-07-16T10:01:14.844 INFO:journalctl@ceph.osd.4.smithi176.stdout:Jul 16 10:01:14 smithi176 ceph-osd[95895]: -- 172.21.15.176:0/1448338064 <== mon.1 v2:172.21.15.176:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55cf34c961e0 con 0x55cf33e49c00 2024-07-16T10:01:14.844 INFO:journalctl@ceph.osd.4.smithi176.stdout:Jul 16 10:01:14 smithi176 bash[95876]: 1bd3a25317b011abe1e64a630e6fb3932de6f017001ef671784eaf4a40b15409 2024-07-16T10:01:14.844 INFO:journalctl@ceph.osd.4.smithi176.stdout:Jul 16 10:01:14 smithi176 systemd[1]: Started Ceph osd.4 for 26c91c26-4358-11ef-bcad-c7b262605968. 2024-07-16T10:01:15.374 INFO:journalctl@ceph.osd.4.smithi176.stdout:Jul 16 10:01:15 smithi176 ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-4[95891]: 2024-07-16T10:01:15.094+0000 7ff6a6856640 -1 Falling back to public interface 2024-07-16T10:01:16.013 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:01:15 smithi176 ceph-mon[81689]: pgmap v128: 65 pgs: 9 active+undersized, 8 peering, 1 stale+active+clean, 6 active+undersized+degraded, 41 active+clean; 253 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 0 op/s; 18/318 objects degraded (5.660%) 2024-07-16T10:01:16.013 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:01:15 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:01:16.013 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:01:15 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:01:16.013 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:01:15 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-16T10:01:16.174 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:01:15 smithi156 ceph-mon[93066]: pgmap v128: 65 pgs: 9 active+undersized, 8 peering, 1 stale+active+clean, 6 active+undersized+degraded, 41 active+clean; 253 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 0 op/s; 18/318 objects degraded (5.660%) 2024-07-16T10:01:16.175 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:01:15 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:01:16.175 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:01:15 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:01:16.175 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:01:15 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-16T10:01:17.058 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:01:16 smithi176 ceph-mon[81689]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-07-16T10:01:17.174 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:01:16 smithi156 ceph-mon[93066]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-07-16T10:01:18.596 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:01:18 smithi176 ceph-mon[81689]: pgmap v129: 65 pgs: 18 active+undersized, 16 active+undersized+degraded, 31 active+clean; 253 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 0 op/s; 53/318 objects degraded (16.667%) 2024-07-16T10:01:18.596 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:01:18 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:01:18.597 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:01:18 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:01:18.597 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:01:18 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:01:18.597 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:01:18 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:01:18.661 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:01:18 smithi156 ceph-mon[93066]: pgmap v129: 65 pgs: 18 active+undersized, 16 active+undersized+degraded, 31 active+clean; 253 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 0 op/s; 53/318 objects degraded (16.667%) 2024-07-16T10:01:18.661 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:01:18 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:01:18.661 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:01:18 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:01:18.661 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:01:18 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:01:18.661 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:01:18 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:01:19.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:01:19 smithi156 ceph-mon[93066]: pgmap v130: 65 pgs: 18 active+undersized, 16 active+undersized+degraded, 31 active+clean; 253 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 53/318 objects degraded (16.667%) 2024-07-16T10:01:19.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:01:19 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:01:19.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:01:19 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-16T10:01:19.701 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:01:19 smithi176 ceph-mon[81689]: pgmap v130: 65 pgs: 18 active+undersized, 16 active+undersized+degraded, 31 active+clean; 253 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 53/318 objects degraded (16.667%) 2024-07-16T10:01:19.702 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:01:19 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:01:19.702 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:01:19 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-16T10:01:20.344 INFO:journalctl@ceph.osd.4.smithi176.stdout:Jul 16 10:01:20 smithi176 ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-4[95891]: 2024-07-16T10:01:20.091+0000 7ff6a6856640 -1 osd.4 69 log_to_monitors true 2024-07-16T10:01:20.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:01:20 smithi156 ceph-mon[93066]: from='osd.4 [v2:172.21.15.176:6808/1802375970,v1:172.21.15.176:6809/1802375970]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-07-16T10:01:20.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:01:20 smithi156 ceph-mon[93066]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-07-16T10:01:20.814 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:01:20 smithi176 ceph-mon[81689]: from='osd.4 [v2:172.21.15.176:6808/1802375970,v1:172.21.15.176:6809/1802375970]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-07-16T10:01:20.814 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:01:20 smithi176 ceph-mon[81689]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-07-16T10:01:21.382 INFO:journalctl@ceph.osd.4.smithi176.stdout:Jul 16 10:01:21 smithi176 ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-4[95891]: 2024-07-16T10:01:21.247+0000 7ff69a5e8640 -1 osd.4 69 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-07-16T10:01:21.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:01:21 smithi156 ceph-mon[93066]: pgmap v131: 65 pgs: 18 active+undersized, 16 active+undersized+degraded, 31 active+clean; 253 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 0 op/s; 53/318 objects degraded (16.667%) 2024-07-16T10:01:21.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:01:21 smithi156 ceph-mon[93066]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-07-16T10:01:21.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:01:21 smithi156 ceph-mon[93066]: osdmap e72: 6 total, 5 up, 6 in 2024-07-16T10:01:21.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:01:21 smithi156 ceph-mon[93066]: from='osd.4 [v2:172.21.15.176:6808/1802375970,v1:172.21.15.176:6809/1802375970]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi176", "root=default"]}]: dispatch 2024-07-16T10:01:21.676 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:01:21 smithi156 ceph-mon[93066]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi176", "root=default"]}]: dispatch 2024-07-16T10:01:21.676 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:01:21 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:01:21.676 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:01:21 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:01:21.676 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:01:21 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-16T10:01:21.676 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:01:21 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-16T10:01:21.676 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:01:21 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:01:21.676 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:01:21 smithi156 ceph-mon[93066]: from='osd.4 ' entity='osd.4' 2024-07-16T10:01:21.676 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:01:21 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-16T10:01:21.676 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:01:21 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:01:21.676 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:01:21 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:01:21.676 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:01:21 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:01:21.676 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:01:21 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-07-16T10:01:21.676 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:01:21 smithi156 ceph-mon[93066]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-07-16T10:01:21.677 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:01:21 smithi156 ceph-mon[93066]: Upgrade: unsafe to stop osd(s) at this time (11 PGs are or would become offline) 2024-07-16T10:01:21.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:01:21 smithi176 ceph-mon[81689]: pgmap v131: 65 pgs: 18 active+undersized, 16 active+undersized+degraded, 31 active+clean; 253 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 0 op/s; 53/318 objects degraded (16.667%) 2024-07-16T10:01:21.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:01:21 smithi176 ceph-mon[81689]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-07-16T10:01:21.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:01:21 smithi176 ceph-mon[81689]: osdmap e72: 6 total, 5 up, 6 in 2024-07-16T10:01:21.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:01:21 smithi176 ceph-mon[81689]: from='osd.4 [v2:172.21.15.176:6808/1802375970,v1:172.21.15.176:6809/1802375970]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi176", "root=default"]}]: dispatch 2024-07-16T10:01:21.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:01:21 smithi176 ceph-mon[81689]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi176", "root=default"]}]: dispatch 2024-07-16T10:01:21.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:01:21 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:01:21.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:01:21 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:01:21.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:01:21 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-16T10:01:21.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:01:21 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-16T10:01:21.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:01:21 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:01:21.845 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:01:21 smithi176 ceph-mon[81689]: from='osd.4 ' entity='osd.4' 2024-07-16T10:01:21.845 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:01:21 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-16T10:01:21.845 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:01:21 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:01:21.845 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:01:21 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:01:21.845 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:01:21 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:01:21.845 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:01:21 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-07-16T10:01:21.845 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:01:21 smithi176 ceph-mon[81689]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-07-16T10:01:21.845 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:01:21 smithi176 ceph-mon[81689]: Upgrade: unsafe to stop osd(s) at this time (11 PGs are or would become offline) 2024-07-16T10:01:22.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:01:22 smithi156 ceph-mon[93066]: Health check update: Degraded data redundancy: 53/318 objects degraded (16.667%), 16 pgs degraded (PG_DEGRADED) 2024-07-16T10:01:22.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:01:22 smithi156 ceph-mon[93066]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-07-16T10:01:22.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:01:22 smithi156 ceph-mon[93066]: osd.4 [v2:172.21.15.176:6808/1802375970,v1:172.21.15.176:6809/1802375970] boot 2024-07-16T10:01:22.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:01:22 smithi156 ceph-mon[93066]: osdmap e73: 6 total, 6 up, 6 in 2024-07-16T10:01:22.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:01:22 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-07-16T10:01:22.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:01:22 smithi176 ceph-mon[81689]: Health check update: Degraded data redundancy: 53/318 objects degraded (16.667%), 16 pgs degraded (PG_DEGRADED) 2024-07-16T10:01:22.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:01:22 smithi176 ceph-mon[81689]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-07-16T10:01:22.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:01:22 smithi176 ceph-mon[81689]: osd.4 [v2:172.21.15.176:6808/1802375970,v1:172.21.15.176:6809/1802375970] boot 2024-07-16T10:01:22.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:01:22 smithi176 ceph-mon[81689]: osdmap e73: 6 total, 6 up, 6 in 2024-07-16T10:01:22.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:01:22 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-07-16T10:01:24.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:01:24 smithi176 ceph-mon[81689]: pgmap v134: 65 pgs: 18 active+undersized, 16 active+undersized+degraded, 31 active+clean; 253 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 2 op/s; 53/318 objects degraded (16.667%) 2024-07-16T10:01:24.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:01:24 smithi176 ceph-mon[81689]: osdmap e74: 6 total, 6 up, 6 in 2024-07-16T10:01:24.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:01:24 smithi156 ceph-mon[93066]: pgmap v134: 65 pgs: 18 active+undersized, 16 active+undersized+degraded, 31 active+clean; 253 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 2 op/s; 53/318 objects degraded (16.667%) 2024-07-16T10:01:24.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:01:24 smithi156 ceph-mon[93066]: osdmap e74: 6 total, 6 up, 6 in 2024-07-16T10:01:26.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:01:26 smithi176 ceph-mon[81689]: pgmap v136: 65 pgs: 14 active+undersized, 15 active+undersized+degraded, 36 active+clean; 253 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s; 50/318 objects degraded (15.723%) 2024-07-16T10:01:26.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:01:26 smithi156 ceph-mon[93066]: pgmap v136: 65 pgs: 14 active+undersized, 15 active+undersized+degraded, 36 active+clean; 253 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s; 50/318 objects degraded (15.723%) 2024-07-16T10:01:27.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:01:27 smithi176 ceph-mon[81689]: Health check update: Degraded data redundancy: 50/318 objects degraded (15.723%), 15 pgs degraded (PG_DEGRADED) 2024-07-16T10:01:27.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:01:27 smithi156 ceph-mon[93066]: Health check update: Degraded data redundancy: 50/318 objects degraded (15.723%), 15 pgs degraded (PG_DEGRADED) 2024-07-16T10:01:28.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:01:28 smithi176 ceph-mon[81689]: pgmap v137: 65 pgs: 3 active+undersized, 3 active+undersized+degraded, 59 active+clean; 253 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.3 KiB/s rd, 3 op/s; 11/318 objects degraded (3.459%) 2024-07-16T10:01:28.659 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:01:28 smithi156 ceph-mon[93066]: pgmap v137: 65 pgs: 3 active+undersized, 3 active+undersized+degraded, 59 active+clean; 253 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.3 KiB/s rd, 3 op/s; 11/318 objects degraded (3.459%) 2024-07-16T10:01:29.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:01:29 smithi176 ceph-mon[81689]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 11/318 objects degraded (3.459%), 3 pgs degraded) 2024-07-16T10:01:29.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:01:29 smithi176 ceph-mon[81689]: Cluster is now healthy 2024-07-16T10:01:29.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:01:29 smithi156 ceph-mon[93066]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 11/318 objects degraded (3.459%), 3 pgs degraded) 2024-07-16T10:01:29.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:01:29 smithi156 ceph-mon[93066]: Cluster is now healthy 2024-07-16T10:01:30.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:01:30 smithi176 ceph-mon[81689]: pgmap v138: 65 pgs: 65 active+clean; 253 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.0 KiB/s rd, 2 op/s 2024-07-16T10:01:30.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:01:30 smithi156 ceph-mon[93066]: pgmap v138: 65 pgs: 65 active+clean; 253 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.0 KiB/s rd, 2 op/s 2024-07-16T10:01:32.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:01:32 smithi176 ceph-mon[81689]: pgmap v139: 65 pgs: 65 active+clean; 253 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.2 KiB/s rd, 3 op/s 2024-07-16T10:01:32.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:01:32 smithi156 ceph-mon[93066]: pgmap v139: 65 pgs: 65 active+clean; 253 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.2 KiB/s rd, 3 op/s 2024-07-16T10:01:34.420 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:34.417+0000 7f92941c7640 1 -- 172.21.15.156:0/2851648129 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f928c0fe310 msgr2=0x7f928c0fa470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:01:34.420 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:34.417+0000 7f92941c7640 1 --2- 172.21.15.156:0/2851648129 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f928c0fe310 0x7f928c0fa470 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f927c0048d0 tx=0x7f927c009da0 comp rx=0 tx=0).stop 2024-07-16T10:01:34.420 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:34.417+0000 7f92941c7640 1 -- 172.21.15.156:0/2851648129 shutdown_connections 2024-07-16T10:01:34.420 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:34.417+0000 7f92941c7640 1 --2- 172.21.15.156:0/2851648129 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f928c0fe310 0x7f928c0fa470 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:01:34.420 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:34.417+0000 7f92941c7640 1 --2- 172.21.15.156:0/2851648129 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f928c0fd9c0 0x7f928c0fddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:01:34.420 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:34.417+0000 7f92941c7640 1 -- 172.21.15.156:0/2851648129 >> 172.21.15.156:0/2851648129 conn(0x7f928c0f61c0 msgr2=0x7f928c0f85f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T10:01:34.420 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:34.417+0000 7f92941c7640 1 -- 172.21.15.156:0/2851648129 shutdown_connections 2024-07-16T10:01:34.420 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:34.417+0000 7f92941c7640 1 -- 172.21.15.156:0/2851648129 wait complete. 2024-07-16T10:01:34.421 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:34.417+0000 7f92941c7640 1 Processor -- start 2024-07-16T10:01:34.421 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:34.417+0000 7f92941c7640 1 -- start start 2024-07-16T10:01:34.421 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:34.417+0000 7f92941c7640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f928c0fd9c0 0x7f928c10cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:01:34.421 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:34.417+0000 7f92941c7640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f928c0fe310 0x7f928c10d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:01:34.421 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:34.417+0000 7f92941c7640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f928c10daf0 con 0x7f928c0fe310 2024-07-16T10:01:34.421 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:34.417+0000 7f92941c7640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f928c10e2a0 con 0x7f928c0fd9c0 2024-07-16T10:01:34.421 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:34.418+0000 7f9291762640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f928c0fe310 0x7f928c10d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:01:34.421 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:34.418+0000 7f9291762640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f928c0fe310 0x7f928c10d4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:59984/0 (socket says 172.21.15.156:59984) 2024-07-16T10:01:34.421 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:34.418+0000 7f9291762640 1 -- 172.21.15.156:0/1207546672 learned_addr learned my addr 172.21.15.156:0/1207546672 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T10:01:34.421 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:34.418+0000 7f9291f63640 1 --2- 172.21.15.156:0/1207546672 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f928c0fd9c0 0x7f928c10cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:01:34.421 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:34.418+0000 7f9291762640 1 -- 172.21.15.156:0/1207546672 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f928c0fd9c0 msgr2=0x7f928c10cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:01:34.421 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:34.418+0000 7f9291762640 1 --2- 172.21.15.156:0/1207546672 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f928c0fd9c0 0x7f928c10cf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:01:34.422 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:34.418+0000 7f9291762640 1 -- 172.21.15.156:0/1207546672 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f927c005040 con 0x7f928c0fe310 2024-07-16T10:01:34.422 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:34.418+0000 7f9291762640 1 --2- 172.21.15.156:0/1207546672 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f928c0fe310 0x7f928c10d4a0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f927c0048a0 tx=0x7f927c009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T10:01:34.422 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:34.418+0000 7f92827fc640 1 -- 172.21.15.156:0/1207546672 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f927c038070 con 0x7f928c0fe310 2024-07-16T10:01:34.422 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:34.418+0000 7f92827fc640 1 -- 172.21.15.156:0/1207546672 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f927c0092b0 con 0x7f928c0fe310 2024-07-16T10:01:34.422 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:34.418+0000 7f92941c7640 1 -- 172.21.15.156:0/1207546672 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f928c10e4d0 con 0x7f928c0fe310 2024-07-16T10:01:34.422 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:34.418+0000 7f92827fc640 1 -- 172.21.15.156:0/1207546672 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f927c033710 con 0x7f928c0fe310 2024-07-16T10:01:34.422 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:34.418+0000 7f92941c7640 1 -- 172.21.15.156:0/1207546672 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f928c10e990 con 0x7f928c0fe310 2024-07-16T10:01:34.423 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:34.420+0000 7f92941c7640 1 -- 172.21.15.156:0/1207546672 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f928c0f9e60 con 0x7f928c0fe310 2024-07-16T10:01:34.423 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:34.420+0000 7f92827fc640 1 -- 172.21.15.156:0/1207546672 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 38) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f927c030020 con 0x7f928c0fe310 2024-07-16T10:01:34.423 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:34.421+0000 7f92827fc640 1 --2- 172.21.15.156:0/1207546672 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f927806e0b0 0x7f9278070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:01:34.423 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:34.421+0000 7f9291f63640 1 --2- 172.21.15.156:0/1207546672 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f927806e0b0 0x7f9278070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:01:34.423 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:34.421+0000 7f92827fc640 1 -- 172.21.15.156:0/1207546672 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f927c0adac0 con 0x7f928c0fe310 2024-07-16T10:01:34.423 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:34.421+0000 7f9291f63640 1 --2- 172.21.15.156:0/1207546672 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f927806e0b0 0x7f9278070560 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f9288009fd0 tx=0x7f928800b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T10:01:34.426 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:34.425+0000 7f92827fc640 1 -- 172.21.15.156:0/1207546672 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187716 (secure 0 0 0) 0x7f927c076ef0 con 0x7f928c0fe310 2024-07-16T10:01:34.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:01:34 smithi176 ceph-mon[81689]: pgmap v140: 65 pgs: 65 active+clean; 253 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.8 KiB/s rd, 3 op/s 2024-07-16T10:01:34.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:01:34 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:01:34.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:01:34 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-16T10:01:34.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:01:34 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:01:34.622 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:34.620+0000 7f92941c7640 1 -- 172.21.15.156:0/1207546672 --> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f928c10ecc0 con 0x7f927806e0b0 2024-07-16T10:01:34.622 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:01:34 smithi156 ceph-mon[93066]: pgmap v140: 65 pgs: 65 active+clean; 253 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.8 KiB/s rd, 3 op/s 2024-07-16T10:01:34.622 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:01:34 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:01:34.622 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:01:34 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-16T10:01:34.623 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:01:34 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:01:34.623 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:34.622+0000 7f92827fc640 1 -- 172.21.15.156:0/1207546672 <== mgr.34104 v2:172.21.15.156:6800/3367765292 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7f928c10ecc0 con 0x7f927806e0b0 2024-07-16T10:01:34.625 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:34.624+0000 7f92941c7640 1 -- 172.21.15.156:0/1207546672 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f927806e0b0 msgr2=0x7f9278070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:01:34.625 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:34.624+0000 7f92941c7640 1 --2- 172.21.15.156:0/1207546672 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f927806e0b0 0x7f9278070560 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f9288009fd0 tx=0x7f928800b040 comp rx=0 tx=0).stop 2024-07-16T10:01:34.626 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:34.624+0000 7f92941c7640 1 -- 172.21.15.156:0/1207546672 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f928c0fe310 msgr2=0x7f928c10d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:01:34.626 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:34.624+0000 7f92941c7640 1 --2- 172.21.15.156:0/1207546672 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f928c0fe310 0x7f928c10d4a0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f927c0048a0 tx=0x7f927c009da0 comp rx=0 tx=0).stop 2024-07-16T10:01:34.626 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:34.624+0000 7f92941c7640 1 -- 172.21.15.156:0/1207546672 shutdown_connections 2024-07-16T10:01:34.626 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:34.624+0000 7f92941c7640 1 --2- 172.21.15.156:0/1207546672 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f927806e0b0 0x7f9278070560 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:01:34.626 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:34.624+0000 7f92941c7640 1 --2- 172.21.15.156:0/1207546672 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f928c0fe310 0x7f928c10d4a0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:01:34.626 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:34.624+0000 7f92941c7640 1 --2- 172.21.15.156:0/1207546672 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f928c0fd9c0 0x7f928c10cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:01:34.626 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:34.624+0000 7f92941c7640 1 -- 172.21.15.156:0/1207546672 >> 172.21.15.156:0/1207546672 conn(0x7f928c0f61c0 msgr2=0x7f928c0f7840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T10:01:34.626 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:34.624+0000 7f92941c7640 1 -- 172.21.15.156:0/1207546672 shutdown_connections 2024-07-16T10:01:34.626 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:34.624+0000 7f92941c7640 1 -- 172.21.15.156:0/1207546672 wait complete. 2024-07-16T10:01:34.634 INFO:teuthology.orchestra.run.smithi156.stdout:true 2024-07-16T10:01:34.708 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:34.705+0000 7f9d5bcdc640 1 -- 172.21.15.156:0/2477246213 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9d540fc340 msgr2=0x7f9d540fc750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:01:34.708 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:34.705+0000 7f9d5bcdc640 1 --2- 172.21.15.156:0/2477246213 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9d540fc340 0x7f9d540fc750 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f9d4c0048d0 tx=0x7f9d4c009da0 comp rx=0 tx=0).stop 2024-07-16T10:01:34.708 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:34.705+0000 7f9d5bcdc640 1 -- 172.21.15.156:0/2477246213 shutdown_connections 2024-07-16T10:01:34.708 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:34.705+0000 7f9d5bcdc640 1 --2- 172.21.15.156:0/2477246213 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9d540fd610 0x7f9d540fda80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:01:34.708 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:34.705+0000 7f9d5bcdc640 1 --2- 172.21.15.156:0/2477246213 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9d540fc340 0x7f9d540fc750 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:01:34.708 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:34.705+0000 7f9d5bcdc640 1 -- 172.21.15.156:0/2477246213 >> 172.21.15.156:0/2477246213 conn(0x7f9d540f7b20 msgr2=0x7f9d540f9f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T10:01:34.708 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:34.705+0000 7f9d5bcdc640 1 -- 172.21.15.156:0/2477246213 shutdown_connections 2024-07-16T10:01:34.708 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:34.705+0000 7f9d5bcdc640 1 -- 172.21.15.156:0/2477246213 wait complete. 2024-07-16T10:01:34.708 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:34.705+0000 7f9d5bcdc640 1 Processor -- start 2024-07-16T10:01:34.708 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:34.706+0000 7f9d5bcdc640 1 -- start start 2024-07-16T10:01:34.709 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:34.706+0000 7f9d5bcdc640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9d540fc340 0x7f9d5410cf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:01:34.709 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:34.706+0000 7f9d5bcdc640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9d540fd610 0x7f9d5410d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:01:34.709 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:34.706+0000 7f9d5bcdc640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d5410dad0 con 0x7f9d540fc340 2024-07-16T10:01:34.709 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:34.706+0000 7f9d5bcdc640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d5410e2f0 con 0x7f9d540fd610 2024-07-16T10:01:34.709 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:34.706+0000 7f9d59a78640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9d540fc340 0x7f9d5410cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:01:34.709 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:34.706+0000 7f9d59a78640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9d540fc340 0x7f9d5410cf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:60004/0 (socket says 172.21.15.156:60004) 2024-07-16T10:01:34.709 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:34.706+0000 7f9d59a78640 1 -- 172.21.15.156:0/851818675 learned_addr learned my addr 172.21.15.156:0/851818675 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T10:01:34.709 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:34.706+0000 7f9d59277640 1 --2- 172.21.15.156:0/851818675 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9d540fd610 0x7f9d5410d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:01:34.709 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:34.706+0000 7f9d59a78640 1 -- 172.21.15.156:0/851818675 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9d540fd610 msgr2=0x7f9d5410d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:01:34.709 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:34.706+0000 7f9d59a78640 1 --2- 172.21.15.156:0/851818675 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9d540fd610 0x7f9d5410d480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:01:34.709 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:34.706+0000 7f9d59a78640 1 -- 172.21.15.156:0/851818675 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d4c005040 con 0x7f9d540fc340 2024-07-16T10:01:34.709 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:34.706+0000 7f9d59a78640 1 --2- 172.21.15.156:0/851818675 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9d540fc340 0x7f9d5410cf40 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f9d4c009f80 tx=0x7f9d4c009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T10:01:34.710 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:34.706+0000 7f9d527fc640 1 -- 172.21.15.156:0/851818675 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9d4c038070 con 0x7f9d540fc340 2024-07-16T10:01:34.710 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:34.706+0000 7f9d527fc640 1 -- 172.21.15.156:0/851818675 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9d4c032970 con 0x7f9d540fc340 2024-07-16T10:01:34.710 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:34.706+0000 7f9d5bcdc640 1 -- 172.21.15.156:0/851818675 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d5410e520 con 0x7f9d540fc340 2024-07-16T10:01:34.710 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:34.706+0000 7f9d527fc640 1 -- 172.21.15.156:0/851818675 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9d4c033860 con 0x7f9d540fc340 2024-07-16T10:01:34.710 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:34.707+0000 7f9d5bcdc640 1 -- 172.21.15.156:0/851818675 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d5410ea60 con 0x7f9d540fc340 2024-07-16T10:01:34.711 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:34.708+0000 7f9d5bcdc640 1 -- 172.21.15.156:0/851818675 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d540fb7a0 con 0x7f9d540fc340 2024-07-16T10:01:34.711 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:34.708+0000 7f9d527fc640 1 -- 172.21.15.156:0/851818675 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 38) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f9d4c030020 con 0x7f9d540fc340 2024-07-16T10:01:34.711 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:34.708+0000 7f9d527fc640 1 --2- 172.21.15.156:0/851818675 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f9d3c06e0b0 0x7f9d3c070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:01:34.711 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:34.708+0000 7f9d527fc640 1 -- 172.21.15.156:0/851818675 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f9d4c0ad9f0 con 0x7f9d540fc340 2024-07-16T10:01:34.711 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:34.708+0000 7f9d59277640 1 --2- 172.21.15.156:0/851818675 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f9d3c06e0b0 0x7f9d3c070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:01:34.711 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:34.709+0000 7f9d59277640 1 --2- 172.21.15.156:0/851818675 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f9d3c06e0b0 0x7f9d3c070560 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f9d5410ccf0 tx=0x7f9d48009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T10:01:34.714 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:34.713+0000 7f9d527fc640 1 -- 172.21.15.156:0/851818675 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187716 (secure 0 0 0) 0x7f9d4c0afe60 con 0x7f9d540fc340 2024-07-16T10:01:34.906 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:34.905+0000 7f9d5bcdc640 1 -- 172.21.15.156:0/851818675 --> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f9d5410ed90 con 0x7f9d3c06e0b0 2024-07-16T10:01:34.907 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:34.906+0000 7f9d527fc640 1 -- 172.21.15.156:0/851818675 <== mgr.34104 v2:172.21.15.156:6800/3367765292 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7f9d5410ed90 con 0x7f9d3c06e0b0 2024-07-16T10:01:34.910 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:34.908+0000 7f9d5bcdc640 1 -- 172.21.15.156:0/851818675 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f9d3c06e0b0 msgr2=0x7f9d3c070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:01:34.910 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:34.908+0000 7f9d5bcdc640 1 --2- 172.21.15.156:0/851818675 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f9d3c06e0b0 0x7f9d3c070560 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f9d5410ccf0 tx=0x7f9d48009040 comp rx=0 tx=0).stop 2024-07-16T10:01:34.910 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:34.908+0000 7f9d5bcdc640 1 -- 172.21.15.156:0/851818675 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9d540fc340 msgr2=0x7f9d5410cf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:01:34.910 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:34.908+0000 7f9d5bcdc640 1 --2- 172.21.15.156:0/851818675 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9d540fc340 0x7f9d5410cf40 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f9d4c009f80 tx=0x7f9d4c009da0 comp rx=0 tx=0).stop 2024-07-16T10:01:34.910 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:34.908+0000 7f9d5bcdc640 1 -- 172.21.15.156:0/851818675 shutdown_connections 2024-07-16T10:01:34.910 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:34.908+0000 7f9d5bcdc640 1 --2- 172.21.15.156:0/851818675 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f9d3c06e0b0 0x7f9d3c070560 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:01:34.910 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:34.908+0000 7f9d5bcdc640 1 --2- 172.21.15.156:0/851818675 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9d540fd610 0x7f9d5410d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:01:34.910 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:34.908+0000 7f9d5bcdc640 1 --2- 172.21.15.156:0/851818675 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9d540fc340 0x7f9d5410cf40 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:01:34.911 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:34.908+0000 7f9d5bcdc640 1 -- 172.21.15.156:0/851818675 >> 172.21.15.156:0/851818675 conn(0x7f9d540f7b20 msgr2=0x7f9d540f91f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T10:01:34.911 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:34.909+0000 7f9d5bcdc640 1 -- 172.21.15.156:0/851818675 shutdown_connections 2024-07-16T10:01:34.911 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:34.909+0000 7f9d5bcdc640 1 -- 172.21.15.156:0/851818675 wait complete. 2024-07-16T10:01:34.995 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:34.988+0000 7fb513b6c640 1 -- 172.21.15.156:0/150392112 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb50c0fc330 msgr2=0x7fb50c0fc740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:01:34.995 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:34.988+0000 7fb513b6c640 1 --2- 172.21.15.156:0/150392112 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb50c0fc330 0x7fb50c0fc740 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fb4fc0048d0 tx=0x7fb4fc009da0 comp rx=0 tx=0).stop 2024-07-16T10:01:34.995 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:34.989+0000 7fb513b6c640 1 -- 172.21.15.156:0/150392112 shutdown_connections 2024-07-16T10:01:34.995 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:34.989+0000 7fb513b6c640 1 --2- 172.21.15.156:0/150392112 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb50c0fd600 0x7fb50c0fda70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:01:34.995 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:34.989+0000 7fb513b6c640 1 --2- 172.21.15.156:0/150392112 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb50c0fc330 0x7fb50c0fc740 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:01:34.995 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:34.989+0000 7fb513b6c640 1 -- 172.21.15.156:0/150392112 >> 172.21.15.156:0/150392112 conn(0x7fb50c0f7af0 msgr2=0x7fb50c0f9f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T10:01:34.996 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:34.989+0000 7fb513b6c640 1 -- 172.21.15.156:0/150392112 shutdown_connections 2024-07-16T10:01:34.996 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:34.989+0000 7fb513b6c640 1 -- 172.21.15.156:0/150392112 wait complete. 2024-07-16T10:01:34.996 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:34.989+0000 7fb513b6c640 1 Processor -- start 2024-07-16T10:01:34.996 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:34.989+0000 7fb513b6c640 1 -- start start 2024-07-16T10:01:34.996 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:34.989+0000 7fb513b6c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb50c0fc330 0x7fb50c10cf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:01:34.996 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:34.989+0000 7fb513b6c640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb50c0fd600 0x7fb50c10d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:01:34.996 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:34.989+0000 7fb513b6c640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb50c10dad0 con 0x7fb50c0fc330 2024-07-16T10:01:34.996 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:34.989+0000 7fb513b6c640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb50c10e2f0 con 0x7fb50c0fd600 2024-07-16T10:01:34.996 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:34.990+0000 7fb511908640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb50c0fc330 0x7fb50c10cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:01:34.996 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:34.990+0000 7fb511908640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb50c0fc330 0x7fb50c10cf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:60022/0 (socket says 172.21.15.156:60022) 2024-07-16T10:01:34.996 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:34.990+0000 7fb511908640 1 -- 172.21.15.156:0/1245712789 learned_addr learned my addr 172.21.15.156:0/1245712789 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T10:01:34.996 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:34.990+0000 7fb511107640 1 --2- 172.21.15.156:0/1245712789 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb50c0fd600 0x7fb50c10d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:01:34.997 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:34.990+0000 7fb511908640 1 -- 172.21.15.156:0/1245712789 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb50c0fd600 msgr2=0x7fb50c10d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:01:34.997 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:34.990+0000 7fb511908640 1 --2- 172.21.15.156:0/1245712789 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb50c0fd600 0x7fb50c10d480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:01:34.997 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:34.990+0000 7fb511908640 1 -- 172.21.15.156:0/1245712789 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4fc005040 con 0x7fb50c0fc330 2024-07-16T10:01:34.997 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:34.990+0000 7fb511908640 1 --2- 172.21.15.156:0/1245712789 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb50c0fc330 0x7fb50c10cf40 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fb4fc009f80 tx=0x7fb4fc009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T10:01:34.997 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:34.990+0000 7fb5027fc640 1 -- 172.21.15.156:0/1245712789 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb4fc038070 con 0x7fb50c0fc330 2024-07-16T10:01:34.997 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:34.990+0000 7fb5027fc640 1 -- 172.21.15.156:0/1245712789 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb4fc032970 con 0x7fb50c0fc330 2024-07-16T10:01:34.997 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:34.990+0000 7fb513b6c640 1 -- 172.21.15.156:0/1245712789 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb50c10e520 con 0x7fb50c0fc330 2024-07-16T10:01:34.997 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:34.990+0000 7fb5027fc640 1 -- 172.21.15.156:0/1245712789 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb4fc033860 con 0x7fb50c0fc330 2024-07-16T10:01:34.997 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:34.990+0000 7fb513b6c640 1 -- 172.21.15.156:0/1245712789 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb50c10ea60 con 0x7fb50c0fc330 2024-07-16T10:01:34.997 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:34.991+0000 7fb513b6c640 1 -- 172.21.15.156:0/1245712789 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb50c0fb790 con 0x7fb50c0fc330 2024-07-16T10:01:34.997 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:34.992+0000 7fb5027fc640 1 -- 172.21.15.156:0/1245712789 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 38) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fb4fc030020 con 0x7fb50c0fc330 2024-07-16T10:01:34.997 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:34.992+0000 7fb5027fc640 1 --2- 172.21.15.156:0/1245712789 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7fb4f806e0b0 0x7fb4f8070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:01:34.998 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:34.992+0000 7fb5027fc640 1 -- 172.21.15.156:0/1245712789 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fb4fc0ada00 con 0x7fb50c0fc330 2024-07-16T10:01:34.998 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:34.992+0000 7fb511107640 1 --2- 172.21.15.156:0/1245712789 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7fb4f806e0b0 0x7fb4f8070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:01:34.998 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:34.993+0000 7fb511107640 1 --2- 172.21.15.156:0/1245712789 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7fb4f806e0b0 0x7fb4f8070560 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fb50c10ccf0 tx=0x7fb50800b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T10:01:34.998 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:34.996+0000 7fb5027fc640 1 -- 172.21.15.156:0/1245712789 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187716 (secure 0 0 0) 0x7fb4fc076e30 con 0x7fb50c0fc330 2024-07-16T10:01:35.183 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:35.182+0000 7fb513b6c640 1 -- 172.21.15.156:0/1245712789 --> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fb50c10edb0 con 0x7fb4f806e0b0 2024-07-16T10:01:35.192 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:35.191+0000 7fb5027fc640 1 -- 172.21.15.156:0/1245712789 <== mgr.34104 v2:172.21.15.156:6800/3367765292 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7fb50c10edb0 con 0x7fb4f806e0b0 2024-07-16T10:01:35.193 INFO:teuthology.orchestra.run.smithi156.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-07-16T10:01:35.193 INFO:teuthology.orchestra.run.smithi156.stdout:alertmanager.smithi156 smithi156 *:9093,9094 running (5m) 81s ago 13m 25.0M - 0.25.0 c8568f914cd2 d524a2c3ff15 2024-07-16T10:01:35.193 INFO:teuthology.orchestra.run.smithi156.stdout:crash.smithi156 smithi156 running (2m) 81s ago 13m 6899k - 19.0.0-3495-ga6ca461b 16f7feb04342 11648cf29af4 2024-07-16T10:01:35.193 INFO:teuthology.orchestra.run.smithi156.stdout:crash.smithi176 smithi176 running (2m) 17s ago 13m 6899k - 19.0.0-3495-ga6ca461b 16f7feb04342 67f3ccc034f8 2024-07-16T10:01:35.193 INFO:teuthology.orchestra.run.smithi156.stdout:grafana.smithi156 smithi156 *:3000 running (5m) 81s ago 13m 77.8M - 9.4.7 954c08fa6188 deb9e1bfeb66 2024-07-16T10:01:35.193 INFO:teuthology.orchestra.run.smithi156.stdout:mds.cephfs.smithi156.evwwaz smithi156 running (8m) 81s ago 8m 67.2M - 17.2.7-1104-ge6231b04 d5de5f953b95 7c0ff9ae43f8 2024-07-16T10:01:35.193 INFO:teuthology.orchestra.run.smithi156.stdout:mds.cephfs.smithi156.rdibys smithi156 running (8m) 81s ago 8m 120M - 17.2.7-1104-ge6231b04 d5de5f953b95 b985cd1e6924 2024-07-16T10:01:35.193 INFO:teuthology.orchestra.run.smithi156.stdout:mds.cephfs.smithi176.dvvbud smithi176 running (8m) 17s ago 8m 108M - 17.2.7-1104-ge6231b04 d5de5f953b95 1b00147950f9 2024-07-16T10:01:35.193 INFO:teuthology.orchestra.run.smithi156.stdout:mds.cephfs.smithi176.wahbsi smithi176 running (8m) 17s ago 8m 62.2M - 17.2.7-1104-ge6231b04 d5de5f953b95 866664af2311 2024-07-16T10:01:35.193 INFO:teuthology.orchestra.run.smithi156.stdout:mgr.smithi156.zkvylv smithi156 *:8443,9283,8765 running (4m) 81s ago 15m 497M - 19.0.0-3495-ga6ca461b 16f7feb04342 ee53bd604b7c 2024-07-16T10:01:35.193 INFO:teuthology.orchestra.run.smithi156.stdout:mgr.smithi176.dqbrvx smithi176 *:8443,9283,8765 running (4m) 17s ago 12m 440M - 19.0.0-3495-ga6ca461b 16f7feb04342 ce9eaae4c2d0 2024-07-16T10:01:35.193 INFO:teuthology.orchestra.run.smithi156.stdout:mon.smithi156 smithi156 running (3m) 81s ago 15m 45.4M 2048M 19.0.0-3495-ga6ca461b 16f7feb04342 77b6ec8c50eb 2024-07-16T10:01:35.194 INFO:teuthology.orchestra.run.smithi156.stdout:mon.smithi176 smithi176 running (3m) 17s ago 12m 39.4M 2048M 19.0.0-3495-ga6ca461b 16f7feb04342 ef92814cec2b 2024-07-16T10:01:35.194 INFO:teuthology.orchestra.run.smithi156.stdout:node-exporter.smithi156 smithi156 *:9100 running (5m) 81s ago 13m 17.2M - 1.5.0 0da6a335fe13 e5d86f23a14c 2024-07-16T10:01:35.194 INFO:teuthology.orchestra.run.smithi156.stdout:node-exporter.smithi176 smithi176 *:9100 running (4m) 17s ago 12m 15.2M - 1.5.0 0da6a335fe13 ea842728144a 2024-07-16T10:01:35.194 INFO:teuthology.orchestra.run.smithi156.stdout:osd.0 smithi156 running (2m) 81s ago 12m 93.7M 1557M 19.0.0-3495-ga6ca461b 16f7feb04342 8633df1689cc 2024-07-16T10:01:35.194 INFO:teuthology.orchestra.run.smithi156.stdout:osd.1 smithi156 running (2m) 81s ago 11m 101M 1557M 19.0.0-3495-ga6ca461b 16f7feb04342 e6fae9653b46 2024-07-16T10:01:35.194 INFO:teuthology.orchestra.run.smithi156.stdout:osd.2 smithi156 running (86s) 81s ago 11m 11.8M 1557M 19.0.0-3495-ga6ca461b 16f7feb04342 c6ecf18de9bf 2024-07-16T10:01:35.194 INFO:teuthology.orchestra.run.smithi156.stdout:osd.3 smithi176 running (52s) 17s ago 10m 98.4M 2581M 19.0.0-3495-ga6ca461b 16f7feb04342 d216534d6027 2024-07-16T10:01:35.194 INFO:teuthology.orchestra.run.smithi156.stdout:osd.4 smithi176 running (20s) 17s ago 10m 13.9M 2581M 19.0.0-3495-ga6ca461b 16f7feb04342 1bd3a25317b0 2024-07-16T10:01:35.194 INFO:teuthology.orchestra.run.smithi156.stdout:osd.5 smithi176 running (10m) 17s ago 10m 225M 2581M 17.2.7-1104-ge6231b04 d5de5f953b95 996e15448a52 2024-07-16T10:01:35.194 INFO:teuthology.orchestra.run.smithi156.stdout:prometheus.smithi156 smithi156 *:9095 running (4m) 81s ago 13m 64.9M - 2.43.0 a07b618ecd1d 20c195ecd665 2024-07-16T10:01:35.195 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:35.193+0000 7fb513b6c640 1 -- 172.21.15.156:0/1245712789 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7fb4f806e0b0 msgr2=0x7fb4f8070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:01:35.195 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:35.193+0000 7fb513b6c640 1 --2- 172.21.15.156:0/1245712789 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7fb4f806e0b0 0x7fb4f8070560 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fb50c10ccf0 tx=0x7fb50800b040 comp rx=0 tx=0).stop 2024-07-16T10:01:35.195 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:35.193+0000 7fb513b6c640 1 -- 172.21.15.156:0/1245712789 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb50c0fc330 msgr2=0x7fb50c10cf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:01:35.195 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:35.193+0000 7fb513b6c640 1 --2- 172.21.15.156:0/1245712789 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb50c0fc330 0x7fb50c10cf40 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fb4fc009f80 tx=0x7fb4fc009da0 comp rx=0 tx=0).stop 2024-07-16T10:01:35.196 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:35.193+0000 7fb513b6c640 1 -- 172.21.15.156:0/1245712789 shutdown_connections 2024-07-16T10:01:35.196 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:35.193+0000 7fb513b6c640 1 --2- 172.21.15.156:0/1245712789 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7fb4f806e0b0 0x7fb4f8070560 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:01:35.196 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:35.193+0000 7fb513b6c640 1 --2- 172.21.15.156:0/1245712789 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb50c0fd600 0x7fb50c10d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:01:35.196 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:35.193+0000 7fb513b6c640 1 --2- 172.21.15.156:0/1245712789 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb50c0fc330 0x7fb50c10cf40 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:01:35.196 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:35.193+0000 7fb513b6c640 1 -- 172.21.15.156:0/1245712789 >> 172.21.15.156:0/1245712789 conn(0x7fb50c0f7af0 msgr2=0x7fb50c0f91d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T10:01:35.196 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:35.193+0000 7fb513b6c640 1 -- 172.21.15.156:0/1245712789 shutdown_connections 2024-07-16T10:01:35.196 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:35.193+0000 7fb513b6c640 1 -- 172.21.15.156:0/1245712789 wait complete. 2024-07-16T10:01:35.275 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:35.272+0000 7f4b07190640 1 -- 172.21.15.156:0/4108925212 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4b000fc380 msgr2=0x7f4b000fc790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:01:35.275 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:35.272+0000 7f4b07190640 1 --2- 172.21.15.156:0/4108925212 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4b000fc380 0x7f4b000fc790 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f4af40048d0 tx=0x7f4af4009da0 comp rx=0 tx=0).stop 2024-07-16T10:01:35.275 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:35.272+0000 7f4b07190640 1 -- 172.21.15.156:0/4108925212 shutdown_connections 2024-07-16T10:01:35.276 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:35.272+0000 7f4b07190640 1 --2- 172.21.15.156:0/4108925212 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4b000fd650 0x7f4b000fdac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:01:35.276 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:35.272+0000 7f4b07190640 1 --2- 172.21.15.156:0/4108925212 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4b000fc380 0x7f4b000fc790 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:01:35.276 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:35.272+0000 7f4b07190640 1 -- 172.21.15.156:0/4108925212 >> 172.21.15.156:0/4108925212 conn(0x7f4b000f7b40 msgr2=0x7f4b000f9f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T10:01:35.276 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:35.272+0000 7f4b07190640 1 -- 172.21.15.156:0/4108925212 shutdown_connections 2024-07-16T10:01:35.276 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:35.272+0000 7f4b07190640 1 -- 172.21.15.156:0/4108925212 wait complete. 2024-07-16T10:01:35.276 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:35.272+0000 7f4b07190640 1 Processor -- start 2024-07-16T10:01:35.276 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:35.272+0000 7f4b07190640 1 -- start start 2024-07-16T10:01:35.276 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:35.273+0000 7f4b07190640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4b000fc380 0x7f4b0010cf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:01:35.276 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:35.273+0000 7f4b07190640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4b000fd650 0x7f4b0010d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:01:35.276 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:35.273+0000 7f4b07190640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b0010db00 con 0x7f4b000fc380 2024-07-16T10:01:35.276 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:35.273+0000 7f4b07190640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b0010e320 con 0x7f4b000fd650 2024-07-16T10:01:35.277 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:35.273+0000 7f4b04f2c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4b000fc380 0x7f4b0010cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:01:35.277 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:35.273+0000 7f4b04f2c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4b000fc380 0x7f4b0010cf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:60046/0 (socket says 172.21.15.156:60046) 2024-07-16T10:01:35.277 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:35.273+0000 7f4b04f2c640 1 -- 172.21.15.156:0/761230073 learned_addr learned my addr 172.21.15.156:0/761230073 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T10:01:35.277 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:35.273+0000 7f4afffff640 1 --2- 172.21.15.156:0/761230073 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4b000fd650 0x7f4b0010d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:01:35.277 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:35.273+0000 7f4b04f2c640 1 -- 172.21.15.156:0/761230073 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4b000fd650 msgr2=0x7f4b0010d4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:01:35.277 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:35.273+0000 7f4b04f2c640 1 --2- 172.21.15.156:0/761230073 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4b000fd650 0x7f4b0010d4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:01:35.277 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:35.273+0000 7f4b04f2c640 1 -- 172.21.15.156:0/761230073 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4af4005040 con 0x7f4b000fc380 2024-07-16T10:01:35.277 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:35.273+0000 7f4afffff640 1 --2- 172.21.15.156:0/761230073 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4b000fd650 0x7f4b0010d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-16T10:01:35.277 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:35.274+0000 7f4b04f2c640 1 --2- 172.21.15.156:0/761230073 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4b000fc380 0x7f4b0010cf70 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f4af40041d0 tx=0x7f4af4009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T10:01:35.277 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:35.274+0000 7f4afd7fa640 1 -- 172.21.15.156:0/761230073 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4af4038070 con 0x7f4b000fc380 2024-07-16T10:01:35.277 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:35.274+0000 7f4b07190640 1 -- 172.21.15.156:0/761230073 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b0010e550 con 0x7f4b000fc380 2024-07-16T10:01:35.277 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:35.274+0000 7f4afd7fa640 1 -- 172.21.15.156:0/761230073 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4af40092b0 con 0x7f4b000fc380 2024-07-16T10:01:35.278 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:35.274+0000 7f4afd7fa640 1 -- 172.21.15.156:0/761230073 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4af4033710 con 0x7f4b000fc380 2024-07-16T10:01:35.278 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:35.274+0000 7f4b07190640 1 -- 172.21.15.156:0/761230073 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b0010ea90 con 0x7f4b000fc380 2024-07-16T10:01:35.278 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:35.275+0000 7f4b07190640 1 -- 172.21.15.156:0/761230073 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b000fb7e0 con 0x7f4b000fc380 2024-07-16T10:01:35.279 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:35.275+0000 7f4afd7fa640 1 -- 172.21.15.156:0/761230073 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 38) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f4af4030020 con 0x7f4b000fc380 2024-07-16T10:01:35.279 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:35.276+0000 7f4afd7fa640 1 --2- 172.21.15.156:0/761230073 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f4af006e0b0 0x7f4af0070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:01:35.279 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:35.276+0000 7f4afd7fa640 1 -- 172.21.15.156:0/761230073 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f4af40ad900 con 0x7f4b000fc380 2024-07-16T10:01:35.279 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:35.276+0000 7f4afffff640 1 --2- 172.21.15.156:0/761230073 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f4af006e0b0 0x7f4af0070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:01:35.279 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:35.276+0000 7f4afffff640 1 --2- 172.21.15.156:0/761230073 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f4af006e0b0 0x7f4af0070560 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f4aec004fd0 tx=0x7f4aec00b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T10:01:35.282 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:35.280+0000 7f4afd7fa640 1 -- 172.21.15.156:0/761230073 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187716 (secure 0 0 0) 0x7f4af4076d30 con 0x7f4b000fc380 2024-07-16T10:01:35.526 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:35.525+0000 7f4b07190640 1 -- 172.21.15.156:0/761230073 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f4b0010ed50 con 0x7f4b000fc380 2024-07-16T10:01:35.527 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:35.526+0000 7f4afd7fa640 1 -- 172.21.15.156:0/761230073 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+838 (secure 0 0 0) 0x7f4af407a3f0 con 0x7f4b000fc380 2024-07-16T10:01:35.527 INFO:teuthology.orchestra.run.smithi156.stdout:{ 2024-07-16T10:01:35.527 INFO:teuthology.orchestra.run.smithi156.stdout: "mon": { 2024-07-16T10:01:35.527 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph version 19.0.0-3495-ga6ca461b (a6ca461be7cc1aff3658c2c6df4cff975a24449a) squid (rc)": 2 2024-07-16T10:01:35.527 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-07-16T10:01:35.528 INFO:teuthology.orchestra.run.smithi156.stdout: "mgr": { 2024-07-16T10:01:35.528 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph version 19.0.0-3495-ga6ca461b (a6ca461be7cc1aff3658c2c6df4cff975a24449a) squid (rc)": 2 2024-07-16T10:01:35.528 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-07-16T10:01:35.528 INFO:teuthology.orchestra.run.smithi156.stdout: "osd": { 2024-07-16T10:01:35.528 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph version 17.2.7-1104-ge6231b04 (e6231b04b52ebf3b9b31803282dd3074439e51ac) quincy (stable)": 1, 2024-07-16T10:01:35.528 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph version 19.0.0-3495-ga6ca461b (a6ca461be7cc1aff3658c2c6df4cff975a24449a) squid (rc)": 5 2024-07-16T10:01:35.528 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-07-16T10:01:35.528 INFO:teuthology.orchestra.run.smithi156.stdout: "mds": { 2024-07-16T10:01:35.528 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph version 17.2.7-1104-ge6231b04 (e6231b04b52ebf3b9b31803282dd3074439e51ac) quincy (stable)": 4 2024-07-16T10:01:35.528 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-07-16T10:01:35.528 INFO:teuthology.orchestra.run.smithi156.stdout: "overall": { 2024-07-16T10:01:35.528 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph version 17.2.7-1104-ge6231b04 (e6231b04b52ebf3b9b31803282dd3074439e51ac) quincy (stable)": 5, 2024-07-16T10:01:35.528 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph version 19.0.0-3495-ga6ca461b (a6ca461be7cc1aff3658c2c6df4cff975a24449a) squid (rc)": 9 2024-07-16T10:01:35.528 INFO:teuthology.orchestra.run.smithi156.stdout: } 2024-07-16T10:01:35.529 INFO:teuthology.orchestra.run.smithi156.stdout:} 2024-07-16T10:01:35.529 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:35.528+0000 7f4b07190640 1 -- 172.21.15.156:0/761230073 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f4af006e0b0 msgr2=0x7f4af0070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:01:35.529 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:35.528+0000 7f4b07190640 1 --2- 172.21.15.156:0/761230073 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f4af006e0b0 0x7f4af0070560 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f4aec004fd0 tx=0x7f4aec00b040 comp rx=0 tx=0).stop 2024-07-16T10:01:35.529 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:35.528+0000 7f4b07190640 1 -- 172.21.15.156:0/761230073 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4b000fc380 msgr2=0x7f4b0010cf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:01:35.529 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:35.528+0000 7f4b07190640 1 --2- 172.21.15.156:0/761230073 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4b000fc380 0x7f4b0010cf70 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f4af40041d0 tx=0x7f4af4009da0 comp rx=0 tx=0).stop 2024-07-16T10:01:35.529 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:35.528+0000 7f4b07190640 1 -- 172.21.15.156:0/761230073 shutdown_connections 2024-07-16T10:01:35.530 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:35.528+0000 7f4b07190640 1 --2- 172.21.15.156:0/761230073 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f4af006e0b0 0x7f4af0070560 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:01:35.530 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:35.528+0000 7f4b07190640 1 --2- 172.21.15.156:0/761230073 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4b000fd650 0x7f4b0010d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:01:35.530 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:35.528+0000 7f4b07190640 1 --2- 172.21.15.156:0/761230073 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4b000fc380 0x7f4b0010cf70 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:01:35.530 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:35.528+0000 7f4b07190640 1 -- 172.21.15.156:0/761230073 >> 172.21.15.156:0/761230073 conn(0x7f4b000f7b40 msgr2=0x7f4b000f91e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T10:01:35.530 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:35.528+0000 7f4b07190640 1 -- 172.21.15.156:0/761230073 shutdown_connections 2024-07-16T10:01:35.530 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:35.528+0000 7f4b07190640 1 -- 172.21.15.156:0/761230073 wait complete. 2024-07-16T10:01:35.610 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:35.608+0000 7f0f96a1f640 1 -- 172.21.15.156:0/110720507 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0f900fb240 msgr2=0x7f0f900fb650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:01:35.610 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:35.608+0000 7f0f96a1f640 1 --2- 172.21.15.156:0/110720507 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0f900fb240 0x7f0f900fb650 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f0f800048d0 tx=0x7f0f80009da0 comp rx=0 tx=0).stop 2024-07-16T10:01:35.610 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:35.608+0000 7f0f96a1f640 1 -- 172.21.15.156:0/110720507 shutdown_connections 2024-07-16T10:01:35.610 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:35.608+0000 7f0f96a1f640 1 --2- 172.21.15.156:0/110720507 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0f900fc510 0x7f0f900fc980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:01:35.610 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:35.608+0000 7f0f96a1f640 1 --2- 172.21.15.156:0/110720507 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0f900fb240 0x7f0f900fb650 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:01:35.610 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:35.609+0000 7f0f96a1f640 1 -- 172.21.15.156:0/110720507 >> 172.21.15.156:0/110720507 conn(0x7f0f900f6a20 msgr2=0x7f0f900f8e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T10:01:35.610 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:35.609+0000 7f0f96a1f640 1 -- 172.21.15.156:0/110720507 shutdown_connections 2024-07-16T10:01:35.611 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:35.609+0000 7f0f96a1f640 1 -- 172.21.15.156:0/110720507 wait complete. 2024-07-16T10:01:35.611 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:35.609+0000 7f0f96a1f640 1 Processor -- start 2024-07-16T10:01:35.612 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:35.609+0000 7f0f96a1f640 1 -- start start 2024-07-16T10:01:35.612 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:35.609+0000 7f0f96a1f640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0f900fb240 0x7f0f900ff360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:01:35.612 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:35.609+0000 7f0f96a1f640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0f900fc510 0x7f0f900fd850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:01:35.612 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:35.609+0000 7f0f96a1f640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f900fde70 con 0x7f0f900fb240 2024-07-16T10:01:35.612 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:35.609+0000 7f0f96a1f640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f900fdfb0 con 0x7f0f900fc510 2024-07-16T10:01:35.612 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:35.609+0000 7f0f8ffff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0f900fb240 0x7f0f900ff360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:01:35.613 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:35.610+0000 7f0f8ffff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0f900fb240 0x7f0f900ff360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:60060/0 (socket says 172.21.15.156:60060) 2024-07-16T10:01:35.613 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:35.610+0000 7f0f8ffff640 1 -- 172.21.15.156:0/316342209 learned_addr learned my addr 172.21.15.156:0/316342209 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T10:01:35.613 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:35.610+0000 7f0f8f7fe640 1 --2- 172.21.15.156:0/316342209 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0f900fc510 0x7f0f900fd850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:01:35.613 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:35.610+0000 7f0f8ffff640 1 -- 172.21.15.156:0/316342209 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0f900fc510 msgr2=0x7f0f900fd850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:01:35.613 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:35.610+0000 7f0f8ffff640 1 --2- 172.21.15.156:0/316342209 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0f900fc510 0x7f0f900fd850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:01:35.613 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:35.610+0000 7f0f8ffff640 1 -- 172.21.15.156:0/316342209 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f80005040 con 0x7f0f900fb240 2024-07-16T10:01:35.613 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:35.610+0000 7f0f8ffff640 1 --2- 172.21.15.156:0/316342209 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0f900fb240 0x7f0f900ff360 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f0f80003080 tx=0x7f0f80009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T10:01:35.613 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:35.610+0000 7f0f8cff9640 1 -- 172.21.15.156:0/316342209 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0f80038070 con 0x7f0f900fb240 2024-07-16T10:01:35.613 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:35.610+0000 7f0f8cff9640 1 -- 172.21.15.156:0/316342209 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0f80032970 con 0x7f0f900fb240 2024-07-16T10:01:35.613 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:35.610+0000 7f0f8cff9640 1 -- 172.21.15.156:0/316342209 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0f80033900 con 0x7f0f900fb240 2024-07-16T10:01:35.613 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:35.610+0000 7f0f96a1f640 1 -- 172.21.15.156:0/316342209 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f900fe230 con 0x7f0f900fb240 2024-07-16T10:01:35.613 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:35.610+0000 7f0f96a1f640 1 -- 172.21.15.156:0/316342209 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f90115050 con 0x7f0f900fb240 2024-07-16T10:01:35.614 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:35.611+0000 7f0f96a1f640 1 -- 172.21.15.156:0/316342209 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f900fa6a0 con 0x7f0f900fb240 2024-07-16T10:01:35.614 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:35.612+0000 7f0f8cff9640 1 -- 172.21.15.156:0/316342209 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 38) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f0f80030020 con 0x7f0f900fb240 2024-07-16T10:01:35.614 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:35.612+0000 7f0f8cff9640 1 --2- 172.21.15.156:0/316342209 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f0f7806e0b0 0x7f0f78070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:01:35.614 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:35.612+0000 7f0f8cff9640 1 -- 172.21.15.156:0/316342209 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f0f800adc40 con 0x7f0f900fb240 2024-07-16T10:01:35.615 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:35.613+0000 7f0f8f7fe640 1 --2- 172.21.15.156:0/316342209 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f0f7806e0b0 0x7f0f78070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:01:35.615 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:35.613+0000 7f0f8f7fe640 1 --2- 172.21.15.156:0/316342209 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f0f7806e0b0 0x7f0f78070560 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f0f84006fd0 tx=0x7f0f84008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T10:01:35.618 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:35.617+0000 7f0f8cff9640 1 -- 172.21.15.156:0/316342209 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187716 (secure 0 0 0) 0x7f0f80076fc0 con 0x7f0f900fb240 2024-07-16T10:01:35.833 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:35.831+0000 7f0f96a1f640 1 -- 172.21.15.156:0/316342209 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f0f900008d0 con 0x7f0f900fb240 2024-07-16T10:01:35.833 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:35.832+0000 7f0f8cff9640 1 -- 172.21.15.156:0/316342209 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 20 v20) v1 ==== 76+0+1920 (secure 0 0 0) 0x7f0f8007a680 con 0x7f0f900fb240 2024-07-16T10:01:35.833 INFO:teuthology.orchestra.run.smithi156.stdout:e20 2024-07-16T10:01:35.833 INFO:teuthology.orchestra.run.smithi156.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-07-16T10:01:35.833 INFO:teuthology.orchestra.run.smithi156.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-07-16T10:01:35.833 INFO:teuthology.orchestra.run.smithi156.stdout:legacy client fscid: 1 2024-07-16T10:01:35.833 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-07-16T10:01:35.834 INFO:teuthology.orchestra.run.smithi156.stdout:Filesystem 'cephfs' (1) 2024-07-16T10:01:35.834 INFO:teuthology.orchestra.run.smithi156.stdout:fs_name cephfs 2024-07-16T10:01:35.834 INFO:teuthology.orchestra.run.smithi156.stdout:epoch 20 2024-07-16T10:01:35.834 INFO:teuthology.orchestra.run.smithi156.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-07-16T10:01:35.834 INFO:teuthology.orchestra.run.smithi156.stdout:created 2024-07-16T09:52:55.214975+0000 2024-07-16T10:01:35.834 INFO:teuthology.orchestra.run.smithi156.stdout:modified 2024-07-16T09:58:56.574272+0000 2024-07-16T10:01:35.834 INFO:teuthology.orchestra.run.smithi156.stdout:tableserver 0 2024-07-16T10:01:35.834 INFO:teuthology.orchestra.run.smithi156.stdout:root 0 2024-07-16T10:01:35.834 INFO:teuthology.orchestra.run.smithi156.stdout:session_timeout 60 2024-07-16T10:01:35.834 INFO:teuthology.orchestra.run.smithi156.stdout:session_autoclose 300 2024-07-16T10:01:35.834 INFO:teuthology.orchestra.run.smithi156.stdout:max_file_size 1099511627776 2024-07-16T10:01:35.834 INFO:teuthology.orchestra.run.smithi156.stdout:max_xattr_size 65536 2024-07-16T10:01:35.835 INFO:teuthology.orchestra.run.smithi156.stdout:required_client_features {} 2024-07-16T10:01:35.835 INFO:teuthology.orchestra.run.smithi156.stdout:last_failure 0 2024-07-16T10:01:35.835 INFO:teuthology.orchestra.run.smithi156.stdout:last_failure_osd_epoch 0 2024-07-16T10:01:35.835 INFO:teuthology.orchestra.run.smithi156.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-07-16T10:01:35.835 INFO:teuthology.orchestra.run.smithi156.stdout:max_mds 2 2024-07-16T10:01:35.835 INFO:teuthology.orchestra.run.smithi156.stdout:in 0,1 2024-07-16T10:01:35.835 INFO:teuthology.orchestra.run.smithi156.stdout:up {0=14458,1=24261} 2024-07-16T10:01:35.835 INFO:teuthology.orchestra.run.smithi156.stdout:failed 2024-07-16T10:01:35.835 INFO:teuthology.orchestra.run.smithi156.stdout:damaged 2024-07-16T10:01:35.835 INFO:teuthology.orchestra.run.smithi156.stdout:stopped 2024-07-16T10:01:35.835 INFO:teuthology.orchestra.run.smithi156.stdout:data_pools [3] 2024-07-16T10:01:35.835 INFO:teuthology.orchestra.run.smithi156.stdout:metadata_pool 2 2024-07-16T10:01:35.835 INFO:teuthology.orchestra.run.smithi156.stdout:inline_data disabled 2024-07-16T10:01:35.835 INFO:teuthology.orchestra.run.smithi156.stdout:balancer 2024-07-16T10:01:35.836 INFO:teuthology.orchestra.run.smithi156.stdout:bal_rank_mask -1 2024-07-16T10:01:35.836 INFO:teuthology.orchestra.run.smithi156.stdout:standby_count_wanted 1 2024-07-16T10:01:35.836 INFO:teuthology.orchestra.run.smithi156.stdout:qdb_cluster leader: 14458 members: 24261,14458 2024-07-16T10:01:35.836 INFO:teuthology.orchestra.run.smithi156.stdout:[mds.cephfs.smithi156.rdibys{0:14458} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.156:6826/1080176013,v1:172.21.15.156:6827/1080176013] compat {c=[1],r=[1],i=[7ff]}] 2024-07-16T10:01:35.836 INFO:teuthology.orchestra.run.smithi156.stdout:[mds.cephfs.smithi156.evwwaz{0:14468} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.156:6828/560329502,v1:172.21.15.156:6829/560329502] compat {c=[1],r=[1],i=[7ff]}] 2024-07-16T10:01:35.836 INFO:teuthology.orchestra.run.smithi156.stdout:[mds.cephfs.smithi176.dvvbud{1:24261} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.176:6824/1577019287,v1:172.21.15.176:6825/1577019287] compat {c=[1],r=[1],i=[7ff]}] 2024-07-16T10:01:35.836 INFO:teuthology.orchestra.run.smithi156.stdout:[mds.cephfs.smithi176.wahbsi{1:24273} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.176:6826/2196775412,v1:172.21.15.176:6827/2196775412] compat {c=[1],r=[1],i=[7ff]}] 2024-07-16T10:01:35.836 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-07-16T10:01:35.836 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-07-16T10:01:35.837 INFO:teuthology.orchestra.run.smithi156.stderr:dumped fsmap epoch 20 2024-07-16T10:01:35.837 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:35.834+0000 7f0f96a1f640 1 -- 172.21.15.156:0/316342209 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f0f7806e0b0 msgr2=0x7f0f78070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:01:35.837 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:35.834+0000 7f0f96a1f640 1 --2- 172.21.15.156:0/316342209 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f0f7806e0b0 0x7f0f78070560 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f0f84006fd0 tx=0x7f0f84008040 comp rx=0 tx=0).stop 2024-07-16T10:01:35.837 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:35.834+0000 7f0f96a1f640 1 -- 172.21.15.156:0/316342209 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0f900fb240 msgr2=0x7f0f900ff360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:01:35.837 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:35.834+0000 7f0f96a1f640 1 --2- 172.21.15.156:0/316342209 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0f900fb240 0x7f0f900ff360 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f0f80003080 tx=0x7f0f80009da0 comp rx=0 tx=0).stop 2024-07-16T10:01:35.837 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:35.834+0000 7f0f96a1f640 1 -- 172.21.15.156:0/316342209 shutdown_connections 2024-07-16T10:01:35.837 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:35.834+0000 7f0f96a1f640 1 --2- 172.21.15.156:0/316342209 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f0f7806e0b0 0x7f0f78070560 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:01:35.837 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:35.834+0000 7f0f96a1f640 1 --2- 172.21.15.156:0/316342209 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0f900fc510 0x7f0f900fd850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:01:35.837 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:35.834+0000 7f0f96a1f640 1 --2- 172.21.15.156:0/316342209 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0f900fb240 0x7f0f900ff360 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:01:35.838 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:35.834+0000 7f0f96a1f640 1 -- 172.21.15.156:0/316342209 >> 172.21.15.156:0/316342209 conn(0x7f0f900f6a20 msgr2=0x7f0f90065400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T10:01:35.838 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:35.834+0000 7f0f96a1f640 1 -- 172.21.15.156:0/316342209 shutdown_connections 2024-07-16T10:01:35.838 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:35.834+0000 7f0f96a1f640 1 -- 172.21.15.156:0/316342209 wait complete. 2024-07-16T10:01:35.916 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:35.915+0000 7fedcb7f7640 1 -- 172.21.15.156:0/1009957368 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fedc40fb260 msgr2=0x7fedc40fb670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:01:35.916 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:35.915+0000 7fedcb7f7640 1 --2- 172.21.15.156:0/1009957368 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fedc40fb260 0x7fedc40fb670 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fedbc0048d0 tx=0x7fedbc009da0 comp rx=0 tx=0).stop 2024-07-16T10:01:35.916 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:35.915+0000 7fedcb7f7640 1 -- 172.21.15.156:0/1009957368 shutdown_connections 2024-07-16T10:01:35.916 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:35.915+0000 7fedcb7f7640 1 --2- 172.21.15.156:0/1009957368 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fedc40fc530 0x7fedc40fc9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:01:35.916 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:35.915+0000 7fedcb7f7640 1 --2- 172.21.15.156:0/1009957368 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fedc40fb260 0x7fedc40fb670 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:01:35.917 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:35.915+0000 7fedcb7f7640 1 -- 172.21.15.156:0/1009957368 >> 172.21.15.156:0/1009957368 conn(0x7fedc40f6a20 msgr2=0x7fedc40f8e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T10:01:35.917 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:35.915+0000 7fedcb7f7640 1 -- 172.21.15.156:0/1009957368 shutdown_connections 2024-07-16T10:01:35.917 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:35.915+0000 7fedcb7f7640 1 -- 172.21.15.156:0/1009957368 wait complete. 2024-07-16T10:01:35.918 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:35.915+0000 7fedcb7f7640 1 Processor -- start 2024-07-16T10:01:35.918 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:35.915+0000 7fedcb7f7640 1 -- start start 2024-07-16T10:01:35.918 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:35.916+0000 7fedcb7f7640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fedc40fb260 0x7fedc410f220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:01:35.918 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:35.916+0000 7fedcb7f7640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fedc40fc530 0x7fedc410f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:01:35.918 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:35.916+0000 7fedcb7f7640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedc410fdb0 con 0x7fedc40fc530 2024-07-16T10:01:35.919 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:35.916+0000 7fedcb7f7640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedc4110580 con 0x7fedc40fb260 2024-07-16T10:01:35.919 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:35.916+0000 7fedc8d92640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fedc40fc530 0x7fedc410f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:01:35.919 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:35.916+0000 7fedc9593640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fedc40fb260 0x7fedc410f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:01:35.919 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:35.916+0000 7fedc8d92640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fedc40fc530 0x7fedc410f760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:60086/0 (socket says 172.21.15.156:60086) 2024-07-16T10:01:35.919 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:35.916+0000 7fedc8d92640 1 -- 172.21.15.156:0/3640591274 learned_addr learned my addr 172.21.15.156:0/3640591274 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T10:01:35.919 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:35.916+0000 7fedc9593640 1 -- 172.21.15.156:0/3640591274 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fedc40fc530 msgr2=0x7fedc410f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:01:35.919 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:35.916+0000 7fedc9593640 1 --2- 172.21.15.156:0/3640591274 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fedc40fc530 0x7fedc410f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:01:35.919 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:35.916+0000 7fedc9593640 1 -- 172.21.15.156:0/3640591274 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedbc005040 con 0x7fedc40fb260 2024-07-16T10:01:35.919 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:35.916+0000 7fedc8d92640 1 --2- 172.21.15.156:0/3640591274 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fedc40fc530 0x7fedc410f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-16T10:01:35.919 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:35.916+0000 7fedc9593640 1 --2- 172.21.15.156:0/3640591274 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fedc40fb260 0x7fedc410f220 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fedbc0048a0 tx=0x7fedbc009da0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T10:01:35.919 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:35.916+0000 7fedc1ffb640 1 -- 172.21.15.156:0/3640591274 <== mon.1 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fedbc038070 con 0x7fedc40fb260 2024-07-16T10:01:35.919 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:35.916+0000 7fedcb7f7640 1 -- 172.21.15.156:0/3640591274 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fedc41107b0 con 0x7fedc40fb260 2024-07-16T10:01:35.919 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:35.916+0000 7fedc1ffb640 1 -- 172.21.15.156:0/3640591274 <== mon.1 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fedbc0092b0 con 0x7fedc40fb260 2024-07-16T10:01:35.920 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:35.916+0000 7fedc1ffb640 1 -- 172.21.15.156:0/3640591274 <== mon.1 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fedbc033660 con 0x7fedc40fb260 2024-07-16T10:01:35.920 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:35.917+0000 7fedcb7f7640 1 -- 172.21.15.156:0/3640591274 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fedc4110cc0 con 0x7fedc40fb260 2024-07-16T10:01:35.920 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:35.917+0000 7fedcb7f7640 1 -- 172.21.15.156:0/3640591274 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fedc40fa6c0 con 0x7fedc40fb260 2024-07-16T10:01:35.920 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:35.918+0000 7fedc1ffb640 1 -- 172.21.15.156:0/3640591274 <== mon.1 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 38) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fedbc030020 con 0x7fedc40fb260 2024-07-16T10:01:35.921 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:35.918+0000 7fedc1ffb640 1 --2- 172.21.15.156:0/3640591274 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7fedac06e0b0 0x7fedac070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:01:35.921 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:35.919+0000 7fedc1ffb640 1 -- 172.21.15.156:0/3640591274 <== mon.1 v2:172.21.15.176:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fedbc0ad830 con 0x7fedc40fb260 2024-07-16T10:01:35.921 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:35.919+0000 7fedc8d92640 1 --2- 172.21.15.156:0/3640591274 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7fedac06e0b0 0x7fedac070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:01:35.921 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:35.919+0000 7fedc8d92640 1 --2- 172.21.15.156:0/3640591274 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7fedac06e0b0 0x7fedac070560 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fedc410efd0 tx=0x7fedb800b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T10:01:35.924 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:35.923+0000 7fedc1ffb640 1 -- 172.21.15.156:0/3640591274 <== mon.1 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187716 (secure 0 0 0) 0x7fedbc076c80 con 0x7fedc40fb260 2024-07-16T10:01:36.121 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:36.120+0000 7fedcb7f7640 1 -- 172.21.15.156:0/3640591274 --> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fedc4110f30 con 0x7fedac06e0b0 2024-07-16T10:01:36.122 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:36.121+0000 7fedc1ffb640 1 -- 172.21.15.156:0/3640591274 <== mgr.34104 v2:172.21.15.156:6800/3367765292 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7fedc4110f30 con 0x7fedac06e0b0 2024-07-16T10:01:36.122 INFO:teuthology.orchestra.run.smithi156.stdout:{ 2024-07-16T10:01:36.122 INFO:teuthology.orchestra.run.smithi156.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:c15ec3aa8e6c0a9c12ee24ff79f3e7dd4d01b3d1a702dfeb7d7fff9b0811a393", 2024-07-16T10:01:36.123 INFO:teuthology.orchestra.run.smithi156.stdout: "in_progress": true, 2024-07-16T10:01:36.123 INFO:teuthology.orchestra.run.smithi156.stdout: "which": "Upgrading all daemon types on all hosts", 2024-07-16T10:01:36.123 INFO:teuthology.orchestra.run.smithi156.stdout: "services_complete": [ 2024-07-16T10:01:36.123 INFO:teuthology.orchestra.run.smithi156.stdout: "mon", 2024-07-16T10:01:36.123 INFO:teuthology.orchestra.run.smithi156.stdout: "mgr", 2024-07-16T10:01:36.123 INFO:teuthology.orchestra.run.smithi156.stdout: "crash" 2024-07-16T10:01:36.123 INFO:teuthology.orchestra.run.smithi156.stdout: ], 2024-07-16T10:01:36.123 INFO:teuthology.orchestra.run.smithi156.stdout: "progress": "11/21 daemons upgraded", 2024-07-16T10:01:36.123 INFO:teuthology.orchestra.run.smithi156.stdout: "message": "Currently upgrading osd daemons", 2024-07-16T10:01:36.123 INFO:teuthology.orchestra.run.smithi156.stdout: "is_paused": false 2024-07-16T10:01:36.123 INFO:teuthology.orchestra.run.smithi156.stdout:} 2024-07-16T10:01:36.125 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:36.124+0000 7fedcb7f7640 1 -- 172.21.15.156:0/3640591274 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7fedac06e0b0 msgr2=0x7fedac070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:01:36.126 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:36.124+0000 7fedcb7f7640 1 --2- 172.21.15.156:0/3640591274 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7fedac06e0b0 0x7fedac070560 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fedc410efd0 tx=0x7fedb800b040 comp rx=0 tx=0).stop 2024-07-16T10:01:36.126 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:36.124+0000 7fedcb7f7640 1 -- 172.21.15.156:0/3640591274 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fedc40fb260 msgr2=0x7fedc410f220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:01:36.126 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:36.124+0000 7fedcb7f7640 1 --2- 172.21.15.156:0/3640591274 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fedc40fb260 0x7fedc410f220 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fedbc0048a0 tx=0x7fedbc009da0 comp rx=0 tx=0).stop 2024-07-16T10:01:36.126 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:36.124+0000 7fedcb7f7640 1 -- 172.21.15.156:0/3640591274 shutdown_connections 2024-07-16T10:01:36.126 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:36.124+0000 7fedcb7f7640 1 --2- 172.21.15.156:0/3640591274 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7fedac06e0b0 0x7fedac070560 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:01:36.126 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:36.124+0000 7fedcb7f7640 1 --2- 172.21.15.156:0/3640591274 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fedc40fc530 0x7fedc410f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:01:36.126 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:36.124+0000 7fedcb7f7640 1 --2- 172.21.15.156:0/3640591274 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fedc40fb260 0x7fedc410f220 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:01:36.126 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:36.124+0000 7fedcb7f7640 1 -- 172.21.15.156:0/3640591274 >> 172.21.15.156:0/3640591274 conn(0x7fedc40f6a20 msgr2=0x7fedc4065ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T10:01:36.126 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:36.124+0000 7fedcb7f7640 1 -- 172.21.15.156:0/3640591274 shutdown_connections 2024-07-16T10:01:36.126 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:36.124+0000 7fedcb7f7640 1 -- 172.21.15.156:0/3640591274 wait complete. 2024-07-16T10:01:36.206 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:36.203+0000 7f54873db640 1 -- 172.21.15.156:0/1492948723 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f54800fb240 msgr2=0x7f54800fb650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:01:36.206 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:36.203+0000 7f54873db640 1 --2- 172.21.15.156:0/1492948723 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f54800fb240 0x7f54800fb650 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f54740048d0 tx=0x7f5474009da0 comp rx=0 tx=0).stop 2024-07-16T10:01:36.206 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:36.204+0000 7f54873db640 1 -- 172.21.15.156:0/1492948723 shutdown_connections 2024-07-16T10:01:36.206 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:36.204+0000 7f54873db640 1 --2- 172.21.15.156:0/1492948723 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f54800fc510 0x7f54800fc980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:01:36.206 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:36.204+0000 7f54873db640 1 --2- 172.21.15.156:0/1492948723 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f54800fb240 0x7f54800fb650 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:01:36.206 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:36.204+0000 7f54873db640 1 -- 172.21.15.156:0/1492948723 >> 172.21.15.156:0/1492948723 conn(0x7f54800f6a20 msgr2=0x7f54800f8e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T10:01:36.207 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:36.204+0000 7f54873db640 1 -- 172.21.15.156:0/1492948723 shutdown_connections 2024-07-16T10:01:36.207 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:36.204+0000 7f54873db640 1 -- 172.21.15.156:0/1492948723 wait complete. 2024-07-16T10:01:36.207 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:36.204+0000 7f54873db640 1 Processor -- start 2024-07-16T10:01:36.207 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:36.204+0000 7f54873db640 1 -- start start 2024-07-16T10:01:36.207 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:36.204+0000 7f54873db640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f54800fb240 0x7f548010fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:01:36.207 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:36.204+0000 7f54873db640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f54800fc510 0x7f54801121d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:01:36.207 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:36.204+0000 7f54873db640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5480065a90 con 0x7f54800fb240 2024-07-16T10:01:36.207 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:36.204+0000 7f54873db640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5480065c00 con 0x7f54800fc510 2024-07-16T10:01:36.207 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:36.204+0000 7f5485177640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f54800fb240 0x7f548010fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:01:36.207 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:36.205+0000 7f5485177640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f54800fb240 0x7f548010fc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:60106/0 (socket says 172.21.15.156:60106) 2024-07-16T10:01:36.207 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:36.205+0000 7f5485177640 1 -- 172.21.15.156:0/1189990355 learned_addr learned my addr 172.21.15.156:0/1189990355 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T10:01:36.207 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:36.205+0000 7f5484976640 1 --2- 172.21.15.156:0/1189990355 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f54800fc510 0x7f54801121d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:01:36.207 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:36.205+0000 7f5485177640 1 -- 172.21.15.156:0/1189990355 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f54800fc510 msgr2=0x7f54801121d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:01:36.208 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:36.205+0000 7f5485177640 1 --2- 172.21.15.156:0/1189990355 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f54800fc510 0x7f54801121d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:01:36.208 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:36.205+0000 7f5485177640 1 -- 172.21.15.156:0/1189990355 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5474005040 con 0x7f54800fb240 2024-07-16T10:01:36.208 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:36.205+0000 7f5485177640 1 --2- 172.21.15.156:0/1189990355 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f54800fb240 0x7f548010fc80 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f5474009f80 tx=0x7f5474009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T10:01:36.209 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:36.205+0000 7f5471ffb640 1 -- 172.21.15.156:0/1189990355 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5474038070 con 0x7f54800fb240 2024-07-16T10:01:36.209 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:36.205+0000 7f5471ffb640 1 -- 172.21.15.156:0/1189990355 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5474032970 con 0x7f54800fb240 2024-07-16T10:01:36.209 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:36.205+0000 7f5471ffb640 1 -- 172.21.15.156:0/1189990355 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5474033860 con 0x7f54800fb240 2024-07-16T10:01:36.209 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:36.205+0000 7f54873db640 1 -- 172.21.15.156:0/1189990355 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5480112710 con 0x7f54800fb240 2024-07-16T10:01:36.209 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:36.205+0000 7f54873db640 1 -- 172.21.15.156:0/1189990355 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5480112c50 con 0x7f54800fb240 2024-07-16T10:01:36.209 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:36.206+0000 7f54873db640 1 -- 172.21.15.156:0/1189990355 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54800fa6a0 con 0x7f54800fb240 2024-07-16T10:01:36.209 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:36.207+0000 7f5471ffb640 1 -- 172.21.15.156:0/1189990355 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 38) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f5474030020 con 0x7f54800fb240 2024-07-16T10:01:36.209 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:36.207+0000 7f5471ffb640 1 --2- 172.21.15.156:0/1189990355 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f546c06e0b0 0x7f546c070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:01:36.210 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:36.207+0000 7f5471ffb640 1 -- 172.21.15.156:0/1189990355 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f54740adaa0 con 0x7f54800fb240 2024-07-16T10:01:36.210 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:36.207+0000 7f5484976640 1 --2- 172.21.15.156:0/1189990355 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f546c06e0b0 0x7f546c070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:01:36.210 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:36.208+0000 7f5484976640 1 --2- 172.21.15.156:0/1189990355 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f546c06e0b0 0x7f546c070560 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f547c004ce0 tx=0x7f547c00b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T10:01:36.213 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:36.212+0000 7f5471ffb640 1 -- 172.21.15.156:0/1189990355 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187716 (secure 0 0 0) 0x7f5474076e20 con 0x7f54800fb240 2024-07-16T10:01:36.424 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:01:36 smithi156 ceph-mon[93066]: pgmap v141: 65 pgs: 65 active+clean; 253 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-07-16T10:01:36.424 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:01:36 smithi156 ceph-mon[93066]: from='client.34342 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-16T10:01:36.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:01:36 smithi156 ceph-mon[93066]: from='client.34346 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-16T10:01:36.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:01:36 smithi156 ceph-mon[93066]: from='client.34350 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-07-16T10:01:36.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:01:36 smithi156 ceph-mon[93066]: from='client.? 172.21.15.156:0/761230073' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:01:36.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:01:36 smithi156 ceph-mon[93066]: from='client.? 172.21.15.156:0/316342209' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-07-16T10:01:36.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:01:36 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-07-16T10:01:36.470 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:36.469+0000 7f54873db640 1 -- 172.21.15.156:0/1189990355 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f548004b530 con 0x7f54800fb240 2024-07-16T10:01:36.470 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:36.469+0000 7f5471ffb640 1 -- 172.21.15.156:0/1189990355 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7f547407a4e0 con 0x7f54800fb240 2024-07-16T10:01:36.471 INFO:teuthology.orchestra.run.smithi156.stdout:HEALTH_OK 2024-07-16T10:01:36.472 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:36.471+0000 7f54873db640 1 -- 172.21.15.156:0/1189990355 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f546c06e0b0 msgr2=0x7f546c070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:01:36.473 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:36.471+0000 7f54873db640 1 --2- 172.21.15.156:0/1189990355 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f546c06e0b0 0x7f546c070560 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f547c004ce0 tx=0x7f547c00b040 comp rx=0 tx=0).stop 2024-07-16T10:01:36.473 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:36.471+0000 7f54873db640 1 -- 172.21.15.156:0/1189990355 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f54800fb240 msgr2=0x7f548010fc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:01:36.473 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:36.471+0000 7f54873db640 1 --2- 172.21.15.156:0/1189990355 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f54800fb240 0x7f548010fc80 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f5474009f80 tx=0x7f5474009da0 comp rx=0 tx=0).stop 2024-07-16T10:01:36.473 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:36.471+0000 7f54873db640 1 -- 172.21.15.156:0/1189990355 shutdown_connections 2024-07-16T10:01:36.473 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:36.471+0000 7f54873db640 1 --2- 172.21.15.156:0/1189990355 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f546c06e0b0 0x7f546c070560 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:01:36.473 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:36.471+0000 7f54873db640 1 --2- 172.21.15.156:0/1189990355 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f54800fc510 0x7f54801121d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:01:36.473 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:36.471+0000 7f54873db640 1 --2- 172.21.15.156:0/1189990355 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f54800fb240 0x7f548010fc80 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:01:36.473 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:36.471+0000 7f54873db640 1 -- 172.21.15.156:0/1189990355 >> 172.21.15.156:0/1189990355 conn(0x7f54800f6a20 msgr2=0x7f54800f8110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T10:01:36.473 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:36.471+0000 7f54873db640 1 -- 172.21.15.156:0/1189990355 shutdown_connections 2024-07-16T10:01:36.473 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:01:36.471+0000 7f54873db640 1 -- 172.21.15.156:0/1189990355 wait complete. 2024-07-16T10:01:36.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:01:36 smithi176 ceph-mon[81689]: pgmap v141: 65 pgs: 65 active+clean; 253 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-07-16T10:01:36.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:01:36 smithi176 ceph-mon[81689]: from='client.34342 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-16T10:01:36.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:01:36 smithi176 ceph-mon[81689]: from='client.34346 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-16T10:01:36.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:01:36 smithi176 ceph-mon[81689]: from='client.34350 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-07-16T10:01:36.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:01:36 smithi176 ceph-mon[81689]: from='client.? 172.21.15.156:0/761230073' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:01:36.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:01:36 smithi176 ceph-mon[81689]: from='client.? 172.21.15.156:0/316342209' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-07-16T10:01:36.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:01:36 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-07-16T10:01:37.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:01:37 smithi176 ceph-mon[81689]: from='client.44257 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-16T10:01:37.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:01:37 smithi176 ceph-mon[81689]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-07-16T10:01:37.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:01:37 smithi176 ceph-mon[81689]: Upgrade: osd.5 is safe to restart 2024-07-16T10:01:37.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:01:37 smithi176 ceph-mon[81689]: from='client.? 172.21.15.156:0/1189990355' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-07-16T10:01:37.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:01:37 smithi156 ceph-mon[93066]: from='client.44257 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-16T10:01:37.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:01:37 smithi156 ceph-mon[93066]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-07-16T10:01:37.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:01:37 smithi156 ceph-mon[93066]: Upgrade: osd.5 is safe to restart 2024-07-16T10:01:37.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:01:37 smithi156 ceph-mon[93066]: from='client.? 172.21.15.156:0/1189990355' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-07-16T10:01:38.382 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:01:38 smithi176 ceph-mon[81689]: pgmap v142: 65 pgs: 65 active+clean; 253 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.9 KiB/s rd, 4 op/s 2024-07-16T10:01:38.382 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:01:38 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:01:38.382 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:01:38 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-07-16T10:01:38.383 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:01:38 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-16T10:01:38.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:01:38 smithi156 ceph-mon[93066]: pgmap v142: 65 pgs: 65 active+clean; 253 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.9 KiB/s rd, 4 op/s 2024-07-16T10:01:38.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:01:38 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:01:38.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:01:38 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-07-16T10:01:38.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:01:38 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-16T10:01:39.498 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:01:39 smithi176 ceph-mon[81689]: Upgrade: Updating osd.5 2024-07-16T10:01:39.498 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:01:39 smithi176 ceph-mon[81689]: Deploying daemon osd.5 on smithi176 2024-07-16T10:01:39.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:01:39 smithi156 ceph-mon[93066]: Upgrade: Updating osd.5 2024-07-16T10:01:39.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:01:39 smithi156 ceph-mon[93066]: Deploying daemon osd.5 on smithi176 2024-07-16T10:01:40.519 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:01:40 smithi176 ceph-mon[81689]: pgmap v143: 65 pgs: 65 active+clean; 253 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-07-16T10:01:40.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:01:40 smithi156 ceph-mon[93066]: pgmap v143: 65 pgs: 65 active+clean; 253 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-07-16T10:01:40.822 INFO:journalctl@ceph.osd.5.smithi176.stdout:Jul 16 10:01:40 smithi176 systemd[1]: Stopping Ceph osd.5 for 26c91c26-4358-11ef-bcad-c7b262605968... 2024-07-16T10:01:41.093 INFO:journalctl@ceph.osd.5.smithi176.stdout:Jul 16 10:01:40 smithi176 ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-5[54868]: 2024-07-16T10:01:40.820+0000 7fb035e8a640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-07-16T10:01:41.093 INFO:journalctl@ceph.osd.5.smithi176.stdout:Jul 16 10:01:40 smithi176 ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-5[54868]: 2024-07-16T10:01:40.820+0000 7fb035e8a640 -1 osd.5 74 *** Got signal Terminated *** 2024-07-16T10:01:41.094 INFO:journalctl@ceph.osd.5.smithi176.stdout:Jul 16 10:01:40 smithi176 ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-5[54868]: 2024-07-16T10:01:40.820+0000 7fb035e8a640 -1 osd.5 74 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-07-16T10:01:41.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:01:41 smithi156 ceph-mon[93066]: pgmap v144: 65 pgs: 65 active+clean; 253 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-07-16T10:01:41.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:01:41 smithi156 ceph-mon[93066]: osd.5 marked itself down and dead 2024-07-16T10:01:41.814 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:01:41 smithi176 ceph-mon[81689]: pgmap v144: 65 pgs: 65 active+clean; 253 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-07-16T10:01:41.815 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:01:41 smithi176 ceph-mon[81689]: osd.5 marked itself down and dead 2024-07-16T10:01:42.094 INFO:journalctl@ceph.osd.5.smithi176.stdout:Jul 16 10:01:41 smithi176 podman[100389]: 2024-07-16 10:01:41.813959319 +0000 UTC m=+1.124663434 container died 996e15448a5255cc2053e0e8b5984a48d7a3c91132cd34778ec883894480d9ad (image=quay.ceph.io/ceph-ci/ceph@sha256:7b86ef6bffe0efc5f1aefc9fef8c854a67a511c50993acee3ac684f23bde83df, name=ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-5, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_CLEAN=True, RELEASE=quincy-e6231b0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240709, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0) 2024-07-16T10:01:42.548 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:01:42 smithi176 ceph-mon[81689]: Health check failed: 1 osds down (OSD_DOWN) 2024-07-16T10:01:42.548 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:01:42 smithi176 ceph-mon[81689]: Health check failed: all OSDs are running squid or later but require_osd_release < squid (OSD_UPGRADE_FINISHED) 2024-07-16T10:01:42.549 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:01:42 smithi176 ceph-mon[81689]: osdmap e75: 6 total, 5 up, 6 in 2024-07-16T10:01:42.549 INFO:journalctl@ceph.osd.5.smithi176.stdout:Jul 16 10:01:42 smithi176 podman[100389]: 2024-07-16 10:01:42.271036406 +0000 UTC m=+1.581740512 container cleanup 996e15448a5255cc2053e0e8b5984a48d7a3c91132cd34778ec883894480d9ad (image=quay.ceph.io/ceph-ci/ceph@sha256:7b86ef6bffe0efc5f1aefc9fef8c854a67a511c50993acee3ac684f23bde83df, name=ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.36.0, RELEASE=quincy-e6231b0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240709, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-07-16T10:01:42.549 INFO:journalctl@ceph.osd.5.smithi176.stdout:Jul 16 10:01:42 smithi176 bash[100389]: ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-5 2024-07-16T10:01:42.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:01:42 smithi156 ceph-mon[93066]: Health check failed: 1 osds down (OSD_DOWN) 2024-07-16T10:01:42.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:01:42 smithi156 ceph-mon[93066]: Health check failed: all OSDs are running squid or later but require_osd_release < squid (OSD_UPGRADE_FINISHED) 2024-07-16T10:01:42.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:01:42 smithi156 ceph-mon[93066]: osdmap e75: 6 total, 5 up, 6 in 2024-07-16T10:01:42.843 INFO:journalctl@ceph.osd.5.smithi176.stdout:Jul 16 10:01:42 smithi176 podman[100406]: 2024-07-16 10:01:42.547724023 +0000 UTC m=+0.728622033 container remove 996e15448a5255cc2053e0e8b5984a48d7a3c91132cd34778ec883894480d9ad (image=quay.ceph.io/ceph-ci/ceph@sha256:7b86ef6bffe0efc5f1aefc9fef8c854a67a511c50993acee3ac684f23bde83df, name=ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-5, org.label-schema.build-date=20240709, RELEASE=quincy-e6231b0, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.36.0) 2024-07-16T10:01:43.121 INFO:journalctl@ceph.osd.5.smithi176.stdout:Jul 16 10:01:42 smithi176 podman[100494]: 2024-07-16 10:01:42.772043788 +0000 UTC m=+0.016867720 image pull 16f7feb043422ce37a2471403d4bc52f550d49cef4b677482112fcc0db3f67ab quay.ceph.io/ceph-ci/ceph@sha256:c15ec3aa8e6c0a9c12ee24ff79f3e7dd4d01b3d1a702dfeb7d7fff9b0811a393 2024-07-16T10:01:43.122 INFO:journalctl@ceph.osd.5.smithi176.stdout:Jul 16 10:01:42 smithi176 podman[100494]: 2024-07-16 10:01:42.90172307 +0000 UTC m=+0.146547013 container create 942143200a4cdfbcf264944524b032d5dc6167c81a29ddffc6bb4e038d7f0fbf (image=quay.ceph.io/ceph-ci/ceph@sha256:c15ec3aa8e6c0a9c12ee24ff79f3e7dd4d01b3d1a702dfeb7d7fff9b0811a393, name=ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-5-deactivate, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.36.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240709, RELEASE=wip-adk4-testing-2024-07-15-1832-squid-a6ca461, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-07-16T10:01:43.122 INFO:journalctl@ceph.osd.5.smithi176.stdout:Jul 16 10:01:43 smithi176 podman[100494]: 2024-07-16 10:01:43.117967625 +0000 UTC m=+0.362791549 container init 942143200a4cdfbcf264944524b032d5dc6167c81a29ddffc6bb4e038d7f0fbf (image=quay.ceph.io/ceph-ci/ceph@sha256:c15ec3aa8e6c0a9c12ee24ff79f3e7dd4d01b3d1a702dfeb7d7fff9b0811a393, name=ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-5-deactivate, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240709, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-adk4-testing-2024-07-15-1832-squid-a6ca461, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-07-16T10:01:43.122 INFO:journalctl@ceph.osd.5.smithi176.stdout:Jul 16 10:01:43 smithi176 podman[100494]: 2024-07-16 10:01:43.121200041 +0000 UTC m=+0.366023962 container start 942143200a4cdfbcf264944524b032d5dc6167c81a29ddffc6bb4e038d7f0fbf (image=quay.ceph.io/ceph-ci/ceph@sha256:c15ec3aa8e6c0a9c12ee24ff79f3e7dd4d01b3d1a702dfeb7d7fff9b0811a393, name=ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-5-deactivate, RELEASE=wip-adk4-testing-2024-07-15-1832-squid-a6ca461, GIT_BRANCH=HEAD, org.label-schema.build-date=20240709, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-07-16T10:01:43.373 INFO:journalctl@ceph.osd.5.smithi176.stdout:Jul 16 10:01:43 smithi176 podman[100494]: 2024-07-16 10:01:43.209557288 +0000 UTC m=+0.454381210 container attach 942143200a4cdfbcf264944524b032d5dc6167c81a29ddffc6bb4e038d7f0fbf (image=quay.ceph.io/ceph-ci/ceph@sha256:c15ec3aa8e6c0a9c12ee24ff79f3e7dd4d01b3d1a702dfeb7d7fff9b0811a393, name=ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-5-deactivate, GIT_BRANCH=HEAD, org.label-schema.build-date=20240709, RELEASE=wip-adk4-testing-2024-07-15-1832-squid-a6ca461, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-07-16T10:01:43.373 INFO:journalctl@ceph.osd.5.smithi176.stdout:Jul 16 10:01:43 smithi176 podman[100494]: 2024-07-16 10:01:43.31418503 +0000 UTC m=+0.559008964 container died 942143200a4cdfbcf264944524b032d5dc6167c81a29ddffc6bb4e038d7f0fbf (image=quay.ceph.io/ceph-ci/ceph@sha256:c15ec3aa8e6c0a9c12ee24ff79f3e7dd4d01b3d1a702dfeb7d7fff9b0811a393, name=ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-5-deactivate, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.36.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-adk4-testing-2024-07-15-1832-squid-a6ca461, org.label-schema.build-date=20240709, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-07-16T10:01:43.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:01:43 smithi156 ceph-mon[93066]: pgmap v146: 65 pgs: 3 peering, 11 stale+active+clean, 51 active+clean; 253 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s 2024-07-16T10:01:43.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:01:43 smithi156 ceph-mon[93066]: Health check failed: Reduced data availability: 2 pgs peering (PG_AVAILABILITY) 2024-07-16T10:01:43.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:01:43 smithi156 ceph-mon[93066]: osdmap e76: 6 total, 5 up, 6 in 2024-07-16T10:01:43.843 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:01:43 smithi176 ceph-mon[81689]: pgmap v146: 65 pgs: 3 peering, 11 stale+active+clean, 51 active+clean; 253 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s 2024-07-16T10:01:43.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:01:43 smithi176 ceph-mon[81689]: Health check failed: Reduced data availability: 2 pgs peering (PG_AVAILABILITY) 2024-07-16T10:01:43.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:01:43 smithi176 ceph-mon[81689]: osdmap e76: 6 total, 5 up, 6 in 2024-07-16T10:01:44.290 INFO:journalctl@ceph.osd.5.smithi176.stdout:Jul 16 10:01:43 smithi176 podman[100517]: 2024-07-16 10:01:43.854925674 +0000 UTC m=+0.530916533 container remove 942143200a4cdfbcf264944524b032d5dc6167c81a29ddffc6bb4e038d7f0fbf (image=quay.ceph.io/ceph-ci/ceph@sha256:c15ec3aa8e6c0a9c12ee24ff79f3e7dd4d01b3d1a702dfeb7d7fff9b0811a393, name=ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-5-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-adk4-testing-2024-07-15-1832-squid-a6ca461, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240709, io.buildah.version=1.36.0, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-07-16T10:01:44.290 INFO:journalctl@ceph.osd.5.smithi176.stdout:Jul 16 10:01:43 smithi176 systemd[1]: ceph-26c91c26-4358-11ef-bcad-c7b262605968@osd.5.service: Deactivated successfully. 2024-07-16T10:01:44.290 INFO:journalctl@ceph.osd.5.smithi176.stdout:Jul 16 10:01:43 smithi176 systemd[1]: Stopped Ceph osd.5 for 26c91c26-4358-11ef-bcad-c7b262605968. 2024-07-16T10:01:44.290 INFO:journalctl@ceph.osd.5.smithi176.stdout:Jul 16 10:01:43 smithi176 systemd[1]: ceph-26c91c26-4358-11ef-bcad-c7b262605968@osd.5.service: Consumed 1min 3.710s CPU time. 2024-07-16T10:01:44.290 INFO:journalctl@ceph.osd.5.smithi176.stdout:Jul 16 10:01:44 smithi176 systemd[1]: Starting Ceph osd.5 for 26c91c26-4358-11ef-bcad-c7b262605968... 2024-07-16T10:01:44.559 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:01:44 smithi176 ceph-mon[81689]: Health check failed: Degraded data redundancy: 6/318 objects degraded (1.887%), 3 pgs degraded (PG_DEGRADED) 2024-07-16T10:01:44.559 INFO:journalctl@ceph.osd.5.smithi176.stdout:Jul 16 10:01:44 smithi176 podman[100619]: 2024-07-16 10:01:44.191680616 +0000 UTC m=+0.017546815 image pull 16f7feb043422ce37a2471403d4bc52f550d49cef4b677482112fcc0db3f67ab quay.ceph.io/ceph-ci/ceph@sha256:c15ec3aa8e6c0a9c12ee24ff79f3e7dd4d01b3d1a702dfeb7d7fff9b0811a393 2024-07-16T10:01:44.559 INFO:journalctl@ceph.osd.5.smithi176.stdout:Jul 16 10:01:44 smithi176 podman[100619]: 2024-07-16 10:01:44.297461461 +0000 UTC m=+0.123327683 container create c425e34d6f492c0b135c161743e6c9056dac01439a82371a9342d105200ae7ec (image=quay.ceph.io/ceph-ci/ceph@sha256:c15ec3aa8e6c0a9c12ee24ff79f3e7dd4d01b3d1a702dfeb7d7fff9b0811a393, name=ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-5-activate, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0, GIT_CLEAN=True, RELEASE=wip-adk4-testing-2024-07-15-1832-squid-a6ca461, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240709, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-07-16T10:01:44.560 INFO:journalctl@ceph.osd.5.smithi176.stdout:Jul 16 10:01:44 smithi176 podman[100619]: 2024-07-16 10:01:44.558479523 +0000 UTC m=+0.384345723 container init c425e34d6f492c0b135c161743e6c9056dac01439a82371a9342d105200ae7ec (image=quay.ceph.io/ceph-ci/ceph@sha256:c15ec3aa8e6c0a9c12ee24ff79f3e7dd4d01b3d1a702dfeb7d7fff9b0811a393, name=ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-5-activate, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-adk4-testing-2024-07-15-1832-squid-a6ca461, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240709, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-07-16T10:01:44.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:01:44 smithi156 ceph-mon[93066]: Health check failed: Degraded data redundancy: 6/318 objects degraded (1.887%), 3 pgs degraded (PG_DEGRADED) 2024-07-16T10:01:44.844 INFO:journalctl@ceph.osd.5.smithi176.stdout:Jul 16 10:01:44 smithi176 podman[100619]: 2024-07-16 10:01:44.561819479 +0000 UTC m=+0.387685681 container start c425e34d6f492c0b135c161743e6c9056dac01439a82371a9342d105200ae7ec (image=quay.ceph.io/ceph-ci/ceph@sha256:c15ec3aa8e6c0a9c12ee24ff79f3e7dd4d01b3d1a702dfeb7d7fff9b0811a393, name=ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-5-activate, org.label-schema.build-date=20240709, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.36.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-adk4-testing-2024-07-15-1832-squid-a6ca461, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-07-16T10:01:44.844 INFO:journalctl@ceph.osd.5.smithi176.stdout:Jul 16 10:01:44 smithi176 podman[100619]: 2024-07-16 10:01:44.658330061 +0000 UTC m=+0.484196273 container attach c425e34d6f492c0b135c161743e6c9056dac01439a82371a9342d105200ae7ec (image=quay.ceph.io/ceph-ci/ceph@sha256:c15ec3aa8e6c0a9c12ee24ff79f3e7dd4d01b3d1a702dfeb7d7fff9b0811a393, name=ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-5-activate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240709, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, RELEASE=wip-adk4-testing-2024-07-15-1832-squid-a6ca461, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, io.buildah.version=1.36.0) 2024-07-16T10:01:44.844 INFO:journalctl@ceph.osd.5.smithi176.stdout:Jul 16 10:01:44 smithi176 ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-5-activate[100634]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-07-16T10:01:44.844 INFO:journalctl@ceph.osd.5.smithi176.stdout:Jul 16 10:01:44 smithi176 bash[100619]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-07-16T10:01:44.844 INFO:journalctl@ceph.osd.5.smithi176.stdout:Jul 16 10:01:44 smithi176 ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-5-activate[100634]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-07-16T10:01:44.844 INFO:journalctl@ceph.osd.5.smithi176.stdout:Jul 16 10:01:44 smithi176 bash[100619]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-07-16T10:01:45.301 INFO:journalctl@ceph.osd.5.smithi176.stdout:Jul 16 10:01:44 smithi176 ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-5-activate[100634]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-07-16T10:01:45.301 INFO:journalctl@ceph.osd.5.smithi176.stdout:Jul 16 10:01:44 smithi176 bash[100619]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-07-16T10:01:45.301 INFO:journalctl@ceph.osd.5.smithi176.stdout:Jul 16 10:01:44 smithi176 ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-5-activate[100634]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-5 --no-mon-config --dev /dev/mapper/vg_nvme-lv_2 2024-07-16T10:01:45.301 INFO:journalctl@ceph.osd.5.smithi176.stdout:Jul 16 10:01:44 smithi176 bash[100619]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-5 --no-mon-config --dev /dev/mapper/vg_nvme-lv_2 2024-07-16T10:01:45.301 INFO:journalctl@ceph.osd.5.smithi176.stdout:Jul 16 10:01:44 smithi176 ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-5-activate[100634]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2024-07-16T10:01:45.301 INFO:journalctl@ceph.osd.5.smithi176.stdout:Jul 16 10:01:44 smithi176 bash[100619]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2024-07-16T10:01:45.301 INFO:journalctl@ceph.osd.5.smithi176.stdout:Jul 16 10:01:44 smithi176 ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-5-activate[100634]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-07-16T10:01:45.301 INFO:journalctl@ceph.osd.5.smithi176.stdout:Jul 16 10:01:44 smithi176 bash[100619]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-07-16T10:01:45.301 INFO:journalctl@ceph.osd.5.smithi176.stdout:Jul 16 10:01:44 smithi176 ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-5-activate[100634]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5/block 2024-07-16T10:01:45.301 INFO:journalctl@ceph.osd.5.smithi176.stdout:Jul 16 10:01:44 smithi176 bash[100619]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5/block 2024-07-16T10:01:45.302 INFO:journalctl@ceph.osd.5.smithi176.stdout:Jul 16 10:01:44 smithi176 ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-5-activate[100634]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-07-16T10:01:45.302 INFO:journalctl@ceph.osd.5.smithi176.stdout:Jul 16 10:01:44 smithi176 bash[100619]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-07-16T10:01:45.302 INFO:journalctl@ceph.osd.5.smithi176.stdout:Jul 16 10:01:44 smithi176 ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-5-activate[100634]: --> ceph-volume raw activate successful for osd ID: 5 2024-07-16T10:01:45.302 INFO:journalctl@ceph.osd.5.smithi176.stdout:Jul 16 10:01:44 smithi176 bash[100619]: --> ceph-volume raw activate successful for osd ID: 5 2024-07-16T10:01:45.302 INFO:journalctl@ceph.osd.5.smithi176.stdout:Jul 16 10:01:44 smithi176 podman[100619]: 2024-07-16 10:01:44.942286717 +0000 UTC m=+0.768152922 container died c425e34d6f492c0b135c161743e6c9056dac01439a82371a9342d105200ae7ec (image=quay.ceph.io/ceph-ci/ceph@sha256:c15ec3aa8e6c0a9c12ee24ff79f3e7dd4d01b3d1a702dfeb7d7fff9b0811a393, name=ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-5-activate, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240709, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-adk4-testing-2024-07-15-1832-squid-a6ca461, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.36.0) 2024-07-16T10:01:45.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:01:45 smithi176 ceph-mon[81689]: pgmap v148: 65 pgs: 2 active+undersized, 3 peering, 8 stale+active+clean, 3 active+undersized+degraded, 49 active+clean; 253 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 6/318 objects degraded (1.887%) 2024-07-16T10:01:45.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:01:45 smithi156 ceph-mon[93066]: pgmap v148: 65 pgs: 2 active+undersized, 3 peering, 8 stale+active+clean, 3 active+undersized+degraded, 49 active+clean; 253 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 6/318 objects degraded (1.887%) 2024-07-16T10:01:45.977 INFO:journalctl@ceph.osd.5.smithi176.stdout:Jul 16 10:01:45 smithi176 podman[100762]: 2024-07-16 10:01:45.670642014 +0000 UTC m=+0.722755440 container remove c425e34d6f492c0b135c161743e6c9056dac01439a82371a9342d105200ae7ec (image=quay.ceph.io/ceph-ci/ceph@sha256:c15ec3aa8e6c0a9c12ee24ff79f3e7dd4d01b3d1a702dfeb7d7fff9b0811a393, name=ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-5-activate, RELEASE=wip-adk4-testing-2024-07-15-1832-squid-a6ca461, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.36.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240709, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-07-16T10:01:45.977 INFO:journalctl@ceph.osd.5.smithi176.stdout:Jul 16 10:01:45 smithi176 podman[100822]: 2024-07-16 10:01:45.879869511 +0000 UTC m=+0.106590865 container create a03640221e7aaf17143732e69641d2ff5b8644e9028fab51caee28aa114e4b5c (image=quay.ceph.io/ceph-ci/ceph@sha256:c15ec3aa8e6c0a9c12ee24ff79f3e7dd4d01b3d1a702dfeb7d7fff9b0811a393, name=ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-5, io.buildah.version=1.36.0, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-adk4-testing-2024-07-15-1832-squid-a6ca461, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240709, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-07-16T10:01:45.977 INFO:journalctl@ceph.osd.5.smithi176.stdout:Jul 16 10:01:45 smithi176 podman[100822]: 2024-07-16 10:01:45.790502075 +0000 UTC m=+0.017223406 image pull 16f7feb043422ce37a2471403d4bc52f550d49cef4b677482112fcc0db3f67ab quay.ceph.io/ceph-ci/ceph@sha256:c15ec3aa8e6c0a9c12ee24ff79f3e7dd4d01b3d1a702dfeb7d7fff9b0811a393 2024-07-16T10:01:46.343 INFO:journalctl@ceph.osd.5.smithi176.stdout:Jul 16 10:01:46 smithi176 podman[100822]: 2024-07-16 10:01:46.090312117 +0000 UTC m=+0.317033429 container init a03640221e7aaf17143732e69641d2ff5b8644e9028fab51caee28aa114e4b5c (image=quay.ceph.io/ceph-ci/ceph@sha256:c15ec3aa8e6c0a9c12ee24ff79f3e7dd4d01b3d1a702dfeb7d7fff9b0811a393, name=ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, RELEASE=wip-adk4-testing-2024-07-15-1832-squid-a6ca461, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240709, CEPH_POINT_RELEASE=, io.buildah.version=1.36.0) 2024-07-16T10:01:46.344 INFO:journalctl@ceph.osd.5.smithi176.stdout:Jul 16 10:01:46 smithi176 podman[100822]: 2024-07-16 10:01:46.093499616 +0000 UTC m=+0.320220932 container start a03640221e7aaf17143732e69641d2ff5b8644e9028fab51caee28aa114e4b5c (image=quay.ceph.io/ceph-ci/ceph@sha256:c15ec3aa8e6c0a9c12ee24ff79f3e7dd4d01b3d1a702dfeb7d7fff9b0811a393, name=ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-adk4-testing-2024-07-15-1832-squid-a6ca461, maintainer=Guillaume Abrioux , io.buildah.version=1.36.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240709, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True) 2024-07-16T10:01:46.344 INFO:journalctl@ceph.osd.5.smithi176.stdout:Jul 16 10:01:46 smithi176 ceph-osd[100842]: -- 172.21.15.176:0/3416387838 <== mon.1 v2:172.21.15.176:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x563b87cf61e0 con 0x563b87c8e000 2024-07-16T10:01:46.344 INFO:journalctl@ceph.osd.5.smithi176.stdout:Jul 16 10:01:46 smithi176 bash[100822]: a03640221e7aaf17143732e69641d2ff5b8644e9028fab51caee28aa114e4b5c 2024-07-16T10:01:46.344 INFO:journalctl@ceph.osd.5.smithi176.stdout:Jul 16 10:01:46 smithi176 systemd[1]: Started Ceph osd.5 for 26c91c26-4358-11ef-bcad-c7b262605968. 2024-07-16T10:01:46.962 INFO:journalctl@ceph.osd.5.smithi176.stdout:Jul 16 10:01:46 smithi176 ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-5[100838]: 2024-07-16T10:01:46.641+0000 7fcd9ae21640 -1 Falling back to public interface 2024-07-16T10:01:47.538 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:01:47 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:01:47.538 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:01:47 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:01:47.538 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:01:47 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-16T10:01:47.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:01:47 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:01:47.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:01:47 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:01:47.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:01:47 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-16T10:01:48.573 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:01:48 smithi176 ceph-mon[81689]: pgmap v149: 65 pgs: 11 active+undersized, 3 peering, 14 active+undersized+degraded, 37 active+clean; 253 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 49/318 objects degraded (15.409%) 2024-07-16T10:01:48.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:01:48 smithi156 ceph-mon[93066]: pgmap v149: 65 pgs: 11 active+undersized, 3 peering, 14 active+undersized+degraded, 37 active+clean; 253 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 49/318 objects degraded (15.409%) 2024-07-16T10:01:49.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:01:49 smithi156 ceph-mon[93066]: pgmap v150: 65 pgs: 14 active+undersized, 14 active+undersized+degraded, 37 active+clean; 253 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 49/318 objects degraded (15.409%) 2024-07-16T10:01:49.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:01:49 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:01:49.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:01:49 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-16T10:01:49.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:01:49 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:01:49.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:01:49 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:01:49.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:01:49 smithi156 ceph-mon[93066]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 2 pgs peering) 2024-07-16T10:01:49.843 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:01:49 smithi176 ceph-mon[81689]: pgmap v150: 65 pgs: 14 active+undersized, 14 active+undersized+degraded, 37 active+clean; 253 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 49/318 objects degraded (15.409%) 2024-07-16T10:01:49.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:01:49 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:01:49.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:01:49 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-16T10:01:49.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:01:49 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:01:49.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:01:49 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:01:49.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:01:49 smithi176 ceph-mon[81689]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 2 pgs peering) 2024-07-16T10:01:50.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:01:50 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:01:50.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:01:50 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:01:50.924 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:01:50 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:01:50.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:01:50 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:01:51.527 INFO:journalctl@ceph.osd.5.smithi176.stdout:Jul 16 10:01:51 smithi176 ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-5[100838]: 2024-07-16T10:01:51.228+0000 7fcd9ae21640 -1 osd.5 74 log_to_monitors true 2024-07-16T10:01:51.843 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:01:51 smithi176 ceph-mon[81689]: pgmap v151: 65 pgs: 14 active+undersized, 14 active+undersized+degraded, 37 active+clean; 253 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 49/318 objects degraded (15.409%) 2024-07-16T10:01:51.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:01:51 smithi176 ceph-mon[81689]: from='osd.5 [v2:172.21.15.176:6816/2260788529,v1:172.21.15.176:6817/2260788529]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-07-16T10:01:51.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:01:51 smithi176 ceph-mon[81689]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-07-16T10:01:51.844 INFO:journalctl@ceph.osd.5.smithi176.stdout:Jul 16 10:01:51 smithi176 ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-5[100838]: 2024-07-16T10:01:51.582+0000 7fcd92bbb640 -1 osd.5 74 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-07-16T10:01:51.924 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:01:51 smithi156 ceph-mon[93066]: pgmap v151: 65 pgs: 14 active+undersized, 14 active+undersized+degraded, 37 active+clean; 253 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 49/318 objects degraded (15.409%) 2024-07-16T10:01:51.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:01:51 smithi156 ceph-mon[93066]: from='osd.5 [v2:172.21.15.176:6816/2260788529,v1:172.21.15.176:6817/2260788529]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-07-16T10:01:51.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:01:51 smithi156 ceph-mon[93066]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-07-16T10:01:52.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:01:52 smithi176 ceph-mon[81689]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-07-16T10:01:52.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:01:52 smithi176 ceph-mon[81689]: osdmap e77: 6 total, 5 up, 6 in 2024-07-16T10:01:52.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:01:52 smithi176 ceph-mon[81689]: from='osd.5 [v2:172.21.15.176:6816/2260788529,v1:172.21.15.176:6817/2260788529]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi176", "root=default"]}]: dispatch 2024-07-16T10:01:52.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:01:52 smithi176 ceph-mon[81689]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi176", "root=default"]}]: dispatch 2024-07-16T10:01:52.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:01:52 smithi176 ceph-mon[81689]: Health check update: Degraded data redundancy: 49/318 objects degraded (15.409%), 14 pgs degraded (PG_DEGRADED) 2024-07-16T10:01:52.924 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:01:52 smithi156 ceph-mon[93066]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-07-16T10:01:52.924 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:01:52 smithi156 ceph-mon[93066]: osdmap e77: 6 total, 5 up, 6 in 2024-07-16T10:01:52.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:01:52 smithi156 ceph-mon[93066]: from='osd.5 [v2:172.21.15.176:6816/2260788529,v1:172.21.15.176:6817/2260788529]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi176", "root=default"]}]: dispatch 2024-07-16T10:01:52.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:01:52 smithi156 ceph-mon[93066]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi176", "root=default"]}]: dispatch 2024-07-16T10:01:52.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:01:52 smithi156 ceph-mon[93066]: Health check update: Degraded data redundancy: 49/318 objects degraded (15.409%), 14 pgs degraded (PG_DEGRADED) 2024-07-16T10:01:53.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:01:53 smithi176 ceph-mon[81689]: pgmap v153: 65 pgs: 14 active+undersized, 14 active+undersized+degraded, 37 active+clean; 253 MiB data, 1013 MiB used, 535 GiB / 536 GiB avail; 49/318 objects degraded (15.409%) 2024-07-16T10:01:53.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:01:53 smithi176 ceph-mon[81689]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-07-16T10:01:53.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:01:53 smithi176 ceph-mon[81689]: osd.5 [v2:172.21.15.176:6816/2260788529,v1:172.21.15.176:6817/2260788529] boot 2024-07-16T10:01:53.845 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:01:53 smithi176 ceph-mon[81689]: osdmap e78: 6 total, 6 up, 6 in 2024-07-16T10:01:53.845 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:01:53 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-07-16T10:01:53.845 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:01:53 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:01:53.845 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:01:53 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:01:53.845 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:01:53 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-16T10:01:53.845 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:01:53 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-16T10:01:53.845 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:01:53 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:01:53.845 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:01:53 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-16T10:01:53.845 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:01:53 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:01:53.845 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:01:53 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:01:53.845 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:01:53 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:01:53.845 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:01:53 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:01:53.845 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:01:53 smithi176 ceph-mon[81689]: Upgrade: Setting container_image for all osd 2024-07-16T10:01:53.846 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:01:53 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:01:53.846 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:01:53 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]: dispatch 2024-07-16T10:01:53.846 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:01:53 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]': finished 2024-07-16T10:01:53.846 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:01:53 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]: dispatch 2024-07-16T10:01:53.846 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:01:53 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]': finished 2024-07-16T10:01:53.846 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:01:53 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]: dispatch 2024-07-16T10:01:53.846 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:01:53 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]': finished 2024-07-16T10:01:53.846 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:01:53 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]: dispatch 2024-07-16T10:01:53.846 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:01:53 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]': finished 2024-07-16T10:01:53.846 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:01:53 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]: dispatch 2024-07-16T10:01:53.846 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:01:53 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]': finished 2024-07-16T10:01:53.846 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:01:53 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]: dispatch 2024-07-16T10:01:53.847 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:01:53 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]': finished 2024-07-16T10:01:53.847 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:01:53 smithi176 ceph-mon[81689]: Upgrade: Setting require_osd_release to 19 squid 2024-07-16T10:01:53.847 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:01:53 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd require-osd-release", "release": "squid"}]: dispatch 2024-07-16T10:01:53.847 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:01:53 smithi176 ceph-mon[81689]: Health check cleared: OSD_UPGRADE_FINISHED (was: all OSDs are running squid or later but require_osd_release < squid) 2024-07-16T10:01:53.847 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:01:53 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd='[{"prefix": "osd require-osd-release", "release": "squid"}]': finished 2024-07-16T10:01:53.847 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:01:53 smithi176 ceph-mon[81689]: osdmap e79: 6 total, 6 up, 6 in 2024-07-16T10:01:53.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:01:53 smithi156 ceph-mon[93066]: pgmap v153: 65 pgs: 14 active+undersized, 14 active+undersized+degraded, 37 active+clean; 253 MiB data, 1013 MiB used, 535 GiB / 536 GiB avail; 49/318 objects degraded (15.409%) 2024-07-16T10:01:53.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:01:53 smithi156 ceph-mon[93066]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-07-16T10:01:53.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:01:53 smithi156 ceph-mon[93066]: osd.5 [v2:172.21.15.176:6816/2260788529,v1:172.21.15.176:6817/2260788529] boot 2024-07-16T10:01:53.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:01:53 smithi156 ceph-mon[93066]: osdmap e78: 6 total, 6 up, 6 in 2024-07-16T10:01:53.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:01:53 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-07-16T10:01:53.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:01:53 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:01:53.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:01:53 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:01:53.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:01:53 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-16T10:01:53.926 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:01:53 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-16T10:01:53.926 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:01:53 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:01:53.926 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:01:53 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-16T10:01:53.926 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:01:53 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:01:53.926 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:01:53 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:01:53.926 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:01:53 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:01:53.926 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:01:53 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:01:53.926 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:01:53 smithi156 ceph-mon[93066]: Upgrade: Setting container_image for all osd 2024-07-16T10:01:53.926 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:01:53 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:01:53.926 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:01:53 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]: dispatch 2024-07-16T10:01:53.926 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:01:53 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]': finished 2024-07-16T10:01:53.927 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:01:53 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]: dispatch 2024-07-16T10:01:53.927 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:01:53 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]': finished 2024-07-16T10:01:53.927 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:01:53 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]: dispatch 2024-07-16T10:01:53.927 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:01:53 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]': finished 2024-07-16T10:01:53.927 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:01:53 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]: dispatch 2024-07-16T10:01:53.927 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:01:53 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]': finished 2024-07-16T10:01:53.927 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:01:53 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]: dispatch 2024-07-16T10:01:53.927 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:01:53 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]': finished 2024-07-16T10:01:53.927 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:01:53 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]: dispatch 2024-07-16T10:01:53.927 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:01:53 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]': finished 2024-07-16T10:01:53.927 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:01:53 smithi156 ceph-mon[93066]: Upgrade: Setting require_osd_release to 19 squid 2024-07-16T10:01:53.927 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:01:53 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd require-osd-release", "release": "squid"}]: dispatch 2024-07-16T10:01:53.927 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:01:53 smithi156 ceph-mon[93066]: Health check cleared: OSD_UPGRADE_FINISHED (was: all OSDs are running squid or later but require_osd_release < squid) 2024-07-16T10:01:53.928 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:01:53 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd='[{"prefix": "osd require-osd-release", "release": "squid"}]': finished 2024-07-16T10:01:53.928 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:01:53 smithi156 ceph-mon[93066]: osdmap e79: 6 total, 6 up, 6 in 2024-07-16T10:01:54.924 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:01:54 smithi156 ceph-mon[93066]: Upgrade: Disabling standby-replay for filesystem cephfs 2024-07-16T10:01:54.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:01:54 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:01:54.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:01:54 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "0"}]: dispatch 2024-07-16T10:01:55.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:01:54 smithi176 ceph-mon[81689]: Upgrade: Disabling standby-replay for filesystem cephfs 2024-07-16T10:01:55.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:01:54 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:01:55.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:01:54 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "0"}]: dispatch 2024-07-16T10:01:55.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:01:55 smithi156 ceph-mon[93066]: pgmap v156: 65 pgs: 13 active+undersized, 13 active+undersized+degraded, 39 active+clean; 253 MiB data, 1013 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s; 48/318 objects degraded (15.094%) 2024-07-16T10:01:55.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:01:55 smithi156 ceph-mon[93066]: Health check failed: insufficient standby MDS daemons available (MDS_INSUFFICIENT_STANDBY) 2024-07-16T10:01:55.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:01:55 smithi156 ceph-mon[93066]: osdmap e80: 6 total, 6 up, 6 in 2024-07-16T10:01:55.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:01:55 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "0"}]': finished 2024-07-16T10:01:55.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:01:55 smithi156 ceph-mon[93066]: fsmap cephfs:2 {0=cephfs.smithi156.rdibys=up:active,1=cephfs.smithi176.dvvbud=up:active} 2024-07-16T10:01:55.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:01:55 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-16T10:01:55.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:01:55 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-16T10:01:55.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:01:55 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-16T10:01:55.926 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:01:55 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:01:55.926 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:01:55 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-16T10:01:55.926 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:01:55 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:01:55.926 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:01:55 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:01:55.926 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:01:55 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:01:55.926 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:01:55 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:01:55.926 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:01:55 smithi156 ceph-mon[93066]: Upgrade: Scaling down filesystem cephfs 2024-07-16T10:01:55.926 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:01:55 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:01:55.926 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:01:55 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"}]: dispatch 2024-07-16T10:01:56.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:01:55 smithi176 ceph-mon[81689]: pgmap v156: 65 pgs: 13 active+undersized, 13 active+undersized+degraded, 39 active+clean; 253 MiB data, 1013 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s; 48/318 objects degraded (15.094%) 2024-07-16T10:01:56.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:01:55 smithi176 ceph-mon[81689]: Health check failed: insufficient standby MDS daemons available (MDS_INSUFFICIENT_STANDBY) 2024-07-16T10:01:56.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:01:55 smithi176 ceph-mon[81689]: osdmap e80: 6 total, 6 up, 6 in 2024-07-16T10:01:56.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:01:55 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "0"}]': finished 2024-07-16T10:01:56.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:01:55 smithi176 ceph-mon[81689]: fsmap cephfs:2 {0=cephfs.smithi156.rdibys=up:active,1=cephfs.smithi176.dvvbud=up:active} 2024-07-16T10:01:56.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:01:55 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-16T10:01:56.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:01:55 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-16T10:01:56.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:01:55 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-16T10:01:56.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:01:55 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:01:56.095 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:01:55 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-16T10:01:56.095 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:01:55 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:01:56.095 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:01:55 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:01:56.095 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:01:55 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:01:56.095 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:01:55 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:01:56.095 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:01:55 smithi176 ceph-mon[81689]: Upgrade: Scaling down filesystem cephfs 2024-07-16T10:01:56.095 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:01:55 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:01:56.095 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:01:55 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"}]: dispatch 2024-07-16T10:01:57.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:01:56 smithi176 ceph-mon[81689]: Health check cleared: MDS_INSUFFICIENT_STANDBY (was: insufficient standby MDS daemons available) 2024-07-16T10:01:57.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:01:56 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"}]': finished 2024-07-16T10:01:57.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:01:56 smithi176 ceph-mon[81689]: mds.? [v2:172.21.15.176:6826/3942496125,v1:172.21.15.176:6827/3942496125] up:boot 2024-07-16T10:01:57.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:01:56 smithi176 ceph-mon[81689]: mds.? [v2:172.21.15.156:6828/333092891,v1:172.21.15.156:6829/333092891] up:boot 2024-07-16T10:01:57.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:01:56 smithi176 ceph-mon[81689]: stopping daemon mds.cephfs.smithi176.dvvbud 2024-07-16T10:01:57.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:01:56 smithi176 ceph-mon[81689]: fsmap cephfs:2 {0=cephfs.smithi156.rdibys=up:active,1=cephfs.smithi176.dvvbud=up:active} 2 up:standby 2024-07-16T10:01:57.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:01:56 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi156.evwwaz"}]: dispatch 2024-07-16T10:01:57.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:01:56 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi176.wahbsi"}]: dispatch 2024-07-16T10:01:57.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:01:56 smithi176 ceph-mon[81689]: fsmap cephfs:2 {0=cephfs.smithi156.rdibys=up:active,1=cephfs.smithi176.dvvbud=up:stopping} 2 up:standby 2024-07-16T10:01:57.095 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:01:56 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-16T10:01:57.095 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:01:56 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-16T10:01:57.095 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:01:56 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-16T10:01:57.095 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:01:56 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:01:57.095 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:01:56 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-16T10:01:57.095 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:01:56 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:01:57.095 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:01:56 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:01:57.095 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:01:56 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:01:57.095 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:01:56 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:01:57.095 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:01:56 smithi176 ceph-mon[81689]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2024-07-16T10:01:57.174 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:01:56 smithi156 ceph-mon[93066]: Health check cleared: MDS_INSUFFICIENT_STANDBY (was: insufficient standby MDS daemons available) 2024-07-16T10:01:57.175 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:01:56 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"}]': finished 2024-07-16T10:01:57.175 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:01:56 smithi156 ceph-mon[93066]: mds.? [v2:172.21.15.176:6826/3942496125,v1:172.21.15.176:6827/3942496125] up:boot 2024-07-16T10:01:57.175 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:01:56 smithi156 ceph-mon[93066]: mds.? [v2:172.21.15.156:6828/333092891,v1:172.21.15.156:6829/333092891] up:boot 2024-07-16T10:01:57.175 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:01:56 smithi156 ceph-mon[93066]: stopping daemon mds.cephfs.smithi176.dvvbud 2024-07-16T10:01:57.175 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:01:56 smithi156 ceph-mon[93066]: fsmap cephfs:2 {0=cephfs.smithi156.rdibys=up:active,1=cephfs.smithi176.dvvbud=up:active} 2 up:standby 2024-07-16T10:01:57.175 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:01:56 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi156.evwwaz"}]: dispatch 2024-07-16T10:01:57.175 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:01:56 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi176.wahbsi"}]: dispatch 2024-07-16T10:01:57.175 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:01:56 smithi156 ceph-mon[93066]: fsmap cephfs:2 {0=cephfs.smithi156.rdibys=up:active,1=cephfs.smithi176.dvvbud=up:stopping} 2 up:standby 2024-07-16T10:01:57.175 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:01:56 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-16T10:01:57.175 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:01:56 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-16T10:01:57.175 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:01:56 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-16T10:01:57.176 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:01:56 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:01:57.176 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:01:56 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-16T10:01:57.176 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:01:56 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:01:57.176 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:01:56 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:01:57.176 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:01:56 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:01:57.176 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:01:56 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:01:57.176 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:01:56 smithi156 ceph-mon[93066]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2024-07-16T10:01:58.093 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:01:57 smithi176 ceph-mon[81689]: pgmap v158: 65 pgs: 9 active+undersized, 5 active+undersized+degraded, 51 active+clean; 253 MiB data, 1013 MiB used, 535 GiB / 536 GiB avail; 638 B/s rd, 2 op/s; 14/318 objects degraded (4.403%) 2024-07-16T10:01:58.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:01:57 smithi176 ceph-mon[81689]: Health check update: Degraded data redundancy: 14/318 objects degraded (4.403%), 5 pgs degraded (PG_DEGRADED) 2024-07-16T10:01:58.174 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:01:57 smithi156 ceph-mon[93066]: pgmap v158: 65 pgs: 9 active+undersized, 5 active+undersized+degraded, 51 active+clean; 253 MiB data, 1013 MiB used, 535 GiB / 536 GiB avail; 638 B/s rd, 2 op/s; 14/318 objects degraded (4.403%) 2024-07-16T10:01:58.174 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:01:57 smithi156 ceph-mon[93066]: Health check update: Degraded data redundancy: 14/318 objects degraded (4.403%), 5 pgs degraded (PG_DEGRADED) 2024-07-16T10:01:59.093 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:01:58 smithi176 ceph-mon[81689]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 14/318 objects degraded (4.403%), 5 pgs degraded) 2024-07-16T10:01:59.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:01:58 smithi176 ceph-mon[81689]: Cluster is now healthy 2024-07-16T10:01:59.174 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:01:58 smithi156 ceph-mon[93066]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 14/318 objects degraded (4.403%), 5 pgs degraded) 2024-07-16T10:01:59.174 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:01:58 smithi156 ceph-mon[93066]: Cluster is now healthy 2024-07-16T10:02:00.093 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:01:59 smithi176 ceph-mon[81689]: pgmap v159: 65 pgs: 65 active+clean; 253 MiB data, 1013 MiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 0 op/s 2024-07-16T10:02:00.174 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:01:59 smithi156 ceph-mon[93066]: pgmap v159: 65 pgs: 65 active+clean; 253 MiB data, 1013 MiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 0 op/s 2024-07-16T10:02:02.093 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:02:01 smithi176 ceph-mon[81689]: pgmap v160: 65 pgs: 65 active+clean; 253 MiB data, 1013 MiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 917 B/s wr, 1 op/s 2024-07-16T10:02:02.174 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:02:01 smithi156 ceph-mon[93066]: pgmap v160: 65 pgs: 65 active+clean; 253 MiB data, 1013 MiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 917 B/s wr, 1 op/s 2024-07-16T10:02:03.093 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:02:02 smithi176 ceph-mon[81689]: fsmap cephfs:2 {0=cephfs.smithi156.rdibys=up:active,1=cephfs.smithi176.dvvbud=up:stopping} 2 up:standby 2024-07-16T10:02:03.174 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:02:02 smithi156 ceph-mon[93066]: fsmap cephfs:2 {0=cephfs.smithi156.rdibys=up:active,1=cephfs.smithi176.dvvbud=up:stopping} 2 up:standby 2024-07-16T10:02:04.093 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:02:03 smithi176 ceph-mon[81689]: pgmap v161: 65 pgs: 65 active+clean; 253 MiB data, 1013 MiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 1.4 KiB/s wr, 1 op/s 2024-07-16T10:02:04.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:02:03 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:02:04.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:02:03 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-16T10:02:04.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:02:03 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:02:04.174 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:02:03 smithi156 ceph-mon[93066]: pgmap v161: 65 pgs: 65 active+clean; 253 MiB data, 1013 MiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 1.4 KiB/s wr, 1 op/s 2024-07-16T10:02:04.175 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:02:03 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:02:04.175 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:02:03 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-16T10:02:04.175 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:02:03 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:02:06.093 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:02:05 smithi176 ceph-mon[81689]: pgmap v162: 65 pgs: 65 active+clean; 253 MiB data, 1013 MiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 1.3 KiB/s wr, 1 op/s 2024-07-16T10:02:06.174 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:02:05 smithi156 ceph-mon[93066]: pgmap v162: 65 pgs: 65 active+clean; 253 MiB data, 1013 MiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 1.3 KiB/s wr, 1 op/s 2024-07-16T10:02:06.565 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:06.563+0000 7ff0b6ef2640 1 -- 172.21.15.156:0/897038558 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff0b00fb260 msgr2=0x7ff0b00fb670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:02:06.565 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:06.563+0000 7ff0b6ef2640 1 --2- 172.21.15.156:0/897038558 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff0b00fb260 0x7ff0b00fb670 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7ff0a00048d0 tx=0x7ff0a0009da0 comp rx=0 tx=0).stop 2024-07-16T10:02:06.565 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:06.563+0000 7ff0b6ef2640 1 -- 172.21.15.156:0/897038558 shutdown_connections 2024-07-16T10:02:06.565 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:06.563+0000 7ff0b6ef2640 1 --2- 172.21.15.156:0/897038558 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff0b00fc530 0x7ff0b00fc9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:02:06.565 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:06.563+0000 7ff0b6ef2640 1 --2- 172.21.15.156:0/897038558 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff0b00fb260 0x7ff0b00fb670 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:02:06.566 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:06.563+0000 7ff0b6ef2640 1 -- 172.21.15.156:0/897038558 >> 172.21.15.156:0/897038558 conn(0x7ff0b00f6a20 msgr2=0x7ff0b00f8e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T10:02:06.566 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:06.563+0000 7ff0b6ef2640 1 -- 172.21.15.156:0/897038558 shutdown_connections 2024-07-16T10:02:06.566 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:06.563+0000 7ff0b6ef2640 1 -- 172.21.15.156:0/897038558 wait complete. 2024-07-16T10:02:06.566 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:06.564+0000 7ff0b6ef2640 1 Processor -- start 2024-07-16T10:02:06.566 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:06.564+0000 7ff0b6ef2640 1 -- start start 2024-07-16T10:02:06.567 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:06.564+0000 7ff0b6ef2640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff0b00fb260 0x7ff0b00ff380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:02:06.568 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:06.564+0000 7ff0b6ef2640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff0b00fc530 0x7ff0b00fd870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:02:06.568 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:06.564+0000 7ff0b6ef2640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0b00fde00 con 0x7ff0b00fb260 2024-07-16T10:02:06.568 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:06.564+0000 7ff0b6ef2640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0b00fdf40 con 0x7ff0b00fc530 2024-07-16T10:02:06.568 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:06.564+0000 7ff0b4c8e640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff0b00fb260 0x7ff0b00ff380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:02:06.568 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:06.564+0000 7ff0b4c8e640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff0b00fb260 0x7ff0b00ff380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:47816/0 (socket says 172.21.15.156:47816) 2024-07-16T10:02:06.568 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:06.564+0000 7ff0b4c8e640 1 -- 172.21.15.156:0/129887312 learned_addr learned my addr 172.21.15.156:0/129887312 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T10:02:06.568 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:06.564+0000 7ff0affff640 1 --2- 172.21.15.156:0/129887312 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff0b00fc530 0x7ff0b00fd870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:02:06.568 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:06.565+0000 7ff0b4c8e640 1 -- 172.21.15.156:0/129887312 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff0b00fc530 msgr2=0x7ff0b00fd870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:02:06.568 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:06.565+0000 7ff0b4c8e640 1 --2- 172.21.15.156:0/129887312 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff0b00fc530 0x7ff0b00fd870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:02:06.568 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:06.565+0000 7ff0b4c8e640 1 -- 172.21.15.156:0/129887312 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0a0005040 con 0x7ff0b00fb260 2024-07-16T10:02:06.568 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:06.565+0000 7ff0b4c8e640 1 --2- 172.21.15.156:0/129887312 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff0b00fb260 0x7ff0b00ff380 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7ff0a00048a0 tx=0x7ff0a0009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T10:02:06.569 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:06.565+0000 7ff0ad7fa640 1 -- 172.21.15.156:0/129887312 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff0a0038070 con 0x7ff0b00fb260 2024-07-16T10:02:06.569 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:06.565+0000 7ff0ad7fa640 1 -- 172.21.15.156:0/129887312 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff0a00092b0 con 0x7ff0b00fb260 2024-07-16T10:02:06.569 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:06.565+0000 7ff0b6ef2640 1 -- 172.21.15.156:0/129887312 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0b00fe1c0 con 0x7ff0b00fb260 2024-07-16T10:02:06.569 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:06.565+0000 7ff0ad7fa640 1 -- 172.21.15.156:0/129887312 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff0a0033820 con 0x7ff0b00fb260 2024-07-16T10:02:06.569 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:06.565+0000 7ff0b6ef2640 1 -- 172.21.15.156:0/129887312 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0b0115040 con 0x7ff0b00fb260 2024-07-16T10:02:06.569 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:06.566+0000 7ff0b6ef2640 1 -- 172.21.15.156:0/129887312 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0b00fa6c0 con 0x7ff0b00fb260 2024-07-16T10:02:06.570 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:06.567+0000 7ff0ad7fa640 1 -- 172.21.15.156:0/129887312 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 38) v1 ==== 92205+0+0 (secure 0 0 0) 0x7ff0a0030020 con 0x7ff0b00fb260 2024-07-16T10:02:06.570 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:06.567+0000 7ff0ad7fa640 1 --2- 172.21.15.156:0/129887312 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7ff09c06e0b0 0x7ff09c070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:02:06.570 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:06.567+0000 7ff0ad7fa640 1 -- 172.21.15.156:0/129887312 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6410+0+0 (secure 0 0 0) 0x7ff0a00ad880 con 0x7ff0b00fb260 2024-07-16T10:02:06.570 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:06.567+0000 7ff0affff640 1 --2- 172.21.15.156:0/129887312 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7ff09c06e0b0 0x7ff09c070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:02:06.570 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:06.568+0000 7ff0affff640 1 --2- 172.21.15.156:0/129887312 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7ff09c06e0b0 0x7ff09c070560 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7ff098009af0 tx=0x7ff09800b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T10:02:06.573 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:06.572+0000 7ff0ad7fa640 1 -- 172.21.15.156:0/129887312 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187716 (secure 0 0 0) 0x7ff0a0076cb0 con 0x7ff0b00fb260 2024-07-16T10:02:06.822 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:06.821+0000 7ff0b6ef2640 1 -- 172.21.15.156:0/129887312 --> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff0b00008d0 con 0x7ff09c06e0b0 2024-07-16T10:02:06.823 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:02:06 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-16T10:02:06.823 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:02:06 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-16T10:02:06.823 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:02:06 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-16T10:02:06.823 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:02:06 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:02:06.823 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:02:06 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-16T10:02:06.823 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:02:06 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:02:06.823 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:02:06 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:02:06.823 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:02:06 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:02:06.823 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:02:06 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:02:06.824 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:02:06 smithi156 ceph-mon[93066]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2024-07-16T10:02:06.824 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:06.822+0000 7ff0ad7fa640 1 -- 172.21.15.156:0/129887312 <== mgr.34104 v2:172.21.15.156:6800/3367765292 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7ff0b00008d0 con 0x7ff09c06e0b0 2024-07-16T10:02:06.826 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:06.825+0000 7ff0b6ef2640 1 -- 172.21.15.156:0/129887312 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7ff09c06e0b0 msgr2=0x7ff09c070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:02:06.826 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:06.825+0000 7ff0b6ef2640 1 --2- 172.21.15.156:0/129887312 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7ff09c06e0b0 0x7ff09c070560 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7ff098009af0 tx=0x7ff09800b040 comp rx=0 tx=0).stop 2024-07-16T10:02:06.826 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:06.825+0000 7ff0b6ef2640 1 -- 172.21.15.156:0/129887312 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff0b00fb260 msgr2=0x7ff0b00ff380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:02:06.827 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:06.825+0000 7ff0b6ef2640 1 --2- 172.21.15.156:0/129887312 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff0b00fb260 0x7ff0b00ff380 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7ff0a00048a0 tx=0x7ff0a0009da0 comp rx=0 tx=0).stop 2024-07-16T10:02:06.827 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:06.825+0000 7ff0b6ef2640 1 -- 172.21.15.156:0/129887312 shutdown_connections 2024-07-16T10:02:06.827 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:06.825+0000 7ff0b6ef2640 1 --2- 172.21.15.156:0/129887312 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7ff09c06e0b0 0x7ff09c070560 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:02:06.827 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:06.825+0000 7ff0b6ef2640 1 --2- 172.21.15.156:0/129887312 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff0b00fc530 0x7ff0b00fd870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:02:06.827 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:06.825+0000 7ff0b6ef2640 1 --2- 172.21.15.156:0/129887312 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff0b00fb260 0x7ff0b00ff380 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:02:06.827 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:06.825+0000 7ff0b6ef2640 1 -- 172.21.15.156:0/129887312 >> 172.21.15.156:0/129887312 conn(0x7ff0b00f6a20 msgr2=0x7ff0b00f8150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T10:02:06.827 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:06.825+0000 7ff0b6ef2640 1 -- 172.21.15.156:0/129887312 shutdown_connections 2024-07-16T10:02:06.827 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:06.825+0000 7ff0b6ef2640 1 -- 172.21.15.156:0/129887312 wait complete. 2024-07-16T10:02:06.835 INFO:teuthology.orchestra.run.smithi156.stdout:true 2024-07-16T10:02:06.910 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:06.907+0000 7f2781896640 1 -- 172.21.15.156:0/2036933761 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f277c0fc510 msgr2=0x7f277c0fc980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:02:06.910 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:06.907+0000 7f2781896640 1 --2- 172.21.15.156:0/2036933761 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f277c0fc510 0x7f277c0fc980 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f27640048d0 tx=0x7f2764009da0 comp rx=0 tx=0).stop 2024-07-16T10:02:06.911 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:06.907+0000 7f2781896640 1 -- 172.21.15.156:0/2036933761 shutdown_connections 2024-07-16T10:02:06.911 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:06.907+0000 7f2781896640 1 --2- 172.21.15.156:0/2036933761 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f277c0fc510 0x7f277c0fc980 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:02:06.911 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:06.907+0000 7f2781896640 1 --2- 172.21.15.156:0/2036933761 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f277c0fb240 0x7f277c0fb650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:02:06.911 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:06.907+0000 7f2781896640 1 -- 172.21.15.156:0/2036933761 >> 172.21.15.156:0/2036933761 conn(0x7f277c0f6a20 msgr2=0x7f277c0f8e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T10:02:06.911 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:06.908+0000 7f2781896640 1 -- 172.21.15.156:0/2036933761 shutdown_connections 2024-07-16T10:02:06.911 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:06.908+0000 7f2781896640 1 -- 172.21.15.156:0/2036933761 wait complete. 2024-07-16T10:02:06.911 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:06.908+0000 7f2781896640 1 Processor -- start 2024-07-16T10:02:06.911 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:06.908+0000 7f2781896640 1 -- start start 2024-07-16T10:02:06.911 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:06.908+0000 7f2781896640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f277c0fb240 0x7f277c10bef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:02:06.911 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:06.908+0000 7f2781896640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f277c0fc510 0x7f277c10c430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:02:06.911 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:06.908+0000 7f2781896640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f277c10ca80 con 0x7f277c0fc510 2024-07-16T10:02:06.912 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:06.908+0000 7f2781896640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f277c10d1c0 con 0x7f277c0fb240 2024-07-16T10:02:06.912 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:06.908+0000 7f2773fff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f277c0fc510 0x7f277c10c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:02:06.912 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:06.908+0000 7f2773fff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f277c0fc510 0x7f277c10c430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:47828/0 (socket says 172.21.15.156:47828) 2024-07-16T10:02:06.912 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:06.908+0000 7f277affd640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f277c0fb240 0x7f277c10bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:02:06.912 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:06.908+0000 7f2773fff640 1 -- 172.21.15.156:0/3180656405 learned_addr learned my addr 172.21.15.156:0/3180656405 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T10:02:06.912 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:06.909+0000 7f2773fff640 1 -- 172.21.15.156:0/3180656405 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f277c0fb240 msgr2=0x7f277c10bef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:02:06.912 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:06.909+0000 7f2773fff640 1 --2- 172.21.15.156:0/3180656405 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f277c0fb240 0x7f277c10bef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:02:06.912 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:06.909+0000 7f2773fff640 1 -- 172.21.15.156:0/3180656405 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2764005040 con 0x7f277c0fc510 2024-07-16T10:02:06.912 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:06.909+0000 7f277affd640 1 --2- 172.21.15.156:0/3180656405 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f277c0fb240 0x7f277c10bef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-16T10:02:06.912 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:06.909+0000 7f2773fff640 1 --2- 172.21.15.156:0/3180656405 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f277c0fc510 0x7f277c10c430 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f27640048a0 tx=0x7f2764009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T10:02:06.912 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:06.909+0000 7f27737fe640 1 -- 172.21.15.156:0/3180656405 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f276403a070 con 0x7f277c0fc510 2024-07-16T10:02:06.913 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:06.909+0000 7f27737fe640 1 -- 172.21.15.156:0/3180656405 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f27640092b0 con 0x7f277c0fc510 2024-07-16T10:02:06.913 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:06.909+0000 7f2781896640 1 -- 172.21.15.156:0/3180656405 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f277c10d3f0 con 0x7f277c0fc510 2024-07-16T10:02:06.913 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:06.909+0000 7f27737fe640 1 -- 172.21.15.156:0/3180656405 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f27640329a0 con 0x7f277c0fc510 2024-07-16T10:02:06.913 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:06.909+0000 7f2781896640 1 -- 172.21.15.156:0/3180656405 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f277c10d930 con 0x7f277c0fc510 2024-07-16T10:02:06.914 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:06.910+0000 7f2781896640 1 -- 172.21.15.156:0/3180656405 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f277c0fa6a0 con 0x7f277c0fc510 2024-07-16T10:02:06.914 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:06.911+0000 7f27737fe640 1 -- 172.21.15.156:0/3180656405 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 38) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f2764030020 con 0x7f277c0fc510 2024-07-16T10:02:06.914 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:06.911+0000 7f27737fe640 1 --2- 172.21.15.156:0/3180656405 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f276806e0b0 0x7f2768070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:02:06.914 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:06.911+0000 7f27737fe640 1 -- 172.21.15.156:0/3180656405 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f27640afaf0 con 0x7f277c0fc510 2024-07-16T10:02:06.914 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:06.911+0000 7f277affd640 1 --2- 172.21.15.156:0/3180656405 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f276806e0b0 0x7f2768070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:02:06.914 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:06.912+0000 7f277affd640 1 --2- 172.21.15.156:0/3180656405 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f276806e0b0 0x7f2768070560 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f277c10bc80 tx=0x7f276c00b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T10:02:06.916 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:06.915+0000 7f27737fe640 1 -- 172.21.15.156:0/3180656405 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187716 (secure 0 0 0) 0x7f2764078f20 con 0x7f277c0fc510 2024-07-16T10:02:07.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:02:06 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-16T10:02:07.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:02:06 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-16T10:02:07.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:02:06 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-16T10:02:07.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:02:06 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:02:07.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:02:06 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-16T10:02:07.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:02:06 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:02:07.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:02:06 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:02:07.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:02:06 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:02:07.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:02:06 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:02:07.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:02:06 smithi176 ceph-mon[81689]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2024-07-16T10:02:07.107 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:07.106+0000 7f2781896640 1 -- 172.21.15.156:0/3180656405 --> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f277c10dd80 con 0x7f276806e0b0 2024-07-16T10:02:07.108 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:07.107+0000 7f27737fe640 1 -- 172.21.15.156:0/3180656405 <== mgr.34104 v2:172.21.15.156:6800/3367765292 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f277c10dd80 con 0x7f276806e0b0 2024-07-16T10:02:07.112 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:07.110+0000 7f2781896640 1 -- 172.21.15.156:0/3180656405 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f276806e0b0 msgr2=0x7f2768070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:02:07.112 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:07.110+0000 7f2781896640 1 --2- 172.21.15.156:0/3180656405 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f276806e0b0 0x7f2768070560 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f277c10bc80 tx=0x7f276c00b040 comp rx=0 tx=0).stop 2024-07-16T10:02:07.112 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:07.110+0000 7f2781896640 1 -- 172.21.15.156:0/3180656405 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f277c0fc510 msgr2=0x7f277c10c430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:02:07.112 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:07.110+0000 7f2781896640 1 --2- 172.21.15.156:0/3180656405 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f277c0fc510 0x7f277c10c430 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f27640048a0 tx=0x7f2764009da0 comp rx=0 tx=0).stop 2024-07-16T10:02:07.112 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:07.110+0000 7f2781896640 1 -- 172.21.15.156:0/3180656405 shutdown_connections 2024-07-16T10:02:07.112 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:07.110+0000 7f2781896640 1 --2- 172.21.15.156:0/3180656405 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f276806e0b0 0x7f2768070560 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:02:07.112 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:07.110+0000 7f2781896640 1 --2- 172.21.15.156:0/3180656405 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f277c0fc510 0x7f277c10c430 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:02:07.112 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:07.110+0000 7f2781896640 1 --2- 172.21.15.156:0/3180656405 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f277c0fb240 0x7f277c10bef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:02:07.112 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:07.110+0000 7f2781896640 1 -- 172.21.15.156:0/3180656405 >> 172.21.15.156:0/3180656405 conn(0x7f277c0f6a20 msgr2=0x7f277c0f8140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T10:02:07.113 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:07.110+0000 7f2781896640 1 -- 172.21.15.156:0/3180656405 shutdown_connections 2024-07-16T10:02:07.113 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:07.110+0000 7f2781896640 1 -- 172.21.15.156:0/3180656405 wait complete. 2024-07-16T10:02:07.193 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:07.190+0000 7f52c5051640 1 -- 172.21.15.156:0/3478648894 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f52c00ff530 msgr2=0x7f52c0101910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:02:07.193 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:07.190+0000 7f52c5051640 1 --2- 172.21.15.156:0/3478648894 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f52c00ff530 0x7f52c0101910 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f52a80048d0 tx=0x7f52a8009da0 comp rx=0 tx=0).stop 2024-07-16T10:02:07.193 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:07.190+0000 7f52c5051640 1 -- 172.21.15.156:0/3478648894 shutdown_connections 2024-07-16T10:02:07.193 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:07.190+0000 7f52c5051640 1 --2- 172.21.15.156:0/3478648894 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f52c00ff530 0x7f52c0101910 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:02:07.193 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:07.190+0000 7f52c5051640 1 --2- 172.21.15.156:0/3478648894 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f52c00664c0 0x7f52c00feff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:02:07.193 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:07.190+0000 7f52c5051640 1 -- 172.21.15.156:0/3478648894 >> 172.21.15.156:0/3478648894 conn(0x7f52c00f6a20 msgr2=0x7f52c00f8e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T10:02:07.193 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:07.190+0000 7f52c5051640 1 -- 172.21.15.156:0/3478648894 shutdown_connections 2024-07-16T10:02:07.193 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:07.190+0000 7f52c5051640 1 -- 172.21.15.156:0/3478648894 wait complete. 2024-07-16T10:02:07.194 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:07.190+0000 7f52c5051640 1 Processor -- start 2024-07-16T10:02:07.194 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:07.190+0000 7f52c5051640 1 -- start start 2024-07-16T10:02:07.194 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:07.191+0000 7f52c5051640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f52c00664c0 0x7f52c010cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:02:07.194 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:07.191+0000 7f52c5051640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f52c00ff530 0x7f52c010d500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:02:07.194 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:07.191+0000 7f52c5051640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52c010db50 con 0x7f52c00664c0 2024-07-16T10:02:07.194 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:07.191+0000 7f52c5051640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52c010e370 con 0x7f52c00ff530 2024-07-16T10:02:07.194 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:07.191+0000 7f52bed9d640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f52c00664c0 0x7f52c010cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:02:07.194 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:07.191+0000 7f52bed9d640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f52c00664c0 0x7f52c010cfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:47852/0 (socket says 172.21.15.156:47852) 2024-07-16T10:02:07.194 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:07.191+0000 7f52bed9d640 1 -- 172.21.15.156:0/2559305129 learned_addr learned my addr 172.21.15.156:0/2559305129 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T10:02:07.194 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:07.191+0000 7f52b7fff640 1 --2- 172.21.15.156:0/2559305129 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f52c00ff530 0x7f52c010d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:02:07.195 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:07.191+0000 7f52bed9d640 1 -- 172.21.15.156:0/2559305129 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f52c00ff530 msgr2=0x7f52c010d500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:02:07.195 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:07.191+0000 7f52bed9d640 1 --2- 172.21.15.156:0/2559305129 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f52c00ff530 0x7f52c010d500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:02:07.195 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:07.191+0000 7f52bed9d640 1 -- 172.21.15.156:0/2559305129 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52a8005040 con 0x7f52c00664c0 2024-07-16T10:02:07.195 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:07.191+0000 7f52bed9d640 1 --2- 172.21.15.156:0/2559305129 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f52c00664c0 0x7f52c010cfc0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f52b000c810 tx=0x7f52b00043c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T10:02:07.195 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:07.191+0000 7f52b77fe640 1 -- 172.21.15.156:0/2559305129 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f52b000a7a0 con 0x7f52c00664c0 2024-07-16T10:02:07.195 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:07.191+0000 7f52b77fe640 1 -- 172.21.15.156:0/2559305129 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f52b000add0 con 0x7f52c00664c0 2024-07-16T10:02:07.195 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:07.191+0000 7f52b77fe640 1 -- 172.21.15.156:0/2559305129 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f52b00115c0 con 0x7f52c00664c0 2024-07-16T10:02:07.195 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:07.191+0000 7f52c5051640 1 -- 172.21.15.156:0/2559305129 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52c010e600 con 0x7f52c00664c0 2024-07-16T10:02:07.195 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:07.192+0000 7f52c5051640 1 -- 172.21.15.156:0/2559305129 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52c010eba0 con 0x7f52c00664c0 2024-07-16T10:02:07.196 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:07.192+0000 7f52c5051640 1 -- 172.21.15.156:0/2559305129 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52c00fa6c0 con 0x7f52c00664c0 2024-07-16T10:02:07.196 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:07.193+0000 7f52b77fe640 1 -- 172.21.15.156:0/2559305129 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 38) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f52b0012420 con 0x7f52c00664c0 2024-07-16T10:02:07.196 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:07.193+0000 7f52b77fe640 1 --2- 172.21.15.156:0/2559305129 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f52ac06e0b0 0x7f52ac070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:02:07.196 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:07.194+0000 7f52b77fe640 1 -- 172.21.15.156:0/2559305129 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f52b008e1e0 con 0x7f52c00664c0 2024-07-16T10:02:07.196 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:07.194+0000 7f52b7fff640 1 --2- 172.21.15.156:0/2559305129 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f52ac06e0b0 0x7f52ac070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:02:07.196 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:07.194+0000 7f52b7fff640 1 --2- 172.21.15.156:0/2559305129 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f52ac06e0b0 0x7f52ac070560 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f52c010cd70 tx=0x7f52a8035040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T10:02:07.199 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:07.198+0000 7f52b77fe640 1 -- 172.21.15.156:0/2559305129 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187716 (secure 0 0 0) 0x7f52b0057560 con 0x7f52c00664c0 2024-07-16T10:02:07.384 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:07.383+0000 7f52c5051640 1 -- 172.21.15.156:0/2559305129 --> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f52c010ef90 con 0x7f52ac06e0b0 2024-07-16T10:02:07.394 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:07.393+0000 7f52b77fe640 1 -- 172.21.15.156:0/2559305129 <== mgr.34104 v2:172.21.15.156:6800/3367765292 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7f52c010ef90 con 0x7f52ac06e0b0 2024-07-16T10:02:07.395 INFO:teuthology.orchestra.run.smithi156.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-07-16T10:02:07.395 INFO:teuthology.orchestra.run.smithi156.stdout:alertmanager.smithi156 smithi156 *:9093,9094 running (5m) 113s ago 14m 25.0M - 0.25.0 c8568f914cd2 d524a2c3ff15 2024-07-16T10:02:07.395 INFO:teuthology.orchestra.run.smithi156.stdout:crash.smithi156 smithi156 running (3m) 113s ago 14m 6899k - 19.0.0-3495-ga6ca461b 16f7feb04342 11648cf29af4 2024-07-16T10:02:07.395 INFO:teuthology.orchestra.run.smithi156.stdout:crash.smithi176 smithi176 running (3m) 18s ago 13m 6899k - 19.0.0-3495-ga6ca461b 16f7feb04342 67f3ccc034f8 2024-07-16T10:02:07.395 INFO:teuthology.orchestra.run.smithi156.stdout:grafana.smithi156 smithi156 *:3000 running (5m) 113s ago 13m 77.8M - 9.4.7 954c08fa6188 deb9e1bfeb66 2024-07-16T10:02:07.395 INFO:teuthology.orchestra.run.smithi156.stdout:mds.cephfs.smithi156.evwwaz smithi156 running (9m) 113s ago 9m 67.2M - 17.2.7-1104-ge6231b04 d5de5f953b95 7c0ff9ae43f8 2024-07-16T10:02:07.395 INFO:teuthology.orchestra.run.smithi156.stdout:mds.cephfs.smithi156.rdibys smithi156 running (9m) 113s ago 9m 120M - 17.2.7-1104-ge6231b04 d5de5f953b95 b985cd1e6924 2024-07-16T10:02:07.395 INFO:teuthology.orchestra.run.smithi156.stdout:mds.cephfs.smithi176.dvvbud smithi176 running (9m) 18s ago 9m 108M - 17.2.7-1104-ge6231b04 d5de5f953b95 1b00147950f9 2024-07-16T10:02:07.395 INFO:teuthology.orchestra.run.smithi156.stdout:mds.cephfs.smithi176.wahbsi smithi176 running (9m) 18s ago 9m 62.3M - 17.2.7-1104-ge6231b04 d5de5f953b95 866664af2311 2024-07-16T10:02:07.395 INFO:teuthology.orchestra.run.smithi156.stdout:mgr.smithi156.zkvylv smithi156 *:8443,9283,8765 running (5m) 113s ago 15m 497M - 19.0.0-3495-ga6ca461b 16f7feb04342 ee53bd604b7c 2024-07-16T10:02:07.395 INFO:teuthology.orchestra.run.smithi156.stdout:mgr.smithi176.dqbrvx smithi176 *:8443,9283,8765 running (4m) 18s ago 13m 441M - 19.0.0-3495-ga6ca461b 16f7feb04342 ce9eaae4c2d0 2024-07-16T10:02:07.395 INFO:teuthology.orchestra.run.smithi156.stdout:mon.smithi156 smithi156 running (4m) 113s ago 15m 45.4M 2048M 19.0.0-3495-ga6ca461b 16f7feb04342 77b6ec8c50eb 2024-07-16T10:02:07.396 INFO:teuthology.orchestra.run.smithi156.stdout:mon.smithi176 smithi176 running (4m) 18s ago 13m 40.4M 2048M 19.0.0-3495-ga6ca461b 16f7feb04342 ef92814cec2b 2024-07-16T10:02:07.396 INFO:teuthology.orchestra.run.smithi156.stdout:node-exporter.smithi156 smithi156 *:9100 running (6m) 113s ago 14m 17.2M - 1.5.0 0da6a335fe13 e5d86f23a14c 2024-07-16T10:02:07.396 INFO:teuthology.orchestra.run.smithi156.stdout:node-exporter.smithi176 smithi176 *:9100 running (5m) 18s ago 13m 15.6M - 1.5.0 0da6a335fe13 ea842728144a 2024-07-16T10:02:07.396 INFO:teuthology.orchestra.run.smithi156.stdout:osd.0 smithi156 running (3m) 113s ago 12m 93.7M 1557M 19.0.0-3495-ga6ca461b 16f7feb04342 8633df1689cc 2024-07-16T10:02:07.396 INFO:teuthology.orchestra.run.smithi156.stdout:osd.1 smithi156 running (2m) 113s ago 12m 101M 1557M 19.0.0-3495-ga6ca461b 16f7feb04342 e6fae9653b46 2024-07-16T10:02:07.396 INFO:teuthology.orchestra.run.smithi156.stdout:osd.2 smithi156 running (119s) 113s ago 11m 11.8M 1557M 19.0.0-3495-ga6ca461b 16f7feb04342 c6ecf18de9bf 2024-07-16T10:02:07.396 INFO:teuthology.orchestra.run.smithi156.stdout:osd.3 smithi176 running (84s) 18s ago 11m 99.7M 2581M 19.0.0-3495-ga6ca461b 16f7feb04342 d216534d6027 2024-07-16T10:02:07.396 INFO:teuthology.orchestra.run.smithi156.stdout:osd.4 smithi176 running (53s) 18s ago 10m 100M 2581M 19.0.0-3495-ga6ca461b 16f7feb04342 1bd3a25317b0 2024-07-16T10:02:07.396 INFO:teuthology.orchestra.run.smithi156.stdout:osd.5 smithi176 running (21s) 18s ago 10m 11.4M 2581M 19.0.0-3495-ga6ca461b 16f7feb04342 a03640221e7a 2024-07-16T10:02:07.396 INFO:teuthology.orchestra.run.smithi156.stdout:prometheus.smithi156 smithi156 *:9095 running (4m) 113s ago 13m 64.9M - 2.43.0 a07b618ecd1d 20c195ecd665 2024-07-16T10:02:07.397 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:07.395+0000 7f52c5051640 1 -- 172.21.15.156:0/2559305129 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f52ac06e0b0 msgr2=0x7f52ac070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:02:07.397 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:07.395+0000 7f52c5051640 1 --2- 172.21.15.156:0/2559305129 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f52ac06e0b0 0x7f52ac070560 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f52c010cd70 tx=0x7f52a8035040 comp rx=0 tx=0).stop 2024-07-16T10:02:07.397 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:07.395+0000 7f52c5051640 1 -- 172.21.15.156:0/2559305129 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f52c00664c0 msgr2=0x7f52c010cfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:02:07.397 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:07.395+0000 7f52c5051640 1 --2- 172.21.15.156:0/2559305129 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f52c00664c0 0x7f52c010cfc0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f52b000c810 tx=0x7f52b00043c0 comp rx=0 tx=0).stop 2024-07-16T10:02:07.397 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:07.395+0000 7f52c5051640 1 -- 172.21.15.156:0/2559305129 shutdown_connections 2024-07-16T10:02:07.398 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:07.395+0000 7f52c5051640 1 --2- 172.21.15.156:0/2559305129 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f52ac06e0b0 0x7f52ac070560 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:02:07.398 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:07.395+0000 7f52c5051640 1 --2- 172.21.15.156:0/2559305129 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f52c00ff530 0x7f52c010d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:02:07.398 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:07.395+0000 7f52c5051640 1 --2- 172.21.15.156:0/2559305129 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f52c00664c0 0x7f52c010cfc0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:02:07.398 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:07.395+0000 7f52c5051640 1 -- 172.21.15.156:0/2559305129 >> 172.21.15.156:0/2559305129 conn(0x7f52c00f6a20 msgr2=0x7f52c00f6e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T10:02:07.398 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:07.395+0000 7f52c5051640 1 -- 172.21.15.156:0/2559305129 shutdown_connections 2024-07-16T10:02:07.398 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:07.395+0000 7f52c5051640 1 -- 172.21.15.156:0/2559305129 wait complete. 2024-07-16T10:02:07.478 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:07.474+0000 7f9ac70ad640 1 -- 172.21.15.156:0/2473340426 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9ac00ff550 msgr2=0x7f9ac0101930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:02:07.478 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:07.474+0000 7f9ac70ad640 1 --2- 172.21.15.156:0/2473340426 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9ac00ff550 0x7f9ac0101930 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f9aa8009c50 tx=0x7f9aa8004d10 comp rx=0 tx=0).stop 2024-07-16T10:02:07.478 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:07.475+0000 7f9ac70ad640 1 -- 172.21.15.156:0/2473340426 shutdown_connections 2024-07-16T10:02:07.478 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:07.475+0000 7f9ac70ad640 1 --2- 172.21.15.156:0/2473340426 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9ac00ff550 0x7f9ac0101930 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:02:07.478 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:07.475+0000 7f9ac70ad640 1 --2- 172.21.15.156:0/2473340426 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9ac0065990 0x7f9ac00ff010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:02:07.478 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:07.475+0000 7f9ac70ad640 1 -- 172.21.15.156:0/2473340426 >> 172.21.15.156:0/2473340426 conn(0x7f9ac00f6a20 msgr2=0x7f9ac00f8e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T10:02:07.478 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:07.475+0000 7f9ac70ad640 1 -- 172.21.15.156:0/2473340426 shutdown_connections 2024-07-16T10:02:07.479 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:07.475+0000 7f9ac70ad640 1 -- 172.21.15.156:0/2473340426 wait complete. 2024-07-16T10:02:07.479 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:07.475+0000 7f9ac70ad640 1 Processor -- start 2024-07-16T10:02:07.479 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:07.475+0000 7f9ac70ad640 1 -- start start 2024-07-16T10:02:07.479 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:07.475+0000 7f9ac70ad640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9ac0065990 0x7f9ac00fc9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:02:07.479 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:07.475+0000 7f9ac70ad640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9ac00ff550 0x7f9ac00faf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:02:07.479 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:07.475+0000 7f9ac70ad640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ac00fb450 con 0x7f9ac0065990 2024-07-16T10:02:07.479 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:07.475+0000 7f9ac70ad640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ac00fb590 con 0x7f9ac00ff550 2024-07-16T10:02:07.479 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:07.476+0000 7f9ac4e49640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9ac0065990 0x7f9ac00fc9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:02:07.479 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:07.476+0000 7f9ac4e49640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9ac0065990 0x7f9ac00fc9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:47864/0 (socket says 172.21.15.156:47864) 2024-07-16T10:02:07.479 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:07.476+0000 7f9ac4e49640 1 -- 172.21.15.156:0/1996467267 learned_addr learned my addr 172.21.15.156:0/1996467267 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T10:02:07.480 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:07.476+0000 7f9abffff640 1 --2- 172.21.15.156:0/1996467267 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9ac00ff550 0x7f9ac00faf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:02:07.480 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:07.476+0000 7f9ac4e49640 1 -- 172.21.15.156:0/1996467267 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9ac00ff550 msgr2=0x7f9ac00faf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:02:07.480 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:07.476+0000 7f9ac4e49640 1 --2- 172.21.15.156:0/1996467267 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9ac00ff550 0x7f9ac00faf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:02:07.480 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:07.476+0000 7f9ac4e49640 1 -- 172.21.15.156:0/1996467267 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9aa80098b0 con 0x7f9ac0065990 2024-07-16T10:02:07.480 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:07.476+0000 7f9ac4e49640 1 --2- 172.21.15.156:0/1996467267 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9ac0065990 0x7f9ac00fc9d0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f9ab8002a80 tx=0x7f9ab8004570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T10:02:07.480 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:07.476+0000 7f9abd7fa640 1 -- 172.21.15.156:0/1996467267 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9ab8007c30 con 0x7f9ac0065990 2024-07-16T10:02:07.480 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:07.476+0000 7f9abd7fa640 1 -- 172.21.15.156:0/1996467267 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9ab800f040 con 0x7f9ac0065990 2024-07-16T10:02:07.480 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:07.476+0000 7f9ac70ad640 1 -- 172.21.15.156:0/1996467267 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ac00fb870 con 0x7f9ac0065990 2024-07-16T10:02:07.480 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:07.476+0000 7f9abd7fa640 1 -- 172.21.15.156:0/1996467267 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9ab8009650 con 0x7f9ac0065990 2024-07-16T10:02:07.480 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:07.476+0000 7f9ac70ad640 1 -- 172.21.15.156:0/1996467267 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ac00fbd90 con 0x7f9ac0065990 2024-07-16T10:02:07.481 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:07.477+0000 7f9ac70ad640 1 -- 172.21.15.156:0/1996467267 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ac00fa6a0 con 0x7f9ac0065990 2024-07-16T10:02:07.481 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:07.478+0000 7f9abd7fa640 1 -- 172.21.15.156:0/1996467267 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 38) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f9ab800d6b0 con 0x7f9ac0065990 2024-07-16T10:02:07.481 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:07.478+0000 7f9abd7fa640 1 --2- 172.21.15.156:0/1996467267 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f9aac06e0b0 0x7f9aac070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:02:07.481 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:07.478+0000 7f9abd7fa640 1 -- 172.21.15.156:0/1996467267 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f9ab808daf0 con 0x7f9ac0065990 2024-07-16T10:02:07.481 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:07.478+0000 7f9abffff640 1 --2- 172.21.15.156:0/1996467267 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f9aac06e0b0 0x7f9aac070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:02:07.481 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:07.479+0000 7f9abffff640 1 --2- 172.21.15.156:0/1996467267 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f9aac06e0b0 0x7f9aac070560 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f9aa8000f80 tx=0x7f9aa8004d10 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T10:02:07.484 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:07.483+0000 7f9abd7fa640 1 -- 172.21.15.156:0/1996467267 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187716 (secure 0 0 0) 0x7f9ab8056e70 con 0x7f9ac0065990 2024-07-16T10:02:07.730 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:07.729+0000 7f9ac70ad640 1 -- 172.21.15.156:0/1996467267 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f9ac0115140 con 0x7f9ac0065990 2024-07-16T10:02:07.731 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:07.729+0000 7f9abd7fa640 1 -- 172.21.15.156:0/1996467267 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+731 (secure 0 0 0) 0x7f9ab805a530 con 0x7f9ac0065990 2024-07-16T10:02:07.731 INFO:teuthology.orchestra.run.smithi156.stdout:{ 2024-07-16T10:02:07.731 INFO:teuthology.orchestra.run.smithi156.stdout: "mon": { 2024-07-16T10:02:07.731 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph version 19.0.0-3495-ga6ca461b (a6ca461be7cc1aff3658c2c6df4cff975a24449a) squid (rc)": 2 2024-07-16T10:02:07.731 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-07-16T10:02:07.731 INFO:teuthology.orchestra.run.smithi156.stdout: "mgr": { 2024-07-16T10:02:07.731 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph version 19.0.0-3495-ga6ca461b (a6ca461be7cc1aff3658c2c6df4cff975a24449a) squid (rc)": 2 2024-07-16T10:02:07.731 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-07-16T10:02:07.732 INFO:teuthology.orchestra.run.smithi156.stdout: "osd": { 2024-07-16T10:02:07.732 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph version 19.0.0-3495-ga6ca461b (a6ca461be7cc1aff3658c2c6df4cff975a24449a) squid (rc)": 6 2024-07-16T10:02:07.732 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-07-16T10:02:07.732 INFO:teuthology.orchestra.run.smithi156.stdout: "mds": { 2024-07-16T10:02:07.732 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph version 17.2.7-1104-ge6231b04 (e6231b04b52ebf3b9b31803282dd3074439e51ac) quincy (stable)": 4 2024-07-16T10:02:07.732 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-07-16T10:02:07.732 INFO:teuthology.orchestra.run.smithi156.stdout: "overall": { 2024-07-16T10:02:07.732 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph version 17.2.7-1104-ge6231b04 (e6231b04b52ebf3b9b31803282dd3074439e51ac) quincy (stable)": 4, 2024-07-16T10:02:07.732 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph version 19.0.0-3495-ga6ca461b (a6ca461be7cc1aff3658c2c6df4cff975a24449a) squid (rc)": 10 2024-07-16T10:02:07.732 INFO:teuthology.orchestra.run.smithi156.stdout: } 2024-07-16T10:02:07.733 INFO:teuthology.orchestra.run.smithi156.stdout:} 2024-07-16T10:02:07.733 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:07.731+0000 7f9ac70ad640 1 -- 172.21.15.156:0/1996467267 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f9aac06e0b0 msgr2=0x7f9aac070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:02:07.733 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:07.731+0000 7f9ac70ad640 1 --2- 172.21.15.156:0/1996467267 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f9aac06e0b0 0x7f9aac070560 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f9aa8000f80 tx=0x7f9aa8004d10 comp rx=0 tx=0).stop 2024-07-16T10:02:07.734 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:07.731+0000 7f9ac70ad640 1 -- 172.21.15.156:0/1996467267 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9ac0065990 msgr2=0x7f9ac00fc9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:02:07.734 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:07.731+0000 7f9ac70ad640 1 --2- 172.21.15.156:0/1996467267 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9ac0065990 0x7f9ac00fc9d0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f9ab8002a80 tx=0x7f9ab8004570 comp rx=0 tx=0).stop 2024-07-16T10:02:07.734 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:07.731+0000 7f9ac70ad640 1 -- 172.21.15.156:0/1996467267 shutdown_connections 2024-07-16T10:02:07.734 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:07.732+0000 7f9ac70ad640 1 --2- 172.21.15.156:0/1996467267 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f9aac06e0b0 0x7f9aac070560 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:02:07.734 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:07.732+0000 7f9ac70ad640 1 --2- 172.21.15.156:0/1996467267 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9ac00ff550 0x7f9ac00faf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:02:07.734 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:07.732+0000 7f9ac70ad640 1 --2- 172.21.15.156:0/1996467267 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9ac0065990 0x7f9ac00fc9d0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:02:07.734 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:07.732+0000 7f9ac70ad640 1 -- 172.21.15.156:0/1996467267 >> 172.21.15.156:0/1996467267 conn(0x7f9ac00f6a20 msgr2=0x7f9ac00f6df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T10:02:07.734 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:07.732+0000 7f9ac70ad640 1 -- 172.21.15.156:0/1996467267 shutdown_connections 2024-07-16T10:02:07.734 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:07.732+0000 7f9ac70ad640 1 -- 172.21.15.156:0/1996467267 wait complete. 2024-07-16T10:02:07.814 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:07.811+0000 7fe193848640 1 -- 172.21.15.156:0/489719095 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe18c066480 msgr2=0x7fe18c0ff030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:02:07.814 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:07.811+0000 7fe193848640 1 --2- 172.21.15.156:0/489719095 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe18c066480 0x7fe18c0ff030 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fe17c0048d0 tx=0x7fe17c009da0 comp rx=0 tx=0).stop 2024-07-16T10:02:07.814 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:07.811+0000 7fe193848640 1 -- 172.21.15.156:0/489719095 shutdown_connections 2024-07-16T10:02:07.814 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:07.811+0000 7fe193848640 1 --2- 172.21.15.156:0/489719095 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe18c0ff570 0x7fe18c101950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:02:07.814 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:07.811+0000 7fe193848640 1 --2- 172.21.15.156:0/489719095 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe18c066480 0x7fe18c0ff030 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:02:07.814 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:07.811+0000 7fe193848640 1 -- 172.21.15.156:0/489719095 >> 172.21.15.156:0/489719095 conn(0x7fe18c0f6a20 msgr2=0x7fe18c0f8e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T10:02:07.815 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:07.811+0000 7fe193848640 1 -- 172.21.15.156:0/489719095 shutdown_connections 2024-07-16T10:02:07.815 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:07.811+0000 7fe193848640 1 -- 172.21.15.156:0/489719095 wait complete. 2024-07-16T10:02:07.815 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:07.811+0000 7fe193848640 1 Processor -- start 2024-07-16T10:02:07.815 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:07.812+0000 7fe193848640 1 -- start start 2024-07-16T10:02:07.815 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:07.812+0000 7fe193848640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe18c066480 0x7fe18c10cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:02:07.815 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:07.812+0000 7fe193848640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe18c0ff570 0x7fe18c10d520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:02:07.815 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:07.812+0000 7fe193848640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe18c10db70 con 0x7fe18c066480 2024-07-16T10:02:07.815 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:07.812+0000 7fe193848640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe18c10e390 con 0x7fe18c0ff570 2024-07-16T10:02:07.815 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:07.812+0000 7fe1915e4640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe18c066480 0x7fe18c10cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:02:07.815 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:07.812+0000 7fe1915e4640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe18c066480 0x7fe18c10cfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:47888/0 (socket says 172.21.15.156:47888) 2024-07-16T10:02:07.815 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:07.812+0000 7fe1915e4640 1 -- 172.21.15.156:0/586902634 learned_addr learned my addr 172.21.15.156:0/586902634 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T10:02:07.816 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:07.812+0000 7fe190de3640 1 --2- 172.21.15.156:0/586902634 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe18c0ff570 0x7fe18c10d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:02:07.816 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:07.812+0000 7fe1915e4640 1 -- 172.21.15.156:0/586902634 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe18c0ff570 msgr2=0x7fe18c10d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:02:07.816 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:07.812+0000 7fe1915e4640 1 --2- 172.21.15.156:0/586902634 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe18c0ff570 0x7fe18c10d520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:02:07.816 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:07.812+0000 7fe1915e4640 1 -- 172.21.15.156:0/586902634 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe17c005040 con 0x7fe18c066480 2024-07-16T10:02:07.816 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:07.812+0000 7fe1915e4640 1 --2- 172.21.15.156:0/586902634 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe18c066480 0x7fe18c10cfe0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fe17c000c00 tx=0x7fe17c009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T10:02:07.816 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:07.813+0000 7fe181ffb640 1 -- 172.21.15.156:0/586902634 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe17c038070 con 0x7fe18c066480 2024-07-16T10:02:07.816 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:07.813+0000 7fe193848640 1 -- 172.21.15.156:0/586902634 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe18c10e5c0 con 0x7fe18c066480 2024-07-16T10:02:07.816 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:07.813+0000 7fe181ffb640 1 -- 172.21.15.156:0/586902634 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe17c032970 con 0x7fe18c066480 2024-07-16T10:02:07.816 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:07.813+0000 7fe181ffb640 1 -- 172.21.15.156:0/586902634 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe17c0338e0 con 0x7fe18c066480 2024-07-16T10:02:07.816 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:07.813+0000 7fe193848640 1 -- 172.21.15.156:0/586902634 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe18c10eb00 con 0x7fe18c066480 2024-07-16T10:02:07.817 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:07.814+0000 7fe193848640 1 -- 172.21.15.156:0/586902634 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe18c0fa6c0 con 0x7fe18c066480 2024-07-16T10:02:07.817 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:07.814+0000 7fe181ffb640 1 -- 172.21.15.156:0/586902634 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 38) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fe17c030020 con 0x7fe18c066480 2024-07-16T10:02:07.817 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:07.815+0000 7fe181ffb640 1 --2- 172.21.15.156:0/586902634 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7fe17806e0b0 0x7fe178070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:02:07.818 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:07.815+0000 7fe181ffb640 1 -- 172.21.15.156:0/586902634 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fe17c0ad980 con 0x7fe18c066480 2024-07-16T10:02:07.818 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:07.815+0000 7fe190de3640 1 --2- 172.21.15.156:0/586902634 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7fe17806e0b0 0x7fe178070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:02:07.818 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:07.815+0000 7fe190de3640 1 --2- 172.21.15.156:0/586902634 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7fe17806e0b0 0x7fe178070560 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fe18c10cd90 tx=0x7fe18800b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T10:02:07.820 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:07.819+0000 7fe181ffb640 1 -- 172.21.15.156:0/586902634 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187716 (secure 0 0 0) 0x7fe17c076db0 con 0x7fe18c066480 2024-07-16T10:02:08.033 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:08.032+0000 7fe193848640 1 -- 172.21.15.156:0/586902634 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fe18c10edc0 con 0x7fe18c066480 2024-07-16T10:02:08.034 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:02:07 smithi156 ceph-mon[93066]: pgmap v163: 65 pgs: 65 active+clean; 253 MiB data, 1013 MiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 1.6 KiB/s wr, 1 op/s 2024-07-16T10:02:08.034 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:02:07 smithi156 ceph-mon[93066]: from='client.34378 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-16T10:02:08.034 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:02:07 smithi156 ceph-mon[93066]: from='client.34382 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-16T10:02:08.034 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:02:07 smithi156 ceph-mon[93066]: from='client.? 172.21.15.156:0/1996467267' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:02:08.034 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:08.032+0000 7fe181ffb640 1 -- 172.21.15.156:0/586902634 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 24 v24) v1 ==== 76+0+1919 (secure 0 0 0) 0x7fe17c07a470 con 0x7fe18c066480 2024-07-16T10:02:08.035 INFO:teuthology.orchestra.run.smithi156.stdout:e24 2024-07-16T10:02:08.035 INFO:teuthology.orchestra.run.smithi156.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-07-16T10:02:08.035 INFO:teuthology.orchestra.run.smithi156.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-07-16T10:02:08.035 INFO:teuthology.orchestra.run.smithi156.stdout:legacy client fscid: 1 2024-07-16T10:02:08.035 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-07-16T10:02:08.035 INFO:teuthology.orchestra.run.smithi156.stdout:Filesystem 'cephfs' (1) 2024-07-16T10:02:08.035 INFO:teuthology.orchestra.run.smithi156.stdout:fs_name cephfs 2024-07-16T10:02:08.035 INFO:teuthology.orchestra.run.smithi156.stdout:epoch 24 2024-07-16T10:02:08.035 INFO:teuthology.orchestra.run.smithi156.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-07-16T10:02:08.035 INFO:teuthology.orchestra.run.smithi156.stdout:created 2024-07-16T09:52:55.214975+0000 2024-07-16T10:02:08.035 INFO:teuthology.orchestra.run.smithi156.stdout:modified 2024-07-16T10:02:01.577797+0000 2024-07-16T10:02:08.036 INFO:teuthology.orchestra.run.smithi156.stdout:tableserver 0 2024-07-16T10:02:08.036 INFO:teuthology.orchestra.run.smithi156.stdout:root 0 2024-07-16T10:02:08.036 INFO:teuthology.orchestra.run.smithi156.stdout:session_timeout 60 2024-07-16T10:02:08.036 INFO:teuthology.orchestra.run.smithi156.stdout:session_autoclose 300 2024-07-16T10:02:08.036 INFO:teuthology.orchestra.run.smithi156.stdout:max_file_size 1099511627776 2024-07-16T10:02:08.036 INFO:teuthology.orchestra.run.smithi156.stdout:max_xattr_size 65536 2024-07-16T10:02:08.036 INFO:teuthology.orchestra.run.smithi156.stdout:required_client_features {} 2024-07-16T10:02:08.036 INFO:teuthology.orchestra.run.smithi156.stdout:last_failure 0 2024-07-16T10:02:08.036 INFO:teuthology.orchestra.run.smithi156.stdout:last_failure_osd_epoch 0 2024-07-16T10:02:08.036 INFO:teuthology.orchestra.run.smithi156.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-07-16T10:02:08.036 INFO:teuthology.orchestra.run.smithi156.stdout:max_mds 1 2024-07-16T10:02:08.036 INFO:teuthology.orchestra.run.smithi156.stdout:in 0,1 2024-07-16T10:02:08.036 INFO:teuthology.orchestra.run.smithi156.stdout:up {0=14458,1=24261} 2024-07-16T10:02:08.037 INFO:teuthology.orchestra.run.smithi156.stdout:failed 2024-07-16T10:02:08.037 INFO:teuthology.orchestra.run.smithi156.stdout:damaged 2024-07-16T10:02:08.037 INFO:teuthology.orchestra.run.smithi156.stdout:stopped 2024-07-16T10:02:08.037 INFO:teuthology.orchestra.run.smithi156.stdout:data_pools [3] 2024-07-16T10:02:08.037 INFO:teuthology.orchestra.run.smithi156.stdout:metadata_pool 2 2024-07-16T10:02:08.037 INFO:teuthology.orchestra.run.smithi156.stdout:inline_data disabled 2024-07-16T10:02:08.037 INFO:teuthology.orchestra.run.smithi156.stdout:balancer 2024-07-16T10:02:08.037 INFO:teuthology.orchestra.run.smithi156.stdout:bal_rank_mask -1 2024-07-16T10:02:08.037 INFO:teuthology.orchestra.run.smithi156.stdout:standby_count_wanted 1 2024-07-16T10:02:08.037 INFO:teuthology.orchestra.run.smithi156.stdout:qdb_cluster leader: 14458 members: 14458 2024-07-16T10:02:08.037 INFO:teuthology.orchestra.run.smithi156.stdout:[mds.cephfs.smithi156.rdibys{0:14458} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.156:6826/1080176013,v1:172.21.15.156:6827/1080176013] compat {c=[1],r=[1],i=[7ff]}] 2024-07-16T10:02:08.037 INFO:teuthology.orchestra.run.smithi156.stdout:[mds.cephfs.smithi176.dvvbud{1:24261} state up:stopping seq 4 export targets 0 join_fscid=1 addr [v2:172.21.15.176:6824/1577019287,v1:172.21.15.176:6825/1577019287] compat {c=[1],r=[1],i=[7ff]}] 2024-07-16T10:02:08.037 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-07-16T10:02:08.038 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-07-16T10:02:08.038 INFO:teuthology.orchestra.run.smithi156.stdout:Standby daemons: 2024-07-16T10:02:08.038 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-07-16T10:02:08.038 INFO:teuthology.orchestra.run.smithi156.stdout:[mds.cephfs.smithi156.evwwaz{-1:34374} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.156:6828/333092891,v1:172.21.15.156:6829/333092891] compat {c=[1],r=[1],i=[7ff]}] 2024-07-16T10:02:08.038 INFO:teuthology.orchestra.run.smithi156.stdout:[mds.cephfs.smithi176.wahbsi{-1:44269} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.176:6826/3942496125,v1:172.21.15.176:6827/3942496125] compat {c=[1],r=[1],i=[7ff]}] 2024-07-16T10:02:08.039 INFO:teuthology.orchestra.run.smithi156.stderr:dumped fsmap epoch 24 2024-07-16T10:02:08.039 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:08.034+0000 7fe193848640 1 -- 172.21.15.156:0/586902634 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7fe17806e0b0 msgr2=0x7fe178070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:02:08.039 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:08.034+0000 7fe193848640 1 --2- 172.21.15.156:0/586902634 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7fe17806e0b0 0x7fe178070560 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fe18c10cd90 tx=0x7fe18800b040 comp rx=0 tx=0).stop 2024-07-16T10:02:08.039 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:08.034+0000 7fe193848640 1 -- 172.21.15.156:0/586902634 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe18c066480 msgr2=0x7fe18c10cfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:02:08.039 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:08.034+0000 7fe193848640 1 --2- 172.21.15.156:0/586902634 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe18c066480 0x7fe18c10cfe0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fe17c000c00 tx=0x7fe17c009da0 comp rx=0 tx=0).stop 2024-07-16T10:02:08.039 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:08.034+0000 7fe193848640 1 -- 172.21.15.156:0/586902634 shutdown_connections 2024-07-16T10:02:08.039 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:08.034+0000 7fe193848640 1 --2- 172.21.15.156:0/586902634 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7fe17806e0b0 0x7fe178070560 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:02:08.039 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:08.034+0000 7fe193848640 1 --2- 172.21.15.156:0/586902634 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe18c0ff570 0x7fe18c10d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:02:08.039 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:08.034+0000 7fe193848640 1 --2- 172.21.15.156:0/586902634 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe18c066480 0x7fe18c10cfe0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:02:08.039 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:08.035+0000 7fe193848640 1 -- 172.21.15.156:0/586902634 >> 172.21.15.156:0/586902634 conn(0x7fe18c0f6a20 msgr2=0x7fe18c0f6e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T10:02:08.040 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:08.035+0000 7fe193848640 1 -- 172.21.15.156:0/586902634 shutdown_connections 2024-07-16T10:02:08.040 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:08.035+0000 7fe193848640 1 -- 172.21.15.156:0/586902634 wait complete. 2024-07-16T10:02:08.119 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:08.115+0000 7f2d11686640 1 -- 172.21.15.156:0/1081682478 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2d0c0fb260 msgr2=0x7f2d0c0fb670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:02:08.119 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:08.115+0000 7f2d11686640 1 --2- 172.21.15.156:0/1081682478 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2d0c0fb260 0x7f2d0c0fb670 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f2cf40048d0 tx=0x7f2cf4009da0 comp rx=0 tx=0).stop 2024-07-16T10:02:08.119 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:08.116+0000 7f2d11686640 1 -- 172.21.15.156:0/1081682478 shutdown_connections 2024-07-16T10:02:08.119 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:08.116+0000 7f2d11686640 1 --2- 172.21.15.156:0/1081682478 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2d0c0fc530 0x7f2d0c0fc9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:02:08.119 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:08.116+0000 7f2d11686640 1 --2- 172.21.15.156:0/1081682478 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2d0c0fb260 0x7f2d0c0fb670 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:02:08.119 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:08.116+0000 7f2d11686640 1 -- 172.21.15.156:0/1081682478 >> 172.21.15.156:0/1081682478 conn(0x7f2d0c0f6a40 msgr2=0x7f2d0c0f8e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T10:02:08.119 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:08.116+0000 7f2d11686640 1 -- 172.21.15.156:0/1081682478 shutdown_connections 2024-07-16T10:02:08.119 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:08.116+0000 7f2d11686640 1 -- 172.21.15.156:0/1081682478 wait complete. 2024-07-16T10:02:08.119 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:08.116+0000 7f2d11686640 1 Processor -- start 2024-07-16T10:02:08.120 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:08.116+0000 7f2d11686640 1 -- start start 2024-07-16T10:02:08.120 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:08.116+0000 7f2d11686640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2d0c0fb260 0x7f2d0c10cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:02:08.120 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:08.116+0000 7f2d11686640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2d0c0fc530 0x7f2d0c10d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:02:08.120 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:08.117+0000 7f2d11686640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d0c10db30 con 0x7f2d0c0fc530 2024-07-16T10:02:08.120 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:08.117+0000 7f2d11686640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d0c10e300 con 0x7f2d0c0fb260 2024-07-16T10:02:08.120 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:08.117+0000 7f2d0a7fc640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2d0c0fc530 0x7f2d0c10d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:02:08.120 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:08.117+0000 7f2d0a7fc640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2d0c0fc530 0x7f2d0c10d4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:47914/0 (socket says 172.21.15.156:47914) 2024-07-16T10:02:08.120 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:08.117+0000 7f2d0a7fc640 1 -- 172.21.15.156:0/2078403630 learned_addr learned my addr 172.21.15.156:0/2078403630 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T10:02:08.120 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:08.117+0000 7f2d0affd640 1 --2- 172.21.15.156:0/2078403630 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2d0c0fb260 0x7f2d0c10cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:02:08.120 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:08.117+0000 7f2d0a7fc640 1 -- 172.21.15.156:0/2078403630 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2d0c0fb260 msgr2=0x7f2d0c10cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:02:08.121 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:08.117+0000 7f2d0a7fc640 1 --2- 172.21.15.156:0/2078403630 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2d0c0fb260 0x7f2d0c10cfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:02:08.121 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:08.117+0000 7f2d0a7fc640 1 -- 172.21.15.156:0/2078403630 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cf4005040 con 0x7f2d0c0fc530 2024-07-16T10:02:08.121 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:08.117+0000 7f2d0a7fc640 1 --2- 172.21.15.156:0/2078403630 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2d0c0fc530 0x7f2d0c10d4e0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f2cfc00eb70 tx=0x7f2cfc00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T10:02:08.121 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:08.117+0000 7f2d037fe640 1 -- 172.21.15.156:0/2078403630 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2cfc007c30 con 0x7f2d0c0fc530 2024-07-16T10:02:08.121 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:08.117+0000 7f2d037fe640 1 -- 172.21.15.156:0/2078403630 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2cfc00f040 con 0x7f2d0c0fc530 2024-07-16T10:02:08.121 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:08.117+0000 7f2d11686640 1 -- 172.21.15.156:0/2078403630 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d0c10e530 con 0x7f2d0c0fc530 2024-07-16T10:02:08.121 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:08.117+0000 7f2d037fe640 1 -- 172.21.15.156:0/2078403630 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2cfc00cad0 con 0x7f2d0c0fc530 2024-07-16T10:02:08.121 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:08.117+0000 7f2d11686640 1 -- 172.21.15.156:0/2078403630 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d0c10ead0 con 0x7f2d0c0fc530 2024-07-16T10:02:08.122 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:08.118+0000 7f2d11686640 1 -- 172.21.15.156:0/2078403630 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d0c0fa6c0 con 0x7f2d0c0fc530 2024-07-16T10:02:08.122 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:08.119+0000 7f2d037fe640 1 -- 172.21.15.156:0/2078403630 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 38) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f2cfc00cc50 con 0x7f2d0c0fc530 2024-07-16T10:02:08.122 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:08.119+0000 7f2d037fe640 1 --2- 172.21.15.156:0/2078403630 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f2cf806e0b0 0x7f2cf8070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:02:08.122 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:08.119+0000 7f2d037fe640 1 -- 172.21.15.156:0/2078403630 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f2cfc08d780 con 0x7f2d0c0fc530 2024-07-16T10:02:08.122 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:08.119+0000 7f2d0affd640 1 --2- 172.21.15.156:0/2078403630 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f2cf806e0b0 0x7f2cf8070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:02:08.122 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:08.120+0000 7f2d0affd640 1 --2- 172.21.15.156:0/2078403630 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f2cf806e0b0 0x7f2cf8070560 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f2cf4009f80 tx=0x7f2cf4035040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T10:02:08.124 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:08.123+0000 7f2d037fe640 1 -- 172.21.15.156:0/2078403630 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187716 (secure 0 0 0) 0x7f2cfc056bb0 con 0x7f2d0c0fc530 2024-07-16T10:02:08.316 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:08.315+0000 7f2d11686640 1 -- 172.21.15.156:0/2078403630 --> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f2d0c103f40 con 0x7f2cf806e0b0 2024-07-16T10:02:08.317 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:08.316+0000 7f2d037fe640 1 -- 172.21.15.156:0/2078403630 <== mgr.34104 v2:172.21.15.156:6800/3367765292 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f2d0c103f40 con 0x7f2cf806e0b0 2024-07-16T10:02:08.318 INFO:teuthology.orchestra.run.smithi156.stdout:{ 2024-07-16T10:02:08.318 INFO:teuthology.orchestra.run.smithi156.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:c15ec3aa8e6c0a9c12ee24ff79f3e7dd4d01b3d1a702dfeb7d7fff9b0811a393", 2024-07-16T10:02:08.318 INFO:teuthology.orchestra.run.smithi156.stdout: "in_progress": true, 2024-07-16T10:02:08.318 INFO:teuthology.orchestra.run.smithi156.stdout: "which": "Upgrading all daemon types on all hosts", 2024-07-16T10:02:08.318 INFO:teuthology.orchestra.run.smithi156.stdout: "services_complete": [ 2024-07-16T10:02:08.318 INFO:teuthology.orchestra.run.smithi156.stdout: "mon", 2024-07-16T10:02:08.318 INFO:teuthology.orchestra.run.smithi156.stdout: "mgr", 2024-07-16T10:02:08.318 INFO:teuthology.orchestra.run.smithi156.stdout: "crash", 2024-07-16T10:02:08.318 INFO:teuthology.orchestra.run.smithi156.stdout: "osd" 2024-07-16T10:02:08.318 INFO:teuthology.orchestra.run.smithi156.stdout: ], 2024-07-16T10:02:08.318 INFO:teuthology.orchestra.run.smithi156.stdout: "progress": "12/21 daemons upgraded", 2024-07-16T10:02:08.319 INFO:teuthology.orchestra.run.smithi156.stdout: "message": "Currently upgrading osd daemons", 2024-07-16T10:02:08.319 INFO:teuthology.orchestra.run.smithi156.stdout: "is_paused": false 2024-07-16T10:02:08.319 INFO:teuthology.orchestra.run.smithi156.stdout:} 2024-07-16T10:02:08.319 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:08.318+0000 7f2d11686640 1 -- 172.21.15.156:0/2078403630 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f2cf806e0b0 msgr2=0x7f2cf8070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:02:08.320 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:08.318+0000 7f2d11686640 1 --2- 172.21.15.156:0/2078403630 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f2cf806e0b0 0x7f2cf8070560 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f2cf4009f80 tx=0x7f2cf4035040 comp rx=0 tx=0).stop 2024-07-16T10:02:08.320 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:08.318+0000 7f2d11686640 1 -- 172.21.15.156:0/2078403630 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2d0c0fc530 msgr2=0x7f2d0c10d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:02:08.320 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:08.318+0000 7f2d11686640 1 --2- 172.21.15.156:0/2078403630 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2d0c0fc530 0x7f2d0c10d4e0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f2cfc00eb70 tx=0x7f2cfc00b040 comp rx=0 tx=0).stop 2024-07-16T10:02:08.320 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:08.318+0000 7f2d11686640 1 -- 172.21.15.156:0/2078403630 shutdown_connections 2024-07-16T10:02:08.320 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:08.318+0000 7f2d11686640 1 --2- 172.21.15.156:0/2078403630 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f2cf806e0b0 0x7f2cf8070560 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:02:08.320 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:08.318+0000 7f2d11686640 1 --2- 172.21.15.156:0/2078403630 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2d0c0fc530 0x7f2d0c10d4e0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:02:08.320 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:08.318+0000 7f2d11686640 1 --2- 172.21.15.156:0/2078403630 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2d0c0fb260 0x7f2d0c10cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:02:08.320 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:08.318+0000 7f2d11686640 1 -- 172.21.15.156:0/2078403630 >> 172.21.15.156:0/2078403630 conn(0x7f2d0c0f6a40 msgr2=0x7f2d0c065e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T10:02:08.320 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:08.318+0000 7f2d11686640 1 -- 172.21.15.156:0/2078403630 shutdown_connections 2024-07-16T10:02:08.320 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:08.318+0000 7f2d11686640 1 -- 172.21.15.156:0/2078403630 wait complete. 2024-07-16T10:02:08.343 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:02:07 smithi176 ceph-mon[81689]: pgmap v163: 65 pgs: 65 active+clean; 253 MiB data, 1013 MiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 1.6 KiB/s wr, 1 op/s 2024-07-16T10:02:08.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:02:07 smithi176 ceph-mon[81689]: from='client.34378 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-16T10:02:08.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:02:07 smithi176 ceph-mon[81689]: from='client.34382 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-16T10:02:08.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:02:07 smithi176 ceph-mon[81689]: from='client.? 172.21.15.156:0/1996467267' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:02:08.401 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:08.398+0000 7f27960db640 1 -- 172.21.15.156:0/2108030520 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2790065a50 msgr2=0x7f27900fef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:02:08.401 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:08.398+0000 7f27960db640 1 --2- 172.21.15.156:0/2108030520 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2790065a50 0x7f27900fef50 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f27800048d0 tx=0x7f2780009da0 comp rx=0 tx=0).stop 2024-07-16T10:02:08.402 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:08.398+0000 7f27960db640 1 -- 172.21.15.156:0/2108030520 shutdown_connections 2024-07-16T10:02:08.402 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:08.398+0000 7f27960db640 1 --2- 172.21.15.156:0/2108030520 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f27900ff490 0x7f2790101870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:02:08.402 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:08.398+0000 7f27960db640 1 --2- 172.21.15.156:0/2108030520 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2790065a50 0x7f27900fef50 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:02:08.402 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:08.398+0000 7f27960db640 1 -- 172.21.15.156:0/2108030520 >> 172.21.15.156:0/2108030520 conn(0x7f27900f6a20 msgr2=0x7f27900f8e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T10:02:08.402 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:08.398+0000 7f27960db640 1 -- 172.21.15.156:0/2108030520 shutdown_connections 2024-07-16T10:02:08.402 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:08.398+0000 7f27960db640 1 -- 172.21.15.156:0/2108030520 wait complete. 2024-07-16T10:02:08.402 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:08.399+0000 7f27960db640 1 Processor -- start 2024-07-16T10:02:08.402 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:08.399+0000 7f27960db640 1 -- start start 2024-07-16T10:02:08.402 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:08.399+0000 7f27960db640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2790065a50 0x7f279010cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:02:08.402 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:08.399+0000 7f27960db640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f27900ff490 0x7f279010d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:02:08.402 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:08.399+0000 7f27960db640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f279010db20 con 0x7f2790065a50 2024-07-16T10:02:08.402 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:08.399+0000 7f27960db640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f279010e340 con 0x7f27900ff490 2024-07-16T10:02:08.403 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:08.399+0000 7f278f7fe640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2790065a50 0x7f279010cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:02:08.403 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:08.399+0000 7f278f7fe640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2790065a50 0x7f279010cf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:47940/0 (socket says 172.21.15.156:47940) 2024-07-16T10:02:08.403 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:08.399+0000 7f278f7fe640 1 -- 172.21.15.156:0/1515071647 learned_addr learned my addr 172.21.15.156:0/1515071647 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T10:02:08.403 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:08.399+0000 7f278effd640 1 --2- 172.21.15.156:0/1515071647 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f27900ff490 0x7f279010d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:02:08.403 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:08.399+0000 7f278f7fe640 1 -- 172.21.15.156:0/1515071647 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f27900ff490 msgr2=0x7f279010d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:02:08.403 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:08.399+0000 7f278f7fe640 1 --2- 172.21.15.156:0/1515071647 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f27900ff490 0x7f279010d4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:02:08.403 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:08.399+0000 7f278f7fe640 1 -- 172.21.15.156:0/1515071647 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2780005040 con 0x7f2790065a50 2024-07-16T10:02:08.403 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:08.399+0000 7f278effd640 1 --2- 172.21.15.156:0/1515071647 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f27900ff490 0x7f279010d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-16T10:02:08.403 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:08.400+0000 7f278f7fe640 1 --2- 172.21.15.156:0/1515071647 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2790065a50 0x7f279010cf90 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f2780000c00 tx=0x7f2780009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T10:02:08.403 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:08.400+0000 7f2777fff640 1 -- 172.21.15.156:0/1515071647 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2780038070 con 0x7f2790065a50 2024-07-16T10:02:08.403 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:08.400+0000 7f27960db640 1 -- 172.21.15.156:0/1515071647 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f279010e570 con 0x7f2790065a50 2024-07-16T10:02:08.404 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:08.400+0000 7f2777fff640 1 -- 172.21.15.156:0/1515071647 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2780032970 con 0x7f2790065a50 2024-07-16T10:02:08.404 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:08.400+0000 7f2777fff640 1 -- 172.21.15.156:0/1515071647 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f27800338e0 con 0x7f2790065a50 2024-07-16T10:02:08.404 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:08.400+0000 7f27960db640 1 -- 172.21.15.156:0/1515071647 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f279010ea30 con 0x7f2790065a50 2024-07-16T10:02:08.405 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:08.401+0000 7f27960db640 1 -- 172.21.15.156:0/1515071647 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27900fa6a0 con 0x7f2790065a50 2024-07-16T10:02:08.405 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:08.401+0000 7f2777fff640 1 -- 172.21.15.156:0/1515071647 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 38) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f2780030020 con 0x7f2790065a50 2024-07-16T10:02:08.405 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:08.402+0000 7f2777fff640 1 --2- 172.21.15.156:0/1515071647 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f277806e0b0 0x7f2778070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:02:08.405 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:08.402+0000 7f2777fff640 1 -- 172.21.15.156:0/1515071647 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f27800ada30 con 0x7f2790065a50 2024-07-16T10:02:08.405 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:08.402+0000 7f278effd640 1 --2- 172.21.15.156:0/1515071647 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f277806e0b0 0x7f2778070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:02:08.405 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:08.402+0000 7f278effd640 1 --2- 172.21.15.156:0/1515071647 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f277806e0b0 0x7f2778070560 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f279010cd40 tx=0x7f2784009ed0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T10:02:08.407 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:08.406+0000 7f2777fff640 1 -- 172.21.15.156:0/1515071647 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187716 (secure 0 0 0) 0x7f2780076db0 con 0x7f2790065a50 2024-07-16T10:02:08.652 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:08.651+0000 7f27960db640 1 -- 172.21.15.156:0/1515071647 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f2790104e90 con 0x7f2790065a50 2024-07-16T10:02:08.653 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:08.651+0000 7f2777fff640 1 -- 172.21.15.156:0/1515071647 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7f278007a470 con 0x7f2790065a50 2024-07-16T10:02:08.653 INFO:teuthology.orchestra.run.smithi156.stdout:HEALTH_OK 2024-07-16T10:02:08.655 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:08.654+0000 7f27960db640 1 -- 172.21.15.156:0/1515071647 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f277806e0b0 msgr2=0x7f2778070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:02:08.655 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:08.654+0000 7f27960db640 1 --2- 172.21.15.156:0/1515071647 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f277806e0b0 0x7f2778070560 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f279010cd40 tx=0x7f2784009ed0 comp rx=0 tx=0).stop 2024-07-16T10:02:08.655 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:08.654+0000 7f27960db640 1 -- 172.21.15.156:0/1515071647 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2790065a50 msgr2=0x7f279010cf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:02:08.655 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:08.654+0000 7f27960db640 1 --2- 172.21.15.156:0/1515071647 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2790065a50 0x7f279010cf90 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f2780000c00 tx=0x7f2780009da0 comp rx=0 tx=0).stop 2024-07-16T10:02:08.655 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:08.654+0000 7f27960db640 1 -- 172.21.15.156:0/1515071647 shutdown_connections 2024-07-16T10:02:08.656 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:08.654+0000 7f27960db640 1 --2- 172.21.15.156:0/1515071647 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f277806e0b0 0x7f2778070560 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:02:08.656 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:08.654+0000 7f27960db640 1 --2- 172.21.15.156:0/1515071647 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f27900ff490 0x7f279010d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:02:08.656 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:08.654+0000 7f27960db640 1 --2- 172.21.15.156:0/1515071647 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2790065a50 0x7f279010cf90 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:02:08.656 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:08.654+0000 7f27960db640 1 -- 172.21.15.156:0/1515071647 >> 172.21.15.156:0/1515071647 conn(0x7f27900f6a20 msgr2=0x7f27900f6df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T10:02:08.656 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:08.654+0000 7f27960db640 1 -- 172.21.15.156:0/1515071647 shutdown_connections 2024-07-16T10:02:08.656 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:08.654+0000 7f27960db640 1 -- 172.21.15.156:0/1515071647 wait complete. 2024-07-16T10:02:08.924 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:02:08 smithi156 ceph-mon[93066]: from='client.34386 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-07-16T10:02:08.924 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:02:08 smithi156 ceph-mon[93066]: from='client.? 172.21.15.156:0/586902634' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-07-16T10:02:08.924 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:02:08 smithi156 ceph-mon[93066]: from='client.34398 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-16T10:02:08.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:02:08 smithi156 ceph-mon[93066]: from='client.? 172.21.15.156:0/1515071647' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-07-16T10:02:09.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:02:08 smithi176 ceph-mon[81689]: from='client.34386 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-07-16T10:02:09.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:02:08 smithi176 ceph-mon[81689]: from='client.? 172.21.15.156:0/586902634' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-07-16T10:02:09.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:02:08 smithi176 ceph-mon[81689]: from='client.34398 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-16T10:02:09.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:02:08 smithi176 ceph-mon[81689]: from='client.? 172.21.15.156:0/1515071647' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-07-16T10:02:10.174 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:02:09 smithi156 ceph-mon[93066]: pgmap v164: 65 pgs: 65 active+clean; 253 MiB data, 1013 MiB used, 535 GiB / 536 GiB avail; 1.6 KiB/s wr, 0 op/s 2024-07-16T10:02:10.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:02:09 smithi176 ceph-mon[81689]: pgmap v164: 65 pgs: 65 active+clean; 253 MiB data, 1013 MiB used, 535 GiB / 536 GiB avail; 1.6 KiB/s wr, 0 op/s 2024-07-16T10:02:12.174 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:02:11 smithi156 ceph-mon[93066]: pgmap v165: 65 pgs: 65 active+clean; 253 MiB data, 1013 MiB used, 535 GiB / 536 GiB avail; 170 B/s rd, 1.8 KiB/s wr, 1 op/s 2024-07-16T10:02:12.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:02:11 smithi176 ceph-mon[81689]: pgmap v165: 65 pgs: 65 active+clean; 253 MiB data, 1013 MiB used, 535 GiB / 536 GiB avail; 170 B/s rd, 1.8 KiB/s wr, 1 op/s 2024-07-16T10:02:14.174 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:02:13 smithi156 ceph-mon[93066]: pgmap v166: 65 pgs: 65 active+clean; 253 MiB data, 1013 MiB used, 535 GiB / 536 GiB avail; 85 B/s rd, 1.2 KiB/s wr, 0 op/s 2024-07-16T10:02:14.343 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:02:13 smithi176 ceph-mon[81689]: pgmap v166: 65 pgs: 65 active+clean; 253 MiB data, 1013 MiB used, 535 GiB / 536 GiB avail; 85 B/s rd, 1.2 KiB/s wr, 0 op/s 2024-07-16T10:02:16.174 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:02:15 smithi156 ceph-mon[93066]: pgmap v167: 65 pgs: 65 active+clean; 249 MiB data, 1009 MiB used, 535 GiB / 536 GiB avail; 85 B/s rd, 853 B/s wr, 1 op/s 2024-07-16T10:02:16.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:02:15 smithi176 ceph-mon[81689]: pgmap v167: 65 pgs: 65 active+clean; 249 MiB data, 1009 MiB used, 535 GiB / 536 GiB avail; 85 B/s rd, 853 B/s wr, 1 op/s 2024-07-16T10:02:17.174 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:02:16 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-16T10:02:17.175 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:02:16 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-16T10:02:17.175 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:02:16 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-16T10:02:17.175 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:02:16 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:02:17.175 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:02:16 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-16T10:02:17.175 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:02:16 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:02:17.175 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:02:16 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:02:17.175 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:02:16 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:02:17.175 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:02:16 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:02:17.218 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:02:16 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-16T10:02:17.219 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:02:16 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-16T10:02:17.219 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:02:16 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-16T10:02:17.219 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:02:16 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:02:17.219 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:02:16 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-16T10:02:17.219 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:02:16 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:02:17.219 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:02:16 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:02:17.219 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:02:16 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:02:17.219 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:02:16 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:02:18.174 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:02:17 smithi156 ceph-mon[93066]: pgmap v168: 65 pgs: 65 active+clean; 225 MiB data, 961 MiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 938 B/s wr, 4 op/s 2024-07-16T10:02:18.175 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:02:17 smithi156 ceph-mon[93066]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2024-07-16T10:02:18.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:02:17 smithi176 ceph-mon[81689]: pgmap v168: 65 pgs: 65 active+clean; 225 MiB data, 961 MiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 938 B/s wr, 4 op/s 2024-07-16T10:02:18.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:02:17 smithi176 ceph-mon[81689]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2024-07-16T10:02:19.175 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:02:18 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-16T10:02:19.343 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:02:18 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-16T10:02:20.174 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:02:19 smithi156 ceph-mon[93066]: pgmap v169: 65 pgs: 65 active+clean; 221 MiB data, 941 MiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 938 B/s wr, 5 op/s 2024-07-16T10:02:20.343 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:02:19 smithi176 ceph-mon[81689]: pgmap v169: 65 pgs: 65 active+clean; 221 MiB data, 941 MiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 938 B/s wr, 5 op/s 2024-07-16T10:02:22.174 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:02:21 smithi156 ceph-mon[93066]: pgmap v170: 65 pgs: 65 active+clean; 189 MiB data, 873 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 938 B/s wr, 6 op/s 2024-07-16T10:02:22.343 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:02:21 smithi176 ceph-mon[81689]: pgmap v170: 65 pgs: 65 active+clean; 189 MiB data, 873 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 938 B/s wr, 6 op/s 2024-07-16T10:02:24.174 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:02:23 smithi156 ceph-mon[93066]: pgmap v171: 65 pgs: 65 active+clean; 145 MiB data, 689 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 853 B/s wr, 7 op/s 2024-07-16T10:02:24.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:02:23 smithi176 ceph-mon[81689]: pgmap v171: 65 pgs: 65 active+clean; 145 MiB data, 689 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 853 B/s wr, 7 op/s 2024-07-16T10:02:26.174 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:02:25 smithi156 ceph-mon[93066]: pgmap v172: 65 pgs: 65 active+clean; 145 MiB data, 689 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 853 B/s wr, 7 op/s 2024-07-16T10:02:26.343 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:02:25 smithi176 ceph-mon[81689]: pgmap v172: 65 pgs: 65 active+clean; 145 MiB data, 689 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 853 B/s wr, 7 op/s 2024-07-16T10:02:27.175 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:02:26 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-16T10:02:27.175 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:02:26 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-16T10:02:27.175 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:02:26 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-16T10:02:27.175 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:02:26 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:02:27.175 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:02:26 smithi156 ceph-mon[93066]: daemon mds.cephfs.smithi176.dvvbud finished stopping rank 1 in filesystem cephfs (now has 1 ranks) 2024-07-16T10:02:27.175 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:02:26 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-16T10:02:27.175 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:02:26 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:02:27.175 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:02:26 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:02:27.175 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:02:26 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:02:27.175 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:02:26 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:02:27.343 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:02:26 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-16T10:02:27.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:02:26 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-16T10:02:27.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:02:26 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-16T10:02:27.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:02:26 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:02:27.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:02:26 smithi176 ceph-mon[81689]: daemon mds.cephfs.smithi176.dvvbud finished stopping rank 1 in filesystem cephfs (now has 1 ranks) 2024-07-16T10:02:27.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:02:26 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-16T10:02:27.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:02:26 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:02:27.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:02:26 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:02:27.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:02:26 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:02:27.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:02:26 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:02:28.174 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:02:27 smithi156 ceph-mon[93066]: pgmap v173: 65 pgs: 65 active+clean; 145 MiB data, 689 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 853 B/s wr, 6 op/s 2024-07-16T10:02:28.175 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:02:27 smithi156 ceph-mon[93066]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2024-07-16T10:02:28.175 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:02:27 smithi156 ceph-mon[93066]: mds.1 [v2:172.21.15.176:6824/1577019287,v1:172.21.15.176:6825/1577019287] down:stopped 2024-07-16T10:02:28.175 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:02:27 smithi156 ceph-mon[93066]: fsmap cephfs:1 {0=cephfs.smithi156.rdibys=up:active} 2 up:standby 2024-07-16T10:02:28.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:02:27 smithi176 ceph-mon[81689]: pgmap v173: 65 pgs: 65 active+clean; 145 MiB data, 689 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 853 B/s wr, 6 op/s 2024-07-16T10:02:28.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:02:27 smithi176 ceph-mon[81689]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2024-07-16T10:02:28.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:02:27 smithi176 ceph-mon[81689]: mds.1 [v2:172.21.15.176:6824/1577019287,v1:172.21.15.176:6825/1577019287] down:stopped 2024-07-16T10:02:28.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:02:27 smithi176 ceph-mon[81689]: fsmap cephfs:1 {0=cephfs.smithi156.rdibys=up:active} 2 up:standby 2024-07-16T10:02:29.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:02:29 smithi176 ceph-mon[81689]: pgmap v174: 65 pgs: 65 active+clean; 145 MiB data, 689 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 938 B/s wr, 3 op/s 2024-07-16T10:02:29.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:02:29 smithi176 ceph-mon[81689]: mds.? [v2:172.21.15.176:6824/3749220343,v1:172.21.15.176:6825/3749220343] up:boot 2024-07-16T10:02:29.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:02:29 smithi176 ceph-mon[81689]: fsmap cephfs:1 {0=cephfs.smithi156.rdibys=up:active} 3 up:standby 2024-07-16T10:02:29.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:02:29 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi176.dvvbud"}]: dispatch 2024-07-16T10:02:29.924 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:02:29 smithi156 ceph-mon[93066]: pgmap v174: 65 pgs: 65 active+clean; 145 MiB data, 689 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 938 B/s wr, 3 op/s 2024-07-16T10:02:29.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:02:29 smithi156 ceph-mon[93066]: mds.? [v2:172.21.15.176:6824/3749220343,v1:172.21.15.176:6825/3749220343] up:boot 2024-07-16T10:02:29.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:02:29 smithi156 ceph-mon[93066]: fsmap cephfs:1 {0=cephfs.smithi156.rdibys=up:active} 3 up:standby 2024-07-16T10:02:29.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:02:29 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi176.dvvbud"}]: dispatch 2024-07-16T10:02:31.843 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:02:31 smithi176 ceph-mon[81689]: pgmap v175: 65 pgs: 65 active+clean; 145 MiB data, 689 MiB used, 536 GiB / 536 GiB avail; 426 B/s wr, 1 op/s 2024-07-16T10:02:31.924 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:02:31 smithi156 ceph-mon[93066]: pgmap v175: 65 pgs: 65 active+clean; 145 MiB data, 689 MiB used, 536 GiB / 536 GiB avail; 426 B/s wr, 1 op/s 2024-07-16T10:02:33.843 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:02:33 smithi176 ceph-mon[81689]: pgmap v176: 65 pgs: 65 active+clean; 145 MiB data, 689 MiB used, 536 GiB / 536 GiB avail; 426 B/s wr, 1 op/s 2024-07-16T10:02:33.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:02:33 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-16T10:02:33.924 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:02:33 smithi156 ceph-mon[93066]: pgmap v176: 65 pgs: 65 active+clean; 145 MiB data, 689 MiB used, 536 GiB / 536 GiB avail; 426 B/s wr, 1 op/s 2024-07-16T10:02:33.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:02:33 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-16T10:02:35.843 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:02:35 smithi176 ceph-mon[81689]: pgmap v177: 65 pgs: 65 active+clean; 145 MiB data, 689 MiB used, 536 GiB / 536 GiB avail; 255 B/s wr, 0 op/s 2024-07-16T10:02:35.924 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:02:35 smithi156 ceph-mon[93066]: pgmap v177: 65 pgs: 65 active+clean; 145 MiB data, 689 MiB used, 536 GiB / 536 GiB avail; 255 B/s wr, 0 op/s 2024-07-16T10:02:37.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:02:37 smithi176 ceph-mon[81689]: pgmap v178: 65 pgs: 65 active+clean; 145 MiB data, 689 MiB used, 536 GiB / 536 GiB avail; 255 B/s wr, 0 op/s 2024-07-16T10:02:37.845 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:02:37 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-16T10:02:37.845 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:02:37 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-16T10:02:37.845 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:02:37 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-16T10:02:37.845 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:02:37 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:02:37.845 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:02:37 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-16T10:02:37.845 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:02:37 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:02:37.845 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:02:37 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:02:37.845 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:02:37 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:02:37.846 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:02:37 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:02:37.846 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:02:37 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "mds ok-to-stop", "ids": ["cephfs.smithi156.rdibys"]}]: dispatch 2024-07-16T10:02:37.846 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:02:37 smithi176 ceph-mon[81689]: Upgrade: It appears safe to stop mds.cephfs.smithi156.rdibys 2024-07-16T10:02:37.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:02:37 smithi156 ceph-mon[93066]: pgmap v178: 65 pgs: 65 active+clean; 145 MiB data, 689 MiB used, 536 GiB / 536 GiB avail; 255 B/s wr, 0 op/s 2024-07-16T10:02:37.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:02:37 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-16T10:02:37.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:02:37 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-16T10:02:37.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:02:37 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-16T10:02:37.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:02:37 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:02:37.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:02:37 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-16T10:02:37.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:02:37 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:02:37.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:02:37 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:02:37.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:02:37 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:02:37.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:02:37 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:02:37.926 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:02:37 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "mds ok-to-stop", "ids": ["cephfs.smithi156.rdibys"]}]: dispatch 2024-07-16T10:02:37.926 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:02:37 smithi156 ceph-mon[93066]: Upgrade: It appears safe to stop mds.cephfs.smithi156.rdibys 2024-07-16T10:02:38.743 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:38.740+0000 7fa741a7c640 1 -- 172.21.15.156:0/3939574404 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa73c0fc360 msgr2=0x7fa73c0fc770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:02:38.744 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:38.740+0000 7fa741a7c640 1 --2- 172.21.15.156:0/3939574404 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa73c0fc360 0x7fa73c0fc770 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fa72c0048d0 tx=0x7fa72c009da0 comp rx=0 tx=0).stop 2024-07-16T10:02:38.744 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:38.740+0000 7fa741a7c640 1 -- 172.21.15.156:0/3939574404 shutdown_connections 2024-07-16T10:02:38.744 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:38.740+0000 7fa741a7c640 1 --2- 172.21.15.156:0/3939574404 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa73c0fd630 0x7fa73c0fdaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:02:38.744 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:38.740+0000 7fa741a7c640 1 --2- 172.21.15.156:0/3939574404 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa73c0fc360 0x7fa73c0fc770 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:02:38.744 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:38.740+0000 7fa741a7c640 1 -- 172.21.15.156:0/3939574404 >> 172.21.15.156:0/3939574404 conn(0x7fa73c0f7b40 msgr2=0x7fa73c0f9f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T10:02:38.744 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:38.740+0000 7fa741a7c640 1 -- 172.21.15.156:0/3939574404 shutdown_connections 2024-07-16T10:02:38.744 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:38.740+0000 7fa741a7c640 1 -- 172.21.15.156:0/3939574404 wait complete. 2024-07-16T10:02:38.744 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:38.741+0000 7fa741a7c640 1 Processor -- start 2024-07-16T10:02:38.744 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:38.741+0000 7fa741a7c640 1 -- start start 2024-07-16T10:02:38.744 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:38.741+0000 7fa741a7c640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa73c0fc360 0x7fa73c10d050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:02:38.744 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:38.741+0000 7fa741a7c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa73c0fd630 0x7fa73c10d590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:02:38.744 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:38.741+0000 7fa741a7c640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa73c10e3b0 con 0x7fa73c0fd630 2024-07-16T10:02:38.744 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:38.741+0000 7fa741a7c640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa73c10e520 con 0x7fa73c0fc360 2024-07-16T10:02:38.745 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:38.741+0000 7fa73affd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa73c0fd630 0x7fa73c10d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:02:38.745 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:38.741+0000 7fa73affd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa73c0fd630 0x7fa73c10d590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:46104/0 (socket says 172.21.15.156:46104) 2024-07-16T10:02:38.745 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:38.741+0000 7fa73affd640 1 -- 172.21.15.156:0/1719884053 learned_addr learned my addr 172.21.15.156:0/1719884053 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T10:02:38.745 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:38.742+0000 7fa73b7fe640 1 --2- 172.21.15.156:0/1719884053 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa73c0fc360 0x7fa73c10d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:02:38.745 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:38.742+0000 7fa73affd640 1 -- 172.21.15.156:0/1719884053 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa73c0fc360 msgr2=0x7fa73c10d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:02:38.745 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:38.742+0000 7fa73affd640 1 --2- 172.21.15.156:0/1719884053 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa73c0fc360 0x7fa73c10d050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:02:38.745 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:38.742+0000 7fa73affd640 1 -- 172.21.15.156:0/1719884053 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa72c005040 con 0x7fa73c0fd630 2024-07-16T10:02:38.745 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:38.742+0000 7fa73affd640 1 --2- 172.21.15.156:0/1719884053 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa73c0fd630 0x7fa73c10d590 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fa73000b790 tx=0x7fa7300091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T10:02:38.745 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:38.742+0000 7fa723fff640 1 -- 172.21.15.156:0/1719884053 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa73000c700 con 0x7fa73c0fd630 2024-07-16T10:02:38.745 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:38.742+0000 7fa741a7c640 1 -- 172.21.15.156:0/1719884053 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa73c10e800 con 0x7fa73c0fd630 2024-07-16T10:02:38.745 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:38.742+0000 7fa723fff640 1 -- 172.21.15.156:0/1719884053 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa73000cd30 con 0x7fa73c0fd630 2024-07-16T10:02:38.745 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:38.742+0000 7fa723fff640 1 -- 172.21.15.156:0/1719884053 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa730013370 con 0x7fa73c0fd630 2024-07-16T10:02:38.746 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:38.742+0000 7fa741a7c640 1 -- 172.21.15.156:0/1719884053 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa73c10ed20 con 0x7fa73c0fd630 2024-07-16T10:02:38.746 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:38.743+0000 7fa741a7c640 1 -- 172.21.15.156:0/1719884053 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa73c0fb7c0 con 0x7fa73c0fd630 2024-07-16T10:02:38.747 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:38.744+0000 7fa723fff640 1 -- 172.21.15.156:0/1719884053 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 38) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fa730012070 con 0x7fa73c0fd630 2024-07-16T10:02:38.747 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:38.744+0000 7fa723fff640 1 --2- 172.21.15.156:0/1719884053 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7fa72406e0b0 0x7fa724070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:02:38.747 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:38.744+0000 7fa723fff640 1 -- 172.21.15.156:0/1719884053 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fa73008d770 con 0x7fa73c0fd630 2024-07-16T10:02:38.747 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:38.744+0000 7fa73b7fe640 1 --2- 172.21.15.156:0/1719884053 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7fa72406e0b0 0x7fa724070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:02:38.747 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:38.745+0000 7fa73b7fe640 1 --2- 172.21.15.156:0/1719884053 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7fa72406e0b0 0x7fa724070560 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fa72c000c00 tx=0x7fa72c035040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T10:02:38.750 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:38.749+0000 7fa723fff640 1 -- 172.21.15.156:0/1719884053 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187716 (secure 0 0 0) 0x7fa730056af0 con 0x7fa73c0fd630 2024-07-16T10:02:38.951 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:38.950+0000 7fa741a7c640 1 -- 172.21.15.156:0/1719884053 --> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa73c10efa0 con 0x7fa72406e0b0 2024-07-16T10:02:38.952 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:38.951+0000 7fa723fff640 1 -- 172.21.15.156:0/1719884053 <== mgr.34104 v2:172.21.15.156:6800/3367765292 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fa73c10efa0 con 0x7fa72406e0b0 2024-07-16T10:02:38.954 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:38.953+0000 7fa741a7c640 1 -- 172.21.15.156:0/1719884053 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7fa72406e0b0 msgr2=0x7fa724070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:02:38.954 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:38.953+0000 7fa741a7c640 1 --2- 172.21.15.156:0/1719884053 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7fa72406e0b0 0x7fa724070560 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fa72c000c00 tx=0x7fa72c035040 comp rx=0 tx=0).stop 2024-07-16T10:02:38.954 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:38.953+0000 7fa741a7c640 1 -- 172.21.15.156:0/1719884053 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa73c0fd630 msgr2=0x7fa73c10d590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:02:38.955 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:38.953+0000 7fa741a7c640 1 --2- 172.21.15.156:0/1719884053 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa73c0fd630 0x7fa73c10d590 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fa73000b790 tx=0x7fa7300091d0 comp rx=0 tx=0).stop 2024-07-16T10:02:38.955 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:38.953+0000 7fa741a7c640 1 -- 172.21.15.156:0/1719884053 shutdown_connections 2024-07-16T10:02:38.955 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:38.953+0000 7fa741a7c640 1 --2- 172.21.15.156:0/1719884053 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7fa72406e0b0 0x7fa724070560 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:02:38.955 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:38.953+0000 7fa741a7c640 1 --2- 172.21.15.156:0/1719884053 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa73c0fd630 0x7fa73c10d590 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:02:38.955 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:38.953+0000 7fa741a7c640 1 --2- 172.21.15.156:0/1719884053 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa73c0fc360 0x7fa73c10d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:02:38.955 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:38.953+0000 7fa741a7c640 1 -- 172.21.15.156:0/1719884053 >> 172.21.15.156:0/1719884053 conn(0x7fa73c0f7b40 msgr2=0x7fa73c0f9220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T10:02:38.955 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:38.953+0000 7fa741a7c640 1 -- 172.21.15.156:0/1719884053 shutdown_connections 2024-07-16T10:02:38.955 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:38.953+0000 7fa741a7c640 1 -- 172.21.15.156:0/1719884053 wait complete. 2024-07-16T10:02:38.964 INFO:teuthology.orchestra.run.smithi156.stdout:true 2024-07-16T10:02:39.036 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:39.034+0000 7f6489e19640 1 -- 172.21.15.156:0/4208274242 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f64840fc330 msgr2=0x7f64840fc740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:02:39.036 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:39.034+0000 7f6489e19640 1 --2- 172.21.15.156:0/4208274242 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f64840fc330 0x7f64840fc740 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f64740048d0 tx=0x7f6474009da0 comp rx=0 tx=0).stop 2024-07-16T10:02:39.036 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:39.034+0000 7f6489e19640 1 -- 172.21.15.156:0/4208274242 shutdown_connections 2024-07-16T10:02:39.036 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:39.034+0000 7f6489e19640 1 --2- 172.21.15.156:0/4208274242 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f64840fd600 0x7f64840fda70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:02:39.036 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:39.034+0000 7f6489e19640 1 --2- 172.21.15.156:0/4208274242 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f64840fc330 0x7f64840fc740 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:02:39.037 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:39.034+0000 7f6489e19640 1 -- 172.21.15.156:0/4208274242 >> 172.21.15.156:0/4208274242 conn(0x7f64840f7af0 msgr2=0x7f64840f9f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T10:02:39.037 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:39.034+0000 7f6489e19640 1 -- 172.21.15.156:0/4208274242 shutdown_connections 2024-07-16T10:02:39.037 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:39.034+0000 7f6489e19640 1 -- 172.21.15.156:0/4208274242 wait complete. 2024-07-16T10:02:39.037 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:39.035+0000 7f6489e19640 1 Processor -- start 2024-07-16T10:02:39.037 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:39.035+0000 7f6489e19640 1 -- start start 2024-07-16T10:02:39.039 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:39.035+0000 7f6489e19640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f64840fc330 0x7f648410d020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:02:39.039 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:39.035+0000 7f6489e19640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f64840fd600 0x7f648410d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:02:39.039 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:39.035+0000 7f6489e19640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f648410e380 con 0x7f64840fd600 2024-07-16T10:02:39.039 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:39.035+0000 7f6489e19640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f648410e4f0 con 0x7f64840fc330 2024-07-16T10:02:39.039 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:39.035+0000 7f6482ffd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f64840fd600 0x7f648410d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:02:39.039 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:39.035+0000 7f6482ffd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f64840fd600 0x7f648410d560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:46114/0 (socket says 172.21.15.156:46114) 2024-07-16T10:02:39.039 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:39.036+0000 7f6482ffd640 1 -- 172.21.15.156:0/2259284922 learned_addr learned my addr 172.21.15.156:0/2259284922 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T10:02:39.039 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:39.036+0000 7f64837fe640 1 --2- 172.21.15.156:0/2259284922 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f64840fc330 0x7f648410d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:02:39.039 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:39.036+0000 7f6482ffd640 1 -- 172.21.15.156:0/2259284922 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f64840fc330 msgr2=0x7f648410d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:02:39.040 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:39.036+0000 7f6482ffd640 1 --2- 172.21.15.156:0/2259284922 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f64840fc330 0x7f648410d020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:02:39.040 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:39.036+0000 7f6482ffd640 1 -- 172.21.15.156:0/2259284922 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6474005040 con 0x7f64840fd600 2024-07-16T10:02:39.040 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:39.036+0000 7f6482ffd640 1 --2- 172.21.15.156:0/2259284922 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f64840fd600 0x7f648410d560 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f647800b790 tx=0x7f64780091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T10:02:39.040 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:39.036+0000 7f646bfff640 1 -- 172.21.15.156:0/2259284922 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f647800c700 con 0x7f64840fd600 2024-07-16T10:02:39.040 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:39.036+0000 7f646bfff640 1 -- 172.21.15.156:0/2259284922 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f647800cd30 con 0x7f64840fd600 2024-07-16T10:02:39.040 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:39.036+0000 7f6489e19640 1 -- 172.21.15.156:0/2259284922 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f648410e7d0 con 0x7f64840fd600 2024-07-16T10:02:39.040 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:39.036+0000 7f646bfff640 1 -- 172.21.15.156:0/2259284922 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6478013370 con 0x7f64840fd600 2024-07-16T10:02:39.040 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:39.036+0000 7f6489e19640 1 -- 172.21.15.156:0/2259284922 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f648410ecf0 con 0x7f64840fd600 2024-07-16T10:02:39.040 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:39.037+0000 7f6489e19640 1 -- 172.21.15.156:0/2259284922 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64840fb790 con 0x7f64840fd600 2024-07-16T10:02:39.040 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:39.038+0000 7f646bfff640 1 -- 172.21.15.156:0/2259284922 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 38) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f6478012070 con 0x7f64840fd600 2024-07-16T10:02:39.041 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:39.038+0000 7f646bfff640 1 --2- 172.21.15.156:0/2259284922 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f646c06e0b0 0x7f646c070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:02:39.041 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:39.038+0000 7f64837fe640 1 --2- 172.21.15.156:0/2259284922 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f646c06e0b0 0x7f646c070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:02:39.041 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:39.038+0000 7f646bfff640 1 -- 172.21.15.156:0/2259284922 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f647808d690 con 0x7f64840fd600 2024-07-16T10:02:39.041 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:39.039+0000 7f64837fe640 1 --2- 172.21.15.156:0/2259284922 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f646c06e0b0 0x7f646c070560 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f6474000c00 tx=0x7f6474035040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T10:02:39.044 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:39.043+0000 7f646bfff640 1 -- 172.21.15.156:0/2259284922 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187716 (secure 0 0 0) 0x7f6478056a90 con 0x7f64840fd600 2024-07-16T10:02:39.245 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:39.243+0000 7f6489e19640 1 -- 172.21.15.156:0/2259284922 --> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f648410efc0 con 0x7f646c06e0b0 2024-07-16T10:02:39.246 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:39.245+0000 7f646bfff640 1 -- 172.21.15.156:0/2259284922 <== mgr.34104 v2:172.21.15.156:6800/3367765292 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f648410efc0 con 0x7f646c06e0b0 2024-07-16T10:02:39.248 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:39.247+0000 7f6489e19640 1 -- 172.21.15.156:0/2259284922 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f646c06e0b0 msgr2=0x7f646c070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:02:39.248 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:39.247+0000 7f6489e19640 1 --2- 172.21.15.156:0/2259284922 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f646c06e0b0 0x7f646c070560 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f6474000c00 tx=0x7f6474035040 comp rx=0 tx=0).stop 2024-07-16T10:02:39.248 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:39.247+0000 7f6489e19640 1 -- 172.21.15.156:0/2259284922 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f64840fd600 msgr2=0x7f648410d560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:02:39.248 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:39.247+0000 7f6489e19640 1 --2- 172.21.15.156:0/2259284922 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f64840fd600 0x7f648410d560 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f647800b790 tx=0x7f64780091d0 comp rx=0 tx=0).stop 2024-07-16T10:02:39.249 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:39.247+0000 7f6489e19640 1 -- 172.21.15.156:0/2259284922 shutdown_connections 2024-07-16T10:02:39.249 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:39.247+0000 7f6489e19640 1 --2- 172.21.15.156:0/2259284922 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f646c06e0b0 0x7f646c070560 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:02:39.249 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:39.247+0000 7f6489e19640 1 --2- 172.21.15.156:0/2259284922 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f64840fd600 0x7f648410d560 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:02:39.249 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:39.247+0000 7f6489e19640 1 --2- 172.21.15.156:0/2259284922 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f64840fc330 0x7f648410d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:02:39.249 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:39.247+0000 7f6489e19640 1 -- 172.21.15.156:0/2259284922 >> 172.21.15.156:0/2259284922 conn(0x7f64840f7af0 msgr2=0x7f64840f9190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T10:02:39.249 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:39.247+0000 7f6489e19640 1 -- 172.21.15.156:0/2259284922 shutdown_connections 2024-07-16T10:02:39.249 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:39.247+0000 7f6489e19640 1 -- 172.21.15.156:0/2259284922 wait complete. 2024-07-16T10:02:39.331 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:39.328+0000 7f165c632640 1 -- 172.21.15.156:0/310985345 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f16540fc510 msgr2=0x7f16540fc980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:02:39.331 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:39.328+0000 7f165c632640 1 --2- 172.21.15.156:0/310985345 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f16540fc510 0x7f16540fc980 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f16500048d0 tx=0x7f1650009da0 comp rx=0 tx=0).stop 2024-07-16T10:02:39.331 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:39.328+0000 7f165c632640 1 -- 172.21.15.156:0/310985345 shutdown_connections 2024-07-16T10:02:39.331 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:39.328+0000 7f165c632640 1 --2- 172.21.15.156:0/310985345 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f16540fc510 0x7f16540fc980 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:02:39.332 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:39.328+0000 7f165c632640 1 --2- 172.21.15.156:0/310985345 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f16540fb240 0x7f16540fb650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:02:39.332 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:39.328+0000 7f165c632640 1 -- 172.21.15.156:0/310985345 >> 172.21.15.156:0/310985345 conn(0x7f16540f6a20 msgr2=0x7f16540f8e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T10:02:39.332 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:39.328+0000 7f165c632640 1 -- 172.21.15.156:0/310985345 shutdown_connections 2024-07-16T10:02:39.332 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:39.328+0000 7f165c632640 1 -- 172.21.15.156:0/310985345 wait complete. 2024-07-16T10:02:39.332 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:39.328+0000 7f165c632640 1 Processor -- start 2024-07-16T10:02:39.332 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:39.329+0000 7f165c632640 1 -- start start 2024-07-16T10:02:39.332 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:39.329+0000 7f165c632640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f16540fb240 0x7f165410fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:02:39.332 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:39.329+0000 7f165c632640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f16540fc510 0x7f1654112230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:02:39.333 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:39.329+0000 7f165c632640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1654065a70 con 0x7f16540fb240 2024-07-16T10:02:39.333 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:39.329+0000 7f165c632640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1654065be0 con 0x7f16540fc510 2024-07-16T10:02:39.333 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:39.329+0000 7f165a3ce640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f16540fb240 0x7f165410fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:02:39.333 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:39.329+0000 7f165a3ce640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f16540fb240 0x7f165410fce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:46128/0 (socket says 172.21.15.156:46128) 2024-07-16T10:02:39.333 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:39.329+0000 7f165a3ce640 1 -- 172.21.15.156:0/2588461411 learned_addr learned my addr 172.21.15.156:0/2588461411 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T10:02:39.333 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:39.329+0000 7f1659bcd640 1 --2- 172.21.15.156:0/2588461411 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f16540fc510 0x7f1654112230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:02:39.334 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:39.329+0000 7f165a3ce640 1 -- 172.21.15.156:0/2588461411 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f16540fc510 msgr2=0x7f1654112230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:02:39.334 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:39.329+0000 7f165a3ce640 1 --2- 172.21.15.156:0/2588461411 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f16540fc510 0x7f1654112230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:02:39.334 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:39.329+0000 7f165a3ce640 1 -- 172.21.15.156:0/2588461411 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1650005040 con 0x7f16540fb240 2024-07-16T10:02:39.334 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:39.329+0000 7f165a3ce640 1 --2- 172.21.15.156:0/2588461411 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f16540fb240 0x7f165410fce0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f164400eb70 tx=0x7f164400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T10:02:39.334 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:39.330+0000 7f164affd640 1 -- 172.21.15.156:0/2588461411 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1644007c30 con 0x7f16540fb240 2024-07-16T10:02:39.334 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:39.330+0000 7f165c632640 1 -- 172.21.15.156:0/2588461411 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1654112770 con 0x7f16540fb240 2024-07-16T10:02:39.334 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:39.330+0000 7f164affd640 1 -- 172.21.15.156:0/2588461411 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f164400f040 con 0x7f16540fb240 2024-07-16T10:02:39.334 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:39.330+0000 7f164affd640 1 -- 172.21.15.156:0/2588461411 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f16440095b0 con 0x7f16540fb240 2024-07-16T10:02:39.334 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:39.330+0000 7f165c632640 1 -- 172.21.15.156:0/2588461411 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1654112ce0 con 0x7f16540fb240 2024-07-16T10:02:39.335 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:39.331+0000 7f165c632640 1 -- 172.21.15.156:0/2588461411 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16540fa6a0 con 0x7f16540fb240 2024-07-16T10:02:39.335 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:39.331+0000 7f164affd640 1 -- 172.21.15.156:0/2588461411 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 38) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f1644009730 con 0x7f16540fb240 2024-07-16T10:02:39.335 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:39.332+0000 7f164affd640 1 --2- 172.21.15.156:0/2588461411 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f164006e0b0 0x7f1640070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:02:39.335 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:39.332+0000 7f164affd640 1 -- 172.21.15.156:0/2588461411 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f164408d8a0 con 0x7f16540fb240 2024-07-16T10:02:39.335 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:39.332+0000 7f1659bcd640 1 --2- 172.21.15.156:0/2588461411 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f164006e0b0 0x7f1640070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:02:39.336 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:39.332+0000 7f1659bcd640 1 --2- 172.21.15.156:0/2588461411 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f164006e0b0 0x7f1640070560 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f1650000c00 tx=0x7f1650035040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T10:02:39.337 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:39.336+0000 7f164affd640 1 -- 172.21.15.156:0/2588461411 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187716 (secure 0 0 0) 0x7f1644056c20 con 0x7f16540fb240 2024-07-16T10:02:39.489 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:02:39 smithi156 ceph-mon[93066]: Upgrade: Updating mds.cephfs.smithi156.rdibys 2024-07-16T10:02:39.489 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:02:39 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:02:39.489 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:02:39 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi156.rdibys", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-16T10:02:39.489 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:02:39 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-16T10:02:39.489 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:02:39 smithi156 ceph-mon[93066]: Deploying daemon mds.cephfs.smithi156.rdibys on smithi156 2024-07-16T10:02:39.528 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:39.526+0000 7f165c632640 1 -- 172.21.15.156:0/2588461411 --> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f1654100880 con 0x7f164006e0b0 2024-07-16T10:02:39.537 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:39.536+0000 7f164affd640 1 -- 172.21.15.156:0/2588461411 <== mgr.34104 v2:172.21.15.156:6800/3367765292 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7f1654100880 con 0x7f164006e0b0 2024-07-16T10:02:39.537 INFO:teuthology.orchestra.run.smithi156.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-07-16T10:02:39.538 INFO:teuthology.orchestra.run.smithi156.stdout:alertmanager.smithi156 smithi156 *:9093,9094 running (6m) 2m ago 14m 25.0M - 0.25.0 c8568f914cd2 d524a2c3ff15 2024-07-16T10:02:39.538 INFO:teuthology.orchestra.run.smithi156.stdout:crash.smithi156 smithi156 running (4m) 2m ago 15m 6899k - 19.0.0-3495-ga6ca461b 16f7feb04342 11648cf29af4 2024-07-16T10:02:39.538 INFO:teuthology.orchestra.run.smithi156.stdout:crash.smithi176 smithi176 running (3m) 50s ago 14m 6899k - 19.0.0-3495-ga6ca461b 16f7feb04342 67f3ccc034f8 2024-07-16T10:02:39.538 INFO:teuthology.orchestra.run.smithi156.stdout:grafana.smithi156 smithi156 *:3000 running (6m) 2m ago 14m 77.8M - 9.4.7 954c08fa6188 deb9e1bfeb66 2024-07-16T10:02:39.538 INFO:teuthology.orchestra.run.smithi156.stdout:mds.cephfs.smithi156.evwwaz smithi156 running (9m) 2m ago 9m 67.2M - 17.2.7-1104-ge6231b04 d5de5f953b95 7c0ff9ae43f8 2024-07-16T10:02:39.538 INFO:teuthology.orchestra.run.smithi156.stdout:mds.cephfs.smithi156.rdibys smithi156 running (9m) 2m ago 9m 120M - 17.2.7-1104-ge6231b04 d5de5f953b95 b985cd1e6924 2024-07-16T10:02:39.538 INFO:teuthology.orchestra.run.smithi156.stdout:mds.cephfs.smithi176.dvvbud smithi176 running (9m) 50s ago 9m 108M - 17.2.7-1104-ge6231b04 d5de5f953b95 1b00147950f9 2024-07-16T10:02:39.539 INFO:teuthology.orchestra.run.smithi156.stdout:mds.cephfs.smithi176.wahbsi smithi176 running (9m) 50s ago 9m 62.3M - 17.2.7-1104-ge6231b04 d5de5f953b95 866664af2311 2024-07-16T10:02:39.539 INFO:teuthology.orchestra.run.smithi156.stdout:mgr.smithi156.zkvylv smithi156 *:8443,9283,8765 running (5m) 2m ago 16m 497M - 19.0.0-3495-ga6ca461b 16f7feb04342 ee53bd604b7c 2024-07-16T10:02:39.539 INFO:teuthology.orchestra.run.smithi156.stdout:mgr.smithi176.dqbrvx smithi176 *:8443,9283,8765 running (5m) 50s ago 13m 441M - 19.0.0-3495-ga6ca461b 16f7feb04342 ce9eaae4c2d0 2024-07-16T10:02:39.539 INFO:teuthology.orchestra.run.smithi156.stdout:mon.smithi156 smithi156 running (4m) 2m ago 16m 45.4M 2048M 19.0.0-3495-ga6ca461b 16f7feb04342 77b6ec8c50eb 2024-07-16T10:02:39.539 INFO:teuthology.orchestra.run.smithi156.stdout:mon.smithi176 smithi176 running (4m) 50s ago 13m 40.4M 2048M 19.0.0-3495-ga6ca461b 16f7feb04342 ef92814cec2b 2024-07-16T10:02:39.539 INFO:teuthology.orchestra.run.smithi156.stdout:node-exporter.smithi156 smithi156 *:9100 running (6m) 2m ago 14m 17.2M - 1.5.0 0da6a335fe13 e5d86f23a14c 2024-07-16T10:02:39.539 INFO:teuthology.orchestra.run.smithi156.stdout:node-exporter.smithi176 smithi176 *:9100 running (5m) 50s ago 14m 15.6M - 1.5.0 0da6a335fe13 ea842728144a 2024-07-16T10:02:39.539 INFO:teuthology.orchestra.run.smithi156.stdout:osd.0 smithi156 running (3m) 2m ago 13m 93.7M 1557M 19.0.0-3495-ga6ca461b 16f7feb04342 8633df1689cc 2024-07-16T10:02:39.540 INFO:teuthology.orchestra.run.smithi156.stdout:osd.1 smithi156 running (3m) 2m ago 12m 101M 1557M 19.0.0-3495-ga6ca461b 16f7feb04342 e6fae9653b46 2024-07-16T10:02:39.540 INFO:teuthology.orchestra.run.smithi156.stdout:osd.2 smithi156 running (2m) 2m ago 12m 11.8M 1557M 19.0.0-3495-ga6ca461b 16f7feb04342 c6ecf18de9bf 2024-07-16T10:02:39.540 INFO:teuthology.orchestra.run.smithi156.stdout:osd.3 smithi176 running (116s) 50s ago 11m 99.7M 2581M 19.0.0-3495-ga6ca461b 16f7feb04342 d216534d6027 2024-07-16T10:02:39.540 INFO:teuthology.orchestra.run.smithi156.stdout:osd.4 smithi176 running (85s) 50s ago 11m 100M 2581M 19.0.0-3495-ga6ca461b 16f7feb04342 1bd3a25317b0 2024-07-16T10:02:39.540 INFO:teuthology.orchestra.run.smithi156.stdout:osd.5 smithi176 running (53s) 50s ago 11m 11.4M 2581M 19.0.0-3495-ga6ca461b 16f7feb04342 a03640221e7a 2024-07-16T10:02:39.540 INFO:teuthology.orchestra.run.smithi156.stdout:prometheus.smithi156 smithi156 *:9095 running (5m) 2m ago 14m 64.9M - 2.43.0 a07b618ecd1d 20c195ecd665 2024-07-16T10:02:39.541 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:39.538+0000 7f165c632640 1 -- 172.21.15.156:0/2588461411 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f164006e0b0 msgr2=0x7f1640070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:02:39.541 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:39.538+0000 7f165c632640 1 --2- 172.21.15.156:0/2588461411 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f164006e0b0 0x7f1640070560 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f1650000c00 tx=0x7f1650035040 comp rx=0 tx=0).stop 2024-07-16T10:02:39.541 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:39.538+0000 7f165c632640 1 -- 172.21.15.156:0/2588461411 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f16540fb240 msgr2=0x7f165410fce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:02:39.541 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:39.538+0000 7f165c632640 1 --2- 172.21.15.156:0/2588461411 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f16540fb240 0x7f165410fce0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f164400eb70 tx=0x7f164400b040 comp rx=0 tx=0).stop 2024-07-16T10:02:39.542 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:39.538+0000 7f165c632640 1 -- 172.21.15.156:0/2588461411 shutdown_connections 2024-07-16T10:02:39.542 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:39.538+0000 7f165c632640 1 --2- 172.21.15.156:0/2588461411 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f164006e0b0 0x7f1640070560 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:02:39.542 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:39.538+0000 7f165c632640 1 --2- 172.21.15.156:0/2588461411 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f16540fc510 0x7f1654112230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:02:39.542 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:39.538+0000 7f165c632640 1 --2- 172.21.15.156:0/2588461411 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f16540fb240 0x7f165410fce0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:02:39.542 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:39.538+0000 7f165c632640 1 -- 172.21.15.156:0/2588461411 >> 172.21.15.156:0/2588461411 conn(0x7f16540f6a20 msgr2=0x7f16540f8140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T10:02:39.542 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:39.538+0000 7f165c632640 1 -- 172.21.15.156:0/2588461411 shutdown_connections 2024-07-16T10:02:39.542 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:39.538+0000 7f165c632640 1 -- 172.21.15.156:0/2588461411 wait complete. 2024-07-16T10:02:39.598 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:02:39 smithi176 ceph-mon[81689]: Upgrade: Updating mds.cephfs.smithi156.rdibys 2024-07-16T10:02:39.598 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:02:39 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:02:39.598 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:02:39 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi156.rdibys", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-16T10:02:39.598 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:02:39 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-16T10:02:39.598 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:02:39 smithi176 ceph-mon[81689]: Deploying daemon mds.cephfs.smithi156.rdibys on smithi156 2024-07-16T10:02:39.618 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:39.616+0000 7f2eb7128640 1 -- 172.21.15.156:0/3849535389 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2eb00fa080 msgr2=0x7f2eb00fa490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:02:39.618 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:39.616+0000 7f2eb7128640 1 --2- 172.21.15.156:0/3849535389 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2eb00fa080 0x7f2eb00fa490 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f2ea80048d0 tx=0x7f2ea8009da0 comp rx=0 tx=0).stop 2024-07-16T10:02:39.618 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:39.616+0000 7f2eb7128640 1 -- 172.21.15.156:0/3849535389 shutdown_connections 2024-07-16T10:02:39.618 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:39.616+0000 7f2eb7128640 1 --2- 172.21.15.156:0/3849535389 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2eb00fa9d0 0x7f2eb0101c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:02:39.618 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:39.616+0000 7f2eb7128640 1 --2- 172.21.15.156:0/3849535389 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2eb00fa080 0x7f2eb00fa490 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:02:39.618 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:39.616+0000 7f2eb7128640 1 -- 172.21.15.156:0/3849535389 >> 172.21.15.156:0/3849535389 conn(0x7f2eb00f5d30 msgr2=0x7f2eb00f8160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T10:02:39.618 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:39.616+0000 7f2eb7128640 1 -- 172.21.15.156:0/3849535389 shutdown_connections 2024-07-16T10:02:39.619 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:39.616+0000 7f2eb7128640 1 -- 172.21.15.156:0/3849535389 wait complete. 2024-07-16T10:02:39.620 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:39.617+0000 7f2eb7128640 1 Processor -- start 2024-07-16T10:02:39.620 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:39.617+0000 7f2eb7128640 1 -- start start 2024-07-16T10:02:39.620 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:39.617+0000 7f2eb7128640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2eb00fa080 0x7f2eb010cf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:02:39.620 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:39.617+0000 7f2eb7128640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2eb00fa9d0 0x7f2eb010d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:02:39.620 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:39.617+0000 7f2eb7128640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2eb010db00 con 0x7f2eb00fa080 2024-07-16T10:02:39.620 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:39.617+0000 7f2eb7128640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2eb010e320 con 0x7f2eb00fa9d0 2024-07-16T10:02:39.620 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:39.618+0000 7f2eb4ec4640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2eb00fa080 0x7f2eb010cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:02:39.620 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:39.618+0000 7f2eb4ec4640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2eb00fa080 0x7f2eb010cf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:46146/0 (socket says 172.21.15.156:46146) 2024-07-16T10:02:39.620 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:39.618+0000 7f2eb4ec4640 1 -- 172.21.15.156:0/1802305078 learned_addr learned my addr 172.21.15.156:0/1802305078 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T10:02:39.621 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:39.618+0000 7f2eaffff640 1 --2- 172.21.15.156:0/1802305078 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2eb00fa9d0 0x7f2eb010d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:02:39.621 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:39.618+0000 7f2eb4ec4640 1 -- 172.21.15.156:0/1802305078 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2eb00fa9d0 msgr2=0x7f2eb010d4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:02:39.621 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:39.618+0000 7f2eb4ec4640 1 --2- 172.21.15.156:0/1802305078 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2eb00fa9d0 0x7f2eb010d4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:02:39.621 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:39.618+0000 7f2eb4ec4640 1 -- 172.21.15.156:0/1802305078 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ea8005040 con 0x7f2eb00fa080 2024-07-16T10:02:39.621 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:39.618+0000 7f2eb4ec4640 1 --2- 172.21.15.156:0/1802305078 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2eb00fa080 0x7f2eb010cf70 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f2ea8000c00 tx=0x7f2ea8009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T10:02:39.621 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:39.618+0000 7f2ead7fa640 1 -- 172.21.15.156:0/1802305078 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2ea8038070 con 0x7f2eb00fa080 2024-07-16T10:02:39.621 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:39.618+0000 7f2eb7128640 1 -- 172.21.15.156:0/1802305078 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2eb010e550 con 0x7f2eb00fa080 2024-07-16T10:02:39.621 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:39.618+0000 7f2ead7fa640 1 -- 172.21.15.156:0/1802305078 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2ea8032970 con 0x7f2eb00fa080 2024-07-16T10:02:39.621 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:39.618+0000 7f2ead7fa640 1 -- 172.21.15.156:0/1802305078 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2ea80338e0 con 0x7f2eb00fa080 2024-07-16T10:02:39.621 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:39.618+0000 7f2eb7128640 1 -- 172.21.15.156:0/1802305078 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2eb010ea10 con 0x7f2eb00fa080 2024-07-16T10:02:39.622 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:39.619+0000 7f2eb7128640 1 -- 172.21.15.156:0/1802305078 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2eb00f99d0 con 0x7f2eb00fa080 2024-07-16T10:02:39.622 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:39.620+0000 7f2ead7fa640 1 -- 172.21.15.156:0/1802305078 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 38) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f2ea8030020 con 0x7f2eb00fa080 2024-07-16T10:02:39.622 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:39.620+0000 7f2ead7fa640 1 --2- 172.21.15.156:0/1802305078 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f2e9c06e0b0 0x7f2e9c070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:02:39.622 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:39.620+0000 7f2ead7fa640 1 -- 172.21.15.156:0/1802305078 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f2ea80ada30 con 0x7f2eb00fa080 2024-07-16T10:02:39.622 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:39.620+0000 7f2eaffff640 1 --2- 172.21.15.156:0/1802305078 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f2e9c06e0b0 0x7f2e9c070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:02:39.622 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:39.621+0000 7f2eaffff640 1 --2- 172.21.15.156:0/1802305078 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f2e9c06e0b0 0x7f2e9c070560 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f2eb010cd20 tx=0x7f2ea000b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T10:02:39.625 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:39.624+0000 7f2ead7fa640 1 -- 172.21.15.156:0/1802305078 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187716 (secure 0 0 0) 0x7f2ea8076db0 con 0x7f2eb00fa080 2024-07-16T10:02:39.891 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:39.888+0000 7f2eb7128640 1 -- 172.21.15.156:0/1802305078 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f2eb004b530 con 0x7f2eb00fa080 2024-07-16T10:02:39.891 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:39.888+0000 7f2ead7fa640 1 -- 172.21.15.156:0/1802305078 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+731 (secure 0 0 0) 0x7f2ea807a470 con 0x7f2eb00fa080 2024-07-16T10:02:39.891 INFO:teuthology.orchestra.run.smithi156.stdout:{ 2024-07-16T10:02:39.891 INFO:teuthology.orchestra.run.smithi156.stdout: "mon": { 2024-07-16T10:02:39.891 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph version 19.0.0-3495-ga6ca461b (a6ca461be7cc1aff3658c2c6df4cff975a24449a) squid (rc)": 2 2024-07-16T10:02:39.891 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-07-16T10:02:39.892 INFO:teuthology.orchestra.run.smithi156.stdout: "mgr": { 2024-07-16T10:02:39.892 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph version 19.0.0-3495-ga6ca461b (a6ca461be7cc1aff3658c2c6df4cff975a24449a) squid (rc)": 2 2024-07-16T10:02:39.892 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-07-16T10:02:39.892 INFO:teuthology.orchestra.run.smithi156.stdout: "osd": { 2024-07-16T10:02:39.892 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph version 19.0.0-3495-ga6ca461b (a6ca461be7cc1aff3658c2c6df4cff975a24449a) squid (rc)": 6 2024-07-16T10:02:39.892 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-07-16T10:02:39.892 INFO:teuthology.orchestra.run.smithi156.stdout: "mds": { 2024-07-16T10:02:39.892 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph version 17.2.7-1104-ge6231b04 (e6231b04b52ebf3b9b31803282dd3074439e51ac) quincy (stable)": 4 2024-07-16T10:02:39.892 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-07-16T10:02:39.892 INFO:teuthology.orchestra.run.smithi156.stdout: "overall": { 2024-07-16T10:02:39.892 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph version 17.2.7-1104-ge6231b04 (e6231b04b52ebf3b9b31803282dd3074439e51ac) quincy (stable)": 4, 2024-07-16T10:02:39.892 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph version 19.0.0-3495-ga6ca461b (a6ca461be7cc1aff3658c2c6df4cff975a24449a) squid (rc)": 10 2024-07-16T10:02:39.893 INFO:teuthology.orchestra.run.smithi156.stdout: } 2024-07-16T10:02:39.893 INFO:teuthology.orchestra.run.smithi156.stdout:} 2024-07-16T10:02:39.893 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:39.891+0000 7f2eb7128640 1 -- 172.21.15.156:0/1802305078 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f2e9c06e0b0 msgr2=0x7f2e9c070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:02:39.893 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:39.891+0000 7f2eb7128640 1 --2- 172.21.15.156:0/1802305078 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f2e9c06e0b0 0x7f2e9c070560 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f2eb010cd20 tx=0x7f2ea000b040 comp rx=0 tx=0).stop 2024-07-16T10:02:39.893 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:39.891+0000 7f2eb7128640 1 -- 172.21.15.156:0/1802305078 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2eb00fa080 msgr2=0x7f2eb010cf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:02:39.893 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:39.891+0000 7f2eb7128640 1 --2- 172.21.15.156:0/1802305078 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2eb00fa080 0x7f2eb010cf70 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f2ea8000c00 tx=0x7f2ea8009da0 comp rx=0 tx=0).stop 2024-07-16T10:02:39.894 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:39.891+0000 7f2eb7128640 1 -- 172.21.15.156:0/1802305078 shutdown_connections 2024-07-16T10:02:39.894 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:39.891+0000 7f2eb7128640 1 --2- 172.21.15.156:0/1802305078 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f2e9c06e0b0 0x7f2e9c070560 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:02:39.894 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:39.891+0000 7f2eb7128640 1 --2- 172.21.15.156:0/1802305078 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2eb00fa9d0 0x7f2eb010d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:02:39.894 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:39.891+0000 7f2eb7128640 1 --2- 172.21.15.156:0/1802305078 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2eb00fa080 0x7f2eb010cf70 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:02:39.894 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:39.891+0000 7f2eb7128640 1 -- 172.21.15.156:0/1802305078 >> 172.21.15.156:0/1802305078 conn(0x7f2eb00f5d30 msgr2=0x7f2eb00ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T10:02:39.894 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:39.891+0000 7f2eb7128640 1 -- 172.21.15.156:0/1802305078 shutdown_connections 2024-07-16T10:02:39.894 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:39.891+0000 7f2eb7128640 1 -- 172.21.15.156:0/1802305078 wait complete. 2024-07-16T10:02:39.979 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:39.977+0000 7fa93ebc8640 1 -- 172.21.15.156:0/2233739749 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa9380fb240 msgr2=0x7fa9380fb650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:02:39.979 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:39.977+0000 7fa93ebc8640 1 --2- 172.21.15.156:0/2233739749 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa9380fb240 0x7fa9380fb650 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fa92c0048d0 tx=0x7fa92c009da0 comp rx=0 tx=0).stop 2024-07-16T10:02:39.980 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:39.977+0000 7fa93ebc8640 1 -- 172.21.15.156:0/2233739749 shutdown_connections 2024-07-16T10:02:39.980 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:39.977+0000 7fa93ebc8640 1 --2- 172.21.15.156:0/2233739749 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa9380fc510 0x7fa9380fc980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:02:39.980 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:39.977+0000 7fa93ebc8640 1 --2- 172.21.15.156:0/2233739749 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa9380fb240 0x7fa9380fb650 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:02:39.980 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:39.977+0000 7fa93ebc8640 1 -- 172.21.15.156:0/2233739749 >> 172.21.15.156:0/2233739749 conn(0x7fa9380f6a20 msgr2=0x7fa9380f8e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T10:02:39.980 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:39.977+0000 7fa93ebc8640 1 -- 172.21.15.156:0/2233739749 shutdown_connections 2024-07-16T10:02:39.980 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:39.977+0000 7fa93ebc8640 1 -- 172.21.15.156:0/2233739749 wait complete. 2024-07-16T10:02:39.980 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:39.977+0000 7fa93ebc8640 1 Processor -- start 2024-07-16T10:02:39.980 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:39.977+0000 7fa93ebc8640 1 -- start start 2024-07-16T10:02:39.980 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:39.978+0000 7fa93ebc8640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa9380fb240 0x7fa93810bee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:02:39.980 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:39.978+0000 7fa93ebc8640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa9380fc510 0x7fa93810c420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:02:39.980 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:39.978+0000 7fa93ebc8640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa93810d150 con 0x7fa9380fb240 2024-07-16T10:02:39.980 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:39.978+0000 7fa93ebc8640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa93810d2c0 con 0x7fa9380fc510 2024-07-16T10:02:39.981 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:39.978+0000 7fa93c964640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa9380fb240 0x7fa93810bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:02:39.981 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:39.978+0000 7fa93c964640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa9380fb240 0x7fa93810bee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:46160/0 (socket says 172.21.15.156:46160) 2024-07-16T10:02:39.981 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:39.978+0000 7fa93c964640 1 -- 172.21.15.156:0/2500794405 learned_addr learned my addr 172.21.15.156:0/2500794405 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T10:02:39.981 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:39.978+0000 7fa937fff640 1 --2- 172.21.15.156:0/2500794405 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa9380fc510 0x7fa93810c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:02:39.981 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:39.978+0000 7fa93c964640 1 -- 172.21.15.156:0/2500794405 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa9380fc510 msgr2=0x7fa93810c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:02:39.981 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:39.978+0000 7fa93c964640 1 --2- 172.21.15.156:0/2500794405 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa9380fc510 0x7fa93810c420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:02:39.981 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:39.978+0000 7fa93c964640 1 -- 172.21.15.156:0/2500794405 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa92c005040 con 0x7fa9380fb240 2024-07-16T10:02:39.983 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:39.979+0000 7fa93c964640 1 --2- 172.21.15.156:0/2500794405 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa9380fb240 0x7fa93810bee0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fa92c009f80 tx=0x7fa92c009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T10:02:39.983 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:39.979+0000 7fa9357fa640 1 -- 172.21.15.156:0/2500794405 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa92c038070 con 0x7fa9380fb240 2024-07-16T10:02:39.983 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:39.979+0000 7fa93ebc8640 1 -- 172.21.15.156:0/2500794405 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa93810d540 con 0x7fa9380fb240 2024-07-16T10:02:39.983 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:39.979+0000 7fa93ebc8640 1 -- 172.21.15.156:0/2500794405 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa93810da80 con 0x7fa9380fb240 2024-07-16T10:02:39.983 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:39.979+0000 7fa9357fa640 1 -- 172.21.15.156:0/2500794405 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa92c032970 con 0x7fa9380fb240 2024-07-16T10:02:39.983 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:39.979+0000 7fa9357fa640 1 -- 172.21.15.156:0/2500794405 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa92c033860 con 0x7fa9380fb240 2024-07-16T10:02:39.983 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:39.980+0000 7fa93ebc8640 1 -- 172.21.15.156:0/2500794405 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa91c005240 con 0x7fa9380fb240 2024-07-16T10:02:39.983 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:39.980+0000 7fa9357fa640 1 -- 172.21.15.156:0/2500794405 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 38) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fa92c030020 con 0x7fa9380fb240 2024-07-16T10:02:39.983 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:39.981+0000 7fa9357fa640 1 --2- 172.21.15.156:0/2500794405 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7fa92806e0b0 0x7fa928070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:02:39.983 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:39.981+0000 7fa9357fa640 1 -- 172.21.15.156:0/2500794405 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fa92c0ad800 con 0x7fa9380fb240 2024-07-16T10:02:39.983 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:39.981+0000 7fa937fff640 1 --2- 172.21.15.156:0/2500794405 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7fa92806e0b0 0x7fa928070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:02:39.984 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:39.981+0000 7fa937fff640 1 --2- 172.21.15.156:0/2500794405 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7fa92806e0b0 0x7fa928070560 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fa93810bc90 tx=0x7fa92400b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T10:02:39.986 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:39.985+0000 7fa9357fa640 1 -- 172.21.15.156:0/2500794405 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187716 (secure 0 0 0) 0x7fa92c076cb0 con 0x7fa9380fb240 2024-07-16T10:02:40.224 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:40.222+0000 7fa93ebc8640 1 -- 172.21.15.156:0/2500794405 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fa91c005d00 con 0x7fa9380fb240 2024-07-16T10:02:40.224 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:40.223+0000 7fa9357fa640 1 -- 172.21.15.156:0/2500794405 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 26 v26) v1 ==== 76+0+1893 (secure 0 0 0) 0x7fa92c07a370 con 0x7fa9380fb240 2024-07-16T10:02:40.224 INFO:teuthology.orchestra.run.smithi156.stdout:e26 2024-07-16T10:02:40.225 INFO:teuthology.orchestra.run.smithi156.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-07-16T10:02:40.225 INFO:teuthology.orchestra.run.smithi156.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-07-16T10:02:40.225 INFO:teuthology.orchestra.run.smithi156.stdout:legacy client fscid: 1 2024-07-16T10:02:40.225 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-07-16T10:02:40.225 INFO:teuthology.orchestra.run.smithi156.stdout:Filesystem 'cephfs' (1) 2024-07-16T10:02:40.225 INFO:teuthology.orchestra.run.smithi156.stdout:fs_name cephfs 2024-07-16T10:02:40.225 INFO:teuthology.orchestra.run.smithi156.stdout:epoch 25 2024-07-16T10:02:40.225 INFO:teuthology.orchestra.run.smithi156.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-07-16T10:02:40.225 INFO:teuthology.orchestra.run.smithi156.stdout:created 2024-07-16T09:52:55.214975+0000 2024-07-16T10:02:40.225 INFO:teuthology.orchestra.run.smithi156.stdout:modified 2024-07-16T10:02:26.579199+0000 2024-07-16T10:02:40.225 INFO:teuthology.orchestra.run.smithi156.stdout:tableserver 0 2024-07-16T10:02:40.225 INFO:teuthology.orchestra.run.smithi156.stdout:root 0 2024-07-16T10:02:40.226 INFO:teuthology.orchestra.run.smithi156.stdout:session_timeout 60 2024-07-16T10:02:40.226 INFO:teuthology.orchestra.run.smithi156.stdout:session_autoclose 300 2024-07-16T10:02:40.226 INFO:teuthology.orchestra.run.smithi156.stdout:max_file_size 1099511627776 2024-07-16T10:02:40.226 INFO:teuthology.orchestra.run.smithi156.stdout:max_xattr_size 65536 2024-07-16T10:02:40.226 INFO:teuthology.orchestra.run.smithi156.stdout:required_client_features {} 2024-07-16T10:02:40.226 INFO:teuthology.orchestra.run.smithi156.stdout:last_failure 0 2024-07-16T10:02:40.226 INFO:teuthology.orchestra.run.smithi156.stdout:last_failure_osd_epoch 0 2024-07-16T10:02:40.226 INFO:teuthology.orchestra.run.smithi156.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-07-16T10:02:40.226 INFO:teuthology.orchestra.run.smithi156.stdout:max_mds 1 2024-07-16T10:02:40.227 INFO:teuthology.orchestra.run.smithi156.stdout:in 0 2024-07-16T10:02:40.227 INFO:teuthology.orchestra.run.smithi156.stdout:up {0=14458} 2024-07-16T10:02:40.227 INFO:teuthology.orchestra.run.smithi156.stdout:failed 2024-07-16T10:02:40.227 INFO:teuthology.orchestra.run.smithi156.stdout:damaged 2024-07-16T10:02:40.227 INFO:teuthology.orchestra.run.smithi156.stdout:stopped 1 2024-07-16T10:02:40.227 INFO:teuthology.orchestra.run.smithi156.stdout:data_pools [3] 2024-07-16T10:02:40.228 INFO:teuthology.orchestra.run.smithi156.stdout:metadata_pool 2 2024-07-16T10:02:40.228 INFO:teuthology.orchestra.run.smithi156.stdout:inline_data disabled 2024-07-16T10:02:40.228 INFO:teuthology.orchestra.run.smithi156.stdout:balancer 2024-07-16T10:02:40.228 INFO:teuthology.orchestra.run.smithi156.stdout:bal_rank_mask -1 2024-07-16T10:02:40.228 INFO:teuthology.orchestra.run.smithi156.stdout:standby_count_wanted 1 2024-07-16T10:02:40.228 INFO:teuthology.orchestra.run.smithi156.stdout:qdb_cluster leader: 14458 members: 14458 2024-07-16T10:02:40.228 INFO:teuthology.orchestra.run.smithi156.stdout:[mds.cephfs.smithi156.rdibys{0:14458} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.156:6826/1080176013,v1:172.21.15.156:6827/1080176013] compat {c=[1],r=[1],i=[7ff]}] 2024-07-16T10:02:40.228 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-07-16T10:02:40.228 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-07-16T10:02:40.228 INFO:teuthology.orchestra.run.smithi156.stdout:Standby daemons: 2024-07-16T10:02:40.228 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-07-16T10:02:40.229 INFO:teuthology.orchestra.run.smithi156.stdout:[mds.cephfs.smithi156.evwwaz{-1:34374} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.156:6828/333092891,v1:172.21.15.156:6829/333092891] compat {c=[1],r=[1],i=[7ff]}] 2024-07-16T10:02:40.229 INFO:teuthology.orchestra.run.smithi156.stdout:[mds.cephfs.smithi176.wahbsi{-1:44269} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.176:6826/3942496125,v1:172.21.15.176:6827/3942496125] compat {c=[1],r=[1],i=[7ff]}] 2024-07-16T10:02:40.229 INFO:teuthology.orchestra.run.smithi156.stdout:[mds.cephfs.smithi176.dvvbud{-1:44301} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.176:6824/3749220343,v1:172.21.15.176:6825/3749220343] compat {c=[1],r=[1],i=[7ff]}] 2024-07-16T10:02:40.229 INFO:teuthology.orchestra.run.smithi156.stderr:dumped fsmap epoch 26 2024-07-16T10:02:40.229 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:40.225+0000 7fa93ebc8640 1 -- 172.21.15.156:0/2500794405 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7fa92806e0b0 msgr2=0x7fa928070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:02:40.229 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:40.225+0000 7fa93ebc8640 1 --2- 172.21.15.156:0/2500794405 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7fa92806e0b0 0x7fa928070560 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fa93810bc90 tx=0x7fa92400b040 comp rx=0 tx=0).stop 2024-07-16T10:02:40.230 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:40.225+0000 7fa93ebc8640 1 -- 172.21.15.156:0/2500794405 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa9380fb240 msgr2=0x7fa93810bee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:02:40.230 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:40.225+0000 7fa93ebc8640 1 --2- 172.21.15.156:0/2500794405 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa9380fb240 0x7fa93810bee0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fa92c009f80 tx=0x7fa92c009da0 comp rx=0 tx=0).stop 2024-07-16T10:02:40.230 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:40.225+0000 7fa93ebc8640 1 -- 172.21.15.156:0/2500794405 shutdown_connections 2024-07-16T10:02:40.230 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:40.225+0000 7fa93ebc8640 1 --2- 172.21.15.156:0/2500794405 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7fa92806e0b0 0x7fa928070560 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:02:40.230 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:40.225+0000 7fa93ebc8640 1 --2- 172.21.15.156:0/2500794405 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa9380fc510 0x7fa93810c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:02:40.230 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:40.225+0000 7fa93ebc8640 1 --2- 172.21.15.156:0/2500794405 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa9380fb240 0x7fa93810bee0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:02:40.230 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:40.225+0000 7fa93ebc8640 1 -- 172.21.15.156:0/2500794405 >> 172.21.15.156:0/2500794405 conn(0x7fa9380f6a20 msgr2=0x7fa9380f8110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T10:02:40.230 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:40.225+0000 7fa93ebc8640 1 -- 172.21.15.156:0/2500794405 shutdown_connections 2024-07-16T10:02:40.230 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:40.225+0000 7fa93ebc8640 1 -- 172.21.15.156:0/2500794405 wait complete. 2024-07-16T10:02:40.323 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:40.321+0000 7efe721f2640 1 -- 172.21.15.156:0/1066696971 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efe6c0fbcc0 msgr2=0x7efe6c0fc130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:02:40.323 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:40.321+0000 7efe721f2640 1 --2- 172.21.15.156:0/1066696971 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efe6c0fbcc0 0x7efe6c0fc130 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7efe600049c0 tx=0x7efe6000aeb0 comp rx=0 tx=0).stop 2024-07-16T10:02:40.323 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:40.321+0000 7efe721f2640 1 -- 172.21.15.156:0/1066696971 shutdown_connections 2024-07-16T10:02:40.323 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:40.321+0000 7efe721f2640 1 --2- 172.21.15.156:0/1066696971 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efe6c0fbcc0 0x7efe6c0fc130 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:02:40.323 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:40.321+0000 7efe721f2640 1 --2- 172.21.15.156:0/1066696971 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7efe6c0fa9f0 0x7efe6c0fae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:02:40.323 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:40.321+0000 7efe721f2640 1 -- 172.21.15.156:0/1066696971 >> 172.21.15.156:0/1066696971 conn(0x7efe6c0f6190 msgr2=0x7efe6c0f85e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T10:02:40.323 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:40.321+0000 7efe721f2640 1 -- 172.21.15.156:0/1066696971 shutdown_connections 2024-07-16T10:02:40.323 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:40.321+0000 7efe721f2640 1 -- 172.21.15.156:0/1066696971 wait complete. 2024-07-16T10:02:40.325 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:40.322+0000 7efe721f2640 1 Processor -- start 2024-07-16T10:02:40.325 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:40.323+0000 7efe721f2640 1 -- start start 2024-07-16T10:02:40.325 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:40.323+0000 7efe721f2640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7efe6c0fa9f0 0x7efe6c10f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:02:40.325 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:40.323+0000 7efe721f2640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efe6c0fbcc0 0x7efe6c10f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:02:40.325 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:40.323+0000 7efe721f2640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe6c110580 con 0x7efe6c0fbcc0 2024-07-16T10:02:40.325 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:40.323+0000 7efe721f2640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe6c1106f0 con 0x7efe6c0fa9f0 2024-07-16T10:02:40.325 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:40.323+0000 7efe6affd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efe6c0fbcc0 0x7efe6c10f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:02:40.325 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:40.323+0000 7efe6affd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efe6c0fbcc0 0x7efe6c10f710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:46172/0 (socket says 172.21.15.156:46172) 2024-07-16T10:02:40.326 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:40.323+0000 7efe6affd640 1 -- 172.21.15.156:0/1886449512 learned_addr learned my addr 172.21.15.156:0/1886449512 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T10:02:40.326 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:40.323+0000 7efe6b7fe640 1 --2- 172.21.15.156:0/1886449512 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7efe6c0fa9f0 0x7efe6c10f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:02:40.326 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:40.323+0000 7efe6affd640 1 -- 172.21.15.156:0/1886449512 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7efe6c0fa9f0 msgr2=0x7efe6c10f1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:02:40.326 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:40.323+0000 7efe6affd640 1 --2- 172.21.15.156:0/1886449512 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7efe6c0fa9f0 0x7efe6c10f1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:02:40.326 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:40.323+0000 7efe6affd640 1 -- 172.21.15.156:0/1886449512 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe60005040 con 0x7efe6c0fbcc0 2024-07-16T10:02:40.326 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:40.324+0000 7efe6affd640 1 --2- 172.21.15.156:0/1886449512 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efe6c0fbcc0 0x7efe6c10f710 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7efe600334d0 tx=0x7efe6000aeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T10:02:40.326 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:40.324+0000 7efe53fff640 1 -- 172.21.15.156:0/1886449512 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efe60033960 con 0x7efe6c0fbcc0 2024-07-16T10:02:40.326 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:40.324+0000 7efe721f2640 1 -- 172.21.15.156:0/1886449512 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe6c110970 con 0x7efe6c0fbcc0 2024-07-16T10:02:40.326 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:40.324+0000 7efe53fff640 1 -- 172.21.15.156:0/1886449512 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7efe60034420 con 0x7efe6c0fbcc0 2024-07-16T10:02:40.326 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:40.324+0000 7efe53fff640 1 -- 172.21.15.156:0/1886449512 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efe6003c5e0 con 0x7efe6c0fbcc0 2024-07-16T10:02:40.326 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:40.324+0000 7efe721f2640 1 -- 172.21.15.156:0/1886449512 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe6c110f10 con 0x7efe6c0fbcc0 2024-07-16T10:02:40.327 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:40.326+0000 7efe721f2640 1 -- 172.21.15.156:0/1886449512 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe6c0f9e50 con 0x7efe6c0fbcc0 2024-07-16T10:02:40.327 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:40.326+0000 7efe53fff640 1 -- 172.21.15.156:0/1886449512 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 38) v1 ==== 92205+0+0 (secure 0 0 0) 0x7efe60031020 con 0x7efe6c0fbcc0 2024-07-16T10:02:40.327 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:40.326+0000 7efe53fff640 1 --2- 172.21.15.156:0/1886449512 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7efe5406e0b0 0x7efe54070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:02:40.328 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:40.326+0000 7efe53fff640 1 -- 172.21.15.156:0/1886449512 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6496+0+0 (secure 0 0 0) 0x7efe600ae3c0 con 0x7efe6c0fbcc0 2024-07-16T10:02:40.328 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:40.327+0000 7efe6b7fe640 1 --2- 172.21.15.156:0/1886449512 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7efe5406e0b0 0x7efe54070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:02:40.328 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:40.327+0000 7efe6b7fe640 1 --2- 172.21.15.156:0/1886449512 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7efe5406e0b0 0x7efe54070560 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7efe5c006fd0 tx=0x7efe5c008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T10:02:40.332 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:40.331+0000 7efe53fff640 1 -- 172.21.15.156:0/1886449512 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187716 (secure 0 0 0) 0x7efe60077770 con 0x7efe6c0fbcc0 2024-07-16T10:02:40.540 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:40.539+0000 7efe721f2640 1 -- 172.21.15.156:0/1886449512 --> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7efe6c111130 con 0x7efe5406e0b0 2024-07-16T10:02:40.540 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:02:40 smithi156 ceph-mon[93066]: pgmap v179: 65 pgs: 65 active+clean; 145 MiB data, 689 MiB used, 536 GiB / 536 GiB avail; 255 B/s wr, 0 op/s 2024-07-16T10:02:40.541 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:02:40 smithi156 ceph-mon[93066]: from='client.34410 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-16T10:02:40.541 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:02:40 smithi156 ceph-mon[93066]: from='client.34414 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-16T10:02:40.541 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:02:40 smithi156 ceph-mon[93066]: from='client.? 172.21.15.156:0/1802305078' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:02:40.541 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:02:40 smithi156 ceph-26c91c26-4358-11ef-bcad-c7b262605968-mon-smithi156[93062]: 2024-07-16T10:02:40.285+0000 7fa4fe7fb640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-07-16T10:02:40.541 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:40.540+0000 7efe53fff640 1 -- 172.21.15.156:0/1886449512 <== mgr.34104 v2:172.21.15.156:6800/3367765292 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7efe6c111130 con 0x7efe5406e0b0 2024-07-16T10:02:40.541 INFO:teuthology.orchestra.run.smithi156.stdout:{ 2024-07-16T10:02:40.542 INFO:teuthology.orchestra.run.smithi156.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:c15ec3aa8e6c0a9c12ee24ff79f3e7dd4d01b3d1a702dfeb7d7fff9b0811a393", 2024-07-16T10:02:40.542 INFO:teuthology.orchestra.run.smithi156.stdout: "in_progress": true, 2024-07-16T10:02:40.542 INFO:teuthology.orchestra.run.smithi156.stdout: "which": "Upgrading all daemon types on all hosts", 2024-07-16T10:02:40.542 INFO:teuthology.orchestra.run.smithi156.stdout: "services_complete": [ 2024-07-16T10:02:40.542 INFO:teuthology.orchestra.run.smithi156.stdout: "mon", 2024-07-16T10:02:40.542 INFO:teuthology.orchestra.run.smithi156.stdout: "mgr", 2024-07-16T10:02:40.542 INFO:teuthology.orchestra.run.smithi156.stdout: "crash", 2024-07-16T10:02:40.542 INFO:teuthology.orchestra.run.smithi156.stdout: "osd" 2024-07-16T10:02:40.542 INFO:teuthology.orchestra.run.smithi156.stdout: ], 2024-07-16T10:02:40.542 INFO:teuthology.orchestra.run.smithi156.stdout: "progress": "12/21 daemons upgraded", 2024-07-16T10:02:40.542 INFO:teuthology.orchestra.run.smithi156.stdout: "message": "Currently upgrading mds daemons", 2024-07-16T10:02:40.543 INFO:teuthology.orchestra.run.smithi156.stdout: "is_paused": false 2024-07-16T10:02:40.543 INFO:teuthology.orchestra.run.smithi156.stdout:} 2024-07-16T10:02:40.544 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:40.542+0000 7efe721f2640 1 -- 172.21.15.156:0/1886449512 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7efe5406e0b0 msgr2=0x7efe54070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:02:40.544 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:40.542+0000 7efe721f2640 1 --2- 172.21.15.156:0/1886449512 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7efe5406e0b0 0x7efe54070560 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7efe5c006fd0 tx=0x7efe5c008040 comp rx=0 tx=0).stop 2024-07-16T10:02:40.544 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:40.542+0000 7efe721f2640 1 -- 172.21.15.156:0/1886449512 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efe6c0fbcc0 msgr2=0x7efe6c10f710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:02:40.544 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:40.542+0000 7efe721f2640 1 --2- 172.21.15.156:0/1886449512 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efe6c0fbcc0 0x7efe6c10f710 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7efe600334d0 tx=0x7efe6000aeb0 comp rx=0 tx=0).stop 2024-07-16T10:02:40.544 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:40.543+0000 7efe721f2640 1 -- 172.21.15.156:0/1886449512 shutdown_connections 2024-07-16T10:02:40.544 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:40.543+0000 7efe721f2640 1 --2- 172.21.15.156:0/1886449512 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7efe5406e0b0 0x7efe54070560 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:02:40.544 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:40.543+0000 7efe721f2640 1 --2- 172.21.15.156:0/1886449512 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efe6c0fbcc0 0x7efe6c10f710 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:02:40.544 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:40.543+0000 7efe721f2640 1 --2- 172.21.15.156:0/1886449512 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7efe6c0fa9f0 0x7efe6c10f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:02:40.544 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:40.543+0000 7efe721f2640 1 -- 172.21.15.156:0/1886449512 >> 172.21.15.156:0/1886449512 conn(0x7efe6c0f6190 msgr2=0x7efe6c0f7810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T10:02:40.544 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:40.543+0000 7efe721f2640 1 -- 172.21.15.156:0/1886449512 shutdown_connections 2024-07-16T10:02:40.544 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:40.543+0000 7efe721f2640 1 -- 172.21.15.156:0/1886449512 wait complete. 2024-07-16T10:02:40.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:02:40 smithi176 ceph-mon[81689]: pgmap v179: 65 pgs: 65 active+clean; 145 MiB data, 689 MiB used, 536 GiB / 536 GiB avail; 255 B/s wr, 0 op/s 2024-07-16T10:02:40.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:02:40 smithi176 ceph-mon[81689]: from='client.34410 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-16T10:02:40.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:02:40 smithi176 ceph-mon[81689]: from='client.34414 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-16T10:02:40.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:02:40 smithi176 ceph-mon[81689]: from='client.? 172.21.15.156:0/1802305078' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:02:40.631 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:40.629+0000 7ff3e94ae640 1 -- 172.21.15.156:0/2851582231 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff3e40fb260 msgr2=0x7ff3e40fb670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:02:40.631 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:40.629+0000 7ff3e94ae640 1 --2- 172.21.15.156:0/2851582231 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff3e40fb260 0x7ff3e40fb670 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7ff3d40048d0 tx=0x7ff3d4009da0 comp rx=0 tx=0).stop 2024-07-16T10:02:40.631 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:40.629+0000 7ff3e94ae640 1 -- 172.21.15.156:0/2851582231 shutdown_connections 2024-07-16T10:02:40.631 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:40.629+0000 7ff3e94ae640 1 --2- 172.21.15.156:0/2851582231 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff3e40fc530 0x7ff3e40fc9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:02:40.631 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:40.629+0000 7ff3e94ae640 1 --2- 172.21.15.156:0/2851582231 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff3e40fb260 0x7ff3e40fb670 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:02:40.631 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:40.629+0000 7ff3e94ae640 1 -- 172.21.15.156:0/2851582231 >> 172.21.15.156:0/2851582231 conn(0x7ff3e40f6a20 msgr2=0x7ff3e40f8e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T10:02:40.631 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:40.629+0000 7ff3e94ae640 1 -- 172.21.15.156:0/2851582231 shutdown_connections 2024-07-16T10:02:40.632 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:40.629+0000 7ff3e94ae640 1 -- 172.21.15.156:0/2851582231 wait complete. 2024-07-16T10:02:40.632 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:40.629+0000 7ff3e94ae640 1 Processor -- start 2024-07-16T10:02:40.632 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:40.629+0000 7ff3e94ae640 1 -- start start 2024-07-16T10:02:40.632 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:40.630+0000 7ff3e94ae640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff3e40fb260 0x7ff3e410f250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:02:40.632 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:40.630+0000 7ff3e94ae640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff3e40fc530 0x7ff3e410f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:02:40.632 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:40.630+0000 7ff3e94ae640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3e4110600 con 0x7ff3e40fc530 2024-07-16T10:02:40.632 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:40.630+0000 7ff3e94ae640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3e4110770 con 0x7ff3e40fb260 2024-07-16T10:02:40.633 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:40.630+0000 7ff3e27fc640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff3e40fc530 0x7ff3e410f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:02:40.633 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:40.630+0000 7ff3e27fc640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff3e40fc530 0x7ff3e410f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:46186/0 (socket says 172.21.15.156:46186) 2024-07-16T10:02:40.633 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:40.630+0000 7ff3e27fc640 1 -- 172.21.15.156:0/3002107791 learned_addr learned my addr 172.21.15.156:0/3002107791 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T10:02:40.633 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:40.630+0000 7ff3e2ffd640 1 --2- 172.21.15.156:0/3002107791 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff3e40fb260 0x7ff3e410f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:02:40.634 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:40.630+0000 7ff3e27fc640 1 -- 172.21.15.156:0/3002107791 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff3e40fb260 msgr2=0x7ff3e410f250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:02:40.634 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:40.630+0000 7ff3e27fc640 1 --2- 172.21.15.156:0/3002107791 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff3e40fb260 0x7ff3e410f250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:02:40.634 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:40.630+0000 7ff3e27fc640 1 -- 172.21.15.156:0/3002107791 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3d4005040 con 0x7ff3e40fc530 2024-07-16T10:02:40.634 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:40.630+0000 7ff3e27fc640 1 --2- 172.21.15.156:0/3002107791 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff3e40fc530 0x7ff3e410f790 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7ff3d800b790 tx=0x7ff3d80091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T10:02:40.634 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:40.631+0000 7ff3cb7fe640 1 -- 172.21.15.156:0/3002107791 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff3d800c700 con 0x7ff3e40fc530 2024-07-16T10:02:40.634 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:40.631+0000 7ff3cb7fe640 1 -- 172.21.15.156:0/3002107791 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff3d800cd30 con 0x7ff3e40fc530 2024-07-16T10:02:40.634 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:40.631+0000 7ff3cb7fe640 1 -- 172.21.15.156:0/3002107791 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff3d8013370 con 0x7ff3e40fc530 2024-07-16T10:02:40.634 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:40.631+0000 7ff3e94ae640 1 -- 172.21.15.156:0/3002107791 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3e4110a50 con 0x7ff3e40fc530 2024-07-16T10:02:40.634 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:40.631+0000 7ff3e94ae640 1 -- 172.21.15.156:0/3002107791 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3e4110ef0 con 0x7ff3e40fc530 2024-07-16T10:02:40.634 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:40.632+0000 7ff3e94ae640 1 -- 172.21.15.156:0/3002107791 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3e40fa6c0 con 0x7ff3e40fc530 2024-07-16T10:02:40.635 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:40.633+0000 7ff3cb7fe640 1 -- 172.21.15.156:0/3002107791 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 38) v1 ==== 92205+0+0 (secure 0 0 0) 0x7ff3d8012070 con 0x7ff3e40fc530 2024-07-16T10:02:40.635 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:40.634+0000 7ff3cb7fe640 1 --2- 172.21.15.156:0/3002107791 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7ff3cc06dea0 0x7ff3cc070350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:02:40.635 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:40.634+0000 7ff3cb7fe640 1 -- 172.21.15.156:0/3002107791 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6496+0+0 (secure 0 0 0) 0x7ff3d808d650 con 0x7ff3e40fc530 2024-07-16T10:02:40.635 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:40.634+0000 7ff3e2ffd640 1 --2- 172.21.15.156:0/3002107791 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7ff3cc06dea0 0x7ff3cc070350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:02:40.636 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:40.635+0000 7ff3e2ffd640 1 --2- 172.21.15.156:0/3002107791 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7ff3cc06dea0 0x7ff3cc070350 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7ff3d4000c00 tx=0x7ff3d4035040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T10:02:40.639 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:40.638+0000 7ff3cb7fe640 1 -- 172.21.15.156:0/3002107791 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187716 (secure 0 0 0) 0x7ff3d8056980 con 0x7ff3e40fc530 2024-07-16T10:02:40.907 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:40.906+0000 7ff3e94ae640 1 -- 172.21.15.156:0/3002107791 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7ff3e4103e30 con 0x7ff3e40fc530 2024-07-16T10:02:40.908 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:40.907+0000 7ff3cb7fe640 1 -- 172.21.15.156:0/3002107791 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+107 (secure 0 0 0) 0x7ff3d8015470 con 0x7ff3e40fc530 2024-07-16T10:02:40.908 INFO:teuthology.orchestra.run.smithi156.stdout:HEALTH_WARN 1 filesystem is degraded 2024-07-16T10:02:40.908 INFO:teuthology.orchestra.run.smithi156.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-07-16T10:02:40.908 INFO:teuthology.orchestra.run.smithi156.stdout: fs cephfs is degraded 2024-07-16T10:02:40.910 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:40.909+0000 7ff3e94ae640 1 -- 172.21.15.156:0/3002107791 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7ff3cc06dea0 msgr2=0x7ff3cc070350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:02:40.910 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:40.909+0000 7ff3e94ae640 1 --2- 172.21.15.156:0/3002107791 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7ff3cc06dea0 0x7ff3cc070350 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7ff3d4000c00 tx=0x7ff3d4035040 comp rx=0 tx=0).stop 2024-07-16T10:02:40.911 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:40.909+0000 7ff3e94ae640 1 -- 172.21.15.156:0/3002107791 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff3e40fc530 msgr2=0x7ff3e410f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:02:40.911 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:40.909+0000 7ff3e94ae640 1 --2- 172.21.15.156:0/3002107791 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff3e40fc530 0x7ff3e410f790 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7ff3d800b790 tx=0x7ff3d80091d0 comp rx=0 tx=0).stop 2024-07-16T10:02:40.911 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:40.909+0000 7ff3e94ae640 1 -- 172.21.15.156:0/3002107791 shutdown_connections 2024-07-16T10:02:40.911 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:40.909+0000 7ff3e94ae640 1 --2- 172.21.15.156:0/3002107791 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7ff3cc06dea0 0x7ff3cc070350 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:02:40.911 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:40.909+0000 7ff3e94ae640 1 --2- 172.21.15.156:0/3002107791 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff3e40fc530 0x7ff3e410f790 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:02:40.911 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:40.909+0000 7ff3e94ae640 1 --2- 172.21.15.156:0/3002107791 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff3e40fb260 0x7ff3e410f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:02:40.911 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:40.909+0000 7ff3e94ae640 1 -- 172.21.15.156:0/3002107791 >> 172.21.15.156:0/3002107791 conn(0x7ff3e40f6a20 msgr2=0x7ff3e4065e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T10:02:40.911 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:40.909+0000 7ff3e94ae640 1 -- 172.21.15.156:0/3002107791 shutdown_connections 2024-07-16T10:02:40.911 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:02:40.909+0000 7ff3e94ae640 1 -- 172.21.15.156:0/3002107791 wait complete. 2024-07-16T10:02:41.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:02:41 smithi176 ceph-mon[81689]: from='client.34418 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-07-16T10:02:41.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:02:41 smithi176 ceph-mon[81689]: from='client.? 172.21.15.156:0/2500794405' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-07-16T10:02:41.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:02:41 smithi176 ceph-mon[81689]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-07-16T10:02:41.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:02:41 smithi176 ceph-mon[81689]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-07-16T10:02:41.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:02:41 smithi176 ceph-mon[81689]: osdmap e81: 6 total, 6 up, 6 in 2024-07-16T10:02:41.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:02:41 smithi176 ceph-mon[81689]: Standby daemon mds.cephfs.smithi156.evwwaz assigned to filesystem cephfs as rank 0 2024-07-16T10:02:41.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:02:41 smithi176 ceph-mon[81689]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-07-16T10:02:41.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:02:41 smithi176 ceph-mon[81689]: fsmap cephfs:0/1 3 up:standby, 1 failed 2024-07-16T10:02:41.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:02:41 smithi176 ceph-mon[81689]: fsmap cephfs:1/1 {0=cephfs.smithi156.evwwaz=up:replay} 2 up:standby 2024-07-16T10:02:41.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:02:41 smithi176 ceph-mon[81689]: from='client.? 172.21.15.156:0/3002107791' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-07-16T10:02:41.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:02:41 smithi156 ceph-mon[93066]: from='client.34418 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-07-16T10:02:41.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:02:41 smithi156 ceph-mon[93066]: from='client.? 172.21.15.156:0/2500794405' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-07-16T10:02:41.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:02:41 smithi156 ceph-mon[93066]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-07-16T10:02:41.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:02:41 smithi156 ceph-mon[93066]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-07-16T10:02:41.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:02:41 smithi156 ceph-mon[93066]: osdmap e81: 6 total, 6 up, 6 in 2024-07-16T10:02:41.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:02:41 smithi156 ceph-mon[93066]: Standby daemon mds.cephfs.smithi156.evwwaz assigned to filesystem cephfs as rank 0 2024-07-16T10:02:41.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:02:41 smithi156 ceph-mon[93066]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-07-16T10:02:41.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:02:41 smithi156 ceph-mon[93066]: fsmap cephfs:0/1 3 up:standby, 1 failed 2024-07-16T10:02:41.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:02:41 smithi156 ceph-mon[93066]: fsmap cephfs:1/1 {0=cephfs.smithi156.evwwaz=up:replay} 2 up:standby 2024-07-16T10:02:41.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:02:41 smithi156 ceph-mon[93066]: from='client.? 172.21.15.156:0/3002107791' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-07-16T10:02:42.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:02:42 smithi176 ceph-mon[81689]: pgmap v181: 65 pgs: 65 active+clean; 145 MiB data, 689 MiB used, 536 GiB / 536 GiB avail 2024-07-16T10:02:42.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:02:42 smithi176 ceph-mon[81689]: from='client.34430 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-16T10:02:42.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:02:42 smithi156 ceph-mon[93066]: pgmap v181: 65 pgs: 65 active+clean; 145 MiB data, 689 MiB used, 536 GiB / 536 GiB avail 2024-07-16T10:02:42.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:02:42 smithi156 ceph-mon[93066]: from='client.34430 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-16T10:02:44.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:02:44 smithi176 ceph-mon[81689]: pgmap v182: 65 pgs: 65 active+clean; 145 MiB data, 689 MiB used, 536 GiB / 536 GiB avail; 3.2 MiB/s rd, 1 op/s 2024-07-16T10:02:44.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:02:44 smithi156 ceph-mon[93066]: pgmap v182: 65 pgs: 65 active+clean; 145 MiB data, 689 MiB used, 536 GiB / 536 GiB avail; 3.2 MiB/s rd, 1 op/s 2024-07-16T10:02:46.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:02:46 smithi176 ceph-mon[81689]: pgmap v183: 65 pgs: 65 active+clean; 145 MiB data, 689 MiB used, 536 GiB / 536 GiB avail; 4.8 MiB/s rd, 2 op/s 2024-07-16T10:02:46.653 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:02:46 smithi156 ceph-mon[93066]: pgmap v183: 65 pgs: 65 active+clean; 145 MiB data, 689 MiB used, 536 GiB / 536 GiB avail; 4.8 MiB/s rd, 2 op/s 2024-07-16T10:02:47.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:02:47 smithi176 ceph-mon[81689]: reconnect by client.14500 192.168.0.1:0/893267894 after 0.000999986 2024-07-16T10:02:47.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:02:47 smithi176 ceph-mon[81689]: reconnect by client.24301 192.168.0.1:0/3627935956 after 0.000999986 2024-07-16T10:02:47.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:02:47 smithi176 ceph-mon[81689]: mds.? [v2:172.21.15.156:6828/333092891,v1:172.21.15.156:6829/333092891] up:reconnect 2024-07-16T10:02:47.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:02:47 smithi176 ceph-mon[81689]: fsmap cephfs:1/1 {0=cephfs.smithi156.evwwaz=up:reconnect} 2 up:standby 2024-07-16T10:02:47.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:02:47 smithi176 ceph-mon[81689]: mds.? [v2:172.21.15.156:6828/333092891,v1:172.21.15.156:6829/333092891] up:rejoin 2024-07-16T10:02:47.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:02:47 smithi156 ceph-mon[93066]: reconnect by client.14500 192.168.0.1:0/893267894 after 0.000999986 2024-07-16T10:02:47.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:02:47 smithi156 ceph-mon[93066]: reconnect by client.24301 192.168.0.1:0/3627935956 after 0.000999986 2024-07-16T10:02:47.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:02:47 smithi156 ceph-mon[93066]: mds.? [v2:172.21.15.156:6828/333092891,v1:172.21.15.156:6829/333092891] up:reconnect 2024-07-16T10:02:47.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:02:47 smithi156 ceph-mon[93066]: fsmap cephfs:1/1 {0=cephfs.smithi156.evwwaz=up:reconnect} 2 up:standby 2024-07-16T10:02:47.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:02:47 smithi156 ceph-mon[93066]: mds.? [v2:172.21.15.156:6828/333092891,v1:172.21.15.156:6829/333092891] up:rejoin 2024-07-16T10:02:48.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:02:48 smithi176 ceph-mon[81689]: pgmap v184: 65 pgs: 65 active+clean; 145 MiB data, 689 MiB used, 536 GiB / 536 GiB avail; 13 MiB/s rd, 4 op/s 2024-07-16T10:02:48.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:02:48 smithi176 ceph-mon[81689]: fsmap cephfs:1/1 {0=cephfs.smithi156.evwwaz=up:rejoin} 2 up:standby 2024-07-16T10:02:48.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:02:48 smithi176 ceph-mon[81689]: daemon mds.cephfs.smithi156.evwwaz is now active in filesystem cephfs as rank 0 2024-07-16T10:02:48.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:02:48 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:02:48.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:02:48 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:02:48.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:02:48 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-16T10:02:48.655 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:02:48 smithi156 ceph-mon[93066]: pgmap v184: 65 pgs: 65 active+clean; 145 MiB data, 689 MiB used, 536 GiB / 536 GiB avail; 13 MiB/s rd, 4 op/s 2024-07-16T10:02:48.655 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:02:48 smithi156 ceph-mon[93066]: fsmap cephfs:1/1 {0=cephfs.smithi156.evwwaz=up:rejoin} 2 up:standby 2024-07-16T10:02:48.655 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:02:48 smithi156 ceph-mon[93066]: daemon mds.cephfs.smithi156.evwwaz is now active in filesystem cephfs as rank 0 2024-07-16T10:02:48.655 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:02:48 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:02:48.655 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:02:48 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:02:48.655 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:02:48 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-16T10:02:49.424 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:02:49 smithi156 ceph-mon[93066]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-07-16T10:02:49.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:02:49 smithi156 ceph-mon[93066]: Cluster is now healthy 2024-07-16T10:02:49.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:02:49 smithi156 ceph-mon[93066]: mds.? [v2:172.21.15.156:6828/333092891,v1:172.21.15.156:6829/333092891] up:active 2024-07-16T10:02:49.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:02:49 smithi156 ceph-mon[93066]: mds.? [v2:172.21.15.156:6826/2852996640,v1:172.21.15.156:6827/2852996640] up:boot 2024-07-16T10:02:49.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:02:49 smithi156 ceph-mon[93066]: fsmap cephfs:1 {0=cephfs.smithi156.evwwaz=up:active} 3 up:standby 2024-07-16T10:02:49.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:02:49 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi156.rdibys"}]: dispatch 2024-07-16T10:02:49.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:02:49 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-16T10:02:49.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:02:49 smithi176 ceph-mon[81689]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-07-16T10:02:49.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:02:49 smithi176 ceph-mon[81689]: Cluster is now healthy 2024-07-16T10:02:49.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:02:49 smithi176 ceph-mon[81689]: mds.? [v2:172.21.15.156:6828/333092891,v1:172.21.15.156:6829/333092891] up:active 2024-07-16T10:02:49.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:02:49 smithi176 ceph-mon[81689]: mds.? [v2:172.21.15.156:6826/2852996640,v1:172.21.15.156:6827/2852996640] up:boot 2024-07-16T10:02:49.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:02:49 smithi176 ceph-mon[81689]: fsmap cephfs:1 {0=cephfs.smithi156.evwwaz=up:active} 3 up:standby 2024-07-16T10:02:49.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:02:49 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi156.rdibys"}]: dispatch 2024-07-16T10:02:49.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:02:49 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-16T10:02:50.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:02:50 smithi176 ceph-mon[81689]: pgmap v185: 65 pgs: 65 active+clean; 145 MiB data, 689 MiB used, 536 GiB / 536 GiB avail; 14 MiB/s rd, 4 op/s 2024-07-16T10:02:50.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:02:50 smithi156 ceph-mon[93066]: pgmap v185: 65 pgs: 65 active+clean; 145 MiB data, 689 MiB used, 536 GiB / 536 GiB avail; 14 MiB/s rd, 4 op/s 2024-07-16T10:02:52.424 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:02:52 smithi156 ceph-mon[93066]: pgmap v186: 65 pgs: 65 active+clean; 145 MiB data, 689 MiB used, 536 GiB / 536 GiB avail; 14 MiB/s rd, 202 B/s wr, 7 op/s 2024-07-16T10:02:52.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:02:52 smithi176 ceph-mon[81689]: pgmap v186: 65 pgs: 65 active+clean; 145 MiB data, 689 MiB used, 536 GiB / 536 GiB avail; 14 MiB/s rd, 202 B/s wr, 7 op/s 2024-07-16T10:02:54.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:02:54 smithi176 ceph-mon[81689]: pgmap v187: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 12 MiB/s rd, 4.7 KiB/s wr, 9 op/s 2024-07-16T10:02:54.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:02:54 smithi156 ceph-mon[93066]: pgmap v187: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 12 MiB/s rd, 4.7 KiB/s wr, 9 op/s 2024-07-16T10:02:56.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:02:56 smithi176 ceph-mon[81689]: pgmap v188: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 8.9 MiB/s rd, 4.7 KiB/s wr, 8 op/s 2024-07-16T10:02:56.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:02:56 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:02:56.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:02:56 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:02:56.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:02:56 smithi156 ceph-mon[93066]: pgmap v188: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 8.9 MiB/s rd, 4.7 KiB/s wr, 8 op/s 2024-07-16T10:02:56.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:02:56 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:02:56.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:02:56 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:02:58.299 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:02:57 smithi156 ceph-mon[93066]: pgmap v189: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 7.6 MiB/s rd, 4.7 KiB/s wr, 7 op/s 2024-07-16T10:02:58.299 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:02:57 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:02:58.299 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:02:57 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:02:58.343 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:02:57 smithi176 ceph-mon[81689]: pgmap v189: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 7.6 MiB/s rd, 4.7 KiB/s wr, 7 op/s 2024-07-16T10:02:58.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:02:57 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:02:58.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:02:57 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:03:00.343 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:02:59 smithi176 ceph-mon[81689]: pgmap v190: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 686 KiB/s rd, 4.7 KiB/s wr, 5 op/s 2024-07-16T10:03:00.424 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:02:59 smithi156 ceph-mon[93066]: pgmap v190: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 686 KiB/s rd, 4.7 KiB/s wr, 5 op/s 2024-07-16T10:03:01.106 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:03:00 smithi156 ceph-mon[93066]: Detected new or changed devices on smithi156 2024-07-16T10:03:01.106 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:03:00 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:03:01.106 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:03:00 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:03:01.106 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:03:00 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-16T10:03:01.106 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:03:00 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-16T10:03:01.106 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:03:00 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:03:01.106 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:03:00 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-16T10:03:01.106 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:03:00 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:03:01.106 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:03:00 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:03:01.106 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:03:00 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:03:01.107 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:03:00 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:03:01.107 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:03:00 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "mds ok-to-stop", "ids": ["cephfs.smithi156.evwwaz"]}]: dispatch 2024-07-16T10:03:01.107 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:03:00 smithi156 ceph-mon[93066]: Upgrade: It appears safe to stop mds.cephfs.smithi156.evwwaz 2024-07-16T10:03:01.343 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:03:00 smithi176 ceph-mon[81689]: Detected new or changed devices on smithi156 2024-07-16T10:03:01.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:03:00 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:03:01.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:03:00 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:03:01.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:03:00 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-16T10:03:01.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:03:00 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-16T10:03:01.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:03:00 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:03:01.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:03:00 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-16T10:03:01.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:03:00 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:03:01.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:03:00 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:03:01.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:03:00 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:03:01.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:03:00 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:03:01.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:03:00 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "mds ok-to-stop", "ids": ["cephfs.smithi156.evwwaz"]}]: dispatch 2024-07-16T10:03:01.345 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:03:00 smithi176 ceph-mon[81689]: Upgrade: It appears safe to stop mds.cephfs.smithi156.evwwaz 2024-07-16T10:03:02.093 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:03:01 smithi156 ceph-mon[93066]: pgmap v191: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 3.3 KiB/s rd, 4.7 KiB/s wr, 5 op/s 2024-07-16T10:03:02.093 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:03:01 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:03:02.094 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:03:01 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi156.evwwaz", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-16T10:03:02.094 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:03:01 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-16T10:03:02.343 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:03:01 smithi176 ceph-mon[81689]: pgmap v191: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 3.3 KiB/s rd, 4.7 KiB/s wr, 5 op/s 2024-07-16T10:03:02.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:03:01 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:03:02.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:03:01 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi156.evwwaz", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-16T10:03:02.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:03:01 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-16T10:03:03.103 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:03:02 smithi156 ceph-mon[93066]: Upgrade: Updating mds.cephfs.smithi156.evwwaz 2024-07-16T10:03:03.103 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:03:02 smithi156 ceph-mon[93066]: Deploying daemon mds.cephfs.smithi156.evwwaz on smithi156 2024-07-16T10:03:03.343 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:03:02 smithi176 ceph-mon[81689]: Upgrade: Updating mds.cephfs.smithi156.evwwaz 2024-07-16T10:03:03.343 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:03:02 smithi176 ceph-mon[81689]: Deploying daemon mds.cephfs.smithi156.evwwaz on smithi156 2024-07-16T10:03:03.426 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:03:03 smithi156 ceph-26c91c26-4358-11ef-bcad-c7b262605968-mon-smithi156[93062]: 2024-07-16T10:03:03.349+0000 7fa4fe7fb640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-07-16T10:03:04.343 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:03:03 smithi176 ceph-mon[81689]: pgmap v192: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 4.6 KiB/s wr, 3 op/s 2024-07-16T10:03:04.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:03:03 smithi176 ceph-mon[81689]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-07-16T10:03:04.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:03:03 smithi176 ceph-mon[81689]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-07-16T10:03:04.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:03:03 smithi176 ceph-mon[81689]: osdmap e82: 6 total, 6 up, 6 in 2024-07-16T10:03:04.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:03:03 smithi176 ceph-mon[81689]: Standby daemon mds.cephfs.smithi156.rdibys assigned to filesystem cephfs as rank 0 2024-07-16T10:03:04.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:03:03 smithi176 ceph-mon[81689]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-07-16T10:03:04.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:03:03 smithi176 ceph-mon[81689]: fsmap cephfs:0/1 3 up:standby, 1 failed 2024-07-16T10:03:04.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:03:03 smithi176 ceph-mon[81689]: fsmap cephfs:1/1 {0=cephfs.smithi156.rdibys=up:replay} 2 up:standby 2024-07-16T10:03:04.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:03:03 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:03:04.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:03:03 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-16T10:03:04.424 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:03:03 smithi156 ceph-mon[93066]: pgmap v192: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 4.6 KiB/s wr, 3 op/s 2024-07-16T10:03:04.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:03:03 smithi156 ceph-mon[93066]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-07-16T10:03:04.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:03:03 smithi156 ceph-mon[93066]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-07-16T10:03:04.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:03:03 smithi156 ceph-mon[93066]: osdmap e82: 6 total, 6 up, 6 in 2024-07-16T10:03:04.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:03:03 smithi156 ceph-mon[93066]: Standby daemon mds.cephfs.smithi156.rdibys assigned to filesystem cephfs as rank 0 2024-07-16T10:03:04.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:03:03 smithi156 ceph-mon[93066]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-07-16T10:03:04.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:03:03 smithi156 ceph-mon[93066]: fsmap cephfs:0/1 3 up:standby, 1 failed 2024-07-16T10:03:04.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:03:03 smithi156 ceph-mon[93066]: fsmap cephfs:1/1 {0=cephfs.smithi156.rdibys=up:replay} 2 up:standby 2024-07-16T10:03:04.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:03:03 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:03:04.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:03:03 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-16T10:03:06.343 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:03:05 smithi176 ceph-mon[81689]: pgmap v194: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 410 KiB/s rd, 0 op/s 2024-07-16T10:03:06.343 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:03:05 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi156.evwwaz"}]: dispatch 2024-07-16T10:03:06.424 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:03:05 smithi156 ceph-mon[93066]: pgmap v194: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 410 KiB/s rd, 0 op/s 2024-07-16T10:03:06.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:03:05 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi156.evwwaz"}]: dispatch 2024-07-16T10:03:08.343 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:03:08 smithi176 ceph-mon[81689]: pgmap v195: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 5.6 MiB/s rd, 2 op/s 2024-07-16T10:03:08.424 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:03:07 smithi156 ceph-mon[93066]: pgmap v195: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 5.6 MiB/s rd, 2 op/s 2024-07-16T10:03:10.343 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:03:10 smithi176 ceph-mon[81689]: pgmap v196: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 9.2 MiB/s rd, 3 op/s 2024-07-16T10:03:10.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:03:10 smithi176 ceph-mon[81689]: reconnect by client.14500 192.168.0.1:0/893267894 after 0.000999985 2024-07-16T10:03:10.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:03:10 smithi176 ceph-mon[81689]: reconnect by client.24301 192.168.0.1:0/3627935956 after 0.000999985 2024-07-16T10:03:10.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:03:10 smithi176 ceph-mon[81689]: mds.? [v2:172.21.15.156:6826/2852996640,v1:172.21.15.156:6827/2852996640] up:reconnect 2024-07-16T10:03:10.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:03:10 smithi176 ceph-mon[81689]: fsmap cephfs:1/1 {0=cephfs.smithi156.rdibys=up:reconnect} 2 up:standby 2024-07-16T10:03:10.424 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:03:10 smithi156 ceph-mon[93066]: pgmap v196: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 9.2 MiB/s rd, 3 op/s 2024-07-16T10:03:10.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:03:10 smithi156 ceph-mon[93066]: reconnect by client.14500 192.168.0.1:0/893267894 after 0.000999985 2024-07-16T10:03:10.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:03:10 smithi156 ceph-mon[93066]: reconnect by client.24301 192.168.0.1:0/3627935956 after 0.000999985 2024-07-16T10:03:10.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:03:10 smithi156 ceph-mon[93066]: mds.? [v2:172.21.15.156:6826/2852996640,v1:172.21.15.156:6827/2852996640] up:reconnect 2024-07-16T10:03:10.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:03:10 smithi156 ceph-mon[93066]: fsmap cephfs:1/1 {0=cephfs.smithi156.rdibys=up:reconnect} 2 up:standby 2024-07-16T10:03:11.005 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:11.002+0000 7f66f0681640 1 -- 172.21.15.156:0/1527259025 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f66e80fe560 msgr2=0x7f66e80fe970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:03:11.005 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:11.002+0000 7f66f0681640 1 --2- 172.21.15.156:0/1527259025 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f66e80fe560 0x7f66e80fe970 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f66e40048d0 tx=0x7f66e4009da0 comp rx=0 tx=0).stop 2024-07-16T10:03:11.005 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:11.002+0000 7f66f0681640 1 -- 172.21.15.156:0/1527259025 shutdown_connections 2024-07-16T10:03:11.006 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:11.002+0000 7f66f0681640 1 --2- 172.21.15.156:0/1527259025 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f66e80ff830 0x7f66e80ffca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:03:11.006 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:11.002+0000 7f66f0681640 1 --2- 172.21.15.156:0/1527259025 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f66e80fe560 0x7f66e80fe970 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:03:11.006 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:11.002+0000 7f66f0681640 1 -- 172.21.15.156:0/1527259025 >> 172.21.15.156:0/1527259025 conn(0x7f66e80fa010 msgr2=0x7f66e80fc3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T10:03:11.006 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:11.002+0000 7f66f0681640 1 -- 172.21.15.156:0/1527259025 shutdown_connections 2024-07-16T10:03:11.006 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:11.002+0000 7f66f0681640 1 -- 172.21.15.156:0/1527259025 wait complete. 2024-07-16T10:03:11.006 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:11.003+0000 7f66f0681640 1 Processor -- start 2024-07-16T10:03:11.006 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:11.003+0000 7f66f0681640 1 -- start start 2024-07-16T10:03:11.006 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:11.003+0000 7f66f0681640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f66e80fe560 0x7f66e810f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:03:11.006 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:11.003+0000 7f66f0681640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f66e80ff830 0x7f66e810f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:03:11.006 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:11.003+0000 7f66f0681640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66e810fcc0 con 0x7f66e80fe560 2024-07-16T10:03:11.006 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:11.003+0000 7f66f0681640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66e81104e0 con 0x7f66e80ff830 2024-07-16T10:03:11.006 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:11.003+0000 7f66ee41d640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f66e80fe560 0x7f66e810f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:03:11.007 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:11.003+0000 7f66ee41d640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f66e80fe560 0x7f66e810f130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:35806/0 (socket says 172.21.15.156:35806) 2024-07-16T10:03:11.007 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:11.003+0000 7f66ee41d640 1 -- 172.21.15.156:0/2896394440 learned_addr learned my addr 172.21.15.156:0/2896394440 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T10:03:11.007 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:11.003+0000 7f66edc1c640 1 --2- 172.21.15.156:0/2896394440 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f66e80ff830 0x7f66e810f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:03:11.007 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:11.003+0000 7f66ee41d640 1 -- 172.21.15.156:0/2896394440 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f66e80ff830 msgr2=0x7f66e810f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:03:11.007 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:11.003+0000 7f66ee41d640 1 --2- 172.21.15.156:0/2896394440 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f66e80ff830 0x7f66e810f670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:03:11.007 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:11.003+0000 7f66ee41d640 1 -- 172.21.15.156:0/2896394440 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66e4005040 con 0x7f66e80fe560 2024-07-16T10:03:11.007 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:11.004+0000 7f66ee41d640 1 --2- 172.21.15.156:0/2896394440 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f66e80fe560 0x7f66e810f130 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f66e4003080 tx=0x7f66e4009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T10:03:11.007 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:11.004+0000 7f66deffd640 1 -- 172.21.15.156:0/2896394440 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f66e4038070 con 0x7f66e80fe560 2024-07-16T10:03:11.007 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:11.004+0000 7f66deffd640 1 -- 172.21.15.156:0/2896394440 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f66e4032970 con 0x7f66e80fe560 2024-07-16T10:03:11.007 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:11.004+0000 7f66f0681640 1 -- 172.21.15.156:0/2896394440 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66e8110710 con 0x7f66e80fe560 2024-07-16T10:03:11.007 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:11.004+0000 7f66deffd640 1 -- 172.21.15.156:0/2896394440 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f66e4033900 con 0x7f66e80fe560 2024-07-16T10:03:11.007 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:11.004+0000 7f66f0681640 1 -- 172.21.15.156:0/2896394440 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66e8110c50 con 0x7f66e80fe560 2024-07-16T10:03:11.008 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:11.005+0000 7f66f0681640 1 -- 172.21.15.156:0/2896394440 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66e80fdb10 con 0x7f66e80fe560 2024-07-16T10:03:11.008 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:11.005+0000 7f66deffd640 1 -- 172.21.15.156:0/2896394440 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 38) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f66e4030020 con 0x7f66e80fe560 2024-07-16T10:03:11.008 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:11.006+0000 7f66deffd640 1 --2- 172.21.15.156:0/2896394440 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f66d406e0b0 0x7f66d4070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:03:11.008 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:11.006+0000 7f66edc1c640 1 --2- 172.21.15.156:0/2896394440 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f66d406e0b0 0x7f66d4070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:03:11.008 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:11.006+0000 7f66deffd640 1 -- 172.21.15.156:0/2896394440 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f66e40add20 con 0x7f66e80fe560 2024-07-16T10:03:11.009 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:11.007+0000 7f66edc1c640 1 --2- 172.21.15.156:0/2896394440 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f66d406e0b0 0x7f66d4070560 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f66e810eee0 tx=0x7f66d8009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T10:03:11.012 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:11.011+0000 7f66deffd640 1 -- 172.21.15.156:0/2896394440 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187716 (secure 0 0 0) 0x7f66e4076f50 con 0x7f66e80fe560 2024-07-16T10:03:11.209 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:11.205+0000 7f66f0681640 1 -- 172.21.15.156:0/2896394440 --> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f66e8110f80 con 0x7f66d406e0b0 2024-07-16T10:03:11.210 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:11.207+0000 7f66deffd640 1 -- 172.21.15.156:0/2896394440 <== mgr.34104 v2:172.21.15.156:6800/3367765292 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f66e8110f80 con 0x7f66d406e0b0 2024-07-16T10:03:11.210 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:11.209+0000 7f66f0681640 1 -- 172.21.15.156:0/2896394440 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f66d406e0b0 msgr2=0x7f66d4070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:03:11.211 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:11.209+0000 7f66f0681640 1 --2- 172.21.15.156:0/2896394440 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f66d406e0b0 0x7f66d4070560 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f66e810eee0 tx=0x7f66d8009040 comp rx=0 tx=0).stop 2024-07-16T10:03:11.211 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:11.209+0000 7f66f0681640 1 -- 172.21.15.156:0/2896394440 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f66e80fe560 msgr2=0x7f66e810f130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:03:11.211 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:11.209+0000 7f66f0681640 1 --2- 172.21.15.156:0/2896394440 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f66e80fe560 0x7f66e810f130 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f66e4003080 tx=0x7f66e4009da0 comp rx=0 tx=0).stop 2024-07-16T10:03:11.211 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:11.209+0000 7f66f0681640 1 -- 172.21.15.156:0/2896394440 shutdown_connections 2024-07-16T10:03:11.211 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:11.209+0000 7f66f0681640 1 --2- 172.21.15.156:0/2896394440 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f66d406e0b0 0x7f66d4070560 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:03:11.211 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:11.209+0000 7f66f0681640 1 --2- 172.21.15.156:0/2896394440 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f66e80ff830 0x7f66e810f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:03:11.211 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:11.209+0000 7f66f0681640 1 --2- 172.21.15.156:0/2896394440 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f66e80fe560 0x7f66e810f130 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:03:11.211 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:11.209+0000 7f66f0681640 1 -- 172.21.15.156:0/2896394440 >> 172.21.15.156:0/2896394440 conn(0x7f66e80fa010 msgr2=0x7f66e80fc6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T10:03:11.211 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:11.209+0000 7f66f0681640 1 -- 172.21.15.156:0/2896394440 shutdown_connections 2024-07-16T10:03:11.211 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:11.209+0000 7f66f0681640 1 -- 172.21.15.156:0/2896394440 wait complete. 2024-07-16T10:03:11.218 INFO:teuthology.orchestra.run.smithi156.stdout:true 2024-07-16T10:03:11.290 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:11.289+0000 7f03e04f6640 1 -- 172.21.15.156:0/4292003512 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f03d80fb260 msgr2=0x7f03d80fb670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:03:11.291 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:11.289+0000 7f03e04f6640 1 --2- 172.21.15.156:0/4292003512 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f03d80fb260 0x7f03d80fb670 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f03d40048d0 tx=0x7f03d4009da0 comp rx=0 tx=0).stop 2024-07-16T10:03:11.291 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:11.289+0000 7f03e04f6640 1 -- 172.21.15.156:0/4292003512 shutdown_connections 2024-07-16T10:03:11.291 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:11.289+0000 7f03e04f6640 1 --2- 172.21.15.156:0/4292003512 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f03d80fc530 0x7f03d80fc9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:03:11.291 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:11.289+0000 7f03e04f6640 1 --2- 172.21.15.156:0/4292003512 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f03d80fb260 0x7f03d80fb670 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:03:11.291 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:11.289+0000 7f03e04f6640 1 -- 172.21.15.156:0/4292003512 >> 172.21.15.156:0/4292003512 conn(0x7f03d80f6a20 msgr2=0x7f03d80f8e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T10:03:11.291 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:03:11 smithi156 ceph-mon[93066]: mds.? [v2:172.21.15.156:6826/2852996640,v1:172.21.15.156:6827/2852996640] up:rejoin 2024-07-16T10:03:11.291 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:03:11 smithi156 ceph-mon[93066]: fsmap cephfs:1/1 {0=cephfs.smithi156.rdibys=up:rejoin} 2 up:standby 2024-07-16T10:03:11.292 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:03:11 smithi156 ceph-mon[93066]: daemon mds.cephfs.smithi156.rdibys is now active in filesystem cephfs as rank 0 2024-07-16T10:03:11.293 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:11.289+0000 7f03e04f6640 1 -- 172.21.15.156:0/4292003512 shutdown_connections 2024-07-16T10:03:11.293 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:11.289+0000 7f03e04f6640 1 -- 172.21.15.156:0/4292003512 wait complete. 2024-07-16T10:03:11.293 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:11.290+0000 7f03e04f6640 1 Processor -- start 2024-07-16T10:03:11.293 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:11.290+0000 7f03e04f6640 1 -- start start 2024-07-16T10:03:11.294 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:11.290+0000 7f03e04f6640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f03d80fb260 0x7f03d810f200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:03:11.294 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:11.290+0000 7f03e04f6640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f03d80fc530 0x7f03d810f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:03:11.294 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:11.290+0000 7f03e04f6640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03d810fd90 con 0x7f03d80fc530 2024-07-16T10:03:11.294 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:11.290+0000 7f03e04f6640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03d8110560 con 0x7f03d80fb260 2024-07-16T10:03:11.294 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:11.290+0000 7f03dda91640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f03d80fc530 0x7f03d810f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:03:11.294 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:11.290+0000 7f03dda91640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f03d80fc530 0x7f03d810f740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:35832/0 (socket says 172.21.15.156:35832) 2024-07-16T10:03:11.294 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:11.290+0000 7f03dda91640 1 -- 172.21.15.156:0/2700969439 learned_addr learned my addr 172.21.15.156:0/2700969439 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T10:03:11.294 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:11.290+0000 7f03de292640 1 --2- 172.21.15.156:0/2700969439 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f03d80fb260 0x7f03d810f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:03:11.294 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:11.290+0000 7f03dda91640 1 -- 172.21.15.156:0/2700969439 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f03d80fb260 msgr2=0x7f03d810f200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:03:11.294 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:11.290+0000 7f03dda91640 1 --2- 172.21.15.156:0/2700969439 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f03d80fb260 0x7f03d810f200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:03:11.294 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:11.291+0000 7f03dda91640 1 -- 172.21.15.156:0/2700969439 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03d4005040 con 0x7f03d80fc530 2024-07-16T10:03:11.295 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:11.291+0000 7f03de292640 1 --2- 172.21.15.156:0/2700969439 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f03d80fb260 0x7f03d810f200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-16T10:03:11.295 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:11.291+0000 7f03dda91640 1 --2- 172.21.15.156:0/2700969439 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f03d80fc530 0x7f03d810f740 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f03c800c830 tx=0x7f03c8004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T10:03:11.295 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:11.291+0000 7f03ceffd640 1 -- 172.21.15.156:0/2700969439 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f03c800a850 con 0x7f03d80fc530 2024-07-16T10:03:11.295 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:11.291+0000 7f03e04f6640 1 -- 172.21.15.156:0/2700969439 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03d81107f0 con 0x7f03d80fc530 2024-07-16T10:03:11.295 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:11.291+0000 7f03ceffd640 1 -- 172.21.15.156:0/2700969439 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f03c800ae80 con 0x7f03d80fc530 2024-07-16T10:03:11.295 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:11.291+0000 7f03e04f6640 1 -- 172.21.15.156:0/2700969439 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03d8110d90 con 0x7f03d80fc530 2024-07-16T10:03:11.295 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:11.292+0000 7f03ceffd640 1 -- 172.21.15.156:0/2700969439 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f03c8012ea0 con 0x7f03d80fc530 2024-07-16T10:03:11.295 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:11.292+0000 7f03e04f6640 1 -- 172.21.15.156:0/2700969439 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03bc005240 con 0x7f03d80fc530 2024-07-16T10:03:11.296 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:11.293+0000 7f03ceffd640 1 -- 172.21.15.156:0/2700969439 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 38) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f03c8011700 con 0x7f03d80fc530 2024-07-16T10:03:11.296 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:11.293+0000 7f03ceffd640 1 --2- 172.21.15.156:0/2700969439 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f03c406e0b0 0x7f03c4070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:03:11.296 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:11.293+0000 7f03ceffd640 1 -- 172.21.15.156:0/2700969439 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f03c808dd50 con 0x7f03d80fc530 2024-07-16T10:03:11.296 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:11.293+0000 7f03de292640 1 --2- 172.21.15.156:0/2700969439 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f03c406e0b0 0x7f03c4070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:03:11.296 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:11.294+0000 7f03de292640 1 --2- 172.21.15.156:0/2700969439 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f03c406e0b0 0x7f03c4070560 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f03d4009f80 tx=0x7f03d4035040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T10:03:11.299 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:11.298+0000 7f03ceffd640 1 -- 172.21.15.156:0/2700969439 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187716 (secure 0 0 0) 0x7f03c8057140 con 0x7f03d80fc530 2024-07-16T10:03:11.343 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:03:11 smithi176 ceph-mon[81689]: mds.? [v2:172.21.15.156:6826/2852996640,v1:172.21.15.156:6827/2852996640] up:rejoin 2024-07-16T10:03:11.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:03:11 smithi176 ceph-mon[81689]: fsmap cephfs:1/1 {0=cephfs.smithi156.rdibys=up:rejoin} 2 up:standby 2024-07-16T10:03:11.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:03:11 smithi176 ceph-mon[81689]: daemon mds.cephfs.smithi156.rdibys is now active in filesystem cephfs as rank 0 2024-07-16T10:03:11.494 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:11.493+0000 7f03e04f6640 1 -- 172.21.15.156:0/2700969439 --> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f03bc000bf0 con 0x7f03c406e0b0 2024-07-16T10:03:11.495 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:11.494+0000 7f03ceffd640 1 -- 172.21.15.156:0/2700969439 <== mgr.34104 v2:172.21.15.156:6800/3367765292 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f03bc000bf0 con 0x7f03c406e0b0 2024-07-16T10:03:11.498 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:11.496+0000 7f03e04f6640 1 -- 172.21.15.156:0/2700969439 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f03c406e0b0 msgr2=0x7f03c4070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:03:11.498 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:11.496+0000 7f03e04f6640 1 --2- 172.21.15.156:0/2700969439 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f03c406e0b0 0x7f03c4070560 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f03d4009f80 tx=0x7f03d4035040 comp rx=0 tx=0).stop 2024-07-16T10:03:11.498 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:11.496+0000 7f03e04f6640 1 -- 172.21.15.156:0/2700969439 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f03d80fc530 msgr2=0x7f03d810f740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:03:11.498 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:11.496+0000 7f03e04f6640 1 --2- 172.21.15.156:0/2700969439 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f03d80fc530 0x7f03d810f740 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f03c800c830 tx=0x7f03c8004430 comp rx=0 tx=0).stop 2024-07-16T10:03:11.498 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:11.497+0000 7f03e04f6640 1 -- 172.21.15.156:0/2700969439 shutdown_connections 2024-07-16T10:03:11.498 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:11.497+0000 7f03e04f6640 1 --2- 172.21.15.156:0/2700969439 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f03c406e0b0 0x7f03c4070560 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:03:11.498 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:11.497+0000 7f03e04f6640 1 --2- 172.21.15.156:0/2700969439 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f03d80fc530 0x7f03d810f740 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:03:11.498 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:11.497+0000 7f03e04f6640 1 --2- 172.21.15.156:0/2700969439 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f03d80fb260 0x7f03d810f200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:03:11.498 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:11.497+0000 7f03e04f6640 1 -- 172.21.15.156:0/2700969439 >> 172.21.15.156:0/2700969439 conn(0x7f03d80f6a20 msgr2=0x7f03d8065e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T10:03:11.499 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:11.497+0000 7f03e04f6640 1 -- 172.21.15.156:0/2700969439 shutdown_connections 2024-07-16T10:03:11.499 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:11.497+0000 7f03e04f6640 1 -- 172.21.15.156:0/2700969439 wait complete. 2024-07-16T10:03:11.577 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:11.576+0000 7f3540080640 1 -- 172.21.15.156:0/3973016742 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f35380664c0 msgr2=0x7f35380feff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:03:11.577 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:11.576+0000 7f3540080640 1 --2- 172.21.15.156:0/3973016742 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f35380664c0 0x7f35380feff0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f35280048d0 tx=0x7f3528009da0 comp rx=0 tx=0).stop 2024-07-16T10:03:11.577 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:11.576+0000 7f3540080640 1 -- 172.21.15.156:0/3973016742 shutdown_connections 2024-07-16T10:03:11.578 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:11.576+0000 7f3540080640 1 --2- 172.21.15.156:0/3973016742 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f35380ff530 0x7f3538101910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:03:11.578 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:11.576+0000 7f3540080640 1 --2- 172.21.15.156:0/3973016742 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f35380664c0 0x7f35380feff0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:03:11.578 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:11.576+0000 7f3540080640 1 -- 172.21.15.156:0/3973016742 >> 172.21.15.156:0/3973016742 conn(0x7f35380f6a20 msgr2=0x7f35380f8e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T10:03:11.578 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:11.576+0000 7f3540080640 1 -- 172.21.15.156:0/3973016742 shutdown_connections 2024-07-16T10:03:11.578 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:11.576+0000 7f3540080640 1 -- 172.21.15.156:0/3973016742 wait complete. 2024-07-16T10:03:11.579 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:11.576+0000 7f3540080640 1 Processor -- start 2024-07-16T10:03:11.579 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:11.577+0000 7f3540080640 1 -- start start 2024-07-16T10:03:11.579 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:11.577+0000 7f3540080640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f35380664c0 0x7f35380fca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:03:11.580 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:11.577+0000 7f3540080640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f35380ff530 0x7f35380faef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:03:11.580 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:11.577+0000 7f3540080640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35380fb480 con 0x7f35380ff530 2024-07-16T10:03:11.580 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:11.577+0000 7f3540080640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35380fb5c0 con 0x7f35380664c0 2024-07-16T10:03:11.580 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:11.577+0000 7f353d61b640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f35380ff530 0x7f35380faef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:03:11.580 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:11.577+0000 7f353d61b640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f35380ff530 0x7f35380faef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:35846/0 (socket says 172.21.15.156:35846) 2024-07-16T10:03:11.580 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:11.577+0000 7f353d61b640 1 -- 172.21.15.156:0/2639977579 learned_addr learned my addr 172.21.15.156:0/2639977579 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T10:03:11.580 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:11.577+0000 7f353de1c640 1 --2- 172.21.15.156:0/2639977579 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f35380664c0 0x7f35380fca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:03:11.580 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:11.577+0000 7f353d61b640 1 -- 172.21.15.156:0/2639977579 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f35380664c0 msgr2=0x7f35380fca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:03:11.580 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:11.577+0000 7f353d61b640 1 --2- 172.21.15.156:0/2639977579 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f35380664c0 0x7f35380fca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:03:11.580 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:11.577+0000 7f353d61b640 1 -- 172.21.15.156:0/2639977579 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3528005040 con 0x7f35380ff530 2024-07-16T10:03:11.580 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:11.577+0000 7f353de1c640 1 --2- 172.21.15.156:0/2639977579 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f35380664c0 0x7f35380fca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-16T10:03:11.580 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:11.577+0000 7f353d61b640 1 --2- 172.21.15.156:0/2639977579 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f35380ff530 0x7f35380faef0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f353400c800 tx=0x7f3534004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T10:03:11.581 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:11.578+0000 7f352e7fc640 1 -- 172.21.15.156:0/2639977579 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f353400a820 con 0x7f35380ff530 2024-07-16T10:03:11.581 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:11.578+0000 7f3540080640 1 -- 172.21.15.156:0/2639977579 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35380fb8a0 con 0x7f35380ff530 2024-07-16T10:03:11.581 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:11.578+0000 7f3540080640 1 -- 172.21.15.156:0/2639977579 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3538115050 con 0x7f35380ff530 2024-07-16T10:03:11.581 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:11.578+0000 7f352e7fc640 1 -- 172.21.15.156:0/2639977579 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f353400ae50 con 0x7f35380ff530 2024-07-16T10:03:11.581 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:11.578+0000 7f352e7fc640 1 -- 172.21.15.156:0/2639977579 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f35340115e0 con 0x7f35380ff530 2024-07-16T10:03:11.582 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:11.579+0000 7f3540080640 1 -- 172.21.15.156:0/2639977579 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35380fa6c0 con 0x7f35380ff530 2024-07-16T10:03:11.582 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:11.579+0000 7f352e7fc640 1 -- 172.21.15.156:0/2639977579 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 38) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f3534012400 con 0x7f35380ff530 2024-07-16T10:03:11.582 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:11.580+0000 7f352e7fc640 1 --2- 172.21.15.156:0/2639977579 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f352406e0b0 0x7f3524070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:03:11.582 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:11.580+0000 7f352e7fc640 1 -- 172.21.15.156:0/2639977579 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f353408dcf0 con 0x7f35380ff530 2024-07-16T10:03:11.582 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:11.580+0000 7f353de1c640 1 --2- 172.21.15.156:0/2639977579 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f352406e0b0 0x7f3524070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:03:11.582 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:11.580+0000 7f353de1c640 1 --2- 172.21.15.156:0/2639977579 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f352406e0b0 0x7f3524070560 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f35380fc7b0 tx=0x7f3528035040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T10:03:11.585 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:11.584+0000 7f352e7fc640 1 -- 172.21.15.156:0/2639977579 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187716 (secure 0 0 0) 0x7f35340570e0 con 0x7f35380ff530 2024-07-16T10:03:11.769 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:11.768+0000 7f3540080640 1 -- 172.21.15.156:0/2639977579 --> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f35380f8560 con 0x7f352406e0b0 2024-07-16T10:03:11.778 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:11.777+0000 7f352e7fc640 1 -- 172.21.15.156:0/2639977579 <== mgr.34104 v2:172.21.15.156:6800/3367765292 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7f35380f8560 con 0x7f352406e0b0 2024-07-16T10:03:11.779 INFO:teuthology.orchestra.run.smithi156.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-07-16T10:03:11.779 INFO:teuthology.orchestra.run.smithi156.stdout:alertmanager.smithi156 smithi156 *:9093,9094 running (6m) 15s ago 15m 24.5M - 0.25.0 c8568f914cd2 d524a2c3ff15 2024-07-16T10:03:11.779 INFO:teuthology.orchestra.run.smithi156.stdout:crash.smithi156 smithi156 running (4m) 15s ago 15m 6899k - 19.0.0-3495-ga6ca461b 16f7feb04342 11648cf29af4 2024-07-16T10:03:11.779 INFO:teuthology.orchestra.run.smithi156.stdout:crash.smithi176 smithi176 running (4m) 82s ago 14m 6899k - 19.0.0-3495-ga6ca461b 16f7feb04342 67f3ccc034f8 2024-07-16T10:03:11.779 INFO:teuthology.orchestra.run.smithi156.stdout:grafana.smithi156 smithi156 *:3000 running (6m) 15s ago 15m 83.9M - 9.4.7 954c08fa6188 deb9e1bfeb66 2024-07-16T10:03:11.779 INFO:teuthology.orchestra.run.smithi156.stdout:mds.cephfs.smithi156.evwwaz smithi156 running (10m) 15s ago 10m 131M - 17.2.7-1104-ge6231b04 d5de5f953b95 7c0ff9ae43f8 2024-07-16T10:03:11.779 INFO:teuthology.orchestra.run.smithi156.stdout:mds.cephfs.smithi156.rdibys smithi156 running (24s) 15s ago 10m 15.2M - 19.0.0-3495-ga6ca461b 16f7feb04342 248f61472011 2024-07-16T10:03:11.779 INFO:teuthology.orchestra.run.smithi156.stdout:mds.cephfs.smithi176.dvvbud smithi176 running (10m) 82s ago 10m 108M - 17.2.7-1104-ge6231b04 d5de5f953b95 1b00147950f9 2024-07-16T10:03:11.779 INFO:teuthology.orchestra.run.smithi156.stdout:mds.cephfs.smithi176.wahbsi smithi176 running (10m) 82s ago 10m 62.3M - 17.2.7-1104-ge6231b04 d5de5f953b95 866664af2311 2024-07-16T10:03:11.779 INFO:teuthology.orchestra.run.smithi156.stdout:mgr.smithi156.zkvylv smithi156 *:8443,9283,8765 running (6m) 15s ago 16m 504M - 19.0.0-3495-ga6ca461b 16f7feb04342 ee53bd604b7c 2024-07-16T10:03:11.779 INFO:teuthology.orchestra.run.smithi156.stdout:mgr.smithi176.dqbrvx smithi176 *:8443,9283,8765 running (5m) 82s ago 14m 441M - 19.0.0-3495-ga6ca461b 16f7feb04342 ce9eaae4c2d0 2024-07-16T10:03:11.780 INFO:teuthology.orchestra.run.smithi156.stdout:mon.smithi156 smithi156 running (5m) 15s ago 16m 50.2M 2048M 19.0.0-3495-ga6ca461b 16f7feb04342 77b6ec8c50eb 2024-07-16T10:03:11.780 INFO:teuthology.orchestra.run.smithi156.stdout:mon.smithi176 smithi176 running (5m) 82s ago 14m 40.4M 2048M 19.0.0-3495-ga6ca461b 16f7feb04342 ef92814cec2b 2024-07-16T10:03:11.780 INFO:teuthology.orchestra.run.smithi156.stdout:node-exporter.smithi156 smithi156 *:9100 running (7m) 15s ago 15m 18.7M - 1.5.0 0da6a335fe13 e5d86f23a14c 2024-07-16T10:03:11.780 INFO:teuthology.orchestra.run.smithi156.stdout:node-exporter.smithi176 smithi176 *:9100 running (6m) 82s ago 14m 15.6M - 1.5.0 0da6a335fe13 ea842728144a 2024-07-16T10:03:11.780 INFO:teuthology.orchestra.run.smithi156.stdout:osd.0 smithi156 running (4m) 15s ago 13m 102M 1557M 19.0.0-3495-ga6ca461b 16f7feb04342 8633df1689cc 2024-07-16T10:03:11.780 INFO:teuthology.orchestra.run.smithi156.stdout:osd.1 smithi156 running (3m) 15s ago 13m 114M 1557M 19.0.0-3495-ga6ca461b 16f7feb04342 e6fae9653b46 2024-07-16T10:03:11.780 INFO:teuthology.orchestra.run.smithi156.stdout:osd.2 smithi156 running (3m) 15s ago 12m 95.7M 1557M 19.0.0-3495-ga6ca461b 16f7feb04342 c6ecf18de9bf 2024-07-16T10:03:11.780 INFO:teuthology.orchestra.run.smithi156.stdout:osd.3 smithi176 running (2m) 82s ago 12m 99.7M 2581M 19.0.0-3495-ga6ca461b 16f7feb04342 d216534d6027 2024-07-16T10:03:11.780 INFO:teuthology.orchestra.run.smithi156.stdout:osd.4 smithi176 running (117s) 82s ago 12m 100M 2581M 19.0.0-3495-ga6ca461b 16f7feb04342 1bd3a25317b0 2024-07-16T10:03:11.780 INFO:teuthology.orchestra.run.smithi156.stdout:osd.5 smithi176 running (85s) 82s ago 11m 11.4M 2581M 19.0.0-3495-ga6ca461b 16f7feb04342 a03640221e7a 2024-07-16T10:03:11.780 INFO:teuthology.orchestra.run.smithi156.stdout:prometheus.smithi156 smithi156 *:9095 running (5m) 15s ago 14m 73.5M - 2.43.0 a07b618ecd1d 20c195ecd665 2024-07-16T10:03:11.781 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:11.779+0000 7f3540080640 1 -- 172.21.15.156:0/2639977579 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f352406e0b0 msgr2=0x7f3524070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:03:11.781 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:11.779+0000 7f3540080640 1 --2- 172.21.15.156:0/2639977579 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f352406e0b0 0x7f3524070560 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f35380fc7b0 tx=0x7f3528035040 comp rx=0 tx=0).stop 2024-07-16T10:03:11.781 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:11.779+0000 7f3540080640 1 -- 172.21.15.156:0/2639977579 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f35380ff530 msgr2=0x7f35380faef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:03:11.781 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:11.779+0000 7f3540080640 1 --2- 172.21.15.156:0/2639977579 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f35380ff530 0x7f35380faef0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f353400c800 tx=0x7f3534004430 comp rx=0 tx=0).stop 2024-07-16T10:03:11.781 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:11.779+0000 7f3540080640 1 -- 172.21.15.156:0/2639977579 shutdown_connections 2024-07-16T10:03:11.781 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:11.779+0000 7f3540080640 1 --2- 172.21.15.156:0/2639977579 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f352406e0b0 0x7f3524070560 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:03:11.782 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:11.779+0000 7f3540080640 1 --2- 172.21.15.156:0/2639977579 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f35380ff530 0x7f35380faef0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:03:11.782 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:11.779+0000 7f3540080640 1 --2- 172.21.15.156:0/2639977579 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f35380664c0 0x7f35380fca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:03:11.782 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:11.779+0000 7f3540080640 1 -- 172.21.15.156:0/2639977579 >> 172.21.15.156:0/2639977579 conn(0x7f35380f6a20 msgr2=0x7f35380f6e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T10:03:11.782 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:11.780+0000 7f3540080640 1 -- 172.21.15.156:0/2639977579 shutdown_connections 2024-07-16T10:03:11.782 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:11.780+0000 7f3540080640 1 -- 172.21.15.156:0/2639977579 wait complete. 2024-07-16T10:03:11.860 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:11.857+0000 7f63bb722640 1 -- 172.21.15.156:0/2896292998 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f63b40ff490 msgr2=0x7f63b4101870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:03:11.860 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:11.857+0000 7f63bb722640 1 --2- 172.21.15.156:0/2896292998 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f63b40ff490 0x7f63b4101870 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f63a40048d0 tx=0x7f63a4009da0 comp rx=0 tx=0).stop 2024-07-16T10:03:11.860 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:11.858+0000 7f63bb722640 1 -- 172.21.15.156:0/2896292998 shutdown_connections 2024-07-16T10:03:11.860 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:11.858+0000 7f63bb722640 1 --2- 172.21.15.156:0/2896292998 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f63b40ff490 0x7f63b4101870 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:03:11.860 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:11.858+0000 7f63bb722640 1 --2- 172.21.15.156:0/2896292998 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f63b4065a50 0x7f63b40fef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:03:11.860 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:11.858+0000 7f63bb722640 1 -- 172.21.15.156:0/2896292998 >> 172.21.15.156:0/2896292998 conn(0x7f63b40f6a20 msgr2=0x7f63b40f8e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T10:03:11.860 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:11.858+0000 7f63bb722640 1 -- 172.21.15.156:0/2896292998 shutdown_connections 2024-07-16T10:03:11.861 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:11.858+0000 7f63bb722640 1 -- 172.21.15.156:0/2896292998 wait complete. 2024-07-16T10:03:11.861 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:11.858+0000 7f63bb722640 1 Processor -- start 2024-07-16T10:03:11.861 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:11.858+0000 7f63bb722640 1 -- start start 2024-07-16T10:03:11.861 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:11.858+0000 7f63bb722640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f63b4065a50 0x7f63b410cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:03:11.861 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:11.858+0000 7f63bb722640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f63b40ff490 0x7f63b410d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:03:11.861 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:11.858+0000 7f63bb722640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63b410db40 con 0x7f63b4065a50 2024-07-16T10:03:11.861 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:11.858+0000 7f63bb722640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63b410e310 con 0x7f63b40ff490 2024-07-16T10:03:11.861 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:11.859+0000 7f63b94be640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f63b4065a50 0x7f63b410cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:03:11.861 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:11.859+0000 7f63b94be640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f63b4065a50 0x7f63b410cfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:35862/0 (socket says 172.21.15.156:35862) 2024-07-16T10:03:11.861 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:11.859+0000 7f63b94be640 1 -- 172.21.15.156:0/2891660215 learned_addr learned my addr 172.21.15.156:0/2891660215 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T10:03:11.863 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:11.859+0000 7f63b94be640 1 -- 172.21.15.156:0/2891660215 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f63b40ff490 msgr2=0x7f63b410d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:03:11.863 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:11.859+0000 7f63b8cbd640 1 --2- 172.21.15.156:0/2891660215 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f63b40ff490 0x7f63b410d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:03:11.863 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:11.859+0000 7f63b94be640 1 --2- 172.21.15.156:0/2891660215 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f63b40ff490 0x7f63b410d4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:03:11.864 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:11.859+0000 7f63b94be640 1 -- 172.21.15.156:0/2891660215 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63a4005040 con 0x7f63b4065a50 2024-07-16T10:03:11.864 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:11.859+0000 7f63b94be640 1 --2- 172.21.15.156:0/2891660215 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f63b4065a50 0x7f63b410cfb0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f63b0002840 tx=0x7f63b0004570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T10:03:11.864 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:11.859+0000 7f63a9ffb640 1 -- 172.21.15.156:0/2891660215 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f63b0007c30 con 0x7f63b4065a50 2024-07-16T10:03:11.864 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:11.859+0000 7f63bb722640 1 -- 172.21.15.156:0/2891660215 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63b410e5a0 con 0x7f63b4065a50 2024-07-16T10:03:11.864 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:11.859+0000 7f63a9ffb640 1 -- 172.21.15.156:0/2891660215 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f63b000f040 con 0x7f63b4065a50 2024-07-16T10:03:11.864 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:11.859+0000 7f63a9ffb640 1 -- 172.21.15.156:0/2891660215 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f63b000da30 con 0x7f63b4065a50 2024-07-16T10:03:11.864 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:11.860+0000 7f63bb722640 1 -- 172.21.15.156:0/2891660215 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63b410eb40 con 0x7f63b4065a50 2024-07-16T10:03:11.864 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:11.861+0000 7f63bb722640 1 -- 172.21.15.156:0/2891660215 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63b40fa6a0 con 0x7f63b4065a50 2024-07-16T10:03:11.864 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:11.861+0000 7f63a9ffb640 1 -- 172.21.15.156:0/2891660215 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 38) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f63b00096a0 con 0x7f63b4065a50 2024-07-16T10:03:11.864 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:11.861+0000 7f63a9ffb640 1 --2- 172.21.15.156:0/2891660215 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f63a006e0b0 0x7f63a0070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:03:11.864 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:11.862+0000 7f63a9ffb640 1 -- 172.21.15.156:0/2891660215 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f63b008db20 con 0x7f63b4065a50 2024-07-16T10:03:11.864 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:11.862+0000 7f63b8cbd640 1 --2- 172.21.15.156:0/2891660215 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f63a006e0b0 0x7f63a0070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:03:11.865 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:11.862+0000 7f63b8cbd640 1 --2- 172.21.15.156:0/2891660215 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f63a006e0b0 0x7f63a0070560 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f63a4000c00 tx=0x7f63a4009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T10:03:11.867 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:11.866+0000 7f63a9ffb640 1 -- 172.21.15.156:0/2891660215 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187716 (secure 0 0 0) 0x7f63b0056df0 con 0x7f63b4065a50 2024-07-16T10:03:12.114 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:12.112+0000 7f63bb722640 1 -- 172.21.15.156:0/2891660215 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f63b404b530 con 0x7f63b4065a50 2024-07-16T10:03:12.114 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:03:12 smithi156 ceph-mon[93066]: pgmap v197: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 13 MiB/s rd, 4 op/s 2024-07-16T10:03:12.114 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:03:12 smithi156 ceph-mon[93066]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-07-16T10:03:12.115 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:03:12 smithi156 ceph-mon[93066]: Cluster is now healthy 2024-07-16T10:03:12.115 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:03:12 smithi156 ceph-mon[93066]: mds.? [v2:172.21.15.156:6826/2852996640,v1:172.21.15.156:6827/2852996640] up:active 2024-07-16T10:03:12.115 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:03:12 smithi156 ceph-mon[93066]: fsmap cephfs:1 {0=cephfs.smithi156.rdibys=up:active} 2 up:standby 2024-07-16T10:03:12.115 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:03:12 smithi156 ceph-mon[93066]: from='client.34442 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-16T10:03:12.115 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:12.113+0000 7f63a9ffb640 1 -- 172.21.15.156:0/2891660215 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+834 (secure 0 0 0) 0x7f63b005a4b0 con 0x7f63b4065a50 2024-07-16T10:03:12.116 INFO:teuthology.orchestra.run.smithi156.stdout:{ 2024-07-16T10:03:12.116 INFO:teuthology.orchestra.run.smithi156.stdout: "mon": { 2024-07-16T10:03:12.116 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph version 19.0.0-3495-ga6ca461b (a6ca461be7cc1aff3658c2c6df4cff975a24449a) squid (rc)": 2 2024-07-16T10:03:12.116 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-07-16T10:03:12.117 INFO:teuthology.orchestra.run.smithi156.stdout: "mgr": { 2024-07-16T10:03:12.117 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph version 19.0.0-3495-ga6ca461b (a6ca461be7cc1aff3658c2c6df4cff975a24449a) squid (rc)": 2 2024-07-16T10:03:12.117 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-07-16T10:03:12.117 INFO:teuthology.orchestra.run.smithi156.stdout: "osd": { 2024-07-16T10:03:12.117 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph version 19.0.0-3495-ga6ca461b (a6ca461be7cc1aff3658c2c6df4cff975a24449a) squid (rc)": 6 2024-07-16T10:03:12.117 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-07-16T10:03:12.117 INFO:teuthology.orchestra.run.smithi156.stdout: "mds": { 2024-07-16T10:03:12.118 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph version 17.2.7-1104-ge6231b04 (e6231b04b52ebf3b9b31803282dd3074439e51ac) quincy (stable)": 2, 2024-07-16T10:03:12.118 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph version 19.0.0-3495-ga6ca461b (a6ca461be7cc1aff3658c2c6df4cff975a24449a) squid (rc)": 1 2024-07-16T10:03:12.118 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-07-16T10:03:12.118 INFO:teuthology.orchestra.run.smithi156.stdout: "overall": { 2024-07-16T10:03:12.118 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph version 17.2.7-1104-ge6231b04 (e6231b04b52ebf3b9b31803282dd3074439e51ac) quincy (stable)": 2, 2024-07-16T10:03:12.118 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph version 19.0.0-3495-ga6ca461b (a6ca461be7cc1aff3658c2c6df4cff975a24449a) squid (rc)": 11 2024-07-16T10:03:12.119 INFO:teuthology.orchestra.run.smithi156.stdout: } 2024-07-16T10:03:12.119 INFO:teuthology.orchestra.run.smithi156.stdout:} 2024-07-16T10:03:12.120 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:12.115+0000 7f63bb722640 1 -- 172.21.15.156:0/2891660215 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f63a006e0b0 msgr2=0x7f63a0070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:03:12.120 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:12.115+0000 7f63bb722640 1 --2- 172.21.15.156:0/2891660215 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f63a006e0b0 0x7f63a0070560 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f63a4000c00 tx=0x7f63a4009da0 comp rx=0 tx=0).stop 2024-07-16T10:03:12.120 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:12.115+0000 7f63bb722640 1 -- 172.21.15.156:0/2891660215 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f63b4065a50 msgr2=0x7f63b410cfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:03:12.120 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:12.115+0000 7f63bb722640 1 --2- 172.21.15.156:0/2891660215 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f63b4065a50 0x7f63b410cfb0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f63b0002840 tx=0x7f63b0004570 comp rx=0 tx=0).stop 2024-07-16T10:03:12.120 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:12.115+0000 7f63bb722640 1 -- 172.21.15.156:0/2891660215 shutdown_connections 2024-07-16T10:03:12.120 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:12.115+0000 7f63bb722640 1 --2- 172.21.15.156:0/2891660215 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f63a006e0b0 0x7f63a0070560 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:03:12.121 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:12.115+0000 7f63bb722640 1 --2- 172.21.15.156:0/2891660215 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f63b40ff490 0x7f63b410d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:03:12.121 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:12.115+0000 7f63bb722640 1 --2- 172.21.15.156:0/2891660215 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f63b4065a50 0x7f63b410cfb0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:03:12.121 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:12.115+0000 7f63bb722640 1 -- 172.21.15.156:0/2891660215 >> 172.21.15.156:0/2891660215 conn(0x7f63b40f6a20 msgr2=0x7f63b40f6df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T10:03:12.121 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:12.115+0000 7f63bb722640 1 -- 172.21.15.156:0/2891660215 shutdown_connections 2024-07-16T10:03:12.121 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:12.115+0000 7f63bb722640 1 -- 172.21.15.156:0/2891660215 wait complete. 2024-07-16T10:03:12.197 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:12.194+0000 7f764481b640 1 -- 172.21.15.156:0/615475205 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f76400fb240 msgr2=0x7f76400fb650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:03:12.197 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:12.194+0000 7f764481b640 1 --2- 172.21.15.156:0/615475205 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f76400fb240 0x7f76400fb650 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f76300048d0 tx=0x7f7630009da0 comp rx=0 tx=0).stop 2024-07-16T10:03:12.197 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:12.194+0000 7f764481b640 1 -- 172.21.15.156:0/615475205 shutdown_connections 2024-07-16T10:03:12.198 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:12.194+0000 7f764481b640 1 --2- 172.21.15.156:0/615475205 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f76400fc510 0x7f76400fc980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:03:12.198 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:12.194+0000 7f764481b640 1 --2- 172.21.15.156:0/615475205 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f76400fb240 0x7f76400fb650 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:03:12.198 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:12.194+0000 7f764481b640 1 -- 172.21.15.156:0/615475205 >> 172.21.15.156:0/615475205 conn(0x7f76400f6a20 msgr2=0x7f76400f8e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T10:03:12.198 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:12.195+0000 7f764481b640 1 -- 172.21.15.156:0/615475205 shutdown_connections 2024-07-16T10:03:12.198 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:12.195+0000 7f764481b640 1 -- 172.21.15.156:0/615475205 wait complete. 2024-07-16T10:03:12.198 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:12.195+0000 7f764481b640 1 Processor -- start 2024-07-16T10:03:12.198 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:12.195+0000 7f764481b640 1 -- start start 2024-07-16T10:03:12.198 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:12.195+0000 7f764481b640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f76400fb240 0x7f764010fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:03:12.198 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:12.195+0000 7f764481b640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f76400fc510 0x7f76401121d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:03:12.198 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:12.195+0000 7f764481b640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7640065a70 con 0x7f76400fc510 2024-07-16T10:03:12.198 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:12.195+0000 7f764481b640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7640065be0 con 0x7f76400fb240 2024-07-16T10:03:12.199 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:12.195+0000 7f7637fff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f76400fc510 0x7f76401121d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:03:12.199 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:12.195+0000 7f7637fff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f76400fc510 0x7f76401121d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:35872/0 (socket says 172.21.15.156:35872) 2024-07-16T10:03:12.199 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:12.195+0000 7f7637fff640 1 -- 172.21.15.156:0/2192636445 learned_addr learned my addr 172.21.15.156:0/2192636445 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T10:03:12.199 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:12.196+0000 7f763e59c640 1 --2- 172.21.15.156:0/2192636445 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f76400fb240 0x7f764010fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:03:12.199 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:12.196+0000 7f7637fff640 1 -- 172.21.15.156:0/2192636445 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f76400fb240 msgr2=0x7f764010fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:03:12.199 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:12.196+0000 7f7637fff640 1 --2- 172.21.15.156:0/2192636445 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f76400fb240 0x7f764010fc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:03:12.199 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:12.196+0000 7f7637fff640 1 -- 172.21.15.156:0/2192636445 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7630005040 con 0x7f76400fc510 2024-07-16T10:03:12.199 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:12.196+0000 7f7637fff640 1 --2- 172.21.15.156:0/2192636445 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f76400fc510 0x7f76401121d0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f7628002a50 tx=0x7f7628004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T10:03:12.199 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:12.196+0000 7f7636ffd640 1 -- 172.21.15.156:0/2192636445 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7628007c30 con 0x7f76400fc510 2024-07-16T10:03:12.199 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:12.196+0000 7f7636ffd640 1 -- 172.21.15.156:0/2192636445 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f762800f040 con 0x7f76400fc510 2024-07-16T10:03:12.199 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:12.196+0000 7f764481b640 1 -- 172.21.15.156:0/2192636445 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7640112770 con 0x7f76400fc510 2024-07-16T10:03:12.199 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:12.196+0000 7f7636ffd640 1 -- 172.21.15.156:0/2192636445 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f762800dbe0 con 0x7f76400fc510 2024-07-16T10:03:12.200 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:12.196+0000 7f764481b640 1 -- 172.21.15.156:0/2192636445 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7640112ce0 con 0x7f76400fc510 2024-07-16T10:03:12.200 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:12.197+0000 7f764481b640 1 -- 172.21.15.156:0/2192636445 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76400fa6a0 con 0x7f76400fc510 2024-07-16T10:03:12.200 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:12.198+0000 7f7636ffd640 1 -- 172.21.15.156:0/2192636445 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 38) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f762800dd40 con 0x7f76400fc510 2024-07-16T10:03:12.201 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:12.198+0000 7f7636ffd640 1 --2- 172.21.15.156:0/2192636445 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f762c06e0b0 0x7f762c070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:03:12.201 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:12.198+0000 7f7636ffd640 1 -- 172.21.15.156:0/2192636445 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f762808d860 con 0x7f76400fc510 2024-07-16T10:03:12.201 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:12.198+0000 7f763e59c640 1 --2- 172.21.15.156:0/2192636445 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f762c06e0b0 0x7f762c070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:03:12.201 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:12.199+0000 7f763e59c640 1 --2- 172.21.15.156:0/2192636445 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f762c06e0b0 0x7f762c070560 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f7630000c00 tx=0x7f7630035040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T10:03:12.204 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:12.203+0000 7f7636ffd640 1 -- 172.21.15.156:0/2192636445 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187716 (secure 0 0 0) 0x7f7628056bb0 con 0x7f76400fc510 2024-07-16T10:03:12.343 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:03:12 smithi176 ceph-mon[81689]: pgmap v197: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 13 MiB/s rd, 4 op/s 2024-07-16T10:03:12.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:03:12 smithi176 ceph-mon[81689]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-07-16T10:03:12.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:03:12 smithi176 ceph-mon[81689]: Cluster is now healthy 2024-07-16T10:03:12.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:03:12 smithi176 ceph-mon[81689]: mds.? [v2:172.21.15.156:6826/2852996640,v1:172.21.15.156:6827/2852996640] up:active 2024-07-16T10:03:12.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:03:12 smithi176 ceph-mon[81689]: fsmap cephfs:1 {0=cephfs.smithi156.rdibys=up:active} 2 up:standby 2024-07-16T10:03:12.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:03:12 smithi176 ceph-mon[81689]: from='client.34442 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-16T10:03:12.420 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:12.418+0000 7f764481b640 1 -- 172.21.15.156:0/2192636445 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f764004b530 con 0x7f76400fc510 2024-07-16T10:03:12.420 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:12.419+0000 7f7636ffd640 1 -- 172.21.15.156:0/2192636445 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 36 v36) v1 ==== 76+0+1763 (secure 0 0 0) 0x7f762805a270 con 0x7f76400fc510 2024-07-16T10:03:12.420 INFO:teuthology.orchestra.run.smithi156.stdout:e36 2024-07-16T10:03:12.420 INFO:teuthology.orchestra.run.smithi156.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-07-16T10:03:12.421 INFO:teuthology.orchestra.run.smithi156.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-07-16T10:03:12.421 INFO:teuthology.orchestra.run.smithi156.stdout:legacy client fscid: 1 2024-07-16T10:03:12.421 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-07-16T10:03:12.421 INFO:teuthology.orchestra.run.smithi156.stdout:Filesystem 'cephfs' (1) 2024-07-16T10:03:12.421 INFO:teuthology.orchestra.run.smithi156.stdout:fs_name cephfs 2024-07-16T10:03:12.421 INFO:teuthology.orchestra.run.smithi156.stdout:epoch 36 2024-07-16T10:03:12.421 INFO:teuthology.orchestra.run.smithi156.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-07-16T10:03:12.421 INFO:teuthology.orchestra.run.smithi156.stdout:created 2024-07-16T09:52:55.214975+0000 2024-07-16T10:03:12.421 INFO:teuthology.orchestra.run.smithi156.stdout:modified 2024-07-16T10:03:11.011991+0000 2024-07-16T10:03:12.421 INFO:teuthology.orchestra.run.smithi156.stdout:tableserver 0 2024-07-16T10:03:12.421 INFO:teuthology.orchestra.run.smithi156.stdout:root 0 2024-07-16T10:03:12.421 INFO:teuthology.orchestra.run.smithi156.stdout:session_timeout 60 2024-07-16T10:03:12.421 INFO:teuthology.orchestra.run.smithi156.stdout:session_autoclose 300 2024-07-16T10:03:12.421 INFO:teuthology.orchestra.run.smithi156.stdout:max_file_size 1099511627776 2024-07-16T10:03:12.422 INFO:teuthology.orchestra.run.smithi156.stdout:max_xattr_size 65536 2024-07-16T10:03:12.422 INFO:teuthology.orchestra.run.smithi156.stdout:required_client_features {} 2024-07-16T10:03:12.422 INFO:teuthology.orchestra.run.smithi156.stdout:last_failure 0 2024-07-16T10:03:12.422 INFO:teuthology.orchestra.run.smithi156.stdout:last_failure_osd_epoch 82 2024-07-16T10:03:12.422 INFO:teuthology.orchestra.run.smithi156.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-07-16T10:03:12.422 INFO:teuthology.orchestra.run.smithi156.stdout:max_mds 1 2024-07-16T10:03:12.422 INFO:teuthology.orchestra.run.smithi156.stdout:in 0 2024-07-16T10:03:12.422 INFO:teuthology.orchestra.run.smithi156.stdout:up {0=34438} 2024-07-16T10:03:12.422 INFO:teuthology.orchestra.run.smithi156.stdout:failed 2024-07-16T10:03:12.422 INFO:teuthology.orchestra.run.smithi156.stdout:damaged 2024-07-16T10:03:12.422 INFO:teuthology.orchestra.run.smithi156.stdout:stopped 1 2024-07-16T10:03:12.422 INFO:teuthology.orchestra.run.smithi156.stdout:data_pools [3] 2024-07-16T10:03:12.422 INFO:teuthology.orchestra.run.smithi156.stdout:metadata_pool 2 2024-07-16T10:03:12.423 INFO:teuthology.orchestra.run.smithi156.stdout:inline_data disabled 2024-07-16T10:03:12.423 INFO:teuthology.orchestra.run.smithi156.stdout:balancer 2024-07-16T10:03:12.423 INFO:teuthology.orchestra.run.smithi156.stdout:bal_rank_mask -1 2024-07-16T10:03:12.423 INFO:teuthology.orchestra.run.smithi156.stdout:standby_count_wanted 1 2024-07-16T10:03:12.423 INFO:teuthology.orchestra.run.smithi156.stdout:qdb_cluster leader: 34438 members: 34438 2024-07-16T10:03:12.423 INFO:teuthology.orchestra.run.smithi156.stdout:[mds.cephfs.smithi156.rdibys{0:34438} state up:active seq 9 join_fscid=1 addr [v2:172.21.15.156:6826/2852996640,v1:172.21.15.156:6827/2852996640] compat {c=[1],r=[1],i=[1fff]}] 2024-07-16T10:03:12.423 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-07-16T10:03:12.423 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-07-16T10:03:12.423 INFO:teuthology.orchestra.run.smithi156.stdout:Standby daemons: 2024-07-16T10:03:12.423 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-07-16T10:03:12.423 INFO:teuthology.orchestra.run.smithi156.stdout:[mds.cephfs.smithi176.wahbsi{-1:44269} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.176:6826/3942496125,v1:172.21.15.176:6827/3942496125] compat {c=[1],r=[1],i=[7ff]}] 2024-07-16T10:03:12.423 INFO:teuthology.orchestra.run.smithi156.stdout:[mds.cephfs.smithi176.dvvbud{-1:44301} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.176:6824/3749220343,v1:172.21.15.176:6825/3749220343] compat {c=[1],r=[1],i=[7ff]}] 2024-07-16T10:03:12.424 INFO:teuthology.orchestra.run.smithi156.stderr:dumped fsmap epoch 36 2024-07-16T10:03:12.424 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:12.422+0000 7f764481b640 1 -- 172.21.15.156:0/2192636445 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f762c06e0b0 msgr2=0x7f762c070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:03:12.424 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:12.422+0000 7f764481b640 1 --2- 172.21.15.156:0/2192636445 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f762c06e0b0 0x7f762c070560 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f7630000c00 tx=0x7f7630035040 comp rx=0 tx=0).stop 2024-07-16T10:03:12.424 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:12.422+0000 7f764481b640 1 -- 172.21.15.156:0/2192636445 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f76400fc510 msgr2=0x7f76401121d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:03:12.424 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:12.422+0000 7f764481b640 1 --2- 172.21.15.156:0/2192636445 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f76400fc510 0x7f76401121d0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f7628002a50 tx=0x7f7628004600 comp rx=0 tx=0).stop 2024-07-16T10:03:12.424 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:12.422+0000 7f764481b640 1 -- 172.21.15.156:0/2192636445 shutdown_connections 2024-07-16T10:03:12.424 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:12.422+0000 7f764481b640 1 --2- 172.21.15.156:0/2192636445 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f762c06e0b0 0x7f762c070560 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:03:12.425 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:12.422+0000 7f764481b640 1 --2- 172.21.15.156:0/2192636445 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f76400fc510 0x7f76401121d0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:03:12.425 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:12.422+0000 7f764481b640 1 --2- 172.21.15.156:0/2192636445 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f76400fb240 0x7f764010fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:03:12.425 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:12.422+0000 7f764481b640 1 -- 172.21.15.156:0/2192636445 >> 172.21.15.156:0/2192636445 conn(0x7f76400f6a20 msgr2=0x7f76400f8110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T10:03:12.425 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:12.422+0000 7f764481b640 1 -- 172.21.15.156:0/2192636445 shutdown_connections 2024-07-16T10:03:12.425 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:12.422+0000 7f764481b640 1 -- 172.21.15.156:0/2192636445 wait complete. 2024-07-16T10:03:12.507 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:12.504+0000 7f4bf8c65640 1 -- 172.21.15.156:0/1215635619 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4bf40fc510 msgr2=0x7f4bf40fc980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:03:12.508 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:12.504+0000 7f4bf8c65640 1 --2- 172.21.15.156:0/1215635619 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4bf40fc510 0x7f4bf40fc980 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f4bdc0048d0 tx=0x7f4bdc009da0 comp rx=0 tx=0).stop 2024-07-16T10:03:12.508 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:12.505+0000 7f4bf8c65640 1 -- 172.21.15.156:0/1215635619 shutdown_connections 2024-07-16T10:03:12.508 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:12.505+0000 7f4bf8c65640 1 --2- 172.21.15.156:0/1215635619 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4bf40fc510 0x7f4bf40fc980 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:03:12.508 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:12.505+0000 7f4bf8c65640 1 --2- 172.21.15.156:0/1215635619 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4bf40fb240 0x7f4bf40fb650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:03:12.508 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:12.505+0000 7f4bf8c65640 1 -- 172.21.15.156:0/1215635619 >> 172.21.15.156:0/1215635619 conn(0x7f4bf40f6a20 msgr2=0x7f4bf40f8e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T10:03:12.508 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:12.505+0000 7f4bf8c65640 1 -- 172.21.15.156:0/1215635619 shutdown_connections 2024-07-16T10:03:12.508 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:12.505+0000 7f4bf8c65640 1 -- 172.21.15.156:0/1215635619 wait complete. 2024-07-16T10:03:12.508 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:12.505+0000 7f4bf8c65640 1 Processor -- start 2024-07-16T10:03:12.508 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:12.505+0000 7f4bf8c65640 1 -- start start 2024-07-16T10:03:12.508 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:12.505+0000 7f4bf8c65640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4bf40fb240 0x7f4bf410cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:03:12.508 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:12.505+0000 7f4bf8c65640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4bf40fc510 0x7f4bf410d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:03:12.509 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:12.505+0000 7f4bf8c65640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bf410db30 con 0x7f4bf40fc510 2024-07-16T10:03:12.509 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:12.505+0000 7f4bf8c65640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bf410e350 con 0x7f4bf40fb240 2024-07-16T10:03:12.509 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:12.505+0000 7f4bebfff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4bf40fc510 0x7f4bf410d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:03:12.509 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:12.506+0000 7f4bebfff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4bf40fc510 0x7f4bf410d4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:35896/0 (socket says 172.21.15.156:35896) 2024-07-16T10:03:12.509 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:12.506+0000 7f4bebfff640 1 -- 172.21.15.156:0/2217538274 learned_addr learned my addr 172.21.15.156:0/2217538274 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T10:03:12.509 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:12.506+0000 7f4bf259c640 1 --2- 172.21.15.156:0/2217538274 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4bf40fb240 0x7f4bf410cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:03:12.509 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:12.506+0000 7f4bebfff640 1 -- 172.21.15.156:0/2217538274 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4bf40fb240 msgr2=0x7f4bf410cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:03:12.509 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:12.506+0000 7f4bebfff640 1 --2- 172.21.15.156:0/2217538274 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4bf40fb240 0x7f4bf410cfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:03:12.509 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:12.506+0000 7f4bebfff640 1 -- 172.21.15.156:0/2217538274 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bdc005040 con 0x7f4bf40fc510 2024-07-16T10:03:12.509 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:12.506+0000 7f4bebfff640 1 --2- 172.21.15.156:0/2217538274 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4bf40fc510 0x7f4bf410d4e0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f4bdc009f80 tx=0x7f4bdc009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T10:03:12.509 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:12.506+0000 7f4beaffd640 1 -- 172.21.15.156:0/2217538274 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4bdc038070 con 0x7f4bf40fc510 2024-07-16T10:03:12.509 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:12.506+0000 7f4beaffd640 1 -- 172.21.15.156:0/2217538274 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4bdc032970 con 0x7f4bf40fc510 2024-07-16T10:03:12.509 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:12.506+0000 7f4beaffd640 1 -- 172.21.15.156:0/2217538274 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4bdc033860 con 0x7f4bf40fc510 2024-07-16T10:03:12.510 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:12.506+0000 7f4bf8c65640 1 -- 172.21.15.156:0/2217538274 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4bf410e580 con 0x7f4bf40fc510 2024-07-16T10:03:12.510 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:12.506+0000 7f4bf8c65640 1 -- 172.21.15.156:0/2217538274 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4bf410eac0 con 0x7f4bf40fc510 2024-07-16T10:03:12.511 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:12.507+0000 7f4bf8c65640 1 -- 172.21.15.156:0/2217538274 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4bf40fa6a0 con 0x7f4bf40fc510 2024-07-16T10:03:12.511 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:12.508+0000 7f4beaffd640 1 -- 172.21.15.156:0/2217538274 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 38) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f4bdc030020 con 0x7f4bf40fc510 2024-07-16T10:03:12.511 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:12.508+0000 7f4beaffd640 1 --2- 172.21.15.156:0/2217538274 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f4be006e0b0 0x7f4be0070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:03:12.511 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:12.508+0000 7f4beaffd640 1 -- 172.21.15.156:0/2217538274 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f4bdc0adb60 con 0x7f4bf40fc510 2024-07-16T10:03:12.511 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:12.508+0000 7f4bf259c640 1 --2- 172.21.15.156:0/2217538274 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f4be006e0b0 0x7f4be0070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:03:12.511 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:12.509+0000 7f4bf259c640 1 --2- 172.21.15.156:0/2217538274 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f4be006e0b0 0x7f4be0070560 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f4be4007fd0 tx=0x7f4be4009e90 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T10:03:12.514 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:12.513+0000 7f4beaffd640 1 -- 172.21.15.156:0/2217538274 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187716 (secure 0 0 0) 0x7f4bdc076e30 con 0x7f4bf40fc510 2024-07-16T10:03:12.709 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:12.708+0000 7f4bf8c65640 1 -- 172.21.15.156:0/2217538274 --> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f4bf40008d0 con 0x7f4be006e0b0 2024-07-16T10:03:12.710 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:12.709+0000 7f4beaffd640 1 -- 172.21.15.156:0/2217538274 <== mgr.34104 v2:172.21.15.156:6800/3367765292 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f4bf40008d0 con 0x7f4be006e0b0 2024-07-16T10:03:12.711 INFO:teuthology.orchestra.run.smithi156.stdout:{ 2024-07-16T10:03:12.711 INFO:teuthology.orchestra.run.smithi156.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:c15ec3aa8e6c0a9c12ee24ff79f3e7dd4d01b3d1a702dfeb7d7fff9b0811a393", 2024-07-16T10:03:12.711 INFO:teuthology.orchestra.run.smithi156.stdout: "in_progress": true, 2024-07-16T10:03:12.711 INFO:teuthology.orchestra.run.smithi156.stdout: "which": "Upgrading all daemon types on all hosts", 2024-07-16T10:03:12.711 INFO:teuthology.orchestra.run.smithi156.stdout: "services_complete": [ 2024-07-16T10:03:12.711 INFO:teuthology.orchestra.run.smithi156.stdout: "mon", 2024-07-16T10:03:12.711 INFO:teuthology.orchestra.run.smithi156.stdout: "mgr", 2024-07-16T10:03:12.711 INFO:teuthology.orchestra.run.smithi156.stdout: "crash", 2024-07-16T10:03:12.711 INFO:teuthology.orchestra.run.smithi156.stdout: "osd" 2024-07-16T10:03:12.711 INFO:teuthology.orchestra.run.smithi156.stdout: ], 2024-07-16T10:03:12.711 INFO:teuthology.orchestra.run.smithi156.stdout: "progress": "13/21 daemons upgraded", 2024-07-16T10:03:12.711 INFO:teuthology.orchestra.run.smithi156.stdout: "message": "Currently upgrading mds daemons", 2024-07-16T10:03:12.712 INFO:teuthology.orchestra.run.smithi156.stdout: "is_paused": false 2024-07-16T10:03:12.712 INFO:teuthology.orchestra.run.smithi156.stdout:} 2024-07-16T10:03:12.713 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:12.711+0000 7f4bf8c65640 1 -- 172.21.15.156:0/2217538274 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f4be006e0b0 msgr2=0x7f4be0070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:03:12.713 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:12.711+0000 7f4bf8c65640 1 --2- 172.21.15.156:0/2217538274 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f4be006e0b0 0x7f4be0070560 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f4be4007fd0 tx=0x7f4be4009e90 comp rx=0 tx=0).stop 2024-07-16T10:03:12.713 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:12.711+0000 7f4bf8c65640 1 -- 172.21.15.156:0/2217538274 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4bf40fc510 msgr2=0x7f4bf410d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:03:12.713 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:12.711+0000 7f4bf8c65640 1 --2- 172.21.15.156:0/2217538274 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4bf40fc510 0x7f4bf410d4e0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f4bdc009f80 tx=0x7f4bdc009da0 comp rx=0 tx=0).stop 2024-07-16T10:03:12.713 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:12.711+0000 7f4bf8c65640 1 -- 172.21.15.156:0/2217538274 shutdown_connections 2024-07-16T10:03:12.713 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:12.711+0000 7f4bf8c65640 1 --2- 172.21.15.156:0/2217538274 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f4be006e0b0 0x7f4be0070560 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:03:12.713 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:12.711+0000 7f4bf8c65640 1 --2- 172.21.15.156:0/2217538274 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4bf40fc510 0x7f4bf410d4e0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:03:12.713 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:12.711+0000 7f4bf8c65640 1 --2- 172.21.15.156:0/2217538274 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4bf40fb240 0x7f4bf410cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:03:12.713 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:12.711+0000 7f4bf8c65640 1 -- 172.21.15.156:0/2217538274 >> 172.21.15.156:0/2217538274 conn(0x7f4bf40f6a20 msgr2=0x7f4bf40f8120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T10:03:12.713 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:12.712+0000 7f4bf8c65640 1 -- 172.21.15.156:0/2217538274 shutdown_connections 2024-07-16T10:03:12.714 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:12.712+0000 7f4bf8c65640 1 -- 172.21.15.156:0/2217538274 wait complete. 2024-07-16T10:03:12.793 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:12.790+0000 7f1edcf4e640 1 -- 172.21.15.156:0/1335780491 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1ed80fc360 msgr2=0x7f1ed80fc770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:03:12.793 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:12.790+0000 7f1edcf4e640 1 --2- 172.21.15.156:0/1335780491 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1ed80fc360 0x7f1ed80fc770 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f1ec80048d0 tx=0x7f1ec8009da0 comp rx=0 tx=0).stop 2024-07-16T10:03:12.793 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:12.790+0000 7f1edcf4e640 1 -- 172.21.15.156:0/1335780491 shutdown_connections 2024-07-16T10:03:12.794 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:12.790+0000 7f1edcf4e640 1 --2- 172.21.15.156:0/1335780491 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1ed80fd630 0x7f1ed80fdaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:03:12.794 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:12.790+0000 7f1edcf4e640 1 --2- 172.21.15.156:0/1335780491 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1ed80fc360 0x7f1ed80fc770 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:03:12.794 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:12.790+0000 7f1edcf4e640 1 -- 172.21.15.156:0/1335780491 >> 172.21.15.156:0/1335780491 conn(0x7f1ed80f7b40 msgr2=0x7f1ed80f9f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T10:03:12.794 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:12.790+0000 7f1edcf4e640 1 -- 172.21.15.156:0/1335780491 shutdown_connections 2024-07-16T10:03:12.794 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:12.790+0000 7f1edcf4e640 1 -- 172.21.15.156:0/1335780491 wait complete. 2024-07-16T10:03:12.794 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:12.790+0000 7f1edcf4e640 1 Processor -- start 2024-07-16T10:03:12.794 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:12.791+0000 7f1edcf4e640 1 -- start start 2024-07-16T10:03:12.794 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:12.791+0000 7f1edcf4e640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1ed80fc360 0x7f1ed810cff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:03:12.794 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:12.791+0000 7f1edcf4e640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1ed80fd630 0x7f1ed810d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:03:12.794 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:12.791+0000 7f1edcf4e640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ed810db80 con 0x7f1ed80fc360 2024-07-16T10:03:12.794 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:12.791+0000 7f1edcf4e640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ed810e350 con 0x7f1ed80fd630 2024-07-16T10:03:12.794 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:12.791+0000 7f1ed659c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1ed80fc360 0x7f1ed810cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:03:12.795 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:12.791+0000 7f1ed659c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1ed80fc360 0x7f1ed810cff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:39422/0 (socket says 172.21.15.156:39422) 2024-07-16T10:03:12.795 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:12.791+0000 7f1ed659c640 1 -- 172.21.15.156:0/2683861308 learned_addr learned my addr 172.21.15.156:0/2683861308 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T10:03:12.795 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:12.791+0000 7f1ecffff640 1 --2- 172.21.15.156:0/2683861308 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1ed80fd630 0x7f1ed810d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:03:12.795 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:12.791+0000 7f1ed659c640 1 -- 172.21.15.156:0/2683861308 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1ed80fd630 msgr2=0x7f1ed810d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:03:12.795 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:12.791+0000 7f1ed659c640 1 --2- 172.21.15.156:0/2683861308 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1ed80fd630 0x7f1ed810d530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:03:12.795 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:12.791+0000 7f1ed659c640 1 -- 172.21.15.156:0/2683861308 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ec8005040 con 0x7f1ed80fc360 2024-07-16T10:03:12.795 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:12.791+0000 7f1ecffff640 1 --2- 172.21.15.156:0/2683861308 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1ed80fd630 0x7f1ed810d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-16T10:03:12.795 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:12.791+0000 7f1ed659c640 1 --2- 172.21.15.156:0/2683861308 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1ed80fc360 0x7f1ed810cff0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f1ec8009f80 tx=0x7f1ec8009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T10:03:12.796 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:12.792+0000 7f1eceffd640 1 -- 172.21.15.156:0/2683861308 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1ec8038070 con 0x7f1ed80fc360 2024-07-16T10:03:12.796 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:12.792+0000 7f1eceffd640 1 -- 172.21.15.156:0/2683861308 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1ec8032970 con 0x7f1ed80fc360 2024-07-16T10:03:12.796 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:12.792+0000 7f1edcf4e640 1 -- 172.21.15.156:0/2683861308 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ed810e580 con 0x7f1ed80fc360 2024-07-16T10:03:12.796 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:12.792+0000 7f1eceffd640 1 -- 172.21.15.156:0/2683861308 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1ec8033860 con 0x7f1ed80fc360 2024-07-16T10:03:12.796 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:12.792+0000 7f1edcf4e640 1 -- 172.21.15.156:0/2683861308 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ed810eac0 con 0x7f1ed80fc360 2024-07-16T10:03:12.798 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:12.793+0000 7f1edcf4e640 1 -- 172.21.15.156:0/2683861308 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ed80fb7c0 con 0x7f1ed80fc360 2024-07-16T10:03:12.798 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:12.793+0000 7f1eceffd640 1 -- 172.21.15.156:0/2683861308 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 38) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f1ec8030020 con 0x7f1ed80fc360 2024-07-16T10:03:12.798 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:12.794+0000 7f1eceffd640 1 --2- 172.21.15.156:0/2683861308 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f1ec406e0b0 0x7f1ec4070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:03:12.798 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:12.794+0000 7f1eceffd640 1 -- 172.21.15.156:0/2683861308 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f1ec80ada50 con 0x7f1ed80fc360 2024-07-16T10:03:12.798 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:12.794+0000 7f1ecffff640 1 --2- 172.21.15.156:0/2683861308 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f1ec406e0b0 0x7f1ec4070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:03:12.799 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:12.794+0000 7f1ecffff640 1 --2- 172.21.15.156:0/2683861308 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f1ec406e0b0 0x7f1ec4070560 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f1ec0007fd0 tx=0x7f1ec0009e90 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T10:03:12.799 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:12.798+0000 7f1eceffd640 1 -- 172.21.15.156:0/2683861308 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187716 (secure 0 0 0) 0x7f1ec8076da0 con 0x7f1ed80fc360 2024-07-16T10:03:13.054 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:13.052+0000 7f1edcf4e640 1 -- 172.21.15.156:0/2683861308 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f1ed804b530 con 0x7f1ed80fc360 2024-07-16T10:03:13.054 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:13.053+0000 7f1eceffd640 1 -- 172.21.15.156:0/2683861308 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7f1ec8041020 con 0x7f1ed80fc360 2024-07-16T10:03:13.054 INFO:teuthology.orchestra.run.smithi156.stdout:HEALTH_OK 2024-07-16T10:03:13.056 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:13.055+0000 7f1edcf4e640 1 -- 172.21.15.156:0/2683861308 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f1ec406e0b0 msgr2=0x7f1ec4070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:03:13.056 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:13.055+0000 7f1edcf4e640 1 --2- 172.21.15.156:0/2683861308 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f1ec406e0b0 0x7f1ec4070560 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f1ec0007fd0 tx=0x7f1ec0009e90 comp rx=0 tx=0).stop 2024-07-16T10:03:13.056 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:13.055+0000 7f1edcf4e640 1 -- 172.21.15.156:0/2683861308 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1ed80fc360 msgr2=0x7f1ed810cff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:03:13.056 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:13.055+0000 7f1edcf4e640 1 --2- 172.21.15.156:0/2683861308 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1ed80fc360 0x7f1ed810cff0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f1ec8009f80 tx=0x7f1ec8009da0 comp rx=0 tx=0).stop 2024-07-16T10:03:13.056 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:13.055+0000 7f1edcf4e640 1 -- 172.21.15.156:0/2683861308 shutdown_connections 2024-07-16T10:03:13.057 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:13.055+0000 7f1edcf4e640 1 --2- 172.21.15.156:0/2683861308 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f1ec406e0b0 0x7f1ec4070560 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:03:13.057 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:13.055+0000 7f1edcf4e640 1 --2- 172.21.15.156:0/2683861308 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1ed80fd630 0x7f1ed810d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:03:13.057 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:13.055+0000 7f1edcf4e640 1 --2- 172.21.15.156:0/2683861308 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1ed80fc360 0x7f1ed810cff0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:03:13.057 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:13.055+0000 7f1edcf4e640 1 -- 172.21.15.156:0/2683861308 >> 172.21.15.156:0/2683861308 conn(0x7f1ed80f7b40 msgr2=0x7f1ed80f9220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T10:03:13.057 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:13.055+0000 7f1edcf4e640 1 -- 172.21.15.156:0/2683861308 shutdown_connections 2024-07-16T10:03:13.057 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:13.055+0000 7f1edcf4e640 1 -- 172.21.15.156:0/2683861308 wait complete. 2024-07-16T10:03:13.322 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:03:13 smithi156 ceph-mon[93066]: from='client.34446 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-16T10:03:13.322 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:03:13 smithi156 ceph-mon[93066]: from='client.34450 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-07-16T10:03:13.322 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:03:13 smithi156 ceph-mon[93066]: from='client.? 172.21.15.156:0/2891660215' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:03:13.322 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:03:13 smithi156 ceph-mon[93066]: from='client.? 172.21.15.156:0/2192636445' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-07-16T10:03:13.343 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:03:13 smithi176 ceph-mon[81689]: from='client.34446 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-16T10:03:13.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:03:13 smithi176 ceph-mon[81689]: from='client.34450 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-07-16T10:03:13.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:03:13 smithi176 ceph-mon[81689]: from='client.? 172.21.15.156:0/2891660215' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:03:13.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:03:13 smithi176 ceph-mon[81689]: from='client.? 172.21.15.156:0/2192636445' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-07-16T10:03:14.343 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:03:14 smithi176 ceph-mon[81689]: pgmap v198: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 14 MiB/s rd, 102 B/s wr, 5 op/s 2024-07-16T10:03:14.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:03:14 smithi176 ceph-mon[81689]: from='client.34462 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-16T10:03:14.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:03:14 smithi176 ceph-mon[81689]: from='client.? 172.21.15.156:0/2683861308' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-07-16T10:03:14.378 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:03:14 smithi156 ceph-mon[93066]: pgmap v198: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 14 MiB/s rd, 102 B/s wr, 5 op/s 2024-07-16T10:03:14.378 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:03:14 smithi156 ceph-mon[93066]: from='client.34462 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-16T10:03:14.378 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:03:14 smithi156 ceph-mon[93066]: from='client.? 172.21.15.156:0/2683861308' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-07-16T10:03:16.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:03:16 smithi156 ceph-mon[93066]: pgmap v199: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 13 MiB/s rd, 92 B/s wr, 6 op/s 2024-07-16T10:03:16.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:03:16 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:03:16.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:03:16 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:03:16.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:03:16 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-16T10:03:16.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:03:16 smithi156 ceph-mon[93066]: mds.? [v2:172.21.15.156:6828/1683974145,v1:172.21.15.156:6829/1683974145] up:boot 2024-07-16T10:03:16.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:03:16 smithi156 ceph-mon[93066]: fsmap cephfs:1 {0=cephfs.smithi156.rdibys=up:active} 3 up:standby 2024-07-16T10:03:16.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:03:16 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi156.evwwaz"}]: dispatch 2024-07-16T10:03:16.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:03:16 smithi176 ceph-mon[81689]: pgmap v199: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 13 MiB/s rd, 92 B/s wr, 6 op/s 2024-07-16T10:03:16.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:03:16 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:03:16.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:03:16 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:03:16.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:03:16 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-16T10:03:16.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:03:16 smithi176 ceph-mon[81689]: mds.? [v2:172.21.15.156:6828/1683974145,v1:172.21.15.156:6829/1683974145] up:boot 2024-07-16T10:03:16.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:03:16 smithi176 ceph-mon[81689]: fsmap cephfs:1 {0=cephfs.smithi156.rdibys=up:active} 3 up:standby 2024-07-16T10:03:16.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:03:16 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi156.evwwaz"}]: dispatch 2024-07-16T10:03:18.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:03:18 smithi156 ceph-mon[93066]: pgmap v200: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 11 MiB/s rd, 4.3 KiB/s wr, 8 op/s 2024-07-16T10:03:18.843 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:03:18 smithi176 ceph-mon[81689]: pgmap v200: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 11 MiB/s rd, 4.3 KiB/s wr, 8 op/s 2024-07-16T10:03:19.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:03:19 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:03:19.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:03:19 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-16T10:03:19.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:03:19 smithi156 ceph-mon[93066]: pgmap v201: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 6.9 MiB/s rd, 4.3 KiB/s wr, 7 op/s 2024-07-16T10:03:19.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:03:19 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:03:19.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:03:19 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-16T10:03:19.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:03:19 smithi176 ceph-mon[81689]: pgmap v201: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 6.9 MiB/s rd, 4.3 KiB/s wr, 7 op/s 2024-07-16T10:03:21.055 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:03:20 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:03:21.056 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:03:20 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:03:21.056 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:03:20 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:03:21.056 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:03:20 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:03:21.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:03:20 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:03:21.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:03:20 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:03:21.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:03:20 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:03:21.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:03:20 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:03:22.034 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:03:21 smithi156 ceph-mon[93066]: pgmap v202: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 3.9 MiB/s rd, 4.3 KiB/s wr, 6 op/s 2024-07-16T10:03:22.093 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:03:21 smithi176 ceph-mon[81689]: pgmap v202: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 3.9 MiB/s rd, 4.3 KiB/s wr, 6 op/s 2024-07-16T10:03:24.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:03:23 smithi176 ceph-mon[81689]: pgmap v203: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 582 KiB/s rd, 4.3 KiB/s wr, 5 op/s 2024-07-16T10:03:24.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:03:23 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:03:24.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:03:23 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:03:24.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:03:23 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-16T10:03:24.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:03:23 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-16T10:03:24.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:03:23 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:03:24.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:03:23 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-16T10:03:24.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:03:23 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:03:24.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:03:23 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:03:24.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:03:23 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:03:24.095 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:03:23 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:03:24.095 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:03:23 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "mds ok-to-stop", "ids": ["cephfs.smithi176.dvvbud"]}]: dispatch 2024-07-16T10:03:24.174 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:03:23 smithi156 ceph-mon[93066]: pgmap v203: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 582 KiB/s rd, 4.3 KiB/s wr, 5 op/s 2024-07-16T10:03:24.175 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:03:23 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:03:24.175 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:03:23 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:03:24.175 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:03:23 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-16T10:03:24.175 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:03:23 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-16T10:03:24.175 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:03:23 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:03:24.175 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:03:23 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-16T10:03:24.175 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:03:23 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:03:24.175 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:03:23 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:03:24.175 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:03:23 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:03:24.175 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:03:23 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:03:24.176 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:03:23 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "mds ok-to-stop", "ids": ["cephfs.smithi176.dvvbud"]}]: dispatch 2024-07-16T10:03:25.093 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:03:24 smithi176 ceph-mon[81689]: Upgrade: It appears safe to stop mds.cephfs.smithi176.dvvbud 2024-07-16T10:03:25.174 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:03:24 smithi156 ceph-mon[93066]: Upgrade: It appears safe to stop mds.cephfs.smithi176.dvvbud 2024-07-16T10:03:26.343 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:03:26 smithi176 ceph-mon[81689]: pgmap v204: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4.2 KiB/s wr, 4 op/s 2024-07-16T10:03:26.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:03:26 smithi176 ceph-mon[81689]: Upgrade: Updating mds.cephfs.smithi176.dvvbud 2024-07-16T10:03:26.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:03:26 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:03:26.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:03:26 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi176.dvvbud", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-16T10:03:26.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:03:26 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-16T10:03:26.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:03:26 smithi176 ceph-mon[81689]: Deploying daemon mds.cephfs.smithi176.dvvbud on smithi176 2024-07-16T10:03:26.424 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:03:26 smithi156 ceph-mon[93066]: pgmap v204: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4.2 KiB/s wr, 4 op/s 2024-07-16T10:03:26.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:03:26 smithi156 ceph-mon[93066]: Upgrade: Updating mds.cephfs.smithi176.dvvbud 2024-07-16T10:03:26.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:03:26 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:03:26.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:03:26 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi176.dvvbud", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-16T10:03:26.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:03:26 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-16T10:03:26.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:03:26 smithi156 ceph-mon[93066]: Deploying daemon mds.cephfs.smithi176.dvvbud on smithi176 2024-07-16T10:03:28.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:03:28 smithi176 ceph-mon[81689]: pgmap v205: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 4.2 KiB/s wr, 3 op/s 2024-07-16T10:03:28.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:03:28 smithi176 ceph-mon[81689]: osdmap e83: 6 total, 6 up, 6 in 2024-07-16T10:03:28.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:03:28 smithi176 ceph-mon[81689]: fsmap cephfs:1 {0=cephfs.smithi156.rdibys=up:active} 2 up:standby 2024-07-16T10:03:28.424 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:03:28 smithi156 ceph-mon[93066]: pgmap v205: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 4.2 KiB/s wr, 3 op/s 2024-07-16T10:03:28.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:03:28 smithi156 ceph-mon[93066]: osdmap e83: 6 total, 6 up, 6 in 2024-07-16T10:03:28.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:03:28 smithi156 ceph-mon[93066]: fsmap cephfs:1 {0=cephfs.smithi156.rdibys=up:active} 2 up:standby 2024-07-16T10:03:29.343 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:03:29 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi176.dvvbud"}]: dispatch 2024-07-16T10:03:29.424 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:03:29 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi176.dvvbud"}]: dispatch 2024-07-16T10:03:30.343 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:03:30 smithi176 ceph-mon[81689]: pgmap v207: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail 2024-07-16T10:03:30.424 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:03:30 smithi156 ceph-mon[93066]: pgmap v207: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail 2024-07-16T10:03:32.260 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:03:32 smithi176 ceph-mon[81689]: pgmap v208: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail 2024-07-16T10:03:32.314 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:03:32 smithi156 ceph-mon[93066]: pgmap v208: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail 2024-07-16T10:03:33.890 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:03:33 smithi176 ceph-mon[81689]: pgmap v209: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail 2024-07-16T10:03:33.891 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:03:33 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:03:33.891 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:03:33 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:03:33.891 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:03:33 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-16T10:03:33.891 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:03:33 smithi176 ceph-mon[81689]: mds.? [v2:172.21.15.176:6824/1700587296,v1:172.21.15.176:6825/1700587296] up:boot 2024-07-16T10:03:33.891 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:03:33 smithi176 ceph-mon[81689]: fsmap cephfs:1 {0=cephfs.smithi156.rdibys=up:active} 3 up:standby 2024-07-16T10:03:33.891 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:03:33 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi176.dvvbud"}]: dispatch 2024-07-16T10:03:33.891 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:03:33 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-16T10:03:33.924 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:03:33 smithi156 ceph-mon[93066]: pgmap v209: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail 2024-07-16T10:03:33.924 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:03:33 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:03:33.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:03:33 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:03:33.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:03:33 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-16T10:03:33.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:03:33 smithi156 ceph-mon[93066]: mds.? [v2:172.21.15.176:6824/1700587296,v1:172.21.15.176:6825/1700587296] up:boot 2024-07-16T10:03:33.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:03:33 smithi156 ceph-mon[93066]: fsmap cephfs:1 {0=cephfs.smithi156.rdibys=up:active} 3 up:standby 2024-07-16T10:03:33.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:03:33 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi176.dvvbud"}]: dispatch 2024-07-16T10:03:33.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:03:33 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-16T10:03:36.190 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:03:36 smithi176 ceph-mon[81689]: pgmap v210: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail 2024-07-16T10:03:36.190 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:03:36 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:03:36.190 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:03:36 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:03:36.190 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:03:36 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:03:36.190 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:03:36 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:03:36.424 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:03:36 smithi156 ceph-mon[93066]: pgmap v210: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail 2024-07-16T10:03:36.424 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:03:36 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:03:36.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:03:36 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:03:36.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:03:36 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:03:36.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:03:36 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:03:38.343 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:03:38 smithi176 ceph-mon[81689]: pgmap v211: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail 2024-07-16T10:03:38.424 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:03:38 smithi156 ceph-mon[93066]: pgmap v211: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail 2024-07-16T10:03:39.843 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:03:39 smithi176 ceph-mon[81689]: pgmap v212: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail 2024-07-16T10:03:39.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:03:39 smithi176 ceph-mon[81689]: Detected new or changed devices on smithi176 2024-07-16T10:03:39.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:03:39 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:03:39.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:03:39 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:03:39.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:03:39 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-16T10:03:39.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:03:39 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-16T10:03:39.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:03:39 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:03:39.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:03:39 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-16T10:03:39.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:03:39 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:03:39.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:03:39 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:03:39.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:03:39 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:03:39.845 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:03:39 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:03:39.845 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:03:39 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "mds ok-to-stop", "ids": ["cephfs.smithi176.wahbsi"]}]: dispatch 2024-07-16T10:03:39.845 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:03:39 smithi176 ceph-mon[81689]: Upgrade: It appears safe to stop mds.cephfs.smithi176.wahbsi 2024-07-16T10:03:40.175 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:03:39 smithi156 ceph-mon[93066]: pgmap v212: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail 2024-07-16T10:03:40.175 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:03:39 smithi156 ceph-mon[93066]: Detected new or changed devices on smithi176 2024-07-16T10:03:40.175 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:03:39 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:03:40.175 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:03:39 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:03:40.175 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:03:39 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-16T10:03:40.175 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:03:39 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-16T10:03:40.175 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:03:39 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:03:40.175 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:03:39 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-16T10:03:40.176 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:03:39 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:03:40.176 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:03:39 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:03:40.176 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:03:39 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:03:40.176 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:03:39 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:03:40.176 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:03:39 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "mds ok-to-stop", "ids": ["cephfs.smithi176.wahbsi"]}]: dispatch 2024-07-16T10:03:40.176 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:03:39 smithi156 ceph-mon[93066]: Upgrade: It appears safe to stop mds.cephfs.smithi176.wahbsi 2024-07-16T10:03:41.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:03:41 smithi176 ceph-mon[81689]: Upgrade: Updating mds.cephfs.smithi176.wahbsi 2024-07-16T10:03:41.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:03:41 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:03:41.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:03:41 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi176.wahbsi", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-16T10:03:41.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:03:41 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-16T10:03:41.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:03:41 smithi176 ceph-mon[81689]: Deploying daemon mds.cephfs.smithi176.wahbsi on smithi176 2024-07-16T10:03:41.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:03:41 smithi156 ceph-mon[93066]: Upgrade: Updating mds.cephfs.smithi176.wahbsi 2024-07-16T10:03:41.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:03:41 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:03:41.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:03:41 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi176.wahbsi", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-16T10:03:41.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:03:41 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-16T10:03:41.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:03:41 smithi156 ceph-mon[93066]: Deploying daemon mds.cephfs.smithi176.wahbsi on smithi176 2024-07-16T10:03:42.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:03:42 smithi176 ceph-mon[81689]: pgmap v213: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail 2024-07-16T10:03:42.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:03:42 smithi156 ceph-mon[93066]: pgmap v213: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail 2024-07-16T10:03:43.151 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:43.146+0000 7fb29ba94640 1 -- 172.21.15.156:0/3017898954 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb294066460 msgr2=0x7fb2940ff030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:03:43.151 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:43.146+0000 7fb29ba94640 1 --2- 172.21.15.156:0/3017898954 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb294066460 0x7fb2940ff030 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fb2900048d0 tx=0x7fb290009da0 comp rx=0 tx=0).stop 2024-07-16T10:03:43.151 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:43.147+0000 7fb29ba94640 1 -- 172.21.15.156:0/3017898954 shutdown_connections 2024-07-16T10:03:43.151 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:43.147+0000 7fb29ba94640 1 --2- 172.21.15.156:0/3017898954 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb2940ff570 0x7fb294101950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:03:43.151 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:43.147+0000 7fb29ba94640 1 --2- 172.21.15.156:0/3017898954 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb294066460 0x7fb2940ff030 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:03:43.151 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:43.147+0000 7fb29ba94640 1 -- 172.21.15.156:0/3017898954 >> 172.21.15.156:0/3017898954 conn(0x7fb2940f6a20 msgr2=0x7fb2940f8e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T10:03:43.151 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:43.147+0000 7fb29ba94640 1 -- 172.21.15.156:0/3017898954 shutdown_connections 2024-07-16T10:03:43.151 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:43.147+0000 7fb29ba94640 1 -- 172.21.15.156:0/3017898954 wait complete. 2024-07-16T10:03:43.151 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:43.147+0000 7fb29ba94640 1 Processor -- start 2024-07-16T10:03:43.152 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:43.147+0000 7fb29ba94640 1 -- start start 2024-07-16T10:03:43.152 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:43.147+0000 7fb29ba94640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb294066460 0x7fb29410d030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:03:43.152 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:43.147+0000 7fb29ba94640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb2940ff570 0x7fb29410d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:03:43.152 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:43.147+0000 7fb29ba94640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb29410dbc0 con 0x7fb294066460 2024-07-16T10:03:43.152 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:43.147+0000 7fb29ba94640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb29410e3e0 con 0x7fb2940ff570 2024-07-16T10:03:43.152 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:43.148+0000 7fb299830640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb294066460 0x7fb29410d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:03:43.152 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:43.148+0000 7fb299830640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb294066460 0x7fb29410d030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:56030/0 (socket says 172.21.15.156:56030) 2024-07-16T10:03:43.152 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:43.148+0000 7fb299830640 1 -- 172.21.15.156:0/4108320385 learned_addr learned my addr 172.21.15.156:0/4108320385 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T10:03:43.152 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:43.148+0000 7fb29902f640 1 --2- 172.21.15.156:0/4108320385 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb2940ff570 0x7fb29410d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:03:43.152 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:43.148+0000 7fb299830640 1 -- 172.21.15.156:0/4108320385 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb2940ff570 msgr2=0x7fb29410d570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:03:43.152 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:43.148+0000 7fb299830640 1 --2- 172.21.15.156:0/4108320385 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb2940ff570 0x7fb29410d570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:03:43.152 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:43.148+0000 7fb299830640 1 -- 172.21.15.156:0/4108320385 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb290005040 con 0x7fb294066460 2024-07-16T10:03:43.153 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:43.148+0000 7fb299830640 1 --2- 172.21.15.156:0/4108320385 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb294066460 0x7fb29410d030 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fb290000c00 tx=0x7fb290009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T10:03:43.153 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:43.148+0000 7fb28a7fc640 1 -- 172.21.15.156:0/4108320385 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb290038070 con 0x7fb294066460 2024-07-16T10:03:43.153 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:43.148+0000 7fb28a7fc640 1 -- 172.21.15.156:0/4108320385 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb2900092b0 con 0x7fb294066460 2024-07-16T10:03:43.153 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:43.148+0000 7fb29ba94640 1 -- 172.21.15.156:0/4108320385 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb29410e610 con 0x7fb294066460 2024-07-16T10:03:43.153 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:43.148+0000 7fb28a7fc640 1 -- 172.21.15.156:0/4108320385 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb290033780 con 0x7fb294066460 2024-07-16T10:03:43.153 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:43.148+0000 7fb29ba94640 1 -- 172.21.15.156:0/4108320385 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb29410eb50 con 0x7fb294066460 2024-07-16T10:03:43.153 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:43.149+0000 7fb29ba94640 1 -- 172.21.15.156:0/4108320385 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2940fa6c0 con 0x7fb294066460 2024-07-16T10:03:43.154 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:43.150+0000 7fb28a7fc640 1 -- 172.21.15.156:0/4108320385 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 38) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fb290030020 con 0x7fb294066460 2024-07-16T10:03:43.154 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:43.151+0000 7fb28a7fc640 1 --2- 172.21.15.156:0/4108320385 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7fb28006e0b0 0x7fb280070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:03:43.154 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:43.151+0000 7fb28a7fc640 1 -- 172.21.15.156:0/4108320385 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7fb2900ad940 con 0x7fb294066460 2024-07-16T10:03:43.154 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:43.151+0000 7fb29902f640 1 --2- 172.21.15.156:0/4108320385 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7fb28006e0b0 0x7fb280070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:03:43.154 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:43.151+0000 7fb29902f640 1 --2- 172.21.15.156:0/4108320385 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7fb28006e0b0 0x7fb280070560 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fb29410cde0 tx=0x7fb28400b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T10:03:43.156 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:43.155+0000 7fb28a7fc640 1 -- 172.21.15.156:0/4108320385 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187716 (secure 0 0 0) 0x7fb290076cc0 con 0x7fb294066460 2024-07-16T10:03:43.351 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:43.350+0000 7fb29ba94640 1 -- 172.21.15.156:0/4108320385 --> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb29410ef20 con 0x7fb28006e0b0 2024-07-16T10:03:43.352 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:43.351+0000 7fb28a7fc640 1 -- 172.21.15.156:0/4108320385 <== mgr.34104 v2:172.21.15.156:6800/3367765292 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fb29410ef20 con 0x7fb28006e0b0 2024-07-16T10:03:43.354 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:43.353+0000 7fb29ba94640 1 -- 172.21.15.156:0/4108320385 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7fb28006e0b0 msgr2=0x7fb280070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:03:43.355 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:43.353+0000 7fb29ba94640 1 --2- 172.21.15.156:0/4108320385 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7fb28006e0b0 0x7fb280070560 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fb29410cde0 tx=0x7fb28400b040 comp rx=0 tx=0).stop 2024-07-16T10:03:43.355 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:43.353+0000 7fb29ba94640 1 -- 172.21.15.156:0/4108320385 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb294066460 msgr2=0x7fb29410d030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:03:43.355 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:43.353+0000 7fb29ba94640 1 --2- 172.21.15.156:0/4108320385 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb294066460 0x7fb29410d030 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fb290000c00 tx=0x7fb290009da0 comp rx=0 tx=0).stop 2024-07-16T10:03:43.355 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:43.353+0000 7fb29ba94640 1 -- 172.21.15.156:0/4108320385 shutdown_connections 2024-07-16T10:03:43.355 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:43.353+0000 7fb29ba94640 1 --2- 172.21.15.156:0/4108320385 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7fb28006e0b0 0x7fb280070560 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:03:43.355 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:43.353+0000 7fb29ba94640 1 --2- 172.21.15.156:0/4108320385 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb2940ff570 0x7fb29410d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:03:43.355 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:43.353+0000 7fb29ba94640 1 --2- 172.21.15.156:0/4108320385 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb294066460 0x7fb29410d030 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:03:43.355 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:43.353+0000 7fb29ba94640 1 -- 172.21.15.156:0/4108320385 >> 172.21.15.156:0/4108320385 conn(0x7fb2940f6a20 msgr2=0x7fb2940f6e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T10:03:43.355 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:43.353+0000 7fb29ba94640 1 -- 172.21.15.156:0/4108320385 shutdown_connections 2024-07-16T10:03:43.355 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:43.353+0000 7fb29ba94640 1 -- 172.21.15.156:0/4108320385 wait complete. 2024-07-16T10:03:43.363 INFO:teuthology.orchestra.run.smithi156.stdout:true 2024-07-16T10:03:43.424 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:03:43 smithi156 ceph-mon[93066]: osdmap e84: 6 total, 6 up, 6 in 2024-07-16T10:03:43.424 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:03:43 smithi156 ceph-mon[93066]: fsmap cephfs:1 {0=cephfs.smithi156.rdibys=up:active} 2 up:standby 2024-07-16T10:03:43.436 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:43.435+0000 7f2a9612d640 1 -- 172.21.15.156:0/1017513418 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2a900fc510 msgr2=0x7f2a900fc980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:03:43.436 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:43.435+0000 7f2a9612d640 1 --2- 172.21.15.156:0/1017513418 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2a900fc510 0x7f2a900fc980 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f2a780048d0 tx=0x7f2a78009da0 comp rx=0 tx=0).stop 2024-07-16T10:03:43.436 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:43.435+0000 7f2a9612d640 1 -- 172.21.15.156:0/1017513418 shutdown_connections 2024-07-16T10:03:43.436 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:43.435+0000 7f2a9612d640 1 --2- 172.21.15.156:0/1017513418 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2a900fc510 0x7f2a900fc980 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:03:43.437 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:43.435+0000 7f2a9612d640 1 --2- 172.21.15.156:0/1017513418 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2a900fb240 0x7f2a900fb650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:03:43.437 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:43.435+0000 7f2a9612d640 1 -- 172.21.15.156:0/1017513418 >> 172.21.15.156:0/1017513418 conn(0x7f2a900f6a20 msgr2=0x7f2a900f8e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T10:03:43.437 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:43.435+0000 7f2a9612d640 1 -- 172.21.15.156:0/1017513418 shutdown_connections 2024-07-16T10:03:43.437 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:43.435+0000 7f2a9612d640 1 -- 172.21.15.156:0/1017513418 wait complete. 2024-07-16T10:03:43.439 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:43.435+0000 7f2a9612d640 1 Processor -- start 2024-07-16T10:03:43.439 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:43.435+0000 7f2a9612d640 1 -- start start 2024-07-16T10:03:43.439 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:43.436+0000 7f2a9612d640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2a900fb240 0x7f2a9010fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:03:43.439 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:43.436+0000 7f2a9612d640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2a900fc510 0x7f2a90112210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:03:43.439 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:43.436+0000 7f2a9612d640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a90065a70 con 0x7f2a900fb240 2024-07-16T10:03:43.439 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:43.436+0000 7f2a9612d640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a90065be0 con 0x7f2a900fc510 2024-07-16T10:03:43.439 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:43.436+0000 7f2a8f7fe640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2a900fb240 0x7f2a9010fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:03:43.439 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:43.436+0000 7f2a8f7fe640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2a900fb240 0x7f2a9010fcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:56046/0 (socket says 172.21.15.156:56046) 2024-07-16T10:03:43.439 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:43.436+0000 7f2a8f7fe640 1 -- 172.21.15.156:0/1099570876 learned_addr learned my addr 172.21.15.156:0/1099570876 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T10:03:43.440 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:43.436+0000 7f2a87fff640 1 --2- 172.21.15.156:0/1099570876 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2a900fc510 0x7f2a90112210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:03:43.440 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:43.436+0000 7f2a8f7fe640 1 -- 172.21.15.156:0/1099570876 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2a900fc510 msgr2=0x7f2a90112210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:03:43.440 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:43.436+0000 7f2a8f7fe640 1 --2- 172.21.15.156:0/1099570876 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2a900fc510 0x7f2a90112210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:03:43.440 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:43.436+0000 7f2a8f7fe640 1 -- 172.21.15.156:0/1099570876 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a78005040 con 0x7f2a900fb240 2024-07-16T10:03:43.440 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:43.436+0000 7f2a8f7fe640 1 --2- 172.21.15.156:0/1099570876 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2a900fb240 0x7f2a9010fcc0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f2a80002a80 tx=0x7f2a80004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T10:03:43.440 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:43.436+0000 7f2a8cff9640 1 -- 172.21.15.156:0/1099570876 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2a80007c30 con 0x7f2a900fb240 2024-07-16T10:03:43.440 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:43.436+0000 7f2a9612d640 1 -- 172.21.15.156:0/1099570876 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a901127b0 con 0x7f2a900fb240 2024-07-16T10:03:43.440 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:43.437+0000 7f2a8cff9640 1 -- 172.21.15.156:0/1099570876 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2a8000f040 con 0x7f2a900fb240 2024-07-16T10:03:43.440 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:43.437+0000 7f2a8cff9640 1 -- 172.21.15.156:0/1099570876 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2a8000db70 con 0x7f2a900fb240 2024-07-16T10:03:43.440 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:43.437+0000 7f2a9612d640 1 -- 172.21.15.156:0/1099570876 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a90112ca0 con 0x7f2a900fb240 2024-07-16T10:03:43.440 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:43.438+0000 7f2a9612d640 1 -- 172.21.15.156:0/1099570876 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a900fa6a0 con 0x7f2a900fb240 2024-07-16T10:03:43.441 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:43.438+0000 7f2a8cff9640 1 -- 172.21.15.156:0/1099570876 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 38) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f2a8000dcd0 con 0x7f2a900fb240 2024-07-16T10:03:43.441 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:43.438+0000 7f2a8cff9640 1 --2- 172.21.15.156:0/1099570876 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f2a7c06e0b0 0x7f2a7c070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:03:43.441 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:43.438+0000 7f2a8cff9640 1 -- 172.21.15.156:0/1099570876 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f2a8008d450 con 0x7f2a900fb240 2024-07-16T10:03:43.441 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:43.439+0000 7f2a87fff640 1 --2- 172.21.15.156:0/1099570876 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f2a7c06e0b0 0x7f2a7c070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:03:43.441 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:43.439+0000 7f2a87fff640 1 --2- 172.21.15.156:0/1099570876 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f2a7c06e0b0 0x7f2a7c070560 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f2a78003080 tx=0x7f2a78035040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T10:03:43.444 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:43.443+0000 7f2a8cff9640 1 -- 172.21.15.156:0/1099570876 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187716 (secure 0 0 0) 0x7f2a800567a0 con 0x7f2a900fb240 2024-07-16T10:03:43.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:03:43 smithi176 ceph-mon[81689]: osdmap e84: 6 total, 6 up, 6 in 2024-07-16T10:03:43.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:03:43 smithi176 ceph-mon[81689]: fsmap cephfs:1 {0=cephfs.smithi156.rdibys=up:active} 2 up:standby 2024-07-16T10:03:43.638 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:43.637+0000 7f2a9612d640 1 -- 172.21.15.156:0/1099570876 --> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f2a90100880 con 0x7f2a7c06e0b0 2024-07-16T10:03:43.639 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:43.638+0000 7f2a8cff9640 1 -- 172.21.15.156:0/1099570876 <== mgr.34104 v2:172.21.15.156:6800/3367765292 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f2a90100880 con 0x7f2a7c06e0b0 2024-07-16T10:03:43.641 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:43.640+0000 7f2a9612d640 1 -- 172.21.15.156:0/1099570876 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f2a7c06e0b0 msgr2=0x7f2a7c070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:03:43.641 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:43.640+0000 7f2a9612d640 1 --2- 172.21.15.156:0/1099570876 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f2a7c06e0b0 0x7f2a7c070560 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f2a78003080 tx=0x7f2a78035040 comp rx=0 tx=0).stop 2024-07-16T10:03:43.641 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:43.640+0000 7f2a9612d640 1 -- 172.21.15.156:0/1099570876 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2a900fb240 msgr2=0x7f2a9010fcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:03:43.642 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:43.640+0000 7f2a9612d640 1 --2- 172.21.15.156:0/1099570876 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2a900fb240 0x7f2a9010fcc0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f2a80002a80 tx=0x7f2a80004600 comp rx=0 tx=0).stop 2024-07-16T10:03:43.642 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:43.640+0000 7f2a9612d640 1 -- 172.21.15.156:0/1099570876 shutdown_connections 2024-07-16T10:03:43.642 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:43.640+0000 7f2a9612d640 1 --2- 172.21.15.156:0/1099570876 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f2a7c06e0b0 0x7f2a7c070560 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:03:43.642 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:43.640+0000 7f2a9612d640 1 --2- 172.21.15.156:0/1099570876 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2a900fc510 0x7f2a90112210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:03:43.642 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:43.640+0000 7f2a9612d640 1 --2- 172.21.15.156:0/1099570876 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2a900fb240 0x7f2a9010fcc0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:03:43.642 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:43.640+0000 7f2a9612d640 1 -- 172.21.15.156:0/1099570876 >> 172.21.15.156:0/1099570876 conn(0x7f2a900f6a20 msgr2=0x7f2a900f8140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T10:03:43.642 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:43.640+0000 7f2a9612d640 1 -- 172.21.15.156:0/1099570876 shutdown_connections 2024-07-16T10:03:43.642 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:43.640+0000 7f2a9612d640 1 -- 172.21.15.156:0/1099570876 wait complete. 2024-07-16T10:03:43.723 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:43.720+0000 7efe6253d640 1 -- 172.21.15.156:0/2621943099 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efe5c0fc530 msgr2=0x7efe5c0fc9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:03:43.723 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:43.720+0000 7efe6253d640 1 --2- 172.21.15.156:0/2621943099 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efe5c0fc530 0x7efe5c0fc9a0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7efe500048d0 tx=0x7efe50009da0 comp rx=0 tx=0).stop 2024-07-16T10:03:43.723 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:43.720+0000 7efe6253d640 1 -- 172.21.15.156:0/2621943099 shutdown_connections 2024-07-16T10:03:43.723 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:43.720+0000 7efe6253d640 1 --2- 172.21.15.156:0/2621943099 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efe5c0fc530 0x7efe5c0fc9a0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:03:43.724 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:43.720+0000 7efe6253d640 1 --2- 172.21.15.156:0/2621943099 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7efe5c0fb260 0x7efe5c0fb670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:03:43.724 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:43.720+0000 7efe6253d640 1 -- 172.21.15.156:0/2621943099 >> 172.21.15.156:0/2621943099 conn(0x7efe5c0f6a20 msgr2=0x7efe5c0f8e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T10:03:43.724 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:43.720+0000 7efe6253d640 1 -- 172.21.15.156:0/2621943099 shutdown_connections 2024-07-16T10:03:43.724 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:43.720+0000 7efe6253d640 1 -- 172.21.15.156:0/2621943099 wait complete. 2024-07-16T10:03:43.724 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:43.721+0000 7efe6253d640 1 Processor -- start 2024-07-16T10:03:43.724 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:43.721+0000 7efe6253d640 1 -- start start 2024-07-16T10:03:43.724 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:43.721+0000 7efe6253d640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efe5c0fb260 0x7efe5c10f200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:03:43.724 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:43.721+0000 7efe6253d640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7efe5c0fc530 0x7efe5c10f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:03:43.724 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:43.721+0000 7efe6253d640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe5c10fd90 con 0x7efe5c0fb260 2024-07-16T10:03:43.724 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:43.721+0000 7efe6253d640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe5c1105b0 con 0x7efe5c0fc530 2024-07-16T10:03:43.724 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:43.721+0000 7efe5bfff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efe5c0fb260 0x7efe5c10f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:03:43.724 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:43.721+0000 7efe5bfff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efe5c0fb260 0x7efe5c10f200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:56060/0 (socket says 172.21.15.156:56060) 2024-07-16T10:03:43.725 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:43.721+0000 7efe5bfff640 1 -- 172.21.15.156:0/1668081648 learned_addr learned my addr 172.21.15.156:0/1668081648 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T10:03:43.725 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:43.721+0000 7efe5b7fe640 1 --2- 172.21.15.156:0/1668081648 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7efe5c0fc530 0x7efe5c10f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:03:43.725 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:43.721+0000 7efe5bfff640 1 -- 172.21.15.156:0/1668081648 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7efe5c0fc530 msgr2=0x7efe5c10f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:03:43.725 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:43.721+0000 7efe5bfff640 1 --2- 172.21.15.156:0/1668081648 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7efe5c0fc530 0x7efe5c10f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:03:43.725 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:43.721+0000 7efe5bfff640 1 -- 172.21.15.156:0/1668081648 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe50005040 con 0x7efe5c0fb260 2024-07-16T10:03:43.725 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:43.722+0000 7efe5bfff640 1 --2- 172.21.15.156:0/1668081648 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efe5c0fb260 0x7efe5c10f200 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7efe4c002a50 tx=0x7efe4c004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T10:03:43.725 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:43.722+0000 7efe58ff9640 1 -- 172.21.15.156:0/1668081648 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efe4c007c30 con 0x7efe5c0fb260 2024-07-16T10:03:43.725 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:43.722+0000 7efe58ff9640 1 -- 172.21.15.156:0/1668081648 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7efe4c00f040 con 0x7efe5c0fb260 2024-07-16T10:03:43.725 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:43.722+0000 7efe6253d640 1 -- 172.21.15.156:0/1668081648 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe5c110840 con 0x7efe5c0fb260 2024-07-16T10:03:43.725 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:43.722+0000 7efe58ff9640 1 -- 172.21.15.156:0/1668081648 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efe4c00dad0 con 0x7efe5c0fb260 2024-07-16T10:03:43.725 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:43.722+0000 7efe6253d640 1 -- 172.21.15.156:0/1668081648 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe5c110db0 con 0x7efe5c0fb260 2024-07-16T10:03:43.726 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:43.723+0000 7efe6253d640 1 -- 172.21.15.156:0/1668081648 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe5c1045e0 con 0x7efe5c0fb260 2024-07-16T10:03:43.726 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:43.724+0000 7efe58ff9640 1 -- 172.21.15.156:0/1668081648 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 38) v1 ==== 92205+0+0 (secure 0 0 0) 0x7efe4c00dd40 con 0x7efe5c0fb260 2024-07-16T10:03:43.726 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:43.724+0000 7efe58ff9640 1 --2- 172.21.15.156:0/1668081648 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7efe4406e0b0 0x7efe44070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:03:43.727 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:43.725+0000 7efe58ff9640 1 -- 172.21.15.156:0/1668081648 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7efe4c08d3e0 con 0x7efe5c0fb260 2024-07-16T10:03:43.727 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:43.725+0000 7efe5b7fe640 1 --2- 172.21.15.156:0/1668081648 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7efe4406e0b0 0x7efe44070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:03:43.727 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:43.725+0000 7efe5b7fe640 1 --2- 172.21.15.156:0/1668081648 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7efe4406e0b0 0x7efe44070560 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7efe500043d0 tx=0x7efe50035040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T10:03:43.730 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:43.729+0000 7efe58ff9640 1 -- 172.21.15.156:0/1668081648 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187716 (secure 0 0 0) 0x7efe4c056760 con 0x7efe5c0fb260 2024-07-16T10:03:43.913 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:43.911+0000 7efe6253d640 1 -- 172.21.15.156:0/1668081648 --> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7efe5c103f80 con 0x7efe4406e0b0 2024-07-16T10:03:43.922 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:43.921+0000 7efe58ff9640 1 -- 172.21.15.156:0/1668081648 <== mgr.34104 v2:172.21.15.156:6800/3367765292 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7efe50039040 con 0x7efe4406e0b0 2024-07-16T10:03:43.922 INFO:teuthology.orchestra.run.smithi156.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-07-16T10:03:43.922 INFO:teuthology.orchestra.run.smithi156.stdout:alertmanager.smithi156 smithi156 *:9093,9094 running (7m) 24s ago 15m 24.5M - 0.25.0 c8568f914cd2 d524a2c3ff15 2024-07-16T10:03:43.922 INFO:teuthology.orchestra.run.smithi156.stdout:crash.smithi156 smithi156 running (5m) 24s ago 16m 6899k - 19.0.0-3495-ga6ca461b 16f7feb04342 11648cf29af4 2024-07-16T10:03:43.922 INFO:teuthology.orchestra.run.smithi156.stdout:crash.smithi176 smithi176 running (5m) 8s ago 15m 6899k - 19.0.0-3495-ga6ca461b 16f7feb04342 67f3ccc034f8 2024-07-16T10:03:43.922 INFO:teuthology.orchestra.run.smithi156.stdout:grafana.smithi156 smithi156 *:3000 running (7m) 24s ago 15m 85.3M - 9.4.7 954c08fa6188 deb9e1bfeb66 2024-07-16T10:03:43.922 INFO:teuthology.orchestra.run.smithi156.stdout:mds.cephfs.smithi156.evwwaz smithi156 running (29s) 24s ago 10m 10.9M - 19.0.0-3495-ga6ca461b 16f7feb04342 75879f672b07 2024-07-16T10:03:43.923 INFO:teuthology.orchestra.run.smithi156.stdout:mds.cephfs.smithi156.rdibys smithi156 running (56s) 24s ago 10m 64.5M - 19.0.0-3495-ga6ca461b 16f7feb04342 248f61472011 2024-07-16T10:03:43.923 INFO:teuthology.orchestra.run.smithi156.stdout:mds.cephfs.smithi176.dvvbud smithi176 running (11s) 8s ago 10m 11.8M - 19.0.0-3495-ga6ca461b 16f7feb04342 3aaf56c30dde 2024-07-16T10:03:43.923 INFO:teuthology.orchestra.run.smithi156.stdout:mds.cephfs.smithi176.wahbsi smithi176 running (10m) 8s ago 10m 11.3M - 17.2.7-1104-ge6231b04 d5de5f953b95 866664af2311 2024-07-16T10:03:43.923 INFO:teuthology.orchestra.run.smithi156.stdout:mgr.smithi156.zkvylv smithi156 *:8443,9283,8765 running (6m) 24s ago 17m 505M - 19.0.0-3495-ga6ca461b 16f7feb04342 ee53bd604b7c 2024-07-16T10:03:43.923 INFO:teuthology.orchestra.run.smithi156.stdout:mgr.smithi176.dqbrvx smithi176 *:8443,9283,8765 running (6m) 8s ago 15m 442M - 19.0.0-3495-ga6ca461b 16f7feb04342 ce9eaae4c2d0 2024-07-16T10:03:43.923 INFO:teuthology.orchestra.run.smithi156.stdout:mon.smithi156 smithi156 running (6m) 24s ago 17m 51.3M 2048M 19.0.0-3495-ga6ca461b 16f7feb04342 77b6ec8c50eb 2024-07-16T10:03:43.923 INFO:teuthology.orchestra.run.smithi156.stdout:mon.smithi176 smithi176 running (5m) 8s ago 14m 39.9M 2048M 19.0.0-3495-ga6ca461b 16f7feb04342 ef92814cec2b 2024-07-16T10:03:43.923 INFO:teuthology.orchestra.run.smithi156.stdout:node-exporter.smithi156 smithi156 *:9100 running (7m) 24s ago 15m 18.7M - 1.5.0 0da6a335fe13 e5d86f23a14c 2024-07-16T10:03:43.923 INFO:teuthology.orchestra.run.smithi156.stdout:node-exporter.smithi176 smithi176 *:9100 running (7m) 8s ago 15m 15.2M - 1.5.0 0da6a335fe13 ea842728144a 2024-07-16T10:03:43.923 INFO:teuthology.orchestra.run.smithi156.stdout:osd.0 smithi156 running (4m) 24s ago 14m 104M 1557M 19.0.0-3495-ga6ca461b 16f7feb04342 8633df1689cc 2024-07-16T10:03:43.923 INFO:teuthology.orchestra.run.smithi156.stdout:osd.1 smithi156 running (4m) 24s ago 13m 115M 1557M 19.0.0-3495-ga6ca461b 16f7feb04342 e6fae9653b46 2024-07-16T10:03:43.923 INFO:teuthology.orchestra.run.smithi156.stdout:osd.2 smithi156 running (3m) 24s ago 13m 96.8M 1557M 19.0.0-3495-ga6ca461b 16f7feb04342 c6ecf18de9bf 2024-07-16T10:03:43.923 INFO:teuthology.orchestra.run.smithi156.stdout:osd.3 smithi176 running (3m) 8s ago 13m 112M 2581M 19.0.0-3495-ga6ca461b 16f7feb04342 d216534d6027 2024-07-16T10:03:43.924 INFO:teuthology.orchestra.run.smithi156.stdout:osd.4 smithi176 running (2m) 8s ago 12m 108M 2581M 19.0.0-3495-ga6ca461b 16f7feb04342 1bd3a25317b0 2024-07-16T10:03:43.924 INFO:teuthology.orchestra.run.smithi156.stdout:osd.5 smithi176 running (118s) 8s ago 12m 94.1M 2581M 19.0.0-3495-ga6ca461b 16f7feb04342 a03640221e7a 2024-07-16T10:03:43.924 INFO:teuthology.orchestra.run.smithi156.stdout:prometheus.smithi156 smithi156 *:9095 running (6m) 24s ago 15m 75.6M - 2.43.0 a07b618ecd1d 20c195ecd665 2024-07-16T10:03:43.925 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:43.923+0000 7efe6253d640 1 -- 172.21.15.156:0/1668081648 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7efe4406e0b0 msgr2=0x7efe44070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:03:43.925 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:43.923+0000 7efe6253d640 1 --2- 172.21.15.156:0/1668081648 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7efe4406e0b0 0x7efe44070560 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7efe500043d0 tx=0x7efe50035040 comp rx=0 tx=0).stop 2024-07-16T10:03:43.925 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:43.923+0000 7efe6253d640 1 -- 172.21.15.156:0/1668081648 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efe5c0fb260 msgr2=0x7efe5c10f200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:03:43.925 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:43.923+0000 7efe6253d640 1 --2- 172.21.15.156:0/1668081648 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efe5c0fb260 0x7efe5c10f200 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7efe4c002a50 tx=0x7efe4c004600 comp rx=0 tx=0).stop 2024-07-16T10:03:43.925 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:43.923+0000 7efe6253d640 1 -- 172.21.15.156:0/1668081648 shutdown_connections 2024-07-16T10:03:43.925 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:43.923+0000 7efe6253d640 1 --2- 172.21.15.156:0/1668081648 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7efe4406e0b0 0x7efe44070560 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:03:43.925 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:43.923+0000 7efe6253d640 1 --2- 172.21.15.156:0/1668081648 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7efe5c0fc530 0x7efe5c10f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:03:43.925 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:43.923+0000 7efe6253d640 1 --2- 172.21.15.156:0/1668081648 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efe5c0fb260 0x7efe5c10f200 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:03:43.925 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:43.923+0000 7efe6253d640 1 -- 172.21.15.156:0/1668081648 >> 172.21.15.156:0/1668081648 conn(0x7efe5c0f6a20 msgr2=0x7efe5c065df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T10:03:43.925 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:43.923+0000 7efe6253d640 1 -- 172.21.15.156:0/1668081648 shutdown_connections 2024-07-16T10:03:43.925 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:43.923+0000 7efe6253d640 1 -- 172.21.15.156:0/1668081648 wait complete. 2024-07-16T10:03:44.004 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:44.002+0000 7f479892a640 1 -- 172.21.15.156:0/1262348913 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f47940fd640 msgr2=0x7f47940fdab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:03:44.004 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:44.002+0000 7f479892a640 1 --2- 172.21.15.156:0/1262348913 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f47940fd640 0x7f47940fdab0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f4788009c50 tx=0x7f4788004d10 comp rx=0 tx=0).stop 2024-07-16T10:03:44.004 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:44.002+0000 7f479892a640 1 -- 172.21.15.156:0/1262348913 shutdown_connections 2024-07-16T10:03:44.004 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:44.002+0000 7f479892a640 1 --2- 172.21.15.156:0/1262348913 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f47940fd640 0x7f47940fdab0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:03:44.004 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:44.002+0000 7f479892a640 1 --2- 172.21.15.156:0/1262348913 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f47940fc370 0x7f47940fc780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:03:44.005 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:44.002+0000 7f479892a640 1 -- 172.21.15.156:0/1262348913 >> 172.21.15.156:0/1262348913 conn(0x7f47940f7b30 msgr2=0x7f47940f9f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T10:03:44.005 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:44.003+0000 7f479892a640 1 -- 172.21.15.156:0/1262348913 shutdown_connections 2024-07-16T10:03:44.005 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:44.003+0000 7f479892a640 1 -- 172.21.15.156:0/1262348913 wait complete. 2024-07-16T10:03:44.006 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:44.003+0000 7f479892a640 1 Processor -- start 2024-07-16T10:03:44.007 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:44.003+0000 7f479892a640 1 -- start start 2024-07-16T10:03:44.007 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:44.004+0000 7f479892a640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f47940fc370 0x7f4794113550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:03:44.007 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:44.004+0000 7f479892a640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f47940fd640 0x7f4794113a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:03:44.007 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:44.004+0000 7f479892a640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47941140e0 con 0x7f47940fc370 2024-07-16T10:03:44.007 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:44.004+0000 7f479892a640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4794114900 con 0x7f47940fd640 2024-07-16T10:03:44.007 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:44.004+0000 7f479259c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f47940fc370 0x7f4794113550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:03:44.007 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:44.004+0000 7f479259c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f47940fc370 0x7f4794113550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:56068/0 (socket says 172.21.15.156:56068) 2024-07-16T10:03:44.007 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:44.004+0000 7f479259c640 1 -- 172.21.15.156:0/1757192305 learned_addr learned my addr 172.21.15.156:0/1757192305 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T10:03:44.007 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:44.004+0000 7f4791d9b640 1 --2- 172.21.15.156:0/1757192305 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f47940fd640 0x7f4794113a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:03:44.007 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:44.004+0000 7f479259c640 1 -- 172.21.15.156:0/1757192305 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f47940fd640 msgr2=0x7f4794113a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:03:44.007 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:44.004+0000 7f479259c640 1 --2- 172.21.15.156:0/1757192305 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f47940fd640 0x7f4794113a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:03:44.008 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:44.004+0000 7f479259c640 1 -- 172.21.15.156:0/1757192305 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47880098b0 con 0x7f47940fc370 2024-07-16T10:03:44.008 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:44.004+0000 7f4791d9b640 1 --2- 172.21.15.156:0/1757192305 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f47940fd640 0x7f4794113a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-16T10:03:44.008 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:44.004+0000 7f479259c640 1 --2- 172.21.15.156:0/1757192305 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f47940fc370 0x7f4794113550 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f478c00c790 tx=0x7f478c0043c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T10:03:44.008 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:44.004+0000 7f4782ffd640 1 -- 172.21.15.156:0/1757192305 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f478c00a820 con 0x7f47940fc370 2024-07-16T10:03:44.008 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:44.004+0000 7f479892a640 1 -- 172.21.15.156:0/1757192305 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4794114b90 con 0x7f47940fc370 2024-07-16T10:03:44.008 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:44.004+0000 7f4782ffd640 1 -- 172.21.15.156:0/1757192305 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f478c00ae50 con 0x7f47940fc370 2024-07-16T10:03:44.008 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:44.004+0000 7f4782ffd640 1 -- 172.21.15.156:0/1757192305 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f478c0115e0 con 0x7f47940fc370 2024-07-16T10:03:44.008 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:44.004+0000 7f479892a640 1 -- 172.21.15.156:0/1757192305 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47941150b0 con 0x7f47940fc370 2024-07-16T10:03:44.009 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:44.006+0000 7f479892a640 1 -- 172.21.15.156:0/1757192305 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47940fb7d0 con 0x7f47940fc370 2024-07-16T10:03:44.009 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:44.006+0000 7f4782ffd640 1 -- 172.21.15.156:0/1757192305 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 38) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f478c012400 con 0x7f47940fc370 2024-07-16T10:03:44.009 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:44.006+0000 7f4782ffd640 1 --2- 172.21.15.156:0/1757192305 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f477c06e0b0 0x7f477c070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:03:44.009 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:44.007+0000 7f4782ffd640 1 -- 172.21.15.156:0/1757192305 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f478c08dd90 con 0x7f47940fc370 2024-07-16T10:03:44.009 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:44.007+0000 7f4791d9b640 1 --2- 172.21.15.156:0/1757192305 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f477c06e0b0 0x7f477c070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:03:44.009 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:44.007+0000 7f4791d9b640 1 --2- 172.21.15.156:0/1757192305 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f477c06e0b0 0x7f477c070560 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f4788000f80 tx=0x7f4788004d10 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T10:03:44.012 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:44.011+0000 7f4782ffd640 1 -- 172.21.15.156:0/1757192305 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187716 (secure 0 0 0) 0x7f478c0570e0 con 0x7f47940fc370 2024-07-16T10:03:44.257 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:44.255+0000 7f479892a640 1 -- 172.21.15.156:0/1757192305 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f4794104fe0 con 0x7f47940fc370 2024-07-16T10:03:44.257 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:03:44 smithi156 ceph-mon[93066]: pgmap v215: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail 2024-07-16T10:03:44.257 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:03:44 smithi156 ceph-mon[93066]: from='client.34474 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-16T10:03:44.258 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:44.256+0000 7f4782ffd640 1 -- 172.21.15.156:0/1757192305 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+618 (secure 0 0 0) 0x7f478c05a7a0 con 0x7f47940fc370 2024-07-16T10:03:44.258 INFO:teuthology.orchestra.run.smithi156.stdout:{ 2024-07-16T10:03:44.258 INFO:teuthology.orchestra.run.smithi156.stdout: "mon": { 2024-07-16T10:03:44.258 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph version 19.0.0-3495-ga6ca461b (a6ca461be7cc1aff3658c2c6df4cff975a24449a) squid (rc)": 2 2024-07-16T10:03:44.258 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-07-16T10:03:44.258 INFO:teuthology.orchestra.run.smithi156.stdout: "mgr": { 2024-07-16T10:03:44.258 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph version 19.0.0-3495-ga6ca461b (a6ca461be7cc1aff3658c2c6df4cff975a24449a) squid (rc)": 2 2024-07-16T10:03:44.258 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-07-16T10:03:44.258 INFO:teuthology.orchestra.run.smithi156.stdout: "osd": { 2024-07-16T10:03:44.259 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph version 19.0.0-3495-ga6ca461b (a6ca461be7cc1aff3658c2c6df4cff975a24449a) squid (rc)": 6 2024-07-16T10:03:44.259 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-07-16T10:03:44.259 INFO:teuthology.orchestra.run.smithi156.stdout: "mds": { 2024-07-16T10:03:44.259 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph version 19.0.0-3495-ga6ca461b (a6ca461be7cc1aff3658c2c6df4cff975a24449a) squid (rc)": 3 2024-07-16T10:03:44.259 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-07-16T10:03:44.259 INFO:teuthology.orchestra.run.smithi156.stdout: "overall": { 2024-07-16T10:03:44.259 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph version 19.0.0-3495-ga6ca461b (a6ca461be7cc1aff3658c2c6df4cff975a24449a) squid (rc)": 13 2024-07-16T10:03:44.259 INFO:teuthology.orchestra.run.smithi156.stdout: } 2024-07-16T10:03:44.259 INFO:teuthology.orchestra.run.smithi156.stdout:} 2024-07-16T10:03:44.260 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:44.258+0000 7f479892a640 1 -- 172.21.15.156:0/1757192305 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f477c06e0b0 msgr2=0x7f477c070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:03:44.260 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:44.258+0000 7f479892a640 1 --2- 172.21.15.156:0/1757192305 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f477c06e0b0 0x7f477c070560 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f4788000f80 tx=0x7f4788004d10 comp rx=0 tx=0).stop 2024-07-16T10:03:44.260 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:44.258+0000 7f479892a640 1 -- 172.21.15.156:0/1757192305 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f47940fc370 msgr2=0x7f4794113550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:03:44.260 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:44.258+0000 7f479892a640 1 --2- 172.21.15.156:0/1757192305 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f47940fc370 0x7f4794113550 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f478c00c790 tx=0x7f478c0043c0 comp rx=0 tx=0).stop 2024-07-16T10:03:44.260 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:44.258+0000 7f479892a640 1 -- 172.21.15.156:0/1757192305 shutdown_connections 2024-07-16T10:03:44.260 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:44.258+0000 7f479892a640 1 --2- 172.21.15.156:0/1757192305 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f477c06e0b0 0x7f477c070560 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:03:44.260 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:44.258+0000 7f479892a640 1 --2- 172.21.15.156:0/1757192305 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f47940fd640 0x7f4794113a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:03:44.260 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:44.258+0000 7f479892a640 1 --2- 172.21.15.156:0/1757192305 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f47940fc370 0x7f4794113550 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:03:44.261 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:44.258+0000 7f479892a640 1 -- 172.21.15.156:0/1757192305 >> 172.21.15.156:0/1757192305 conn(0x7f47940f7b30 msgr2=0x7f47940f9040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T10:03:44.261 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:44.258+0000 7f479892a640 1 -- 172.21.15.156:0/1757192305 shutdown_connections 2024-07-16T10:03:44.261 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:44.258+0000 7f479892a640 1 -- 172.21.15.156:0/1757192305 wait complete. 2024-07-16T10:03:44.341 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:44.338+0000 7f9564cf2640 1 -- 172.21.15.156:0/3056402991 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f95600fc530 msgr2=0x7f95600fc9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:03:44.342 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:44.338+0000 7f9564cf2640 1 --2- 172.21.15.156:0/3056402991 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f95600fc530 0x7f95600fc9a0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f9554009c50 tx=0x7f9554004d10 comp rx=0 tx=0).stop 2024-07-16T10:03:44.342 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:44.338+0000 7f9564cf2640 1 -- 172.21.15.156:0/3056402991 shutdown_connections 2024-07-16T10:03:44.342 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:44.338+0000 7f9564cf2640 1 --2- 172.21.15.156:0/3056402991 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f95600fc530 0x7f95600fc9a0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:03:44.342 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:44.338+0000 7f9564cf2640 1 --2- 172.21.15.156:0/3056402991 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f95600fb260 0x7f95600fb670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:03:44.342 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:44.338+0000 7f9564cf2640 1 -- 172.21.15.156:0/3056402991 >> 172.21.15.156:0/3056402991 conn(0x7f95600f6a40 msgr2=0x7f95600f8e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T10:03:44.342 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:44.338+0000 7f9564cf2640 1 -- 172.21.15.156:0/3056402991 shutdown_connections 2024-07-16T10:03:44.342 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:44.338+0000 7f9564cf2640 1 -- 172.21.15.156:0/3056402991 wait complete. 2024-07-16T10:03:44.342 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:44.339+0000 7f9564cf2640 1 Processor -- start 2024-07-16T10:03:44.342 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:44.339+0000 7f9564cf2640 1 -- start start 2024-07-16T10:03:44.343 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:44.339+0000 7f9564cf2640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f95600fb260 0x7f956010fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:03:44.343 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:44.339+0000 7f9564cf2640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f95600fc530 0x7f9560112270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:03:44.343 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:44.339+0000 7f9564cf2640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9560066580 con 0x7f95600fb260 2024-07-16T10:03:44.343 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:44.339+0000 7f9564cf2640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95600666f0 con 0x7f95600fc530 2024-07-16T10:03:44.343 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:44.339+0000 7f955e59c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f95600fb260 0x7f956010fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:03:44.343 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:44.339+0000 7f955e59c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f95600fb260 0x7f956010fd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:56078/0 (socket says 172.21.15.156:56078) 2024-07-16T10:03:44.343 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:44.339+0000 7f955e59c640 1 -- 172.21.15.156:0/2304081375 learned_addr learned my addr 172.21.15.156:0/2304081375 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T10:03:44.343 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:44.339+0000 7f955dd9b640 1 --2- 172.21.15.156:0/2304081375 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f95600fc530 0x7f9560112270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:03:44.343 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:44.339+0000 7f955e59c640 1 -- 172.21.15.156:0/2304081375 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f95600fc530 msgr2=0x7f9560112270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:03:44.343 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:44.339+0000 7f955e59c640 1 --2- 172.21.15.156:0/2304081375 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f95600fc530 0x7f9560112270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:03:44.343 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:44.339+0000 7f955e59c640 1 -- 172.21.15.156:0/2304081375 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95540098b0 con 0x7f95600fb260 2024-07-16T10:03:44.343 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:44.340+0000 7f955e59c640 1 --2- 172.21.15.156:0/2304081375 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f95600fb260 0x7f956010fd20 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f9558002a50 tx=0x7f9558004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T10:03:44.344 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:44.340+0000 7f954effd640 1 -- 172.21.15.156:0/2304081375 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9558007c30 con 0x7f95600fb260 2024-07-16T10:03:44.344 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:44.340+0000 7f954effd640 1 -- 172.21.15.156:0/2304081375 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f955800f040 con 0x7f95600fb260 2024-07-16T10:03:44.344 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:44.340+0000 7f9564cf2640 1 -- 172.21.15.156:0/2304081375 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9560112810 con 0x7f95600fb260 2024-07-16T10:03:44.344 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:44.340+0000 7f954effd640 1 -- 172.21.15.156:0/2304081375 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f955800db50 con 0x7f95600fb260 2024-07-16T10:03:44.344 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:44.340+0000 7f9564cf2640 1 -- 172.21.15.156:0/2304081375 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9560112d00 con 0x7f95600fb260 2024-07-16T10:03:44.345 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:44.341+0000 7f9564cf2640 1 -- 172.21.15.156:0/2304081375 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95600fa6c0 con 0x7f95600fb260 2024-07-16T10:03:44.345 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:44.341+0000 7f954effd640 1 -- 172.21.15.156:0/2304081375 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 38) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f955801c400 con 0x7f95600fb260 2024-07-16T10:03:44.345 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:44.342+0000 7f954effd640 1 --2- 172.21.15.156:0/2304081375 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f954806e0b0 0x7f9548070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:03:44.345 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:44.342+0000 7f954effd640 1 -- 172.21.15.156:0/2304081375 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f955808d4d0 con 0x7f95600fb260 2024-07-16T10:03:44.345 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:44.342+0000 7f955dd9b640 1 --2- 172.21.15.156:0/2304081375 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f954806e0b0 0x7f9548070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:03:44.345 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:44.342+0000 7f955dd9b640 1 --2- 172.21.15.156:0/2304081375 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f954806e0b0 0x7f9548070560 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f955402a8b0 tx=0x7f9554004d10 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T10:03:44.347 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:44.346+0000 7f954effd640 1 -- 172.21.15.156:0/2304081375 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187716 (secure 0 0 0) 0x7f95580567a0 con 0x7f95600fb260 2024-07-16T10:03:44.560 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:44.559+0000 7f9564cf2640 1 -- 172.21.15.156:0/2304081375 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f9560113080 con 0x7f95600fb260 2024-07-16T10:03:44.561 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:44.559+0000 7f954effd640 1 -- 172.21.15.156:0/2304081375 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 40 v40) v1 ==== 76+0+1765 (secure 0 0 0) 0x7f9558059e60 con 0x7f95600fb260 2024-07-16T10:03:44.561 INFO:teuthology.orchestra.run.smithi156.stdout:e40 2024-07-16T10:03:44.561 INFO:teuthology.orchestra.run.smithi156.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-07-16T10:03:44.561 INFO:teuthology.orchestra.run.smithi156.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-07-16T10:03:44.561 INFO:teuthology.orchestra.run.smithi156.stdout:legacy client fscid: 1 2024-07-16T10:03:44.561 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-07-16T10:03:44.561 INFO:teuthology.orchestra.run.smithi156.stdout:Filesystem 'cephfs' (1) 2024-07-16T10:03:44.562 INFO:teuthology.orchestra.run.smithi156.stdout:fs_name cephfs 2024-07-16T10:03:44.562 INFO:teuthology.orchestra.run.smithi156.stdout:epoch 36 2024-07-16T10:03:44.562 INFO:teuthology.orchestra.run.smithi156.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-07-16T10:03:44.562 INFO:teuthology.orchestra.run.smithi156.stdout:created 2024-07-16T09:52:55.214975+0000 2024-07-16T10:03:44.562 INFO:teuthology.orchestra.run.smithi156.stdout:modified 2024-07-16T10:03:11.011991+0000 2024-07-16T10:03:44.562 INFO:teuthology.orchestra.run.smithi156.stdout:tableserver 0 2024-07-16T10:03:44.562 INFO:teuthology.orchestra.run.smithi156.stdout:root 0 2024-07-16T10:03:44.562 INFO:teuthology.orchestra.run.smithi156.stdout:session_timeout 60 2024-07-16T10:03:44.562 INFO:teuthology.orchestra.run.smithi156.stdout:session_autoclose 300 2024-07-16T10:03:44.562 INFO:teuthology.orchestra.run.smithi156.stdout:max_file_size 1099511627776 2024-07-16T10:03:44.563 INFO:teuthology.orchestra.run.smithi156.stdout:max_xattr_size 65536 2024-07-16T10:03:44.563 INFO:teuthology.orchestra.run.smithi156.stdout:required_client_features {} 2024-07-16T10:03:44.563 INFO:teuthology.orchestra.run.smithi156.stdout:last_failure 0 2024-07-16T10:03:44.563 INFO:teuthology.orchestra.run.smithi156.stdout:last_failure_osd_epoch 82 2024-07-16T10:03:44.563 INFO:teuthology.orchestra.run.smithi156.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-07-16T10:03:44.563 INFO:teuthology.orchestra.run.smithi156.stdout:max_mds 1 2024-07-16T10:03:44.563 INFO:teuthology.orchestra.run.smithi156.stdout:in 0 2024-07-16T10:03:44.563 INFO:teuthology.orchestra.run.smithi156.stdout:up {0=34438} 2024-07-16T10:03:44.563 INFO:teuthology.orchestra.run.smithi156.stdout:failed 2024-07-16T10:03:44.563 INFO:teuthology.orchestra.run.smithi156.stdout:damaged 2024-07-16T10:03:44.563 INFO:teuthology.orchestra.run.smithi156.stdout:stopped 1 2024-07-16T10:03:44.563 INFO:teuthology.orchestra.run.smithi156.stdout:data_pools [3] 2024-07-16T10:03:44.564 INFO:teuthology.orchestra.run.smithi156.stdout:metadata_pool 2 2024-07-16T10:03:44.564 INFO:teuthology.orchestra.run.smithi156.stdout:inline_data disabled 2024-07-16T10:03:44.564 INFO:teuthology.orchestra.run.smithi156.stdout:balancer 2024-07-16T10:03:44.564 INFO:teuthology.orchestra.run.smithi156.stdout:bal_rank_mask -1 2024-07-16T10:03:44.564 INFO:teuthology.orchestra.run.smithi156.stdout:standby_count_wanted 1 2024-07-16T10:03:44.564 INFO:teuthology.orchestra.run.smithi156.stdout:qdb_cluster leader: 34438 members: 34438 2024-07-16T10:03:44.564 INFO:teuthology.orchestra.run.smithi156.stdout:[mds.cephfs.smithi156.rdibys{0:34438} state up:active seq 9 join_fscid=1 addr [v2:172.21.15.156:6826/2852996640,v1:172.21.15.156:6827/2852996640] compat {c=[1],r=[1],i=[1fff]}] 2024-07-16T10:03:44.564 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-07-16T10:03:44.564 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-07-16T10:03:44.564 INFO:teuthology.orchestra.run.smithi156.stdout:Standby daemons: 2024-07-16T10:03:44.564 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-07-16T10:03:44.564 INFO:teuthology.orchestra.run.smithi156.stdout:[mds.cephfs.smithi156.evwwaz{-1:34470} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.156:6828/1683974145,v1:172.21.15.156:6829/1683974145] compat {c=[1],r=[1],i=[1fff]}] 2024-07-16T10:03:44.564 INFO:teuthology.orchestra.run.smithi156.stdout:[mds.cephfs.smithi176.dvvbud{-1:44347} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.176:6824/1700587296,v1:172.21.15.176:6825/1700587296] compat {c=[1],r=[1],i=[1fff]}] 2024-07-16T10:03:44.565 INFO:teuthology.orchestra.run.smithi156.stderr:dumped fsmap epoch 40 2024-07-16T10:03:44.565 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:44.561+0000 7f9564cf2640 1 -- 172.21.15.156:0/2304081375 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f954806e0b0 msgr2=0x7f9548070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:03:44.565 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:44.561+0000 7f9564cf2640 1 --2- 172.21.15.156:0/2304081375 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f954806e0b0 0x7f9548070560 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f955402a8b0 tx=0x7f9554004d10 comp rx=0 tx=0).stop 2024-07-16T10:03:44.565 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:44.561+0000 7f9564cf2640 1 -- 172.21.15.156:0/2304081375 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f95600fb260 msgr2=0x7f956010fd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:03:44.565 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:44.561+0000 7f9564cf2640 1 --2- 172.21.15.156:0/2304081375 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f95600fb260 0x7f956010fd20 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f9558002a50 tx=0x7f9558004600 comp rx=0 tx=0).stop 2024-07-16T10:03:44.566 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:44.562+0000 7f9564cf2640 1 -- 172.21.15.156:0/2304081375 shutdown_connections 2024-07-16T10:03:44.566 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:44.562+0000 7f9564cf2640 1 --2- 172.21.15.156:0/2304081375 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f954806e0b0 0x7f9548070560 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:03:44.566 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:44.562+0000 7f9564cf2640 1 --2- 172.21.15.156:0/2304081375 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f95600fc530 0x7f9560112270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:03:44.566 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:44.562+0000 7f9564cf2640 1 --2- 172.21.15.156:0/2304081375 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f95600fb260 0x7f956010fd20 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:03:44.566 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:44.562+0000 7f9564cf2640 1 -- 172.21.15.156:0/2304081375 >> 172.21.15.156:0/2304081375 conn(0x7f95600f6a40 msgr2=0x7f95600f8100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T10:03:44.566 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:44.562+0000 7f9564cf2640 1 -- 172.21.15.156:0/2304081375 shutdown_connections 2024-07-16T10:03:44.566 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:44.562+0000 7f9564cf2640 1 -- 172.21.15.156:0/2304081375 wait complete. 2024-07-16T10:03:44.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:03:44 smithi176 ceph-mon[81689]: pgmap v215: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail 2024-07-16T10:03:44.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:03:44 smithi176 ceph-mon[81689]: from='client.34474 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-16T10:03:44.643 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:44.642+0000 7ff472936640 1 -- 172.21.15.156:0/339955096 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff46c0fc360 msgr2=0x7ff46c0fc770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:03:44.643 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:44.642+0000 7ff472936640 1 --2- 172.21.15.156:0/339955096 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff46c0fc360 0x7ff46c0fc770 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7ff45c0048d0 tx=0x7ff45c009da0 comp rx=0 tx=0).stop 2024-07-16T10:03:44.644 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:44.642+0000 7ff472936640 1 -- 172.21.15.156:0/339955096 shutdown_connections 2024-07-16T10:03:44.644 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:44.642+0000 7ff472936640 1 --2- 172.21.15.156:0/339955096 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff46c0fd630 0x7ff46c0fdaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:03:44.644 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:44.642+0000 7ff472936640 1 --2- 172.21.15.156:0/339955096 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff46c0fc360 0x7ff46c0fc770 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:03:44.644 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:44.642+0000 7ff472936640 1 -- 172.21.15.156:0/339955096 >> 172.21.15.156:0/339955096 conn(0x7ff46c0f7b40 msgr2=0x7ff46c0f9f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T10:03:44.644 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:44.642+0000 7ff472936640 1 -- 172.21.15.156:0/339955096 shutdown_connections 2024-07-16T10:03:44.644 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:44.642+0000 7ff472936640 1 -- 172.21.15.156:0/339955096 wait complete. 2024-07-16T10:03:44.644 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:44.642+0000 7ff472936640 1 Processor -- start 2024-07-16T10:03:44.644 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:44.642+0000 7ff472936640 1 -- start start 2024-07-16T10:03:44.646 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:44.643+0000 7ff472936640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff46c0fc360 0x7ff46c10cff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:03:44.646 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:44.643+0000 7ff472936640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff46c0fd630 0x7ff46c10d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:03:44.646 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:44.643+0000 7ff472936640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff46c10db80 con 0x7ff46c0fd630 2024-07-16T10:03:44.646 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:44.643+0000 7ff472936640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff46c10e350 con 0x7ff46c0fc360 2024-07-16T10:03:44.646 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:44.643+0000 7ff46b7fe640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff46c0fd630 0x7ff46c10d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:03:44.646 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:44.643+0000 7ff46b7fe640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff46c0fd630 0x7ff46c10d530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:56110/0 (socket says 172.21.15.156:56110) 2024-07-16T10:03:44.646 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:44.643+0000 7ff46b7fe640 1 -- 172.21.15.156:0/95510521 learned_addr learned my addr 172.21.15.156:0/95510521 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T10:03:44.646 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:44.643+0000 7ff46bfff640 1 --2- 172.21.15.156:0/95510521 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff46c0fc360 0x7ff46c10cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:03:44.646 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:44.643+0000 7ff46b7fe640 1 -- 172.21.15.156:0/95510521 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff46c0fc360 msgr2=0x7ff46c10cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:03:44.647 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:44.643+0000 7ff46b7fe640 1 --2- 172.21.15.156:0/95510521 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff46c0fc360 0x7ff46c10cff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:03:44.647 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:44.643+0000 7ff46b7fe640 1 -- 172.21.15.156:0/95510521 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff45c005040 con 0x7ff46c0fd630 2024-07-16T10:03:44.647 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:44.643+0000 7ff46b7fe640 1 --2- 172.21.15.156:0/95510521 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff46c0fd630 0x7ff46c10d530 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7ff46000eb70 tx=0x7ff46000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T10:03:44.647 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:44.643+0000 7ff468ff9640 1 -- 172.21.15.156:0/95510521 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff460007c30 con 0x7ff46c0fd630 2024-07-16T10:03:44.647 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:44.643+0000 7ff468ff9640 1 -- 172.21.15.156:0/95510521 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff46000f040 con 0x7ff46c0fd630 2024-07-16T10:03:44.647 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:44.643+0000 7ff472936640 1 -- 172.21.15.156:0/95510521 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff46c10e580 con 0x7ff46c0fd630 2024-07-16T10:03:44.647 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:44.643+0000 7ff468ff9640 1 -- 172.21.15.156:0/95510521 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff46000cad0 con 0x7ff46c0fd630 2024-07-16T10:03:44.647 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:44.644+0000 7ff472936640 1 -- 172.21.15.156:0/95510521 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff46c10eb20 con 0x7ff46c0fd630 2024-07-16T10:03:44.647 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:44.645+0000 7ff472936640 1 -- 172.21.15.156:0/95510521 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff46c0fb7c0 con 0x7ff46c0fd630 2024-07-16T10:03:44.648 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:44.645+0000 7ff468ff9640 1 -- 172.21.15.156:0/95510521 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 38) v1 ==== 92205+0+0 (secure 0 0 0) 0x7ff46000cc30 con 0x7ff46c0fd630 2024-07-16T10:03:44.648 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:44.646+0000 7ff468ff9640 1 --2- 172.21.15.156:0/95510521 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7ff45406e0b0 0x7ff454070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:03:44.648 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:44.646+0000 7ff468ff9640 1 -- 172.21.15.156:0/95510521 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7ff46008d750 con 0x7ff46c0fd630 2024-07-16T10:03:44.648 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:44.646+0000 7ff46bfff640 1 --2- 172.21.15.156:0/95510521 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7ff45406e0b0 0x7ff454070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:03:44.648 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:44.646+0000 7ff46bfff640 1 --2- 172.21.15.156:0/95510521 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7ff45406e0b0 0x7ff454070560 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7ff45c009f80 tx=0x7ff45c035040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T10:03:44.651 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:44.650+0000 7ff468ff9640 1 -- 172.21.15.156:0/95510521 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187716 (secure 0 0 0) 0x7ff460056b50 con 0x7ff46c0fd630 2024-07-16T10:03:44.842 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:44.841+0000 7ff472936640 1 -- 172.21.15.156:0/95510521 --> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff46c10eea0 con 0x7ff45406e0b0 2024-07-16T10:03:44.844 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:44.843+0000 7ff468ff9640 1 -- 172.21.15.156:0/95510521 <== mgr.34104 v2:172.21.15.156:6800/3367765292 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7ff46c10eea0 con 0x7ff45406e0b0 2024-07-16T10:03:44.844 INFO:teuthology.orchestra.run.smithi156.stdout:{ 2024-07-16T10:03:44.844 INFO:teuthology.orchestra.run.smithi156.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:c15ec3aa8e6c0a9c12ee24ff79f3e7dd4d01b3d1a702dfeb7d7fff9b0811a393", 2024-07-16T10:03:44.844 INFO:teuthology.orchestra.run.smithi156.stdout: "in_progress": true, 2024-07-16T10:03:44.844 INFO:teuthology.orchestra.run.smithi156.stdout: "which": "Upgrading all daemon types on all hosts", 2024-07-16T10:03:44.844 INFO:teuthology.orchestra.run.smithi156.stdout: "services_complete": [ 2024-07-16T10:03:44.844 INFO:teuthology.orchestra.run.smithi156.stdout: "mon", 2024-07-16T10:03:44.844 INFO:teuthology.orchestra.run.smithi156.stdout: "mgr", 2024-07-16T10:03:44.844 INFO:teuthology.orchestra.run.smithi156.stdout: "crash", 2024-07-16T10:03:44.845 INFO:teuthology.orchestra.run.smithi156.stdout: "osd" 2024-07-16T10:03:44.845 INFO:teuthology.orchestra.run.smithi156.stdout: ], 2024-07-16T10:03:44.845 INFO:teuthology.orchestra.run.smithi156.stdout: "progress": "15/21 daemons upgraded", 2024-07-16T10:03:44.845 INFO:teuthology.orchestra.run.smithi156.stdout: "message": "Currently upgrading mds daemons", 2024-07-16T10:03:44.845 INFO:teuthology.orchestra.run.smithi156.stdout: "is_paused": false 2024-07-16T10:03:44.845 INFO:teuthology.orchestra.run.smithi156.stdout:} 2024-07-16T10:03:44.846 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:44.845+0000 7ff472936640 1 -- 172.21.15.156:0/95510521 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7ff45406e0b0 msgr2=0x7ff454070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:03:44.846 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:44.845+0000 7ff472936640 1 --2- 172.21.15.156:0/95510521 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7ff45406e0b0 0x7ff454070560 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7ff45c009f80 tx=0x7ff45c035040 comp rx=0 tx=0).stop 2024-07-16T10:03:44.846 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:44.845+0000 7ff472936640 1 -- 172.21.15.156:0/95510521 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff46c0fd630 msgr2=0x7ff46c10d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:03:44.846 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:44.845+0000 7ff472936640 1 --2- 172.21.15.156:0/95510521 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff46c0fd630 0x7ff46c10d530 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7ff46000eb70 tx=0x7ff46000b040 comp rx=0 tx=0).stop 2024-07-16T10:03:44.846 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:44.845+0000 7ff472936640 1 -- 172.21.15.156:0/95510521 shutdown_connections 2024-07-16T10:03:44.846 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:44.845+0000 7ff472936640 1 --2- 172.21.15.156:0/95510521 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7ff45406e0b0 0x7ff454070560 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:03:44.846 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:44.845+0000 7ff472936640 1 --2- 172.21.15.156:0/95510521 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff46c0fd630 0x7ff46c10d530 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:03:44.846 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:44.845+0000 7ff472936640 1 --2- 172.21.15.156:0/95510521 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff46c0fc360 0x7ff46c10cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:03:44.847 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:44.845+0000 7ff472936640 1 -- 172.21.15.156:0/95510521 >> 172.21.15.156:0/95510521 conn(0x7ff46c0f7b40 msgr2=0x7ff46c0f9220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T10:03:44.847 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:44.845+0000 7ff472936640 1 -- 172.21.15.156:0/95510521 shutdown_connections 2024-07-16T10:03:44.847 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:44.845+0000 7ff472936640 1 -- 172.21.15.156:0/95510521 wait complete. 2024-07-16T10:03:44.926 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:44.924+0000 7f92605ee640 1 -- 172.21.15.156:0/809117026 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f92580fd610 msgr2=0x7f92580fda80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:03:44.926 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:44.924+0000 7f92605ee640 1 --2- 172.21.15.156:0/809117026 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f92580fd610 0x7f92580fda80 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f92480048d0 tx=0x7f9248009da0 comp rx=0 tx=0).stop 2024-07-16T10:03:44.926 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:44.924+0000 7f92605ee640 1 -- 172.21.15.156:0/809117026 shutdown_connections 2024-07-16T10:03:44.926 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:44.924+0000 7f92605ee640 1 --2- 172.21.15.156:0/809117026 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f92580fd610 0x7f92580fda80 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:03:44.926 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:44.924+0000 7f92605ee640 1 --2- 172.21.15.156:0/809117026 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f92580fc340 0x7f92580fc750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:03:44.926 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:44.924+0000 7f92605ee640 1 -- 172.21.15.156:0/809117026 >> 172.21.15.156:0/809117026 conn(0x7f92580f7b20 msgr2=0x7f92580f9f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T10:03:44.926 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:44.924+0000 7f92605ee640 1 -- 172.21.15.156:0/809117026 shutdown_connections 2024-07-16T10:03:44.926 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:44.924+0000 7f92605ee640 1 -- 172.21.15.156:0/809117026 wait complete. 2024-07-16T10:03:44.928 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:44.925+0000 7f92605ee640 1 Processor -- start 2024-07-16T10:03:44.928 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:44.925+0000 7f92605ee640 1 -- start start 2024-07-16T10:03:44.928 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:44.925+0000 7f92605ee640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f92580fc340 0x7f9258071380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:03:44.928 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:44.925+0000 7f92605ee640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f92580fd610 0x7f925806f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:03:44.928 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:44.925+0000 7f92605ee640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f925806fe00 con 0x7f92580fc340 2024-07-16T10:03:44.928 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:44.925+0000 7f92605ee640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f925806ff40 con 0x7f92580fd610 2024-07-16T10:03:44.928 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:44.925+0000 7f925e38a640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f92580fc340 0x7f9258071380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:03:44.929 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:44.925+0000 7f925e38a640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f92580fc340 0x7f9258071380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:56122/0 (socket says 172.21.15.156:56122) 2024-07-16T10:03:44.929 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:44.925+0000 7f925e38a640 1 -- 172.21.15.156:0/647139676 learned_addr learned my addr 172.21.15.156:0/647139676 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T10:03:44.929 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:44.925+0000 7f925db89640 1 --2- 172.21.15.156:0/647139676 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f92580fd610 0x7f925806f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:03:44.929 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:44.926+0000 7f925e38a640 1 -- 172.21.15.156:0/647139676 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f92580fd610 msgr2=0x7f925806f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:03:44.929 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:44.926+0000 7f925e38a640 1 --2- 172.21.15.156:0/647139676 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f92580fd610 0x7f925806f870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:03:44.929 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:44.926+0000 7f925e38a640 1 -- 172.21.15.156:0/647139676 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9248005040 con 0x7f92580fc340 2024-07-16T10:03:44.929 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:44.926+0000 7f925e38a640 1 --2- 172.21.15.156:0/647139676 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f92580fc340 0x7f9258071380 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f925400c830 tx=0x7f9254004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T10:03:44.929 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:44.926+0000 7f924effd640 1 -- 172.21.15.156:0/647139676 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f925400a850 con 0x7f92580fc340 2024-07-16T10:03:44.929 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:44.926+0000 7f924effd640 1 -- 172.21.15.156:0/647139676 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f925400ae80 con 0x7f92580fc340 2024-07-16T10:03:44.929 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:44.926+0000 7f92605ee640 1 -- 172.21.15.156:0/647139676 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9258070220 con 0x7f92580fc340 2024-07-16T10:03:44.929 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:44.926+0000 7f924effd640 1 -- 172.21.15.156:0/647139676 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9254012ea0 con 0x7f92580fc340 2024-07-16T10:03:44.929 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:44.926+0000 7f92605ee640 1 -- 172.21.15.156:0/647139676 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9258114fa0 con 0x7f92580fc340 2024-07-16T10:03:44.930 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:44.927+0000 7f92605ee640 1 -- 172.21.15.156:0/647139676 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92580fb7a0 con 0x7f92580fc340 2024-07-16T10:03:44.930 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:44.927+0000 7f924effd640 1 -- 172.21.15.156:0/647139676 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 38) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f9254012400 con 0x7f92580fc340 2024-07-16T10:03:44.930 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:44.928+0000 7f924effd640 1 --2- 172.21.15.156:0/647139676 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f924406e0b0 0x7f9244070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:03:44.930 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:44.928+0000 7f924effd640 1 -- 172.21.15.156:0/647139676 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f925408de30 con 0x7f92580fc340 2024-07-16T10:03:44.930 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:44.928+0000 7f925db89640 1 --2- 172.21.15.156:0/647139676 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f924406e0b0 0x7f9244070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:03:44.931 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:44.928+0000 7f925db89640 1 --2- 172.21.15.156:0/647139676 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f924406e0b0 0x7f9244070560 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f9248000c00 tx=0x7f9248035040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T10:03:44.933 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:44.932+0000 7f924effd640 1 -- 172.21.15.156:0/647139676 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187716 (secure 0 0 0) 0x7f9254057180 con 0x7f92580fc340 2024-07-16T10:03:45.184 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:45.183+0000 7f92605ee640 1 -- 172.21.15.156:0/647139676 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f925804b530 con 0x7f92580fc340 2024-07-16T10:03:45.185 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:45.183+0000 7f924effd640 1 -- 172.21.15.156:0/647139676 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7f92540126b0 con 0x7f92580fc340 2024-07-16T10:03:45.185 INFO:teuthology.orchestra.run.smithi156.stdout:HEALTH_OK 2024-07-16T10:03:45.186 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:45.185+0000 7f92605ee640 1 -- 172.21.15.156:0/647139676 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f924406e0b0 msgr2=0x7f9244070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:03:45.187 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:45.185+0000 7f92605ee640 1 --2- 172.21.15.156:0/647139676 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f924406e0b0 0x7f9244070560 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f9248000c00 tx=0x7f9248035040 comp rx=0 tx=0).stop 2024-07-16T10:03:45.187 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:45.185+0000 7f92605ee640 1 -- 172.21.15.156:0/647139676 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f92580fc340 msgr2=0x7f9258071380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:03:45.187 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:45.185+0000 7f92605ee640 1 --2- 172.21.15.156:0/647139676 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f92580fc340 0x7f9258071380 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f925400c830 tx=0x7f9254004430 comp rx=0 tx=0).stop 2024-07-16T10:03:45.187 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:45.185+0000 7f92605ee640 1 -- 172.21.15.156:0/647139676 shutdown_connections 2024-07-16T10:03:45.187 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:45.185+0000 7f92605ee640 1 --2- 172.21.15.156:0/647139676 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f924406e0b0 0x7f9244070560 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:03:45.187 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:45.185+0000 7f92605ee640 1 --2- 172.21.15.156:0/647139676 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f92580fd610 0x7f925806f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:03:45.187 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:45.185+0000 7f92605ee640 1 --2- 172.21.15.156:0/647139676 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f92580fc340 0x7f9258071380 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:03:45.187 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:45.185+0000 7f92605ee640 1 -- 172.21.15.156:0/647139676 >> 172.21.15.156:0/647139676 conn(0x7f92580f7b20 msgr2=0x7f92580f91e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T10:03:45.187 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:45.185+0000 7f92605ee640 1 -- 172.21.15.156:0/647139676 shutdown_connections 2024-07-16T10:03:45.187 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:03:45.185+0000 7f92605ee640 1 -- 172.21.15.156:0/647139676 wait complete. 2024-07-16T10:03:45.448 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:03:45 smithi156 ceph-mon[93066]: from='client.34478 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-16T10:03:45.449 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:03:45 smithi156 ceph-mon[93066]: from='client.34482 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-07-16T10:03:45.449 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:03:45 smithi156 ceph-mon[93066]: from='client.? 172.21.15.156:0/1757192305' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:03:45.449 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:03:45 smithi156 ceph-mon[93066]: from='client.? 172.21.15.156:0/2304081375' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-07-16T10:03:45.449 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:03:45 smithi156 ceph-mon[93066]: from='client.? 172.21.15.156:0/647139676' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-07-16T10:03:45.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:03:45 smithi176 ceph-mon[81689]: from='client.34478 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-16T10:03:45.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:03:45 smithi176 ceph-mon[81689]: from='client.34482 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-07-16T10:03:45.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:03:45 smithi176 ceph-mon[81689]: from='client.? 172.21.15.156:0/1757192305' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:03:45.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:03:45 smithi176 ceph-mon[81689]: from='client.? 172.21.15.156:0/2304081375' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-07-16T10:03:45.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:03:45 smithi176 ceph-mon[81689]: from='client.? 172.21.15.156:0/647139676' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-07-16T10:03:46.571 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:03:46 smithi176 ceph-mon[81689]: pgmap v216: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail 2024-07-16T10:03:46.571 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:03:46 smithi176 ceph-mon[81689]: from='client.34494 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-16T10:03:46.571 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:03:46 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi176.wahbsi"}]: dispatch 2024-07-16T10:03:46.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:03:46 smithi156 ceph-mon[93066]: pgmap v216: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail 2024-07-16T10:03:46.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:03:46 smithi156 ceph-mon[93066]: from='client.34494 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-16T10:03:46.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:03:46 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi176.wahbsi"}]: dispatch 2024-07-16T10:03:48.486 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:03:48 smithi176 ceph-mon[81689]: pgmap v217: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail 2024-07-16T10:03:48.486 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:03:48 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:03:48.486 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:03:48 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:03:48.486 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:03:48 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-16T10:03:48.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:03:48 smithi156 ceph-mon[93066]: pgmap v217: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail 2024-07-16T10:03:48.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:03:48 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:03:48.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:03:48 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:03:48.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:03:48 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-16T10:03:49.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:03:49 smithi176 ceph-mon[81689]: mds.? [v2:172.21.15.176:6826/2277003188,v1:172.21.15.176:6827/2277003188] up:boot 2024-07-16T10:03:49.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:03:49 smithi176 ceph-mon[81689]: fsmap cephfs:1 {0=cephfs.smithi156.rdibys=up:active} 3 up:standby 2024-07-16T10:03:49.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:03:49 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi176.wahbsi"}]: dispatch 2024-07-16T10:03:49.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:03:49 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-16T10:03:49.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:03:49 smithi156 ceph-mon[93066]: mds.? [v2:172.21.15.176:6826/2277003188,v1:172.21.15.176:6827/2277003188] up:boot 2024-07-16T10:03:49.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:03:49 smithi156 ceph-mon[93066]: fsmap cephfs:1 {0=cephfs.smithi156.rdibys=up:active} 3 up:standby 2024-07-16T10:03:49.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:03:49 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi176.wahbsi"}]: dispatch 2024-07-16T10:03:49.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:03:49 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-16T10:03:50.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:03:50 smithi176 ceph-mon[81689]: pgmap v218: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail 2024-07-16T10:03:50.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:03:50 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:03:50.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:03:50 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:03:50.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:03:50 smithi156 ceph-mon[93066]: pgmap v218: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail 2024-07-16T10:03:50.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:03:50 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:03:50.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:03:50 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:03:51.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:03:51 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:03:51.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:03:51 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:03:51.764 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:03:51 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:03:51.764 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:03:51 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:03:52.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:03:52 smithi156 ceph-mon[93066]: pgmap v219: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail 2024-07-16T10:03:52.814 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:03:52 smithi176 ceph-mon[81689]: pgmap v219: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail 2024-07-16T10:03:53.563 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:03:53 smithi176 ceph-mon[81689]: pgmap v220: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail 2024-07-16T10:03:53.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:03:53 smithi156 ceph-mon[93066]: pgmap v220: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail 2024-07-16T10:03:54.924 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:03:54 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:03:54.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:03:54 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:03:54.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:03:54 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-16T10:03:54.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:03:54 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-16T10:03:54.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:03:54 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:03:54.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:03:54 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-16T10:03:54.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:03:54 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:03:54.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:03:54 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:03:54.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:03:54 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:03:54.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:03:54 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:03:54.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:03:54 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:03:54.926 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:03:54 smithi156 ceph-mon[93066]: Upgrade: Setting container_image for all mds 2024-07-16T10:03:54.926 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:03:54 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:03:54.926 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:03:54 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi156.evwwaz"}]: dispatch 2024-07-16T10:03:54.926 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:03:54 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi156.evwwaz"}]': finished 2024-07-16T10:03:54.926 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:03:54 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi156.rdibys"}]: dispatch 2024-07-16T10:03:54.926 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:03:54 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi156.rdibys"}]': finished 2024-07-16T10:03:54.926 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:03:54 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi176.dvvbud"}]: dispatch 2024-07-16T10:03:54.926 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:03:54 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi176.dvvbud"}]': finished 2024-07-16T10:03:54.926 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:03:54 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi176.wahbsi"}]: dispatch 2024-07-16T10:03:54.926 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:03:54 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi176.wahbsi"}]': finished 2024-07-16T10:03:54.926 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:03:54 smithi156 ceph-mon[93066]: Upgrade: Scaling up filesystem cephfs max_mds to 2 2024-07-16T10:03:54.927 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:03:54 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"}]: dispatch 2024-07-16T10:03:55.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:03:54 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:03:55.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:03:54 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:03:55.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:03:54 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-16T10:03:55.095 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:03:54 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-16T10:03:55.095 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:03:54 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:03:55.095 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:03:54 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-16T10:03:55.095 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:03:54 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:03:55.095 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:03:54 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:03:55.095 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:03:54 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:03:55.096 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:03:54 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:03:55.096 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:03:54 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:03:55.096 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:03:54 smithi176 ceph-mon[81689]: Upgrade: Setting container_image for all mds 2024-07-16T10:03:55.096 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:03:54 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:03:55.096 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:03:54 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi156.evwwaz"}]: dispatch 2024-07-16T10:03:55.097 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:03:54 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi156.evwwaz"}]': finished 2024-07-16T10:03:55.097 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:03:54 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi156.rdibys"}]: dispatch 2024-07-16T10:03:55.097 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:03:54 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi156.rdibys"}]': finished 2024-07-16T10:03:55.097 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:03:54 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi176.dvvbud"}]: dispatch 2024-07-16T10:03:55.097 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:03:54 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi176.dvvbud"}]': finished 2024-07-16T10:03:55.097 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:03:54 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi176.wahbsi"}]: dispatch 2024-07-16T10:03:55.097 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:03:54 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi176.wahbsi"}]': finished 2024-07-16T10:03:55.098 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:03:54 smithi176 ceph-mon[81689]: Upgrade: Scaling up filesystem cephfs max_mds to 2 2024-07-16T10:03:55.098 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:03:54 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"}]: dispatch 2024-07-16T10:03:55.924 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:03:55 smithi156 ceph-mon[93066]: pgmap v221: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail 2024-07-16T10:03:55.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:03:55 smithi156 ceph-mon[93066]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-07-16T10:03:55.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:03:55 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"}]': finished 2024-07-16T10:03:55.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:03:55 smithi156 ceph-mon[93066]: daemon mds.cephfs.smithi156.evwwaz assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-07-16T10:03:55.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:03:55 smithi156 ceph-mon[93066]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-07-16T10:03:55.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:03:55 smithi156 ceph-mon[93066]: Cluster is now healthy 2024-07-16T10:03:55.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:03:55 smithi156 ceph-mon[93066]: fsmap cephfs:1 {0=cephfs.smithi156.rdibys=up:active} 3 up:standby 2024-07-16T10:03:55.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:03:55 smithi156 ceph-mon[93066]: fsmap cephfs:2 {0=cephfs.smithi156.rdibys=up:active,1=cephfs.smithi156.evwwaz=up:starting} 2 up:standby 2024-07-16T10:03:55.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:03:55 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:03:55.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:03:55 smithi156 ceph-mon[93066]: Upgrade: Enabling allow_standby_replay on filesystem cephfs 2024-07-16T10:03:55.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:03:55 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "1"}]: dispatch 2024-07-16T10:03:55.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:03:55 smithi156 ceph-mon[93066]: daemon mds.cephfs.smithi156.evwwaz is now active in filesystem cephfs as rank 1 2024-07-16T10:03:56.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:03:55 smithi176 ceph-mon[81689]: pgmap v221: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail 2024-07-16T10:03:56.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:03:55 smithi176 ceph-mon[81689]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-07-16T10:03:56.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:03:55 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"}]': finished 2024-07-16T10:03:56.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:03:55 smithi176 ceph-mon[81689]: daemon mds.cephfs.smithi156.evwwaz assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-07-16T10:03:56.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:03:55 smithi176 ceph-mon[81689]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-07-16T10:03:56.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:03:55 smithi176 ceph-mon[81689]: Cluster is now healthy 2024-07-16T10:03:56.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:03:55 smithi176 ceph-mon[81689]: fsmap cephfs:1 {0=cephfs.smithi156.rdibys=up:active} 3 up:standby 2024-07-16T10:03:56.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:03:55 smithi176 ceph-mon[81689]: fsmap cephfs:2 {0=cephfs.smithi156.rdibys=up:active,1=cephfs.smithi156.evwwaz=up:starting} 2 up:standby 2024-07-16T10:03:56.095 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:03:55 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:03:56.095 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:03:55 smithi176 ceph-mon[81689]: Upgrade: Enabling allow_standby_replay on filesystem cephfs 2024-07-16T10:03:56.095 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:03:55 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "1"}]: dispatch 2024-07-16T10:03:56.095 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:03:55 smithi176 ceph-mon[81689]: daemon mds.cephfs.smithi156.evwwaz is now active in filesystem cephfs as rank 1 2024-07-16T10:03:56.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:03:56 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "1"}]': finished 2024-07-16T10:03:56.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:03:56 smithi156 ceph-mon[93066]: mds.? [v2:172.21.15.156:6828/1683974145,v1:172.21.15.156:6829/1683974145] up:active 2024-07-16T10:03:56.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:03:56 smithi156 ceph-mon[93066]: fsmap cephfs:2 {0=cephfs.smithi156.rdibys=up:active,1=cephfs.smithi156.evwwaz=up:active} 2 up:standby 2024-07-16T10:03:56.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:03:56 smithi156 ceph-mon[93066]: fsmap cephfs:2 {0=cephfs.smithi156.rdibys=up:active,1=cephfs.smithi156.evwwaz=up:active} 2 up:standby-replay 2024-07-16T10:03:56.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:03:56 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:03:56.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:03:56 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:03:56.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:03:56 smithi156 ceph-mon[93066]: Upgrade: Setting container_image for all rgw 2024-07-16T10:03:56.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:03:56 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:03:56.926 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:03:56 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:03:56.926 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:03:56 smithi156 ceph-mon[93066]: Upgrade: Setting container_image for all rbd-mirror 2024-07-16T10:03:56.926 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:03:56 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:03:56.926 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:03:56 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:03:56.926 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:03:56 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:03:56.926 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:03:56 smithi156 ceph-mon[93066]: Upgrade: Setting container_image for all ceph-exporter 2024-07-16T10:03:56.926 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:03:56 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:03:56.926 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:03:56 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:03:56.926 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:03:56 smithi156 ceph-mon[93066]: Upgrade: Setting container_image for all iscsi 2024-07-16T10:03:56.926 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:03:56 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:03:56.926 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:03:56 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:03:56.926 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:03:56 smithi156 ceph-mon[93066]: Upgrade: Setting container_image for all nfs 2024-07-16T10:03:56.926 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:03:56 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:03:56.927 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:03:56 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:03:56.927 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:03:56 smithi156 ceph-mon[93066]: Upgrade: Setting container_image for all nvmeof 2024-07-16T10:03:56.927 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:03:56 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:03:57.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:03:56 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "1"}]': finished 2024-07-16T10:03:57.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:03:56 smithi176 ceph-mon[81689]: mds.? [v2:172.21.15.156:6828/1683974145,v1:172.21.15.156:6829/1683974145] up:active 2024-07-16T10:03:57.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:03:56 smithi176 ceph-mon[81689]: fsmap cephfs:2 {0=cephfs.smithi156.rdibys=up:active,1=cephfs.smithi156.evwwaz=up:active} 2 up:standby 2024-07-16T10:03:57.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:03:56 smithi176 ceph-mon[81689]: fsmap cephfs:2 {0=cephfs.smithi156.rdibys=up:active,1=cephfs.smithi156.evwwaz=up:active} 2 up:standby-replay 2024-07-16T10:03:57.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:03:56 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:03:57.095 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:03:56 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:03:57.095 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:03:56 smithi176 ceph-mon[81689]: Upgrade: Setting container_image for all rgw 2024-07-16T10:03:57.095 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:03:56 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:03:57.095 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:03:56 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:03:57.095 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:03:56 smithi176 ceph-mon[81689]: Upgrade: Setting container_image for all rbd-mirror 2024-07-16T10:03:57.095 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:03:56 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:03:57.095 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:03:56 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:03:57.095 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:03:56 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:03:57.095 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:03:56 smithi176 ceph-mon[81689]: Upgrade: Setting container_image for all ceph-exporter 2024-07-16T10:03:57.095 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:03:56 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:03:57.095 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:03:56 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:03:57.095 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:03:56 smithi176 ceph-mon[81689]: Upgrade: Setting container_image for all iscsi 2024-07-16T10:03:57.096 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:03:56 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:03:57.096 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:03:56 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:03:57.096 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:03:56 smithi176 ceph-mon[81689]: Upgrade: Setting container_image for all nfs 2024-07-16T10:03:57.096 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:03:56 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:03:57.096 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:03:56 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:03:57.096 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:03:56 smithi176 ceph-mon[81689]: Upgrade: Setting container_image for all nvmeof 2024-07-16T10:03:57.096 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:03:56 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:03:57.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:03:57 smithi156 ceph-mon[93066]: pgmap v222: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 255 B/s rd, 0 op/s 2024-07-16T10:03:57.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:03:57 smithi156 ceph-mon[93066]: Upgrade: Updating node-exporter.smithi156 (1/2) 2024-07-16T10:03:57.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:03:57 smithi156 ceph-mon[93066]: Deploying daemon node-exporter.smithi156 on smithi156 2024-07-16T10:03:58.093 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:03:57 smithi176 ceph-mon[81689]: pgmap v222: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 255 B/s rd, 0 op/s 2024-07-16T10:03:58.093 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:03:57 smithi176 ceph-mon[81689]: Upgrade: Updating node-exporter.smithi156 (1/2) 2024-07-16T10:03:58.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:03:57 smithi176 ceph-mon[81689]: Deploying daemon node-exporter.smithi156 on smithi156 2024-07-16T10:03:59.924 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:03:59 smithi156 ceph-mon[93066]: pgmap v223: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.0 MiB/s rd, 85 B/s wr, 3 op/s 2024-07-16T10:03:59.924 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:03:59 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:03:59.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:03:59 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:04:00.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:03:59 smithi176 ceph-mon[81689]: pgmap v223: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.0 MiB/s rd, 85 B/s wr, 3 op/s 2024-07-16T10:04:00.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:03:59 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:04:00.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:03:59 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:04:02.055 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:04:01 smithi156 ceph-mon[93066]: pgmap v224: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 8.3 MiB/s rd, 170 B/s wr, 8 op/s 2024-07-16T10:04:02.055 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:04:01 smithi156 ceph-mon[93066]: Upgrade: Updating node-exporter.smithi176 (2/2) 2024-07-16T10:04:02.055 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:04:01 smithi156 ceph-mon[93066]: Deploying daemon node-exporter.smithi176 on smithi176 2024-07-16T10:04:02.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:04:01 smithi176 ceph-mon[81689]: pgmap v224: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 8.3 MiB/s rd, 170 B/s wr, 8 op/s 2024-07-16T10:04:02.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:04:01 smithi176 ceph-mon[81689]: Upgrade: Updating node-exporter.smithi176 (2/2) 2024-07-16T10:04:02.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:04:01 smithi176 ceph-mon[81689]: Deploying daemon node-exporter.smithi176 on smithi176 2024-07-16T10:04:04.093 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:04:03 smithi176 ceph-mon[81689]: pgmap v225: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 12 MiB/s rd, 170 B/s wr, 11 op/s 2024-07-16T10:04:04.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:04:03 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-16T10:04:04.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:04:03 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:04:04.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:04:03 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:04:04.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:04:03 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-16T10:04:04.174 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:04:03 smithi156 ceph-mon[93066]: pgmap v225: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 12 MiB/s rd, 170 B/s wr, 11 op/s 2024-07-16T10:04:04.174 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:04:03 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-16T10:04:04.174 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:04:03 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:04:04.175 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:04:03 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:04:04.175 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:04:03 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-16T10:04:06.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:04:05 smithi176 ceph-mon[81689]: pgmap v226: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 12 MiB/s rd, 170 B/s wr, 11 op/s 2024-07-16T10:04:06.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:04:05 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:04:06.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:04:05 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:04:06.174 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:04:05 smithi156 ceph-mon[93066]: pgmap v226: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 12 MiB/s rd, 170 B/s wr, 11 op/s 2024-07-16T10:04:06.175 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:04:05 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:04:06.175 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:04:05 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:04:07.424 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:04:07 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:04:07.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:04:07 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:04:07.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:04:07 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:04:07.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:04:07 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:04:08.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:04:08 smithi176 ceph-mon[81689]: pgmap v227: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 12 MiB/s rd, 170 B/s wr, 12 op/s 2024-07-16T10:04:08.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:04:08 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:04:08.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:04:08 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:04:08.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:04:08 smithi156 ceph-mon[93066]: pgmap v227: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 12 MiB/s rd, 170 B/s wr, 12 op/s 2024-07-16T10:04:08.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:04:08 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:04:08.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:04:08 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:04:09.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:04:09 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:04:09.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:04:09 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:04:09.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:04:09 smithi156 ceph-mon[93066]: pgmap v228: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 12 MiB/s rd, 170 B/s wr, 12 op/s 2024-07-16T10:04:09.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:04:09 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:04:09.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:04:09 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:04:09.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:04:09 smithi176 ceph-mon[81689]: pgmap v228: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 12 MiB/s rd, 170 B/s wr, 12 op/s 2024-07-16T10:04:11.560 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:04:11 smithi156 ceph-mon[93066]: pgmap v229: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 9.6 MiB/s rd, 85 B/s wr, 10 op/s 2024-07-16T10:04:11.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:04:11 smithi176 ceph-mon[81689]: pgmap v229: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 9.6 MiB/s rd, 85 B/s wr, 10 op/s 2024-07-16T10:04:12.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:04:12 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:04:12.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:04:12 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:04:12.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:04:12 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:04:12.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:04:12 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:04:12.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:04:12 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-16T10:04:12.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:04:12 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-16T10:04:12.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:04:12 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:04:12.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:04:12 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-16T10:04:12.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:04:12 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:04:12.845 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:04:12 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:04:12.845 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:04:12 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:04:12.845 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:04:12 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:04:12.845 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:04:12 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:04:12.845 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:04:12 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:04:12.845 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:04:12 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:04:12.845 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:04:12 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:04:12.845 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:04:12 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:04:12.845 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:04:12 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:04:12.845 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:04:12 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:04:12.845 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:04:12 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:04:12.845 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:04:12 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:04:12.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:04:12 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:04:12.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:04:12 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:04:12.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:04:12 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:04:12.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:04:12 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:04:12.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:04:12 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-16T10:04:12.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:04:12 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-16T10:04:12.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:04:12 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:04:12.926 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:04:12 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-16T10:04:12.926 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:04:12 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:04:12.926 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:04:12 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:04:12.926 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:04:12 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:04:12.926 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:04:12 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:04:12.926 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:04:12 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:04:12.926 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:04:12 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:04:12.926 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:04:12 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:04:12.926 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:04:12 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:04:12.926 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:04:12 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:04:12.926 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:04:12 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:04:12.926 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:04:12 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:04:12.926 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:04:12 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:04:12.927 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:04:12 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:04:14.343 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:04:14 smithi176 ceph-mon[81689]: pgmap v230: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 3.2 MiB/s rd, 0 B/s wr, 5 op/s 2024-07-16T10:04:14.353 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:04:14 smithi156 ceph-mon[93066]: pgmap v230: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 3.2 MiB/s rd, 0 B/s wr, 5 op/s 2024-07-16T10:04:15.271 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:15.269+0000 7f32738d2640 1 -- 172.21.15.156:0/4084718078 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f326c0fb240 msgr2=0x7f326c0fb650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:04:15.272 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:15.269+0000 7f32738d2640 1 --2- 172.21.15.156:0/4084718078 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f326c0fb240 0x7f326c0fb650 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f32680048d0 tx=0x7f3268009da0 comp rx=0 tx=0).stop 2024-07-16T10:04:15.272 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:15.269+0000 7f32738d2640 1 -- 172.21.15.156:0/4084718078 shutdown_connections 2024-07-16T10:04:15.272 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:15.269+0000 7f32738d2640 1 --2- 172.21.15.156:0/4084718078 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f326c0fc510 0x7f326c0fc980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:04:15.272 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:15.269+0000 7f32738d2640 1 --2- 172.21.15.156:0/4084718078 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f326c0fb240 0x7f326c0fb650 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:04:15.272 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:15.269+0000 7f32738d2640 1 -- 172.21.15.156:0/4084718078 >> 172.21.15.156:0/4084718078 conn(0x7f326c0f6a20 msgr2=0x7f326c0f8e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T10:04:15.272 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:04:15 smithi156 ceph-mon[93066]: Upgrade: Updating prometheus.smithi156 2024-07-16T10:04:15.272 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:04:15 smithi156 ceph-mon[93066]: Deploying daemon prometheus.smithi156 on smithi156 2024-07-16T10:04:15.274 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:15.269+0000 7f32738d2640 1 -- 172.21.15.156:0/4084718078 shutdown_connections 2024-07-16T10:04:15.274 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:15.269+0000 7f32738d2640 1 -- 172.21.15.156:0/4084718078 wait complete. 2024-07-16T10:04:15.274 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:15.270+0000 7f32738d2640 1 Processor -- start 2024-07-16T10:04:15.274 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:15.270+0000 7f32738d2640 1 -- start start 2024-07-16T10:04:15.274 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:15.270+0000 7f32738d2640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f326c0fb240 0x7f326c10f160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:04:15.274 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:15.270+0000 7f32738d2640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f326c0fc510 0x7f326c10f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:04:15.274 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:15.270+0000 7f32738d2640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f326c10fcf0 con 0x7f326c0fb240 2024-07-16T10:04:15.274 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:15.270+0000 7f32738d2640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f326c110510 con 0x7f326c0fc510 2024-07-16T10:04:15.274 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:15.270+0000 7f327166e640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f326c0fb240 0x7f326c10f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:04:15.274 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:15.270+0000 7f327166e640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f326c0fb240 0x7f326c10f160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:47128/0 (socket says 172.21.15.156:47128) 2024-07-16T10:04:15.275 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:15.270+0000 7f327166e640 1 -- 172.21.15.156:0/292596589 learned_addr learned my addr 172.21.15.156:0/292596589 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T10:04:15.275 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:15.270+0000 7f327166e640 1 -- 172.21.15.156:0/292596589 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f326c0fc510 msgr2=0x7f326c10f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:04:15.275 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:15.270+0000 7f3270e6d640 1 --2- 172.21.15.156:0/292596589 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f326c0fc510 0x7f326c10f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:04:15.275 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:15.270+0000 7f327166e640 1 --2- 172.21.15.156:0/292596589 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f326c0fc510 0x7f326c10f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:04:15.275 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:15.270+0000 7f327166e640 1 -- 172.21.15.156:0/292596589 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3268005040 con 0x7f326c0fb240 2024-07-16T10:04:15.275 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:15.270+0000 7f3270e6d640 1 --2- 172.21.15.156:0/292596589 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f326c0fc510 0x7f326c10f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-16T10:04:15.275 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:15.271+0000 7f327166e640 1 --2- 172.21.15.156:0/292596589 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f326c0fb240 0x7f326c10f160 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f32680048a0 tx=0x7f3268009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T10:04:15.275 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:15.271+0000 7f3261ffb640 1 -- 172.21.15.156:0/292596589 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3268038070 con 0x7f326c0fb240 2024-07-16T10:04:15.275 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:15.271+0000 7f32738d2640 1 -- 172.21.15.156:0/292596589 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f326c110740 con 0x7f326c0fb240 2024-07-16T10:04:15.275 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:15.271+0000 7f3261ffb640 1 -- 172.21.15.156:0/292596589 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f32680092b0 con 0x7f326c0fb240 2024-07-16T10:04:15.275 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:15.271+0000 7f3261ffb640 1 -- 172.21.15.156:0/292596589 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f32680338a0 con 0x7f326c0fb240 2024-07-16T10:04:15.275 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:15.271+0000 7f32738d2640 1 -- 172.21.15.156:0/292596589 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f326c110c80 con 0x7f326c0fb240 2024-07-16T10:04:15.276 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:15.272+0000 7f32738d2640 1 -- 172.21.15.156:0/292596589 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f326c0fa6a0 con 0x7f326c0fb240 2024-07-16T10:04:15.277 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:15.273+0000 7f3261ffb640 1 -- 172.21.15.156:0/292596589 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 38) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f3268030020 con 0x7f326c0fb240 2024-07-16T10:04:15.277 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:15.273+0000 7f3261ffb640 1 --2- 172.21.15.156:0/292596589 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f325806e0b0 0x7f3258070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:04:15.277 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:15.273+0000 7f3261ffb640 1 -- 172.21.15.156:0/292596589 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f32680ad900 con 0x7f326c0fb240 2024-07-16T10:04:15.277 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:15.273+0000 7f3270e6d640 1 --2- 172.21.15.156:0/292596589 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f325806e0b0 0x7f3258070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:04:15.277 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:15.274+0000 7f3270e6d640 1 --2- 172.21.15.156:0/292596589 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f325806e0b0 0x7f3258070560 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f326c10ef10 tx=0x7f325c00b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T10:04:15.281 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:15.278+0000 7f3261ffb640 1 -- 172.21.15.156:0/292596589 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187716 (secure 0 0 0) 0x7f3268076d00 con 0x7f326c0fb240 2024-07-16T10:04:15.343 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:04:15 smithi176 ceph-mon[81689]: Upgrade: Updating prometheus.smithi156 2024-07-16T10:04:15.343 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:04:15 smithi176 ceph-mon[81689]: Deploying daemon prometheus.smithi156 on smithi156 2024-07-16T10:04:15.491 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:15.488+0000 7f32738d2640 1 -- 172.21.15.156:0/292596589 --> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f326c065670 con 0x7f325806e0b0 2024-07-16T10:04:15.491 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:15.489+0000 7f3261ffb640 1 -- 172.21.15.156:0/292596589 <== mgr.34104 v2:172.21.15.156:6800/3367765292 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+438 (secure 0 0 0) 0x7f326c065670 con 0x7f325806e0b0 2024-07-16T10:04:15.494 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:15.491+0000 7f32738d2640 1 -- 172.21.15.156:0/292596589 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f325806e0b0 msgr2=0x7f3258070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:04:15.494 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:15.491+0000 7f32738d2640 1 --2- 172.21.15.156:0/292596589 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f325806e0b0 0x7f3258070560 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f326c10ef10 tx=0x7f325c00b040 comp rx=0 tx=0).stop 2024-07-16T10:04:15.494 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:15.491+0000 7f32738d2640 1 -- 172.21.15.156:0/292596589 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f326c0fb240 msgr2=0x7f326c10f160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:04:15.494 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:15.491+0000 7f32738d2640 1 --2- 172.21.15.156:0/292596589 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f326c0fb240 0x7f326c10f160 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f32680048a0 tx=0x7f3268009da0 comp rx=0 tx=0).stop 2024-07-16T10:04:15.494 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:15.491+0000 7f32738d2640 1 -- 172.21.15.156:0/292596589 shutdown_connections 2024-07-16T10:04:15.494 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:15.491+0000 7f32738d2640 1 --2- 172.21.15.156:0/292596589 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f325806e0b0 0x7f3258070560 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:04:15.494 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:15.491+0000 7f32738d2640 1 --2- 172.21.15.156:0/292596589 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f326c0fc510 0x7f326c10f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:04:15.495 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:15.491+0000 7f32738d2640 1 --2- 172.21.15.156:0/292596589 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f326c0fb240 0x7f326c10f160 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:04:15.495 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:15.491+0000 7f32738d2640 1 -- 172.21.15.156:0/292596589 >> 172.21.15.156:0/292596589 conn(0x7f326c0f6a20 msgr2=0x7f326c0f7350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T10:04:15.495 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:15.492+0000 7f32738d2640 1 -- 172.21.15.156:0/292596589 shutdown_connections 2024-07-16T10:04:15.495 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:15.492+0000 7f32738d2640 1 -- 172.21.15.156:0/292596589 wait complete. 2024-07-16T10:04:15.502 INFO:teuthology.orchestra.run.smithi156.stdout:true 2024-07-16T10:04:15.577 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:15.575+0000 7f4d6265f640 1 -- 172.21.15.156:0/3055354678 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4d5c0fc330 msgr2=0x7f4d5c0fc740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:04:15.578 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:15.575+0000 7f4d6265f640 1 --2- 172.21.15.156:0/3055354678 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4d5c0fc330 0x7f4d5c0fc740 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f4d4c0048d0 tx=0x7f4d4c009da0 comp rx=0 tx=0).stop 2024-07-16T10:04:15.578 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:15.575+0000 7f4d6265f640 1 -- 172.21.15.156:0/3055354678 shutdown_connections 2024-07-16T10:04:15.578 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:15.575+0000 7f4d6265f640 1 --2- 172.21.15.156:0/3055354678 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4d5c0fd600 0x7f4d5c0fda70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:04:15.578 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:15.575+0000 7f4d6265f640 1 --2- 172.21.15.156:0/3055354678 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4d5c0fc330 0x7f4d5c0fc740 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:04:15.578 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:15.575+0000 7f4d6265f640 1 -- 172.21.15.156:0/3055354678 >> 172.21.15.156:0/3055354678 conn(0x7f4d5c0f7af0 msgr2=0x7f4d5c0f9f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T10:04:15.578 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:15.575+0000 7f4d6265f640 1 -- 172.21.15.156:0/3055354678 shutdown_connections 2024-07-16T10:04:15.578 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:15.575+0000 7f4d6265f640 1 -- 172.21.15.156:0/3055354678 wait complete. 2024-07-16T10:04:15.579 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:15.576+0000 7f4d6265f640 1 Processor -- start 2024-07-16T10:04:15.579 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:15.576+0000 7f4d6265f640 1 -- start start 2024-07-16T10:04:15.579 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:15.576+0000 7f4d6265f640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4d5c0fc330 0x7f4d5c10cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:04:15.580 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:15.576+0000 7f4d6265f640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4d5c0fd600 0x7f4d5c10d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:04:15.580 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:15.577+0000 7f4d6265f640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d5c10daf0 con 0x7f4d5c0fd600 2024-07-16T10:04:15.580 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:15.577+0000 7f4d6265f640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d5c10e310 con 0x7f4d5c0fc330 2024-07-16T10:04:15.580 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:15.577+0000 7f4d5b7fe640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4d5c0fd600 0x7f4d5c10d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:04:15.580 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:15.577+0000 7f4d5b7fe640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4d5c0fd600 0x7f4d5c10d4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:47140/0 (socket says 172.21.15.156:47140) 2024-07-16T10:04:15.580 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:15.577+0000 7f4d5b7fe640 1 -- 172.21.15.156:0/3341940184 learned_addr learned my addr 172.21.15.156:0/3341940184 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T10:04:15.580 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:15.577+0000 7f4d5bfff640 1 --2- 172.21.15.156:0/3341940184 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4d5c0fc330 0x7f4d5c10cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:04:15.580 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:15.577+0000 7f4d5b7fe640 1 -- 172.21.15.156:0/3341940184 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4d5c0fc330 msgr2=0x7f4d5c10cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:04:15.580 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:15.577+0000 7f4d5b7fe640 1 --2- 172.21.15.156:0/3341940184 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4d5c0fc330 0x7f4d5c10cf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:04:15.580 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:15.577+0000 7f4d5b7fe640 1 -- 172.21.15.156:0/3341940184 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d4c005040 con 0x7f4d5c0fd600 2024-07-16T10:04:15.580 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:15.577+0000 7f4d5b7fe640 1 --2- 172.21.15.156:0/3341940184 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4d5c0fd600 0x7f4d5c10d4a0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f4d50002a50 tx=0x7f4d50004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T10:04:15.581 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:15.578+0000 7f4d58ff9640 1 -- 172.21.15.156:0/3341940184 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4d50007c30 con 0x7f4d5c0fd600 2024-07-16T10:04:15.581 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:15.578+0000 7f4d6265f640 1 -- 172.21.15.156:0/3341940184 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d5c10e5a0 con 0x7f4d5c0fd600 2024-07-16T10:04:15.581 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:15.578+0000 7f4d58ff9640 1 -- 172.21.15.156:0/3341940184 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4d5000f040 con 0x7f4d5c0fd600 2024-07-16T10:04:15.581 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:15.578+0000 7f4d6265f640 1 -- 172.21.15.156:0/3341940184 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d5c10eb40 con 0x7f4d5c0fd600 2024-07-16T10:04:15.581 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:15.578+0000 7f4d58ff9640 1 -- 172.21.15.156:0/3341940184 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4d5000dad0 con 0x7f4d5c0fd600 2024-07-16T10:04:15.582 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:15.579+0000 7f4d6265f640 1 -- 172.21.15.156:0/3341940184 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d5c0fb790 con 0x7f4d5c0fd600 2024-07-16T10:04:15.582 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:15.579+0000 7f4d58ff9640 1 -- 172.21.15.156:0/3341940184 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 38) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f4d5000dd10 con 0x7f4d5c0fd600 2024-07-16T10:04:15.582 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:15.580+0000 7f4d58ff9640 1 --2- 172.21.15.156:0/3341940184 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f4d4406e0b0 0x7f4d44070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:04:15.582 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:15.580+0000 7f4d58ff9640 1 -- 172.21.15.156:0/3341940184 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f4d5008d300 con 0x7f4d5c0fd600 2024-07-16T10:04:15.583 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:15.580+0000 7f4d5bfff640 1 --2- 172.21.15.156:0/3341940184 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f4d4406e0b0 0x7f4d44070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:04:15.583 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:15.580+0000 7f4d5bfff640 1 --2- 172.21.15.156:0/3341940184 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f4d4406e0b0 0x7f4d44070560 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f4d4c0043d0 tx=0x7f4d4c035040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T10:04:15.586 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:15.584+0000 7f4d58ff9640 1 -- 172.21.15.156:0/3341940184 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187716 (secure 0 0 0) 0x7f4d50056700 con 0x7f4d5c0fd600 2024-07-16T10:04:15.790 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:15.788+0000 7f4d6265f640 1 -- 172.21.15.156:0/3341940184 --> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f4d5c10eec0 con 0x7f4d4406e0b0 2024-07-16T10:04:15.791 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:15.789+0000 7f4d58ff9640 1 -- 172.21.15.156:0/3341940184 <== mgr.34104 v2:172.21.15.156:6800/3367765292 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+438 (secure 0 0 0) 0x7f4d5c10eec0 con 0x7f4d4406e0b0 2024-07-16T10:04:15.794 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:15.791+0000 7f4d6265f640 1 -- 172.21.15.156:0/3341940184 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f4d4406e0b0 msgr2=0x7f4d44070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:04:15.794 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:15.791+0000 7f4d6265f640 1 --2- 172.21.15.156:0/3341940184 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f4d4406e0b0 0x7f4d44070560 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f4d4c0043d0 tx=0x7f4d4c035040 comp rx=0 tx=0).stop 2024-07-16T10:04:15.794 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:15.791+0000 7f4d6265f640 1 -- 172.21.15.156:0/3341940184 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4d5c0fd600 msgr2=0x7f4d5c10d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:04:15.794 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:15.791+0000 7f4d6265f640 1 --2- 172.21.15.156:0/3341940184 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4d5c0fd600 0x7f4d5c10d4a0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f4d50002a50 tx=0x7f4d50004600 comp rx=0 tx=0).stop 2024-07-16T10:04:15.794 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:15.791+0000 7f4d6265f640 1 -- 172.21.15.156:0/3341940184 shutdown_connections 2024-07-16T10:04:15.794 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:15.791+0000 7f4d6265f640 1 --2- 172.21.15.156:0/3341940184 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f4d4406e0b0 0x7f4d44070560 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:04:15.794 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:15.791+0000 7f4d6265f640 1 --2- 172.21.15.156:0/3341940184 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4d5c0fd600 0x7f4d5c10d4a0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:04:15.794 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:15.791+0000 7f4d6265f640 1 --2- 172.21.15.156:0/3341940184 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4d5c0fc330 0x7f4d5c10cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:04:15.794 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:15.791+0000 7f4d6265f640 1 -- 172.21.15.156:0/3341940184 >> 172.21.15.156:0/3341940184 conn(0x7f4d5c0f7af0 msgr2=0x7f4d5c0f91d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T10:04:15.794 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:15.792+0000 7f4d6265f640 1 -- 172.21.15.156:0/3341940184 shutdown_connections 2024-07-16T10:04:15.794 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:15.792+0000 7f4d6265f640 1 -- 172.21.15.156:0/3341940184 wait complete. 2024-07-16T10:04:15.878 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:15.875+0000 7f49713e7640 1 -- 172.21.15.156:0/1170451654 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f496c0fc510 msgr2=0x7f496c0fc980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:04:15.878 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:15.875+0000 7f49713e7640 1 --2- 172.21.15.156:0/1170451654 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f496c0fc510 0x7f496c0fc980 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f49600048d0 tx=0x7f4960009da0 comp rx=0 tx=0).stop 2024-07-16T10:04:15.878 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:15.876+0000 7f49713e7640 1 -- 172.21.15.156:0/1170451654 shutdown_connections 2024-07-16T10:04:15.879 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:15.876+0000 7f49713e7640 1 --2- 172.21.15.156:0/1170451654 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f496c0fc510 0x7f496c0fc980 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:04:15.879 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:15.876+0000 7f49713e7640 1 --2- 172.21.15.156:0/1170451654 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f496c0fb240 0x7f496c0fb650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:04:15.879 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:15.876+0000 7f49713e7640 1 -- 172.21.15.156:0/1170451654 >> 172.21.15.156:0/1170451654 conn(0x7f496c0f6a20 msgr2=0x7f496c0f8e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T10:04:15.879 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:15.876+0000 7f49713e7640 1 -- 172.21.15.156:0/1170451654 shutdown_connections 2024-07-16T10:04:15.879 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:15.876+0000 7f49713e7640 1 -- 172.21.15.156:0/1170451654 wait complete. 2024-07-16T10:04:15.881 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:15.876+0000 7f49713e7640 1 Processor -- start 2024-07-16T10:04:15.881 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:15.876+0000 7f49713e7640 1 -- start start 2024-07-16T10:04:15.881 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:15.877+0000 7f49713e7640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f496c0fb240 0x7f496c10cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:04:15.881 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:15.877+0000 7f49713e7640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f496c0fc510 0x7f496c10d500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:04:15.881 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:15.877+0000 7f49713e7640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f496c10db50 con 0x7f496c0fb240 2024-07-16T10:04:15.882 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:15.877+0000 7f49713e7640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f496c10e320 con 0x7f496c0fc510 2024-07-16T10:04:15.882 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:15.877+0000 7f496affd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f496c0fb240 0x7f496c10cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:04:15.882 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:15.877+0000 7f496affd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f496c0fb240 0x7f496c10cfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:47170/0 (socket says 172.21.15.156:47170) 2024-07-16T10:04:15.882 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:15.877+0000 7f496affd640 1 -- 172.21.15.156:0/1718630572 learned_addr learned my addr 172.21.15.156:0/1718630572 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T10:04:15.882 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:15.877+0000 7f496a7fc640 1 --2- 172.21.15.156:0/1718630572 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f496c0fc510 0x7f496c10d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:04:15.882 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:15.877+0000 7f496affd640 1 -- 172.21.15.156:0/1718630572 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f496c0fc510 msgr2=0x7f496c10d500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:04:15.882 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:15.877+0000 7f496affd640 1 --2- 172.21.15.156:0/1718630572 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f496c0fc510 0x7f496c10d500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:04:15.882 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:15.877+0000 7f496affd640 1 -- 172.21.15.156:0/1718630572 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4960005040 con 0x7f496c0fb240 2024-07-16T10:04:15.882 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:15.877+0000 7f496affd640 1 --2- 172.21.15.156:0/1718630572 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f496c0fb240 0x7f496c10cfc0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f495c002a80 tx=0x7f495c004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T10:04:15.882 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:15.877+0000 7f49537fe640 1 -- 172.21.15.156:0/1718630572 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f495c007c30 con 0x7f496c0fb240 2024-07-16T10:04:15.883 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:15.878+0000 7f49537fe640 1 -- 172.21.15.156:0/1718630572 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f495c00f040 con 0x7f496c0fb240 2024-07-16T10:04:15.883 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:15.878+0000 7f49713e7640 1 -- 172.21.15.156:0/1718630572 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f496c10e5b0 con 0x7f496c0fb240 2024-07-16T10:04:15.883 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:15.878+0000 7f49537fe640 1 -- 172.21.15.156:0/1718630572 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f495c00db70 con 0x7f496c0fb240 2024-07-16T10:04:15.883 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:15.878+0000 7f49713e7640 1 -- 172.21.15.156:0/1718630572 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f496c10eaa0 con 0x7f496c0fb240 2024-07-16T10:04:15.883 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:15.879+0000 7f49713e7640 1 -- 172.21.15.156:0/1718630572 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f496c1045a0 con 0x7f496c0fb240 2024-07-16T10:04:15.883 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:15.879+0000 7f49537fe640 1 -- 172.21.15.156:0/1718630572 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 38) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f495c00dcf0 con 0x7f496c0fb240 2024-07-16T10:04:15.884 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:15.880+0000 7f49537fe640 1 --2- 172.21.15.156:0/1718630572 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f495406e0b0 0x7f4954070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:04:15.884 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:15.880+0000 7f49537fe640 1 -- 172.21.15.156:0/1718630572 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f495c08d490 con 0x7f496c0fb240 2024-07-16T10:04:15.884 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:15.880+0000 7f496a7fc640 1 --2- 172.21.15.156:0/1718630572 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f495406e0b0 0x7f4954070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:04:15.884 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:15.880+0000 7f496a7fc640 1 --2- 172.21.15.156:0/1718630572 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f495406e0b0 0x7f4954070560 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f49600043d0 tx=0x7f4960035040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T10:04:15.886 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:15.884+0000 7f49537fe640 1 -- 172.21.15.156:0/1718630572 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187716 (secure 0 0 0) 0x7f495c056810 con 0x7f496c0fb240 2024-07-16T10:04:16.076 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:16.074+0000 7f49713e7640 1 -- 172.21.15.156:0/1718630572 --> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f496c0008d0 con 0x7f495406e0b0 2024-07-16T10:04:16.086 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:16.083+0000 7f49537fe640 1 -- 172.21.15.156:0/1718630572 <== mgr.34104 v2:172.21.15.156:6800/3367765292 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f496c0008d0 con 0x7f495406e0b0 2024-07-16T10:04:16.086 INFO:teuthology.orchestra.run.smithi156.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-07-16T10:04:16.086 INFO:teuthology.orchestra.run.smithi156.stdout:alertmanager.smithi156 smithi156 *:9093,9094 running (8m) 8s ago 16m 24.5M - 0.25.0 c8568f914cd2 d524a2c3ff15 2024-07-16T10:04:16.086 INFO:teuthology.orchestra.run.smithi156.stdout:crash.smithi156 smithi156 running (5m) 8s ago 16m 6899k - 19.0.0-3495-ga6ca461b 16f7feb04342 11648cf29af4 2024-07-16T10:04:16.086 INFO:teuthology.orchestra.run.smithi156.stdout:crash.smithi176 smithi176 running (5m) 10s ago 15m 6899k - 19.0.0-3495-ga6ca461b 16f7feb04342 67f3ccc034f8 2024-07-16T10:04:16.086 INFO:teuthology.orchestra.run.smithi156.stdout:grafana.smithi156 smithi156 *:3000 running (7m) 8s ago 16m 86.2M - 9.4.7 954c08fa6188 deb9e1bfeb66 2024-07-16T10:04:16.086 INFO:teuthology.orchestra.run.smithi156.stdout:mds.cephfs.smithi156.evwwaz smithi156 running (61s) 8s ago 11m 14.7M - 19.0.0-3495-ga6ca461b 16f7feb04342 75879f672b07 2024-07-16T10:04:16.086 INFO:teuthology.orchestra.run.smithi156.stdout:mds.cephfs.smithi156.rdibys smithi156 running (88s) 8s ago 11m 64.5M - 19.0.0-3495-ga6ca461b 16f7feb04342 248f61472011 2024-07-16T10:04:16.086 INFO:teuthology.orchestra.run.smithi156.stdout:mds.cephfs.smithi176.dvvbud smithi176 running (43s) 10s ago 11m 133M - 19.0.0-3495-ga6ca461b 16f7feb04342 3aaf56c30dde 2024-07-16T10:04:16.087 INFO:teuthology.orchestra.run.smithi156.stdout:mds.cephfs.smithi176.wahbsi smithi176 running (28s) 10s ago 11m 14.3M - 19.0.0-3495-ga6ca461b 16f7feb04342 d30f4f63a683 2024-07-16T10:04:16.087 INFO:teuthology.orchestra.run.smithi156.stdout:mgr.smithi156.zkvylv smithi156 *:8443,9283,8765 running (7m) 8s ago 17m 507M - 19.0.0-3495-ga6ca461b 16f7feb04342 ee53bd604b7c 2024-07-16T10:04:16.087 INFO:teuthology.orchestra.run.smithi156.stdout:mgr.smithi176.dqbrvx smithi176 *:8443,9283,8765 running (6m) 10s ago 15m 442M - 19.0.0-3495-ga6ca461b 16f7feb04342 ce9eaae4c2d0 2024-07-16T10:04:16.087 INFO:teuthology.orchestra.run.smithi156.stdout:mon.smithi156 smithi156 running (6m) 8s ago 17m 52.5M 2048M 19.0.0-3495-ga6ca461b 16f7feb04342 77b6ec8c50eb 2024-07-16T10:04:16.087 INFO:teuthology.orchestra.run.smithi156.stdout:mon.smithi176 smithi176 running (6m) 10s ago 15m 39.8M 2048M 19.0.0-3495-ga6ca461b 16f7feb04342 ef92814cec2b 2024-07-16T10:04:16.087 INFO:teuthology.orchestra.run.smithi156.stdout:node-exporter.smithi156 smithi156 *:9100 running (16s) 8s ago 16m 5440k - 1.5.0 0da6a335fe13 a67719f4e14a 2024-07-16T10:04:16.087 INFO:teuthology.orchestra.run.smithi156.stdout:node-exporter.smithi176 smithi176 *:9100 running (12s) 10s ago 15m 3598k - 1.5.0 0da6a335fe13 87cdbf1674aa 2024-07-16T10:04:16.087 INFO:teuthology.orchestra.run.smithi156.stdout:osd.0 smithi156 running (5m) 8s ago 14m 105M 1557M 19.0.0-3495-ga6ca461b 16f7feb04342 8633df1689cc 2024-07-16T10:04:16.087 INFO:teuthology.orchestra.run.smithi156.stdout:osd.1 smithi156 running (4m) 8s ago 14m 116M 1557M 19.0.0-3495-ga6ca461b 16f7feb04342 e6fae9653b46 2024-07-16T10:04:16.087 INFO:teuthology.orchestra.run.smithi156.stdout:osd.2 smithi156 running (4m) 8s ago 13m 97.6M 1557M 19.0.0-3495-ga6ca461b 16f7feb04342 c6ecf18de9bf 2024-07-16T10:04:16.087 INFO:teuthology.orchestra.run.smithi156.stdout:osd.3 smithi176 running (3m) 10s ago 13m 113M 2581M 19.0.0-3495-ga6ca461b 16f7feb04342 d216534d6027 2024-07-16T10:04:16.087 INFO:teuthology.orchestra.run.smithi156.stdout:osd.4 smithi176 running (3m) 10s ago 13m 109M 2581M 19.0.0-3495-ga6ca461b 16f7feb04342 1bd3a25317b0 2024-07-16T10:04:16.088 INFO:teuthology.orchestra.run.smithi156.stdout:osd.5 smithi176 running (2m) 10s ago 12m 95.7M 2581M 19.0.0-3495-ga6ca461b 16f7feb04342 a03640221e7a 2024-07-16T10:04:16.088 INFO:teuthology.orchestra.run.smithi156.stdout:prometheus.smithi156 smithi156 *:9095 running (6m) 8s ago 15m 75.7M - 2.43.0 a07b618ecd1d 20c195ecd665 2024-07-16T10:04:16.088 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:16.086+0000 7f49713e7640 1 -- 172.21.15.156:0/1718630572 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f495406e0b0 msgr2=0x7f4954070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:04:16.088 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:16.086+0000 7f49713e7640 1 --2- 172.21.15.156:0/1718630572 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f495406e0b0 0x7f4954070560 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f49600043d0 tx=0x7f4960035040 comp rx=0 tx=0).stop 2024-07-16T10:04:16.088 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:16.086+0000 7f49713e7640 1 -- 172.21.15.156:0/1718630572 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f496c0fb240 msgr2=0x7f496c10cfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:04:16.088 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:16.086+0000 7f49713e7640 1 --2- 172.21.15.156:0/1718630572 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f496c0fb240 0x7f496c10cfc0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f495c002a80 tx=0x7f495c004600 comp rx=0 tx=0).stop 2024-07-16T10:04:16.089 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:16.086+0000 7f49713e7640 1 -- 172.21.15.156:0/1718630572 shutdown_connections 2024-07-16T10:04:16.089 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:16.086+0000 7f49713e7640 1 --2- 172.21.15.156:0/1718630572 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f495406e0b0 0x7f4954070560 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:04:16.089 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:16.086+0000 7f49713e7640 1 --2- 172.21.15.156:0/1718630572 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f496c0fc510 0x7f496c10d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:04:16.089 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:16.086+0000 7f49713e7640 1 --2- 172.21.15.156:0/1718630572 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f496c0fb240 0x7f496c10cfc0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:04:16.089 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:16.086+0000 7f49713e7640 1 -- 172.21.15.156:0/1718630572 >> 172.21.15.156:0/1718630572 conn(0x7f496c0f6a20 msgr2=0x7f496c065380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T10:04:16.089 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:16.086+0000 7f49713e7640 1 -- 172.21.15.156:0/1718630572 shutdown_connections 2024-07-16T10:04:16.089 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:16.086+0000 7f49713e7640 1 -- 172.21.15.156:0/1718630572 wait complete. 2024-07-16T10:04:16.167 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:16.165+0000 7f4948267640 1 -- 172.21.15.156:0/4273360316 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f49400ff510 msgr2=0x7f49401018f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:04:16.167 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:16.165+0000 7f4948267640 1 --2- 172.21.15.156:0/4273360316 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f49400ff510 0x7f49401018f0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f49300048d0 tx=0x7f4930009da0 comp rx=0 tx=0).stop 2024-07-16T10:04:16.168 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:04:16 smithi156 ceph-mon[93066]: pgmap v231: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 1.5 KiB/s rd, 3 op/s 2024-07-16T10:04:16.170 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:16.165+0000 7f4948267640 1 -- 172.21.15.156:0/4273360316 shutdown_connections 2024-07-16T10:04:16.170 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:16.165+0000 7f4948267640 1 --2- 172.21.15.156:0/4273360316 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f49400ff510 0x7f49401018f0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:04:16.170 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:16.165+0000 7f4948267640 1 --2- 172.21.15.156:0/4273360316 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f49400664c0 0x7f49400fefd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:04:16.170 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:16.165+0000 7f4948267640 1 -- 172.21.15.156:0/4273360316 >> 172.21.15.156:0/4273360316 conn(0x7f49400f6a20 msgr2=0x7f49400f8e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T10:04:16.170 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:16.165+0000 7f4948267640 1 -- 172.21.15.156:0/4273360316 shutdown_connections 2024-07-16T10:04:16.170 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:16.165+0000 7f4948267640 1 -- 172.21.15.156:0/4273360316 wait complete. 2024-07-16T10:04:16.170 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:16.166+0000 7f4948267640 1 Processor -- start 2024-07-16T10:04:16.170 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:16.166+0000 7f4948267640 1 -- start start 2024-07-16T10:04:16.170 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:16.166+0000 7f4948267640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f49400664c0 0x7f49400fc9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:04:16.170 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:16.166+0000 7f4948267640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f49400ff510 0x7f49400faed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:04:16.171 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:16.166+0000 7f4948267640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49400fb460 con 0x7f49400ff510 2024-07-16T10:04:16.171 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:16.166+0000 7f4948267640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49400fb5a0 con 0x7f49400664c0 2024-07-16T10:04:16.171 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:16.166+0000 7f4945802640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f49400ff510 0x7f49400faed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:04:16.171 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:16.166+0000 7f4945802640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f49400ff510 0x7f49400faed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:47180/0 (socket says 172.21.15.156:47180) 2024-07-16T10:04:16.171 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:16.166+0000 7f4945802640 1 -- 172.21.15.156:0/2394423785 learned_addr learned my addr 172.21.15.156:0/2394423785 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T10:04:16.171 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:16.166+0000 7f4946003640 1 --2- 172.21.15.156:0/2394423785 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f49400664c0 0x7f49400fc9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:04:16.171 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:16.167+0000 7f4945802640 1 -- 172.21.15.156:0/2394423785 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f49400664c0 msgr2=0x7f49400fc9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:04:16.171 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:16.167+0000 7f4945802640 1 --2- 172.21.15.156:0/2394423785 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f49400664c0 0x7f49400fc9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:04:16.171 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:16.167+0000 7f4945802640 1 -- 172.21.15.156:0/2394423785 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4930005040 con 0x7f49400ff510 2024-07-16T10:04:16.171 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:16.167+0000 7f4945802640 1 --2- 172.21.15.156:0/2394423785 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f49400ff510 0x7f49400faed0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f4930009ae0 tx=0x7f4930009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T10:04:16.171 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:16.167+0000 7f4936ffd640 1 -- 172.21.15.156:0/2394423785 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4930038070 con 0x7f49400ff510 2024-07-16T10:04:16.171 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:16.167+0000 7f4936ffd640 1 -- 172.21.15.156:0/2394423785 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4930032d60 con 0x7f49400ff510 2024-07-16T10:04:16.172 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:16.167+0000 7f4948267640 1 -- 172.21.15.156:0/2394423785 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49400fb820 con 0x7f49400ff510 2024-07-16T10:04:16.172 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:16.167+0000 7f4936ffd640 1 -- 172.21.15.156:0/2394423785 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4930033420 con 0x7f49400ff510 2024-07-16T10:04:16.172 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:16.167+0000 7f4948267640 1 -- 172.21.15.156:0/2394423785 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49400fbd10 con 0x7f49400ff510 2024-07-16T10:04:16.172 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:16.169+0000 7f4948267640 1 -- 172.21.15.156:0/2394423785 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49400fa6c0 con 0x7f49400ff510 2024-07-16T10:04:16.173 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:16.170+0000 7f4936ffd640 1 -- 172.21.15.156:0/2394423785 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 38) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f4930030020 con 0x7f49400ff510 2024-07-16T10:04:16.173 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:16.170+0000 7f4936ffd640 1 --2- 172.21.15.156:0/2394423785 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f492c06e0b0 0x7f492c070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:04:16.173 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:16.170+0000 7f4936ffd640 1 -- 172.21.15.156:0/2394423785 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f49300ae630 con 0x7f49400ff510 2024-07-16T10:04:16.173 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:16.171+0000 7f4946003640 1 --2- 172.21.15.156:0/2394423785 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f492c06e0b0 0x7f492c070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:04:16.173 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:16.171+0000 7f4946003640 1 --2- 172.21.15.156:0/2394423785 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f492c06e0b0 0x7f492c070560 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f49400fc790 tx=0x7f493c00b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T10:04:16.177 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:16.175+0000 7f4936ffd640 1 -- 172.21.15.156:0/2394423785 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187716 (secure 0 0 0) 0x7f49300779c0 con 0x7f49400ff510 2024-07-16T10:04:16.343 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:04:16 smithi176 ceph-mon[81689]: pgmap v231: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 1.5 KiB/s rd, 3 op/s 2024-07-16T10:04:16.427 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:16.425+0000 7f4948267640 1 -- 172.21.15.156:0/2394423785 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f494004b530 con 0x7f49400ff510 2024-07-16T10:04:16.428 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:16.426+0000 7f4936ffd640 1 -- 172.21.15.156:0/2394423785 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+618 (secure 0 0 0) 0x7f493007b080 con 0x7f49400ff510 2024-07-16T10:04:16.428 INFO:teuthology.orchestra.run.smithi156.stdout:{ 2024-07-16T10:04:16.428 INFO:teuthology.orchestra.run.smithi156.stdout: "mon": { 2024-07-16T10:04:16.428 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph version 19.0.0-3495-ga6ca461b (a6ca461be7cc1aff3658c2c6df4cff975a24449a) squid (rc)": 2 2024-07-16T10:04:16.428 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-07-16T10:04:16.428 INFO:teuthology.orchestra.run.smithi156.stdout: "mgr": { 2024-07-16T10:04:16.428 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph version 19.0.0-3495-ga6ca461b (a6ca461be7cc1aff3658c2c6df4cff975a24449a) squid (rc)": 2 2024-07-16T10:04:16.428 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-07-16T10:04:16.429 INFO:teuthology.orchestra.run.smithi156.stdout: "osd": { 2024-07-16T10:04:16.429 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph version 19.0.0-3495-ga6ca461b (a6ca461be7cc1aff3658c2c6df4cff975a24449a) squid (rc)": 6 2024-07-16T10:04:16.429 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-07-16T10:04:16.429 INFO:teuthology.orchestra.run.smithi156.stdout: "mds": { 2024-07-16T10:04:16.429 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph version 19.0.0-3495-ga6ca461b (a6ca461be7cc1aff3658c2c6df4cff975a24449a) squid (rc)": 4 2024-07-16T10:04:16.429 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-07-16T10:04:16.429 INFO:teuthology.orchestra.run.smithi156.stdout: "overall": { 2024-07-16T10:04:16.429 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph version 19.0.0-3495-ga6ca461b (a6ca461be7cc1aff3658c2c6df4cff975a24449a) squid (rc)": 14 2024-07-16T10:04:16.429 INFO:teuthology.orchestra.run.smithi156.stdout: } 2024-07-16T10:04:16.429 INFO:teuthology.orchestra.run.smithi156.stdout:} 2024-07-16T10:04:16.430 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:16.428+0000 7f4948267640 1 -- 172.21.15.156:0/2394423785 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f492c06e0b0 msgr2=0x7f492c070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:04:16.430 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:16.428+0000 7f4948267640 1 --2- 172.21.15.156:0/2394423785 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f492c06e0b0 0x7f492c070560 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f49400fc790 tx=0x7f493c00b040 comp rx=0 tx=0).stop 2024-07-16T10:04:16.430 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:16.428+0000 7f4948267640 1 -- 172.21.15.156:0/2394423785 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f49400ff510 msgr2=0x7f49400faed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:04:16.430 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:16.428+0000 7f4948267640 1 --2- 172.21.15.156:0/2394423785 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f49400ff510 0x7f49400faed0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f4930009ae0 tx=0x7f4930009da0 comp rx=0 tx=0).stop 2024-07-16T10:04:16.430 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:16.428+0000 7f4948267640 1 -- 172.21.15.156:0/2394423785 shutdown_connections 2024-07-16T10:04:16.431 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:16.428+0000 7f4948267640 1 --2- 172.21.15.156:0/2394423785 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f492c06e0b0 0x7f492c070560 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:04:16.431 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:16.428+0000 7f4948267640 1 --2- 172.21.15.156:0/2394423785 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f49400ff510 0x7f49400faed0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:04:16.431 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:16.428+0000 7f4948267640 1 --2- 172.21.15.156:0/2394423785 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f49400664c0 0x7f49400fc9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:04:16.431 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:16.428+0000 7f4948267640 1 -- 172.21.15.156:0/2394423785 >> 172.21.15.156:0/2394423785 conn(0x7f49400f6a20 msgr2=0x7f49400f6e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T10:04:16.431 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:16.428+0000 7f4948267640 1 -- 172.21.15.156:0/2394423785 shutdown_connections 2024-07-16T10:04:16.431 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:16.428+0000 7f4948267640 1 -- 172.21.15.156:0/2394423785 wait complete. 2024-07-16T10:04:16.512 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:16.507+0000 7f45d5e05640 1 -- 172.21.15.156:0/4218100756 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f45d00fc360 msgr2=0x7f45d00fc770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:04:16.512 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:16.507+0000 7f45d5e05640 1 --2- 172.21.15.156:0/4218100756 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f45d00fc360 0x7f45d00fc770 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f45c00048d0 tx=0x7f45c0009da0 comp rx=0 tx=0).stop 2024-07-16T10:04:16.512 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:16.507+0000 7f45d5e05640 1 -- 172.21.15.156:0/4218100756 shutdown_connections 2024-07-16T10:04:16.512 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:16.507+0000 7f45d5e05640 1 --2- 172.21.15.156:0/4218100756 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f45d00fd630 0x7f45d00fdaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:04:16.512 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:16.507+0000 7f45d5e05640 1 --2- 172.21.15.156:0/4218100756 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f45d00fc360 0x7f45d00fc770 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:04:16.512 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:16.507+0000 7f45d5e05640 1 -- 172.21.15.156:0/4218100756 >> 172.21.15.156:0/4218100756 conn(0x7f45d00f7b40 msgr2=0x7f45d00f9f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T10:04:16.512 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:16.507+0000 7f45d5e05640 1 -- 172.21.15.156:0/4218100756 shutdown_connections 2024-07-16T10:04:16.512 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:16.507+0000 7f45d5e05640 1 -- 172.21.15.156:0/4218100756 wait complete. 2024-07-16T10:04:16.512 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:16.508+0000 7f45d5e05640 1 Processor -- start 2024-07-16T10:04:16.512 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:16.508+0000 7f45d5e05640 1 -- start start 2024-07-16T10:04:16.512 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:16.508+0000 7f45d5e05640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f45d00fc360 0x7f45d0071330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:04:16.513 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:16.508+0000 7f45d5e05640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f45d00fd630 0x7f45d006f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:04:16.513 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:16.508+0000 7f45d5e05640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45d006fdb0 con 0x7f45d00fd630 2024-07-16T10:04:16.513 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:16.508+0000 7f45d5e05640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45d006fef0 con 0x7f45d00fc360 2024-07-16T10:04:16.513 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:16.508+0000 7f45c7fff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f45d00fd630 0x7f45d006f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:04:16.513 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:16.508+0000 7f45c7fff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f45d00fd630 0x7f45d006f870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:47190/0 (socket says 172.21.15.156:47190) 2024-07-16T10:04:16.513 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:16.508+0000 7f45c7fff640 1 -- 172.21.15.156:0/1596977631 learned_addr learned my addr 172.21.15.156:0/1596977631 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T10:04:16.513 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:16.508+0000 7f45cf7fe640 1 --2- 172.21.15.156:0/1596977631 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f45d00fc360 0x7f45d0071330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:04:16.513 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:16.509+0000 7f45c7fff640 1 -- 172.21.15.156:0/1596977631 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f45d00fc360 msgr2=0x7f45d0071330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:04:16.513 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:16.509+0000 7f45c7fff640 1 --2- 172.21.15.156:0/1596977631 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f45d00fc360 0x7f45d0071330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:04:16.513 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:16.509+0000 7f45c7fff640 1 -- 172.21.15.156:0/1596977631 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45c0005040 con 0x7f45d00fd630 2024-07-16T10:04:16.513 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:16.509+0000 7f45c7fff640 1 --2- 172.21.15.156:0/1596977631 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f45d00fd630 0x7f45d006f870 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f45b8002a50 tx=0x7f45b8004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T10:04:16.513 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:16.509+0000 7f45ccff9640 1 -- 172.21.15.156:0/1596977631 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f45b8007c30 con 0x7f45d00fd630 2024-07-16T10:04:16.513 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:16.509+0000 7f45ccff9640 1 -- 172.21.15.156:0/1596977631 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f45b800f040 con 0x7f45d00fd630 2024-07-16T10:04:16.514 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:16.509+0000 7f45d5e05640 1 -- 172.21.15.156:0/1596977631 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45d00701d0 con 0x7f45d00fd630 2024-07-16T10:04:16.514 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:16.509+0000 7f45ccff9640 1 -- 172.21.15.156:0/1596977631 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f45b800dad0 con 0x7f45d00fd630 2024-07-16T10:04:16.514 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:16.509+0000 7f45d5e05640 1 -- 172.21.15.156:0/1596977631 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45d0115040 con 0x7f45d00fd630 2024-07-16T10:04:16.515 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:16.510+0000 7f45d5e05640 1 -- 172.21.15.156:0/1596977631 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45d00fb7c0 con 0x7f45d00fd630 2024-07-16T10:04:16.515 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:16.510+0000 7f45ccff9640 1 -- 172.21.15.156:0/1596977631 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 38) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f45b800dd40 con 0x7f45d00fd630 2024-07-16T10:04:16.515 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:16.511+0000 7f45ccff9640 1 --2- 172.21.15.156:0/1596977631 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f45bc06e0b0 0x7f45bc070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:04:16.515 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:16.511+0000 7f45ccff9640 1 -- 172.21.15.156:0/1596977631 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f45b808d3f0 con 0x7f45d00fd630 2024-07-16T10:04:16.515 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:16.511+0000 7f45cf7fe640 1 --2- 172.21.15.156:0/1596977631 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f45bc06e0b0 0x7f45bc070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:04:16.515 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:16.511+0000 7f45cf7fe640 1 --2- 172.21.15.156:0/1596977631 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f45bc06e0b0 0x7f45bc070560 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f45c00043d0 tx=0x7f45c0035040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T10:04:16.517 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:16.515+0000 7f45ccff9640 1 -- 172.21.15.156:0/1596977631 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187716 (secure 0 0 0) 0x7f45b8056770 con 0x7f45d00fd630 2024-07-16T10:04:16.738 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:16.735+0000 7f45d5e05640 1 -- 172.21.15.156:0/1596977631 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f45d004b530 con 0x7f45d00fd630 2024-07-16T10:04:16.738 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:16.735+0000 7f45ccff9640 1 -- 172.21.15.156:0/1596977631 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 45 v45) v1 ==== 76+0+1972 (secure 0 0 0) 0x7f45b8059e30 con 0x7f45d00fd630 2024-07-16T10:04:16.739 INFO:teuthology.orchestra.run.smithi156.stdout:e45 2024-07-16T10:04:16.739 INFO:teuthology.orchestra.run.smithi156.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-07-16T10:04:16.739 INFO:teuthology.orchestra.run.smithi156.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-07-16T10:04:16.739 INFO:teuthology.orchestra.run.smithi156.stdout:legacy client fscid: 1 2024-07-16T10:04:16.739 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-07-16T10:04:16.739 INFO:teuthology.orchestra.run.smithi156.stdout:Filesystem 'cephfs' (1) 2024-07-16T10:04:16.739 INFO:teuthology.orchestra.run.smithi156.stdout:fs_name cephfs 2024-07-16T10:04:16.739 INFO:teuthology.orchestra.run.smithi156.stdout:epoch 45 2024-07-16T10:04:16.739 INFO:teuthology.orchestra.run.smithi156.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-07-16T10:04:16.739 INFO:teuthology.orchestra.run.smithi156.stdout:created 2024-07-16T09:52:55.214975+0000 2024-07-16T10:04:16.739 INFO:teuthology.orchestra.run.smithi156.stdout:modified 2024-07-16T10:03:55.772577+0000 2024-07-16T10:04:16.739 INFO:teuthology.orchestra.run.smithi156.stdout:tableserver 0 2024-07-16T10:04:16.739 INFO:teuthology.orchestra.run.smithi156.stdout:root 0 2024-07-16T10:04:16.740 INFO:teuthology.orchestra.run.smithi156.stdout:session_timeout 60 2024-07-16T10:04:16.740 INFO:teuthology.orchestra.run.smithi156.stdout:session_autoclose 300 2024-07-16T10:04:16.740 INFO:teuthology.orchestra.run.smithi156.stdout:max_file_size 1099511627776 2024-07-16T10:04:16.740 INFO:teuthology.orchestra.run.smithi156.stdout:max_xattr_size 65536 2024-07-16T10:04:16.740 INFO:teuthology.orchestra.run.smithi156.stdout:required_client_features {} 2024-07-16T10:04:16.740 INFO:teuthology.orchestra.run.smithi156.stdout:last_failure 0 2024-07-16T10:04:16.740 INFO:teuthology.orchestra.run.smithi156.stdout:last_failure_osd_epoch 82 2024-07-16T10:04:16.740 INFO:teuthology.orchestra.run.smithi156.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-07-16T10:04:16.740 INFO:teuthology.orchestra.run.smithi156.stdout:max_mds 2 2024-07-16T10:04:16.740 INFO:teuthology.orchestra.run.smithi156.stdout:in 0,1 2024-07-16T10:04:16.740 INFO:teuthology.orchestra.run.smithi156.stdout:up {0=34438,1=34470} 2024-07-16T10:04:16.740 INFO:teuthology.orchestra.run.smithi156.stdout:failed 2024-07-16T10:04:16.741 INFO:teuthology.orchestra.run.smithi156.stdout:damaged 2024-07-16T10:04:16.741 INFO:teuthology.orchestra.run.smithi156.stdout:stopped 2024-07-16T10:04:16.741 INFO:teuthology.orchestra.run.smithi156.stdout:data_pools [3] 2024-07-16T10:04:16.741 INFO:teuthology.orchestra.run.smithi156.stdout:metadata_pool 2 2024-07-16T10:04:16.741 INFO:teuthology.orchestra.run.smithi156.stdout:inline_data disabled 2024-07-16T10:04:16.741 INFO:teuthology.orchestra.run.smithi156.stdout:balancer 2024-07-16T10:04:16.741 INFO:teuthology.orchestra.run.smithi156.stdout:bal_rank_mask -1 2024-07-16T10:04:16.741 INFO:teuthology.orchestra.run.smithi156.stdout:standby_count_wanted 1 2024-07-16T10:04:16.741 INFO:teuthology.orchestra.run.smithi156.stdout:qdb_cluster leader: 34438 members: 34470,34438 2024-07-16T10:04:16.741 INFO:teuthology.orchestra.run.smithi156.stdout:[mds.cephfs.smithi156.rdibys{0:34438} state up:active seq 9 join_fscid=1 addr [v2:172.21.15.156:6826/2852996640,v1:172.21.15.156:6827/2852996640] compat {c=[1],r=[1],i=[1fff]}] 2024-07-16T10:04:16.741 INFO:teuthology.orchestra.run.smithi156.stdout:[mds.cephfs.smithi176.dvvbud{0:44347} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.176:6824/1700587296,v1:172.21.15.176:6825/1700587296] compat {c=[1],r=[1],i=[1fff]}] 2024-07-16T10:04:16.741 INFO:teuthology.orchestra.run.smithi156.stdout:[mds.cephfs.smithi156.evwwaz{1:34470} state up:active seq 11 join_fscid=1 addr [v2:172.21.15.156:6828/1683974145,v1:172.21.15.156:6829/1683974145] compat {c=[1],r=[1],i=[1fff]}] 2024-07-16T10:04:16.741 INFO:teuthology.orchestra.run.smithi156.stdout:[mds.cephfs.smithi176.wahbsi{1:44373} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.176:6826/2277003188,v1:172.21.15.176:6827/2277003188] compat {c=[1],r=[1],i=[1fff]}] 2024-07-16T10:04:16.742 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-07-16T10:04:16.742 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-07-16T10:04:16.742 INFO:teuthology.orchestra.run.smithi156.stderr:dumped fsmap epoch 45 2024-07-16T10:04:16.742 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:16.738+0000 7f45d5e05640 1 -- 172.21.15.156:0/1596977631 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f45bc06e0b0 msgr2=0x7f45bc070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:04:16.742 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:16.738+0000 7f45d5e05640 1 --2- 172.21.15.156:0/1596977631 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f45bc06e0b0 0x7f45bc070560 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f45c00043d0 tx=0x7f45c0035040 comp rx=0 tx=0).stop 2024-07-16T10:04:16.743 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:16.738+0000 7f45d5e05640 1 -- 172.21.15.156:0/1596977631 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f45d00fd630 msgr2=0x7f45d006f870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:04:16.743 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:16.738+0000 7f45d5e05640 1 --2- 172.21.15.156:0/1596977631 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f45d00fd630 0x7f45d006f870 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f45b8002a50 tx=0x7f45b8004600 comp rx=0 tx=0).stop 2024-07-16T10:04:16.743 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:16.738+0000 7f45d5e05640 1 -- 172.21.15.156:0/1596977631 shutdown_connections 2024-07-16T10:04:16.743 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:16.738+0000 7f45d5e05640 1 --2- 172.21.15.156:0/1596977631 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f45bc06e0b0 0x7f45bc070560 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:04:16.743 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:16.738+0000 7f45d5e05640 1 --2- 172.21.15.156:0/1596977631 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f45d00fd630 0x7f45d006f870 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:04:16.743 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:16.738+0000 7f45d5e05640 1 --2- 172.21.15.156:0/1596977631 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f45d00fc360 0x7f45d0071330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:04:16.743 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:16.738+0000 7f45d5e05640 1 -- 172.21.15.156:0/1596977631 >> 172.21.15.156:0/1596977631 conn(0x7f45d00f7b40 msgr2=0x7f45d00f9220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T10:04:16.743 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:16.738+0000 7f45d5e05640 1 -- 172.21.15.156:0/1596977631 shutdown_connections 2024-07-16T10:04:16.743 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:16.738+0000 7f45d5e05640 1 -- 172.21.15.156:0/1596977631 wait complete. 2024-07-16T10:04:16.831 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:16.828+0000 7f2c5cb9e640 1 -- 172.21.15.156:0/3507238004 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2c580fd470 msgr2=0x7f2c580fd8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:04:16.831 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:16.828+0000 7f2c5cb9e640 1 --2- 172.21.15.156:0/3507238004 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2c580fd470 0x7f2c580fd8e0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f2c4c0048d0 tx=0x7f2c4c009da0 comp rx=0 tx=0).stop 2024-07-16T10:04:16.832 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:16.828+0000 7f2c5cb9e640 1 -- 172.21.15.156:0/3507238004 shutdown_connections 2024-07-16T10:04:16.832 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:16.828+0000 7f2c5cb9e640 1 --2- 172.21.15.156:0/3507238004 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2c580fd470 0x7f2c580fd8e0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:04:16.832 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:16.828+0000 7f2c5cb9e640 1 --2- 172.21.15.156:0/3507238004 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2c580fc1a0 0x7f2c580fc5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:04:16.832 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:16.828+0000 7f2c5cb9e640 1 -- 172.21.15.156:0/3507238004 >> 172.21.15.156:0/3507238004 conn(0x7f2c580f79a0 msgr2=0x7f2c580f9db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T10:04:16.832 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:16.828+0000 7f2c5cb9e640 1 -- 172.21.15.156:0/3507238004 shutdown_connections 2024-07-16T10:04:16.832 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:16.828+0000 7f2c5cb9e640 1 -- 172.21.15.156:0/3507238004 wait complete. 2024-07-16T10:04:16.832 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:16.828+0000 7f2c5cb9e640 1 Processor -- start 2024-07-16T10:04:16.832 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:16.828+0000 7f2c5cb9e640 1 -- start start 2024-07-16T10:04:16.832 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:16.829+0000 7f2c5cb9e640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2c580fc1a0 0x7f2c58071380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:04:16.832 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:16.829+0000 7f2c5cb9e640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2c580fd470 0x7f2c5806f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:04:16.832 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:16.829+0000 7f2c5cb9e640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c5806fe00 con 0x7f2c580fc1a0 2024-07-16T10:04:16.832 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:16.829+0000 7f2c5cb9e640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c5806ff40 con 0x7f2c580fd470 2024-07-16T10:04:16.833 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:16.829+0000 7f2c577fe640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2c580fc1a0 0x7f2c58071380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:04:16.833 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:16.829+0000 7f2c577fe640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2c580fc1a0 0x7f2c58071380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:47216/0 (socket says 172.21.15.156:47216) 2024-07-16T10:04:16.833 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:16.829+0000 7f2c577fe640 1 -- 172.21.15.156:0/1584516479 learned_addr learned my addr 172.21.15.156:0/1584516479 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T10:04:16.833 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:16.829+0000 7f2c56ffd640 1 --2- 172.21.15.156:0/1584516479 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2c580fd470 0x7f2c5806f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:04:16.833 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:16.829+0000 7f2c577fe640 1 -- 172.21.15.156:0/1584516479 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2c580fd470 msgr2=0x7f2c5806f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:04:16.833 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:16.829+0000 7f2c577fe640 1 --2- 172.21.15.156:0/1584516479 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2c580fd470 0x7f2c5806f870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:04:16.833 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:16.829+0000 7f2c577fe640 1 -- 172.21.15.156:0/1584516479 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c4c005040 con 0x7f2c580fc1a0 2024-07-16T10:04:16.834 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:16.829+0000 7f2c577fe640 1 --2- 172.21.15.156:0/1584516479 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2c580fc1a0 0x7f2c58071380 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f2c4800c830 tx=0x7f2c48004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T10:04:16.834 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:16.829+0000 7f2c3ffff640 1 -- 172.21.15.156:0/1584516479 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2c4800a7d0 con 0x7f2c580fc1a0 2024-07-16T10:04:16.834 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:16.829+0000 7f2c3ffff640 1 -- 172.21.15.156:0/1584516479 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2c4800ae00 con 0x7f2c580fc1a0 2024-07-16T10:04:16.835 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:16.829+0000 7f2c5cb9e640 1 -- 172.21.15.156:0/1584516479 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c58070220 con 0x7f2c580fc1a0 2024-07-16T10:04:16.835 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:16.829+0000 7f2c3ffff640 1 -- 172.21.15.156:0/1584516479 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2c480113a0 con 0x7f2c580fc1a0 2024-07-16T10:04:16.835 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:16.830+0000 7f2c5cb9e640 1 -- 172.21.15.156:0/1584516479 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c58114dc0 con 0x7f2c580fc1a0 2024-07-16T10:04:16.835 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:16.831+0000 7f2c5cb9e640 1 -- 172.21.15.156:0/1584516479 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c5805eb80 con 0x7f2c580fc1a0 2024-07-16T10:04:16.835 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:16.831+0000 7f2c3ffff640 1 -- 172.21.15.156:0/1584516479 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 38) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f2c48012400 con 0x7f2c580fc1a0 2024-07-16T10:04:16.835 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:16.832+0000 7f2c3ffff640 1 --2- 172.21.15.156:0/1584516479 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f2c4006e0b0 0x7f2c40070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:04:16.835 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:16.832+0000 7f2c3ffff640 1 -- 172.21.15.156:0/1584516479 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f2c4808dcf0 con 0x7f2c580fc1a0 2024-07-16T10:04:16.835 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:16.832+0000 7f2c56ffd640 1 --2- 172.21.15.156:0/1584516479 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f2c4006e0b0 0x7f2c40070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:04:16.835 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:16.832+0000 7f2c56ffd640 1 --2- 172.21.15.156:0/1584516479 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f2c4006e0b0 0x7f2c40070560 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f2c4c000c00 tx=0x7f2c4c035040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T10:04:16.842 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:16.840+0000 7f2c3ffff640 1 -- 172.21.15.156:0/1584516479 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187716 (secure 0 0 0) 0x7f2c48057040 con 0x7f2c580fc1a0 2024-07-16T10:04:17.058 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:17.056+0000 7f2c5cb9e640 1 -- 172.21.15.156:0/1584516479 --> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f2c58101790 con 0x7f2c4006e0b0 2024-07-16T10:04:17.059 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:17.057+0000 7f2c3ffff640 1 -- 172.21.15.156:0/1584516479 <== mgr.34104 v2:172.21.15.156:6800/3367765292 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+438 (secure 0 0 0) 0x7f2c58101790 con 0x7f2c4006e0b0 2024-07-16T10:04:17.060 INFO:teuthology.orchestra.run.smithi156.stdout:{ 2024-07-16T10:04:17.060 INFO:teuthology.orchestra.run.smithi156.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:c15ec3aa8e6c0a9c12ee24ff79f3e7dd4d01b3d1a702dfeb7d7fff9b0811a393", 2024-07-16T10:04:17.060 INFO:teuthology.orchestra.run.smithi156.stdout: "in_progress": true, 2024-07-16T10:04:17.060 INFO:teuthology.orchestra.run.smithi156.stdout: "which": "Upgrading all daemon types on all hosts", 2024-07-16T10:04:17.060 INFO:teuthology.orchestra.run.smithi156.stdout: "services_complete": [ 2024-07-16T10:04:17.060 INFO:teuthology.orchestra.run.smithi156.stdout: "mgr", 2024-07-16T10:04:17.060 INFO:teuthology.orchestra.run.smithi156.stdout: "crash", 2024-07-16T10:04:17.060 INFO:teuthology.orchestra.run.smithi156.stdout: "mds", 2024-07-16T10:04:17.060 INFO:teuthology.orchestra.run.smithi156.stdout: "mon", 2024-07-16T10:04:17.060 INFO:teuthology.orchestra.run.smithi156.stdout: "osd" 2024-07-16T10:04:17.060 INFO:teuthology.orchestra.run.smithi156.stdout: ], 2024-07-16T10:04:17.061 INFO:teuthology.orchestra.run.smithi156.stdout: "progress": "16/21 daemons upgraded", 2024-07-16T10:04:17.061 INFO:teuthology.orchestra.run.smithi156.stdout: "message": "Currently upgrading prometheus daemons", 2024-07-16T10:04:17.061 INFO:teuthology.orchestra.run.smithi156.stdout: "is_paused": false 2024-07-16T10:04:17.061 INFO:teuthology.orchestra.run.smithi156.stdout:} 2024-07-16T10:04:17.062 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:17.060+0000 7f2c5cb9e640 1 -- 172.21.15.156:0/1584516479 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f2c4006e0b0 msgr2=0x7f2c40070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:04:17.062 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:17.060+0000 7f2c5cb9e640 1 --2- 172.21.15.156:0/1584516479 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f2c4006e0b0 0x7f2c40070560 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f2c4c000c00 tx=0x7f2c4c035040 comp rx=0 tx=0).stop 2024-07-16T10:04:17.063 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:17.060+0000 7f2c5cb9e640 1 -- 172.21.15.156:0/1584516479 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2c580fc1a0 msgr2=0x7f2c58071380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:04:17.063 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:17.060+0000 7f2c5cb9e640 1 --2- 172.21.15.156:0/1584516479 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2c580fc1a0 0x7f2c58071380 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f2c4800c830 tx=0x7f2c48004430 comp rx=0 tx=0).stop 2024-07-16T10:04:17.063 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:17.060+0000 7f2c5cb9e640 1 -- 172.21.15.156:0/1584516479 shutdown_connections 2024-07-16T10:04:17.063 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:17.060+0000 7f2c5cb9e640 1 --2- 172.21.15.156:0/1584516479 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f2c4006e0b0 0x7f2c40070560 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:04:17.063 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:17.060+0000 7f2c5cb9e640 1 --2- 172.21.15.156:0/1584516479 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2c580fd470 0x7f2c5806f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:04:17.063 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:17.060+0000 7f2c5cb9e640 1 --2- 172.21.15.156:0/1584516479 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2c580fc1a0 0x7f2c58071380 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:04:17.063 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:17.060+0000 7f2c5cb9e640 1 -- 172.21.15.156:0/1584516479 >> 172.21.15.156:0/1584516479 conn(0x7f2c580f79a0 msgr2=0x7f2c580f9040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T10:04:17.063 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:17.060+0000 7f2c5cb9e640 1 -- 172.21.15.156:0/1584516479 shutdown_connections 2024-07-16T10:04:17.063 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:17.060+0000 7f2c5cb9e640 1 -- 172.21.15.156:0/1584516479 wait complete. 2024-07-16T10:04:17.150 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:17.146+0000 7f119cc6f640 1 -- 172.21.15.156:0/3803250098 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f11980ff510 msgr2=0x7f11981018f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:04:17.150 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:17.146+0000 7f119cc6f640 1 --2- 172.21.15.156:0/3803250098 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f11980ff510 0x7f11981018f0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f118c0048d0 tx=0x7f118c009da0 comp rx=0 tx=0).stop 2024-07-16T10:04:17.150 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:17.146+0000 7f119cc6f640 1 -- 172.21.15.156:0/3803250098 shutdown_connections 2024-07-16T10:04:17.150 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:17.146+0000 7f119cc6f640 1 --2- 172.21.15.156:0/3803250098 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f11980ff510 0x7f11981018f0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:04:17.150 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:17.146+0000 7f119cc6f640 1 --2- 172.21.15.156:0/3803250098 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f11980659d0 0x7f11980fefd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:04:17.151 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:17.146+0000 7f119cc6f640 1 -- 172.21.15.156:0/3803250098 >> 172.21.15.156:0/3803250098 conn(0x7f11980f6a20 msgr2=0x7f11980f8e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T10:04:17.151 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:17.146+0000 7f119cc6f640 1 -- 172.21.15.156:0/3803250098 shutdown_connections 2024-07-16T10:04:17.151 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:17.146+0000 7f119cc6f640 1 -- 172.21.15.156:0/3803250098 wait complete. 2024-07-16T10:04:17.151 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:17.147+0000 7f119cc6f640 1 Processor -- start 2024-07-16T10:04:17.151 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:17.147+0000 7f119cc6f640 1 -- start start 2024-07-16T10:04:17.151 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:17.147+0000 7f119cc6f640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f11980659d0 0x7f119810cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:04:17.151 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:17.147+0000 7f119cc6f640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f11980ff510 0x7f119810d500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:04:17.151 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:17.147+0000 7f119cc6f640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f119810db50 con 0x7f11980659d0 2024-07-16T10:04:17.151 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:17.147+0000 7f119cc6f640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f119810e320 con 0x7f11980ff510 2024-07-16T10:04:17.151 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:17.147+0000 7f119659c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f11980659d0 0x7f119810cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:04:17.151 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:17.147+0000 7f119659c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f11980659d0 0x7f119810cfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:47228/0 (socket says 172.21.15.156:47228) 2024-07-16T10:04:17.151 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:17.147+0000 7f119659c640 1 -- 172.21.15.156:0/3002921826 learned_addr learned my addr 172.21.15.156:0/3002921826 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T10:04:17.152 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:17.147+0000 7f1195d9b640 1 --2- 172.21.15.156:0/3002921826 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f11980ff510 0x7f119810d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:04:17.152 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:17.147+0000 7f119659c640 1 -- 172.21.15.156:0/3002921826 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f11980ff510 msgr2=0x7f119810d500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:04:17.152 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:17.147+0000 7f119659c640 1 --2- 172.21.15.156:0/3002921826 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f11980ff510 0x7f119810d500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:04:17.152 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:17.147+0000 7f119659c640 1 -- 172.21.15.156:0/3002921826 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f118c005040 con 0x7f11980659d0 2024-07-16T10:04:17.152 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:17.147+0000 7f1195d9b640 1 --2- 172.21.15.156:0/3002921826 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f11980ff510 0x7f119810d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-16T10:04:17.152 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:17.147+0000 7f119659c640 1 --2- 172.21.15.156:0/3002921826 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f11980659d0 0x7f119810cfc0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f118800cae0 tx=0x7f1188007ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T10:04:17.152 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:17.148+0000 7f1186ffd640 1 -- 172.21.15.156:0/3002921826 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f118800a9d0 con 0x7f11980659d0 2024-07-16T10:04:17.152 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:17.148+0000 7f119cc6f640 1 -- 172.21.15.156:0/3002921826 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f119810e5b0 con 0x7f11980659d0 2024-07-16T10:04:17.152 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:17.148+0000 7f1186ffd640 1 -- 172.21.15.156:0/3002921826 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1188012440 con 0x7f11980659d0 2024-07-16T10:04:17.152 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:17.148+0000 7f1186ffd640 1 -- 172.21.15.156:0/3002921826 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f11880115c0 con 0x7f11980659d0 2024-07-16T10:04:17.152 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:17.148+0000 7f119cc6f640 1 -- 172.21.15.156:0/3002921826 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f119810eb50 con 0x7f11980659d0 2024-07-16T10:04:17.153 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:17.149+0000 7f119cc6f640 1 -- 172.21.15.156:0/3002921826 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11980fa6a0 con 0x7f11980659d0 2024-07-16T10:04:17.153 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:17.149+0000 7f1186ffd640 1 -- 172.21.15.156:0/3002921826 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 38) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f118800ab30 con 0x7f11980659d0 2024-07-16T10:04:17.153 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:17.150+0000 7f1186ffd640 1 --2- 172.21.15.156:0/3002921826 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f118006e0b0 0x7f1180070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:04:17.154 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:17.150+0000 7f1186ffd640 1 -- 172.21.15.156:0/3002921826 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f118808e340 con 0x7f11980659d0 2024-07-16T10:04:17.154 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:17.150+0000 7f1195d9b640 1 --2- 172.21.15.156:0/3002921826 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f118006e0b0 0x7f1180070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:04:17.154 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:17.150+0000 7f1195d9b640 1 --2- 172.21.15.156:0/3002921826 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f118006e0b0 0x7f1180070560 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f118c0047f0 tx=0x7f118c009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T10:04:17.156 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:17.154+0000 7f1186ffd640 1 -- 172.21.15.156:0/3002921826 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187716 (secure 0 0 0) 0x7f1188057690 con 0x7f11980659d0 2024-07-16T10:04:17.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:04:17 smithi176 ceph-mon[81689]: from='client.34502 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-16T10:04:17.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:04:17 smithi176 ceph-mon[81689]: from='client.34506 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-16T10:04:17.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:04:17 smithi176 ceph-mon[81689]: from='client.34510 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-07-16T10:04:17.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:04:17 smithi176 ceph-mon[81689]: from='client.? 172.21.15.156:0/2394423785' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:04:17.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:04:17 smithi176 ceph-mon[81689]: from='client.? 172.21.15.156:0/1596977631' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-07-16T10:04:17.411 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:17.408+0000 7f119cc6f640 1 -- 172.21.15.156:0/3002921826 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f119804b530 con 0x7f11980659d0 2024-07-16T10:04:17.411 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:04:17 smithi156 ceph-mon[93066]: from='client.34502 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-16T10:04:17.411 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:04:17 smithi156 ceph-mon[93066]: from='client.34506 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-16T10:04:17.411 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:04:17 smithi156 ceph-mon[93066]: from='client.34510 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-07-16T10:04:17.411 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:04:17 smithi156 ceph-mon[93066]: from='client.? 172.21.15.156:0/2394423785' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:04:17.411 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:04:17 smithi156 ceph-mon[93066]: from='client.? 172.21.15.156:0/1596977631' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-07-16T10:04:17.412 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:17.408+0000 7f1186ffd640 1 -- 172.21.15.156:0/3002921826 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7f118805ad50 con 0x7f11980659d0 2024-07-16T10:04:17.412 INFO:teuthology.orchestra.run.smithi156.stdout:HEALTH_OK 2024-07-16T10:04:17.413 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:17.411+0000 7f119cc6f640 1 -- 172.21.15.156:0/3002921826 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f118006e0b0 msgr2=0x7f1180070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:04:17.413 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:17.411+0000 7f119cc6f640 1 --2- 172.21.15.156:0/3002921826 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f118006e0b0 0x7f1180070560 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f118c0047f0 tx=0x7f118c009da0 comp rx=0 tx=0).stop 2024-07-16T10:04:17.414 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:17.411+0000 7f119cc6f640 1 -- 172.21.15.156:0/3002921826 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f11980659d0 msgr2=0x7f119810cfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:04:17.414 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:17.411+0000 7f119cc6f640 1 --2- 172.21.15.156:0/3002921826 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f11980659d0 0x7f119810cfc0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f118800cae0 tx=0x7f1188007ed0 comp rx=0 tx=0).stop 2024-07-16T10:04:17.414 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:17.411+0000 7f119cc6f640 1 -- 172.21.15.156:0/3002921826 shutdown_connections 2024-07-16T10:04:17.414 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:17.411+0000 7f119cc6f640 1 --2- 172.21.15.156:0/3002921826 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f118006e0b0 0x7f1180070560 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:04:17.414 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:17.411+0000 7f119cc6f640 1 --2- 172.21.15.156:0/3002921826 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f11980ff510 0x7f119810d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:04:17.414 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:17.411+0000 7f119cc6f640 1 --2- 172.21.15.156:0/3002921826 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f11980659d0 0x7f119810cfc0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:04:17.414 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:17.411+0000 7f119cc6f640 1 -- 172.21.15.156:0/3002921826 >> 172.21.15.156:0/3002921826 conn(0x7f11980f6a20 msgr2=0x7f11980f6df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T10:04:17.414 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:17.411+0000 7f119cc6f640 1 -- 172.21.15.156:0/3002921826 shutdown_connections 2024-07-16T10:04:17.414 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:17.411+0000 7f119cc6f640 1 -- 172.21.15.156:0/3002921826 wait complete. 2024-07-16T10:04:18.040 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:04:18 smithi156 ceph-mon[93066]: pgmap v232: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 1.9 KiB/s rd, 4 op/s 2024-07-16T10:04:18.040 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:04:18 smithi156 ceph-mon[93066]: from='client.34522 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-16T10:04:18.040 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:04:18 smithi156 ceph-mon[93066]: from='client.? 172.21.15.156:0/3002921826' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-07-16T10:04:18.343 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:04:18 smithi176 ceph-mon[81689]: pgmap v232: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 1.9 KiB/s rd, 4 op/s 2024-07-16T10:04:18.343 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:04:18 smithi176 ceph-mon[81689]: from='client.34522 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-16T10:04:18.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:04:18 smithi176 ceph-mon[81689]: from='client.? 172.21.15.156:0/3002921826' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-07-16T10:04:19.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:04:19 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-16T10:04:19.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:04:19 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:04:19.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:04:19 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:04:19.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:04:19 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-16T10:04:19.418 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:04:19 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-16T10:04:19.418 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:04:19 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:04:19.418 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:04:19 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:04:19.418 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:04:19 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-16T10:04:20.283 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:04:20 smithi156 ceph-mon[93066]: pgmap v233: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-07-16T10:04:20.343 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:04:20 smithi176 ceph-mon[81689]: pgmap v233: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-07-16T10:04:22.286 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:04:22 smithi156 ceph-mon[93066]: pgmap v234: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:04:22.343 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:04:22 smithi176 ceph-mon[81689]: pgmap v234: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:04:24.225 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:04:23 smithi156 ceph-mon[93066]: pgmap v235: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:04:24.225 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:04:23 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:04:24.225 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:04:23 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:04:24.225 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:04:23 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:04:24.225 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:04:23 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:04:24.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:04:23 smithi176 ceph-mon[81689]: pgmap v235: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:04:24.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:04:23 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:04:24.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:04:23 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:04:24.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:04:23 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:04:24.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:04:23 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:04:26.343 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:04:25 smithi176 ceph-mon[81689]: pgmap v236: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-07-16T10:04:26.395 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:04:25 smithi156 ceph-mon[93066]: pgmap v236: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-07-16T10:04:27.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:04:27 smithi156 ceph-mon[93066]: pgmap v237: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:04:27.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:04:27 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:04:27.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:04:27 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:04:27.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:04:27 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-16T10:04:27.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:04:27 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-16T10:04:27.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:04:27 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:04:27.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:04:27 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-07-16T10:04:27.926 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:04:27 smithi156 ceph-mon[93066]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-07-16T10:04:27.926 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:04:27 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-16T10:04:27.926 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:04:27 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:04:27.926 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:04:27 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:04:27.926 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:04:27 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:04:27.926 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:04:27 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:04:27.926 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:04:27 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:04:27.926 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:04:27 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:04:27.926 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:04:27 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:04:27.926 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:04:27 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:04:27.926 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:04:27 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:04:27.927 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:04:27 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:04:27.927 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:04:27 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:04:27.927 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:04:27 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:04:27.927 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:04:27 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:04:27.927 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:04:27 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:04:28.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:04:27 smithi176 ceph-mon[81689]: pgmap v237: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:04:28.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:04:27 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:04:28.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:04:27 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:04:28.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:04:27 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-16T10:04:28.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:04:27 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-16T10:04:28.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:04:27 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:04:28.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:04:27 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-07-16T10:04:28.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:04:27 smithi176 ceph-mon[81689]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-07-16T10:04:28.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:04:27 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-16T10:04:28.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:04:27 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:04:28.095 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:04:27 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:04:28.095 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:04:27 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:04:28.095 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:04:27 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:04:28.095 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:04:27 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:04:28.095 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:04:27 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:04:28.095 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:04:27 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:04:28.095 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:04:27 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:04:28.095 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:04:27 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:04:28.096 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:04:27 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:04:28.096 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:04:27 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:04:28.096 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:04:27 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:04:28.096 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:04:27 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:04:28.096 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:04:27 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:04:28.995 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:04:28 smithi156 ceph-mon[93066]: Upgrade: Updating alertmanager.smithi156 2024-07-16T10:04:28.995 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:04:28 smithi156 ceph-mon[93066]: Deploying daemon alertmanager.smithi156 on smithi156 2024-07-16T10:04:29.093 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:04:28 smithi176 ceph-mon[81689]: Upgrade: Updating alertmanager.smithi156 2024-07-16T10:04:29.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:04:28 smithi176 ceph-mon[81689]: Deploying daemon alertmanager.smithi156 on smithi156 2024-07-16T10:04:30.093 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:04:29 smithi176 ceph-mon[81689]: pgmap v238: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-07-16T10:04:30.094 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:04:29 smithi156 ceph-mon[93066]: pgmap v238: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-07-16T10:04:32.093 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:04:31 smithi176 ceph-mon[81689]: pgmap v239: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:04:32.101 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:04:31 smithi156 ceph-mon[93066]: pgmap v239: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:04:34.093 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:04:33 smithi176 ceph-mon[81689]: pgmap v240: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:04:34.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:04:33 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-16T10:04:34.174 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:04:33 smithi156 ceph-mon[93066]: pgmap v240: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:04:34.175 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:04:33 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-16T10:04:35.343 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:04:35 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:04:35.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:04:35 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:04:35.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:04:35 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-16T10:04:35.405 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:04:35 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:04:35.406 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:04:35 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:04:35.406 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:04:35 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-16T10:04:36.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:04:36 smithi176 ceph-mon[81689]: pgmap v241: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-07-16T10:04:36.353 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:04:36 smithi156 ceph-mon[93066]: pgmap v241: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-07-16T10:04:38.343 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:04:38 smithi176 ceph-mon[81689]: pgmap v242: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:04:38.424 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:04:38 smithi156 ceph-mon[93066]: pgmap v242: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:04:39.730 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:04:39 smithi156 ceph-mon[93066]: pgmap v243: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-07-16T10:04:39.730 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:04:39 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:04:39.730 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:04:39 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:04:39.730 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:04:39 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:04:39.730 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:04:39 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:04:39.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:04:39 smithi176 ceph-mon[81689]: pgmap v243: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-07-16T10:04:39.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:04:39 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:04:39.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:04:39 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:04:39.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:04:39 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:04:39.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:04:39 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:04:41.843 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:04:41 smithi176 ceph-mon[81689]: pgmap v244: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:04:41.900 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:04:41 smithi156 ceph-mon[93066]: pgmap v244: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:04:43.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:04:43 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:04:43.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:04:43 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:04:43.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:04:43 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-16T10:04:43.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:04:43 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-16T10:04:43.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:04:43 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:04:43.426 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:04:43 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-07-16T10:04:43.426 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:04:43 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-16T10:04:43.426 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:04:43 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:04:43.426 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:04:43 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:04:43.426 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:04:43 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:04:43.426 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:04:43 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:04:43.426 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:04:43 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:04:43.426 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:04:43 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:04:43.426 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:04:43 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:04:43.426 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:04:43 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:04:43.426 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:04:43 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:04:43.427 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:04:43 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:04:43.427 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:04:43 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:04:43.427 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:04:43 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:04:43.427 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:04:43 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:04:43.427 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:04:43 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:04:43.427 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:04:43 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:04:43.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:04:43 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:04:43.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:04:43 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:04:43.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:04:43 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-16T10:04:43.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:04:43 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-16T10:04:43.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:04:43 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:04:43.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:04:43 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-07-16T10:04:43.595 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:04:43 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-16T10:04:43.595 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:04:43 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:04:43.595 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:04:43 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:04:43.595 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:04:43 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:04:43.595 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:04:43 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:04:43.595 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:04:43 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:04:43.595 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:04:43 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:04:43.595 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:04:43 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:04:43.595 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:04:43 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:04:43.595 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:04:43 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:04:43.595 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:04:43 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:04:43.595 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:04:43 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:04:43.595 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:04:43 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:04:43.595 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:04:43 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:04:43.596 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:04:43 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:04:43.596 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:04:43 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:04:44.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:04:44 smithi176 ceph-mon[81689]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-07-16T10:04:44.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:04:44 smithi176 ceph-mon[81689]: pgmap v245: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:04:44.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:04:44 smithi156 ceph-mon[93066]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-07-16T10:04:44.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:04:44 smithi156 ceph-mon[93066]: pgmap v245: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:04:45.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:04:45 smithi176 ceph-mon[81689]: Upgrade: Updating grafana.smithi156 2024-07-16T10:04:45.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:04:45 smithi176 ceph-mon[81689]: Deploying daemon grafana.smithi156 on smithi156 2024-07-16T10:04:45.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:04:45 smithi156 ceph-mon[93066]: Upgrade: Updating grafana.smithi156 2024-07-16T10:04:45.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:04:45 smithi156 ceph-mon[93066]: Deploying daemon grafana.smithi156 on smithi156 2024-07-16T10:04:46.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:04:46 smithi176 ceph-mon[81689]: pgmap v246: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-07-16T10:04:46.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:04:46 smithi156 ceph-mon[93066]: pgmap v246: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-07-16T10:04:47.499 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:47.496+0000 7ffa9bba7640 1 -- 172.21.15.156:0/3169405109 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffa940fc360 msgr2=0x7ffa940fc770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:04:47.499 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:47.496+0000 7ffa9bba7640 1 --2- 172.21.15.156:0/3169405109 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffa940fc360 0x7ffa940fc770 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7ffa8c0048d0 tx=0x7ffa8c009da0 comp rx=0 tx=0).stop 2024-07-16T10:04:47.499 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:47.496+0000 7ffa9bba7640 1 -- 172.21.15.156:0/3169405109 shutdown_connections 2024-07-16T10:04:47.499 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:47.496+0000 7ffa9bba7640 1 --2- 172.21.15.156:0/3169405109 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ffa940fd630 0x7ffa940fdaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:04:47.499 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:47.496+0000 7ffa9bba7640 1 --2- 172.21.15.156:0/3169405109 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffa940fc360 0x7ffa940fc770 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:04:47.499 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:47.496+0000 7ffa9bba7640 1 -- 172.21.15.156:0/3169405109 >> 172.21.15.156:0/3169405109 conn(0x7ffa940f7b40 msgr2=0x7ffa940f9f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T10:04:47.499 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:47.496+0000 7ffa9bba7640 1 -- 172.21.15.156:0/3169405109 shutdown_connections 2024-07-16T10:04:47.499 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:47.496+0000 7ffa9bba7640 1 -- 172.21.15.156:0/3169405109 wait complete. 2024-07-16T10:04:47.500 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:47.496+0000 7ffa9bba7640 1 Processor -- start 2024-07-16T10:04:47.500 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:47.497+0000 7ffa9bba7640 1 -- start start 2024-07-16T10:04:47.500 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:47.497+0000 7ffa9bba7640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ffa940fc360 0x7ffa9410cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:04:47.500 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:47.497+0000 7ffa9bba7640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffa940fd630 0x7ffa9410d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:04:47.500 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:47.497+0000 7ffa9bba7640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa9410db60 con 0x7ffa940fd630 2024-07-16T10:04:47.500 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:47.497+0000 7ffa9bba7640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa9410e330 con 0x7ffa940fc360 2024-07-16T10:04:47.500 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:47.497+0000 7ffa99142640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffa940fd630 0x7ffa9410d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:04:47.500 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:47.497+0000 7ffa99142640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffa940fd630 0x7ffa9410d510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:37148/0 (socket says 172.21.15.156:37148) 2024-07-16T10:04:47.500 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:47.497+0000 7ffa99142640 1 -- 172.21.15.156:0/4287399970 learned_addr learned my addr 172.21.15.156:0/4287399970 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T10:04:47.500 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:47.497+0000 7ffa99943640 1 --2- 172.21.15.156:0/4287399970 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ffa940fc360 0x7ffa9410cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:04:47.500 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:47.497+0000 7ffa99142640 1 -- 172.21.15.156:0/4287399970 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ffa940fc360 msgr2=0x7ffa9410cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:04:47.500 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:47.497+0000 7ffa99142640 1 --2- 172.21.15.156:0/4287399970 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ffa940fc360 0x7ffa9410cfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:04:47.500 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:47.497+0000 7ffa99142640 1 -- 172.21.15.156:0/4287399970 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa8c005040 con 0x7ffa940fd630 2024-07-16T10:04:47.501 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:47.497+0000 7ffa99142640 1 --2- 172.21.15.156:0/4287399970 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffa940fd630 0x7ffa9410d510 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7ffa88002a50 tx=0x7ffa88004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T10:04:47.501 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:47.498+0000 7ffa927fc640 1 -- 172.21.15.156:0/4287399970 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffa88007c30 con 0x7ffa940fd630 2024-07-16T10:04:47.501 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:47.498+0000 7ffa9bba7640 1 -- 172.21.15.156:0/4287399970 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa9410e5c0 con 0x7ffa940fd630 2024-07-16T10:04:47.501 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:47.498+0000 7ffa927fc640 1 -- 172.21.15.156:0/4287399970 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ffa8800f040 con 0x7ffa940fd630 2024-07-16T10:04:47.501 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:47.498+0000 7ffa927fc640 1 -- 172.21.15.156:0/4287399970 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffa8800db60 con 0x7ffa940fd630 2024-07-16T10:04:47.501 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:47.498+0000 7ffa9bba7640 1 -- 172.21.15.156:0/4287399970 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa9410eb60 con 0x7ffa940fd630 2024-07-16T10:04:47.502 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:47.499+0000 7ffa9bba7640 1 -- 172.21.15.156:0/4287399970 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa940fb7c0 con 0x7ffa940fd630 2024-07-16T10:04:47.502 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:47.499+0000 7ffa927fc640 1 -- 172.21.15.156:0/4287399970 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 38) v1 ==== 92205+0+0 (secure 0 0 0) 0x7ffa8800dcc0 con 0x7ffa940fd630 2024-07-16T10:04:47.502 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:47.499+0000 7ffa927fc640 1 --2- 172.21.15.156:0/4287399970 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7ffa7c06e0b0 0x7ffa7c070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:04:47.502 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:47.500+0000 7ffa927fc640 1 -- 172.21.15.156:0/4287399970 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7ffa8808dab0 con 0x7ffa940fd630 2024-07-16T10:04:47.502 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:47.500+0000 7ffa99943640 1 --2- 172.21.15.156:0/4287399970 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7ffa7c06e0b0 0x7ffa7c070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:04:47.502 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:47.500+0000 7ffa99943640 1 --2- 172.21.15.156:0/4287399970 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7ffa7c06e0b0 0x7ffa7c070560 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7ffa8c0043d0 tx=0x7ffa8c035040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T10:04:47.505 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:47.504+0000 7ffa927fc640 1 -- 172.21.15.156:0/4287399970 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187716 (secure 0 0 0) 0x7ffa88056d80 con 0x7ffa940fd630 2024-07-16T10:04:47.698 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:47.697+0000 7ffa9bba7640 1 -- 172.21.15.156:0/4287399970 --> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ffa9410ee90 con 0x7ffa7c06e0b0 2024-07-16T10:04:47.699 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:47.698+0000 7ffa927fc640 1 -- 172.21.15.156:0/4287399970 <== mgr.34104 v2:172.21.15.156:6800/3367765292 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+435 (secure 0 0 0) 0x7ffa9410ee90 con 0x7ffa7c06e0b0 2024-07-16T10:04:47.702 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:47.700+0000 7ffa9bba7640 1 -- 172.21.15.156:0/4287399970 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7ffa7c06e0b0 msgr2=0x7ffa7c070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:04:47.702 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:47.700+0000 7ffa9bba7640 1 --2- 172.21.15.156:0/4287399970 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7ffa7c06e0b0 0x7ffa7c070560 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7ffa8c0043d0 tx=0x7ffa8c035040 comp rx=0 tx=0).stop 2024-07-16T10:04:47.702 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:47.700+0000 7ffa9bba7640 1 -- 172.21.15.156:0/4287399970 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffa940fd630 msgr2=0x7ffa9410d510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:04:47.702 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:47.700+0000 7ffa9bba7640 1 --2- 172.21.15.156:0/4287399970 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffa940fd630 0x7ffa9410d510 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7ffa88002a50 tx=0x7ffa88004600 comp rx=0 tx=0).stop 2024-07-16T10:04:47.702 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:47.700+0000 7ffa9bba7640 1 -- 172.21.15.156:0/4287399970 shutdown_connections 2024-07-16T10:04:47.702 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:47.700+0000 7ffa9bba7640 1 --2- 172.21.15.156:0/4287399970 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7ffa7c06e0b0 0x7ffa7c070560 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:04:47.702 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:47.700+0000 7ffa9bba7640 1 --2- 172.21.15.156:0/4287399970 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffa940fd630 0x7ffa9410d510 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:04:47.702 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:47.700+0000 7ffa9bba7640 1 --2- 172.21.15.156:0/4287399970 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ffa940fc360 0x7ffa9410cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:04:47.702 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:47.700+0000 7ffa9bba7640 1 -- 172.21.15.156:0/4287399970 >> 172.21.15.156:0/4287399970 conn(0x7ffa940f7b40 msgr2=0x7ffa940f9220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T10:04:47.702 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:47.701+0000 7ffa9bba7640 1 -- 172.21.15.156:0/4287399970 shutdown_connections 2024-07-16T10:04:47.702 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:47.701+0000 7ffa9bba7640 1 -- 172.21.15.156:0/4287399970 wait complete. 2024-07-16T10:04:47.711 INFO:teuthology.orchestra.run.smithi156.stdout:true 2024-07-16T10:04:47.783 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:47.781+0000 7f930501a640 1 -- 172.21.15.156:0/1713036577 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f93000fb230 msgr2=0x7f93000fb640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:04:47.783 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:47.781+0000 7f930501a640 1 --2- 172.21.15.156:0/1713036577 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f93000fb230 0x7f93000fb640 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f92f00048d0 tx=0x7f92f0009da0 comp rx=0 tx=0).stop 2024-07-16T10:04:47.783 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:47.781+0000 7f930501a640 1 -- 172.21.15.156:0/1713036577 shutdown_connections 2024-07-16T10:04:47.783 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:47.781+0000 7f930501a640 1 --2- 172.21.15.156:0/1713036577 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f93000fc500 0x7f93000fc970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:04:47.783 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:47.781+0000 7f930501a640 1 --2- 172.21.15.156:0/1713036577 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f93000fb230 0x7f93000fb640 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:04:47.783 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:47.781+0000 7f930501a640 1 -- 172.21.15.156:0/1713036577 >> 172.21.15.156:0/1713036577 conn(0x7f93000f6a10 msgr2=0x7f93000f8e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T10:04:47.783 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:47.781+0000 7f930501a640 1 -- 172.21.15.156:0/1713036577 shutdown_connections 2024-07-16T10:04:47.784 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:47.781+0000 7f930501a640 1 -- 172.21.15.156:0/1713036577 wait complete. 2024-07-16T10:04:47.785 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:47.782+0000 7f930501a640 1 Processor -- start 2024-07-16T10:04:47.785 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:47.782+0000 7f930501a640 1 -- start start 2024-07-16T10:04:47.785 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:47.782+0000 7f930501a640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f93000fb230 0x7f930010fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:04:47.785 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:47.782+0000 7f930501a640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f93000fc500 0x7f9300112160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:04:47.785 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:47.782+0000 7f930501a640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9300066360 con 0x7f93000fc500 2024-07-16T10:04:47.786 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:47.782+0000 7f930501a640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93000664d0 con 0x7f93000fb230 2024-07-16T10:04:47.786 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:47.783+0000 7f92fe59c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f93000fc500 0x7f9300112160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:04:47.786 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:47.783+0000 7f92fe59c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f93000fc500 0x7f9300112160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:37160/0 (socket says 172.21.15.156:37160) 2024-07-16T10:04:47.786 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:47.783+0000 7f92fe59c640 1 -- 172.21.15.156:0/777135707 learned_addr learned my addr 172.21.15.156:0/777135707 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T10:04:47.786 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:47.783+0000 7f92fed9d640 1 --2- 172.21.15.156:0/777135707 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f93000fb230 0x7f930010fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:04:47.786 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:47.783+0000 7f92fe59c640 1 -- 172.21.15.156:0/777135707 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f93000fb230 msgr2=0x7f930010fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:04:47.786 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:47.783+0000 7f92fe59c640 1 --2- 172.21.15.156:0/777135707 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f93000fb230 0x7f930010fc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:04:47.786 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:47.783+0000 7f92fe59c640 1 -- 172.21.15.156:0/777135707 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92f0005040 con 0x7f93000fc500 2024-07-16T10:04:47.786 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:47.783+0000 7f92fe59c640 1 --2- 172.21.15.156:0/777135707 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f93000fc500 0x7f9300112160 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f92f400c790 tx=0x7f92f40043c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T10:04:47.786 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:47.783+0000 7f92e77fe640 1 -- 172.21.15.156:0/777135707 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f92f400a7a0 con 0x7f93000fc500 2024-07-16T10:04:47.786 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:47.783+0000 7f92e77fe640 1 -- 172.21.15.156:0/777135707 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f92f400add0 con 0x7f93000fc500 2024-07-16T10:04:47.786 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:47.783+0000 7f930501a640 1 -- 172.21.15.156:0/777135707 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9300112760 con 0x7f93000fc500 2024-07-16T10:04:47.786 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:47.783+0000 7f92e77fe640 1 -- 172.21.15.156:0/777135707 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f92f40115e0 con 0x7f93000fc500 2024-07-16T10:04:47.787 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:47.783+0000 7f930501a640 1 -- 172.21.15.156:0/777135707 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9300112cd0 con 0x7f93000fc500 2024-07-16T10:04:47.787 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:47.784+0000 7f930501a640 1 -- 172.21.15.156:0/777135707 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93000fa690 con 0x7f93000fc500 2024-07-16T10:04:47.787 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:47.785+0000 7f92e77fe640 1 -- 172.21.15.156:0/777135707 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 38) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f92f4012400 con 0x7f93000fc500 2024-07-16T10:04:47.787 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:47.785+0000 7f92e77fe640 1 --2- 172.21.15.156:0/777135707 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f92e806e0b0 0x7f92e8070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:04:47.788 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:47.785+0000 7f92e77fe640 1 -- 172.21.15.156:0/777135707 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f92f408dc70 con 0x7f93000fc500 2024-07-16T10:04:47.788 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:47.786+0000 7f92fed9d640 1 --2- 172.21.15.156:0/777135707 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f92e806e0b0 0x7f92e8070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:04:47.788 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:47.786+0000 7f92fed9d640 1 --2- 172.21.15.156:0/777135707 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f92e806e0b0 0x7f92e8070560 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f92f0000c00 tx=0x7f92f0035040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T10:04:47.791 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:47.790+0000 7f92e77fe640 1 -- 172.21.15.156:0/777135707 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187716 (secure 0 0 0) 0x7f92f4056fc0 con 0x7f93000fc500 2024-07-16T10:04:47.990 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:47.989+0000 7f930501a640 1 -- 172.21.15.156:0/777135707 --> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f9300100810 con 0x7f92e806e0b0 2024-07-16T10:04:47.991 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:47.990+0000 7f92e77fe640 1 -- 172.21.15.156:0/777135707 <== mgr.34104 v2:172.21.15.156:6800/3367765292 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+435 (secure 0 0 0) 0x7f9300100810 con 0x7f92e806e0b0 2024-07-16T10:04:47.994 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:47.992+0000 7f930501a640 1 -- 172.21.15.156:0/777135707 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f92e806e0b0 msgr2=0x7f92e8070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:04:47.994 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:47.992+0000 7f930501a640 1 --2- 172.21.15.156:0/777135707 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f92e806e0b0 0x7f92e8070560 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f92f0000c00 tx=0x7f92f0035040 comp rx=0 tx=0).stop 2024-07-16T10:04:47.994 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:47.992+0000 7f930501a640 1 -- 172.21.15.156:0/777135707 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f93000fc500 msgr2=0x7f9300112160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:04:47.994 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:47.992+0000 7f930501a640 1 --2- 172.21.15.156:0/777135707 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f93000fc500 0x7f9300112160 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f92f400c790 tx=0x7f92f40043c0 comp rx=0 tx=0).stop 2024-07-16T10:04:47.994 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:47.992+0000 7f930501a640 1 -- 172.21.15.156:0/777135707 shutdown_connections 2024-07-16T10:04:47.994 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:47.992+0000 7f930501a640 1 --2- 172.21.15.156:0/777135707 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f92e806e0b0 0x7f92e8070560 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:04:47.994 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:47.992+0000 7f930501a640 1 --2- 172.21.15.156:0/777135707 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f93000fc500 0x7f9300112160 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:04:47.995 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:47.992+0000 7f930501a640 1 --2- 172.21.15.156:0/777135707 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f93000fb230 0x7f930010fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:04:47.995 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:47.992+0000 7f930501a640 1 -- 172.21.15.156:0/777135707 >> 172.21.15.156:0/777135707 conn(0x7f93000f6a10 msgr2=0x7f93000f8040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T10:04:47.995 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:47.993+0000 7f930501a640 1 -- 172.21.15.156:0/777135707 shutdown_connections 2024-07-16T10:04:47.995 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:47.993+0000 7f930501a640 1 -- 172.21.15.156:0/777135707 wait complete. 2024-07-16T10:04:48.074 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:48.073+0000 7fab77db7640 1 -- 172.21.15.156:0/88134358 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fab700fda40 msgr2=0x7fab700fde50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:04:48.075 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:48.073+0000 7fab77db7640 1 --2- 172.21.15.156:0/88134358 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fab700fda40 0x7fab700fde50 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fab640048d0 tx=0x7fab64009da0 comp rx=0 tx=0).stop 2024-07-16T10:04:48.075 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:48.073+0000 7fab77db7640 1 -- 172.21.15.156:0/88134358 shutdown_connections 2024-07-16T10:04:48.075 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:48.073+0000 7fab77db7640 1 --2- 172.21.15.156:0/88134358 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fab700fe390 0x7fab700fa4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:04:48.075 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:48.073+0000 7fab77db7640 1 --2- 172.21.15.156:0/88134358 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fab700fda40 0x7fab700fde50 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:04:48.075 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:48.073+0000 7fab77db7640 1 -- 172.21.15.156:0/88134358 >> 172.21.15.156:0/88134358 conn(0x7fab700f61a0 msgr2=0x7fab700f85b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T10:04:48.075 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:48.073+0000 7fab77db7640 1 -- 172.21.15.156:0/88134358 shutdown_connections 2024-07-16T10:04:48.075 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:48.073+0000 7fab77db7640 1 -- 172.21.15.156:0/88134358 wait complete. 2024-07-16T10:04:48.077 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:48.074+0000 7fab77db7640 1 Processor -- start 2024-07-16T10:04:48.077 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:48.074+0000 7fab77db7640 1 -- start start 2024-07-16T10:04:48.077 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:48.074+0000 7fab77db7640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fab700fda40 0x7fab700fd230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:04:48.077 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:48.074+0000 7fab77db7640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fab700fe390 0x7fab700fb720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:04:48.077 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:48.074+0000 7fab77db7640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab700fbcb0 con 0x7fab700fe390 2024-07-16T10:04:48.078 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:48.074+0000 7fab77db7640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab700fbdf0 con 0x7fab700fda40 2024-07-16T10:04:48.078 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:48.074+0000 7fab75352640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fab700fe390 0x7fab700fb720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:04:48.078 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:48.074+0000 7fab75352640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fab700fe390 0x7fab700fb720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:37174/0 (socket says 172.21.15.156:37174) 2024-07-16T10:04:48.078 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:48.074+0000 7fab75352640 1 -- 172.21.15.156:0/2614426228 learned_addr learned my addr 172.21.15.156:0/2614426228 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T10:04:48.078 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:48.074+0000 7fab75b53640 1 --2- 172.21.15.156:0/2614426228 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fab700fda40 0x7fab700fd230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:04:48.078 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:48.074+0000 7fab75352640 1 -- 172.21.15.156:0/2614426228 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fab700fda40 msgr2=0x7fab700fd230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:04:48.078 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:48.074+0000 7fab75352640 1 --2- 172.21.15.156:0/2614426228 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fab700fda40 0x7fab700fd230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:04:48.078 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:48.074+0000 7fab75352640 1 -- 172.21.15.156:0/2614426228 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab64005040 con 0x7fab700fe390 2024-07-16T10:04:48.078 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:48.075+0000 7fab75352640 1 --2- 172.21.15.156:0/2614426228 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fab700fe390 0x7fab700fb720 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fab6c002f40 tx=0x7fab6c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T10:04:48.079 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:48.075+0000 7fab627fc640 1 -- 172.21.15.156:0/2614426228 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fab6c007c30 con 0x7fab700fe390 2024-07-16T10:04:48.079 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:48.075+0000 7fab77db7640 1 -- 172.21.15.156:0/2614426228 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab700fc070 con 0x7fab700fe390 2024-07-16T10:04:48.079 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:48.075+0000 7fab627fc640 1 -- 172.21.15.156:0/2614426228 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fab6c00f040 con 0x7fab700fe390 2024-07-16T10:04:48.079 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:48.075+0000 7fab627fc640 1 -- 172.21.15.156:0/2614426228 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fab6c00cc30 con 0x7fab700fe390 2024-07-16T10:04:48.079 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:48.075+0000 7fab77db7640 1 -- 172.21.15.156:0/2614426228 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab70114fe0 con 0x7fab700fe390 2024-07-16T10:04:48.080 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:48.076+0000 7fab77db7640 1 -- 172.21.15.156:0/2614426228 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab700f9e20 con 0x7fab700fe390 2024-07-16T10:04:48.080 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:48.076+0000 7fab627fc640 1 -- 172.21.15.156:0/2614426228 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 38) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fab6c0090d0 con 0x7fab700fe390 2024-07-16T10:04:48.080 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:48.077+0000 7fab627fc640 1 --2- 172.21.15.156:0/2614426228 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7fab5c06e0b0 0x7fab5c070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:04:48.080 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:48.077+0000 7fab627fc640 1 -- 172.21.15.156:0/2614426228 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7fab6c08d740 con 0x7fab700fe390 2024-07-16T10:04:48.080 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:48.077+0000 7fab75b53640 1 --2- 172.21.15.156:0/2614426228 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7fab5c06e0b0 0x7fab5c070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:04:48.080 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:48.078+0000 7fab75b53640 1 --2- 172.21.15.156:0/2614426228 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7fab5c06e0b0 0x7fab5c070560 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fab64009f80 tx=0x7fab64035040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T10:04:48.083 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:48.082+0000 7fab627fc640 1 -- 172.21.15.156:0/2614426228 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187716 (secure 0 0 0) 0x7fab6c056b40 con 0x7fab700fe390 2024-07-16T10:04:48.274 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:48.273+0000 7fab77db7640 1 -- 172.21.15.156:0/2614426228 --> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fab70101b20 con 0x7fab5c06e0b0 2024-07-16T10:04:48.289 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:48.288+0000 7fab627fc640 1 -- 172.21.15.156:0/2614426228 <== mgr.34104 v2:172.21.15.156:6800/3367765292 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7fab70101b20 con 0x7fab5c06e0b0 2024-07-16T10:04:48.290 INFO:teuthology.orchestra.run.smithi156.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-07-16T10:04:48.290 INFO:teuthology.orchestra.run.smithi156.stdout:alertmanager.smithi156 smithi156 *:9093,9094 running (14s) 9s ago 16m 16.2M - 0.25.0 c8568f914cd2 10ea5635099c 2024-07-16T10:04:48.290 INFO:teuthology.orchestra.run.smithi156.stdout:crash.smithi156 smithi156 running (6m) 9s ago 17m 6899k - 19.0.0-3495-ga6ca461b 16f7feb04342 11648cf29af4 2024-07-16T10:04:48.290 INFO:teuthology.orchestra.run.smithi156.stdout:crash.smithi176 smithi176 running (6m) 42s ago 16m 6899k - 19.0.0-3495-ga6ca461b 16f7feb04342 67f3ccc034f8 2024-07-16T10:04:48.290 INFO:teuthology.orchestra.run.smithi156.stdout:grafana.smithi156 smithi156 *:3000 running (8m) 9s ago 16m 86.7M - 9.4.7 954c08fa6188 deb9e1bfeb66 2024-07-16T10:04:48.290 INFO:teuthology.orchestra.run.smithi156.stdout:mds.cephfs.smithi156.evwwaz smithi156 running (93s) 9s ago 11m 15.6M - 19.0.0-3495-ga6ca461b 16f7feb04342 75879f672b07 2024-07-16T10:04:48.290 INFO:teuthology.orchestra.run.smithi156.stdout:mds.cephfs.smithi156.rdibys smithi156 running (2m) 9s ago 11m 64.5M - 19.0.0-3495-ga6ca461b 16f7feb04342 248f61472011 2024-07-16T10:04:48.290 INFO:teuthology.orchestra.run.smithi156.stdout:mds.cephfs.smithi176.dvvbud smithi176 running (76s) 42s ago 11m 133M - 19.0.0-3495-ga6ca461b 16f7feb04342 3aaf56c30dde 2024-07-16T10:04:48.291 INFO:teuthology.orchestra.run.smithi156.stdout:mds.cephfs.smithi176.wahbsi smithi176 running (60s) 42s ago 11m 14.3M - 19.0.0-3495-ga6ca461b 16f7feb04342 d30f4f63a683 2024-07-16T10:04:48.291 INFO:teuthology.orchestra.run.smithi156.stdout:mgr.smithi156.zkvylv smithi156 *:8443,9283,8765 running (7m) 9s ago 18m 509M - 19.0.0-3495-ga6ca461b 16f7feb04342 ee53bd604b7c 2024-07-16T10:04:48.291 INFO:teuthology.orchestra.run.smithi156.stdout:mgr.smithi176.dqbrvx smithi176 *:8443,9283,8765 running (7m) 42s ago 16m 442M - 19.0.0-3495-ga6ca461b 16f7feb04342 ce9eaae4c2d0 2024-07-16T10:04:48.291 INFO:teuthology.orchestra.run.smithi156.stdout:mon.smithi156 smithi156 running (7m) 9s ago 18m 53.0M 2048M 19.0.0-3495-ga6ca461b 16f7feb04342 77b6ec8c50eb 2024-07-16T10:04:48.291 INFO:teuthology.orchestra.run.smithi156.stdout:mon.smithi176 smithi176 running (6m) 42s ago 16m 39.8M 2048M 19.0.0-3495-ga6ca461b 16f7feb04342 ef92814cec2b 2024-07-16T10:04:48.291 INFO:teuthology.orchestra.run.smithi156.stdout:node-exporter.smithi156 smithi156 *:9100 running (48s) 9s ago 17m 8308k - 1.5.0 0da6a335fe13 a67719f4e14a 2024-07-16T10:04:48.291 INFO:teuthology.orchestra.run.smithi156.stdout:node-exporter.smithi176 smithi176 *:9100 running (45s) 42s ago 16m 3598k - 1.5.0 0da6a335fe13 87cdbf1674aa 2024-07-16T10:04:48.291 INFO:teuthology.orchestra.run.smithi156.stdout:osd.0 smithi156 running (5m) 9s ago 15m 105M 1557M 19.0.0-3495-ga6ca461b 16f7feb04342 8633df1689cc 2024-07-16T10:04:48.291 INFO:teuthology.orchestra.run.smithi156.stdout:osd.1 smithi156 running (5m) 9s ago 14m 117M 1557M 19.0.0-3495-ga6ca461b 16f7feb04342 e6fae9653b46 2024-07-16T10:04:48.291 INFO:teuthology.orchestra.run.smithi156.stdout:osd.2 smithi156 running (4m) 9s ago 14m 97.7M 1557M 19.0.0-3495-ga6ca461b 16f7feb04342 c6ecf18de9bf 2024-07-16T10:04:48.291 INFO:teuthology.orchestra.run.smithi156.stdout:osd.3 smithi176 running (4m) 42s ago 14m 113M 2581M 19.0.0-3495-ga6ca461b 16f7feb04342 d216534d6027 2024-07-16T10:04:48.292 INFO:teuthology.orchestra.run.smithi156.stdout:osd.4 smithi176 running (3m) 42s ago 13m 109M 2581M 19.0.0-3495-ga6ca461b 16f7feb04342 1bd3a25317b0 2024-07-16T10:04:48.292 INFO:teuthology.orchestra.run.smithi156.stdout:osd.5 smithi176 running (3m) 42s ago 13m 95.7M 2581M 19.0.0-3495-ga6ca461b 16f7feb04342 a03640221e7a 2024-07-16T10:04:48.292 INFO:teuthology.orchestra.run.smithi156.stdout:prometheus.smithi156 smithi156 *:9095 running (29s) 9s ago 16m 44.5M - 2.43.0 a07b618ecd1d f0897441abda 2024-07-16T10:04:48.292 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:48.291+0000 7fab77db7640 1 -- 172.21.15.156:0/2614426228 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7fab5c06e0b0 msgr2=0x7fab5c070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:04:48.293 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:48.291+0000 7fab77db7640 1 --2- 172.21.15.156:0/2614426228 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7fab5c06e0b0 0x7fab5c070560 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fab64009f80 tx=0x7fab64035040 comp rx=0 tx=0).stop 2024-07-16T10:04:48.293 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:48.291+0000 7fab77db7640 1 -- 172.21.15.156:0/2614426228 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fab700fe390 msgr2=0x7fab700fb720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:04:48.293 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:48.291+0000 7fab77db7640 1 --2- 172.21.15.156:0/2614426228 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fab700fe390 0x7fab700fb720 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fab6c002f40 tx=0x7fab6c00b040 comp rx=0 tx=0).stop 2024-07-16T10:04:48.293 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:48.291+0000 7fab77db7640 1 -- 172.21.15.156:0/2614426228 shutdown_connections 2024-07-16T10:04:48.293 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:48.291+0000 7fab77db7640 1 --2- 172.21.15.156:0/2614426228 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7fab5c06e0b0 0x7fab5c070560 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:04:48.293 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:48.291+0000 7fab77db7640 1 --2- 172.21.15.156:0/2614426228 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fab700fe390 0x7fab700fb720 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:04:48.293 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:48.291+0000 7fab77db7640 1 --2- 172.21.15.156:0/2614426228 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fab700fda40 0x7fab700fd230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:04:48.293 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:48.291+0000 7fab77db7640 1 -- 172.21.15.156:0/2614426228 >> 172.21.15.156:0/2614426228 conn(0x7fab700f61a0 msgr2=0x7fab700f77c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T10:04:48.293 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:48.291+0000 7fab77db7640 1 -- 172.21.15.156:0/2614426228 shutdown_connections 2024-07-16T10:04:48.293 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:48.291+0000 7fab77db7640 1 -- 172.21.15.156:0/2614426228 wait complete. 2024-07-16T10:04:48.380 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:48.377+0000 7f274c38d640 1 -- 172.21.15.156:0/311651967 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f27440ff510 msgr2=0x7f27441018f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:04:48.380 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:48.377+0000 7f274c38d640 1 --2- 172.21.15.156:0/311651967 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f27440ff510 0x7f27441018f0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f27340048d0 tx=0x7f2734009da0 comp rx=0 tx=0).stop 2024-07-16T10:04:48.380 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:48.377+0000 7f274c38d640 1 -- 172.21.15.156:0/311651967 shutdown_connections 2024-07-16T10:04:48.380 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:48.377+0000 7f274c38d640 1 --2- 172.21.15.156:0/311651967 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f27440ff510 0x7f27441018f0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:04:48.381 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:48.377+0000 7f274c38d640 1 --2- 172.21.15.156:0/311651967 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f27440659d0 0x7f27440fefd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:04:48.381 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:48.377+0000 7f274c38d640 1 -- 172.21.15.156:0/311651967 >> 172.21.15.156:0/311651967 conn(0x7f27440f6a20 msgr2=0x7f27440f8e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T10:04:48.381 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:48.377+0000 7f274c38d640 1 -- 172.21.15.156:0/311651967 shutdown_connections 2024-07-16T10:04:48.381 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:48.377+0000 7f274c38d640 1 -- 172.21.15.156:0/311651967 wait complete. 2024-07-16T10:04:48.381 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:48.378+0000 7f274c38d640 1 Processor -- start 2024-07-16T10:04:48.381 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:48.378+0000 7f274c38d640 1 -- start start 2024-07-16T10:04:48.381 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:48.378+0000 7f274c38d640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f27440659d0 0x7f274410cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:04:48.381 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:48.378+0000 7f274c38d640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f27440ff510 0x7f274410d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:04:48.381 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:48.378+0000 7f274c38d640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f274410db30 con 0x7f27440659d0 2024-07-16T10:04:48.381 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:48.378+0000 7f274c38d640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f274410e350 con 0x7f27440ff510 2024-07-16T10:04:48.381 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:48.378+0000 7f274a129640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f27440659d0 0x7f274410cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:04:48.381 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:48.378+0000 7f274a129640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f27440659d0 0x7f274410cfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:37196/0 (socket says 172.21.15.156:37196) 2024-07-16T10:04:48.381 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:48.378+0000 7f274a129640 1 -- 172.21.15.156:0/3063471801 learned_addr learned my addr 172.21.15.156:0/3063471801 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T10:04:48.382 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:48.378+0000 7f2749928640 1 --2- 172.21.15.156:0/3063471801 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f27440ff510 0x7f274410d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:04:48.382 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:48.378+0000 7f274a129640 1 -- 172.21.15.156:0/3063471801 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f27440ff510 msgr2=0x7f274410d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:04:48.382 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:48.378+0000 7f274a129640 1 --2- 172.21.15.156:0/3063471801 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f27440ff510 0x7f274410d4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:04:48.382 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:48.378+0000 7f274a129640 1 -- 172.21.15.156:0/3063471801 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2734005040 con 0x7f27440659d0 2024-07-16T10:04:48.382 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:48.379+0000 7f274a129640 1 --2- 172.21.15.156:0/3063471801 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f27440659d0 0x7f274410cfa0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f274000eb70 tx=0x7f274000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T10:04:48.382 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:48.379+0000 7f273affd640 1 -- 172.21.15.156:0/3063471801 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2740007c30 con 0x7f27440659d0 2024-07-16T10:04:48.382 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:48.379+0000 7f274c38d640 1 -- 172.21.15.156:0/3063471801 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f274410e5e0 con 0x7f27440659d0 2024-07-16T10:04:48.382 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:48.379+0000 7f274c38d640 1 -- 172.21.15.156:0/3063471801 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f274410eb50 con 0x7f27440659d0 2024-07-16T10:04:48.382 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:48.379+0000 7f273affd640 1 -- 172.21.15.156:0/3063471801 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f274000f040 con 0x7f27440659d0 2024-07-16T10:04:48.382 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:48.379+0000 7f273affd640 1 -- 172.21.15.156:0/3063471801 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f274000ccb0 con 0x7f27440659d0 2024-07-16T10:04:48.383 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:48.380+0000 7f274c38d640 1 -- 172.21.15.156:0/3063471801 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27441056b0 con 0x7f27440659d0 2024-07-16T10:04:48.383 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:48.381+0000 7f273affd640 1 -- 172.21.15.156:0/3063471801 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 38) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f2740009110 con 0x7f27440659d0 2024-07-16T10:04:48.383 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:48.381+0000 7f273affd640 1 --2- 172.21.15.156:0/3063471801 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f273006e0b0 0x7f2730070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:04:48.384 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:48.381+0000 7f273affd640 1 -- 172.21.15.156:0/3063471801 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f274008dc40 con 0x7f27440659d0 2024-07-16T10:04:48.384 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:48.381+0000 7f2749928640 1 --2- 172.21.15.156:0/3063471801 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f273006e0b0 0x7f2730070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:04:48.384 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:48.382+0000 7f2749928640 1 --2- 172.21.15.156:0/3063471801 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f273006e0b0 0x7f2730070560 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f2734000c00 tx=0x7f2734035040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T10:04:48.387 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:48.386+0000 7f273affd640 1 -- 172.21.15.156:0/3063471801 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187716 (secure 0 0 0) 0x7f2740056f90 con 0x7f27440659d0 2024-07-16T10:04:48.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:04:48 smithi176 ceph-mon[81689]: pgmap v247: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:04:48.663 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:48.661+0000 7f274c38d640 1 -- 172.21.15.156:0/3063471801 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f274404b5b0 con 0x7f27440659d0 2024-07-16T10:04:48.663 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:04:48 smithi156 ceph-mon[93066]: pgmap v247: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:04:48.663 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:48.662+0000 7f273affd640 1 -- 172.21.15.156:0/3063471801 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+618 (secure 0 0 0) 0x7f274404b5b0 con 0x7f27440659d0 2024-07-16T10:04:48.664 INFO:teuthology.orchestra.run.smithi156.stdout:{ 2024-07-16T10:04:48.664 INFO:teuthology.orchestra.run.smithi156.stdout: "mon": { 2024-07-16T10:04:48.664 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph version 19.0.0-3495-ga6ca461b (a6ca461be7cc1aff3658c2c6df4cff975a24449a) squid (rc)": 2 2024-07-16T10:04:48.664 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-07-16T10:04:48.664 INFO:teuthology.orchestra.run.smithi156.stdout: "mgr": { 2024-07-16T10:04:48.664 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph version 19.0.0-3495-ga6ca461b (a6ca461be7cc1aff3658c2c6df4cff975a24449a) squid (rc)": 2 2024-07-16T10:04:48.664 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-07-16T10:04:48.664 INFO:teuthology.orchestra.run.smithi156.stdout: "osd": { 2024-07-16T10:04:48.664 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph version 19.0.0-3495-ga6ca461b (a6ca461be7cc1aff3658c2c6df4cff975a24449a) squid (rc)": 6 2024-07-16T10:04:48.664 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-07-16T10:04:48.664 INFO:teuthology.orchestra.run.smithi156.stdout: "mds": { 2024-07-16T10:04:48.664 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph version 19.0.0-3495-ga6ca461b (a6ca461be7cc1aff3658c2c6df4cff975a24449a) squid (rc)": 4 2024-07-16T10:04:48.664 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-07-16T10:04:48.664 INFO:teuthology.orchestra.run.smithi156.stdout: "overall": { 2024-07-16T10:04:48.665 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph version 19.0.0-3495-ga6ca461b (a6ca461be7cc1aff3658c2c6df4cff975a24449a) squid (rc)": 14 2024-07-16T10:04:48.665 INFO:teuthology.orchestra.run.smithi156.stdout: } 2024-07-16T10:04:48.665 INFO:teuthology.orchestra.run.smithi156.stdout:} 2024-07-16T10:04:48.665 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:48.664+0000 7f274c38d640 1 -- 172.21.15.156:0/3063471801 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f273006e0b0 msgr2=0x7f2730070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:04:48.665 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:48.664+0000 7f274c38d640 1 --2- 172.21.15.156:0/3063471801 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f273006e0b0 0x7f2730070560 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f2734000c00 tx=0x7f2734035040 comp rx=0 tx=0).stop 2024-07-16T10:04:48.665 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:48.664+0000 7f274c38d640 1 -- 172.21.15.156:0/3063471801 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f27440659d0 msgr2=0x7f274410cfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:04:48.666 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:48.664+0000 7f274c38d640 1 --2- 172.21.15.156:0/3063471801 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f27440659d0 0x7f274410cfa0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f274000eb70 tx=0x7f274000b040 comp rx=0 tx=0).stop 2024-07-16T10:04:48.666 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:48.664+0000 7f274c38d640 1 -- 172.21.15.156:0/3063471801 shutdown_connections 2024-07-16T10:04:48.666 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:48.664+0000 7f274c38d640 1 --2- 172.21.15.156:0/3063471801 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f273006e0b0 0x7f2730070560 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:04:48.666 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:48.664+0000 7f274c38d640 1 --2- 172.21.15.156:0/3063471801 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f27440ff510 0x7f274410d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:04:48.666 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:48.664+0000 7f274c38d640 1 --2- 172.21.15.156:0/3063471801 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f27440659d0 0x7f274410cfa0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:04:48.666 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:48.664+0000 7f274c38d640 1 -- 172.21.15.156:0/3063471801 >> 172.21.15.156:0/3063471801 conn(0x7f27440f6a20 msgr2=0x7f27440f6df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T10:04:48.666 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:48.664+0000 7f274c38d640 1 -- 172.21.15.156:0/3063471801 shutdown_connections 2024-07-16T10:04:48.666 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:48.664+0000 7f274c38d640 1 -- 172.21.15.156:0/3063471801 wait complete. 2024-07-16T10:04:48.758 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:48.756+0000 7f2b0c96c640 1 -- 172.21.15.156:0/1914660043 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2b080ff4f0 msgr2=0x7f2b081018d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:04:48.759 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:48.756+0000 7f2b0c96c640 1 --2- 172.21.15.156:0/1914660043 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2b080ff4f0 0x7f2b081018d0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f2af00048d0 tx=0x7f2af0009da0 comp rx=0 tx=0).stop 2024-07-16T10:04:48.759 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:48.756+0000 7f2b0c96c640 1 -- 172.21.15.156:0/1914660043 shutdown_connections 2024-07-16T10:04:48.759 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:48.756+0000 7f2b0c96c640 1 --2- 172.21.15.156:0/1914660043 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2b080ff4f0 0x7f2b081018d0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:04:48.759 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:48.756+0000 7f2b0c96c640 1 --2- 172.21.15.156:0/1914660043 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2b080659f0 0x7f2b080fefb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:04:48.759 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:48.756+0000 7f2b0c96c640 1 -- 172.21.15.156:0/1914660043 >> 172.21.15.156:0/1914660043 conn(0x7f2b080f6a20 msgr2=0x7f2b080f8e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T10:04:48.759 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:48.756+0000 7f2b0c96c640 1 -- 172.21.15.156:0/1914660043 shutdown_connections 2024-07-16T10:04:48.759 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:48.756+0000 7f2b0c96c640 1 -- 172.21.15.156:0/1914660043 wait complete. 2024-07-16T10:04:48.759 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:48.756+0000 7f2b0c96c640 1 Processor -- start 2024-07-16T10:04:48.759 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:48.756+0000 7f2b0c96c640 1 -- start start 2024-07-16T10:04:48.759 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:48.756+0000 7f2b0c96c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2b080659f0 0x7f2b0810cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:04:48.759 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:48.756+0000 7f2b0c96c640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2b080ff4f0 0x7f2b0810d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:04:48.759 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:48.756+0000 7f2b0c96c640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b0810db30 con 0x7f2b080659f0 2024-07-16T10:04:48.760 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:48.756+0000 7f2b0c96c640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b0810e300 con 0x7f2b080ff4f0 2024-07-16T10:04:48.760 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:48.757+0000 7f2b0659c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2b080659f0 0x7f2b0810cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:04:48.760 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:48.757+0000 7f2b0659c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2b080659f0 0x7f2b0810cfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:37220/0 (socket says 172.21.15.156:37220) 2024-07-16T10:04:48.760 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:48.757+0000 7f2b0659c640 1 -- 172.21.15.156:0/2901017069 learned_addr learned my addr 172.21.15.156:0/2901017069 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T10:04:48.760 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:48.757+0000 7f2afffff640 1 --2- 172.21.15.156:0/2901017069 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2b080ff4f0 0x7f2b0810d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:04:48.760 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:48.757+0000 7f2b0659c640 1 -- 172.21.15.156:0/2901017069 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2b080ff4f0 msgr2=0x7f2b0810d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:04:48.760 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:48.757+0000 7f2b0659c640 1 --2- 172.21.15.156:0/2901017069 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2b080ff4f0 0x7f2b0810d4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:04:48.760 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:48.757+0000 7f2b0659c640 1 -- 172.21.15.156:0/2901017069 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2af0005040 con 0x7f2b080659f0 2024-07-16T10:04:48.760 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:48.757+0000 7f2b0659c640 1 --2- 172.21.15.156:0/2901017069 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2b080659f0 0x7f2b0810cfa0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f2af800c810 tx=0x7f2af80043c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T10:04:48.760 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:48.757+0000 7f2afeffd640 1 -- 172.21.15.156:0/2901017069 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2af800a7a0 con 0x7f2b080659f0 2024-07-16T10:04:48.760 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:48.757+0000 7f2b0c96c640 1 -- 172.21.15.156:0/2901017069 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b0810e590 con 0x7f2b080659f0 2024-07-16T10:04:48.760 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:48.757+0000 7f2afeffd640 1 -- 172.21.15.156:0/2901017069 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2af800add0 con 0x7f2b080659f0 2024-07-16T10:04:48.761 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:48.757+0000 7f2afeffd640 1 -- 172.21.15.156:0/2901017069 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2af80115c0 con 0x7f2b080659f0 2024-07-16T10:04:48.761 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:48.757+0000 7f2b0c96c640 1 -- 172.21.15.156:0/2901017069 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b0810eb00 con 0x7f2b080659f0 2024-07-16T10:04:48.761 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:48.758+0000 7f2b0c96c640 1 -- 172.21.15.156:0/2901017069 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b081056b0 con 0x7f2b080659f0 2024-07-16T10:04:48.762 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:48.759+0000 7f2afeffd640 1 -- 172.21.15.156:0/2901017069 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 38) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f2af8012420 con 0x7f2b080659f0 2024-07-16T10:04:48.762 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:48.760+0000 7f2afeffd640 1 --2- 172.21.15.156:0/2901017069 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f2af406e0b0 0x7f2af4070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:04:48.762 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:48.760+0000 7f2afeffd640 1 -- 172.21.15.156:0/2901017069 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f2af808ded0 con 0x7f2b080659f0 2024-07-16T10:04:48.762 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:48.760+0000 7f2afffff640 1 --2- 172.21.15.156:0/2901017069 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f2af406e0b0 0x7f2af4070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:04:48.762 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:48.760+0000 7f2afffff640 1 --2- 172.21.15.156:0/2901017069 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f2af406e0b0 0x7f2af4070560 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f2af0009a10 tx=0x7f2af0035040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T10:04:48.766 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:48.765+0000 7f2afeffd640 1 -- 172.21.15.156:0/2901017069 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187716 (secure 0 0 0) 0x7f2af8057220 con 0x7f2b080659f0 2024-07-16T10:04:49.005 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:49.004+0000 7f2b0c96c640 1 -- 172.21.15.156:0/2901017069 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f2b0810eea0 con 0x7f2b080659f0 2024-07-16T10:04:49.005 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:49.004+0000 7f2afeffd640 1 -- 172.21.15.156:0/2901017069 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 45 v45) v1 ==== 76+0+1972 (secure 0 0 0) 0x7f2af80167d0 con 0x7f2b080659f0 2024-07-16T10:04:49.006 INFO:teuthology.orchestra.run.smithi156.stdout:e45 2024-07-16T10:04:49.006 INFO:teuthology.orchestra.run.smithi156.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-07-16T10:04:49.006 INFO:teuthology.orchestra.run.smithi156.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-07-16T10:04:49.006 INFO:teuthology.orchestra.run.smithi156.stdout:legacy client fscid: 1 2024-07-16T10:04:49.006 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-07-16T10:04:49.006 INFO:teuthology.orchestra.run.smithi156.stdout:Filesystem 'cephfs' (1) 2024-07-16T10:04:49.006 INFO:teuthology.orchestra.run.smithi156.stdout:fs_name cephfs 2024-07-16T10:04:49.006 INFO:teuthology.orchestra.run.smithi156.stdout:epoch 45 2024-07-16T10:04:49.006 INFO:teuthology.orchestra.run.smithi156.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-07-16T10:04:49.007 INFO:teuthology.orchestra.run.smithi156.stdout:created 2024-07-16T09:52:55.214975+0000 2024-07-16T10:04:49.007 INFO:teuthology.orchestra.run.smithi156.stdout:modified 2024-07-16T10:03:55.772577+0000 2024-07-16T10:04:49.007 INFO:teuthology.orchestra.run.smithi156.stdout:tableserver 0 2024-07-16T10:04:49.007 INFO:teuthology.orchestra.run.smithi156.stdout:root 0 2024-07-16T10:04:49.007 INFO:teuthology.orchestra.run.smithi156.stdout:session_timeout 60 2024-07-16T10:04:49.007 INFO:teuthology.orchestra.run.smithi156.stdout:session_autoclose 300 2024-07-16T10:04:49.007 INFO:teuthology.orchestra.run.smithi156.stdout:max_file_size 1099511627776 2024-07-16T10:04:49.007 INFO:teuthology.orchestra.run.smithi156.stdout:max_xattr_size 65536 2024-07-16T10:04:49.007 INFO:teuthology.orchestra.run.smithi156.stdout:required_client_features {} 2024-07-16T10:04:49.008 INFO:teuthology.orchestra.run.smithi156.stdout:last_failure 0 2024-07-16T10:04:49.008 INFO:teuthology.orchestra.run.smithi156.stdout:last_failure_osd_epoch 82 2024-07-16T10:04:49.008 INFO:teuthology.orchestra.run.smithi156.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-07-16T10:04:49.008 INFO:teuthology.orchestra.run.smithi156.stdout:max_mds 2 2024-07-16T10:04:49.008 INFO:teuthology.orchestra.run.smithi156.stdout:in 0,1 2024-07-16T10:04:49.008 INFO:teuthology.orchestra.run.smithi156.stdout:up {0=34438,1=34470} 2024-07-16T10:04:49.008 INFO:teuthology.orchestra.run.smithi156.stdout:failed 2024-07-16T10:04:49.008 INFO:teuthology.orchestra.run.smithi156.stdout:damaged 2024-07-16T10:04:49.008 INFO:teuthology.orchestra.run.smithi156.stdout:stopped 2024-07-16T10:04:49.008 INFO:teuthology.orchestra.run.smithi156.stdout:data_pools [3] 2024-07-16T10:04:49.008 INFO:teuthology.orchestra.run.smithi156.stdout:metadata_pool 2 2024-07-16T10:04:49.009 INFO:teuthology.orchestra.run.smithi156.stdout:inline_data disabled 2024-07-16T10:04:49.009 INFO:teuthology.orchestra.run.smithi156.stdout:balancer 2024-07-16T10:04:49.009 INFO:teuthology.orchestra.run.smithi156.stdout:bal_rank_mask -1 2024-07-16T10:04:49.009 INFO:teuthology.orchestra.run.smithi156.stdout:standby_count_wanted 1 2024-07-16T10:04:49.009 INFO:teuthology.orchestra.run.smithi156.stdout:qdb_cluster leader: 34438 members: 34470,34438 2024-07-16T10:04:49.009 INFO:teuthology.orchestra.run.smithi156.stdout:[mds.cephfs.smithi156.rdibys{0:34438} state up:active seq 9 join_fscid=1 addr [v2:172.21.15.156:6826/2852996640,v1:172.21.15.156:6827/2852996640] compat {c=[1],r=[1],i=[1fff]}] 2024-07-16T10:04:49.009 INFO:teuthology.orchestra.run.smithi156.stdout:[mds.cephfs.smithi176.dvvbud{0:44347} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.176:6824/1700587296,v1:172.21.15.176:6825/1700587296] compat {c=[1],r=[1],i=[1fff]}] 2024-07-16T10:04:49.009 INFO:teuthology.orchestra.run.smithi156.stdout:[mds.cephfs.smithi156.evwwaz{1:34470} state up:active seq 11 join_fscid=1 addr [v2:172.21.15.156:6828/1683974145,v1:172.21.15.156:6829/1683974145] compat {c=[1],r=[1],i=[1fff]}] 2024-07-16T10:04:49.009 INFO:teuthology.orchestra.run.smithi156.stdout:[mds.cephfs.smithi176.wahbsi{1:44373} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.176:6826/2277003188,v1:172.21.15.176:6827/2277003188] compat {c=[1],r=[1],i=[1fff]}] 2024-07-16T10:04:49.009 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-07-16T10:04:49.009 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-07-16T10:04:49.010 INFO:teuthology.orchestra.run.smithi156.stderr:dumped fsmap epoch 45 2024-07-16T10:04:49.010 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:49.006+0000 7f2b0c96c640 1 -- 172.21.15.156:0/2901017069 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f2af406e0b0 msgr2=0x7f2af4070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:04:49.010 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:49.006+0000 7f2b0c96c640 1 --2- 172.21.15.156:0/2901017069 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f2af406e0b0 0x7f2af4070560 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f2af0009a10 tx=0x7f2af0035040 comp rx=0 tx=0).stop 2024-07-16T10:04:49.010 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:49.006+0000 7f2b0c96c640 1 -- 172.21.15.156:0/2901017069 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2b080659f0 msgr2=0x7f2b0810cfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:04:49.010 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:49.006+0000 7f2b0c96c640 1 --2- 172.21.15.156:0/2901017069 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2b080659f0 0x7f2b0810cfa0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f2af800c810 tx=0x7f2af80043c0 comp rx=0 tx=0).stop 2024-07-16T10:04:49.010 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:49.006+0000 7f2b0c96c640 1 -- 172.21.15.156:0/2901017069 shutdown_connections 2024-07-16T10:04:49.011 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:49.006+0000 7f2b0c96c640 1 --2- 172.21.15.156:0/2901017069 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f2af406e0b0 0x7f2af4070560 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:04:49.011 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:49.006+0000 7f2b0c96c640 1 --2- 172.21.15.156:0/2901017069 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2b080ff4f0 0x7f2b0810d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:04:49.011 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:49.006+0000 7f2b0c96c640 1 --2- 172.21.15.156:0/2901017069 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2b080659f0 0x7f2b0810cfa0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:04:49.011 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:49.006+0000 7f2b0c96c640 1 -- 172.21.15.156:0/2901017069 >> 172.21.15.156:0/2901017069 conn(0x7f2b080f6a20 msgr2=0x7f2b080f6df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T10:04:49.011 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:49.006+0000 7f2b0c96c640 1 -- 172.21.15.156:0/2901017069 shutdown_connections 2024-07-16T10:04:49.011 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:49.006+0000 7f2b0c96c640 1 -- 172.21.15.156:0/2901017069 wait complete. 2024-07-16T10:04:49.095 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:49.092+0000 7f4b3c86e640 1 -- 172.21.15.156:0/559063731 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4b380fc330 msgr2=0x7f4b380fc740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:04:49.095 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:49.092+0000 7f4b3c86e640 1 --2- 172.21.15.156:0/559063731 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4b380fc330 0x7f4b380fc740 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f4b280048d0 tx=0x7f4b28009da0 comp rx=0 tx=0).stop 2024-07-16T10:04:49.096 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:49.092+0000 7f4b3c86e640 1 -- 172.21.15.156:0/559063731 shutdown_connections 2024-07-16T10:04:49.096 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:49.092+0000 7f4b3c86e640 1 --2- 172.21.15.156:0/559063731 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4b380fd600 0x7f4b380fda70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:04:49.096 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:49.092+0000 7f4b3c86e640 1 --2- 172.21.15.156:0/559063731 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4b380fc330 0x7f4b380fc740 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:04:49.096 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:49.092+0000 7f4b3c86e640 1 -- 172.21.15.156:0/559063731 >> 172.21.15.156:0/559063731 conn(0x7f4b380f7af0 msgr2=0x7f4b380f9f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T10:04:49.096 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:49.092+0000 7f4b3c86e640 1 -- 172.21.15.156:0/559063731 shutdown_connections 2024-07-16T10:04:49.096 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:49.092+0000 7f4b3c86e640 1 -- 172.21.15.156:0/559063731 wait complete. 2024-07-16T10:04:49.096 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:49.092+0000 7f4b3c86e640 1 Processor -- start 2024-07-16T10:04:49.096 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:49.092+0000 7f4b3c86e640 1 -- start start 2024-07-16T10:04:49.096 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:49.093+0000 7f4b3c86e640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4b380fc330 0x7f4b3810cf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:04:49.097 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:49.093+0000 7f4b3c86e640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4b380fd600 0x7f4b3810d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:04:49.097 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:49.093+0000 7f4b3c86e640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b3810dad0 con 0x7f4b380fd600 2024-07-16T10:04:49.097 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:49.093+0000 7f4b3c86e640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b3810e2f0 con 0x7f4b380fc330 2024-07-16T10:04:49.097 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:49.093+0000 7f4b3659c640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4b380fc330 0x7f4b3810cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:04:49.097 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:49.093+0000 7f4b35d9b640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4b380fd600 0x7f4b3810d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:04:49.097 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:49.093+0000 7f4b3659c640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4b380fc330 0x7f4b3810cf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.156:43990/0 (socket says 172.21.15.156:43990) 2024-07-16T10:04:49.097 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:49.093+0000 7f4b35d9b640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4b380fd600 0x7f4b3810d480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:37238/0 (socket says 172.21.15.156:37238) 2024-07-16T10:04:49.097 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:49.093+0000 7f4b3659c640 1 -- 172.21.15.156:0/120646605 learned_addr learned my addr 172.21.15.156:0/120646605 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T10:04:49.097 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:49.093+0000 7f4b35d9b640 1 -- 172.21.15.156:0/120646605 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4b380fc330 msgr2=0x7f4b3810cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:04:49.097 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:49.093+0000 7f4b35d9b640 1 --2- 172.21.15.156:0/120646605 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4b380fc330 0x7f4b3810cf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:04:49.097 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:49.093+0000 7f4b35d9b640 1 -- 172.21.15.156:0/120646605 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b28005040 con 0x7f4b380fd600 2024-07-16T10:04:49.098 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:49.093+0000 7f4b3659c640 1 --2- 172.21.15.156:0/120646605 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4b380fc330 0x7f4b3810cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-16T10:04:49.098 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:49.093+0000 7f4b35d9b640 1 --2- 172.21.15.156:0/120646605 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4b380fd600 0x7f4b3810d480 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f4b2c002820 tx=0x7f4b2c004570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T10:04:49.098 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:49.094+0000 7f4b26ffd640 1 -- 172.21.15.156:0/120646605 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4b2c002db0 con 0x7f4b380fd600 2024-07-16T10:04:49.098 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:49.094+0000 7f4b3c86e640 1 -- 172.21.15.156:0/120646605 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b3810e580 con 0x7f4b380fd600 2024-07-16T10:04:49.098 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:49.094+0000 7f4b26ffd640 1 -- 172.21.15.156:0/120646605 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4b2c00f040 con 0x7f4b380fd600 2024-07-16T10:04:49.099 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:49.094+0000 7f4b26ffd640 1 -- 172.21.15.156:0/120646605 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4b2c009680 con 0x7f4b380fd600 2024-07-16T10:04:49.099 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:49.094+0000 7f4b3c86e640 1 -- 172.21.15.156:0/120646605 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b3810eaa0 con 0x7f4b380fd600 2024-07-16T10:04:49.099 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:49.095+0000 7f4b3c86e640 1 -- 172.21.15.156:0/120646605 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b380fb790 con 0x7f4b380fd600 2024-07-16T10:04:49.099 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:49.096+0000 7f4b26ffd640 1 -- 172.21.15.156:0/120646605 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 38) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f4b2c007c30 con 0x7f4b380fd600 2024-07-16T10:04:49.099 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:49.096+0000 7f4b26ffd640 1 --2- 172.21.15.156:0/120646605 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f4b2006e0b0 0x7f4b20070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:04:49.099 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:49.096+0000 7f4b26ffd640 1 -- 172.21.15.156:0/120646605 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f4b2c08d360 con 0x7f4b380fd600 2024-07-16T10:04:49.099 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:49.096+0000 7f4b3659c640 1 --2- 172.21.15.156:0/120646605 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f4b2006e0b0 0x7f4b20070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:04:49.099 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:49.097+0000 7f4b3659c640 1 --2- 172.21.15.156:0/120646605 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f4b2006e0b0 0x7f4b20070560 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f4b28004540 tx=0x7f4b28035040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T10:04:49.102 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:49.101+0000 7f4b26ffd640 1 -- 172.21.15.156:0/120646605 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187716 (secure 0 0 0) 0x7f4b2c0566e0 con 0x7f4b380fd600 2024-07-16T10:04:49.337 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:49.336+0000 7f4b3c86e640 1 -- 172.21.15.156:0/120646605 --> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f4b3810ee40 con 0x7f4b2006e0b0 2024-07-16T10:04:49.338 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:49.337+0000 7f4b26ffd640 1 -- 172.21.15.156:0/120646605 <== mgr.34104 v2:172.21.15.156:6800/3367765292 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+435 (secure 0 0 0) 0x7f4b3810ee40 con 0x7f4b2006e0b0 2024-07-16T10:04:49.339 INFO:teuthology.orchestra.run.smithi156.stdout:{ 2024-07-16T10:04:49.339 INFO:teuthology.orchestra.run.smithi156.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:c15ec3aa8e6c0a9c12ee24ff79f3e7dd4d01b3d1a702dfeb7d7fff9b0811a393", 2024-07-16T10:04:49.339 INFO:teuthology.orchestra.run.smithi156.stdout: "in_progress": true, 2024-07-16T10:04:49.339 INFO:teuthology.orchestra.run.smithi156.stdout: "which": "Upgrading all daemon types on all hosts", 2024-07-16T10:04:49.339 INFO:teuthology.orchestra.run.smithi156.stdout: "services_complete": [ 2024-07-16T10:04:49.339 INFO:teuthology.orchestra.run.smithi156.stdout: "mgr", 2024-07-16T10:04:49.339 INFO:teuthology.orchestra.run.smithi156.stdout: "crash", 2024-07-16T10:04:49.339 INFO:teuthology.orchestra.run.smithi156.stdout: "mds", 2024-07-16T10:04:49.339 INFO:teuthology.orchestra.run.smithi156.stdout: "mon", 2024-07-16T10:04:49.339 INFO:teuthology.orchestra.run.smithi156.stdout: "osd" 2024-07-16T10:04:49.339 INFO:teuthology.orchestra.run.smithi156.stdout: ], 2024-07-16T10:04:49.339 INFO:teuthology.orchestra.run.smithi156.stdout: "progress": "16/21 daemons upgraded", 2024-07-16T10:04:49.340 INFO:teuthology.orchestra.run.smithi156.stdout: "message": "Currently upgrading grafana daemons", 2024-07-16T10:04:49.340 INFO:teuthology.orchestra.run.smithi156.stdout: "is_paused": false 2024-07-16T10:04:49.340 INFO:teuthology.orchestra.run.smithi156.stdout:} 2024-07-16T10:04:49.341 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:49.339+0000 7f4b3c86e640 1 -- 172.21.15.156:0/120646605 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f4b2006e0b0 msgr2=0x7f4b20070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:04:49.341 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:49.339+0000 7f4b3c86e640 1 --2- 172.21.15.156:0/120646605 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f4b2006e0b0 0x7f4b20070560 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f4b28004540 tx=0x7f4b28035040 comp rx=0 tx=0).stop 2024-07-16T10:04:49.341 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:49.339+0000 7f4b3c86e640 1 -- 172.21.15.156:0/120646605 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4b380fd600 msgr2=0x7f4b3810d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:04:49.341 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:49.339+0000 7f4b3c86e640 1 --2- 172.21.15.156:0/120646605 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4b380fd600 0x7f4b3810d480 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f4b2c002820 tx=0x7f4b2c004570 comp rx=0 tx=0).stop 2024-07-16T10:04:49.341 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:49.340+0000 7f4b3c86e640 1 -- 172.21.15.156:0/120646605 shutdown_connections 2024-07-16T10:04:49.341 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:49.340+0000 7f4b3c86e640 1 --2- 172.21.15.156:0/120646605 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f4b2006e0b0 0x7f4b20070560 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:04:49.342 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:49.340+0000 7f4b3c86e640 1 --2- 172.21.15.156:0/120646605 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4b380fd600 0x7f4b3810d480 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:04:49.342 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:49.340+0000 7f4b3c86e640 1 --2- 172.21.15.156:0/120646605 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4b380fc330 0x7f4b3810cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:04:49.342 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:49.340+0000 7f4b3c86e640 1 -- 172.21.15.156:0/120646605 >> 172.21.15.156:0/120646605 conn(0x7f4b380f7af0 msgr2=0x7f4b380f91d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T10:04:49.342 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:49.340+0000 7f4b3c86e640 1 -- 172.21.15.156:0/120646605 shutdown_connections 2024-07-16T10:04:49.342 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:49.340+0000 7f4b3c86e640 1 -- 172.21.15.156:0/120646605 wait complete. 2024-07-16T10:04:49.435 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:49.433+0000 7fb6e2492640 1 -- 172.21.15.156:0/3583833857 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb6dc0fc500 msgr2=0x7fb6dc0fc970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:04:49.435 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:49.433+0000 7fb6e2492640 1 --2- 172.21.15.156:0/3583833857 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb6dc0fc500 0x7fb6dc0fc970 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fb6d00048d0 tx=0x7fb6d0009da0 comp rx=0 tx=0).stop 2024-07-16T10:04:49.435 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:49.433+0000 7fb6e2492640 1 -- 172.21.15.156:0/3583833857 shutdown_connections 2024-07-16T10:04:49.435 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:49.433+0000 7fb6e2492640 1 --2- 172.21.15.156:0/3583833857 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb6dc0fc500 0x7fb6dc0fc970 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:04:49.436 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:49.433+0000 7fb6e2492640 1 --2- 172.21.15.156:0/3583833857 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb6dc0fb230 0x7fb6dc0fb640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:04:49.436 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:49.433+0000 7fb6e2492640 1 -- 172.21.15.156:0/3583833857 >> 172.21.15.156:0/3583833857 conn(0x7fb6dc0f6a10 msgr2=0x7fb6dc0f8e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T10:04:49.436 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:49.434+0000 7fb6e2492640 1 -- 172.21.15.156:0/3583833857 shutdown_connections 2024-07-16T10:04:49.436 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:49.434+0000 7fb6e2492640 1 -- 172.21.15.156:0/3583833857 wait complete. 2024-07-16T10:04:49.436 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:49.434+0000 7fb6e2492640 1 Processor -- start 2024-07-16T10:04:49.438 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:49.434+0000 7fb6e2492640 1 -- start start 2024-07-16T10:04:49.438 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:49.435+0000 7fb6e2492640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb6dc0fb230 0x7fb6dc10fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:04:49.438 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:49.435+0000 7fb6e2492640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb6dc0fc500 0x7fb6dc112190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:04:49.438 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:49.435+0000 7fb6e2492640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6dc066320 con 0x7fb6dc0fc500 2024-07-16T10:04:49.438 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:49.435+0000 7fb6e2492640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6dc066490 con 0x7fb6dc0fb230 2024-07-16T10:04:49.438 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:49.435+0000 7fb6db7fe640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb6dc0fc500 0x7fb6dc112190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:04:49.438 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:49.435+0000 7fb6db7fe640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb6dc0fc500 0x7fb6dc112190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:37258/0 (socket says 172.21.15.156:37258) 2024-07-16T10:04:49.439 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:49.435+0000 7fb6db7fe640 1 -- 172.21.15.156:0/1981518343 learned_addr learned my addr 172.21.15.156:0/1981518343 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T10:04:49.439 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:49.435+0000 7fb6dbfff640 1 --2- 172.21.15.156:0/1981518343 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb6dc0fb230 0x7fb6dc10fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:04:49.439 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:49.435+0000 7fb6db7fe640 1 -- 172.21.15.156:0/1981518343 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb6dc0fb230 msgr2=0x7fb6dc10fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:04:49.439 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:49.435+0000 7fb6db7fe640 1 --2- 172.21.15.156:0/1981518343 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb6dc0fb230 0x7fb6dc10fc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:04:49.439 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:49.435+0000 7fb6db7fe640 1 -- 172.21.15.156:0/1981518343 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6d0005040 con 0x7fb6dc0fc500 2024-07-16T10:04:49.439 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:49.435+0000 7fb6dbfff640 1 --2- 172.21.15.156:0/1981518343 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb6dc0fb230 0x7fb6dc10fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-16T10:04:49.439 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:49.435+0000 7fb6db7fe640 1 --2- 172.21.15.156:0/1981518343 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb6dc0fc500 0x7fb6dc112190 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fb6d00048a0 tx=0x7fb6d0009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T10:04:49.439 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:49.435+0000 7fb6d8ff9640 1 -- 172.21.15.156:0/1981518343 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb6d0038070 con 0x7fb6dc0fc500 2024-07-16T10:04:49.439 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:49.436+0000 7fb6d8ff9640 1 -- 172.21.15.156:0/1981518343 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb6d0032970 con 0x7fb6dc0fc500 2024-07-16T10:04:49.439 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:49.436+0000 7fb6e2492640 1 -- 172.21.15.156:0/1981518343 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6dc1126d0 con 0x7fb6dc0fc500 2024-07-16T10:04:49.439 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:49.436+0000 7fb6d8ff9640 1 -- 172.21.15.156:0/1981518343 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb6d003d9a0 con 0x7fb6dc0fc500 2024-07-16T10:04:49.439 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:49.436+0000 7fb6e2492640 1 -- 172.21.15.156:0/1981518343 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6dc112c10 con 0x7fb6dc0fc500 2024-07-16T10:04:49.440 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:49.437+0000 7fb6e2492640 1 -- 172.21.15.156:0/1981518343 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6dc0fa690 con 0x7fb6dc0fc500 2024-07-16T10:04:49.440 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:49.437+0000 7fb6d8ff9640 1 -- 172.21.15.156:0/1981518343 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 38) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fb6d0030020 con 0x7fb6dc0fc500 2024-07-16T10:04:49.440 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:49.438+0000 7fb6d8ff9640 1 --2- 172.21.15.156:0/1981518343 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7fb6c406e0b0 0x7fb6c4070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:04:49.441 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:49.438+0000 7fb6d8ff9640 1 -- 172.21.15.156:0/1981518343 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7fb6d00ae270 con 0x7fb6dc0fc500 2024-07-16T10:04:49.441 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:49.438+0000 7fb6dbfff640 1 --2- 172.21.15.156:0/1981518343 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7fb6c406e0b0 0x7fb6c4070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:04:49.441 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:49.439+0000 7fb6dbfff640 1 --2- 172.21.15.156:0/1981518343 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7fb6c406e0b0 0x7fb6c4070560 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fb6cc0041d0 tx=0x7fb6cc009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T10:04:49.443 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:49.442+0000 7fb6d8ff9640 1 -- 172.21.15.156:0/1981518343 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187716 (secure 0 0 0) 0x7fb6d0077680 con 0x7fb6dc0fc500 2024-07-16T10:04:49.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:04:49 smithi176 ceph-mon[81689]: from='client.34530 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-16T10:04:49.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:04:49 smithi176 ceph-mon[81689]: from='client.34534 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-16T10:04:49.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:04:49 smithi176 ceph-mon[81689]: from='client.34538 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-07-16T10:04:49.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:04:49 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-16T10:04:49.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:04:49 smithi176 ceph-mon[81689]: from='client.? 172.21.15.156:0/3063471801' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:04:49.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:04:49 smithi176 ceph-mon[81689]: from='client.? 172.21.15.156:0/2901017069' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-07-16T10:04:49.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:04:49 smithi156 ceph-mon[93066]: from='client.34530 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-16T10:04:49.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:04:49 smithi156 ceph-mon[93066]: from='client.34534 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-16T10:04:49.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:04:49 smithi156 ceph-mon[93066]: from='client.34538 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-07-16T10:04:49.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:04:49 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-16T10:04:49.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:04:49 smithi156 ceph-mon[93066]: from='client.? 172.21.15.156:0/3063471801' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:04:49.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:04:49 smithi156 ceph-mon[93066]: from='client.? 172.21.15.156:0/2901017069' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-07-16T10:04:49.740 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:49.739+0000 7fb6e2492640 1 -- 172.21.15.156:0/1981518343 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fb6dc103d80 con 0x7fb6dc0fc500 2024-07-16T10:04:49.741 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:49.739+0000 7fb6d8ff9640 1 -- 172.21.15.156:0/1981518343 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7fb6d007ad40 con 0x7fb6dc0fc500 2024-07-16T10:04:49.741 INFO:teuthology.orchestra.run.smithi156.stdout:HEALTH_OK 2024-07-16T10:04:49.743 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:49.742+0000 7fb6e2492640 1 -- 172.21.15.156:0/1981518343 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7fb6c406e0b0 msgr2=0x7fb6c4070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:04:49.743 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:49.742+0000 7fb6e2492640 1 --2- 172.21.15.156:0/1981518343 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7fb6c406e0b0 0x7fb6c4070560 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fb6cc0041d0 tx=0x7fb6cc009040 comp rx=0 tx=0).stop 2024-07-16T10:04:49.743 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:49.742+0000 7fb6e2492640 1 -- 172.21.15.156:0/1981518343 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb6dc0fc500 msgr2=0x7fb6dc112190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:04:49.743 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:49.742+0000 7fb6e2492640 1 --2- 172.21.15.156:0/1981518343 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb6dc0fc500 0x7fb6dc112190 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fb6d00048a0 tx=0x7fb6d0009da0 comp rx=0 tx=0).stop 2024-07-16T10:04:49.743 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:49.742+0000 7fb6e2492640 1 -- 172.21.15.156:0/1981518343 shutdown_connections 2024-07-16T10:04:49.743 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:49.742+0000 7fb6e2492640 1 --2- 172.21.15.156:0/1981518343 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7fb6c406e0b0 0x7fb6c4070560 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:04:49.743 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:49.742+0000 7fb6e2492640 1 --2- 172.21.15.156:0/1981518343 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb6dc0fc500 0x7fb6dc112190 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:04:49.744 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:49.742+0000 7fb6e2492640 1 --2- 172.21.15.156:0/1981518343 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb6dc0fb230 0x7fb6dc10fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:04:49.744 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:49.742+0000 7fb6e2492640 1 -- 172.21.15.156:0/1981518343 >> 172.21.15.156:0/1981518343 conn(0x7fb6dc0f6a10 msgr2=0x7fb6dc0f80b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T10:04:49.744 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:49.742+0000 7fb6e2492640 1 -- 172.21.15.156:0/1981518343 shutdown_connections 2024-07-16T10:04:49.744 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:04:49.742+0000 7fb6e2492640 1 -- 172.21.15.156:0/1981518343 wait complete. 2024-07-16T10:04:50.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:04:50 smithi176 ceph-mon[81689]: pgmap v248: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-07-16T10:04:50.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:04:50 smithi176 ceph-mon[81689]: from='client.34550 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-16T10:04:50.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:04:50 smithi176 ceph-mon[81689]: from='client.? 172.21.15.156:0/1981518343' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-07-16T10:04:50.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:04:50 smithi156 ceph-mon[93066]: pgmap v248: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-07-16T10:04:50.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:04:50 smithi156 ceph-mon[93066]: from='client.34550 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-16T10:04:50.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:04:50 smithi156 ceph-mon[93066]: from='client.? 172.21.15.156:0/1981518343' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-07-16T10:04:52.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:04:52 smithi176 ceph-mon[81689]: pgmap v249: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:04:52.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:04:52 smithi156 ceph-mon[93066]: pgmap v249: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:04:54.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:04:54 smithi176 ceph-mon[81689]: pgmap v250: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:04:54.656 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:04:54 smithi156 ceph-mon[93066]: pgmap v250: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:04:56.343 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:04:55 smithi176 ceph-mon[81689]: pgmap v251: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-07-16T10:04:56.343 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:04:55 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:04:56.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:04:55 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:04:56.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:04:55 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-16T10:04:56.424 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:04:55 smithi156 ceph-mon[93066]: pgmap v251: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-07-16T10:04:56.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:04:55 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:04:56.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:04:55 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:04:56.425 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:04:55 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-16T10:04:58.174 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:04:57 smithi156 ceph-mon[93066]: pgmap v252: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:04:58.343 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:04:57 smithi176 ceph-mon[81689]: pgmap v252: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:05:00.591 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:05:00 smithi156 ceph-mon[93066]: pgmap v253: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-07-16T10:05:00.591 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:05:00 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:05:00.592 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:05:00 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:05:00.592 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:05:00 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:05:00.592 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:05:00 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:05:00.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:05:00 smithi176 ceph-mon[81689]: pgmap v253: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-07-16T10:05:00.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:05:00 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:05:00.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:05:00 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:05:00.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:05:00 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:05:00.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:05:00 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:05:02.528 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:05:02 smithi156 ceph-mon[93066]: pgmap v254: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:05:02.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:05:02 smithi176 ceph-mon[81689]: pgmap v254: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:05:04.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:05:04 smithi176 ceph-mon[81689]: pgmap v255: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:05:04.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:05:04 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:05:04.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:05:04 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:05:04.595 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:05:04 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-16T10:05:04.595 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:05:04 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-16T10:05:04.595 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:05:04 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:05:04.595 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:05:04 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-07-16T10:05:04.595 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:05:04 smithi176 ceph-mon[81689]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-07-16T10:05:04.595 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:05:04 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-16T10:05:04.595 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:05:04 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:05:04.595 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:05:04 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-16T10:05:04.595 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:05:04 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:05:04.595 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:05:04 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:05:04.596 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:05:04 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:05:04.596 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:05:04 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:05:04.596 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:05:04 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:05:04.596 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:05:04 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:05:04.596 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:05:04 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:05:04.596 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:05:04 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:05:04.596 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:05:04 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:05:04.596 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:05:04 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:05:04.596 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:05:04 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:05:04.596 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:05:04 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:05:04.596 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:05:04 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:05:04.596 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:05:04 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:05:04.596 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:05:04 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:05:04.596 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:05:04 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:05:04.597 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:05:04 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:05:04.597 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:05:04 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:05:04.597 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:05:04 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]: dispatch 2024-07-16T10:05:04.597 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:05:04 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]': finished 2024-07-16T10:05:04.597 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:05:04 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-07-16T10:05:04.597 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:05:04 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon"}]': finished 2024-07-16T10:05:04.597 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:05:04 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]: dispatch 2024-07-16T10:05:04.597 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:05:04 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]': finished 2024-07-16T10:05:04.597 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:05:04 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd"}]: dispatch 2024-07-16T10:05:04.597 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:05:04 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd"}]': finished 2024-07-16T10:05:04.597 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:05:04 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds"}]: dispatch 2024-07-16T10:05:04.597 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:05:04 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds"}]': finished 2024-07-16T10:05:04.597 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:05:04 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]: dispatch 2024-07-16T10:05:04.597 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:05:04 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]': finished 2024-07-16T10:05:04.598 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:05:04 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]: dispatch 2024-07-16T10:05:04.598 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:05:04 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]': finished 2024-07-16T10:05:04.598 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:05:04 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-07-16T10:05:04.598 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:05:04 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]: dispatch 2024-07-16T10:05:04.598 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:05:04 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]': finished 2024-07-16T10:05:04.598 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:05:04 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]: dispatch 2024-07-16T10:05:04.598 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:05:04 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]': finished 2024-07-16T10:05:04.598 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:05:04 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]: dispatch 2024-07-16T10:05:04.598 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:05:04 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]': finished 2024-07-16T10:05:04.598 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:05:04 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]: dispatch 2024-07-16T10:05:04.598 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:05:04 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]': finished 2024-07-16T10:05:04.599 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:05:04 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-07-16T10:05:04.599 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:05:04 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-07-16T10:05:04.599 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:05:04 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-07-16T10:05:04.599 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:05:04 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-07-16T10:05:04.599 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:05:04 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-07-16T10:05:04.599 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:05:04 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-07-16T10:05:04.599 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:05:04 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]: dispatch 2024-07-16T10:05:04.599 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:05:04 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd='[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]': finished 2024-07-16T10:05:04.599 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:05:04 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-16T10:05:04.599 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:05:04 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-16T10:05:04.599 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:05:04 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-16T10:05:04.599 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:05:04 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:05:04.599 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:05:04 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:05:04.600 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:05:04 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-16T10:05:04.600 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:05:04 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-16T10:05:04.600 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:05:04 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-16T10:05:04.600 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:05:04 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:05:04.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:05:04 smithi156 ceph-mon[93066]: pgmap v255: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:05:04.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:05:04 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:05:04.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:05:04 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:05:04.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:05:04 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-16T10:05:04.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:05:04 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-16T10:05:04.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:05:04 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:05:04.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:05:04 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-07-16T10:05:04.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:05:04 smithi156 ceph-mon[93066]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-07-16T10:05:04.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:05:04 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-16T10:05:04.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:05:04 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:05:04.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:05:04 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-16T10:05:04.676 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:05:04 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:05:04.676 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:05:04 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:05:04.676 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:05:04 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:05:04.676 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:05:04 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:05:04.676 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:05:04 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:05:04.676 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:05:04 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:05:04.676 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:05:04 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:05:04.676 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:05:04 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:05:04.676 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:05:04 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:05:04.676 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:05:04 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:05:04.676 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:05:04 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:05:04.676 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:05:04 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:05:04.676 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:05:04 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:05:04.677 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:05:04 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:05:04.677 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:05:04 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:05:04.677 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:05:04 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:05:04.677 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:05:04 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:05:04.677 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:05:04 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:05:04.677 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:05:04 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]: dispatch 2024-07-16T10:05:04.677 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:05:04 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]': finished 2024-07-16T10:05:04.677 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:05:04 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-07-16T10:05:04.677 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:05:04 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon"}]': finished 2024-07-16T10:05:04.677 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:05:04 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]: dispatch 2024-07-16T10:05:04.677 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:05:04 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]': finished 2024-07-16T10:05:04.677 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:05:04 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd"}]: dispatch 2024-07-16T10:05:04.677 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:05:04 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd"}]': finished 2024-07-16T10:05:04.678 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:05:04 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds"}]: dispatch 2024-07-16T10:05:04.678 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:05:04 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds"}]': finished 2024-07-16T10:05:04.678 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:05:04 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]: dispatch 2024-07-16T10:05:04.678 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:05:04 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]': finished 2024-07-16T10:05:04.678 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:05:04 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]: dispatch 2024-07-16T10:05:04.678 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:05:04 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]': finished 2024-07-16T10:05:04.678 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:05:04 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-07-16T10:05:04.678 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:05:04 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]: dispatch 2024-07-16T10:05:04.678 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:05:04 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]': finished 2024-07-16T10:05:04.678 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:05:04 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]: dispatch 2024-07-16T10:05:04.679 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:05:04 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]': finished 2024-07-16T10:05:04.679 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:05:04 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]: dispatch 2024-07-16T10:05:04.679 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:05:04 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]': finished 2024-07-16T10:05:04.679 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:05:04 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]: dispatch 2024-07-16T10:05:04.679 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:05:04 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]': finished 2024-07-16T10:05:04.679 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:05:04 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-07-16T10:05:04.679 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:05:04 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-07-16T10:05:04.679 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:05:04 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-07-16T10:05:04.679 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:05:04 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-07-16T10:05:04.679 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:05:04 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-07-16T10:05:04.679 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:05:04 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-07-16T10:05:04.680 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:05:04 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]: dispatch 2024-07-16T10:05:04.680 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:05:04 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd='[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]': finished 2024-07-16T10:05:04.680 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:05:04 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-16T10:05:04.680 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:05:04 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-16T10:05:04.680 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:05:04 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-16T10:05:04.680 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:05:04 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:05:04.680 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:05:04 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:05:04.680 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:05:04 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-16T10:05:04.680 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:05:04 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-16T10:05:04.680 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:05:04 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-16T10:05:04.680 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:05:04 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:05:05.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:05:05 smithi176 ceph-mon[81689]: Upgrade: Finalizing container_image settings 2024-07-16T10:05:05.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:05:05 smithi176 ceph-mon[81689]: Upgrade: Complete! 2024-07-16T10:05:05.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:05:05 smithi156 ceph-mon[93066]: Upgrade: Finalizing container_image settings 2024-07-16T10:05:05.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:05:05 smithi156 ceph-mon[93066]: Upgrade: Complete! 2024-07-16T10:05:06.495 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:05:06 smithi176 ceph-mon[81689]: pgmap v256: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-07-16T10:05:06.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:05:06 smithi156 ceph-mon[93066]: pgmap v256: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-07-16T10:05:08.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:05:08 smithi176 ceph-mon[81689]: pgmap v257: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:05:08.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:05:08 smithi156 ceph-mon[93066]: pgmap v257: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:05:10.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:05:10 smithi176 ceph-mon[81689]: pgmap v258: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-07-16T10:05:10.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:05:10 smithi156 ceph-mon[93066]: pgmap v258: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-07-16T10:05:12.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:05:12 smithi176 ceph-mon[81689]: pgmap v259: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:05:12.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:05:12 smithi156 ceph-mon[93066]: pgmap v259: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:05:14.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:05:14 smithi176 ceph-mon[81689]: pgmap v260: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:05:14.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:05:14 smithi156 ceph-mon[93066]: pgmap v260: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:05:16.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:05:16 smithi176 ceph-mon[81689]: pgmap v261: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-07-16T10:05:16.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:05:16 smithi156 ceph-mon[93066]: pgmap v261: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-07-16T10:05:18.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:05:18 smithi176 ceph-mon[81689]: pgmap v262: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:05:18.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:05:18 smithi156 ceph-mon[93066]: pgmap v262: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:05:19.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:05:19 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-16T10:05:19.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:05:19 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-16T10:05:19.839 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:05:19.836+0000 7f56bbf64640 1 -- 172.21.15.156:0/887495680 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f56b40fc330 msgr2=0x7f56b40fc740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:05:19.839 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:05:19.836+0000 7f56bbf64640 1 --2- 172.21.15.156:0/887495680 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f56b40fc330 0x7f56b40fc740 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f56a80048d0 tx=0x7f56a8009da0 comp rx=0 tx=0).stop 2024-07-16T10:05:19.839 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:05:19.836+0000 7f56bbf64640 1 -- 172.21.15.156:0/887495680 shutdown_connections 2024-07-16T10:05:19.839 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:05:19.836+0000 7f56bbf64640 1 --2- 172.21.15.156:0/887495680 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f56b40fd600 0x7f56b40fda70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:05:19.839 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:05:19.836+0000 7f56bbf64640 1 --2- 172.21.15.156:0/887495680 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f56b40fc330 0x7f56b40fc740 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:05:19.839 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:05:19.836+0000 7f56bbf64640 1 -- 172.21.15.156:0/887495680 >> 172.21.15.156:0/887495680 conn(0x7f56b40f7af0 msgr2=0x7f56b40f9f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T10:05:19.839 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:05:19.836+0000 7f56bbf64640 1 -- 172.21.15.156:0/887495680 shutdown_connections 2024-07-16T10:05:19.839 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:05:19.836+0000 7f56bbf64640 1 -- 172.21.15.156:0/887495680 wait complete. 2024-07-16T10:05:19.839 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:05:19.837+0000 7f56bbf64640 1 Processor -- start 2024-07-16T10:05:19.840 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:05:19.837+0000 7f56bbf64640 1 -- start start 2024-07-16T10:05:19.840 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:05:19.837+0000 7f56bbf64640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f56b40fc330 0x7f56b4071380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:05:19.840 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:05:19.837+0000 7f56bbf64640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f56b40fd600 0x7f56b406f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:05:19.840 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:05:19.837+0000 7f56bbf64640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56b406fe00 con 0x7f56b40fd600 2024-07-16T10:05:19.840 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:05:19.837+0000 7f56bbf64640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56b406ff40 con 0x7f56b40fc330 2024-07-16T10:05:19.840 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:05:19.837+0000 7f56b94ff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f56b40fd600 0x7f56b406f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:05:19.840 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:05:19.837+0000 7f56b94ff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f56b40fd600 0x7f56b406f870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:49182/0 (socket says 172.21.15.156:49182) 2024-07-16T10:05:19.840 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:05:19.837+0000 7f56b94ff640 1 -- 172.21.15.156:0/2075079396 learned_addr learned my addr 172.21.15.156:0/2075079396 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T10:05:19.840 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:05:19.837+0000 7f56b9d00640 1 --2- 172.21.15.156:0/2075079396 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f56b40fc330 0x7f56b4071380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:05:19.840 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:05:19.838+0000 7f56b94ff640 1 -- 172.21.15.156:0/2075079396 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f56b40fc330 msgr2=0x7f56b4071380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:05:19.840 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:05:19.838+0000 7f56b94ff640 1 --2- 172.21.15.156:0/2075079396 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f56b40fc330 0x7f56b4071380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:05:19.840 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:05:19.838+0000 7f56b94ff640 1 -- 172.21.15.156:0/2075079396 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56a8005040 con 0x7f56b40fd600 2024-07-16T10:05:19.841 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:05:19.838+0000 7f56b94ff640 1 --2- 172.21.15.156:0/2075079396 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f56b40fd600 0x7f56b406f870 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f56b0002a50 tx=0x7f56b0004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T10:05:19.842 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:05:19.838+0000 7f56a67fc640 1 -- 172.21.15.156:0/2075079396 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f56b0007c30 con 0x7f56b40fd600 2024-07-16T10:05:19.842 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:05:19.838+0000 7f56a67fc640 1 -- 172.21.15.156:0/2075079396 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f56b000f040 con 0x7f56b40fd600 2024-07-16T10:05:19.842 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:05:19.838+0000 7f56bbf64640 1 -- 172.21.15.156:0/2075079396 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56b4070220 con 0x7f56b40fd600 2024-07-16T10:05:19.842 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:05:19.838+0000 7f56a67fc640 1 -- 172.21.15.156:0/2075079396 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f56b000dad0 con 0x7f56b40fd600 2024-07-16T10:05:19.842 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:05:19.838+0000 7f56bbf64640 1 -- 172.21.15.156:0/2075079396 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56b4114fb0 con 0x7f56b40fd600 2024-07-16T10:05:19.842 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:05:19.839+0000 7f56bbf64640 1 -- 172.21.15.156:0/2075079396 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56b40fb790 con 0x7f56b40fd600 2024-07-16T10:05:19.842 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:05:19.840+0000 7f56a67fc640 1 -- 172.21.15.156:0/2075079396 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 38) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f56b000dd40 con 0x7f56b40fd600 2024-07-16T10:05:19.842 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:05:19.840+0000 7f56a67fc640 1 --2- 172.21.15.156:0/2075079396 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f56a006e0b0 0x7f56a0070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:05:19.842 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:05:19.840+0000 7f56a67fc640 1 -- 172.21.15.156:0/2075079396 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f56b008d3e0 con 0x7f56b40fd600 2024-07-16T10:05:19.842 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:05:19.840+0000 7f56b9d00640 1 --2- 172.21.15.156:0/2075079396 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f56a006e0b0 0x7f56a0070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:05:19.842 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:05:19.841+0000 7f56b9d00640 1 --2- 172.21.15.156:0/2075079396 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f56a006e0b0 0x7f56a0070560 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f56a8000c00 tx=0x7f56a8035040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T10:05:19.846 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:05:19.845+0000 7f56a67fc640 1 -- 172.21.15.156:0/2075079396 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187716 (secure 0 0 0) 0x7f56b0056760 con 0x7f56b40fd600 2024-07-16T10:05:20.040 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:05:20.039+0000 7f56bbf64640 1 -- 172.21.15.156:0/2075079396 --> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f56b4101930 con 0x7f56a006e0b0 2024-07-16T10:05:20.041 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:05:20.040+0000 7f56a67fc640 1 -- 172.21.15.156:0/2075079396 <== mgr.34104 v2:172.21.15.156:6800/3367765292 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+175 (secure 0 0 0) 0x7f56b4101930 con 0x7f56a006e0b0 2024-07-16T10:05:20.044 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:05:20.042+0000 7f56bbf64640 1 -- 172.21.15.156:0/2075079396 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f56a006e0b0 msgr2=0x7f56a0070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:05:20.044 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:05:20.042+0000 7f56bbf64640 1 --2- 172.21.15.156:0/2075079396 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f56a006e0b0 0x7f56a0070560 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f56a8000c00 tx=0x7f56a8035040 comp rx=0 tx=0).stop 2024-07-16T10:05:20.044 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:05:20.042+0000 7f56bbf64640 1 -- 172.21.15.156:0/2075079396 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f56b40fd600 msgr2=0x7f56b406f870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:05:20.044 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:05:20.042+0000 7f56bbf64640 1 --2- 172.21.15.156:0/2075079396 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f56b40fd600 0x7f56b406f870 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f56b0002a50 tx=0x7f56b0004600 comp rx=0 tx=0).stop 2024-07-16T10:05:20.044 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:05:20.042+0000 7f56bbf64640 1 -- 172.21.15.156:0/2075079396 shutdown_connections 2024-07-16T10:05:20.044 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:05:20.042+0000 7f56bbf64640 1 --2- 172.21.15.156:0/2075079396 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f56a006e0b0 0x7f56a0070560 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:05:20.044 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:05:20.042+0000 7f56bbf64640 1 --2- 172.21.15.156:0/2075079396 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f56b40fd600 0x7f56b406f870 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:05:20.044 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:05:20.042+0000 7f56bbf64640 1 --2- 172.21.15.156:0/2075079396 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f56b40fc330 0x7f56b4071380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:05:20.044 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:05:20.042+0000 7f56bbf64640 1 -- 172.21.15.156:0/2075079396 >> 172.21.15.156:0/2075079396 conn(0x7f56b40f7af0 msgr2=0x7f56b40f91d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T10:05:20.044 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:05:20.042+0000 7f56bbf64640 1 -- 172.21.15.156:0/2075079396 shutdown_connections 2024-07-16T10:05:20.044 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:05:20.042+0000 7f56bbf64640 1 -- 172.21.15.156:0/2075079396 wait complete. 2024-07-16T10:05:20.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:05:20 smithi176 ceph-mon[81689]: pgmap v263: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-07-16T10:05:20.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:05:20 smithi156 ceph-mon[93066]: pgmap v263: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-07-16T10:05:20.748 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 26c91c26-4358-11ef-bcad-c7b262605968 -e sha1=a6ca461be7cc1aff3658c2c6df4cff975a24449a -- bash -c 'ceph orch ps' 2024-07-16T10:05:21.050 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/26c91c26-4358-11ef-bcad-c7b262605968/mon.smithi156/config 2024-07-16T10:05:21.424 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:05:21 smithi156 ceph-mon[93066]: from='client.34558 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-16T10:05:21.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:05:21 smithi176 ceph-mon[81689]: from='client.34558 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-16T10:05:22.390 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:05:22 smithi156 ceph-mon[93066]: pgmap v264: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:05:22.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:05:22 smithi176 ceph-mon[81689]: pgmap v264: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:05:22.604 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:05:22.602+0000 7f637a8d5640 1 -- 172.21.15.156:0/1699466882 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6374101c00 msgr2=0x7f63740fb290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:05:22.604 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:05:22.602+0000 7f637a8d5640 1 --2- 172.21.15.156:0/1699466882 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6374101c00 0x7f63740fb290 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f635c0048d0 tx=0x7f635c009da0 comp rx=0 tx=0).stop 2024-07-16T10:05:22.604 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:05:22.602+0000 7f637a8d5640 1 -- 172.21.15.156:0/1699466882 shutdown_connections 2024-07-16T10:05:22.604 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:05:22.602+0000 7f637a8d5640 1 --2- 172.21.15.156:0/1699466882 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6374101c00 0x7f63740fb290 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:05:22.604 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:05:22.602+0000 7f637a8d5640 1 --2- 172.21.15.156:0/1699466882 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6374101260 0x7f6374101630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:05:22.604 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:05:22.602+0000 7f637a8d5640 1 -- 172.21.15.156:0/1699466882 >> 172.21.15.156:0/1699466882 conn(0x7f63740f6e10 msgr2=0x7f63740f9220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T10:05:22.604 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:05:22.602+0000 7f637a8d5640 1 -- 172.21.15.156:0/1699466882 shutdown_connections 2024-07-16T10:05:22.604 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:05:22.602+0000 7f637a8d5640 1 -- 172.21.15.156:0/1699466882 wait complete. 2024-07-16T10:05:22.606 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:05:22.603+0000 7f637a8d5640 1 Processor -- start 2024-07-16T10:05:22.606 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:05:22.603+0000 7f637a8d5640 1 -- start start 2024-07-16T10:05:22.606 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:05:22.603+0000 7f637a8d5640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6374101260 0x7f6374111fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:05:22.606 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:05:22.603+0000 7f637a8d5640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6374101c00 0x7f6374112510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:05:22.606 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:05:22.603+0000 7f637a8d5640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6374112b10 con 0x7f6374101c00 2024-07-16T10:05:22.606 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:05:22.603+0000 7f637a8d5640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f637410bf50 con 0x7f6374101260 2024-07-16T10:05:22.606 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:05:22.603+0000 7f636bfff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6374101c00 0x7f6374112510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:05:22.606 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:05:22.604+0000 7f636bfff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6374101c00 0x7f6374112510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:49202/0 (socket says 172.21.15.156:49202) 2024-07-16T10:05:22.606 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:05:22.604+0000 7f636bfff640 1 -- 172.21.15.156:0/2909677155 learned_addr learned my addr 172.21.15.156:0/2909677155 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T10:05:22.606 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:05:22.604+0000 7f6373fff640 1 --2- 172.21.15.156:0/2909677155 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6374101260 0x7f6374111fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:05:22.606 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:05:22.604+0000 7f636bfff640 1 -- 172.21.15.156:0/2909677155 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6374101260 msgr2=0x7f6374111fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:05:22.607 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:05:22.604+0000 7f636bfff640 1 --2- 172.21.15.156:0/2909677155 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6374101260 0x7f6374111fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:05:22.607 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:05:22.604+0000 7f636bfff640 1 -- 172.21.15.156:0/2909677155 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f635c005040 con 0x7f6374101c00 2024-07-16T10:05:22.607 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:05:22.604+0000 7f636bfff640 1 --2- 172.21.15.156:0/2909677155 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6374101c00 0x7f6374112510 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f635c003080 tx=0x7f635c009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T10:05:22.607 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:05:22.604+0000 7f63717fa640 1 -- 172.21.15.156:0/2909677155 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f635c038070 con 0x7f6374101c00 2024-07-16T10:05:22.607 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:05:22.604+0000 7f637a8d5640 1 -- 172.21.15.156:0/2909677155 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f637410c1d0 con 0x7f6374101c00 2024-07-16T10:05:22.607 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:05:22.604+0000 7f63717fa640 1 -- 172.21.15.156:0/2909677155 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f635c008ee0 con 0x7f6374101c00 2024-07-16T10:05:22.607 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:05:22.604+0000 7f63717fa640 1 -- 172.21.15.156:0/2909677155 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f635c03d870 con 0x7f6374101c00 2024-07-16T10:05:22.607 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:05:22.604+0000 7f637a8d5640 1 -- 172.21.15.156:0/2909677155 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f637410c710 con 0x7f6374101c00 2024-07-16T10:05:22.608 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:05:22.605+0000 7f636a7fc640 1 -- 172.21.15.156:0/2909677155 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63740fc9d0 con 0x7f6374101c00 2024-07-16T10:05:22.608 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:05:22.606+0000 7f63717fa640 1 -- 172.21.15.156:0/2909677155 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 38) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f635c030020 con 0x7f6374101c00 2024-07-16T10:05:22.608 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:05:22.606+0000 7f63717fa640 1 --2- 172.21.15.156:0/2909677155 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f636006e0b0 0x7f6360070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:05:22.608 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:05:22.606+0000 7f63717fa640 1 -- 172.21.15.156:0/2909677155 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f635c0ad8b0 con 0x7f6374101c00 2024-07-16T10:05:22.608 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:05:22.606+0000 7f6373fff640 1 --2- 172.21.15.156:0/2909677155 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f636006e0b0 0x7f6360070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:05:22.608 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:05:22.607+0000 7f6373fff640 1 --2- 172.21.15.156:0/2909677155 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f636006e0b0 0x7f6360070560 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f63640041d0 tx=0x7f6364004330 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T10:05:22.611 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:05:22.611+0000 7f63717fa640 1 -- 172.21.15.156:0/2909677155 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187716 (secure 0 0 0) 0x7f635c076c00 con 0x7f6374101c00 2024-07-16T10:05:22.797 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:05:22.795+0000 7f636a7fc640 1 -- 172.21.15.156:0/2909677155 --> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f63740faa50 con 0x7f636006e0b0 2024-07-16T10:05:22.806 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:05:22.804+0000 7f63717fa640 1 -- 172.21.15.156:0/2909677155 <== mgr.34104 v2:172.21.15.156:6800/3367765292 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7f63740faa50 con 0x7f636006e0b0 2024-07-16T10:05:22.806 INFO:teuthology.orchestra.run.smithi156.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-07-16T10:05:22.806 INFO:teuthology.orchestra.run.smithi156.stdout:alertmanager.smithi156 smithi156 *:9093,9094 running (49s) 23s ago 17m 16.2M - 0.25.0 c8568f914cd2 10ea5635099c 2024-07-16T10:05:22.806 INFO:teuthology.orchestra.run.smithi156.stdout:crash.smithi156 smithi156 running (6m) 23s ago 17m 6899k - 19.0.0-3495-ga6ca461b 16f7feb04342 11648cf29af4 2024-07-16T10:05:22.806 INFO:teuthology.orchestra.run.smithi156.stdout:crash.smithi176 smithi176 running (6m) 77s ago 16m 6899k - 19.0.0-3495-ga6ca461b 16f7feb04342 67f3ccc034f8 2024-07-16T10:05:22.806 INFO:teuthology.orchestra.run.smithi156.stdout:grafana.smithi156 smithi156 *:3000 running (28s) 23s ago 17m 45.4M - 9.4.12 2bacad6d85d8 b5801e6c9d2d 2024-07-16T10:05:22.806 INFO:teuthology.orchestra.run.smithi156.stdout:mds.cephfs.smithi156.evwwaz smithi156 running (2m) 23s ago 12m 16.2M - 19.0.0-3495-ga6ca461b 16f7feb04342 75879f672b07 2024-07-16T10:05:22.806 INFO:teuthology.orchestra.run.smithi156.stdout:mds.cephfs.smithi156.rdibys smithi156 running (2m) 23s ago 12m 64.6M - 19.0.0-3495-ga6ca461b 16f7feb04342 248f61472011 2024-07-16T10:05:22.807 INFO:teuthology.orchestra.run.smithi156.stdout:mds.cephfs.smithi176.dvvbud smithi176 running (110s) 77s ago 12m 133M - 19.0.0-3495-ga6ca461b 16f7feb04342 3aaf56c30dde 2024-07-16T10:05:22.807 INFO:teuthology.orchestra.run.smithi156.stdout:mds.cephfs.smithi176.wahbsi smithi176 running (95s) 77s ago 12m 14.3M - 19.0.0-3495-ga6ca461b 16f7feb04342 d30f4f63a683 2024-07-16T10:05:22.807 INFO:teuthology.orchestra.run.smithi156.stdout:mgr.smithi156.zkvylv smithi156 *:8443,9283,8765 running (8m) 23s ago 18m 516M - 19.0.0-3495-ga6ca461b 16f7feb04342 ee53bd604b7c 2024-07-16T10:05:22.807 INFO:teuthology.orchestra.run.smithi156.stdout:mgr.smithi176.dqbrvx smithi176 *:8443,9283,8765 running (7m) 77s ago 16m 442M - 19.0.0-3495-ga6ca461b 16f7feb04342 ce9eaae4c2d0 2024-07-16T10:05:22.807 INFO:teuthology.orchestra.run.smithi156.stdout:mon.smithi156 smithi156 running (7m) 23s ago 18m 53.8M 2048M 19.0.0-3495-ga6ca461b 16f7feb04342 77b6ec8c50eb 2024-07-16T10:05:22.807 INFO:teuthology.orchestra.run.smithi156.stdout:mon.smithi176 smithi176 running (7m) 77s ago 16m 39.8M 2048M 19.0.0-3495-ga6ca461b 16f7feb04342 ef92814cec2b 2024-07-16T10:05:22.807 INFO:teuthology.orchestra.run.smithi156.stdout:node-exporter.smithi156 smithi156 *:9100 running (83s) 23s ago 17m 8178k - 1.5.0 0da6a335fe13 a67719f4e14a 2024-07-16T10:05:22.807 INFO:teuthology.orchestra.run.smithi156.stdout:node-exporter.smithi176 smithi176 *:9100 running (79s) 77s ago 16m 3598k - 1.5.0 0da6a335fe13 87cdbf1674aa 2024-07-16T10:05:22.807 INFO:teuthology.orchestra.run.smithi156.stdout:osd.0 smithi156 running (6m) 23s ago 15m 105M 1557M 19.0.0-3495-ga6ca461b 16f7feb04342 8633df1689cc 2024-07-16T10:05:22.807 INFO:teuthology.orchestra.run.smithi156.stdout:osd.1 smithi156 running (5m) 23s ago 15m 116M 1557M 19.0.0-3495-ga6ca461b 16f7feb04342 e6fae9653b46 2024-07-16T10:05:22.807 INFO:teuthology.orchestra.run.smithi156.stdout:osd.2 smithi156 running (5m) 23s ago 15m 97.8M 1557M 19.0.0-3495-ga6ca461b 16f7feb04342 c6ecf18de9bf 2024-07-16T10:05:22.807 INFO:teuthology.orchestra.run.smithi156.stdout:osd.3 smithi176 running (4m) 77s ago 14m 113M 2581M 19.0.0-3495-ga6ca461b 16f7feb04342 d216534d6027 2024-07-16T10:05:22.807 INFO:teuthology.orchestra.run.smithi156.stdout:osd.4 smithi176 running (4m) 77s ago 14m 109M 2581M 19.0.0-3495-ga6ca461b 16f7feb04342 1bd3a25317b0 2024-07-16T10:05:22.808 INFO:teuthology.orchestra.run.smithi156.stdout:osd.5 smithi176 running (3m) 77s ago 13m 95.7M 2581M 19.0.0-3495-ga6ca461b 16f7feb04342 a03640221e7a 2024-07-16T10:05:22.808 INFO:teuthology.orchestra.run.smithi156.stdout:prometheus.smithi156 smithi156 *:9095 running (64s) 23s ago 17m 46.9M - 2.43.0 a07b618ecd1d f0897441abda 2024-07-16T10:05:22.808 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:05:22.807+0000 7f636a7fc640 1 -- 172.21.15.156:0/2909677155 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f636006e0b0 msgr2=0x7f6360070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:05:22.808 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:05:22.807+0000 7f636a7fc640 1 --2- 172.21.15.156:0/2909677155 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f636006e0b0 0x7f6360070560 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f63640041d0 tx=0x7f6364004330 comp rx=0 tx=0).stop 2024-07-16T10:05:22.809 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:05:22.807+0000 7f636a7fc640 1 -- 172.21.15.156:0/2909677155 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6374101c00 msgr2=0x7f6374112510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:05:22.809 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:05:22.807+0000 7f636a7fc640 1 --2- 172.21.15.156:0/2909677155 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6374101c00 0x7f6374112510 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f635c003080 tx=0x7f635c009da0 comp rx=0 tx=0).stop 2024-07-16T10:05:22.809 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:05:22.807+0000 7f636a7fc640 1 -- 172.21.15.156:0/2909677155 shutdown_connections 2024-07-16T10:05:22.809 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:05:22.807+0000 7f636a7fc640 1 --2- 172.21.15.156:0/2909677155 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f636006e0b0 0x7f6360070560 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:05:22.809 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:05:22.807+0000 7f636a7fc640 1 --2- 172.21.15.156:0/2909677155 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6374101c00 0x7f6374112510 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:05:22.809 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:05:22.807+0000 7f636a7fc640 1 --2- 172.21.15.156:0/2909677155 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6374101260 0x7f6374111fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:05:22.809 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:05:22.807+0000 7f636a7fc640 1 -- 172.21.15.156:0/2909677155 >> 172.21.15.156:0/2909677155 conn(0x7f63740f6e10 msgr2=0x7f63740f7d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T10:05:22.809 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:05:22.807+0000 7f636a7fc640 1 -- 172.21.15.156:0/2909677155 shutdown_connections 2024-07-16T10:05:22.809 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:05:22.807+0000 7f636a7fc640 1 -- 172.21.15.156:0/2909677155 wait complete. 2024-07-16T10:05:23.480 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 26c91c26-4358-11ef-bcad-c7b262605968 -e sha1=a6ca461be7cc1aff3658c2c6df4cff975a24449a -- bash -c 'ceph orch upgrade status' 2024-07-16T10:05:23.740 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/26c91c26-4358-11ef-bcad-c7b262605968/mon.smithi156/config 2024-07-16T10:05:24.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:05:24 smithi176 ceph-mon[81689]: pgmap v265: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:05:24.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:05:24 smithi176 ceph-mon[81689]: from='client.34562 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-07-16T10:05:24.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:05:24 smithi156 ceph-mon[93066]: pgmap v265: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:05:24.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:05:24 smithi156 ceph-mon[93066]: from='client.34562 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-07-16T10:05:25.315 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:05:25.300+0000 7ff7a6d87640 1 -- 172.21.15.156:0/1280696012 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff7a0101250 msgr2=0x7ff7a0101620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:05:25.315 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:05:25.300+0000 7ff7a6d87640 1 --2- 172.21.15.156:0/1280696012 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff7a0101250 0x7ff7a0101620 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7ff79c0048d0 tx=0x7ff79c009da0 comp rx=0 tx=0).stop 2024-07-16T10:05:25.315 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:05:25.300+0000 7ff7a6d87640 1 -- 172.21.15.156:0/1280696012 shutdown_connections 2024-07-16T10:05:25.315 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:05:25.300+0000 7ff7a6d87640 1 --2- 172.21.15.156:0/1280696012 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff7a0101bf0 0x7ff7a00fb280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:05:25.315 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:05:25.300+0000 7ff7a6d87640 1 --2- 172.21.15.156:0/1280696012 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff7a0101250 0x7ff7a0101620 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:05:25.316 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:05:25.300+0000 7ff7a6d87640 1 -- 172.21.15.156:0/1280696012 >> 172.21.15.156:0/1280696012 conn(0x7ff7a00f6e00 msgr2=0x7ff7a00f9210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T10:05:25.316 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:05:25.300+0000 7ff7a6d87640 1 -- 172.21.15.156:0/1280696012 shutdown_connections 2024-07-16T10:05:25.316 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:05:25.300+0000 7ff7a6d87640 1 -- 172.21.15.156:0/1280696012 wait complete. 2024-07-16T10:05:25.316 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:05:25.300+0000 7ff7a6d87640 1 Processor -- start 2024-07-16T10:05:25.316 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:05:25.300+0000 7ff7a6d87640 1 -- start start 2024-07-16T10:05:25.316 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:05:25.301+0000 7ff7a6d87640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff7a0101250 0x7ff7a0111fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:05:25.316 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:05:25.301+0000 7ff7a6d87640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff7a0101bf0 0x7ff7a01124f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:05:25.316 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:05:25.301+0000 7ff7a6d87640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7a0112af0 con 0x7ff7a0101bf0 2024-07-16T10:05:25.316 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:05:25.301+0000 7ff7a6d87640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7a010bf30 con 0x7ff7a0101250 2024-07-16T10:05:25.316 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:05:25.301+0000 7ff797fff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff7a0101bf0 0x7ff7a01124f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:05:25.316 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:05:25.301+0000 7ff797fff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff7a0101bf0 0x7ff7a01124f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:43650/0 (socket says 172.21.15.156:43650) 2024-07-16T10:05:25.316 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:05:25.301+0000 7ff797fff640 1 -- 172.21.15.156:0/1037486258 learned_addr learned my addr 172.21.15.156:0/1037486258 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T10:05:25.316 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:05:25.301+0000 7ff7a4b23640 1 --2- 172.21.15.156:0/1037486258 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff7a0101250 0x7ff7a0111fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:05:25.316 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:05:25.301+0000 7ff797fff640 1 -- 172.21.15.156:0/1037486258 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff7a0101250 msgr2=0x7ff7a0111fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:05:25.317 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:05:25.301+0000 7ff797fff640 1 --2- 172.21.15.156:0/1037486258 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff7a0101250 0x7ff7a0111fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:05:25.317 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:05:25.301+0000 7ff797fff640 1 -- 172.21.15.156:0/1037486258 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff79c005040 con 0x7ff7a0101bf0 2024-07-16T10:05:25.317 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:05:25.301+0000 7ff797fff640 1 --2- 172.21.15.156:0/1037486258 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff7a0101bf0 0x7ff7a01124f0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7ff790002a50 tx=0x7ff790004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T10:05:25.317 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:05:25.301+0000 7ff7957fa640 1 -- 172.21.15.156:0/1037486258 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff790007c30 con 0x7ff7a0101bf0 2024-07-16T10:05:25.317 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:05:25.301+0000 7ff7a6d87640 1 -- 172.21.15.156:0/1037486258 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7a010c210 con 0x7ff7a0101bf0 2024-07-16T10:05:25.317 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:05:25.301+0000 7ff7957fa640 1 -- 172.21.15.156:0/1037486258 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff79000f040 con 0x7ff7a0101bf0 2024-07-16T10:05:25.317 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:05:25.302+0000 7ff7957fa640 1 -- 172.21.15.156:0/1037486258 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff79000db50 con 0x7ff7a0101bf0 2024-07-16T10:05:25.317 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:05:25.302+0000 7ff7a6d87640 1 -- 172.21.15.156:0/1037486258 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7a010c7b0 con 0x7ff7a0101bf0 2024-07-16T10:05:25.317 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:05:25.303+0000 7ff7a6d87640 1 -- 172.21.15.156:0/1037486258 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7a00fc9c0 con 0x7ff7a0101bf0 2024-07-16T10:05:25.317 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:05:25.303+0000 7ff7957fa640 1 -- 172.21.15.156:0/1037486258 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 38) v1 ==== 92205+0+0 (secure 0 0 0) 0x7ff790009070 con 0x7ff7a0101bf0 2024-07-16T10:05:25.317 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:05:25.303+0000 7ff7957fa640 1 --2- 172.21.15.156:0/1037486258 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7ff78806e0b0 0x7ff788070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:05:25.317 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:05:25.304+0000 7ff7957fa640 1 -- 172.21.15.156:0/1037486258 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7ff79008d500 con 0x7ff7a0101bf0 2024-07-16T10:05:25.318 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:05:25.304+0000 7ff7a4b23640 1 --2- 172.21.15.156:0/1037486258 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7ff78806e0b0 0x7ff788070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:05:25.318 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:05:25.304+0000 7ff7a4b23640 1 --2- 172.21.15.156:0/1037486258 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7ff78806e0b0 0x7ff788070560 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7ff79c000c00 tx=0x7ff79c035040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T10:05:25.318 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:05:25.308+0000 7ff7957fa640 1 -- 172.21.15.156:0/1037486258 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187716 (secure 0 0 0) 0x7ff7900567d0 con 0x7ff7a0101bf0 2024-07-16T10:05:25.505 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:05:25.503+0000 7ff7a6d87640 1 -- 172.21.15.156:0/1037486258 --> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff7a01083b0 con 0x7ff78806e0b0 2024-07-16T10:05:25.506 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:05:25.504+0000 7ff7957fa640 1 -- 172.21.15.156:0/1037486258 <== mgr.34104 v2:172.21.15.156:6800/3367765292 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+175 (secure 0 0 0) 0x7ff7a01083b0 con 0x7ff78806e0b0 2024-07-16T10:05:25.506 INFO:teuthology.orchestra.run.smithi156.stdout:{ 2024-07-16T10:05:25.506 INFO:teuthology.orchestra.run.smithi156.stdout: "target_image": null, 2024-07-16T10:05:25.507 INFO:teuthology.orchestra.run.smithi156.stdout: "in_progress": false, 2024-07-16T10:05:25.507 INFO:teuthology.orchestra.run.smithi156.stdout: "which": "", 2024-07-16T10:05:25.507 INFO:teuthology.orchestra.run.smithi156.stdout: "services_complete": [], 2024-07-16T10:05:25.507 INFO:teuthology.orchestra.run.smithi156.stdout: "progress": null, 2024-07-16T10:05:25.507 INFO:teuthology.orchestra.run.smithi156.stdout: "message": "", 2024-07-16T10:05:25.507 INFO:teuthology.orchestra.run.smithi156.stdout: "is_paused": false 2024-07-16T10:05:25.507 INFO:teuthology.orchestra.run.smithi156.stdout:} 2024-07-16T10:05:25.508 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:05:25.506+0000 7ff7a6d87640 1 -- 172.21.15.156:0/1037486258 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7ff78806e0b0 msgr2=0x7ff788070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:05:25.509 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:05:25.506+0000 7ff7a6d87640 1 --2- 172.21.15.156:0/1037486258 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7ff78806e0b0 0x7ff788070560 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7ff79c000c00 tx=0x7ff79c035040 comp rx=0 tx=0).stop 2024-07-16T10:05:25.509 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:05:25.506+0000 7ff7a6d87640 1 -- 172.21.15.156:0/1037486258 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff7a0101bf0 msgr2=0x7ff7a01124f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:05:25.509 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:05:25.506+0000 7ff7a6d87640 1 --2- 172.21.15.156:0/1037486258 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff7a0101bf0 0x7ff7a01124f0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7ff790002a50 tx=0x7ff790004600 comp rx=0 tx=0).stop 2024-07-16T10:05:25.509 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:05:25.506+0000 7ff7a6d87640 1 -- 172.21.15.156:0/1037486258 shutdown_connections 2024-07-16T10:05:25.509 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:05:25.506+0000 7ff7a6d87640 1 --2- 172.21.15.156:0/1037486258 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7ff78806e0b0 0x7ff788070560 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:05:25.509 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:05:25.506+0000 7ff7a6d87640 1 --2- 172.21.15.156:0/1037486258 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff7a0101bf0 0x7ff7a01124f0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:05:25.509 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:05:25.506+0000 7ff7a6d87640 1 --2- 172.21.15.156:0/1037486258 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff7a0101250 0x7ff7a0111fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:05:25.509 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:05:25.506+0000 7ff7a6d87640 1 -- 172.21.15.156:0/1037486258 >> 172.21.15.156:0/1037486258 conn(0x7ff7a00f6e00 msgr2=0x7ff7a00f7dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T10:05:25.509 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:05:25.506+0000 7ff7a6d87640 1 -- 172.21.15.156:0/1037486258 shutdown_connections 2024-07-16T10:05:25.509 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:05:25.506+0000 7ff7a6d87640 1 -- 172.21.15.156:0/1037486258 wait complete. 2024-07-16T10:05:26.135 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 26c91c26-4358-11ef-bcad-c7b262605968 -e sha1=a6ca461be7cc1aff3658c2c6df4cff975a24449a -- bash -c 'ceph health detail' 2024-07-16T10:05:26.393 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/26c91c26-4358-11ef-bcad-c7b262605968/mon.smithi156/config 2024-07-16T10:05:26.424 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:05:26 smithi156 ceph-mon[93066]: pgmap v266: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-07-16T10:05:26.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:05:26 smithi176 ceph-mon[81689]: pgmap v266: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-07-16T10:05:27.477 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:05:27 smithi156 ceph-mon[93066]: from='client.34566 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-16T10:05:27.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:05:27 smithi176 ceph-mon[81689]: from='client.34566 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-16T10:05:27.985 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:05:27.982+0000 7f9a51cc2640 1 -- 172.21.15.156:0/3672869324 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9a4c1023a0 msgr2=0x7f9a4c102770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:05:27.985 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:05:27.982+0000 7f9a51cc2640 1 --2- 172.21.15.156:0/3672869324 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9a4c1023a0 0x7f9a4c102770 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f9a3c009c50 tx=0x7f9a3c004d10 comp rx=0 tx=0).stop 2024-07-16T10:05:27.985 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:05:27.982+0000 7f9a51cc2640 1 -- 172.21.15.156:0/3672869324 shutdown_connections 2024-07-16T10:05:27.985 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:05:27.982+0000 7f9a51cc2640 1 --2- 172.21.15.156:0/3672869324 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9a4c102d40 0x7f9a4c0faa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:05:27.985 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:05:27.982+0000 7f9a51cc2640 1 --2- 172.21.15.156:0/3672869324 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9a4c1023a0 0x7f9a4c102770 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:05:27.985 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:05:27.982+0000 7f9a51cc2640 1 -- 172.21.15.156:0/3672869324 >> 172.21.15.156:0/3672869324 conn(0x7f9a4c070af0 msgr2=0x7f9a4c070ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T10:05:27.985 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:05:27.982+0000 7f9a51cc2640 1 -- 172.21.15.156:0/3672869324 shutdown_connections 2024-07-16T10:05:27.985 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:05:27.982+0000 7f9a51cc2640 1 -- 172.21.15.156:0/3672869324 wait complete. 2024-07-16T10:05:27.986 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:05:27.983+0000 7f9a51cc2640 1 Processor -- start 2024-07-16T10:05:27.987 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:05:27.983+0000 7f9a51cc2640 1 -- start start 2024-07-16T10:05:27.987 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:05:27.983+0000 7f9a51cc2640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9a4c1023a0 0x7f9a4c113150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:05:27.987 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:05:27.983+0000 7f9a51cc2640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9a4c102d40 0x7f9a4c113690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:05:27.987 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:05:27.983+0000 7f9a51cc2640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a4c113c90 con 0x7f9a4c102d40 2024-07-16T10:05:27.987 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:05:27.983+0000 7f9a51cc2640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a4c10d0d0 con 0x7f9a4c1023a0 2024-07-16T10:05:27.988 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:05:27.983+0000 7f9a4affd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9a4c102d40 0x7f9a4c113690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:05:27.988 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:05:27.983+0000 7f9a4affd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9a4c102d40 0x7f9a4c113690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:43662/0 (socket says 172.21.15.156:43662) 2024-07-16T10:05:27.988 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:05:27.983+0000 7f9a4affd640 1 -- 172.21.15.156:0/1448387357 learned_addr learned my addr 172.21.15.156:0/1448387357 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T10:05:27.988 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:05:27.984+0000 7f9a4b7fe640 1 --2- 172.21.15.156:0/1448387357 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9a4c1023a0 0x7f9a4c113150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:05:27.988 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:05:27.984+0000 7f9a4affd640 1 -- 172.21.15.156:0/1448387357 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9a4c1023a0 msgr2=0x7f9a4c113150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:05:27.988 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:05:27.984+0000 7f9a4affd640 1 --2- 172.21.15.156:0/1448387357 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9a4c1023a0 0x7f9a4c113150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:05:27.988 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:05:27.984+0000 7f9a4affd640 1 -- 172.21.15.156:0/1448387357 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a3c0098b0 con 0x7f9a4c102d40 2024-07-16T10:05:27.988 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:05:27.984+0000 7f9a4b7fe640 1 --2- 172.21.15.156:0/1448387357 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9a4c1023a0 0x7f9a4c113150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-16T10:05:27.988 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:05:27.984+0000 7f9a4affd640 1 --2- 172.21.15.156:0/1448387357 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9a4c102d40 0x7f9a4c113690 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f9a40002a80 tx=0x7f9a40004570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T10:05:27.988 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:05:27.984+0000 7f9a33fff640 1 -- 172.21.15.156:0/1448387357 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9a40007c30 con 0x7f9a4c102d40 2024-07-16T10:05:27.988 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:05:27.984+0000 7f9a51cc2640 1 -- 172.21.15.156:0/1448387357 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a4c10d3b0 con 0x7f9a4c102d40 2024-07-16T10:05:27.988 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:05:27.984+0000 7f9a33fff640 1 -- 172.21.15.156:0/1448387357 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9a4000f040 con 0x7f9a4c102d40 2024-07-16T10:05:27.988 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:05:27.984+0000 7f9a33fff640 1 -- 172.21.15.156:0/1448387357 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9a40009640 con 0x7f9a4c102d40 2024-07-16T10:05:27.989 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:05:27.984+0000 7f9a51cc2640 1 -- 172.21.15.156:0/1448387357 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a4c10d8d0 con 0x7f9a4c102d40 2024-07-16T10:05:27.989 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:05:27.985+0000 7f9a51cc2640 1 -- 172.21.15.156:0/1448387357 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a4c0655c0 con 0x7f9a4c102d40 2024-07-16T10:05:27.989 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:05:27.986+0000 7f9a33fff640 1 -- 172.21.15.156:0/1448387357 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 38) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f9a4000d680 con 0x7f9a4c102d40 2024-07-16T10:05:27.989 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:05:27.986+0000 7f9a33fff640 1 --2- 172.21.15.156:0/1448387357 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f9a3406e0b0 0x7f9a34070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:05:27.990 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:05:27.986+0000 7f9a33fff640 1 -- 172.21.15.156:0/1448387357 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f9a4008d560 con 0x7f9a4c102d40 2024-07-16T10:05:27.990 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:05:27.986+0000 7f9a4b7fe640 1 --2- 172.21.15.156:0/1448387357 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f9a3406e0b0 0x7f9a34070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:05:27.990 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:05:27.987+0000 7f9a4b7fe640 1 --2- 172.21.15.156:0/1448387357 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f9a3406e0b0 0x7f9a34070560 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f9a3c000f80 tx=0x7f9a3c004d10 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T10:05:27.993 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:05:27.991+0000 7f9a33fff640 1 -- 172.21.15.156:0/1448387357 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187716 (secure 0 0 0) 0x7f9a40056830 con 0x7f9a4c102d40 2024-07-16T10:05:28.236 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:05:28.234+0000 7f9a51cc2640 1 -- 172.21.15.156:0/1448387357 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f9a4c109570 con 0x7f9a4c102d40 2024-07-16T10:05:28.236 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:05:28.234+0000 7f9a33fff640 1 -- 172.21.15.156:0/1448387357 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7f9a40059ef0 con 0x7f9a4c102d40 2024-07-16T10:05:28.237 INFO:teuthology.orchestra.run.smithi156.stdout:HEALTH_OK 2024-07-16T10:05:28.238 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:05:28.236+0000 7f9a51cc2640 1 -- 172.21.15.156:0/1448387357 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f9a3406e0b0 msgr2=0x7f9a34070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:05:28.239 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:05:28.236+0000 7f9a51cc2640 1 --2- 172.21.15.156:0/1448387357 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f9a3406e0b0 0x7f9a34070560 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f9a3c000f80 tx=0x7f9a3c004d10 comp rx=0 tx=0).stop 2024-07-16T10:05:28.239 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:05:28.236+0000 7f9a51cc2640 1 -- 172.21.15.156:0/1448387357 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9a4c102d40 msgr2=0x7f9a4c113690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:05:28.239 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:05:28.236+0000 7f9a51cc2640 1 --2- 172.21.15.156:0/1448387357 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9a4c102d40 0x7f9a4c113690 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f9a40002a80 tx=0x7f9a40004570 comp rx=0 tx=0).stop 2024-07-16T10:05:28.239 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:05:28.236+0000 7f9a51cc2640 1 -- 172.21.15.156:0/1448387357 shutdown_connections 2024-07-16T10:05:28.239 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:05:28.236+0000 7f9a51cc2640 1 --2- 172.21.15.156:0/1448387357 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f9a3406e0b0 0x7f9a34070560 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:05:28.239 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:05:28.236+0000 7f9a51cc2640 1 --2- 172.21.15.156:0/1448387357 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9a4c102d40 0x7f9a4c113690 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:05:28.239 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:05:28.236+0000 7f9a51cc2640 1 --2- 172.21.15.156:0/1448387357 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9a4c1023a0 0x7f9a4c113150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:05:28.239 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:05:28.236+0000 7f9a51cc2640 1 -- 172.21.15.156:0/1448387357 >> 172.21.15.156:0/1448387357 conn(0x7f9a4c070af0 msgr2=0x7f9a4c105160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T10:05:28.239 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:05:28.236+0000 7f9a51cc2640 1 -- 172.21.15.156:0/1448387357 shutdown_connections 2024-07-16T10:05:28.239 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:05:28.236+0000 7f9a51cc2640 1 -- 172.21.15.156:0/1448387357 wait complete. 2024-07-16T10:05:28.551 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:05:28 smithi156 ceph-mon[93066]: pgmap v267: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:05:28.551 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:05:28 smithi156 ceph-mon[93066]: from='client.? 172.21.15.156:0/1448387357' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-07-16T10:05:28.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:05:28 smithi176 ceph-mon[81689]: pgmap v267: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:05:28.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:05:28 smithi176 ceph-mon[81689]: from='client.? 172.21.15.156:0/1448387357' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-07-16T10:05:28.886 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 26c91c26-4358-11ef-bcad-c7b262605968 -e sha1=a6ca461be7cc1aff3658c2c6df4cff975a24449a -- bash -c 'ceph versions' 2024-07-16T10:05:29.142 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/26c91c26-4358-11ef-bcad-c7b262605968/mon.smithi156/config 2024-07-16T10:05:30.497 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:05:30 smithi156 ceph-mon[93066]: pgmap v268: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-07-16T10:05:30.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:05:30 smithi176 ceph-mon[81689]: pgmap v268: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-07-16T10:05:30.694 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:05:30.690+0000 7f139c1ed640 1 -- 172.21.15.156:0/2474907902 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f13940fab00 msgr2=0x7f13940faf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:05:30.694 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:05:30.690+0000 7f139c1ed640 1 --2- 172.21.15.156:0/2474907902 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f13940fab00 0x7f13940faf70 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f1388009c50 tx=0x7f1388004d10 comp rx=0 tx=0).stop 2024-07-16T10:05:30.694 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:05:30.690+0000 7f139c1ed640 1 -- 172.21.15.156:0/2474907902 shutdown_connections 2024-07-16T10:05:30.694 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:05:30.690+0000 7f139c1ed640 1 --2- 172.21.15.156:0/2474907902 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f13940fab00 0x7f13940faf70 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:05:30.694 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:05:30.690+0000 7f139c1ed640 1 --2- 172.21.15.156:0/2474907902 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f13941024d0 0x7f13941028a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:05:30.694 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:05:30.690+0000 7f139c1ed640 1 -- 172.21.15.156:0/2474907902 >> 172.21.15.156:0/2474907902 conn(0x7f1394070b60 msgr2=0x7f1394070f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T10:05:30.695 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:05:30.690+0000 7f139c1ed640 1 -- 172.21.15.156:0/2474907902 shutdown_connections 2024-07-16T10:05:30.695 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:05:30.690+0000 7f139c1ed640 1 -- 172.21.15.156:0/2474907902 wait complete. 2024-07-16T10:05:30.695 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:05:30.690+0000 7f139c1ed640 1 Processor -- start 2024-07-16T10:05:30.695 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:05:30.691+0000 7f139c1ed640 1 -- start start 2024-07-16T10:05:30.695 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:05:30.691+0000 7f139c1ed640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f13940fab00 0x7f13940fbce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:05:30.695 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:05:30.691+0000 7f139c1ed640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f13941024d0 0x7f13940fc220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:05:30.695 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:05:30.691+0000 7f139c1ed640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13940fc760 con 0x7f13940fab00 2024-07-16T10:05:30.695 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:05:30.691+0000 7f139c1ed640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13940fc8d0 con 0x7f13941024d0 2024-07-16T10:05:30.695 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:05:30.691+0000 7f1399f89640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f13940fab00 0x7f13940fbce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:05:30.695 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:05:30.691+0000 7f1399f89640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f13940fab00 0x7f13940fbce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:43676/0 (socket says 172.21.15.156:43676) 2024-07-16T10:05:30.695 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:05:30.691+0000 7f1399f89640 1 -- 172.21.15.156:0/690845078 learned_addr learned my addr 172.21.15.156:0/690845078 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T10:05:30.696 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:05:30.691+0000 7f1399788640 1 --2- 172.21.15.156:0/690845078 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f13941024d0 0x7f13940fc220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:05:30.696 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:05:30.691+0000 7f1399f89640 1 -- 172.21.15.156:0/690845078 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f13941024d0 msgr2=0x7f13940fc220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:05:30.696 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:05:30.691+0000 7f1399f89640 1 --2- 172.21.15.156:0/690845078 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f13941024d0 0x7f13940fc220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:05:30.696 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:05:30.691+0000 7f1399f89640 1 -- 172.21.15.156:0/690845078 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13880098b0 con 0x7f13940fab00 2024-07-16T10:05:30.696 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:05:30.691+0000 7f1399f89640 1 --2- 172.21.15.156:0/690845078 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f13940fab00 0x7f13940fbce0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f138c002a50 tx=0x7f138c004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T10:05:30.697 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:05:30.691+0000 7f13927fc640 1 -- 172.21.15.156:0/690845078 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f138c007c30 con 0x7f13940fab00 2024-07-16T10:05:30.697 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:05:30.692+0000 7f139c1ed640 1 -- 172.21.15.156:0/690845078 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13940fcb60 con 0x7f13940fab00 2024-07-16T10:05:30.697 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:05:30.692+0000 7f13927fc640 1 -- 172.21.15.156:0/690845078 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f138c00f040 con 0x7f13940fab00 2024-07-16T10:05:30.697 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:05:30.692+0000 7f13927fc640 1 -- 172.21.15.156:0/690845078 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f138c00db50 con 0x7f13940fab00 2024-07-16T10:05:30.697 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:05:30.692+0000 7f139c1ed640 1 -- 172.21.15.156:0/690845078 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13941197d0 con 0x7f13940fab00 2024-07-16T10:05:30.698 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:05:30.693+0000 7f139c1ed640 1 -- 172.21.15.156:0/690845078 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1394065600 con 0x7f13940fab00 2024-07-16T10:05:30.698 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:05:30.693+0000 7f13927fc640 1 -- 172.21.15.156:0/690845078 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 38) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f138c01c400 con 0x7f13940fab00 2024-07-16T10:05:30.699 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:05:30.693+0000 7f13927fc640 1 --2- 172.21.15.156:0/690845078 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f137c06e0b0 0x7f137c070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:05:30.699 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:05:30.694+0000 7f13927fc640 1 -- 172.21.15.156:0/690845078 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f138c08d4d0 con 0x7f13940fab00 2024-07-16T10:05:30.699 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:05:30.694+0000 7f1399788640 1 --2- 172.21.15.156:0/690845078 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f137c06e0b0 0x7f137c070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:05:30.699 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:05:30.694+0000 7f1399788640 1 --2- 172.21.15.156:0/690845078 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f137c06e0b0 0x7f137c070560 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f1388000f80 tx=0x7f1388004d10 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T10:05:30.700 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:05:30.698+0000 7f13927fc640 1 -- 172.21.15.156:0/690845078 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187716 (secure 0 0 0) 0x7f138c0567a0 con 0x7f13940fab00 2024-07-16T10:05:30.945 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:05:30.943+0000 7f139c1ed640 1 -- 172.21.15.156:0/690845078 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f13940fd3d0 con 0x7f13940fab00 2024-07-16T10:05:30.945 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:05:30.943+0000 7f13927fc640 1 -- 172.21.15.156:0/690845078 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+618 (secure 0 0 0) 0x7f138c059e60 con 0x7f13940fab00 2024-07-16T10:05:30.946 INFO:teuthology.orchestra.run.smithi156.stdout:{ 2024-07-16T10:05:30.946 INFO:teuthology.orchestra.run.smithi156.stdout: "mon": { 2024-07-16T10:05:30.946 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph version 19.0.0-3495-ga6ca461b (a6ca461be7cc1aff3658c2c6df4cff975a24449a) squid (rc)": 2 2024-07-16T10:05:30.946 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-07-16T10:05:30.946 INFO:teuthology.orchestra.run.smithi156.stdout: "mgr": { 2024-07-16T10:05:30.946 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph version 19.0.0-3495-ga6ca461b (a6ca461be7cc1aff3658c2c6df4cff975a24449a) squid (rc)": 2 2024-07-16T10:05:30.946 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-07-16T10:05:30.946 INFO:teuthology.orchestra.run.smithi156.stdout: "osd": { 2024-07-16T10:05:30.946 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph version 19.0.0-3495-ga6ca461b (a6ca461be7cc1aff3658c2c6df4cff975a24449a) squid (rc)": 6 2024-07-16T10:05:30.947 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-07-16T10:05:30.947 INFO:teuthology.orchestra.run.smithi156.stdout: "mds": { 2024-07-16T10:05:30.947 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph version 19.0.0-3495-ga6ca461b (a6ca461be7cc1aff3658c2c6df4cff975a24449a) squid (rc)": 4 2024-07-16T10:05:30.947 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-07-16T10:05:30.947 INFO:teuthology.orchestra.run.smithi156.stdout: "overall": { 2024-07-16T10:05:30.947 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph version 19.0.0-3495-ga6ca461b (a6ca461be7cc1aff3658c2c6df4cff975a24449a) squid (rc)": 14 2024-07-16T10:05:30.947 INFO:teuthology.orchestra.run.smithi156.stdout: } 2024-07-16T10:05:30.947 INFO:teuthology.orchestra.run.smithi156.stdout:} 2024-07-16T10:05:30.948 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:05:30.946+0000 7f139c1ed640 1 -- 172.21.15.156:0/690845078 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f137c06e0b0 msgr2=0x7f137c070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:05:30.948 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:05:30.946+0000 7f139c1ed640 1 --2- 172.21.15.156:0/690845078 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f137c06e0b0 0x7f137c070560 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f1388000f80 tx=0x7f1388004d10 comp rx=0 tx=0).stop 2024-07-16T10:05:30.948 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:05:30.946+0000 7f139c1ed640 1 -- 172.21.15.156:0/690845078 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f13940fab00 msgr2=0x7f13940fbce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:05:30.948 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:05:30.946+0000 7f139c1ed640 1 --2- 172.21.15.156:0/690845078 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f13940fab00 0x7f13940fbce0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f138c002a50 tx=0x7f138c004600 comp rx=0 tx=0).stop 2024-07-16T10:05:30.948 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:05:30.946+0000 7f139c1ed640 1 -- 172.21.15.156:0/690845078 shutdown_connections 2024-07-16T10:05:30.949 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:05:30.946+0000 7f139c1ed640 1 --2- 172.21.15.156:0/690845078 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f137c06e0b0 0x7f137c070560 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:05:30.949 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:05:30.946+0000 7f139c1ed640 1 --2- 172.21.15.156:0/690845078 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f13941024d0 0x7f13940fc220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:05:30.949 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:05:30.946+0000 7f139c1ed640 1 --2- 172.21.15.156:0/690845078 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f13940fab00 0x7f13940fbce0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:05:30.949 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:05:30.946+0000 7f139c1ed640 1 -- 172.21.15.156:0/690845078 >> 172.21.15.156:0/690845078 conn(0x7f1394070b60 msgr2=0x7f13940f9e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T10:05:30.949 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:05:30.946+0000 7f139c1ed640 1 -- 172.21.15.156:0/690845078 shutdown_connections 2024-07-16T10:05:30.949 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:05:30.946+0000 7f139c1ed640 1 -- 172.21.15.156:0/690845078 wait complete. 2024-07-16T10:05:31.584 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:05:31 smithi156 ceph-mon[93066]: from='client.? 172.21.15.156:0/690845078' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:05:31.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:05:31 smithi176 ceph-mon[81689]: from='client.? 172.21.15.156:0/690845078' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:05:31.609 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 26c91c26-4358-11ef-bcad-c7b262605968 -e sha1=a6ca461be7cc1aff3658c2c6df4cff975a24449a -- bash -c 'echo "wait for servicemap items w/ changing names to refresh"' 2024-07-16T10:05:31.868 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/26c91c26-4358-11ef-bcad-c7b262605968/mon.smithi156/config 2024-07-16T10:05:32.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:05:32 smithi176 ceph-mon[81689]: pgmap v269: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:05:32.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:05:32 smithi156 ceph-mon[93066]: pgmap v269: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:05:33.525 INFO:teuthology.orchestra.run.smithi156.stdout:wait for servicemap items w/ changing names to refresh 2024-07-16T10:05:34.184 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 26c91c26-4358-11ef-bcad-c7b262605968 -e sha1=a6ca461be7cc1aff3658c2c6df4cff975a24449a -- bash -c 'sleep 60' 2024-07-16T10:05:34.424 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:05:34 smithi156 ceph-mon[93066]: pgmap v270: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:05:34.424 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:05:34 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-16T10:05:34.441 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/26c91c26-4358-11ef-bcad-c7b262605968/mon.smithi156/config 2024-07-16T10:05:34.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:05:34 smithi176 ceph-mon[81689]: pgmap v270: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:05:34.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:05:34 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-16T10:05:36.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:05:36 smithi176 ceph-mon[81689]: pgmap v271: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-07-16T10:05:36.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:05:36 smithi156 ceph-mon[93066]: pgmap v271: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-07-16T10:05:38.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:05:38 smithi176 ceph-mon[81689]: pgmap v272: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:05:38.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:05:38 smithi156 ceph-mon[93066]: pgmap v272: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:05:40.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:05:40 smithi176 ceph-mon[81689]: pgmap v273: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-07-16T10:05:40.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:05:40 smithi156 ceph-mon[93066]: pgmap v273: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-07-16T10:05:42.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:05:42 smithi176 ceph-mon[81689]: pgmap v274: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:05:42.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:05:42 smithi156 ceph-mon[93066]: pgmap v274: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:05:44.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:05:44 smithi176 ceph-mon[81689]: pgmap v275: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:05:44.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:05:44 smithi156 ceph-mon[93066]: pgmap v275: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:05:46.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:05:46 smithi176 ceph-mon[81689]: pgmap v276: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-07-16T10:05:46.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:05:46 smithi156 ceph-mon[93066]: pgmap v276: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-07-16T10:05:48.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:05:48 smithi176 ceph-mon[81689]: pgmap v277: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:05:48.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:05:48 smithi156 ceph-mon[93066]: pgmap v277: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:05:49.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:05:49 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-16T10:05:49.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:05:49 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-16T10:05:50.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:05:50 smithi176 ceph-mon[81689]: pgmap v278: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-07-16T10:05:50.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:05:50 smithi156 ceph-mon[93066]: pgmap v278: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-07-16T10:05:52.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:05:52 smithi176 ceph-mon[81689]: pgmap v279: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:05:52.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:05:52 smithi156 ceph-mon[93066]: pgmap v279: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:05:54.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:05:54 smithi176 ceph-mon[81689]: pgmap v280: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:05:54.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:05:54 smithi156 ceph-mon[93066]: pgmap v280: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:05:56.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:05:56 smithi176 ceph-mon[81689]: pgmap v281: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-07-16T10:05:56.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:05:56 smithi156 ceph-mon[93066]: pgmap v281: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-07-16T10:05:58.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:05:58 smithi156 ceph-mon[93066]: pgmap v282: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:05:58.843 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:05:58 smithi176 ceph-mon[81689]: pgmap v282: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:06:00.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:06:00 smithi156 ceph-mon[93066]: pgmap v283: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-07-16T10:06:00.843 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:06:00 smithi176 ceph-mon[81689]: pgmap v283: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-07-16T10:06:02.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:06:02 smithi156 ceph-mon[93066]: pgmap v284: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:06:02.843 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:06:02 smithi176 ceph-mon[81689]: pgmap v284: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:06:04.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:06:04 smithi156 ceph-mon[93066]: pgmap v285: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:06:04.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:06:04 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-16T10:06:04.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:06:04 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-16T10:06:04.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:06:04 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-16T10:06:04.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:06:04 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-16T10:06:04.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:06:04 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:06:04.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:06:04 smithi176 ceph-mon[81689]: pgmap v285: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:06:04.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:06:04 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-16T10:06:04.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:06:04 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-16T10:06:04.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:06:04 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-16T10:06:04.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:06:04 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-16T10:06:04.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:06:04 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:06:06.646 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:06:06 smithi176 ceph-mon[81689]: pgmap v286: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-07-16T10:06:06.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:06:06 smithi156 ceph-mon[93066]: pgmap v286: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-07-16T10:06:08.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:06:08 smithi156 ceph-mon[93066]: pgmap v287: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:06:08.843 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:06:08 smithi176 ceph-mon[81689]: pgmap v287: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:06:10.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:06:10 smithi156 ceph-mon[93066]: pgmap v288: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-07-16T10:06:10.843 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:06:10 smithi176 ceph-mon[81689]: pgmap v288: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-07-16T10:06:12.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:06:12 smithi156 ceph-mon[93066]: pgmap v289: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:06:12.843 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:06:12 smithi176 ceph-mon[81689]: pgmap v289: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:06:14.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:06:14 smithi156 ceph-mon[93066]: pgmap v290: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:06:14.843 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:06:14 smithi176 ceph-mon[81689]: pgmap v290: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:06:16.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:06:16 smithi156 ceph-mon[93066]: pgmap v291: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-07-16T10:06:16.843 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:06:16 smithi176 ceph-mon[81689]: pgmap v291: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-07-16T10:06:18.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:06:18 smithi156 ceph-mon[93066]: pgmap v292: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:06:18.843 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:06:18 smithi176 ceph-mon[81689]: pgmap v292: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:06:19.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:06:19 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-16T10:06:19.843 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:06:19 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-16T10:06:20.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:06:20 smithi156 ceph-mon[93066]: pgmap v293: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-07-16T10:06:20.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:06:20 smithi176 ceph-mon[81689]: pgmap v293: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-07-16T10:06:22.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:06:22 smithi156 ceph-mon[93066]: pgmap v294: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:06:22.843 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:06:22 smithi176 ceph-mon[81689]: pgmap v294: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:06:24.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:06:24 smithi156 ceph-mon[93066]: pgmap v295: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:06:24.843 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:06:24 smithi176 ceph-mon[81689]: pgmap v295: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:06:26.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:06:26 smithi156 ceph-mon[93066]: pgmap v296: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-07-16T10:06:26.843 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:06:26 smithi176 ceph-mon[81689]: pgmap v296: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-07-16T10:06:28.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:06:28 smithi156 ceph-mon[93066]: pgmap v297: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:06:28.843 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:06:28 smithi176 ceph-mon[81689]: pgmap v297: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:06:29.843 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:06:29 smithi176 ceph-mon[81689]: pgmap v298: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-07-16T10:06:29.924 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:06:29 smithi156 ceph-mon[93066]: pgmap v298: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-07-16T10:06:31.843 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:06:31 smithi176 ceph-mon[81689]: pgmap v299: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:06:31.924 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:06:31 smithi156 ceph-mon[93066]: pgmap v299: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:06:33.843 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:06:33 smithi176 ceph-mon[81689]: pgmap v300: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:06:33.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:06:33 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-16T10:06:33.924 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:06:33 smithi156 ceph-mon[93066]: pgmap v300: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:06:33.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:06:33 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-16T10:06:35.794 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:06:35 smithi156 ceph-mon[93066]: pgmap v301: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-07-16T10:06:35.843 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:06:35 smithi176 ceph-mon[81689]: pgmap v301: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-07-16T10:06:36.419 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 26c91c26-4358-11ef-bcad-c7b262605968 -e sha1=a6ca461be7cc1aff3658c2c6df4cff975a24449a -- bash -c 'ceph orch ps' 2024-07-16T10:06:36.686 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/26c91c26-4358-11ef-bcad-c7b262605968/mon.smithi156/config 2024-07-16T10:06:37.761 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:06:37 smithi156 ceph-mon[93066]: pgmap v302: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:06:37.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:06:37 smithi176 ceph-mon[81689]: pgmap v302: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:06:38.247 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:38.242+0000 7fcc0f431640 1 -- 172.21.15.156:0/3481548887 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcc08102d10 msgr2=0x7fcc080fc3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:06:38.247 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:38.242+0000 7fcc0f431640 1 --2- 172.21.15.156:0/3481548887 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcc08102d10 0x7fcc080fc3a0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fcc040048d0 tx=0x7fcc04009da0 comp rx=0 tx=0).stop 2024-07-16T10:06:38.247 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:38.242+0000 7fcc0f431640 1 -- 172.21.15.156:0/3481548887 shutdown_connections 2024-07-16T10:06:38.247 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:38.242+0000 7fcc0f431640 1 --2- 172.21.15.156:0/3481548887 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcc08102d10 0x7fcc080fc3a0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:06:38.247 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:38.242+0000 7fcc0f431640 1 --2- 172.21.15.156:0/3481548887 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fcc08102370 0x7fcc08102740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:06:38.247 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:38.242+0000 7fcc0f431640 1 -- 172.21.15.156:0/3481548887 >> 172.21.15.156:0/3481548887 conn(0x7fcc080f7f00 msgr2=0x7fcc080fa310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T10:06:38.247 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:38.242+0000 7fcc0f431640 1 -- 172.21.15.156:0/3481548887 shutdown_connections 2024-07-16T10:06:38.247 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:38.242+0000 7fcc0f431640 1 -- 172.21.15.156:0/3481548887 wait complete. 2024-07-16T10:06:38.248 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:38.243+0000 7fcc0f431640 1 Processor -- start 2024-07-16T10:06:38.248 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:38.243+0000 7fcc0f431640 1 -- start start 2024-07-16T10:06:38.248 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:38.243+0000 7fcc0f431640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcc08102370 0x7fcc0806f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:06:38.248 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:38.243+0000 7fcc0f431640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fcc08102d10 0x7fcc0806fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:06:38.248 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:38.243+0000 7fcc0f431640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc080702a0 con 0x7fcc08102370 2024-07-16T10:06:38.248 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:38.243+0000 7fcc0f431640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc08070410 con 0x7fcc08102d10 2024-07-16T10:06:38.248 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:38.243+0000 7fcc0d1cd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcc08102370 0x7fcc0806f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:06:38.248 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:38.243+0000 7fcc0d1cd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcc08102370 0x7fcc0806f820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:56642/0 (socket says 172.21.15.156:56642) 2024-07-16T10:06:38.248 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:38.243+0000 7fcc0d1cd640 1 -- 172.21.15.156:0/4173293260 learned_addr learned my addr 172.21.15.156:0/4173293260 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T10:06:38.248 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:38.244+0000 7fcc0c9cc640 1 --2- 172.21.15.156:0/4173293260 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fcc08102d10 0x7fcc0806fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:06:38.248 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:38.244+0000 7fcc0d1cd640 1 -- 172.21.15.156:0/4173293260 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fcc08102d10 msgr2=0x7fcc0806fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:06:38.249 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:38.244+0000 7fcc0d1cd640 1 --2- 172.21.15.156:0/4173293260 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fcc08102d10 0x7fcc0806fd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:06:38.249 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:38.244+0000 7fcc0d1cd640 1 -- 172.21.15.156:0/4173293260 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc04005040 con 0x7fcc08102370 2024-07-16T10:06:38.249 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:38.244+0000 7fcc0d1cd640 1 --2- 172.21.15.156:0/4173293260 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcc08102370 0x7fcc0806f820 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fcbf800eb70 tx=0x7fcbf800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T10:06:38.249 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:38.244+0000 7fcbfdffb640 1 -- 172.21.15.156:0/4173293260 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcbf8007c30 con 0x7fcc08102370 2024-07-16T10:06:38.249 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:38.244+0000 7fcbfdffb640 1 -- 172.21.15.156:0/4173293260 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcbf800f040 con 0x7fcc08102370 2024-07-16T10:06:38.249 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:38.244+0000 7fcc0f431640 1 -- 172.21.15.156:0/4173293260 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc08070640 con 0x7fcc08102370 2024-07-16T10:06:38.249 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:38.244+0000 7fcbfdffb640 1 -- 172.21.15.156:0/4173293260 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcbf80095b0 con 0x7fcc08102370 2024-07-16T10:06:38.249 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:38.244+0000 7fcc0f431640 1 -- 172.21.15.156:0/4173293260 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc08119af0 con 0x7fcc08102370 2024-07-16T10:06:38.250 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:38.245+0000 7fcc0f431640 1 -- 172.21.15.156:0/4173293260 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc080fdae0 con 0x7fcc08102370 2024-07-16T10:06:38.250 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:38.246+0000 7fcbfdffb640 1 -- 172.21.15.156:0/4173293260 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 38) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fcbf8009730 con 0x7fcc08102370 2024-07-16T10:06:38.251 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:38.246+0000 7fcbfdffb640 1 --2- 172.21.15.156:0/4173293260 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7fcbf406e0b0 0x7fcbf4070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:06:38.251 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:38.246+0000 7fcbfdffb640 1 -- 172.21.15.156:0/4173293260 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7fcbf808d970 con 0x7fcc08102370 2024-07-16T10:06:38.251 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:38.246+0000 7fcc0c9cc640 1 --2- 172.21.15.156:0/4173293260 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7fcbf406e0b0 0x7fcbf4070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:06:38.251 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:38.247+0000 7fcc0c9cc640 1 --2- 172.21.15.156:0/4173293260 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7fcbf406e0b0 0x7fcbf4070560 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fcc04009f80 tx=0x7fcc04035040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T10:06:38.253 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:38.251+0000 7fcbfdffb640 1 -- 172.21.15.156:0/4173293260 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187716 (secure 0 0 0) 0x7fcbf8056c40 con 0x7fcc08102370 2024-07-16T10:06:38.441 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:38.438+0000 7fcc0f431640 1 -- 172.21.15.156:0/4173293260 --> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fcc08070f10 con 0x7fcbf406e0b0 2024-07-16T10:06:38.449 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:38.447+0000 7fcbfdffb640 1 -- 172.21.15.156:0/4173293260 <== mgr.34104 v2:172.21.15.156:6800/3367765292 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7fcc08070f10 con 0x7fcbf406e0b0 2024-07-16T10:06:38.450 INFO:teuthology.orchestra.run.smithi156.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-07-16T10:06:38.450 INFO:teuthology.orchestra.run.smithi156.stdout:alertmanager.smithi156 smithi156 *:9093,9094 running (2m) 99s ago 18m 16.2M - 0.25.0 c8568f914cd2 10ea5635099c 2024-07-16T10:06:38.450 INFO:teuthology.orchestra.run.smithi156.stdout:crash.smithi156 smithi156 running (8m) 99s ago 18m 6899k - 19.0.0-3495-ga6ca461b 16f7feb04342 11648cf29af4 2024-07-16T10:06:38.450 INFO:teuthology.orchestra.run.smithi156.stdout:crash.smithi176 smithi176 running (7m) 2m ago 18m 6899k - 19.0.0-3495-ga6ca461b 16f7feb04342 67f3ccc034f8 2024-07-16T10:06:38.450 INFO:teuthology.orchestra.run.smithi156.stdout:grafana.smithi156 smithi156 *:3000 running (103s) 99s ago 18m 45.4M - 9.4.12 2bacad6d85d8 b5801e6c9d2d 2024-07-16T10:06:38.450 INFO:teuthology.orchestra.run.smithi156.stdout:mds.cephfs.smithi156.evwwaz smithi156 running (3m) 99s ago 13m 16.2M - 19.0.0-3495-ga6ca461b 16f7feb04342 75879f672b07 2024-07-16T10:06:38.450 INFO:teuthology.orchestra.run.smithi156.stdout:mds.cephfs.smithi156.rdibys smithi156 running (3m) 99s ago 13m 64.6M - 19.0.0-3495-ga6ca461b 16f7feb04342 248f61472011 2024-07-16T10:06:38.450 INFO:teuthology.orchestra.run.smithi156.stdout:mds.cephfs.smithi176.dvvbud smithi176 running (3m) 2m ago 13m 133M - 19.0.0-3495-ga6ca461b 16f7feb04342 3aaf56c30dde 2024-07-16T10:06:38.450 INFO:teuthology.orchestra.run.smithi156.stdout:mds.cephfs.smithi176.wahbsi smithi176 running (2m) 2m ago 13m 14.3M - 19.0.0-3495-ga6ca461b 16f7feb04342 d30f4f63a683 2024-07-16T10:06:38.451 INFO:teuthology.orchestra.run.smithi156.stdout:mgr.smithi156.zkvylv smithi156 *:8443,9283,8765 running (9m) 99s ago 20m 516M - 19.0.0-3495-ga6ca461b 16f7feb04342 ee53bd604b7c 2024-07-16T10:06:38.451 INFO:teuthology.orchestra.run.smithi156.stdout:mgr.smithi176.dqbrvx smithi176 *:8443,9283,8765 running (9m) 2m ago 17m 442M - 19.0.0-3495-ga6ca461b 16f7feb04342 ce9eaae4c2d0 2024-07-16T10:06:38.451 INFO:teuthology.orchestra.run.smithi156.stdout:mon.smithi156 smithi156 running (8m) 99s ago 20m 53.8M 2048M 19.0.0-3495-ga6ca461b 16f7feb04342 77b6ec8c50eb 2024-07-16T10:06:38.451 INFO:teuthology.orchestra.run.smithi156.stdout:mon.smithi176 smithi176 running (8m) 2m ago 17m 39.8M 2048M 19.0.0-3495-ga6ca461b 16f7feb04342 ef92814cec2b 2024-07-16T10:06:38.451 INFO:teuthology.orchestra.run.smithi156.stdout:node-exporter.smithi156 smithi156 *:9100 running (2m) 99s ago 18m 8178k - 1.5.0 0da6a335fe13 a67719f4e14a 2024-07-16T10:06:38.451 INFO:teuthology.orchestra.run.smithi156.stdout:node-exporter.smithi176 smithi176 *:9100 running (2m) 2m ago 17m 3598k - 1.5.0 0da6a335fe13 87cdbf1674aa 2024-07-16T10:06:38.451 INFO:teuthology.orchestra.run.smithi156.stdout:osd.0 smithi156 running (7m) 99s ago 17m 105M 1557M 19.0.0-3495-ga6ca461b 16f7feb04342 8633df1689cc 2024-07-16T10:06:38.451 INFO:teuthology.orchestra.run.smithi156.stdout:osd.1 smithi156 running (7m) 99s ago 16m 116M 1557M 19.0.0-3495-ga6ca461b 16f7feb04342 e6fae9653b46 2024-07-16T10:06:38.451 INFO:teuthology.orchestra.run.smithi156.stdout:osd.2 smithi156 running (6m) 99s ago 16m 97.8M 1557M 19.0.0-3495-ga6ca461b 16f7feb04342 c6ecf18de9bf 2024-07-16T10:06:38.451 INFO:teuthology.orchestra.run.smithi156.stdout:osd.3 smithi176 running (5m) 2m ago 15m 113M 2581M 19.0.0-3495-ga6ca461b 16f7feb04342 d216534d6027 2024-07-16T10:06:38.452 INFO:teuthology.orchestra.run.smithi156.stdout:osd.4 smithi176 running (5m) 2m ago 15m 109M 2581M 19.0.0-3495-ga6ca461b 16f7feb04342 1bd3a25317b0 2024-07-16T10:06:38.452 INFO:teuthology.orchestra.run.smithi156.stdout:osd.5 smithi176 running (4m) 2m ago 15m 95.7M 2581M 19.0.0-3495-ga6ca461b 16f7feb04342 a03640221e7a 2024-07-16T10:06:38.452 INFO:teuthology.orchestra.run.smithi156.stdout:prometheus.smithi156 smithi156 *:9095 running (2m) 99s ago 18m 46.9M - 2.43.0 a07b618ecd1d f0897441abda 2024-07-16T10:06:38.453 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:38.449+0000 7fcc0f431640 1 -- 172.21.15.156:0/4173293260 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7fcbf406e0b0 msgr2=0x7fcbf4070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:06:38.453 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:38.449+0000 7fcc0f431640 1 --2- 172.21.15.156:0/4173293260 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7fcbf406e0b0 0x7fcbf4070560 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fcc04009f80 tx=0x7fcc04035040 comp rx=0 tx=0).stop 2024-07-16T10:06:38.453 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:38.449+0000 7fcc0f431640 1 -- 172.21.15.156:0/4173293260 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcc08102370 msgr2=0x7fcc0806f820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:06:38.453 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:38.449+0000 7fcc0f431640 1 --2- 172.21.15.156:0/4173293260 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcc08102370 0x7fcc0806f820 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fcbf800eb70 tx=0x7fcbf800b040 comp rx=0 tx=0).stop 2024-07-16T10:06:38.453 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:38.450+0000 7fcc0f431640 1 -- 172.21.15.156:0/4173293260 shutdown_connections 2024-07-16T10:06:38.453 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:38.450+0000 7fcc0f431640 1 --2- 172.21.15.156:0/4173293260 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7fcbf406e0b0 0x7fcbf4070560 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:06:38.453 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:38.450+0000 7fcc0f431640 1 --2- 172.21.15.156:0/4173293260 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fcc08102d10 0x7fcc0806fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:06:38.454 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:38.450+0000 7fcc0f431640 1 --2- 172.21.15.156:0/4173293260 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcc08102370 0x7fcc0806f820 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:06:38.454 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:38.450+0000 7fcc0f431640 1 -- 172.21.15.156:0/4173293260 >> 172.21.15.156:0/4173293260 conn(0x7fcc080f7f00 msgr2=0x7fcc080f8e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T10:06:38.454 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:38.450+0000 7fcc0f431640 1 -- 172.21.15.156:0/4173293260 shutdown_connections 2024-07-16T10:06:38.454 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:38.450+0000 7fcc0f431640 1 -- 172.21.15.156:0/4173293260 wait complete. 2024-07-16T10:06:39.069 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 26c91c26-4358-11ef-bcad-c7b262605968 -e sha1=a6ca461be7cc1aff3658c2c6df4cff975a24449a -- bash -c 'ceph versions' 2024-07-16T10:06:39.335 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/26c91c26-4358-11ef-bcad-c7b262605968/mon.smithi156/config 2024-07-16T10:06:39.602 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:06:39 smithi156 ceph-mon[93066]: pgmap v303: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-07-16T10:06:39.602 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:06:39 smithi156 ceph-mon[93066]: from='client.34578 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-07-16T10:06:39.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:06:39 smithi176 ceph-mon[81689]: pgmap v303: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-07-16T10:06:39.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:06:39 smithi176 ceph-mon[81689]: from='client.34578 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-07-16T10:06:40.923 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:40.909+0000 7ff5cac92640 1 -- 172.21.15.156:0/3281600330 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff5c4102350 msgr2=0x7ff5c4102720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:06:40.923 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:40.909+0000 7ff5cac92640 1 --2- 172.21.15.156:0/3281600330 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff5c4102350 0x7ff5c4102720 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7ff5c00048d0 tx=0x7ff5c0009da0 comp rx=0 tx=0).stop 2024-07-16T10:06:40.923 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:40.909+0000 7ff5cac92640 1 -- 172.21.15.156:0/3281600330 shutdown_connections 2024-07-16T10:06:40.923 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:40.909+0000 7ff5cac92640 1 --2- 172.21.15.156:0/3281600330 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff5c4102cf0 0x7ff5c40fc380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:06:40.923 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:40.909+0000 7ff5cac92640 1 --2- 172.21.15.156:0/3281600330 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff5c4102350 0x7ff5c4102720 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:06:40.923 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:40.909+0000 7ff5cac92640 1 -- 172.21.15.156:0/3281600330 >> 172.21.15.156:0/3281600330 conn(0x7ff5c40f7f20 msgr2=0x7ff5c40fa330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T10:06:40.923 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:40.909+0000 7ff5cac92640 1 -- 172.21.15.156:0/3281600330 shutdown_connections 2024-07-16T10:06:40.923 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:40.909+0000 7ff5cac92640 1 -- 172.21.15.156:0/3281600330 wait complete. 2024-07-16T10:06:40.923 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:40.910+0000 7ff5cac92640 1 Processor -- start 2024-07-16T10:06:40.923 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:40.910+0000 7ff5cac92640 1 -- start start 2024-07-16T10:06:40.924 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:40.910+0000 7ff5cac92640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff5c4102350 0x7ff5c406f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:06:40.924 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:40.910+0000 7ff5cac92640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff5c4102cf0 0x7ff5c406fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:06:40.924 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:40.910+0000 7ff5cac92640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5c40702a0 con 0x7ff5c4102cf0 2024-07-16T10:06:40.924 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:40.910+0000 7ff5cac92640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5c4070410 con 0x7ff5c4102350 2024-07-16T10:06:40.924 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:40.910+0000 7ff5bbfff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff5c4102cf0 0x7ff5c406fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:06:40.924 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:40.910+0000 7ff5bbfff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff5c4102cf0 0x7ff5c406fd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:56664/0 (socket says 172.21.15.156:56664) 2024-07-16T10:06:40.924 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:40.910+0000 7ff5bbfff640 1 -- 172.21.15.156:0/525109692 learned_addr learned my addr 172.21.15.156:0/525109692 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T10:06:40.924 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:40.911+0000 7ff5c8a2e640 1 --2- 172.21.15.156:0/525109692 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff5c4102350 0x7ff5c406f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:06:40.924 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:40.911+0000 7ff5bbfff640 1 -- 172.21.15.156:0/525109692 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff5c4102350 msgr2=0x7ff5c406f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:06:40.924 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:40.911+0000 7ff5bbfff640 1 --2- 172.21.15.156:0/525109692 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff5c4102350 0x7ff5c406f820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:06:40.924 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:40.911+0000 7ff5bbfff640 1 -- 172.21.15.156:0/525109692 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5c0005040 con 0x7ff5c4102cf0 2024-07-16T10:06:40.925 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:40.911+0000 7ff5bbfff640 1 --2- 172.21.15.156:0/525109692 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff5c4102cf0 0x7ff5c406fd60 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7ff5b400c830 tx=0x7ff5b4004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T10:06:40.925 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:40.911+0000 7ff5b97fa640 1 -- 172.21.15.156:0/525109692 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff5b400a850 con 0x7ff5c4102cf0 2024-07-16T10:06:40.925 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:40.911+0000 7ff5b97fa640 1 -- 172.21.15.156:0/525109692 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff5b400ae80 con 0x7ff5c4102cf0 2024-07-16T10:06:40.925 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:40.911+0000 7ff5cac92640 1 -- 172.21.15.156:0/525109692 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5c40706a0 con 0x7ff5c4102cf0 2024-07-16T10:06:40.925 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:40.911+0000 7ff5b97fa640 1 -- 172.21.15.156:0/525109692 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff5b4012ea0 con 0x7ff5c4102cf0 2024-07-16T10:06:40.925 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:40.911+0000 7ff5cac92640 1 -- 172.21.15.156:0/525109692 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5c41197c0 con 0x7ff5c4102cf0 2024-07-16T10:06:40.925 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:40.912+0000 7ff5cac92640 1 -- 172.21.15.156:0/525109692 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5c40fdac0 con 0x7ff5c4102cf0 2024-07-16T10:06:40.925 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:40.913+0000 7ff5b97fa640 1 -- 172.21.15.156:0/525109692 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 38) v1 ==== 92205+0+0 (secure 0 0 0) 0x7ff5b4012400 con 0x7ff5c4102cf0 2024-07-16T10:06:40.925 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:40.913+0000 7ff5b97fa640 1 --2- 172.21.15.156:0/525109692 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7ff5ac06e0b0 0x7ff5ac070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:06:40.925 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:40.913+0000 7ff5b97fa640 1 -- 172.21.15.156:0/525109692 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7ff5b408de40 con 0x7ff5c4102cf0 2024-07-16T10:06:40.925 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:40.913+0000 7ff5c8a2e640 1 --2- 172.21.15.156:0/525109692 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7ff5ac06e0b0 0x7ff5ac070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:06:40.925 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:40.913+0000 7ff5c8a2e640 1 --2- 172.21.15.156:0/525109692 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7ff5ac06e0b0 0x7ff5ac070560 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7ff5c0000c00 tx=0x7ff5c0035040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T10:06:40.926 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:40.917+0000 7ff5b97fa640 1 -- 172.21.15.156:0/525109692 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187716 (secure 0 0 0) 0x7ff5b4057190 con 0x7ff5c4102cf0 2024-07-16T10:06:41.168 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:41.165+0000 7ff5cac92640 1 -- 172.21.15.156:0/525109692 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7ff5c41094c0 con 0x7ff5c4102cf0 2024-07-16T10:06:41.168 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:41.166+0000 7ff5b97fa640 1 -- 172.21.15.156:0/525109692 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+618 (secure 0 0 0) 0x7ff5b40126b0 con 0x7ff5c4102cf0 2024-07-16T10:06:41.169 INFO:teuthology.orchestra.run.smithi156.stdout:{ 2024-07-16T10:06:41.169 INFO:teuthology.orchestra.run.smithi156.stdout: "mon": { 2024-07-16T10:06:41.169 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph version 19.0.0-3495-ga6ca461b (a6ca461be7cc1aff3658c2c6df4cff975a24449a) squid (rc)": 2 2024-07-16T10:06:41.169 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-07-16T10:06:41.169 INFO:teuthology.orchestra.run.smithi156.stdout: "mgr": { 2024-07-16T10:06:41.169 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph version 19.0.0-3495-ga6ca461b (a6ca461be7cc1aff3658c2c6df4cff975a24449a) squid (rc)": 2 2024-07-16T10:06:41.169 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-07-16T10:06:41.169 INFO:teuthology.orchestra.run.smithi156.stdout: "osd": { 2024-07-16T10:06:41.169 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph version 19.0.0-3495-ga6ca461b (a6ca461be7cc1aff3658c2c6df4cff975a24449a) squid (rc)": 6 2024-07-16T10:06:41.169 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-07-16T10:06:41.170 INFO:teuthology.orchestra.run.smithi156.stdout: "mds": { 2024-07-16T10:06:41.170 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph version 19.0.0-3495-ga6ca461b (a6ca461be7cc1aff3658c2c6df4cff975a24449a) squid (rc)": 4 2024-07-16T10:06:41.170 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-07-16T10:06:41.170 INFO:teuthology.orchestra.run.smithi156.stdout: "overall": { 2024-07-16T10:06:41.170 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph version 19.0.0-3495-ga6ca461b (a6ca461be7cc1aff3658c2c6df4cff975a24449a) squid (rc)": 14 2024-07-16T10:06:41.170 INFO:teuthology.orchestra.run.smithi156.stdout: } 2024-07-16T10:06:41.170 INFO:teuthology.orchestra.run.smithi156.stdout:} 2024-07-16T10:06:41.171 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:41.168+0000 7ff5cac92640 1 -- 172.21.15.156:0/525109692 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7ff5ac06e0b0 msgr2=0x7ff5ac070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:06:41.171 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:41.168+0000 7ff5cac92640 1 --2- 172.21.15.156:0/525109692 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7ff5ac06e0b0 0x7ff5ac070560 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7ff5c0000c00 tx=0x7ff5c0035040 comp rx=0 tx=0).stop 2024-07-16T10:06:41.171 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:41.168+0000 7ff5cac92640 1 -- 172.21.15.156:0/525109692 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff5c4102cf0 msgr2=0x7ff5c406fd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:06:41.172 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:41.168+0000 7ff5cac92640 1 --2- 172.21.15.156:0/525109692 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff5c4102cf0 0x7ff5c406fd60 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7ff5b400c830 tx=0x7ff5b4004430 comp rx=0 tx=0).stop 2024-07-16T10:06:41.172 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:41.168+0000 7ff5cac92640 1 -- 172.21.15.156:0/525109692 shutdown_connections 2024-07-16T10:06:41.172 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:41.168+0000 7ff5cac92640 1 --2- 172.21.15.156:0/525109692 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7ff5ac06e0b0 0x7ff5ac070560 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:06:41.172 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:41.168+0000 7ff5cac92640 1 --2- 172.21.15.156:0/525109692 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff5c4102cf0 0x7ff5c406fd60 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:06:41.172 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:41.168+0000 7ff5cac92640 1 --2- 172.21.15.156:0/525109692 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff5c4102350 0x7ff5c406f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:06:41.172 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:41.168+0000 7ff5cac92640 1 -- 172.21.15.156:0/525109692 >> 172.21.15.156:0/525109692 conn(0x7ff5c40f7f20 msgr2=0x7ff5c40f8ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T10:06:41.172 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:41.168+0000 7ff5cac92640 1 -- 172.21.15.156:0/525109692 shutdown_connections 2024-07-16T10:06:41.172 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:41.168+0000 7ff5cac92640 1 -- 172.21.15.156:0/525109692 wait complete. 2024-07-16T10:06:41.785 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:06:41 smithi156 ceph-mon[93066]: pgmap v304: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:06:41.785 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:06:41 smithi156 ceph-mon[93066]: from='client.? 172.21.15.156:0/525109692' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:06:41.809 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 26c91c26-4358-11ef-bcad-c7b262605968 -e sha1=a6ca461be7cc1aff3658c2c6df4cff975a24449a -- bash -c 'ceph versions | jq -e '"'"'.overall | length == 1'"'"'' 2024-07-16T10:06:41.843 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:06:41 smithi176 ceph-mon[81689]: pgmap v304: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:06:41.843 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:06:41 smithi176 ceph-mon[81689]: from='client.? 172.21.15.156:0/525109692' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:06:42.068 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/26c91c26-4358-11ef-bcad-c7b262605968/mon.smithi156/config 2024-07-16T10:06:43.605 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:43.598+0000 7ff15b93f640 1 -- 172.21.15.156:0/373684776 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff1540fb620 msgr2=0x7ff1540fb9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:06:43.605 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:43.598+0000 7ff15b93f640 1 --2- 172.21.15.156:0/373684776 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff1540fb620 0x7ff1540fb9f0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7ff14c0048d0 tx=0x7ff14c009da0 comp rx=0 tx=0).stop 2024-07-16T10:06:43.605 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:43.599+0000 7ff15b93f640 1 -- 172.21.15.156:0/373684776 shutdown_connections 2024-07-16T10:06:43.605 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:43.599+0000 7ff15b93f640 1 --2- 172.21.15.156:0/373684776 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff1540fbf30 0x7ff1540fce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:06:43.605 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:43.599+0000 7ff15b93f640 1 --2- 172.21.15.156:0/373684776 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff1540fb620 0x7ff1540fb9f0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:06:43.605 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:43.599+0000 7ff15b93f640 1 -- 172.21.15.156:0/373684776 >> 172.21.15.156:0/373684776 conn(0x7ff1540721c0 msgr2=0x7ff1540725c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T10:06:43.605 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:43.599+0000 7ff15b93f640 1 -- 172.21.15.156:0/373684776 shutdown_connections 2024-07-16T10:06:43.606 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:43.599+0000 7ff15b93f640 1 -- 172.21.15.156:0/373684776 wait complete. 2024-07-16T10:06:43.606 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:43.599+0000 7ff15b93f640 1 Processor -- start 2024-07-16T10:06:43.606 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:43.599+0000 7ff15b93f640 1 -- start start 2024-07-16T10:06:43.606 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:43.599+0000 7ff15b93f640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff1540fb620 0x7ff15410be60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:06:43.606 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:43.599+0000 7ff15b93f640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff1540fbf30 0x7ff15410c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:06:43.606 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:43.599+0000 7ff15b93f640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff15410c8e0 con 0x7ff1540fb620 2024-07-16T10:06:43.606 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:43.599+0000 7ff15b93f640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff15410ca20 con 0x7ff1540fbf30 2024-07-16T10:06:43.606 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:43.599+0000 7ff1596db640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff1540fb620 0x7ff15410be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:06:43.606 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:43.600+0000 7ff1596db640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff1540fb620 0x7ff15410be60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:39904/0 (socket says 172.21.15.156:39904) 2024-07-16T10:06:43.606 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:43.600+0000 7ff1596db640 1 -- 172.21.15.156:0/27499822 learned_addr learned my addr 172.21.15.156:0/27499822 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T10:06:43.606 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:43.600+0000 7ff158eda640 1 --2- 172.21.15.156:0/27499822 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff1540fbf30 0x7ff15410c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:06:43.606 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:43.600+0000 7ff1596db640 1 -- 172.21.15.156:0/27499822 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff1540fbf30 msgr2=0x7ff15410c3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:06:43.607 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:43.600+0000 7ff1596db640 1 --2- 172.21.15.156:0/27499822 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff1540fbf30 0x7ff15410c3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:06:43.607 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:43.600+0000 7ff1596db640 1 -- 172.21.15.156:0/27499822 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff14c005040 con 0x7ff1540fb620 2024-07-16T10:06:43.607 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:43.600+0000 7ff1596db640 1 --2- 172.21.15.156:0/27499822 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff1540fb620 0x7ff15410be60 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7ff14c0043d0 tx=0x7ff14c009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T10:06:43.607 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:43.600+0000 7ff151ffb640 1 -- 172.21.15.156:0/27499822 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff14c038070 con 0x7ff1540fb620 2024-07-16T10:06:43.607 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:43.600+0000 7ff151ffb640 1 -- 172.21.15.156:0/27499822 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff14c032970 con 0x7ff1540fb620 2024-07-16T10:06:43.607 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:43.600+0000 7ff15b93f640 1 -- 172.21.15.156:0/27499822 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff15410cca0 con 0x7ff1540fb620 2024-07-16T10:06:43.607 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:43.600+0000 7ff151ffb640 1 -- 172.21.15.156:0/27499822 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff14c03da60 con 0x7ff1540fb620 2024-07-16T10:06:43.607 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:43.600+0000 7ff15b93f640 1 -- 172.21.15.156:0/27499822 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1541197a0 con 0x7ff1540fb620 2024-07-16T10:06:43.607 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:43.602+0000 7ff15b93f640 1 -- 172.21.15.156:0/27499822 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1540fc9d0 con 0x7ff1540fb620 2024-07-16T10:06:43.607 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:43.602+0000 7ff151ffb640 1 -- 172.21.15.156:0/27499822 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 38) v1 ==== 92205+0+0 (secure 0 0 0) 0x7ff14c030020 con 0x7ff1540fb620 2024-07-16T10:06:43.607 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:43.602+0000 7ff151ffb640 1 --2- 172.21.15.156:0/27499822 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7ff13c06e0b0 0x7ff13c070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:06:43.607 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:43.602+0000 7ff151ffb640 1 -- 172.21.15.156:0/27499822 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7ff14c0adca0 con 0x7ff1540fb620 2024-07-16T10:06:43.607 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:43.602+0000 7ff158eda640 1 --2- 172.21.15.156:0/27499822 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7ff13c06e0b0 0x7ff13c070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:06:43.608 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:43.603+0000 7ff158eda640 1 --2- 172.21.15.156:0/27499822 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7ff13c06e0b0 0x7ff13c070560 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7ff148007fd0 tx=0x7ff148009e90 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T10:06:43.608 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:43.607+0000 7ff151ffb640 1 -- 172.21.15.156:0/27499822 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187716 (secure 0 0 0) 0x7ff14c0b2070 con 0x7ff1540fb620 2024-07-16T10:06:43.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:06:43 smithi156 ceph-mon[93066]: pgmap v305: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:06:43.843 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:06:43 smithi176 ceph-mon[81689]: pgmap v305: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:06:43.854 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:43.853+0000 7ff15b93f640 1 -- 172.21.15.156:0/27499822 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7ff154072cd0 con 0x7ff1540fb620 2024-07-16T10:06:43.855 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:43.854+0000 7ff151ffb640 1 -- 172.21.15.156:0/27499822 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+618 (secure 0 0 0) 0x7ff14c076f70 con 0x7ff1540fb620 2024-07-16T10:06:43.857 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:43.856+0000 7ff15b93f640 1 -- 172.21.15.156:0/27499822 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7ff13c06e0b0 msgr2=0x7ff13c070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:06:43.858 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:43.856+0000 7ff15b93f640 1 --2- 172.21.15.156:0/27499822 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7ff13c06e0b0 0x7ff13c070560 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7ff148007fd0 tx=0x7ff148009e90 comp rx=0 tx=0).stop 2024-07-16T10:06:43.858 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:43.856+0000 7ff15b93f640 1 -- 172.21.15.156:0/27499822 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff1540fb620 msgr2=0x7ff15410be60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:06:43.858 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:43.856+0000 7ff15b93f640 1 --2- 172.21.15.156:0/27499822 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff1540fb620 0x7ff15410be60 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7ff14c0043d0 tx=0x7ff14c009da0 comp rx=0 tx=0).stop 2024-07-16T10:06:43.858 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:43.856+0000 7ff15b93f640 1 -- 172.21.15.156:0/27499822 shutdown_connections 2024-07-16T10:06:43.858 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:43.856+0000 7ff15b93f640 1 --2- 172.21.15.156:0/27499822 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7ff13c06e0b0 0x7ff13c070560 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:06:43.858 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:43.856+0000 7ff15b93f640 1 --2- 172.21.15.156:0/27499822 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff1540fbf30 0x7ff15410c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:06:43.858 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:43.856+0000 7ff15b93f640 1 --2- 172.21.15.156:0/27499822 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff1540fb620 0x7ff15410be60 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:06:43.858 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:43.856+0000 7ff15b93f640 1 -- 172.21.15.156:0/27499822 >> 172.21.15.156:0/27499822 conn(0x7ff1540721c0 msgr2=0x7ff1540f8680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T10:06:43.858 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:43.856+0000 7ff15b93f640 1 -- 172.21.15.156:0/27499822 shutdown_connections 2024-07-16T10:06:43.858 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:43.856+0000 7ff15b93f640 1 -- 172.21.15.156:0/27499822 wait complete. 2024-07-16T10:06:43.866 INFO:teuthology.orchestra.run.smithi156.stdout:true 2024-07-16T10:06:44.510 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 26c91c26-4358-11ef-bcad-c7b262605968 -e sha1=a6ca461be7cc1aff3658c2c6df4cff975a24449a -- bash -c 'ceph versions | jq -e '"'"'.overall | keys'"'"' | grep $sha1' 2024-07-16T10:06:44.766 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/26c91c26-4358-11ef-bcad-c7b262605968/mon.smithi156/config 2024-07-16T10:06:44.843 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:06:44 smithi176 ceph-mon[81689]: from='client.? 172.21.15.156:0/27499822' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:06:44.888 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:06:44 smithi156 ceph-mon[93066]: from='client.? 172.21.15.156:0/27499822' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:06:45.827 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:06:45 smithi156 ceph-mon[93066]: pgmap v306: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-07-16T10:06:45.843 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:06:45 smithi176 ceph-mon[81689]: pgmap v306: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-07-16T10:06:46.330 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:46.314+0000 7f9359a4e640 1 -- 172.21.15.156:0/1858986926 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9354102370 msgr2=0x7f9354102740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:06:46.330 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:46.314+0000 7f9359a4e640 1 --2- 172.21.15.156:0/1858986926 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9354102370 0x7f9354102740 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f93440048d0 tx=0x7f9344009da0 comp rx=0 tx=0).stop 2024-07-16T10:06:46.330 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:46.315+0000 7f9359a4e640 1 -- 172.21.15.156:0/1858986926 shutdown_connections 2024-07-16T10:06:46.331 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:46.315+0000 7f9359a4e640 1 --2- 172.21.15.156:0/1858986926 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9354102d10 0x7f93540fc3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:06:46.331 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:46.315+0000 7f9359a4e640 1 --2- 172.21.15.156:0/1858986926 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9354102370 0x7f9354102740 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:06:46.331 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:46.315+0000 7f9359a4e640 1 -- 172.21.15.156:0/1858986926 >> 172.21.15.156:0/1858986926 conn(0x7f93540f7f00 msgr2=0x7f93540fa310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T10:06:46.331 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:46.315+0000 7f9359a4e640 1 -- 172.21.15.156:0/1858986926 shutdown_connections 2024-07-16T10:06:46.331 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:46.315+0000 7f9359a4e640 1 -- 172.21.15.156:0/1858986926 wait complete. 2024-07-16T10:06:46.331 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:46.315+0000 7f9359a4e640 1 Processor -- start 2024-07-16T10:06:46.331 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:46.315+0000 7f9359a4e640 1 -- start start 2024-07-16T10:06:46.331 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:46.315+0000 7f9359a4e640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9354102370 0x7f93541133a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:06:46.331 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:46.315+0000 7f9359a4e640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9354102d10 0x7f93541138e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:06:46.331 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:46.315+0000 7f9359a4e640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9354113ee0 con 0x7f9354102d10 2024-07-16T10:06:46.331 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:46.315+0000 7f9359a4e640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9354114050 con 0x7f9354102370 2024-07-16T10:06:46.331 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:46.315+0000 7f934bfff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9354102d10 0x7f93541138e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:06:46.332 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:46.316+0000 7f934bfff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9354102d10 0x7f93541138e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:39920/0 (socket says 172.21.15.156:39920) 2024-07-16T10:06:46.332 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:46.316+0000 7f934bfff640 1 -- 172.21.15.156:0/1857414115 learned_addr learned my addr 172.21.15.156:0/1857414115 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T10:06:46.332 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:46.316+0000 7f9352ffd640 1 --2- 172.21.15.156:0/1857414115 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9354102370 0x7f93541133a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:06:46.332 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:46.316+0000 7f934bfff640 1 -- 172.21.15.156:0/1857414115 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9354102370 msgr2=0x7f93541133a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:06:46.332 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:46.316+0000 7f934bfff640 1 --2- 172.21.15.156:0/1857414115 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9354102370 0x7f93541133a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:06:46.332 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:46.316+0000 7f934bfff640 1 -- 172.21.15.156:0/1857414115 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9344005040 con 0x7f9354102d10 2024-07-16T10:06:46.332 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:46.316+0000 7f934bfff640 1 --2- 172.21.15.156:0/1857414115 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9354102d10 0x7f93541138e0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f933c00c790 tx=0x7f933c0043c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T10:06:46.332 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:46.316+0000 7f934b7fe640 1 -- 172.21.15.156:0/1857414115 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f933c00a820 con 0x7f9354102d10 2024-07-16T10:06:46.332 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:46.316+0000 7f9359a4e640 1 -- 172.21.15.156:0/1857414115 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f935410d4d0 con 0x7f9354102d10 2024-07-16T10:06:46.332 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:46.316+0000 7f934b7fe640 1 -- 172.21.15.156:0/1857414115 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f933c00ae50 con 0x7f9354102d10 2024-07-16T10:06:46.332 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:46.316+0000 7f934b7fe640 1 -- 172.21.15.156:0/1857414115 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f933c0115e0 con 0x7f9354102d10 2024-07-16T10:06:46.332 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:46.316+0000 7f9359a4e640 1 -- 172.21.15.156:0/1857414115 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f935410da70 con 0x7f9354102d10 2024-07-16T10:06:46.332 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:46.317+0000 7f9359a4e640 1 -- 172.21.15.156:0/1857414115 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93540fdae0 con 0x7f9354102d10 2024-07-16T10:06:46.333 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:46.318+0000 7f934b7fe640 1 -- 172.21.15.156:0/1857414115 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 38) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f933c012400 con 0x7f9354102d10 2024-07-16T10:06:46.333 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:46.318+0000 7f934b7fe640 1 --2- 172.21.15.156:0/1857414115 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f934006e0b0 0x7f9340070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:06:46.333 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:46.318+0000 7f934b7fe640 1 -- 172.21.15.156:0/1857414115 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f933c08dd90 con 0x7f9354102d10 2024-07-16T10:06:46.333 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:46.318+0000 7f9352ffd640 1 --2- 172.21.15.156:0/1857414115 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f934006e0b0 0x7f9340070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:06:46.333 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:46.319+0000 7f9352ffd640 1 --2- 172.21.15.156:0/1857414115 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f934006e0b0 0x7f9340070560 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f9344000c00 tx=0x7f9344035040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T10:06:46.333 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:46.322+0000 7f934b7fe640 1 -- 172.21.15.156:0/1857414115 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187716 (secure 0 0 0) 0x7f933c0570e0 con 0x7f9354102d10 2024-07-16T10:06:46.572 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:46.571+0000 7f9359a4e640 1 -- 172.21.15.156:0/1857414115 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f93540fbb60 con 0x7f9354102d10 2024-07-16T10:06:46.572 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:46.571+0000 7f934b7fe640 1 -- 172.21.15.156:0/1857414115 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+618 (secure 0 0 0) 0x7f933c05a7a0 con 0x7f9354102d10 2024-07-16T10:06:46.574 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:46.573+0000 7f9359a4e640 1 -- 172.21.15.156:0/1857414115 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f934006e0b0 msgr2=0x7f9340070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:06:46.575 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:46.573+0000 7f9359a4e640 1 --2- 172.21.15.156:0/1857414115 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f934006e0b0 0x7f9340070560 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f9344000c00 tx=0x7f9344035040 comp rx=0 tx=0).stop 2024-07-16T10:06:46.575 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:46.573+0000 7f9359a4e640 1 -- 172.21.15.156:0/1857414115 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9354102d10 msgr2=0x7f93541138e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:06:46.575 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:46.573+0000 7f9359a4e640 1 --2- 172.21.15.156:0/1857414115 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9354102d10 0x7f93541138e0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f933c00c790 tx=0x7f933c0043c0 comp rx=0 tx=0).stop 2024-07-16T10:06:46.575 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:46.573+0000 7f9359a4e640 1 -- 172.21.15.156:0/1857414115 shutdown_connections 2024-07-16T10:06:46.575 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:46.573+0000 7f9359a4e640 1 --2- 172.21.15.156:0/1857414115 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f934006e0b0 0x7f9340070560 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:06:46.575 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:46.573+0000 7f9359a4e640 1 --2- 172.21.15.156:0/1857414115 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9354102d10 0x7f93541138e0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:06:46.575 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:46.573+0000 7f9359a4e640 1 --2- 172.21.15.156:0/1857414115 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9354102370 0x7f93541133a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:06:46.575 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:46.573+0000 7f9359a4e640 1 -- 172.21.15.156:0/1857414115 >> 172.21.15.156:0/1857414115 conn(0x7f93540f7f00 msgr2=0x7f93540f8e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T10:06:46.575 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:46.573+0000 7f9359a4e640 1 -- 172.21.15.156:0/1857414115 shutdown_connections 2024-07-16T10:06:46.575 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:46.574+0000 7f9359a4e640 1 -- 172.21.15.156:0/1857414115 wait complete. 2024-07-16T10:06:46.583 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph version 19.0.0-3495-ga6ca461b (a6ca461be7cc1aff3658c2c6df4cff975a24449a) squid (rc)" 2024-07-16T10:06:47.245 DEBUG:teuthology.parallel:result is None 2024-07-16T10:06:47.246 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-07-16T10:06:47.257 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi156.front.sepia.ceph.com 2024-07-16T10:06:47.258 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 26c91c26-4358-11ef-bcad-c7b262605968 -- bash -c 'ceph fs dump' 2024-07-16T10:06:47.517 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/26c91c26-4358-11ef-bcad-c7b262605968/mon.smithi156/config 2024-07-16T10:06:47.734 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:06:47 smithi156 ceph-mon[93066]: pgmap v307: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:06:47.734 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:06:47 smithi156 ceph-mon[93066]: from='client.? 172.21.15.156:0/1857414115' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:06:47.843 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:06:47 smithi176 ceph-mon[81689]: pgmap v307: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:06:47.843 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:06:47 smithi176 ceph-mon[81689]: from='client.? 172.21.15.156:0/1857414115' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-07-16T10:06:48.757 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:06:48 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-16T10:06:48.843 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:06:48 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-16T10:06:49.123 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:49.098+0000 7f12295e1640 1 -- 172.21.15.156:0/3792860564 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1224101c20 msgr2=0x7f12240fb2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:06:49.123 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:49.098+0000 7f12295e1640 1 --2- 172.21.15.156:0/3792860564 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1224101c20 0x7f12240fb2b0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f12180048d0 tx=0x7f1218009da0 comp rx=0 tx=0).stop 2024-07-16T10:06:49.123 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:49.098+0000 7f12295e1640 1 -- 172.21.15.156:0/3792860564 shutdown_connections 2024-07-16T10:06:49.123 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:49.098+0000 7f12295e1640 1 --2- 172.21.15.156:0/3792860564 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1224101c20 0x7f12240fb2b0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:06:49.123 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:49.098+0000 7f12295e1640 1 --2- 172.21.15.156:0/3792860564 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1224101280 0x7f1224101650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:06:49.123 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:49.098+0000 7f12295e1640 1 -- 172.21.15.156:0/3792860564 >> 172.21.15.156:0/3792860564 conn(0x7f12240f6e70 msgr2=0x7f12240f9280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T10:06:49.124 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:49.098+0000 7f12295e1640 1 -- 172.21.15.156:0/3792860564 shutdown_connections 2024-07-16T10:06:49.124 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:49.098+0000 7f12295e1640 1 -- 172.21.15.156:0/3792860564 wait complete. 2024-07-16T10:06:49.124 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:49.099+0000 7f12295e1640 1 Processor -- start 2024-07-16T10:06:49.124 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:49.099+0000 7f12295e1640 1 -- start start 2024-07-16T10:06:49.124 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:49.099+0000 7f12295e1640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1224101280 0x7f122410e080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:06:49.124 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:49.099+0000 7f12295e1640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1224101c20 0x7f122410e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:06:49.124 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:49.099+0000 7f12295e1640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f122410ec10 con 0x7f1224101c20 2024-07-16T10:06:49.124 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:49.099+0000 7f12295e1640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f122410ed80 con 0x7f1224101280 2024-07-16T10:06:49.124 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:49.099+0000 7f12227fc640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1224101c20 0x7f122410e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:06:49.125 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:49.099+0000 7f12227fc640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1224101c20 0x7f122410e5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:39932/0 (socket says 172.21.15.156:39932) 2024-07-16T10:06:49.125 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:49.099+0000 7f12227fc640 1 -- 172.21.15.156:0/2652656593 learned_addr learned my addr 172.21.15.156:0/2652656593 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T10:06:49.125 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:49.099+0000 7f1222ffd640 1 --2- 172.21.15.156:0/2652656593 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1224101280 0x7f122410e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:06:49.125 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:49.099+0000 7f12227fc640 1 -- 172.21.15.156:0/2652656593 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1224101280 msgr2=0x7f122410e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:06:49.125 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:49.099+0000 7f12227fc640 1 --2- 172.21.15.156:0/2652656593 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1224101280 0x7f122410e080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:06:49.125 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:49.099+0000 7f12227fc640 1 -- 172.21.15.156:0/2652656593 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1218005040 con 0x7f1224101c20 2024-07-16T10:06:49.125 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:49.100+0000 7f12227fc640 1 --2- 172.21.15.156:0/2652656593 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1224101c20 0x7f122410e5c0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f1218003080 tx=0x7f1218009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T10:06:49.126 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:49.100+0000 7f120b7fe640 1 -- 172.21.15.156:0/2652656593 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1218038070 con 0x7f1224101c20 2024-07-16T10:06:49.126 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:49.100+0000 7f12295e1640 1 -- 172.21.15.156:0/2652656593 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12241170f0 con 0x7f1224101c20 2024-07-16T10:06:49.126 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:49.100+0000 7f120b7fe640 1 -- 172.21.15.156:0/2652656593 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1218008ee0 con 0x7f1224101c20 2024-07-16T10:06:49.126 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:49.100+0000 7f120b7fe640 1 -- 172.21.15.156:0/2652656593 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f121803d870 con 0x7f1224101c20 2024-07-16T10:06:49.126 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:49.100+0000 7f12295e1640 1 -- 172.21.15.156:0/2652656593 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1224117550 con 0x7f1224101c20 2024-07-16T10:06:49.126 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:49.101+0000 7f12295e1640 1 -- 172.21.15.156:0/2652656593 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12240fc9f0 con 0x7f1224101c20 2024-07-16T10:06:49.126 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:49.102+0000 7f120b7fe640 1 -- 172.21.15.156:0/2652656593 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 38) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f1218030020 con 0x7f1224101c20 2024-07-16T10:06:49.127 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:49.102+0000 7f120b7fe640 1 --2- 172.21.15.156:0/2652656593 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f120c06e0b0 0x7f120c070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:06:49.127 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:49.102+0000 7f120b7fe640 1 -- 172.21.15.156:0/2652656593 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f12180ad8a0 con 0x7f1224101c20 2024-07-16T10:06:49.127 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:49.102+0000 7f1222ffd640 1 --2- 172.21.15.156:0/2652656593 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f120c06e0b0 0x7f120c070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:06:49.127 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:49.103+0000 7f1222ffd640 1 --2- 172.21.15.156:0/2652656593 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f120c06e0b0 0x7f120c070560 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f1214007fd0 tx=0x7f1214009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T10:06:49.127 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:49.106+0000 7f120b7fe640 1 -- 172.21.15.156:0/2652656593 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187716 (secure 0 0 0) 0x7f1218076bf0 con 0x7f1224101c20 2024-07-16T10:06:49.319 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:49.317+0000 7f12295e1640 1 -- 172.21.15.156:0/2652656593 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f1224108450 con 0x7f1224101c20 2024-07-16T10:06:49.319 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:49.318+0000 7f120b7fe640 1 -- 172.21.15.156:0/2652656593 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 45 v45) v1 ==== 76+0+1972 (secure 0 0 0) 0x7f121807a2b0 con 0x7f1224101c20 2024-07-16T10:06:49.319 INFO:teuthology.orchestra.run.smithi156.stdout:e45 2024-07-16T10:06:49.319 INFO:teuthology.orchestra.run.smithi156.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-07-16T10:06:49.320 INFO:teuthology.orchestra.run.smithi156.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-07-16T10:06:49.320 INFO:teuthology.orchestra.run.smithi156.stdout:legacy client fscid: 1 2024-07-16T10:06:49.320 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-07-16T10:06:49.320 INFO:teuthology.orchestra.run.smithi156.stdout:Filesystem 'cephfs' (1) 2024-07-16T10:06:49.320 INFO:teuthology.orchestra.run.smithi156.stdout:fs_name cephfs 2024-07-16T10:06:49.320 INFO:teuthology.orchestra.run.smithi156.stdout:epoch 45 2024-07-16T10:06:49.320 INFO:teuthology.orchestra.run.smithi156.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-07-16T10:06:49.320 INFO:teuthology.orchestra.run.smithi156.stdout:created 2024-07-16T09:52:55.214975+0000 2024-07-16T10:06:49.320 INFO:teuthology.orchestra.run.smithi156.stdout:modified 2024-07-16T10:03:55.772577+0000 2024-07-16T10:06:49.320 INFO:teuthology.orchestra.run.smithi156.stdout:tableserver 0 2024-07-16T10:06:49.320 INFO:teuthology.orchestra.run.smithi156.stdout:root 0 2024-07-16T10:06:49.321 INFO:teuthology.orchestra.run.smithi156.stdout:session_timeout 60 2024-07-16T10:06:49.321 INFO:teuthology.orchestra.run.smithi156.stdout:session_autoclose 300 2024-07-16T10:06:49.321 INFO:teuthology.orchestra.run.smithi156.stdout:max_file_size 1099511627776 2024-07-16T10:06:49.321 INFO:teuthology.orchestra.run.smithi156.stdout:max_xattr_size 65536 2024-07-16T10:06:49.321 INFO:teuthology.orchestra.run.smithi156.stdout:required_client_features {} 2024-07-16T10:06:49.321 INFO:teuthology.orchestra.run.smithi156.stdout:last_failure 0 2024-07-16T10:06:49.321 INFO:teuthology.orchestra.run.smithi156.stdout:last_failure_osd_epoch 82 2024-07-16T10:06:49.321 INFO:teuthology.orchestra.run.smithi156.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-07-16T10:06:49.321 INFO:teuthology.orchestra.run.smithi156.stdout:max_mds 2 2024-07-16T10:06:49.322 INFO:teuthology.orchestra.run.smithi156.stdout:in 0,1 2024-07-16T10:06:49.322 INFO:teuthology.orchestra.run.smithi156.stdout:up {0=34438,1=34470} 2024-07-16T10:06:49.322 INFO:teuthology.orchestra.run.smithi156.stdout:failed 2024-07-16T10:06:49.322 INFO:teuthology.orchestra.run.smithi156.stdout:damaged 2024-07-16T10:06:49.322 INFO:teuthology.orchestra.run.smithi156.stdout:stopped 2024-07-16T10:06:49.322 INFO:teuthology.orchestra.run.smithi156.stdout:data_pools [3] 2024-07-16T10:06:49.322 INFO:teuthology.orchestra.run.smithi156.stdout:metadata_pool 2 2024-07-16T10:06:49.322 INFO:teuthology.orchestra.run.smithi156.stdout:inline_data disabled 2024-07-16T10:06:49.322 INFO:teuthology.orchestra.run.smithi156.stdout:balancer 2024-07-16T10:06:49.322 INFO:teuthology.orchestra.run.smithi156.stdout:bal_rank_mask -1 2024-07-16T10:06:49.323 INFO:teuthology.orchestra.run.smithi156.stdout:standby_count_wanted 1 2024-07-16T10:06:49.323 INFO:teuthology.orchestra.run.smithi156.stdout:qdb_cluster leader: 34438 members: 34470,34438 2024-07-16T10:06:49.323 INFO:teuthology.orchestra.run.smithi156.stdout:[mds.cephfs.smithi156.rdibys{0:34438} state up:active seq 9 join_fscid=1 addr [v2:172.21.15.156:6826/2852996640,v1:172.21.15.156:6827/2852996640] compat {c=[1],r=[1],i=[1fff]}] 2024-07-16T10:06:49.323 INFO:teuthology.orchestra.run.smithi156.stdout:[mds.cephfs.smithi176.dvvbud{0:44347} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.176:6824/1700587296,v1:172.21.15.176:6825/1700587296] compat {c=[1],r=[1],i=[1fff]}] 2024-07-16T10:06:49.323 INFO:teuthology.orchestra.run.smithi156.stdout:[mds.cephfs.smithi156.evwwaz{1:34470} state up:active seq 11 join_fscid=1 addr [v2:172.21.15.156:6828/1683974145,v1:172.21.15.156:6829/1683974145] compat {c=[1],r=[1],i=[1fff]}] 2024-07-16T10:06:49.323 INFO:teuthology.orchestra.run.smithi156.stdout:[mds.cephfs.smithi176.wahbsi{1:44373} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.176:6826/2277003188,v1:172.21.15.176:6827/2277003188] compat {c=[1],r=[1],i=[1fff]}] 2024-07-16T10:06:49.323 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-07-16T10:06:49.323 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-07-16T10:06:49.324 INFO:teuthology.orchestra.run.smithi156.stderr:dumped fsmap epoch 45 2024-07-16T10:06:49.324 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:49.320+0000 7f12295e1640 1 -- 172.21.15.156:0/2652656593 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f120c06e0b0 msgr2=0x7f120c070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:06:49.324 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:49.320+0000 7f12295e1640 1 --2- 172.21.15.156:0/2652656593 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f120c06e0b0 0x7f120c070560 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f1214007fd0 tx=0x7f1214009040 comp rx=0 tx=0).stop 2024-07-16T10:06:49.325 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:49.320+0000 7f12295e1640 1 -- 172.21.15.156:0/2652656593 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1224101c20 msgr2=0x7f122410e5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:06:49.325 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:49.320+0000 7f12295e1640 1 --2- 172.21.15.156:0/2652656593 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1224101c20 0x7f122410e5c0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f1218003080 tx=0x7f1218009da0 comp rx=0 tx=0).stop 2024-07-16T10:06:49.325 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:49.320+0000 7f12295e1640 1 -- 172.21.15.156:0/2652656593 shutdown_connections 2024-07-16T10:06:49.325 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:49.320+0000 7f12295e1640 1 --2- 172.21.15.156:0/2652656593 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f120c06e0b0 0x7f120c070560 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:06:49.325 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:49.320+0000 7f12295e1640 1 --2- 172.21.15.156:0/2652656593 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1224101c20 0x7f122410e5c0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:06:49.325 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:49.320+0000 7f12295e1640 1 --2- 172.21.15.156:0/2652656593 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1224101280 0x7f122410e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:06:49.325 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:49.320+0000 7f12295e1640 1 -- 172.21.15.156:0/2652656593 >> 172.21.15.156:0/2652656593 conn(0x7f12240f6e70 msgr2=0x7f12240f7d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T10:06:49.325 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:49.320+0000 7f12295e1640 1 -- 172.21.15.156:0/2652656593 shutdown_connections 2024-07-16T10:06:49.325 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:49.320+0000 7f12295e1640 1 -- 172.21.15.156:0/2652656593 wait complete. 2024-07-16T10:06:49.647 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:06:49 smithi156 ceph-mon[93066]: pgmap v308: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-07-16T10:06:49.647 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:06:49 smithi156 ceph-mon[93066]: from='client.? 172.21.15.156:0/2652656593' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-07-16T10:06:49.843 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:06:49 smithi176 ceph-mon[81689]: pgmap v308: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-07-16T10:06:49.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:06:49 smithi176 ceph-mon[81689]: from='client.? 172.21.15.156:0/2652656593' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-07-16T10:06:49.958 INFO:teuthology.run_tasks:Running task fs.post_upgrade_checks... 2024-07-16T10:06:49.972 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 26c91c26-4358-11ef-bcad-c7b262605968 -- ceph fs dump --format=json 2024-07-16T10:06:50.230 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/26c91c26-4358-11ef-bcad-c7b262605968/mon.smithi156/config 2024-07-16T10:06:51.605 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:06:51 smithi156 ceph-mon[93066]: pgmap v309: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:06:51.797 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:51.796+0000 7f7741f5f640 1 -- 172.21.15.156:0/4253100628 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f773c0714f0 msgr2=0x7f773c06f8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:06:51.798 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:51.796+0000 7f7741f5f640 1 --2- 172.21.15.156:0/4253100628 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f773c0714f0 0x7f773c06f8c0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f77240048d0 tx=0x7f7724009da0 comp rx=0 tx=0).stop 2024-07-16T10:06:51.798 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:51.796+0000 7f7741f5f640 1 -- 172.21.15.156:0/4253100628 shutdown_connections 2024-07-16T10:06:51.798 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:51.796+0000 7f7741f5f640 1 --2- 172.21.15.156:0/4253100628 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f773c06fe00 0x7f773c070270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:06:51.798 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:51.796+0000 7f7741f5f640 1 --2- 172.21.15.156:0/4253100628 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f773c0714f0 0x7f773c06f8c0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:06:51.798 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:51.796+0000 7f7741f5f640 1 -- 172.21.15.156:0/4253100628 >> 172.21.15.156:0/4253100628 conn(0x7f773c0f7fb0 msgr2=0x7f773c0fa3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T10:06:51.798 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:51.796+0000 7f7741f5f640 1 -- 172.21.15.156:0/4253100628 shutdown_connections 2024-07-16T10:06:51.798 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:51.796+0000 7f7741f5f640 1 -- 172.21.15.156:0/4253100628 wait complete. 2024-07-16T10:06:51.798 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:51.796+0000 7f7741f5f640 1 Processor -- start 2024-07-16T10:06:51.800 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:51.797+0000 7f7741f5f640 1 -- start start 2024-07-16T10:06:51.800 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:51.797+0000 7f7741f5f640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f773c06fe00 0x7f773c119720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:06:51.800 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:51.797+0000 7f7741f5f640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f773c0714f0 0x7f773c119c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:06:51.800 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:51.797+0000 7f7741f5f640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f773c11a2b0 con 0x7f773c06fe00 2024-07-16T10:06:51.800 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:51.797+0000 7f7741f5f640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f773c1136c0 con 0x7f773c0714f0 2024-07-16T10:06:51.800 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:51.797+0000 7f773b7fe640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f773c06fe00 0x7f773c119720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:06:51.800 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:51.797+0000 7f773b7fe640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f773c06fe00 0x7f773c119720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:39948/0 (socket says 172.21.15.156:39948) 2024-07-16T10:06:51.800 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:51.797+0000 7f773b7fe640 1 -- 172.21.15.156:0/3483895279 learned_addr learned my addr 172.21.15.156:0/3483895279 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T10:06:51.800 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:51.797+0000 7f773affd640 1 --2- 172.21.15.156:0/3483895279 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f773c0714f0 0x7f773c119c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:06:51.800 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:51.797+0000 7f773b7fe640 1 -- 172.21.15.156:0/3483895279 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f773c0714f0 msgr2=0x7f773c119c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:06:51.800 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:51.797+0000 7f773b7fe640 1 --2- 172.21.15.156:0/3483895279 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f773c0714f0 0x7f773c119c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:06:51.801 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:51.797+0000 7f773b7fe640 1 -- 172.21.15.156:0/3483895279 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7724005040 con 0x7f773c06fe00 2024-07-16T10:06:51.801 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:51.797+0000 7f773b7fe640 1 --2- 172.21.15.156:0/3483895279 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f773c06fe00 0x7f773c119720 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f77240043d0 tx=0x7f7724009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T10:06:51.801 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:51.798+0000 7f7733fff640 1 -- 172.21.15.156:0/3483895279 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7724038070 con 0x7f773c06fe00 2024-07-16T10:06:51.801 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:51.798+0000 7f7733fff640 1 -- 172.21.15.156:0/3483895279 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7724032d30 con 0x7f773c06fe00 2024-07-16T10:06:51.801 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:51.798+0000 7f7741f5f640 1 -- 172.21.15.156:0/3483895279 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f773c113940 con 0x7f773c06fe00 2024-07-16T10:06:51.801 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:51.798+0000 7f7733fff640 1 -- 172.21.15.156:0/3483895279 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f772403dcd0 con 0x7f773c06fe00 2024-07-16T10:06:51.801 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:51.798+0000 7f7741f5f640 1 -- 172.21.15.156:0/3483895279 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f773c113e00 con 0x7f773c06fe00 2024-07-16T10:06:51.802 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:51.799+0000 7f7741f5f640 1 -- 172.21.15.156:0/3483895279 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f773c071000 con 0x7f773c06fe00 2024-07-16T10:06:51.802 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:51.799+0000 7f7733fff640 1 -- 172.21.15.156:0/3483895279 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 38) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f7724030020 con 0x7f773c06fe00 2024-07-16T10:06:51.802 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:51.800+0000 7f7733fff640 1 --2- 172.21.15.156:0/3483895279 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f772806e0b0 0x7f7728070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:06:51.802 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:51.800+0000 7f773affd640 1 --2- 172.21.15.156:0/3483895279 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f772806e0b0 0x7f7728070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:06:51.802 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:51.800+0000 7f7733fff640 1 -- 172.21.15.156:0/3483895279 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f77240ae850 con 0x7f773c06fe00 2024-07-16T10:06:51.802 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:51.800+0000 7f773affd640 1 --2- 172.21.15.156:0/3483895279 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f772806e0b0 0x7f7728070560 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f773c1149a0 tx=0x7f772c009dc0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T10:06:51.806 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:51.804+0000 7f7733fff640 1 -- 172.21.15.156:0/3483895279 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187716 (secure 0 0 0) 0x7f7724077b20 con 0x7f773c06fe00 2024-07-16T10:06:51.843 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:06:51 smithi176 ceph-mon[81689]: pgmap v309: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:06:52.021 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:52.020+0000 7f7741f5f640 1 -- 172.21.15.156:0/3483895279 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f773c10faa0 con 0x7f773c06fe00 2024-07-16T10:06:52.021 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:52.020+0000 7f7733fff640 1 -- 172.21.15.156:0/3483895279 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 45 v45) v1 ==== 94+0+5257 (secure 0 0 0) 0x7f772407b1e0 con 0x7f773c06fe00 2024-07-16T10:06:52.022 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-07-16T10:06:52.022 INFO:teuthology.orchestra.run.smithi156.stdout:{"epoch":45,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":45,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-16T09:52:55.214975+0000","modified":"2024-07-16T10:03:55.772577+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":82,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":34438,"mds_1":34470},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34438":{"gid":34438,"name":"cephfs.smithi156.rdibys","rank":0,"incarnation":33,"state":"up:active","state_seq":9,"addr":"172.21.15.156:6827/2852996640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":2852996640},{"type":"v1","addr":"172.21.15.156:6827","nonce":2852996640}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34470":{"gid":34470,"name":"cephfs.smithi156.evwwaz","rank":1,"incarnation":43,"state":"up:active","state_seq":11,"addr":"172.21.15.156:6829/1683974145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":1683974145},{"type":"v1","addr":"172.21.15.156:6829","nonce":1683974145}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_44347":{"gid":44347,"name":"cephfs.smithi176.dvvbud","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.176:6825/1700587296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6824","nonce":1700587296},{"type":"v1","addr":"172.21.15.176:6825","nonce":1700587296}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_44373":{"gid":44373,"name":"cephfs.smithi176.wahbsi","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.176:6827/2277003188","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6826","nonce":2277003188},{"type":"v1","addr":"172.21.15.176:6827","nonce":2277003188}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":34438,"qdb_cluster":[34470,34438]},"id":1}]} 2024-07-16T10:06:52.022 INFO:teuthology.orchestra.run.smithi156.stderr:dumped fsmap epoch 45 2024-07-16T10:06:52.024 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:52.022+0000 7f7741f5f640 1 -- 172.21.15.156:0/3483895279 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f772806e0b0 msgr2=0x7f7728070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:06:52.024 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:52.022+0000 7f7741f5f640 1 --2- 172.21.15.156:0/3483895279 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f772806e0b0 0x7f7728070560 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f773c1149a0 tx=0x7f772c009dc0 comp rx=0 tx=0).stop 2024-07-16T10:06:52.024 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:52.022+0000 7f7741f5f640 1 -- 172.21.15.156:0/3483895279 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f773c06fe00 msgr2=0x7f773c119720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:06:52.024 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:52.022+0000 7f7741f5f640 1 --2- 172.21.15.156:0/3483895279 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f773c06fe00 0x7f773c119720 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f77240043d0 tx=0x7f7724009da0 comp rx=0 tx=0).stop 2024-07-16T10:06:52.024 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:52.023+0000 7f7741f5f640 1 -- 172.21.15.156:0/3483895279 shutdown_connections 2024-07-16T10:06:52.024 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:52.023+0000 7f7741f5f640 1 --2- 172.21.15.156:0/3483895279 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f772806e0b0 0x7f7728070560 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:06:52.025 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:52.023+0000 7f7741f5f640 1 --2- 172.21.15.156:0/3483895279 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f773c0714f0 0x7f773c119c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:06:52.025 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:52.023+0000 7f7741f5f640 1 --2- 172.21.15.156:0/3483895279 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f773c06fe00 0x7f773c119720 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:06:52.025 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:52.023+0000 7f7741f5f640 1 -- 172.21.15.156:0/3483895279 >> 172.21.15.156:0/3483895279 conn(0x7f773c0f7fb0 msgr2=0x7f773c0f9130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T10:06:52.025 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:52.023+0000 7f7741f5f640 1 -- 172.21.15.156:0/3483895279 shutdown_connections 2024-07-16T10:06:52.025 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:52.023+0000 7f7741f5f640 1 -- 172.21.15.156:0/3483895279 wait complete. 2024-07-16T10:06:52.615 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:06:52 smithi156 ceph-mon[93066]: from='client.? 172.21.15.156:0/3483895279' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-16T10:06:52.639 DEBUG:tasks.fs:checking fs fscid=1,name=cephfs state = {'epoch': 16, 'max_mds': 2, 'flags': 50} 2024-07-16T10:06:52.639 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 26c91c26-4358-11ef-bcad-c7b262605968 -- ceph fs dump --format=json 17 2024-07-16T10:06:52.843 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:06:52 smithi176 ceph-mon[81689]: from='client.? 172.21.15.156:0/3483895279' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-16T10:06:52.899 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/26c91c26-4358-11ef-bcad-c7b262605968/mon.smithi156/config 2024-07-16T10:06:53.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:06:53 smithi156 ceph-mon[93066]: pgmap v310: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:06:53.843 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:06:53 smithi176 ceph-mon[81689]: pgmap v310: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:06:54.431 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:54.429+0000 7fc0fca43640 1 -- 172.21.15.156:0/1928677901 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc0f8102cf0 msgr2=0x7fc0f80fc380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:06:54.432 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:54.429+0000 7fc0fca43640 1 --2- 172.21.15.156:0/1928677901 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc0f8102cf0 0x7fc0f80fc380 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fc0e00048d0 tx=0x7fc0e0009da0 comp rx=0 tx=0).stop 2024-07-16T10:06:54.432 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:54.429+0000 7fc0fca43640 1 -- 172.21.15.156:0/1928677901 shutdown_connections 2024-07-16T10:06:54.432 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:54.429+0000 7fc0fca43640 1 --2- 172.21.15.156:0/1928677901 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc0f8102cf0 0x7fc0f80fc380 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:06:54.432 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:54.429+0000 7fc0fca43640 1 --2- 172.21.15.156:0/1928677901 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc0f8102350 0x7fc0f8102720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:06:54.432 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:54.429+0000 7fc0fca43640 1 -- 172.21.15.156:0/1928677901 >> 172.21.15.156:0/1928677901 conn(0x7fc0f80f7f00 msgr2=0x7fc0f80fa310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T10:06:54.432 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:54.429+0000 7fc0fca43640 1 -- 172.21.15.156:0/1928677901 shutdown_connections 2024-07-16T10:06:54.432 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:54.429+0000 7fc0fca43640 1 -- 172.21.15.156:0/1928677901 wait complete. 2024-07-16T10:06:54.432 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:54.430+0000 7fc0fca43640 1 Processor -- start 2024-07-16T10:06:54.432 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:54.430+0000 7fc0fca43640 1 -- start start 2024-07-16T10:06:54.432 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:54.430+0000 7fc0fca43640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc0f8102350 0x7fc0f8115290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:06:54.432 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:54.430+0000 7fc0fca43640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc0f8102cf0 0x7fc0f81157d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:06:54.432 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:54.430+0000 7fc0fca43640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0f8115dd0 con 0x7fc0f8102350 2024-07-16T10:06:54.433 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:54.430+0000 7fc0fca43640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0f810f210 con 0x7fc0f8102cf0 2024-07-16T10:06:54.433 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:54.430+0000 7fc0f659c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc0f8102350 0x7fc0f8115290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:06:54.433 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:54.430+0000 7fc0f659c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc0f8102350 0x7fc0f8115290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:41164/0 (socket says 172.21.15.156:41164) 2024-07-16T10:06:54.433 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:54.430+0000 7fc0f659c640 1 -- 172.21.15.156:0/1816768667 learned_addr learned my addr 172.21.15.156:0/1816768667 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T10:06:54.433 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:54.430+0000 7fc0f5d9b640 1 --2- 172.21.15.156:0/1816768667 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc0f8102cf0 0x7fc0f81157d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:06:54.433 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:54.430+0000 7fc0f659c640 1 -- 172.21.15.156:0/1816768667 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc0f8102cf0 msgr2=0x7fc0f81157d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:06:54.433 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:54.430+0000 7fc0f659c640 1 --2- 172.21.15.156:0/1816768667 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc0f8102cf0 0x7fc0f81157d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:06:54.433 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:54.430+0000 7fc0f659c640 1 -- 172.21.15.156:0/1816768667 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0e0005040 con 0x7fc0f8102350 2024-07-16T10:06:54.433 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:54.431+0000 7fc0f659c640 1 --2- 172.21.15.156:0/1816768667 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc0f8102350 0x7fc0f8115290 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fc0e800db80 tx=0x7fc0e8004f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T10:06:54.434 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:54.431+0000 7fc0eeffd640 1 -- 172.21.15.156:0/1816768667 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc0e800b8d0 con 0x7fc0f8102350 2024-07-16T10:06:54.434 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:54.431+0000 7fc0eeffd640 1 -- 172.21.15.156:0/1816768667 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc0e800f040 con 0x7fc0f8102350 2024-07-16T10:06:54.435 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:54.431+0000 7fc0fca43640 1 -- 172.21.15.156:0/1816768667 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0f810f4f0 con 0x7fc0f8102350 2024-07-16T10:06:54.435 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:54.431+0000 7fc0eeffd640 1 -- 172.21.15.156:0/1816768667 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc0e8012390 con 0x7fc0f8102350 2024-07-16T10:06:54.435 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:54.431+0000 7fc0fca43640 1 -- 172.21.15.156:0/1816768667 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0f810f9e0 con 0x7fc0f8102350 2024-07-16T10:06:54.435 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:54.432+0000 7fc0fca43640 1 -- 172.21.15.156:0/1816768667 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0f80fdac0 con 0x7fc0f8102350 2024-07-16T10:06:54.435 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:54.432+0000 7fc0eeffd640 1 -- 172.21.15.156:0/1816768667 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 38) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fc0e800ba30 con 0x7fc0f8102350 2024-07-16T10:06:54.435 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:54.433+0000 7fc0eeffd640 1 --2- 172.21.15.156:0/1816768667 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7fc0e406e0b0 0x7fc0e4070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:06:54.435 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:54.433+0000 7fc0eeffd640 1 -- 172.21.15.156:0/1816768667 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7fc0e808dad0 con 0x7fc0f8102350 2024-07-16T10:06:54.435 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:54.433+0000 7fc0f5d9b640 1 --2- 172.21.15.156:0/1816768667 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7fc0e406e0b0 0x7fc0e4070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:06:54.435 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:54.433+0000 7fc0f5d9b640 1 --2- 172.21.15.156:0/1816768667 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7fc0e406e0b0 0x7fc0e4070560 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fc0e0032430 tx=0x7fc0e0035040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T10:06:54.438 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:54.437+0000 7fc0eeffd640 1 -- 172.21.15.156:0/1816768667 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187716 (secure 0 0 0) 0x7fc0e8056da0 con 0x7fc0f8102350 2024-07-16T10:06:54.654 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:54.653+0000 7fc0fca43640 1 -- 172.21.15.156:0/1816768667 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 17, "format": "json"} v 0) v1 -- 0x7fc0f804b530 con 0x7fc0f8102350 2024-07-16T10:06:54.655 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:54.654+0000 7fc0eeffd640 1 -- 172.21.15.156:0/1816768667 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 17, "format": "json"}]=0 dumped fsmap epoch 17 v45) v1 ==== 107+0+4896 (secure 0 0 0) 0x7fc0e805a460 con 0x7fc0f8102350 2024-07-16T10:06:54.655 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-07-16T10:06:54.655 INFO:teuthology.orchestra.run.smithi156.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":17,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-16T09:52:55.214975+0000","modified":"2024-07-16T09:54:55.108622+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":14458,"mds_1":24261},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14458":{"gid":14458,"name":"cephfs.smithi156.rdibys","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.156:6827/1080176013","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":1080176013},{"type":"v1","addr":"172.21.15.156:6827","nonce":1080176013}]},"join_fscid":1,"export_targets":[1],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14468":{"gid":14468,"name":"cephfs.smithi156.evwwaz","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":2,"addr":"172.21.15.156:6829/560329502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":560329502},{"type":"v1","addr":"172.21.15.156:6829","nonce":560329502}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24261":{"gid":24261,"name":"cephfs.smithi176.dvvbud","rank":1,"incarnation":7,"state":"up:active","state_seq":4,"addr":"172.21.15.176:6825/1577019287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6824","nonce":1577019287},{"type":"v1","addr":"172.21.15.176:6825","nonce":1577019287}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24273":{"gid":24273,"name":"cephfs.smithi176.wahbsi","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":2,"addr":"172.21.15.176:6827/2196775412","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6826","nonce":2196775412},{"type":"v1","addr":"172.21.15.176:6827","nonce":2196775412}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-07-16T10:06:54.656 INFO:teuthology.orchestra.run.smithi156.stderr:dumped fsmap epoch 17 2024-07-16T10:06:54.657 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:54.656+0000 7fc0fca43640 1 -- 172.21.15.156:0/1816768667 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7fc0e406e0b0 msgr2=0x7fc0e4070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:06:54.658 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:54.656+0000 7fc0fca43640 1 --2- 172.21.15.156:0/1816768667 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7fc0e406e0b0 0x7fc0e4070560 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fc0e0032430 tx=0x7fc0e0035040 comp rx=0 tx=0).stop 2024-07-16T10:06:54.658 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:54.656+0000 7fc0fca43640 1 -- 172.21.15.156:0/1816768667 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc0f8102350 msgr2=0x7fc0f8115290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:06:54.658 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:54.656+0000 7fc0fca43640 1 --2- 172.21.15.156:0/1816768667 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc0f8102350 0x7fc0f8115290 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fc0e800db80 tx=0x7fc0e8004f00 comp rx=0 tx=0).stop 2024-07-16T10:06:54.658 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:54.656+0000 7fc0fca43640 1 -- 172.21.15.156:0/1816768667 shutdown_connections 2024-07-16T10:06:54.658 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:54.656+0000 7fc0fca43640 1 --2- 172.21.15.156:0/1816768667 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7fc0e406e0b0 0x7fc0e4070560 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:06:54.658 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:54.656+0000 7fc0fca43640 1 --2- 172.21.15.156:0/1816768667 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc0f8102cf0 0x7fc0f81157d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:06:54.658 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:54.656+0000 7fc0fca43640 1 --2- 172.21.15.156:0/1816768667 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc0f8102350 0x7fc0f8115290 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:06:54.658 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:54.656+0000 7fc0fca43640 1 -- 172.21.15.156:0/1816768667 >> 172.21.15.156:0/1816768667 conn(0x7fc0f80f7f00 msgr2=0x7fc0f80f8df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T10:06:54.658 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:54.656+0000 7fc0fca43640 1 -- 172.21.15.156:0/1816768667 shutdown_connections 2024-07-16T10:06:54.658 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:54.656+0000 7fc0fca43640 1 -- 172.21.15.156:0/1816768667 wait complete. 2024-07-16T10:06:55.314 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 26c91c26-4358-11ef-bcad-c7b262605968 -- ceph fs dump --format=json 18 2024-07-16T10:06:55.571 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/26c91c26-4358-11ef-bcad-c7b262605968/mon.smithi156/config 2024-07-16T10:06:55.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:06:55 smithi156 ceph-mon[93066]: pgmap v311: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-07-16T10:06:55.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:06:55 smithi156 ceph-mon[93066]: from='client.? 172.21.15.156:0/1816768667' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 17, "format": "json"}]: dispatch 2024-07-16T10:06:55.843 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:06:55 smithi176 ceph-mon[81689]: pgmap v311: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-07-16T10:06:55.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:06:55 smithi176 ceph-mon[81689]: from='client.? 172.21.15.156:0/1816768667' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 17, "format": "json"}]: dispatch 2024-07-16T10:06:57.206 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:57.203+0000 7f0f74e66640 1 -- 172.21.15.156:0/2004621687 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0f70102370 msgr2=0x7f0f70102740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:06:57.207 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:57.203+0000 7f0f74e66640 1 --2- 172.21.15.156:0/2004621687 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0f70102370 0x7f0f70102740 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f0f600048d0 tx=0x7f0f60009da0 comp rx=0 tx=0).stop 2024-07-16T10:06:57.207 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:57.203+0000 7f0f74e66640 1 -- 172.21.15.156:0/2004621687 shutdown_connections 2024-07-16T10:06:57.207 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:57.203+0000 7f0f74e66640 1 --2- 172.21.15.156:0/2004621687 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0f70102d10 0x7f0f700fc3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:06:57.207 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:57.203+0000 7f0f74e66640 1 --2- 172.21.15.156:0/2004621687 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0f70102370 0x7f0f70102740 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:06:57.207 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:57.203+0000 7f0f74e66640 1 -- 172.21.15.156:0/2004621687 >> 172.21.15.156:0/2004621687 conn(0x7f0f700f7f00 msgr2=0x7f0f700fa310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T10:06:57.207 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:57.203+0000 7f0f74e66640 1 -- 172.21.15.156:0/2004621687 shutdown_connections 2024-07-16T10:06:57.207 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:57.203+0000 7f0f74e66640 1 -- 172.21.15.156:0/2004621687 wait complete. 2024-07-16T10:06:57.207 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:57.204+0000 7f0f74e66640 1 Processor -- start 2024-07-16T10:06:57.207 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:57.204+0000 7f0f74e66640 1 -- start start 2024-07-16T10:06:57.207 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:57.204+0000 7f0f74e66640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0f70102370 0x7f0f701133a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:06:57.208 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:57.204+0000 7f0f74e66640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0f70102d10 0x7f0f701138e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:06:57.208 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:57.204+0000 7f0f74e66640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f70113ee0 con 0x7f0f70102370 2024-07-16T10:06:57.208 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:57.204+0000 7f0f74e66640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f70114050 con 0x7f0f70102d10 2024-07-16T10:06:57.208 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:57.204+0000 7f0f6e59c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0f70102370 0x7f0f701133a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:06:57.208 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:57.204+0000 7f0f6e59c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0f70102370 0x7f0f701133a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:41178/0 (socket says 172.21.15.156:41178) 2024-07-16T10:06:57.208 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:57.204+0000 7f0f6e59c640 1 -- 172.21.15.156:0/2239629507 learned_addr learned my addr 172.21.15.156:0/2239629507 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T10:06:57.208 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:57.205+0000 7f0f67fff640 1 --2- 172.21.15.156:0/2239629507 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0f70102d10 0x7f0f701138e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:06:57.208 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:57.205+0000 7f0f6e59c640 1 -- 172.21.15.156:0/2239629507 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0f70102d10 msgr2=0x7f0f701138e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:06:57.208 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:57.205+0000 7f0f6e59c640 1 --2- 172.21.15.156:0/2239629507 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0f70102d10 0x7f0f701138e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:06:57.208 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:57.205+0000 7f0f6e59c640 1 -- 172.21.15.156:0/2239629507 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f60005040 con 0x7f0f70102370 2024-07-16T10:06:57.208 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:57.205+0000 7f0f67fff640 1 --2- 172.21.15.156:0/2239629507 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0f70102d10 0x7f0f701138e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-16T10:06:57.209 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:57.205+0000 7f0f6e59c640 1 --2- 172.21.15.156:0/2239629507 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0f70102370 0x7f0f701133a0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f0f600043d0 tx=0x7f0f60009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T10:06:57.209 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:57.205+0000 7f0f66ffd640 1 -- 172.21.15.156:0/2239629507 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0f60038070 con 0x7f0f70102370 2024-07-16T10:06:57.209 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:57.205+0000 7f0f66ffd640 1 -- 172.21.15.156:0/2239629507 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0f60032970 con 0x7f0f70102370 2024-07-16T10:06:57.209 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:57.205+0000 7f0f74e66640 1 -- 172.21.15.156:0/2239629507 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f7010d470 con 0x7f0f70102370 2024-07-16T10:06:57.209 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:57.205+0000 7f0f66ffd640 1 -- 172.21.15.156:0/2239629507 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0f6003da60 con 0x7f0f70102370 2024-07-16T10:06:57.209 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:57.205+0000 7f0f74e66640 1 -- 172.21.15.156:0/2239629507 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f7010d9b0 con 0x7f0f70102370 2024-07-16T10:06:57.210 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:57.206+0000 7f0f74e66640 1 -- 172.21.15.156:0/2239629507 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f700fdae0 con 0x7f0f70102370 2024-07-16T10:06:57.210 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:57.207+0000 7f0f66ffd640 1 -- 172.21.15.156:0/2239629507 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 38) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f0f60030020 con 0x7f0f70102370 2024-07-16T10:06:57.210 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:57.207+0000 7f0f66ffd640 1 --2- 172.21.15.156:0/2239629507 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f0f5c06e0b0 0x7f0f5c070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:06:57.210 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:57.207+0000 7f0f66ffd640 1 -- 172.21.15.156:0/2239629507 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f0f600adcc0 con 0x7f0f70102370 2024-07-16T10:06:57.210 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:57.207+0000 7f0f67fff640 1 --2- 172.21.15.156:0/2239629507 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f0f5c06e0b0 0x7f0f5c070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:06:57.210 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:57.208+0000 7f0f67fff640 1 --2- 172.21.15.156:0/2239629507 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f0f5c06e0b0 0x7f0f5c070560 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f0f7010eca0 tx=0x7f0f5800b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T10:06:57.213 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:57.212+0000 7f0f66ffd640 1 -- 172.21.15.156:0/2239629507 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187716 (secure 0 0 0) 0x7f0f60076f90 con 0x7f0f70102370 2024-07-16T10:06:57.427 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:57.426+0000 7f0f74e66640 1 -- 172.21.15.156:0/2239629507 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 18, "format": "json"} v 0) v1 -- 0x7f0f700fbb60 con 0x7f0f70102370 2024-07-16T10:06:57.428 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:57.427+0000 7f0f66ffd640 1 -- 172.21.15.156:0/2239629507 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 18, "format": "json"}]=0 dumped fsmap epoch 18 v45) v1 ==== 107+0+4897 (secure 0 0 0) 0x7f0f6007a650 con 0x7f0f70102370 2024-07-16T10:06:57.428 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-07-16T10:06:57.428 INFO:teuthology.orchestra.run.smithi156.stdout:{"epoch":18,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":18,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-16T09:52:55.214975+0000","modified":"2024-07-16T09:55:41.914320+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":14458,"mds_1":24261},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14458":{"gid":14458,"name":"cephfs.smithi156.rdibys","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.156:6827/1080176013","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":1080176013},{"type":"v1","addr":"172.21.15.156:6827","nonce":1080176013}]},"join_fscid":1,"export_targets":[1],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14468":{"gid":14468,"name":"cephfs.smithi156.evwwaz","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":2,"addr":"172.21.15.156:6829/560329502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":560329502},{"type":"v1","addr":"172.21.15.156:6829","nonce":560329502}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24261":{"gid":24261,"name":"cephfs.smithi176.dvvbud","rank":1,"incarnation":7,"state":"up:active","state_seq":4,"addr":"172.21.15.176:6825/1577019287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6824","nonce":1577019287},{"type":"v1","addr":"172.21.15.176:6825","nonce":1577019287}]},"join_fscid":1,"export_targets":[0],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24273":{"gid":24273,"name":"cephfs.smithi176.wahbsi","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":2,"addr":"172.21.15.176:6827/2196775412","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6826","nonce":2196775412},{"type":"v1","addr":"172.21.15.176:6827","nonce":2196775412}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-07-16T10:06:57.429 INFO:teuthology.orchestra.run.smithi156.stderr:dumped fsmap epoch 18 2024-07-16T10:06:57.430 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:57.429+0000 7f0f74e66640 1 -- 172.21.15.156:0/2239629507 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f0f5c06e0b0 msgr2=0x7f0f5c070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:06:57.430 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:57.429+0000 7f0f74e66640 1 --2- 172.21.15.156:0/2239629507 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f0f5c06e0b0 0x7f0f5c070560 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f0f7010eca0 tx=0x7f0f5800b040 comp rx=0 tx=0).stop 2024-07-16T10:06:57.430 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:57.429+0000 7f0f74e66640 1 -- 172.21.15.156:0/2239629507 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0f70102370 msgr2=0x7f0f701133a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:06:57.431 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:57.429+0000 7f0f74e66640 1 --2- 172.21.15.156:0/2239629507 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0f70102370 0x7f0f701133a0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f0f600043d0 tx=0x7f0f60009da0 comp rx=0 tx=0).stop 2024-07-16T10:06:57.431 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:57.429+0000 7f0f74e66640 1 -- 172.21.15.156:0/2239629507 shutdown_connections 2024-07-16T10:06:57.431 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:57.429+0000 7f0f74e66640 1 --2- 172.21.15.156:0/2239629507 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f0f5c06e0b0 0x7f0f5c070560 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:06:57.431 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:57.429+0000 7f0f74e66640 1 --2- 172.21.15.156:0/2239629507 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0f70102d10 0x7f0f701138e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:06:57.431 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:57.429+0000 7f0f74e66640 1 --2- 172.21.15.156:0/2239629507 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0f70102370 0x7f0f701133a0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:06:57.431 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:57.429+0000 7f0f74e66640 1 -- 172.21.15.156:0/2239629507 >> 172.21.15.156:0/2239629507 conn(0x7f0f700f7f00 msgr2=0x7f0f700f8e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T10:06:57.431 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:57.429+0000 7f0f74e66640 1 -- 172.21.15.156:0/2239629507 shutdown_connections 2024-07-16T10:06:57.431 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:57.429+0000 7f0f74e66640 1 -- 172.21.15.156:0/2239629507 wait complete. 2024-07-16T10:06:57.730 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:06:57 smithi156 ceph-mon[93066]: pgmap v312: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:06:57.730 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:06:57 smithi156 ceph-mon[93066]: from='client.? 172.21.15.156:0/2239629507' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 18, "format": "json"}]: dispatch 2024-07-16T10:06:57.843 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:06:57 smithi176 ceph-mon[81689]: pgmap v312: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:06:57.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:06:57 smithi176 ceph-mon[81689]: from='client.? 172.21.15.156:0/2239629507' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 18, "format": "json"}]: dispatch 2024-07-16T10:06:58.014 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 26c91c26-4358-11ef-bcad-c7b262605968 -- ceph fs dump --format=json 19 2024-07-16T10:06:58.271 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/26c91c26-4358-11ef-bcad-c7b262605968/mon.smithi156/config 2024-07-16T10:06:59.632 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:06:59 smithi156 ceph-mon[93066]: pgmap v313: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-07-16T10:06:59.843 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:06:59 smithi176 ceph-mon[81689]: pgmap v313: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-07-16T10:06:59.858 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:59.835+0000 7f94dce08640 1 -- 172.21.15.156:0/2808336330 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f94d8101bf0 msgr2=0x7f94d80fb280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:06:59.858 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:59.835+0000 7f94dce08640 1 --2- 172.21.15.156:0/2808336330 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f94d8101bf0 0x7f94d80fb280 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f94cc0048d0 tx=0x7f94cc009da0 comp rx=0 tx=0).stop 2024-07-16T10:06:59.858 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:59.835+0000 7f94dce08640 1 -- 172.21.15.156:0/2808336330 shutdown_connections 2024-07-16T10:06:59.858 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:59.835+0000 7f94dce08640 1 --2- 172.21.15.156:0/2808336330 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f94d8101bf0 0x7f94d80fb280 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:06:59.858 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:59.835+0000 7f94dce08640 1 --2- 172.21.15.156:0/2808336330 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f94d8101250 0x7f94d8101620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:06:59.858 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:59.835+0000 7f94dce08640 1 -- 172.21.15.156:0/2808336330 >> 172.21.15.156:0/2808336330 conn(0x7f94d80f6e40 msgr2=0x7f94d80f9250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T10:06:59.858 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:59.835+0000 7f94dce08640 1 -- 172.21.15.156:0/2808336330 shutdown_connections 2024-07-16T10:06:59.858 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:59.835+0000 7f94dce08640 1 -- 172.21.15.156:0/2808336330 wait complete. 2024-07-16T10:06:59.858 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:59.835+0000 7f94dce08640 1 Processor -- start 2024-07-16T10:06:59.858 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:59.835+0000 7f94dce08640 1 -- start start 2024-07-16T10:06:59.859 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:59.835+0000 7f94dce08640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f94d8101250 0x7f94d810e240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:06:59.859 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:59.835+0000 7f94dce08640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f94d8101bf0 0x7f94d810e780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:06:59.859 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:59.835+0000 7f94dce08640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94d810ecc0 con 0x7f94d8101250 2024-07-16T10:06:59.859 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:59.835+0000 7f94dce08640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94d810ee30 con 0x7f94d8101bf0 2024-07-16T10:06:59.859 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:59.836+0000 7f94d659c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f94d8101250 0x7f94d810e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:06:59.859 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:59.836+0000 7f94d659c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f94d8101250 0x7f94d810e240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:41204/0 (socket says 172.21.15.156:41204) 2024-07-16T10:06:59.859 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:59.836+0000 7f94d659c640 1 -- 172.21.15.156:0/370290409 learned_addr learned my addr 172.21.15.156:0/370290409 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T10:06:59.859 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:59.836+0000 7f94d5d9b640 1 --2- 172.21.15.156:0/370290409 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f94d8101bf0 0x7f94d810e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:06:59.859 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:59.836+0000 7f94d659c640 1 -- 172.21.15.156:0/370290409 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f94d8101bf0 msgr2=0x7f94d810e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:06:59.859 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:59.836+0000 7f94d659c640 1 --2- 172.21.15.156:0/370290409 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f94d8101bf0 0x7f94d810e780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:06:59.859 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:59.836+0000 7f94d659c640 1 -- 172.21.15.156:0/370290409 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94cc005040 con 0x7f94d8101250 2024-07-16T10:06:59.859 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:59.836+0000 7f94d659c640 1 --2- 172.21.15.156:0/370290409 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f94d8101250 0x7f94d810e240 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f94c800bbb0 tx=0x7f94c8009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T10:06:59.860 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:59.836+0000 7f94c6ffd640 1 -- 172.21.15.156:0/370290409 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f94c800c9c0 con 0x7f94d8101250 2024-07-16T10:06:59.860 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:59.836+0000 7f94c6ffd640 1 -- 172.21.15.156:0/370290409 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f94c80092e0 con 0x7f94d8101250 2024-07-16T10:06:59.860 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:59.836+0000 7f94dce08640 1 -- 172.21.15.156:0/370290409 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94d8117170 con 0x7f94d8101250 2024-07-16T10:06:59.860 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:59.836+0000 7f94c6ffd640 1 -- 172.21.15.156:0/370290409 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f94c8007640 con 0x7f94d8101250 2024-07-16T10:06:59.860 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:59.836+0000 7f94dce08640 1 -- 172.21.15.156:0/370290409 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94d8117690 con 0x7f94d8101250 2024-07-16T10:06:59.860 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:59.837+0000 7f94dce08640 1 -- 172.21.15.156:0/370290409 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94d80fc9c0 con 0x7f94d8101250 2024-07-16T10:06:59.860 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:59.838+0000 7f94c6ffd640 1 -- 172.21.15.156:0/370290409 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 38) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f94c8012070 con 0x7f94d8101250 2024-07-16T10:06:59.860 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:59.838+0000 7f94c6ffd640 1 --2- 172.21.15.156:0/370290409 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f94c006e0b0 0x7f94c0070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:06:59.860 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:59.838+0000 7f94d5d9b640 1 --2- 172.21.15.156:0/370290409 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f94c006e0b0 0x7f94c0070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:06:59.860 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:59.839+0000 7f94c6ffd640 1 -- 172.21.15.156:0/370290409 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f94c808dba0 con 0x7f94d8101250 2024-07-16T10:06:59.860 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:59.839+0000 7f94d5d9b640 1 --2- 172.21.15.156:0/370290409 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f94c006e0b0 0x7f94c0070560 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f94cc009f80 tx=0x7f94cc035040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T10:06:59.860 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:06:59.843+0000 7f94c6ffd640 1 -- 172.21.15.156:0/370290409 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187716 (secure 0 0 0) 0x7f94c8056ef0 con 0x7f94d8101250 2024-07-16T10:07:00.056 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:00.054+0000 7f94dce08640 1 -- 172.21.15.156:0/370290409 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 19, "format": "json"} v 0) v1 -- 0x7f94d80faa40 con 0x7f94d8101250 2024-07-16T10:07:00.056 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:00.055+0000 7f94c6ffd640 1 -- 172.21.15.156:0/370290409 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 19, "format": "json"}]=0 dumped fsmap epoch 19 v45) v1 ==== 107+0+4911 (secure 0 0 0) 0x7f94c805a5b0 con 0x7f94d8101250 2024-07-16T10:07:00.057 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-07-16T10:07:00.057 INFO:teuthology.orchestra.run.smithi156.stdout:{"epoch":19,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":19,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-16T09:52:55.214975+0000","modified":"2024-07-16T09:58:14.542481+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":14458,"mds_1":24261},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14458":{"gid":14458,"name":"cephfs.smithi156.rdibys","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.156:6827/1080176013","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":1080176013},{"type":"v1","addr":"172.21.15.156:6827","nonce":1080176013}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14468":{"gid":14468,"name":"cephfs.smithi156.evwwaz","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":2,"addr":"172.21.15.156:6829/560329502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":560329502},{"type":"v1","addr":"172.21.15.156:6829","nonce":560329502}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24261":{"gid":24261,"name":"cephfs.smithi176.dvvbud","rank":1,"incarnation":7,"state":"up:active","state_seq":4,"addr":"172.21.15.176:6825/1577019287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6824","nonce":1577019287},{"type":"v1","addr":"172.21.15.176:6825","nonce":1577019287}]},"join_fscid":1,"export_targets":[0],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24273":{"gid":24273,"name":"cephfs.smithi176.wahbsi","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":2,"addr":"172.21.15.176:6827/2196775412","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6826","nonce":2196775412},{"type":"v1","addr":"172.21.15.176:6827","nonce":2196775412}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":14458,"qdb_cluster":[14458,24261]},"id":1}]} 2024-07-16T10:07:00.057 INFO:teuthology.orchestra.run.smithi156.stderr:dumped fsmap epoch 19 2024-07-16T10:07:00.058 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:00.057+0000 7f94dce08640 1 -- 172.21.15.156:0/370290409 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f94c006e0b0 msgr2=0x7f94c0070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:07:00.058 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:00.057+0000 7f94dce08640 1 --2- 172.21.15.156:0/370290409 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f94c006e0b0 0x7f94c0070560 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f94cc009f80 tx=0x7f94cc035040 comp rx=0 tx=0).stop 2024-07-16T10:07:00.058 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:00.057+0000 7f94dce08640 1 -- 172.21.15.156:0/370290409 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f94d8101250 msgr2=0x7f94d810e240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:07:00.058 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:00.057+0000 7f94dce08640 1 --2- 172.21.15.156:0/370290409 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f94d8101250 0x7f94d810e240 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f94c800bbb0 tx=0x7f94c8009840 comp rx=0 tx=0).stop 2024-07-16T10:07:00.058 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:00.057+0000 7f94dce08640 1 -- 172.21.15.156:0/370290409 shutdown_connections 2024-07-16T10:07:00.059 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:00.057+0000 7f94dce08640 1 --2- 172.21.15.156:0/370290409 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f94c006e0b0 0x7f94c0070560 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:07:00.059 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:00.057+0000 7f94dce08640 1 --2- 172.21.15.156:0/370290409 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f94d8101bf0 0x7f94d810e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:07:00.059 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:00.057+0000 7f94dce08640 1 --2- 172.21.15.156:0/370290409 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f94d8101250 0x7f94d810e240 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:07:00.059 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:00.057+0000 7f94dce08640 1 -- 172.21.15.156:0/370290409 >> 172.21.15.156:0/370290409 conn(0x7f94d80f6e40 msgr2=0x7f94d80f7d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T10:07:00.059 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:00.057+0000 7f94dce08640 1 -- 172.21.15.156:0/370290409 shutdown_connections 2024-07-16T10:07:00.059 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:00.057+0000 7f94dce08640 1 -- 172.21.15.156:0/370290409 wait complete. 2024-07-16T10:07:00.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:07:00 smithi156 ceph-mon[93066]: from='client.? 172.21.15.156:0/370290409' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 19, "format": "json"}]: dispatch 2024-07-16T10:07:00.712 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 26c91c26-4358-11ef-bcad-c7b262605968 -- ceph fs dump --format=json 20 2024-07-16T10:07:00.843 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:07:00 smithi176 ceph-mon[81689]: from='client.? 172.21.15.156:0/370290409' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 19, "format": "json"}]: dispatch 2024-07-16T10:07:00.971 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/26c91c26-4358-11ef-bcad-c7b262605968/mon.smithi156/config 2024-07-16T10:07:01.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:07:01 smithi156 ceph-mon[93066]: pgmap v314: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:07:01.843 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:07:01 smithi176 ceph-mon[81689]: pgmap v314: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:07:02.544 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:02.543+0000 7fe1abb0c640 1 -- 172.21.15.156:0/1312638742 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe1a4101280 msgr2=0x7fe1a4101650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:07:02.545 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:02.543+0000 7fe1abb0c640 1 --2- 172.21.15.156:0/1312638742 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe1a4101280 0x7fe1a4101650 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fe1a00048d0 tx=0x7fe1a0009da0 comp rx=0 tx=0).stop 2024-07-16T10:07:02.545 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:02.543+0000 7fe1abb0c640 1 -- 172.21.15.156:0/1312638742 shutdown_connections 2024-07-16T10:07:02.545 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:02.543+0000 7fe1abb0c640 1 --2- 172.21.15.156:0/1312638742 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe1a4101c20 0x7fe1a40fb2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:07:02.545 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:02.543+0000 7fe1abb0c640 1 --2- 172.21.15.156:0/1312638742 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe1a4101280 0x7fe1a4101650 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:07:02.545 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:02.543+0000 7fe1abb0c640 1 -- 172.21.15.156:0/1312638742 >> 172.21.15.156:0/1312638742 conn(0x7fe1a40f6e30 msgr2=0x7fe1a40f9240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T10:07:02.545 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:02.543+0000 7fe1abb0c640 1 -- 172.21.15.156:0/1312638742 shutdown_connections 2024-07-16T10:07:02.545 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:02.543+0000 7fe1abb0c640 1 -- 172.21.15.156:0/1312638742 wait complete. 2024-07-16T10:07:02.547 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:02.544+0000 7fe1abb0c640 1 Processor -- start 2024-07-16T10:07:02.547 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:02.544+0000 7fe1abb0c640 1 -- start start 2024-07-16T10:07:02.547 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:02.544+0000 7fe1abb0c640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe1a4101280 0x7fe1a4111fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:07:02.547 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:02.544+0000 7fe1abb0c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe1a4101c20 0x7fe1a41124e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:07:02.547 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:02.544+0000 7fe1abb0c640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1a4112ae0 con 0x7fe1a4101c20 2024-07-16T10:07:02.547 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:02.544+0000 7fe1abb0c640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1a410bf20 con 0x7fe1a4101280 2024-07-16T10:07:02.547 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:02.544+0000 7fe1a90a7640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe1a4101c20 0x7fe1a41124e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:07:02.547 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:02.544+0000 7fe1a90a7640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe1a4101c20 0x7fe1a41124e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:41230/0 (socket says 172.21.15.156:41230) 2024-07-16T10:07:02.547 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:02.544+0000 7fe1a90a7640 1 -- 172.21.15.156:0/2235447085 learned_addr learned my addr 172.21.15.156:0/2235447085 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T10:07:02.548 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:02.544+0000 7fe1a98a8640 1 --2- 172.21.15.156:0/2235447085 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe1a4101280 0x7fe1a4111fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:07:02.548 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:02.545+0000 7fe1a90a7640 1 -- 172.21.15.156:0/2235447085 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe1a4101280 msgr2=0x7fe1a4111fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:07:02.548 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:02.545+0000 7fe1a90a7640 1 --2- 172.21.15.156:0/2235447085 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe1a4101280 0x7fe1a4111fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:07:02.548 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:02.545+0000 7fe1a90a7640 1 -- 172.21.15.156:0/2235447085 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1a0005040 con 0x7fe1a4101c20 2024-07-16T10:07:02.548 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:02.545+0000 7fe1a90a7640 1 --2- 172.21.15.156:0/2235447085 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe1a4101c20 0x7fe1a41124e0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fe19400c790 tx=0x7fe1940043c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T10:07:02.548 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:02.545+0000 7fe19a7fc640 1 -- 172.21.15.156:0/2235447085 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe19400a7a0 con 0x7fe1a4101c20 2024-07-16T10:07:02.548 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:02.545+0000 7fe19a7fc640 1 -- 172.21.15.156:0/2235447085 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe19400add0 con 0x7fe1a4101c20 2024-07-16T10:07:02.548 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:02.545+0000 7fe1abb0c640 1 -- 172.21.15.156:0/2235447085 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1a410c200 con 0x7fe1a4101c20 2024-07-16T10:07:02.548 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:02.545+0000 7fe19a7fc640 1 -- 172.21.15.156:0/2235447085 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe1940115e0 con 0x7fe1a4101c20 2024-07-16T10:07:02.548 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:02.545+0000 7fe1abb0c640 1 -- 172.21.15.156:0/2235447085 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1a410c7a0 con 0x7fe1a4101c20 2024-07-16T10:07:02.548 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:02.546+0000 7fe1abb0c640 1 -- 172.21.15.156:0/2235447085 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1a40fc9f0 con 0x7fe1a4101c20 2024-07-16T10:07:02.549 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:02.547+0000 7fe19a7fc640 1 -- 172.21.15.156:0/2235447085 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 38) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fe194012400 con 0x7fe1a4101c20 2024-07-16T10:07:02.549 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:02.547+0000 7fe19a7fc640 1 --2- 172.21.15.156:0/2235447085 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7fe19006e0b0 0x7fe190070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:07:02.549 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:02.547+0000 7fe19a7fc640 1 -- 172.21.15.156:0/2235447085 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7fe19408db50 con 0x7fe1a4101c20 2024-07-16T10:07:02.549 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:02.547+0000 7fe1a98a8640 1 --2- 172.21.15.156:0/2235447085 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7fe19006e0b0 0x7fe190070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:07:02.550 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:02.548+0000 7fe1a98a8640 1 --2- 172.21.15.156:0/2235447085 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7fe19006e0b0 0x7fe190070560 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fe1a0009f80 tx=0x7fe1a0035040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T10:07:02.552 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:02.551+0000 7fe19a7fc640 1 -- 172.21.15.156:0/2235447085 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187716 (secure 0 0 0) 0x7fe194056f40 con 0x7fe1a4101c20 2024-07-16T10:07:02.762 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:02.761+0000 7fe1abb0c640 1 -- 172.21.15.156:0/2235447085 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 20, "format": "json"} v 0) v1 -- 0x7fe1a4108440 con 0x7fe1a4101c20 2024-07-16T10:07:02.763 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:02.762+0000 7fe19a7fc640 1 -- 172.21.15.156:0/2235447085 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 20, "format": "json"}]=0 dumped fsmap epoch 20 v45) v1 ==== 107+0+4910 (secure 0 0 0) 0x7fe19405a600 con 0x7fe1a4101c20 2024-07-16T10:07:02.763 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-07-16T10:07:02.763 INFO:teuthology.orchestra.run.smithi156.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-16T09:52:55.214975+0000","modified":"2024-07-16T09:58:56.574272+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":14458,"mds_1":24261},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14458":{"gid":14458,"name":"cephfs.smithi156.rdibys","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.156:6827/1080176013","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":1080176013},{"type":"v1","addr":"172.21.15.156:6827","nonce":1080176013}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14468":{"gid":14468,"name":"cephfs.smithi156.evwwaz","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":2,"addr":"172.21.15.156:6829/560329502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":560329502},{"type":"v1","addr":"172.21.15.156:6829","nonce":560329502}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24261":{"gid":24261,"name":"cephfs.smithi176.dvvbud","rank":1,"incarnation":7,"state":"up:active","state_seq":4,"addr":"172.21.15.176:6825/1577019287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6824","nonce":1577019287},{"type":"v1","addr":"172.21.15.176:6825","nonce":1577019287}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24273":{"gid":24273,"name":"cephfs.smithi176.wahbsi","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":2,"addr":"172.21.15.176:6827/2196775412","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6826","nonce":2196775412},{"type":"v1","addr":"172.21.15.176:6827","nonce":2196775412}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":14458,"qdb_cluster":[24261,14458]},"id":1}]} 2024-07-16T10:07:02.764 INFO:teuthology.orchestra.run.smithi156.stderr:dumped fsmap epoch 20 2024-07-16T10:07:02.765 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:02.764+0000 7fe1abb0c640 1 -- 172.21.15.156:0/2235447085 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7fe19006e0b0 msgr2=0x7fe190070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:07:02.765 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:02.764+0000 7fe1abb0c640 1 --2- 172.21.15.156:0/2235447085 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7fe19006e0b0 0x7fe190070560 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fe1a0009f80 tx=0x7fe1a0035040 comp rx=0 tx=0).stop 2024-07-16T10:07:02.766 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:02.764+0000 7fe1abb0c640 1 -- 172.21.15.156:0/2235447085 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe1a4101c20 msgr2=0x7fe1a41124e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:07:02.766 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:02.764+0000 7fe1abb0c640 1 --2- 172.21.15.156:0/2235447085 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe1a4101c20 0x7fe1a41124e0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fe19400c790 tx=0x7fe1940043c0 comp rx=0 tx=0).stop 2024-07-16T10:07:02.766 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:02.764+0000 7fe1abb0c640 1 -- 172.21.15.156:0/2235447085 shutdown_connections 2024-07-16T10:07:02.766 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:02.764+0000 7fe1abb0c640 1 --2- 172.21.15.156:0/2235447085 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7fe19006e0b0 0x7fe190070560 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:07:02.766 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:02.764+0000 7fe1abb0c640 1 --2- 172.21.15.156:0/2235447085 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe1a4101c20 0x7fe1a41124e0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:07:02.766 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:02.764+0000 7fe1abb0c640 1 --2- 172.21.15.156:0/2235447085 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe1a4101280 0x7fe1a4111fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:07:02.766 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:02.764+0000 7fe1abb0c640 1 -- 172.21.15.156:0/2235447085 >> 172.21.15.156:0/2235447085 conn(0x7fe1a40f6e30 msgr2=0x7fe1a40f7db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T10:07:02.766 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:02.764+0000 7fe1abb0c640 1 -- 172.21.15.156:0/2235447085 shutdown_connections 2024-07-16T10:07:02.766 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:02.764+0000 7fe1abb0c640 1 -- 172.21.15.156:0/2235447085 wait complete. 2024-07-16T10:07:03.464 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 26c91c26-4358-11ef-bcad-c7b262605968 -- ceph fs dump --format=json 21 2024-07-16T10:07:03.725 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/26c91c26-4358-11ef-bcad-c7b262605968/mon.smithi156/config 2024-07-16T10:07:03.843 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:07:03 smithi176 ceph-mon[81689]: pgmap v315: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:07:03.843 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:07:03 smithi176 ceph-mon[81689]: from='client.? 172.21.15.156:0/2235447085' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 20, "format": "json"}]: dispatch 2024-07-16T10:07:03.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:07:03 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-16T10:07:03.848 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:07:03 smithi156 ceph-mon[93066]: pgmap v315: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:07:03.848 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:07:03 smithi156 ceph-mon[93066]: from='client.? 172.21.15.156:0/2235447085' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 20, "format": "json"}]: dispatch 2024-07-16T10:07:03.848 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:07:03 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-16T10:07:04.843 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:07:04 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-16T10:07:04.881 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:07:04 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-16T10:07:05.481 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:05.458+0000 7f951aa94640 1 -- 172.21.15.156:0/3070619693 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f95140650e0 msgr2=0x7f95140654b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:07:05.481 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:05.458+0000 7f951aa94640 1 --2- 172.21.15.156:0/3070619693 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f95140650e0 0x7f95140654b0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f950c0048d0 tx=0x7f950c009da0 comp rx=0 tx=0).stop 2024-07-16T10:07:05.481 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:05.458+0000 7f951aa94640 1 -- 172.21.15.156:0/3070619693 shutdown_connections 2024-07-16T10:07:05.481 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:05.458+0000 7f951aa94640 1 --2- 172.21.15.156:0/3070619693 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f95140fdf50 0x7f95140fe3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:07:05.481 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:05.458+0000 7f951aa94640 1 --2- 172.21.15.156:0/3070619693 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f95140650e0 0x7f95140654b0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:07:05.481 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:05.458+0000 7f951aa94640 1 -- 172.21.15.156:0/3070619693 >> 172.21.15.156:0/3070619693 conn(0x7f95140721c0 msgr2=0x7f95140725c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T10:07:05.482 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:05.458+0000 7f951aa94640 1 -- 172.21.15.156:0/3070619693 shutdown_connections 2024-07-16T10:07:05.482 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:05.458+0000 7f951aa94640 1 -- 172.21.15.156:0/3070619693 wait complete. 2024-07-16T10:07:05.482 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:05.458+0000 7f951aa94640 1 Processor -- start 2024-07-16T10:07:05.482 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:05.458+0000 7f951aa94640 1 -- start start 2024-07-16T10:07:05.482 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:05.459+0000 7f951aa94640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f95140650e0 0x7f95141130f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:07:05.482 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:05.459+0000 7f951aa94640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f95140fdf50 0x7f9514113630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:07:05.482 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:05.459+0000 7f951aa94640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9514113c30 con 0x7f95140fdf50 2024-07-16T10:07:05.482 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:05.459+0000 7f951aa94640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f951410d0c0 con 0x7f95140650e0 2024-07-16T10:07:05.482 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:05.459+0000 7f9513fff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f95140fdf50 0x7f9514113630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:07:05.482 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:05.459+0000 7f9513fff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f95140fdf50 0x7f9514113630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:55106/0 (socket says 172.21.15.156:55106) 2024-07-16T10:07:05.482 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:05.459+0000 7f9513fff640 1 -- 172.21.15.156:0/4105185139 learned_addr learned my addr 172.21.15.156:0/4105185139 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T10:07:05.482 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:05.459+0000 7f9518830640 1 --2- 172.21.15.156:0/4105185139 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f95140650e0 0x7f95141130f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:07:05.483 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:05.459+0000 7f9513fff640 1 -- 172.21.15.156:0/4105185139 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f95140650e0 msgr2=0x7f95141130f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:07:05.483 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:05.459+0000 7f9513fff640 1 --2- 172.21.15.156:0/4105185139 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f95140650e0 0x7f95141130f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:07:05.483 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:05.459+0000 7f9513fff640 1 -- 172.21.15.156:0/4105185139 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f950c005040 con 0x7f95140fdf50 2024-07-16T10:07:05.483 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:05.459+0000 7f9513fff640 1 --2- 172.21.15.156:0/4105185139 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f95140fdf50 0x7f9514113630 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f94fc00eb70 tx=0x7f94fc00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T10:07:05.483 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:05.459+0000 7f95117fa640 1 -- 172.21.15.156:0/4105185139 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f94fc007c30 con 0x7f95140fdf50 2024-07-16T10:07:05.483 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:05.460+0000 7f95117fa640 1 -- 172.21.15.156:0/4105185139 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f94fc00f040 con 0x7f95140fdf50 2024-07-16T10:07:05.483 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:05.460+0000 7f951aa94640 1 -- 172.21.15.156:0/4105185139 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f951410d340 con 0x7f95140fdf50 2024-07-16T10:07:05.483 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:05.460+0000 7f95117fa640 1 -- 172.21.15.156:0/4105185139 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f94fc00cad0 con 0x7f95140fdf50 2024-07-16T10:07:05.483 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:05.460+0000 7f951aa94640 1 -- 172.21.15.156:0/4105185139 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f951410d860 con 0x7f95140fdf50 2024-07-16T10:07:05.483 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:05.461+0000 7f951aa94640 1 -- 172.21.15.156:0/4105185139 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95140fdaf0 con 0x7f95140fdf50 2024-07-16T10:07:05.483 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:05.461+0000 7f95117fa640 1 -- 172.21.15.156:0/4105185139 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 38) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f94fc00cc50 con 0x7f95140fdf50 2024-07-16T10:07:05.483 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:05.462+0000 7f95117fa640 1 --2- 172.21.15.156:0/4105185139 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f950006e0b0 0x7f9500070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:07:05.483 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:05.462+0000 7f9518830640 1 --2- 172.21.15.156:0/4105185139 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f950006e0b0 0x7f9500070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:07:05.484 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:05.462+0000 7f95117fa640 1 -- 172.21.15.156:0/4105185139 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f94fc08d830 con 0x7f95140fdf50 2024-07-16T10:07:05.484 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:05.462+0000 7f9518830640 1 --2- 172.21.15.156:0/4105185139 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f950006e0b0 0x7f9500070560 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f950c0048a0 tx=0x7f950c035040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T10:07:05.484 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:05.466+0000 7f95117fa640 1 -- 172.21.15.156:0/4105185139 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187716 (secure 0 0 0) 0x7f94fc056bb0 con 0x7f95140fdf50 2024-07-16T10:07:05.683 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:05.682+0000 7f951aa94640 1 -- 172.21.15.156:0/4105185139 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 21, "format": "json"} v 0) v1 -- 0x7f951410e760 con 0x7f95140fdf50 2024-07-16T10:07:05.684 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:05.682+0000 7f95117fa640 1 -- 172.21.15.156:0/4105185139 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 21, "format": "json"}]=0 dumped fsmap epoch 21 v45) v1 ==== 107+0+3336 (secure 0 0 0) 0x7f94fc019d20 con 0x7f95140fdf50 2024-07-16T10:07:05.684 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-07-16T10:07:05.684 INFO:teuthology.orchestra.run.smithi156.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":21,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-16T09:52:55.214975+0000","modified":"2024-07-16T10:01:53.664538+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":14458,"mds_1":24261},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14458":{"gid":14458,"name":"cephfs.smithi156.rdibys","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.156:6827/1080176013","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":1080176013},{"type":"v1","addr":"172.21.15.156:6827","nonce":1080176013}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24261":{"gid":24261,"name":"cephfs.smithi176.dvvbud","rank":1,"incarnation":7,"state":"up:active","state_seq":4,"addr":"172.21.15.176:6825/1577019287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6824","nonce":1577019287},{"type":"v1","addr":"172.21.15.176:6825","nonce":1577019287}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":14458,"qdb_cluster":[14458,24261]},"id":1}]} 2024-07-16T10:07:05.684 INFO:teuthology.orchestra.run.smithi156.stderr:dumped fsmap epoch 21 2024-07-16T10:07:05.686 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:05.685+0000 7f951aa94640 1 -- 172.21.15.156:0/4105185139 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f950006e0b0 msgr2=0x7f9500070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:07:05.686 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:05.685+0000 7f951aa94640 1 --2- 172.21.15.156:0/4105185139 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f950006e0b0 0x7f9500070560 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f950c0048a0 tx=0x7f950c035040 comp rx=0 tx=0).stop 2024-07-16T10:07:05.686 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:05.685+0000 7f951aa94640 1 -- 172.21.15.156:0/4105185139 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f95140fdf50 msgr2=0x7f9514113630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:07:05.686 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:05.685+0000 7f951aa94640 1 --2- 172.21.15.156:0/4105185139 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f95140fdf50 0x7f9514113630 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f94fc00eb70 tx=0x7f94fc00b040 comp rx=0 tx=0).stop 2024-07-16T10:07:05.687 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:05.685+0000 7f951aa94640 1 -- 172.21.15.156:0/4105185139 shutdown_connections 2024-07-16T10:07:05.687 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:05.685+0000 7f951aa94640 1 --2- 172.21.15.156:0/4105185139 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f950006e0b0 0x7f9500070560 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:07:05.687 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:05.685+0000 7f951aa94640 1 --2- 172.21.15.156:0/4105185139 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f95140fdf50 0x7f9514113630 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:07:05.687 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:05.685+0000 7f951aa94640 1 --2- 172.21.15.156:0/4105185139 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f95140650e0 0x7f95141130f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:07:05.687 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:05.685+0000 7f951aa94640 1 -- 172.21.15.156:0/4105185139 >> 172.21.15.156:0/4105185139 conn(0x7f95140721c0 msgr2=0x7f95141051d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T10:07:05.687 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:05.685+0000 7f951aa94640 1 -- 172.21.15.156:0/4105185139 shutdown_connections 2024-07-16T10:07:05.687 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:05.685+0000 7f951aa94640 1 -- 172.21.15.156:0/4105185139 wait complete. 2024-07-16T10:07:05.843 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:07:05 smithi176 ceph-mon[81689]: pgmap v316: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-07-16T10:07:05.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:07:05 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-16T10:07:05.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:07:05 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-16T10:07:05.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:07:05 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:07:05.924 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:07:05 smithi156 ceph-mon[93066]: pgmap v316: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-07-16T10:07:05.924 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:07:05 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-16T10:07:05.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:07:05 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-16T10:07:05.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:07:05 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:07:06.310 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 21 2024-07-16T10:07:06.310 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 26c91c26-4358-11ef-bcad-c7b262605968 -- ceph fs dump --format=json 22 2024-07-16T10:07:06.570 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/26c91c26-4358-11ef-bcad-c7b262605968/mon.smithi156/config 2024-07-16T10:07:06.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:07:06 smithi156 ceph-mon[93066]: from='client.? 172.21.15.156:0/4105185139' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 21, "format": "json"}]: dispatch 2024-07-16T10:07:06.843 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:07:06 smithi176 ceph-mon[81689]: from='client.? 172.21.15.156:0/4105185139' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 21, "format": "json"}]: dispatch 2024-07-16T10:07:07.642 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:07:07 smithi156 ceph-mon[93066]: pgmap v317: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:07:07.843 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:07:07 smithi176 ceph-mon[81689]: pgmap v317: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:07:08.221 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:08.215+0000 7fa7621cd640 1 -- 172.21.15.156:0/2718912695 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa75c102370 msgr2=0x7fa75c102740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:07:08.221 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:08.215+0000 7fa7621cd640 1 --2- 172.21.15.156:0/2718912695 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa75c102370 0x7fa75c102740 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fa74c0048d0 tx=0x7fa74c009da0 comp rx=0 tx=0).stop 2024-07-16T10:07:08.221 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:08.215+0000 7fa7621cd640 1 -- 172.21.15.156:0/2718912695 shutdown_connections 2024-07-16T10:07:08.221 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:08.215+0000 7fa7621cd640 1 --2- 172.21.15.156:0/2718912695 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa75c102d10 0x7fa75c0fc3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:07:08.221 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:08.215+0000 7fa7621cd640 1 --2- 172.21.15.156:0/2718912695 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa75c102370 0x7fa75c102740 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:07:08.221 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:08.215+0000 7fa7621cd640 1 -- 172.21.15.156:0/2718912695 >> 172.21.15.156:0/2718912695 conn(0x7fa75c0f7f00 msgr2=0x7fa75c0fa310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T10:07:08.221 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:08.215+0000 7fa7621cd640 1 -- 172.21.15.156:0/2718912695 shutdown_connections 2024-07-16T10:07:08.221 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:08.215+0000 7fa7621cd640 1 -- 172.21.15.156:0/2718912695 wait complete. 2024-07-16T10:07:08.222 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:08.216+0000 7fa7621cd640 1 Processor -- start 2024-07-16T10:07:08.222 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:08.216+0000 7fa7621cd640 1 -- start start 2024-07-16T10:07:08.222 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:08.216+0000 7fa7621cd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa75c102370 0x7fa75c1133c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:07:08.222 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:08.216+0000 7fa7621cd640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa75c102d10 0x7fa75c113900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:07:08.222 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:08.216+0000 7fa7621cd640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa75c113f00 con 0x7fa75c102370 2024-07-16T10:07:08.222 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:08.216+0000 7fa7621cd640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa75c10d340 con 0x7fa75c102d10 2024-07-16T10:07:08.222 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:08.216+0000 7fa75b7fe640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa75c102370 0x7fa75c1133c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:07:08.222 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:08.216+0000 7fa75b7fe640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa75c102370 0x7fa75c1133c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:55118/0 (socket says 172.21.15.156:55118) 2024-07-16T10:07:08.222 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:08.216+0000 7fa75b7fe640 1 -- 172.21.15.156:0/1318517519 learned_addr learned my addr 172.21.15.156:0/1318517519 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T10:07:08.222 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:08.216+0000 7fa753fff640 1 --2- 172.21.15.156:0/1318517519 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa75c102d10 0x7fa75c113900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:07:08.222 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:08.216+0000 7fa75b7fe640 1 -- 172.21.15.156:0/1318517519 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa75c102d10 msgr2=0x7fa75c113900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:07:08.222 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:08.216+0000 7fa75b7fe640 1 --2- 172.21.15.156:0/1318517519 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa75c102d10 0x7fa75c113900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:07:08.223 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:08.216+0000 7fa75b7fe640 1 -- 172.21.15.156:0/1318517519 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa74c005040 con 0x7fa75c102370 2024-07-16T10:07:08.223 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:08.216+0000 7fa75b7fe640 1 --2- 172.21.15.156:0/1318517519 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa75c102370 0x7fa75c1133c0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fa74c0043d0 tx=0x7fa74c009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T10:07:08.223 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:08.217+0000 7fa758ff9640 1 -- 172.21.15.156:0/1318517519 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa74c038070 con 0x7fa75c102370 2024-07-16T10:07:08.223 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:08.217+0000 7fa758ff9640 1 -- 172.21.15.156:0/1318517519 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa74c032970 con 0x7fa75c102370 2024-07-16T10:07:08.223 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:08.217+0000 7fa7621cd640 1 -- 172.21.15.156:0/1318517519 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa75c10d5c0 con 0x7fa75c102370 2024-07-16T10:07:08.223 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:08.217+0000 7fa758ff9640 1 -- 172.21.15.156:0/1318517519 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa74c03da60 con 0x7fa75c102370 2024-07-16T10:07:08.223 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:08.217+0000 7fa7621cd640 1 -- 172.21.15.156:0/1318517519 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa75c10db00 con 0x7fa75c102370 2024-07-16T10:07:08.223 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:08.218+0000 7fa7621cd640 1 -- 172.21.15.156:0/1318517519 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa75c0fdae0 con 0x7fa75c102370 2024-07-16T10:07:08.223 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:08.218+0000 7fa758ff9640 1 -- 172.21.15.156:0/1318517519 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 38) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fa74c030020 con 0x7fa75c102370 2024-07-16T10:07:08.223 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:08.219+0000 7fa758ff9640 1 --2- 172.21.15.156:0/1318517519 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7fa74806e0b0 0x7fa748070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:07:08.223 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:08.219+0000 7fa758ff9640 1 -- 172.21.15.156:0/1318517519 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7fa74c0adc50 con 0x7fa75c102370 2024-07-16T10:07:08.223 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:08.219+0000 7fa753fff640 1 --2- 172.21.15.156:0/1318517519 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7fa74806e0b0 0x7fa748070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:07:08.223 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:08.219+0000 7fa753fff640 1 --2- 172.21.15.156:0/1318517519 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7fa74806e0b0 0x7fa748070560 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fa75c10ec80 tx=0x7fa74400b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T10:07:08.224 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:08.223+0000 7fa758ff9640 1 -- 172.21.15.156:0/1318517519 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187716 (secure 0 0 0) 0x7fa74c076f20 con 0x7fa75c102370 2024-07-16T10:07:08.437 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:08.435+0000 7fa7621cd640 1 -- 172.21.15.156:0/1318517519 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 22, "format": "json"} v 0) v1 -- 0x7fa75c0fbb60 con 0x7fa75c102370 2024-07-16T10:07:08.437 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:08.436+0000 7fa758ff9640 1 -- 172.21.15.156:0/1318517519 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 22, "format": "json"}]=0 dumped fsmap epoch 22 v45) v1 ==== 107+0+4896 (secure 0 0 0) 0x7fa74c07a5e0 con 0x7fa75c102370 2024-07-16T10:07:08.438 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-07-16T10:07:08.438 INFO:teuthology.orchestra.run.smithi156.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34374,"name":"cephfs.smithi156.evwwaz","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6829/333092891","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":333092891},{"type":"v1","addr":"172.21.15.156:6829","nonce":333092891}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":22},{"gid":44269,"name":"cephfs.smithi176.wahbsi","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6827/3942496125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6826","nonce":3942496125},{"type":"v1","addr":"172.21.15.176:6827","nonce":3942496125}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-16T09:52:55.214975+0000","modified":"2024-07-16T10:01:54.812215+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0,1],"up":{"mds_0":14458,"mds_1":24261},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14458":{"gid":14458,"name":"cephfs.smithi156.rdibys","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.156:6827/1080176013","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":1080176013},{"type":"v1","addr":"172.21.15.156:6827","nonce":1080176013}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24261":{"gid":24261,"name":"cephfs.smithi176.dvvbud","rank":1,"incarnation":7,"state":"up:active","state_seq":4,"addr":"172.21.15.176:6825/1577019287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6824","nonce":1577019287},{"type":"v1","addr":"172.21.15.176:6825","nonce":1577019287}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":14458,"qdb_cluster":[24261,14458]},"id":1}]} 2024-07-16T10:07:08.438 INFO:teuthology.orchestra.run.smithi156.stderr:dumped fsmap epoch 22 2024-07-16T10:07:08.439 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:08.438+0000 7fa7621cd640 1 -- 172.21.15.156:0/1318517519 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7fa74806e0b0 msgr2=0x7fa748070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:07:08.440 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:08.438+0000 7fa7621cd640 1 --2- 172.21.15.156:0/1318517519 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7fa74806e0b0 0x7fa748070560 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fa75c10ec80 tx=0x7fa74400b040 comp rx=0 tx=0).stop 2024-07-16T10:07:08.440 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:08.438+0000 7fa7621cd640 1 -- 172.21.15.156:0/1318517519 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa75c102370 msgr2=0x7fa75c1133c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:07:08.440 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:08.438+0000 7fa7621cd640 1 --2- 172.21.15.156:0/1318517519 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa75c102370 0x7fa75c1133c0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fa74c0043d0 tx=0x7fa74c009da0 comp rx=0 tx=0).stop 2024-07-16T10:07:08.440 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:08.438+0000 7fa7621cd640 1 -- 172.21.15.156:0/1318517519 shutdown_connections 2024-07-16T10:07:08.440 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:08.438+0000 7fa7621cd640 1 --2- 172.21.15.156:0/1318517519 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7fa74806e0b0 0x7fa748070560 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:07:08.440 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:08.438+0000 7fa7621cd640 1 --2- 172.21.15.156:0/1318517519 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa75c102d10 0x7fa75c113900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:07:08.440 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:08.438+0000 7fa7621cd640 1 --2- 172.21.15.156:0/1318517519 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa75c102370 0x7fa75c1133c0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:07:08.440 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:08.438+0000 7fa7621cd640 1 -- 172.21.15.156:0/1318517519 >> 172.21.15.156:0/1318517519 conn(0x7fa75c0f7f00 msgr2=0x7fa75c0f8e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T10:07:08.440 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:08.438+0000 7fa7621cd640 1 -- 172.21.15.156:0/1318517519 shutdown_connections 2024-07-16T10:07:08.440 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:08.438+0000 7fa7621cd640 1 -- 172.21.15.156:0/1318517519 wait complete. 2024-07-16T10:07:08.691 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:07:08 smithi156 ceph-mon[93066]: from='client.? 172.21.15.156:0/1318517519' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 22, "format": "json"}]: dispatch 2024-07-16T10:07:08.843 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:07:08 smithi176 ceph-mon[81689]: from='client.? 172.21.15.156:0/1318517519' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 22, "format": "json"}]: dispatch 2024-07-16T10:07:09.086 DEBUG:tasks.fs:max_mds reduced in epoch 22 2024-07-16T10:07:09.086 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 22 2024-07-16T10:07:09.086 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 26c91c26-4358-11ef-bcad-c7b262605968 -- ceph fs dump --format=json 23 2024-07-16T10:07:09.351 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/26c91c26-4358-11ef-bcad-c7b262605968/mon.smithi156/config 2024-07-16T10:07:09.737 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:07:09 smithi156 ceph-mon[93066]: pgmap v318: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-07-16T10:07:09.843 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:07:09 smithi176 ceph-mon[81689]: pgmap v318: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-07-16T10:07:10.986 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:10.973+0000 7f6e09fd1640 1 -- 172.21.15.156:0/2786738091 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6e04102cf0 msgr2=0x7f6e040fc380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:07:10.986 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:10.973+0000 7f6e09fd1640 1 --2- 172.21.15.156:0/2786738091 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6e04102cf0 0x7f6e040fc380 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f6dec0048d0 tx=0x7f6dec009da0 comp rx=0 tx=0).stop 2024-07-16T10:07:10.987 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:10.973+0000 7f6e09fd1640 1 -- 172.21.15.156:0/2786738091 shutdown_connections 2024-07-16T10:07:10.987 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:10.973+0000 7f6e09fd1640 1 --2- 172.21.15.156:0/2786738091 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6e04102cf0 0x7f6e040fc380 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:07:10.987 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:10.973+0000 7f6e09fd1640 1 --2- 172.21.15.156:0/2786738091 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6e04102350 0x7f6e04102720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:07:10.987 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:10.973+0000 7f6e09fd1640 1 -- 172.21.15.156:0/2786738091 >> 172.21.15.156:0/2786738091 conn(0x7f6e040f7f00 msgr2=0x7f6e040fa310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T10:07:10.987 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:10.973+0000 7f6e09fd1640 1 -- 172.21.15.156:0/2786738091 shutdown_connections 2024-07-16T10:07:10.987 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:10.973+0000 7f6e09fd1640 1 -- 172.21.15.156:0/2786738091 wait complete. 2024-07-16T10:07:10.987 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:10.974+0000 7f6e09fd1640 1 Processor -- start 2024-07-16T10:07:10.987 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:10.974+0000 7f6e09fd1640 1 -- start start 2024-07-16T10:07:10.987 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:10.974+0000 7f6e09fd1640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6e04102350 0x7f6e041130d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:07:10.987 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:10.974+0000 7f6e09fd1640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6e04102cf0 0x7f6e04113610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:07:10.987 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:10.974+0000 7f6e09fd1640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e04113c10 con 0x7f6e04102350 2024-07-16T10:07:10.988 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:10.974+0000 7f6e09fd1640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e0410d050 con 0x7f6e04102cf0 2024-07-16T10:07:10.988 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:10.974+0000 7f6e037fe640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6e04102350 0x7f6e041130d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:07:10.988 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:10.974+0000 7f6e037fe640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6e04102350 0x7f6e041130d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:55140/0 (socket says 172.21.15.156:55140) 2024-07-16T10:07:10.988 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:10.974+0000 7f6e037fe640 1 -- 172.21.15.156:0/2363961895 learned_addr learned my addr 172.21.15.156:0/2363961895 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T10:07:10.988 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:10.974+0000 7f6dfbfff640 1 --2- 172.21.15.156:0/2363961895 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6e04102cf0 0x7f6e04113610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:07:10.988 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:10.974+0000 7f6e037fe640 1 -- 172.21.15.156:0/2363961895 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6e04102cf0 msgr2=0x7f6e04113610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:07:10.988 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:10.974+0000 7f6e037fe640 1 --2- 172.21.15.156:0/2363961895 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6e04102cf0 0x7f6e04113610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:07:10.988 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:10.974+0000 7f6e037fe640 1 -- 172.21.15.156:0/2363961895 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6dec005040 con 0x7f6e04102350 2024-07-16T10:07:10.988 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:10.975+0000 7f6e037fe640 1 --2- 172.21.15.156:0/2363961895 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6e04102350 0x7f6e041130d0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f6df4002a50 tx=0x7f6df4004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T10:07:10.988 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:10.975+0000 7f6e00ff9640 1 -- 172.21.15.156:0/2363961895 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6df4007c30 con 0x7f6e04102350 2024-07-16T10:07:10.988 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:10.975+0000 7f6e00ff9640 1 -- 172.21.15.156:0/2363961895 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6df400f040 con 0x7f6e04102350 2024-07-16T10:07:10.988 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:10.975+0000 7f6e00ff9640 1 -- 172.21.15.156:0/2363961895 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6df400db50 con 0x7f6e04102350 2024-07-16T10:07:10.989 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:10.975+0000 7f6e09fd1640 1 -- 172.21.15.156:0/2363961895 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e0410d330 con 0x7f6e04102350 2024-07-16T10:07:10.989 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:10.975+0000 7f6e09fd1640 1 -- 172.21.15.156:0/2363961895 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e0410d880 con 0x7f6e04102350 2024-07-16T10:07:10.989 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:10.976+0000 7f6e09fd1640 1 -- 172.21.15.156:0/2363961895 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e040fdac0 con 0x7f6e04102350 2024-07-16T10:07:10.989 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:10.976+0000 7f6e00ff9640 1 -- 172.21.15.156:0/2363961895 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 38) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f6df401c400 con 0x7f6e04102350 2024-07-16T10:07:10.989 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:10.977+0000 7f6e00ff9640 1 --2- 172.21.15.156:0/2363961895 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f6df006e0b0 0x7f6df0070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:07:10.989 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:10.977+0000 7f6e00ff9640 1 -- 172.21.15.156:0/2363961895 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f6df408d530 con 0x7f6e04102350 2024-07-16T10:07:10.989 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:10.977+0000 7f6dfbfff640 1 --2- 172.21.15.156:0/2363961895 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f6df006e0b0 0x7f6df0070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:07:10.989 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:10.977+0000 7f6dfbfff640 1 --2- 172.21.15.156:0/2363961895 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f6df006e0b0 0x7f6df0070560 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f6dec000c00 tx=0x7f6dec009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T10:07:10.989 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:10.981+0000 7f6e00ff9640 1 -- 172.21.15.156:0/2363961895 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187716 (secure 0 0 0) 0x7f6df40567a0 con 0x7f6e04102350 2024-07-16T10:07:11.194 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:11.193+0000 7f6e09fd1640 1 -- 172.21.15.156:0/2363961895 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 23, "format": "json"} v 0) v1 -- 0x7f6e041094e0 con 0x7f6e04102350 2024-07-16T10:07:11.194 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:11.193+0000 7f6e00ff9640 1 -- 172.21.15.156:0/2363961895 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 23, "format": "json"}]=0 dumped fsmap epoch 23 v45) v1 ==== 107+0+4892 (secure 0 0 0) 0x7f6df4059e60 con 0x7f6e04102350 2024-07-16T10:07:11.195 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-07-16T10:07:11.195 INFO:teuthology.orchestra.run.smithi156.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34374,"name":"cephfs.smithi156.evwwaz","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6829/333092891","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":333092891},{"type":"v1","addr":"172.21.15.156:6829","nonce":333092891}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":22},{"gid":44269,"name":"cephfs.smithi176.wahbsi","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6827/3942496125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6826","nonce":3942496125},{"type":"v1","addr":"172.21.15.176:6827","nonce":3942496125}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-16T09:52:55.214975+0000","modified":"2024-07-16T10:01:55.685998+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0,1],"up":{"mds_0":14458,"mds_1":24261},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14458":{"gid":14458,"name":"cephfs.smithi156.rdibys","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.156:6827/1080176013","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":1080176013},{"type":"v1","addr":"172.21.15.156:6827","nonce":1080176013}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24261":{"gid":24261,"name":"cephfs.smithi176.dvvbud","rank":1,"incarnation":7,"state":"up:stopping","state_seq":4,"addr":"172.21.15.176:6825/1577019287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6824","nonce":1577019287},{"type":"v1","addr":"172.21.15.176:6825","nonce":1577019287}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":14458,"qdb_cluster":[14458]},"id":1}]} 2024-07-16T10:07:11.195 INFO:teuthology.orchestra.run.smithi156.stderr:dumped fsmap epoch 23 2024-07-16T10:07:11.197 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:11.195+0000 7f6e09fd1640 1 -- 172.21.15.156:0/2363961895 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f6df006e0b0 msgr2=0x7f6df0070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:07:11.197 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:11.195+0000 7f6e09fd1640 1 --2- 172.21.15.156:0/2363961895 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f6df006e0b0 0x7f6df0070560 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f6dec000c00 tx=0x7f6dec009da0 comp rx=0 tx=0).stop 2024-07-16T10:07:11.197 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:11.195+0000 7f6e09fd1640 1 -- 172.21.15.156:0/2363961895 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6e04102350 msgr2=0x7f6e041130d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:07:11.197 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:11.195+0000 7f6e09fd1640 1 --2- 172.21.15.156:0/2363961895 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6e04102350 0x7f6e041130d0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f6df4002a50 tx=0x7f6df4004600 comp rx=0 tx=0).stop 2024-07-16T10:07:11.197 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:11.195+0000 7f6e09fd1640 1 -- 172.21.15.156:0/2363961895 shutdown_connections 2024-07-16T10:07:11.197 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:11.195+0000 7f6e09fd1640 1 --2- 172.21.15.156:0/2363961895 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f6df006e0b0 0x7f6df0070560 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:07:11.197 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:11.195+0000 7f6e09fd1640 1 --2- 172.21.15.156:0/2363961895 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6e04102cf0 0x7f6e04113610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:07:11.197 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:11.195+0000 7f6e09fd1640 1 --2- 172.21.15.156:0/2363961895 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6e04102350 0x7f6e041130d0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:07:11.197 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:11.195+0000 7f6e09fd1640 1 -- 172.21.15.156:0/2363961895 >> 172.21.15.156:0/2363961895 conn(0x7f6e040f7f00 msgr2=0x7f6e040f8ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T10:07:11.197 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:11.196+0000 7f6e09fd1640 1 -- 172.21.15.156:0/2363961895 shutdown_connections 2024-07-16T10:07:11.198 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:11.196+0000 7f6e09fd1640 1 -- 172.21.15.156:0/2363961895 wait complete. 2024-07-16T10:07:11.791 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:07:11 smithi156 ceph-mon[93066]: pgmap v319: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:07:11.791 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:07:11 smithi156 ceph-mon[93066]: from='client.? 172.21.15.156:0/2363961895' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 23, "format": "json"}]: dispatch 2024-07-16T10:07:11.816 DEBUG:tasks.fs:max_mds reduced in epoch 23 2024-07-16T10:07:11.817 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 23 2024-07-16T10:07:11.817 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 26c91c26-4358-11ef-bcad-c7b262605968 -- ceph fs dump --format=json 24 2024-07-16T10:07:11.843 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:07:11 smithi176 ceph-mon[81689]: pgmap v319: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:07:11.843 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:07:11 smithi176 ceph-mon[81689]: from='client.? 172.21.15.156:0/2363961895' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 23, "format": "json"}]: dispatch 2024-07-16T10:07:12.073 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/26c91c26-4358-11ef-bcad-c7b262605968/mon.smithi156/config 2024-07-16T10:07:13.669 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:13.656+0000 7f305c317640 1 -- 172.21.15.156:0/2351773167 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3054102cf0 msgr2=0x7f30540fc380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:07:13.669 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:13.656+0000 7f305c317640 1 --2- 172.21.15.156:0/2351773167 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3054102cf0 0x7f30540fc380 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f30440048d0 tx=0x7f3044009da0 comp rx=0 tx=0).stop 2024-07-16T10:07:13.669 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:13.656+0000 7f305c317640 1 -- 172.21.15.156:0/2351773167 shutdown_connections 2024-07-16T10:07:13.670 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:13.656+0000 7f305c317640 1 --2- 172.21.15.156:0/2351773167 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3054102cf0 0x7f30540fc380 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:07:13.670 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:13.656+0000 7f305c317640 1 --2- 172.21.15.156:0/2351773167 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3054102350 0x7f3054102720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:07:13.670 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:13.656+0000 7f305c317640 1 -- 172.21.15.156:0/2351773167 >> 172.21.15.156:0/2351773167 conn(0x7f30540f7f00 msgr2=0x7f30540fa310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T10:07:13.670 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:13.656+0000 7f305c317640 1 -- 172.21.15.156:0/2351773167 shutdown_connections 2024-07-16T10:07:13.670 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:13.656+0000 7f305c317640 1 -- 172.21.15.156:0/2351773167 wait complete. 2024-07-16T10:07:13.670 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:13.656+0000 7f305c317640 1 Processor -- start 2024-07-16T10:07:13.670 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:13.656+0000 7f305c317640 1 -- start start 2024-07-16T10:07:13.670 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:13.657+0000 7f305c317640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3054102350 0x7f30541152f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:07:13.670 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:13.657+0000 7f305c317640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3054102cf0 0x7f3054115830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:07:13.670 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:13.657+0000 7f305c317640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3054115e30 con 0x7f3054102cf0 2024-07-16T10:07:13.670 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:13.657+0000 7f305c317640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f305410f270 con 0x7f3054102350 2024-07-16T10:07:13.671 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:13.657+0000 7f30598b2640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3054102cf0 0x7f3054115830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:07:13.671 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:13.657+0000 7f30598b2640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3054102cf0 0x7f3054115830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:33202/0 (socket says 172.21.15.156:33202) 2024-07-16T10:07:13.671 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:13.657+0000 7f30598b2640 1 -- 172.21.15.156:0/1276883967 learned_addr learned my addr 172.21.15.156:0/1276883967 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T10:07:13.671 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:13.657+0000 7f305a0b3640 1 --2- 172.21.15.156:0/1276883967 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3054102350 0x7f30541152f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:07:13.671 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:13.657+0000 7f30598b2640 1 -- 172.21.15.156:0/1276883967 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3054102350 msgr2=0x7f30541152f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:07:13.671 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:13.657+0000 7f30598b2640 1 --2- 172.21.15.156:0/1276883967 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3054102350 0x7f30541152f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:07:13.671 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:13.657+0000 7f30598b2640 1 -- 172.21.15.156:0/1276883967 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3044005040 con 0x7f3054102cf0 2024-07-16T10:07:13.671 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:13.657+0000 7f30598b2640 1 --2- 172.21.15.156:0/1276883967 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3054102cf0 0x7f3054115830 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f30440043d0 tx=0x7f3044009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T10:07:13.671 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:13.657+0000 7f304affd640 1 -- 172.21.15.156:0/1276883967 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3044038070 con 0x7f3054102cf0 2024-07-16T10:07:13.671 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:13.657+0000 7f304affd640 1 -- 172.21.15.156:0/1276883967 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3044032970 con 0x7f3054102cf0 2024-07-16T10:07:13.671 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:13.657+0000 7f305c317640 1 -- 172.21.15.156:0/1276883967 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f305410f4f0 con 0x7f3054102cf0 2024-07-16T10:07:13.671 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:13.657+0000 7f304affd640 1 -- 172.21.15.156:0/1276883967 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f304403da60 con 0x7f3054102cf0 2024-07-16T10:07:13.672 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:13.657+0000 7f305c317640 1 -- 172.21.15.156:0/1276883967 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f305410fa30 con 0x7f3054102cf0 2024-07-16T10:07:13.672 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:13.658+0000 7f305c317640 1 -- 172.21.15.156:0/1276883967 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30540fdac0 con 0x7f3054102cf0 2024-07-16T10:07:13.672 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:13.659+0000 7f304affd640 1 -- 172.21.15.156:0/1276883967 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 38) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f3044030020 con 0x7f3054102cf0 2024-07-16T10:07:13.672 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:13.659+0000 7f304affd640 1 --2- 172.21.15.156:0/1276883967 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f304006e0b0 0x7f3040070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:07:13.672 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:13.659+0000 7f304affd640 1 -- 172.21.15.156:0/1276883967 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f30440adcf0 con 0x7f3054102cf0 2024-07-16T10:07:13.672 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:13.659+0000 7f305a0b3640 1 --2- 172.21.15.156:0/1276883967 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f304006e0b0 0x7f3040070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:07:13.672 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:13.660+0000 7f305a0b3640 1 --2- 172.21.15.156:0/1276883967 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f304006e0b0 0x7f3040070560 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f3050004ed0 tx=0x7f305000b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T10:07:13.672 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:13.664+0000 7f304affd640 1 -- 172.21.15.156:0/1276883967 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187716 (secure 0 0 0) 0x7f3044076fc0 con 0x7f3054102cf0 2024-07-16T10:07:13.843 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:07:13 smithi176 ceph-mon[81689]: pgmap v320: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:07:13.882 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:13.881+0000 7f305c317640 1 -- 172.21.15.156:0/1276883967 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 24, "format": "json"} v 0) v1 -- 0x7f30541094b0 con 0x7f3054102cf0 2024-07-16T10:07:13.882 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:07:13 smithi156 ceph-mon[93066]: pgmap v320: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:07:13.883 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:13.881+0000 7f304affd640 1 -- 172.21.15.156:0/1276883967 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 24, "format": "json"}]=0 dumped fsmap epoch 24 v45) v1 ==== 107+0+4893 (secure 0 0 0) 0x7f304407a680 con 0x7f3054102cf0 2024-07-16T10:07:13.883 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-07-16T10:07:13.883 INFO:teuthology.orchestra.run.smithi156.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34374,"name":"cephfs.smithi156.evwwaz","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6829/333092891","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":333092891},{"type":"v1","addr":"172.21.15.156:6829","nonce":333092891}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":22},{"gid":44269,"name":"cephfs.smithi176.wahbsi","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6827/3942496125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6826","nonce":3942496125},{"type":"v1","addr":"172.21.15.176:6827","nonce":3942496125}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-16T09:52:55.214975+0000","modified":"2024-07-16T10:02:01.577797+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0,1],"up":{"mds_0":14458,"mds_1":24261},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14458":{"gid":14458,"name":"cephfs.smithi156.rdibys","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.156:6827/1080176013","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":1080176013},{"type":"v1","addr":"172.21.15.156:6827","nonce":1080176013}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24261":{"gid":24261,"name":"cephfs.smithi176.dvvbud","rank":1,"incarnation":7,"state":"up:stopping","state_seq":4,"addr":"172.21.15.176:6825/1577019287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6824","nonce":1577019287},{"type":"v1","addr":"172.21.15.176:6825","nonce":1577019287}]},"join_fscid":1,"export_targets":[0],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":14458,"qdb_cluster":[14458]},"id":1}]} 2024-07-16T10:07:13.883 INFO:teuthology.orchestra.run.smithi156.stderr:dumped fsmap epoch 24 2024-07-16T10:07:13.885 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:13.883+0000 7f305c317640 1 -- 172.21.15.156:0/1276883967 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f304006e0b0 msgr2=0x7f3040070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:07:13.885 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:13.883+0000 7f305c317640 1 --2- 172.21.15.156:0/1276883967 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f304006e0b0 0x7f3040070560 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f3050004ed0 tx=0x7f305000b040 comp rx=0 tx=0).stop 2024-07-16T10:07:13.885 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:13.883+0000 7f305c317640 1 -- 172.21.15.156:0/1276883967 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3054102cf0 msgr2=0x7f3054115830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:07:13.885 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:13.883+0000 7f305c317640 1 --2- 172.21.15.156:0/1276883967 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3054102cf0 0x7f3054115830 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f30440043d0 tx=0x7f3044009da0 comp rx=0 tx=0).stop 2024-07-16T10:07:13.885 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:13.884+0000 7f305c317640 1 -- 172.21.15.156:0/1276883967 shutdown_connections 2024-07-16T10:07:13.885 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:13.884+0000 7f305c317640 1 --2- 172.21.15.156:0/1276883967 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f304006e0b0 0x7f3040070560 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:07:13.885 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:13.884+0000 7f305c317640 1 --2- 172.21.15.156:0/1276883967 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3054102cf0 0x7f3054115830 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:07:13.885 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:13.884+0000 7f305c317640 1 --2- 172.21.15.156:0/1276883967 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3054102350 0x7f30541152f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:07:13.886 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:13.884+0000 7f305c317640 1 -- 172.21.15.156:0/1276883967 >> 172.21.15.156:0/1276883967 conn(0x7f30540f7f00 msgr2=0x7f30540f8e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T10:07:13.886 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:13.884+0000 7f305c317640 1 -- 172.21.15.156:0/1276883967 shutdown_connections 2024-07-16T10:07:13.886 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:13.884+0000 7f305c317640 1 -- 172.21.15.156:0/1276883967 wait complete. 2024-07-16T10:07:14.525 DEBUG:tasks.fs:max_mds reduced in epoch 24 2024-07-16T10:07:14.525 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 24 2024-07-16T10:07:14.525 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 26c91c26-4358-11ef-bcad-c7b262605968 -- ceph fs dump --format=json 25 2024-07-16T10:07:14.783 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/26c91c26-4358-11ef-bcad-c7b262605968/mon.smithi156/config 2024-07-16T10:07:14.843 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:07:14 smithi176 ceph-mon[81689]: from='client.? 172.21.15.156:0/1276883967' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 24, "format": "json"}]: dispatch 2024-07-16T10:07:14.882 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:07:14 smithi156 ceph-mon[93066]: from='client.? 172.21.15.156:0/1276883967' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 24, "format": "json"}]: dispatch 2024-07-16T10:07:15.822 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:07:15 smithi156 ceph-mon[93066]: pgmap v321: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-07-16T10:07:15.843 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:07:15 smithi176 ceph-mon[81689]: pgmap v321: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-07-16T10:07:16.273 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:16.270+0000 7fe694bba640 1 -- 172.21.15.156:0/3955529393 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe690101c00 msgr2=0x7fe6900fb290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:07:16.273 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:16.270+0000 7fe694bba640 1 --2- 172.21.15.156:0/3955529393 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe690101c00 0x7fe6900fb290 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fe6780048d0 tx=0x7fe678009da0 comp rx=0 tx=0).stop 2024-07-16T10:07:16.274 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:16.270+0000 7fe694bba640 1 -- 172.21.15.156:0/3955529393 shutdown_connections 2024-07-16T10:07:16.274 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:16.270+0000 7fe694bba640 1 --2- 172.21.15.156:0/3955529393 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe690101c00 0x7fe6900fb290 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:07:16.274 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:16.270+0000 7fe694bba640 1 --2- 172.21.15.156:0/3955529393 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe690101260 0x7fe690101630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:07:16.274 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:16.270+0000 7fe694bba640 1 -- 172.21.15.156:0/3955529393 >> 172.21.15.156:0/3955529393 conn(0x7fe6900f6e10 msgr2=0x7fe6900f9220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T10:07:16.274 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:16.270+0000 7fe694bba640 1 -- 172.21.15.156:0/3955529393 shutdown_connections 2024-07-16T10:07:16.274 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:16.270+0000 7fe694bba640 1 -- 172.21.15.156:0/3955529393 wait complete. 2024-07-16T10:07:16.274 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:16.271+0000 7fe694bba640 1 Processor -- start 2024-07-16T10:07:16.275 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:16.271+0000 7fe694bba640 1 -- start start 2024-07-16T10:07:16.275 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:16.271+0000 7fe694bba640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe690101260 0x7fe690113180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:07:16.275 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:16.271+0000 7fe694bba640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe690101c00 0x7fe6901136c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:07:16.275 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:16.271+0000 7fe694bba640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe690113cc0 con 0x7fe690101c00 2024-07-16T10:07:16.275 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:16.271+0000 7fe694bba640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe69010d100 con 0x7fe690101260 2024-07-16T10:07:16.275 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:16.271+0000 7fe687fff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe690101c00 0x7fe6901136c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:07:16.275 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:16.271+0000 7fe687fff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe690101c00 0x7fe6901136c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:33228/0 (socket says 172.21.15.156:33228) 2024-07-16T10:07:16.275 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:16.271+0000 7fe687fff640 1 -- 172.21.15.156:0/2920586467 learned_addr learned my addr 172.21.15.156:0/2920586467 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T10:07:16.275 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:16.272+0000 7fe68e59c640 1 --2- 172.21.15.156:0/2920586467 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe690101260 0x7fe690113180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:07:16.275 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:16.272+0000 7fe687fff640 1 -- 172.21.15.156:0/2920586467 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe690101260 msgr2=0x7fe690113180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:07:16.275 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:16.272+0000 7fe687fff640 1 --2- 172.21.15.156:0/2920586467 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe690101260 0x7fe690113180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:07:16.276 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:16.272+0000 7fe687fff640 1 -- 172.21.15.156:0/2920586467 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe678005040 con 0x7fe690101c00 2024-07-16T10:07:16.276 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:16.272+0000 7fe68e59c640 1 --2- 172.21.15.156:0/2920586467 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe690101260 0x7fe690113180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-16T10:07:16.276 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:16.272+0000 7fe687fff640 1 --2- 172.21.15.156:0/2920586467 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe690101c00 0x7fe6901136c0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fe678003080 tx=0x7fe678009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T10:07:16.276 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:16.272+0000 7fe686ffd640 1 -- 172.21.15.156:0/2920586467 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe678038070 con 0x7fe690101c00 2024-07-16T10:07:16.276 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:16.272+0000 7fe686ffd640 1 -- 172.21.15.156:0/2920586467 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe678032970 con 0x7fe690101c00 2024-07-16T10:07:16.276 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:16.272+0000 7fe694bba640 1 -- 172.21.15.156:0/2920586467 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe69010d380 con 0x7fe690101c00 2024-07-16T10:07:16.276 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:16.272+0000 7fe686ffd640 1 -- 172.21.15.156:0/2920586467 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe67803d960 con 0x7fe690101c00 2024-07-16T10:07:16.276 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:16.272+0000 7fe694bba640 1 -- 172.21.15.156:0/2920586467 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe69010d840 con 0x7fe690101c00 2024-07-16T10:07:16.277 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:16.273+0000 7fe694bba640 1 -- 172.21.15.156:0/2920586467 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6900fc9d0 con 0x7fe690101c00 2024-07-16T10:07:16.277 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:16.274+0000 7fe686ffd640 1 -- 172.21.15.156:0/2920586467 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 38) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fe678030020 con 0x7fe690101c00 2024-07-16T10:07:16.277 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:16.274+0000 7fe686ffd640 1 --2- 172.21.15.156:0/2920586467 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7fe67c06e0b0 0x7fe67c070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:07:16.277 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:16.274+0000 7fe68e59c640 1 --2- 172.21.15.156:0/2920586467 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7fe67c06e0b0 0x7fe67c070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:07:16.277 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:16.274+0000 7fe686ffd640 1 -- 172.21.15.156:0/2920586467 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7fe6780adc50 con 0x7fe690101c00 2024-07-16T10:07:16.277 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:16.275+0000 7fe68e59c640 1 --2- 172.21.15.156:0/2920586467 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7fe67c06e0b0 0x7fe67c070560 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fe6800049d0 tx=0x7fe68000b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T10:07:16.280 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:16.279+0000 7fe686ffd640 1 -- 172.21.15.156:0/2920586467 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187716 (secure 0 0 0) 0x7fe678076f20 con 0x7fe690101c00 2024-07-16T10:07:16.495 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:16.494+0000 7fe694bba640 1 -- 172.21.15.156:0/2920586467 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 25, "format": "json"} v 0) v1 -- 0x7fe6901083c0 con 0x7fe690101c00 2024-07-16T10:07:16.496 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:16.494+0000 7fe686ffd640 1 -- 172.21.15.156:0/2920586467 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 25, "format": "json"}]=0 dumped fsmap epoch 25 v45) v1 ==== 107+0+4094 (secure 0 0 0) 0x7fe67807a5e0 con 0x7fe690101c00 2024-07-16T10:07:16.496 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-07-16T10:07:16.496 INFO:teuthology.orchestra.run.smithi156.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34374,"name":"cephfs.smithi156.evwwaz","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6829/333092891","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":333092891},{"type":"v1","addr":"172.21.15.156:6829","nonce":333092891}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":22},{"gid":44269,"name":"cephfs.smithi176.wahbsi","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6827/3942496125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6826","nonce":3942496125},{"type":"v1","addr":"172.21.15.176:6827","nonce":3942496125}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-16T09:52:55.214975+0000","modified":"2024-07-16T10:02:26.579199+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":14458},"failed":[],"damaged":[],"stopped":[1],"info":{"gid_14458":{"gid":14458,"name":"cephfs.smithi156.rdibys","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.156:6827/1080176013","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":1080176013},{"type":"v1","addr":"172.21.15.156:6827","nonce":1080176013}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":14458,"qdb_cluster":[14458]},"id":1}]} 2024-07-16T10:07:16.496 INFO:teuthology.orchestra.run.smithi156.stderr:dumped fsmap epoch 25 2024-07-16T10:07:16.498 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:16.497+0000 7fe694bba640 1 -- 172.21.15.156:0/2920586467 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7fe67c06e0b0 msgr2=0x7fe67c070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:07:16.498 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:16.497+0000 7fe694bba640 1 --2- 172.21.15.156:0/2920586467 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7fe67c06e0b0 0x7fe67c070560 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fe6800049d0 tx=0x7fe68000b040 comp rx=0 tx=0).stop 2024-07-16T10:07:16.498 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:16.497+0000 7fe694bba640 1 -- 172.21.15.156:0/2920586467 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe690101c00 msgr2=0x7fe6901136c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:07:16.498 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:16.497+0000 7fe694bba640 1 --2- 172.21.15.156:0/2920586467 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe690101c00 0x7fe6901136c0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fe678003080 tx=0x7fe678009da0 comp rx=0 tx=0).stop 2024-07-16T10:07:16.498 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:16.497+0000 7fe694bba640 1 -- 172.21.15.156:0/2920586467 shutdown_connections 2024-07-16T10:07:16.499 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:16.497+0000 7fe694bba640 1 --2- 172.21.15.156:0/2920586467 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7fe67c06e0b0 0x7fe67c070560 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:07:16.499 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:16.497+0000 7fe694bba640 1 --2- 172.21.15.156:0/2920586467 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe690101c00 0x7fe6901136c0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:07:16.499 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:16.497+0000 7fe694bba640 1 --2- 172.21.15.156:0/2920586467 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe690101260 0x7fe690113180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:07:16.499 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:16.497+0000 7fe694bba640 1 -- 172.21.15.156:0/2920586467 >> 172.21.15.156:0/2920586467 conn(0x7fe6900f6e10 msgr2=0x7fe6900f8520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T10:07:16.499 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:16.497+0000 7fe694bba640 1 -- 172.21.15.156:0/2920586467 shutdown_connections 2024-07-16T10:07:16.499 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:16.497+0000 7fe694bba640 1 -- 172.21.15.156:0/2920586467 wait complete. 2024-07-16T10:07:16.804 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:07:16 smithi156 ceph-mon[93066]: from='client.? 172.21.15.156:0/2920586467' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 25, "format": "json"}]: dispatch 2024-07-16T10:07:16.843 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:07:16 smithi176 ceph-mon[81689]: from='client.? 172.21.15.156:0/2920586467' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 25, "format": "json"}]: dispatch 2024-07-16T10:07:17.145 DEBUG:tasks.fs:max_mds reduced in epoch 25 2024-07-16T10:07:17.145 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 25 2024-07-16T10:07:17.145 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 26c91c26-4358-11ef-bcad-c7b262605968 -- ceph fs dump --format=json 26 2024-07-16T10:07:17.398 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/26c91c26-4358-11ef-bcad-c7b262605968/mon.smithi156/config 2024-07-16T10:07:17.843 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:07:17 smithi176 ceph-mon[81689]: pgmap v322: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:07:17.849 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:07:17 smithi156 ceph-mon[93066]: pgmap v322: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:07:18.740 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:07:18 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-16T10:07:18.843 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:07:18 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-16T10:07:18.974 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:18.967+0000 7f15fe1cd640 1 -- 172.21.15.156:0/4017113405 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f15f8101250 msgr2=0x7f15f8101620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:07:18.974 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:18.967+0000 7f15fe1cd640 1 --2- 172.21.15.156:0/4017113405 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f15f8101250 0x7f15f8101620 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f15e80048d0 tx=0x7f15e8009da0 comp rx=0 tx=0).stop 2024-07-16T10:07:18.974 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:18.967+0000 7f15fe1cd640 1 -- 172.21.15.156:0/4017113405 shutdown_connections 2024-07-16T10:07:18.974 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:18.967+0000 7f15fe1cd640 1 --2- 172.21.15.156:0/4017113405 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f15f8101bf0 0x7f15f80fb280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:07:18.974 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:18.967+0000 7f15fe1cd640 1 --2- 172.21.15.156:0/4017113405 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f15f8101250 0x7f15f8101620 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:07:18.974 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:18.967+0000 7f15fe1cd640 1 -- 172.21.15.156:0/4017113405 >> 172.21.15.156:0/4017113405 conn(0x7f15f80f6e00 msgr2=0x7f15f80f9210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T10:07:18.974 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:18.967+0000 7f15fe1cd640 1 -- 172.21.15.156:0/4017113405 shutdown_connections 2024-07-16T10:07:18.974 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:18.967+0000 7f15fe1cd640 1 -- 172.21.15.156:0/4017113405 wait complete. 2024-07-16T10:07:18.974 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:18.968+0000 7f15fe1cd640 1 Processor -- start 2024-07-16T10:07:18.975 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:18.968+0000 7f15fe1cd640 1 -- start start 2024-07-16T10:07:18.975 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:18.968+0000 7f15fe1cd640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f15f8101250 0x7f15f8111fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:07:18.975 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:18.968+0000 7f15fe1cd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f15f8101bf0 0x7f15f81124f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:07:18.975 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:18.968+0000 7f15fe1cd640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15f8112af0 con 0x7f15f8101bf0 2024-07-16T10:07:18.975 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:18.968+0000 7f15fe1cd640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15f810bf30 con 0x7f15f8101250 2024-07-16T10:07:18.975 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:18.968+0000 7f15effff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f15f8101bf0 0x7f15f81124f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:07:18.975 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:18.969+0000 7f15effff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f15f8101bf0 0x7f15f81124f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:33238/0 (socket says 172.21.15.156:33238) 2024-07-16T10:07:18.975 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:18.969+0000 7f15effff640 1 -- 172.21.15.156:0/1361868476 learned_addr learned my addr 172.21.15.156:0/1361868476 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T10:07:18.975 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:18.969+0000 7f15f77fe640 1 --2- 172.21.15.156:0/1361868476 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f15f8101250 0x7f15f8111fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:07:18.975 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:18.969+0000 7f15effff640 1 -- 172.21.15.156:0/1361868476 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f15f8101250 msgr2=0x7f15f8111fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:07:18.975 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:18.969+0000 7f15effff640 1 --2- 172.21.15.156:0/1361868476 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f15f8101250 0x7f15f8111fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:07:18.975 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:18.969+0000 7f15effff640 1 -- 172.21.15.156:0/1361868476 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15e8005040 con 0x7f15f8101bf0 2024-07-16T10:07:18.975 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:18.969+0000 7f15effff640 1 --2- 172.21.15.156:0/1361868476 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f15f8101bf0 0x7f15f81124f0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f15e000c790 tx=0x7f15e00043c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T10:07:18.976 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:18.969+0000 7f15f4ff9640 1 -- 172.21.15.156:0/1361868476 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f15e000a820 con 0x7f15f8101bf0 2024-07-16T10:07:18.976 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:18.969+0000 7f15fe1cd640 1 -- 172.21.15.156:0/1361868476 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15f810c210 con 0x7f15f8101bf0 2024-07-16T10:07:18.976 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:18.969+0000 7f15f4ff9640 1 -- 172.21.15.156:0/1361868476 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f15e000ae50 con 0x7f15f8101bf0 2024-07-16T10:07:18.976 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:18.969+0000 7f15f4ff9640 1 -- 172.21.15.156:0/1361868476 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f15e00115e0 con 0x7f15f8101bf0 2024-07-16T10:07:18.976 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:18.969+0000 7f15fe1cd640 1 -- 172.21.15.156:0/1361868476 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15f810c7b0 con 0x7f15f8101bf0 2024-07-16T10:07:18.976 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:18.970+0000 7f15fe1cd640 1 -- 172.21.15.156:0/1361868476 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15f80fc9c0 con 0x7f15f8101bf0 2024-07-16T10:07:18.976 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:18.971+0000 7f15f4ff9640 1 -- 172.21.15.156:0/1361868476 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 38) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f15e0012400 con 0x7f15f8101bf0 2024-07-16T10:07:18.976 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:18.971+0000 7f15f4ff9640 1 --2- 172.21.15.156:0/1361868476 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f15e406e0b0 0x7f15e4070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:07:18.976 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:18.971+0000 7f15f4ff9640 1 -- 172.21.15.156:0/1361868476 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f15e008dd90 con 0x7f15f8101bf0 2024-07-16T10:07:18.976 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:18.971+0000 7f15f77fe640 1 --2- 172.21.15.156:0/1361868476 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f15e406e0b0 0x7f15e4070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:07:18.976 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:18.972+0000 7f15f77fe640 1 --2- 172.21.15.156:0/1361868476 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f15e406e0b0 0x7f15e4070560 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f15e80047f0 tx=0x7f15e8009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T10:07:18.977 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:18.976+0000 7f15f4ff9640 1 -- 172.21.15.156:0/1361868476 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187716 (secure 0 0 0) 0x7f15e00570e0 con 0x7f15f8101bf0 2024-07-16T10:07:19.192 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:19.191+0000 7f15fe1cd640 1 -- 172.21.15.156:0/1361868476 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 26, "format": "json"} v 0) v1 -- 0x7f15f81083e0 con 0x7f15f8101bf0 2024-07-16T10:07:19.193 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:19.191+0000 7f15f4ff9640 1 -- 172.21.15.156:0/1361868476 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 26, "format": "json"}]=0 dumped fsmap epoch 26 v45) v1 ==== 107+0+4876 (secure 0 0 0) 0x7f15e005a7a0 con 0x7f15f8101bf0 2024-07-16T10:07:19.193 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-07-16T10:07:19.193 INFO:teuthology.orchestra.run.smithi156.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34374,"name":"cephfs.smithi156.evwwaz","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6829/333092891","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":333092891},{"type":"v1","addr":"172.21.15.156:6829","nonce":333092891}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":22},{"gid":44269,"name":"cephfs.smithi176.wahbsi","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6827/3942496125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6826","nonce":3942496125},{"type":"v1","addr":"172.21.15.176:6827","nonce":3942496125}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":22},{"gid":44301,"name":"cephfs.smithi176.dvvbud","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6825/3749220343","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6824","nonce":3749220343},{"type":"v1","addr":"172.21.15.176:6825","nonce":3749220343}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-16T09:52:55.214975+0000","modified":"2024-07-16T10:02:26.579199+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":14458},"failed":[],"damaged":[],"stopped":[1],"info":{"gid_14458":{"gid":14458,"name":"cephfs.smithi156.rdibys","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.156:6827/1080176013","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":1080176013},{"type":"v1","addr":"172.21.15.156:6827","nonce":1080176013}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":14458,"qdb_cluster":[14458]},"id":1}]} 2024-07-16T10:07:19.193 INFO:teuthology.orchestra.run.smithi156.stderr:dumped fsmap epoch 26 2024-07-16T10:07:19.195 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:19.193+0000 7f15fe1cd640 1 -- 172.21.15.156:0/1361868476 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f15e406e0b0 msgr2=0x7f15e4070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:07:19.195 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:19.193+0000 7f15fe1cd640 1 --2- 172.21.15.156:0/1361868476 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f15e406e0b0 0x7f15e4070560 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f15e80047f0 tx=0x7f15e8009da0 comp rx=0 tx=0).stop 2024-07-16T10:07:19.195 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:19.193+0000 7f15fe1cd640 1 -- 172.21.15.156:0/1361868476 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f15f8101bf0 msgr2=0x7f15f81124f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:07:19.195 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:19.193+0000 7f15fe1cd640 1 --2- 172.21.15.156:0/1361868476 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f15f8101bf0 0x7f15f81124f0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f15e000c790 tx=0x7f15e00043c0 comp rx=0 tx=0).stop 2024-07-16T10:07:19.195 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:19.194+0000 7f15fe1cd640 1 -- 172.21.15.156:0/1361868476 shutdown_connections 2024-07-16T10:07:19.195 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:19.194+0000 7f15fe1cd640 1 --2- 172.21.15.156:0/1361868476 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f15e406e0b0 0x7f15e4070560 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:07:19.196 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:19.194+0000 7f15fe1cd640 1 --2- 172.21.15.156:0/1361868476 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f15f8101bf0 0x7f15f81124f0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:07:19.196 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:19.194+0000 7f15fe1cd640 1 --2- 172.21.15.156:0/1361868476 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f15f8101250 0x7f15f8111fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:07:19.196 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:19.194+0000 7f15fe1cd640 1 -- 172.21.15.156:0/1361868476 >> 172.21.15.156:0/1361868476 conn(0x7f15f80f6e00 msgr2=0x7f15f80f7dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T10:07:19.196 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:19.194+0000 7f15fe1cd640 1 -- 172.21.15.156:0/1361868476 shutdown_connections 2024-07-16T10:07:19.196 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:19.194+0000 7f15fe1cd640 1 -- 172.21.15.156:0/1361868476 wait complete. 2024-07-16T10:07:19.761 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:07:19 smithi156 ceph-mon[93066]: pgmap v323: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-07-16T10:07:19.761 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:07:19 smithi156 ceph-mon[93066]: from='client.? 172.21.15.156:0/1361868476' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 26, "format": "json"}]: dispatch 2024-07-16T10:07:19.787 DEBUG:tasks.fs:max_mds reduced in epoch 26 2024-07-16T10:07:19.787 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 26 2024-07-16T10:07:19.788 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 26c91c26-4358-11ef-bcad-c7b262605968 -- ceph fs dump --format=json 27 2024-07-16T10:07:19.843 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:07:19 smithi176 ceph-mon[81689]: pgmap v323: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-07-16T10:07:19.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:07:19 smithi176 ceph-mon[81689]: from='client.? 172.21.15.156:0/1361868476' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 26, "format": "json"}]: dispatch 2024-07-16T10:07:20.046 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/26c91c26-4358-11ef-bcad-c7b262605968/mon.smithi156/config 2024-07-16T10:07:21.555 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:21.551+0000 7fdb0f7a0640 1 -- 172.21.15.156:0/443718836 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdb080fb620 msgr2=0x7fdb080fb9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:07:21.555 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:21.551+0000 7fdb0f7a0640 1 --2- 172.21.15.156:0/443718836 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdb080fb620 0x7fdb080fb9f0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fdaf80048d0 tx=0x7fdaf8009da0 comp rx=0 tx=0).stop 2024-07-16T10:07:21.555 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:21.551+0000 7fdb0f7a0640 1 -- 172.21.15.156:0/443718836 shutdown_connections 2024-07-16T10:07:21.555 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:21.551+0000 7fdb0f7a0640 1 --2- 172.21.15.156:0/443718836 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fdb080fbf30 0x7fdb080fce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:07:21.555 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:21.551+0000 7fdb0f7a0640 1 --2- 172.21.15.156:0/443718836 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdb080fb620 0x7fdb080fb9f0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:07:21.555 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:21.551+0000 7fdb0f7a0640 1 -- 172.21.15.156:0/443718836 >> 172.21.15.156:0/443718836 conn(0x7fdb080721c0 msgr2=0x7fdb080725c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T10:07:21.555 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:21.552+0000 7fdb0f7a0640 1 -- 172.21.15.156:0/443718836 shutdown_connections 2024-07-16T10:07:21.555 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:21.552+0000 7fdb0f7a0640 1 -- 172.21.15.156:0/443718836 wait complete. 2024-07-16T10:07:21.556 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:21.552+0000 7fdb0f7a0640 1 Processor -- start 2024-07-16T10:07:21.556 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:21.552+0000 7fdb0f7a0640 1 -- start start 2024-07-16T10:07:21.556 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:21.552+0000 7fdb0f7a0640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdb080fb620 0x7fdb08113100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:07:21.556 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:21.552+0000 7fdb0f7a0640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fdb080fbf30 0x7fdb08113640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:07:21.556 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:21.552+0000 7fdb0f7a0640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb08113c40 con 0x7fdb080fb620 2024-07-16T10:07:21.556 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:21.552+0000 7fdb0f7a0640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb0810d080 con 0x7fdb080fbf30 2024-07-16T10:07:21.556 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:21.553+0000 7fdb0d53c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdb080fb620 0x7fdb08113100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:07:21.556 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:21.553+0000 7fdb0d53c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdb080fb620 0x7fdb08113100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:33256/0 (socket says 172.21.15.156:33256) 2024-07-16T10:07:21.556 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:21.553+0000 7fdb0d53c640 1 -- 172.21.15.156:0/2613307044 learned_addr learned my addr 172.21.15.156:0/2613307044 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T10:07:21.556 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:21.553+0000 7fdb0cd3b640 1 --2- 172.21.15.156:0/2613307044 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fdb080fbf30 0x7fdb08113640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:07:21.556 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:21.553+0000 7fdb0d53c640 1 -- 172.21.15.156:0/2613307044 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fdb080fbf30 msgr2=0x7fdb08113640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:07:21.556 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:21.553+0000 7fdb0d53c640 1 --2- 172.21.15.156:0/2613307044 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fdb080fbf30 0x7fdb08113640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:07:21.556 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:21.553+0000 7fdb0d53c640 1 -- 172.21.15.156:0/2613307044 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdaf8005040 con 0x7fdb080fb620 2024-07-16T10:07:21.557 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:21.553+0000 7fdb0d53c640 1 --2- 172.21.15.156:0/2613307044 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdb080fb620 0x7fdb08113100 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fdaf80043d0 tx=0x7fdaf8009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T10:07:21.557 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:21.553+0000 7fdafdffb640 1 -- 172.21.15.156:0/2613307044 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdaf8038070 con 0x7fdb080fb620 2024-07-16T10:07:21.557 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:21.553+0000 7fdafdffb640 1 -- 172.21.15.156:0/2613307044 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdaf802abd0 con 0x7fdb080fb620 2024-07-16T10:07:21.557 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:21.553+0000 7fdb0f7a0640 1 -- 172.21.15.156:0/2613307044 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb0810d300 con 0x7fdb080fb620 2024-07-16T10:07:21.557 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:21.553+0000 7fdafdffb640 1 -- 172.21.15.156:0/2613307044 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdaf80336f0 con 0x7fdb080fb620 2024-07-16T10:07:21.557 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:21.554+0000 7fdb0f7a0640 1 -- 172.21.15.156:0/2613307044 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb0810d840 con 0x7fdb080fb620 2024-07-16T10:07:21.558 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:21.555+0000 7fdb0f7a0640 1 -- 172.21.15.156:0/2613307044 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb080fc9d0 con 0x7fdb080fb620 2024-07-16T10:07:21.558 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:21.555+0000 7fdafdffb640 1 -- 172.21.15.156:0/2613307044 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 38) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fdaf8030020 con 0x7fdb080fb620 2024-07-16T10:07:21.558 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:21.556+0000 7fdafdffb640 1 --2- 172.21.15.156:0/2613307044 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7fdaf406e0b0 0x7fdaf4070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:07:21.558 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:21.556+0000 7fdafdffb640 1 -- 172.21.15.156:0/2613307044 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7fdaf80ad8e0 con 0x7fdb080fb620 2024-07-16T10:07:21.558 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:21.556+0000 7fdb0cd3b640 1 --2- 172.21.15.156:0/2613307044 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7fdaf406e0b0 0x7fdaf4070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:07:21.558 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:21.556+0000 7fdb0cd3b640 1 --2- 172.21.15.156:0/2613307044 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7fdaf406e0b0 0x7fdaf4070560 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fdb0810e9c0 tx=0x7fdb0400b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T10:07:21.561 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:21.560+0000 7fdafdffb640 1 -- 172.21.15.156:0/2613307044 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187716 (secure 0 0 0) 0x7fdaf8076c60 con 0x7fdb080fb620 2024-07-16T10:07:21.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:07:21 smithi156 ceph-mon[93066]: pgmap v324: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:07:21.778 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:21.776+0000 7fdb0f7a0640 1 -- 172.21.15.156:0/2613307044 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 27, "format": "json"} v 0) v1 -- 0x7fdb0804b530 con 0x7fdb080fb620 2024-07-16T10:07:21.778 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:21.777+0000 7fdafdffb640 1 -- 172.21.15.156:0/2613307044 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 27, "format": "json"}]=0 dumped fsmap epoch 27 v45) v1 ==== 107+0+4076 (secure 0 0 0) 0x7fdaf807a320 con 0x7fdb080fb620 2024-07-16T10:07:21.779 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-07-16T10:07:21.779 INFO:teuthology.orchestra.run.smithi156.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34374,"name":"cephfs.smithi156.evwwaz","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6829/333092891","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":333092891},{"type":"v1","addr":"172.21.15.156:6829","nonce":333092891}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":22},{"gid":44269,"name":"cephfs.smithi176.wahbsi","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6827/3942496125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6826","nonce":3942496125},{"type":"v1","addr":"172.21.15.176:6827","nonce":3942496125}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":22},{"gid":44301,"name":"cephfs.smithi176.dvvbud","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6825/3749220343","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6824","nonce":3749220343},{"type":"v1","addr":"172.21.15.176:6825","nonce":3749220343}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-16T09:52:55.214975+0000","modified":"2024-07-16T10:02:40.286225+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":81,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[1],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-07-16T10:07:21.779 INFO:teuthology.orchestra.run.smithi156.stderr:dumped fsmap epoch 27 2024-07-16T10:07:21.780 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:21.779+0000 7fdb0f7a0640 1 -- 172.21.15.156:0/2613307044 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7fdaf406e0b0 msgr2=0x7fdaf4070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:07:21.781 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:21.779+0000 7fdb0f7a0640 1 --2- 172.21.15.156:0/2613307044 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7fdaf406e0b0 0x7fdaf4070560 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fdb0810e9c0 tx=0x7fdb0400b040 comp rx=0 tx=0).stop 2024-07-16T10:07:21.781 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:21.779+0000 7fdb0f7a0640 1 -- 172.21.15.156:0/2613307044 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdb080fb620 msgr2=0x7fdb08113100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:07:21.781 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:21.779+0000 7fdb0f7a0640 1 --2- 172.21.15.156:0/2613307044 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdb080fb620 0x7fdb08113100 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fdaf80043d0 tx=0x7fdaf8009da0 comp rx=0 tx=0).stop 2024-07-16T10:07:21.781 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:21.779+0000 7fdb0f7a0640 1 -- 172.21.15.156:0/2613307044 shutdown_connections 2024-07-16T10:07:21.781 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:21.779+0000 7fdb0f7a0640 1 --2- 172.21.15.156:0/2613307044 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7fdaf406e0b0 0x7fdaf4070560 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:07:21.781 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:21.779+0000 7fdb0f7a0640 1 --2- 172.21.15.156:0/2613307044 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fdb080fbf30 0x7fdb08113640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:07:21.781 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:21.779+0000 7fdb0f7a0640 1 --2- 172.21.15.156:0/2613307044 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdb080fb620 0x7fdb08113100 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:07:21.781 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:21.779+0000 7fdb0f7a0640 1 -- 172.21.15.156:0/2613307044 >> 172.21.15.156:0/2613307044 conn(0x7fdb080721c0 msgr2=0x7fdb080fff50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T10:07:21.781 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:21.779+0000 7fdb0f7a0640 1 -- 172.21.15.156:0/2613307044 shutdown_connections 2024-07-16T10:07:21.781 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:21.779+0000 7fdb0f7a0640 1 -- 172.21.15.156:0/2613307044 wait complete. 2024-07-16T10:07:21.843 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:07:21 smithi176 ceph-mon[81689]: pgmap v324: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:07:22.393 DEBUG:tasks.fs:max_mds reduced in epoch 27 2024-07-16T10:07:22.393 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 27 2024-07-16T10:07:22.393 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 26c91c26-4358-11ef-bcad-c7b262605968 -- ceph fs dump --format=json 28 2024-07-16T10:07:22.651 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/26c91c26-4358-11ef-bcad-c7b262605968/mon.smithi156/config 2024-07-16T10:07:22.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:07:22 smithi156 ceph-mon[93066]: from='client.? 172.21.15.156:0/2613307044' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 27, "format": "json"}]: dispatch 2024-07-16T10:07:22.843 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:07:22 smithi176 ceph-mon[81689]: from='client.? 172.21.15.156:0/2613307044' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 27, "format": "json"}]: dispatch 2024-07-16T10:07:23.726 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:07:23 smithi156 ceph-mon[93066]: pgmap v325: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:07:23.843 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:07:23 smithi176 ceph-mon[81689]: pgmap v325: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:07:24.204 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:24.182+0000 7fbae19ca640 1 -- 172.21.15.156:0/2307907992 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbadc0fa9f0 msgr2=0x7fbadc0fae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:07:24.204 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:24.182+0000 7fbae19ca640 1 --2- 172.21.15.156:0/2307907992 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbadc0fa9f0 0x7fbadc0fae40 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fbac40048d0 tx=0x7fbac4009da0 comp rx=0 tx=0).stop 2024-07-16T10:07:24.204 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:24.182+0000 7fbae19ca640 1 -- 172.21.15.156:0/2307907992 shutdown_connections 2024-07-16T10:07:24.204 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:24.182+0000 7fbae19ca640 1 --2- 172.21.15.156:0/2307907992 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbadc0fa9f0 0x7fbadc0fae40 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:07:24.204 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:24.182+0000 7fbae19ca640 1 --2- 172.21.15.156:0/2307907992 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbadc0fe270 0x7fbadc0fe640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:07:24.204 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:24.182+0000 7fbae19ca640 1 -- 172.21.15.156:0/2307907992 >> 172.21.15.156:0/2307907992 conn(0x7fbadc071910 msgr2=0x7fbadc071d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T10:07:24.204 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:24.183+0000 7fbae19ca640 1 -- 172.21.15.156:0/2307907992 shutdown_connections 2024-07-16T10:07:24.204 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:24.183+0000 7fbae19ca640 1 -- 172.21.15.156:0/2307907992 wait complete. 2024-07-16T10:07:24.205 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:24.183+0000 7fbae19ca640 1 Processor -- start 2024-07-16T10:07:24.205 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:24.183+0000 7fbae19ca640 1 -- start start 2024-07-16T10:07:24.205 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:24.183+0000 7fbae19ca640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbadc0fa9f0 0x7fbadc113080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:07:24.205 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:24.183+0000 7fbae19ca640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbadc0fe270 0x7fbadc1135c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:07:24.205 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:24.183+0000 7fbae19ca640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbadc113bc0 con 0x7fbadc0fa9f0 2024-07-16T10:07:24.205 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:24.183+0000 7fbae19ca640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbadc10d000 con 0x7fbadc0fe270 2024-07-16T10:07:24.205 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:24.183+0000 7fbadaffd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbadc0fa9f0 0x7fbadc113080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:07:24.205 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:24.184+0000 7fbadaffd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbadc0fa9f0 0x7fbadc113080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:58864/0 (socket says 172.21.15.156:58864) 2024-07-16T10:07:24.205 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:24.184+0000 7fbadaffd640 1 -- 172.21.15.156:0/3344861124 learned_addr learned my addr 172.21.15.156:0/3344861124 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T10:07:24.205 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:24.184+0000 7fbad27fc640 1 --2- 172.21.15.156:0/3344861124 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbadc0fe270 0x7fbadc1135c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:07:24.205 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:24.184+0000 7fbadaffd640 1 -- 172.21.15.156:0/3344861124 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbadc0fe270 msgr2=0x7fbadc1135c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:07:24.205 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:24.184+0000 7fbadaffd640 1 --2- 172.21.15.156:0/3344861124 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbadc0fe270 0x7fbadc1135c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:07:24.206 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:24.184+0000 7fbadaffd640 1 -- 172.21.15.156:0/3344861124 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbac4005040 con 0x7fbadc0fa9f0 2024-07-16T10:07:24.206 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:24.184+0000 7fbad27fc640 1 --2- 172.21.15.156:0/3344861124 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbadc0fe270 0x7fbadc1135c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-16T10:07:24.206 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:24.184+0000 7fbadaffd640 1 --2- 172.21.15.156:0/3344861124 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbadc0fa9f0 0x7fbadc113080 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fbacc002a50 tx=0x7fbacc004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T10:07:24.206 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:24.184+0000 7fbad3fff640 1 -- 172.21.15.156:0/3344861124 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbacc007c30 con 0x7fbadc0fa9f0 2024-07-16T10:07:24.206 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:24.184+0000 7fbae19ca640 1 -- 172.21.15.156:0/3344861124 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbadc10d2e0 con 0x7fbadc0fa9f0 2024-07-16T10:07:24.206 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:24.184+0000 7fbad3fff640 1 -- 172.21.15.156:0/3344861124 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbacc00f040 con 0x7fbadc0fa9f0 2024-07-16T10:07:24.206 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:24.184+0000 7fbad3fff640 1 -- 172.21.15.156:0/3344861124 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbacc00dbe0 con 0x7fbadc0fa9f0 2024-07-16T10:07:24.206 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:24.184+0000 7fbae19ca640 1 -- 172.21.15.156:0/3344861124 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbadc10d880 con 0x7fbadc0fa9f0 2024-07-16T10:07:24.206 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:24.185+0000 7fbae19ca640 1 -- 172.21.15.156:0/3344861124 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbadc064b60 con 0x7fbadc0fa9f0 2024-07-16T10:07:24.206 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:24.185+0000 7fbad3fff640 1 -- 172.21.15.156:0/3344861124 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 38) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fbacc00dd40 con 0x7fbadc0fa9f0 2024-07-16T10:07:24.206 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:24.186+0000 7fbad3fff640 1 --2- 172.21.15.156:0/3344861124 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7fbac806e0b0 0x7fbac8070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:07:24.206 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:24.186+0000 7fbad3fff640 1 -- 172.21.15.156:0/3344861124 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7fbacc08dbd0 con 0x7fbadc0fa9f0 2024-07-16T10:07:24.207 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:24.186+0000 7fbad27fc640 1 --2- 172.21.15.156:0/3344861124 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7fbac806e0b0 0x7fbac8070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:07:24.207 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:24.186+0000 7fbad27fc640 1 --2- 172.21.15.156:0/3344861124 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7fbac806e0b0 0x7fbac8070560 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fbac4009f80 tx=0x7fbac4009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T10:07:24.207 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:24.190+0000 7fbad3fff640 1 -- 172.21.15.156:0/3344861124 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187716 (secure 0 0 0) 0x7fbacc056ea0 con 0x7fbadc0fa9f0 2024-07-16T10:07:24.405 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:24.404+0000 7fbae19ca640 1 -- 172.21.15.156:0/3344861124 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 28, "format": "json"} v 0) v1 -- 0x7fbadc04b5b0 con 0x7fbadc0fa9f0 2024-07-16T10:07:24.405 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:24.404+0000 7fbad3fff640 1 -- 172.21.15.156:0/3344861124 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 28, "format": "json"}]=0 dumped fsmap epoch 28 v45) v1 ==== 107+0+4087 (secure 0 0 0) 0x7fbadc04b5b0 con 0x7fbadc0fa9f0 2024-07-16T10:07:24.406 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-07-16T10:07:24.406 INFO:teuthology.orchestra.run.smithi156.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":44269,"name":"cephfs.smithi176.wahbsi","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6827/3942496125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6826","nonce":3942496125},{"type":"v1","addr":"172.21.15.176:6827","nonce":3942496125}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":22},{"gid":44301,"name":"cephfs.smithi176.dvvbud","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6825/3749220343","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6824","nonce":3749220343},{"type":"v1","addr":"172.21.15.176:6825","nonce":3749220343}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26}],"filesystems":[{"mdsmap":{"epoch":28,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-16T09:52:55.214975+0000","modified":"2024-07-16T10:02:40.290493+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":81,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":34374},"failed":[],"damaged":[],"stopped":[1],"info":{"gid_34374":{"gid":34374,"name":"cephfs.smithi156.evwwaz","rank":0,"incarnation":28,"state":"up:replay","state_seq":1,"addr":"172.21.15.156:6829/333092891","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":333092891},{"type":"v1","addr":"172.21.15.156:6829","nonce":333092891}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-07-16T10:07:24.406 INFO:teuthology.orchestra.run.smithi156.stderr:dumped fsmap epoch 28 2024-07-16T10:07:24.407 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:24.406+0000 7fbae19ca640 1 -- 172.21.15.156:0/3344861124 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7fbac806e0b0 msgr2=0x7fbac8070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:07:24.408 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:24.406+0000 7fbae19ca640 1 --2- 172.21.15.156:0/3344861124 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7fbac806e0b0 0x7fbac8070560 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fbac4009f80 tx=0x7fbac4009da0 comp rx=0 tx=0).stop 2024-07-16T10:07:24.408 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:24.406+0000 7fbae19ca640 1 -- 172.21.15.156:0/3344861124 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbadc0fa9f0 msgr2=0x7fbadc113080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:07:24.408 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:24.406+0000 7fbae19ca640 1 --2- 172.21.15.156:0/3344861124 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbadc0fa9f0 0x7fbadc113080 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fbacc002a50 tx=0x7fbacc004600 comp rx=0 tx=0).stop 2024-07-16T10:07:24.408 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:24.406+0000 7fbae19ca640 1 -- 172.21.15.156:0/3344861124 shutdown_connections 2024-07-16T10:07:24.408 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:24.406+0000 7fbae19ca640 1 --2- 172.21.15.156:0/3344861124 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7fbac806e0b0 0x7fbac8070560 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:07:24.408 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:24.406+0000 7fbae19ca640 1 --2- 172.21.15.156:0/3344861124 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbadc0fe270 0x7fbadc1135c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:07:24.408 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:24.406+0000 7fbae19ca640 1 --2- 172.21.15.156:0/3344861124 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbadc0fa9f0 0x7fbadc113080 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:07:24.408 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:24.406+0000 7fbae19ca640 1 -- 172.21.15.156:0/3344861124 >> 172.21.15.156:0/3344861124 conn(0x7fbadc071910 msgr2=0x7fbadc105220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T10:07:24.408 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:24.406+0000 7fbae19ca640 1 -- 172.21.15.156:0/3344861124 shutdown_connections 2024-07-16T10:07:24.408 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:24.406+0000 7fbae19ca640 1 -- 172.21.15.156:0/3344861124 wait complete. 2024-07-16T10:07:24.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:07:24 smithi156 ceph-mon[93066]: from='client.? 172.21.15.156:0/3344861124' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 28, "format": "json"}]: dispatch 2024-07-16T10:07:24.843 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:07:24 smithi176 ceph-mon[81689]: from='client.? 172.21.15.156:0/3344861124' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 28, "format": "json"}]: dispatch 2024-07-16T10:07:25.008 DEBUG:tasks.fs:max_mds reduced in epoch 28 2024-07-16T10:07:25.008 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 28 2024-07-16T10:07:25.009 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 26c91c26-4358-11ef-bcad-c7b262605968 -- ceph fs dump --format=json 29 2024-07-16T10:07:25.272 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/26c91c26-4358-11ef-bcad-c7b262605968/mon.smithi156/config 2024-07-16T10:07:25.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:07:25 smithi156 ceph-mon[93066]: pgmap v326: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-07-16T10:07:25.843 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:07:25 smithi176 ceph-mon[81689]: pgmap v326: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-07-16T10:07:26.874 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:26.871+0000 7f2a74637640 1 -- 172.21.15.156:0/3896065073 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2a6c0fdf50 msgr2=0x7f2a6c0fe3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:07:26.875 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:26.871+0000 7f2a74637640 1 --2- 172.21.15.156:0/3896065073 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2a6c0fdf50 0x7f2a6c0fe3c0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f2a60009c50 tx=0x7f2a60004d10 comp rx=0 tx=0).stop 2024-07-16T10:07:26.875 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:26.871+0000 7f2a74637640 1 -- 172.21.15.156:0/3896065073 shutdown_connections 2024-07-16T10:07:26.875 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:26.871+0000 7f2a74637640 1 --2- 172.21.15.156:0/3896065073 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2a6c0fdf50 0x7f2a6c0fe3c0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:07:26.875 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:26.871+0000 7f2a74637640 1 --2- 172.21.15.156:0/3896065073 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2a6c0650e0 0x7f2a6c0654b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:07:26.875 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:26.871+0000 7f2a74637640 1 -- 172.21.15.156:0/3896065073 >> 172.21.15.156:0/3896065073 conn(0x7f2a6c0721c0 msgr2=0x7f2a6c0725c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T10:07:26.875 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:26.871+0000 7f2a74637640 1 -- 172.21.15.156:0/3896065073 shutdown_connections 2024-07-16T10:07:26.875 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:26.871+0000 7f2a74637640 1 -- 172.21.15.156:0/3896065073 wait complete. 2024-07-16T10:07:26.875 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:26.872+0000 7f2a74637640 1 Processor -- start 2024-07-16T10:07:26.875 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:26.872+0000 7f2a74637640 1 -- start start 2024-07-16T10:07:26.875 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:26.872+0000 7f2a74637640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2a6c0650e0 0x7f2a6c113180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:07:26.875 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:26.872+0000 7f2a74637640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2a6c0fdf50 0x7f2a6c1136c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:07:26.876 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:26.872+0000 7f2a74637640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a6c113d50 con 0x7f2a6c0fdf50 2024-07-16T10:07:26.876 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:26.872+0000 7f2a74637640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a6c10d150 con 0x7f2a6c0650e0 2024-07-16T10:07:26.876 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:26.872+0000 7f2a71bd2640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2a6c0fdf50 0x7f2a6c1136c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:07:26.876 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:26.872+0000 7f2a71bd2640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2a6c0fdf50 0x7f2a6c1136c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:58892/0 (socket says 172.21.15.156:58892) 2024-07-16T10:07:26.876 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:26.872+0000 7f2a71bd2640 1 -- 172.21.15.156:0/1620403901 learned_addr learned my addr 172.21.15.156:0/1620403901 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T10:07:26.876 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:26.872+0000 7f2a723d3640 1 --2- 172.21.15.156:0/1620403901 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2a6c0650e0 0x7f2a6c113180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:07:26.876 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:26.873+0000 7f2a71bd2640 1 -- 172.21.15.156:0/1620403901 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2a6c0650e0 msgr2=0x7f2a6c113180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:07:26.876 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:26.873+0000 7f2a71bd2640 1 --2- 172.21.15.156:0/1620403901 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2a6c0650e0 0x7f2a6c113180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:07:26.876 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:26.873+0000 7f2a71bd2640 1 -- 172.21.15.156:0/1620403901 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a600098b0 con 0x7f2a6c0fdf50 2024-07-16T10:07:26.876 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:26.873+0000 7f2a71bd2640 1 --2- 172.21.15.156:0/1620403901 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2a6c0fdf50 0x7f2a6c1136c0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f2a60004f80 tx=0x7f2a60004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T10:07:26.877 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:26.873+0000 7f2a6affd640 1 -- 172.21.15.156:0/1620403901 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2a60008de0 con 0x7f2a6c0fdf50 2024-07-16T10:07:26.877 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:26.873+0000 7f2a74637640 1 -- 172.21.15.156:0/1620403901 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a6c10d3d0 con 0x7f2a6c0fdf50 2024-07-16T10:07:26.877 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:26.873+0000 7f2a6affd640 1 -- 172.21.15.156:0/1620403901 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2a60032c00 con 0x7f2a6c0fdf50 2024-07-16T10:07:26.877 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:26.873+0000 7f2a6affd640 1 -- 172.21.15.156:0/1620403901 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2a6003dbc0 con 0x7f2a6c0fdf50 2024-07-16T10:07:26.877 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:26.873+0000 7f2a74637640 1 -- 172.21.15.156:0/1620403901 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a6c10d8f0 con 0x7f2a6c0fdf50 2024-07-16T10:07:26.878 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:26.874+0000 7f2a74637640 1 -- 172.21.15.156:0/1620403901 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a6c0fdaf0 con 0x7f2a6c0fdf50 2024-07-16T10:07:26.878 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:26.875+0000 7f2a6affd640 1 -- 172.21.15.156:0/1620403901 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 38) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f2a6002f030 con 0x7f2a6c0fdf50 2024-07-16T10:07:26.878 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:26.875+0000 7f2a6affd640 1 --2- 172.21.15.156:0/1620403901 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f2a5406e0b0 0x7f2a54070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:07:26.878 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:26.875+0000 7f2a6affd640 1 -- 172.21.15.156:0/1620403901 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f2a600af1b0 con 0x7f2a6c0fdf50 2024-07-16T10:07:26.878 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:26.875+0000 7f2a723d3640 1 --2- 172.21.15.156:0/1620403901 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f2a5406e0b0 0x7f2a54070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:07:26.878 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:26.876+0000 7f2a723d3640 1 --2- 172.21.15.156:0/1620403901 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f2a5406e0b0 0x7f2a54070560 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f2a64009b50 tx=0x7f2a64009120 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T10:07:26.880 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:26.879+0000 7f2a6affd640 1 -- 172.21.15.156:0/1620403901 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187716 (secure 0 0 0) 0x7f2a60078540 con 0x7f2a6c0fdf50 2024-07-16T10:07:27.094 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:27.093+0000 7f2a74637640 1 -- 172.21.15.156:0/1620403901 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 29, "format": "json"} v 0) v1 -- 0x7f2a6c10e7f0 con 0x7f2a6c0fdf50 2024-07-16T10:07:27.095 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:27.094+0000 7f2a6affd640 1 -- 172.21.15.156:0/1620403901 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 29, "format": "json"}]=0 dumped fsmap epoch 29 v45) v1 ==== 107+0+4091 (secure 0 0 0) 0x7f2a6007bc00 con 0x7f2a6c0fdf50 2024-07-16T10:07:27.095 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-07-16T10:07:27.095 INFO:teuthology.orchestra.run.smithi156.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":44269,"name":"cephfs.smithi176.wahbsi","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6827/3942496125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6826","nonce":3942496125},{"type":"v1","addr":"172.21.15.176:6827","nonce":3942496125}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":22},{"gid":44301,"name":"cephfs.smithi176.dvvbud","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6825/3749220343","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6824","nonce":3749220343},{"type":"v1","addr":"172.21.15.176:6825","nonce":3749220343}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26}],"filesystems":[{"mdsmap":{"epoch":29,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-16T09:52:55.214975+0000","modified":"2024-07-16T10:02:45.422152+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":81,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":34374},"failed":[],"damaged":[],"stopped":[1],"info":{"gid_34374":{"gid":34374,"name":"cephfs.smithi156.evwwaz","rank":0,"incarnation":28,"state":"up:reconnect","state_seq":14,"addr":"172.21.15.156:6829/333092891","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":333092891},{"type":"v1","addr":"172.21.15.156:6829","nonce":333092891}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-07-16T10:07:27.096 INFO:teuthology.orchestra.run.smithi156.stderr:dumped fsmap epoch 29 2024-07-16T10:07:27.097 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:27.096+0000 7f2a74637640 1 -- 172.21.15.156:0/1620403901 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f2a5406e0b0 msgr2=0x7f2a54070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:07:27.097 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:27.096+0000 7f2a74637640 1 --2- 172.21.15.156:0/1620403901 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f2a5406e0b0 0x7f2a54070560 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f2a64009b50 tx=0x7f2a64009120 comp rx=0 tx=0).stop 2024-07-16T10:07:27.097 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:27.096+0000 7f2a74637640 1 -- 172.21.15.156:0/1620403901 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2a6c0fdf50 msgr2=0x7f2a6c1136c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:07:27.097 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:27.096+0000 7f2a74637640 1 --2- 172.21.15.156:0/1620403901 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2a6c0fdf50 0x7f2a6c1136c0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f2a60004f80 tx=0x7f2a60004d10 comp rx=0 tx=0).stop 2024-07-16T10:07:27.097 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:27.096+0000 7f2a74637640 1 -- 172.21.15.156:0/1620403901 shutdown_connections 2024-07-16T10:07:27.098 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:27.096+0000 7f2a74637640 1 --2- 172.21.15.156:0/1620403901 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f2a5406e0b0 0x7f2a54070560 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:07:27.098 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:27.096+0000 7f2a74637640 1 --2- 172.21.15.156:0/1620403901 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2a6c0fdf50 0x7f2a6c1136c0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:07:27.098 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:27.096+0000 7f2a74637640 1 --2- 172.21.15.156:0/1620403901 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2a6c0650e0 0x7f2a6c113180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:07:27.098 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:27.096+0000 7f2a74637640 1 -- 172.21.15.156:0/1620403901 >> 172.21.15.156:0/1620403901 conn(0x7f2a6c0721c0 msgr2=0x7f2a6c1051d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T10:07:27.098 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:27.096+0000 7f2a74637640 1 -- 172.21.15.156:0/1620403901 shutdown_connections 2024-07-16T10:07:27.098 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:27.096+0000 7f2a74637640 1 -- 172.21.15.156:0/1620403901 wait complete. 2024-07-16T10:07:27.673 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:07:27 smithi156 ceph-mon[93066]: pgmap v327: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:07:27.673 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:07:27 smithi156 ceph-mon[93066]: from='client.? 172.21.15.156:0/1620403901' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 29, "format": "json"}]: dispatch 2024-07-16T10:07:27.701 DEBUG:tasks.fs:max_mds reduced in epoch 29 2024-07-16T10:07:27.701 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 29 2024-07-16T10:07:27.702 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 26c91c26-4358-11ef-bcad-c7b262605968 -- ceph fs dump --format=json 30 2024-07-16T10:07:27.843 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:07:27 smithi176 ceph-mon[81689]: pgmap v327: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:07:27.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:07:27 smithi176 ceph-mon[81689]: from='client.? 172.21.15.156:0/1620403901' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 29, "format": "json"}]: dispatch 2024-07-16T10:07:27.961 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/26c91c26-4358-11ef-bcad-c7b262605968/mon.smithi156/config 2024-07-16T10:07:29.305 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:29.302+0000 7f540d666640 1 -- 172.21.15.156:0/1697364462 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5408101c10 msgr2=0x7f54080fb2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:07:29.305 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:29.302+0000 7f540d666640 1 --2- 172.21.15.156:0/1697364462 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5408101c10 0x7f54080fb2a0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f53f00048d0 tx=0x7f53f0009da0 comp rx=0 tx=0).stop 2024-07-16T10:07:29.305 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:29.302+0000 7f540d666640 1 -- 172.21.15.156:0/1697364462 shutdown_connections 2024-07-16T10:07:29.306 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:29.302+0000 7f540d666640 1 --2- 172.21.15.156:0/1697364462 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5408101c10 0x7f54080fb2a0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:07:29.306 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:29.302+0000 7f540d666640 1 --2- 172.21.15.156:0/1697364462 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5408101270 0x7f5408101640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:07:29.306 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:29.302+0000 7f540d666640 1 -- 172.21.15.156:0/1697364462 >> 172.21.15.156:0/1697364462 conn(0x7f54080f6e20 msgr2=0x7f54080f9230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T10:07:29.306 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:29.302+0000 7f540d666640 1 -- 172.21.15.156:0/1697364462 shutdown_connections 2024-07-16T10:07:29.306 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:29.302+0000 7f540d666640 1 -- 172.21.15.156:0/1697364462 wait complete. 2024-07-16T10:07:29.306 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:29.302+0000 7f540d666640 1 Processor -- start 2024-07-16T10:07:29.306 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:29.303+0000 7f540d666640 1 -- start start 2024-07-16T10:07:29.306 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:29.303+0000 7f540d666640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5408101270 0x7f54080fcf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:07:29.306 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:29.303+0000 7f540d666640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5408101c10 0x7f54080fd450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:07:29.307 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:29.303+0000 7f540d666640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54080fd990 con 0x7f5408101c10 2024-07-16T10:07:29.307 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:29.303+0000 7f540d666640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54080fdb00 con 0x7f5408101270 2024-07-16T10:07:29.307 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:29.303+0000 7f53fe7fc640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5408101c10 0x7f54080fd450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:07:29.307 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:29.303+0000 7f53fe7fc640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5408101c10 0x7f54080fd450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:58908/0 (socket says 172.21.15.156:58908) 2024-07-16T10:07:29.307 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:29.303+0000 7f53fe7fc640 1 -- 172.21.15.156:0/3989658263 learned_addr learned my addr 172.21.15.156:0/3989658263 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T10:07:29.307 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:29.303+0000 7f5406ffd640 1 --2- 172.21.15.156:0/3989658263 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5408101270 0x7f54080fcf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:07:29.307 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:29.303+0000 7f53fe7fc640 1 -- 172.21.15.156:0/3989658263 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5408101270 msgr2=0x7f54080fcf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:07:29.307 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:29.303+0000 7f53fe7fc640 1 --2- 172.21.15.156:0/3989658263 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5408101270 0x7f54080fcf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:07:29.307 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:29.303+0000 7f53fe7fc640 1 -- 172.21.15.156:0/3989658263 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53f0005040 con 0x7f5408101c10 2024-07-16T10:07:29.307 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:29.303+0000 7f53fe7fc640 1 --2- 172.21.15.156:0/3989658263 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5408101c10 0x7f54080fd450 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f53f00043d0 tx=0x7f53f0009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T10:07:29.307 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:29.304+0000 7f53fffff640 1 -- 172.21.15.156:0/3989658263 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f53f0038070 con 0x7f5408101c10 2024-07-16T10:07:29.307 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:29.304+0000 7f540d666640 1 -- 172.21.15.156:0/3989658263 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54080fdd30 con 0x7f5408101c10 2024-07-16T10:07:29.308 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:29.304+0000 7f53fffff640 1 -- 172.21.15.156:0/3989658263 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f53f0032970 con 0x7f5408101c10 2024-07-16T10:07:29.308 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:29.304+0000 7f53fffff640 1 -- 172.21.15.156:0/3989658263 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f53f003da60 con 0x7f5408101c10 2024-07-16T10:07:29.308 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:29.304+0000 7f540d666640 1 -- 172.21.15.156:0/3989658263 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5408119740 con 0x7f5408101c10 2024-07-16T10:07:29.309 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:29.305+0000 7f540d666640 1 -- 172.21.15.156:0/3989658263 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54080fc9e0 con 0x7f5408101c10 2024-07-16T10:07:29.309 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:29.305+0000 7f53fffff640 1 -- 172.21.15.156:0/3989658263 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 38) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f53f0030020 con 0x7f5408101c10 2024-07-16T10:07:29.309 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:29.306+0000 7f53fffff640 1 --2- 172.21.15.156:0/3989658263 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f53f406e0b0 0x7f53f4070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:07:29.309 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:29.306+0000 7f53fffff640 1 -- 172.21.15.156:0/3989658263 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f53f00addb0 con 0x7f5408101c10 2024-07-16T10:07:29.309 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:29.306+0000 7f5406ffd640 1 --2- 172.21.15.156:0/3989658263 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f53f406e0b0 0x7f53f4070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:07:29.309 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:29.306+0000 7f5406ffd640 1 --2- 172.21.15.156:0/3989658263 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f53f406e0b0 0x7f53f4070560 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f53f8007fd0 tx=0x7f53f8009e90 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T10:07:29.311 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:29.310+0000 7f53fffff640 1 -- 172.21.15.156:0/3989658263 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187716 (secure 0 0 0) 0x7f53f0077080 con 0x7f5408101c10 2024-07-16T10:07:29.523 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:29.522+0000 7f540d666640 1 -- 172.21.15.156:0/3989658263 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 30, "format": "json"} v 0) v1 -- 0x7f540804b530 con 0x7f5408101c10 2024-07-16T10:07:29.523 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:29.522+0000 7f53fffff640 1 -- 172.21.15.156:0/3989658263 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 30, "format": "json"}]=0 dumped fsmap epoch 30 v45) v1 ==== 107+0+4088 (secure 0 0 0) 0x7f53f007a740 con 0x7f5408101c10 2024-07-16T10:07:29.524 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-07-16T10:07:29.524 INFO:teuthology.orchestra.run.smithi156.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":44269,"name":"cephfs.smithi176.wahbsi","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6827/3942496125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6826","nonce":3942496125},{"type":"v1","addr":"172.21.15.176:6827","nonce":3942496125}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":22},{"gid":44301,"name":"cephfs.smithi176.dvvbud","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6825/3749220343","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6824","nonce":3749220343},{"type":"v1","addr":"172.21.15.176:6825","nonce":3749220343}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26}],"filesystems":[{"mdsmap":{"epoch":30,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-16T09:52:55.214975+0000","modified":"2024-07-16T10:02:46.301684+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":81,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":34374},"failed":[],"damaged":[],"stopped":[1],"info":{"gid_34374":{"gid":34374,"name":"cephfs.smithi156.evwwaz","rank":0,"incarnation":28,"state":"up:rejoin","state_seq":15,"addr":"172.21.15.156:6829/333092891","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":333092891},{"type":"v1","addr":"172.21.15.156:6829","nonce":333092891}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-07-16T10:07:29.524 INFO:teuthology.orchestra.run.smithi156.stderr:dumped fsmap epoch 30 2024-07-16T10:07:29.526 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:29.524+0000 7f540d666640 1 -- 172.21.15.156:0/3989658263 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f53f406e0b0 msgr2=0x7f53f4070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:07:29.526 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:29.524+0000 7f540d666640 1 --2- 172.21.15.156:0/3989658263 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f53f406e0b0 0x7f53f4070560 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f53f8007fd0 tx=0x7f53f8009e90 comp rx=0 tx=0).stop 2024-07-16T10:07:29.526 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:29.524+0000 7f540d666640 1 -- 172.21.15.156:0/3989658263 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5408101c10 msgr2=0x7f54080fd450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:07:29.526 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:29.524+0000 7f540d666640 1 --2- 172.21.15.156:0/3989658263 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5408101c10 0x7f54080fd450 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f53f00043d0 tx=0x7f53f0009da0 comp rx=0 tx=0).stop 2024-07-16T10:07:29.526 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:29.524+0000 7f540d666640 1 -- 172.21.15.156:0/3989658263 shutdown_connections 2024-07-16T10:07:29.526 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:29.524+0000 7f540d666640 1 --2- 172.21.15.156:0/3989658263 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f53f406e0b0 0x7f53f4070560 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:07:29.526 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:29.524+0000 7f540d666640 1 --2- 172.21.15.156:0/3989658263 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5408101c10 0x7f54080fd450 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:07:29.526 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:29.524+0000 7f540d666640 1 --2- 172.21.15.156:0/3989658263 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5408101270 0x7f54080fcf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:07:29.526 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:29.524+0000 7f540d666640 1 -- 172.21.15.156:0/3989658263 >> 172.21.15.156:0/3989658263 conn(0x7f54080f6e20 msgr2=0x7f5408065910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T10:07:29.526 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:29.524+0000 7f540d666640 1 -- 172.21.15.156:0/3989658263 shutdown_connections 2024-07-16T10:07:29.527 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:29.525+0000 7f540d666640 1 -- 172.21.15.156:0/3989658263 wait complete. 2024-07-16T10:07:29.843 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:07:29 smithi176 ceph-mon[81689]: pgmap v328: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-07-16T10:07:29.843 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:07:29 smithi176 ceph-mon[81689]: from='client.? 172.21.15.156:0/3989658263' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 30, "format": "json"}]: dispatch 2024-07-16T10:07:29.888 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:07:29 smithi156 ceph-mon[93066]: pgmap v328: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-07-16T10:07:29.888 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:07:29 smithi156 ceph-mon[93066]: from='client.? 172.21.15.156:0/3989658263' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 30, "format": "json"}]: dispatch 2024-07-16T10:07:30.198 DEBUG:tasks.fs:max_mds reduced in epoch 30 2024-07-16T10:07:30.198 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 30 2024-07-16T10:07:30.198 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 26c91c26-4358-11ef-bcad-c7b262605968 -- ceph fs dump --format=json 31 2024-07-16T10:07:30.456 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/26c91c26-4358-11ef-bcad-c7b262605968/mon.smithi156/config 2024-07-16T10:07:31.781 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:07:31 smithi156 ceph-mon[93066]: pgmap v329: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:07:31.843 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:07:31 smithi176 ceph-mon[81689]: pgmap v329: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:07:32.018 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:31.996+0000 7f78071a1640 1 -- 172.21.15.156:0/3415172581 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7800102cf0 msgr2=0x7f78000fc380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:07:32.018 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:31.996+0000 7f78071a1640 1 --2- 172.21.15.156:0/3415172581 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7800102cf0 0x7f78000fc380 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f77e80048d0 tx=0x7f77e8009da0 comp rx=0 tx=0).stop 2024-07-16T10:07:32.018 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:31.996+0000 7f78071a1640 1 -- 172.21.15.156:0/3415172581 shutdown_connections 2024-07-16T10:07:32.018 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:31.996+0000 7f78071a1640 1 --2- 172.21.15.156:0/3415172581 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7800102cf0 0x7f78000fc380 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:07:32.018 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:31.996+0000 7f78071a1640 1 --2- 172.21.15.156:0/3415172581 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7800102350 0x7f7800102720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:07:32.018 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:31.996+0000 7f78071a1640 1 -- 172.21.15.156:0/3415172581 >> 172.21.15.156:0/3415172581 conn(0x7f78000f7f00 msgr2=0x7f78000fa310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T10:07:32.018 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:31.996+0000 7f78071a1640 1 -- 172.21.15.156:0/3415172581 shutdown_connections 2024-07-16T10:07:32.018 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:31.997+0000 7f78071a1640 1 -- 172.21.15.156:0/3415172581 wait complete. 2024-07-16T10:07:32.018 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:31.997+0000 7f78071a1640 1 Processor -- start 2024-07-16T10:07:32.018 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:31.997+0000 7f78071a1640 1 -- start start 2024-07-16T10:07:32.018 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:31.997+0000 7f78071a1640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7800102350 0x7f7800113160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:07:32.019 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:31.997+0000 7f78071a1640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7800102cf0 0x7f78001136a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:07:32.019 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:31.997+0000 7f78071a1640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7800113d30 con 0x7f7800102cf0 2024-07-16T10:07:32.019 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:31.997+0000 7f78071a1640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f780010d0e0 con 0x7f7800102350 2024-07-16T10:07:32.019 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:31.998+0000 7f77fffff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7800102cf0 0x7f78001136a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:07:32.019 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:31.998+0000 7f77fffff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7800102cf0 0x7f78001136a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:58916/0 (socket says 172.21.15.156:58916) 2024-07-16T10:07:32.019 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:31.998+0000 7f77fffff640 1 -- 172.21.15.156:0/3998818556 learned_addr learned my addr 172.21.15.156:0/3998818556 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T10:07:32.019 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:31.998+0000 7f7804f3d640 1 --2- 172.21.15.156:0/3998818556 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7800102350 0x7f7800113160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:07:32.019 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:31.998+0000 7f77fffff640 1 -- 172.21.15.156:0/3998818556 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7800102350 msgr2=0x7f7800113160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:07:32.019 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:31.998+0000 7f77fffff640 1 --2- 172.21.15.156:0/3998818556 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7800102350 0x7f7800113160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:07:32.019 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:31.998+0000 7f77fffff640 1 -- 172.21.15.156:0/3998818556 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77e8005040 con 0x7f7800102cf0 2024-07-16T10:07:32.020 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:31.998+0000 7f7804f3d640 1 --2- 172.21.15.156:0/3998818556 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7800102350 0x7f7800113160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-16T10:07:32.020 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:31.998+0000 7f77fffff640 1 --2- 172.21.15.156:0/3998818556 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7800102cf0 0x7f78001136a0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f77e8032430 tx=0x7f77e8009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T10:07:32.020 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:31.998+0000 7f77fd7fa640 1 -- 172.21.15.156:0/3998818556 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f77e8038070 con 0x7f7800102cf0 2024-07-16T10:07:32.020 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:31.998+0000 7f77fd7fa640 1 -- 172.21.15.156:0/3998818556 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f77e8008ee0 con 0x7f7800102cf0 2024-07-16T10:07:32.020 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:31.998+0000 7f78071a1640 1 -- 172.21.15.156:0/3998818556 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f780010d360 con 0x7f7800102cf0 2024-07-16T10:07:32.020 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:31.998+0000 7f77fd7fa640 1 -- 172.21.15.156:0/3998818556 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f77e803d7a0 con 0x7f7800102cf0 2024-07-16T10:07:32.020 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:31.998+0000 7f78071a1640 1 -- 172.21.15.156:0/3998818556 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f780010d820 con 0x7f7800102cf0 2024-07-16T10:07:32.020 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:31.999+0000 7f78071a1640 1 -- 172.21.15.156:0/3998818556 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78000fdac0 con 0x7f7800102cf0 2024-07-16T10:07:32.020 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:32.000+0000 7f77fd7fa640 1 -- 172.21.15.156:0/3998818556 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 38) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f77e8030020 con 0x7f7800102cf0 2024-07-16T10:07:32.020 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:32.000+0000 7f77fd7fa640 1 --2- 172.21.15.156:0/3998818556 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f77ec06e0b0 0x7f77ec070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:07:32.020 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:32.000+0000 7f77fd7fa640 1 -- 172.21.15.156:0/3998818556 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f77e80ae170 con 0x7f7800102cf0 2024-07-16T10:07:32.020 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:32.000+0000 7f7804f3d640 1 --2- 172.21.15.156:0/3998818556 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f77ec06e0b0 0x7f77ec070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:07:32.020 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:32.001+0000 7f7804f3d640 1 --2- 172.21.15.156:0/3998818556 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f77ec06e0b0 0x7f77ec070560 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f77f0001370 tx=0x7f77f00096a0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T10:07:32.021 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:32.005+0000 7f77fd7fa640 1 -- 172.21.15.156:0/3998818556 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187716 (secure 0 0 0) 0x7f77e8077580 con 0x7f7800102cf0 2024-07-16T10:07:32.218 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:32.216+0000 7f78071a1640 1 -- 172.21.15.156:0/3998818556 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 31, "format": "json"} v 0) v1 -- 0x7f7800109570 con 0x7f7800102cf0 2024-07-16T10:07:32.218 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:32.217+0000 7f77fd7fa640 1 -- 172.21.15.156:0/3998818556 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 31, "format": "json"}]=0 dumped fsmap epoch 31 v45) v1 ==== 107+0+4947 (secure 0 0 0) 0x7f77e807ac40 con 0x7f7800102cf0 2024-07-16T10:07:32.219 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-07-16T10:07:32.219 INFO:teuthology.orchestra.run.smithi156.stdout:{"epoch":31,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34438,"name":"cephfs.smithi156.rdibys","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6827/2852996640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":2852996640},{"type":"v1","addr":"172.21.15.156:6827","nonce":2852996640}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":44269,"name":"cephfs.smithi176.wahbsi","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6827/3942496125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6826","nonce":3942496125},{"type":"v1","addr":"172.21.15.176:6827","nonce":3942496125}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":22},{"gid":44301,"name":"cephfs.smithi176.dvvbud","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6825/3749220343","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6824","nonce":3749220343},{"type":"v1","addr":"172.21.15.176:6825","nonce":3749220343}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26}],"filesystems":[{"mdsmap":{"epoch":31,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-16T09:52:55.214975+0000","modified":"2024-07-16T10:02:48.307021+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":81,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":34374},"failed":[],"damaged":[],"stopped":[1],"info":{"gid_34374":{"gid":34374,"name":"cephfs.smithi156.evwwaz","rank":0,"incarnation":28,"state":"up:active","state_seq":16,"addr":"172.21.15.156:6829/333092891","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":333092891},{"type":"v1","addr":"172.21.15.156:6829","nonce":333092891}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":34374,"qdb_cluster":[34374]},"id":1}]} 2024-07-16T10:07:32.219 INFO:teuthology.orchestra.run.smithi156.stderr:dumped fsmap epoch 31 2024-07-16T10:07:32.221 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:32.219+0000 7f78071a1640 1 -- 172.21.15.156:0/3998818556 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f77ec06e0b0 msgr2=0x7f77ec070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:07:32.221 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:32.219+0000 7f78071a1640 1 --2- 172.21.15.156:0/3998818556 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f77ec06e0b0 0x7f77ec070560 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f77f0001370 tx=0x7f77f00096a0 comp rx=0 tx=0).stop 2024-07-16T10:07:32.221 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:32.219+0000 7f78071a1640 1 -- 172.21.15.156:0/3998818556 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7800102cf0 msgr2=0x7f78001136a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:07:32.221 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:32.219+0000 7f78071a1640 1 --2- 172.21.15.156:0/3998818556 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7800102cf0 0x7f78001136a0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f77e8032430 tx=0x7f77e8009da0 comp rx=0 tx=0).stop 2024-07-16T10:07:32.221 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:32.219+0000 7f78071a1640 1 -- 172.21.15.156:0/3998818556 shutdown_connections 2024-07-16T10:07:32.221 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:32.219+0000 7f78071a1640 1 --2- 172.21.15.156:0/3998818556 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f77ec06e0b0 0x7f77ec070560 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:07:32.221 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:32.219+0000 7f78071a1640 1 --2- 172.21.15.156:0/3998818556 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7800102cf0 0x7f78001136a0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:07:32.222 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:32.219+0000 7f78071a1640 1 --2- 172.21.15.156:0/3998818556 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7800102350 0x7f7800113160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:07:32.222 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:32.219+0000 7f78071a1640 1 -- 172.21.15.156:0/3998818556 >> 172.21.15.156:0/3998818556 conn(0x7f78000f7f00 msgr2=0x7f78000f8df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T10:07:32.222 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:32.219+0000 7f78071a1640 1 -- 172.21.15.156:0/3998818556 shutdown_connections 2024-07-16T10:07:32.222 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:32.219+0000 7f78071a1640 1 -- 172.21.15.156:0/3998818556 wait complete. 2024-07-16T10:07:32.837 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:07:32 smithi156 ceph-mon[93066]: from='client.? 172.21.15.156:0/3998818556' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 31, "format": "json"}]: dispatch 2024-07-16T10:07:32.839 DEBUG:tasks.fs:max_mds reduced in epoch 31 2024-07-16T10:07:32.839 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 31 2024-07-16T10:07:32.840 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 26c91c26-4358-11ef-bcad-c7b262605968 -- ceph fs dump --format=json 32 2024-07-16T10:07:32.843 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:07:32 smithi176 ceph-mon[81689]: from='client.? 172.21.15.156:0/3998818556' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 31, "format": "json"}]: dispatch 2024-07-16T10:07:33.096 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/26c91c26-4358-11ef-bcad-c7b262605968/mon.smithi156/config 2024-07-16T10:07:33.836 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:07:33 smithi156 ceph-mon[93066]: pgmap v330: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:07:33.836 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:07:33 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-16T10:07:33.843 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:07:33 smithi176 ceph-mon[81689]: pgmap v330: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:07:33.843 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:07:33 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-16T10:07:34.629 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:34.626+0000 7fe338a9a640 1 -- 172.21.15.156:0/50381077 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe334102d10 msgr2=0x7fe3340fc3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:07:34.629 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:34.626+0000 7fe338a9a640 1 --2- 172.21.15.156:0/50381077 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe334102d10 0x7fe3340fc3a0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fe3280048d0 tx=0x7fe328009da0 comp rx=0 tx=0).stop 2024-07-16T10:07:34.630 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:34.627+0000 7fe338a9a640 1 -- 172.21.15.156:0/50381077 shutdown_connections 2024-07-16T10:07:34.630 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:34.627+0000 7fe338a9a640 1 --2- 172.21.15.156:0/50381077 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe334102d10 0x7fe3340fc3a0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:07:34.630 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:34.627+0000 7fe338a9a640 1 --2- 172.21.15.156:0/50381077 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe334102370 0x7fe334102740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:07:34.630 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:34.627+0000 7fe338a9a640 1 -- 172.21.15.156:0/50381077 >> 172.21.15.156:0/50381077 conn(0x7fe3340f7f00 msgr2=0x7fe3340fa310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T10:07:34.630 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:34.627+0000 7fe338a9a640 1 -- 172.21.15.156:0/50381077 shutdown_connections 2024-07-16T10:07:34.630 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:34.627+0000 7fe338a9a640 1 -- 172.21.15.156:0/50381077 wait complete. 2024-07-16T10:07:34.630 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:34.627+0000 7fe338a9a640 1 Processor -- start 2024-07-16T10:07:34.630 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:34.627+0000 7fe338a9a640 1 -- start start 2024-07-16T10:07:34.630 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:34.627+0000 7fe338a9a640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe334102370 0x7fe33406f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:07:34.630 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:34.627+0000 7fe338a9a640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe334102d10 0x7fe33406fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:07:34.630 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:34.627+0000 7fe338a9a640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3340702a0 con 0x7fe334102370 2024-07-16T10:07:34.631 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:34.627+0000 7fe338a9a640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe334070410 con 0x7fe334102d10 2024-07-16T10:07:34.631 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:34.627+0000 7fe33259c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe334102370 0x7fe33406f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:07:34.631 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:34.627+0000 7fe33259c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe334102370 0x7fe33406f820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:42030/0 (socket says 172.21.15.156:42030) 2024-07-16T10:07:34.631 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:34.627+0000 7fe33259c640 1 -- 172.21.15.156:0/3909187891 learned_addr learned my addr 172.21.15.156:0/3909187891 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T10:07:34.631 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:34.628+0000 7fe33259c640 1 -- 172.21.15.156:0/3909187891 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe334102d10 msgr2=0x7fe33406fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:07:34.631 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:34.628+0000 7fe33259c640 1 --2- 172.21.15.156:0/3909187891 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe334102d10 0x7fe33406fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:07:34.631 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:34.628+0000 7fe33259c640 1 -- 172.21.15.156:0/3909187891 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe328005040 con 0x7fe334102370 2024-07-16T10:07:34.631 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:34.628+0000 7fe33259c640 1 --2- 172.21.15.156:0/3909187891 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe334102370 0x7fe33406f820 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fe324002a80 tx=0x7fe324004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T10:07:34.631 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:34.628+0000 7fe322ffd640 1 -- 172.21.15.156:0/3909187891 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe324007c30 con 0x7fe334102370 2024-07-16T10:07:34.631 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:34.628+0000 7fe338a9a640 1 -- 172.21.15.156:0/3909187891 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3340706a0 con 0x7fe334102370 2024-07-16T10:07:34.631 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:34.628+0000 7fe322ffd640 1 -- 172.21.15.156:0/3909187891 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe32400f040 con 0x7fe334102370 2024-07-16T10:07:34.631 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:34.628+0000 7fe322ffd640 1 -- 172.21.15.156:0/3909187891 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe3240096b0 con 0x7fe334102370 2024-07-16T10:07:34.631 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:34.628+0000 7fe338a9a640 1 -- 172.21.15.156:0/3909187891 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe334119ab0 con 0x7fe334102370 2024-07-16T10:07:34.632 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:34.629+0000 7fe338a9a640 1 -- 172.21.15.156:0/3909187891 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3340fdae0 con 0x7fe334102370 2024-07-16T10:07:34.632 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:34.630+0000 7fe322ffd640 1 -- 172.21.15.156:0/3909187891 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 38) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fe32400d6b0 con 0x7fe334102370 2024-07-16T10:07:34.633 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:34.630+0000 7fe322ffd640 1 --2- 172.21.15.156:0/3909187891 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7fe31c06e0b0 0x7fe31c070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:07:34.633 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:34.630+0000 7fe322ffd640 1 -- 172.21.15.156:0/3909187891 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7fe32408d5d0 con 0x7fe334102370 2024-07-16T10:07:34.633 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:34.630+0000 7fe331d9b640 1 --2- 172.21.15.156:0/3909187891 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7fe31c06e0b0 0x7fe31c070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:07:34.633 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:34.631+0000 7fe331d9b640 1 --2- 172.21.15.156:0/3909187891 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7fe31c06e0b0 0x7fe31c070560 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fe328032430 tx=0x7fe328009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T10:07:34.635 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:34.634+0000 7fe322ffd640 1 -- 172.21.15.156:0/3909187891 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187716 (secure 0 0 0) 0x7fe3240568a0 con 0x7fe334102370 2024-07-16T10:07:34.850 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:34.849+0000 7fe338a9a640 1 -- 172.21.15.156:0/3909187891 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 32, "format": "json"} v 0) v1 -- 0x7fe3340fbb60 con 0x7fe334102370 2024-07-16T10:07:34.851 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:34.850+0000 7fe322ffd640 1 -- 172.21.15.156:0/3909187891 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 32, "format": "json"}]=0 dumped fsmap epoch 32 v45) v1 ==== 107+0+4147 (secure 0 0 0) 0x7fe324056030 con 0x7fe334102370 2024-07-16T10:07:34.851 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-07-16T10:07:34.851 INFO:teuthology.orchestra.run.smithi156.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34438,"name":"cephfs.smithi156.rdibys","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6827/2852996640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":2852996640},{"type":"v1","addr":"172.21.15.156:6827","nonce":2852996640}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":44269,"name":"cephfs.smithi176.wahbsi","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6827/3942496125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6826","nonce":3942496125},{"type":"v1","addr":"172.21.15.176:6827","nonce":3942496125}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":22},{"gid":44301,"name":"cephfs.smithi176.dvvbud","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6825/3749220343","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6824","nonce":3749220343},{"type":"v1","addr":"172.21.15.176:6825","nonce":3749220343}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-16T09:52:55.214975+0000","modified":"2024-07-16T10:03:03.351410+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":82,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[1],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-07-16T10:07:34.852 INFO:teuthology.orchestra.run.smithi156.stderr:dumped fsmap epoch 32 2024-07-16T10:07:34.854 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:34.852+0000 7fe338a9a640 1 -- 172.21.15.156:0/3909187891 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7fe31c06e0b0 msgr2=0x7fe31c070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:07:34.854 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:34.852+0000 7fe338a9a640 1 --2- 172.21.15.156:0/3909187891 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7fe31c06e0b0 0x7fe31c070560 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fe328032430 tx=0x7fe328009da0 comp rx=0 tx=0).stop 2024-07-16T10:07:34.854 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:34.852+0000 7fe338a9a640 1 -- 172.21.15.156:0/3909187891 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe334102370 msgr2=0x7fe33406f820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:07:34.854 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:34.852+0000 7fe338a9a640 1 --2- 172.21.15.156:0/3909187891 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe334102370 0x7fe33406f820 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fe324002a80 tx=0x7fe324004600 comp rx=0 tx=0).stop 2024-07-16T10:07:34.854 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:34.852+0000 7fe338a9a640 1 -- 172.21.15.156:0/3909187891 shutdown_connections 2024-07-16T10:07:34.854 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:34.852+0000 7fe338a9a640 1 --2- 172.21.15.156:0/3909187891 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7fe31c06e0b0 0x7fe31c070560 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:07:34.854 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:34.852+0000 7fe338a9a640 1 --2- 172.21.15.156:0/3909187891 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe334102d10 0x7fe33406fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:07:34.854 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:34.852+0000 7fe338a9a640 1 --2- 172.21.15.156:0/3909187891 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe334102370 0x7fe33406f820 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:07:34.855 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:34.852+0000 7fe338a9a640 1 -- 172.21.15.156:0/3909187891 >> 172.21.15.156:0/3909187891 conn(0x7fe3340f7f00 msgr2=0x7fe3340f8e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T10:07:34.855 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:34.852+0000 7fe338a9a640 1 -- 172.21.15.156:0/3909187891 shutdown_connections 2024-07-16T10:07:34.855 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:34.852+0000 7fe338a9a640 1 -- 172.21.15.156:0/3909187891 wait complete. 2024-07-16T10:07:35.721 DEBUG:tasks.fs:max_mds reduced in epoch 32 2024-07-16T10:07:35.721 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 32 2024-07-16T10:07:35.722 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 26c91c26-4358-11ef-bcad-c7b262605968 -- ceph fs dump --format=json 33 2024-07-16T10:07:35.843 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:07:35 smithi176 ceph-mon[81689]: pgmap v331: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-07-16T10:07:35.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:07:35 smithi176 ceph-mon[81689]: from='client.? 172.21.15.156:0/3909187891' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 32, "format": "json"}]: dispatch 2024-07-16T10:07:35.924 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:07:35 smithi156 ceph-mon[93066]: pgmap v331: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-07-16T10:07:35.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:07:35 smithi156 ceph-mon[93066]: from='client.? 172.21.15.156:0/3909187891' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 32, "format": "json"}]: dispatch 2024-07-16T10:07:35.984 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/26c91c26-4358-11ef-bcad-c7b262605968/mon.smithi156/config 2024-07-16T10:07:37.445 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:37.431+0000 7fe350031640 1 -- 172.21.15.156:0/3961369499 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe3480fdd40 msgr2=0x7fe348106380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:07:37.446 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:37.431+0000 7fe350031640 1 --2- 172.21.15.156:0/3961369499 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe3480fdd40 0x7fe348106380 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fe3380048d0 tx=0x7fe338009da0 comp rx=0 tx=0).stop 2024-07-16T10:07:37.446 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:37.431+0000 7fe350031640 1 -- 172.21.15.156:0/3961369499 shutdown_connections 2024-07-16T10:07:37.446 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:37.431+0000 7fe350031640 1 --2- 172.21.15.156:0/3961369499 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe3480fdd40 0x7fe348106380 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:07:37.446 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:37.431+0000 7fe350031640 1 --2- 172.21.15.156:0/3961369499 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe3480fd3a0 0x7fe3480fd770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:07:37.446 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:37.431+0000 7fe350031640 1 -- 172.21.15.156:0/3961369499 >> 172.21.15.156:0/3961369499 conn(0x7fe3480f6e30 msgr2=0x7fe3480f9240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T10:07:37.446 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:37.432+0000 7fe350031640 1 -- 172.21.15.156:0/3961369499 shutdown_connections 2024-07-16T10:07:37.446 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:37.432+0000 7fe350031640 1 -- 172.21.15.156:0/3961369499 wait complete. 2024-07-16T10:07:37.446 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:37.432+0000 7fe350031640 1 Processor -- start 2024-07-16T10:07:37.446 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:37.432+0000 7fe350031640 1 -- start start 2024-07-16T10:07:37.446 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:37.432+0000 7fe350031640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe3480fd3a0 0x7fe3480fb1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:07:37.446 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:37.432+0000 7fe350031640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe3480fdd40 0x7fe3480fb710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:07:37.447 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:37.432+0000 7fe350031640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3480fbc50 con 0x7fe3480fdd40 2024-07-16T10:07:37.447 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:37.432+0000 7fe350031640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3480fbdc0 con 0x7fe3480fd3a0 2024-07-16T10:07:37.447 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:37.432+0000 7fe34d5cc640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe3480fdd40 0x7fe3480fb710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:07:37.447 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:37.433+0000 7fe34d5cc640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe3480fdd40 0x7fe3480fb710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:42042/0 (socket says 172.21.15.156:42042) 2024-07-16T10:07:37.447 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:37.433+0000 7fe34d5cc640 1 -- 172.21.15.156:0/770208743 learned_addr learned my addr 172.21.15.156:0/770208743 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T10:07:37.447 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:37.433+0000 7fe34ddcd640 1 --2- 172.21.15.156:0/770208743 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe3480fd3a0 0x7fe3480fb1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:07:37.447 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:37.433+0000 7fe34d5cc640 1 -- 172.21.15.156:0/770208743 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe3480fd3a0 msgr2=0x7fe3480fb1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:07:37.447 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:37.433+0000 7fe34d5cc640 1 --2- 172.21.15.156:0/770208743 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe3480fd3a0 0x7fe3480fb1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:07:37.447 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:37.433+0000 7fe34d5cc640 1 -- 172.21.15.156:0/770208743 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe338005040 con 0x7fe3480fdd40 2024-07-16T10:07:37.447 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:37.433+0000 7fe34d5cc640 1 --2- 172.21.15.156:0/770208743 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe3480fdd40 0x7fe3480fb710 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fe338003080 tx=0x7fe338009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T10:07:37.447 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:37.433+0000 7fe33e7fc640 1 -- 172.21.15.156:0/770208743 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe338038070 con 0x7fe3480fdd40 2024-07-16T10:07:37.447 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:37.433+0000 7fe33e7fc640 1 -- 172.21.15.156:0/770208743 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe338008ee0 con 0x7fe3480fdd40 2024-07-16T10:07:37.448 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:37.433+0000 7fe350031640 1 -- 172.21.15.156:0/770208743 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3480fbff0 con 0x7fe3480fdd40 2024-07-16T10:07:37.448 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:37.433+0000 7fe33e7fc640 1 -- 172.21.15.156:0/770208743 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe33803d870 con 0x7fe3480fdd40 2024-07-16T10:07:37.448 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:37.433+0000 7fe350031640 1 -- 172.21.15.156:0/770208743 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3481197c0 con 0x7fe3480fdd40 2024-07-16T10:07:37.448 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:37.434+0000 7fe350031640 1 -- 172.21.15.156:0/770208743 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe348066160 con 0x7fe3480fdd40 2024-07-16T10:07:37.448 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:37.435+0000 7fe33e7fc640 1 -- 172.21.15.156:0/770208743 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 38) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fe338030020 con 0x7fe3480fdd40 2024-07-16T10:07:37.448 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:37.435+0000 7fe33e7fc640 1 --2- 172.21.15.156:0/770208743 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7fe33406e0b0 0x7fe334070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:07:37.448 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:37.435+0000 7fe33e7fc640 1 -- 172.21.15.156:0/770208743 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7fe3380ad8b0 con 0x7fe3480fdd40 2024-07-16T10:07:37.448 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:37.435+0000 7fe34ddcd640 1 --2- 172.21.15.156:0/770208743 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7fe33406e0b0 0x7fe334070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:07:37.448 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:37.436+0000 7fe34ddcd640 1 --2- 172.21.15.156:0/770208743 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7fe33406e0b0 0x7fe334070560 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fe3440041d0 tx=0x7fe344004330 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T10:07:37.448 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:37.440+0000 7fe33e7fc640 1 -- 172.21.15.156:0/770208743 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187716 (secure 0 0 0) 0x7fe338076c00 con 0x7fe3480fdd40 2024-07-16T10:07:37.661 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:37.658+0000 7fe350031640 1 -- 172.21.15.156:0/770208743 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 33, "format": "json"} v 0) v1 -- 0x7fe348109520 con 0x7fe3480fdd40 2024-07-16T10:07:37.662 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:37.659+0000 7fe33e7fc640 1 -- 172.21.15.156:0/770208743 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 33, "format": "json"}]=0 dumped fsmap epoch 33 v45) v1 ==== 107+0+4226 (secure 0 0 0) 0x7fe33807a2c0 con 0x7fe3480fdd40 2024-07-16T10:07:37.662 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-07-16T10:07:37.662 INFO:teuthology.orchestra.run.smithi156.stdout:{"epoch":33,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":44269,"name":"cephfs.smithi176.wahbsi","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6827/3942496125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6826","nonce":3942496125},{"type":"v1","addr":"172.21.15.176:6827","nonce":3942496125}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":22},{"gid":44301,"name":"cephfs.smithi176.dvvbud","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6825/3749220343","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6824","nonce":3749220343},{"type":"v1","addr":"172.21.15.176:6825","nonce":3749220343}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-16T09:52:55.214975+0000","modified":"2024-07-16T10:03:03.355768+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":82,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":34438},"failed":[],"damaged":[],"stopped":[1],"info":{"gid_34438":{"gid":34438,"name":"cephfs.smithi156.rdibys","rank":0,"incarnation":33,"state":"up:replay","state_seq":1,"addr":"172.21.15.156:6827/2852996640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":2852996640},{"type":"v1","addr":"172.21.15.156:6827","nonce":2852996640}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-07-16T10:07:37.663 INFO:teuthology.orchestra.run.smithi156.stderr:dumped fsmap epoch 33 2024-07-16T10:07:37.663 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:37.661+0000 7fe350031640 1 -- 172.21.15.156:0/770208743 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7fe33406e0b0 msgr2=0x7fe334070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:07:37.663 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:37.661+0000 7fe350031640 1 --2- 172.21.15.156:0/770208743 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7fe33406e0b0 0x7fe334070560 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fe3440041d0 tx=0x7fe344004330 comp rx=0 tx=0).stop 2024-07-16T10:07:37.663 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:37.661+0000 7fe350031640 1 -- 172.21.15.156:0/770208743 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe3480fdd40 msgr2=0x7fe3480fb710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:07:37.663 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:37.661+0000 7fe350031640 1 --2- 172.21.15.156:0/770208743 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe3480fdd40 0x7fe3480fb710 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fe338003080 tx=0x7fe338009da0 comp rx=0 tx=0).stop 2024-07-16T10:07:37.663 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:37.661+0000 7fe350031640 1 -- 172.21.15.156:0/770208743 shutdown_connections 2024-07-16T10:07:37.663 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:37.661+0000 7fe350031640 1 --2- 172.21.15.156:0/770208743 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7fe33406e0b0 0x7fe334070560 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:07:37.663 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:37.661+0000 7fe350031640 1 --2- 172.21.15.156:0/770208743 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe3480fdd40 0x7fe3480fb710 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:07:37.663 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:37.661+0000 7fe350031640 1 --2- 172.21.15.156:0/770208743 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe3480fd3a0 0x7fe3480fb1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:07:37.663 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:37.661+0000 7fe350031640 1 -- 172.21.15.156:0/770208743 >> 172.21.15.156:0/770208743 conn(0x7fe3480f6e30 msgr2=0x7fe3480f8230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T10:07:37.663 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:37.661+0000 7fe350031640 1 -- 172.21.15.156:0/770208743 shutdown_connections 2024-07-16T10:07:37.664 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:37.661+0000 7fe350031640 1 -- 172.21.15.156:0/770208743 wait complete. 2024-07-16T10:07:37.843 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:07:37 smithi176 ceph-mon[81689]: pgmap v332: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:07:37.924 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:07:37 smithi156 ceph-mon[93066]: pgmap v332: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:07:38.342 DEBUG:tasks.fs:max_mds reduced in epoch 33 2024-07-16T10:07:38.342 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 33 2024-07-16T10:07:38.342 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 26c91c26-4358-11ef-bcad-c7b262605968 -- ceph fs dump --format=json 34 2024-07-16T10:07:38.601 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/26c91c26-4358-11ef-bcad-c7b262605968/mon.smithi156/config 2024-07-16T10:07:38.836 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:07:38 smithi156 ceph-mon[93066]: from='client.? 172.21.15.156:0/770208743' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 33, "format": "json"}]: dispatch 2024-07-16T10:07:38.843 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:07:38 smithi176 ceph-mon[81689]: from='client.? 172.21.15.156:0/770208743' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 33, "format": "json"}]: dispatch 2024-07-16T10:07:39.843 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:07:39 smithi176 ceph-mon[81689]: pgmap v333: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-07-16T10:07:39.880 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:07:39 smithi156 ceph-mon[93066]: pgmap v333: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-07-16T10:07:40.114 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:40.112+0000 7fd7284d4640 1 -- 172.21.15.156:0/4097748408 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd7201023a0 msgr2=0x7fd720102770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:07:40.115 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:40.112+0000 7fd7284d4640 1 --2- 172.21.15.156:0/4097748408 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd7201023a0 0x7fd720102770 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fd71c0048d0 tx=0x7fd71c009da0 comp rx=0 tx=0).stop 2024-07-16T10:07:40.115 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:40.112+0000 7fd7284d4640 1 -- 172.21.15.156:0/4097748408 shutdown_connections 2024-07-16T10:07:40.115 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:40.112+0000 7fd7284d4640 1 --2- 172.21.15.156:0/4097748408 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd720102d40 0x7fd7200faa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:07:40.115 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:40.112+0000 7fd7284d4640 1 --2- 172.21.15.156:0/4097748408 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd7201023a0 0x7fd720102770 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:07:40.115 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:40.112+0000 7fd7284d4640 1 -- 172.21.15.156:0/4097748408 >> 172.21.15.156:0/4097748408 conn(0x7fd720070af0 msgr2=0x7fd720070ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T10:07:40.115 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:40.112+0000 7fd7284d4640 1 -- 172.21.15.156:0/4097748408 shutdown_connections 2024-07-16T10:07:40.115 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:40.112+0000 7fd7284d4640 1 -- 172.21.15.156:0/4097748408 wait complete. 2024-07-16T10:07:40.115 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:40.113+0000 7fd7284d4640 1 Processor -- start 2024-07-16T10:07:40.115 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:40.113+0000 7fd7284d4640 1 -- start start 2024-07-16T10:07:40.115 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:40.113+0000 7fd7284d4640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd7201023a0 0x7fd720113190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:07:40.115 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:40.113+0000 7fd7284d4640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd720102d40 0x7fd7201136d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:07:40.115 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:40.113+0000 7fd7284d4640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd720113cd0 con 0x7fd7201023a0 2024-07-16T10:07:40.116 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:40.113+0000 7fd7284d4640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd72010d110 con 0x7fd720102d40 2024-07-16T10:07:40.116 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:40.113+0000 7fd726270640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd7201023a0 0x7fd720113190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:07:40.116 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:40.113+0000 7fd726270640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd7201023a0 0x7fd720113190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:42066/0 (socket says 172.21.15.156:42066) 2024-07-16T10:07:40.116 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:40.113+0000 7fd726270640 1 -- 172.21.15.156:0/168799065 learned_addr learned my addr 172.21.15.156:0/168799065 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T10:07:40.116 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:40.113+0000 7fd725a6f640 1 --2- 172.21.15.156:0/168799065 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd720102d40 0x7fd7201136d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:07:40.116 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:40.113+0000 7fd726270640 1 -- 172.21.15.156:0/168799065 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd720102d40 msgr2=0x7fd7201136d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:07:40.116 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:40.113+0000 7fd726270640 1 --2- 172.21.15.156:0/168799065 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd720102d40 0x7fd7201136d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:07:40.116 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:40.113+0000 7fd726270640 1 -- 172.21.15.156:0/168799065 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd71c005040 con 0x7fd7201023a0 2024-07-16T10:07:40.116 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:40.113+0000 7fd725a6f640 1 --2- 172.21.15.156:0/168799065 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd720102d40 0x7fd7201136d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-16T10:07:40.116 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:40.114+0000 7fd726270640 1 --2- 172.21.15.156:0/168799065 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd7201023a0 0x7fd720113190 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fd71c0043d0 tx=0x7fd71c009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T10:07:40.117 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:40.114+0000 7fd716ffd640 1 -- 172.21.15.156:0/168799065 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd71c038070 con 0x7fd7201023a0 2024-07-16T10:07:40.118 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:40.114+0000 7fd716ffd640 1 -- 172.21.15.156:0/168799065 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd71c02abd0 con 0x7fd7201023a0 2024-07-16T10:07:40.118 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:40.114+0000 7fd7284d4640 1 -- 172.21.15.156:0/168799065 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd72010d390 con 0x7fd7201023a0 2024-07-16T10:07:40.118 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:40.114+0000 7fd716ffd640 1 -- 172.21.15.156:0/168799065 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd71c033640 con 0x7fd7201023a0 2024-07-16T10:07:40.118 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:40.114+0000 7fd7284d4640 1 -- 172.21.15.156:0/168799065 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd72010d7f0 con 0x7fd7201023a0 2024-07-16T10:07:40.118 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:40.115+0000 7fd7284d4640 1 -- 172.21.15.156:0/168799065 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7200655c0 con 0x7fd7201023a0 2024-07-16T10:07:40.118 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:40.115+0000 7fd716ffd640 1 -- 172.21.15.156:0/168799065 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 38) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fd71c030020 con 0x7fd7201023a0 2024-07-16T10:07:40.118 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:40.116+0000 7fd716ffd640 1 --2- 172.21.15.156:0/168799065 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7fd70c06e0b0 0x7fd70c070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:07:40.118 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:40.116+0000 7fd725a6f640 1 --2- 172.21.15.156:0/168799065 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7fd70c06e0b0 0x7fd70c070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:07:40.118 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:40.116+0000 7fd716ffd640 1 -- 172.21.15.156:0/168799065 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7fd71c0ada20 con 0x7fd7201023a0 2024-07-16T10:07:40.118 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:40.116+0000 7fd725a6f640 1 --2- 172.21.15.156:0/168799065 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7fd70c06e0b0 0x7fd70c070560 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fd710009fd0 tx=0x7fd710009e50 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T10:07:40.121 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:40.120+0000 7fd716ffd640 1 -- 172.21.15.156:0/168799065 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187716 (secure 0 0 0) 0x7fd71c076cf0 con 0x7fd7201023a0 2024-07-16T10:07:40.339 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:40.337+0000 7fd7284d4640 1 -- 172.21.15.156:0/168799065 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 34, "format": "json"} v 0) v1 -- 0x7fd72004b530 con 0x7fd7201023a0 2024-07-16T10:07:40.339 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:40.338+0000 7fd716ffd640 1 -- 172.21.15.156:0/168799065 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 34, "format": "json"}]=0 dumped fsmap epoch 34 v45) v1 ==== 107+0+4229 (secure 0 0 0) 0x7fd71c07a3b0 con 0x7fd7201023a0 2024-07-16T10:07:40.340 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-07-16T10:07:40.340 INFO:teuthology.orchestra.run.smithi156.stdout:{"epoch":34,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":44269,"name":"cephfs.smithi176.wahbsi","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6827/3942496125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6826","nonce":3942496125},{"type":"v1","addr":"172.21.15.176:6827","nonce":3942496125}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":22},{"gid":44301,"name":"cephfs.smithi176.dvvbud","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6825/3749220343","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6824","nonce":3749220343},{"type":"v1","addr":"172.21.15.176:6825","nonce":3749220343}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-16T09:52:55.214975+0000","modified":"2024-07-16T10:03:08.411783+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":82,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":34438},"failed":[],"damaged":[],"stopped":[1],"info":{"gid_34438":{"gid":34438,"name":"cephfs.smithi156.rdibys","rank":0,"incarnation":33,"state":"up:reconnect","state_seq":7,"addr":"172.21.15.156:6827/2852996640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":2852996640},{"type":"v1","addr":"172.21.15.156:6827","nonce":2852996640}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-07-16T10:07:40.340 INFO:teuthology.orchestra.run.smithi156.stderr:dumped fsmap epoch 34 2024-07-16T10:07:40.342 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:40.340+0000 7fd7284d4640 1 -- 172.21.15.156:0/168799065 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7fd70c06e0b0 msgr2=0x7fd70c070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:07:40.342 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:40.340+0000 7fd7284d4640 1 --2- 172.21.15.156:0/168799065 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7fd70c06e0b0 0x7fd70c070560 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fd710009fd0 tx=0x7fd710009e50 comp rx=0 tx=0).stop 2024-07-16T10:07:40.342 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:40.340+0000 7fd7284d4640 1 -- 172.21.15.156:0/168799065 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd7201023a0 msgr2=0x7fd720113190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:07:40.342 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:40.340+0000 7fd7284d4640 1 --2- 172.21.15.156:0/168799065 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd7201023a0 0x7fd720113190 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fd71c0043d0 tx=0x7fd71c009da0 comp rx=0 tx=0).stop 2024-07-16T10:07:40.342 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:40.340+0000 7fd7284d4640 1 -- 172.21.15.156:0/168799065 shutdown_connections 2024-07-16T10:07:40.342 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:40.340+0000 7fd7284d4640 1 --2- 172.21.15.156:0/168799065 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7fd70c06e0b0 0x7fd70c070560 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:07:40.342 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:40.340+0000 7fd7284d4640 1 --2- 172.21.15.156:0/168799065 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd720102d40 0x7fd7201136d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:07:40.342 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:40.340+0000 7fd7284d4640 1 --2- 172.21.15.156:0/168799065 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd7201023a0 0x7fd720113190 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:07:40.342 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:40.340+0000 7fd7284d4640 1 -- 172.21.15.156:0/168799065 >> 172.21.15.156:0/168799065 conn(0x7fd720070af0 msgr2=0x7fd720105160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T10:07:40.342 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:40.341+0000 7fd7284d4640 1 -- 172.21.15.156:0/168799065 shutdown_connections 2024-07-16T10:07:40.342 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:40.341+0000 7fd7284d4640 1 -- 172.21.15.156:0/168799065 wait complete. 2024-07-16T10:07:40.732 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:07:40 smithi156 ceph-mon[93066]: from='client.? 172.21.15.156:0/168799065' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 34, "format": "json"}]: dispatch 2024-07-16T10:07:41.040 DEBUG:tasks.fs:max_mds reduced in epoch 34 2024-07-16T10:07:41.040 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 34 2024-07-16T10:07:41.040 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 26c91c26-4358-11ef-bcad-c7b262605968 -- ceph fs dump --format=json 35 2024-07-16T10:07:41.093 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:07:40 smithi176 ceph-mon[81689]: from='client.? 172.21.15.156:0/168799065' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 34, "format": "json"}]: dispatch 2024-07-16T10:07:41.307 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/26c91c26-4358-11ef-bcad-c7b262605968/mon.smithi156/config 2024-07-16T10:07:41.667 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:07:41 smithi156 ceph-mon[93066]: pgmap v334: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:07:42.093 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:07:41 smithi176 ceph-mon[81689]: pgmap v334: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:07:42.895 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:42.894+0000 7f4b1d7a2640 1 -- 172.21.15.156:0/540694628 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4b180fdd60 msgr2=0x7f4b181063a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:07:42.895 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:42.894+0000 7f4b1d7a2640 1 --2- 172.21.15.156:0/540694628 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4b180fdd60 0x7f4b181063a0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f4b00004900 tx=0x7f4b00009da0 comp rx=0 tx=0).stop 2024-07-16T10:07:42.895 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:42.894+0000 7f4b1d7a2640 1 -- 172.21.15.156:0/540694628 shutdown_connections 2024-07-16T10:07:42.896 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:42.894+0000 7f4b1d7a2640 1 --2- 172.21.15.156:0/540694628 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4b180fdd60 0x7f4b181063a0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:07:42.896 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:42.894+0000 7f4b1d7a2640 1 --2- 172.21.15.156:0/540694628 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4b180fd3c0 0x7f4b180fd790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:07:42.896 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:42.894+0000 7f4b1d7a2640 1 -- 172.21.15.156:0/540694628 >> 172.21.15.156:0/540694628 conn(0x7f4b180f6e10 msgr2=0x7f4b180f9220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T10:07:42.896 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:42.894+0000 7f4b1d7a2640 1 -- 172.21.15.156:0/540694628 shutdown_connections 2024-07-16T10:07:42.896 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:42.894+0000 7f4b1d7a2640 1 -- 172.21.15.156:0/540694628 wait complete. 2024-07-16T10:07:42.898 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:42.894+0000 7f4b1d7a2640 1 Processor -- start 2024-07-16T10:07:42.898 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:42.895+0000 7f4b1d7a2640 1 -- start start 2024-07-16T10:07:42.898 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:42.895+0000 7f4b1d7a2640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4b180fd3c0 0x7f4b181131b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:07:42.898 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:42.895+0000 7f4b1d7a2640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4b180fdd60 0x7f4b181136f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:07:42.898 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:42.895+0000 7f4b1d7a2640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b18113cf0 con 0x7f4b180fdd60 2024-07-16T10:07:42.898 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:42.895+0000 7f4b1d7a2640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b1810d130 con 0x7f4b180fd3c0 2024-07-16T10:07:42.898 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:42.895+0000 7f4b167fc640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4b180fdd60 0x7f4b181136f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:07:42.898 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:42.895+0000 7f4b167fc640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4b180fdd60 0x7f4b181136f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:44330/0 (socket says 172.21.15.156:44330) 2024-07-16T10:07:42.898 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:42.895+0000 7f4b167fc640 1 -- 172.21.15.156:0/2191698459 learned_addr learned my addr 172.21.15.156:0/2191698459 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T10:07:42.898 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:42.895+0000 7f4b16ffd640 1 --2- 172.21.15.156:0/2191698459 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4b180fd3c0 0x7f4b181131b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:07:42.898 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:42.895+0000 7f4b167fc640 1 -- 172.21.15.156:0/2191698459 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4b180fd3c0 msgr2=0x7f4b181131b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:07:42.899 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:42.895+0000 7f4b167fc640 1 --2- 172.21.15.156:0/2191698459 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4b180fd3c0 0x7f4b181131b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:07:42.899 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:42.895+0000 7f4b167fc640 1 -- 172.21.15.156:0/2191698459 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b00005040 con 0x7f4b180fdd60 2024-07-16T10:07:42.899 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:42.895+0000 7f4b167fc640 1 --2- 172.21.15.156:0/2191698459 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4b180fdd60 0x7f4b181136f0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f4b000043d0 tx=0x7f4b00009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T10:07:42.899 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:42.896+0000 7f4b0f7fe640 1 -- 172.21.15.156:0/2191698459 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4b00038070 con 0x7f4b180fdd60 2024-07-16T10:07:42.899 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:42.896+0000 7f4b0f7fe640 1 -- 172.21.15.156:0/2191698459 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4b0002abc0 con 0x7f4b180fdd60 2024-07-16T10:07:42.899 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:42.896+0000 7f4b1d7a2640 1 -- 172.21.15.156:0/2191698459 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b1810d3b0 con 0x7f4b180fdd60 2024-07-16T10:07:42.899 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:42.896+0000 7f4b0f7fe640 1 -- 172.21.15.156:0/2191698459 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4b0003c630 con 0x7f4b180fdd60 2024-07-16T10:07:42.899 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:42.896+0000 7f4b1d7a2640 1 -- 172.21.15.156:0/2191698459 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b1810d870 con 0x7f4b180fdd60 2024-07-16T10:07:42.899 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:42.897+0000 7f4b1d7a2640 1 -- 172.21.15.156:0/2191698459 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b18066100 con 0x7f4b180fdd60 2024-07-16T10:07:42.900 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:42.897+0000 7f4b0f7fe640 1 -- 172.21.15.156:0/2191698459 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 38) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f4b00030020 con 0x7f4b180fdd60 2024-07-16T10:07:42.900 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:42.898+0000 7f4b0f7fe640 1 --2- 172.21.15.156:0/2191698459 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f4b0406e0b0 0x7f4b04070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:07:42.900 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:42.898+0000 7f4b0f7fe640 1 -- 172.21.15.156:0/2191698459 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f4b000ad920 con 0x7f4b180fdd60 2024-07-16T10:07:42.900 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:42.898+0000 7f4b16ffd640 1 --2- 172.21.15.156:0/2191698459 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f4b0406e0b0 0x7f4b04070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:07:42.900 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:42.898+0000 7f4b16ffd640 1 --2- 172.21.15.156:0/2191698459 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f4b0406e0b0 0x7f4b04070560 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f4b10004fd0 tx=0x7f4b10009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T10:07:42.903 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:42.902+0000 7f4b0f7fe640 1 -- 172.21.15.156:0/2191698459 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187716 (secure 0 0 0) 0x7f4b00076cc0 con 0x7f4b180fdd60 2024-07-16T10:07:43.118 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:43.116+0000 7f4b1d7a2640 1 -- 172.21.15.156:0/2191698459 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 35, "format": "json"} v 0) v1 -- 0x7f4b18109480 con 0x7f4b180fdd60 2024-07-16T10:07:43.118 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:43.117+0000 7f4b0f7fe640 1 -- 172.21.15.156:0/2191698459 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 35, "format": "json"}]=0 dumped fsmap epoch 35 v45) v1 ==== 107+0+4226 (secure 0 0 0) 0x7f4b0007a380 con 0x7f4b180fdd60 2024-07-16T10:07:43.119 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-07-16T10:07:43.119 INFO:teuthology.orchestra.run.smithi156.stdout:{"epoch":35,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":44269,"name":"cephfs.smithi176.wahbsi","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6827/3942496125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6826","nonce":3942496125},{"type":"v1","addr":"172.21.15.176:6827","nonce":3942496125}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":22},{"gid":44301,"name":"cephfs.smithi176.dvvbud","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6825/3749220343","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6824","nonce":3749220343},{"type":"v1","addr":"172.21.15.176:6825","nonce":3749220343}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-16T09:52:55.214975+0000","modified":"2024-07-16T10:03:09.006790+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":82,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":34438},"failed":[],"damaged":[],"stopped":[1],"info":{"gid_34438":{"gid":34438,"name":"cephfs.smithi156.rdibys","rank":0,"incarnation":33,"state":"up:rejoin","state_seq":8,"addr":"172.21.15.156:6827/2852996640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":2852996640},{"type":"v1","addr":"172.21.15.156:6827","nonce":2852996640}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-07-16T10:07:43.119 INFO:teuthology.orchestra.run.smithi156.stderr:dumped fsmap epoch 35 2024-07-16T10:07:43.121 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:43.119+0000 7f4b1d7a2640 1 -- 172.21.15.156:0/2191698459 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f4b0406e0b0 msgr2=0x7f4b04070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:07:43.121 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:43.119+0000 7f4b1d7a2640 1 --2- 172.21.15.156:0/2191698459 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f4b0406e0b0 0x7f4b04070560 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f4b10004fd0 tx=0x7f4b10009040 comp rx=0 tx=0).stop 2024-07-16T10:07:43.121 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:43.119+0000 7f4b1d7a2640 1 -- 172.21.15.156:0/2191698459 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4b180fdd60 msgr2=0x7f4b181136f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:07:43.121 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:43.119+0000 7f4b1d7a2640 1 --2- 172.21.15.156:0/2191698459 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4b180fdd60 0x7f4b181136f0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f4b000043d0 tx=0x7f4b00009da0 comp rx=0 tx=0).stop 2024-07-16T10:07:43.121 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:43.119+0000 7f4b1d7a2640 1 -- 172.21.15.156:0/2191698459 shutdown_connections 2024-07-16T10:07:43.121 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:43.119+0000 7f4b1d7a2640 1 --2- 172.21.15.156:0/2191698459 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f4b0406e0b0 0x7f4b04070560 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:07:43.121 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:43.119+0000 7f4b1d7a2640 1 --2- 172.21.15.156:0/2191698459 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4b180fdd60 0x7f4b181136f0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:07:43.121 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:43.119+0000 7f4b1d7a2640 1 --2- 172.21.15.156:0/2191698459 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4b180fd3c0 0x7f4b181131b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:07:43.121 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:43.119+0000 7f4b1d7a2640 1 -- 172.21.15.156:0/2191698459 >> 172.21.15.156:0/2191698459 conn(0x7f4b180f6e10 msgr2=0x7f4b180f82c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T10:07:43.121 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:43.119+0000 7f4b1d7a2640 1 -- 172.21.15.156:0/2191698459 shutdown_connections 2024-07-16T10:07:43.121 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:43.119+0000 7f4b1d7a2640 1 -- 172.21.15.156:0/2191698459 wait complete. 2024-07-16T10:07:43.745 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:07:43 smithi156 ceph-mon[93066]: pgmap v335: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:07:43.745 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:07:43 smithi156 ceph-mon[93066]: from='client.? 172.21.15.156:0/2191698459' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 35, "format": "json"}]: dispatch 2024-07-16T10:07:43.772 DEBUG:tasks.fs:max_mds reduced in epoch 35 2024-07-16T10:07:43.772 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 35 2024-07-16T10:07:43.772 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 26c91c26-4358-11ef-bcad-c7b262605968 -- ceph fs dump --format=json 36 2024-07-16T10:07:44.030 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/26c91c26-4358-11ef-bcad-c7b262605968/mon.smithi156/config 2024-07-16T10:07:44.093 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:07:43 smithi176 ceph-mon[81689]: pgmap v335: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:07:44.093 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:07:43 smithi176 ceph-mon[81689]: from='client.? 172.21.15.156:0/2191698459' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 35, "format": "json"}]: dispatch 2024-07-16T10:07:45.602 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:45.601+0000 7f7c6a448640 1 -- 172.21.15.156:0/1258508331 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7c64102d10 msgr2=0x7f7c640fc3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:07:45.602 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:45.601+0000 7f7c6a448640 1 --2- 172.21.15.156:0/1258508331 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7c64102d10 0x7f7c640fc3a0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f7c580048d0 tx=0x7f7c58009da0 comp rx=0 tx=0).stop 2024-07-16T10:07:45.602 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:45.601+0000 7f7c6a448640 1 -- 172.21.15.156:0/1258508331 shutdown_connections 2024-07-16T10:07:45.603 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:45.601+0000 7f7c6a448640 1 --2- 172.21.15.156:0/1258508331 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7c64102d10 0x7f7c640fc3a0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:07:45.603 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:45.601+0000 7f7c6a448640 1 --2- 172.21.15.156:0/1258508331 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7c64102370 0x7f7c64102740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:07:45.603 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:45.601+0000 7f7c6a448640 1 -- 172.21.15.156:0/1258508331 >> 172.21.15.156:0/1258508331 conn(0x7f7c640f7f00 msgr2=0x7f7c640fa310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T10:07:45.603 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:45.601+0000 7f7c6a448640 1 -- 172.21.15.156:0/1258508331 shutdown_connections 2024-07-16T10:07:45.603 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:45.601+0000 7f7c6a448640 1 -- 172.21.15.156:0/1258508331 wait complete. 2024-07-16T10:07:45.603 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:45.601+0000 7f7c6a448640 1 Processor -- start 2024-07-16T10:07:45.604 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:45.602+0000 7f7c6a448640 1 -- start start 2024-07-16T10:07:45.604 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:45.602+0000 7f7c6a448640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7c64102370 0x7f7c641155e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:07:45.604 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:45.602+0000 7f7c6a448640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7c64102d10 0x7f7c64115b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:07:45.604 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:45.602+0000 7f7c6a448640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c64116190 con 0x7f7c64102d10 2024-07-16T10:07:45.604 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:45.602+0000 7f7c6a448640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c6410f580 con 0x7f7c64102370 2024-07-16T10:07:45.605 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:45.602+0000 7f7c637fe640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7c64102d10 0x7f7c64115b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:07:45.605 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:45.602+0000 7f7c637fe640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7c64102d10 0x7f7c64115b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:44350/0 (socket says 172.21.15.156:44350) 2024-07-16T10:07:45.605 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:45.602+0000 7f7c637fe640 1 -- 172.21.15.156:0/3476196173 learned_addr learned my addr 172.21.15.156:0/3476196173 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T10:07:45.605 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:45.602+0000 7f7c63fff640 1 --2- 172.21.15.156:0/3476196173 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7c64102370 0x7f7c641155e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:07:45.605 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:45.602+0000 7f7c637fe640 1 -- 172.21.15.156:0/3476196173 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7c64102370 msgr2=0x7f7c641155e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:07:45.605 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:45.602+0000 7f7c637fe640 1 --2- 172.21.15.156:0/3476196173 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7c64102370 0x7f7c641155e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:07:45.605 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:45.602+0000 7f7c637fe640 1 -- 172.21.15.156:0/3476196173 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c58005040 con 0x7f7c64102d10 2024-07-16T10:07:45.605 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:45.603+0000 7f7c637fe640 1 --2- 172.21.15.156:0/3476196173 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7c64102d10 0x7f7c64115b40 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f7c58003080 tx=0x7f7c58009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T10:07:45.606 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:45.604+0000 7f7c60ff9640 1 -- 172.21.15.156:0/3476196173 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7c58038070 con 0x7f7c64102d10 2024-07-16T10:07:45.606 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:45.604+0000 7f7c60ff9640 1 -- 172.21.15.156:0/3476196173 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7c58032970 con 0x7f7c64102d10 2024-07-16T10:07:45.606 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:45.604+0000 7f7c6a448640 1 -- 172.21.15.156:0/3476196173 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c6410f800 con 0x7f7c64102d10 2024-07-16T10:07:45.606 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:45.604+0000 7f7c60ff9640 1 -- 172.21.15.156:0/3476196173 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7c5803d960 con 0x7f7c64102d10 2024-07-16T10:07:45.606 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:45.604+0000 7f7c6a448640 1 -- 172.21.15.156:0/3476196173 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c6410fcc0 con 0x7f7c64102d10 2024-07-16T10:07:45.607 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:45.605+0000 7f7c6a448640 1 -- 172.21.15.156:0/3476196173 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c640fdae0 con 0x7f7c64102d10 2024-07-16T10:07:45.607 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:45.605+0000 7f7c60ff9640 1 -- 172.21.15.156:0/3476196173 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 38) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f7c58030020 con 0x7f7c64102d10 2024-07-16T10:07:45.607 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:45.606+0000 7f7c60ff9640 1 --2- 172.21.15.156:0/3476196173 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f7c4c06e0b0 0x7f7c4c070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:07:45.607 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:45.606+0000 7f7c60ff9640 1 -- 172.21.15.156:0/3476196173 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f7c580adb90 con 0x7f7c64102d10 2024-07-16T10:07:45.608 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:45.606+0000 7f7c63fff640 1 --2- 172.21.15.156:0/3476196173 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f7c4c06e0b0 0x7f7c4c070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:07:45.608 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:45.606+0000 7f7c63fff640 1 --2- 172.21.15.156:0/3476196173 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f7c4c06e0b0 0x7f7c4c070560 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f7c640fb760 tx=0x7f7c5400b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T10:07:45.611 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:45.610+0000 7f7c60ff9640 1 -- 172.21.15.156:0/3476196173 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187716 (secure 0 0 0) 0x7f7c58076e60 con 0x7f7c64102d10 2024-07-16T10:07:45.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:07:45 smithi156 ceph-mon[93066]: pgmap v336: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-07-16T10:07:45.825 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:45.824+0000 7f7c6a448640 1 -- 172.21.15.156:0/3476196173 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 36, "format": "json"} v 0) v1 -- 0x7f7c640fbb60 con 0x7f7c64102d10 2024-07-16T10:07:45.826 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:45.824+0000 7f7c60ff9640 1 -- 172.21.15.156:0/3476196173 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 36, "format": "json"}]=0 dumped fsmap epoch 36 v45) v1 ==== 107+0+4235 (secure 0 0 0) 0x7f7c5807a520 con 0x7f7c64102d10 2024-07-16T10:07:45.826 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-07-16T10:07:45.826 INFO:teuthology.orchestra.run.smithi156.stdout:{"epoch":36,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":44269,"name":"cephfs.smithi176.wahbsi","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6827/3942496125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6826","nonce":3942496125},{"type":"v1","addr":"172.21.15.176:6827","nonce":3942496125}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":22},{"gid":44301,"name":"cephfs.smithi176.dvvbud","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6825/3749220343","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6824","nonce":3749220343},{"type":"v1","addr":"172.21.15.176:6825","nonce":3749220343}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-16T09:52:55.214975+0000","modified":"2024-07-16T10:03:11.011991+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":82,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":34438},"failed":[],"damaged":[],"stopped":[1],"info":{"gid_34438":{"gid":34438,"name":"cephfs.smithi156.rdibys","rank":0,"incarnation":33,"state":"up:active","state_seq":9,"addr":"172.21.15.156:6827/2852996640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":2852996640},{"type":"v1","addr":"172.21.15.156:6827","nonce":2852996640}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":34438,"qdb_cluster":[34438]},"id":1}]} 2024-07-16T10:07:45.826 INFO:teuthology.orchestra.run.smithi156.stderr:dumped fsmap epoch 36 2024-07-16T10:07:45.828 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:45.827+0000 7f7c6a448640 1 -- 172.21.15.156:0/3476196173 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f7c4c06e0b0 msgr2=0x7f7c4c070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:07:45.828 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:45.827+0000 7f7c6a448640 1 --2- 172.21.15.156:0/3476196173 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f7c4c06e0b0 0x7f7c4c070560 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f7c640fb760 tx=0x7f7c5400b040 comp rx=0 tx=0).stop 2024-07-16T10:07:45.828 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:45.827+0000 7f7c6a448640 1 -- 172.21.15.156:0/3476196173 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7c64102d10 msgr2=0x7f7c64115b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:07:45.828 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:45.827+0000 7f7c6a448640 1 --2- 172.21.15.156:0/3476196173 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7c64102d10 0x7f7c64115b40 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f7c58003080 tx=0x7f7c58009da0 comp rx=0 tx=0).stop 2024-07-16T10:07:45.829 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:45.827+0000 7f7c6a448640 1 -- 172.21.15.156:0/3476196173 shutdown_connections 2024-07-16T10:07:45.829 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:45.827+0000 7f7c6a448640 1 --2- 172.21.15.156:0/3476196173 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f7c4c06e0b0 0x7f7c4c070560 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:07:45.829 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:45.827+0000 7f7c6a448640 1 --2- 172.21.15.156:0/3476196173 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7c64102d10 0x7f7c64115b40 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:07:45.829 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:45.827+0000 7f7c6a448640 1 --2- 172.21.15.156:0/3476196173 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7c64102370 0x7f7c641155e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:07:45.829 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:45.827+0000 7f7c6a448640 1 -- 172.21.15.156:0/3476196173 >> 172.21.15.156:0/3476196173 conn(0x7f7c640f7f00 msgr2=0x7f7c64101750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T10:07:45.829 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:45.827+0000 7f7c6a448640 1 -- 172.21.15.156:0/3476196173 shutdown_connections 2024-07-16T10:07:45.829 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:45.827+0000 7f7c6a448640 1 -- 172.21.15.156:0/3476196173 wait complete. 2024-07-16T10:07:46.093 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:07:45 smithi176 ceph-mon[81689]: pgmap v336: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-07-16T10:07:46.519 DEBUG:tasks.fs:max_mds reduced in epoch 36 2024-07-16T10:07:46.519 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 36 2024-07-16T10:07:46.519 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 26c91c26-4358-11ef-bcad-c7b262605968 -- ceph fs dump --format=json 37 2024-07-16T10:07:46.780 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/26c91c26-4358-11ef-bcad-c7b262605968/mon.smithi156/config 2024-07-16T10:07:46.903 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:07:46 smithi156 ceph-mon[93066]: from='client.? 172.21.15.156:0/3476196173' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 36, "format": "json"}]: dispatch 2024-07-16T10:07:47.093 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:07:46 smithi176 ceph-mon[81689]: from='client.? 172.21.15.156:0/3476196173' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 36, "format": "json"}]: dispatch 2024-07-16T10:07:47.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:07:47 smithi156 ceph-mon[93066]: pgmap v337: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:07:48.093 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:07:47 smithi176 ceph-mon[81689]: pgmap v337: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:07:48.270 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:48.264+0000 7f702cdd9640 1 -- 172.21.15.156:0/2374737161 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7028102380 msgr2=0x7f7028102750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:07:48.270 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:48.264+0000 7f702cdd9640 1 --2- 172.21.15.156:0/2374737161 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7028102380 0x7f7028102750 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f7018009c50 tx=0x7f7018004d10 comp rx=0 tx=0).stop 2024-07-16T10:07:48.270 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:48.264+0000 7f702cdd9640 1 -- 172.21.15.156:0/2374737161 shutdown_connections 2024-07-16T10:07:48.270 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:48.264+0000 7f702cdd9640 1 --2- 172.21.15.156:0/2374737161 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7028102d20 0x7f70280fc3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:07:48.271 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:48.264+0000 7f702cdd9640 1 --2- 172.21.15.156:0/2374737161 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7028102380 0x7f7028102750 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:07:48.271 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:48.264+0000 7f702cdd9640 1 -- 172.21.15.156:0/2374737161 >> 172.21.15.156:0/2374737161 conn(0x7f70280f7f10 msgr2=0x7f70280fa320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T10:07:48.271 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:48.264+0000 7f702cdd9640 1 -- 172.21.15.156:0/2374737161 shutdown_connections 2024-07-16T10:07:48.271 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:48.264+0000 7f702cdd9640 1 -- 172.21.15.156:0/2374737161 wait complete. 2024-07-16T10:07:48.271 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:48.264+0000 7f702cdd9640 1 Processor -- start 2024-07-16T10:07:48.271 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:48.264+0000 7f702cdd9640 1 -- start start 2024-07-16T10:07:48.271 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:48.265+0000 7f702cdd9640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7028102380 0x7f70281130b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:07:48.271 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:48.265+0000 7f702cdd9640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7028102d20 0x7f70281135f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:07:48.271 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:48.265+0000 7f702cdd9640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7028113bf0 con 0x7f7028102d20 2024-07-16T10:07:48.271 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:48.265+0000 7f702cdd9640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f702810d030 con 0x7f7028102380 2024-07-16T10:07:48.271 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:48.265+0000 7f7025d9b640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7028102d20 0x7f70281135f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:07:48.271 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:48.265+0000 7f7025d9b640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7028102d20 0x7f70281135f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:44362/0 (socket says 172.21.15.156:44362) 2024-07-16T10:07:48.271 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:48.265+0000 7f7025d9b640 1 -- 172.21.15.156:0/1834980031 learned_addr learned my addr 172.21.15.156:0/1834980031 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T10:07:48.272 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:48.265+0000 7f702659c640 1 --2- 172.21.15.156:0/1834980031 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7028102380 0x7f70281130b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:07:48.272 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:48.265+0000 7f7025d9b640 1 -- 172.21.15.156:0/1834980031 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7028102380 msgr2=0x7f70281130b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:07:48.272 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:48.265+0000 7f7025d9b640 1 --2- 172.21.15.156:0/1834980031 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7028102380 0x7f70281130b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:07:48.272 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:48.265+0000 7f7025d9b640 1 -- 172.21.15.156:0/1834980031 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70180098b0 con 0x7f7028102d20 2024-07-16T10:07:48.272 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:48.265+0000 7f702659c640 1 --2- 172.21.15.156:0/1834980031 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7028102380 0x7f70281130b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-16T10:07:48.272 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:48.265+0000 7f7025d9b640 1 --2- 172.21.15.156:0/1834980031 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7028102d20 0x7f70281135f0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f7010002a50 tx=0x7f7010004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T10:07:48.272 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:48.265+0000 7f701effd640 1 -- 172.21.15.156:0/1834980031 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7010007c30 con 0x7f7028102d20 2024-07-16T10:07:48.272 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:48.265+0000 7f701effd640 1 -- 172.21.15.156:0/1834980031 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f701000f040 con 0x7f7028102d20 2024-07-16T10:07:48.272 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:48.265+0000 7f702cdd9640 1 -- 172.21.15.156:0/1834980031 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f702810d310 con 0x7f7028102d20 2024-07-16T10:07:48.272 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:48.265+0000 7f701effd640 1 -- 172.21.15.156:0/1834980031 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f701000db50 con 0x7f7028102d20 2024-07-16T10:07:48.272 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:48.265+0000 7f702cdd9640 1 -- 172.21.15.156:0/1834980031 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f702810d830 con 0x7f7028102d20 2024-07-16T10:07:48.272 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:48.266+0000 7f702cdd9640 1 -- 172.21.15.156:0/1834980031 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70280fdaf0 con 0x7f7028102d20 2024-07-16T10:07:48.272 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:48.267+0000 7f701effd640 1 -- 172.21.15.156:0/1834980031 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 38) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f701000dcf0 con 0x7f7028102d20 2024-07-16T10:07:48.273 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:48.267+0000 7f701effd640 1 --2- 172.21.15.156:0/1834980031 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f701406e0b0 0x7f7014070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:07:48.273 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:48.268+0000 7f701effd640 1 -- 172.21.15.156:0/1834980031 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f701008d430 con 0x7f7028102d20 2024-07-16T10:07:48.273 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:48.268+0000 7f702659c640 1 --2- 172.21.15.156:0/1834980031 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f701406e0b0 0x7f7014070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:07:48.273 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:48.268+0000 7f702659c640 1 --2- 172.21.15.156:0/1834980031 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f701406e0b0 0x7f7014070560 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f7018004750 tx=0x7f7018004d10 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T10:07:48.274 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:48.272+0000 7f701effd640 1 -- 172.21.15.156:0/1834980031 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187716 (secure 0 0 0) 0x7f7010056780 con 0x7f7028102d20 2024-07-16T10:07:48.498 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:48.496+0000 7f702cdd9640 1 -- 172.21.15.156:0/1834980031 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 37, "format": "json"} v 0) v1 -- 0x7f70281094e0 con 0x7f7028102d20 2024-07-16T10:07:48.499 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:48.497+0000 7f701effd640 1 -- 172.21.15.156:0/1834980031 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 37, "format": "json"}]=0 dumped fsmap epoch 37 v45) v1 ==== 107+0+5085 (secure 0 0 0) 0x7f7010059e40 con 0x7f7028102d20 2024-07-16T10:07:48.499 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-07-16T10:07:48.499 INFO:teuthology.orchestra.run.smithi156.stdout:{"epoch":37,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34470,"name":"cephfs.smithi156.evwwaz","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6829/1683974145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":1683974145},{"type":"v1","addr":"172.21.15.156:6829","nonce":1683974145}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":37},{"gid":44269,"name":"cephfs.smithi176.wahbsi","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6827/3942496125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6826","nonce":3942496125},{"type":"v1","addr":"172.21.15.176:6827","nonce":3942496125}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":22},{"gid":44301,"name":"cephfs.smithi176.dvvbud","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6825/3749220343","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6824","nonce":3749220343},{"type":"v1","addr":"172.21.15.176:6825","nonce":3749220343}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-16T09:52:55.214975+0000","modified":"2024-07-16T10:03:11.011991+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":82,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":34438},"failed":[],"damaged":[],"stopped":[1],"info":{"gid_34438":{"gid":34438,"name":"cephfs.smithi156.rdibys","rank":0,"incarnation":33,"state":"up:active","state_seq":9,"addr":"172.21.15.156:6827/2852996640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":2852996640},{"type":"v1","addr":"172.21.15.156:6827","nonce":2852996640}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":34438,"qdb_cluster":[34438]},"id":1}]} 2024-07-16T10:07:48.500 INFO:teuthology.orchestra.run.smithi156.stderr:dumped fsmap epoch 37 2024-07-16T10:07:48.502 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:48.499+0000 7f702cdd9640 1 -- 172.21.15.156:0/1834980031 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f701406e0b0 msgr2=0x7f7014070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:07:48.502 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:48.499+0000 7f702cdd9640 1 --2- 172.21.15.156:0/1834980031 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f701406e0b0 0x7f7014070560 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f7018004750 tx=0x7f7018004d10 comp rx=0 tx=0).stop 2024-07-16T10:07:48.502 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:48.499+0000 7f702cdd9640 1 -- 172.21.15.156:0/1834980031 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7028102d20 msgr2=0x7f70281135f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:07:48.502 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:48.499+0000 7f702cdd9640 1 --2- 172.21.15.156:0/1834980031 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7028102d20 0x7f70281135f0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f7010002a50 tx=0x7f7010004600 comp rx=0 tx=0).stop 2024-07-16T10:07:48.502 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:48.499+0000 7f702cdd9640 1 -- 172.21.15.156:0/1834980031 shutdown_connections 2024-07-16T10:07:48.502 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:48.499+0000 7f702cdd9640 1 --2- 172.21.15.156:0/1834980031 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f701406e0b0 0x7f7014070560 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:07:48.503 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:48.499+0000 7f702cdd9640 1 --2- 172.21.15.156:0/1834980031 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7028102d20 0x7f70281135f0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:07:48.503 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:48.499+0000 7f702cdd9640 1 --2- 172.21.15.156:0/1834980031 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7028102380 0x7f70281130b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:07:48.503 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:48.500+0000 7f702cdd9640 1 -- 172.21.15.156:0/1834980031 >> 172.21.15.156:0/1834980031 conn(0x7f70280f7f10 msgr2=0x7f70280f8ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T10:07:48.503 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:48.500+0000 7f702cdd9640 1 -- 172.21.15.156:0/1834980031 shutdown_connections 2024-07-16T10:07:48.503 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:48.500+0000 7f702cdd9640 1 -- 172.21.15.156:0/1834980031 wait complete. 2024-07-16T10:07:48.924 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:07:48 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-16T10:07:48.924 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:07:48 smithi156 ceph-mon[93066]: from='client.? 172.21.15.156:0/1834980031' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 37, "format": "json"}]: dispatch 2024-07-16T10:07:49.093 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:07:48 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-16T10:07:49.093 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:07:48 smithi176 ceph-mon[81689]: from='client.? 172.21.15.156:0/1834980031' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 37, "format": "json"}]: dispatch 2024-07-16T10:07:49.098 DEBUG:tasks.fs:max_mds reduced in epoch 37 2024-07-16T10:07:49.098 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 37 2024-07-16T10:07:49.098 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 26c91c26-4358-11ef-bcad-c7b262605968 -- ceph fs dump --format=json 38 2024-07-16T10:07:49.353 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/26c91c26-4358-11ef-bcad-c7b262605968/mon.smithi156/config 2024-07-16T10:07:49.794 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:07:49 smithi156 ceph-mon[93066]: pgmap v338: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-07-16T10:07:50.093 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:07:49 smithi176 ceph-mon[81689]: pgmap v338: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-07-16T10:07:50.967 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:50.953+0000 7f9eb5a5b640 1 -- 172.21.15.156:0/3413272515 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9eb0102d10 msgr2=0x7f9eb00fc3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:07:50.967 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:50.953+0000 7f9eb5a5b640 1 --2- 172.21.15.156:0/3413272515 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9eb0102d10 0x7f9eb00fc3a0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f9ea40048d0 tx=0x7f9ea4009da0 comp rx=0 tx=0).stop 2024-07-16T10:07:50.968 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:50.954+0000 7f9eb5a5b640 1 -- 172.21.15.156:0/3413272515 shutdown_connections 2024-07-16T10:07:50.968 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:50.954+0000 7f9eb5a5b640 1 --2- 172.21.15.156:0/3413272515 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9eb0102d10 0x7f9eb00fc3a0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:07:50.968 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:50.954+0000 7f9eb5a5b640 1 --2- 172.21.15.156:0/3413272515 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9eb0102370 0x7f9eb0102740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:07:50.968 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:50.954+0000 7f9eb5a5b640 1 -- 172.21.15.156:0/3413272515 >> 172.21.15.156:0/3413272515 conn(0x7f9eb00f7f00 msgr2=0x7f9eb00fa310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T10:07:50.968 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:50.954+0000 7f9eb5a5b640 1 -- 172.21.15.156:0/3413272515 shutdown_connections 2024-07-16T10:07:50.968 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:50.954+0000 7f9eb5a5b640 1 -- 172.21.15.156:0/3413272515 wait complete. 2024-07-16T10:07:50.968 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:50.954+0000 7f9eb5a5b640 1 Processor -- start 2024-07-16T10:07:50.968 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:50.954+0000 7f9eb5a5b640 1 -- start start 2024-07-16T10:07:50.968 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:50.954+0000 7f9eb5a5b640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9eb0102370 0x7f9eb01133c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:07:50.968 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:50.955+0000 7f9eb5a5b640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9eb0102d10 0x7f9eb0113900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:07:50.968 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:50.955+0000 7f9eb5a5b640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9eb0113f00 con 0x7f9eb0102d10 2024-07-16T10:07:50.968 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:50.955+0000 7f9eb5a5b640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9eb0114070 con 0x7f9eb0102370 2024-07-16T10:07:50.969 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:50.955+0000 7f9eae7fc640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9eb0102d10 0x7f9eb0113900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:07:50.969 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:50.955+0000 7f9eae7fc640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9eb0102d10 0x7f9eb0113900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:44372/0 (socket says 172.21.15.156:44372) 2024-07-16T10:07:50.969 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:50.955+0000 7f9eae7fc640 1 -- 172.21.15.156:0/363376606 learned_addr learned my addr 172.21.15.156:0/363376606 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T10:07:50.969 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:50.955+0000 7f9eaeffd640 1 --2- 172.21.15.156:0/363376606 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9eb0102370 0x7f9eb01133c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:07:50.969 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:50.955+0000 7f9eae7fc640 1 -- 172.21.15.156:0/363376606 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9eb0102370 msgr2=0x7f9eb01133c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:07:50.969 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:50.955+0000 7f9eae7fc640 1 --2- 172.21.15.156:0/363376606 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9eb0102370 0x7f9eb01133c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:07:50.969 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:50.955+0000 7f9eae7fc640 1 -- 172.21.15.156:0/363376606 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ea4005040 con 0x7f9eb0102d10 2024-07-16T10:07:50.969 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:50.955+0000 7f9eae7fc640 1 --2- 172.21.15.156:0/363376606 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9eb0102d10 0x7f9eb0113900 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f9ea40043d0 tx=0x7f9ea4009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T10:07:50.969 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:50.955+0000 7f9e977fe640 1 -- 172.21.15.156:0/363376606 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9ea4038070 con 0x7f9eb0102d10 2024-07-16T10:07:50.969 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:50.955+0000 7f9eb5a5b640 1 -- 172.21.15.156:0/363376606 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9eb010d490 con 0x7f9eb0102d10 2024-07-16T10:07:50.969 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:50.955+0000 7f9e977fe640 1 -- 172.21.15.156:0/363376606 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9ea4032970 con 0x7f9eb0102d10 2024-07-16T10:07:50.969 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:50.955+0000 7f9e977fe640 1 -- 172.21.15.156:0/363376606 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9ea403da60 con 0x7f9eb0102d10 2024-07-16T10:07:50.969 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:50.955+0000 7f9eb5a5b640 1 -- 172.21.15.156:0/363376606 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9eb010d9d0 con 0x7f9eb0102d10 2024-07-16T10:07:50.970 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:50.956+0000 7f9eb5a5b640 1 -- 172.21.15.156:0/363376606 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9eb00fdae0 con 0x7f9eb0102d10 2024-07-16T10:07:50.970 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:50.957+0000 7f9e977fe640 1 -- 172.21.15.156:0/363376606 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 38) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f9ea4030020 con 0x7f9eb0102d10 2024-07-16T10:07:50.970 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:50.957+0000 7f9e977fe640 1 --2- 172.21.15.156:0/363376606 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f9e9806e0b0 0x7f9e98070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:07:50.970 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:50.957+0000 7f9e977fe640 1 -- 172.21.15.156:0/363376606 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f9ea40add30 con 0x7f9eb0102d10 2024-07-16T10:07:50.970 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:50.957+0000 7f9eaeffd640 1 --2- 172.21.15.156:0/363376606 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f9e9806e0b0 0x7f9e98070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:07:50.970 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:50.958+0000 7f9eaeffd640 1 --2- 172.21.15.156:0/363376606 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f9e9806e0b0 0x7f9e98070560 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f9ea0004ed0 tx=0x7f9ea000b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T10:07:50.970 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:50.962+0000 7f9e977fe640 1 -- 172.21.15.156:0/363376606 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187716 (secure 0 0 0) 0x7f9ea4077000 con 0x7f9eb0102d10 2024-07-16T10:07:51.180 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:51.177+0000 7f9eb5a5b640 1 -- 172.21.15.156:0/363376606 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 38, "format": "json"} v 0) v1 -- 0x7f9eb01094d0 con 0x7f9eb0102d10 2024-07-16T10:07:51.180 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:51.178+0000 7f9e977fe640 1 -- 172.21.15.156:0/363376606 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 38, "format": "json"}]=0 dumped fsmap epoch 38 v45) v1 ==== 107+0+4303 (secure 0 0 0) 0x7f9ea407a6c0 con 0x7f9eb0102d10 2024-07-16T10:07:51.180 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-07-16T10:07:51.180 INFO:teuthology.orchestra.run.smithi156.stdout:{"epoch":38,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34470,"name":"cephfs.smithi156.evwwaz","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6829/1683974145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":1683974145},{"type":"v1","addr":"172.21.15.156:6829","nonce":1683974145}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":37},{"gid":44269,"name":"cephfs.smithi176.wahbsi","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6827/3942496125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6826","nonce":3942496125},{"type":"v1","addr":"172.21.15.176:6827","nonce":3942496125}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-16T09:52:55.214975+0000","modified":"2024-07-16T10:03:11.011991+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":82,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":34438},"failed":[],"damaged":[],"stopped":[1],"info":{"gid_34438":{"gid":34438,"name":"cephfs.smithi156.rdibys","rank":0,"incarnation":33,"state":"up:active","state_seq":9,"addr":"172.21.15.156:6827/2852996640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":2852996640},{"type":"v1","addr":"172.21.15.156:6827","nonce":2852996640}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":34438,"qdb_cluster":[34438]},"id":1}]} 2024-07-16T10:07:51.181 INFO:teuthology.orchestra.run.smithi156.stderr:dumped fsmap epoch 38 2024-07-16T10:07:51.183 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:51.181+0000 7f9eb5a5b640 1 -- 172.21.15.156:0/363376606 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f9e9806e0b0 msgr2=0x7f9e98070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:07:51.183 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:51.181+0000 7f9eb5a5b640 1 --2- 172.21.15.156:0/363376606 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f9e9806e0b0 0x7f9e98070560 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f9ea0004ed0 tx=0x7f9ea000b040 comp rx=0 tx=0).stop 2024-07-16T10:07:51.183 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:51.181+0000 7f9eb5a5b640 1 -- 172.21.15.156:0/363376606 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9eb0102d10 msgr2=0x7f9eb0113900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:07:51.183 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:51.181+0000 7f9eb5a5b640 1 --2- 172.21.15.156:0/363376606 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9eb0102d10 0x7f9eb0113900 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f9ea40043d0 tx=0x7f9ea4009da0 comp rx=0 tx=0).stop 2024-07-16T10:07:51.183 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:51.181+0000 7f9eb5a5b640 1 -- 172.21.15.156:0/363376606 shutdown_connections 2024-07-16T10:07:51.183 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:51.181+0000 7f9eb5a5b640 1 --2- 172.21.15.156:0/363376606 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f9e9806e0b0 0x7f9e98070560 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:07:51.183 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:51.181+0000 7f9eb5a5b640 1 --2- 172.21.15.156:0/363376606 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9eb0102d10 0x7f9eb0113900 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:07:51.184 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:51.181+0000 7f9eb5a5b640 1 --2- 172.21.15.156:0/363376606 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9eb0102370 0x7f9eb01133c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:07:51.184 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:51.181+0000 7f9eb5a5b640 1 -- 172.21.15.156:0/363376606 >> 172.21.15.156:0/363376606 conn(0x7f9eb00f7f00 msgr2=0x7f9eb00f8ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T10:07:51.184 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:51.181+0000 7f9eb5a5b640 1 -- 172.21.15.156:0/363376606 shutdown_connections 2024-07-16T10:07:51.184 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:51.181+0000 7f9eb5a5b640 1 -- 172.21.15.156:0/363376606 wait complete. 2024-07-16T10:07:51.822 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:07:51 smithi156 ceph-mon[93066]: pgmap v339: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:07:51.822 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:07:51 smithi156 ceph-mon[93066]: from='client.? 172.21.15.156:0/363376606' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 38, "format": "json"}]: dispatch 2024-07-16T10:07:51.846 DEBUG:tasks.fs:max_mds reduced in epoch 38 2024-07-16T10:07:51.846 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 38 2024-07-16T10:07:51.846 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 26c91c26-4358-11ef-bcad-c7b262605968 -- ceph fs dump --format=json 39 2024-07-16T10:07:52.093 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:07:51 smithi176 ceph-mon[81689]: pgmap v339: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:07:52.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:07:51 smithi176 ceph-mon[81689]: from='client.? 172.21.15.156:0/363376606' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 38, "format": "json"}]: dispatch 2024-07-16T10:07:52.103 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/26c91c26-4358-11ef-bcad-c7b262605968/mon.smithi156/config 2024-07-16T10:07:53.603 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:53.602+0000 7f19b38b7640 1 -- 172.21.15.156:0/1297694215 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f19ac102cf0 msgr2=0x7f19ac0fc380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:07:53.604 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:53.602+0000 7f19b38b7640 1 --2- 172.21.15.156:0/1297694215 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f19ac102cf0 0x7f19ac0fc380 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f199c0048d0 tx=0x7f199c009da0 comp rx=0 tx=0).stop 2024-07-16T10:07:53.604 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:53.602+0000 7f19b38b7640 1 -- 172.21.15.156:0/1297694215 shutdown_connections 2024-07-16T10:07:53.604 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:53.602+0000 7f19b38b7640 1 --2- 172.21.15.156:0/1297694215 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f19ac102cf0 0x7f19ac0fc380 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:07:53.604 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:53.602+0000 7f19b38b7640 1 --2- 172.21.15.156:0/1297694215 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f19ac102350 0x7f19ac102720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:07:53.604 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:53.602+0000 7f19b38b7640 1 -- 172.21.15.156:0/1297694215 >> 172.21.15.156:0/1297694215 conn(0x7f19ac0f7f00 msgr2=0x7f19ac0fa310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T10:07:53.604 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:53.602+0000 7f19b38b7640 1 -- 172.21.15.156:0/1297694215 shutdown_connections 2024-07-16T10:07:53.604 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:53.602+0000 7f19b38b7640 1 -- 172.21.15.156:0/1297694215 wait complete. 2024-07-16T10:07:53.605 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:53.603+0000 7f19b38b7640 1 Processor -- start 2024-07-16T10:07:53.605 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:53.603+0000 7f19b38b7640 1 -- start start 2024-07-16T10:07:53.606 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:53.603+0000 7f19b38b7640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f19ac102350 0x7f19ac1130d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:07:53.606 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:53.603+0000 7f19b38b7640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f19ac102cf0 0x7f19ac113610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:07:53.606 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:53.603+0000 7f19b38b7640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19ac113c10 con 0x7f19ac102cf0 2024-07-16T10:07:53.606 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:53.603+0000 7f19b38b7640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19ac10d050 con 0x7f19ac102350 2024-07-16T10:07:53.606 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:53.603+0000 7f19b0e52640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f19ac102cf0 0x7f19ac113610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:07:53.606 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:53.603+0000 7f19b0e52640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f19ac102cf0 0x7f19ac113610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:45592/0 (socket says 172.21.15.156:45592) 2024-07-16T10:07:53.606 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:53.603+0000 7f19b1653640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f19ac102350 0x7f19ac1130d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:07:53.606 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:53.603+0000 7f19b0e52640 1 -- 172.21.15.156:0/1364205792 learned_addr learned my addr 172.21.15.156:0/1364205792 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T10:07:53.606 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:53.604+0000 7f19b0e52640 1 -- 172.21.15.156:0/1364205792 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f19ac102350 msgr2=0x7f19ac1130d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:07:53.607 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:53.604+0000 7f19b0e52640 1 --2- 172.21.15.156:0/1364205792 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f19ac102350 0x7f19ac1130d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:07:53.607 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:53.604+0000 7f19b0e52640 1 -- 172.21.15.156:0/1364205792 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f199c005040 con 0x7f19ac102cf0 2024-07-16T10:07:53.607 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:53.604+0000 7f19b0e52640 1 --2- 172.21.15.156:0/1364205792 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f19ac102cf0 0x7f19ac113610 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f199c0043d0 tx=0x7f199c009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T10:07:53.607 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:53.604+0000 7f19a1ffb640 1 -- 172.21.15.156:0/1364205792 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f199c038070 con 0x7f19ac102cf0 2024-07-16T10:07:53.607 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:53.604+0000 7f19a1ffb640 1 -- 172.21.15.156:0/1364205792 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f199c032970 con 0x7f19ac102cf0 2024-07-16T10:07:53.607 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:53.604+0000 7f19b38b7640 1 -- 172.21.15.156:0/1364205792 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19ac10d2d0 con 0x7f19ac102cf0 2024-07-16T10:07:53.607 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:53.604+0000 7f19a1ffb640 1 -- 172.21.15.156:0/1364205792 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f199c03da60 con 0x7f19ac102cf0 2024-07-16T10:07:53.607 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:53.604+0000 7f19b38b7640 1 -- 172.21.15.156:0/1364205792 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19ac10d810 con 0x7f19ac102cf0 2024-07-16T10:07:53.608 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:53.605+0000 7f19b38b7640 1 -- 172.21.15.156:0/1364205792 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19ac0fdac0 con 0x7f19ac102cf0 2024-07-16T10:07:53.608 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:53.606+0000 7f19a1ffb640 1 -- 172.21.15.156:0/1364205792 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 38) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f199c030020 con 0x7f19ac102cf0 2024-07-16T10:07:53.608 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:53.606+0000 7f19a1ffb640 1 --2- 172.21.15.156:0/1364205792 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f199806e0b0 0x7f1998070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:07:53.608 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:53.606+0000 7f19a1ffb640 1 -- 172.21.15.156:0/1364205792 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f199c0adcf0 con 0x7f19ac102cf0 2024-07-16T10:07:53.609 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:53.606+0000 7f19b1653640 1 --2- 172.21.15.156:0/1364205792 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f199806e0b0 0x7f1998070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:07:53.609 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:53.607+0000 7f19b1653640 1 --2- 172.21.15.156:0/1364205792 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f199806e0b0 0x7f1998070560 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f19a8004fd0 tx=0x7f19a8009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T10:07:53.612 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:53.611+0000 7f19a1ffb640 1 -- 172.21.15.156:0/1364205792 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187716 (secure 0 0 0) 0x7f199c076fc0 con 0x7f19ac102cf0 2024-07-16T10:07:53.825 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:53.823+0000 7f19b38b7640 1 -- 172.21.15.156:0/1364205792 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 39, "format": "json"} v 0) v1 -- 0x7f19ac1094b0 con 0x7f19ac102cf0 2024-07-16T10:07:53.825 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:53.824+0000 7f19a1ffb640 1 -- 172.21.15.156:0/1364205792 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 39, "format": "json"}]=0 dumped fsmap epoch 39 v45) v1 ==== 107+0+5153 (secure 0 0 0) 0x7f199c07a680 con 0x7f19ac102cf0 2024-07-16T10:07:53.825 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-07-16T10:07:53.825 INFO:teuthology.orchestra.run.smithi156.stdout:{"epoch":39,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34470,"name":"cephfs.smithi156.evwwaz","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6829/1683974145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":1683974145},{"type":"v1","addr":"172.21.15.156:6829","nonce":1683974145}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":37},{"gid":44269,"name":"cephfs.smithi176.wahbsi","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6827/3942496125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6826","nonce":3942496125},{"type":"v1","addr":"172.21.15.176:6827","nonce":3942496125}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":22},{"gid":44347,"name":"cephfs.smithi176.dvvbud","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6825/1700587296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6824","nonce":1700587296},{"type":"v1","addr":"172.21.15.176:6825","nonce":1700587296}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-16T09:52:55.214975+0000","modified":"2024-07-16T10:03:11.011991+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":82,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":34438},"failed":[],"damaged":[],"stopped":[1],"info":{"gid_34438":{"gid":34438,"name":"cephfs.smithi156.rdibys","rank":0,"incarnation":33,"state":"up:active","state_seq":9,"addr":"172.21.15.156:6827/2852996640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":2852996640},{"type":"v1","addr":"172.21.15.156:6827","nonce":2852996640}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":34438,"qdb_cluster":[34438]},"id":1}]} 2024-07-16T10:07:53.826 INFO:teuthology.orchestra.run.smithi156.stderr:dumped fsmap epoch 39 2024-07-16T10:07:53.827 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:53.826+0000 7f19b38b7640 1 -- 172.21.15.156:0/1364205792 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f199806e0b0 msgr2=0x7f1998070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:07:53.827 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:53.826+0000 7f19b38b7640 1 --2- 172.21.15.156:0/1364205792 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f199806e0b0 0x7f1998070560 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f19a8004fd0 tx=0x7f19a8009040 comp rx=0 tx=0).stop 2024-07-16T10:07:53.827 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:53.826+0000 7f19b38b7640 1 -- 172.21.15.156:0/1364205792 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f19ac102cf0 msgr2=0x7f19ac113610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:07:53.827 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:53.826+0000 7f19b38b7640 1 --2- 172.21.15.156:0/1364205792 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f19ac102cf0 0x7f19ac113610 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f199c0043d0 tx=0x7f199c009da0 comp rx=0 tx=0).stop 2024-07-16T10:07:53.827 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:53.826+0000 7f19b38b7640 1 -- 172.21.15.156:0/1364205792 shutdown_connections 2024-07-16T10:07:53.827 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:53.826+0000 7f19b38b7640 1 --2- 172.21.15.156:0/1364205792 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f199806e0b0 0x7f1998070560 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:07:53.828 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:53.826+0000 7f19b38b7640 1 --2- 172.21.15.156:0/1364205792 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f19ac102cf0 0x7f19ac113610 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:07:53.828 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:53.826+0000 7f19b38b7640 1 --2- 172.21.15.156:0/1364205792 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f19ac102350 0x7f19ac1130d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:07:53.828 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:53.826+0000 7f19b38b7640 1 -- 172.21.15.156:0/1364205792 >> 172.21.15.156:0/1364205792 conn(0x7f19ac0f7f00 msgr2=0x7f19ac0f8ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T10:07:53.828 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:53.826+0000 7f19b38b7640 1 -- 172.21.15.156:0/1364205792 shutdown_connections 2024-07-16T10:07:53.828 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:53.826+0000 7f19b38b7640 1 -- 172.21.15.156:0/1364205792 wait complete. 2024-07-16T10:07:53.924 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:07:53 smithi156 ceph-mon[93066]: pgmap v340: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:07:54.093 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:07:53 smithi176 ceph-mon[81689]: pgmap v340: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:07:54.477 DEBUG:tasks.fs:max_mds reduced in epoch 39 2024-07-16T10:07:54.478 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 39 2024-07-16T10:07:54.478 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 26c91c26-4358-11ef-bcad-c7b262605968 -- ceph fs dump --format=json 40 2024-07-16T10:07:54.735 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/26c91c26-4358-11ef-bcad-c7b262605968/mon.smithi156/config 2024-07-16T10:07:54.903 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:07:54 smithi156 ceph-mon[93066]: from='client.? 172.21.15.156:0/1364205792' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 39, "format": "json"}]: dispatch 2024-07-16T10:07:55.093 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:07:54 smithi176 ceph-mon[81689]: from='client.? 172.21.15.156:0/1364205792' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 39, "format": "json"}]: dispatch 2024-07-16T10:07:55.718 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:07:55 smithi156 ceph-mon[93066]: pgmap v341: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-07-16T10:07:56.093 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:07:55 smithi176 ceph-mon[81689]: pgmap v341: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-07-16T10:07:56.220 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:56.215+0000 7f3a7186c640 1 -- 172.21.15.156:0/185818299 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3a6c0fbf30 msgr2=0x7f3a6c0fce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:07:56.221 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:56.215+0000 7f3a7186c640 1 --2- 172.21.15.156:0/185818299 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3a6c0fbf30 0x7f3a6c0fce30 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f3a600048d0 tx=0x7f3a60009da0 comp rx=0 tx=0).stop 2024-07-16T10:07:56.221 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:56.215+0000 7f3a7186c640 1 -- 172.21.15.156:0/185818299 shutdown_connections 2024-07-16T10:07:56.221 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:56.215+0000 7f3a7186c640 1 --2- 172.21.15.156:0/185818299 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3a6c0fbf30 0x7f3a6c0fce30 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:07:56.221 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:56.215+0000 7f3a7186c640 1 --2- 172.21.15.156:0/185818299 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3a6c0fb620 0x7f3a6c0fb9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:07:56.221 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:56.215+0000 7f3a7186c640 1 -- 172.21.15.156:0/185818299 >> 172.21.15.156:0/185818299 conn(0x7f3a6c0721c0 msgr2=0x7f3a6c0725c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T10:07:56.221 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:56.215+0000 7f3a7186c640 1 -- 172.21.15.156:0/185818299 shutdown_connections 2024-07-16T10:07:56.221 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:56.215+0000 7f3a7186c640 1 -- 172.21.15.156:0/185818299 wait complete. 2024-07-16T10:07:56.221 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:56.215+0000 7f3a7186c640 1 Processor -- start 2024-07-16T10:07:56.221 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:56.215+0000 7f3a7186c640 1 -- start start 2024-07-16T10:07:56.221 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:56.216+0000 7f3a7186c640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3a6c0fb620 0x7f3a6c10e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:07:56.221 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:56.216+0000 7f3a7186c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3a6c0fbf30 0x7f3a6c10e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:07:56.222 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:56.216+0000 7f3a7186c640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a6c10ec70 con 0x7f3a6c0fbf30 2024-07-16T10:07:56.222 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:56.216+0000 7f3a7186c640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a6c10ede0 con 0x7f3a6c0fb620 2024-07-16T10:07:56.222 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:56.216+0000 7f3a6a7fc640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3a6c0fbf30 0x7f3a6c10e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:07:56.222 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:56.216+0000 7f3a6a7fc640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3a6c0fbf30 0x7f3a6c10e5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:45624/0 (socket says 172.21.15.156:45624) 2024-07-16T10:07:56.222 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:56.216+0000 7f3a6a7fc640 1 -- 172.21.15.156:0/326409118 learned_addr learned my addr 172.21.15.156:0/326409118 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T10:07:56.222 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:56.216+0000 7f3a6affd640 1 --2- 172.21.15.156:0/326409118 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3a6c0fb620 0x7f3a6c10e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:07:56.222 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:56.216+0000 7f3a6a7fc640 1 -- 172.21.15.156:0/326409118 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3a6c0fb620 msgr2=0x7f3a6c10e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:07:56.222 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:56.216+0000 7f3a6a7fc640 1 --2- 172.21.15.156:0/326409118 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3a6c0fb620 0x7f3a6c10e0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:07:56.222 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:56.216+0000 7f3a6a7fc640 1 -- 172.21.15.156:0/326409118 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a60005040 con 0x7f3a6c0fbf30 2024-07-16T10:07:56.222 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:56.216+0000 7f3a6affd640 1 --2- 172.21.15.156:0/326409118 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3a6c0fb620 0x7f3a6c10e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-16T10:07:56.222 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:56.216+0000 7f3a6a7fc640 1 --2- 172.21.15.156:0/326409118 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3a6c0fbf30 0x7f3a6c10e5e0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f3a60003080 tx=0x7f3a60009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T10:07:56.222 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:56.217+0000 7f3a537fe640 1 -- 172.21.15.156:0/326409118 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3a60038070 con 0x7f3a6c0fbf30 2024-07-16T10:07:56.222 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:56.217+0000 7f3a537fe640 1 -- 172.21.15.156:0/326409118 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3a60008ee0 con 0x7f3a6c0fbf30 2024-07-16T10:07:56.223 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:56.217+0000 7f3a537fe640 1 -- 172.21.15.156:0/326409118 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3a6003d870 con 0x7f3a6c0fbf30 2024-07-16T10:07:56.223 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:56.217+0000 7f3a7186c640 1 -- 172.21.15.156:0/326409118 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a6c117110 con 0x7f3a6c0fbf30 2024-07-16T10:07:56.223 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:56.217+0000 7f3a7186c640 1 -- 172.21.15.156:0/326409118 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a6c117580 con 0x7f3a6c0fbf30 2024-07-16T10:07:56.223 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:56.218+0000 7f3a7186c640 1 -- 172.21.15.156:0/326409118 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a6c0fc9d0 con 0x7f3a6c0fbf30 2024-07-16T10:07:56.223 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:56.218+0000 7f3a537fe640 1 -- 172.21.15.156:0/326409118 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 38) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f3a60030020 con 0x7f3a6c0fbf30 2024-07-16T10:07:56.223 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:56.219+0000 7f3a537fe640 1 --2- 172.21.15.156:0/326409118 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f3a5406e0b0 0x7f3a54070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:07:56.223 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:56.219+0000 7f3a537fe640 1 -- 172.21.15.156:0/326409118 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f3a600ad980 con 0x7f3a6c0fbf30 2024-07-16T10:07:56.223 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:56.219+0000 7f3a6affd640 1 --2- 172.21.15.156:0/326409118 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f3a5406e0b0 0x7f3a54070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:07:56.223 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:56.219+0000 7f3a6affd640 1 --2- 172.21.15.156:0/326409118 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f3a5406e0b0 0x7f3a54070560 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f3a5c001250 tx=0x7f3a5c009610 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T10:07:56.224 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:56.223+0000 7f3a537fe640 1 -- 172.21.15.156:0/326409118 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187716 (secure 0 0 0) 0x7f3a60076cd0 con 0x7f3a6c0fbf30 2024-07-16T10:07:56.436 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:56.435+0000 7f3a7186c640 1 -- 172.21.15.156:0/326409118 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 40, "format": "json"} v 0) v1 -- 0x7f3a6c04b530 con 0x7f3a6c0fbf30 2024-07-16T10:07:56.436 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:56.435+0000 7f3a537fe640 1 -- 172.21.15.156:0/326409118 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 40, "format": "json"}]=0 dumped fsmap epoch 40 v45) v1 ==== 107+0+4371 (secure 0 0 0) 0x7f3a6007a390 con 0x7f3a6c0fbf30 2024-07-16T10:07:56.437 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-07-16T10:07:56.437 INFO:teuthology.orchestra.run.smithi156.stdout:{"epoch":40,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34470,"name":"cephfs.smithi156.evwwaz","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6829/1683974145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":1683974145},{"type":"v1","addr":"172.21.15.156:6829","nonce":1683974145}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":37},{"gid":44347,"name":"cephfs.smithi176.dvvbud","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6825/1700587296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6824","nonce":1700587296},{"type":"v1","addr":"172.21.15.176:6825","nonce":1700587296}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-16T09:52:55.214975+0000","modified":"2024-07-16T10:03:11.011991+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":82,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":34438},"failed":[],"damaged":[],"stopped":[1],"info":{"gid_34438":{"gid":34438,"name":"cephfs.smithi156.rdibys","rank":0,"incarnation":33,"state":"up:active","state_seq":9,"addr":"172.21.15.156:6827/2852996640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":2852996640},{"type":"v1","addr":"172.21.15.156:6827","nonce":2852996640}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":34438,"qdb_cluster":[34438]},"id":1}]} 2024-07-16T10:07:56.437 INFO:teuthology.orchestra.run.smithi156.stderr:dumped fsmap epoch 40 2024-07-16T10:07:56.439 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:56.437+0000 7f3a7186c640 1 -- 172.21.15.156:0/326409118 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f3a5406e0b0 msgr2=0x7f3a54070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:07:56.439 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:56.437+0000 7f3a7186c640 1 --2- 172.21.15.156:0/326409118 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f3a5406e0b0 0x7f3a54070560 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f3a5c001250 tx=0x7f3a5c009610 comp rx=0 tx=0).stop 2024-07-16T10:07:56.439 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:56.437+0000 7f3a7186c640 1 -- 172.21.15.156:0/326409118 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3a6c0fbf30 msgr2=0x7f3a6c10e5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:07:56.439 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:56.437+0000 7f3a7186c640 1 --2- 172.21.15.156:0/326409118 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3a6c0fbf30 0x7f3a6c10e5e0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f3a60003080 tx=0x7f3a60009da0 comp rx=0 tx=0).stop 2024-07-16T10:07:56.439 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:56.437+0000 7f3a7186c640 1 -- 172.21.15.156:0/326409118 shutdown_connections 2024-07-16T10:07:56.439 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:56.437+0000 7f3a7186c640 1 --2- 172.21.15.156:0/326409118 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f3a5406e0b0 0x7f3a54070560 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:07:56.439 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:56.437+0000 7f3a7186c640 1 --2- 172.21.15.156:0/326409118 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3a6c0fbf30 0x7f3a6c10e5e0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:07:56.439 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:56.437+0000 7f3a7186c640 1 --2- 172.21.15.156:0/326409118 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3a6c0fb620 0x7f3a6c10e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:07:56.439 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:56.437+0000 7f3a7186c640 1 -- 172.21.15.156:0/326409118 >> 172.21.15.156:0/326409118 conn(0x7f3a6c0721c0 msgr2=0x7f3a6c0f86d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T10:07:56.439 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:56.438+0000 7f3a7186c640 1 -- 172.21.15.156:0/326409118 shutdown_connections 2024-07-16T10:07:56.440 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:56.438+0000 7f3a7186c640 1 -- 172.21.15.156:0/326409118 wait complete. 2024-07-16T10:07:56.749 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:07:56 smithi156 ceph-mon[93066]: from='client.? 172.21.15.156:0/326409118' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 40, "format": "json"}]: dispatch 2024-07-16T10:07:57.059 DEBUG:tasks.fs:max_mds reduced in epoch 40 2024-07-16T10:07:57.059 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 40 2024-07-16T10:07:57.060 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 26c91c26-4358-11ef-bcad-c7b262605968 -- ceph fs dump --format=json 41 2024-07-16T10:07:57.093 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:07:56 smithi176 ceph-mon[81689]: from='client.? 172.21.15.156:0/326409118' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 40, "format": "json"}]: dispatch 2024-07-16T10:07:57.317 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/26c91c26-4358-11ef-bcad-c7b262605968/mon.smithi156/config 2024-07-16T10:07:57.720 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:07:57 smithi156 ceph-mon[93066]: pgmap v342: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:07:58.093 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:07:57 smithi176 ceph-mon[81689]: pgmap v342: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:07:58.977 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:58.964+0000 7f0ac53a1640 1 -- 172.21.15.156:0/2261256663 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0ac0065150 msgr2=0x7f0ac0065520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:07:58.977 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:58.964+0000 7f0ac53a1640 1 --2- 172.21.15.156:0/2261256663 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0ac0065150 0x7f0ac0065520 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f0ab00048d0 tx=0x7f0ab0009da0 comp rx=0 tx=0).stop 2024-07-16T10:07:58.977 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:58.964+0000 7f0ac53a1640 1 -- 172.21.15.156:0/2261256663 shutdown_connections 2024-07-16T10:07:58.977 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:58.964+0000 7f0ac53a1640 1 --2- 172.21.15.156:0/2261256663 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0ac00fdf60 0x7f0ac00fe390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:07:58.977 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:58.964+0000 7f0ac53a1640 1 --2- 172.21.15.156:0/2261256663 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0ac0065150 0x7f0ac0065520 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:07:58.977 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:58.964+0000 7f0ac53a1640 1 -- 172.21.15.156:0/2261256663 >> 172.21.15.156:0/2261256663 conn(0x7f0ac00721c0 msgr2=0x7f0ac00725c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T10:07:58.978 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:58.964+0000 7f0ac53a1640 1 -- 172.21.15.156:0/2261256663 shutdown_connections 2024-07-16T10:07:58.978 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:58.964+0000 7f0ac53a1640 1 -- 172.21.15.156:0/2261256663 wait complete. 2024-07-16T10:07:58.978 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:58.964+0000 7f0ac53a1640 1 Processor -- start 2024-07-16T10:07:58.978 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:58.965+0000 7f0ac53a1640 1 -- start start 2024-07-16T10:07:58.978 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:58.965+0000 7f0ac53a1640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0ac0065150 0x7f0ac0113020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:07:58.978 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:58.965+0000 7f0ac53a1640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0ac00fdf60 0x7f0ac0113560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:07:58.978 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:58.965+0000 7f0ac53a1640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ac0113b60 con 0x7f0ac0065150 2024-07-16T10:07:58.978 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:58.965+0000 7f0ac53a1640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ac0113cd0 con 0x7f0ac00fdf60 2024-07-16T10:07:58.978 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:58.965+0000 7f0abeffd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0ac0065150 0x7f0ac0113020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:07:58.978 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:58.965+0000 7f0abeffd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0ac0065150 0x7f0ac0113020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:45656/0 (socket says 172.21.15.156:45656) 2024-07-16T10:07:58.978 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:58.965+0000 7f0abeffd640 1 -- 172.21.15.156:0/3936995001 learned_addr learned my addr 172.21.15.156:0/3936995001 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T10:07:58.978 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:58.965+0000 7f0ab67fc640 1 --2- 172.21.15.156:0/3936995001 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0ac00fdf60 0x7f0ac0113560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:07:58.979 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:58.965+0000 7f0abeffd640 1 -- 172.21.15.156:0/3936995001 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0ac00fdf60 msgr2=0x7f0ac0113560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:07:58.979 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:58.965+0000 7f0abeffd640 1 --2- 172.21.15.156:0/3936995001 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0ac00fdf60 0x7f0ac0113560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:07:58.979 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:58.965+0000 7f0abeffd640 1 -- 172.21.15.156:0/3936995001 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ab0005040 con 0x7f0ac0065150 2024-07-16T10:07:58.979 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:58.965+0000 7f0abeffd640 1 --2- 172.21.15.156:0/3936995001 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0ac0065150 0x7f0ac0113020 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f0ab00043d0 tx=0x7f0ab0009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T10:07:58.979 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:58.966+0000 7f0ab7fff640 1 -- 172.21.15.156:0/3936995001 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0ab0038070 con 0x7f0ac0065150 2024-07-16T10:07:58.979 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:58.966+0000 7f0ac53a1640 1 -- 172.21.15.156:0/3936995001 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ac010d160 con 0x7f0ac0065150 2024-07-16T10:07:58.979 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:58.966+0000 7f0ab7fff640 1 -- 172.21.15.156:0/3936995001 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0ab0032970 con 0x7f0ac0065150 2024-07-16T10:07:58.979 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:58.966+0000 7f0ab7fff640 1 -- 172.21.15.156:0/3936995001 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0ab003da60 con 0x7f0ac0065150 2024-07-16T10:07:58.979 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:58.966+0000 7f0ac53a1640 1 -- 172.21.15.156:0/3936995001 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ac010d6a0 con 0x7f0ac0065150 2024-07-16T10:07:58.979 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:58.967+0000 7f0ac53a1640 1 -- 172.21.15.156:0/3936995001 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ac00fdb00 con 0x7f0ac0065150 2024-07-16T10:07:58.979 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:58.967+0000 7f0ab7fff640 1 -- 172.21.15.156:0/3936995001 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 38) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f0ab0030020 con 0x7f0ac0065150 2024-07-16T10:07:58.979 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:58.968+0000 7f0ab7fff640 1 --2- 172.21.15.156:0/3936995001 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f0aac06e0b0 0x7f0aac070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:07:58.980 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:58.968+0000 7f0ab7fff640 1 -- 172.21.15.156:0/3936995001 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f0ab00adcf0 con 0x7f0ac0065150 2024-07-16T10:07:58.980 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:58.968+0000 7f0ab67fc640 1 --2- 172.21.15.156:0/3936995001 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f0aac06e0b0 0x7f0aac070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:07:58.980 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:58.968+0000 7f0ab67fc640 1 --2- 172.21.15.156:0/3936995001 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f0aac06e0b0 0x7f0aac070560 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f0ac0072800 tx=0x7f0aa800b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T10:07:58.980 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:58.972+0000 7f0ab7fff640 1 -- 172.21.15.156:0/3936995001 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187716 (secure 0 0 0) 0x7f0ab0076fc0 con 0x7f0ac0065150 2024-07-16T10:07:59.187 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:59.186+0000 7f0ac53a1640 1 -- 172.21.15.156:0/3936995001 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 41, "format": "json"} v 0) v1 -- 0x7f0ac0072cd0 con 0x7f0ac0065150 2024-07-16T10:07:59.187 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:59.186+0000 7f0ab7fff640 1 -- 172.21.15.156:0/3936995001 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 41, "format": "json"}]=0 dumped fsmap epoch 41 v45) v1 ==== 107+0+5221 (secure 0 0 0) 0x7f0ab007a680 con 0x7f0ac0065150 2024-07-16T10:07:59.188 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-07-16T10:07:59.188 INFO:teuthology.orchestra.run.smithi156.stdout:{"epoch":41,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34470,"name":"cephfs.smithi156.evwwaz","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6829/1683974145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":1683974145},{"type":"v1","addr":"172.21.15.156:6829","nonce":1683974145}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":37},{"gid":44347,"name":"cephfs.smithi176.dvvbud","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6825/1700587296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6824","nonce":1700587296},{"type":"v1","addr":"172.21.15.176:6825","nonce":1700587296}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":44373,"name":"cephfs.smithi176.wahbsi","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6827/2277003188","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6826","nonce":2277003188},{"type":"v1","addr":"172.21.15.176:6827","nonce":2277003188}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-16T09:52:55.214975+0000","modified":"2024-07-16T10:03:11.011991+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":82,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":34438},"failed":[],"damaged":[],"stopped":[1],"info":{"gid_34438":{"gid":34438,"name":"cephfs.smithi156.rdibys","rank":0,"incarnation":33,"state":"up:active","state_seq":9,"addr":"172.21.15.156:6827/2852996640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":2852996640},{"type":"v1","addr":"172.21.15.156:6827","nonce":2852996640}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":34438,"qdb_cluster":[34438]},"id":1}]} 2024-07-16T10:07:59.188 INFO:teuthology.orchestra.run.smithi156.stderr:dumped fsmap epoch 41 2024-07-16T10:07:59.190 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:59.188+0000 7f0ac53a1640 1 -- 172.21.15.156:0/3936995001 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f0aac06e0b0 msgr2=0x7f0aac070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:07:59.190 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:59.188+0000 7f0ac53a1640 1 --2- 172.21.15.156:0/3936995001 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f0aac06e0b0 0x7f0aac070560 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f0ac0072800 tx=0x7f0aa800b040 comp rx=0 tx=0).stop 2024-07-16T10:07:59.190 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:59.188+0000 7f0ac53a1640 1 -- 172.21.15.156:0/3936995001 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0ac0065150 msgr2=0x7f0ac0113020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:07:59.190 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:59.188+0000 7f0ac53a1640 1 --2- 172.21.15.156:0/3936995001 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0ac0065150 0x7f0ac0113020 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f0ab00043d0 tx=0x7f0ab0009da0 comp rx=0 tx=0).stop 2024-07-16T10:07:59.190 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:59.188+0000 7f0ac53a1640 1 -- 172.21.15.156:0/3936995001 shutdown_connections 2024-07-16T10:07:59.190 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:59.188+0000 7f0ac53a1640 1 --2- 172.21.15.156:0/3936995001 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f0aac06e0b0 0x7f0aac070560 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:07:59.190 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:59.188+0000 7f0ac53a1640 1 --2- 172.21.15.156:0/3936995001 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0ac00fdf60 0x7f0ac0113560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:07:59.190 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:59.188+0000 7f0ac53a1640 1 --2- 172.21.15.156:0/3936995001 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0ac0065150 0x7f0ac0113020 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:07:59.191 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:59.188+0000 7f0ac53a1640 1 -- 172.21.15.156:0/3936995001 >> 172.21.15.156:0/3936995001 conn(0x7f0ac00721c0 msgr2=0x7f0ac00f86c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T10:07:59.191 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:59.189+0000 7f0ac53a1640 1 -- 172.21.15.156:0/3936995001 shutdown_connections 2024-07-16T10:07:59.191 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:07:59.189+0000 7f0ac53a1640 1 -- 172.21.15.156:0/3936995001 wait complete. 2024-07-16T10:07:59.897 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:07:59 smithi156 ceph-mon[93066]: pgmap v343: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-07-16T10:07:59.897 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:07:59 smithi156 ceph-mon[93066]: from='client.? 172.21.15.156:0/3936995001' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 41, "format": "json"}]: dispatch 2024-07-16T10:07:59.923 DEBUG:tasks.fs:max_mds reduced in epoch 41 2024-07-16T10:07:59.923 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 41 2024-07-16T10:07:59.923 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 26c91c26-4358-11ef-bcad-c7b262605968 -- ceph fs dump --format=json 42 2024-07-16T10:08:00.093 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:07:59 smithi176 ceph-mon[81689]: pgmap v343: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-07-16T10:08:00.093 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:07:59 smithi176 ceph-mon[81689]: from='client.? 172.21.15.156:0/3936995001' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 41, "format": "json"}]: dispatch 2024-07-16T10:08:00.180 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/26c91c26-4358-11ef-bcad-c7b262605968/mon.smithi156/config 2024-07-16T10:08:01.833 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:08:01.818+0000 7fd10cf5a640 1 -- 172.21.15.156:0/2216057448 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd1081023a0 msgr2=0x7fd108102770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:08:01.833 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:08:01.818+0000 7fd10cf5a640 1 --2- 172.21.15.156:0/2216057448 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd1081023a0 0x7fd108102770 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fd0f80048d0 tx=0x7fd0f8009da0 comp rx=0 tx=0).stop 2024-07-16T10:08:01.834 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:08:01.818+0000 7fd10cf5a640 1 -- 172.21.15.156:0/2216057448 shutdown_connections 2024-07-16T10:08:01.834 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:08:01.818+0000 7fd10cf5a640 1 --2- 172.21.15.156:0/2216057448 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd108102d40 0x7fd1080faa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:08:01.834 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:08:01.818+0000 7fd10cf5a640 1 --2- 172.21.15.156:0/2216057448 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd1081023a0 0x7fd108102770 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:08:01.834 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:08:01.818+0000 7fd10cf5a640 1 -- 172.21.15.156:0/2216057448 >> 172.21.15.156:0/2216057448 conn(0x7fd108070af0 msgr2=0x7fd108070ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T10:08:01.834 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:08:01.818+0000 7fd10cf5a640 1 -- 172.21.15.156:0/2216057448 shutdown_connections 2024-07-16T10:08:01.834 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:08:01.818+0000 7fd10cf5a640 1 -- 172.21.15.156:0/2216057448 wait complete. 2024-07-16T10:08:01.834 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:08:01.819+0000 7fd10cf5a640 1 Processor -- start 2024-07-16T10:08:01.834 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:08:01.819+0000 7fd10cf5a640 1 -- start start 2024-07-16T10:08:01.834 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:08:01.819+0000 7fd10cf5a640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd1081023a0 0x7fd1081131f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:08:01.834 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:08:01.819+0000 7fd10cf5a640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd108102d40 0x7fd108113730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:08:01.834 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:08:01.819+0000 7fd10cf5a640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd108113dc0 con 0x7fd1081023a0 2024-07-16T10:08:01.834 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:08:01.819+0000 7fd10cf5a640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd10810d170 con 0x7fd108102d40 2024-07-16T10:08:01.835 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:08:01.819+0000 7fd10659c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd1081023a0 0x7fd1081131f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:08:01.835 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:08:01.819+0000 7fd10659c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd1081023a0 0x7fd1081131f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:45674/0 (socket says 172.21.15.156:45674) 2024-07-16T10:08:01.835 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:08:01.819+0000 7fd10659c640 1 -- 172.21.15.156:0/998818622 learned_addr learned my addr 172.21.15.156:0/998818622 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T10:08:01.835 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:08:01.819+0000 7fd10659c640 1 -- 172.21.15.156:0/998818622 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd108102d40 msgr2=0x7fd108113730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:08:01.835 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:08:01.819+0000 7fd10659c640 1 --2- 172.21.15.156:0/998818622 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd108102d40 0x7fd108113730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:08:01.835 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:08:01.819+0000 7fd10659c640 1 -- 172.21.15.156:0/998818622 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0f8005040 con 0x7fd1081023a0 2024-07-16T10:08:01.835 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:08:01.819+0000 7fd10659c640 1 --2- 172.21.15.156:0/998818622 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd1081023a0 0x7fd1081131f0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fd0f8008fc0 tx=0x7fd0f8009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T10:08:01.835 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:08:01.820+0000 7fd0f6ffd640 1 -- 172.21.15.156:0/998818622 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd0f8038070 con 0x7fd1081023a0 2024-07-16T10:08:01.835 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:08:01.820+0000 7fd0f6ffd640 1 -- 172.21.15.156:0/998818622 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd0f8032b80 con 0x7fd1081023a0 2024-07-16T10:08:01.835 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:08:01.820+0000 7fd10cf5a640 1 -- 172.21.15.156:0/998818622 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd10810d3f0 con 0x7fd1081023a0 2024-07-16T10:08:01.835 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:08:01.820+0000 7fd0f6ffd640 1 -- 172.21.15.156:0/998818622 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd0f803dc20 con 0x7fd1081023a0 2024-07-16T10:08:01.835 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:08:01.820+0000 7fd10cf5a640 1 -- 172.21.15.156:0/998818622 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd10810d8b0 con 0x7fd1081023a0 2024-07-16T10:08:01.836 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:08:01.821+0000 7fd10cf5a640 1 -- 172.21.15.156:0/998818622 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1080655c0 con 0x7fd1081023a0 2024-07-16T10:08:01.836 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:08:01.821+0000 7fd0f6ffd640 1 -- 172.21.15.156:0/998818622 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 38) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fd0f8030020 con 0x7fd1081023a0 2024-07-16T10:08:01.836 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:08:01.822+0000 7fd0f6ffd640 1 --2- 172.21.15.156:0/998818622 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7fd0f006e0b0 0x7fd0f0070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:08:01.836 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:08:01.822+0000 7fd0f6ffd640 1 -- 172.21.15.156:0/998818622 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7fd0f80ae560 con 0x7fd1081023a0 2024-07-16T10:08:01.836 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:08:01.822+0000 7fd105d9b640 1 --2- 172.21.15.156:0/998818622 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7fd0f006e0b0 0x7fd0f0070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:08:01.836 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:08:01.822+0000 7fd105d9b640 1 --2- 172.21.15.156:0/998818622 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7fd0f006e0b0 0x7fd0f0070560 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fd10810eab0 tx=0x7fd0fc006d20 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T10:08:01.836 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:08:01.826+0000 7fd0f6ffd640 1 -- 172.21.15.156:0/998818622 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187716 (secure 0 0 0) 0x7fd0f80778b0 con 0x7fd1081023a0 2024-07-16T10:08:01.924 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:08:01 smithi156 ceph-mon[93066]: pgmap v344: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:08:02.044 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:08:02.042+0000 7fd10cf5a640 1 -- 172.21.15.156:0/998818622 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 42, "format": "json"} v 0) v1 -- 0x7fd108074a10 con 0x7fd1081023a0 2024-07-16T10:08:02.044 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:08:02.043+0000 7fd0f6ffd640 1 -- 172.21.15.156:0/998818622 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 42, "format": "json"}]=0 dumped fsmap epoch 42 v45) v1 ==== 107+0+5221 (secure 0 0 0) 0x7fd108074a10 con 0x7fd1081023a0 2024-07-16T10:08:02.044 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-07-16T10:08:02.045 INFO:teuthology.orchestra.run.smithi156.stdout:{"epoch":42,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34470,"name":"cephfs.smithi156.evwwaz","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6829/1683974145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":1683974145},{"type":"v1","addr":"172.21.15.156:6829","nonce":1683974145}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":37},{"gid":44347,"name":"cephfs.smithi176.dvvbud","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6825/1700587296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6824","nonce":1700587296},{"type":"v1","addr":"172.21.15.176:6825","nonce":1700587296}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":44373,"name":"cephfs.smithi176.wahbsi","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6827/2277003188","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6826","nonce":2277003188},{"type":"v1","addr":"172.21.15.176:6827","nonce":2277003188}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41}],"filesystems":[{"mdsmap":{"epoch":42,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-16T09:52:55.214975+0000","modified":"2024-07-16T10:03:53.758433+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":82,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0],"up":{"mds_0":34438},"failed":[],"damaged":[],"stopped":[1],"info":{"gid_34438":{"gid":34438,"name":"cephfs.smithi156.rdibys","rank":0,"incarnation":33,"state":"up:active","state_seq":9,"addr":"172.21.15.156:6827/2852996640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":2852996640},{"type":"v1","addr":"172.21.15.156:6827","nonce":2852996640}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":34438,"qdb_cluster":[34438]},"id":1}]} 2024-07-16T10:08:02.045 INFO:teuthology.orchestra.run.smithi156.stderr:dumped fsmap epoch 42 2024-07-16T10:08:02.046 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:08:02.045+0000 7fd10cf5a640 1 -- 172.21.15.156:0/998818622 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7fd0f006e0b0 msgr2=0x7fd0f0070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:08:02.047 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:08:02.045+0000 7fd10cf5a640 1 --2- 172.21.15.156:0/998818622 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7fd0f006e0b0 0x7fd0f0070560 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fd10810eab0 tx=0x7fd0fc006d20 comp rx=0 tx=0).stop 2024-07-16T10:08:02.047 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:08:02.045+0000 7fd10cf5a640 1 -- 172.21.15.156:0/998818622 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd1081023a0 msgr2=0x7fd1081131f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:08:02.047 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:08:02.045+0000 7fd10cf5a640 1 --2- 172.21.15.156:0/998818622 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd1081023a0 0x7fd1081131f0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fd0f8008fc0 tx=0x7fd0f8009da0 comp rx=0 tx=0).stop 2024-07-16T10:08:02.047 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:08:02.045+0000 7fd10cf5a640 1 -- 172.21.15.156:0/998818622 shutdown_connections 2024-07-16T10:08:02.047 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:08:02.045+0000 7fd10cf5a640 1 --2- 172.21.15.156:0/998818622 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7fd0f006e0b0 0x7fd0f0070560 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:08:02.047 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:08:02.045+0000 7fd10cf5a640 1 --2- 172.21.15.156:0/998818622 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd108102d40 0x7fd108113730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:08:02.047 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:08:02.045+0000 7fd10cf5a640 1 --2- 172.21.15.156:0/998818622 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd1081023a0 0x7fd1081131f0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:08:02.047 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:08:02.045+0000 7fd10cf5a640 1 -- 172.21.15.156:0/998818622 >> 172.21.15.156:0/998818622 conn(0x7fd108070af0 msgr2=0x7fd108105160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T10:08:02.047 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:08:02.045+0000 7fd10cf5a640 1 -- 172.21.15.156:0/998818622 shutdown_connections 2024-07-16T10:08:02.047 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:08:02.045+0000 7fd10cf5a640 1 -- 172.21.15.156:0/998818622 wait complete. 2024-07-16T10:08:02.093 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:08:01 smithi176 ceph-mon[81689]: pgmap v344: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:08:02.789 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 42 2024-07-16T10:08:02.789 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 26c91c26-4358-11ef-bcad-c7b262605968 -- ceph fs dump --format=json 43 2024-07-16T10:08:02.924 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:08:02 smithi156 ceph-mon[93066]: from='client.? 172.21.15.156:0/998818622' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 42, "format": "json"}]: dispatch 2024-07-16T10:08:03.047 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/26c91c26-4358-11ef-bcad-c7b262605968/mon.smithi156/config 2024-07-16T10:08:03.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:08:02 smithi176 ceph-mon[81689]: from='client.? 172.21.15.156:0/998818622' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 42, "format": "json"}]: dispatch 2024-07-16T10:08:03.828 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:08:03 smithi156 ceph-mon[93066]: pgmap v345: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:08:03.828 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:08:03 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-16T10:08:04.093 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:08:03 smithi176 ceph-mon[81689]: pgmap v345: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:08:04.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:08:03 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-16T10:08:04.723 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:08:04.711+0000 7fceec19c640 1 -- 172.21.15.156:0/787094614 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcee4101280 msgr2=0x7fcee4101650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:08:04.723 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:08:04.711+0000 7fceec19c640 1 --2- 172.21.15.156:0/787094614 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcee4101280 0x7fcee4101650 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fcee00048d0 tx=0x7fcee0009da0 comp rx=0 tx=0).stop 2024-07-16T10:08:04.723 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:08:04.711+0000 7fceec19c640 1 -- 172.21.15.156:0/787094614 shutdown_connections 2024-07-16T10:08:04.723 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:08:04.711+0000 7fceec19c640 1 --2- 172.21.15.156:0/787094614 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fcee4101c20 0x7fcee40fb2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:08:04.723 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:08:04.711+0000 7fceec19c640 1 --2- 172.21.15.156:0/787094614 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcee4101280 0x7fcee4101650 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:08:04.723 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:08:04.711+0000 7fceec19c640 1 -- 172.21.15.156:0/787094614 >> 172.21.15.156:0/787094614 conn(0x7fcee40f6e70 msgr2=0x7fcee40f9280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T10:08:04.724 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:08:04.711+0000 7fceec19c640 1 -- 172.21.15.156:0/787094614 shutdown_connections 2024-07-16T10:08:04.724 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:08:04.711+0000 7fceec19c640 1 -- 172.21.15.156:0/787094614 wait complete. 2024-07-16T10:08:04.724 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:08:04.711+0000 7fceec19c640 1 Processor -- start 2024-07-16T10:08:04.724 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:08:04.711+0000 7fceec19c640 1 -- start start 2024-07-16T10:08:04.724 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:08:04.711+0000 7fceec19c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcee4101280 0x7fcee4102100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:08:04.724 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:08:04.711+0000 7fceec19c640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fcee4101c20 0x7fcee4102640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:08:04.724 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:08:04.711+0000 7fceec19c640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcee4102c10 con 0x7fcee4101280 2024-07-16T10:08:04.724 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:08:04.711+0000 7fceec19c640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcee4102d80 con 0x7fcee4101c20 2024-07-16T10:08:04.724 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:08:04.712+0000 7fcee9f38640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcee4101280 0x7fcee4102100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:08:04.724 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:08:04.712+0000 7fcee9f38640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcee4101280 0x7fcee4102100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:56800/0 (socket says 172.21.15.156:56800) 2024-07-16T10:08:04.724 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:08:04.712+0000 7fcee9f38640 1 -- 172.21.15.156:0/1355961333 learned_addr learned my addr 172.21.15.156:0/1355961333 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T10:08:04.724 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:08:04.712+0000 7fcee9737640 1 --2- 172.21.15.156:0/1355961333 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fcee4101c20 0x7fcee4102640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:08:04.724 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:08:04.712+0000 7fcee9f38640 1 -- 172.21.15.156:0/1355961333 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fcee4101c20 msgr2=0x7fcee4102640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:08:04.725 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:08:04.712+0000 7fcee9f38640 1 --2- 172.21.15.156:0/1355961333 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fcee4101c20 0x7fcee4102640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:08:04.725 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:08:04.712+0000 7fcee9f38640 1 -- 172.21.15.156:0/1355961333 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcee0005040 con 0x7fcee4101280 2024-07-16T10:08:04.725 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:08:04.712+0000 7fcee9737640 1 --2- 172.21.15.156:0/1355961333 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fcee4101c20 0x7fcee4102640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-16T10:08:04.725 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:08:04.712+0000 7fcee9f38640 1 --2- 172.21.15.156:0/1355961333 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcee4101280 0x7fcee4102100 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fcee0003080 tx=0x7fcee0009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T10:08:04.725 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:08:04.712+0000 7fceda7fc640 1 -- 172.21.15.156:0/1355961333 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcee0038070 con 0x7fcee4101280 2024-07-16T10:08:04.725 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:08:04.712+0000 7fceda7fc640 1 -- 172.21.15.156:0/1355961333 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcee0008ee0 con 0x7fcee4101280 2024-07-16T10:08:04.725 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:08:04.712+0000 7fceec19c640 1 -- 172.21.15.156:0/1355961333 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcee4119400 con 0x7fcee4101280 2024-07-16T10:08:04.725 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:08:04.712+0000 7fceda7fc640 1 -- 172.21.15.156:0/1355961333 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcee003d870 con 0x7fcee4101280 2024-07-16T10:08:04.725 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:08:04.712+0000 7fceec19c640 1 -- 172.21.15.156:0/1355961333 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcee41197e0 con 0x7fcee4101280 2024-07-16T10:08:04.725 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:08:04.713+0000 7fceec19c640 1 -- 172.21.15.156:0/1355961333 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcee40fc9f0 con 0x7fcee4101280 2024-07-16T10:08:04.725 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:08:04.714+0000 7fceda7fc640 1 -- 172.21.15.156:0/1355961333 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 38) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fcee0030020 con 0x7fcee4101280 2024-07-16T10:08:04.725 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:08:04.714+0000 7fceda7fc640 1 --2- 172.21.15.156:0/1355961333 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7fced006e0b0 0x7fced0070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:08:04.725 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:08:04.714+0000 7fceda7fc640 1 -- 172.21.15.156:0/1355961333 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7fcee00ad810 con 0x7fcee4101280 2024-07-16T10:08:04.726 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:08:04.714+0000 7fcee9737640 1 --2- 172.21.15.156:0/1355961333 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7fced006e0b0 0x7fced0070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:08:04.726 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:08:04.715+0000 7fcee9737640 1 --2- 172.21.15.156:0/1355961333 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7fced006e0b0 0x7fced0070560 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fcee40f8880 tx=0x7fced4008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T10:08:04.726 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:08:04.719+0000 7fceda7fc640 1 -- 172.21.15.156:0/1355961333 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187716 (secure 0 0 0) 0x7fcee0076c00 con 0x7fcee4101280 2024-07-16T10:08:04.944 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:08:04.942+0000 7fceec19c640 1 -- 172.21.15.156:0/1355961333 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 43, "format": "json"} v 0) v1 -- 0x7fcee40faa70 con 0x7fcee4101280 2024-07-16T10:08:04.944 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:08:04.943+0000 7fceda7fc640 1 -- 172.21.15.156:0/1355961333 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 43, "format": "json"}]=0 dumped fsmap epoch 43 v45) v1 ==== 107+0+5238 (secure 0 0 0) 0x7fcee007a2c0 con 0x7fcee4101280 2024-07-16T10:08:04.944 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-07-16T10:08:04.945 INFO:teuthology.orchestra.run.smithi156.stdout:{"epoch":43,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":44347,"name":"cephfs.smithi176.dvvbud","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6825/1700587296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6824","nonce":1700587296},{"type":"v1","addr":"172.21.15.176:6825","nonce":1700587296}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":44373,"name":"cephfs.smithi176.wahbsi","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6827/2277003188","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6826","nonce":2277003188},{"type":"v1","addr":"172.21.15.176:6827","nonce":2277003188}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41}],"filesystems":[{"mdsmap":{"epoch":43,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-16T09:52:55.214975+0000","modified":"2024-07-16T10:03:54.764145+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":82,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":34438,"mds_1":34470},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34438":{"gid":34438,"name":"cephfs.smithi156.rdibys","rank":0,"incarnation":33,"state":"up:active","state_seq":9,"addr":"172.21.15.156:6827/2852996640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":2852996640},{"type":"v1","addr":"172.21.15.156:6827","nonce":2852996640}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34470":{"gid":34470,"name":"cephfs.smithi156.evwwaz","rank":1,"incarnation":43,"state":"up:starting","state_seq":1,"addr":"172.21.15.156:6829/1683974145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":1683974145},{"type":"v1","addr":"172.21.15.156:6829","nonce":1683974145}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":34438,"qdb_cluster":[34438]},"id":1}]} 2024-07-16T10:08:04.945 INFO:teuthology.orchestra.run.smithi156.stderr:dumped fsmap epoch 43 2024-07-16T10:08:04.947 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:08:04.945+0000 7fceec19c640 1 -- 172.21.15.156:0/1355961333 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7fced006e0b0 msgr2=0x7fced0070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:08:04.947 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:08:04.945+0000 7fceec19c640 1 --2- 172.21.15.156:0/1355961333 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7fced006e0b0 0x7fced0070560 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fcee40f8880 tx=0x7fced4008040 comp rx=0 tx=0).stop 2024-07-16T10:08:04.947 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:08:04.945+0000 7fceec19c640 1 -- 172.21.15.156:0/1355961333 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcee4101280 msgr2=0x7fcee4102100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:08:04.947 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:08:04.945+0000 7fceec19c640 1 --2- 172.21.15.156:0/1355961333 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcee4101280 0x7fcee4102100 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fcee0003080 tx=0x7fcee0009da0 comp rx=0 tx=0).stop 2024-07-16T10:08:04.947 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:08:04.945+0000 7fceec19c640 1 -- 172.21.15.156:0/1355961333 shutdown_connections 2024-07-16T10:08:04.947 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:08:04.945+0000 7fceec19c640 1 --2- 172.21.15.156:0/1355961333 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7fced006e0b0 0x7fced0070560 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:08:04.947 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:08:04.945+0000 7fceec19c640 1 --2- 172.21.15.156:0/1355961333 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fcee4101c20 0x7fcee4102640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:08:04.947 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:08:04.945+0000 7fceec19c640 1 --2- 172.21.15.156:0/1355961333 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcee4101280 0x7fcee4102100 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:08:04.947 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:08:04.945+0000 7fceec19c640 1 -- 172.21.15.156:0/1355961333 >> 172.21.15.156:0/1355961333 conn(0x7fcee40f6e70 msgr2=0x7fcee4065880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T10:08:04.947 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:08:04.946+0000 7fceec19c640 1 -- 172.21.15.156:0/1355961333 shutdown_connections 2024-07-16T10:08:04.947 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:08:04.946+0000 7fceec19c640 1 -- 172.21.15.156:0/1355961333 wait complete. 2024-07-16T10:08:05.604 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 43 2024-07-16T10:08:05.604 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 26c91c26-4358-11ef-bcad-c7b262605968 -- ceph fs dump --format=json 44 2024-07-16T10:08:05.903 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/26c91c26-4358-11ef-bcad-c7b262605968/mon.smithi156/config 2024-07-16T10:08:05.924 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:08:05 smithi156 ceph-mon[93066]: pgmap v346: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-07-16T10:08:05.924 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:08:05 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-16T10:08:05.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:08:05 smithi156 ceph-mon[93066]: from='client.? 172.21.15.156:0/1355961333' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 43, "format": "json"}]: dispatch 2024-07-16T10:08:05.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:08:05 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:08:05.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:08:05 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:08:06.093 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:08:05 smithi176 ceph-mon[81689]: pgmap v346: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-07-16T10:08:06.093 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:08:05 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-16T10:08:06.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:08:05 smithi176 ceph-mon[81689]: from='client.? 172.21.15.156:0/1355961333' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 43, "format": "json"}]: dispatch 2024-07-16T10:08:06.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:08:05 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:08:06.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:08:05 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:08:06.924 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:08:06 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-16T10:08:06.924 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:08:06 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-16T10:08:06.924 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:08:06 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:08:07.093 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:08:06 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-16T10:08:07.093 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:08:06 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-16T10:08:07.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:08:06 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:08:07.643 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:08:07.640+0000 7f303c6f2640 1 -- 172.21.15.156:0/3103803234 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f30340fbf30 msgr2=0x7f30340fce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:08:07.643 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:08:07.640+0000 7f303c6f2640 1 --2- 172.21.15.156:0/3103803234 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f30340fbf30 0x7f30340fce30 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f30280048d0 tx=0x7f3028009da0 comp rx=0 tx=0).stop 2024-07-16T10:08:07.644 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:08:07.640+0000 7f303c6f2640 1 -- 172.21.15.156:0/3103803234 shutdown_connections 2024-07-16T10:08:07.644 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:08:07.640+0000 7f303c6f2640 1 --2- 172.21.15.156:0/3103803234 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f30340fbf30 0x7f30340fce30 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:08:07.644 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:08:07.640+0000 7f303c6f2640 1 --2- 172.21.15.156:0/3103803234 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f30340fb620 0x7f30340fb9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:08:07.644 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:08:07.640+0000 7f303c6f2640 1 -- 172.21.15.156:0/3103803234 >> 172.21.15.156:0/3103803234 conn(0x7f30340721c0 msgr2=0x7f30340725c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T10:08:07.644 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:08:07.640+0000 7f303c6f2640 1 -- 172.21.15.156:0/3103803234 shutdown_connections 2024-07-16T10:08:07.644 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:08:07.640+0000 7f303c6f2640 1 -- 172.21.15.156:0/3103803234 wait complete. 2024-07-16T10:08:07.644 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:08:07.640+0000 7f303c6f2640 1 Processor -- start 2024-07-16T10:08:07.644 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:08:07.640+0000 7f303c6f2640 1 -- start start 2024-07-16T10:08:07.644 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:08:07.641+0000 7f303c6f2640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f30340fb620 0x7f3034112000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:08:07.644 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:08:07.641+0000 7f303c6f2640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f30340fbf30 0x7f3034112540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:08:07.644 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:08:07.641+0000 7f303c6f2640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3034112bd0 con 0x7f30340fbf30 2024-07-16T10:08:07.644 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:08:07.641+0000 7f303c6f2640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f303410bf80 con 0x7f30340fb620 2024-07-16T10:08:07.645 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:08:07.641+0000 7f3039c8d640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f30340fbf30 0x7f3034112540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:08:07.645 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:08:07.641+0000 7f3039c8d640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f30340fbf30 0x7f3034112540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:56814/0 (socket says 172.21.15.156:56814) 2024-07-16T10:08:07.645 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:08:07.641+0000 7f3039c8d640 1 -- 172.21.15.156:0/2363311968 learned_addr learned my addr 172.21.15.156:0/2363311968 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T10:08:07.645 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:08:07.641+0000 7f303a48e640 1 --2- 172.21.15.156:0/2363311968 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f30340fb620 0x7f3034112000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:08:07.645 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:08:07.641+0000 7f3039c8d640 1 -- 172.21.15.156:0/2363311968 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f30340fb620 msgr2=0x7f3034112000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:08:07.645 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:08:07.641+0000 7f3039c8d640 1 --2- 172.21.15.156:0/2363311968 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f30340fb620 0x7f3034112000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:08:07.645 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:08:07.641+0000 7f3039c8d640 1 -- 172.21.15.156:0/2363311968 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3028005040 con 0x7f30340fbf30 2024-07-16T10:08:07.645 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:08:07.641+0000 7f303a48e640 1 --2- 172.21.15.156:0/2363311968 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f30340fb620 0x7f3034112000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-16T10:08:07.645 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:08:07.641+0000 7f3039c8d640 1 --2- 172.21.15.156:0/2363311968 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f30340fbf30 0x7f3034112540 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f3028003080 tx=0x7f3028009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T10:08:07.645 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:08:07.642+0000 7f3026ffd640 1 -- 172.21.15.156:0/2363311968 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3028038070 con 0x7f30340fbf30 2024-07-16T10:08:07.645 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:08:07.642+0000 7f3026ffd640 1 -- 172.21.15.156:0/2363311968 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3028008ee0 con 0x7f30340fbf30 2024-07-16T10:08:07.645 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:08:07.642+0000 7f3026ffd640 1 -- 172.21.15.156:0/2363311968 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f302803d870 con 0x7f30340fbf30 2024-07-16T10:08:07.646 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:08:07.642+0000 7f303c6f2640 1 -- 172.21.15.156:0/2363311968 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f303410c200 con 0x7f30340fbf30 2024-07-16T10:08:07.646 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:08:07.642+0000 7f303c6f2640 1 -- 172.21.15.156:0/2363311968 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f303410c6c0 con 0x7f30340fbf30 2024-07-16T10:08:07.647 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:08:07.643+0000 7f303c6f2640 1 -- 172.21.15.156:0/2363311968 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30340fc9d0 con 0x7f30340fbf30 2024-07-16T10:08:07.647 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:08:07.643+0000 7f3026ffd640 1 -- 172.21.15.156:0/2363311968 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 38) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f3028030020 con 0x7f30340fbf30 2024-07-16T10:08:07.647 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:08:07.644+0000 7f3026ffd640 1 --2- 172.21.15.156:0/2363311968 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f302006e0b0 0x7f3020070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:08:07.647 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:08:07.644+0000 7f3026ffd640 1 -- 172.21.15.156:0/2363311968 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f30280ad860 con 0x7f30340fbf30 2024-07-16T10:08:07.647 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:08:07.644+0000 7f303a48e640 1 --2- 172.21.15.156:0/2363311968 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f302006e0b0 0x7f3020070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:08:07.647 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:08:07.644+0000 7f303a48e640 1 --2- 172.21.15.156:0/2363311968 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f302006e0b0 0x7f3020070560 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f30300099f0 tx=0x7f3030008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T10:08:07.649 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:08:07.648+0000 7f3026ffd640 1 -- 172.21.15.156:0/2363311968 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187716 (secure 0 0 0) 0x7f3028076c50 con 0x7f30340fbf30 2024-07-16T10:08:07.873 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:08:07.871+0000 7f303c6f2640 1 -- 172.21.15.156:0/2363311968 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 44, "format": "json"} v 0) v1 -- 0x7f3034072cd0 con 0x7f30340fbf30 2024-07-16T10:08:07.873 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:08:07.871+0000 7f3026ffd640 1 -- 172.21.15.156:0/2363311968 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 44, "format": "json"}]=0 dumped fsmap epoch 44 v45) v1 ==== 107+0+5242 (secure 0 0 0) 0x7f302807a310 con 0x7f30340fbf30 2024-07-16T10:08:07.873 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-07-16T10:08:07.873 INFO:teuthology.orchestra.run.smithi156.stdout:{"epoch":44,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":44347,"name":"cephfs.smithi176.dvvbud","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6825/1700587296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6824","nonce":1700587296},{"type":"v1","addr":"172.21.15.176:6825","nonce":1700587296}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":44373,"name":"cephfs.smithi176.wahbsi","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6827/2277003188","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6826","nonce":2277003188},{"type":"v1","addr":"172.21.15.176:6827","nonce":2277003188}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41}],"filesystems":[{"mdsmap":{"epoch":44,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-16T09:52:55.214975+0000","modified":"2024-07-16T10:03:55.768394+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":82,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":34438,"mds_1":34470},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34438":{"gid":34438,"name":"cephfs.smithi156.rdibys","rank":0,"incarnation":33,"state":"up:active","state_seq":9,"addr":"172.21.15.156:6827/2852996640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":2852996640},{"type":"v1","addr":"172.21.15.156:6827","nonce":2852996640}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34470":{"gid":34470,"name":"cephfs.smithi156.evwwaz","rank":1,"incarnation":43,"state":"up:active","state_seq":11,"addr":"172.21.15.156:6829/1683974145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":1683974145},{"type":"v1","addr":"172.21.15.156:6829","nonce":1683974145}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":34438,"qdb_cluster":[34438,34470]},"id":1}]} 2024-07-16T10:08:07.874 INFO:teuthology.orchestra.run.smithi156.stderr:dumped fsmap epoch 44 2024-07-16T10:08:07.875 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:08:07.874+0000 7f303c6f2640 1 -- 172.21.15.156:0/2363311968 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f302006e0b0 msgr2=0x7f3020070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:08:07.875 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:08:07.874+0000 7f303c6f2640 1 --2- 172.21.15.156:0/2363311968 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f302006e0b0 0x7f3020070560 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f30300099f0 tx=0x7f3030008040 comp rx=0 tx=0).stop 2024-07-16T10:08:07.875 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:08:07.874+0000 7f303c6f2640 1 -- 172.21.15.156:0/2363311968 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f30340fbf30 msgr2=0x7f3034112540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:08:07.875 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:08:07.874+0000 7f303c6f2640 1 --2- 172.21.15.156:0/2363311968 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f30340fbf30 0x7f3034112540 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f3028003080 tx=0x7f3028009da0 comp rx=0 tx=0).stop 2024-07-16T10:08:07.876 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:08:07.874+0000 7f303c6f2640 1 -- 172.21.15.156:0/2363311968 shutdown_connections 2024-07-16T10:08:07.876 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:08:07.874+0000 7f303c6f2640 1 --2- 172.21.15.156:0/2363311968 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f302006e0b0 0x7f3020070560 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:08:07.876 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:08:07.874+0000 7f303c6f2640 1 --2- 172.21.15.156:0/2363311968 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f30340fbf30 0x7f3034112540 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:08:07.876 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:08:07.874+0000 7f303c6f2640 1 --2- 172.21.15.156:0/2363311968 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f30340fb620 0x7f3034112000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:08:07.876 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:08:07.874+0000 7f303c6f2640 1 -- 172.21.15.156:0/2363311968 >> 172.21.15.156:0/2363311968 conn(0x7f30340721c0 msgr2=0x7f30340f86d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T10:08:07.876 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:08:07.874+0000 7f303c6f2640 1 -- 172.21.15.156:0/2363311968 shutdown_connections 2024-07-16T10:08:07.876 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:08:07.874+0000 7f303c6f2640 1 -- 172.21.15.156:0/2363311968 wait complete. 2024-07-16T10:08:08.093 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:08:07 smithi176 ceph-mon[81689]: pgmap v347: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:08:08.166 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:08:07 smithi156 ceph-mon[93066]: pgmap v347: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:08:08.475 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2024-07-16T10:08:08.488 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2024-07-16T10:08:08.488 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2024-07-16T10:08:08.489 DEBUG:teuthology.orchestra.run.smithi156:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-16T10:08:08.507 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2024-07-16T10:08:08.507 DEBUG:teuthology.orchestra.run.smithi156:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-16T10:08:08.564 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 26c91c26-4358-11ef-bcad-c7b262605968 -- ceph osd blocklist ls 2024-07-16T10:08:08.862 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/26c91c26-4358-11ef-bcad-c7b262605968/mon.smithi156/config 2024-07-16T10:08:09.093 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:08:08 smithi176 ceph-mon[81689]: from='client.? 172.21.15.156:0/2363311968' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 44, "format": "json"}]: dispatch 2024-07-16T10:08:09.128 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:08:08 smithi156 ceph-mon[93066]: from='client.? 172.21.15.156:0/2363311968' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 44, "format": "json"}]: dispatch 2024-07-16T10:08:10.093 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:08:09 smithi176 ceph-mon[81689]: pgmap v348: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-07-16T10:08:10.163 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:08:09 smithi156 ceph-mon[93066]: pgmap v348: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-07-16T10:08:10.569 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:08:10.545+0000 7fc763e38640 1 -- 172.21.15.156:0/3128276673 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc75c1023a0 msgr2=0x7fc75c102770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:08:10.569 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:08:10.545+0000 7fc763e38640 1 --2- 172.21.15.156:0/3128276673 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc75c1023a0 0x7fc75c102770 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fc74c0048d0 tx=0x7fc74c009da0 comp rx=0 tx=0).stop 2024-07-16T10:08:10.569 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:08:10.545+0000 7fc763e38640 1 -- 172.21.15.156:0/3128276673 shutdown_connections 2024-07-16T10:08:10.569 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:08:10.545+0000 7fc763e38640 1 --2- 172.21.15.156:0/3128276673 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc75c102d40 0x7fc75c0faa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:08:10.569 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:08:10.545+0000 7fc763e38640 1 --2- 172.21.15.156:0/3128276673 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc75c1023a0 0x7fc75c102770 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:08:10.569 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:08:10.545+0000 7fc763e38640 1 -- 172.21.15.156:0/3128276673 >> 172.21.15.156:0/3128276673 conn(0x7fc75c070af0 msgr2=0x7fc75c070ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T10:08:10.569 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:08:10.545+0000 7fc763e38640 1 -- 172.21.15.156:0/3128276673 shutdown_connections 2024-07-16T10:08:10.570 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:08:10.545+0000 7fc763e38640 1 -- 172.21.15.156:0/3128276673 wait complete. 2024-07-16T10:08:10.570 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:08:10.546+0000 7fc763e38640 1 Processor -- start 2024-07-16T10:08:10.570 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:08:10.546+0000 7fc763e38640 1 -- start start 2024-07-16T10:08:10.570 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:08:10.546+0000 7fc763e38640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc75c1023a0 0x7fc75c1131b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:08:10.570 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:08:10.546+0000 7fc763e38640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc75c102d40 0x7fc75c1136f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:08:10.570 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:08:10.546+0000 7fc763e38640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc75c113cf0 con 0x7fc75c1023a0 2024-07-16T10:08:10.570 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:08:10.546+0000 7fc763e38640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc75c10d130 con 0x7fc75c102d40 2024-07-16T10:08:10.570 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:08:10.546+0000 7fc761bd4640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc75c1023a0 0x7fc75c1131b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:08:10.570 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:08:10.546+0000 7fc761bd4640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc75c1023a0 0x7fc75c1131b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:56828/0 (socket says 172.21.15.156:56828) 2024-07-16T10:08:10.570 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:08:10.546+0000 7fc761bd4640 1 -- 172.21.15.156:0/1849667137 learned_addr learned my addr 172.21.15.156:0/1849667137 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T10:08:10.570 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:08:10.546+0000 7fc7613d3640 1 --2- 172.21.15.156:0/1849667137 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc75c102d40 0x7fc75c1136f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:08:10.570 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:08:10.546+0000 7fc761bd4640 1 -- 172.21.15.156:0/1849667137 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc75c102d40 msgr2=0x7fc75c1136f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:08:10.571 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:08:10.546+0000 7fc761bd4640 1 --2- 172.21.15.156:0/1849667137 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc75c102d40 0x7fc75c1136f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:08:10.571 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:08:10.546+0000 7fc761bd4640 1 -- 172.21.15.156:0/1849667137 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc74c005040 con 0x7fc75c1023a0 2024-07-16T10:08:10.571 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:08:10.546+0000 7fc7613d3640 1 --2- 172.21.15.156:0/1849667137 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc75c102d40 0x7fc75c1136f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-16T10:08:10.571 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:08:10.547+0000 7fc761bd4640 1 --2- 172.21.15.156:0/1849667137 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc75c1023a0 0x7fc75c1131b0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fc74c0043d0 tx=0x7fc74c009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T10:08:10.571 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:08:10.547+0000 7fc7527fc640 1 -- 172.21.15.156:0/1849667137 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc74c038070 con 0x7fc75c1023a0 2024-07-16T10:08:10.571 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:08:10.547+0000 7fc7527fc640 1 -- 172.21.15.156:0/1849667137 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc74c02abd0 con 0x7fc75c1023a0 2024-07-16T10:08:10.571 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:08:10.547+0000 7fc763e38640 1 -- 172.21.15.156:0/1849667137 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc75c10d3b0 con 0x7fc75c1023a0 2024-07-16T10:08:10.571 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:08:10.547+0000 7fc7527fc640 1 -- 172.21.15.156:0/1849667137 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc74c033640 con 0x7fc75c1023a0 2024-07-16T10:08:10.571 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:08:10.547+0000 7fc763e38640 1 -- 172.21.15.156:0/1849667137 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc75c10d810 con 0x7fc75c1023a0 2024-07-16T10:08:10.571 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:08:10.548+0000 7fc763e38640 1 -- 172.21.15.156:0/1849667137 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc75c0655c0 con 0x7fc75c1023a0 2024-07-16T10:08:10.571 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:08:10.548+0000 7fc7527fc640 1 -- 172.21.15.156:0/1849667137 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 38) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fc74c030020 con 0x7fc75c1023a0 2024-07-16T10:08:10.571 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:08:10.549+0000 7fc7527fc640 1 --2- 172.21.15.156:0/1849667137 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7fc74806e0b0 0x7fc748070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:08:10.571 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:08:10.549+0000 7fc7527fc640 1 -- 172.21.15.156:0/1849667137 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7fc74c0ada20 con 0x7fc75c1023a0 2024-07-16T10:08:10.572 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:08:10.549+0000 7fc7613d3640 1 --2- 172.21.15.156:0/1849667137 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7fc74806e0b0 0x7fc748070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:08:10.572 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:08:10.549+0000 7fc7613d3640 1 --2- 172.21.15.156:0/1849667137 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7fc74806e0b0 0x7fc748070560 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fc75c10ea70 tx=0x7fc75800b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T10:08:10.572 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:08:10.553+0000 7fc7527fc640 1 -- 172.21.15.156:0/1849667137 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187716 (secure 0 0 0) 0x7fc74c076cf0 con 0x7fc75c1023a0 2024-07-16T10:08:10.742 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:08:10.741+0000 7fc763e38640 1 -- 172.21.15.156:0/1849667137 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fc75c04b530 con 0x7fc75c1023a0 2024-07-16T10:08:10.743 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:08:10.741+0000 7fc7527fc640 1 -- 172.21.15.156:0/1849667137 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 41 entries v84) v1 ==== 81+0+2456 (secure 0 0 0) 0x7fc74c07a3b0 con 0x7fc75c1023a0 2024-07-16T10:08:10.743 INFO:teuthology.orchestra.run.smithi156.stdout:172.21.15.156:6828/333092891 2024-07-17T10:03:03.351199+0000 2024-07-16T10:08:10.743 INFO:teuthology.orchestra.run.smithi156.stdout:172.21.15.156:6827/1080176013 2024-07-17T10:02:40.286020+0000 2024-07-16T10:08:10.743 INFO:teuthology.orchestra.run.smithi156.stdout:172.21.15.156:6826/1080176013 2024-07-17T10:02:40.286020+0000 2024-07-16T10:08:10.743 INFO:teuthology.orchestra.run.smithi156.stdout:172.21.15.156:0/2582266353 2024-07-17T09:57:48.301524+0000 2024-07-16T10:08:10.744 INFO:teuthology.orchestra.run.smithi156.stdout:172.21.15.156:0/788626231 2024-07-17T09:57:48.301524+0000 2024-07-16T10:08:10.744 INFO:teuthology.orchestra.run.smithi156.stdout:172.21.15.156:6801/55311371 2024-07-17T09:57:48.301524+0000 2024-07-16T10:08:10.744 INFO:teuthology.orchestra.run.smithi156.stdout:172.21.15.156:6800/55311371 2024-07-17T09:57:48.301524+0000 2024-07-16T10:08:10.744 INFO:teuthology.orchestra.run.smithi156.stdout:172.21.15.156:0/1414712577 2024-07-17T09:57:48.301524+0000 2024-07-16T10:08:10.744 INFO:teuthology.orchestra.run.smithi156.stdout:172.21.15.176:6829/673792884 2024-07-17T09:57:08.232362+0000 2024-07-16T10:08:10.744 INFO:teuthology.orchestra.run.smithi156.stdout:172.21.15.156:0/3487646818 2024-07-17T09:46:53.642082+0000 2024-07-16T10:08:10.744 INFO:teuthology.orchestra.run.smithi156.stdout:172.21.15.156:6801/1023000918 2024-07-17T09:47:22.566842+0000 2024-07-16T10:08:10.745 INFO:teuthology.orchestra.run.smithi156.stdout:172.21.15.156:0/2322505969 2024-07-17T09:47:22.566842+0000 2024-07-16T10:08:10.745 INFO:teuthology.orchestra.run.smithi156.stdout:172.21.15.156:0/2320156657 2024-07-17T09:48:24.722579+0000 2024-07-16T10:08:10.745 INFO:teuthology.orchestra.run.smithi156.stdout:172.21.15.156:0/3143160297 2024-07-17T09:57:48.301524+0000 2024-07-16T10:08:10.745 INFO:teuthology.orchestra.run.smithi156.stdout:172.21.15.156:6800/1023000918 2024-07-17T09:47:22.566842+0000 2024-07-16T10:08:10.745 INFO:teuthology.orchestra.run.smithi156.stdout:172.21.15.156:0/590617513 2024-07-17T09:55:49.326453+0000 2024-07-16T10:08:10.745 INFO:teuthology.orchestra.run.smithi156.stdout:172.21.15.156:0/3148313406 2024-07-17T09:55:49.326453+0000 2024-07-16T10:08:10.745 INFO:teuthology.orchestra.run.smithi156.stdout:172.21.15.156:0/1864977086 2024-07-17T09:47:22.566842+0000 2024-07-16T10:08:10.745 INFO:teuthology.orchestra.run.smithi156.stdout:172.21.15.156:0/3191234605 2024-07-17T09:48:24.722579+0000 2024-07-16T10:08:10.746 INFO:teuthology.orchestra.run.smithi156.stdout:172.21.15.176:0/1657551005 2024-07-17T09:57:08.232362+0000 2024-07-16T10:08:10.746 INFO:teuthology.orchestra.run.smithi156.stdout:172.21.15.156:0/1490472205 2024-07-17T09:57:48.301524+0000 2024-07-16T10:08:10.746 INFO:teuthology.orchestra.run.smithi156.stdout:172.21.15.176:6828/673792884 2024-07-17T09:57:08.232362+0000 2024-07-16T10:08:10.746 INFO:teuthology.orchestra.run.smithi156.stdout:172.21.15.156:0/3579914066 2024-07-17T09:55:49.326453+0000 2024-07-16T10:08:10.746 INFO:teuthology.orchestra.run.smithi156.stdout:172.21.15.156:0/4134717169 2024-07-17T09:57:48.301524+0000 2024-07-16T10:08:10.746 INFO:teuthology.orchestra.run.smithi156.stdout:172.21.15.176:0/4058805282 2024-07-17T09:57:08.232362+0000 2024-07-16T10:08:10.746 INFO:teuthology.orchestra.run.smithi156.stdout:172.21.15.176:0/1245693665 2024-07-17T09:57:08.232362+0000 2024-07-16T10:08:10.747 INFO:teuthology.orchestra.run.smithi156.stdout:172.21.15.176:0/3647663194 2024-07-17T09:57:08.232362+0000 2024-07-16T10:08:10.747 INFO:teuthology.orchestra.run.smithi156.stdout:172.21.15.156:6829/333092891 2024-07-17T10:03:03.351199+0000 2024-07-16T10:08:10.747 INFO:teuthology.orchestra.run.smithi156.stdout:172.21.15.176:0/1691847637 2024-07-17T09:57:08.232362+0000 2024-07-16T10:08:10.747 INFO:teuthology.orchestra.run.smithi156.stdout:172.21.15.156:0/211214934 2024-07-17T09:55:49.326453+0000 2024-07-16T10:08:10.747 INFO:teuthology.orchestra.run.smithi156.stdout:172.21.15.156:6800/2871070852 2024-07-17T09:46:53.642082+0000 2024-07-16T10:08:10.747 INFO:teuthology.orchestra.run.smithi156.stdout:172.21.15.156:0/3390422489 2024-07-17T09:46:53.642082+0000 2024-07-16T10:08:10.747 INFO:teuthology.orchestra.run.smithi156.stdout:172.21.15.156:6801/2615741717 2024-07-17T09:55:49.326453+0000 2024-07-16T10:08:10.747 INFO:teuthology.orchestra.run.smithi156.stdout:172.21.15.156:6800/2615741717 2024-07-17T09:55:49.326453+0000 2024-07-16T10:08:10.748 INFO:teuthology.orchestra.run.smithi156.stdout:172.21.15.176:0/4270704268 2024-07-17T09:57:08.232362+0000 2024-07-16T10:08:10.748 INFO:teuthology.orchestra.run.smithi156.stdout:172.21.15.156:6801/2871070852 2024-07-17T09:46:53.642082+0000 2024-07-16T10:08:10.748 INFO:teuthology.orchestra.run.smithi156.stdout:172.21.15.156:0/1325014720 2024-07-17T09:48:24.722579+0000 2024-07-16T10:08:10.748 INFO:teuthology.orchestra.run.smithi156.stdout:172.21.15.156:6800/3984987611 2024-07-17T09:48:24.722579+0000 2024-07-16T10:08:10.748 INFO:teuthology.orchestra.run.smithi156.stdout:172.21.15.156:6801/3984987611 2024-07-17T09:48:24.722579+0000 2024-07-16T10:08:10.748 INFO:teuthology.orchestra.run.smithi156.stdout:172.21.15.156:0/2120399071 2024-07-17T09:47:22.566842+0000 2024-07-16T10:08:10.748 INFO:teuthology.orchestra.run.smithi156.stdout:172.21.15.156:0/4158213614 2024-07-17T09:46:53.642082+0000 2024-07-16T10:08:10.749 INFO:teuthology.orchestra.run.smithi156.stderr:listed 41 entries 2024-07-16T10:08:10.749 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:08:10.743+0000 7fc763e38640 1 -- 172.21.15.156:0/1849667137 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7fc74806e0b0 msgr2=0x7fc748070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:08:10.749 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:08:10.743+0000 7fc763e38640 1 --2- 172.21.15.156:0/1849667137 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7fc74806e0b0 0x7fc748070560 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fc75c10ea70 tx=0x7fc75800b040 comp rx=0 tx=0).stop 2024-07-16T10:08:10.750 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:08:10.743+0000 7fc763e38640 1 -- 172.21.15.156:0/1849667137 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc75c1023a0 msgr2=0x7fc75c1131b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:08:10.750 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:08:10.743+0000 7fc763e38640 1 --2- 172.21.15.156:0/1849667137 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc75c1023a0 0x7fc75c1131b0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fc74c0043d0 tx=0x7fc74c009da0 comp rx=0 tx=0).stop 2024-07-16T10:08:10.750 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:08:10.743+0000 7fc763e38640 1 -- 172.21.15.156:0/1849667137 shutdown_connections 2024-07-16T10:08:10.750 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:08:10.743+0000 7fc763e38640 1 --2- 172.21.15.156:0/1849667137 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7fc74806e0b0 0x7fc748070560 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:08:10.750 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:08:10.743+0000 7fc763e38640 1 --2- 172.21.15.156:0/1849667137 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc75c102d40 0x7fc75c1136f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:08:10.750 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:08:10.743+0000 7fc763e38640 1 --2- 172.21.15.156:0/1849667137 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc75c1023a0 0x7fc75c1131b0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:08:10.750 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:08:10.743+0000 7fc763e38640 1 -- 172.21.15.156:0/1849667137 >> 172.21.15.156:0/1849667137 conn(0x7fc75c070af0 msgr2=0x7fc75c105160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T10:08:10.751 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:08:10.744+0000 7fc763e38640 1 -- 172.21.15.156:0/1849667137 shutdown_connections 2024-07-16T10:08:10.751 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:08:10.744+0000 7fc763e38640 1 -- 172.21.15.156:0/1849667137 wait complete. 2024-07-16T10:08:11.039 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:08:10 smithi156 ceph-mon[93066]: from='client.? 172.21.15.156:0/1849667137' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-07-16T10:08:11.093 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:08:10 smithi176 ceph-mon[81689]: from='client.? 172.21.15.156:0/1849667137' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-07-16T10:08:11.373 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2024-07-16T10:08:11.373 DEBUG:teuthology.orchestra.run.smithi156:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-16T10:08:11.391 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 26c91c26-4358-11ef-bcad-c7b262605968 -- ceph osd blocklist ls 2024-07-16T10:08:11.700 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/26c91c26-4358-11ef-bcad-c7b262605968/mon.smithi156/config 2024-07-16T10:08:11.924 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:08:11 smithi156 ceph-mon[93066]: pgmap v349: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:08:12.093 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:08:11 smithi176 ceph-mon[81689]: pgmap v349: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:08:13.287 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:08:13.286+0000 7f21112f5640 1 -- 172.21.15.156:0/2700765558 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f210c102350 msgr2=0x7f210c102720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:08:13.288 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:08:13.286+0000 7f21112f5640 1 --2- 172.21.15.156:0/2700765558 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f210c102350 0x7f210c102720 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f20fc0048d0 tx=0x7f20fc009da0 comp rx=0 tx=0).stop 2024-07-16T10:08:13.288 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:08:13.286+0000 7f21112f5640 1 -- 172.21.15.156:0/2700765558 shutdown_connections 2024-07-16T10:08:13.288 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:08:13.286+0000 7f21112f5640 1 --2- 172.21.15.156:0/2700765558 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f210c102cf0 0x7f210c0fc380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:08:13.288 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:08:13.286+0000 7f21112f5640 1 --2- 172.21.15.156:0/2700765558 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f210c102350 0x7f210c102720 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:08:13.288 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:08:13.286+0000 7f21112f5640 1 -- 172.21.15.156:0/2700765558 >> 172.21.15.156:0/2700765558 conn(0x7f210c0f7f00 msgr2=0x7f210c0fa310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T10:08:13.288 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:08:13.286+0000 7f21112f5640 1 -- 172.21.15.156:0/2700765558 shutdown_connections 2024-07-16T10:08:13.288 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:08:13.286+0000 7f21112f5640 1 -- 172.21.15.156:0/2700765558 wait complete. 2024-07-16T10:08:13.288 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:08:13.286+0000 7f21112f5640 1 Processor -- start 2024-07-16T10:08:13.288 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:08:13.287+0000 7f21112f5640 1 -- start start 2024-07-16T10:08:13.289 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:08:13.287+0000 7f21112f5640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f210c102350 0x7f210c1130b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:08:13.289 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:08:13.287+0000 7f21112f5640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f210c102cf0 0x7f210c1135f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:08:13.289 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:08:13.287+0000 7f21112f5640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f210c113bf0 con 0x7f210c102350 2024-07-16T10:08:13.290 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:08:13.287+0000 7f21112f5640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f210c10d030 con 0x7f210c102cf0 2024-07-16T10:08:13.290 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:08:13.287+0000 7f210affd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f210c102350 0x7f210c1130b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:08:13.290 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:08:13.287+0000 7f210affd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f210c102350 0x7f210c1130b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:51340/0 (socket says 172.21.15.156:51340) 2024-07-16T10:08:13.290 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:08:13.287+0000 7f210affd640 1 -- 172.21.15.156:0/2393645525 learned_addr learned my addr 172.21.15.156:0/2393645525 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-07-16T10:08:13.290 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:08:13.287+0000 7f21027fc640 1 --2- 172.21.15.156:0/2393645525 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f210c102cf0 0x7f210c1135f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:08:13.290 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:08:13.287+0000 7f210affd640 1 -- 172.21.15.156:0/2393645525 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f210c102cf0 msgr2=0x7f210c1135f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:08:13.290 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:08:13.287+0000 7f210affd640 1 --2- 172.21.15.156:0/2393645525 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f210c102cf0 0x7f210c1135f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:08:13.290 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:08:13.287+0000 7f210affd640 1 -- 172.21.15.156:0/2393645525 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20fc005040 con 0x7f210c102350 2024-07-16T10:08:13.290 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:08:13.287+0000 7f210affd640 1 --2- 172.21.15.156:0/2393645525 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f210c102350 0x7f210c1130b0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f20fc003080 tx=0x7f20fc009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T10:08:13.290 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:08:13.288+0000 7f2103fff640 1 -- 172.21.15.156:0/2393645525 <== mon.0 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f20fc038070 con 0x7f210c102350 2024-07-16T10:08:13.290 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:08:13.288+0000 7f2103fff640 1 -- 172.21.15.156:0/2393645525 <== mon.0 v2:172.21.15.156:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f20fc008ee0 con 0x7f210c102350 2024-07-16T10:08:13.291 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:08:13.288+0000 7f21112f5640 1 -- 172.21.15.156:0/2393645525 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f210c10d2b0 con 0x7f210c102350 2024-07-16T10:08:13.291 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:08:13.288+0000 7f2103fff640 1 -- 172.21.15.156:0/2393645525 <== mon.0 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f20fc03d870 con 0x7f210c102350 2024-07-16T10:08:13.291 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:08:13.288+0000 7f21112f5640 1 -- 172.21.15.156:0/2393645525 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f210c10d7f0 con 0x7f210c102350 2024-07-16T10:08:13.291 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:08:13.289+0000 7f21112f5640 1 -- 172.21.15.156:0/2393645525 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f210c0fdac0 con 0x7f210c102350 2024-07-16T10:08:13.292 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:08:13.289+0000 7f2103fff640 1 -- 172.21.15.156:0/2393645525 <== mon.0 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 38) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f20fc030020 con 0x7f210c102350 2024-07-16T10:08:13.292 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:08:13.290+0000 7f2103fff640 1 --2- 172.21.15.156:0/2393645525 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f20f806e0b0 0x7f20f8070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-16T10:08:13.292 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:08:13.290+0000 7f2103fff640 1 -- 172.21.15.156:0/2393645525 <== mon.0 v2:172.21.15.156:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f20fc0ad8a0 con 0x7f210c102350 2024-07-16T10:08:13.292 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:08:13.290+0000 7f21027fc640 1 --2- 172.21.15.156:0/2393645525 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f20f806e0b0 0x7f20f8070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-16T10:08:13.292 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:08:13.290+0000 7f21027fc640 1 --2- 172.21.15.156:0/2393645525 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f20f806e0b0 0x7f20f8070560 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f210c10e970 tx=0x7f20f4004330 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-16T10:08:13.295 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:08:13.294+0000 7f2103fff640 1 -- 172.21.15.156:0/2393645525 <== mon.0 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187716 (secure 0 0 0) 0x7f20fc076bf0 con 0x7f210c102350 2024-07-16T10:08:13.486 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:08:13.484+0000 7f21112f5640 1 -- 172.21.15.156:0/2393645525 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f210c0fbb40 con 0x7f210c102350 2024-07-16T10:08:13.486 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:08:13.485+0000 7f2103fff640 1 -- 172.21.15.156:0/2393645525 <== mon.0 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 41 entries v84) v1 ==== 81+0+2456 (secure 0 0 0) 0x7f20fc07a2b0 con 0x7f210c102350 2024-07-16T10:08:13.486 INFO:teuthology.orchestra.run.smithi156.stdout:172.21.15.156:6828/333092891 2024-07-17T10:03:03.351199+0000 2024-07-16T10:08:13.486 INFO:teuthology.orchestra.run.smithi156.stdout:172.21.15.156:6827/1080176013 2024-07-17T10:02:40.286020+0000 2024-07-16T10:08:13.487 INFO:teuthology.orchestra.run.smithi156.stdout:172.21.15.156:6826/1080176013 2024-07-17T10:02:40.286020+0000 2024-07-16T10:08:13.487 INFO:teuthology.orchestra.run.smithi156.stdout:172.21.15.156:0/2582266353 2024-07-17T09:57:48.301524+0000 2024-07-16T10:08:13.487 INFO:teuthology.orchestra.run.smithi156.stdout:172.21.15.156:0/788626231 2024-07-17T09:57:48.301524+0000 2024-07-16T10:08:13.487 INFO:teuthology.orchestra.run.smithi156.stdout:172.21.15.156:6801/55311371 2024-07-17T09:57:48.301524+0000 2024-07-16T10:08:13.487 INFO:teuthology.orchestra.run.smithi156.stdout:172.21.15.156:6800/55311371 2024-07-17T09:57:48.301524+0000 2024-07-16T10:08:13.487 INFO:teuthology.orchestra.run.smithi156.stdout:172.21.15.156:0/1414712577 2024-07-17T09:57:48.301524+0000 2024-07-16T10:08:13.487 INFO:teuthology.orchestra.run.smithi156.stdout:172.21.15.176:6829/673792884 2024-07-17T09:57:08.232362+0000 2024-07-16T10:08:13.487 INFO:teuthology.orchestra.run.smithi156.stdout:172.21.15.156:0/3487646818 2024-07-17T09:46:53.642082+0000 2024-07-16T10:08:13.487 INFO:teuthology.orchestra.run.smithi156.stdout:172.21.15.156:6801/1023000918 2024-07-17T09:47:22.566842+0000 2024-07-16T10:08:13.487 INFO:teuthology.orchestra.run.smithi156.stdout:172.21.15.156:0/2322505969 2024-07-17T09:47:22.566842+0000 2024-07-16T10:08:13.488 INFO:teuthology.orchestra.run.smithi156.stdout:172.21.15.156:0/2320156657 2024-07-17T09:48:24.722579+0000 2024-07-16T10:08:13.488 INFO:teuthology.orchestra.run.smithi156.stdout:172.21.15.156:0/3143160297 2024-07-17T09:57:48.301524+0000 2024-07-16T10:08:13.488 INFO:teuthology.orchestra.run.smithi156.stdout:172.21.15.156:6800/1023000918 2024-07-17T09:47:22.566842+0000 2024-07-16T10:08:13.488 INFO:teuthology.orchestra.run.smithi156.stdout:172.21.15.156:0/590617513 2024-07-17T09:55:49.326453+0000 2024-07-16T10:08:13.488 INFO:teuthology.orchestra.run.smithi156.stdout:172.21.15.156:0/3148313406 2024-07-17T09:55:49.326453+0000 2024-07-16T10:08:13.488 INFO:teuthology.orchestra.run.smithi156.stdout:172.21.15.156:0/1864977086 2024-07-17T09:47:22.566842+0000 2024-07-16T10:08:13.489 INFO:teuthology.orchestra.run.smithi156.stdout:172.21.15.156:0/3191234605 2024-07-17T09:48:24.722579+0000 2024-07-16T10:08:13.489 INFO:teuthology.orchestra.run.smithi156.stdout:172.21.15.176:0/1657551005 2024-07-17T09:57:08.232362+0000 2024-07-16T10:08:13.489 INFO:teuthology.orchestra.run.smithi156.stdout:172.21.15.156:0/1490472205 2024-07-17T09:57:48.301524+0000 2024-07-16T10:08:13.489 INFO:teuthology.orchestra.run.smithi156.stdout:172.21.15.176:6828/673792884 2024-07-17T09:57:08.232362+0000 2024-07-16T10:08:13.489 INFO:teuthology.orchestra.run.smithi156.stdout:172.21.15.156:0/3579914066 2024-07-17T09:55:49.326453+0000 2024-07-16T10:08:13.489 INFO:teuthology.orchestra.run.smithi156.stdout:172.21.15.156:0/4134717169 2024-07-17T09:57:48.301524+0000 2024-07-16T10:08:13.490 INFO:teuthology.orchestra.run.smithi156.stdout:172.21.15.176:0/4058805282 2024-07-17T09:57:08.232362+0000 2024-07-16T10:08:13.490 INFO:teuthology.orchestra.run.smithi156.stdout:172.21.15.176:0/1245693665 2024-07-17T09:57:08.232362+0000 2024-07-16T10:08:13.490 INFO:teuthology.orchestra.run.smithi156.stdout:172.21.15.176:0/3647663194 2024-07-17T09:57:08.232362+0000 2024-07-16T10:08:13.490 INFO:teuthology.orchestra.run.smithi156.stdout:172.21.15.156:6829/333092891 2024-07-17T10:03:03.351199+0000 2024-07-16T10:08:13.490 INFO:teuthology.orchestra.run.smithi156.stdout:172.21.15.176:0/1691847637 2024-07-17T09:57:08.232362+0000 2024-07-16T10:08:13.490 INFO:teuthology.orchestra.run.smithi156.stdout:172.21.15.156:0/211214934 2024-07-17T09:55:49.326453+0000 2024-07-16T10:08:13.491 INFO:teuthology.orchestra.run.smithi156.stdout:172.21.15.156:6800/2871070852 2024-07-17T09:46:53.642082+0000 2024-07-16T10:08:13.491 INFO:teuthology.orchestra.run.smithi156.stdout:172.21.15.156:0/3390422489 2024-07-17T09:46:53.642082+0000 2024-07-16T10:08:13.491 INFO:teuthology.orchestra.run.smithi156.stdout:172.21.15.156:6801/2615741717 2024-07-17T09:55:49.326453+0000 2024-07-16T10:08:13.491 INFO:teuthology.orchestra.run.smithi156.stdout:172.21.15.156:6800/2615741717 2024-07-17T09:55:49.326453+0000 2024-07-16T10:08:13.491 INFO:teuthology.orchestra.run.smithi156.stdout:172.21.15.176:0/4270704268 2024-07-17T09:57:08.232362+0000 2024-07-16T10:08:13.491 INFO:teuthology.orchestra.run.smithi156.stdout:172.21.15.156:6801/2871070852 2024-07-17T09:46:53.642082+0000 2024-07-16T10:08:13.491 INFO:teuthology.orchestra.run.smithi156.stdout:172.21.15.156:0/1325014720 2024-07-17T09:48:24.722579+0000 2024-07-16T10:08:13.492 INFO:teuthology.orchestra.run.smithi156.stdout:172.21.15.156:6800/3984987611 2024-07-17T09:48:24.722579+0000 2024-07-16T10:08:13.492 INFO:teuthology.orchestra.run.smithi156.stdout:172.21.15.156:6801/3984987611 2024-07-17T09:48:24.722579+0000 2024-07-16T10:08:13.492 INFO:teuthology.orchestra.run.smithi156.stdout:172.21.15.156:0/2120399071 2024-07-17T09:47:22.566842+0000 2024-07-16T10:08:13.492 INFO:teuthology.orchestra.run.smithi156.stdout:172.21.15.156:0/4158213614 2024-07-17T09:46:53.642082+0000 2024-07-16T10:08:13.493 INFO:teuthology.orchestra.run.smithi156.stderr:listed 41 entries 2024-07-16T10:08:13.493 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:08:13.487+0000 7f21112f5640 1 -- 172.21.15.156:0/2393645525 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f20f806e0b0 msgr2=0x7f20f8070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:08:13.494 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:08:13.487+0000 7f21112f5640 1 --2- 172.21.15.156:0/2393645525 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f20f806e0b0 0x7f20f8070560 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f210c10e970 tx=0x7f20f4004330 comp rx=0 tx=0).stop 2024-07-16T10:08:13.494 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:08:13.487+0000 7f21112f5640 1 -- 172.21.15.156:0/2393645525 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f210c102350 msgr2=0x7f210c1130b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-16T10:08:13.494 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:08:13.487+0000 7f21112f5640 1 --2- 172.21.15.156:0/2393645525 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f210c102350 0x7f210c1130b0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f20fc003080 tx=0x7f20fc009da0 comp rx=0 tx=0).stop 2024-07-16T10:08:13.494 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:08:13.487+0000 7f21112f5640 1 -- 172.21.15.156:0/2393645525 shutdown_connections 2024-07-16T10:08:13.494 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:08:13.487+0000 7f21112f5640 1 --2- 172.21.15.156:0/2393645525 >> [v2:172.21.15.156:6800/3367765292,v1:172.21.15.156:6801/3367765292] conn(0x7f20f806e0b0 0x7f20f8070560 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:08:13.494 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:08:13.487+0000 7f21112f5640 1 --2- 172.21.15.156:0/2393645525 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f210c102cf0 0x7f210c1135f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:08:13.495 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:08:13.487+0000 7f21112f5640 1 --2- 172.21.15.156:0/2393645525 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f210c102350 0x7f210c1130b0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-16T10:08:13.495 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:08:13.487+0000 7f21112f5640 1 -- 172.21.15.156:0/2393645525 >> 172.21.15.156:0/2393645525 conn(0x7f210c0f7f00 msgr2=0x7f210c0f8ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-16T10:08:13.495 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:08:13.487+0000 7f21112f5640 1 -- 172.21.15.156:0/2393645525 shutdown_connections 2024-07-16T10:08:13.495 INFO:teuthology.orchestra.run.smithi156.stderr:2024-07-16T10:08:13.487+0000 7f21112f5640 1 -- 172.21.15.156:0/2393645525 wait complete. 2024-07-16T10:08:14.093 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:08:13 smithi176 ceph-mon[81689]: pgmap v350: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:08:14.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:08:13 smithi176 ceph-mon[81689]: from='client.? 172.21.15.156:0/2393645525' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-07-16T10:08:14.129 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:08:13 smithi156 ceph-mon[93066]: pgmap v350: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:08:14.129 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:08:13 smithi156 ceph-mon[93066]: from='client.? 172.21.15.156:0/2393645525' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-07-16T10:08:14.155 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi156.front.sepia.ceph.com... 2024-07-16T10:08:14.155 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-16T10:08:14.155 DEBUG:teuthology.orchestra.run.smithi156:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-07-16T10:08:14.183 INFO:teuthology.orchestra.run:waiting for 300 2024-07-16T10:08:16.093 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:08:15 smithi176 ceph-mon[81689]: pgmap v351: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-07-16T10:08:16.174 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:08:15 smithi156 ceph-mon[93066]: pgmap v351: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-07-16T10:08:18.093 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:08:17 smithi176 ceph-mon[81689]: pgmap v352: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:08:18.174 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:08:17 smithi156 ceph-mon[93066]: pgmap v352: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:08:19.093 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:08:18 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-16T10:08:19.174 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:08:18 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-16T10:08:20.093 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:08:19 smithi176 ceph-mon[81689]: pgmap v353: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-07-16T10:08:20.174 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:08:19 smithi156 ceph-mon[93066]: pgmap v353: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-07-16T10:08:22.093 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:08:21 smithi176 ceph-mon[81689]: pgmap v354: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:08:22.174 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:08:21 smithi156 ceph-mon[93066]: pgmap v354: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:08:24.093 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:08:23 smithi176 ceph-mon[81689]: pgmap v355: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:08:24.174 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:08:23 smithi156 ceph-mon[93066]: pgmap v355: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:08:26.093 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:08:25 smithi176 ceph-mon[81689]: pgmap v356: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-07-16T10:08:26.174 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:08:25 smithi156 ceph-mon[93066]: pgmap v356: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-07-16T10:08:28.343 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:08:27 smithi176 ceph-mon[81689]: pgmap v357: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:08:28.424 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:08:27 smithi156 ceph-mon[93066]: pgmap v357: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:08:30.343 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:08:29 smithi176 ceph-mon[81689]: pgmap v358: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-07-16T10:08:30.424 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:08:29 smithi156 ceph-mon[93066]: pgmap v358: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-07-16T10:08:32.315 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:08:31 smithi156 ceph-mon[93066]: pgmap v359: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:08:32.343 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:08:31 smithi176 ceph-mon[81689]: pgmap v359: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:08:34.348 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:08:33 smithi176 ceph-mon[81689]: pgmap v360: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:08:34.348 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:08:33 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-16T10:08:34.424 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:08:33 smithi156 ceph-mon[93066]: pgmap v360: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:08:34.424 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:08:33 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-16T10:08:36.343 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:08:35 smithi176 ceph-mon[81689]: pgmap v361: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-07-16T10:08:36.424 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:08:35 smithi156 ceph-mon[93066]: pgmap v361: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-07-16T10:08:38.343 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:08:37 smithi176 ceph-mon[81689]: pgmap v362: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:08:38.424 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:08:37 smithi156 ceph-mon[93066]: pgmap v362: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:08:40.343 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:08:39 smithi176 ceph-mon[81689]: pgmap v363: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-07-16T10:08:40.424 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:08:39 smithi156 ceph-mon[93066]: pgmap v363: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-07-16T10:08:42.093 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:08:41 smithi176 ceph-mon[81689]: pgmap v364: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:08:42.315 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:08:41 smithi156 ceph-mon[93066]: pgmap v364: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:08:44.343 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:08:43 smithi176 ceph-mon[81689]: pgmap v365: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:08:44.424 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:08:43 smithi156 ceph-mon[93066]: pgmap v365: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:08:46.343 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:08:45 smithi176 ceph-mon[81689]: pgmap v366: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-07-16T10:08:46.424 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:08:45 smithi156 ceph-mon[93066]: pgmap v366: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-07-16T10:08:48.343 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:08:47 smithi176 ceph-mon[81689]: pgmap v367: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:08:48.424 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:08:47 smithi156 ceph-mon[93066]: pgmap v367: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:08:49.343 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:08:48 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-16T10:08:49.424 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:08:48 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-16T10:08:50.343 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:08:49 smithi176 ceph-mon[81689]: pgmap v368: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-07-16T10:08:50.424 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:08:49 smithi156 ceph-mon[93066]: pgmap v368: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-07-16T10:08:52.315 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:08:51 smithi156 ceph-mon[93066]: pgmap v369: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:08:52.343 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:08:51 smithi176 ceph-mon[81689]: pgmap v369: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:08:54.343 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:08:53 smithi176 ceph-mon[81689]: pgmap v370: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:08:54.424 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:08:53 smithi156 ceph-mon[93066]: pgmap v370: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:08:56.343 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:08:55 smithi176 ceph-mon[81689]: pgmap v371: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-07-16T10:08:56.424 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:08:55 smithi156 ceph-mon[93066]: pgmap v371: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-07-16T10:08:58.343 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:08:57 smithi176 ceph-mon[81689]: pgmap v372: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:08:58.424 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:08:57 smithi156 ceph-mon[93066]: pgmap v372: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:09:00.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:08:59 smithi176 ceph-mon[81689]: pgmap v373: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-07-16T10:09:00.424 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:08:59 smithi156 ceph-mon[93066]: pgmap v373: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-07-16T10:09:02.315 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:09:02 smithi156 ceph-mon[93066]: pgmap v374: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:09:02.343 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:09:02 smithi176 ceph-mon[81689]: pgmap v374: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:09:04.343 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:09:04 smithi176 ceph-mon[81689]: pgmap v375: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:09:04.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:09:04 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-16T10:09:04.424 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:09:04 smithi156 ceph-mon[93066]: pgmap v375: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:09:04.424 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:09:04 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-16T10:09:06.343 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:09:06 smithi176 ceph-mon[81689]: pgmap v376: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-07-16T10:09:06.344 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:09:06 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-16T10:09:06.424 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:09:06 smithi156 ceph-mon[93066]: pgmap v376: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-07-16T10:09:06.424 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:09:06 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-16T10:09:07.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:09:07 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:09:07.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:09:07 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:09:07.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:09:07 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-16T10:09:07.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:09:07 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-16T10:09:07.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:09:07 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:09:07.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:09:07 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:09:07.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:09:07 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:09:07.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:09:07 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-16T10:09:07.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:09:07 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-16T10:09:07.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:09:07 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:09:08.495 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:09:08 smithi176 ceph-mon[81689]: pgmap v377: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:09:08.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:09:08 smithi156 ceph-mon[93066]: pgmap v377: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:09:10.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:09:10 smithi176 ceph-mon[81689]: pgmap v378: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-07-16T10:09:10.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:09:10 smithi156 ceph-mon[93066]: pgmap v378: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-07-16T10:09:12.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:09:12 smithi176 ceph-mon[81689]: pgmap v379: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:09:12.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:09:12 smithi156 ceph-mon[93066]: pgmap v379: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:09:14.477 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:09:14 smithi156 ceph-mon[93066]: pgmap v380: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:09:14.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:09:14 smithi176 ceph-mon[81689]: pgmap v380: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:09:16.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:09:16 smithi176 ceph-mon[81689]: pgmap v381: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-07-16T10:09:16.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:09:16 smithi156 ceph-mon[93066]: pgmap v381: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-07-16T10:09:18.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:09:18 smithi176 ceph-mon[81689]: pgmap v382: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:09:18.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:09:18 smithi156 ceph-mon[93066]: pgmap v382: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:09:19.488 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:09:19 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-16T10:09:19.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:09:19 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-16T10:09:20.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:09:20 smithi176 ceph-mon[81689]: pgmap v383: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-07-16T10:09:20.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:09:20 smithi156 ceph-mon[93066]: pgmap v383: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-07-16T10:09:22.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:09:22 smithi176 ceph-mon[81689]: pgmap v384: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:09:22.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:09:22 smithi156 ceph-mon[93066]: pgmap v384: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:09:24.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:09:24 smithi176 ceph-mon[81689]: pgmap v385: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:09:24.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:09:24 smithi156 ceph-mon[93066]: pgmap v385: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:09:26.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:09:26 smithi176 ceph-mon[81689]: pgmap v386: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-07-16T10:09:26.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:09:26 smithi156 ceph-mon[93066]: pgmap v386: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-07-16T10:09:28.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:09:28 smithi176 ceph-mon[81689]: pgmap v387: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:09:28.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:09:28 smithi156 ceph-mon[93066]: pgmap v387: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:09:30.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:09:30 smithi176 ceph-mon[81689]: pgmap v388: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-07-16T10:09:30.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:09:30 smithi156 ceph-mon[93066]: pgmap v388: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-07-16T10:09:32.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:09:32 smithi176 ceph-mon[81689]: pgmap v389: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:09:32.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:09:32 smithi156 ceph-mon[93066]: pgmap v389: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:09:34.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:09:34 smithi176 ceph-mon[81689]: pgmap v390: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:09:34.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:09:34 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-16T10:09:34.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:09:34 smithi156 ceph-mon[93066]: pgmap v390: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:09:34.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:09:34 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-16T10:09:36.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:09:36 smithi176 ceph-mon[81689]: pgmap v391: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-07-16T10:09:36.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:09:36 smithi156 ceph-mon[93066]: pgmap v391: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-07-16T10:09:38.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:09:38 smithi176 ceph-mon[81689]: pgmap v392: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:09:38.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:09:38 smithi156 ceph-mon[93066]: pgmap v392: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:09:40.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:09:40 smithi176 ceph-mon[81689]: pgmap v393: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-07-16T10:09:40.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:09:40 smithi156 ceph-mon[93066]: pgmap v393: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-07-16T10:09:42.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:09:42 smithi176 ceph-mon[81689]: pgmap v394: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:09:42.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:09:42 smithi156 ceph-mon[93066]: pgmap v394: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:09:44.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:09:44 smithi176 ceph-mon[81689]: pgmap v395: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:09:44.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:09:44 smithi156 ceph-mon[93066]: pgmap v395: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:09:46.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:09:46 smithi176 ceph-mon[81689]: pgmap v396: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-07-16T10:09:46.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:09:46 smithi156 ceph-mon[93066]: pgmap v396: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-07-16T10:09:48.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:09:48 smithi176 ceph-mon[81689]: pgmap v397: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:09:48.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:09:48 smithi156 ceph-mon[93066]: pgmap v397: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:09:49.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:09:49 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-16T10:09:49.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:09:49 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-16T10:09:50.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:09:50 smithi176 ceph-mon[81689]: pgmap v398: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-07-16T10:09:50.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:09:50 smithi156 ceph-mon[93066]: pgmap v398: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-07-16T10:09:52.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:09:52 smithi176 ceph-mon[81689]: pgmap v399: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:09:52.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:09:52 smithi156 ceph-mon[93066]: pgmap v399: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:09:54.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:09:54 smithi176 ceph-mon[81689]: pgmap v400: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:09:54.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:09:54 smithi156 ceph-mon[93066]: pgmap v400: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:09:56.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:09:56 smithi176 ceph-mon[81689]: pgmap v401: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-07-16T10:09:56.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:09:56 smithi156 ceph-mon[93066]: pgmap v401: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-07-16T10:09:58.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:09:58 smithi176 ceph-mon[81689]: pgmap v402: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:09:58.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:09:58 smithi156 ceph-mon[93066]: pgmap v402: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:10:00.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:10:00 smithi176 ceph-mon[81689]: pgmap v403: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-07-16T10:10:00.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:10:00 smithi176 ceph-mon[81689]: overall HEALTH_OK 2024-07-16T10:10:00.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:10:00 smithi156 ceph-mon[93066]: pgmap v403: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-07-16T10:10:00.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:10:00 smithi156 ceph-mon[93066]: overall HEALTH_OK 2024-07-16T10:10:02.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:10:02 smithi176 ceph-mon[81689]: pgmap v404: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:10:02.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:10:02 smithi156 ceph-mon[93066]: pgmap v404: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:10:04.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:10:04 smithi176 ceph-mon[81689]: pgmap v405: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:10:04.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:10:04 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-16T10:10:04.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:10:04 smithi156 ceph-mon[93066]: pgmap v405: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:10:04.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:10:04 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-16T10:10:06.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:10:06 smithi176 ceph-mon[81689]: pgmap v406: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-07-16T10:10:06.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:10:06 smithi156 ceph-mon[93066]: pgmap v406: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-07-16T10:10:07.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:10:07 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-16T10:10:07.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:10:07 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-16T10:10:07.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:10:07 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-16T10:10:07.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:10:07 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:10:07.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:10:07 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-16T10:10:07.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:10:07 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-16T10:10:07.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:10:07 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-16T10:10:07.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:10:07 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:10:08.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:10:08 smithi176 ceph-mon[81689]: pgmap v407: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:10:08.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:10:08 smithi156 ceph-mon[93066]: pgmap v407: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:10:10.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:10:10 smithi176 ceph-mon[81689]: pgmap v408: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-07-16T10:10:10.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:10:10 smithi156 ceph-mon[93066]: pgmap v408: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-07-16T10:10:12.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:10:12 smithi176 ceph-mon[81689]: pgmap v409: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:10:12.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:10:12 smithi156 ceph-mon[93066]: pgmap v409: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:10:14.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:10:14 smithi176 ceph-mon[81689]: pgmap v410: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:10:14.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:10:14 smithi156 ceph-mon[93066]: pgmap v410: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:10:16.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:10:16 smithi176 ceph-mon[81689]: pgmap v411: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-07-16T10:10:16.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:10:16 smithi156 ceph-mon[93066]: pgmap v411: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-07-16T10:10:18.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:10:18 smithi176 ceph-mon[81689]: pgmap v412: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:10:18.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:10:18 smithi156 ceph-mon[93066]: pgmap v412: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:10:19.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:10:19 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-16T10:10:19.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:10:19 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-16T10:10:20.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:10:20 smithi176 ceph-mon[81689]: pgmap v413: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-07-16T10:10:20.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:10:20 smithi156 ceph-mon[93066]: pgmap v413: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-07-16T10:10:22.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:10:22 smithi176 ceph-mon[81689]: pgmap v414: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:10:22.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:10:22 smithi156 ceph-mon[93066]: pgmap v414: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:10:24.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:10:24 smithi176 ceph-mon[81689]: pgmap v415: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:10:24.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:10:24 smithi156 ceph-mon[93066]: pgmap v415: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:10:26.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:10:26 smithi176 ceph-mon[81689]: pgmap v416: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-07-16T10:10:26.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:10:26 smithi156 ceph-mon[93066]: pgmap v416: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-07-16T10:10:28.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:10:28 smithi176 ceph-mon[81689]: pgmap v417: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:10:28.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:10:28 smithi156 ceph-mon[93066]: pgmap v417: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:10:30.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:10:30 smithi176 ceph-mon[81689]: pgmap v418: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-07-16T10:10:30.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:10:30 smithi156 ceph-mon[93066]: pgmap v418: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-07-16T10:10:32.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:10:32 smithi156 ceph-mon[93066]: pgmap v419: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:10:32.843 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:10:32 smithi176 ceph-mon[81689]: pgmap v419: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:10:34.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:10:34 smithi156 ceph-mon[93066]: pgmap v420: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:10:34.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:10:34 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-16T10:10:34.843 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:10:34 smithi176 ceph-mon[81689]: pgmap v420: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:10:34.843 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:10:34 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-16T10:10:36.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:10:36 smithi156 ceph-mon[93066]: pgmap v421: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-07-16T10:10:36.843 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:10:36 smithi176 ceph-mon[81689]: pgmap v421: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-07-16T10:10:38.646 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:10:38 smithi176 ceph-mon[81689]: pgmap v422: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:10:38.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:10:38 smithi156 ceph-mon[93066]: pgmap v422: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:10:40.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:10:40 smithi156 ceph-mon[93066]: pgmap v423: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-07-16T10:10:40.843 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:10:40 smithi176 ceph-mon[81689]: pgmap v423: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-07-16T10:10:42.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:10:42 smithi156 ceph-mon[93066]: pgmap v424: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:10:42.843 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:10:42 smithi176 ceph-mon[81689]: pgmap v424: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:10:44.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:10:44 smithi156 ceph-mon[93066]: pgmap v425: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:10:44.843 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:10:44 smithi176 ceph-mon[81689]: pgmap v425: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:10:46.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:10:46 smithi156 ceph-mon[93066]: pgmap v426: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-07-16T10:10:46.843 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:10:46 smithi176 ceph-mon[81689]: pgmap v426: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-07-16T10:10:48.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:10:48 smithi156 ceph-mon[93066]: pgmap v427: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:10:48.843 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:10:48 smithi176 ceph-mon[81689]: pgmap v427: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:10:49.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:10:49 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-16T10:10:49.843 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:10:49 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-16T10:10:50.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:10:50 smithi156 ceph-mon[93066]: pgmap v428: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-07-16T10:10:50.843 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:10:50 smithi176 ceph-mon[81689]: pgmap v428: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-07-16T10:10:52.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:10:52 smithi156 ceph-mon[93066]: pgmap v429: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:10:52.843 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:10:52 smithi176 ceph-mon[81689]: pgmap v429: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:10:54.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:10:54 smithi156 ceph-mon[93066]: pgmap v430: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:10:54.843 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:10:54 smithi176 ceph-mon[81689]: pgmap v430: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:10:56.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:10:56 smithi156 ceph-mon[93066]: pgmap v431: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-07-16T10:10:56.843 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:10:56 smithi176 ceph-mon[81689]: pgmap v431: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-07-16T10:10:58.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:10:58 smithi156 ceph-mon[93066]: pgmap v432: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:10:58.843 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:10:58 smithi176 ceph-mon[81689]: pgmap v432: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:11:00.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:11:00 smithi156 ceph-mon[93066]: pgmap v433: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-07-16T10:11:00.843 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:11:00 smithi176 ceph-mon[81689]: pgmap v433: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-07-16T10:11:02.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:11:02 smithi156 ceph-mon[93066]: pgmap v434: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:11:02.843 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:11:02 smithi176 ceph-mon[81689]: pgmap v434: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:11:03.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:11:03 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-16T10:11:03.843 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:11:03 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-16T10:11:04.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:11:04 smithi156 ceph-mon[93066]: pgmap v435: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:11:04.843 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:11:04 smithi176 ceph-mon[81689]: pgmap v435: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:11:06.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:11:06 smithi156 ceph-mon[93066]: pgmap v436: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-07-16T10:11:06.843 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:11:06 smithi176 ceph-mon[81689]: pgmap v436: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-07-16T10:11:07.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:11:07 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-16T10:11:07.843 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:11:07 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-16T10:11:08.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:11:08 smithi156 ceph-mon[93066]: pgmap v437: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:11:08.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:11:08 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-16T10:11:08.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:11:08 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-16T10:11:08.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:11:08 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:11:08.745 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:11:08 smithi176 ceph-mon[81689]: pgmap v437: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:11:08.745 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:11:08 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-16T10:11:08.745 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:11:08 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-16T10:11:08.745 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:11:08 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:11:10.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:11:10 smithi156 ceph-mon[93066]: pgmap v438: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-07-16T10:11:10.843 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:11:10 smithi176 ceph-mon[81689]: pgmap v438: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-07-16T10:11:12.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:11:12 smithi156 ceph-mon[93066]: pgmap v439: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:11:12.843 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:11:12 smithi176 ceph-mon[81689]: pgmap v439: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:11:14.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:11:14 smithi156 ceph-mon[93066]: pgmap v440: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:11:14.843 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:11:14 smithi176 ceph-mon[81689]: pgmap v440: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:11:16.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:11:16 smithi156 ceph-mon[93066]: pgmap v441: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-07-16T10:11:16.843 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:11:16 smithi176 ceph-mon[81689]: pgmap v441: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-07-16T10:11:18.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:11:18 smithi156 ceph-mon[93066]: pgmap v442: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:11:18.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:11:18 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-16T10:11:18.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:11:18 smithi176 ceph-mon[81689]: pgmap v442: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:11:18.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:11:18 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-16T10:11:20.843 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:11:20 smithi176 ceph-mon[81689]: pgmap v443: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-07-16T10:11:20.924 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:11:20 smithi156 ceph-mon[93066]: pgmap v443: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-07-16T10:11:22.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:11:22 smithi156 ceph-mon[93066]: pgmap v444: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:11:22.845 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:11:22 smithi176 ceph-mon[81689]: pgmap v444: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:11:24.843 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:11:24 smithi176 ceph-mon[81689]: pgmap v445: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:11:24.924 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:11:24 smithi156 ceph-mon[93066]: pgmap v445: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:11:26.843 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:11:26 smithi176 ceph-mon[81689]: pgmap v446: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-07-16T10:11:26.924 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:11:26 smithi156 ceph-mon[93066]: pgmap v446: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-07-16T10:11:28.843 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:11:28 smithi176 ceph-mon[81689]: pgmap v447: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:11:28.924 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:11:28 smithi156 ceph-mon[93066]: pgmap v447: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:11:29.843 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:11:29 smithi176 ceph-mon[81689]: pgmap v448: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-07-16T10:11:29.924 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:11:29 smithi156 ceph-mon[93066]: pgmap v448: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-07-16T10:11:31.843 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:11:31 smithi176 ceph-mon[81689]: pgmap v449: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:11:31.924 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:11:31 smithi156 ceph-mon[93066]: pgmap v449: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:11:33.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:11:33 smithi176 ceph-mon[81689]: pgmap v450: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:11:33.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:11:33 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-16T10:11:33.924 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:11:33 smithi156 ceph-mon[93066]: pgmap v450: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:11:33.924 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:11:33 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-16T10:11:35.843 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:11:35 smithi176 ceph-mon[81689]: pgmap v451: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-07-16T10:11:35.924 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:11:35 smithi156 ceph-mon[93066]: pgmap v451: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-07-16T10:11:37.843 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:11:37 smithi176 ceph-mon[81689]: pgmap v452: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:11:37.924 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:11:37 smithi156 ceph-mon[93066]: pgmap v452: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:11:39.843 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:11:39 smithi176 ceph-mon[81689]: pgmap v453: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-07-16T10:11:39.924 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:11:39 smithi156 ceph-mon[93066]: pgmap v453: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-07-16T10:11:41.843 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:11:41 smithi176 ceph-mon[81689]: pgmap v454: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:11:41.924 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:11:41 smithi156 ceph-mon[93066]: pgmap v454: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:11:43.843 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:11:43 smithi176 ceph-mon[81689]: pgmap v455: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:11:43.924 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:11:43 smithi156 ceph-mon[93066]: pgmap v455: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:11:45.843 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:11:45 smithi176 ceph-mon[81689]: pgmap v456: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-07-16T10:11:45.924 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:11:45 smithi156 ceph-mon[93066]: pgmap v456: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-07-16T10:11:47.843 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:11:47 smithi176 ceph-mon[81689]: pgmap v457: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:11:47.924 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:11:47 smithi156 ceph-mon[93066]: pgmap v457: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:11:48.924 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:11:48 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-16T10:11:49.093 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:11:48 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-16T10:11:49.924 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:11:49 smithi156 ceph-mon[93066]: pgmap v458: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-07-16T10:11:50.093 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:11:49 smithi176 ceph-mon[81689]: pgmap v458: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-07-16T10:11:51.924 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:11:51 smithi156 ceph-mon[93066]: pgmap v459: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:11:52.093 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:11:51 smithi176 ceph-mon[81689]: pgmap v459: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:11:53.924 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:11:53 smithi156 ceph-mon[93066]: pgmap v460: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:11:54.093 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:11:53 smithi176 ceph-mon[81689]: pgmap v460: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:11:55.924 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:11:55 smithi156 ceph-mon[93066]: pgmap v461: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-07-16T10:11:56.093 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:11:55 smithi176 ceph-mon[81689]: pgmap v461: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-07-16T10:11:57.924 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:11:57 smithi156 ceph-mon[93066]: pgmap v462: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:11:58.093 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:11:57 smithi176 ceph-mon[81689]: pgmap v462: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:11:59.924 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:11:59 smithi156 ceph-mon[93066]: pgmap v463: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-07-16T10:12:00.093 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:11:59 smithi176 ceph-mon[81689]: pgmap v463: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-07-16T10:12:01.924 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:12:01 smithi156 ceph-mon[93066]: pgmap v464: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:12:02.093 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:12:01 smithi176 ceph-mon[81689]: pgmap v464: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:12:03.924 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:12:03 smithi156 ceph-mon[93066]: pgmap v465: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:12:03.924 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:12:03 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-16T10:12:04.093 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:12:03 smithi176 ceph-mon[81689]: pgmap v465: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:12:04.093 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:12:03 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-16T10:12:05.924 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:12:05 smithi156 ceph-mon[93066]: pgmap v466: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-07-16T10:12:06.093 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:12:05 smithi176 ceph-mon[81689]: pgmap v466: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-07-16T10:12:07.924 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:12:07 smithi156 ceph-mon[93066]: pgmap v467: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:12:08.093 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:12:07 smithi176 ceph-mon[81689]: pgmap v467: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:12:08.924 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:12:08 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-16T10:12:08.924 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:12:08 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-16T10:12:08.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:12:08 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-16T10:12:08.925 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:12:08 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:12:09.093 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:12:08 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-16T10:12:09.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:12:08 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-16T10:12:09.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:12:08 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-16T10:12:09.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:12:08 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:12:10.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:12:10 smithi176 ceph-mon[81689]: pgmap v468: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-07-16T10:12:10.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:12:10 smithi156 ceph-mon[93066]: pgmap v468: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-07-16T10:12:12.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:12:12 smithi176 ceph-mon[81689]: pgmap v469: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:12:12.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:12:12 smithi156 ceph-mon[93066]: pgmap v469: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:12:14.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:12:14 smithi176 ceph-mon[81689]: pgmap v470: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:12:14.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:12:14 smithi156 ceph-mon[93066]: pgmap v470: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:12:16.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:12:16 smithi176 ceph-mon[81689]: pgmap v471: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-07-16T10:12:16.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:12:16 smithi156 ceph-mon[93066]: pgmap v471: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-07-16T10:12:18.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:12:18 smithi176 ceph-mon[81689]: pgmap v472: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:12:18.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:12:18 smithi156 ceph-mon[93066]: pgmap v472: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:12:19.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:12:19 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-16T10:12:19.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:12:19 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-16T10:12:20.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:12:20 smithi176 ceph-mon[81689]: pgmap v473: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-07-16T10:12:20.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:12:20 smithi156 ceph-mon[93066]: pgmap v473: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-07-16T10:12:22.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:12:22 smithi176 ceph-mon[81689]: pgmap v474: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:12:22.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:12:22 smithi156 ceph-mon[93066]: pgmap v474: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:12:24.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:12:24 smithi176 ceph-mon[81689]: pgmap v475: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:12:24.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:12:24 smithi156 ceph-mon[93066]: pgmap v475: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:12:26.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:12:26 smithi176 ceph-mon[81689]: pgmap v476: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-07-16T10:12:26.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:12:26 smithi156 ceph-mon[93066]: pgmap v476: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-07-16T10:12:28.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:12:28 smithi176 ceph-mon[81689]: pgmap v477: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:12:28.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:12:28 smithi156 ceph-mon[93066]: pgmap v477: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:12:30.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:12:30 smithi176 ceph-mon[81689]: pgmap v478: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-07-16T10:12:30.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:12:30 smithi156 ceph-mon[93066]: pgmap v478: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-07-16T10:12:32.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:12:32 smithi176 ceph-mon[81689]: pgmap v479: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:12:32.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:12:32 smithi156 ceph-mon[93066]: pgmap v479: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:12:34.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:12:34 smithi176 ceph-mon[81689]: pgmap v480: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:12:34.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:12:34 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-16T10:12:34.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:12:34 smithi156 ceph-mon[93066]: pgmap v480: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:12:34.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:12:34 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-16T10:12:36.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:12:36 smithi176 ceph-mon[81689]: pgmap v481: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-07-16T10:12:36.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:12:36 smithi156 ceph-mon[93066]: pgmap v481: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-07-16T10:12:38.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:12:38 smithi176 ceph-mon[81689]: pgmap v482: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:12:38.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:12:38 smithi156 ceph-mon[93066]: pgmap v482: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:12:40.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:12:40 smithi176 ceph-mon[81689]: pgmap v483: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-07-16T10:12:40.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:12:40 smithi156 ceph-mon[93066]: pgmap v483: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-07-16T10:12:42.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:12:42 smithi176 ceph-mon[81689]: pgmap v484: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:12:42.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:12:42 smithi156 ceph-mon[93066]: pgmap v484: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:12:44.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:12:44 smithi176 ceph-mon[81689]: pgmap v485: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:12:44.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:12:44 smithi156 ceph-mon[93066]: pgmap v485: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:12:46.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:12:46 smithi176 ceph-mon[81689]: pgmap v486: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-07-16T10:12:46.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:12:46 smithi156 ceph-mon[93066]: pgmap v486: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-07-16T10:12:48.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:12:48 smithi176 ceph-mon[81689]: pgmap v487: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.0 KiB/s rd, 4 op/s 2024-07-16T10:12:48.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:12:48 smithi156 ceph-mon[93066]: pgmap v487: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.0 KiB/s rd, 4 op/s 2024-07-16T10:12:49.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:12:49 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-16T10:12:49.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:12:49 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-16T10:12:50.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:12:50 smithi176 ceph-mon[81689]: pgmap v488: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s 2024-07-16T10:12:50.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:12:50 smithi156 ceph-mon[93066]: pgmap v488: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s 2024-07-16T10:12:52.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:12:52 smithi176 ceph-mon[81689]: pgmap v489: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.0 KiB/s rd, 4 op/s 2024-07-16T10:12:52.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:12:52 smithi156 ceph-mon[93066]: pgmap v489: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.0 KiB/s rd, 4 op/s 2024-07-16T10:12:54.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:12:54 smithi176 ceph-mon[81689]: pgmap v490: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.0 KiB/s rd, 4 op/s 2024-07-16T10:12:54.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:12:54 smithi156 ceph-mon[93066]: pgmap v490: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.0 KiB/s rd, 4 op/s 2024-07-16T10:12:56.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:12:56 smithi176 ceph-mon[81689]: pgmap v491: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 1.6 KiB/s rd, 3 op/s 2024-07-16T10:12:56.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:12:56 smithi156 ceph-mon[93066]: pgmap v491: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 1.6 KiB/s rd, 3 op/s 2024-07-16T10:12:58.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:12:58 smithi176 ceph-mon[81689]: pgmap v492: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.0 KiB/s rd, 4 op/s 2024-07-16T10:12:58.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:12:58 smithi156 ceph-mon[93066]: pgmap v492: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.0 KiB/s rd, 4 op/s 2024-07-16T10:13:00.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:13:00 smithi176 ceph-mon[81689]: pgmap v493: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-07-16T10:13:00.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:13:00 smithi156 ceph-mon[93066]: pgmap v493: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-07-16T10:13:02.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:13:02 smithi176 ceph-mon[81689]: pgmap v494: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:13:02.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:13:02 smithi156 ceph-mon[93066]: pgmap v494: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:13:04.593 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:13:04 smithi176 ceph-mon[81689]: pgmap v495: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:13:04.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:13:04 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-16T10:13:04.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:13:04 smithi156 ceph-mon[93066]: pgmap v495: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:13:04.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:13:04 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-07-16T10:13:06.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:13:06 smithi156 ceph-mon[93066]: pgmap v496: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-07-16T10:13:06.843 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:13:06 smithi176 ceph-mon[81689]: pgmap v496: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-07-16T10:13:08.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:13:08 smithi156 ceph-mon[93066]: pgmap v497: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:13:08.843 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:13:08 smithi176 ceph-mon[81689]: pgmap v497: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:13:09.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:13:09 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-16T10:13:09.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:13:09 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-16T10:13:09.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:13:09 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-16T10:13:09.675 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:13:09 smithi156 ceph-mon[93066]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:13:09.843 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:13:09 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-16T10:13:09.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:13:09 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-16T10:13:09.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:13:09 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-16T10:13:09.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:13:09 smithi176 ceph-mon[81689]: from='mgr.34104 172.21.15.156:0/3709414419' entity='mgr.smithi156.zkvylv' 2024-07-16T10:13:10.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:13:10 smithi156 ceph-mon[93066]: pgmap v498: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-07-16T10:13:10.745 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:13:10 smithi176 ceph-mon[81689]: pgmap v498: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-07-16T10:13:12.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:13:12 smithi156 ceph-mon[93066]: pgmap v499: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:13:12.843 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:13:12 smithi176 ceph-mon[81689]: pgmap v499: 65 pgs: 65 active+clean; 145 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-07-16T10:13:13.254 ERROR:tasks.cephfs.fuse_mount:process failed to terminate after unmount. This probably indicates a bug within ceph-fuse. 2024-07-16T10:13:13.255 ERROR:teuthology.run_tasks:Manager failed: ceph-fuse Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_5bc696040f54eb999ca02a20c94bfd735f22b93c/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.10/contextlib.py", line 142, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_a6ca461be7cc1aff3658c2c6df4cff975a24449a/qa/tasks/ceph_fuse.py", line 248, in task mount.umount_wait() File "/home/teuthworker/src/git.ceph.com_ceph-c_a6ca461be7cc1aff3658c2c6df4cff975a24449a/qa/tasks/cephfs/fuse_mount.py", line 403, in umount_wait run.wait([self.fuse_daemon], timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_5bc696040f54eb999ca02a20c94bfd735f22b93c/teuthology/orchestra/run.py", line 473, in wait check_time() File "/home/teuthworker/src/git.ceph.com_teuthology_5bc696040f54eb999ca02a20c94bfd735f22b93c/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (51) after waiting for 300 seconds 2024-07-16T10:13:13.257 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-07-16T10:13:13.269 INFO:tasks.cephadm:Teardown begin 2024-07-16T10:13:13.269 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_5bc696040f54eb999ca02a20c94bfd735f22b93c/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_ceph-c_a6ca461be7cc1aff3658c2c6df4cff975a24449a/qa/tasks/cephadm.py", line 2226, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_5bc696040f54eb999ca02a20c94bfd735f22b93c/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.10/contextlib.py", line 142, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_a6ca461be7cc1aff3658c2c6df4cff975a24449a/qa/tasks/ceph_fuse.py", line 248, in task mount.umount_wait() File "/home/teuthworker/src/git.ceph.com_ceph-c_a6ca461be7cc1aff3658c2c6df4cff975a24449a/qa/tasks/cephfs/fuse_mount.py", line 403, in umount_wait run.wait([self.fuse_daemon], timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_5bc696040f54eb999ca02a20c94bfd735f22b93c/teuthology/orchestra/run.py", line 473, in wait check_time() File "/home/teuthworker/src/git.ceph.com_teuthology_5bc696040f54eb999ca02a20c94bfd735f22b93c/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (51) after waiting for 300 seconds 2024-07-16T10:13:13.270 DEBUG:teuthology.orchestra.run.smithi156:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-07-16T10:13:13.304 DEBUG:teuthology.orchestra.run.smithi176:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-07-16T10:13:13.337 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-07-16T10:13:13.337 DEBUG:teuthology.orchestra.run.smithi156:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-07-16T10:13:13.362 DEBUG:teuthology.orchestra.run.smithi176:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-07-16T10:13:13.395 INFO:tasks.cephadm:Stopping all daemons... 2024-07-16T10:13:13.395 INFO:tasks.cephadm.mon.smithi156:Stopping mon.smithi156... 2024-07-16T10:13:13.395 DEBUG:teuthology.orchestra.run.smithi156:> sudo systemctl stop ceph-26c91c26-4358-11ef-bcad-c7b262605968@mon.smithi156 2024-07-16T10:13:13.674 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:13:13 smithi156 systemd[1]: Stopping Ceph mon.smithi156 for 26c91c26-4358-11ef-bcad-c7b262605968... 2024-07-16T10:13:13.994 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:13:13 smithi156 ceph-26c91c26-4358-11ef-bcad-c7b262605968-mon-smithi156[93062]: 2024-07-16T10:13:13.719+0000 7fa504006640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi156 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-07-16T10:13:13.994 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:13:13 smithi156 ceph-26c91c26-4358-11ef-bcad-c7b262605968-mon-smithi156[93062]: 2024-07-16T10:13:13.719+0000 7fa504006640 -1 mon.smithi156@0(leader) e3 *** Got Signal Terminated *** 2024-07-16T10:13:13.994 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:13:13 smithi156 podman[152447]: 2024-07-16 10:13:13.873134233 +0000 UTC m=+0.269348563 container died 77b6ec8c50eb5165a042b8c22a28eb8ecd167c807708bd75457f3a70b576c408 (image=quay.ceph.io/ceph-ci/ceph@sha256:c15ec3aa8e6c0a9c12ee24ff79f3e7dd4d01b3d1a702dfeb7d7fff9b0811a393, name=ceph-26c91c26-4358-11ef-bcad-c7b262605968-mon-smithi156, org.label-schema.license=GPLv2, RELEASE=wip-adk4-testing-2024-07-15-1832-squid-a6ca461, CEPH_POINT_RELEASE=, io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240709, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-07-16T10:13:14.589 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:13:14 smithi156 podman[152447]: 2024-07-16 10:13:14.328641575 +0000 UTC m=+0.724855905 container cleanup 77b6ec8c50eb5165a042b8c22a28eb8ecd167c807708bd75457f3a70b576c408 (image=quay.ceph.io/ceph-ci/ceph@sha256:c15ec3aa8e6c0a9c12ee24ff79f3e7dd4d01b3d1a702dfeb7d7fff9b0811a393, name=ceph-26c91c26-4358-11ef-bcad-c7b262605968-mon-smithi156, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-adk4-testing-2024-07-15-1832-squid-a6ca461, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240709, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-07-16T10:13:14.589 INFO:journalctl@ceph.mon.smithi156.smithi156.stdout:Jul 16 10:13:14 smithi156 bash[152447]: ceph-26c91c26-4358-11ef-bcad-c7b262605968-mon-smithi156 2024-07-16T10:13:14.756 DEBUG:teuthology.orchestra.run.smithi156:> sudo pkill -f 'journalctl -f -n 0 -u ceph-26c91c26-4358-11ef-bcad-c7b262605968@mon.smithi156.service' 2024-07-16T10:13:14.815 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-07-16T10:13:14.815 INFO:tasks.cephadm.mon.smithi156:Stopped mon.smithi156 2024-07-16T10:13:14.816 INFO:tasks.cephadm.mon.smithi176:Stopping mon.smithi176... 2024-07-16T10:13:14.816 DEBUG:teuthology.orchestra.run.smithi176:> sudo systemctl stop ceph-26c91c26-4358-11ef-bcad-c7b262605968@mon.smithi176 2024-07-16T10:13:15.094 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:13:14 smithi176 systemd[1]: Stopping Ceph mon.smithi176 for 26c91c26-4358-11ef-bcad-c7b262605968... 2024-07-16T10:13:15.511 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:13:15 smithi176 ceph-26c91c26-4358-11ef-bcad-c7b262605968-mon-smithi176[81685]: 2024-07-16T10:13:15.126+0000 7f10eb077640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi176 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-07-16T10:13:15.511 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:13:15 smithi176 ceph-26c91c26-4358-11ef-bcad-c7b262605968-mon-smithi176[81685]: 2024-07-16T10:13:15.126+0000 7f10eb077640 -1 mon.smithi176@1(peon) e3 *** Got Signal Terminated *** 2024-07-16T10:13:15.512 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:13:15 smithi176 podman[118273]: 2024-07-16 10:13:15.249511524 +0000 UTC m=+0.233916218 container died ef92814cec2bc58325935136a26d0a5a2db0964b14e066585628006ac725759b (image=quay.ceph.io/ceph-ci/ceph@sha256:c15ec3aa8e6c0a9c12ee24ff79f3e7dd4d01b3d1a702dfeb7d7fff9b0811a393, name=ceph-26c91c26-4358-11ef-bcad-c7b262605968-mon-smithi176, RELEASE=wip-adk4-testing-2024-07-15-1832-squid-a6ca461, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.36.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240709, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-07-16T10:13:15.843 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:13:15 smithi176 podman[118273]: 2024-07-16 10:13:15.685417485 +0000 UTC m=+0.669822219 container cleanup ef92814cec2bc58325935136a26d0a5a2db0964b14e066585628006ac725759b (image=quay.ceph.io/ceph-ci/ceph@sha256:c15ec3aa8e6c0a9c12ee24ff79f3e7dd4d01b3d1a702dfeb7d7fff9b0811a393, name=ceph-26c91c26-4358-11ef-bcad-c7b262605968-mon-smithi176, org.label-schema.schema-version=1.0, io.buildah.version=1.36.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240709, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, RELEASE=wip-adk4-testing-2024-07-15-1832-squid-a6ca461, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-07-16T10:13:15.843 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Jul 16 10:13:15 smithi176 bash[118273]: ceph-26c91c26-4358-11ef-bcad-c7b262605968-mon-smithi176 2024-07-16T10:13:16.136 DEBUG:teuthology.orchestra.run.smithi176:> sudo pkill -f 'journalctl -f -n 0 -u ceph-26c91c26-4358-11ef-bcad-c7b262605968@mon.smithi176.service' 2024-07-16T10:13:16.181 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-07-16T10:13:16.181 INFO:tasks.cephadm.mon.smithi176:Stopped mon.smithi176 2024-07-16T10:13:16.181 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-07-16T10:13:16.181 DEBUG:teuthology.orchestra.run.smithi156:> sudo systemctl stop ceph-26c91c26-4358-11ef-bcad-c7b262605968@osd.0 2024-07-16T10:13:16.508 INFO:journalctl@ceph.osd.0.smithi156.stdout:Jul 16 10:13:16 smithi156 systemd[1]: Stopping Ceph osd.0 for 26c91c26-4358-11ef-bcad-c7b262605968... 2024-07-16T10:13:16.769 INFO:journalctl@ceph.osd.0.smithi156.stdout:Jul 16 10:13:16 smithi156 ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-0[101600]: 2024-07-16T10:13:16.507+0000 7f7f12bc2640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-07-16T10:13:16.769 INFO:journalctl@ceph.osd.0.smithi156.stdout:Jul 16 10:13:16 smithi156 ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-0[101600]: 2024-07-16T10:13:16.507+0000 7f7f12bc2640 -1 osd.0 84 *** Got signal Terminated *** 2024-07-16T10:13:16.769 INFO:journalctl@ceph.osd.0.smithi156.stdout:Jul 16 10:13:16 smithi156 ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-0[101600]: 2024-07-16T10:13:16.507+0000 7f7f12bc2640 -1 osd.0 84 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-07-16T10:13:22.126 INFO:journalctl@ceph.osd.0.smithi156.stdout:Jul 16 10:13:21 smithi156 podman[152582]: 2024-07-16 10:13:21.868321432 +0000 UTC m=+5.505035418 container died 8633df1689cccb177b417e4827d55d249d75ad77e20260e3883d2368b89e44d9 (image=quay.ceph.io/ceph-ci/ceph@sha256:c15ec3aa8e6c0a9c12ee24ff79f3e7dd4d01b3d1a702dfeb7d7fff9b0811a393, name=ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240709, RELEASE=wip-adk4-testing-2024-07-15-1832-squid-a6ca461, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.36.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-07-16T10:13:22.424 INFO:journalctl@ceph.osd.0.smithi156.stdout:Jul 16 10:13:22 smithi156 podman[152582]: 2024-07-16 10:13:22.208141846 +0000 UTC m=+5.844855832 container cleanup 8633df1689cccb177b417e4827d55d249d75ad77e20260e3883d2368b89e44d9 (image=quay.ceph.io/ceph-ci/ceph@sha256:c15ec3aa8e6c0a9c12ee24ff79f3e7dd4d01b3d1a702dfeb7d7fff9b0811a393, name=ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240709, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-adk4-testing-2024-07-15-1832-squid-a6ca461, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.36.0) 2024-07-16T10:13:22.425 INFO:journalctl@ceph.osd.0.smithi156.stdout:Jul 16 10:13:22 smithi156 bash[152582]: ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-0 2024-07-16T10:13:22.838 INFO:journalctl@ceph.osd.0.smithi156.stdout:Jul 16 10:13:22 smithi156 podman[152612]: 2024-07-16 10:13:22.428493045 +0000 UTC m=+0.553260679 container remove 8633df1689cccb177b417e4827d55d249d75ad77e20260e3883d2368b89e44d9 (image=quay.ceph.io/ceph-ci/ceph@sha256:c15ec3aa8e6c0a9c12ee24ff79f3e7dd4d01b3d1a702dfeb7d7fff9b0811a393, name=ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.36.0, RELEASE=wip-adk4-testing-2024-07-15-1832-squid-a6ca461, org.label-schema.build-date=20240709) 2024-07-16T10:13:23.099 INFO:journalctl@ceph.osd.0.smithi156.stdout:Jul 16 10:13:22 smithi156 podman[152697]: 2024-07-16 10:13:22.838002541 +0000 UTC m=+0.112242366 container create 2079e5beb8f57556e2ea8a6aed84df42459a908afb065745b53eff9d7f649795 (image=quay.ceph.io/ceph-ci/ceph@sha256:c15ec3aa8e6c0a9c12ee24ff79f3e7dd4d01b3d1a702dfeb7d7fff9b0811a393, name=ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-0-deactivate, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.36.0, RELEASE=wip-adk4-testing-2024-07-15-1832-squid-a6ca461, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240709, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-07-16T10:13:23.100 INFO:journalctl@ceph.osd.0.smithi156.stdout:Jul 16 10:13:22 smithi156 podman[152697]: 2024-07-16 10:13:22.743253273 +0000 UTC m=+0.017493063 image pull 16f7feb043422ce37a2471403d4bc52f550d49cef4b677482112fcc0db3f67ab quay.ceph.io/ceph-ci/ceph@sha256:c15ec3aa8e6c0a9c12ee24ff79f3e7dd4d01b3d1a702dfeb7d7fff9b0811a393 2024-07-16T10:13:23.100 INFO:journalctl@ceph.osd.0.smithi156.stdout:Jul 16 10:13:23 smithi156 podman[152697]: 2024-07-16 10:13:23.099252904 +0000 UTC m=+0.373492699 container init 2079e5beb8f57556e2ea8a6aed84df42459a908afb065745b53eff9d7f649795 (image=quay.ceph.io/ceph-ci/ceph@sha256:c15ec3aa8e6c0a9c12ee24ff79f3e7dd4d01b3d1a702dfeb7d7fff9b0811a393, name=ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-0-deactivate, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240709, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux , RELEASE=wip-adk4-testing-2024-07-15-1832-squid-a6ca461, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-07-16T10:13:23.425 INFO:journalctl@ceph.osd.0.smithi156.stdout:Jul 16 10:13:23 smithi156 podman[152697]: 2024-07-16 10:13:23.102761402 +0000 UTC m=+0.377001190 container start 2079e5beb8f57556e2ea8a6aed84df42459a908afb065745b53eff9d7f649795 (image=quay.ceph.io/ceph-ci/ceph@sha256:c15ec3aa8e6c0a9c12ee24ff79f3e7dd4d01b3d1a702dfeb7d7fff9b0811a393, name=ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-0-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.build-date=20240709, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.36.0, RELEASE=wip-adk4-testing-2024-07-15-1832-squid-a6ca461, GIT_BRANCH=HEAD) 2024-07-16T10:13:23.425 INFO:journalctl@ceph.osd.0.smithi156.stdout:Jul 16 10:13:23 smithi156 podman[152697]: 2024-07-16 10:13:23.190823288 +0000 UTC m=+0.465063082 container attach 2079e5beb8f57556e2ea8a6aed84df42459a908afb065745b53eff9d7f649795 (image=quay.ceph.io/ceph-ci/ceph@sha256:c15ec3aa8e6c0a9c12ee24ff79f3e7dd4d01b3d1a702dfeb7d7fff9b0811a393, name=ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-0-deactivate, org.label-schema.build-date=20240709, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.36.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, RELEASE=wip-adk4-testing-2024-07-15-1832-squid-a6ca461, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD) 2024-07-16T10:13:23.425 INFO:journalctl@ceph.osd.0.smithi156.stdout:Jul 16 10:13:23 smithi156 podman[152697]: 2024-07-16 10:13:23.299367467 +0000 UTC m=+0.573607247 container died 2079e5beb8f57556e2ea8a6aed84df42459a908afb065745b53eff9d7f649795 (image=quay.ceph.io/ceph-ci/ceph@sha256:c15ec3aa8e6c0a9c12ee24ff79f3e7dd4d01b3d1a702dfeb7d7fff9b0811a393, name=ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-0-deactivate, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240709, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0, RELEASE=wip-adk4-testing-2024-07-15-1832-squid-a6ca461, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-07-16T10:13:23.975 DEBUG:teuthology.orchestra.run.smithi156:> sudo pkill -f 'journalctl -f -n 0 -u ceph-26c91c26-4358-11ef-bcad-c7b262605968@osd.0.service' 2024-07-16T10:13:24.015 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-07-16T10:13:24.015 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-07-16T10:13:24.015 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-07-16T10:13:24.016 DEBUG:teuthology.orchestra.run.smithi156:> sudo systemctl stop ceph-26c91c26-4358-11ef-bcad-c7b262605968@osd.1 2024-07-16T10:13:24.332 INFO:journalctl@ceph.osd.1.smithi156.stdout:Jul 16 10:13:24 smithi156 systemd[1]: Stopping Ceph osd.1 for 26c91c26-4358-11ef-bcad-c7b262605968... 2024-07-16T10:13:24.674 INFO:journalctl@ceph.osd.1.smithi156.stdout:Jul 16 10:13:24 smithi156 ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-1[106943]: 2024-07-16T10:13:24.331+0000 7f640194a640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-07-16T10:13:24.674 INFO:journalctl@ceph.osd.1.smithi156.stdout:Jul 16 10:13:24 smithi156 ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-1[106943]: 2024-07-16T10:13:24.331+0000 7f640194a640 -1 osd.1 84 *** Got signal Terminated *** 2024-07-16T10:13:24.674 INFO:journalctl@ceph.osd.1.smithi156.stdout:Jul 16 10:13:24 smithi156 ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-1[106943]: 2024-07-16T10:13:24.331+0000 7f640194a640 -1 osd.1 84 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-07-16T10:13:29.924 INFO:journalctl@ceph.osd.1.smithi156.stdout:Jul 16 10:13:29 smithi156 podman[152800]: 2024-07-16 10:13:29.686080249 +0000 UTC m=+5.449856342 container died e6fae9653b460a3931256034fdf7318a813be739a41d87ec058129ace21cd2e4 (image=quay.ceph.io/ceph-ci/ceph@sha256:c15ec3aa8e6c0a9c12ee24ff79f3e7dd4d01b3d1a702dfeb7d7fff9b0811a393, name=ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-1, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , RELEASE=wip-adk4-testing-2024-07-15-1832-squid-a6ca461, org.label-schema.build-date=20240709, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True) 2024-07-16T10:13:30.424 INFO:journalctl@ceph.osd.1.smithi156.stdout:Jul 16 10:13:30 smithi156 podman[152800]: 2024-07-16 10:13:30.056708506 +0000 UTC m=+5.820484598 container cleanup e6fae9653b460a3931256034fdf7318a813be739a41d87ec058129ace21cd2e4 (image=quay.ceph.io/ceph-ci/ceph@sha256:c15ec3aa8e6c0a9c12ee24ff79f3e7dd4d01b3d1a702dfeb7d7fff9b0811a393, name=ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-1, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240709, org.label-schema.license=GPLv2, io.buildah.version=1.36.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-adk4-testing-2024-07-15-1832-squid-a6ca461, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS) 2024-07-16T10:13:30.425 INFO:journalctl@ceph.osd.1.smithi156.stdout:Jul 16 10:13:30 smithi156 bash[152800]: ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-1 2024-07-16T10:13:30.425 INFO:journalctl@ceph.osd.1.smithi156.stdout:Jul 16 10:13:30 smithi156 podman[152814]: 2024-07-16 10:13:30.220641525 +0000 UTC m=+0.532913370 container remove e6fae9653b460a3931256034fdf7318a813be739a41d87ec058129ace21cd2e4 (image=quay.ceph.io/ceph-ci/ceph@sha256:c15ec3aa8e6c0a9c12ee24ff79f3e7dd4d01b3d1a702dfeb7d7fff9b0811a393, name=ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-1, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, RELEASE=wip-adk4-testing-2024-07-15-1832-squid-a6ca461, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.36.0, ceph=True, org.label-schema.build-date=20240709, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-07-16T10:13:30.995 INFO:journalctl@ceph.osd.1.smithi156.stdout:Jul 16 10:13:30 smithi156 podman[152904]: 2024-07-16 10:13:30.581097212 +0000 UTC m=+0.017620586 image pull 16f7feb043422ce37a2471403d4bc52f550d49cef4b677482112fcc0db3f67ab quay.ceph.io/ceph-ci/ceph@sha256:c15ec3aa8e6c0a9c12ee24ff79f3e7dd4d01b3d1a702dfeb7d7fff9b0811a393 2024-07-16T10:13:30.996 INFO:journalctl@ceph.osd.1.smithi156.stdout:Jul 16 10:13:30 smithi156 podman[152904]: 2024-07-16 10:13:30.71744516 +0000 UTC m=+0.153968523 container create 39c24e952a28d9b4166cd1975b1c4a668e7745dbf6d92fb39d61f64a88dcb369 (image=quay.ceph.io/ceph-ci/ceph@sha256:c15ec3aa8e6c0a9c12ee24ff79f3e7dd4d01b3d1a702dfeb7d7fff9b0811a393, name=ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-1-deactivate, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-adk4-testing-2024-07-15-1832-squid-a6ca461, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240709, io.buildah.version=1.36.0, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-07-16T10:13:30.996 INFO:journalctl@ceph.osd.1.smithi156.stdout:Jul 16 10:13:30 smithi156 podman[152904]: 2024-07-16 10:13:30.921585876 +0000 UTC m=+0.358109250 container init 39c24e952a28d9b4166cd1975b1c4a668e7745dbf6d92fb39d61f64a88dcb369 (image=quay.ceph.io/ceph-ci/ceph@sha256:c15ec3aa8e6c0a9c12ee24ff79f3e7dd4d01b3d1a702dfeb7d7fff9b0811a393, name=ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-1-deactivate, org.label-schema.license=GPLv2, io.buildah.version=1.36.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240709, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=wip-adk4-testing-2024-07-15-1832-squid-a6ca461, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-07-16T10:13:30.996 INFO:journalctl@ceph.osd.1.smithi156.stdout:Jul 16 10:13:30 smithi156 podman[152904]: 2024-07-16 10:13:30.924962191 +0000 UTC m=+0.361485567 container start 39c24e952a28d9b4166cd1975b1c4a668e7745dbf6d92fb39d61f64a88dcb369 (image=quay.ceph.io/ceph-ci/ceph@sha256:c15ec3aa8e6c0a9c12ee24ff79f3e7dd4d01b3d1a702dfeb7d7fff9b0811a393, name=ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-1-deactivate, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240709, io.buildah.version=1.36.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-adk4-testing-2024-07-15-1832-squid-a6ca461, org.label-schema.name=CentOS Stream 9 Base Image) 2024-07-16T10:13:30.996 INFO:journalctl@ceph.osd.1.smithi156.stdout:Jul 16 10:13:30 smithi156 podman[152904]: 2024-07-16 10:13:30.995214022 +0000 UTC m=+0.431737398 container attach 39c24e952a28d9b4166cd1975b1c4a668e7745dbf6d92fb39d61f64a88dcb369 (image=quay.ceph.io/ceph-ci/ceph@sha256:c15ec3aa8e6c0a9c12ee24ff79f3e7dd4d01b3d1a702dfeb7d7fff9b0811a393, name=ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-1-deactivate, org.label-schema.license=GPLv2, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240709, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , RELEASE=wip-adk4-testing-2024-07-15-1832-squid-a6ca461) 2024-07-16T10:13:31.407 INFO:journalctl@ceph.osd.1.smithi156.stdout:Jul 16 10:13:31 smithi156 podman[152904]: 2024-07-16 10:13:31.122881914 +0000 UTC m=+0.559405290 container died 39c24e952a28d9b4166cd1975b1c4a668e7745dbf6d92fb39d61f64a88dcb369 (image=quay.ceph.io/ceph-ci/ceph@sha256:c15ec3aa8e6c0a9c12ee24ff79f3e7dd4d01b3d1a702dfeb7d7fff9b0811a393, name=ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-1-deactivate, maintainer=Guillaume Abrioux , RELEASE=wip-adk4-testing-2024-07-15-1832-squid-a6ca461, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240709, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.36.0) 2024-07-16T10:13:31.700 DEBUG:teuthology.orchestra.run.smithi156:> sudo pkill -f 'journalctl -f -n 0 -u ceph-26c91c26-4358-11ef-bcad-c7b262605968@osd.1.service' 2024-07-16T10:13:31.740 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-07-16T10:13:31.740 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-07-16T10:13:31.740 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-07-16T10:13:31.740 DEBUG:teuthology.orchestra.run.smithi156:> sudo systemctl stop ceph-26c91c26-4358-11ef-bcad-c7b262605968@osd.2 2024-07-16T10:13:32.078 INFO:journalctl@ceph.osd.2.smithi156.stdout:Jul 16 10:13:31 smithi156 systemd[1]: Stopping Ceph osd.2 for 26c91c26-4358-11ef-bcad-c7b262605968... 2024-07-16T10:13:32.424 INFO:journalctl@ceph.osd.2.smithi156.stdout:Jul 16 10:13:32 smithi156 ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-2[112427]: 2024-07-16T10:13:32.076+0000 7fe545834640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-07-16T10:13:32.424 INFO:journalctl@ceph.osd.2.smithi156.stdout:Jul 16 10:13:32 smithi156 ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-2[112427]: 2024-07-16T10:13:32.076+0000 7fe545834640 -1 osd.2 84 *** Got signal Terminated *** 2024-07-16T10:13:32.425 INFO:journalctl@ceph.osd.2.smithi156.stdout:Jul 16 10:13:32 smithi156 ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-2[112427]: 2024-07-16T10:13:32.076+0000 7fe545834640 -1 osd.2 84 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-07-16T10:13:37.852 INFO:journalctl@ceph.osd.2.smithi156.stdout:Jul 16 10:13:37 smithi156 podman[153009]: 2024-07-16 10:13:37.528366439 +0000 UTC m=+5.562624369 container died c6ecf18de9bfff8ebce934b708f7a97c99b90a23559675f76cf95efaf7ce61ba (image=quay.ceph.io/ceph-ci/ceph@sha256:c15ec3aa8e6c0a9c12ee24ff79f3e7dd4d01b3d1a702dfeb7d7fff9b0811a393, name=ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-2, org.label-schema.build-date=20240709, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.36.0, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-adk4-testing-2024-07-15-1832-squid-a6ca461, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-07-16T10:13:37.852 INFO:journalctl@ceph.osd.2.smithi156.stdout:Jul 16 10:13:37 smithi156 podman[153009]: 2024-07-16 10:13:37.851276977 +0000 UTC m=+5.885534906 container cleanup c6ecf18de9bfff8ebce934b708f7a97c99b90a23559675f76cf95efaf7ce61ba (image=quay.ceph.io/ceph-ci/ceph@sha256:c15ec3aa8e6c0a9c12ee24ff79f3e7dd4d01b3d1a702dfeb7d7fff9b0811a393, name=ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-adk4-testing-2024-07-15-1832-squid-a6ca461, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240709, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.36.0, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2) 2024-07-16T10:13:38.174 INFO:journalctl@ceph.osd.2.smithi156.stdout:Jul 16 10:13:37 smithi156 bash[153009]: ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-2 2024-07-16T10:13:38.175 INFO:journalctl@ceph.osd.2.smithi156.stdout:Jul 16 10:13:38 smithi156 podman[153026]: 2024-07-16 10:13:38.061151792 +0000 UTC m=+0.532557889 container remove c6ecf18de9bfff8ebce934b708f7a97c99b90a23559675f76cf95efaf7ce61ba (image=quay.ceph.io/ceph-ci/ceph@sha256:c15ec3aa8e6c0a9c12ee24ff79f3e7dd4d01b3d1a702dfeb7d7fff9b0811a393, name=ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-2, ceph=True, RELEASE=wip-adk4-testing-2024-07-15-1832-squid-a6ca461, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, io.buildah.version=1.36.0, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240709, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-07-16T10:13:38.798 INFO:journalctl@ceph.osd.2.smithi156.stdout:Jul 16 10:13:38 smithi156 podman[153113]: 2024-07-16 10:13:38.381325179 +0000 UTC m=+0.017604541 image pull 16f7feb043422ce37a2471403d4bc52f550d49cef4b677482112fcc0db3f67ab quay.ceph.io/ceph-ci/ceph@sha256:c15ec3aa8e6c0a9c12ee24ff79f3e7dd4d01b3d1a702dfeb7d7fff9b0811a393 2024-07-16T10:13:38.798 INFO:journalctl@ceph.osd.2.smithi156.stdout:Jul 16 10:13:38 smithi156 podman[153113]: 2024-07-16 10:13:38.495411433 +0000 UTC m=+0.131690804 container create 12fb89c58e34050bd8f57c55338e50bf9cd57a6da3b0c5fdb6971d3e8fdb04de (image=quay.ceph.io/ceph-ci/ceph@sha256:c15ec3aa8e6c0a9c12ee24ff79f3e7dd4d01b3d1a702dfeb7d7fff9b0811a393, name=ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-2-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-adk4-testing-2024-07-15-1832-squid-a6ca461, GIT_CLEAN=True, io.buildah.version=1.36.0, ceph=True, org.label-schema.build-date=20240709, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-07-16T10:13:38.798 INFO:journalctl@ceph.osd.2.smithi156.stdout:Jul 16 10:13:38 smithi156 podman[153113]: 2024-07-16 10:13:38.723026765 +0000 UTC m=+0.359306116 container init 12fb89c58e34050bd8f57c55338e50bf9cd57a6da3b0c5fdb6971d3e8fdb04de (image=quay.ceph.io/ceph-ci/ceph@sha256:c15ec3aa8e6c0a9c12ee24ff79f3e7dd4d01b3d1a702dfeb7d7fff9b0811a393, name=ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-2-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.36.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240709, org.label-schema.license=GPLv2, RELEASE=wip-adk4-testing-2024-07-15-1832-squid-a6ca461, GIT_CLEAN=True) 2024-07-16T10:13:38.798 INFO:journalctl@ceph.osd.2.smithi156.stdout:Jul 16 10:13:38 smithi156 podman[153113]: 2024-07-16 10:13:38.726478887 +0000 UTC m=+0.362758237 container start 12fb89c58e34050bd8f57c55338e50bf9cd57a6da3b0c5fdb6971d3e8fdb04de (image=quay.ceph.io/ceph-ci/ceph@sha256:c15ec3aa8e6c0a9c12ee24ff79f3e7dd4d01b3d1a702dfeb7d7fff9b0811a393, name=ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-2-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, io.buildah.version=1.36.0, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, RELEASE=wip-adk4-testing-2024-07-15-1832-squid-a6ca461, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240709, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-07-16T10:13:38.799 INFO:journalctl@ceph.osd.2.smithi156.stdout:Jul 16 10:13:38 smithi156 podman[153113]: 2024-07-16 10:13:38.797774156 +0000 UTC m=+0.434053519 container attach 12fb89c58e34050bd8f57c55338e50bf9cd57a6da3b0c5fdb6971d3e8fdb04de (image=quay.ceph.io/ceph-ci/ceph@sha256:c15ec3aa8e6c0a9c12ee24ff79f3e7dd4d01b3d1a702dfeb7d7fff9b0811a393, name=ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-2-deactivate, RELEASE=wip-adk4-testing-2024-07-15-1832-squid-a6ca461, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240709, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-07-16T10:13:39.174 INFO:journalctl@ceph.osd.2.smithi156.stdout:Jul 16 10:13:38 smithi156 podman[153113]: 2024-07-16 10:13:38.913567154 +0000 UTC m=+0.549846517 container died 12fb89c58e34050bd8f57c55338e50bf9cd57a6da3b0c5fdb6971d3e8fdb04de (image=quay.ceph.io/ceph-ci/ceph@sha256:c15ec3aa8e6c0a9c12ee24ff79f3e7dd4d01b3d1a702dfeb7d7fff9b0811a393, name=ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-2-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, RELEASE=wip-adk4-testing-2024-07-15-1832-squid-a6ca461, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.36.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240709) 2024-07-16T10:13:39.667 DEBUG:teuthology.orchestra.run.smithi156:> sudo pkill -f 'journalctl -f -n 0 -u ceph-26c91c26-4358-11ef-bcad-c7b262605968@osd.2.service' 2024-07-16T10:13:39.706 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-07-16T10:13:39.706 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-07-16T10:13:39.707 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-07-16T10:13:39.707 DEBUG:teuthology.orchestra.run.smithi176:> sudo systemctl stop ceph-26c91c26-4358-11ef-bcad-c7b262605968@osd.3 2024-07-16T10:13:40.013 INFO:journalctl@ceph.osd.3.smithi176.stdout:Jul 16 10:13:39 smithi176 systemd[1]: Stopping Ceph osd.3 for 26c91c26-4358-11ef-bcad-c7b262605968... 2024-07-16T10:13:40.267 INFO:journalctl@ceph.osd.3.smithi176.stdout:Jul 16 10:13:40 smithi176 ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-3[91068]: 2024-07-16T10:13:40.011+0000 7f9d12c37640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-07-16T10:13:40.267 INFO:journalctl@ceph.osd.3.smithi176.stdout:Jul 16 10:13:40 smithi176 ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-3[91068]: 2024-07-16T10:13:40.011+0000 7f9d12c37640 -1 osd.3 84 *** Got signal Terminated *** 2024-07-16T10:13:40.267 INFO:journalctl@ceph.osd.3.smithi176.stdout:Jul 16 10:13:40 smithi176 ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-3[91068]: 2024-07-16T10:13:40.011+0000 7f9d12c37640 -1 osd.3 84 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-07-16T10:13:45.093 INFO:journalctl@ceph.osd.3.smithi176.stdout:Jul 16 10:13:44 smithi176 ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-3[91068]: 2024-07-16T10:13:44.706+0000 7f9d0fa3d640 -1 osd.3 84 heartbeat_check: no reply from 172.21.15.156:6806 osd.0 since back 2024-07-16T10:13:19.068462+0000 front 2024-07-16T10:13:19.068502+0000 (oldest deadline 2024-07-16T10:13:43.768275+0000) 2024-07-16T10:13:45.614 INFO:journalctl@ceph.osd.3.smithi176.stdout:Jul 16 10:13:45 smithi176 podman[118397]: 2024-07-16 10:13:45.355190461 +0000 UTC m=+5.445224570 container died d216534d60278450eeca86761d90f8e64ee51d272a186899f32cb2836e07576a (image=quay.ceph.io/ceph-ci/ceph@sha256:c15ec3aa8e6c0a9c12ee24ff79f3e7dd4d01b3d1a702dfeb7d7fff9b0811a393, name=ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-3, ceph=True, org.label-schema.build-date=20240709, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-adk4-testing-2024-07-15-1832-squid-a6ca461, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-07-16T10:13:45.614 INFO:journalctl@ceph.osd.4.smithi176.stdout:Jul 16 10:13:45 smithi176 ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-4[95891]: 2024-07-16T10:13:45.448+0000 7ff69bdeb640 -1 osd.4 84 heartbeat_check: no reply from 172.21.15.156:6806 osd.0 since back 2024-07-16T10:13:20.983543+0000 front 2024-07-16T10:13:20.983625+0000 (oldest deadline 2024-07-16T10:13:45.083324+0000) 2024-07-16T10:13:45.614 INFO:journalctl@ceph.osd.5.smithi176.stdout:Jul 16 10:13:45 smithi176 ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-5[100838]: 2024-07-16T10:13:45.551+0000 7fcd943be640 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.156:6806 osd.0 since back 2024-07-16T10:13:20.515373+0000 front 2024-07-16T10:13:20.515256+0000 (oldest deadline 2024-07-16T10:13:44.615176+0000) 2024-07-16T10:13:45.997 INFO:journalctl@ceph.osd.3.smithi176.stdout:Jul 16 10:13:45 smithi176 podman[118397]: 2024-07-16 10:13:45.704040574 +0000 UTC m=+5.794074683 container cleanup d216534d60278450eeca86761d90f8e64ee51d272a186899f32cb2836e07576a (image=quay.ceph.io/ceph-ci/ceph@sha256:c15ec3aa8e6c0a9c12ee24ff79f3e7dd4d01b3d1a702dfeb7d7fff9b0811a393, name=ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-3, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-adk4-testing-2024-07-15-1832-squid-a6ca461, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240709, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0) 2024-07-16T10:13:45.998 INFO:journalctl@ceph.osd.3.smithi176.stdout:Jul 16 10:13:45 smithi176 bash[118397]: ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-3 2024-07-16T10:13:46.330 INFO:journalctl@ceph.osd.3.smithi176.stdout:Jul 16 10:13:45 smithi176 podman[118425]: 2024-07-16 10:13:45.9971031 +0000 UTC m=+0.642204959 container remove d216534d60278450eeca86761d90f8e64ee51d272a186899f32cb2836e07576a (image=quay.ceph.io/ceph-ci/ceph@sha256:c15ec3aa8e6c0a9c12ee24ff79f3e7dd4d01b3d1a702dfeb7d7fff9b0811a393, name=ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-3, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240709, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , RELEASE=wip-adk4-testing-2024-07-15-1832-squid-a6ca461, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.36.0, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-07-16T10:13:46.590 INFO:journalctl@ceph.osd.4.smithi176.stdout:Jul 16 10:13:46 smithi176 ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-4[95891]: 2024-07-16T10:13:46.466+0000 7ff69bdeb640 -1 osd.4 84 heartbeat_check: no reply from 172.21.15.156:6806 osd.0 since back 2024-07-16T10:13:20.983543+0000 front 2024-07-16T10:13:20.983625+0000 (oldest deadline 2024-07-16T10:13:45.083324+0000) 2024-07-16T10:13:46.590 INFO:journalctl@ceph.osd.5.smithi176.stdout:Jul 16 10:13:46 smithi176 ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-5[100838]: 2024-07-16T10:13:46.576+0000 7fcd943be640 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.156:6806 osd.0 since back 2024-07-16T10:13:20.515373+0000 front 2024-07-16T10:13:20.515256+0000 (oldest deadline 2024-07-16T10:13:44.615176+0000) 2024-07-16T10:13:46.591 INFO:journalctl@ceph.osd.3.smithi176.stdout:Jul 16 10:13:46 smithi176 podman[118513]: 2024-07-16 10:13:46.232123359 +0000 UTC m=+0.018448323 image pull 16f7feb043422ce37a2471403d4bc52f550d49cef4b677482112fcc0db3f67ab quay.ceph.io/ceph-ci/ceph@sha256:c15ec3aa8e6c0a9c12ee24ff79f3e7dd4d01b3d1a702dfeb7d7fff9b0811a393 2024-07-16T10:13:46.591 INFO:journalctl@ceph.osd.3.smithi176.stdout:Jul 16 10:13:46 smithi176 podman[118513]: 2024-07-16 10:13:46.448128464 +0000 UTC m=+0.234453436 container create 06c5797e6a3b52ef7e3611200023bb996afdfeb511d7ffb8e8237fd8f092ad93 (image=quay.ceph.io/ceph-ci/ceph@sha256:c15ec3aa8e6c0a9c12ee24ff79f3e7dd4d01b3d1a702dfeb7d7fff9b0811a393, name=ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-3-deactivate, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.36.0, RELEASE=wip-adk4-testing-2024-07-15-1832-squid-a6ca461, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240709, GIT_CLEAN=True) 2024-07-16T10:13:46.843 INFO:journalctl@ceph.osd.3.smithi176.stdout:Jul 16 10:13:46 smithi176 podman[118513]: 2024-07-16 10:13:46.66705687 +0000 UTC m=+0.453381821 container init 06c5797e6a3b52ef7e3611200023bb996afdfeb511d7ffb8e8237fd8f092ad93 (image=quay.ceph.io/ceph-ci/ceph@sha256:c15ec3aa8e6c0a9c12ee24ff79f3e7dd4d01b3d1a702dfeb7d7fff9b0811a393, name=ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-3-deactivate, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240709, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux , RELEASE=wip-adk4-testing-2024-07-15-1832-squid-a6ca461, ceph=True) 2024-07-16T10:13:46.844 INFO:journalctl@ceph.osd.3.smithi176.stdout:Jul 16 10:13:46 smithi176 podman[118513]: 2024-07-16 10:13:46.670661573 +0000 UTC m=+0.456986527 container start 06c5797e6a3b52ef7e3611200023bb996afdfeb511d7ffb8e8237fd8f092ad93 (image=quay.ceph.io/ceph-ci/ceph@sha256:c15ec3aa8e6c0a9c12ee24ff79f3e7dd4d01b3d1a702dfeb7d7fff9b0811a393, name=ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-3-deactivate, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240709, RELEASE=wip-adk4-testing-2024-07-15-1832-squid-a6ca461, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0, GIT_CLEAN=True) 2024-07-16T10:13:46.844 INFO:journalctl@ceph.osd.3.smithi176.stdout:Jul 16 10:13:46 smithi176 podman[118513]: 2024-07-16 10:13:46.742043124 +0000 UTC m=+0.528368074 container attach 06c5797e6a3b52ef7e3611200023bb996afdfeb511d7ffb8e8237fd8f092ad93 (image=quay.ceph.io/ceph-ci/ceph@sha256:c15ec3aa8e6c0a9c12ee24ff79f3e7dd4d01b3d1a702dfeb7d7fff9b0811a393, name=ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-3-deactivate, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, RELEASE=wip-adk4-testing-2024-07-15-1832-squid-a6ca461, org.label-schema.build-date=20240709, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-07-16T10:13:47.146 INFO:journalctl@ceph.osd.3.smithi176.stdout:Jul 16 10:13:46 smithi176 podman[118513]: 2024-07-16 10:13:46.861116036 +0000 UTC m=+0.647440986 container died 06c5797e6a3b52ef7e3611200023bb996afdfeb511d7ffb8e8237fd8f092ad93 (image=quay.ceph.io/ceph-ci/ceph@sha256:c15ec3aa8e6c0a9c12ee24ff79f3e7dd4d01b3d1a702dfeb7d7fff9b0811a393, name=ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-3-deactivate, RELEASE=wip-adk4-testing-2024-07-15-1832-squid-a6ca461, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.36.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240709, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-07-16T10:13:47.440 DEBUG:teuthology.orchestra.run.smithi176:> sudo pkill -f 'journalctl -f -n 0 -u ceph-26c91c26-4358-11ef-bcad-c7b262605968@osd.3.service' 2024-07-16T10:13:47.480 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-07-16T10:13:47.480 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-07-16T10:13:47.481 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-07-16T10:13:47.481 DEBUG:teuthology.orchestra.run.smithi176:> sudo systemctl stop ceph-26c91c26-4358-11ef-bcad-c7b262605968@osd.4 2024-07-16T10:13:47.833 INFO:journalctl@ceph.osd.4.smithi176.stdout:Jul 16 10:13:47 smithi176 ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-4[95891]: 2024-07-16T10:13:47.427+0000 7ff69bdeb640 -1 osd.4 84 heartbeat_check: no reply from 172.21.15.156:6806 osd.0 since back 2024-07-16T10:13:20.983543+0000 front 2024-07-16T10:13:20.983625+0000 (oldest deadline 2024-07-16T10:13:45.083324+0000) 2024-07-16T10:13:47.833 INFO:journalctl@ceph.osd.4.smithi176.stdout:Jul 16 10:13:47 smithi176 systemd[1]: Stopping Ceph osd.4 for 26c91c26-4358-11ef-bcad-c7b262605968... 2024-07-16T10:13:47.833 INFO:journalctl@ceph.osd.5.smithi176.stdout:Jul 16 10:13:47 smithi176 ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-5[100838]: 2024-07-16T10:13:47.567+0000 7fcd943be640 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.156:6806 osd.0 since back 2024-07-16T10:13:20.515373+0000 front 2024-07-16T10:13:20.515256+0000 (oldest deadline 2024-07-16T10:13:44.615176+0000) 2024-07-16T10:13:48.093 INFO:journalctl@ceph.osd.4.smithi176.stdout:Jul 16 10:13:47 smithi176 ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-4[95891]: 2024-07-16T10:13:47.831+0000 7ff6a2fed640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-07-16T10:13:48.093 INFO:journalctl@ceph.osd.4.smithi176.stdout:Jul 16 10:13:47 smithi176 ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-4[95891]: 2024-07-16T10:13:47.831+0000 7ff6a2fed640 -1 osd.4 84 *** Got signal Terminated *** 2024-07-16T10:13:48.094 INFO:journalctl@ceph.osd.4.smithi176.stdout:Jul 16 10:13:47 smithi176 ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-4[95891]: 2024-07-16T10:13:47.831+0000 7ff6a2fed640 -1 osd.4 84 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-07-16T10:13:48.843 INFO:journalctl@ceph.osd.4.smithi176.stdout:Jul 16 10:13:48 smithi176 ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-4[95891]: 2024-07-16T10:13:48.437+0000 7ff69bdeb640 -1 osd.4 84 heartbeat_check: no reply from 172.21.15.156:6806 osd.0 since back 2024-07-16T10:13:20.983543+0000 front 2024-07-16T10:13:20.983625+0000 (oldest deadline 2024-07-16T10:13:45.083324+0000) 2024-07-16T10:13:48.844 INFO:journalctl@ceph.osd.5.smithi176.stdout:Jul 16 10:13:48 smithi176 ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-5[100838]: 2024-07-16T10:13:48.527+0000 7fcd943be640 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.156:6806 osd.0 since back 2024-07-16T10:13:20.515373+0000 front 2024-07-16T10:13:20.515256+0000 (oldest deadline 2024-07-16T10:13:44.615176+0000) 2024-07-16T10:13:49.844 INFO:journalctl@ceph.osd.4.smithi176.stdout:Jul 16 10:13:49 smithi176 ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-4[95891]: 2024-07-16T10:13:49.429+0000 7ff69bdeb640 -1 osd.4 84 heartbeat_check: no reply from 172.21.15.156:6806 osd.0 since back 2024-07-16T10:13:20.983543+0000 front 2024-07-16T10:13:20.983625+0000 (oldest deadline 2024-07-16T10:13:45.083324+0000) 2024-07-16T10:13:49.844 INFO:journalctl@ceph.osd.5.smithi176.stdout:Jul 16 10:13:49 smithi176 ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-5[100838]: 2024-07-16T10:13:49.512+0000 7fcd943be640 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.156:6806 osd.0 since back 2024-07-16T10:13:20.515373+0000 front 2024-07-16T10:13:20.515256+0000 (oldest deadline 2024-07-16T10:13:44.615176+0000) 2024-07-16T10:13:50.708 INFO:journalctl@ceph.osd.4.smithi176.stdout:Jul 16 10:13:50 smithi176 ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-4[95891]: 2024-07-16T10:13:50.385+0000 7ff69bdeb640 -1 osd.4 84 heartbeat_check: no reply from 172.21.15.156:6806 osd.0 since back 2024-07-16T10:13:20.983543+0000 front 2024-07-16T10:13:20.983625+0000 (oldest deadline 2024-07-16T10:13:45.083324+0000) 2024-07-16T10:13:50.708 INFO:journalctl@ceph.osd.5.smithi176.stdout:Jul 16 10:13:50 smithi176 ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-5[100838]: 2024-07-16T10:13:50.537+0000 7fcd943be640 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.156:6806 osd.0 since back 2024-07-16T10:13:20.515373+0000 front 2024-07-16T10:13:20.515256+0000 (oldest deadline 2024-07-16T10:13:44.615176+0000) 2024-07-16T10:13:50.708 INFO:journalctl@ceph.osd.5.smithi176.stdout:Jul 16 10:13:50 smithi176 ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-5[100838]: 2024-07-16T10:13:50.537+0000 7fcd943be640 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.156:6814 osd.1 since back 2024-07-16T10:13:29.316095+0000 front 2024-07-16T10:13:29.316079+0000 (oldest deadline 2024-07-16T10:13:50.415894+0000) 2024-07-16T10:13:51.593 INFO:journalctl@ceph.osd.4.smithi176.stdout:Jul 16 10:13:51 smithi176 ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-4[95891]: 2024-07-16T10:13:51.419+0000 7ff69bdeb640 -1 osd.4 84 heartbeat_check: no reply from 172.21.15.156:6806 osd.0 since back 2024-07-16T10:13:20.983543+0000 front 2024-07-16T10:13:20.983625+0000 (oldest deadline 2024-07-16T10:13:45.083324+0000) 2024-07-16T10:13:51.594 INFO:journalctl@ceph.osd.4.smithi176.stdout:Jul 16 10:13:51 smithi176 ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-4[95891]: 2024-07-16T10:13:51.419+0000 7ff69bdeb640 -1 osd.4 84 heartbeat_check: no reply from 172.21.15.156:6814 osd.1 since back 2024-07-16T10:13:25.083960+0000 front 2024-07-16T10:13:25.083890+0000 (oldest deadline 2024-07-16T10:13:50.983718+0000) 2024-07-16T10:13:51.594 INFO:journalctl@ceph.osd.5.smithi176.stdout:Jul 16 10:13:51 smithi176 ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-5[100838]: 2024-07-16T10:13:51.503+0000 7fcd943be640 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.156:6806 osd.0 since back 2024-07-16T10:13:20.515373+0000 front 2024-07-16T10:13:20.515256+0000 (oldest deadline 2024-07-16T10:13:44.615176+0000) 2024-07-16T10:13:51.594 INFO:journalctl@ceph.osd.5.smithi176.stdout:Jul 16 10:13:51 smithi176 ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-5[100838]: 2024-07-16T10:13:51.503+0000 7fcd943be640 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.156:6814 osd.1 since back 2024-07-16T10:13:29.316095+0000 front 2024-07-16T10:13:29.316079+0000 (oldest deadline 2024-07-16T10:13:50.415894+0000) 2024-07-16T10:13:52.844 INFO:journalctl@ceph.osd.4.smithi176.stdout:Jul 16 10:13:52 smithi176 ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-4[95891]: 2024-07-16T10:13:52.385+0000 7ff69bdeb640 -1 osd.4 84 heartbeat_check: no reply from 172.21.15.156:6806 osd.0 since back 2024-07-16T10:13:20.983543+0000 front 2024-07-16T10:13:20.983625+0000 (oldest deadline 2024-07-16T10:13:45.083324+0000) 2024-07-16T10:13:52.844 INFO:journalctl@ceph.osd.4.smithi176.stdout:Jul 16 10:13:52 smithi176 ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-4[95891]: 2024-07-16T10:13:52.385+0000 7ff69bdeb640 -1 osd.4 84 heartbeat_check: no reply from 172.21.15.156:6814 osd.1 since back 2024-07-16T10:13:25.083960+0000 front 2024-07-16T10:13:25.083890+0000 (oldest deadline 2024-07-16T10:13:50.983718+0000) 2024-07-16T10:13:52.844 INFO:journalctl@ceph.osd.5.smithi176.stdout:Jul 16 10:13:52 smithi176 ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-5[100838]: 2024-07-16T10:13:52.473+0000 7fcd943be640 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.156:6806 osd.0 since back 2024-07-16T10:13:20.515373+0000 front 2024-07-16T10:13:20.515256+0000 (oldest deadline 2024-07-16T10:13:44.615176+0000) 2024-07-16T10:13:52.844 INFO:journalctl@ceph.osd.5.smithi176.stdout:Jul 16 10:13:52 smithi176 ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-5[100838]: 2024-07-16T10:13:52.473+0000 7fcd943be640 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.156:6814 osd.1 since back 2024-07-16T10:13:29.316095+0000 front 2024-07-16T10:13:29.316079+0000 (oldest deadline 2024-07-16T10:13:50.415894+0000) 2024-07-16T10:13:53.558 INFO:journalctl@ceph.osd.4.smithi176.stdout:Jul 16 10:13:53 smithi176 podman[118616]: 2024-07-16 10:13:53.292125347 +0000 UTC m=+5.577993557 container died 1bd3a25317b011abe1e64a630e6fb3932de6f017001ef671784eaf4a40b15409 (image=quay.ceph.io/ceph-ci/ceph@sha256:c15ec3aa8e6c0a9c12ee24ff79f3e7dd4d01b3d1a702dfeb7d7fff9b0811a393, name=ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-4, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240709, maintainer=Guillaume Abrioux , RELEASE=wip-adk4-testing-2024-07-15-1832-squid-a6ca461, io.buildah.version=1.36.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-07-16T10:13:53.558 INFO:journalctl@ceph.osd.5.smithi176.stdout:Jul 16 10:13:53 smithi176 ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-5[100838]: 2024-07-16T10:13:53.469+0000 7fcd943be640 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.156:6806 osd.0 since back 2024-07-16T10:13:20.515373+0000 front 2024-07-16T10:13:20.515256+0000 (oldest deadline 2024-07-16T10:13:44.615176+0000) 2024-07-16T10:13:53.558 INFO:journalctl@ceph.osd.5.smithi176.stdout:Jul 16 10:13:53 smithi176 ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-5[100838]: 2024-07-16T10:13:53.469+0000 7fcd943be640 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.156:6814 osd.1 since back 2024-07-16T10:13:29.316095+0000 front 2024-07-16T10:13:29.316079+0000 (oldest deadline 2024-07-16T10:13:50.415894+0000) 2024-07-16T10:13:53.844 INFO:journalctl@ceph.osd.4.smithi176.stdout:Jul 16 10:13:53 smithi176 podman[118616]: 2024-07-16 10:13:53.648262783 +0000 UTC m=+5.934130995 container cleanup 1bd3a25317b011abe1e64a630e6fb3932de6f017001ef671784eaf4a40b15409 (image=quay.ceph.io/ceph-ci/ceph@sha256:c15ec3aa8e6c0a9c12ee24ff79f3e7dd4d01b3d1a702dfeb7d7fff9b0811a393, name=ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-4, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-adk4-testing-2024-07-15-1832-squid-a6ca461, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240709, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.36.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-07-16T10:13:53.844 INFO:journalctl@ceph.osd.4.smithi176.stdout:Jul 16 10:13:53 smithi176 bash[118616]: ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-4 2024-07-16T10:13:54.274 INFO:journalctl@ceph.osd.4.smithi176.stdout:Jul 16 10:13:53 smithi176 podman[118631]: 2024-07-16 10:13:53.932771028 +0000 UTC m=+0.633136436 container remove 1bd3a25317b011abe1e64a630e6fb3932de6f017001ef671784eaf4a40b15409 (image=quay.ceph.io/ceph-ci/ceph@sha256:c15ec3aa8e6c0a9c12ee24ff79f3e7dd4d01b3d1a702dfeb7d7fff9b0811a393, name=ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-4, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-adk4-testing-2024-07-15-1832-squid-a6ca461, io.buildah.version=1.36.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.build-date=20240709) 2024-07-16T10:13:54.593 INFO:journalctl@ceph.osd.5.smithi176.stdout:Jul 16 10:13:54 smithi176 ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-5[100838]: 2024-07-16T10:13:54.490+0000 7fcd943be640 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.156:6806 osd.0 since back 2024-07-16T10:13:20.515373+0000 front 2024-07-16T10:13:20.515256+0000 (oldest deadline 2024-07-16T10:13:44.615176+0000) 2024-07-16T10:13:54.594 INFO:journalctl@ceph.osd.5.smithi176.stdout:Jul 16 10:13:54 smithi176 ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-5[100838]: 2024-07-16T10:13:54.490+0000 7fcd943be640 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.156:6814 osd.1 since back 2024-07-16T10:13:29.316095+0000 front 2024-07-16T10:13:29.316079+0000 (oldest deadline 2024-07-16T10:13:50.415894+0000) 2024-07-16T10:13:54.594 INFO:journalctl@ceph.osd.4.smithi176.stdout:Jul 16 10:13:54 smithi176 podman[118722]: 2024-07-16 10:13:54.176815412 +0000 UTC m=+0.017954040 image pull 16f7feb043422ce37a2471403d4bc52f550d49cef4b677482112fcc0db3f67ab quay.ceph.io/ceph-ci/ceph@sha256:c15ec3aa8e6c0a9c12ee24ff79f3e7dd4d01b3d1a702dfeb7d7fff9b0811a393 2024-07-16T10:13:54.594 INFO:journalctl@ceph.osd.4.smithi176.stdout:Jul 16 10:13:54 smithi176 podman[118722]: 2024-07-16 10:13:54.308988416 +0000 UTC m=+0.150127082 container create 0194066e4b62531aac6892a58fe6af80c4c4571dfbbd3410dec5a3bb218ea4ee (image=quay.ceph.io/ceph-ci/ceph@sha256:c15ec3aa8e6c0a9c12ee24ff79f3e7dd4d01b3d1a702dfeb7d7fff9b0811a393, name=ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-4-deactivate, RELEASE=wip-adk4-testing-2024-07-15-1832-squid-a6ca461, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, io.buildah.version=1.36.0, org.label-schema.build-date=20240709, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-07-16T10:13:54.594 INFO:journalctl@ceph.osd.4.smithi176.stdout:Jul 16 10:13:54 smithi176 podman[118722]: 2024-07-16 10:13:54.519604697 +0000 UTC m=+0.360743338 container init 0194066e4b62531aac6892a58fe6af80c4c4571dfbbd3410dec5a3bb218ea4ee (image=quay.ceph.io/ceph-ci/ceph@sha256:c15ec3aa8e6c0a9c12ee24ff79f3e7dd4d01b3d1a702dfeb7d7fff9b0811a393, name=ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-4-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0, org.label-schema.build-date=20240709, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, RELEASE=wip-adk4-testing-2024-07-15-1832-squid-a6ca461, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-07-16T10:13:54.594 INFO:journalctl@ceph.osd.4.smithi176.stdout:Jul 16 10:13:54 smithi176 podman[118722]: 2024-07-16 10:13:54.523787497 +0000 UTC m=+0.364926130 container start 0194066e4b62531aac6892a58fe6af80c4c4571dfbbd3410dec5a3bb218ea4ee (image=quay.ceph.io/ceph-ci/ceph@sha256:c15ec3aa8e6c0a9c12ee24ff79f3e7dd4d01b3d1a702dfeb7d7fff9b0811a393, name=ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-4-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240709, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-adk4-testing-2024-07-15-1832-squid-a6ca461, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True) 2024-07-16T10:13:55.006 INFO:journalctl@ceph.osd.4.smithi176.stdout:Jul 16 10:13:54 smithi176 podman[118722]: 2024-07-16 10:13:54.594447024 +0000 UTC m=+0.435585648 container attach 0194066e4b62531aac6892a58fe6af80c4c4571dfbbd3410dec5a3bb218ea4ee (image=quay.ceph.io/ceph-ci/ceph@sha256:c15ec3aa8e6c0a9c12ee24ff79f3e7dd4d01b3d1a702dfeb7d7fff9b0811a393, name=ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-4-deactivate, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240709, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-adk4-testing-2024-07-15-1832-squid-a6ca461, io.buildah.version=1.36.0, CEPH_POINT_RELEASE=) 2024-07-16T10:13:55.006 INFO:journalctl@ceph.osd.4.smithi176.stdout:Jul 16 10:13:54 smithi176 podman[118722]: 2024-07-16 10:13:54.711957161 +0000 UTC m=+0.553095788 container died 0194066e4b62531aac6892a58fe6af80c4c4571dfbbd3410dec5a3bb218ea4ee (image=quay.ceph.io/ceph-ci/ceph@sha256:c15ec3aa8e6c0a9c12ee24ff79f3e7dd4d01b3d1a702dfeb7d7fff9b0811a393, name=ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-4-deactivate, maintainer=Guillaume Abrioux , RELEASE=wip-adk4-testing-2024-07-15-1832-squid-a6ca461, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240709, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.36.0) 2024-07-16T10:13:55.312 DEBUG:teuthology.orchestra.run.smithi176:> sudo pkill -f 'journalctl -f -n 0 -u ceph-26c91c26-4358-11ef-bcad-c7b262605968@osd.4.service' 2024-07-16T10:13:55.351 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-07-16T10:13:55.351 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-07-16T10:13:55.351 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-07-16T10:13:55.351 DEBUG:teuthology.orchestra.run.smithi176:> sudo systemctl stop ceph-26c91c26-4358-11ef-bcad-c7b262605968@osd.5 2024-07-16T10:13:55.594 INFO:journalctl@ceph.osd.5.smithi176.stdout:Jul 16 10:13:55 smithi176 systemd[1]: Stopping Ceph osd.5 for 26c91c26-4358-11ef-bcad-c7b262605968... 2024-07-16T10:13:55.594 INFO:journalctl@ceph.osd.5.smithi176.stdout:Jul 16 10:13:55 smithi176 ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-5[100838]: 2024-07-16T10:13:55.508+0000 7fcd943be640 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.156:6806 osd.0 since back 2024-07-16T10:13:20.515373+0000 front 2024-07-16T10:13:20.515256+0000 (oldest deadline 2024-07-16T10:13:44.615176+0000) 2024-07-16T10:13:55.594 INFO:journalctl@ceph.osd.5.smithi176.stdout:Jul 16 10:13:55 smithi176 ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-5[100838]: 2024-07-16T10:13:55.508+0000 7fcd943be640 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.156:6814 osd.1 since back 2024-07-16T10:13:29.316095+0000 front 2024-07-16T10:13:29.316079+0000 (oldest deadline 2024-07-16T10:13:50.415894+0000) 2024-07-16T10:13:56.093 INFO:journalctl@ceph.osd.5.smithi176.stdout:Jul 16 10:13:55 smithi176 ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-5[100838]: 2024-07-16T10:13:55.684+0000 7fcd975b8640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-07-16T10:13:56.094 INFO:journalctl@ceph.osd.5.smithi176.stdout:Jul 16 10:13:55 smithi176 ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-5[100838]: 2024-07-16T10:13:55.684+0000 7fcd975b8640 -1 osd.5 84 *** Got signal Terminated *** 2024-07-16T10:13:56.094 INFO:journalctl@ceph.osd.5.smithi176.stdout:Jul 16 10:13:55 smithi176 ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-5[100838]: 2024-07-16T10:13:55.684+0000 7fcd975b8640 -1 osd.5 84 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-07-16T10:13:56.843 INFO:journalctl@ceph.osd.5.smithi176.stdout:Jul 16 10:13:56 smithi176 ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-5[100838]: 2024-07-16T10:13:56.510+0000 7fcd943be640 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.156:6806 osd.0 since back 2024-07-16T10:13:20.515373+0000 front 2024-07-16T10:13:20.515256+0000 (oldest deadline 2024-07-16T10:13:44.615176+0000) 2024-07-16T10:13:56.844 INFO:journalctl@ceph.osd.5.smithi176.stdout:Jul 16 10:13:56 smithi176 ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-5[100838]: 2024-07-16T10:13:56.510+0000 7fcd943be640 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.156:6814 osd.1 since back 2024-07-16T10:13:29.316095+0000 front 2024-07-16T10:13:29.316079+0000 (oldest deadline 2024-07-16T10:13:50.415894+0000) 2024-07-16T10:13:57.843 INFO:journalctl@ceph.osd.5.smithi176.stdout:Jul 16 10:13:57 smithi176 ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-5[100838]: 2024-07-16T10:13:57.542+0000 7fcd943be640 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.156:6806 osd.0 since back 2024-07-16T10:13:20.515373+0000 front 2024-07-16T10:13:20.515256+0000 (oldest deadline 2024-07-16T10:13:44.615176+0000) 2024-07-16T10:13:57.843 INFO:journalctl@ceph.osd.5.smithi176.stdout:Jul 16 10:13:57 smithi176 ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-5[100838]: 2024-07-16T10:13:57.542+0000 7fcd943be640 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.156:6814 osd.1 since back 2024-07-16T10:13:29.316095+0000 front 2024-07-16T10:13:29.316079+0000 (oldest deadline 2024-07-16T10:13:50.415894+0000) 2024-07-16T10:13:58.843 INFO:journalctl@ceph.osd.5.smithi176.stdout:Jul 16 10:13:58 smithi176 ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-5[100838]: 2024-07-16T10:13:58.506+0000 7fcd943be640 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.156:6806 osd.0 since back 2024-07-16T10:13:20.515373+0000 front 2024-07-16T10:13:20.515256+0000 (oldest deadline 2024-07-16T10:13:44.615176+0000) 2024-07-16T10:13:58.844 INFO:journalctl@ceph.osd.5.smithi176.stdout:Jul 16 10:13:58 smithi176 ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-5[100838]: 2024-07-16T10:13:58.506+0000 7fcd943be640 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.156:6814 osd.1 since back 2024-07-16T10:13:29.316095+0000 front 2024-07-16T10:13:29.316079+0000 (oldest deadline 2024-07-16T10:13:50.415894+0000) 2024-07-16T10:13:59.843 INFO:journalctl@ceph.osd.5.smithi176.stdout:Jul 16 10:13:59 smithi176 ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-5[100838]: 2024-07-16T10:13:59.547+0000 7fcd943be640 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.156:6806 osd.0 since back 2024-07-16T10:13:20.515373+0000 front 2024-07-16T10:13:20.515256+0000 (oldest deadline 2024-07-16T10:13:44.615176+0000) 2024-07-16T10:13:59.844 INFO:journalctl@ceph.osd.5.smithi176.stdout:Jul 16 10:13:59 smithi176 ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-5[100838]: 2024-07-16T10:13:59.547+0000 7fcd943be640 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.156:6814 osd.1 since back 2024-07-16T10:13:29.316095+0000 front 2024-07-16T10:13:29.316079+0000 (oldest deadline 2024-07-16T10:13:50.415894+0000) 2024-07-16T10:13:59.844 INFO:journalctl@ceph.osd.5.smithi176.stdout:Jul 16 10:13:59 smithi176 ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-5[100838]: 2024-07-16T10:13:59.547+0000 7fcd943be640 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.156:6822 osd.2 since back 2024-07-16T10:13:36.117354+0000 front 2024-07-16T10:13:36.117379+0000 (oldest deadline 2024-07-16T10:13:59.517795+0000) 2024-07-16T10:14:00.844 INFO:journalctl@ceph.osd.5.smithi176.stdout:Jul 16 10:14:00 smithi176 ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-5[100838]: 2024-07-16T10:14:00.536+0000 7fcd943be640 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.156:6806 osd.0 since back 2024-07-16T10:13:20.515373+0000 front 2024-07-16T10:13:20.515256+0000 (oldest deadline 2024-07-16T10:13:44.615176+0000) 2024-07-16T10:14:00.844 INFO:journalctl@ceph.osd.5.smithi176.stdout:Jul 16 10:14:00 smithi176 ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-5[100838]: 2024-07-16T10:14:00.536+0000 7fcd943be640 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.156:6814 osd.1 since back 2024-07-16T10:13:29.316095+0000 front 2024-07-16T10:13:29.316079+0000 (oldest deadline 2024-07-16T10:13:50.415894+0000) 2024-07-16T10:14:00.844 INFO:journalctl@ceph.osd.5.smithi176.stdout:Jul 16 10:14:00 smithi176 ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-5[100838]: 2024-07-16T10:14:00.536+0000 7fcd943be640 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.156:6822 osd.2 since back 2024-07-16T10:13:36.117354+0000 front 2024-07-16T10:13:36.117379+0000 (oldest deadline 2024-07-16T10:13:59.517795+0000) 2024-07-16T10:14:01.285 INFO:journalctl@ceph.osd.5.smithi176.stdout:Jul 16 10:14:01 smithi176 podman[118826]: 2024-07-16 10:14:01.013166055 +0000 UTC m=+5.439930232 container died a03640221e7aaf17143732e69641d2ff5b8644e9028fab51caee28aa114e4b5c (image=quay.ceph.io/ceph-ci/ceph@sha256:c15ec3aa8e6c0a9c12ee24ff79f3e7dd4d01b3d1a702dfeb7d7fff9b0811a393, name=ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240709, CEPH_POINT_RELEASE=, io.buildah.version=1.36.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-adk4-testing-2024-07-15-1832-squid-a6ca461, GIT_CLEAN=True) 2024-07-16T10:14:01.593 INFO:journalctl@ceph.osd.5.smithi176.stdout:Jul 16 10:14:01 smithi176 podman[118826]: 2024-07-16 10:14:01.37584108 +0000 UTC m=+5.802605257 container cleanup a03640221e7aaf17143732e69641d2ff5b8644e9028fab51caee28aa114e4b5c (image=quay.ceph.io/ceph-ci/ceph@sha256:c15ec3aa8e6c0a9c12ee24ff79f3e7dd4d01b3d1a702dfeb7d7fff9b0811a393, name=ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240709, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0, RELEASE=wip-adk4-testing-2024-07-15-1832-squid-a6ca461, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-07-16T10:14:01.594 INFO:journalctl@ceph.osd.5.smithi176.stdout:Jul 16 10:14:01 smithi176 bash[118826]: ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-5 2024-07-16T10:14:02.004 INFO:journalctl@ceph.osd.5.smithi176.stdout:Jul 16 10:14:01 smithi176 podman[118842]: 2024-07-16 10:14:01.636722366 +0000 UTC m=+0.616014803 container remove a03640221e7aaf17143732e69641d2ff5b8644e9028fab51caee28aa114e4b5c (image=quay.ceph.io/ceph-ci/ceph@sha256:c15ec3aa8e6c0a9c12ee24ff79f3e7dd4d01b3d1a702dfeb7d7fff9b0811a393, name=ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-5, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240709, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.36.0, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-adk4-testing-2024-07-15-1832-squid-a6ca461, maintainer=Guillaume Abrioux ) 2024-07-16T10:14:02.305 INFO:journalctl@ceph.osd.5.smithi176.stdout:Jul 16 10:14:02 smithi176 podman[118936]: 2024-07-16 10:14:01.906553786 +0000 UTC m=+0.017356064 image pull 16f7feb043422ce37a2471403d4bc52f550d49cef4b677482112fcc0db3f67ab quay.ceph.io/ceph-ci/ceph@sha256:c15ec3aa8e6c0a9c12ee24ff79f3e7dd4d01b3d1a702dfeb7d7fff9b0811a393 2024-07-16T10:14:02.306 INFO:journalctl@ceph.osd.5.smithi176.stdout:Jul 16 10:14:02 smithi176 podman[118936]: 2024-07-16 10:14:02.019478539 +0000 UTC m=+0.130280796 container create 3977da643e92d742a5eef21f1c259f6015bc82f9a81d7db9204f11f778553ab5 (image=quay.ceph.io/ceph-ci/ceph@sha256:c15ec3aa8e6c0a9c12ee24ff79f3e7dd4d01b3d1a702dfeb7d7fff9b0811a393, name=ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-5-deactivate, org.label-schema.build-date=20240709, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.36.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-adk4-testing-2024-07-15-1832-squid-a6ca461, CEPH_POINT_RELEASE=) 2024-07-16T10:14:02.306 INFO:journalctl@ceph.osd.5.smithi176.stdout:Jul 16 10:14:02 smithi176 podman[118936]: 2024-07-16 10:14:02.230318067 +0000 UTC m=+0.341120343 container init 3977da643e92d742a5eef21f1c259f6015bc82f9a81d7db9204f11f778553ab5 (image=quay.ceph.io/ceph-ci/ceph@sha256:c15ec3aa8e6c0a9c12ee24ff79f3e7dd4d01b3d1a702dfeb7d7fff9b0811a393, name=ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-5-deactivate, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240709, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.36.0, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-adk4-testing-2024-07-15-1832-squid-a6ca461, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-07-16T10:14:02.306 INFO:journalctl@ceph.osd.5.smithi176.stdout:Jul 16 10:14:02 smithi176 podman[118936]: 2024-07-16 10:14:02.234494454 +0000 UTC m=+0.345296731 container start 3977da643e92d742a5eef21f1c259f6015bc82f9a81d7db9204f11f778553ab5 (image=quay.ceph.io/ceph-ci/ceph@sha256:c15ec3aa8e6c0a9c12ee24ff79f3e7dd4d01b3d1a702dfeb7d7fff9b0811a393, name=ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-5-deactivate, org.label-schema.schema-version=1.0, io.buildah.version=1.36.0, ceph=True, GIT_CLEAN=True, RELEASE=wip-adk4-testing-2024-07-15-1832-squid-a6ca461, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240709, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-07-16T10:14:02.306 INFO:journalctl@ceph.osd.5.smithi176.stdout:Jul 16 10:14:02 smithi176 podman[118936]: 2024-07-16 10:14:02.30511165 +0000 UTC m=+0.415913930 container attach 3977da643e92d742a5eef21f1c259f6015bc82f9a81d7db9204f11f778553ab5 (image=quay.ceph.io/ceph-ci/ceph@sha256:c15ec3aa8e6c0a9c12ee24ff79f3e7dd4d01b3d1a702dfeb7d7fff9b0811a393, name=ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-5-deactivate, io.buildah.version=1.36.0, org.label-schema.build-date=20240709, ceph=True, RELEASE=wip-adk4-testing-2024-07-15-1832-squid-a6ca461, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-07-16T10:14:02.594 INFO:journalctl@ceph.osd.5.smithi176.stdout:Jul 16 10:14:02 smithi176 podman[118936]: 2024-07-16 10:14:02.427302801 +0000 UTC m=+0.538105079 container died 3977da643e92d742a5eef21f1c259f6015bc82f9a81d7db9204f11f778553ab5 (image=quay.ceph.io/ceph-ci/ceph@sha256:c15ec3aa8e6c0a9c12ee24ff79f3e7dd4d01b3d1a702dfeb7d7fff9b0811a393, name=ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-5-deactivate, org.label-schema.build-date=20240709, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.36.0, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-adk4-testing-2024-07-15-1832-squid-a6ca461, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-07-16T10:14:02.986 INFO:journalctl@ceph.osd.5.smithi176.stdout:Jul 16 10:14:02 smithi176 podman[118959]: 2024-07-16 10:14:02.975449672 +0000 UTC m=+0.542093142 container remove 3977da643e92d742a5eef21f1c259f6015bc82f9a81d7db9204f11f778553ab5 (image=quay.ceph.io/ceph-ci/ceph@sha256:c15ec3aa8e6c0a9c12ee24ff79f3e7dd4d01b3d1a702dfeb7d7fff9b0811a393, name=ceph-26c91c26-4358-11ef-bcad-c7b262605968-osd-5-deactivate, RELEASE=wip-adk4-testing-2024-07-15-1832-squid-a6ca461, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240709, org.label-schema.schema-version=1.0) 2024-07-16T10:14:02.992 DEBUG:teuthology.orchestra.run.smithi176:> sudo pkill -f 'journalctl -f -n 0 -u ceph-26c91c26-4358-11ef-bcad-c7b262605968@osd.5.service' 2024-07-16T10:14:03.031 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-07-16T10:14:03.031 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-07-16T10:14:03.031 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 26c91c26-4358-11ef-bcad-c7b262605968 --force --keep-logs 2024-07-16T10:14:06.440 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi156.stderr:ceph-fuse[76406]: fuse finished with error 0 and tester_r 0 2024-07-16T10:14:21.497 DEBUG:teuthology.orchestra.run.smithi176:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 26c91c26-4358-11ef-bcad-c7b262605968 --force --keep-logs 2024-07-16T10:14:33.182 DEBUG:teuthology.orchestra.run.smithi156:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-07-16T10:14:33.215 DEBUG:teuthology.orchestra.run.smithi176:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-07-16T10:14:33.242 INFO:tasks.cephadm:Archiving crash dumps... 2024-07-16T10:14:33.244 DEBUG:teuthology.misc:Transferring archived files from smithi156:/var/lib/ceph/26c91c26-4358-11ef-bcad-c7b262605968/crash to /home/teuthworker/archive/adking-2024-07-16_04:15:24-orch:cephadm-wip-adk4-testing-2024-07-15-1832-squid-distro-default-smithi/7803693/remote/smithi156/crash 2024-07-16T10:14:33.245 DEBUG:teuthology.orchestra.run.smithi156:> sudo tar c -f - -C /var/lib/ceph/26c91c26-4358-11ef-bcad-c7b262605968/crash -- . 2024-07-16T10:14:33.283 INFO:teuthology.orchestra.run.smithi156.stderr:tar: /var/lib/ceph/26c91c26-4358-11ef-bcad-c7b262605968/crash: Cannot open: No such file or directory 2024-07-16T10:14:33.283 INFO:teuthology.orchestra.run.smithi156.stderr:tar: Error is not recoverable: exiting now 2024-07-16T10:14:33.285 DEBUG:teuthology.misc:Transferring archived files from smithi176:/var/lib/ceph/26c91c26-4358-11ef-bcad-c7b262605968/crash to /home/teuthworker/archive/adking-2024-07-16_04:15:24-orch:cephadm-wip-adk4-testing-2024-07-15-1832-squid-distro-default-smithi/7803693/remote/smithi176/crash 2024-07-16T10:14:33.286 DEBUG:teuthology.orchestra.run.smithi176:> sudo tar c -f - -C /var/lib/ceph/26c91c26-4358-11ef-bcad-c7b262605968/crash -- . 2024-07-16T10:14:33.310 INFO:teuthology.orchestra.run.smithi176.stderr:tar: /var/lib/ceph/26c91c26-4358-11ef-bcad-c7b262605968/crash: Cannot open: No such file or directory 2024-07-16T10:14:33.310 INFO:teuthology.orchestra.run.smithi176.stderr:tar: Error is not recoverable: exiting now 2024-07-16T10:14:33.311 INFO:tasks.cephadm:Checking cluster log for badness... 2024-07-16T10:14:33.312 DEBUG:teuthology.orchestra.run.smithi156:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/26c91c26-4358-11ef-bcad-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v OSD_DOWN | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | head -n 1 2024-07-16T10:14:33.366 INFO:tasks.cephadm:Compressing logs... 2024-07-16T10:14:33.366 DEBUG:teuthology.orchestra.run.smithi156:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-07-16T10:14:33.408 DEBUG:teuthology.orchestra.run.smithi176:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-07-16T10:14:33.432 INFO:teuthology.orchestra.run.smithi156.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/ceph-client.0.log 2024-07-16T10:14:33.432 INFO:teuthology.orchestra.run.smithi156.stderr:: No such file or directory 2024-07-16T10:14:33.432 INFO:teuthology.orchestra.run.smithi156.stderr:gzip -5 --verbose -- /var/log/ceph/26c91c26-4358-11ef-bcad-c7b262605968/ceph-mds.cephfs.smithi156.evwwaz.log 2024-07-16T10:14:33.432 INFO:teuthology.orchestra.run.smithi156.stderr:/var/log/ceph/ceph-client.0.log: gzip -5 --verbose -- /var/log/ceph/26c91c26-4358-11ef-bcad-c7b262605968/ceph.log 2024-07-16T10:14:33.432 INFO:teuthology.orchestra.run.smithi156.stderr:gzip -5/var/log/ceph/26c91c26-4358-11ef-bcad-c7b262605968/ceph-mds.cephfs.smithi156.evwwaz.log: --verbose -- /var/log/ceph/26c91c26-4358-11ef-bcad-c7b262605968/ceph-osd.2.log 2024-07-16T10:14:33.436 INFO:teuthology.orchestra.run.smithi176.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/ceph-client.1.log 2024-07-16T10:14:33.436 INFO:teuthology.orchestra.run.smithi176.stderr:: No such file or directory 2024-07-16T10:14:33.436 INFO:teuthology.orchestra.run.smithi176.stderr:gzip -5 --verbose -- /var/log/ceph/26c91c26-4358-11ef-bcad-c7b262605968/ceph.log 2024-07-16T10:14:33.436 INFO:teuthology.orchestra.run.smithi176.stderr:gzip -5 --verbose -- /var/log/ceph/26c91c26-4358-11ef-bcad-c7b262605968/ceph.audit.log 2024-07-16T10:14:33.436 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.1.log: gzip -5 --verbose -- /var/log/ceph/26c91c26-4358-11ef-bcad-c7b262605968/ceph-osd.5.log 2024-07-16T10:14:33.436 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/26c91c26-4358-11ef-bcad-c7b262605968/ceph.log: /var/log/ceph/26c91c26-4358-11ef-bcad-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/26c91c26-4358-11ef-bcad-c7b262605968/ceph-osd.3.log 2024-07-16T10:14:33.436 INFO:teuthology.orchestra.run.smithi176.stderr:gzip -5 --verbose -- /var/log/ceph/26c91c26-4358-11ef-bcad-c7b262605968/ceph-mon.smithi176.log 2024-07-16T10:14:33.436 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/26c91c26-4358-11ef-bcad-c7b262605968/ceph-osd.5.log: /var/log/ceph/26c91c26-4358-11ef-bcad-c7b262605968/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/26c91c26-4358-11ef-bcad-c7b262605968/ceph-osd.4.log 2024-07-16T10:14:33.437 INFO:teuthology.orchestra.run.smithi156.stderr:/var/log/ceph/26c91c26-4358-11ef-bcad-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/26c91c26-4358-11ef-bcad-c7b262605968/ceph.audit.log 2024-07-16T10:14:33.438 INFO:teuthology.orchestra.run.smithi156.stderr:gzip -5 --verbose -- /var/log/ceph/26c91c26-4358-11ef-bcad-c7b262605968/ceph-mgr.smithi156.zkvylv.log 2024-07-16T10:14:33.438 INFO:teuthology.orchestra.run.smithi156.stderr:/var/log/ceph/26c91c26-4358-11ef-bcad-c7b262605968/ceph-osd.2.log: /var/log/ceph/26c91c26-4358-11ef-bcad-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/26c91c26-4358-11ef-bcad-c7b262605968/ceph-osd.1.log 2024-07-16T10:14:33.438 INFO:teuthology.orchestra.run.smithi156.stderr:/var/log/ceph/26c91c26-4358-11ef-bcad-c7b262605968/ceph-mgr.smithi156.zkvylv.log: gzip -5 --verbose -- /var/log/ceph/26c91c26-4358-11ef-bcad-c7b262605968/ceph-osd.0.log 2024-07-16T10:14:33.438 INFO:teuthology.orchestra.run.smithi156.stderr:/var/log/ceph/26c91c26-4358-11ef-bcad-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/26c91c26-4358-11ef-bcad-c7b262605968/ceph.cephadm.log 2024-07-16T10:14:33.438 INFO:teuthology.orchestra.run.smithi156.stderr:/var/log/ceph/26c91c26-4358-11ef-bcad-c7b262605968/ceph-osd.0.log: 88.4% -- replaced with /var/log/ceph/26c91c26-4358-11ef-bcad-c7b262605968/ceph.log.gz 2024-07-16T10:14:33.439 INFO:teuthology.orchestra.run.smithi156.stderr: 91.3% -- replaced with /var/log/ceph/26c91c26-4358-11ef-bcad-c7b262605968/ceph.audit.log.gz 2024-07-16T10:14:33.439 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/26c91c26-4358-11ef-bcad-c7b262605968/ceph-mon.smithi176.log: gzip -5 --verbose -- /var/log/ceph/26c91c26-4358-11ef-bcad-c7b262605968/ceph-mgr.smithi176.dqbrvx.log 2024-07-16T10:14:33.439 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/26c91c26-4358-11ef-bcad-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/26c91c26-4358-11ef-bcad-c7b262605968/ceph.cephadm.log 2024-07-16T10:14:33.439 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/26c91c26-4358-11ef-bcad-c7b262605968/ceph-mgr.smithi176.dqbrvx.log: 88.4% -- replaced with /var/log/ceph/26c91c26-4358-11ef-bcad-c7b262605968/ceph.log.gz 2024-07-16T10:14:33.439 INFO:teuthology.orchestra.run.smithi176.stderr: 91.5% -- replaced with /var/log/ceph/26c91c26-4358-11ef-bcad-c7b262605968/ceph.audit.log.gz 2024-07-16T10:14:33.439 INFO:teuthology.orchestra.run.smithi176.stderr:gzip -5 --verbose -- /var/log/ceph/26c91c26-4358-11ef-bcad-c7b262605968/ceph-mds.cephfs.smithi176.wahbsi.log 2024-07-16T10:14:33.439 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/26c91c26-4358-11ef-bcad-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/26c91c26-4358-11ef-bcad-c7b262605968/ceph-mds.cephfs.smithi176.dvvbud.log 2024-07-16T10:14:33.439 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/26c91c26-4358-11ef-bcad-c7b262605968/ceph-mds.cephfs.smithi176.wahbsi.log: 85.4% -- replaced with /var/log/ceph/26c91c26-4358-11ef-bcad-c7b262605968/ceph.cephadm.log.gz 2024-07-16T10:14:33.451 INFO:teuthology.orchestra.run.smithi176.stderr:gzip -5 --verbose -- /var/log/ceph/26c91c26-4358-11ef-bcad-c7b262605968/ceph-volume.log 2024-07-16T10:14:33.454 INFO:teuthology.orchestra.run.smithi156.stderr:gzip -5 --verbose -- /var/log/ceph/26c91c26-4358-11ef-bcad-c7b262605968/ceph-mds.cephfs.smithi156.rdibys.log 2024-07-16T10:14:33.455 INFO:teuthology.orchestra.run.smithi156.stderr:/var/log/ceph/26c91c26-4358-11ef-bcad-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/26c91c26-4358-11ef-bcad-c7b262605968/ceph-volume.log 2024-07-16T10:14:33.457 INFO:teuthology.orchestra.run.smithi156.stderr:/var/log/ceph/26c91c26-4358-11ef-bcad-c7b262605968/ceph-mds.cephfs.smithi156.rdibys.log: 86.5% -- replaced with /var/log/ceph/26c91c26-4358-11ef-bcad-c7b262605968/ceph.cephadm.log.gz 2024-07-16T10:14:33.457 INFO:teuthology.orchestra.run.smithi156.stderr:gzip -5 --verbose -- /var/log/ceph/26c91c26-4358-11ef-bcad-c7b262605968/ceph-mon.smithi156.log 2024-07-16T10:14:33.464 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/26c91c26-4358-11ef-bcad-c7b262605968/ceph-mds.cephfs.smithi176.dvvbud.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-07-16T10:14:33.471 INFO:teuthology.orchestra.run.smithi156.stderr:/var/log/ceph/26c91c26-4358-11ef-bcad-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-07-16T10:14:33.480 INFO:teuthology.orchestra.run.smithi156.stderr:/var/log/ceph/26c91c26-4358-11ef-bcad-c7b262605968/ceph-mon.smithi156.log: /var/log/ceph/cephadm.log: 91.1% -- replaced with /var/log/ceph/26c91c26-4358-11ef-bcad-c7b262605968/ceph-volume.log.gz 2024-07-16T10:14:33.480 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/26c91c26-4358-11ef-bcad-c7b262605968/ceph-volume.log: /var/log/ceph/cephadm.log: 92.3% -- replaced with /var/log/ceph/cephadm.log.gz 2024-07-16T10:14:33.494 INFO:teuthology.orchestra.run.smithi176.stderr: 89.8% -- replaced with /var/log/ceph/26c91c26-4358-11ef-bcad-c7b262605968/ceph-mgr.smithi176.dqbrvx.log.gz 2024-07-16T10:14:33.495 INFO:teuthology.orchestra.run.smithi176.stderr: 91.1% -- replaced with /var/log/ceph/26c91c26-4358-11ef-bcad-c7b262605968/ceph-volume.log.gz 2024-07-16T10:14:33.514 INFO:teuthology.orchestra.run.smithi156.stderr: 92.1% -- replaced with /var/log/ceph/cephadm.log.gz 2024-07-16T10:14:33.825 INFO:teuthology.orchestra.run.smithi156.stderr: 89.1% -- replaced with /var/log/ceph/26c91c26-4358-11ef-bcad-c7b262605968/ceph-mgr.smithi156.zkvylv.log.gz 2024-07-16T10:14:33.884 INFO:teuthology.orchestra.run.smithi176.stderr: 92.5% -- replaced with /var/log/ceph/26c91c26-4358-11ef-bcad-c7b262605968/ceph-mon.smithi176.log.gz 2024-07-16T10:14:34.427 INFO:teuthology.orchestra.run.smithi156.stderr: 90.5% -- replaced with /var/log/ceph/26c91c26-4358-11ef-bcad-c7b262605968/ceph-mon.smithi156.log.gz 2024-07-16T10:14:38.195 INFO:teuthology.orchestra.run.smithi176.stderr: 95.2% -- replaced with /var/log/ceph/26c91c26-4358-11ef-bcad-c7b262605968/ceph-mds.cephfs.smithi176.wahbsi.log.gz 2024-07-16T10:14:39.130 INFO:teuthology.orchestra.run.smithi156.stderr: 93.4% -- replaced with /var/log/ceph/26c91c26-4358-11ef-bcad-c7b262605968/ceph-osd.1.log.gz 2024-07-16T10:14:39.416 INFO:teuthology.orchestra.run.smithi156.stderr: 93.3% -- replaced with /var/log/ceph/26c91c26-4358-11ef-bcad-c7b262605968/ceph-osd.2.log.gz 2024-07-16T10:14:39.628 INFO:teuthology.orchestra.run.smithi176.stderr: 93.3% -- replaced with /var/log/ceph/26c91c26-4358-11ef-bcad-c7b262605968/ceph-osd.4.log.gz 2024-07-16T10:14:40.032 INFO:teuthology.orchestra.run.smithi156.stderr: 93.4% -- replaced with /var/log/ceph/26c91c26-4358-11ef-bcad-c7b262605968/ceph-osd.0.log.gz 2024-07-16T10:14:40.482 INFO:teuthology.orchestra.run.smithi176.stderr: 93.5% -- replaced with /var/log/ceph/26c91c26-4358-11ef-bcad-c7b262605968/ceph-osd.5.log.gz 2024-07-16T10:14:41.581 INFO:teuthology.orchestra.run.smithi176.stderr: 93.4% -- replaced with /var/log/ceph/26c91c26-4358-11ef-bcad-c7b262605968/ceph-osd.3.log.gz 2024-07-16T10:14:43.160 INFO:teuthology.orchestra.run.smithi156.stderr: 95.2% -- replaced with /var/log/ceph/26c91c26-4358-11ef-bcad-c7b262605968/ceph-mds.cephfs.smithi156.evwwaz.log.gz 2024-07-16T10:14:47.357 INFO:teuthology.orchestra.run.smithi176.stderr:gzip: /var/log/ceph/ceph-client.1.log: file size changed while zipping 2024-07-16T10:14:47.357 INFO:teuthology.orchestra.run.smithi176.stderr: 93.1% -- replaced with /var/log/ceph/ceph-client.1.log.gz 2024-07-16T10:14:49.675 INFO:teuthology.orchestra.run.smithi156.stderr:gzip: /var/log/ceph/ceph-client.0.log: file size changed while zipping 2024-07-16T10:14:49.868 INFO:teuthology.orchestra.run.smithi156.stderr: 93.1% -- replaced with /var/log/ceph/ceph-client.0.log.gz 2024-07-16T10:15:14.580 INFO:teuthology.orchestra.run.smithi176.stderr: 93.6% -- replaced with /var/log/ceph/26c91c26-4358-11ef-bcad-c7b262605968/ceph-mds.cephfs.smithi176.dvvbud.log.gz 2024-07-16T10:15:14.582 INFO:teuthology.orchestra.run.smithi176.stderr: 2024-07-16T10:15:14.582 INFO:teuthology.orchestra.run.smithi176.stderr:real 0m41.159s 2024-07-16T10:15:14.582 INFO:teuthology.orchestra.run.smithi176.stderr:user 1m19.104s 2024-07-16T10:15:14.583 INFO:teuthology.orchestra.run.smithi176.stderr:sys 0m2.471s 2024-07-16T10:15:23.536 INFO:teuthology.orchestra.run.smithi156.stderr: 93.3% -- replaced with /var/log/ceph/26c91c26-4358-11ef-bcad-c7b262605968/ceph-mds.cephfs.smithi156.rdibys.log.gz 2024-07-16T10:15:23.538 INFO:teuthology.orchestra.run.smithi156.stderr: 2024-07-16T10:15:23.538 INFO:teuthology.orchestra.run.smithi156.stderr:real 0m50.116s 2024-07-16T10:15:23.538 INFO:teuthology.orchestra.run.smithi156.stderr:user 1m32.784s 2024-07-16T10:15:23.538 INFO:teuthology.orchestra.run.smithi156.stderr:sys 0m2.945s 2024-07-16T10:15:23.538 INFO:tasks.cephadm:Archiving logs... 2024-07-16T10:15:23.539 DEBUG:teuthology.misc:Transferring archived files from smithi156:/var/log/ceph to /home/teuthworker/archive/adking-2024-07-16_04:15:24-orch:cephadm-wip-adk4-testing-2024-07-15-1832-squid-distro-default-smithi/7803693/remote/smithi156/log 2024-07-16T10:15:23.540 DEBUG:teuthology.orchestra.run.smithi156:> sudo tar c -f - -C /var/log/ceph -- . 2024-07-16T10:15:35.955 DEBUG:teuthology.misc:Transferring archived files from smithi176:/var/log/ceph to /home/teuthworker/archive/adking-2024-07-16_04:15:24-orch:cephadm-wip-adk4-testing-2024-07-15-1832-squid-distro-default-smithi/7803693/remote/smithi176/log 2024-07-16T10:15:35.956 DEBUG:teuthology.orchestra.run.smithi176:> sudo tar c -f - -C /var/log/ceph -- . 2024-07-16T10:15:46.860 INFO:tasks.cephadm:Removing cluster... 2024-07-16T10:15:46.860 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 26c91c26-4358-11ef-bcad-c7b262605968 --force 2024-07-16T10:15:47.438 DEBUG:teuthology.orchestra.run.smithi176:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 26c91c26-4358-11ef-bcad-c7b262605968 --force 2024-07-16T10:15:47.992 INFO:tasks.cephadm:Removing cephadm ... 2024-07-16T10:15:47.993 DEBUG:teuthology.orchestra.run.smithi156:> rm -rf /home/ubuntu/cephtest/cephadm 2024-07-16T10:15:48.009 DEBUG:teuthology.orchestra.run.smithi176:> rm -rf /home/ubuntu/cephtest/cephadm 2024-07-16T10:15:48.026 INFO:tasks.cephadm:Teardown complete 2024-07-16T10:15:48.026 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-07-16T10:15:48.038 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_5bc696040f54eb999ca02a20c94bfd735f22b93c/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_teuthology_5bc696040f54eb999ca02a20c94bfd735f22b93c/teuthology/task/install/__init__.py", line 641, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_5bc696040f54eb999ca02a20c94bfd735f22b93c/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.10/contextlib.py", line 142, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_a6ca461be7cc1aff3658c2c6df4cff975a24449a/qa/tasks/ceph_fuse.py", line 248, in task mount.umount_wait() File "/home/teuthworker/src/git.ceph.com_ceph-c_a6ca461be7cc1aff3658c2c6df4cff975a24449a/qa/tasks/cephfs/fuse_mount.py", line 403, in umount_wait run.wait([self.fuse_daemon], timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_5bc696040f54eb999ca02a20c94bfd735f22b93c/teuthology/orchestra/run.py", line 473, in wait check_time() File "/home/teuthworker/src/git.ceph.com_teuthology_5bc696040f54eb999ca02a20c94bfd735f22b93c/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (51) after waiting for 300 seconds 2024-07-16T10:15:48.039 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-07-16T10:15:48.039 DEBUG:teuthology.orchestra.run.smithi156:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-07-16T10:15:48.053 DEBUG:teuthology.orchestra.run.smithi176:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-07-16T10:15:48.137 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-07-16T10:15:48.151 INFO:teuthology.task.clock:Checking final clock skew... 2024-07-16T10:15:48.151 DEBUG:teuthology.orchestra.run.smithi156:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-07-16T10:15:48.154 DEBUG:teuthology.orchestra.run.smithi176:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-07-16T10:15:48.167 INFO:teuthology.orchestra.run.smithi156.stderr:bash: line 1: ntpq: command not found 2024-07-16T10:15:48.168 INFO:teuthology.orchestra.run.smithi176.stderr:bash: line 1: ntpq: command not found 2024-07-16T10:15:48.172 INFO:teuthology.orchestra.run.smithi156.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-07-16T10:15:48.172 INFO:teuthology.orchestra.run.smithi156.stdout:=============================================================================== 2024-07-16T10:15:48.172 INFO:teuthology.orchestra.run.smithi156.stdout:^* hv01.front.sepia.ceph.com 3 6 377 65 +216us[ +243us] +/- 40ms 2024-07-16T10:15:48.172 INFO:teuthology.orchestra.run.smithi156.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 1 +122us[ +122us] +/- 60ms 2024-07-16T10:15:48.172 INFO:teuthology.orchestra.run.smithi156.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 3 -579us[ -579us] +/- 82ms 2024-07-16T10:15:48.172 INFO:teuthology.orchestra.run.smithi156.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-07-16T10:15:48.174 INFO:teuthology.orchestra.run.smithi176.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-07-16T10:15:48.174 INFO:teuthology.orchestra.run.smithi176.stdout:=============================================================================== 2024-07-16T10:15:48.174 INFO:teuthology.orchestra.run.smithi176.stdout:^* hv01.front.sepia.ceph.com 3 6 377 3 +199us[ +199us] +/- 41ms 2024-07-16T10:15:48.174 INFO:teuthology.orchestra.run.smithi176.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 2 +104us[ +104us] +/- 60ms 2024-07-16T10:15:48.174 INFO:teuthology.orchestra.run.smithi176.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 2 -603us[ -603us] +/- 82ms 2024-07-16T10:15:48.174 INFO:teuthology.orchestra.run.smithi176.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-07-16T10:15:48.175 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-07-16T10:15:48.186 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-07-16T10:15:48.187 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-07-16T10:15:48.201 DEBUG:teuthology.orchestra.run.smithi156:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-07-16T10:15:48.296 DEBUG:teuthology.orchestra.run.smithi176:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-07-16T10:15:48.371 DEBUG:teuthology.orchestra.run.smithi156:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-07-16T10:15:48.400 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-07-16T10:15:48.401 DEBUG:teuthology.orchestra.run.smithi176:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-07-16T10:15:48.428 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-07-16T10:15:48.429 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-07-16T10:15:48.440 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-07-16T10:15:48.522 INFO:teuthology.task.internal:Duration was 2216.487263 seconds 2024-07-16T10:15:48.523 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-07-16T10:15:48.537 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-07-16T10:15:48.537 DEBUG:teuthology.orchestra.run.smithi156:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-07-16T10:15:48.540 DEBUG:teuthology.orchestra.run.smithi176:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-07-16T10:15:48.576 INFO:teuthology.orchestra.run.smithi156.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-07-16T10:15:48.578 INFO:teuthology.orchestra.run.smithi176.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-07-16T10:15:48.889 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-07-16T10:15:48.889 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi156.front.sepia.ceph.com 2024-07-16T10:15:48.889 DEBUG:teuthology.orchestra.run.smithi156:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-07-16T10:15:48.910 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi176.front.sepia.ceph.com 2024-07-16T10:15:48.910 DEBUG:teuthology.orchestra.run.smithi176:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-07-16T10:15:48.953 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-07-16T10:15:48.953 DEBUG:teuthology.orchestra.run.smithi156:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-07-16T10:15:48.955 DEBUG:teuthology.orchestra.run.smithi176:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-07-16T10:15:49.020 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-07-16T10:15:49.020 DEBUG:teuthology.orchestra.run.smithi156:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-07-16T10:15:49.022 DEBUG:teuthology.orchestra.run.smithi176:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-07-16T10:15:49.736 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-07-16T10:15:49.750 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-07-16T10:15:49.750 DEBUG:teuthology.orchestra.run.smithi156:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-07-16T10:15:49.775 DEBUG:teuthology.orchestra.run.smithi176:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-07-16T10:15:49.802 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-07-16T10:15:49.816 DEBUG:teuthology.orchestra.run.smithi156:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-07-16T10:15:49.818 DEBUG:teuthology.orchestra.run.smithi176:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-07-16T10:15:49.839 INFO:teuthology.orchestra.run.smithi156.stdout:kernel.core_pattern = core 2024-07-16T10:15:49.866 INFO:teuthology.orchestra.run.smithi176.stdout:kernel.core_pattern = core 2024-07-16T10:15:49.912 DEBUG:teuthology.orchestra.run.smithi156:> test -e /home/ubuntu/cephtest/archive/coredump 2024-07-16T10:15:49.926 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-07-16T10:15:49.927 DEBUG:teuthology.orchestra.run.smithi176:> test -e /home/ubuntu/cephtest/archive/coredump 2024-07-16T10:15:49.967 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-07-16T10:15:49.967 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-07-16T10:15:49.980 INFO:teuthology.task.internal:Transferring archived files... 2024-07-16T10:15:49.981 DEBUG:teuthology.misc:Transferring archived files from smithi156:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/adking-2024-07-16_04:15:24-orch:cephadm-wip-adk4-testing-2024-07-15-1832-squid-distro-default-smithi/7803693/remote/smithi156 2024-07-16T10:15:49.981 DEBUG:teuthology.orchestra.run.smithi156:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-07-16T10:15:50.032 DEBUG:teuthology.misc:Transferring archived files from smithi176:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/adking-2024-07-16_04:15:24-orch:cephadm-wip-adk4-testing-2024-07-15-1832-squid-distro-default-smithi/7803693/remote/smithi176 2024-07-16T10:15:50.032 DEBUG:teuthology.orchestra.run.smithi176:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-07-16T10:15:50.071 INFO:teuthology.task.internal:Removing archive directory... 2024-07-16T10:15:50.071 DEBUG:teuthology.orchestra.run.smithi156:> rm -rf -- /home/ubuntu/cephtest/archive 2024-07-16T10:15:50.073 DEBUG:teuthology.orchestra.run.smithi176:> rm -rf -- /home/ubuntu/cephtest/archive 2024-07-16T10:15:50.119 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-07-16T10:15:50.132 INFO:teuthology.task.internal:Not uploading archives. 2024-07-16T10:15:50.133 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-07-16T10:15:50.146 INFO:teuthology.task.internal:Tidying up after the test... 2024-07-16T10:15:50.146 DEBUG:teuthology.orchestra.run.smithi156:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-07-16T10:15:50.149 DEBUG:teuthology.orchestra.run.smithi176:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-07-16T10:15:50.162 INFO:teuthology.orchestra.run.smithi156.stdout: 265390 4 drwxr-xr-x 3 ubuntu ubuntu 4096 Jul 16 10:15 /home/ubuntu/cephtest 2024-07-16T10:15:50.162 INFO:teuthology.orchestra.run.smithi156.stdout: 669154 4 d--------- 2 ubuntu ubuntu 4096 Jul 16 09:53 /home/ubuntu/cephtest/mnt.0 2024-07-16T10:15:50.162 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-07-16T10:15:50.163 INFO:teuthology.orchestra.run.smithi156.stderr:find: ‘/home/ubuntu/cephtest/mnt.0’: Permission denied 2024-07-16T10:15:50.163 INFO:teuthology.orchestra.run.smithi156.stderr:rmdir: failed to remove '/home/ubuntu/cephtest': Directory not empty 2024-07-16T10:15:50.163 ERROR:teuthology.run_tasks:Manager failed: internal.base Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_5bc696040f54eb999ca02a20c94bfd735f22b93c/teuthology/task/internal/__init__.py", line 48, in base yield File "/home/teuthworker/src/git.ceph.com_teuthology_5bc696040f54eb999ca02a20c94bfd735f22b93c/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.10/contextlib.py", line 142, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_a6ca461be7cc1aff3658c2c6df4cff975a24449a/qa/tasks/ceph_fuse.py", line 248, in task mount.umount_wait() File "/home/teuthworker/src/git.ceph.com_ceph-c_a6ca461be7cc1aff3658c2c6df4cff975a24449a/qa/tasks/cephfs/fuse_mount.py", line 403, in umount_wait run.wait([self.fuse_daemon], timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_5bc696040f54eb999ca02a20c94bfd735f22b93c/teuthology/orchestra/run.py", line 473, in wait check_time() File "/home/teuthworker/src/git.ceph.com_teuthology_5bc696040f54eb999ca02a20c94bfd735f22b93c/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (51) after waiting for 300 seconds During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_5bc696040f54eb999ca02a20c94bfd735f22b93c/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.10/contextlib.py", line 153, in __exit__ self.gen.throw(typ, value, traceback) File "/home/teuthworker/src/git.ceph.com_teuthology_5bc696040f54eb999ca02a20c94bfd735f22b93c/teuthology/task/internal/__init__.py", line 53, in base run.wait( File "/home/teuthworker/src/git.ceph.com_teuthology_5bc696040f54eb999ca02a20c94bfd735f22b93c/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_5bc696040f54eb999ca02a20c94bfd735f22b93c/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_5bc696040f54eb999ca02a20c94bfd735f22b93c/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi156 with status 1: 'find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest' 2024-07-16T10:15:50.164 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-07-16T10:15:50.175 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-07-16T10:15:50.191 DEBUG:teuthology.run_tasks:Exception was not quenched, exiting: MaxWhileTries: reached maximum tries (51) after waiting for 300 seconds 2024-07-16T10:15:50.194 INFO:teuthology.run:Summary data: description: orch:cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn pg_health syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} duration: 2216.4872629642487 failure_reason: reached maximum tries (51) after waiting for 300 seconds flavor: default owner: scheduled_adking@teuthology status: fail success: false 2024-07-16T10:15:50.194 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-07-16T10:15:50.279 INFO:teuthology.run:FAIL